Stop hitting API rate limits. Shield your token budget.
The Local Token Shield & Context Optimization Engine
TZRO runs a transparent <50MB native loopback proxy that locks KV-cache prefixes (70–99% cache read hit rate, benchmarked across 8 models), skeletonizes source code via Tree-sitter, and provides sub-millisecond local discovery.
# Output pipeline expectation:
➔ Downloading runtime binaries from secure s3 allocation...
✔ System architecture validated (x86_64 / arm64)
✔ Symlinked tzro binary wrapper to local user bin space.
How Tzro v2 Works
Tzro sits transparently between your AI coding agent and cloud providers, intercepting heavy context and locking prompt cache prefixes.
Transparent Proxy
Loopback Port 7878
- › Agents send requests to localhost:7878. System instructions and tool schemas are pinned in deterministic byte order.
- › Benchmarked at 70–99% prompt cache hit rates across 8 LLM providers, avoiding the 12.5x cache miss penalty.
- › E2E benchmarks show kvlock normalization improves cache effectiveness by 5–10 percentage points over native provider caching.
Tree-Sitter Engine
On-Device Indexing
- › Code reads are skeletonized: method bodies are elided into SHA-256 hashes (#hash) and stored in local SQLite.
-
›
tzro probescans codebases via ripgrep in <5ms, turning 10-turn discovery loops into single 500-token queries.
Cloud LLM
Claude 3.5 / GPT-4o
- › The model receives dense, high-signal, cache-locked context without transient log bloat or secret leakage.
- › SSE tokens stream straight back to your editor with sub-millisecond latency.
KV-Cache Prefix Locking — Benchmarked Across Real Models
We run two-phase benchmarks against live cloud APIs (via OpenRouter): a 20-turn incremental agent loop tracking per-turn cache hit ratios, then a controlled token-size sweep to find the cache cliff. All numbers below are from real runs — not synthetic micro-benchmarks.
| What We Measure | Without kvlock | With kvlock | Impact |
|---|---|---|---|
| Prompt Cache Hit Rate | ~13% (provider default) | 63–96% (avg 79% warm) | 5–10pp improvement over native caching 🟢 |
| Prefix Hash Stability | Varies across turns | 100% stable (20/20 turns) | Byte-for-byte prefix reproducibility across every turn 🟢 |
| E2E Agent Loop Cost | $0.0080 (19 turns, direct) | $0.00035 (2 turns, proxied) | 95.6% cost reduction — same task, same model 🟢 |
How Prefix Locking Eliminates the 12.5× Cache Miss Penalty
Cloud LLM providers charge 12.5× more for cache misses than cache reads. A single reordered tool definition or volatile timestamp in the system prompt breaks the prefix — and every subsequent turn pays full price.
Cache hit ratio swings wildly — 0% on turn 1, 40% on turn 6, 48% on turn 10. Every cache miss triggers the full 1.25× write cost.
Prefix hash f9e6d771245a locked stable across all
20 turns. Avg cache hit ratio: 63.5%.
Peak: 96.4% (turn 18). Zero prefix drift.
Core Execution Surfaces
Tzro intercepts every outgoing LLM request through a transparent loopback proxy — normalizing prefixes, redacting secrets, and compacting context before the request leaves your machine. A native CLI toolkit handles sub-millisecond codebase discovery with zero cloud tokens.
Token Shield Proxy
TRANSPARENT<50 MB RAM · Zero dependencies · Transparent loopback
CLI Discovery Engine
NATIVESub-millisecond discovery · AST-aware · Zero cloud tokens
Every OS primitive, implemented for AI agents
tzro maps every classical operating system concept to an agentic equivalent. Click any block to explore.
One command. Zero config.
Install TZRO with a single command. It detects your system, downloads the right binary, and symlinks it to your PATH.
What else ships inside the jumpdrive
Six more proprietary subsystems — all implemented, tested, and shipping.
Zero Syntax Failures
GBNF Grammar + Semantic Validator
The Local Model generates high-speed XML under shallow GBNF structural constraints, then a deterministic Semantic Validator coerces it into strict JSON tool parameters. Type coercion, default imputation, fuzzy matching — all in one boundary seam. 0% syntax failure rate for tool arguments.
Zero Chat Latency
Priority KV Cache Preemption
When a user sends a chat message, the OS dumps the running background task's KV attention state to disk, erases the slot, and processes the chat instantly — sub-450ms time-to-first-token. When chat completes, background state restores from disk without re-evaluating historical tokens.
Pure Local Memory
Hybrid Vector Search & Knowledge Graph
Zero cloud dependencies for long-term recall. SQLite FTS5 keyword pre-filtering → ONNX cosine similarity ranking → Neighborhood Multi-Hop traversal across a relational knowledge graph. Rich, contextual subgraphs injected directly into the model.
Self-Improving Inference
Dual Micro-Skill Extraction
Successful trajectories synthesize Procedural Micro-Skills. Failed-then-succeeded pairs extract Corrective Micro-Skills — teaching the local model to self-correct on specific failure patterns without weight updates. The OS gets better the more you use it.
Background Intelligence
Observer + Sentinel + Attention Scheduler
Three background agents form an autonomous nervous system. The Observer reflects on completed tasks. The Sentinel proactively correlates workspace activity. The Attention Scheduler enforces preemption, safety gates (L0–L4), and resource budgets.
Generative Dashboard
Agent-Composed Observability
The dashboard isn't a static monitoring page — it's a Generative UI surface where the Local Model analyzes system state and composes the layout from 15 primitives. The agent decides panel ordering, emphasis, and which tasks deserve spotlight attention.