Caveman, RTK, Headroom, and context-mode compared

Your agent runs a test suite. Two thousand lines land in context, ready to be carried into later requests alongside everything else the agent has already read. Then it writes six paragraphs explaining a one-line fix.

Caveman, RTK, Headroom, and context-mode tackle different parts of that sequence. Start where your tokens go. A smaller shell response, a shorter answer, and less context sent to the provider each measure something different.

We build Caveman. This comparison covers local tooling, with documentation reviewed on September 19, 2026. Feature descriptions summarize each project's linked documentation. Published measurements keep their original scope; we did not run a new benchmark for this article.

Start with what each tool can reach

RTK's interception boundary matters. Its Claude Code hook rewrites Bash commands, while built-in Read, Grep, and Glob calls bypass it. An agent using rg through the shell can benefit; one using native Grep needs a different path. RTK documents this explicitly in its quick start.

Caveman's skill changes how the agent writes prose. Its proxy works on supported content in requests routed through it, so setup and payload shape determine coverage. Installing the skill alone does not compress incoming tool output. Caveman README

Headroom also reaches both sides. Alongside its proxy, library, and MCP tools, an optional output shaper adds terseness instructions and adjusts reasoning effort for selected turns. That capability belongs in the comparison. Headroom output reduction

context-mode changes the workflow earlier. Commands run in a sandbox, and the agent receives selected output or searches an index for relevant passages. Its documented search returns snippets around matches. That is useful for finding evidence in a large document; it does not establish a byte-exact replay contract for the complete original output. context-mode architecture

Recovery must cover the detail your agent missed

Suppose compression hides the one log line that explains a failing deployment. The next step costs another tool call, and the agent must recognize that it needs more evidence before answering.

Caveman's proxy stores the original before publishing a lossy view. If storage fails, the original passes through. A recovery call without a query returns the stored bytes exactly; adding a query returns selected relevant sections. The local store must remain available. This promise applies to content the proxy compresses, not to hypothetical longer replies the skill prevented. Recoverable compression

Keeping bytes available gives the agent a way to correct an incomplete view. It cannot make the agent notice every omission. A quality check still has to verify the answer, including tasks where the decisive detail was removed. Recovery calls belong in the token count too.

Local processing and telemetry are separate choices

The table describes reporting back to the tool's developer. Your configured model provider still receives requests, and fetching a web page still contacts its server even when every telemetry switch is off. Local tooling can still participate in an online workflow.

Caveman's CLI telemetry excludes prompt bodies, code, file paths, and tool results. Interactive use discloses its default; CI and non-interactive runs do not send it. Managed mode has a different data path: requests through the managed Caveman gateway transit Caveman Cloud before reaching your configured model provider. Caveman security and privacy

Read the denominator before comparing percentages

Each headline below describes its own workload and measurement. The rows cannot rank the tools against one another.

Consider a hypothetical 10,000-token request. Suppose shell output accounts for 2,000 tokens: removing 90% of it saves 1,800 tokens, reducing the whole request by 18%. Everything else remains. Later requests may carry the smaller output again, changing the session total.

Dollar cost needs another calculation. Fresh input, cache reads, cache writes, and generated output can have different prices, so keep those categories separate and include tokens spent on extra searches or retries.

Our paired report covers Caveman and Headroom

The CaveBench Wrap report records six fixed workloads, three repetitions per arm, and 54 runs total. It pins Claude Code 2.1.223, claude-sonnet-5, and Headroom 0.33.0.

Caveman used 591,673 input tokens against 885,793 for direct. Its HTML case used 9.9% more input than direct because compression did not apply and overhead remained. Headroom's three failed YAML runs stay visible as failures; the 6.7% compares only its passing pairs with their matched baselines.

This is our controlled report. The repository publishes the method and provenance hashes, but lacks the raw harness and run artifacts needed for independent reproduction. These are input-token results for pinned versions, not current-version rankings or measured reductions in a customer's bill.

RTK and context-mode were outside that experiment. It supplied data through MCP fixtures, so it would not exercise RTK's shell filter.

Choose a layer, then test a finished task

Start with RTK when shell output dominates your sessions. If your workflow can inspect large outputs through sandbox processing and search, consider context-mode; its approach asks the agent to select what it needs.

For compression in the request path, evaluate Caveman and Headroom on the payloads your agent sends. If shorter explanations would solve your immediate problem, start with Caveman's standalone skill and check whether input compression helps later.

Take a task with an answer you can check. Run it with your normal setup, then repeat it with one change, recording correctness and elapsed time alongside provider usage by token category. Include every recovery call and retry in those counts. Repeat the pair. A result that survives several attempts gives you more to work with than a single lucky run.

Keep failed attempts in the results. If you combine tools, measure that combination separately: reductions from overlapping stages cannot be added. Use our measurement guide to account for the complete run, then keep the setup that improves your cost per correct task.