What prompt compression does to your bytes, and how to check it

Prompt compression replaces the bulky parts of what your agent sends with a shorter view of the same content, and the original is written to your disk before the replacement happens. Nothing is thrown away. If any step of that fails, the original goes to the model untouched.

The constraint behind every check on this page: a smaller prompt is worthless if the agent then needs three more turns to recover what you took out. The number that decides this is the cost of one finished task, not a compression ratio.

The original is stored first, and the failure path is the original

Here is the order of operations on a tool result that is large enough to be worth shrinking.

1. Caveman picks a compressor by content type. There are nine, for the things agents actually send: JSON, logs, code, tables, diffs, search results and bulk context.
2. The original bytes are written to a local store.
3. The model receives the shorter view, plus a short ID it can use to read the original text back.
4. When the model needs the part that is missing, it calls that ID and gets the original.

Three things stop the transform, and all three end the same way. Your original goes to the model exactly as it was.

• The content does not parse as the type it looked like.
• The store write fails.
• The shrunk version came out no smaller than the input.

Passing through is not a failure state. It is the design. Your bytes also stay on the machine: prompts and responses go from there straight to your provider, and Caveman's cloud is not in that path.

Shrink one file before you point any of this at an agent

Use a fixture you are allowed to keep, a saved log or a real tool result:

npm install -g @caveman-ai/cli
caveman setup --install
caveman shrink --file ./fixtures/tool-output.json

Or through a pipe:

caveman compress < ./fixtures/tool-output.json > tool-output.view.txt

Read the view and the report side by side. What stayed, what went, and did you get an ID back. A small or unsuitable input passes through unchanged, and that is a real result. Record it. Do not hunt for a file that makes the demo look better.

Prove the recovery yourself, with cmp

This is the part you should not take on trust. Copy the actual ID your run printed into the command below, in place of YOUR_RECOVERY_HANDLE:

caveman retrieve YOUR_RECOVERY_HANDLE > recovered-original.json
cmp ./fixtures/tool-output.json recovered-original.json

No output from cmp means the two files are identical, byte for byte. Retrieval with no query always returns the stored original whole.

Adding a query returns a relevant slice of the original instead:

caveman retrieve YOUR_RECOVERY_HANDLE "connection pool"

Use the query form when the agent needs one fact. Use full retrieval when order, exact bytes or completeness decide the answer. A query result is another selection, not proof that everything survived.

The store lives on your host with a default payload budget of 512 MiB, and at capacity existing IDs are kept rather than evicted, so recovery that already works keeps working. WASM storage holds its contents in memory and has a shorter life. The recovery contract lists the paths and overrides.

Give the agent a way to fetch the original, or it will guess

An ID in a prompt is inert on its own. The agent needs a tool or a proxy-side mechanism to call it, and that mechanism needs to reach the same store.

For a supported local coding agent:

caveman doctor claude
caveman claude

Then run a task whose answer depends on a detail a large result would drop, and watch whether the agent retrieves it. Streaming and subscription-auth paths need agent-side recovery. Requests that return one response instead of a live stream, on an API key, can use proxy-side handling. If the agent and the store sit on different machines, fix that first. A local ID does not create remote storage.

Match the reduction to what you actually have

These are not interchangeable. A shorter answer does not mean the input shrank. Caching lowers the price of repeated input without removing a byte of it. Read the caching guide before you disturb a prefix that already gets good reuse.

Compression loses money in three specific ways

It broke a cache that was working. You rewrote a prefix the provider was already serving cheaply from cache, so every turn now pays full price for context that used to be nearly free. Check cache-read fields before and after.

The agent never asked for the missing detail. The ID was there, the recovery path was not wired up, and the model answered from what it had. That shows up as a wrong patch, not as an error.

A lossy view reached a typed consumer. A model-facing view and an application data object have different contracts. Code that parses a complete schema gets the original or it gets nothing.

Run the same task twice and judge the artifact

Run the task directly and through Caveman, in two fresh sessions, with the model, reasoning setting, tools, prompt and starting workspace all held fixed. Check the output yourself. Run the tests, read the patch, compare extracted fields against known answers. Count every model request, recovery and retries included. Cover more than the content type the compressor is best at: short output, long structured data, raw logs, and one task that turns on an exact earlier detail. The measurement guide explains how to write those results up.

Then pick where it runs

Use coding-agent setup for local launchers, gateway migration for an API-path trial you can reverse, and middleware setup for an agent you wrote yourself. For other compressors, read Headroom, LLMLingua and RTK. Their recovery contracts matter more than their ratios.