Cut the MCP token overhead you pay on every request

Every MCP server you connect writes its tool definitions into the top of every request, used or not. You pay for the whole list on the first turn, the last turn and every turn between.

The constraint is that this cost is invisible. It has no line item, no warning and no relationship to what you asked for. It is the floor under your session, and you cannot cut a number you have never looked at.

Open a fresh session, type one word, and read the input count

Start a new session in your agent with your usual MCP servers connected. Send hi. Nothing else. Then look at the input tokens on that turn in your provider usage or your agent's own counter.

That number is your floor. You paid it for a two letter question, and you will pay it again on every turn of every session until you change something. Write it down before you touch anything.

Now disconnect every MCP server and repeat. The difference between the two numbers is what MCP costs you per request.

Three different costs get called MCP overhead

The first row is the one people miss, because it is the only one that has nothing to do with what the agent is doing. The protocol says how hosts and servers exchange tools, not how a host loads them into a model request. Some hosts load lazily or search first. Check the MCP architecture and then check the host you actually run.

Disconnect the servers you are not using today

This is the largest cut and the easiest to undo. A browser server, a database server and two API servers you added in March are all still being described to the model while you write a CSS fix.

Turn off what today's work does not need, where your host lets you. Then run a task that needs the servers you kept, and confirm the agent still finds the right tool.

Shorten the prose, keep the contract

Argument names, types, required fields, enums and constraints stay exactly as they are. They are the contract. What you can cut is repeated boilerplate, worked examples and the third sentence explaining what the first sentence already said.

Read a description before you trim it. Some of those words are the only thing telling the model that one tool reads and its neighbour writes. Deleting them makes the schema smaller and the agent worse.

Caveman has explicit tool-schema controls, and broad stripping is not on by default anywhere. Proxy configuration lists what is supported. Do not switch a transform on because a different compressor produced a good ratio on a different kind of content.

Do not merge precise tools into one vague tool

Replacing eight named tools with one do_anything tool makes the definitions block smaller and the session worse. Selection errors go up, permission boundaries disappear, and every mistake buys you a repair turn.

Deferred discovery has the same trap in reverse. A small opening prompt is not a win if the agent burns five calls finding the tool it needs. Count the calls, not the prompt.

Compress the results, and keep the original readable

Take a real tool result you are allowed to keep and run it through the compressor:

npm install -g @caveman-ai/cli
caveman setup --install
caveman shrink --file ./fixtures/mcp-tool-result.json

Read what came out. Check what survived, what went, and whether you got a short ID the model can use to read the original text back. Then ask a follow-up question that needs a detail the smaller view dropped, and see if the agent retrieves it. The compression guide covers recovery and how to judge a finished task.

Retrieval needs a path. Once the required binary is present, the CLI can register the recovery server with your host:

caveman tools mcp install claude --server caveman

Check what actually got registered, then make one recovery request. The local server speaks stdio, so stdout has to stay clean protocol traffic. A shell wrapper that prints a startup banner there kills the connection. Store access is local too: an ID created on your laptop means nothing to an agent running somewhere else. The Caveman MCP reference lists the tools and errors.

One rule stands above all of this. A consumer that expects a complete typed object never gets a smaller view instead. That is a contract change, not a compression setting.

Let the local report rank the sinks for you

Caveman can read supported local agent sessions and sort your token sinks by size:

caveman learn
caveman learn report --json

Read it before you apply anything. It names candidates, and a candidate is not a billed saving. Coverage depends on which sessions and formats exist on your machine, so treat gaps as gaps.

To see exactly what a suggested change would do, preview it with the real ID from your report. YOUR_SINK_ID is a placeholder:

caveman learn apply YOUR_SINK_ID --dry-run

The learn documentation explains the supported path.

Measure the empty turn again, then measure a real task

Send hi to a fresh session with your new configuration. Compare that input count with the one you wrote down. That is the per request floor you removed, and it is the cleanest number on this page.

Then run a real task on both configurations, same host, same model, same starting workspace. Count failed tool calls, retries and recovery, and check the final answer yourself. Keep the skill state identical in both arms so the input change stays the only change.

A good outcome is specific: this server off for this workflow, these descriptions shorter with selection unchanged, these results compressed with the answer still correct. Adopt what passed. Cost measurement keeps the result tied to work you accepted, and the setup guide covers the launchers.