How to add token compression to a Mastra agent

Your Mastra agent stays as it is. You wrap it once with withCavemanMastra, call generate or stream on the wrapped agent, and run a small local runtime next to your app. For the background, read Caveman vs Mastra.

What you need

• @mastra/core >=1.65 <2. Outside that range the adapter declines, reports unsupported_version, and passes calls through untouched.
• Node.js 22.13 or newer for the caveman CLI that runs the runtime.
• One agent whose tools return a lot of text, such as a log reader, a file tool or an API tool with big JSON responses. Pick one that answers in plain text; structured-output calls get no recovery tool, so they go out uncompressed.

Step 1: Record a baseline

Run the task a few times on the unwrapped agent. Save the answer, whether it passed your check, and the usage Mastra reports for the run. Keep the model, instructions and tools fixed from here on.

Step 2: Install the packages

npm install @caveman-ai/middleware @caveman-ai/sdk

@caveman-ai/middleware (0.1.0-alpha.2) holds the adapters and brings in no framework of its own, so it uses the Mastra you already have. @caveman-ai/sdk (1.1.0) holds the runtime client.

Step 3: Start the local runtime

npm install -g @caveman-ai/cli && caveman setup --install
CAVEMAN_MODE=compress caveman start

Check it from another shell:

curl -s http://127.0.0.1:8787/caveman/v1/middleware/capabilities | jq '{mode, persistent, recovery, retention_seconds}'

You want "mode": "compress" and "recovery": true. Plain caveman start runs record, which measures and changes nothing.

Step 4: Wrap the agent

Create one runtime for the process and pass your agent through withCavemanMastra. The call below is the docs example; the runtime lines come from the TypeScript middleware guide.

import { createMiddlewareRuntime } from '@caveman-ai/sdk/middleware';
import { withCavemanMastra } from '@caveman-ai/middleware/mastra';

const runtime = createMiddlewareRuntime({
  endpoint: 'http://127.0.0.1:8787',
  mode: 'compress',
  onReport: report => console.error('Caveman:', JSON.stringify(report)),
});

export async function run({ agent, messages, scope }) {
  if (runtime.mode !== "off") await runtime.ready(); // ready() throws in off mode
  const wrapped = withCavemanMastra(agent, { runtime, scope });
  return wrapped.generate(messages, { maxSteps: 4 });
}

A few notes from the adapter source:

• The wrapper does not touch your Agent. On each call it puts its processor in front of your configured input processors and keeps your prepareStep. The original agent still works unwrapped.
• Scope keys the stored originals. It is four strings: namespace, session_id, branch_id, cache_epoch. You can pass a fixed object or a function that receives { requestContext }. Build it from your authenticated user and conversation, never from model input, and keep it the same across turns so earlier recovery handles still resolve.
• Leave room for recovery. If the model decides to fetch an original, that is one more tool step. If maxSteps is tight, raise it by one.

Step 5: Check it worked

onReport logs one line per provider call. Expect this pattern:

• First call: skipped with no_candidate, since no tool has run yet.
• After the first large tool result: applied, with a transform id such as caveman.engine.log.v1.
• Later calls: reused. The same shortened bytes go out again, so the provider's cached prefix holds.

Then confirm:

1. Mastra's stored messages and memory still hold the full tool results.
2. The task passes the same check as the baseline.
3. Provider-reported input tokens are lower across the whole run, counting any recovery calls and retries.

If nothing shrinks, read the reason. runtime_unavailable means the endpoint did not answer. recorded means the runtime is in record mode. unsupported_version means @mastra/core is outside the gate; npm ls @mastra/core shows what is installed.

How to roll back

• Call agent.generate or agent.stream on the original agent instead of the wrapped one. Nothing else was changed.
• Or set mode: 'off' on the runtime. The adapter reports disabled and sends the original bytes.

Close the runtime client with runtime.close() at shutdown. The runtime process keeps stored originals for 24 hours after their last use, so conversations already in flight can still recover.

Next steps

Judge the change on finished tasks, not one request: how to measure agent cost covers the method. The middleware overview explains report statuses, and the deployment page covers running the runtime beside a deployed app.