---
title: "Caveman vs Vercel AI SDK: tool-result compression"
description: "The Vercel AI SDK runs your model calls, tool loop and streaming. Caveman Middleware plugs into it and shrinks large tool results before each call goes out. They do different jobs, so the usual answer"
canonical: https://caveman.ac/compare/vercel-ai-sdk
last-updated: 2026-10-04
---

# Caveman vs Vercel AI SDK: tool-result compression

The Vercel AI SDK runs your model calls, tool loop and streaming. Caveman Middleware plugs into it and shrinks large tool results before each call goes out. They do different jobs, so the usual answer is to keep the AI SDK and add the Caveman adapter where tool output is heavy.

## What is the Vercel AI SDK?

The AI SDK is Vercel's free, open-source TypeScript toolkit for building AI apps and agents, from the team behind Next.js. It is Apache 2.0 licensed and sits at about 27k stars on GitHub. It comes in three parts: AI SDK Core (one API for text, structured objects, tool calls and agents), AI SDK UI (hooks for chat and generative interfaces) and AI SDK Harnesses (a uniform API for running established agent harnesses).

The boring parts are done well, which is why so many TypeScript agents start here:

- One call shape across providers through packages like `@ai-sdk/openai`, `@ai-sdk/anthropic` and `@ai-sdk/google`.
- `ToolLoopAgent`, which runs the model, calls tools and feeds results back until a stop condition. It stops after 20 steps by default.
- `prepareStep`, which lets you change the model, tools or messages between steps, plus a `pruneMessages` helper for simple compaction.
- Streaming straight into React, Next.js, Svelte, Vue and plain Node.
- A language model middleware system: `wrapLanguageModel` with `transformParams`, `wrapGenerate` and `wrapStream`.

That last one matters here, because it is the seam Caveman uses.

## Does Caveman work with the Vercel AI SDK?

Yes. Caveman ships a native adapter at `@caveman-ai/middleware/ai-sdk` for `ai >=7.0.94 <8` and `@ai-sdk/provider >=4.0.11 <5`. Its `withCaveman` function takes the options you already pass to `streamText`, `generateText` or a `ToolLoopAgent` and returns a new object: your model wrapped with the SDK's own `wrapLanguageModel`, plus one extra tool called `caveman_retrieve`.

```diff
+import { createMiddlewareRuntime } from "@caveman-ai/sdk/middleware";
+import { withCaveman } from "@caveman-ai/middleware/ai-sdk";
+
+const runtime = createMiddlewareRuntime({ endpoint: "http://127.0.0.1:8787", mode: "compress" });
+const scope = { namespace: "support", session_id: conversationId, branch_id: "main", cache_epoch: "0" };
+
-const result = streamText(options);
+const result = streamText(withCaveman(options, { runtime, scope }));
```

On each provider call, the adapter copies the outbound request, sends eligible tool-result text to a local Caveman runtime, and swaps in the shorter version it gets back. Your `messages` array and `result.steps` keep the original bytes. If the model needs a line that got cut, it calls `caveman_retrieve` and reads the original back. Your provider, retries, streaming and tools stay where they were.

Here is what that looks like on a real shape of data. The middleware docs send a 400-line worker log as a tool result. The runtime counted 8,399 tokens before and 177 to 187 after across sixteen runs. Those are inferred counts from the `o200k_base` tokenizer on one synthetic log, not a provider bill. The model sees the first and last lines, a note saying how many lines were elided, and a handle it can use to fetch the rest.

User messages, system prompts, reasoning, errors and image or PDF bytes are left alone. Only eligible tool-result text is touched.

## Vercel AI SDK vs Caveman: feature comparison

| | Vercel AI SDK | Caveman Middleware |
| --- | --- | --- |
| What it is | TypeScript toolkit for AI apps and agents | Adapter that shrinks tool results before model calls |
| Languages | TypeScript | TypeScript and Python |
| License | Apache 2.0 | MIT client and adapters, BSL local runtime |
| Calls the model | Yes, through provider packages | No, the AI SDK still makes the call |
| Tool loop and streaming | `ToolLoopAgent`, `streamText`, UI hooks | Uses yours |
| Trimming history | `pruneMessages` inside `prepareStep` | Leaves your history as is |
| Shrinks large tool results | Not built in | Yes, on the outbound copy |
| Getting cut content back | Pruned content is not sent | `caveman_retrieve` returns the original |
| Extra process | None | Local runtime on port 8787 |
| Maturity | Established, v7 line | Alpha, `0.1.0-alpha.2` |

## When should you use the Vercel AI SDK alone?

Use it alone when your tool results are small. A weather lookup or a row count has nothing worth compressing, and the first call of any loop carries no tool result at all, so Caveman reports `no_candidate` and does nothing.

Skip Caveman too if your cost problem is long model output. Caveman shortens input; if output dominates, change the response format instead.

You also need a runtime you can reach. That matters for AI SDK apps on serverless functions: the runtime is a process with a store on disk, so it cannot ride along inside a function. You would point the function at a runtime on a host you keep running, over HTTPS with a token, or run without compression.

And if you already prune aggressively with `prepareStep` and the model still answers well, you may not need more. Measure first.

## When does adding Caveman pay off?

It pays off in agents whose tools return a lot of text: file reads, API calls that return big JSON, shell commands, document extraction. In a tool loop every earlier result gets sent again on every later step. A 20k-token JSON response read at step 2 of a 10-step run rides along on eight more calls.

Caveman shortens that result on each resend. After the first cut it resends the same shortened bytes, reported as `reused`, so the provider's cached prefix stays byte identical from call to call.

It also stacks with what you have. `pruneMessages` decides which messages go out. Caveman shrinks the tool results in the messages that stay. The two do not fight, because the adapter works on the request that is about to be sent.

If the runtime is down, the call goes out with the original input and the report says `runtime_unavailable`. After three failures in a row the adapter stops asking for 30 seconds. Your agent keeps working either way.

## How to add Caveman to the Vercel AI SDK

The short version:

1. `npm install @caveman-ai/middleware @caveman-ai/sdk ai`
2. Start the local runtime with `CAVEMAN_MODE=compress caveman start`.
3. Wrap your options with `withCaveman(options, { runtime, scope })`.
4. Log `onReport` and look for `applied` and `reused`.

The [step-by-step switch guide](/switch/vercel-ai-sdk) covers the baseline, scope, rollback and how to read the reports. The [Vercel AI SDK middleware docs](https://docs.caveman.so/docs/sdk/middleware/vercel-ai-sdk) have a complete runnable example and a local fixture test that makes no provider calls. For measuring the change on finished tasks, see [how to measure agent cost](/guides/measure-agent-cost).

## FAQ

**Does Caveman replace the Vercel AI SDK?**
No. The AI SDK still makes every model call, runs the tool loop and streams to your UI. Caveman only changes the tool-result text inside the outbound copy.

**Does it work with `streamText` and `ToolLoopAgent`?**
Yes. `withCaveman` wraps the options object for `streamText`, `generateText` or a `ToolLoopAgent`. Keep the `tools` it returns, because replacing them after the wrap removes `caveman_retrieve`.

**Does it rewrite my stored messages?**
No. The docs' test asserts that the application's own messages and tool results keep their original bytes. Only the copy sent to the provider shrinks.

**Is this the same as Vercel AI Gateway?**
No. AI Gateway routes model calls to providers. Caveman shrinks what those calls carry. See [Caveman vs Vercel AI Gateway](/compare/vercel-ai-gateway).

**What does it cost?**
The client and adapters are MIT licensed and the runtime runs on your own machine under BSL terms. Inference is still billed by your provider as before.

## Sources

- [AI SDK introduction](https://ai-sdk.dev/docs/introduction)
- [AI SDK language model middleware](https://ai-sdk.dev/docs/ai-sdk-core/middleware)
- [AI SDK loop control](https://ai-sdk.dev/docs/agents/loop-control)
- [vercel/ai on GitHub](https://github.com/vercel/ai)
- [Caveman middleware: Vercel AI SDK](https://docs.caveman.so/docs/sdk/middleware/vercel-ai-sdk)
- [Caveman middleware: supported frameworks](https://docs.caveman.so/docs/sdk/middleware/frameworks)