---
title: "Caveman vs Pydantic AI: token compression for agents"
description: "Pydantic AI is the Python framework your agent runs on. Caveman Middleware plugs into it as a capability that shrinks large tool returns before each model request. Keep Pydantic AI for typed agents, t"
canonical: https://caveman.ac/compare/pydanticai
last-updated: 2026-10-04
---

# Caveman vs Pydantic AI: token compression for agents

Pydantic AI is the Python framework your agent runs on. Caveman Middleware plugs into it as a capability that shrinks large tool returns before each model request. Keep Pydantic AI for typed agents, tools and outputs, and add `CavemanCapability` where tools return more text than the next request needs.

## What is Pydantic AI?

Pydantic AI is a Python agent framework from the team behind Pydantic, the validation layer inside the OpenAI SDK, the Anthropic SDK and Google ADK. It is MIT licensed with about 20k GitHub stars. The pitch is "typed end to end": structured outputs, typed dependency injection and typed tools, so mistakes surface while you develop.

What it does well:

- **Model agnostic.** One Python API for OpenAI, Anthropic, Google, Bedrock, Azure, Groq, Mistral, xAI, Ollama and dozens more.
- **Capabilities.** Reusable, composable units that bundle tools, lifecycle hooks, instructions and model settings. You pass them in `capabilities=[...]`.
- **Observability.** OpenTelemetry-native instrumentation that feeds Pydantic Logfire.
- **Durable execution** on engines like Temporal, DBOS and Prefect.
- **Extras** like MCP support, Pydantic Graph for typed control flow, and Pydantic Evals.

It also takes context seriously. The core Compaction capability uses provider-native compaction on OpenAI and Anthropic. Pydantic AI Harness, the official capability library, adds model-agnostic compaction (tool-result clearing, sliding windows, LLM summarization) and Tool Output Limits, which truncate, spill to a queryable file, or summarize oversized tool returns at the source. The `ProcessHistory` capability lets you rewrite message history yourself before each request.

## Does Caveman work with Pydantic AI?

Yes. `caveman_middleware.pydantic_ai` is a native adapter for `pydantic-ai-slim >=2.42 <3`, installed with the `[pydantic-ai]` extra on Python 3.13 or newer. It is a capability, so it fits the way Pydantic AI already composes behaviour. This is the docs example:

```python
from pydantic_ai import Agent
from caveman_middleware.pydantic_ai import CavemanCapability

def build(*, model, runtime, scope):
    return Agent(model, capabilities=[CavemanCapability(runtime=runtime, scope=scope)])
```

The capability does two things. It contributes a `caveman_retrieve` tool through its own toolset, and it wraps each model request through the `wrap_model_request` hook. On every request it copies the outbound messages, sends eligible `ToolReturnPart` text to a local Caveman runtime, and swaps in the shorter version. The agent's message history keeps the originals. If the model needs a detail that was cut, it calls `caveman_retrieve` and gets the exact original back.

It works on streamed runs too, and it accepts the regular synchronous `MiddlewareRuntime`, converting it to an async view for Pydantic AI's async internals. `scope_from_run(ctx, namespace=...)` builds the scope from the run's native `conversation_id`.

Two limits to know before a trial. The adapter compresses only when the model underneath is Pydantic AI's `OpenAIChatModel` or `AnthropicModel`; other model classes pass through untouched and report `unsupported_provider`. And it shortens only tool returns that are plain strings, so a tool that returns a dict or a Pydantic model is sent as is.

For a feel of the size change: the Caveman docs send a 400-line worker log as a tool result, and the runtime counted 8,399 tokens before and 177 to 187 after across sixteen runs. Those are inferred `o200k_base` counts on one synthetic log, not a bill.

## Pydantic AI vs Caveman: feature comparison

| | Pydantic AI | Caveman Middleware |
| --- | --- | --- |
| What it is | Typed Python agent framework | Capability that shrinks tool returns before model requests |
| Languages | Python | Python and TypeScript |
| License | MIT | MIT client and adapters, BSL local runtime |
| Calls the model | Yes, across many providers | No, Pydantic AI still makes the call |
| Typed tools, outputs, deps | Yes | Uses yours |
| Context management | Compaction, Tool Output Limits (Harness), `ProcessHistory` | Shortens tool returns, keeps history intact |
| Where the cut happens | In history or at the tool source | In the outbound copy only |
| Getting cut content back | Depends on the strategy you pick | `caveman_retrieve` returns the exact original |
| Observability | Logfire via OpenTelemetry | Per-request decision reports |
| Maturity | Established, 2.x | Alpha, `0.1.0a1` |

## When should you use Pydantic AI alone?

Use it alone when tool returns are short. Caveman reports `no_candidate` or `not_smaller` and sends the original when there is nothing worth cutting.

Use it alone when your agent mostly returns structured output. The capability only offers the recovery tool on plain text-output runs with automatic tool choice. With a structured `output_type` or a forced tool choice, it does not register recovery for that request, and without recovery the request goes out uncompressed.

And if Tool Output Limits already caps your big returns and answers stay good, that may be all you need. It runs at the source and needs no extra process.

## When does adding Caveman pay off?

It pays off when the agent keeps needing its tool output. A support agent that pulls a 20k-token order history at step 2 and reasons over it for eight more steps sends it eight more times. Truncation loses the tail. Summarizing costs a model call and blurs detail. Clearing drops it. Caveman leaves the full return in the agent's history, sends a shorter copy, and keeps the exact text one `caveman_retrieve` call away.

After the first cut, Caveman resends the same shortened bytes on later requests, reported as `reused`, so the provider's cached prefix stays byte identical.

It stacks with the built-in capabilities. Compaction and history processors decide what history is sent. Caveman shrinks the tool returns inside the request that actually goes out.

If the runtime is unreachable, requests go out with the original messages. After three failures in a row the adapter skips the runtime for 30 seconds. The agent keeps running either way.

## How to add Caveman to Pydantic AI

1. `pip install caveman-sdk "caveman-middleware[pydantic-ai]==0.1.0a1"`
2. Start the runtime with `CAVEMAN_MODE=compress caveman start`.
3. Add `CavemanCapability(runtime=runtime, scope=scope)` to `capabilities`.
4. Log reports and look for `applied` and `reused`.

The [Pydantic AI switch guide](/switch/pydanticai) walks through each step with scope and rollback. The [Python middleware docs](https://docs.caveman.so/docs/sdk/middleware/python) list every Python adapter, and [how to measure agent cost](/guides/measure-agent-cost) covers judging the change on finished tasks.

## FAQ

**Does Caveman replace Pydantic AI?**
No. Pydantic AI still runs the agent, validates outputs and calls the model. Caveman only shortens tool-return text in the copy of the request sent to the provider.

**Is it a capability or a model wrapper?**
Both exist. `CavemanCapability` is the one to use, because it registers the recovery tool. `with_caveman_model` wraps a model on its own with no recovery executor.

**Does it change my message history?**
No. The adapter is copy-on-write: it builds a shortened view for the outbound request and leaves the agent's messages as they were.

**How is this different from Tool Output Limits?**
Tool Output Limits acts on the tool return at the source. Caveman leaves the return as is in history and shortens only what is sent, with the exact original recoverable by the model.

## Sources

- [Pydantic AI overview](https://pydantic.dev/docs/ai/overview/)
- [Pydantic AI capabilities](https://pydantic.dev/docs/ai/core-concepts/capabilities/)
- [Pydantic AI message history](https://pydantic.dev/docs/ai/core-concepts/message-history/)
- [pydantic/pydantic-ai on GitHub](https://github.com/pydantic/pydantic-ai)
- [Caveman middleware: Python adapters](https://docs.caveman.so/docs/sdk/middleware/python)
- [Caveman middleware: supported frameworks](https://docs.caveman.so/docs/sdk/middleware/frameworks)