Caveman MiddlewareAlpha

Keep your stack.
Lose the extra tokens.

Your models. Your frameworks. Your tools.
Smaller context between them, with the originals still in reach.

Add to your agent Explore the savings

TypeScript & Python · Runs locally · No account needed

Captured local run

orders.json

7,202input tokens

[
  {
    "id": "order-1000",
    "customer": "customer-0",
    "region": "eu-west",
    "amount": 49,
    "status": "completed"
  },
  {
    "id": "order-1001",
    "customer": "customer-1",
    "region": "eu-west",
    "amount": 50,
    "status": "completed"
  },
  {
    "id": "order-1002",
    "customer": "customer-2",
    "region": "eu-west",
    "amount": 51,
    "status": "completed"
  },
  {
    "id": "order-1003",
    "customer": "customer-3",
    "region": "eu-west",
    "amount": 52,
    "status": "completed"
  },
  {
    "id": "order-1004",
    "customer": "customer-4",
    "region": "eu-west",
    "amount": 53,
    "status": "completed"
  },
  {
    "id": "order-1005",
    "customer": "customer-5",
    "region": "eu-west",
    "amount": 54,
    "status": "completed"
  },
  {
    "id": "order-1006",
    "customer": "customer-6",
    "region": "eu-west",
    "amount": 55,
    "status": "completed"
  },
  {
    "id": "order-1007",
    "customer": "customer-7",
    "region": "eu-west",
    "amount": 56,
    "status": "completed"
  },
  {
    "id": "order-1008",
    "customer": "customer-8",
    "region": "eu-west",
    "amount": 57,
    "status": "completed"
  },
  {
    "id": "order-1009",
    "customer": "customer-9",
    "region": "eu-west",
    "amount": 58,
    "status": "completed"
  },
  {
    "id": "order-1010",
    "customer": "customer-10",
    "region": "eu-w

With Caveman

465input tokens*

Original kept. Exact recovery verified.

93.5%fewer estimated input tokens
in this JSON sample

*Includes recovery-tool overhead. Synthetic sample, not a whole-task or billing result. Inspect all formats

Made for the stack you already use.

AI SDKOpenAIAnthropicGoogleLangChainmastra

21 adapters across two languages

A smaller change than you think

You built your agent.
Keep every part of it.

Caveman sits where tool results become model input. Your framework still runs the show. Your provider still gets the call.

Your applicationYour keys. Your control.

Your toolsFiles, APIs, search

CavemanSmaller tool results

Your modelSame provider SDK

Original history stays in your application

Keep your framework.

Same orchestration, tool execution, retries, and streaming.

Keep your models.

Same provider, credentials, and inference path.

Keep your originals.

Same saved messages and checkpoints. Compression works on the outbound copy.

Less to send. More room to work.

Small files. Big files.
See what you save.

Logs, exports, source code, and everything in between.
Select a format. Inspect the before and after.

And when it doesn’t help:

Measured synthetic samples, including recovery-tool overhead. Results vary with your content.

orders.json

JSON,
with less repetition.

An API response with repeated field names across 160 records.

−93.5%estimated input tokens

7,202 before465 with Caveman + tool

Details can be omitted from the compact view. Exact original recovery was verified for this sample.

Excerpt

[caveman: shortened; exact original via caveman_retrieve handle=cmw_e70ee2505e012351cf20a46a644d682d7dc4f0b043d3a5be]
[{"amount":49,"customer":"customer-0","id":"order-1000","region":"eu-west","status":"completed"},{"amount":50,"customer":"customer-1","id":"order-1001","region":"eu-west","status":"completed"},{"amount":51,"customer":"customer-2","id":"order-1002","region":"eu-west","status":"completed"},{"__caveman_elided__":76,"__caveman_invariants__":"all region=eu-west status=completed; customer: 32 distinct, customer-0..customer-9; id: order-1003..order-1078 all 76 present; range amount=52..127","__caveman_note__":"… caveman: elided elements resemble shown elements and match the stated invariants — answer from this view; if you truly cannot, one broad caveman_retrieve …"},{"amount":-249,"customer":"customer-15","id":"order-1079","region":"eu-west","status":"review"},{"amount":129,"customer":"customer-16","id":"order-1080","region":"eu-west","status":"completed"},{"__caveman_elided__":77,"__caveman_invariants__":"all region=eu-west status=completed; customer: 32 distinct, customer-0..customer-9; id: order-1081..order-1157 all 77 present; range amount=130..206"},{"amount":207,"customer":"customer-30","id":"order-1158","region":"eu-west","status":"completed"},{"amount":208,"customer":"customer-31","id":"order-1159","region":"eu-west","status":"completed"}]

Exact original recoveredFull data

How these savings were measured +

13 synthetic samples passed through the actual local middleware runtime. Counts are inferred o200k_base estimates, including 62 tokens for the declared recovery tool. Source build 4971c00, not a published runtime release. These are sample reductions, not promised bill savings. Task quality, later recovery calls, cache effects, and provider charges were not measured.

What counts as a file?

Text your tools return: a file read, API response, document extraction, or command output. Raw PDF bytes, images, audio, and video stay untouched. So do user messages, system instructions, reasoning, errors, and protected content.

Smaller doesn’t mean gone

Less context.
Still your content.

Your agent starts with the compact view. When the task needs more detail, it can go back to the exact original.

Recorded local run. Replay the captured excerpts.

worker.logCompact model input279 tokens

[caveman: shortened; exact original via caveman_retrieve handle=cmw_924074d9d5fd8e8a2d295a82cb7c0389f9da1b1f56a55f6a]
2026-09-20 09:00:00 INFO worker=orders health=healthy request=0 latency_ms=12
2026-09-20 09:00:01 INFO worker=orders health=healthy request=1 latency_ms=12
… 77 lines elided (caveman): all worker=orders health=healthy latency_ms=12; range request=2..78 …
2026-09-20 09:01:19 ERROR database timeout request=79 retry_after_ms=250
… 78 lines elided (caveman): all worker=orders health=healthy latency_ms=12; range request=80..157 …
2026-09-20 09:02:38 INFO worker=orders health=healthy request=158 latency_ms=12
2026-09-20 09:02:39 INFO worker=orders health=healthy request=159 latency_ms=12
… caveman: elided lines resemble shown lines and match the stated invariants — answer from this view; if you truly cannot, one broad caveman_retrieve …

Exact original available through caveman_retrieve

Stored before it shrinks.

The runtime keeps the original before making a recoverable replacement.

Read deeper when needed.

Your agent can call caveman_retrieve for the original details.

Scoped to your session.

Recovery follows your application, session, branch, and cache epoch. Handles expire; keep your own history.

Make it your numbers

Every resend adds up.
So does every token saved.

Take one measured sample. Add your volume and input price.
See the estimated input-cost difference.

Use a measured sampleTimes sent per monthInput price per 1M tokens (USD)

Original sample$216.06

With Caveman + tool$13.95

Estimated monthly input-cost reduction$202.11

6,737 tokens removed × 10,000 sends × $3 / 1M

Illustration based on one synthetic sample, not a forecast. Count every resend. The $3 default is an editable example price. Excludes output, cache pricing, recovery calls, retries, runtime costs, and quality changes.

Fits right into your code

Your next model call.
A little lighter.

Start the local runtime. Install your adapter.
Wrap the call you already make.

Check compatibility ↗

Works withVercel AI SDK OpenAI Anthropic Google GenAI LangChain Strands Mastra MCP

01
Start the local runtime.
Install the CLI and runtime once. Run compression beside your application.
```
npm install -g @caveman-ai/[email protected]
caveman setup --install
CAVEMAN_MODE=compress caveman start
```
Process & container setup ↗

Add your adapter.

TypeScript packages, pinned to the published alpha.

npm install @caveman-ai/[email protected] @caveman-ai/[email protected]

03
Wrap your existing call.
Vercel AI SDK example shown. Keep your model, tools, and messages. Use a stable conversation ID for the recovery scope.
Open the complete TypeScript quickstart ↗

agent.ts · Vercel AI SDK

01import { streamText } from "ai";02import { createMiddlewareRuntime } from "@caveman-ai/sdk/middleware";03import { withCaveman } from "@caveman-ai/middleware/ai-sdk";04 05const runtime = createMiddlewareRuntime({06  endpoint: "http://127.0.0.1:8787",07  mode: "compress",08});09await runtime.ready();10 11const scope = {12  namespace: "my-app",13  session_id: conversationId,14  branch_id: "main",15  cache_epoch: "0",16};17 18const result = streamText(19  withCaveman(existingOptions, { runtime, scope }),20);21// Consume the stream as usual.22// Close runtime on application shutdown.

AI SDK ≥7.0.94 <8 · Provider ≥4.0.11 <5

Integration excerpt: existing options, conversation IDs, and messages come from your application. Run the linked quickstart for a complete example. 21 adapters across two languages; some frameworks appear in both.

Start with your workload.
Let the results decide.

Measurement guide

record

See the candidates.

Inspect eligible results while keeping request text unchanged. Understand where compression could help.

compress

Compare complete tasks.

Turn on replacements. Measure answer quality, provider usage, latency, recovery, and retries.

off

Keep your baseline.

Disable the client for a direct comparison. Roll back whenever your workload calls for it.

What about whole-agent results? +

Our earlier Proxy wrap + Skill study reported 33.2% fewer input tokens across six tasks and three paired runs each, with 18/18 exact answers in both groups. HTML used 9.9% more tokens. This study did not test middleware. Cache buckets were summed without price weighting, so it is not an invoice-savings claim. Raw run artifacts are not public.

Published August 6, 2026. Claude Code 2.1.223, Sonnet 5. 95% interval: 14.6% to 48.5%.

Read the earlier study

Good questions

Before you
plug it in.

Read the docs

Do I need to change models or frameworks?+

No. The adapter works around your existing framework or provider client. Your application still owns model calls, tools, retries, and streaming. Check the supported version range for your adapter.

Does inference go through Caveman Cloud?+

No. With a local runtime, compression and recovery stay beside your application. Your existing SDK sends inference to your chosen provider. A remote runtime requires explicit configuration and consent.

What if the runtime is unavailable?+

In normal operation, the adapter falls back to original inference input. Explicit startup readiness checks, strict mode, cancellation, and failed recovery calls have their own error behavior.

Will every file get smaller?+

No. Content, size, recovery availability, and the runtime’s transforms decide what can be compressed. Ineligible content passes through. Recovery tooling can still add prompt overhead, so compare the complete task.

Is it free? Is it production-ready?+

Local compression needs no Caveman account. Client packages and the engine runtime are Apache-2.0. Middleware is alpha. Validate framework versions and your workload before production.

All your tools.
Less excess baggage.

Keep what you built. Add Caveman.

Add to your agent Alpha · TypeScript & Python · Local runtime

You built your agent.Keep every part of it.