Caveman vs OpenRouter: compression and auto routing

Keep OpenRouter. Caveman works with it. Your bill is roughly price per token times tokens per task. OpenRouter works on the first half: one key, hundreds of models, provider routing that favours cheaper healthy endpoints, and an Auto Router that picks a model for you. Caveman Middleware works on the second half by shrinking the tool output your agent resends on every turn. The two multiply, so you can use both on the same call.

What is OpenRouter?

OpenRouter is one OpenAI-compatible API in front of hundreds of models from many providers. You point an OpenAI SDK client at https://openrouter.ai/api/v1, use an OpenRouter key, and call models by slug. If a provider is down or rate limited, OpenRouter falls back to another one automatically.

Provider routing is where it quietly does a lot. By default it load balances across providers for the same model, prefers ones without a significant outage in the last 30 seconds, and weights the rest by the inverse square of price. You can override that with a provider object: order, only, allow_fallbacks, sort by price, throughput or latency, data_collection, and zdr for zero data retention endpoints. A models array adds fallback models if your first choice is unavailable.

Pricing is pay as you go with no markup on the provider's price. Buying credits carries a fee (5.5% on the standard plan, per the pricing page), and bring-your-own-key usage is free up to $25,000 of list-price inference a month, then 5%. There is a free tier with 25+ free models and a daily request cap, and an enterprise tier with SSO and SLAs.

What is OpenRouter's Auto Router?

The Auto Router picks a model for each request when you set the model to openrouter/auto. A lightweight classifier assigns the prompt one of about 30 task types. The router then ranks models by what the OpenRouter community actually spends on for that task type over a trailing seven-day window, within a price band you choose with cost_tier (low, medium, high, xhigh or max). With no setting it routes roughly like low.

You can restrict the pool with allowed_models, which takes patterns like anthropic/*. There is no extra fee: you pay the rate of whichever model it picks, and the response's model field tells you which one that was. It is a good default when you don't have strong opinions yet and want a sensible, market-informed choice with zero setup.

Does Caveman work with OpenRouter?

Yes. There is no OpenRouter-specific adapter because the OpenAI one covers it. You wrap the OpenAI client you already point at OpenRouter: withCavemanOpenAITools from @caveman-ai/middleware/openai in TypeScript, with_caveman_openai_tools from caveman_middleware.openai in Python. The base URL, key and model slugs stay the same. If you use the Vercel AI SDK, LangChain or another supported framework with OpenRouter as the provider, wrap the framework instead. The frameworks page lists the adapters.

Before each call, the wrapper finds eligible tool results (file reads, API responses, command output, extracted documents) and replaces them with a shorter copy from a local Caveman runtime. Your app keeps the original history. The model gets a caveman_retrieve tool to pull back the full text when the short copy leaves out something it needs. System prompts, user messages, reasoning, errors and image or PDF bytes are not touched. Headers such as HTTP-Referer and X-OpenRouter-Title pass through unchanged.

OpenRouter vs Caveman: feature comparison

OpenRouter Auto Router vs Caveman Router

This is the one place the two products compete, so here is the straight version. Caveman Router is eval-gated model routing: it sends a call to the most efficient model in your pool that passes your own evals, and leaves traffic alone when it can't help. It is not released yet, so it is not something you can use today.

The approaches differ. OpenRouter ranks models by aggregate community spend per task type. That signal is broad and free, and it knows nothing about your tasks. Caveman Router's gate is your evals, which is closer to your quality bar and costs you the work of having evals. If you want automatic routing this week, openrouter/auto is the answer. If you already run evals and want routing decisions checked against them, read about Caveman Router and keep using OpenRouter meanwhile. The model routing guide covers when a cheaper model actually pays off, and Caveman vs Not Diamond covers another routing approach.

Should you use Auto Router and Caveman together?

You can, with one caution about measurement. If openrouter/auto picks a different model on the compressed run than on the baseline, you can't tell which change moved the cost. Pin one model while you measure compression, then turn Auto Router back on.

Recovery needs tool calls. caveman_retrieve is a normal function tool, so every model Auto Router might pick has to handle tool calling. Use allowed_models to keep the pool to models you trust with tools.

What does OpenRouter see with Caveman in the path?

OpenRouter receives, logs and bills the shortened request. Compression happens in your process before the HTTP call, so the usage on your OpenRouter account reflects the smaller input. Your app still holds every original byte. A caveman_retrieve call is an ordinary tool round trip and costs tokens, so count it.

When should you use OpenRouter alone?

Use OpenRouter alone when your prompts are short. A chat app sending a couple of thousand tokens per turn with little tool output should shop on price, and there is nothing to compress.

Use it alone while you are comparing models. One key and a full catalog make bake-offs quick. Adding compression at the same time just blurs the result.

Use it alone when your code runs somewhere that can't reach a Caveman runtime. The runtime is a process with a disk, so a serverless function needs a runtime hosted elsewhere over HTTPS, or no compression.

When does adding Caveman help?

Add Caveman when the model is settled and the bill still climbs because each turn drags a big tool result along. A research agent that pastes a 20,000-token search result into context and keeps it for eight more turns is the typical case. A cheaper model lowers the price of those tokens. Caveman lowers the count.

How much it helps depends on your traffic. The middleware docs show one 400-line log going from 8,399 to between 177 and 187 tokens, counted with o200k_base and labelled inferred. Your OpenRouter usage on a pinned model is how you get your own number.

For coding agents like Claude Code or Codex, use the Caveman Proxy instead. Middleware is for apps you build.

FAQ

Is Caveman an OpenRouter alternative? No. Caveman doesn't host models or route between providers. It compresses tool output in your app, and OpenRouter stays the API you call.

Does Caveman work with openrouter/auto? Yes. The request goes to OpenRouter as usual. Pin a model while measuring, and restrict the pool to tool-capable models so recovery works.

Does Caveman change OpenRouter's price per token? No. It reduces input tokens on eligible calls. The price per token is whatever OpenRouter charges for the model it serves.

Does Caveman break prompt caching on OpenRouter? It is designed not to. Within a conversation, Caveman reuses the replacement it chose on an earlier turn, so the prompt prefix stays byte identical from call to call.

Sources

• OpenRouter homepage and quickstart
• OpenRouter Auto Router
• OpenRouter provider routing
• OpenRouter pricing
• Caveman Middleware and supported frameworks
• Next: add Caveman to OpenRouter, reduce LLM costs