---
title: "Add Caveman to OpenRouter: wrap the OpenAI client"
description: "This guide adds Caveman Middleware to an app that already calls OpenRouter through the OpenAI SDK. The key, base URL, model slugs and attribution headers stay the same. You wrap the client, pin one mo"
canonical: https://caveman.ac/switch/openrouter
last-updated: 2026-10-04
---

# Add Caveman to OpenRouter: wrap the OpenAI client

This guide adds Caveman Middleware to an app that already calls OpenRouter through the OpenAI SDK. The key, base URL, model slugs and attribution headers stay the same. You wrap the client, pin one model while you measure, and compare the two runs in your OpenRouter usage. For the comparison itself, including Auto Router, see [Caveman vs OpenRouter](/compare/openrouter).

## What do you need first?

An app using the OpenAI SDK against `https://openrouter.ai/api/v1`, and one agent task you can grade. Choose a task where a tool returns something large, such as a search result, a JSON API response or a file. Versions: TypeScript needs `openai >=7.12 <8`; Python needs Python 3.13+ and `openai >=3.10 <4`. Out of range, the adapter passes calls through untouched.

## Step 1: Start the runtime

The middleware sends eligible tool output to a local Caveman runtime, which returns the short copy and keeps the original on disk.

```bash
npm install -g @caveman-ai/cli && caveman setup --install
caveman start
```

That runs in `record` mode: it measures and changes nothing. Check it:

```bash
curl -s http://127.0.0.1:8787/caveman/v1/middleware/capabilities | jq '{mode, persistent, recovery, retention_seconds}'
```

## Step 2: Wrap the OpenRouter client

```bash
npm install @caveman-ai/middleware @caveman-ai/sdk openai
```

Build the client as you do today, and pass the same `fetch` to both the client and the adapter:

```ts
import OpenAI from "openai";
import { createMiddlewareRuntime } from "@caveman-ai/sdk/middleware";
import { withCavemanOpenAITools } from "@caveman-ai/middleware/openai";

const fetch = globalThis.fetch;
const client = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
  defaultHeaders: { "HTTP-Referer": "https://example.com", "X-OpenRouter-Title": "Research agent" },
  fetch,
});

const mode = (process.env.CAVEMAN_CLIENT_MODE ?? "compress") as "off" | "record" | "compress";
const runtime = createMiddlewareRuntime({
  endpoint: "http://127.0.0.1:8787",
  mode,
  onReport: report => console.error("caveman", JSON.stringify(report)),
});
const scope = { namespace: "research-agent", session_id: conversationId, branch_id: "main", cache_epoch: "0" };
if (mode !== "off") await runtime.ready();

const loop = withCavemanOpenAITools(client, {
  runtime, scope, fetch,
  protocol: "openai-chat",
  tools: [webSearchTool],
  functions: { web_search: webSearch },
});

const history = structuredClone(messages);
for (let step = 0; step < 8; step++) {
  const response = await loop.client.chat.completions.create({
    model: process.env.OPENROUTER_MODEL,
    messages: history,
    tools: loop.tools,
  });
  const message = response.choices[0].message;
  history.push(message);
  if (!message.tool_calls?.length) break;
  for (const call of message.tool_calls) {
    const output = await loop.functions[call.function.name](JSON.parse(call.function.arguments));
    history.push({ role: "tool", tool_call_id: call.id, content: typeof output === "string" ? output : JSON.stringify(output) });
  }
}
```

`loop.tools` is your tool list plus `caveman_retrieve`, and `loop.functions` runs both, which is how the model gets an original back when it needs a detail. `history` keeps the full tool output. Only the copy sent to OpenRouter is shortened. Keep `session_id` stable for the conversation, because a recovery handle only resolves under the scope that made it.

In Python, the same pattern uses `with_caveman_openai_tools(client, runtime=runtime, scope=scope, protocol="openai-chat", tools=[...], functions={...})` from `caveman_middleware.openai`, installed with `pip install 'caveman-sdk==1.1.0' 'caveman-middleware[openai]==0.1.0a1'`. The [Cloudflare guide](/switch/cloudflare-ai-gateway) shows a full Python loop that differs only in the base URL.

## Step 3: Pin a model and run both arms

OpenRouter can change the provider or the model between runs: provider load balancing, fallbacks, and `openrouter/auto` if you use it. Any of those can move cost more than compression does. For the test, set `OPENROUTER_MODEL` to one concrete slug and, if you want to be strict, pin the provider with OpenRouter's `provider` options (`order` plus `allow_fallbacks: false`) as described in its [provider routing docs](https://openrouter.ai/docs/guides/routing/provider-selection).

Run the task once with the runtime in `record` mode. That is your baseline, through the same code. Then restart in compress mode and run it again with a new `conversationId`:

```bash
CAVEMAN_MODE=compress caveman start
```

Same model, same prompt, same tool data.

## How do you check it worked?

Read the `onReport` output first. `applied` means a tool result was replaced, `reused` means an earlier replacement was kept so the prefix stays byte stable, and `skipped` carries a reason such as `no_candidate` or `not_smaller`. If every call is `skipped`, this task had nothing to compress.

Then compare the two runs in your OpenRouter activity: input tokens and cost per request. Add any `caveman_retrieve` calls to the candidate total, because recovery costs tokens. Keep OpenRouter's billed numbers and Caveman's local estimates in separate columns; the local ones are inferred.

Then grade the answer. If the compressed run needed more retries or got the task wrong, it didn't save anything. The [cost measurement guide](/guides/measure-agent-cost) covers the bookkeeping.

## Can you turn Auto Router back on afterwards?

Yes. Once you've measured compression on a pinned model, switch the model back to `openrouter/auto` if that's how you run in production. Two things to set. Use `allowed_models` on the `auto-router` plugin to keep the pool to models that handle tool calls well, since `caveman_retrieve` is a function tool. And read the `model` field on responses so you know which model served each call when you look at cost later.

## How do you roll back?

Set `CAVEMAN_CLIENT_MODE=off` and restart. The snippet passes it as `mode` on the runtime client and skips `ready()`, which throws in off mode. Nothing goes to the runtime, and OpenRouter receives requests exactly as your code built them. To remove Caveman entirely, call `client` with your own tool list instead of `loop.client` and `loop.tools`. The wrapper builds a new client with `withOptions`, so the original is untouched.

Restarting the runtime with plain `caveman start` stops compression but keeps measuring. If the runtime goes down, the adapter sends the original request, so your OpenRouter traffic keeps flowing.

## Where to go next

- [Caveman Middleware](https://docs.caveman.so/docs/sdk/middleware), [Python adapters](https://docs.caveman.so/docs/sdk/middleware/python) and [supported frameworks](https://docs.caveman.so/docs/sdk/middleware/frameworks)
- [OpenRouter quickstart](https://openrouter.ai/docs/quickstart) and [Auto Router](https://openrouter.ai/docs/guides/routing/routers/auto-router)
- [Caveman vs OpenRouter](/compare/openrouter) and [the model routing guide](/guides/model-routing)