Caveman vs Langfuse: tracing plus token compression

Langfuse tells you what your agent sent, what it cost and whether the answer was any good. Caveman Middleware changes what the agent sends: it shortens large tool results before each model call. One measures and the other cuts, so the normal setup is both. Keep Langfuse, add the Caveman adapter, and use Langfuse's own experiments to prove the cut did not cost you answers.

What is Langfuse?

Langfuse is an open-source LLM engineering platform for tracing, prompt management, evaluations and experiments. The repository is MIT licensed apart from its ee folders, has about 35k GitHub stars, and self-hosts with Docker Compose, Kubernetes via Helm, or Terraform templates for AWS, Azure and GCP. ClickHouse acquired Langfuse in January 2026; both companies said Langfuse stays open source and self-hostable and that Langfuse Cloud keeps running as before.

What it does well:

• Hierarchical traces of LLM calls, tool calls and retrieval steps, with sessions for multi-turn conversations.
• Prompt management with versions, deployment labels and rollback.
• Evaluations from LLM-as-a-judge, code checks and human review.
• Datasets and experiments, where you run a task over a fixed set of inputs and compare runs side by side.
• Cost and latency dashboards. Langfuse uses the token usage your integration reports and only tokenizes the text itself when usage is missing.

Langfuse Cloud has a free Hobby tier with 50k units a month, then Core at $29 and Pro at $199 a month, with overage from $8 per 100k units. Self-hosting is free.

What does Caveman add to Langfuse?

Caveman lowers the input tokens your traces are counting. Caveman Middleware wraps the client or framework you already call. Before each provider call it replaces eligible tool results (file reads, API responses, command output, extracted documents) with a shorter copy in the outbound request, and registers a caveman_retrieve tool so the model can pull the exact original back when it needs a detail. Your app keeps its own history byte for byte. User messages, system prompts, images and errors are never touched, and inference still goes straight to your provider.

For scale: in the docs example, a 400-line worker log went from 8,399 tokens to roughly 180 in the copy the model saw. That is an inferred o200k_base count on one synthetic log, not a bill.

Adapters are published in alpha for TypeScript (@caveman-ai/middleware) and Python (caveman-middleware, Python 3.13+): OpenAI, Anthropic, Google Gen AI, LangChain and LangGraph, the Vercel AI SDK, LiteLLM, Pydantic AI, CrewAI, Mastra and more. The compression runs in a Caveman runtime you start yourself; no Caveman account is involved.

Which copy of the request does a Langfuse trace show?

It depends on where Langfuse hooks in, and it is worth knowing before you read your first trace.

The reasons come from where each tool sits. Caveman's LangChain adapter is agent middleware that hands the chat model a modified copy of the request, so the LLM run Langfuse records is that copy, marked with a [caveman: shortened; exact original via caveman_retrieve …] header. The graph state and the tool spans keep the originals.

The OpenAI adapter rewrites the HTTP request below chat.completions.create, which is the method Langfuse's drop-in patches. Use its with_caveman_openai_tools form: the plain with_caveman_openai wrapper has no recovery tool, so it sends every call unchanged. So Langfuse logs the messages you passed, while the usage on the span comes from the response and describes the smaller request. The AI SDK behaves the same way: version 7 hands telemetry the call's messages before the wrapped model applies Caveman's transform.

One practical catch on the second and third rows: if usage is missing from the response, Langfuse tokenizes the input it recorded, which is the original, and overstates what you sent. With streamed OpenAI calls, set stream_options={"include_usage": True} so the real usage lands on the span.

Langfuse vs Caveman: feature comparison

How do you check Caveman did not hurt quality in Langfuse?

Run the same Langfuse dataset experiment twice, once with the Caveman client in record mode and once in compress, and compare the scores. record measures and sends every byte unchanged, so it is an honest baseline on the same code path.

Pick items where tool output is big and the answer depends on a detail inside it: a stack trace buried in a long log, one field in a 20k-token JSON response. Those are the cases where a shortened view could drop something. Compare three things per run: the evaluator scores, total input tokens for the whole task, and how often the model called caveman_retrieve. Recovery calls are extra round trips, so they belong in the token total. If scores hold and total tokens fall, you have a real result. The switch guide has the code.

When should you use Langfuse alone?

Use Langfuse alone when your tool results are small. Chat apps with short prompts and no big tool payloads give Caveman nothing to shorten, and each call will report skipped with reason no_candidate.

Also hold off on Caveman when you have not measured yet. A week of Langfuse traces will show if your spend is really in tool output or somewhere duller: a retry loop, a judge you forgot was paid, one heavy user. And Caveman Middleware is alpha, so pin versions and run your own acceptance checks before production.

Caveman does none of Langfuse's jobs. The middleware has no trace viewer, prompt registry or alerting. Caveman Platform, the hosted workspace, is not generally available, so if you need a team workspace for traces today, Langfuse is the answer on this page.

FAQ

Does Caveman replace Langfuse? No. Caveman changes the tokens sent; Langfuse records and scores what happened. They run side by side with no data to migrate.

Will Langfuse cost numbers drop after adding Caveman? They should when compression applies, because Langfuse prices the usage your provider reports for the smaller request. Check the task total, recovery calls included, over a real experiment before you believe it.

Does Caveman send my prompts anywhere? The adapter talks to a Caveman runtime you run, on localhost by default, which stores the originals so the model can recover them. Langfuse receives whatever your Langfuse integration already sends.

Can I self-host both? Yes. Langfuse self-hosts on Docker, Kubernetes or Terraform, and the Caveman runtime is a local process or container beside your app.

Sources

• Langfuse homepage
• Langfuse pricing
• Langfuse on GitHub
• Langfuse token and cost tracking
• Langfuse experiments via SDK
• ClickHouse acquires Langfuse
• Caveman middleware docs and supported frameworks

Next: add Caveman to a Langfuse setup, agent evaluations, measure agent cost, and Caveman Middleware.