---
title: "Caveman vs LangSmith: traces, evals, fewer tokens"
description: "LangSmith is where a lot of LangChain and LangGraph teams watch their agents and grade them. Caveman Middleware sits inside the same agent and makes the requests smaller by shortening big tool results"
canonical: https://caveman.ac/compare/langsmith
last-updated: 2026-10-04
---

# Caveman vs LangSmith: traces, evals, fewer tokens

LangSmith is where a lot of LangChain and LangGraph teams watch their agents and grade them. Caveman Middleware sits inside the same agent and makes the requests smaller by shortening big tool results before each model call. They do different jobs and fit together cleanly: Caveman's LangChain adapter is ordinary agent middleware, so LangSmith traces it like any other step, and LangSmith's experiment comparison is the right tool to prove the smaller requests did not cost you answers.

## What is LangSmith?

LangSmith is LangChain's platform for agent observability, evaluation and deployment. It works with any framework, with SDKs for Python, TypeScript, Go and Java and support for OpenTelemetry, and it traces apps built on the OpenAI SDK, Anthropic SDK, Vercel AI SDK or LlamaIndex as well as LangChain.

What it does well:

- **Tracing with no code change for LangChain.** Set `LANGSMITH_TRACING=true` and an API key, and every chain, graph node, model call and tool call is logged.
- **Evaluation.** Datasets, offline experiments through `evaluate()`, and online LLM-as-judge and code evaluators on production traffic.
- **Experiment comparison.** Pick two runs over the same dataset and LangSmith marks each example red where it regressed against the source experiment and green where it improved.
- **The rest of the agent stack.** Deployment, sandboxes for agent-generated code, an LLM gateway for model calls, and Fleet, a no-code agent builder.

Pricing is per seat plus usage. Developer is $0 per seat with up to 5k base traces a month. Plus is $39 per seat a month with up to 10k base traces. Base traces keep 14 days, extended traces 180. Enterprise is custom and is the only plan with self-hosted and hybrid deployment. The `langsmith` SDK is MIT licensed on GitHub; the platform is a commercial product.

## Does Caveman work with LangSmith?

Yes, with nothing to configure on the LangSmith side. Caveman ships native LangChain adapters in both languages: `caveman_middleware.langchain` for Python (`langchain >=1.4 <2`, `langgraph >=1.2 <2`) and `@caveman-ai/middleware/langchain` for TypeScript. `with_caveman_agent` takes the arguments you pass to `create_agent` and adds two things: a Caveman agent middleware and a `caveman_retrieve` tool.

On each model call, the middleware sends eligible tool results (a file read, an API response, command output) to a Caveman runtime you run beside your app, and swaps in a shorter copy for that call only. Your graph state keeps the originals. If the model needs a detail the short copy dropped, it calls `caveman_retrieve` and gets the exact original back. User messages, system prompts and errors are not touched, and inference stays with your provider.

For scale: in the Caveman docs example, a 400-line worker log went from 8,399 tokens to roughly 180 in the copy the model saw. That is an inferred `o200k_base` count on one synthetic log, not a billing figure.

## Which copy of the request does a LangSmith trace show?

With LangChain, the LLM run shows the shortened copy the model received. Caveman's middleware hands the chat model a modified request, and LangChain's tracer records the messages the model was actually invoked with. Shortened tool results start with `[caveman: shortened; exact original via caveman_retrieve handle=…]`. The tool runs and the graph state still show the full originals, so one trace holds both views.

Other LangSmith integrations see a different copy:

| Your setup | Input shown on the LLM run | Token usage on the run |
| --- | --- | --- |
| LangChain or LangGraph + Caveman LangChain adapter | Shortened copy the model read | Provider usage for that copy |
| `wrap_openai` + Caveman `with_caveman_openai_tools` | Your original messages | Provider usage for the shortened request |
| Vercel AI SDK 7 telemetry + Caveman `withCaveman` | Your original messages | Provider usage for the shortened request |

The OpenAI adapter rewrites the HTTP request below `chat.completions.create`, which is the method `wrap_openai` wraps, so LangSmith logs what you passed and reads usage from the response. Use the `with_caveman_openai_tools` form, which registers `caveman_retrieve` for a loop you own; the plain `with_caveman_openai` wrapper has no recovery tool and sends every call unchanged. Caveman returns a new client, so apply `wrap_openai` to that one, or the tracing wrapper is left on the old client.

## LangSmith vs Caveman: feature comparison

| | LangSmith | Caveman Middleware |
| --- | --- | --- |
| Main job | Trace, evaluate, deploy agents | Fewer input tokens from tool output |
| Modifies requests | No | Yes, an outbound copy per call |
| Tracing | Yes, many frameworks + OTel | No |
| Datasets and experiments | Yes | No, use LangSmith's |
| Online evaluators | Yes | No |
| LangChain integration | Native, env vars | Native agent middleware |
| Recovery of originals | Not applicable | `caveman_retrieve` per conversation |
| Self-hosting | Enterprise add-on | Runtime runs beside your app |
| License | MIT SDK, commercial platform | MIT client, BSL runtime |
| Status | Generally available | Alpha |

## How do you check Caveman did not hurt quality with LangSmith?

Run `client.evaluate` twice on the same dataset: once with the Caveman client in `record` mode and once in `compress`. `record` measures and sends every byte unchanged, so it is a clean baseline through the same code. Then open both experiments, click Compare, and set the record run as the source. Every red row is an answer that got worse.

Build the dataset from tasks where the answer hides inside a large tool result: a failing assertion in a 3,000-line CI log, one status field in a 20k-token JSON payload. Also compare total tokens for the whole task and count `caveman_retrieve` calls, since each one is a round trip that brings the original back. [The switch guide](/switch/langsmith) has the code.

## When should you use LangSmith alone?

Use LangSmith alone if your agents do not pull large tool output into context. Short chat turns and small tool results give Caveman nothing to shorten, and its reports will say `skipped` with `no_candidate`.

Also start with LangSmith alone if you have not looked at your traces yet. They will show whether the tokens go to tool output at all, or to retries, long system prompts or an evaluator you run on every call. Caveman Middleware is alpha, so pin versions and run an experiment before it touches production. Caveman has no tracing, evaluation or deployment of its own, and Caveman Platform, the hosted workspace, is not generally available.

## FAQ

**Does Caveman replace LangSmith?**
No. LangSmith shows and grades what your agent did. Caveman reduces what it sends. You keep LangSmith and add the adapter.

**Will I see the compressed text in LangSmith?**
With the LangChain adapter, yes: the LLM run shows the shortened copy and the tool run shows the original. With `wrap_openai`, the input shows your originals and only the token usage reflects the smaller request.

**Does it work with LangGraph checkpoints?**
Yes. The middleware only changes the copy sent on each call, so checkpoints keep the originals. `scope_from_config` builds the Caveman scope from your LangGraph `thread_id`.

**What about the LangSmith LLM gateway?**
Caveman has no gateway-level adapter for it. Wrap your framework or provider client with Caveman and keep the client pointed wherever it points today.

## Sources

- [LangSmith product page](https://www.langchain.com/langsmith)
- [LangChain pricing](https://www.langchain.com/pricing)
- [Trace with LangChain](https://docs.langchain.com/langsmith/trace-with-langchain)
- [Compare experiment results](https://docs.langchain.com/langsmith/compare-experiment-results)
- [Self-hosted LangSmith](https://docs.langchain.com/langsmith/self-hosted)
- [Caveman Python middleware](https://docs.caveman.so/docs/sdk/middleware/python) and [supported frameworks](https://docs.caveman.so/docs/sdk/middleware/frameworks)

Next: [add Caveman to a LangSmith setup](/switch/langsmith), [agent evaluations](/guides/agent-evaluations), [agent observability](/guides/agent-observability), and [Caveman Middleware](/products/caveman-middleware).