# Middleware format samples

These are synthetic, individual tool-result measurements through the real local
middleware HTTP runtime. They do not measure a native framework integration,
model answer quality, complete tasks, provider usage, or billed savings.

The runtime was built from a clean archive of Caveman commit
`4971c00dfa488dfac4dbc4f319ca696d8a30eadd`. This is a source snapshot, not a
published runtime release. Its identity and binary hash are in
`measurements.json`. The site did not modify the backend.

`reproduce.mjs` constructs all 13 fixtures, gets the runtime's advertised
transforms, submits one tool-result segment per fixture, and retrieves every
page of each replaced original. Exact recovery must match both the original
text and its SHA-256. Every fixture stays in the report, including no-ops.

## Reproduce

Use Node 22+ and the Go version declared by the pinned repository. In a separate
working directory:

```sh
git clone https://github.com/JuliusBrussee/caveman.git
cd caveman
git checkout 4971c00dfa488dfac4dbc4f319ca696d8a30eadd
go build -ldflags '-X main.version=4971c00-middleware-samples' -o ./sample-proxy ./proxy/cmd/caveman-proxy
```

Start an isolated process with a new empty data directory. Do not point this at
your existing runtime state. The environment below contains no provider keys:

```sh
sample_state=$(mktemp -d)
env -i PATH="$PATH" CAVEMAN_HOME="$sample_state" CAVEMAN_LISTEN=127.0.0.1:19471 CAVEMAN_MODE=compress ./sample-proxy serve
```

In another terminal, download the script from `/middleware/reproduce.mjs` on
caveman.so, then run:

```sh
node reproduce.mjs http://127.0.0.1:19471 > measurements.json
```

Stop the temporary runtime with Ctrl-C when finished. No provider calls or paid
inference are needed. Random recovery handles can change the inferred count by
a few tokens between runs. Source-only behavior can differ from a released build.

## Numbers displayed

- `before`: `plan.measurement.tokens_before`.
- `after`: `plan.measurement.tokens_after` plus `recovery_overhead_tokens`.
- Reduction: `100 × (before − after) / before`.
- The declared recovery schema contributes 62 tokens in this snapshot, even
  when a result passes through. The replacement's recovery marker is already
  included in `tokens_after`.
- Tokenizer: `o200k_base`. Basis: `inferred`. Verified dollars saved: zero.
- Future recovery calls, other prompt overhead, output tokens, cache pricing,
  retries, latency, runtime costs and answer quality are outside this measurement.
- The calculator extrapolates one sample at a visitor-supplied input rate. Its
  default $3 per million tokens is an example, not a provider price quotation.

The displayed text is an excerpt of the captured original and replacement.
The recovery control replays captured excerpts; it does not call a runtime from
the visitor's browser. Full inputs, replacements, measurements and recovery
hashes are available in `measurements.json`. Recovery handles in this historical
snapshot are not permanent retrieval links.

## Refresh the page data

After capturing a reviewed new snapshot, run from the site repository:

```sh
node scripts/sync-middleware-samples.mjs
```

This validates the snapshot and derives the compact page data without inventing
or manually copying numeric results. Preserve or update the source provenance
when refreshing measurements.