Mirror of headroomlabs-ai/headroom (AI context compression proxy)
Find a file
Parideboy 3752458022
fix(cache): enforce Anthropic's 1h-before-5m cache_control ordering before forwarding (#2941)
## Description

Anthropic evaluates prompt-cache breakpoints in **one pass over the
whole request** — `tools`, then `system`, then `messages` — and rejects
the request outright when a `ttl='1h'` breakpoint appears after a
5-minute one. A bare `{"type": "ephemeral"}` marker counts as 5 minutes,
so this is easy to trip without any `ttl` field being visibly wrong:

```
API Error: 400 messages.15.content.1.cache_control.ttl: a ttl='1h' cache_control block must not
come after a ttl='5m' cache_control block. Note that blocks are processed in the following order:
tools, system, messages.
```

Headroom rewrites `cache_control` markers in several independent places,
each looking at one section, and nothing checked the invariant that
spans them. The failure mode is a dead turn, not a silent cost
regression.

Two paths can leave the forwarded body illegal today:

1. **Replayed 1h marker in a 5m request.** Claude Code picks its TTL
lane per request, not per session: the main loop asks for 1h and sends
the `extended-cache-ttl` beta header, while a side question (`/btw` in
the report) goes out in the 5m lane with bare markers and no beta
header. Headroom replays part of the previous turn's forwarded bytes
into `messages` to keep the prefix stable, and those bytes still carry
`ttl: "1h"`. `tools`/`system` at 5m, `messages` at 1h — 400.
2. **Tools breakpoint downgraded.** `inject_tool_search_deferral`
re-places the *last* marker it stripped, so a bare marker on a later
deferred tool overwrites a `ttl='1h'` one. The tools prefix goes
upstream at 5m while message breakpoints are still 1h — 400. Reported
separately as #2767.

The rule spans `tools`/`system`/`messages`, so no individual transform
is in a position to check it. The fix is a guard at the last seam before
the body goes on the wire, plus the one Python/Rust divergence that
manufactures the violation upstream of it.

Closes #2939

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/helpers.py`: new `enforce_cache_control_ttl_order`,
plus `cache_control_ttl_lane` / `cache_control_ttl_lanes` /
`walk_cache_control`. The walk visits markers in Anthropic's documented
order and matches the traversal `count_cache_breakpoints` already
performs, so the two cannot disagree about what counts as a breakpoint.
TTL ranking is ported verbatim from the Rust `TtlOrderingWalk::observe`:
absent or `"5m"` is short, `"1h"` is long, any other value is left alone
rather than guessed at. Two repairs:
- **Lane containment** — when the client sent no 1h marker of its own,
strip `ttl` from any 1h marker that leaked in. That request never sent
the `extended-cache-ttl` beta header, so it could not have written a 1h
entry anyway; nothing is lost. Other marker fields (`scope`, …) are
preserved.
- **Ordering** — when the client did ask for 1h, promote every 5m marker
preceding the last 1h one. Demoting would also make the request legal
but would discard 1h caching the client is explicitly paying for, which
is the regression #2375 / #2382 / #2651 were filed to stop. A violation
seen on the way out means headroom downgraded or introduced a marker, so
promoting restores what the client's own (legal) request asked for at
that position.
- Copy-on-write: a legal body is returned by identity, so the hot path
pays only a walk over at most a handful of markers. Kill switch
`HEADROOM_CACHE_CONTROL_TTL_GUARD=0`, matching the
`HEADROOM_TOOL_SEARCH=0` convention.
- `headroom/proxy/handlers/anthropic.py`:
- Capture the client's TTL lane from the inbound snapshot, before any
transform runs. This cannot be inferred from the session or from config
— the lane is a per-request property of Claude Code, which is the whole
reason the `/btw` case exists.
- Call the guard immediately before `log_cache_breakpoints`, i.e. after
every transform, the tool sort, the deferral, CCR injection and the
pipeline extensions. Mark the body mutated when a repair fires, and log
a WARNING carrying the repair kind, the counts and the offending
sections — the diagnostic the next report of this class will need.
- `_sort_tools_deterministically` now skips the sort when any tool
carries `cache_control`, logging `event=tool_sort_skipped
reason=marker_present`. A breakpoint on a tool means "cache through
here", so reordering changes what is inside the cached prefix and can
move a 1h-marked tool behind a 5m-marked one. The Rust proxy already
refuses for exactly this reason (`any_tool_has_cache_control` in
`crates/headroom-proxy/src/compression/live_zone_anthropic.rs:651`); the
Python path never got the same guard. Putting the check in
`_sort_tools_deterministically` rather than `_tools_for_forwarding`
covers all call sites including the batch path.
- `tests/test_cache_control_ttl_order.py`: new, 24 cases.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

The new tests validate against an independent reimplementation of
Anthropic's rule rather than against the guard's own walk, so a bug in
the walk cannot make the assertions pass. Coverage: lane classification
(bare marker is 5m, unknown TTLs are `other`); containment of a replayed
1h marker including preservation of non-`ttl` fields; promotion across
`tools`→`messages`, `system`→`messages` and within `messages`; markers
nested in `tool_result` sub-blocks; only markers before the *last* 1h
one are rewritten, so a legal 1h-then-5m ordering is left alone; legal
bodies returned by `is` identity; unknown `ttl` untouched; kill switch;
the tool sort skipping on marked tools and still sorting unmarked ones,
with one test pinning that the sort *would* have created a violation
without the guard; and an end-to-end regression running
`inject_tool_search_deferral` then the guard on the #2767 shape.

### Test Output

```text
$ pytest tests/test_cache_control_ttl_order.py -q
24 passed in 0.77s

$ pytest tests/test_cache_ttl_preserved.py tests/test_cache_control_move_bust.py \
         tests/test_cache_breakpoint_diagnostics.py tests/test_issue_746_tool_search.py -q
86 passed in 1.36s

$ pytest tests/test_cache/ tests/test_proxy/ -q
16 failed, 489 passed, 2 skipped in 91.41s

$ ruff check headroom/ tests/test_cache_control_ttl_order.py
All checks passed!

$ ruff format --check headroom/ tests/test_cache_control_ttl_order.py
1 file would be reformatted, 520 files already formatted

$ mypy --python-version 3.13 headroom --ignore-missing-imports
Found 12 errors in 3 files (checked 517 source files)
```

The three non-green results above are all pre-existing on a clean
`upstream/main` in this environment, verified by stashing the changes
and re-running:

- The 16 failures are all in
`tests/test_cache/test_client_integration.py` and are a Windows
temp-path problem in this sandbox (`OSError: [WinError 123] ...
'\\C:\\Users\\...\\Temp'`), not a code failure. They fail identically
with the branch stashed.
- `ruff format --check` flags `headroom/testing/README.md`, a docs code
block untouched by this PR.
- The mypy errors are in `headroom/memory/mcp_server.py` and
`headroom/release_version.py`; none are in the files this PR changes.
`--python-version 3.13` is needed locally because the pinned
`python_version = "3.10"` makes mypy reject the installed numpy stubs
before it checks anything.

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11, headroom at this branch's
head, run against the real `headroom.proxy` forwarding helpers. No
Anthropic API key is available in this environment, so Anthropic's
validator is reimplemented locally from its documented rule and its own
error string; the request bodies are produced by the real code path
(`_sort_tools_deterministically` then `inject_tool_search_deferral` then
the guard), not hand-written.
- Exact command / steps: build two request shapes — (A) a 5m-lane
request whose `messages` carries a replayed `ttl:"1h"` marker, the
`/btw` case; (B) 13 tools with markers on two deferred tools plus 1h
message breakpoints, the #2767 case — push each through the forwarding
helpers twice, once with `HEADROOM_CACHE_CONTROL_TTL_GUARD=0` and once
with the guard at its default, and validate the resulting body.
- Observed result: both scenarios are rejected with the issue's exact
400 when the guard is off, and both are legal with it on. Scenario A is
repaired by lane containment (the leaked 1h ttl is stripped), scenario B
by promotion (the downgraded tools breakpoint goes back to 1h). Full
output:

```text
########## Scenario A: /btw side question replays a 1h marker into a 5m request ##########

===== BEFORE (HEADROOM_CACHE_CONTROL_TTL_GUARD=0) =====
  tools.0                      5m
  system.0                     5m
  messages.1.content.0         1h
  RESULT: API Error: 400 messages.1.content.0.cache_control.ttl: a ttl='1h' cache_control block
  must not come after a ttl='5m' cache_control block. Note that blocks are processed in the
  following order: tools, system, messages.

===== AFTER (default) =====
WARNING event=cache_control_ttl_order request_id=repro-2939 repair=lane_containment demoted=1
leaked_from_section=messages; the client sent no 1h marker, so a replayed 1h breakpoint would
have been rejected upstream
  tools.0                      5m
  system.0                     5m
  messages.1.content.0         5m
  RESULT: 200 OK (request satisfies the ordering rule)

########## Scenario B: tool-search deferral downgrades the tools breakpoint ##########

===== BEFORE (HEADROOM_CACHE_CONTROL_TTL_GUARD=0) =====
INFO event=tool_sort_skipped reason=marker_present tool_count=13 marked=2
  tools.1                      5m
  messages.1.content.0         1h
  RESULT: API Error: 400 messages.1.content.0.cache_control.ttl: a ttl='1h' cache_control block
  must not come after a ttl='5m' cache_control block. Note that blocks are processed in the
  following order: tools, system, messages.

===== AFTER (default) =====
INFO event=tool_sort_skipped reason=marker_present tool_count=13 marked=2
WARNING event=cache_control_ttl_order request_id=repro-2939 repair=promote_to_1h promoted=1
first_short_section=tools first_long_section=messages; a 5m breakpoint preceded a 1h one, which
Anthropic rejects outright
  tools.1                      1h
  messages.1.content.0         1h
  RESULT: 200 OK (request satisfies the ordering rule)
```

- Not tested: no live call to `api.anthropic.com` — no credentials in
this environment — so the 400/200 above come from a local
reimplementation of the rule, not from the API itself. The reporter's
original `/btw` flow was not reproduced end to end through `headroom
wrap claude`. Cache-hit-rate impact of promoting a 5m marker to 1h was
not measured against real traffic; the reasoning for promoting over
demoting is argued above, not benchmarked. Nothing on the Rust side was
exercised.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Documentation was not updated: the new env var is a kill switch for an
internal correctness guard with no user-facing behaviour when things are
working, matching how `HEADROOM_KOMPRESS_BACKGROUND_WARM` is handled.
Happy to add a line to the env-var reference if maintainers prefer.

**One trade-off worth a maintainer's eye.** `affinity_tools` feeds a
`segment_fingerprint` used for prefix-tracker affinity. Skipping the
sort makes that fingerprint depend on the client's tool order. Clients
that mark tools must already keep a stable order for their own prefix
cache to work, so this should be safe, but it is stated rather than
assumed.

**Deliberately out of scope, flagged rather than dropped:**

- The sibling hole in `inject_tool_search_deferral`: when
`resident_has_cache_control` is already true at 5m, a dropped 1h marker
is discarded outright. That is a cost regression rather than a 400, and
it sits in the same handful of lines that the open PR #2771 rewrites, so
touching it here would conflict. Better raised on #2767.
- `TtlOrderingWalk` in `crates/headroom-core/src/cache_control.rs` is
instantiated separately per field list, so it only ever sees violations
*within* `messages`, `system` or `tools` — never across them — and it
only warns. Its module doc justifies warn-only with "Anthropic itself
accepts both orderings (just with potentially-suboptimal cache
eviction)". #2939 and #2767 both show that premise is now stale.
Changing Rust behaviour is a separate blast radius.
- `cold_prefix._cache_control_ttls` never scans `tools[]`, so a client
whose only 1h marker rides on `tools` is read as 300s. Separate bug,
separate PR.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-08-12 16:32:12 -05:00
.claude-plugin chore: release main (#2679) 2026-08-04 19:39:34 -07:00
.codegraph perf(compression): take large cold-start contexts off the synchronous kompress path (#1171) (#1298) 2026-06-23 10:48:06 -05:00
.devcontainer fix(transforms): use thread-local tree-sitter parsers to prevent pyo3 Unsendable panic (#604) 2026-06-10 18:30:00 -05:00
.github fix: publish headroom-opencode in release workflow (#2372) 2026-08-11 23:56:40 -05:00
.serena fix(wrap/serena): stop creating serena_config.yml, unbricking Serena on fresh installs (#2676) 2026-07-30 20:55:47 -07:00
benchmarks fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
crates fix(proxy): port session-sticky beta headers to the Rust proxy (#2381) 2026-08-12 00:04:18 -05:00
deploy/beacon feat(beacon): hourly R2 compaction, per-strategy savings, and a stack that reports (#2853) 2026-08-09 15:42:12 -07:00
docker fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
docs fix(observability): aggregate tool savings in OTEL (#2936) 2026-08-11 22:54:55 -07:00
e2e feat(claude): support Claude Code in VS Code (#2752) 2026-08-03 20:14:13 -07:00
examples docs(metrics): ship an importable Grafana dashboard (#2168) 2026-07-14 16:07:25 -04:00
headroom fix(cache): enforce Anthropic's 1h-before-5m cache_control ordering before forwarding (#2941) 2026-08-12 16:32:12 -05:00
plugins fix: publish headroom-opencode in release workflow (#2372) 2026-08-11 23:56:40 -05:00
REALIGNMENT fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
sbom fix(deps): remediate dependency CVEs and publish SBOM (#1509) 2026-06-27 15:28:12 -07:00
scripts fix(install): don't crash the PowerShell installer when $PROFILE is unset (#2469) 2026-08-12 00:12:30 -05:00
sdk/typescript deps: bump postcss from 8.5.19 to 8.5.25 in /sdk/typescript (#2747) 2026-08-04 21:58:58 -05:00
sql feat(telemetry): add headroom_stack and install_mode identity fields 2026-04-17 17:12:38 +02:00
tests fix(cache): enforce Anthropic's 1h-before-5m cache_control ordering before forwarding (#2941) 2026-08-12 16:32:12 -05:00
wiki fix(compress): resolve the /v1/compress tokenizer per model, and document the real contract (#2743) 2026-08-03 12:20:33 -07:00
.actrc feat: add act testing config, fix gitignore, make workflow production-ready 2026-04-15 20:28:29 -05:00
.actrc.local.example feat: add act testing config, fix gitignore, make workflow production-ready 2026-04-15 20:28:29 -05:00
.changelog.md fix: use /tmp for changelog artifact to avoid . file matching issues 2026-04-15 22:54:34 -05:00
.commitlintrc.json ci: fix smart_crusher branch CI failures + add make ci-precheck pre-push gate 2026-04-27 11:13:47 -07:00
.dockerignore fix(docker): report source build version (#1862) 2026-07-08 13:32:04 -05:00
.env.act.example feat: add act testing config, fix gitignore, make workflow production-ready 2026-04-15 20:28:29 -05:00
.env.example fix(security): patch loopback guard, retry None raise, blocking subprocess, and cache stats race 2026-06-04 16:09:53 -04:00
.git-blame-ignore-revs chore: add .git-blame-ignore-revs 2026-04-24 15:35:29 +02:00
.gitattributes chore: union-merge CHANGELOG.md to stop cross-PR merge conflicts (#2118) 2026-07-13 10:18:04 -04:00
.gitguardian.yaml fix: harden Copilot API auth token handling (#557) 2026-06-11 12:57:48 -05:00
.gitignore feat(rust): port CodeCompressor AST compressor to Rust (parity-only) (#1154) 2026-07-27 09:21:57 -07:00
.gitleaks.toml fix(ci): extend gitleaks allowlist to cover test fixtures + verified examples (#1539) 2026-06-28 13:10:58 -07:00
.pre-commit-config.yaml deps: bump ruff from 0.15.17 to 0.15.22 in the pip-minor-patch group (#2501) 2026-08-10 17:31:36 -05:00
.release-please-config.json fix: publish headroom-opencode in release workflow (#2372) 2026-08-11 23:56:40 -05:00
.release-please-manifest.json chore: release main (#2679) 2026-08-04 19:39:34 -07:00
.releasemetadata chore: release main (#2679) 2026-08-04 19:39:34 -07:00
Cargo.lock deps: bump hf-hub from 0.4.3 to 0.5.0 (#2285) 2026-08-10 17:32:59 -05:00
Cargo.toml feat(simulators): add provider simulator service (#2014) 2026-07-11 09:41:49 -07:00
CHANGELOG.md chore: release main (#2679) 2026-08-04 19:39:34 -07:00
claude_analysis_ttl.py chore: add cache TTL cost analysis script 2026-05-13 10:49:17 -07:00
CODE_OF_CONDUCT.md chore(telemetry): remove Supabase anonymous beacon; fix contact domain to headroomlabs.ai (#1526) 2026-06-27 22:48:26 -07:00
codecov.yml fix(codex): poll /wham/usage for subscription limits (handshake no longer sends x-codex-* headers) (#924) 2026-06-12 17:03:14 -05:00
CONTRIBUTING.md ci(changelog): stop the CHANGELOG cascade — release-please owns it (#2329) 2026-07-16 14:28:57 -07:00
dashboard-cache-ttl-main.png perf(proxy): bound upstream calls and hot-path costs (#2852) 2026-08-09 16:24:33 -07:00
deny.toml feat(rust): scaffold workspace + parity harness (phase-0) 2026-04-24 13:39:48 -07:00
docker-bake.hcl refactor(docker): migrate to bake with multi-variant distroless images 2026-04-04 21:51:37 +05:30
docker-compose.yml fix: remove rtk and lean-ctx CLI context tools (#2677) 2026-07-30 22:59:41 -07:00
Dockerfile Fix Dockerfile (#2337) 2026-07-17 16:12:20 -07:00
Headroom-2.gif Add demo GIF to README 2026-01-20 18:57:17 -08:00
headroom-savings.png docs: rewrite README for clarity and highlight Kompress-base, leaderboard, RTK 2026-04-18 09:36:49 -07:00
headroom_learn.gif docs: add headroom learn demo GIF to README 2026-03-07 17:56:17 -08:00
HeadroomDemo-Fast.gif Replace demo GIF with HeadroomDemo-Fast.gif 2026-04-10 15:05:25 -07:00
LICENSE Prepare for OSS release v0.2.0 2026-01-07 11:36:44 -08:00
llms.txt feat(wrap): add omp target (Oh My Pi) with models.yml override and unwrap (#1811) 2026-07-15 19:30:19 +00:00
Makefile ci(parity): make the parity harness a real per-PR gate (#2567) 2026-07-26 11:00:14 -07:00
NOTICE fix(dashboard): serve tailwind/htmx/alpine locally instead of from CDNs (#2734) 2026-08-03 06:07:27 -07:00
pyproject.toml deps: bump ruff from 0.15.17 to 0.15.22 in the pip-minor-patch group (#2501) 2026-08-10 17:31:36 -05:00
README.md docs(readme): surface the Serena opt-out in the wrap quickstart (#2790) 2026-08-04 19:22:42 -07:00
run-all-plugins.sh perf(proxy): bound upstream calls and hot-path costs (#2852) 2026-08-09 16:24:33 -07:00
rust-toolchain.toml fix(rust): clippy 1.95 unnecessary_sort_by + pin toolchain 2026-04-27 12:11:49 -07:00
RUST_DEV.md chore: remove committed node_modules + stray/internal markdown (repo hygiene) (#1528) 2026-06-27 23:32:54 -07:00
SECURITY.md chore(telemetry): remove Supabase anonymous beacon; fix contact domain to headroomlabs.ai (#1526) 2026-06-27 22:48:26 -07:00
server.json chore: release main (#2679) 2026-08-04 19:39:34 -07:00
TESTING-copilot-subscription.md fix(copilot): restore generic endpoint for non-subscription OAuth (#610) (#612) 2026-06-04 16:27:54 -07:00
uv.lock deps: bump ruff from 0.15.17 to 0.15.22 in the pip-minor-patch group (#2501) 2026-08-10 17:31:36 -05:00

  ██╗  ██╗███████╗ █████╗ ██████╗ ██████╗  ██████╗  ██████╗ ███╗   ███╗
  ██║  ██║██╔════╝██╔══██╗██╔══██╗██╔══██╗██╔═══██╗██╔═══██╗████╗ ████║
  ███████║█████╗  ███████║██║  ██║██████╔╝██║   ██║██║   ██║██╔████╔██║
  ██╔══██║██╔══╝  ██╔══██║██║  ██║██╔══██╗██║   ██║██║   ██║██║╚██╔╝██║
  ██║  ██║███████╗██║  ██║██████╔╝██║  ██║╚██████╔╝╚██████╔╝██║ ╚═╝ ██║
  ╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝╚═════╝ ╚═╝  ╚═╝ ╚═════╝  ╚═════╝ ╚═╝     ╚═╝
              The context compression layer for AI agents

6095% fewer tokens (for JSON data), 15-20% fewer tokens (for coding agents) · library · proxy · MCP · content-aware compressors · local-first · reversible

CI codecov PyPI npm Model: Kompress-v2-base License: Apache 2.0 Docs

Docs · Install · Proof · Agents · Discord · llms.txt

AI agents / LLMs: read /llms.txt here, or fetch the live index / full docs blob.


chopratejas%2Fheadroom | Trendshift

Headroom compresses everything your AI agent reads — tool outputs, logs, RAG chunks, files, and conversation history — before it reaches the LLM. Same answers, fraction of the tokens.

Headroom in action
Live: 10,144 → 1,260 tokens — same FATAL found.

What it does

  • Librarycompress(messages) in Python or TypeScript, inline in any app
  • Proxyheadroom proxy --port 8787, zero code changes, any language
  • Agent wrapheadroom wrap claude|codex|grok|copilot|cursor|aider|opencode|cline|continue|goose|openhands|openclaw|vibe|omp|zcode in one command; undo with headroom unwrap <tool>
  • MCP serverheadroom_compress, headroom_retrieve, headroom_stats for any MCP client
  • Cross-agent memory — shared store across Claude, Codex, Gemini, Grok, auto-dedup
  • headroom learn — mines failed sessions, writes corrections to CLAUDE.local.md (default, gitignored) or CLAUDE.md / AGENTS.md / GEMINI.md / GROK.md
  • Output token reduction — trims what the model writes back (not just what you send): drops ceremony/restated code and skips deep "thinking" on routine steps. See Output token reduction.
  • Reversible (CCR) — originals are cached for retrieval on demand

How it works (30 seconds)

 Your agent / app
   (Claude Code, Cursor, Codex, LangChain, Agno, Strands, your own code…)
        │   prompts · tool outputs · logs · RAG results · files
        ▼
    ┌────────────────────────────────────────────────────┐
    │  Headroom   (runs locally — your data stays here)  │
    │  ────────────────────────────────────────────────  │
    │  CacheAligner  →  ContentRouter  →  CCR            │
    │                    ├─ SmartCrusher   (JSON)        │
    │                    ├─ CodeCompressor (AST)         │
    │                    └─ Kompress-v2-base (text, HF)  │
    │                                                    │
    │  Cross-agent memory  ·  headroom learn  ·  MCP     │
    └────────────────────────────────────────────────────┘
        │   compressed prompt  +  retrieval tool
        ▼
 LLM provider  (Anthropic · OpenAI · Bedrock · …)
  • ContentRouter — detects content type, selects the right compressor
  • SmartCrusher / CodeCompressor / Kompress-v2-base — compress JSON, AST, or prose
  • CacheAligner - detects and warns about volatile content that can bust provider KV cache prefixes; never rewrites prompts
  • CCR — stores originals locally; LLM calls headroom_retrieve if it needs them

Architecture · CCR reversible compression · Kompress-v2-base model card

Get started (60 seconds)

# 1 — Install
uv tool install --python 3.13 "headroom-ai[all]"  # CLI as a global tool in a self-contained virtual env
pip install "headroom-ai[all]"                    # Python — ships the `headroom` CLI
npm install headroom-ai                           # TypeScript SDK only — no `headroom` CLI

# 2 — Pick your mode  (the `headroom` commands below come from the uv or pip install)
headroom deploy                         # turnkey local deployment + agent config
headroom wrap claude                    # wrap a coding agent
headroom proxy --port 8787              # drop-in proxy, zero code changes
# or: from headroom import compress      # inline library

# 3 — Verify setup and see the savings
headroom doctor                         # health check — confirms routing is working
headroom perf
headroom dashboard                      # live savings dashboard (proxy must be running)

To use headroom, it is recommended you launch a wrapped agent session each time so that all necessary setup is completed. When wrapping a coding agent, headroom starts a local proxy, installs Serena for semantic code navigation, and launches a coding agent session configured to proxy requests through headroom.

Serena is registered at user scope (for Claude Code, in ~/.claude.json), so it stays available in your other projects until you run headroom unwrap. To skip it entirely, wrap with --code-memory none.

The headroom CLI ships only via the PyPI package. The npm headroom-ai is the TypeScript SDK — a library you import (import { compress } from 'headroom-ai'), not a CLI, so it provides no headroom command.

Granular extras: [proxy], [mcp], [ml], [code], [memory], [vector] (optional HNSW backend — needs a C++ toolchain, not in [all]), [relevance], [image], [agno], [langchain], [evals], [pytorch-mps] (Apple-GPU memory-embedder offload — set HEADROOM_EMBEDDER_RUNTIME=pytorch_mps). Requires Python 3.10+.

Codex / global install

If Codex or another MCP client cannot inherit a shell PATH reliably, install Headroom as a persistent uv tool and point the client at the absolute binary path:

uv tool install "headroom-ai[all]"
command -v headroom

Then use the returned path in MCP config:

[mcp_servers.headroom]
command = "/absolute/path/from/command-v/headroom"
args = ["mcp", "serve"]

command = "headroom" only works when the client starts with a PATH that already includes the uv tool directory.

Proof

Savings on real agent workloads:

Workload Before After Savings
Code search (100 results) 17,765 1,408 92%
SRE incident debugging 65,694 5,118 92%
GitHub issue triage 54,174 14,761 73%
Codebase exploration 78,502 41,254 47%

Accuracy preserved on standard benchmarks:

Benchmark Category N Baseline Headroom Delta
GSM8K Math 100 0.870 0.870 ±0.000
TruthfulQA Factual 100 0.530 0.560 +0.030
SQuAD v2 QA 100 97% 19% compression
BFCL Tools 100 97% 32% compression

Reproduce: python -m headroom.evals suite --tier 1 · Full benchmarks & methodology

Output token reduction (cut what the model writes back)

Everything above shrinks the prompt you send. But you also pay for every token the model writes back — and on Opus-class models output costs 5× input. A lot of that output is waste: "Great, let me…" preambles, re-printing code you just showed it, and deep "thinking" on routine steps like reading a file.

Headroom can trim that too, from the proxy, without you changing any code:

  • Verbosity steering — appends a short "be terse, don't restate context" note to the end of the system prompt (so your prompt cache still hits).
  • Effort routing — when a turn is just the model resuming after a tool result (a file read, a passing test), it dials the model's thinking effort down. New questions and errors keep full effort.

Applies to Anthropic /v1/messages and OpenAI-compatible endpoints (/v1/chat/completions, /v1/responses). Effort routing uses reasoning_effort on OpenAI, thinking.budget_tokens / output_config.effort on Anthropic — same clamp-only invariant on both paths, same output_shaper:* label vocabulary.

Turn it on:

export HEADROOM_OUTPUT_SHAPER=1     # off by default
headroom proxy --port 8787

Already running a proxy? These switches are read live on every request, so a proxy that headroom wrap reused (rather than started) would not see a value you export afterwards — its environment was snapshotted at launch. headroom wrap now hot-syncs your current settings to the running proxy via a loopback POST /admin/runtime-env, so they take effect immediately with no restart (no cold start, no dropped requests, no lost caches). Set them before you wrap. On a shared proxy these overrides are global — the last explicit setting wins.

Learn the right terseness for you. People don't say how terse they want answers — they show it (they interrupt long replies, or move on before they could have read them). headroom learn --verbosity reads your past sessions and picks the level automatically:

headroom learn --verbosity            # preview what it found (dry run)
headroom learn --verbosity --apply    # save it; the proxy uses it from now on

See how many output tokens you saved. Output savings are counterfactual — we never see what the model would have written — so Headroom reports an honest estimate with a confidence range, never a made-up number:

headroom output-savings
# Reduction: 31.7%  (95% CI 27.7% … 35.7%)   [estimated]

Want a measured number instead of an estimate? Leave 10% of conversations unshaped as a control group: export HEADROOM_OUTPUT_HOLDOUT=0.1. The dashboard shows an Output Tokens Saved card next to input compression, labelled measured or estimated with the confidence band.

→ Full write-up incl. the measurement methodology: Output token reduction

Star History Chart

Agent compatibility matrix

Agent headroom wrap Notes
Claude Code --memory · --code-graph · --1m · --tool-search
Codex shares memory with Claude
Grok CLI routes via GROK_MODELS_BASE_URL
Cursor Manual setup starts proxy and prints base URLs for Cursor settings
Aider starts proxy + launches
Copilot CLI starts proxy + launches
VS Code Copilot transparent proxy; preserves selected model
OpenClaw installs as ContextEngine plugin
OpenCode injects config · starts proxy + launches
Cline starts proxy + injects config
Continue starts proxy + injects config
Goose starts proxy + launches
OpenHands starts proxy + launches
Mistral Vibe starts proxy + launches
Oh My Pi injects config · starts proxy + launches
Cortex Code Library only 6065% savings (library mode; no wrap)
Kimi CLI OAuth bearer forwarded — log in once
ZCode starts proxy and prints base URLs for ZCode settings

Any OpenAI-compatible client works via headroom proxy. MCP-native: headroom mcp install. Undo durable wrapping with headroom unwrap <tool> (supports: claude, copilot, codex, grok, kimi, omp, opencode, openclaw, zcode). Registry authors can use the canonical server.json in the repo root instead of reconstructing the headroom mcp serve contract from prose.

GitHub Copilot CLI subscription mode

Headroom can route GitHub Copilot CLI subscription traffic through the local proxy:

headroom copilot-auth login
headroom wrap copilot --subscription -- --model gpt-4o

This lets Headroom intercept OpenAI-compatible Copilot CLI requests and apply the same proxy compression pipeline before forwarding to GitHub Copilot's hosted API. The wrapper exchanges Headroom's reusable GitHub OAuth token for Copilot's short-lived API token and prints the upstream endpoint as COPILOT_PROVIDER_API_URL=... during launch.

headroom copilot-auth login stores a Headroom-specific Copilot OAuth token. This avoids relying on generic GitHub or Copilot CLI tokens that can read Copilot account metadata but may still be rejected by Copilot's token-exchange endpoint.

For GitHub Enterprise Server or custom-domain Copilot deployments, set one of these before launching:

export GITHUB_COPILOT_ENTERPRISE_DOMAIN=ghe.example.com
# or
export GITHUB_COPILOT_ENTERPRISE_URL=https://ghe.example.com

Both variables are supported. If both are set, GITHUB_COPILOT_ENTERPRISE_URL takes precedence.

For GitHub.com Enterprise Cloud URLs such as github.com/enterprises/your-enterprise, do not set an enterprise-domain override. Headroom uses GitHub's normal token-exchange endpoint and the Copilot API endpoint advertised for the signed-in account.

Platform support note: macOS auth reuse via Copilot CLI Keychain storage has been smoke-tested. Windows Credential Manager, Linux Secret Service / secret-tool, and Docker/CI token-injection paths are implemented or planned as auth-discovery paths, but still need real OS validation before they should be considered fully vetted. For Docker and CI, prefer passing an explicit GITHUB_COPILOT_TOKEN or GITHUB_COPILOT_GITHUB_TOKEN rather than relying on host keychain access.

GitHub Copilot in Visual Studio Code

Headroom transparently overrides Copilot's API proxy endpoint, so the normal VS Code model picker remains authoritative. GPT-5.5, GPT-5.6 Luna/Sol/Terra, Claude Sonnet/Opus, and other Copilot models keep their original model IDs while traffic passes through the local compression proxy. Headroom does not patch VS Code or change Codex settings:

headroom copilot-auth login
headroom wrap vscode

Keep the command running and use Copilot normally. Headroom holds the short-lived upstream Copilot token only in the proxy process. See the cross-platform VS Code Copilot guide for paths, credential flow, remote-development notes, undo steps, and troubleshooting.

Claude Code in Visual Studio Code

The official Claude Code extension embeds Claude Code and reads the same user settings as the CLI. Install Headroom's proxy dependencies, then run the wrapper from the project you plan to open in VS Code:

pip install "headroom-ai[proxy]"
headroom wrap vscode-claude

On the first run, reload the VS Code window. Keep the wrapper terminal running while you use the Claude Code panel; inspect the dashboard or proxy log printed at startup to see requests and savings. Headroom preserves your Anthropic authentication and selected model.

Press Ctrl+C to stop the proxy. Restart the same command before using Claude Code again, or completely restore the settings that existed before setup:

headroom unwrap vscode-claude

See the VS Code Claude Code guide for verification, configuration paths, custom profiles, remote development, and troubleshooting.

When to use · When to skip

Great fit if you…

  • run AI coding agents daily and want savings without changing your code
  • work across multiple agents and want shared memory
  • need reversible compression — originals are retrievable via CCR within the configured TTL

Skip it if you…

  • only use a single provider's native compaction and don't need cross-agent memory
  • work in a sandboxed environment where local processes can't run
Integrations — drop Headroom into any stack
Your setup Hook in with
Any Python app compress(messages, model=…)
Any TypeScript app await compress(messages, { model })
Anthropic / OpenAI SDK withHeadroom(new Anthropic()) · withHeadroom(new OpenAI())
Vercel AI SDK wrapLanguageModel({ model, middleware: headroomMiddleware() })
LiteLLM litellm.callbacks = [HeadroomCallback()]
LangChain HeadroomChatModel(your_llm)
Agno HeadroomAgnoModel(your_model)
Strands Strands guide
ASGI apps app.add_middleware(CompressionMiddleware)
Multi-agent SharedContext().put / .get
MCP clients headroom mcp install
What's inside
  • SmartCrusher — universal JSON: arrays of dicts, nested objects, mixed types.
  • CodeCompressor — AST-aware for Python, JS/TS, Go, Rust, Java, C/C++, Perl.
  • Kompress-v2-base — our HuggingFace model, trained on agentic traces.
  • Image compression — 4090% reduction via trained ML router.
  • CacheAligner - detects and warns about volatile content that can bust provider KV cache prefixes; never rewrites prompts.
  • Live-zone compression — compresses only new bytes (fresh tool output, latest turn); frozen prefix stays byte-identical so provider cache is not busted. History is never dropped.
  • CCR — reversible compression; LLM retrieves originals on demand.
  • Cross-agent memory — shared store, agent provenance, auto-dedup.
  • SharedContext — compressed context passing across multi-agent workflows.
  • headroom learn — plugin-based failure mining for Claude, Codex, Gemini.
Pipeline internals

Headroom exposes one stable request lifecycle across compress(), the SDK, and the proxy:

SetupPre-StartPost-StartInput ReceivedInput CachedInput RoutedInput CompressedInput RememberedPre-SendPost-SendResponse Received

  • Transforms do the work: CacheAligner → ContentRouter → SmartCrusher / CodeCompressor / Kompress-base (live-zone only; IntelligentContext and RollingWindow were retired in PR-B1).
  • Pipeline extensions observe or customize lifecycle stages via on_pipeline_event(...).
  • Compression hooks sit alongside the canonical lifecycle as an additional extension seam.
  • Proxy extensions remain the server/app integration seam for ASGI middleware, routes, and startup policy.

Provider and tool-specific behavior lives under headroom/providers/ so core orchestration stays focused on lifecycle, sequencing, and policy.

  • CLI/tool slices: headroom/providers/claude, copilot, codex, grok, openclaw
  • Provider runtime slices: headroom/providers/claude, gemini, plus shared backend/runtime dispatch in headroom/providers/registry.py
  • Core files stay orchestration-first: wrap.py, client.py, cli/proxy.py, and proxy/server.py delegate provider-specific env shaping, API target normalization, backend selection, and transport dispatch.

Headroom for teams

Headroom OSS is built for individual developers: run headroom proxy or headroom wrap on your laptop and start cutting tokens in minutes — free, local-first, your data never leaves your machine.

Running it across a whole engineering org is a different job: a shared, always-on deployment; centralized config and version rollout; org-wide savings dashboards; SSO and access controls; air-gapped / VPC installs; and someone to call when it matters. That's what we help companies with — self-hosted with support, or fully managed.

If your team is spending real money on LLM tokens — Claude Code, Codex, Cursor, or agents running in CI — and you want those savings across everyone, not just one laptop:

→ Email hello@headroomlabs.ai with your stack and rough monthly LLM spend, and we'll help you roll Headroom out across your organization.

Everything in this repo stays open source (Apache 2.0). The managed offering is simply for teams that would rather have it deployed, supported, and scaled for them.

Install

uv tool install --python 3.13 "headroom-ai[all]"  # CLI, isolated app env
pip install "headroom-ai[all]"                    # Python, everything — includes the `headroom` CLI
npm install headroom-ai                           # TypeScript SDK (library only — no `headroom` CLI)
docker pull ghcr.io/chopratejas/headroom:latest

Granular extras: [proxy], [mcp], [ml] (Kompress-v2-base), [code], [memory], [vector] (optional HNSW backend — needs a C++ toolchain, not in [all]), [relevance], [image], [agno], [langchain], [evals], [pytorch-mps] (Apple-GPU memory-embedder offload — set HEADROOM_EMBEDDER_RUNTIME=pytorch_mps). Requires Python 3.10+.

Note

: [all] covers the core stack but excludes framework adapters. Install them separately: pip install "headroom-ai[langchain]" (also [agno], [strands], [anyllm], [bedrock]).

Using uv for the headroom CLI? Prefer uv tool install so the command lives in an isolated app environment. On macOS, pass --python 3.13 if your default python3 is newer than the current wheel set:

brew install python@3.13  # if Python 3.13 is not already available
uv tool install --python 3.13 "headroom-ai[all]"
uv tool update-shell      # if ~/.local/bin is not already on PATH
headroom --version

For MCP clients such as Codex that do not inherit your interactive shell PATH, configure the absolute executable path returned by command -v headroom:

[mcp_servers.headroom]
command = "/Users/you/.local/bin/headroom"
args = ["mcp", "serve"]

Current native wheels cover macOS Apple Silicon and Linux. On Intel macOS, use Docker-native install until native wheel support lands.

Using pipx? Choose a supported interpreter explicitly:

pipx install --python python3.13 "headroom-ai[all]"

Pick 3.13 if you want dollar savings. The dashboard's Proxy $ Saved tile prices compression with LiteLLM, and LiteLLM can't be installed on Python 3.14+. On 3.14 token savings still track, but the dollar figure stays $0.00. If you already installed on 3.14, switch with pipx reinstall headroom-ai --python python3.13 and restart the proxy.

Installation guide — Docker tags, persistent service, PowerShell, devcontainers.

CPU requirement (x86/x86_64): the ONNX-backed features — Magika content detection and embedding relevance — use a precompiled ONNX Runtime that needs AVX2. On x86 hosts without AVX2 (some Docker/QEMU setups and older cloud VMs) Headroom automatically falls back to its non-ONNX paths (BM25 relevance, heuristic detection) rather than crashing. arm64/Apple Silicon needs no AVX2.

Updating

headroom update          # detects pip / pipx / uv tool and upgrades in place
headroom update --check  # report the latest release without upgrading
headroom update --pre    # include pre-releases

headroom update figures out how Headroom was installed (pip/venv, pip --user, pipx, uv tool) and runs the matching upgrade across macOS, Linux, and Windows. For git checkouts, editable installs, Docker images, and externally-managed system Pythons (PEP 668) it prints the correct manual step instead of guessing.

The proxy also shows a one-line "update available" notice on startup. It checks PyPI at most once a day, in the background, and never blocks. Opt out with HEADROOM_UPDATE_CHECK=off (also skipped in --stateless mode and CI).

Corporate / SSL-inspection environments

If pip install "headroom-ai[all]" fails with CERTIFICATE_VERIFY_FAILED (unable to get local issuer certificate), your network uses SSL inspection — a MITM proxy presenting a company-issued CA. The build backend (maturin) downloads rustup over a connection your TLS stack doesn't trust. Install Rust first so the build doesn't fetch it:

# macOS / Linux
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh && rustup default stable
# Windows
winget install Rustlang.Rustup && rustup default stable

Restart your shell, then pip install "headroom-ai[all]". A prebuilt wheel avoids the Rust build entirely where available: pip install --only-binary headroom-ai headroom-ai. Prebuilt wheels are published for Windows (win_amd64), Linux (x86_64 / aarch64), and macOS (Apple Silicon and Intel), so installs on those platforms never need a local Rust toolchain — the Rust-first dance above is only for the platform-independent sdist fallback when no wheel matches.

Two runtime assets are fetched over TLS; if they are blocked, trust your corporate CA via REQUESTS_CA_BUNDLE / SSL_CERT_FILE / CURL_CA_BUNDLE:

  • cdn.pyke.io — the ONNX Runtime for the Rust core. Alternatively pre-provide it with ORT_STRATEGY=system and ORT_LIB_LOCATION=/path/to/onnxruntime.
  • huggingface.co — the kompress-base compression model. Pre-download it and run with HF_HUB_OFFLINE=1, or set HF_ENDPOINT to a trusted mirror.

Running with compression disabled (pure gateway) requires neither asset.

Intel macOS (x86_64-apple-darwin): no prebuilt ONNX Runtime binary (#941)

ort-sys ships no prebuilt ONNX Runtime binary for Intel macOS, so a source build fails by default even outside a corporate-proxy environment. The same ORT_STRATEGY=system mechanism above fixes it — point it at a system ONNX Runtime instead:

brew install onnxruntime
ORT_STRATEGY=system \
ORT_LIB_LOCATION="$(brew --prefix onnxruntime)/lib" \
ORT_PREFER_DYNAMIC_LINK=1 \
  pip install "headroom-ai[all]"

# ORT is dlopen'd at runtime too:
export ORT_DYLIB_PATH="$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib"

ORT_LIB_LOCATION must point at lib/ (not the bare prefix) and ORT_PREFER_DYNAMIC_LINK=1 is required, or ORT_STRATEGY=system still attempts static linking, which the Homebrew keg doesn't provide.

"Basic Constraints of CA cert not marked critical" (Python 3.13+ strict mode)

A different failure from the one above. If TLS fails with:

[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
Basic Constraints of CA cert not marked critical

then the corporate CA is found and trusted — adding it to a CA bundle changes nothing. Python 3.13 + OpenSSL 3.x enable VERIFY_X509_STRICT by default, which enforces RFC 5280 §4.2.1.9: a CA cert's basicConstraints must be marked critical. Inspection roots like Zscaler set CA:TRUE without the critical bit, so the chain is rejected.

Set HEADROOM_TLS_STRICT=0 to clear only the strict flag from every TLS context Headroom controls — the proxy's httpx upstream client and the urllib3/huggingface_hub path used for model downloads. Chain validation, signature, expiry, and hostname checks all stay on; this is strictly narrower than disabling verification.

HEADROOM_TLS_STRICT=0 headroom proxy --port 8787

The Rust core's ONNX download (cdn.pyke.io) uses a separate TLS stack (rustls / OS trust store), unaffected by HEADROOM_TLS_STRICT. On Windows the corporate root must be in the machine certificate store (browsers already trust it there); or pre-provision ONNX Runtime with ORT_STRATEGY=system + ORT_LIB_LOCATION=/path/to/onnxruntime to skip the download entirely.

headroom learn

headroom learn in action

headroom learn — mines failed sessions, writes corrections to CLAUDE.local.md (default, gitignored; use --target CLAUDE.md for the shared team file) / AGENTS.md / GEMINI.md.

Documentation

Start here Go deeper
Quickstart Architecture
Proxy How compression works
MCP tools CCR — reversible compression
Memory Cache optimization
Failure learning Benchmarks
Configuration Limitations
Persistent installs (headroom init / headroom install apply) Savings analytics (headroom savings / headroom perf / headroom doctor)

Compared to

Headroom runs locally, covers every content type, works with every major framework, and is reversible.

Scope Deploy Local Reversible
Headroom All context — tools, RAG, logs, files, history Proxy · library · middleware · MCP Yes Yes
Compresr, Token Co. Text sent to their API Hosted API call No No
OpenAI Compaction Conversation history Provider-native No No

Stack & integrations. Headroom is the proxy — that's what we build and offer, and it compresses everything flowing through it no matter what sits upstream. Our recommended companion is Serena (installed by default when you wrap an agent) for semantic code navigation — plus Ponytail if you want leaner model output. Everything else is your call: you're free to attach your own tooling — code-memory MCP, Graphify, Caveman, or any MCP server — and Headroom compresses downstream of all of it.

Contributing

git clone https://github.com/chopratejas/headroom.git && cd headroom
uv sync --extra dev && uv run pytest

Devcontainers in .devcontainer/ (default + memory-stack with Qdrant & Neo4j). See CONTRIBUTING.md.

Community

Community projects

  • Claude Code status-line indicator — a Claude Code plugin that shows live Headroom usage in your status line: idle until headroom_compress fires, then the running total of tokens saved.

License

Apache 2.0 — see LICENSE.