Phase B step 2 of the live-zone-only realignment. Replaces PR-A1's
unconditional "passthrough" stub with a real dispatcher that
inspects the Anthropic /v1/messages body, identifies the live zone
(latest user message at index >= frozen_message_count), and routes
each block to a per-type compressor. PR-B2 wires every per-type
compressor to a no-op, so the dispatcher returns
LiveZoneOutcome::NoChange on every call — bytes-in == bytes-out.
PR-B3+ replaces the no-ops with SmartCrusher, Log, Search, Diff,
and Code compressors.
Adds:
- crates/headroom-core/src/transforms/live_zone.rs — public API:
- `compress_live_zone(body, frozen_message_count, AuthMode)`
- `LiveZoneOutcome::{NoChange, Modified}`
- `CompressionManifest` with per-block outcomes (message_index,
block_index, block_type, BlockAction).
- `BlockAction::{NoOpSkeleton, Excluded { reason }}`. The
HOT_ZONE_BLOCK_TYPES list (`tool_use`, `thinking`,
`redacted_thinking`, `compaction`) excludes blocks even when
they appear in the latest user message.
- `AuthMode::{Payg, OAuth, Subscription}` — accepted but unused
in B2; PR-F2 wires the auth-mode gate.
- 12 unit tests pin: empty messages, no messages field, invalid
JSON, latest user message selection, frozen_count respect,
hot-zone block exclusion, string-shaped content, no user msg
in live zone, AuthMode no-op, NoChange contract, manifest
counters, frozen-count clamping.
- crates/headroom-proxy/src/compression/live_zone_anthropic.rs —
new entry point. `compress_anthropic_request` parses the body,
resolves frozen_count via `resolve_frozen_count` (PR-A4 helper),
dispatches via `compress_live_zone`, and returns
`Outcome::NoCompression` on PR-B2 success / `Outcome::Passthrough
{ reason: NotJson | NoMessages | ModeOff }` on body-shape /
policy issues. Six unit tests pin: mode_off short-circuit, no
messages field, invalid JSON, valid body NoCompression,
empty body, cache_control disabled.
Modifies:
- compression/mod.rs — re-exports `compress_anthropic_request` from
`live_zone_anthropic` instead of `anthropic`. The old anthropic
module is reduced to the `resolve_frozen_count` helper only
(not deleted, because its CacheControlAutoFrozen-policy gate is
reused).
- proxy.rs — passes `state.config.cache_control_auto_frozen` into
the dispatcher. Drops the obsolete "live_zone reserved for
Phase B" warning that PR-A1 emitted on every request.
- compression/anthropic.rs — pruned to the resolve_frozen_count
helper plus its tests. The PR-A1 passthrough stub
`compress_anthropic_request` is gone (live_zone_anthropic owns
the name now).
- config.rs — `compression_mode` doc updated to reflect the wired
dispatcher (no longer "reserved for Phase B").
- tests/integration_compression.rs — `compression_decision_logged`
pins the new log contract (`decision="no_change"`,
`reason="no_op_skeleton_pr_b2"`, plus manifest fields
`frozen_message_count`, `messages_total`, `live_zone_blocks`).
Asserts the obsolete Phase A warning is NOT emitted.
- proxy.rs no longer imports CompressionMode (only used inside the
retired warning).
Benchmark cleanup (B1 leftovers that surfaced now):
- benchmarks/proxy_mode_benchmark.py + claude_session_mode_benchmark.py:
drop `intelligent_context=False` arg from ProxyConfig (the field
was retired in B1; tests/test_proxy_mode_benchmark.py and
tests/test_claude_session_mode_benchmark.py imported these
factories and started failing).
- benchmarks/bench_transforms.py: delete TestRollingWindowBenchmarks
class; rewire TestTransformPipelineBenchmarks fixture without
RollingWindow.
- benchmarks/conftest.py: drop rolling_window_config fixture.
- benchmarks/run_benchmarks.py: drop the `window` suite + table
rows referencing RollingWindow.
Cache-safety invariant:
- PR-B2 dispatcher never mutates body bytes (no-op skeleton). The
proxy forwards the original buffered bytes byte-equal. Phase A's
SHA-256 fixtures pin this.
- `passthrough_mode_live_zone_currently_passthrough_byte_equal_sha256`
retitled comment to reflect the dispatcher being live but
no-op.
Acceptance:
- cargo build --workspace + clippy + fmt: green.
- cargo test --workspace --exclude headroom-py: all green
(777 + 12 new live_zone + 6 new live_zone_anthropic tests).
- pytest: 4678 passed, 240 skipped, 0 failed.
- Anthropic decision log includes manifest fields per the
observability contract documented in
REALIGNMENT/02-architecture.md.
Per-PR-B2 plan: REALIGNMENT/04-phase-B-live-zone.md.
Phase B step 1 of the live-zone-only realignment. Removes ~10K LOC of
"drop messages from history" machinery that became unreachable after
PR-A1 made `/v1/messages` a passthrough on the proxy. Live-zone-only
compression (PR-B2..B7) operates on content blocks within messages;
message-list mutation no longer happens in the pipeline.
Python deletes:
- headroom/transforms/intelligent_context.py (1077 LOC)
- headroom/transforms/rolling_window.py (395 LOC)
- headroom/transforms/progressive_summarizer.py (508 LOC)
- headroom/transforms/scoring.py (459 LOC)
- headroom/transforms/tool_crusher.py (338 LOC)
- 5 corresponding tests/test_transforms/* and tests/test_proxy_intelligent_context.py
Rust deletes:
- crates/headroom-core/src/context/* (manager, config, workspace,
candidate, ccr_drop, strategy/, mod) + safety.rs replaced
- crates/headroom-core/src/scoring/* (mod, score, scorer, traits, weights)
- MessageScorerComparator from crates/headroom-parity (PR #338/#343
becomes deletable; sunk cost stays sunk)
- 13 message_scorer fixtures + record_message_scorer.py
Rust adds (move + rewrite):
- crates/headroom-core/src/transforms/safety.rs — `tool_pair_indices`
preserves the OpenAI/Anthropic tool_use ↔ tool_result pairing rule
the live-zone dispatcher (PR-B2) needs. No IcmConfig dependency.
Surface refactors:
- HeadroomConfig: drop `tool_crusher`, `rolling_window`,
`intelligent_context` fields; hoist `output_buffer_tokens` to top
level (used by client.py).
- ProxyConfig: drop `intelligent_context*` fields.
- `headroom wrap` proxy server: retire IntelligentContextManager
and RollingWindow imports + branch; pipeline is CacheAligner →
ContentRouter (smart_routing) or CacheAligner → SmartCrusher
(legacy).
- CLI: drop `--no-intelligent-context`, `--no-intelligent-scoring`,
`--no-compress-first` flags.
- LangChain memory integration: rename `_apply_rolling_window` →
`_apply_compression`, drop RollingWindowConfig dep. Threshold is
now advisory — B6 will rework the contract.
- TransformPipeline.create_pipeline now takes only cache_aligner_config.
- headroom/__init__.py + headroom/transforms/__init__.py: strip
exports of deleted symbols.
Bug fixes uncovered by full pytest sweep:
- providers/copilot/wrap.py: `environ or os.environ` collapsed
empty-dict to falsy → callers passing `environ={}` accidentally
pulled from os.environ. Use `environ if environ is not None else
os.environ`.
Test correctness fixes:
- _DummyAnthropicHandler._retry_request gains **_kwargs to match
the real handler signature post-A8.
- test_ws_http_fallback extracts JSON from `content=` (post-A3
byte-faithful) rather than the obsolete `json=` kwarg.
- test_ccr_response_handler_extra fixture joins SSE events with
`\n\n` per spec (post-A8 byte-buffer parser requirement).
- test_proxy_responses_phase_preservation: capture via direct
handler attached to the named logger, so the assertion is
order-independent (proxy `_setup_file_logging` flips
`headroom.propagate=False` once any earlier test triggers it).
- conftest.py autouse fixture resets `headroom.propagate=True`
before each test as a defensive measure for the same pollution.
- test_wrap_copilot_translated_backend_still_requires_byok:
monkeypatch.delenv every provider key so the BYOK error
actually fires.
- test_native_installers: skip when system bash < 4.3 (macOS ships 3.2).
- TestGeminiEmbedContent / TestGeminiBatchEmbedContents:
pytest.mark.skip — proxy currently has no :embedContent route;
feature gap, not regression.
Acceptance:
- cargo build --workspace + cargo clippy + cargo fmt --check: green.
- cargo test --workspace --exclude headroom-py: 777 passed.
- pytest: 4892 passed, 240 skipped, 0 failed.
- git grep returns only intentional comments referencing the deletion.
Per-PR-B1 plan: REALIGNMENT/04-phase-B-live-zone.md.
Codex CLI in subscription mode polls /backend-api/wham/usage, fetches
agent identity JWKS from /backend-api/wham/agent-identities/jwks, and
hits other auxiliary /backend-api/* endpoints during startup. The HTTP
catchall in _select_passthrough_base_url ignored ChatGPT auth and
routed all unmatched paths to api.openai.com, which 404s on every
backend-api path. Codex interprets that as "session invalid" and
refuses subscription auth.
Add a single branch at the top of _select_passthrough_base_url: when
_resolve_codex_routing_headers reports ChatGPT auth (explicit
ChatGPT-Account-Id header or JWT with chatgpt_account_id claim), return
https://chatgpt.com so the catchall forwards to the right host.
No-op for Anthropic (x-api-key, no JWT), Gemini (x-goog-api-key, no
JWT), OpenAI API key (sk- tokens fail JWT decode), and explicit-route
OpenAI passthroughs (/v1/embeddings, /v1/moderations, etc. don't go
through the catchall). Only behavior change is the targeted unblock for
subscription Codex.
Eliminates the Python wire-format hotfix bugs gated on Phase A's
lockdown so the proxy is safe through Phase H's Python retirement.
Bugs retired:
- P0-7 / P4-44: Codex `phase` field is now explicitly preserved
through the Responses-API ↔ Chat-Completions round-trip; multi
text-part rebuild collapses to a single text part (no more
content doubling).
- P1-8: Bytes-level SSE event splitter
`parse_sse_events_from_byte_buffer`; emoji/CJK split across
chunks survive intact. Buffer is `bytearray`; UTF-8 decode happens
only AFTER the `\n\n` event terminator is located in bytes.
Invalid UTF-8 in a *complete* event raises (operator-visible
diagnostic, not silent corruption).
- P1-9: `_parse_sse_to_response` handles all delta types per
Anthropic guide §5.1: `thinking_delta`, `signature_delta`,
`citations_delta`. Block map keyed by `index` so out-of-order
events reconstruct correctly. `redacted_thinking.data` preserved.
- P4-47: Unknown Responses-API item types now log a structured
`unknown_responses_item_type` warning so operators see new
Codex item types in flight before they break.
- P5-57: Rust proxy captures upstream `request-id` (Anthropic) and
`x-request-id` (OpenAI); surfaced as `headroom-upstream-request-id`
on the response and as a tracing span field. Distinct from the
proxy's own `x-request-id`.
- P5-59: Body-too-large now returns 413 (was 400). Pre-checks
`Content-Length` and rejects without consuming the body when
present; chunked uploads still buffer-then-fail with 413.
Configurability (no hardcodes):
- HEADROOM_SSE_BUFFER_MAX_BYTES (default 1 MiB) — per-event cap.
- HEADROOM_PROXY_BODY_TOO_LARGE_STATUS (default 413) — operator
override for body-too-large status.
A7 follow-up: `_DummyAnthropicHandler._retry_request` accepts the
A3 byte-faithful kwargs (`original_body_bytes`, `body_mutated`,
`mutation_reasons`, `request_id`, `forwarder_name`, `path_for_log`)
so the existing 20 backpressure tests stay green against the real
handler signature.
The project-wide grep
git grep 'errors="ignore"\|errors="replace"' headroom/proxy/handlers/ headroom/ccr/
returns nothing; the single remaining lossy-decode site (response-
body diagnostics, not SSE) routes through `safe_decode_for_logging`
in `headroom/proxy/helpers.py`.
Tests:
- tests/test_sse_thinking_blocks.py (4 tests)
- tests/test_sse_utf8_split.py (3 tests)
- tests/test_proxy_responses_phase_preservation.py (4 tests)
- crates/headroom-proxy/tests/integration_request_id.rs (2 tests)
- crates/headroom-proxy/tests/integration_body_size.rs (2 tests)
Closes the second half of P0-6: once memory injects memory_save / memory_search
into body["tools"] for a session, every subsequent turn injects the byte-equal
same definitions — even if memory is disabled mid-session. Toggling tool list
mid-session busts Anthropic prefix cache per guide §6.3 #2.
Adds in headroom/proxy/helpers.py:
* SessionToolTracker — bounded LRU keyed by (provider, session_id) storing
GOLDEN tool-definition bytes from the first injection. Tracker is
provider-aware so the same session_id under Anthropic and OpenAI keeps
independent state. Reentrant lock for concurrent access; LRU eviction at
HEADROOM_TOOL_TRACKER_MAX_SESSIONS (default 1000).
* apply_session_sticky_memory_tools — single coordination point with three
paths: first-time inject (record golden bytes), sticky replay (always
inject golden bytes regardless of inject_this_turn), and skip. Honors
HEADROOM_TOOL_INJECTION_STICKY=disabled as a loud operator opt-in for
rollback (NOT a fallback).
* serialize_tool_definition_canonical — deterministic byte serialization
via the same separators=(",",":")/ensure_ascii=False rules as
serialize_body_canonical.
* log_tool_injection_decision — structured per-decision log line; never
logs the tool definition contents.
Wires the helper into all four memory tool injection sites:
* handlers/anthropic.py — /v1/messages
* handlers/openai.py — /v1/chat/completions
* handlers/openai.py — /v1/responses
* handlers/openai.py — Codex WS path
memory_handler.MemoryHandler gains compute_memory_tool_definitions(provider) —
a pure builder that returns the tool definitions without mutating a tools
list, so the proxy can route through the sticky tracker. The legacy
inject_tools(...) is preserved for callers without a session_id.
Tests: tests/test_memory_tool_session_sticky.py — 29 unit + integration
cases covering: turn-1→turn-2 byte-equality (Anthropic + OpenAI), sticky
replay after memory disabled, golden-fixture pin, LRU eviction, provider
isolation under shared session_id, thread-safe concurrent access, env-var
contract, disabled-mode passthrough, dedupe with client tools.
Golden fixtures pin canonical bytes:
* tests/fixtures/memory_tool_definitions/anthropic.json
* tests/fixtures/memory_tool_definitions/openai.json
No regex. No hardcodes (env-configurable: HEADROOM_TOOL_INJECTION_STICKY,
HEADROOM_TOOL_TRACKER_MAX_SESSIONS). No silent fallbacks. Per-decision
structured logging. Realignment build constraints satisfied.
PR-A6 of the Phase A cache-safety lockdown. Eliminates P5-50 and preps
P0-6 (memory tool injection toggling).
Two cache-killer patterns the merge + tracker defeat:
1. Mid-session mutation: when memory was enabled the proxy did an
ad-hoc concat of `context-management-2025-06-27` onto the client
value (anthropic.py:1244-1248). The order varied with the client
value, breaking byte-stable headers across turns.
2. Token drop-out across turns: clients (Claude Code, Codex CLI) MAY
drop a beta token between turn N and turn N+1 even when the proxy
mutated turn N to add it. The cache hot zone is positional, so the
next turn's prefix bytes hash differently and the prefix-cache
read misses.
Changes
-------
`headroom/proxy/helpers.py`
* `merge_anthropic_beta` / `merge_openai_beta`: pure, deterministic,
order-preserving merge. Client tokens first (in their original
order), then Headroom-required tokens (in the order passed). Dedupe
is case-insensitive but preserves the original casing of the first
occurrence. No regex.
* `SessionBetaTracker`: bounded LRU keyed by (provider, session_id),
unioning client tokens with previously-seen tokens. OrderedDict
LRU; threading.RLock for thread safety (mirrors the
CompressionCache pattern from compression_cache.py).
* `get_session_beta_tracker` / `_reset_session_beta_tracker_for_test`
process-wide singleton with test reset.
* `log_beta_header_merge`: structured log per cache-affecting merge.
* Env-var knobs (NO HARDCODES):
- HEADROOM_BETA_HEADER_STICKY=enabled|disabled (default enabled).
- HEADROOM_BETA_TRACKER_MAX_SESSIONS (default 1000).
`headroom/proxy/handlers/anthropic.py`
* After `compute_session_id` (line ~744): record client
`anthropic-beta` against the session tracker, write the sticky
value back into `headers` if changed. Order matters: sticky-merge
FIRST so memory-injection has the canonical baseline.
* Memory-injection site (line ~1244): replace the ad-hoc concat with
`merge_anthropic_beta(headers["anthropic-beta"], required_tokens)`.
`headroom/proxy/handlers/openai.py`
* Chat-completions (line ~360): record/merge `openai-beta`.
* /v1/responses HTTP (line ~1213): compute `_responses_session_id`
and record/merge `openai-beta`.
* /v1/responses WS (line ~1711): replace the ad-hoc absent-only
inject with `merge_openai_beta(sticky, ["responses_websockets=
2026-02-06"])`. Replaces any case-variants of the existing key.
Tests
-----
`tests/test_anthropic_beta_session_sticky.py` (26 tests):
* Pure helper: empty inputs, only-client, only-headroom, ordering,
dedupe casing, deterministic memory-injection order, no-double-
inject when token already present.
* Tracker: sticky-on across turns even when client drops, casing
preservation, provider namespace independence, LRU eviction at
max_sessions, env-var validation (loud failures), thread safety
under 16-thread concurrent access, blank-input rejection.
`tests/test_openai_beta_session_sticky.py` (17 tests):
* Mirror of the anthropic suite for `OpenAI-Beta`.
* Plus WS-specific coverage: sticky-then-merge of
`responses_websockets=2026-02-06` against client baseline.
`tests/test_openai_codex_routing.py`
* Add `session_tracker_store` stub to `_DummyOpenAIHandler` so the
routing tests still exercise the responses HTTP handler now that
it computes a session_id for beta-merge.
Notes
-----
Build constraints honored:
* Configurable: HEADROOM_BETA_HEADER_STICKY,
HEADROOM_BETA_TRACKER_MAX_SESSIONS.
* No regex, no hardcodes (env-var bounds), no fallbacks (disabled
mode is operator opt-in for diagnostics, loud failures on invalid
values).
* Structured tracing log via `log_beta_header_merge`.
Acceptance:
* 43 new tests pass.
* `cargo test --workspace` green (no Rust changes).
* `make ci-precheck` green.
Eliminate P5-49: every Python forwarder and the Rust transparent proxy
now drop internal `x-headroom-*` request headers (`x-headroom-bypass`,
`x-headroom-mode`, `x-headroom-user-id`, `x-headroom-stack`,
`x-headroom-base-url`) before the upstream call. Stops fingerprinting
of the proxy by subscription-revocation enforcers and prevents leakage
of internal user-id / stack / base-url internals to whichever vendor
terminates the request.
Python:
- `_strip_internal_headers(headers)` in `headroom/proxy/helpers.py`
returns a NEW dict with `x-headroom-*` keys removed (case-insensitive
prefix match, no regex). Pure function. Operator opt-in
`HEADROOM_STRIP_INTERNAL_HEADERS=disabled` keeps internal headers in
the upstream-bound dict for diagnostic shadow tracing — explicit, not
a fallback.
- Strip applied at every handler entry capture in `anthropic.py`,
`openai.py`, `batch.py`, `gemini.py` (chat completions, responses,
WebSocket handshake, Copilot passthrough, batch passthroughs, Gemini
generate / stream / countTokens / cloudcode-assist, Anthropic
passthrough + batch results). Inbound reads of x-headroom (bypass
gating, memory user-id) migrated to `request.headers.get(...)` so
they continue working off the original dict.
- `log_outbound_headers` emits `event=outbound_headers forwarder=...
stripped_count=N request_id=...` per call. Never logs header values.
Rust (crates/headroom-proxy):
- `strip_internal_headers(&mut HeaderMap)` and `is_internal_header`
helpers in `src/headers.rs`. `build_forward_request_headers` accepts
a `strip_internal: bool` so the same path serves HTTP and WebSocket.
- `Config::strip_internal_headers: StripInternalHeaders` driven by CLI
flag `--strip-internal-headers` and env var
`HEADROOM_PROXY_STRIP_INTERNAL_HEADERS` (default `enabled`).
- `proxy.rs` and `websocket.rs` call `build_forward_request_headers`
with the resolved policy; structured `tracing::info!` /
`tracing::warn!` line per request describes the strip decision.
Tests: 24 Python (`tests/test_header_isolation.py`) + 4 Rust
integration (`crates/headroom-proxy/tests/integration_headers.rs`) +
4 Rust unit tests in `headers.rs`. Covers every named header
(`bypass`, `mode`, `user-id`, `stack`, `base-url`), case-insensitive
prefix matching, legitimate-headers passthrough, the `disabled`
operator-opt-in mode, and that the inbound bypass-gating read path
is unaffected by the strip.
Acceptance: targeted `pytest -x` suite green (87 tests across
test_header_isolation, test_proxy_byte_faithful_forwarding,
test_proxy_anthropic_cache_stability, test_proxy_system_prompt_immutable,
test_proxy_openai_cache_stability, test_proxy_pipeline_lifecycle).
`cargo test -p headroom-proxy` green (23 tests across all integrations
plus 7 lib unit tests). `cargo clippy -p headroom-proxy -- -D warnings`
clean. `cargo fmt --all -- --check` clean. `cargo test --workspace`
green (~900 tests total).
Per realignment build constraints: configurable (env + CLI), no
hardcodes, no regex (pure `.lower().starts_with()` match), no silent
fallbacks (`disabled` is loud operator opt-in), structured logs
(`event=outbound_headers`).
Remaining `x-headroom-` references in `headroom/proxy/handlers/` are
inbound-read sites only: `request.headers.get("x-headroom-bypass")` /
`x-headroom-mode` for behavior gating, `request.headers.get
("x-headroom-user-id")` for memory user-id resolution, and `ws_headers
.get(...)` on the WebSocket inbound path. Response-side `X-Headroom-*`
injection (e.g. `x-headroom-tokens-saved`) is unrelated to upstream
forwarding and untouched.
Eliminates P0-2 universally. Every Python forwarder (server.py
`_retry_request`, handlers/streaming.py `_stream_response`,
handlers/openai.py `_ws_http_fallback`, handlers/batch.py `_batch_passthrough`
+ batch-create + Google batch passthrough, handlers/anthropic.py CCR
continuation + batch endpoint) now switches from `httpx ... json=body` to
`httpx ... content=raw_bytes`. The default httpx JSON encoder was
re-serializing every request with `, `/`: ` separators and `\\uXXXX` ASCII
escapes — collapsing Anthropic prompt-cache hit-rate.
Forwarder strategy:
- unmutated body → forward `await request.body()` verbatim;
- mutated body → re-serialize once via the new
`serialize_body_canonical(body) -> bytes` helper (compact separators,
`ensure_ascii=False`, dict insertion order preserved).
`HEADROOM_PROXY_PYTHON_FORWARDER_MODE` env var configures the mode:
- `byte_faithful` (default) — the new behavior;
- `legacy_json_kwarg` — explicit operator opt-in for emergency rollback.
Documented in `docs/content/docs/configuration.mdx`. NOT a fallback —
unknown values raise loudly per build constraint #4.
`BodyMutationTracker` accompanies each request through the handler so
transform sites mark the tracker (`memory_injection`,
`image_compression`, `compression_*`, `batch_compression`,
`ccr_continuation`, etc.). At forwarder dispatch we additionally compare
the final body dict against the parsed original bytes as a structural
safety net — any silent mutation we missed still triggers canonical
re-serialization.
A2 follow-up: `handlers/openai.py:534-540` (Chat Completions memory
injection) was prepending a system message; replaced with
`append_text_to_latest_user_chat_message`, the OpenAI Chat Completions
analog of `_append_context_to_latest_non_frozen_user_turn`. The cache
hot zone (system messages) is now sacrosanct on /v1/chat/completions
too. Honors `HEADROOM_MEMORY_INJECTION_MODE=disabled`.
Structured logging: every forwarder emits an `event=outbound_request`
log line with `forwarder`, `path`, `body_bytes`, `body_mutated`,
`mutation_reasons`, `source` (passthrough|canonical|legacy),
`request_id`. Never logs Authorization or full body.
`_read_request_json` factored to share `_read_request_body_bytes` with
new `read_request_json_with_bytes` so the anthropic handler can capture
both the parsed dict and the original (decompressed) bytes.
Tests:
- `tests/test_proxy_byte_faithful_forwarding.py` (28 tests):
SHA-256 byte-equality on /v1/messages and streaming, unicode
preservation, numeric precision, mutation-tracker invariants,
canonical-serializer properties, legacy-mode rollback, OpenAI
Chat memory routing.
- Existing test mocks updated to accept the new `**kwargs` on
`_retry_request` (no behavior change).
- `tests/test_proxy_handlers_batch.py` updated to read the captured
`content=` bytes (formerly `json=`).
- One A2 test corrected (`test_anthropic_tool_sort_and_context_append_helpers`)
to match the live-zone-tail semantics introduced by A2.
Constraints satisfied: configurable env var; no new regex / hardcodes;
no silent fallback (`legacy_json_kwarg` is operator opt-in);
performant (`prepare_outbound_body_bytes` is O(1) for passthrough);
elegant single-responsibility helpers; structured tracing logs.
P0-1: Delete `_inject_system_context` from `proxy/server.py`. Memory
context now routes exclusively to the first text block of the latest
non-frozen user message via `_append_context_to_latest_non_frozen_user_turn`
(promoted to the canonical default in handlers/anthropic.py). Mirror
applied to OpenAI Responses API at handlers/openai.py: `body["instructions"]`
is no longer mutated; memory context appends to the latest user item in
`body["input"]`.
P2-23: Replace `headroom/transforms/cache_aligner.py` with a detector-only
implementation. The legacy rewrite path (~400 LOC) is removed. The volatile-
content detector uses no regex — UUIDs via `uuid.UUID`, ISO 8601 via
`datetime.fromisoformat`, JWT shape via base64url segment-count check, hex
hashes via length + `int(token, 16)` validation. Volatile findings surface
through `cache_metrics`/`warnings`/`logger.warning`; the prompt is never
mutated.
Configurability: new env var `HEADROOM_MEMORY_INJECTION_MODE` with values
`live_zone_tail` (default) and `disabled`. No `system_prompt` value — that
path is permanently retired.
Structured logs: every memory injection emits `event=memory_injection`
with `decision`, `bytes_injected`, `query_hash` (BLAKE2b, never raw query),
`session_id`, `request_id`. Auth is never logged.
Tests:
- Add `tests/test_proxy_system_prompt_immutable.py` (7 tests).
- Add `tests/test_cache_aligner_detector_only.py` (20 tests).
- Replace `tests/test_transforms/test_cache_aligner.py` (rewrite-path
tests, 58 cases) with detector-only behavior.
- Update `tests/test_acceptance.py::TestDateTrap` to pin the new
detector-only contract.
Acceptance:
- `git grep -n "_inject_system_context\|_inject_to_system_or_instructions" headroom/`
returns nothing.
- `git grep -n "import re\|from re import" headroom/transforms/cache_aligner.py`
returns nothing.
- Targeted suite (`test_proxy_system_prompt_immutable.py`,
`test_cache_aligner_detector_only.py`, `test_proxy_anthropic_cache_stability.py`,
`test_acceptance.py::TestDateTrap`, `test_memory*.py`, `test_cli/`) green.
PR-A4 of the Realignment Phase A lockdown
(REALIGNMENT/03-phase-A-lockdown.md). Eliminates P0-3 (Rust proxy
ignores customer cache_control markers) and P0-5 (numeric precision
lost via serde_json::Value round-trip) at the library level; Phase B
PR-B2 wires the helper into the live-zone block dispatcher.
Cargo.toml — add `arbitrary_precision` and `raw_value` to
`serde_json` workspace features. `arbitrary_precision` keeps `1.0`
from collapsing to `1` and preserves >2^53 integers; `raw_value`
exposes `&RawValue` so PR-B2 can forward unmodified `messages[*]`
entries as exact byte copies.
crates/headroom-core/src/cache_control.rs (new) — `compute_frozen_count`
walks `messages[i].content[*].cache_control` via serde_json
accessors only (no regex) and returns the smallest N such that
`messages[i]` is frozen for every i < N. Markers in `system` or
`tools[*]` log at debug! but never bump the floor (those fields are
unconditionally cache-hot per invariant I2). TTL ordering violations
(5m before 1h, guide §2.19) emit `tracing::warn!` but the function
computes the correct count regardless — the customer's request, not
ours to reject.
crates/headroom-core/src/lib.rs — re-export `compute_frozen_count` at
crate root so the proxy crate has a stable import path.
crates/headroom-proxy/src/compression/anthropic.rs — add
`resolve_frozen_count` thin wrapper that consults the
`cache_control_auto_frozen` config flag. When `disabled`, returns 0
regardless of body content (operator opt-out for benchmarking).
crates/headroom-proxy/src/config.rs — add `CacheControlAutoFrozen`
enum and the matching CLI flag `--cache-control-auto-frozen` /
env var `HEADROOM_PROXY_CACHE_CONTROL_AUTO_FROZEN`. Default is
`enabled`. Documented in the doc comments.
Tests
- crates/headroom-core/src/cache_control.rs (inline): 11 unit tests
covering marker detection, system/tools negative cases, ordering
state machine, defensive (missing fields, non-array messages,
non-object content blocks).
- crates/headroom-core/tests/cache_control.rs: 11 unit + 3 property
tests (monotonic non-decrease as markers are added; system/tools
markers don't change count; empty messages → 0).
- crates/headroom-proxy/tests/integration_cache_control.rs: 8 tests
exercising the proxy wrapper (configurability gate; tracing
capture for the 5m-before-1h warn path).
Acceptance gates: `cargo build --workspace`, `cargo test --workspace`
(33 new tests green), `cargo clippy --workspace -- -D warnings`,
`cargo fmt --all --check` all clean. No new `regex::` imports;
`git grep -n 'regex::' crates/{headroom-core/src/cache_control.rs,
headroom-core/tests/cache_control.rs, headroom-proxy/tests/
integration_cache_control.rs}` empty.
Honors the realignment build constraints: configurable (CLI + env),
no hardcodes (TTL strings live as const), no regex (serde_json
accessor walk), no fallbacks (one impl), structured logging
(debug!/warn! with field/index/ttl/rule context), tests
comprehensive (unit + property + integration + tracing capture).
Stop calling IntelligentContextManager from the Rust proxy on
/v1/messages. The proxy is now a byte-faithful passthrough on this
endpoint. Eliminates the C1+C2+C3+C4 cache-killer cluster (P0-3,
P0-4, P0-5, P1-13) by not running ICM with `frozen_message_count: 0`
hardcoded — Phase B PR-B2 brings live-zone-only compression back.
Per REALIGNMENT/03-phase-A-lockdown.md.
Changes:
- Add `--compression-mode {off,live_zone}` flag and
`HEADROOM_PROXY_COMPRESSION_MODE` env var. Default `off`. Both
modes passthrough in PR-A1; `live_zone` warns loudly because
Phase B isn't implemented yet (no silent fallback).
- Replace `compress_anthropic_request` body with a passthrough
stub that emits a structured `tracing::info!` decision log line
(request_id, path, method, compression_mode, decision,
reason="phase_a_lockdown", body_bytes) and returns
`Outcome::NoCompression`. Function signature preserved so
Phase B PR-B2 is a pure body swap.
- Delete `compression/icm.rs` (per the realignment plan: ICM
modules in headroom-core are deleted in PR-B1).
- Drop the `Arc<IntelligentContextManager>` field from `AppState`
— no longer used.
- Add request-entry `tracing::debug!` with auth_mode_placeholder
("unknown" until Phase F PR-F1 wires the auth-mode classifier).
- Add `debug_assert!` on the NoCompression branch that the
buffered bytes length is stable, locking in Phase A's
cache-safety invariant at the call site.
- Tighten existing tests from `len()` equality to SHA-256 byte
equality. Rename `compression_on_oversized_body_trims_messages`
→ `compression_on_long_body_passes_through_in_phase_a` and
flip the assertion to byte-equal.
- Add new tests: passthrough_mode_off_byte_equal_sha256,
passthrough_mode_live_zone_currently_passthrough_byte_equal_sha256,
passthrough_preserves_numeric_precision (literal-byte body so
serde_json's f64 quantization can't mask a regression),
passthrough_preserves_cache_control_markers,
passthrough_preserves_thinking_signature,
passthrough_preserves_redacted_thinking_data,
passthrough_recorded_fixture_byte_equal_sha256,
tracing_capture::compression_decision_logged.
- Add fixture
`crates/headroom-proxy/tests/fixtures/anthropic_messages_request_real.json`
with system block list + cache_control markers, tools with
nested JSON Schema, messages containing text + thinking +
signature + tool_use + tool_result + image, non-ASCII content,
large numbers. Used as the canonical SHA-256 round-trip gate.
Constraints honored: configurable (compression_mode is the only
new knob), no hardcoded thresholds, no regex usage, no silent
fallbacks (live_zone-not-implemented warns), structured tracing
on every cache-affecting decision, comprehensive tests.
Acceptance criteria from PR-A1 spec:
- `cargo build --workspace` clean
- `cargo test --workspace` green (886 tests pass)
- `cargo clippy --workspace -- -D warnings` clean
- `cargo fmt --all --check` clean
- `make ci-precheck` green
- New SHA-256 byte-equality tests pass against the recorded fixture
- `tracing::info!` decision-log line is observable
- `--compression-mode` CLI + env var work
- No regex import added
The bounded compression executor introduced in this PR moved every
handler's compression call from `asyncio.wait_for(asyncio.to_thread(...))`
to `self._run_compression_in_executor(...)`, which lives on
`HeadroomProxy` (server.py) and is inherited by handler mixins at
runtime.
The test's `_DummyOpenAIHandler` only inherits `OpenAIHandlerMixin`,
not `HeadroomProxy`, so it lacks the method. The Responses API
compression path caught the AttributeError and silently fell back —
which made `test_handle_openai_responses_stream_keeps_compression`
fail with `apply.call_count == 0`.
Add a synchronous stub that just invokes the callable; tests don't
need real thread-pool semantics.
Adds an opt-in compression interceptor that buffers Anthropic
/v1/messages requests, runs IntelligentContextManager over the
messages array, and forwards the (possibly trimmed) body upstream.
All other paths, methods, and content-types stay on the original
streaming passthrough — so existing operators see zero change.
Behaviour gates ALL must be true to buffer + compress:
- --compression flag (or HEADROOM_PROXY_COMPRESSION=1)
- method == POST
- path == /v1/messages
- Content-Type: application/json
- ICM constructed successfully at startup
Falls through to streaming on any failure: parse, missing fields,
unknown model, body-too-large. Compression must never break a
request — that's the safety contract.
Model context windows come from a vendored LiteLLM snapshot at
crates/headroom-proxy/data/model_prices_and_context_window.json
parsed once into an OnceLock<HashMap>. Refresh via
scripts/refresh_model_limits.sh. Rationale documented inline:
hardcoded tables silently rot; LiteLLM is the canonical source
the entire LLM-tooling ecosystem relies on.
New tests:
- 16 unit tests across compression::{anthropic, icm, model_limits}
- 5 integration tests: off-passthrough, on-short-passthrough,
on-oversized-trim, on-non-json-skip, on-non-llm-path-skip
Verification:
- cargo test --workspace -> 884 passed, 0 failed
- cargo clippy --workspace -- -D warnings -> clean
- cargo fmt --check -> clean
Three audit follow-ups from issue #327's deep-dive review.
C1 — CompressionCache concurrency lock
======================================
`CompressionCache` instances are shared per `session_id` and accessed from
async-dispatched threadpool workers. Pre-fix, concurrent requests for the
same session raced on `_cache`, `_stable_hashes`, `_first_seen`, and
`_total_tokens_saved` with no synchronization. Observable failures:
* Lost-update on `_total_tokens_saved` (read-modify-write).
* `RuntimeError: OrderedDict mutated during iteration` from `apply_cached`
when a concurrent `store_compressed` evicts during the walk.
* Lost stable-hash records — next-turn compute_frozen_count reads
inconsistent state.
May also explain part of SvenMeyer's `_cache: 0 entries / 1003 misses`
observation: the cache was being clobbered concurrently.
Added `threading.RLock` guarding all mutating methods. `RLock` (not `Lock`)
so future code can call locked methods from inside another locked method
without self-deadlock. Also locked `HeadroomProxy._compression_caches`
dict-of-caches access via a separate `_compression_caches_lock` so two
concurrent calls for the same session_id can't each create distinct
CompressionCache objects (which would split the cache state between them).
The `/stats` endpoint snapshots the cache list under the dict lock before
iterating to avoid eviction-during-iteration.
C2 — Multi-worker CCR fragmentation: documented + startup warning
=================================================================
The in-memory `InMemoryCcrStore` (Rust), `_compression_caches` (Python),
`session_tracker_store` (Python), and TOIN learner state are ALL
per-process. Multi-worker uvicorn round-robins requests across workers,
so a session whose turn-1 lands on worker A may have turn-2 land on
worker B. Worker B has zero knowledge of A's CCR markers, replay cache,
or prefix-cache state. Result: `Retrieve original: hash=X` markers stay
in-context as opaque directives, every fresh tool_result is recompressed
from scratch, and `frozen_message_count=0` causes Anthropic prefix-cache
busts on every cross-worker turn.
Added a "Multi-worker deployment — CCR fragmentation" section in
`RUST_DEV.md` documenting the failure modes, the supported configuration
(`--workers 1`), and the sticky-session workaround for horizontal scale.
The proxy emits a `WARNING`-level log line on startup if `workers > 1` is
detected, pointing at the doc section.
C3 — Bounded compression executor with cancel-aware metrics
===========================================================
`asyncio.wait_for(asyncio.to_thread(pipeline.apply), timeout=...)`
cancellation does NOT propagate into the threadpool worker that's running
Rust code. Once the worker has picked up the task,
`concurrent.futures.Future.cancel()` returns False and the thread runs to
completion. Stuck threads accumulated invisibly on asyncio's default
executor, contending with unrelated `to_thread` callers (file IO, etc.).
Replaced all 7 `asyncio.to_thread` call sites for `pipeline.apply()`
across `proxy/handlers/anthropic.py` (3) and `proxy/handlers/openai.py` (4)
with a new `HeadroomProxy._run_compression_in_executor(fn, *, timeout)`
helper that:
1. Submits to a dedicated bounded `ThreadPoolExecutor` named
`headroom-compress` (configurable via
`ProxyConfig.compression_max_workers`; defaults to
`min(32, (cpu_count or 1) * 4)`).
2. Increments `_compression_in_flight` (gauge) when work starts and
decrements when work completes; tracks `_compression_in_flight_max`
as a high-water mark.
3. Detects "leaked threads" by comparing wall-clock elapsed against the
timeout in the worker's `finally` block. Increments
`_compression_leaked_threads` when a worker finishes after its
asyncio future was cancelled. Operators can see the leaked-thread
rate climbing in `/stats runtime.compression_executor` BEFORE the
pool fills up.
Tests
=====
* `TestCompressionCacheConcurrency` (3 tests) — many threads
store_compressed / apply_cached / update_from_result on a single
CompressionCache; assert no exceptions, no lost updates, no partial
state.
* `test_get_compression_cache_returns_same_instance_under_contention` —
32 concurrent `_get_compression_cache(same_id)` calls return the
identical instance (would split pre-lock).
* `test_proxy_compression_executor.py` (8 tests) — pool size respects
config, in-flight gauge tracks running compressions, high-water mark
is monotonic, timeout propagates to awaiter, leaked-thread counter
increments on post-deadline completion, `/stats` surfaces all three
gauges.
Verification
============
* All 123 targeted regression tests pass.
* `make ci-precheck` clean.
* No `Co-Authored-By` trailer; conventional `fix:` prefix; no
`--no-verify`.
The @1.95.0 git ref of dtolnay/rust-toolchain shipped action code
that errors on ubuntu-latest with:
failed to install component: 'clippy-preview-x86_64-unknown-linux-gnu',
detected conflict: 'bin/cargo-clippy'
The runner's pre-installed Rust ships cargo-clippy at $HOME/.cargo/bin,
and the older action code's rustup invocation hits a path conflict
when adding the clippy-preview component for 1.95.0.
The @stable ref of the action has the fix; pass toolchain: 1.95.0 as
input so the version stays pinned. rust-toolchain.toml continues to
be the source of truth for the version (used by cargo's
auto-detection); this keeps the action's install in sync.
The Anthropic token-mode handler walked past prefix_tracker.frozen_message_count
whenever an upcoming tool_result's content-hash matched comp_cache._stable_hashes
or should_defer_compression returned True. That conflated content equality with
positional cache membership.
Anthropic's prefix cache is POSITIONAL: bytes 0..K cached, anything past K is
fresh. _stable_hashes is content-keyed and grows unbounded. In long Claude Code
sessions where tool_result content rhymes across turns (repeated system prompts,
repeated file reads, repeated tool descriptions), the walker advanced
frozen_message_count to len(messages) on every turn and the pipeline produced
transforms_applied=[] on 73% of requests in user SvenMeyer's reported session
(headroom-stats-2026-05-01.json: 74 of 101 eligible requests "prefix_frozen") —
even after the prior fix in 44944fb. The 15 requests that did compress averaged
21%, proving compression itself works when reached.
Fix: delete the walker. The freeze boundary is now
frozen_message_count = min(
prefix_tracker.frozen_message_count, # positional ground truth
comp_cache.compute_frozen_count(messages), # local cache lower bound
)
compute_frozen_count's use of _stable_hashes can only LOWER the freeze via the
min clamp, never raise it past prefix_tracker's value. For any position in the
gap [compute_frozen_count, prefix_tracker.frozen_count], recompressing produces
byte-stable output (compression is deterministic on input content), so
Anthropic's prefix cache stays valid.
Cross-handler verification:
* OpenAI handler (proxy/handlers/openai.py:358-382) does not have this walker
— uses only compute_frozen_count. Codex routes through OpenAI handler. Both
unaffected.
* Streaming and non-streaming both invoke anthropic_pipeline.apply() before the
upstream call. One fix covers both paths.
* Cache mode (is_cache_mode) takes the _extract_cache_stable_delta path and is
independent of the walker. Unaffected.
Tests: six new regression tests lock down the post-fix invariants — clamp to
min(prefix_tracker, compute_frozen_count); fresh tool_result whose hash matches
old _stable_hashes entry is not frozen; frozen prefix byte-stable across the
pipeline; 10-turn session produces non-empty compression suffix every turn;
streaming and non-streaming compute identical frozen_message_count; OpenAI
handler never calls the walker functions. Plus scripts/smoke_issue_327.py
(gated by RUN_LIVE_API=1) drives a 10-turn conversation against
api.anthropic.com in both shapes (string + list-of-blocks) and both modes
(streaming + non-streaming).
ci-precheck clean. 191 tests pass.
Follow-ups (separate PRs):
* Fix _cache perpetually empty (anthropic.py result.messages != working_messages
comparison rarely fires in token mode).
* Cap _stable_hashes with bounded LRU + 1h TTL — hygiene only after the freeze
gate is removed.
* List-shape tool_result content gates at content_router.py:1975 and
intelligent_context.py:657 (cluster A from the audit).
Direct port of `headroom.transforms.scoring.MessageScorer` (459 LOC).
Foundation piece for the IntelligentContext port (PR-B onward).
What's wired:
- Deterministic factors fully ported: recency (exp-decay), forward
references (tool_call_id graph), token density (unique/total).
- External-dep factors gated behind traits: `EmbeddingProvider` and
`ToinProvider`. No concrete impls yet — both default to neutral
values matching Python's `embedding_provider=None` / `toin=None`.
PR-A1 wires fastembed; PR-A2 plugs in a PyO3 ToinProvider.
- ScoringWeights + MessageScore with serde + BTreeMap-ordered
breakdown for stable JSON.
Parity:
- 13 fixtures recorded from Python, byte-equal under the comparator.
- Floats rounded to 5 decimals on both sides — absorbs f32-vs-f64
drift in the weighted sum without masking real bugs.
Drive-by: re-fix three pre-existing clippy errors in
smart_crusher/crusher.rs that re-emerged with new test additions
(field_reassign_with_default + dead hash_array_for_ccr).
Closes findings from the post-Phase-3g audit. Five surgical fixes
plus telemetry-discoverability docs. PyO3 0.22 → 0.24 security
upgrade is its own PR (issue #335).
1. DiffCompressor cache_key persistence (production bug)
---------------------------------------------------------
Pre-fix: `RustDiffCompressor.compress()` minted a `cache_key`,
embedded `[... hash=abc123]` in the wire marker, and returned
without storing the original anywhere. Python ContentRouter then
returned the compressed text with a dangling marker — every
retrieval tool call from the LLM 404'd.
Sibling compressors (LogCompressor, SearchCompressor) already had
the right pattern: Rust mints the key, Python's
`_persist_to_python_ccr` writes the original to the production
`CompressionStore`. DiffCompressor was the asymmetric one.
Fix:
- Rust: add `DiffCompressor::compress_with_store(content, context,
Option<&dyn CcrStore>)` mirroring siblings. Calls `store.put`
when a key is minted; legacy `compress()` and
`compress_with_stats()` delegate with `None` for parity.
- Python: add `_persist_to_python_ccr` helper to
`headroom/transforms/diff_compressor.py.compress()` mirroring
`log_compressor.py` and `search_compressor.py`.
- Pipeline `DiffOffload`: switch to `compress_with_store(Some(store))`
and drop the post-hoc double-store hack that papered over this
bug at the orchestrator boundary.
2. CCR store TOCTOU race in `get()`
-----------------------------------
`InMemoryCcrStore::get()` checked TTL under a read lock, dropped
the lock, then called `remove()`. Between drop and remove a
concurrent `put()` of the same hash with fresh data could land —
and our `remove` would then wipe that fresh entry. Under
multi-worker proxy load this manifested as "I just stored it; why
is it gone?"
Fix: use `DashMap::remove_if`. Predicate runs under the shard
write lock so check-and-remove is atomic. New regression test
exercises a tight contention loop between writer and reader on
the same key.
3. Pre-existing clippy debt in smart_crusher
--------------------------------------------
- 3× `field_reassign_with_default` in `crusher.rs` test setup —
switch to struct-update syntax `Config { field: x, ..Default }`.
- `hash_array_for_ccr` was `#[cfg(test)]` but unused; deleted with
a comment so a future test can reintroduce it as a one-liner.
`cargo clippy --workspace --all-targets -- -D warnings` is now
clean across the whole workspace; previous CI patches that allowed
these warnings can be removed in a follow-up.
4. Tokenizers dependency dedup
------------------------------
`tokenizers 0.21` (direct dep) + `tokenizers 0.22` (transitive via
fastembed) compiled twice into the binary. Bumped direct dep to
`0.22` to align; API is compatible (verified by full tokenizer
test suite). Saves compile time + binary bloat.
5. Telemetry-discoverability doc (no new code)
----------------------------------------------
The audit recommended a per-transform invocation counter to
inform the next Python → Rust port. Discovered the infrastructure
already exists at `/stats`:
- `compressions_by_strategy` — invocation count per strategy
- `pipeline_timing` — count + avg/max ms per transform name
- `tokens_saved_by_strategy` — savings attribution
Added a section to `RUST_DEV.md` showing the `curl + jq` recipes
to read this data, with example output highlighting how to spot
zero-invocation deferral candidates (e.g. `code_compressor`).
Verification: workspace tests 734 + 14 + 5 + 4 + 6 + 5 + 2 + 2 +
3 + 4 + 2 + 2 + 1 = all green; cargo fmt clean; cargo clippy
--all-targets clean; Python tests 185 pass; commitlint clean.
The proxy startup chain eagerly loads `headroom.memory.adapters`, which
imports `ports.py` and `sqlite.py`. Both files declared `import numpy as np`
at module top even though numpy is not in base dependencies (it is only
under `[all]`, `[dev]`, `[evals]`, and `[relevance]` extras). As a result,
`pipx install 'headroom-ai[mcp]' && headroom proxy` fails with
`ModuleNotFoundError: No module named 'numpy'` even when memory features
are disabled (the default).
Both files already use `from __future__ import annotations`, so the
`np.ndarray` type hints are strings at runtime - numpy is only needed
for static type checking. Move the module-level `import numpy as np`
under a `TYPE_CHECKING` guard. For the only runtime users
(`SQLiteMemoryStore._serialize_embedding` and `_deserialize_embedding`),
add a local `import numpy as np` so they raise a clear `ImportError`
only when embeddings are actually persisted.
Net effect: the proxy boots without numpy installed; memory features
that actually use numpy still work when its optional deps are present.
Fixes#332
test_anthropic_hooks_do_not_break_extract_user_query_lookup mocks
pipeline.apply to return tokens_after=40 and a tiny compressed
message. The pre-Bug-3 proxy trusted the mock's tokens_after and
emitted x-headroom-tokens-after: 40. After issue #327 Bug 3 the
proxy recounts optimized_tokens from result.messages with its
own tokenizer (the mocked "compressed" string counts to 11), so the
header asserted against the wrong tokenizer's number.
Compute the expected value from the same tokenizer the proxy uses
(get_tokenizer("claude-sonnet-4-6")) and assert the recounted
header matches that. Add a tokens_before > tokens_after invariant
so the spirit of the test (compression actually reduced bytes) is
preserved without coupling to a specific tokenizer's calibration.
Three bugs combined to drive end-to-end compression on the Anthropic
backend to ~0% in token mode (the default). User report #327 saw a
~9× drop in dashboard savings from one day to the next on Claude
Code traffic; the dashboard headline was technically correct but the
underlying compression genuinely was not running. After this change
the same Claude Code-shape multi-turn conversation goes from
14987 → 14371 tokens at the request boundary on turn 1 and only
recompresses the freshest tool_result on subsequent turns, with the
prior turns frozen byte-identical to preserve the upstream prefix
cache.
Bug 1 — IntelligentContextManager inner ContentRouter has no observer
PR #302 (commit cf979958, 2026-04-28) wired CompressionObserver onto
the outer ContentRouter in proxy/server.py and onto SmartCrusher.
The inner ContentRouter constructed lazily inside
IntelligentContextManager._get_content_router (added Jan 18, 2026
in 57b2de5 alongside the COMPRESS_FIRST strategy) was missed. That
inner router handles the bulk of Claude Code's tool_result-block
compression, so per-strategy counters surfaced by PR #314 in v0.15.0
showed compressions_by_strategy={"text": 6} while
summary.compression.total_tokens_removed=1.3M — math-impossible.
Fix: add observer= parameter to IntelligentContextManager.__init__,
forward it to the inner ContentRouter at intelligent_context.py:525,
and pass observer=self.metrics from proxy/server.py.
Bug 2 — TTL deferral marks every fresh tool_result as stable
should_defer_compression in compression_cache.py returned True on
first-sight (added 2026-04-07 in commit 22dad13 with the intent of
batching first-time compressions near the 5-min cache TTL boundary
to trade many small busts for one). The token-mode walker at
anthropic.py:766-787 walks every message past frozen_message_count,
calls should_defer_compression on each fresh tool_result, gets True,
and advances ttl_frozen += 1 — every iteration. Result:
frozen_message_count grows to len(messages), the pipeline freezes
the entire request, and nothing reaches a real compressor.
The defer-first-sight rationale assumes recurring content within
TTL. Real Claude Code traffic produces unique content per turn, so
"defer until next sight" defers forever. Compressing fresh content
on first sight does not bust any prefix cache because Anthropic has
not cached that byte position yet — it's a cache write either way.
Fix: should_defer_compression returns False on first-sight (record
the timestamp; compress now). Subsequent sightings within TTL still
defer (batch window preserved for genuinely repeating content).
Updated tests in test_compression_cache.py to assert the corrected
semantics and verify _first_seen is recorded on first call.
Bug 3 — cross-tokenizer comparison in token-mode inflation guard
anthropic.py:634 sets original_tokens = tokenizer.count_messages(...)
using the proxy-side EstimatingTokenCounter. The token-mode branch
at line 816 set optimized_tokens = result.tokens_after from
pipeline, which uses the provider-side AnthropicProvider tiktoken
estimator. The two tokenizers disagree by ~25% on the same payload.
The inflation guard at line 901
(if optimized_tokens > original_tokens: revert to originals) treats
those two numbers as comparable. After a real 12% compression the
provider-tokenizer figure was still higher than the proxy-tokenizer
baseline, so the guard fired, optimized_messages was reset to the
original input, transforms_applied was emptied, and tokens_saved
went to 0. The dashboard showed no compression even when the
pipeline successfully compressed.
Fix: recount optimized_tokens with the proxy tokenizer right after
the pipeline returns, so the guard compares apples-to-apples. The
recount cost is a few ms on a 50K-token request and is dwarfed by
upstream call latency.
Verification
* 80 targeted tests across test_compression_cache,
test_compression_observability, test_proxy_anthropic_cache_stability,
test_proxy_intelligent_context pass.
* make ci-precheck clean.
* End-to-end real-API run against api.anthropic.com via local proxy:
- Turn 1 fresh: 14987 → 14371 (4.1%) on a 3-tool-round payload;
smart_crusher and diff strategies fired with non-zero savings.
- Turn 2 (turn 1 history + 1 new tool_result): 23161 → 21928 (5.3%);
only the new tool_result compressed; older turns marked
router:protected:user_message; Anthropic returned
cache_creation_input_tokens > 0 confirming the prefix was not
busted.
Two new regression tests in test_compression_observability lock down
the inner ContentRouter observer wiring so a future copy of Bug 1
fails the suite the day it lands.
Stacked on rust-stage-3g-reformat-offload-rework. Adds the JSON
offload — the last big content shape that was deferred from PR1 —
by wrapping the existing SmartCrusher subsystem in the
OffloadTransform contract.
Why this is a thin wrapper, not a refactor: SmartCrusher is a
3000+ line subsystem (tabular IR, cell classifier, document
compactor, formatters, parity fixtures, PyO3 bridge) that's already
shadow-validated against Python. Reimplementing it under the trait
shape would risk parity. Instead JsonOffload composes:
- estimate_bloat: cheap byte scan that spots array-of-objects shape
and counts row separators (`},{`, `}, {`, `},\n`). No JSON parse —
reserved for apply.
- apply: delegates to SmartCrusher::crush(content, ctx.query, 0.0).
On modification, hashes the WHOLE input and stashes it through
the orchestrator-supplied CCR store under that hash, appends a
`[json_offload CCR: hash=...]` marker. Honors the trait contract:
cache_key resolves in the store the orchestrator passed.
SmartCrusher's per-array CCR markers in the compressed body remain
informational. The wrapper-level outer hash is what the LLM
retrieves to recover the full original payload — single
authoritative recovery path for the orchestrator pipeline.
Config goes in [offload.json] of pipeline.toml:
- min_array_rows = 5 (estimator floor)
- saturation_rows = 50 (score saturates to 1.0 here)
12 new JsonOffload tests + 2 orchestrator E2E tests; full
headroom-core suite (730) green; cargo fmt clean; no regex per
project convention.
Replaces PR1's lossless/lossy split with ReformatTransform (pack
denser, no info lost) and OffloadTransform (drop bytes, CCR-stash
original via required cache_key). With CCR every transform is
information-preserving end-to-end, so the lossless/lossy distinction
misnamed the architecture.
OffloadTransform carries a cheap, structural estimate_bloat() method
scoped to its domain — generic byte-redundancy heuristics miss domain
semantics. The orchestrator runs reformat phase + per-offload bloat
estimation in parallel via rayon::join + par_iter, then runs offload
iff bloat clears threshold OR reformat underwhelmed.
Transforms shipped:
REFORMATS (lossless):
- JsonMinifier: serde_json round-trip whitespace stripping.
- LogTemplate: Drain-inspired order-preserving template miner.
Collapses consecutive runs of same-template lines into
[Template Tn: ...] (Nx) + variant table. Win comes from emitting
the constant-token prefix once instead of N times. Lossless: every
original line reconstructible from template + variants.
OFFLOADS (drop bytes, stash original via CCR):
- LogOffload: wraps existing LogCompressor; bloat = repetition x
uniqueness_weight + dilution x priority_dilution_weight.
- DiffOffload: wraps existing DiffCompressor; bloat = context-to-
change ratio. Bug-fix-on-port — persists original under the
cache_key the parity-bound DiffCompressor mints (closes a leak).
- DiffNoise: drops lockfile hunks (Cargo.lock, package-lock.json,
yarn.lock, etc., suffix list configurable in TOML) and
whitespace-only hunks. Stashes original via CCR for retrieval.
Search offload exists but is not in default re-exports — modern
agents (Claude Code, Codex) use scoped rg/grep, the marginal value
didn't justify default registration. Reach via the explicit module
path if opting in.
JSON Offload is intentionally absent from this PR — already lives at
SmartCrusher; Phase 3g PR3 wraps it in the OffloadTransform contract.
Thresholds and weights live in config/pipeline.toml, embedded via
include_str!; PipelineConfig::from_toml_str loads runtime overrides.
98 new pipeline tests; full headroom-core suite (714) and workspace
tests green; cargo fmt clean. No regex, per project convention.
CI flagged two issues on the rebased branch:
1. ruff format --check failed on server.py and test_traffic_learner.py
after the rebase; line-collapse / trailing-whitespace nits.
2. Codecov reported 80% patch coverage with 20 lines missing in the
matcher helpers — mostly branches not exercised by the high-level
tests (empty Levenshtein inputs, source-prefix Bash parsing, env-var
skip, equal-string short-circuit in binary match, the substantive-
token path that beats the edit-distance gate, error_recovery patterns
with non-canonical content in _drop_contradictions).
Adds 16 targeted unit tests for those branches and applies ruff format.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The recovery matchers paired any failed and successful tool call within
a 5-call window with no semantic check that the pair was actually a
retry. This produced confidently-wrong rules like:
File `state.rs` does not exist. The correct path is `lib.rs`.
…where the user simply read two unrelated files in the same directory.
Across sessions the same user can also typo in opposite directions,
producing directly contradictory rules side by side.
This commit adds three structural checks:
1. Read recovery: require the failed and successful basenames to be
identical or close in Levenshtein distance. Rejects the "same dir,
different file" case that was the most common noise source.
2. Bash recovery: require both commands to share a binary (allowing
path-prefixed variants and short prefix-versions like
`python` ↔ `python3`) AND either have low normalized edit distance
or share a substantive non-flag token. Rejects pairs that share only
the binary name but differ in every meaningful argument.
3. Contradiction filter on flush: detect A→B and B→A pairs in
error_recovery patterns and drop both. They almost always indicate
opposite-direction typos in different sessions, not stable advice.
Also: stash failed_path in metadata so the contradiction filter and
downstream consumers can reason about pairs without parsing content.
Tests: 13 new tests covering the heuristics directly. Existing tests
exercising legitimate recoveries (`python`→`python3`, `ruff`→`.venv/bin/ruff`,
`pip install`→success) continue to pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Asserts that stop()'s final flush_to_file does not bypass the evidence
threshold. Earlier behavior collapsed the gate to 1 at shutdown,
persisting every singleton pattern. This guards against that change
sneaking back in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The traffic learner was emitting one-shot error_recovery patterns that
contradicted each other and bloated MEMORY.md with low-signal noise. Two
issues drove this:
1. The shutdown flush bypassed the evidence gate: the in-memory
_min_evidence was set to 2, but on stop() the gate dropped to 1, so
every singleton pattern got persisted at session end. This is the
opposite of how evidence thresholding should work — singletons are
the least trustworthy patterns, not the most.
2. The default min_evidence of 2 is too low to filter noise from the
matchers, which pair up failed/successful tool calls within a small
sliding window without a strong semantic check that the calls are
actually related.
Changes:
- Raise default min_evidence from 2 to 5 in TrafficLearner.
- Remove the shutdown-relaxation in flush_to_files; require
self._min_evidence at all times, including on stop().
- Add traffic_learning_min_evidence to ProxyConfig (default 5).
- Add --min-evidence CLI flag with HEADROOM_MIN_EVIDENCE envvar so
users and embedded clients (desktop apps, plugins) can tune the
threshold without source changes.
- Thread the config value through HeadroomProxy into TrafficLearner.
- Tests: cover default propagation and custom value flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces `CompressionPipeline` — the formal lossless-then-lossy
orchestrator described in issue #315. Before this PR each compressor
carried its own ad-hoc decision tree (SmartCrusher's 3c.2 refactor
made it explicit for one transform; the rest still decide privately).
This PR replaces that scaffolding with two traits, one orchestrator,
and one real impl per trait.
# Surface
* `LosslessTransform` — `name()`, `applies_to()`, `apply(content)`.
Preserves all information; orchestrator runs these first and stops
early if the cumulative savings hit `lossless_target_ratio`.
* `LossyTransform` — same shape plus `apply(content, ctx)` taking a
`CompressionContext` (query + token budget) and a `confidence()`
calibration score for telemetry.
* `TransformResult { output, bytes_saved, structure_preserved,
reversible_via }` — common return shape.
* `TransformError { InvalidInput, Skipped, Internal }` — orchestrator
treats all three as skip-this-transform; never panics.
* `CompressionPipeline` + `CompressionPipelineBuilder` — sequential
dispatch keyed on `ContentType`. Acceptance gate is
`min_savings_ratio` (default 5%); lossless stop gate is
`lossless_target_ratio` (default 50% of original). Per-step
`name()` reporting feeds the strategy-stats JSONB nest from 3e.0.
# Concrete impls (one real impl per trait — no speculative extraction)
* `JsonMinifier` (lossless) — `serde_json::Value` round-trip.
Pretty-printed JSON shrinks 25-35%; already-compact returns
`bytes_saved == 0` and the orchestrator rejects it.
* `LineImportanceFilter` (lossy) — consumes the existing
`signals::LineImportanceDetector` trait. Walks `str::lines()`,
scores each, drops below threshold, anchor-preserves first/last,
collapses gaps into `[... N lines omitted ...]` markers.
# No regex
By project convention, nothing in this module uses the `regex` crate.
JsonMinifier is pure `serde_json`. LineImportanceFilter walks lines
and consumes the signals trait (aho-corasick + ASCII word-boundary
post-filter, also no regex).
# Touches outside the new module
* `ContentType` gains `Hash` derive so the orchestrator can key
transforms by content type. Trivially safe — the enum is already
`Eq + PartialEq`.
* `transforms/mod.rs` re-exports the new public surface.
# Test plan (all green)
* 41 unit tests across the four submodules:
- 9 trait + error-handling tests
- 9 JsonMinifier tests (pretty/compact/empty/malformed/Unicode/
deeply-nested/structure-preserved/reversible-via/applies-to)
- 11 LineImportanceFilter tests (drop/keep/anchor windows/single-
line/empty/gap-counting/confidence/structure-preserved/Unicode/
overlapping anchors/applies-to narrowing)
- 12 orchestrator tests (empty pipeline/no-applicable/lossless-
runs/rejection-below-min-ratio/error-recovery/lossless-target-
stop/lossy-runs/lossy-compounds/structure-preserved-flag/
is-acceptable-zero-cases/min-savings-ratio/builder-dispatch/
builder-order)
* `make ci-precheck` clean
* `cargo fmt --all` + `cargo clippy` happy
# Out of scope (lands in later PRs)
* PR2: wrap existing structural transforms (Diff/Log/Search/Tag) in
trait shape. Begins parallel-execution candidate via subagents.
* PR3: SmartCrusher refactor to use the orchestrator + retire Python
glue.
* PR4: `ProseFieldCompressor` (parser/model boundary primitive,
blocked on labeled corpus).
* PR5: migrate text/search/log compressors and delete Python
ContentRouter strategy dispatch.
`headroom/transforms/tag_protector.py` was a regex-driven scan-and-
replace loop that ran on every kompress call from ContentRouter
(`content_router.py:1089`). The Python implementation had five real
bugs we now fix in the port — the most consequential being a
`str.replace(.., .., 1)` first-occurrence-replace bug that silently
collapsed two identical custom-tag blocks in the same input to a
single placeholder + a stray duplicate of the second block.
# Bug fixes (each pinned by a `fixed_in_3e4` test)
* **#1: O(n²) on nested custom tags.** Python's `while changed` loop
restarted a full regex scan after every replacement. Rust walks
once in linear time on input length.
* **#2: First-occurrence replace bug.** `result.replace(orig, ph, 1)`
replaces the FIRST textual match, not the matched offset. Two
identical custom-tag blocks collapsed to one placeholder + a stray
duplicate of the second block. The Rust walker stitches output by
offset so distinct blocks always get distinct placeholders.
* **#3: Silent 50-iteration cap.** Python had a hard `max_iterations
= 50` safety limit that quietly truncated tag protection on deeply
nested input. The Rust walker is bounded by input length only.
* **#4: Self-closing pass duplicate-replace risk.** Python ran a
second loop with the same `replace_first` bug for self-closers.
Rust handles self-closers in the same single pass.
* **#5: Placeholder collision.** If the input contained a literal
`{{HEADROOM_TAG_…}}` substring, Python silently let the collision
break restoration. Rust salts the prefix and reports it in stats.
# Architecture
Two-phase walker:
* Phase 1 (`identify_spans`): linear scan over input bytes, hand-
rolled tag-open / tag-close lexer (no regex). Maintains a stack of
open custom tags; on a matching close, collapses the inner span
into a single `Span { start, end, Block }`. Self-closing custom
tags become `Span { ..., SelfClosing }` immediately. Marker-only
mode (`compress_tagged_content=true`) emits Open/CloseMarker spans
instead. Orphan opens stay un-protected (matches Python behavior).
Orphan closes are emitted verbatim and counted in stats.
* Phase 2 (`emit_output`): walks `text` once, splicing placeholders
for span ranges and copying everything else verbatim. Offset-based,
never `str.replace`.
PyO3 surface: `protect_tags`, `restore_tags`, `is_html_tag`,
`known_html_tag_names`. The Python shim retires the regex internals
and re-exports `KNOWN_HTML_TAGS` (rebuilt from the Rust list) +
`_is_html_tag` for backwards compat with `content_router.py` and the
existing test surface.
# Test plan
* 25 Rust unit tests including 4 `fixed_in_3e4_*` bug-fix tests
* 27 Python tests (23 existing + 4 new `fixed_in_3e4` parity tests)
* 5 integration tests in `test_tag_protection_integration.py` pass
* `make ci-precheck` clean
`headroom/transforms/query_echo.py` was already disabled in all three
proxy handlers (`anthropic.py`, `openai.py`, `gemini.py`) — each
carried the same comment: 'disabled — hurts prefix caching in long
conversations. The echo changes every turn, invalidating the cached
prefix.' That call was right: the echo's per-turn variability would
bust the Anthropic/OpenAI/Gemini prompt cache and cost more in TTFT
than the recall benefit ever paid back. The module was orphaned but
still living in the tree, with its own 70-test file pinning the
disabled behavior.
# Removed
* `headroom/transforms/query_echo.py` (123 LOC)
* `tests/test_query_echo.py` (whole file)
* The three 'disabled — hurts prefix caching' comment blocks in the
proxy handlers (the rationale lives in this commit message and the
PR description; no need to leave dead-code breadcrumbs in the hot
path).
# Kept
* `headroom.utils.extract_user_query` is a different function with
the same name and is still used elsewhere — untouched.
# Test plan
* `make ci-precheck` clean
* No remaining references to `query_echo`/`QueryEcho`/`Query Echo`/
`inject_query_echo` in the tree.
