Replaces the ICM-based interceptor on /v1/messages and adds OpenAI
/v1/chat/completions and /v1/responses arms. The proxy now follows
exactly one rule, applied per-request:
Find the last user message. Freeze every byte before it. In the
new turn, walk for tool outputs and route them through the
CompressionPipeline (Diff/Log/Json offloads + JsonMinifier/
LogTemplate reformats). Everything else passes through.
Why this design:
- Provider prefix caches (Anthropic cache_control, OpenAI auto
prefix, OpenAI prompt_cache_key) are positional. We never modify
prefix bytes, so cache hits are preserved 100%.
- User text, system prompts, assistant text are never touched —
accuracy preserved.
- Tool outputs (file reads, search results, build logs, diffs)
are where the tokens are. The CompressionPipeline already in
headroom-core compresses them with CCR backup.
What this PR removes:
- ICM (IntelligentContextManager) is no longer wired into the
proxy. It's still in headroom-core for future use, but the
proxy crate doesn't import it. ICM message-dropping risks
cache busts that token mode rules out by construction.
What this PR adds (compression module rewrite):
- compression/walker.rs — compress_blob() shared helper
- compression/pipeline.rs — build CompressionPipeline + CcrStore
once at startup
- compression/anthropic.rs — token-mode walker (tool_result blocks)
- compression/openai.rs — token-mode walker (role:tool messages)
- compression/responses.rs — token-mode walker (function_call_output
items; previous_response_id skips)
- compression/mod.rs — endpoint classifier + dispatch
Per-shape rules documented per file. Highlights:
- Anthropic: walk tool_result blocks (string OR list); compress
text inside list-shaped blocks; preserve images
- OpenAI chat: only role:tool with string content
- OpenAI responses: only function_call_output with string output;
reasoning items, *_call items, image_generation_call all
preserved verbatim; previous_response_id triggers full
passthrough (server holds the conversation)
Tests:
- Unit (32 in compression module + 3 walker tests)
- Mock integration (8 tests, including prefix-byte-identical
assertion across Anthropic / OpenAI / Responses)
- Real OpenAI e2e (3 tests, gated on HEADROOM_E2E=1):
* /v1/chat/completions tool message: 70 prompt tokens received
(compressed from a ~120KB / ~30K-token raw log payload)
* /v1/responses function_call_output: 68 input tokens
(same payload, same compression)
* Prefix cache preservation: 3,456 of 3,518 tokens cached
on the second turn (98%) — proves byte-stable prefix
Verification:
- cargo test --workspace -> 904 passed, 0 failed
- cargo clippy --workspace --all-targets -- -D warnings -> clean
- cargo fmt --check -> clean
- HEADROOM_E2E=1 cargo test --test e2e_token_mode -> 3/3 pass
- make ci-precheck -> green
The bounded compression executor introduced in this PR moved every
handler's compression call from `asyncio.wait_for(asyncio.to_thread(...))`
to `self._run_compression_in_executor(...)`, which lives on
`HeadroomProxy` (server.py) and is inherited by handler mixins at
runtime.
The test's `_DummyOpenAIHandler` only inherits `OpenAIHandlerMixin`,
not `HeadroomProxy`, so it lacks the method. The Responses API
compression path caught the AttributeError and silently fell back —
which made `test_handle_openai_responses_stream_keeps_compression`
fail with `apply.call_count == 0`.
Add a synchronous stub that just invokes the callable; tests don't
need real thread-pool semantics.
Adds an opt-in compression interceptor that buffers Anthropic
/v1/messages requests, runs IntelligentContextManager over the
messages array, and forwards the (possibly trimmed) body upstream.
All other paths, methods, and content-types stay on the original
streaming passthrough — so existing operators see zero change.
Behaviour gates ALL must be true to buffer + compress:
- --compression flag (or HEADROOM_PROXY_COMPRESSION=1)
- method == POST
- path == /v1/messages
- Content-Type: application/json
- ICM constructed successfully at startup
Falls through to streaming on any failure: parse, missing fields,
unknown model, body-too-large. Compression must never break a
request — that's the safety contract.
Model context windows come from a vendored LiteLLM snapshot at
crates/headroom-proxy/data/model_prices_and_context_window.json
parsed once into an OnceLock<HashMap>. Refresh via
scripts/refresh_model_limits.sh. Rationale documented inline:
hardcoded tables silently rot; LiteLLM is the canonical source
the entire LLM-tooling ecosystem relies on.
New tests:
- 16 unit tests across compression::{anthropic, icm, model_limits}
- 5 integration tests: off-passthrough, on-short-passthrough,
on-oversized-trim, on-non-json-skip, on-non-llm-path-skip
Verification:
- cargo test --workspace -> 884 passed, 0 failed
- cargo clippy --workspace -- -D warnings -> clean
- cargo fmt --check -> clean
Three audit follow-ups from issue #327's deep-dive review.
C1 — CompressionCache concurrency lock
======================================
`CompressionCache` instances are shared per `session_id` and accessed from
async-dispatched threadpool workers. Pre-fix, concurrent requests for the
same session raced on `_cache`, `_stable_hashes`, `_first_seen`, and
`_total_tokens_saved` with no synchronization. Observable failures:
* Lost-update on `_total_tokens_saved` (read-modify-write).
* `RuntimeError: OrderedDict mutated during iteration` from `apply_cached`
when a concurrent `store_compressed` evicts during the walk.
* Lost stable-hash records — next-turn compute_frozen_count reads
inconsistent state.
May also explain part of SvenMeyer's `_cache: 0 entries / 1003 misses`
observation: the cache was being clobbered concurrently.
Added `threading.RLock` guarding all mutating methods. `RLock` (not `Lock`)
so future code can call locked methods from inside another locked method
without self-deadlock. Also locked `HeadroomProxy._compression_caches`
dict-of-caches access via a separate `_compression_caches_lock` so two
concurrent calls for the same session_id can't each create distinct
CompressionCache objects (which would split the cache state between them).
The `/stats` endpoint snapshots the cache list under the dict lock before
iterating to avoid eviction-during-iteration.
C2 — Multi-worker CCR fragmentation: documented + startup warning
=================================================================
The in-memory `InMemoryCcrStore` (Rust), `_compression_caches` (Python),
`session_tracker_store` (Python), and TOIN learner state are ALL
per-process. Multi-worker uvicorn round-robins requests across workers,
so a session whose turn-1 lands on worker A may have turn-2 land on
worker B. Worker B has zero knowledge of A's CCR markers, replay cache,
or prefix-cache state. Result: `Retrieve original: hash=X` markers stay
in-context as opaque directives, every fresh tool_result is recompressed
from scratch, and `frozen_message_count=0` causes Anthropic prefix-cache
busts on every cross-worker turn.
Added a "Multi-worker deployment — CCR fragmentation" section in
`RUST_DEV.md` documenting the failure modes, the supported configuration
(`--workers 1`), and the sticky-session workaround for horizontal scale.
The proxy emits a `WARNING`-level log line on startup if `workers > 1` is
detected, pointing at the doc section.
C3 — Bounded compression executor with cancel-aware metrics
===========================================================
`asyncio.wait_for(asyncio.to_thread(pipeline.apply), timeout=...)`
cancellation does NOT propagate into the threadpool worker that's running
Rust code. Once the worker has picked up the task,
`concurrent.futures.Future.cancel()` returns False and the thread runs to
completion. Stuck threads accumulated invisibly on asyncio's default
executor, contending with unrelated `to_thread` callers (file IO, etc.).
Replaced all 7 `asyncio.to_thread` call sites for `pipeline.apply()`
across `proxy/handlers/anthropic.py` (3) and `proxy/handlers/openai.py` (4)
with a new `HeadroomProxy._run_compression_in_executor(fn, *, timeout)`
helper that:
1. Submits to a dedicated bounded `ThreadPoolExecutor` named
`headroom-compress` (configurable via
`ProxyConfig.compression_max_workers`; defaults to
`min(32, (cpu_count or 1) * 4)`).
2. Increments `_compression_in_flight` (gauge) when work starts and
decrements when work completes; tracks `_compression_in_flight_max`
as a high-water mark.
3. Detects "leaked threads" by comparing wall-clock elapsed against the
timeout in the worker's `finally` block. Increments
`_compression_leaked_threads` when a worker finishes after its
asyncio future was cancelled. Operators can see the leaked-thread
rate climbing in `/stats runtime.compression_executor` BEFORE the
pool fills up.
Tests
=====
* `TestCompressionCacheConcurrency` (3 tests) — many threads
store_compressed / apply_cached / update_from_result on a single
CompressionCache; assert no exceptions, no lost updates, no partial
state.
* `test_get_compression_cache_returns_same_instance_under_contention` —
32 concurrent `_get_compression_cache(same_id)` calls return the
identical instance (would split pre-lock).
* `test_proxy_compression_executor.py` (8 tests) — pool size respects
config, in-flight gauge tracks running compressions, high-water mark
is monotonic, timeout propagates to awaiter, leaked-thread counter
increments on post-deadline completion, `/stats` surfaces all three
gauges.
Verification
============
* All 123 targeted regression tests pass.
* `make ci-precheck` clean.
* No `Co-Authored-By` trailer; conventional `fix:` prefix; no
`--no-verify`.
The @1.95.0 git ref of dtolnay/rust-toolchain shipped action code
that errors on ubuntu-latest with:
failed to install component: 'clippy-preview-x86_64-unknown-linux-gnu',
detected conflict: 'bin/cargo-clippy'
The runner's pre-installed Rust ships cargo-clippy at $HOME/.cargo/bin,
and the older action code's rustup invocation hits a path conflict
when adding the clippy-preview component for 1.95.0.
The @stable ref of the action has the fix; pass toolchain: 1.95.0 as
input so the version stays pinned. rust-toolchain.toml continues to
be the source of truth for the version (used by cargo's
auto-detection); this keeps the action's install in sync.
The Anthropic token-mode handler walked past prefix_tracker.frozen_message_count
whenever an upcoming tool_result's content-hash matched comp_cache._stable_hashes
or should_defer_compression returned True. That conflated content equality with
positional cache membership.
Anthropic's prefix cache is POSITIONAL: bytes 0..K cached, anything past K is
fresh. _stable_hashes is content-keyed and grows unbounded. In long Claude Code
sessions where tool_result content rhymes across turns (repeated system prompts,
repeated file reads, repeated tool descriptions), the walker advanced
frozen_message_count to len(messages) on every turn and the pipeline produced
transforms_applied=[] on 73% of requests in user SvenMeyer's reported session
(headroom-stats-2026-05-01.json: 74 of 101 eligible requests "prefix_frozen") —
even after the prior fix in 44944fb. The 15 requests that did compress averaged
21%, proving compression itself works when reached.
Fix: delete the walker. The freeze boundary is now
frozen_message_count = min(
prefix_tracker.frozen_message_count, # positional ground truth
comp_cache.compute_frozen_count(messages), # local cache lower bound
)
compute_frozen_count's use of _stable_hashes can only LOWER the freeze via the
min clamp, never raise it past prefix_tracker's value. For any position in the
gap [compute_frozen_count, prefix_tracker.frozen_count], recompressing produces
byte-stable output (compression is deterministic on input content), so
Anthropic's prefix cache stays valid.
Cross-handler verification:
* OpenAI handler (proxy/handlers/openai.py:358-382) does not have this walker
— uses only compute_frozen_count. Codex routes through OpenAI handler. Both
unaffected.
* Streaming and non-streaming both invoke anthropic_pipeline.apply() before the
upstream call. One fix covers both paths.
* Cache mode (is_cache_mode) takes the _extract_cache_stable_delta path and is
independent of the walker. Unaffected.
Tests: six new regression tests lock down the post-fix invariants — clamp to
min(prefix_tracker, compute_frozen_count); fresh tool_result whose hash matches
old _stable_hashes entry is not frozen; frozen prefix byte-stable across the
pipeline; 10-turn session produces non-empty compression suffix every turn;
streaming and non-streaming compute identical frozen_message_count; OpenAI
handler never calls the walker functions. Plus scripts/smoke_issue_327.py
(gated by RUN_LIVE_API=1) drives a 10-turn conversation against
api.anthropic.com in both shapes (string + list-of-blocks) and both modes
(streaming + non-streaming).
ci-precheck clean. 191 tests pass.
Follow-ups (separate PRs):
* Fix _cache perpetually empty (anthropic.py result.messages != working_messages
comparison rarely fires in token mode).
* Cap _stable_hashes with bounded LRU + 1h TTL — hygiene only after the freeze
gate is removed.
* List-shape tool_result content gates at content_router.py:1975 and
intelligent_context.py:657 (cluster A from the audit).
Direct port of `headroom.transforms.scoring.MessageScorer` (459 LOC).
Foundation piece for the IntelligentContext port (PR-B onward).
What's wired:
- Deterministic factors fully ported: recency (exp-decay), forward
references (tool_call_id graph), token density (unique/total).
- External-dep factors gated behind traits: `EmbeddingProvider` and
`ToinProvider`. No concrete impls yet — both default to neutral
values matching Python's `embedding_provider=None` / `toin=None`.
PR-A1 wires fastembed; PR-A2 plugs in a PyO3 ToinProvider.
- ScoringWeights + MessageScore with serde + BTreeMap-ordered
breakdown for stable JSON.
Parity:
- 13 fixtures recorded from Python, byte-equal under the comparator.
- Floats rounded to 5 decimals on both sides — absorbs f32-vs-f64
drift in the weighted sum without masking real bugs.
Drive-by: re-fix three pre-existing clippy errors in
smart_crusher/crusher.rs that re-emerged with new test additions
(field_reassign_with_default + dead hash_array_for_ccr).
Closes findings from the post-Phase-3g audit. Five surgical fixes
plus telemetry-discoverability docs. PyO3 0.22 → 0.24 security
upgrade is its own PR (issue #335).
1. DiffCompressor cache_key persistence (production bug)
---------------------------------------------------------
Pre-fix: `RustDiffCompressor.compress()` minted a `cache_key`,
embedded `[... hash=abc123]` in the wire marker, and returned
without storing the original anywhere. Python ContentRouter then
returned the compressed text with a dangling marker — every
retrieval tool call from the LLM 404'd.
Sibling compressors (LogCompressor, SearchCompressor) already had
the right pattern: Rust mints the key, Python's
`_persist_to_python_ccr` writes the original to the production
`CompressionStore`. DiffCompressor was the asymmetric one.
Fix:
- Rust: add `DiffCompressor::compress_with_store(content, context,
Option<&dyn CcrStore>)` mirroring siblings. Calls `store.put`
when a key is minted; legacy `compress()` and
`compress_with_stats()` delegate with `None` for parity.
- Python: add `_persist_to_python_ccr` helper to
`headroom/transforms/diff_compressor.py.compress()` mirroring
`log_compressor.py` and `search_compressor.py`.
- Pipeline `DiffOffload`: switch to `compress_with_store(Some(store))`
and drop the post-hoc double-store hack that papered over this
bug at the orchestrator boundary.
2. CCR store TOCTOU race in `get()`
-----------------------------------
`InMemoryCcrStore::get()` checked TTL under a read lock, dropped
the lock, then called `remove()`. Between drop and remove a
concurrent `put()` of the same hash with fresh data could land —
and our `remove` would then wipe that fresh entry. Under
multi-worker proxy load this manifested as "I just stored it; why
is it gone?"
Fix: use `DashMap::remove_if`. Predicate runs under the shard
write lock so check-and-remove is atomic. New regression test
exercises a tight contention loop between writer and reader on
the same key.
3. Pre-existing clippy debt in smart_crusher
--------------------------------------------
- 3× `field_reassign_with_default` in `crusher.rs` test setup —
switch to struct-update syntax `Config { field: x, ..Default }`.
- `hash_array_for_ccr` was `#[cfg(test)]` but unused; deleted with
a comment so a future test can reintroduce it as a one-liner.
`cargo clippy --workspace --all-targets -- -D warnings` is now
clean across the whole workspace; previous CI patches that allowed
these warnings can be removed in a follow-up.
4. Tokenizers dependency dedup
------------------------------
`tokenizers 0.21` (direct dep) + `tokenizers 0.22` (transitive via
fastembed) compiled twice into the binary. Bumped direct dep to
`0.22` to align; API is compatible (verified by full tokenizer
test suite). Saves compile time + binary bloat.
5. Telemetry-discoverability doc (no new code)
----------------------------------------------
The audit recommended a per-transform invocation counter to
inform the next Python → Rust port. Discovered the infrastructure
already exists at `/stats`:
- `compressions_by_strategy` — invocation count per strategy
- `pipeline_timing` — count + avg/max ms per transform name
- `tokens_saved_by_strategy` — savings attribution
Added a section to `RUST_DEV.md` showing the `curl + jq` recipes
to read this data, with example output highlighting how to spot
zero-invocation deferral candidates (e.g. `code_compressor`).
Verification: workspace tests 734 + 14 + 5 + 4 + 6 + 5 + 2 + 2 +
3 + 4 + 2 + 2 + 1 = all green; cargo fmt clean; cargo clippy
--all-targets clean; Python tests 185 pass; commitlint clean.
The proxy startup chain eagerly loads `headroom.memory.adapters`, which
imports `ports.py` and `sqlite.py`. Both files declared `import numpy as np`
at module top even though numpy is not in base dependencies (it is only
under `[all]`, `[dev]`, `[evals]`, and `[relevance]` extras). As a result,
`pipx install 'headroom-ai[mcp]' && headroom proxy` fails with
`ModuleNotFoundError: No module named 'numpy'` even when memory features
are disabled (the default).
Both files already use `from __future__ import annotations`, so the
`np.ndarray` type hints are strings at runtime - numpy is only needed
for static type checking. Move the module-level `import numpy as np`
under a `TYPE_CHECKING` guard. For the only runtime users
(`SQLiteMemoryStore._serialize_embedding` and `_deserialize_embedding`),
add a local `import numpy as np` so they raise a clear `ImportError`
only when embeddings are actually persisted.
Net effect: the proxy boots without numpy installed; memory features
that actually use numpy still work when its optional deps are present.
Fixes#332
test_anthropic_hooks_do_not_break_extract_user_query_lookup mocks
pipeline.apply to return tokens_after=40 and a tiny compressed
message. The pre-Bug-3 proxy trusted the mock's tokens_after and
emitted x-headroom-tokens-after: 40. After issue #327 Bug 3 the
proxy recounts optimized_tokens from result.messages with its
own tokenizer (the mocked "compressed" string counts to 11), so the
header asserted against the wrong tokenizer's number.
Compute the expected value from the same tokenizer the proxy uses
(get_tokenizer("claude-sonnet-4-6")) and assert the recounted
header matches that. Add a tokens_before > tokens_after invariant
so the spirit of the test (compression actually reduced bytes) is
preserved without coupling to a specific tokenizer's calibration.
Three bugs combined to drive end-to-end compression on the Anthropic
backend to ~0% in token mode (the default). User report #327 saw a
~9× drop in dashboard savings from one day to the next on Claude
Code traffic; the dashboard headline was technically correct but the
underlying compression genuinely was not running. After this change
the same Claude Code-shape multi-turn conversation goes from
14987 → 14371 tokens at the request boundary on turn 1 and only
recompresses the freshest tool_result on subsequent turns, with the
prior turns frozen byte-identical to preserve the upstream prefix
cache.
Bug 1 — IntelligentContextManager inner ContentRouter has no observer
PR #302 (commit cf979958, 2026-04-28) wired CompressionObserver onto
the outer ContentRouter in proxy/server.py and onto SmartCrusher.
The inner ContentRouter constructed lazily inside
IntelligentContextManager._get_content_router (added Jan 18, 2026
in 57b2de5 alongside the COMPRESS_FIRST strategy) was missed. That
inner router handles the bulk of Claude Code's tool_result-block
compression, so per-strategy counters surfaced by PR #314 in v0.15.0
showed compressions_by_strategy={"text": 6} while
summary.compression.total_tokens_removed=1.3M — math-impossible.
Fix: add observer= parameter to IntelligentContextManager.__init__,
forward it to the inner ContentRouter at intelligent_context.py:525,
and pass observer=self.metrics from proxy/server.py.
Bug 2 — TTL deferral marks every fresh tool_result as stable
should_defer_compression in compression_cache.py returned True on
first-sight (added 2026-04-07 in commit 22dad13 with the intent of
batching first-time compressions near the 5-min cache TTL boundary
to trade many small busts for one). The token-mode walker at
anthropic.py:766-787 walks every message past frozen_message_count,
calls should_defer_compression on each fresh tool_result, gets True,
and advances ttl_frozen += 1 — every iteration. Result:
frozen_message_count grows to len(messages), the pipeline freezes
the entire request, and nothing reaches a real compressor.
The defer-first-sight rationale assumes recurring content within
TTL. Real Claude Code traffic produces unique content per turn, so
"defer until next sight" defers forever. Compressing fresh content
on first sight does not bust any prefix cache because Anthropic has
not cached that byte position yet — it's a cache write either way.
Fix: should_defer_compression returns False on first-sight (record
the timestamp; compress now). Subsequent sightings within TTL still
defer (batch window preserved for genuinely repeating content).
Updated tests in test_compression_cache.py to assert the corrected
semantics and verify _first_seen is recorded on first call.
Bug 3 — cross-tokenizer comparison in token-mode inflation guard
anthropic.py:634 sets original_tokens = tokenizer.count_messages(...)
using the proxy-side EstimatingTokenCounter. The token-mode branch
at line 816 set optimized_tokens = result.tokens_after from
pipeline, which uses the provider-side AnthropicProvider tiktoken
estimator. The two tokenizers disagree by ~25% on the same payload.
The inflation guard at line 901
(if optimized_tokens > original_tokens: revert to originals) treats
those two numbers as comparable. After a real 12% compression the
provider-tokenizer figure was still higher than the proxy-tokenizer
baseline, so the guard fired, optimized_messages was reset to the
original input, transforms_applied was emptied, and tokens_saved
went to 0. The dashboard showed no compression even when the
pipeline successfully compressed.
Fix: recount optimized_tokens with the proxy tokenizer right after
the pipeline returns, so the guard compares apples-to-apples. The
recount cost is a few ms on a 50K-token request and is dwarfed by
upstream call latency.
Verification
* 80 targeted tests across test_compression_cache,
test_compression_observability, test_proxy_anthropic_cache_stability,
test_proxy_intelligent_context pass.
* make ci-precheck clean.
* End-to-end real-API run against api.anthropic.com via local proxy:
- Turn 1 fresh: 14987 → 14371 (4.1%) on a 3-tool-round payload;
smart_crusher and diff strategies fired with non-zero savings.
- Turn 2 (turn 1 history + 1 new tool_result): 23161 → 21928 (5.3%);
only the new tool_result compressed; older turns marked
router:protected:user_message; Anthropic returned
cache_creation_input_tokens > 0 confirming the prefix was not
busted.
Two new regression tests in test_compression_observability lock down
the inner ContentRouter observer wiring so a future copy of Bug 1
fails the suite the day it lands.
Stacked on rust-stage-3g-reformat-offload-rework. Adds the JSON
offload — the last big content shape that was deferred from PR1 —
by wrapping the existing SmartCrusher subsystem in the
OffloadTransform contract.
Why this is a thin wrapper, not a refactor: SmartCrusher is a
3000+ line subsystem (tabular IR, cell classifier, document
compactor, formatters, parity fixtures, PyO3 bridge) that's already
shadow-validated against Python. Reimplementing it under the trait
shape would risk parity. Instead JsonOffload composes:
- estimate_bloat: cheap byte scan that spots array-of-objects shape
and counts row separators (`},{`, `}, {`, `},\n`). No JSON parse —
reserved for apply.
- apply: delegates to SmartCrusher::crush(content, ctx.query, 0.0).
On modification, hashes the WHOLE input and stashes it through
the orchestrator-supplied CCR store under that hash, appends a
`[json_offload CCR: hash=...]` marker. Honors the trait contract:
cache_key resolves in the store the orchestrator passed.
SmartCrusher's per-array CCR markers in the compressed body remain
informational. The wrapper-level outer hash is what the LLM
retrieves to recover the full original payload — single
authoritative recovery path for the orchestrator pipeline.
Config goes in [offload.json] of pipeline.toml:
- min_array_rows = 5 (estimator floor)
- saturation_rows = 50 (score saturates to 1.0 here)
12 new JsonOffload tests + 2 orchestrator E2E tests; full
headroom-core suite (730) green; cargo fmt clean; no regex per
project convention.
Replaces PR1's lossless/lossy split with ReformatTransform (pack
denser, no info lost) and OffloadTransform (drop bytes, CCR-stash
original via required cache_key). With CCR every transform is
information-preserving end-to-end, so the lossless/lossy distinction
misnamed the architecture.
OffloadTransform carries a cheap, structural estimate_bloat() method
scoped to its domain — generic byte-redundancy heuristics miss domain
semantics. The orchestrator runs reformat phase + per-offload bloat
estimation in parallel via rayon::join + par_iter, then runs offload
iff bloat clears threshold OR reformat underwhelmed.
Transforms shipped:
REFORMATS (lossless):
- JsonMinifier: serde_json round-trip whitespace stripping.
- LogTemplate: Drain-inspired order-preserving template miner.
Collapses consecutive runs of same-template lines into
[Template Tn: ...] (Nx) + variant table. Win comes from emitting
the constant-token prefix once instead of N times. Lossless: every
original line reconstructible from template + variants.
OFFLOADS (drop bytes, stash original via CCR):
- LogOffload: wraps existing LogCompressor; bloat = repetition x
uniqueness_weight + dilution x priority_dilution_weight.
- DiffOffload: wraps existing DiffCompressor; bloat = context-to-
change ratio. Bug-fix-on-port — persists original under the
cache_key the parity-bound DiffCompressor mints (closes a leak).
- DiffNoise: drops lockfile hunks (Cargo.lock, package-lock.json,
yarn.lock, etc., suffix list configurable in TOML) and
whitespace-only hunks. Stashes original via CCR for retrieval.
Search offload exists but is not in default re-exports — modern
agents (Claude Code, Codex) use scoped rg/grep, the marginal value
didn't justify default registration. Reach via the explicit module
path if opting in.
JSON Offload is intentionally absent from this PR — already lives at
SmartCrusher; Phase 3g PR3 wraps it in the OffloadTransform contract.
Thresholds and weights live in config/pipeline.toml, embedded via
include_str!; PipelineConfig::from_toml_str loads runtime overrides.
98 new pipeline tests; full headroom-core suite (714) and workspace
tests green; cargo fmt clean. No regex, per project convention.
CI flagged two issues on the rebased branch:
1. ruff format --check failed on server.py and test_traffic_learner.py
after the rebase; line-collapse / trailing-whitespace nits.
2. Codecov reported 80% patch coverage with 20 lines missing in the
matcher helpers — mostly branches not exercised by the high-level
tests (empty Levenshtein inputs, source-prefix Bash parsing, env-var
skip, equal-string short-circuit in binary match, the substantive-
token path that beats the edit-distance gate, error_recovery patterns
with non-canonical content in _drop_contradictions).
Adds 16 targeted unit tests for those branches and applies ruff format.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The recovery matchers paired any failed and successful tool call within
a 5-call window with no semantic check that the pair was actually a
retry. This produced confidently-wrong rules like:
File `state.rs` does not exist. The correct path is `lib.rs`.
…where the user simply read two unrelated files in the same directory.
Across sessions the same user can also typo in opposite directions,
producing directly contradictory rules side by side.
This commit adds three structural checks:
1. Read recovery: require the failed and successful basenames to be
identical or close in Levenshtein distance. Rejects the "same dir,
different file" case that was the most common noise source.
2. Bash recovery: require both commands to share a binary (allowing
path-prefixed variants and short prefix-versions like
`python` ↔ `python3`) AND either have low normalized edit distance
or share a substantive non-flag token. Rejects pairs that share only
the binary name but differ in every meaningful argument.
3. Contradiction filter on flush: detect A→B and B→A pairs in
error_recovery patterns and drop both. They almost always indicate
opposite-direction typos in different sessions, not stable advice.
Also: stash failed_path in metadata so the contradiction filter and
downstream consumers can reason about pairs without parsing content.
Tests: 13 new tests covering the heuristics directly. Existing tests
exercising legitimate recoveries (`python`→`python3`, `ruff`→`.venv/bin/ruff`,
`pip install`→success) continue to pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Asserts that stop()'s final flush_to_file does not bypass the evidence
threshold. Earlier behavior collapsed the gate to 1 at shutdown,
persisting every singleton pattern. This guards against that change
sneaking back in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The traffic learner was emitting one-shot error_recovery patterns that
contradicted each other and bloated MEMORY.md with low-signal noise. Two
issues drove this:
1. The shutdown flush bypassed the evidence gate: the in-memory
_min_evidence was set to 2, but on stop() the gate dropped to 1, so
every singleton pattern got persisted at session end. This is the
opposite of how evidence thresholding should work — singletons are
the least trustworthy patterns, not the most.
2. The default min_evidence of 2 is too low to filter noise from the
matchers, which pair up failed/successful tool calls within a small
sliding window without a strong semantic check that the calls are
actually related.
Changes:
- Raise default min_evidence from 2 to 5 in TrafficLearner.
- Remove the shutdown-relaxation in flush_to_files; require
self._min_evidence at all times, including on stop().
- Add traffic_learning_min_evidence to ProxyConfig (default 5).
- Add --min-evidence CLI flag with HEADROOM_MIN_EVIDENCE envvar so
users and embedded clients (desktop apps, plugins) can tune the
threshold without source changes.
- Thread the config value through HeadroomProxy into TrafficLearner.
- Tests: cover default propagation and custom value flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces `CompressionPipeline` — the formal lossless-then-lossy
orchestrator described in issue #315. Before this PR each compressor
carried its own ad-hoc decision tree (SmartCrusher's 3c.2 refactor
made it explicit for one transform; the rest still decide privately).
This PR replaces that scaffolding with two traits, one orchestrator,
and one real impl per trait.
# Surface
* `LosslessTransform` — `name()`, `applies_to()`, `apply(content)`.
Preserves all information; orchestrator runs these first and stops
early if the cumulative savings hit `lossless_target_ratio`.
* `LossyTransform` — same shape plus `apply(content, ctx)` taking a
`CompressionContext` (query + token budget) and a `confidence()`
calibration score for telemetry.
* `TransformResult { output, bytes_saved, structure_preserved,
reversible_via }` — common return shape.
* `TransformError { InvalidInput, Skipped, Internal }` — orchestrator
treats all three as skip-this-transform; never panics.
* `CompressionPipeline` + `CompressionPipelineBuilder` — sequential
dispatch keyed on `ContentType`. Acceptance gate is
`min_savings_ratio` (default 5%); lossless stop gate is
`lossless_target_ratio` (default 50% of original). Per-step
`name()` reporting feeds the strategy-stats JSONB nest from 3e.0.
# Concrete impls (one real impl per trait — no speculative extraction)
* `JsonMinifier` (lossless) — `serde_json::Value` round-trip.
Pretty-printed JSON shrinks 25-35%; already-compact returns
`bytes_saved == 0` and the orchestrator rejects it.
* `LineImportanceFilter` (lossy) — consumes the existing
`signals::LineImportanceDetector` trait. Walks `str::lines()`,
scores each, drops below threshold, anchor-preserves first/last,
collapses gaps into `[... N lines omitted ...]` markers.
# No regex
By project convention, nothing in this module uses the `regex` crate.
JsonMinifier is pure `serde_json`. LineImportanceFilter walks lines
and consumes the signals trait (aho-corasick + ASCII word-boundary
post-filter, also no regex).
# Touches outside the new module
* `ContentType` gains `Hash` derive so the orchestrator can key
transforms by content type. Trivially safe — the enum is already
`Eq + PartialEq`.
* `transforms/mod.rs` re-exports the new public surface.
# Test plan (all green)
* 41 unit tests across the four submodules:
- 9 trait + error-handling tests
- 9 JsonMinifier tests (pretty/compact/empty/malformed/Unicode/
deeply-nested/structure-preserved/reversible-via/applies-to)
- 11 LineImportanceFilter tests (drop/keep/anchor windows/single-
line/empty/gap-counting/confidence/structure-preserved/Unicode/
overlapping anchors/applies-to narrowing)
- 12 orchestrator tests (empty pipeline/no-applicable/lossless-
runs/rejection-below-min-ratio/error-recovery/lossless-target-
stop/lossy-runs/lossy-compounds/structure-preserved-flag/
is-acceptable-zero-cases/min-savings-ratio/builder-dispatch/
builder-order)
* `make ci-precheck` clean
* `cargo fmt --all` + `cargo clippy` happy
# Out of scope (lands in later PRs)
* PR2: wrap existing structural transforms (Diff/Log/Search/Tag) in
trait shape. Begins parallel-execution candidate via subagents.
* PR3: SmartCrusher refactor to use the orchestrator + retire Python
glue.
* PR4: `ProseFieldCompressor` (parser/model boundary primitive,
blocked on labeled corpus).
* PR5: migrate text/search/log compressors and delete Python
ContentRouter strategy dispatch.
`headroom/transforms/tag_protector.py` was a regex-driven scan-and-
replace loop that ran on every kompress call from ContentRouter
(`content_router.py:1089`). The Python implementation had five real
bugs we now fix in the port — the most consequential being a
`str.replace(.., .., 1)` first-occurrence-replace bug that silently
collapsed two identical custom-tag blocks in the same input to a
single placeholder + a stray duplicate of the second block.
# Bug fixes (each pinned by a `fixed_in_3e4` test)
* **#1: O(n²) on nested custom tags.** Python's `while changed` loop
restarted a full regex scan after every replacement. Rust walks
once in linear time on input length.
* **#2: First-occurrence replace bug.** `result.replace(orig, ph, 1)`
replaces the FIRST textual match, not the matched offset. Two
identical custom-tag blocks collapsed to one placeholder + a stray
duplicate of the second block. The Rust walker stitches output by
offset so distinct blocks always get distinct placeholders.
* **#3: Silent 50-iteration cap.** Python had a hard `max_iterations
= 50` safety limit that quietly truncated tag protection on deeply
nested input. The Rust walker is bounded by input length only.
* **#4: Self-closing pass duplicate-replace risk.** Python ran a
second loop with the same `replace_first` bug for self-closers.
Rust handles self-closers in the same single pass.
* **#5: Placeholder collision.** If the input contained a literal
`{{HEADROOM_TAG_…}}` substring, Python silently let the collision
break restoration. Rust salts the prefix and reports it in stats.
# Architecture
Two-phase walker:
* Phase 1 (`identify_spans`): linear scan over input bytes, hand-
rolled tag-open / tag-close lexer (no regex). Maintains a stack of
open custom tags; on a matching close, collapses the inner span
into a single `Span { start, end, Block }`. Self-closing custom
tags become `Span { ..., SelfClosing }` immediately. Marker-only
mode (`compress_tagged_content=true`) emits Open/CloseMarker spans
instead. Orphan opens stay un-protected (matches Python behavior).
Orphan closes are emitted verbatim and counted in stats.
* Phase 2 (`emit_output`): walks `text` once, splicing placeholders
for span ranges and copying everything else verbatim. Offset-based,
never `str.replace`.
PyO3 surface: `protect_tags`, `restore_tags`, `is_html_tag`,
`known_html_tag_names`. The Python shim retires the regex internals
and re-exports `KNOWN_HTML_TAGS` (rebuilt from the Rust list) +
`_is_html_tag` for backwards compat with `content_router.py` and the
existing test surface.
# Test plan
* 25 Rust unit tests including 4 `fixed_in_3e4_*` bug-fix tests
* 27 Python tests (23 existing + 4 new `fixed_in_3e4` parity tests)
* 5 integration tests in `test_tag_protection_integration.py` pass
* `make ci-precheck` clean
`headroom/transforms/query_echo.py` was already disabled in all three
proxy handlers (`anthropic.py`, `openai.py`, `gemini.py`) — each
carried the same comment: 'disabled — hurts prefix caching in long
conversations. The echo changes every turn, invalidating the cached
prefix.' That call was right: the echo's per-turn variability would
bust the Anthropic/OpenAI/Gemini prompt cache and cost more in TTFT
than the recall benefit ever paid back. The module was orphaned but
still living in the tree, with its own 70-test file pinning the
disabled behavior.
# Removed
* `headroom/transforms/query_echo.py` (123 LOC)
* `tests/test_query_echo.py` (whole file)
* The three 'disabled — hurts prefix caching' comment blocks in the
proxy handlers (the rationale lives in this commit message and the
PR description; no need to leave dead-code breadcrumbs in the hot
path).
# Kept
* `headroom.utils.extract_user_query` is a different function with
the same name and is still used elsewhere — untouched.
# Test plan
* `make ci-precheck` clean
* No remaining references to `query_echo`/`QueryEcho`/`Query Echo`/
`inject_query_echo` in the tree.
`headroom/transforms/text_compressor.py` was a regex-line-sampling
fallback that nothing in the runtime called. ContentRouter routes
`CompressionStrategy.TEXT` straight to the Kompress ML compressor at
`content_router.py:1046` — the comment there literally says 'Prefer
Kompress ML compressor for text'. The Python file was orphaned but
still imported by its own test class, making it look live in the 3e
queue.
Drops the 3e.3 port from the queue: there's nothing to port.
# Removed
* `headroom/transforms/text_compressor.py` (255 LOC, unused)
* `tests/test_text_compressors.py::TestTextCompressor` (3 tests)
* `text_compressor` mention in `error_detection.py` shim docstring
* `text_compressor` mention in `test_signals_keyword_parity.py` docstring
* `TextCompressor` mention in `bench_latency.py` scenario comment
# Kept (defensive)
The legacy marker regex in `ccr/tool_injection.py:213` stays — it
parses an even older TextCompressor output format (pre-2026), is
purely defensive, and removal buys nothing. Test references to that
format in `test_ccr_tool_injection.py` document the regex contract
and stay too.
# Test plan
* `make ci-precheck` clean
* `tests/test_text_compressors.py` 19 passes (was 22, dropped 3)
Ports `headroom.transforms.log_compressor` to Rust. The biggest-by-
impact remaining compressor port: build/test logs are where the
10-50x compression wins live.
* Stack-trace state machine: per-flavor dispatcher (Python Traceback,
JS, Java, Rust error, Go); each flavor has its own termination
rule. Python terminated on any blank line, dropping mid-trace
lines from chained-exception traces.
* Conservative dedupe: preserves message prefix (everything before
first `:` or `=`); only trailing region is tokenised. Python's
blanket normalisation collapsed segfaults at different addresses.
* Loud CCR failures: `tracing::warn!` + `logger.warning` instead of
bare `except: pass`.
* `LogLevel::FAIL` documented as cosmetic-equivalent to ERROR.
Same shape as search_compressor port. Rust `LogCompressor`
orchestrates format detect -> classify -> score -> select ->
format -> CCR. Inline static-table format detector (YAGNI),
aho-corasick level classifier with word-boundary post-filter
(`signals::keyword_detector` technique), hand-rolled per-flavor
stack-trace state machine. `signals::LineImportanceDetector` NOT
consumed -- log levels are structural, not prose-style importance.
`headroom.transforms.log_compressor` becomes a thin shim:
`compress()` delegates to Rust end-to-end; internal helpers
preserved for the existing 50-test surface. Two existing tests
updated for new dedupe semantics + new compress orchestration.
* 17 Rust unit tests
* 50 Python tests pass
* `make ci-precheck` clean
The 12 langchain integration evals generate fixture data via
`random.choice`/`random.randint` without seeding. SmartCrusher's
anchor selection consumes the same global random state, so a handful
of unseeded inputs (~1% of seed values) skip the first/last anchor
preservation and the eval flakes — surfaced on PR #319 CI even though
this PR doesn't touch SmartCrusher.
Confirmed pre-existing: identical 5/500 seed failures on `main`
@ `cf3877d`. The fix is the smallest one that doesn't paper over the
underlying selector behavior — seed `random` per-test via an autouse
fixture so dataset generation is reproducible.
Ports `headroom.transforms.search_compressor` to Rust as the first
consumer of the `signals::LineImportanceDetector` trait shipped in
Phase 3e.1.
The Python regex registry (`_GREP_PATTERN`/`_RG_CONTEXT_PATTERN`)
silently misparsed two real-world inputs. The hand-rolled Rust parser
fixes both:
* **Windows paths.** `^([^:]+):(\d+):(.*)$` captured only the drive
letter from `C:\Users\foo\bar.py:42:line`, then the `\d+` group
failed on `\`. Result: every Windows-formatted line was silently
dropped from `file_matches`. The Rust parser detects the drive
prefix and starts the line-number scan after the drive colon.
* **Filenames with `-`.** `_RG_CONTEXT_PATTERN`'s `[^:-]+` excluded
dashes from the path, so legitimate names like
`pre-commit-config.yaml-42-line` parsed wrong. The Rust parser
anchors on the *line-number marker* (`<sep>\d+<sep>`), so paths
can contain dashes freely.
Two further hardening changes:
* CCR storage failures are loud (Python silently swallowed them).
* Per-file dedup is `O(n log n)` via `BTreeSet<(line_no, content_hash)>`
(Python used linear `match not in file_selected`, worst-case
quadratic for big files).
The Rust `SearchCompressor` owns a `Box<dyn LineImportanceDetector>`
defaulting to `KeywordDetector`. Priority scoring routes through the
trait instead of a hardcoded regex list, so a future BGE classifier
head (per the trait extension docs) can take over without touching
the compressor.
Sidecar `SearchCompressorStats` captures lines unparsed, files
dropped by `max_files`, matches dropped by per-file vs global caps,
and the CCR skip reason -- diagnostics Python never emitted.
`headroom.transforms.search_compressor` is now a thin shim that
delegates `compress()` to Rust end-to-end (so the parser bug fixes
land in production), and keeps the legacy `_parse_search_results`
helper routed through the same Rust parser. The other internal
helpers (`_score_matches`, `_select_matches`, `_format_output`)
stay Python -- they're heavily covered by existing direct-call tests
and Rust scoring is byte-equivalent.
The 4 public dataclasses are unchanged. Tests that monkeypatched the
old internal `_store_in_ccr` helper are updated to exercise the new
`_persist_to_python_ccr` boundary instead.
* 16 Rust unit tests (parser, scoring, selection, CCR round-trip) +
3 explicit `fixed_in_3e2` markers for the bug-fix lines
* 53 Python tests (existing suite intact; 2 updated for new shape)
* `make ci-precheck` clean
Stacks on PR #317 (signals trait module).
This test pinned the pre-3e.1 behavior on three lines that the new
KeywordDetector intentionally changes:
1. `'token'` was asserted to be in SECURITY_KEYWORDS. It was dropped
from the security set in 3e.1 because it false-positived on every
LLM-token reference in our own product. Updated to assert the new
set (`security|password|auth|secret`) and explicitly that `token`
is gone.
2. SECURITY_PATTERN was tested via "rotate the auth token" (matched
via the now-removed `token` keyword). Now tested via "rotate the
auth header" (matches via `auth`, which is the real security
signal) plus a negative assertion that LLM-metric strings no
longer fire.
3. ERROR_PATTERN test added an assertion that "Connection timeout"
now flags as an error (3e.1 fixed the keyword/regex drift).
PRIORITY_PATTERNS_TEXT indices were also corrected: the Rust-supplied
markdown_prefixes table is ordered `# `, `## `, `### `, `#### `, `**`,
`> ` (six prefixes) so the bold/blockquote assertions live at indices
6 and 7, not 3 and 4. New `# ` and `## ` checks pin the lower indices.
Each diverging assertion carries a `fixed_in_3e1` comment so the
audit trail stays clear.
Establish `crates/headroom-core/src/signals/` as a top-level module
holding cross-cutting detection traits. Phase 3e.1 ports
`error_detection.py` to a `LineImportanceDetector` trait + a
`Tiered<T>` combinator + a single concrete `KeywordDetector` impl
backed by aho-corasick. Three traits at three granularities are
sketched (line / blob / item); only line-importance is implemented
today.
Two bug fixes from the Python source bake into both the Rust impl
and the Python regex shim:
1. `ERROR_KEYWORDS` listed `timeout|abort|denied|rejected` but
`ERROR_PATTERN` regex omitted them. Lines like `"Connection
timeout"` were silently neutral despite the keyword being canonical.
Both surfaces now flag them.
2. `SECURITY_KEYWORDS` carried `token`, which false-positived on
every reference to LLM tokens (`input_tokens`, `tokens_saved`, ...)
in our own product. Dropped from the security set.
The Python `error_detection.py` shim now reflects keyword data out of
Rust via `keyword_registry_snapshot()` and recompiles the legacy
`re.Pattern` objects on the fly. Existing callers (text_compressor,
search_compressor, intelligent_context) continue to import the same
names with no source changes; caller migration to the trait API
happens in their own port PRs.
The trait architecture is the seam where a future ML detector slots
in without touching `KeywordDetector` or any caller. The canonical
extension is documented in `signals/README.md` as a classifier head
on the existing `bge-small-en-v1.5` embedder loaded by
`relevance::EmbeddingScorer` -- 384-dim -> 4-class softmax,
~1.5 KB head, ~1 ms inference, no extra model file. Two alternatives
(distilled tinyBERT in ONNX, logistic regression on lexical
features) are kept open in case BGE-head underfits.
Per the no-silent-fallbacks rule: only `KeywordDetector` lands as a
concrete impl. No NoOp, no MockDetector, no stub-ML -- those will
arrive with their real implementations.
Phase 3g (Compression Pipeline Formalization, issue #315) is queued
as the cross-cutting follow-up that will make lossless-then-lossy-
then-CCR ordering an explicit, observable architecture rather than
implicit per-compressor logic. Trait shapes there will reuse the
signals primitive landed in this PR.
