PR-A6 of the Phase A cache-safety lockdown. Eliminates P5-50 and preps
P0-6 (memory tool injection toggling).
Two cache-killer patterns the merge + tracker defeat:
1. Mid-session mutation: when memory was enabled the proxy did an
ad-hoc concat of `context-management-2025-06-27` onto the client
value (anthropic.py:1244-1248). The order varied with the client
value, breaking byte-stable headers across turns.
2. Token drop-out across turns: clients (Claude Code, Codex CLI) MAY
drop a beta token between turn N and turn N+1 even when the proxy
mutated turn N to add it. The cache hot zone is positional, so the
next turn's prefix bytes hash differently and the prefix-cache
read misses.
Changes
-------
`headroom/proxy/helpers.py`
* `merge_anthropic_beta` / `merge_openai_beta`: pure, deterministic,
order-preserving merge. Client tokens first (in their original
order), then Headroom-required tokens (in the order passed). Dedupe
is case-insensitive but preserves the original casing of the first
occurrence. No regex.
* `SessionBetaTracker`: bounded LRU keyed by (provider, session_id),
unioning client tokens with previously-seen tokens. OrderedDict
LRU; threading.RLock for thread safety (mirrors the
CompressionCache pattern from compression_cache.py).
* `get_session_beta_tracker` / `_reset_session_beta_tracker_for_test`
process-wide singleton with test reset.
* `log_beta_header_merge`: structured log per cache-affecting merge.
* Env-var knobs (NO HARDCODES):
- HEADROOM_BETA_HEADER_STICKY=enabled|disabled (default enabled).
- HEADROOM_BETA_TRACKER_MAX_SESSIONS (default 1000).
`headroom/proxy/handlers/anthropic.py`
* After `compute_session_id` (line ~744): record client
`anthropic-beta` against the session tracker, write the sticky
value back into `headers` if changed. Order matters: sticky-merge
FIRST so memory-injection has the canonical baseline.
* Memory-injection site (line ~1244): replace the ad-hoc concat with
`merge_anthropic_beta(headers["anthropic-beta"], required_tokens)`.
`headroom/proxy/handlers/openai.py`
* Chat-completions (line ~360): record/merge `openai-beta`.
* /v1/responses HTTP (line ~1213): compute `_responses_session_id`
and record/merge `openai-beta`.
* /v1/responses WS (line ~1711): replace the ad-hoc absent-only
inject with `merge_openai_beta(sticky, ["responses_websockets=
2026-02-06"])`. Replaces any case-variants of the existing key.
Tests
-----
`tests/test_anthropic_beta_session_sticky.py` (26 tests):
* Pure helper: empty inputs, only-client, only-headroom, ordering,
dedupe casing, deterministic memory-injection order, no-double-
inject when token already present.
* Tracker: sticky-on across turns even when client drops, casing
preservation, provider namespace independence, LRU eviction at
max_sessions, env-var validation (loud failures), thread safety
under 16-thread concurrent access, blank-input rejection.
`tests/test_openai_beta_session_sticky.py` (17 tests):
* Mirror of the anthropic suite for `OpenAI-Beta`.
* Plus WS-specific coverage: sticky-then-merge of
`responses_websockets=2026-02-06` against client baseline.
`tests/test_openai_codex_routing.py`
* Add `session_tracker_store` stub to `_DummyOpenAIHandler` so the
routing tests still exercise the responses HTTP handler now that
it computes a session_id for beta-merge.
Notes
-----
Build constraints honored:
* Configurable: HEADROOM_BETA_HEADER_STICKY,
HEADROOM_BETA_TRACKER_MAX_SESSIONS.
* No regex, no hardcodes (env-var bounds), no fallbacks (disabled
mode is operator opt-in for diagnostics, loud failures on invalid
values).
* Structured tracing log via `log_beta_header_merge`.
Acceptance:
* 43 new tests pass.
* `cargo test --workspace` green (no Rust changes).
* `make ci-precheck` green.
Adds an opt-in compression interceptor that buffers Anthropic
/v1/messages requests, runs IntelligentContextManager over the
messages array, and forwards the (possibly trimmed) body upstream.
All other paths, methods, and content-types stay on the original
streaming passthrough — so existing operators see zero change.
Behaviour gates ALL must be true to buffer + compress:
- --compression flag (or HEADROOM_PROXY_COMPRESSION=1)
- method == POST
- path == /v1/messages
- Content-Type: application/json
- ICM constructed successfully at startup
Falls through to streaming on any failure: parse, missing fields,
unknown model, body-too-large. Compression must never break a
request — that's the safety contract.
Model context windows come from a vendored LiteLLM snapshot at
crates/headroom-proxy/data/model_prices_and_context_window.json
parsed once into an OnceLock<HashMap>. Refresh via
scripts/refresh_model_limits.sh. Rationale documented inline:
hardcoded tables silently rot; LiteLLM is the canonical source
the entire LLM-tooling ecosystem relies on.
New tests:
- 16 unit tests across compression::{anthropic, icm, model_limits}
- 5 integration tests: off-passthrough, on-short-passthrough,
on-oversized-trim, on-non-json-skip, on-non-llm-path-skip
Verification:
- cargo test --workspace -> 884 passed, 0 failed
- cargo clippy --workspace -- -D warnings -> clean
- cargo fmt --check -> clean
Replaces PR1's lossless/lossy split with ReformatTransform (pack
denser, no info lost) and OffloadTransform (drop bytes, CCR-stash
original via required cache_key). With CCR every transform is
information-preserving end-to-end, so the lossless/lossy distinction
misnamed the architecture.
OffloadTransform carries a cheap, structural estimate_bloat() method
scoped to its domain — generic byte-redundancy heuristics miss domain
semantics. The orchestrator runs reformat phase + per-offload bloat
estimation in parallel via rayon::join + par_iter, then runs offload
iff bloat clears threshold OR reformat underwhelmed.
Transforms shipped:
REFORMATS (lossless):
- JsonMinifier: serde_json round-trip whitespace stripping.
- LogTemplate: Drain-inspired order-preserving template miner.
Collapses consecutive runs of same-template lines into
[Template Tn: ...] (Nx) + variant table. Win comes from emitting
the constant-token prefix once instead of N times. Lossless: every
original line reconstructible from template + variants.
OFFLOADS (drop bytes, stash original via CCR):
- LogOffload: wraps existing LogCompressor; bloat = repetition x
uniqueness_weight + dilution x priority_dilution_weight.
- DiffOffload: wraps existing DiffCompressor; bloat = context-to-
change ratio. Bug-fix-on-port — persists original under the
cache_key the parity-bound DiffCompressor mints (closes a leak).
- DiffNoise: drops lockfile hunks (Cargo.lock, package-lock.json,
yarn.lock, etc., suffix list configurable in TOML) and
whitespace-only hunks. Stashes original via CCR for retrieval.
Search offload exists but is not in default re-exports — modern
agents (Claude Code, Codex) use scoped rg/grep, the marginal value
didn't justify default registration. Reach via the explicit module
path if opting in.
JSON Offload is intentionally absent from this PR — already lives at
SmartCrusher; Phase 3g PR3 wraps it in the OffloadTransform contract.
Thresholds and weights live in config/pipeline.toml, embedded via
include_str!; PipelineConfig::from_toml_str loads runtime overrides.
98 new pipeline tests; full headroom-core suite (714) and workspace
tests green; cargo fmt clean. No regex, per project convention.
Ports `headroom.transforms.log_compressor` to Rust. The biggest-by-
impact remaining compressor port: build/test logs are where the
10-50x compression wins live.
* Stack-trace state machine: per-flavor dispatcher (Python Traceback,
JS, Java, Rust error, Go); each flavor has its own termination
rule. Python terminated on any blank line, dropping mid-trace
lines from chained-exception traces.
* Conservative dedupe: preserves message prefix (everything before
first `:` or `=`); only trailing region is tokenised. Python's
blanket normalisation collapsed segfaults at different addresses.
* Loud CCR failures: `tracing::warn!` + `logger.warning` instead of
bare `except: pass`.
* `LogLevel::FAIL` documented as cosmetic-equivalent to ERROR.
Same shape as search_compressor port. Rust `LogCompressor`
orchestrates format detect -> classify -> score -> select ->
format -> CCR. Inline static-table format detector (YAGNI),
aho-corasick level classifier with word-boundary post-filter
(`signals::keyword_detector` technique), hand-rolled per-flavor
stack-trace state machine. `signals::LineImportanceDetector` NOT
consumed -- log levels are structural, not prose-style importance.
`headroom.transforms.log_compressor` becomes a thin shim:
`compress()` delegates to Rust end-to-end; internal helpers
preserved for the existing 50-test surface. Two existing tests
updated for new dedupe semantics + new compress orchestration.
* 17 Rust unit tests
* 50 Python tests pass
* `make ci-precheck` clean
Establish `crates/headroom-core/src/signals/` as a top-level module
holding cross-cutting detection traits. Phase 3e.1 ports
`error_detection.py` to a `LineImportanceDetector` trait + a
`Tiered<T>` combinator + a single concrete `KeywordDetector` impl
backed by aho-corasick. Three traits at three granularities are
sketched (line / blob / item); only line-importance is implemented
today.
Two bug fixes from the Python source bake into both the Rust impl
and the Python regex shim:
1. `ERROR_KEYWORDS` listed `timeout|abort|denied|rejected` but
`ERROR_PATTERN` regex omitted them. Lines like `"Connection
timeout"` were silently neutral despite the keyword being canonical.
Both surfaces now flag them.
2. `SECURITY_KEYWORDS` carried `token`, which false-positived on
every reference to LLM tokens (`input_tokens`, `tokens_saved`, ...)
in our own product. Dropped from the security set.
The Python `error_detection.py` shim now reflects keyword data out of
Rust via `keyword_registry_snapshot()` and recompiles the legacy
`re.Pattern` objects on the fly. Existing callers (text_compressor,
search_compressor, intelligent_context) continue to import the same
names with no source changes; caller migration to the trait API
happens in their own port PRs.
The trait architecture is the seam where a future ML detector slots
in without touching `KeywordDetector` or any caller. The canonical
extension is documented in `signals/README.md` as a classifier head
on the existing `bge-small-en-v1.5` embedder loaded by
`relevance::EmbeddingScorer` -- 384-dim -> 4-class softmax,
~1.5 KB head, ~1 ms inference, no extra model file. Two alternatives
(distilled tinyBERT in ONNX, logistic regression on lexical
features) are kept open in case BGE-head underfits.
Per the no-silent-fallbacks rule: only `KeywordDetector` lands as a
concrete impl. No NoOp, no MockDetector, no stub-ML -- those will
arrive with their real implementations.
Phase 3g (Compression Pipeline Formalization, issue #315) is queued
as the cross-cutting follow-up that will make lossless-then-lossy-
then-CCR ordering an explicit, observable architecture rather than
implicit per-compressor logic. Trait shapes there will reuse the
signals primitive landed in this PR.
Re-lands two audit fixes that were marked "merged" on GitHub but never
reached main: squash-merging the parent stack changed its commit SHA,
which silently dropped the contents of the stacked PRs (#301, #305).
Single PR this time — no stacking risk.
What lands:
1. **`enable_ccr_marker` Rust gate** — new field on `SmartCrusherConfig`
(default `true`). `crush_array` checks it before emitting the
`<<ccr:HASH>>` marker text and the CCR store write. PyO3 surface +
parity-fixture tolerance updated; recorded fixtures predate the
field and inherit the `true` default.
2. **Python shim collapses both flags to the gate** — both
`ccr_config.enabled=False` and `ccr_config.inject_retrieval_marker
=False` now flip the Rust gate off. Storing a payload nothing in
the prompt can reference is pointless, and storing under
`enabled=False` would be a surprise side effect the user
explicitly opted out of.
3. **Custom `scorer` / `relevance_config` fails loud** — replaces the
prior WARNING-and-drop. Silently dropping a user-supplied scorer
is a textbook silent fallback. `NotImplementedError` instead.
Verified zero production callers pass these args; full plumbing
arrives with Stage-3c.2's relevance-crate Python bridge.
Tests:
- 2 new Rust unit tests in `crusher.rs::tests`
- 6 new Python tests in `test_smart_crusher_toin_attachment.py`
(3 CCR marker-knob behaviors + 3 scorer fail-loud)
- Removed `test_ccr_inject_marker_false_logs_warning` (the WARNING is
gone now that the flag is honored)
- `make ci-precheck` green; eval suite + observability tests run
twice consecutively to verify no TOIN file pollution leaks into the
regular+coverage double-run on Python 3.11
RUST_DEV.md audit table reflects both gaps closed.
The PR8 marker injection appends a sentinel object
{"_ccr_dropped": "<<ccr:HASH N_rows_offloaded>>"} to the kept-items
array on the lossy path so the LLM sees the retrieval pointer in the
prompt. Tests that iterate compressed arrays via subscript access
(e["level"], r["status"], i["labels"], m["text"]) hit KeyError on
the sentinel because it doesn't share the record schema.
Same root cause as the test_quality_retention fixes in PR8 -- these
integration tests were left out of that pass.
Ship a public helper headroom.transforms.smart_crusher.strip_ccr_sentinels
so tests can use it cleanly: `for e in strip_ccr_sentinels(entries):`
and production callers iterating compressed output get a single
canonical filter instead of inlining the _ccr_dropped check.
The 7 previously-failing tests in PR #292 CI now pass:
- langchain test_100_percent_errors_preserved_logs
- langchain test_errors_preserved_with_many_errors
- langchain test_search_results_with_query_term
- mcp test_all_log_errors_preserved
- mcp test_slack_significant_compression_with_content
- mcp test_database_error_status_preserved
- mcp test_github_bugs_partial_preservation
753 tests across the integration + transforms + retention suites pass
locally. Plugin manifests auto-bumped 0.13.3 -> 0.13.4 by the
sync-plugin-versions hook (unrelated to this fix).
The action was set to @stable, which installs whatever the latest
stable is (1.95.0 right now). Then maturin invokes cargo, which reads
rust-toolchain.toml and re-resolves to "1.95.0 + clippy + rustfmt".
rustup treats stable and 1.95.0 as distinct toolchain identities and
refuses the second install with:
failed to install component 'clippy-preview-x86_64-unknown-linux-gnu',
detected conflict: 'bin/cargo-clippy'
This was intermittent across the matrix (only test (3.10) tripped on
the most recent run; others got lucky on cache state). Pinning the
action ref to 1.95.0 makes both sides ask for the exact same toolchain
identity, so the second install is a no-op and the conflict can't fire.
Bump procedure stays the same: when rust-toolchain.toml's channel
changes, update these refs in lock-step.
Plugin manifests auto-bumped 0.11.0 -> 0.13.2 by sync-plugin-versions
hook (unrelated to the workflow fix).
Stage 3c.2 PR2. Adds an opt-in compaction stage that runs BEFORE the
existing lossy pipeline. When configured, it tries to losslessly
re-shape arrays of objects into a recursive Compaction IR and renders
that to bytes via a pluggable Formatter trait. When not configured
(default OSS), behavior is byte-equal with the pre-PR2 path — all 17
SmartCrusher parity fixtures stay green.
# What lands
- Recursive Compaction IR (`compaction/ir.rs`): Table / Buckets /
OpaqueRef / Untouched. CellValue can hold a nested Compaction so
multi-level cases (stringified-JSON inside cells, heterogeneous
arrays bucketed by discriminator, opaque blobs CCR-substituted)
share one tree shape.
- Cell classifier (`compaction/classifier.rs`): per-cell decision —
Scalar / JsonObject / JsonArray / StringifiedJson(parsed) /
Opaque(kind). Conservative: in doubt, return Scalar.
- TabularCompactor (`compaction/compactor.rs`): array → IR. Handles
uniform-nested flattening into dotted columns ("meta.region",
"meta.tier"), stringified-JSON parsing + recursion, opaque-blob
CCR-substitution (12-char SHA-256 prefix), and heterogeneous
bucketing by discriminator. Falls through to a sparse Table when
no clean discriminator exists, so we always do better than the
lossy path for object arrays.
- Formatter trait (`compaction/formatter.rs`) + two impls:
- JsonFormatter: structured JSON for debugging / programmatic use.
- CsvSchemaFormatter: [N]{col:type,col:type} declaration + CSV
rows. Steals TOON's row-count-and-shape declaration without
adopting TOON's bespoke escaping. CSV is the format LLMs are
strongest at — every model has seen millions of examples in
training. >30% smaller than raw JSON serialization on tabular
fixtures.
- Wiring (`crusher.rs`, `builder.rs`): SmartCrusher gains an optional
compaction stage. Builder methods with_compaction(stage) and
with_default_compaction() opt in. CrushArrayResult gets two new
fields (compacted, compaction_kind) populated only when the stage
runs. strategy_info becomes compaction kind when compaction won.
# Why this design
- Three-trait extension surface preserved. PR1 added Constraint /
Observer / Scorer; PR2 adds Formatter as the fourth pluggable
seam. Enterprise plug-ins land cleanly without forking core.
- Empty default builder rule held. SmartCrusherBuilder::new() still
produces a no-compaction crusher. with_default_compaction() is
the explicit OSS preset. No silent fallbacks.
- Recursive IR was the unlock. A flat table-of-scalars IR would have
collapsed the moment a cell held nested JSON. Making
CellValue::Nested hold another Compaction made stringified-JSON
parsing + heterogeneous bucketing + opaque substitution all share
one renderer pass.
- CCR substitution for opaque cells. Strings classified as
base64/HTML/long-opaque become structured markers keyed by 12-char
SHA-256 prefix. The full bytes round-trip via the CCR store (PyO3
bridge owns actual storage; this PR emits the marker and computes
the hash).
# Tests
- 60 new unit tests across IR / classifier / compactor / formatter /
wiring (448 total in headroom-core, was 388).
- 17/17 SmartCrusher parity fixtures byte-equal — default-config
path completely unchanged.
- 21/21 Python parity tests pass via PyO3 bridge.
- make ci-precheck green: ruff, mypy, cargo fmt/clippy/test
(1.95.0), commitlint.
# Deferred to follow-up PRs
- ToonFormatter (small; ship after eval harness compares formats)
- Diff/code detection in cells → routes to DiffCompressor /
CodeCompressor (coupled to ContentRouter Phase 4)
- Budget-aware row dropping (Constraint-respecting) when rendered
size exceeds budget
- Format A/B eval harness
- ContentRouter unification (Phase 4)
Modules: crates/headroom-core/src/transforms/smart_crusher/compaction/*, builder.rs, crusher.rs, mod.rs
The cosign signing step passed bake metadata via env var:
env:
BAKE_META: ${{ steps.bake.outputs.metadata }}
run: echo "$BAKE_META" | jq ...
For large bake targets (code-nonroot, runtime-code-nonroot) the
metadata JSON is large enough that combined argv+env at bash spawn
exceeds Linux ARG_MAX (~128 KiB on ubuntu-latest), so bash dies with
E2BIG before the script even runs.
Switch to writing metadata into a heredoc-backed temp file, then read
it via jq file input. Heredocs put the JSON in the script body itself,
which bash reads from a temp file (no ARG_MAX limit), bypassing the
env-size ceiling entirely.
Module: .github/workflows/docker.yml
