`CI / lint (pull_request)` failed because `ruff format --check` detected
formatting drift in the new PR governance script and its tests. This PR
aligns those files with repository formatting rules so the lint job can
pass.
- **Root cause**
- `ruff format --check .` reported two files as non-canonical:
- `scripts/pr-governance.py`
- `scripts/tests/test_pr_governance.py`
- **Change set**
- Applied `ruff` formatting to only the two flagged files.
- No behavioral or logic changes; edits are line-wrap/format
normalization only.
- **Representative update**
```python
parser.add_argument(
"--event", type=Path, required=True, help="Path to the GitHub event
payload JSON."
)
```
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
## Summary
- Lazy-import `create_app`/`run_server` in `headroom/proxy/__init__.py`
via PEP 562 `__getattr__` to prevent CLI crash when `fastapi` is not
installed (i.e., installed without `[proxy]` extras)
- Fix `.pre-commit-config.yaml` to use `python3` instead of `python`
(unavailable on macOS Homebrew)
- Add graceful `ImportError` skip in `scripts/sync-plugin-versions.py`
for environments without dev dependencies
Fixes#441
## Test plan
- [x] `headroom --help` works without `[proxy]` extras installed
- [x] `headroom proxy --help` works with `[proxy]` extras installed
- [x] `headroom proxy --port 18787` starts and serves traffic
- [x] Lazy imports resolve correctly: `from headroom.proxy import
create_app, run_server`
- [x] `AttributeError` raised for invalid attributes on `headroom.proxy`
- [x] Pre-commit hooks pass (ruff, ruff-format, mypy,
sync-plugin-versions)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Code Bot <claude-code@smartwatermelon.github>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
Replaces `chopratejas/kompress-base` with
**`chopratejas/kompress-v2-base`** as the default Kompress
text-compression model (the fallback for content not handled by
structured compressors), using a new **weight-only int8 ONNX** artifact
that is fp32-equivalent at 2.2x less memory.
## Why
v2 is the same dual-head ModernBERT (token classifier + span CNN),
LoRA-trained on the v2 dataset. The HF repo originally shipped PyTorch
weights only — pointing Headroom at it naively would have forced the
heavy `[ml]` (torch) path on every `[proxy]` install. We exported ONNX
artifacts reproducing the v1 loader contract (single `final_scores`
output) and published them to the HF repo.
## Eval (labeled dataset_v2 test split, n=500, threshold 0.5)
| artifact | size | f1 | must_keep_recall | keep_rate | fp32 agreement |
|---|---|---|---|---|---|
| fp32 (reference) | 601 MB | 0.9128 | 0.9770 | 0.8100 | 100% |
| **int8-wo (default)** | **261 MB** | **0.9130** | **0.9765** |
**0.8097** | **99.6%** |
| fp16 | 301 MB | 0.9129 | 0.9771 | 0.8099 | 99.9% |
| int4-wo | 230 MB | 0.9102 | 0.9825 | 0.8354 ⚠️ | 94.5% |
| int8-dynamic | 271 MB | 0.9041 | 0.9868 | 0.8857 ⚠️ | 90.5% |
Weight-only int8 (MatMulNBits) keeps activations fp32, avoiding the
upward score bias that makes dynamic int8 keep ~7% more tokens (≈40%
less compression savings). Quantized candidates were generated and
eval-gated by a Modal job in the kompress repo
(`modal_jobs/export_onnx_v2.py`) against the labeled test split.
## Changes
- Default model id → `chopratejas/kompress-v2-base`
- ONNX artifact resolution tries candidates in order (**int8-wo → fp32 →
v1 int8**), falling through on download miss **or session-load failure**
— onnxruntime builds without the MatMulNBits 8-bit kernel fall back to
fp32 instead of losing Kompress
- `HEADROOM_KOMPRESS_ONNX_FILENAME` env pins an exact artifact
- `scripts/export_kompress_v2_onnx.py`: reproducible fp32 export (loads
the merged v2 checkpoint, traces the `final_scores` contract, verifies
vs PyTorch)
- `.gitignore`: local `onnx/` artifacts dir; allowlist the export script
## Testing
- End-to-end: fresh `KompressCompressor().preload()` resolves int8-wo
from HF, loads on onnxruntime 1.23 (CPU, no torch), compresses with
error/traceback content preserved
- fp32 export verified lossless vs PyTorch (max |Δscore| = 0.0, 100%
keep agreement)
- ruff check + format clean, mypy clean, 63 targeted tests pass
## Problem
`.pre-commit-config.yaml` already has `ruff` + `ruff-format` configured,
and `pre-commit>=3.0.0` is already in `[dev]` deps — but `make
install-git-hooks` never called `pre-commit install`. Every
contributor's repo had the hook **config** but no running hook.
PR #772 merged with inline-comment spacing and import-order violations
that ruff would have caught automatically. The maintainer had to add a
separate fixup commit (`fix: format issue 728 regression test`) to clean
it up.
## Changes
**`scripts/install-git-hooks.sh`** — after installing the pre-push hook,
also run `pre-commit install`. Falls back to `.venv/bin/pre-commit` when
`pre-commit` is not on `PATH`, with a clear warning if neither is found:
```
✅ installed: .git/hooks/pre-push
Runs 'make ci-precheck' before every git push.
✅ installed: .git/hooks/pre-commit (ruff lint + format via pre-commit)
```
**`CONTRIBUTING.md`** — update PR workflow step 2 to mention `make
install-git-hooks` so contributors know to run it after `pip install`:
```
2. pip install -e ".[dev]" then make install-git-hooks — installs ruff on
every commit and ci-precheck on every push.
```
## No behaviour change for existing code
Only the local dev setup script is touched. Nothing in the proxy, tests,
or CI pipeline changes.
## Real behavior proof
- **OS**: macOS darwin arm64
- **Steps**: ran `bash scripts/install-git-hooks.sh` with venv
available, then attempted a commit with a badly-formatted file
- **Result**: ruff caught and auto-fixed it before the commit landed
```
✅ installed: .git/hooks/pre-push
Runs 'make ci-precheck' before every git push.
Bypass (use sparingly): git push --no-verify
pre-commit installed at .git/hooks/pre-commit
✅ installed: .git/hooks/pre-commit (ruff lint + format via pre-commit)
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
scripts/validate-workflows.sh exercises release.yml with an `act`
dry-run that posted a synthesized push-to-main event. After the
previous commit retired that trigger in favor of `release:
published`, the dry-run started failing in CI because release.yml
no longer responds to push events.
Simulate the new trigger instead: feed release.yml a
release-published event (.github/act/release-published.json) and
move the push-to-main dry-run onto release-please.yml — the
workflow that now owns that event.
PR #484 added branch-awareness to ``scripts/sync-plugin-versions.py``
(no-op on feature branches unless ``HEADROOM_SYNC_VERSIONS=1``). The
existing ``test_main_runs_plugin_only_version_sync`` test broke
because it didn't account for the new ``_should_sync`` gate — on a
feature branch the main() function early-returns and the subprocess
mock was never invoked, but actually the test broke earlier because
``_current_branch`` calls ``subprocess.run(..., capture_output=True,
text=True, check=False)`` and the test's lambda only accepted
``(command, cwd, check)``.
Fix: force ``_should_sync`` True in the existing test so it locks
the run-path, then add 4 new tests covering the branch-aware logic
itself (env override, main vs feature, git-unavailable defensive
no-op).
Three independent contract-pattern follow-ons bundled into one PR.
Same frozen-dataclass + factory + apply_to_tags + Rust-portable
shape that PR #473 / #477 / #483 established.
## (1) MemoryRanker + RecencyBoostRanker
Pre-this-PR Headroom ranked memory candidates by pure cosine
similarity. Every other memory system we surveyed (Letta, Mem0,
Cognee, Supermemory) re-ranks beyond cosine.
* ``MemoryRanker`` Protocol — pluggable re-ranker; future PRs add
source-weight + access-count rankers behind the same interface.
* ``RecencyBoostRanker`` — first concrete impl. Final score is
``cosine × exp(-age_days / decay_days)``. Default decay 30 days
(half-life ~21 days; 60-day-old factor 0.135, 90-day-old 0.050).
* ``MemoryCandidate`` — backend-agnostic frozen value type that
flows through the ranker. ``MemoryCandidate.from_backend_result``
adapter converts the existing ``MemoryResult`` shape (with nested
``memory.created_at``) into the ranker's flatter form.
* Wired into ``memory_handler.search_and_format_context`` as an
optional ``ranker=`` kwarg — backwards-compat: ``None`` (default)
preserves the pure-cosine path identically.
Defensive:
* ``created_at=None`` → factor 1.0 (recency-neutral, back-compat with
legacy rows / migrating backends)
* Negative age (clock skew) → clamped to factor 1.0 (a future-dated
row can't outrank a real fresh memory)
* Sort is stable on ties — same input → same output every turn, so
consecutive turns inject memories in the same order (prefix-cache
friendly)
Performance: O(N) over candidates where N=top_k≈10. One ``math.exp``
per candidate. Sub-microsecond. Zero new I/O.
## (2) ImageCompressionDecision
Mirror of :class:`CompressionDecision` for image compression. Two
sites today (``openai.py:1203``, ``anthropic.py:868``) gate inline;
both already respect bypass (no Gemini-class drift bug like text
compression had), but consolidating into a value type:
* Locks bypass-respect via AST contract test — future sites can't
drift on it
* Surfaces ``image_skip_reason`` in ``RequestOutcome.tags`` for
dashboard slicing (same observability surface as
``passthrough_reason`` and ``memory_skip_reason``)
* Same Rust-port shape as the other decision types
Precedence: ``bypass_header`` > ``image_optimize_disabled`` >
``no_messages`` > ``should_compress=True``.
Anthropic's extra ``is_cache_mode`` check stays inline because it's
Anthropic-specific (openai/gemini don't have it). Documented in a
code comment.
## (3) Branch-aware sync-plugin-versions hook
Pre-this-fix the pre-commit ``sync-plugin-versions`` hook ran on
every commit and bumped manifests to the predicted-next-release
version. Every PR ended up carrying the prediction as collateral
("Why are we bumping ``.claude-plugin/marketplace.json`` — we
should not, right??" - user, on PR #483).
Fix: the hook is now a NO-OP unless EITHER:
* We're on the ``main`` branch, OR
* ``HEADROOM_SYNC_VERSIONS=1`` is set explicitly (release workflow)
On feature branches the hook prints a single line explaining the
skip and exits cleanly. The release workflow opts in via the env
var; behaviour on main / at release time is unchanged.
## Test coverage
* 16 new tests on ``MemoryRanker`` / ``RecencyBoostRanker``
(frozen, equal cosine wins by recency, decay configurable, NULL
timestamp neutral, no-mutation contract, Rust-port shape)
* 17 new tests on ``ImageCompressionDecision`` (frozen, all 3
skip reasons, precedence, observability fields, apply_to_tags)
* 1 new AST invariant test (extends
``test_handler_outcome_tag_invariant.py``) — locks "no raw
``if self.config.image_optimize and messages and not _bypass:``
conjunction in any handler"
All existing memory + cache-stability + handler tests pass (203 ✓).
``make ci-precheck`` clean.
## Rust portability
All three new value types port cleanly to frozen Rust structs +
pure functions. Same migration pattern as ``CompressionDecision``
(already locked in for the SmartCrusher Rust port).
## Zero-regression contract
* Default ``ranker=None`` → memory_handler behaves identically to
pre-this-PR (pure cosine; no perf change)
* Image decision migration is identity at the bypass/optimize/messages
gate — no behaviour change, just contract consolidation
* Hook fix is no-op on feature branches (less churn) and unchanged
on main (release flow preserved)
CI ran the Codex compression scheduler stress test on python 3.10/3.11/
3.12/3.13 and all four versions failed with:
ModuleNotFoundError: No module named 'scripts.replay_codex_ws_load'
The test imports ``boot_proxy``, ``warmup``, ``replay_session``, ``Frame``,
and ``Scenario`` from the replay tool to drive 30 concurrent compression
calls and assert no p99 contention tail (the very regression this PR
fixes). The tool was untracked because ``scripts/*`` is gitignored by
allowlist — local-only tools work fine for measurement but CI cannot
import them.
Add the replay tool to the allowlist so it ships. Same pattern as
``scripts/smoke_issue_327.py`` and other single-purpose scripts already
on the allowlist. The tool is genuinely useful beyond this PR: it lets
any contributor reproduce the Codex slowness baseline numbers and
measure their own fix against the same workload shape.
Local re-run with the tool committed: 3 passed, 1 skipped — identical
to the pre-fix branch run.
Derive source-tree versions from release history so headroom --version no longer reports stale project metadata.
Restart stale idle proxies after an upgrade, but leave active sessions running to avoid interrupting ongoing conversations.
Remove _version.py from version-sync ownership and make version verification catch package/plugin manifest drift.
PR #396's X2 dry-run caught a wheel-import failure on the manylinux_2_28
floor matrix entry (both x86_64 and aarch64). Same class as #355:
ImportError: ... undefined symbol: __libc_single_threaded
`__libc_single_threaded` is a single-byte char added in glibc 2.32.
Newer libstdc++ (gcc 11+) reads it inside `__cxa_thread_atexit_impl`
to elide locking on the single-threaded fast path. ORT prebuilt static
archives compiled with gcc-14.2.1 against glibc-2.38+ headers bake in
the reference. Users with glibc < 2.32 hit ImportError on
`import headroom._core`.
Latent since the ORT artifact bump that started using gcc 14. X1 is
the gate that catches it at release time; X2 caught it at PR time —
exactly as designed.
Fix:
1. glibc_compat.c adds Section B: `char __libc_single_threaded = 0;`
Setting to 0 (multi-threaded) is safe; libstdc++ takes the locked
slow path. Setting to 1 would race in any multithreaded Rust wheel.
2. build.rs adds `-Wl,-u,__libc_single_threaded` so the shim's archive
members are pulled regardless of scan order.
3. audit_wheel_glibc_symbols.py POST_FLOOR_SYMBOLS adds the new
symbol — verified locally: the audit now rejects the failing
PR #396 wheel with the right message.
Issue #355: published headroom_ai-*-manylinux_2_28_*.whl fails to import on Ubuntu 22.04, Debian 11/12, Conda envs with libc < 2.38: 'ImportError: undefined symbol: __isoc23_strtoll'.
Root cause: ORT prebuilt artifacts (downloaded via fastembed's ort-download-binaries-rustls-tls feature) are compiled with gcc-14.2.1 on a glibc-2.38+ host and reference __isoc23_strtoll. Our manylinux build host has glibc 2.38 so the link succeeds; end users with older glibc don't.
Two-part fix: (1) crates/headroom-py/glibc_compat.c provides weak-alias definitions for __isoc23_strtol/strtoll/strtoul/strtoull delegating to the older strtol* family, compiled by build.rs on Linux/glibc only. The dynamic linker prefers glibc's strong symbol when present (>= 2.38) and falls back to ours when not (< 2.38). (2) scripts/audit_wheel_glibc_symbols.py is a release.yml gate that runs objdump -T on every Linux wheel and rejects any UND symbol whose required glibc version exceeds the wheel's manylinux floor.
Validated: the audit correctly rejects the actually-broken v0.20.26 wheel with a precise diagnostic. The shim itself is a tiny static link with zero runtime cost.
Regression tests in tests/test_release_workflows.py pin the shim's load-bearing pieces (.c file, build.rs trigger, [build-dependencies] cc dep) and the audit invocation in release.yml. Future drift fails at PR time, not in the next release.
This is the same bug class as PR #371 (rustls-everywhere). Each instance gets fixed; the audit gate now catches the *class* — any future static linkage that introduces a post-floor symbol gets blocked before publish.
Eliminates the dual-package architecture that was the root cause of #355.
`pip install headroom-ai` now produces ONE wheel containing both the Python
source (headroom/*.py) and the compiled Rust extension (headroom/_core.so).
No more separate `headroom-core-py` package, no more chicken-and-egg with
PyPI publication, no more wheelhouse / PIP_FIND_LINKS / composite-action
plumbing in CI.
This is the canonical pattern used by cryptography, polars, ruff,
pydantic-core, and other Rust-as-core Python packages. Honors the
"Rust as core engine" direction.
## What changed
- pyproject.toml: `[build-system]` swapped from hatchling to maturin.
`[tool.hatch.*]` deleted; `[tool.maturin]` added pointing at
`crates/headroom-py/Cargo.toml` for the cdylib. `python-source = "."`
picks up the root `headroom/` package directly (dashboard HTML
templates and other non-Python files included automatically).
- crates/headroom-py/pyproject.toml: deleted. The crate is no longer a
separate published package; its Cargo.toml stays as the cdylib build
target invoked via `[tool.maturin] manifest-path`.
- crates/headroom-py/python/: deleted (placeholder layout for the old
separate package).
## CI updates
- ci.yml: `test` / `test-extras` / `test-agno` jobs simplified — Rust
toolchain set up before `pip install -e .` (which now invokes maturin
via build-system). Removed the "build wheel + symlink .so" dance.
`build` job swapped from `python -m build` (hatch) to
`maturin build` + `maturin sdist`.
- release.yml: collapsed dual-package matrix into one. New `build-wheels`
matrix produces cross-platform wheels for cp310/11/12/13 ×
{linux x86_64, linux aarch64, macos x86_64, macos aarch64}. New
`collect-dist` aggregator merges artifacts. publish-pypi consumes the
merged dist.
- init-native-e2e.yml: dropped windows-latest from the matrix —
upstream `esaxx-rs` (/MT) and `ort-sys` (/MD) link with conflicting
MSVC C runtime libraries, so the Rust extension cannot build for
win_amd64 today. Tracked as a follow-up; not a blocker for Linux+macOS.
- headroom-e2e-setup: composite action now sets up Rust toolchain +
Swatinem/rust-cache before `pip install -e .[proxy]`.
- eval.yml, publish.yml, rust.yml: same pattern — rust toolchain before
install. rust.yml's wheels job builds from root pyproject.toml (no
more `-m crates/headroom-py/Cargo.toml`).
- e2e/init/Dockerfile, e2e/wrap/Dockerfile: install rust + maturin in
the build stage; copy `crates/` + workspace `Cargo.toml/lock` so the
install can build the extension. Dropped `HEADROOM_REQUIRE_RUST_CORE=false`
from wrap-e2e — the image now ships the full Rust core.
- Dockerfile (main): simplified — no more Layer 2/3 dance with
`headroom-core-py` install + symlink. Single `uv pip install` builds
+ installs everything.
- .devcontainer/Dockerfile: rust toolchain + libssl-dev + maturin
added so `uv sync` builds the extension inside the devcontainer.
## Lockfile + script
- uv.lock: regenerated. No `headroom-core-py` entries remain.
- scripts/build_rust_extension.sh: simplified from a symlink-into-tree
workaround to a thin wrapper around `pip install -e .`. The maturin
build-backend handles placement automatically.
## Local validation (all green on macOS aarch64)
1. Clean venv `pip install -e .` → `from headroom._core import …` works.
2. `maturin build --release` → 13.8 MB wheel, 336 files including
`headroom/_core.cpython-311-darwin.so` (32 MB cdylib) and
`headroom/dashboard/templates/dashboard.html`.
3. `pip install <wheel>` in fresh venv → import works.
4. Wheel contents verified via `unzip -l`.
5. `pytest tests/test_transforms/test_diff_compressor.py` — 29 passed.
6. `pytest tests/test_relevance.py` — 30 passed.
7. `cargo build --workspace` + `cargo test --workspace` — all green.
8. `make ci-precheck` — 176 Python tests + Rust + commitlint green.
## Migration notes
Users on `pip install headroom-ai` get the Rust core automatically
(linux + macos wheels). sdist installs require rust toolchain available
locally — pip will build via maturin.
Closes#355
Supersedes #357 (workarounds-based fix abandoned in favor of
architectural fix)
Production incident (Finding #2 of HEADROOM_PROXY_LOG_FINDINGS_2026_05_03.md):
on this customer's deployment the Rust extension `headroom._core` was
never installed into the runtime Docker image. Diff compression failed
54 times in a single day; "Optimization failed: ModuleNotFoundError" hit
379 times. The failure rate climbed every day and reached ~223/day on
2026-05-03 — effectively 100% of requests on the Rust path. Every Rust
PR we'd merged (MessageScorer, ICM, DiffCompressor, etc.) was providing
zero customer value because the module wasn't loadable at all.
Root cause: the Dockerfile builder stage installed Python deps and the
in-tree `headroom-ai` package but never ran `maturin build` for the
`headroom-py` crate, so the runtime image shipped without `_core.so`.
The Python proxy continued to start because the extension's absence is
caught and routed through Python-only fallbacks that either silently
no-op or raise per-request.
This change makes that mode impossible by default:
* `headroom.proxy.server._check_rust_core()` runs as the first step of
the FastAPI lifespan. If the import fails it prints a structured
diagnostic, logs `event=rust_core_missing`, and calls `sys.exit(78)`
(sysexits.h `EX_CONFIG`). Process supervisors (systemd / k8s /
docker) treat this as a deliberate config error and stop restart
loops.
* `HEADROOM_REQUIRE_RUST_CORE=false` is the explicit opt-out for
Python-only `pip install -e .` developer flows; lifespan logs
`event=rust_core_disabled` and continues. Any other value (including
unset) keeps the fail-loud default.
* `/health` now surfaces `rust_core: "loaded" | "disabled" | "missing"`
(plus `rust_core_error` when non-loaded) so operators can alert on
the degraded state rather than discovering it via a customer ticket.
* `scripts/build_rust_extension.sh` is the single dev-time path: build
→ install → import-verify with the same `hello()` marker the lifespan
checks. Failures are loud at every step.
* `Makefile` exposes the script as `make verify-rust-core`.
* `Dockerfile` now installs `rustup` + `maturin`, builds the wheel from
`crates/headroom-py`, force-installs it into site-packages, and runs
the same `hello()` import-verify in the build image so a broken build
fails the docker-build, not the next runtime restart.
Tests:
* `tests/test_rust_core_smoke.py` pins all four contracts:
- `_core.hello()` returns `"headroom-core"`
- missing extension + default env → `SystemExit(78)`
- missing extension + opt-out env → lifespan starts, `/health`
returns `rust_core: "disabled"` with the underlying error
- present extension + default env → `("loaded", None)`
Per-finding-#2: ~/Desktop/HEADROOM_PROXY_LOG_FINDINGS_2026_05_03.md.
Adds an opt-in compression interceptor that buffers Anthropic
/v1/messages requests, runs IntelligentContextManager over the
messages array, and forwards the (possibly trimmed) body upstream.
All other paths, methods, and content-types stay on the original
streaming passthrough — so existing operators see zero change.
Behaviour gates ALL must be true to buffer + compress:
- --compression flag (or HEADROOM_PROXY_COMPRESSION=1)
- method == POST
- path == /v1/messages
- Content-Type: application/json
- ICM constructed successfully at startup
Falls through to streaming on any failure: parse, missing fields,
unknown model, body-too-large. Compression must never break a
request — that's the safety contract.
Model context windows come from a vendored LiteLLM snapshot at
crates/headroom-proxy/data/model_prices_and_context_window.json
parsed once into an OnceLock<HashMap>. Refresh via
scripts/refresh_model_limits.sh. Rationale documented inline:
hardcoded tables silently rot; LiteLLM is the canonical source
the entire LLM-tooling ecosystem relies on.
New tests:
- 16 unit tests across compression::{anthropic, icm, model_limits}
- 5 integration tests: off-passthrough, on-short-passthrough,
on-oversized-trim, on-non-json-skip, on-non-llm-path-skip
Verification:
- cargo test --workspace -> 884 passed, 0 failed
- cargo clippy --workspace -- -D warnings -> clean
- cargo fmt --check -> clean
The Anthropic token-mode handler walked past prefix_tracker.frozen_message_count
whenever an upcoming tool_result's content-hash matched comp_cache._stable_hashes
or should_defer_compression returned True. That conflated content equality with
positional cache membership.
Anthropic's prefix cache is POSITIONAL: bytes 0..K cached, anything past K is
fresh. _stable_hashes is content-keyed and grows unbounded. In long Claude Code
sessions where tool_result content rhymes across turns (repeated system prompts,
repeated file reads, repeated tool descriptions), the walker advanced
frozen_message_count to len(messages) on every turn and the pipeline produced
transforms_applied=[] on 73% of requests in user SvenMeyer's reported session
(headroom-stats-2026-05-01.json: 74 of 101 eligible requests "prefix_frozen") —
even after the prior fix in 44944fb. The 15 requests that did compress averaged
21%, proving compression itself works when reached.
Fix: delete the walker. The freeze boundary is now
frozen_message_count = min(
prefix_tracker.frozen_message_count, # positional ground truth
comp_cache.compute_frozen_count(messages), # local cache lower bound
)
compute_frozen_count's use of _stable_hashes can only LOWER the freeze via the
min clamp, never raise it past prefix_tracker's value. For any position in the
gap [compute_frozen_count, prefix_tracker.frozen_count], recompressing produces
byte-stable output (compression is deterministic on input content), so
Anthropic's prefix cache stays valid.
Cross-handler verification:
* OpenAI handler (proxy/handlers/openai.py:358-382) does not have this walker
— uses only compute_frozen_count. Codex routes through OpenAI handler. Both
unaffected.
* Streaming and non-streaming both invoke anthropic_pipeline.apply() before the
upstream call. One fix covers both paths.
* Cache mode (is_cache_mode) takes the _extract_cache_stable_delta path and is
independent of the walker. Unaffected.
Tests: six new regression tests lock down the post-fix invariants — clamp to
min(prefix_tracker, compute_frozen_count); fresh tool_result whose hash matches
old _stable_hashes entry is not frozen; frozen prefix byte-stable across the
pipeline; 10-turn session produces non-empty compression suffix every turn;
streaming and non-streaming compute identical frozen_message_count; OpenAI
handler never calls the walker functions. Plus scripts/smoke_issue_327.py
(gated by RUN_LIVE_API=1) drives a 10-turn conversation against
api.anthropic.com in both shapes (string + list-of-blocks) and both modes
(streaming + non-streaming).
ci-precheck clean. 191 tests pass.
Follow-ups (separate PRs):
* Fix _cache perpetually empty (anthropic.py result.messages != working_messages
comparison rarely fires in token mode).
* Cap _stable_hashes with bounded LRU + 1h TTL — hygiene only after the freeze
gate is removed.
* List-shape tool_result content gates at content_router.py:1975 and
intelligent_context.py:657 (cluster A from the audit).
Five gates broke on the 2026-04-27 push of the smart_crusher branch.
Each is fixed below; the second half adds a `make ci-precheck` target
(plus an installable git pre-push hook) so the same dance never happens
again.
Failures fixed:
1. cargo fmt — 22 files had formatting drift introduced over the
stage 3c.1 work. `cargo fmt --all` reformatted them; no semantic
changes. `cargo test --workspace` still green (388 + supporting).
2. wheels job (macOS x86_64) — `fastembed -> ort -> ort-sys` does not
publish prebuilt ONNX Runtime binaries for `x86_64-apple-darwin`.
Removed that target from `.github/workflows/rust.yml`'s wheels
matrix. Apple Silicon (`aarch64-apple-darwin`) covers macOS
distribution; Intel macOS users can build from source. The matrix
now has 2 targets: linux x86_64 + macOS aarch64.
3. test-extras (relevance.py) — `tests/test_relevance.py::TestSmartCrusherIntegration`
constructs a `SmartCrusher`, which hard-imports `headroom._core`
since the python implementation was retired in stage 3c.1b. The
test-extras job didn't build the rust extension. Added the same
`maturin build + symlink` block the main `test` job uses.
4. smoke-test (eval.yml) — same root cause:
`compression_only.evaluate_ccr_lossless` instantiates a SmartCrusher.
Same fix: build the rust extension before the smoke test runs.
5. commitlint — three rules tripped:
- `subject-case` rejects PascalCase identifiers in subjects, but
the project deliberately names classes (SmartCrusher, HfTokenizer,
ContentRouter, DiffCompressor) in commit subjects. Disabled.
- `footer-leading-blank` is a warning that the wagoid action turns
into a CI failure; lines like `Module: foo.rs` in our bodies
match the conventional footer pattern and trip it. Disabled.
- `type-enum` doesn't include `parity`, but the project ships
parity-test infrastructure as its own concern (separate from
`test:`); added `parity` to the allowed types.
Pre-push verification — the prevention half:
`make ci-precheck` runs all of the above CI gates locally:
- `ci-precheck-rust`: cargo fmt --check + clippy + test --workspace.
- `ci-precheck-python`: builds the rust extension via maturin, then
runs the smart_crusher-affected python test files (185 tests across
test_transforms/, test_relevance*, test_ccr, test_acceptance,
test_critical_fixes, test_quality_retention).
- `ci-precheck-commitlint`: `npx commitlint --from origin/main --to
HEAD` against the same config CI uses. Skipped silently if npx is
not on PATH (install Node 18+ to enable).
`make install-git-hooks` (or `scripts/install-git-hooks.sh`) installs
a git pre-push hook that runs `make ci-precheck` automatically.
Bypass with `--no-verify` only when truly needed.
When new CI gates land in `.github/workflows/`, mirror them into a
`make ci-precheck-*` target. The Makefile is the local mirror of the
CI configuration; keeping them in sync is a load-bearing invariant.
Verification: `make ci-precheck` runs green on this commit.
The python `DiffCompressor` is replaced by a thin pyo3-backed shim that
delegates to `headroom._core.DiffCompressor`. There is no python
implementation and no env-var fallback — the wheel is a hard import.
Why now: opt-in defaults don't drive python retirement. Byte-equal
parity was already proven across 27 fixtures (stage 3a); keeping a
shadow python impl behind a flag is a permanent maintenance cost with
no operational benefit. Stage 3b deletes ~700 lines of python parser /
scorer / formatter code; the rust crate has its own coverage.
Surface preserved:
- `headroom.transforms.diff_compressor.DiffCompressor` — same class
name, same `__init__`, same `compress(content, context)` shape.
Returns python `DiffCompressionResult` dataclasses so call sites that
destructure with `asdict()` work unchanged.
- `DiffCompressorConfig` and `DiffCompressionResult` dataclasses kept.
- Sidecar `compress_with_stats(...)` exposes the rust-only
`DiffCompressorStats` (per-file hunk drops, context lines trimmed,
file_mode normalizations) for observability.
Removed:
- Python parser / scorer / formatter (~700 lines).
- Internal `DiffHunk` / `DiffFile` parser dataclasses (rust crate has
parallel coverage).
- 12 tests that probed `_parse_diff` / `_score_hunks` or the deleted
parser dataclasses. The 29 public-API tests in `test_diff_compressor.py`
remain and now exercise the rust backend through the same import path.
- `HEADROOM_DIFF_COMPRESSOR_BACKEND` env var — no longer meaningful.
Build:
- `scripts/build_rust_extension.sh` runs `maturin develop` and symlinks
the built `.so` into `headroom/` so `import headroom._core` resolves
past the in-tree package shadowing the maturin overlay.
- `.gitignore` excludes the symlinks and allowlists the build script.
Tests:
- 544 transforms pass; 33 rust-vs-fixture parity tests guard the pyo3
bridge against regressions (`tests/test_transforms/test_diff_compressor_rust_parity.py`).
- Mypy clean.
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but
74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings,
violating that contract. Every macOS/Linux clone reports these files as
"modified" on fresh checkout because git's diff engine sees the stored
bytes don't match the attribute contract, even though the working tree
and index match byte-for-byte.
Running `git add --renormalize .` rewrites each affected blob so the
stored form matches the attribute declaration. No semantic changes —
every affected file's diff is "N insertions, N deletions" with inserts
and deletes being the same lines modulo line endings.
Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub
blame skip this mechanical commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
asyncio.timeout was added in Python 3.11, but the project supports >=3.10.
The test_repro_codex_replay_smoke test was crashing on Python 3.10 CI with
"AttributeError: module 'asyncio' has no attribute 'timeout'".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add websockets to the dev extra so the repro harness smoke test can import
its websocket client dependency in the CI test matrix. Also apply ruff
formatting to the files the formatter check was rejecting so the 3.12 lint
job passes.
- Route --json output cleanly: JSON goes to stdout (machine-readable),
human-readable summary goes to stderr. Without --json, the human
summary stays on stdout as before. Makes piping the harness into
jq / other tools trivial without losing operator visibility.
- Add distinct exit codes so CI and shell wrappers can branch on the
failure class: 0 success, 1 crash, 2 proxy_unreachable, 3 livez
threshold exceeded, 4 warmup failed, 130 SIGINT (already correct).
Smoke test updated to assert EXIT_PROXY_UNREACHABLE instead of 1.
- Replace flat 50-250ms jitter in the Anthropic-client retry loop with
exponential backoff + 50-150% jitter
(base=250ms, max=5000ms, attempt counter). Matches the proxy's own
jitter_delay_ms helper; inlined to keep the script free of proxy
package imports.
Add a workflow-validation CI job that installs actionlint and act,
checks the release and Docker workflows against checked-in event
fixtures, and shares the same validation script developers can run
locally.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- publish-pypi: add permissions: id-token: write for OIDC trusted publishing
- publish-npm: add npm run build before npm publish for both packages
- publish-github-packages: add npm run build, use --registry for GPR
- version-sync: add update_openclaw_package_json to sync headroom-ai dep range
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Align Docker-native wrapper help and runtime behavior with the Python install contract, including persistent deployment metadata, baked install-image defaults, and explicit unsupported wrap targets.
Harden the Python persistent-install path with profile validation, safer provider-scope handling, Windows environment restoration, runtime parity improvements, and rollback-safe apply/update behavior.
Update README, Docker install docs, CI, and focused regressions to cover the Windows BOM failure, wrapper parity, compose coverage, and Docker-native wrap behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tighten Docker-native bash and PowerShell wrapper validation for wrap and proxy flows, pin the bash wrapper to the install-time interpreter, clean up failed persistent container starts, and extend docs, CI, e2e, and native installer coverage for persistent Docker installs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add host-managed OpenClaw wrap and unwrap flows to the Docker-native wrappers so the installed headroom script can configure the OpenClaw plugin on the host while keeping Headroom itself in Docker. Reuse hidden prepare-only hooks for OpenClaw config payloads, preserve existing plugin metadata on unwrap, and update the Docker-native and integration docs to reflect the supported flow.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add system-native install scripts and host wrappers for running Headroom from Docker while keeping wrapped tools on the host. Document the Docker-native path, add a complete CLI reference with help output and parity details, and add support for root help/version aliases and proxy env-based binding behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
