PR #411 reintroduced an older `--code-aware` is_flag option and a
duplicate `code_aware_enabled=` kwarg in the ProxyConfig call, which
collided with the canonical tristate `--code-aware/--no-code-aware`
introduced in #260. The result: every CLI entry point (`headroom proxy`,
`headroom wrap codex`, `headroom wrap claude`, etc.) raised at import
time:
File ".../headroom/cli/proxy.py", line 575
code_aware_enabled=code_aware or _get_env_bool(...)
SyntaxError: keyword argument repeated: code_aware_enabled
Removes:
- The legacy `@click.option("--code-aware", is_flag=True, ...)`
- The legacy `code_aware: bool` function parameter
- The duplicate `code_aware_enabled=` kwarg
Keeps the tristate `--code-aware/--no-code-aware` > env-var >
default-off resolver. Behavior is unchanged for all flag combinations
covered by tests/test_cli_proxy_env.py.
Test mocks for `run_server` updated to accept `**kwargs` to match
the real signature (config plus run-time options like print_banner).
Without this the four code-aware tests added in #411 raised
TypeError on each invocation.
Plugin marketplace/manifest version bump 0.21.5 → 0.21.7 carried in
this commit by the sync-plugin-versions pre-commit hook.
Every release since v0.20.16 has uploaded 12 wheels but no sdist. The
underlying failure is a 400 from PyPI:
400 License-File NOTICE does not exist in distribution file
headroom_ai-X.Y.Z.tar.gz at headroom_ai-X.Y.Z/NOTICE
Two-part regression:
1. The hatch -> maturin migration in 2a91cbb (single-wheel maturin build
backend, May 4) replaced `[tool.hatch.build.targets.sdist].include`,
which listed both `LICENSE` and `NOTICE`, with maturin's own include
directive that only carried `LICENSE` over. Maturin's PEP 639 license
auto-discovery still emits `License-File: NOTICE` into the sdist's
PKG-INFO (because NOTICE exists at the project root and matches the
default glob), so the sdist tarball declares a license file it
doesn't physically contain. PyPI's PEP 639 validator rejects with
400. Wheels were unaffected because maturin auto-injects both files
into `*.dist-info/licenses/`.
2. CI showed "publish-pypi" green for ~22 releases despite this break
because twine was bailing earlier with `400 File already exists` on
the wheels (the version detector kept computing the same v0.21.5).
PR #412 added `skip-existing: true` (May 6) to make wheel re-uploads
idempotent. With wheels now silently skipping, twine proceeded to
upload the sdist for the first time in three weeks - and the
dormant License-File error surfaced as a hard 400.
Fix:
- Add `NOTICE` alongside `LICENSE` in `[tool.maturin].include` for the
`sdist` format. Both files now ship in the tarball, matching what
PEP 639 already declares in PKG-INFO.
- Replace the existing "verify sdist contains LICENSE" check with a
generic "every License-File entry in PKG-INFO resolves to a real
tarball member" check. This catches the same bug class for any
future addition (COPYING, AUTHORS, etc.) without another bespoke
literal.
Verified locally:
$ maturin sdist --out dist
Including license file `LICENSE`
Including license file `NOTICE`
Including files matching "LICENSE"
Including files matching "NOTICE"
Built source distribution to dist/headroom_ai-0.9.1.tar.gz
$ tar -tzf dist/headroom_ai-0.9.1.tar.gz | grep -E '(LICENSE|NOTICE)$'
headroom_ai-0.9.1/LICENSE
headroom_ai-0.9.1/NOTICE
$ twine check dist/headroom_ai-0.9.1.tar.gz
Checking dist/headroom_ai-0.9.1.tar.gz: PASSED
Builds on PR #406 (HTTP /v1/responses PyO3) and PR #410 (WS first-frame
PyO3) which landed the binding for `compress_openai_responses_live_zone`.
This change closes the remaining gaps so every (provider × endpoint ×
auth-mode × streaming) combination compresses AND surfaces in the
dashboard.
Telemetry: extend the PyO3 binding return tuple from `(bytes, modified)`
to `(bytes, modified, tokens_saved, transforms_applied)` by adding
`CompressionManifest::tokens_saved()` and `transforms_applied()`
accessors on the existing manifest. The Python proxy populates
request-log telemetry from the binding output instead of recounting
tokens. Updates the existing 2-tuple call sites in HTTP and WS
first-frame, plus the unpacks in tests.
WebSocket multi-frame compression: subscription Codex users keep a
long-lived WS open and send multiple `response.create` events per
session. PR #410 only compressed the first frame; subsequent frames
went raw. Added `_maybe_compress_response_create_frame` closure inside
`_client_to_upstream` that runs the same Rust dispatcher on every
client→upstream `response.create` text frame, passes other event
types (response.cancel, session.update, etc.) through unchanged, and
accumulates `tokens_saved` / `transforms_applied` /
`ws_frames_compressed` counters across the session.
Pre-existing dashboard gap: `streaming.py` and `anthropic.py` write
`RequestLog` entries; the non-streaming OpenAI HTTP and WS handlers
did not. Result: /transformations/feed was invisible for every Codex
turn and every Cline / OpenClaude / Aider turn. Added the same wiring
in `handle_openai_chat` (non-streaming), `handle_openai_responses`
(non-streaming HTTP), and `handle_openai_responses_ws` (session-end).
All three populate `auth_mode` + `endpoint` tags so the dashboard can
break compression activity down by client class (PAYG / OAuth /
Subscription) and surface (`chat_completions` / `responses_http` /
`responses_ws`). The WS metric record is now unconditional — was
previously gated on `tokens_saved > 0`, so first-frame no-changes
never registered.
compute_frozen_count over-freeze for prose-format clients:
`compute_frozen_count` walked until it found an unstable
`tool_result` / `role: "tool"` block. Cline / OpenClaude / Aider —
clients that embed tool calls as XML inside plain text — never
produce such a boundary, so the function returned `len(messages)` and
the pipeline froze 100% of messages including the brand-new user
turn. Live zone empty → `Transform content_router: 16414 → 16414
tokens (saved 0)`. Reported on Discord 2026-05-07 with Cline+DeepSeek.
Fix: cap at `max(0, len(messages) - 1)`. Updates 3 existing test
assertions whose expected values encoded the old over-freeze. Adds 6
new prose-format invariant tests.
CodeQL "clear-text logging of sensitive information" fix:
`tests/e2e_real_compression.py` previously stored API keys in local
variables in the same scope as diagnostic prints, which CodeQL flagged
via data-flow analysis. Refactored to read keys from `os.environ`
inside the request helper — the credentials never enter the runner's
main scope, so the taint flow never reaches the print.
End-to-end verification with real keys (.env):
/v1/messages (PAYG, non-stream) tok 14109 → 969 saved 13140
/v1/messages (PAYG, stream) tok 14109 → 969 saved 13140
/v1/chat/completions (PAYG, non-stream) tok 18460 → 1374 saved 17086
/v1/chat/completions (PAYG, stream) tok 18460 → 1374 saved 17086 (cache_hit=100%)
/v1/responses HTTP (PAYG, non-stream) bytes 50138 → 597 saved 18391
/v1/responses WS (frame 1) bytes 46429 → 488 saved 16791
/v1/responses WS (frame 2 multi) bytes 46429 → 488 saved 16791
/v1/responses WS (response.cancel) passthrough untouched
Tests: cargo workspace + pytest (4846 pass, 0 fail), make ci-precheck
passed, two E2E scripts (multi-turn HTTP, WS fake-upstream) all green.
Every push to main since v0.21.5 was first published has failed the
publish-pypi job with `400 File already exists`. The workflow's
detect-version step has been computing v0.21.5 repeatedly (the
canonical+commit-height algorithm hasn't bumped past it for the
recent fix-only commits), so each run rebuilds the same wheels with
the same version and twine rejects the duplicates.
Failed runs:
- 25443521479 (PR #406 merge, 15:04 UTC)
- 25452026402 (PR #409 merge, 17:55 UTC)
- 25452038283 (next push, 17:56 UTC)
PyPA's recommended pattern for this scenario is `skip-existing: true`
on the publish action — duplicate uploads become no-ops, fresh
versions still publish normally. Idempotent.
This unblocks main without touching the version-detection algorithm.
A follow-up audit of `headroom/release_version.py` is the right
deeper fix (so a series of `fix:` commits between releases produces
a sequence of patch bumps), but that's a deeper investigation; this
patch just stops the publish job from going red on every push.
Effect after this lands:
- Push to main → wheels rebuilt with whatever version detect-version
computes
- If that version's wheels are already on PyPI → twine skips them,
exit 0, downstream jobs (publish-npm, publish-docker, create-release)
run normally
- If detect-version computes a NEW version not on PyPI → wheels
publish as before, no behaviour change
PR-C5 (May 3) retired the Python `/v1/responses` compression pipeline
with the comment "Rust handles item-aware compression natively" — but
the standalone `crates/headroom-proxy` binary that was supposed to do
that compression is not deployed by the CLI today (`headroom proxy`
and `headroom wrap codex` both run only the Python proxy via uvicorn).
Result: every `/v1/responses` request since v0.20.16 has been
forwarded uncompressed. Codex CLI is the flagship consumer of this
endpoint; this is the regression users have been reporting.
Closes Bug 1 of the Codex regression by exposing the existing
`headroom_core::transforms::compress_openai_responses_live_zone` as
a PyO3 binding so the Python proxy can call the live-zone dispatcher
in-process. The `headroom._core` extension is already loaded at
proxy startup (PR-A0 verifies), so adding one more callable is
mechanical.
Why PyO3 inline (Layer 1) vs originally-intended two-process chain
(Layer 2): the inline call requires zero deployment changes — the
wheel already ships `headroom._core`. Layer 2 (build + ship the
standalone `headroom-proxy` binary, teach CLI to spawn both
processes) is the right long-term move; Layer 1 restores v0.5.21
functional behaviour today.
# Returns
`(body, modified)`. On change → `(new_body_bytes, True)`; on
passthrough → `(input_bytes, False)`.
# Failure mode
Never raises. The dispatcher's `LiveZoneError` cases (body not JSON,
no input array) are passthrough conditions matching the Rust proxy's
`compress_openai_responses_request` contract.
# Tests
14 new tests in `tests/test_responses_pyo3_compression.py`:
binding exposed, passthrough cases, every F1 AuthMode variant,
empty-model default, no-raise on garbage bytes.
The smoke-test job in .github/workflows/eval.yml has been silently
broken on every PR that touches headroom/transforms/**, evals/**, or
compress.py. Root cause: the public chopratejas/headroom repo has zero
Actions secrets configured (`gh api repos/chopratejas/headroom/actions/secrets`
returns `{"total_count":0,"secrets":[]}`), so OPENAI_API_KEY resolves
to the empty string and `openai.OpenAI()` raises OpenAIError at client
init before any compression code runs.
Fix: gate the live OpenAI eval step on a non-empty key and emit a
GitHub `:⚠️:` annotation when skipped, so the skip is loud in
the run summary (per the project's "no silent fallbacks" rule). The
CCR round-trip step above remains the mandatory gate — it tests real
compression logic with zero external dependencies.
Operators who DO wire OPENAI_API_KEY as a repo secret get the live
eval as before.
Test plan:
- yaml syntax validated locally
- Will re-run on PR #400 push; expect smoke-test to pass with the
:⚠️: annotation visible in the run summary.
Refs: F2.1 (unblocks merge of #400)
PR #396's first dry-run on its own changes failed three smoke matrix
entries — two fixed by PR #397's __libc_single_threaded shim (now on
main, this PR is rebased on top), one orthogonal: ubuntu:20.04 + 3.10.
Failure mode on ubuntu:20.04 + 3.10: deadsnakes PPA install path
stopped reliably provisioning python3.10-venv on focal once Ubuntu
20.04 hit End of Standard Support in May 2025. The error is from
apt-get, NOT from `import headroom._core` — the wheel never gets a
chance to load:
E: Unable to locate package python3.10-venv
Continuing to promise wheel-runtime correctness on glibc 2.31 in CI
would require either pulling ESM-tier ubuntu:focal images (paid /
auth-gated) or pinning a specific deadsnakes snapshot URL — neither
of which we want owning long-term.
Floor coverage we keep:
- manylinux_2_28_x86_64 (glibc 2.28 — the floor we promise)
- manylinux_2_28_aarch64 (glibc 2.28 — same)
- ubuntu:22.04 + 3.12 x86_64 (glibc 2.35 — issue #355's reporter env)
- ubuntu:22.04 + 3.12 aarch64 (glibc 2.35 — arm equivalent)
- macos-14 + 3.13 (Apple Silicon native)
Test pin in tests/test_release_workflows.py
(test_release_workflow_has_smoke_import_wheel_gate) only requires the
manylinux floors, ubuntu:22.04, and macos-14, so this drop doesn't
break the structural invariant.
The X1 smoke-import gate (PR #387) catches runtime symbol mismatches
on the wheel before publish, but only at release time. Recent break
patterns were upstream of that:
- #379 (docker bake `name=` regression in PR #376)
- #382 (sdist `os: ubuntu-latest` → `ubuntu-24.04` rename)
- #384 / #385 / #386 (glibc shim alias / link-order iterations)
- #387's own heredoc-indent regression that broke main on the FIRST
release run after merge — the heredoc was inside `bash -ec '...'`,
no PR-time check exercised it
X2 adds a `pull_request:` trigger to release.yml with a NARROW path
filter so the dry-run runs at PR time for changes that affect wheel
layout / release pipeline, but skips for source-only PRs to
`crates/headroom-core` / `crates/headroom-proxy`.
Path filter: release.yml, docker.yml, crates/headroom-py/**,
pyproject.toml, root Cargo.toml, Cargo.lock.
publish-pypi / publish-npm / publish-github-packages / publish-docker
/ create-release all gate on `github.event_name != 'pull_request'`,
so a PR run never publishes — the dry-run is build + collect-dist +
smoke-import only.
concurrency rules:
- PR runs: namespaced by PR number (`pr-N`), cancel-in-progress=true.
- main runs: namespaced by ref_name, cancel-in-progress=false (a
tag-push release that's mid-flight must not be cancelled).
Test pin covers all four invariants (trigger, path filter, publish
gates, concurrency split). 21 tests in test_release_workflows.py,
all green locally.
CodeQL flagged `compression_store.store()`'s default MD5 cache key as
`py/weak-sensitive-data-hashing` after the explicit_hash refactor in
PR #395 brought the line into a new diff context.
First attempt: `usedforsecurity=False` + `# lgtm[...]` comment to
silence the alert without changing the hash. Both failed — CodeQL
ignores the hashlib parameter and our LGTM marker, the alert stayed
open on the PR.
Second attempt (this commit): drop MD5 entirely. The cache key is for
deduplication / lookup, not security or integrity, so any deterministic
function works. SHA-256[:24] gives the same 96-bit collision space as
MD5[:24] (~280 trillion entries for 50% collision under birthday
bound), is FIPS-clean, and CodeQL won't flag it.
Behaviour impact: zero. The cache is in-memory (no disk persistence),
so the same content always hashes deterministically under whichever
function is in use — there is no upgrade-time mismatch to manage.
Drive-by: pre-commit's sync-plugin-versions hook bumped marketplace +
plugin manifests to 0.20.28 since v0.20.27 was tagged on main.
The X1 smoke-import job (PR #387) embedded the smoke check as a
`<<PY ... PY` heredoc inside `bash -ec '...'`. The outer bash
single-quote preserves whitespace, so the heredoc body and the
closing `PY` retained their YAML indentation (column 14 inside
`bash -ec`). Bash never found a column-0 `PY` and read past EOF:
bash: line 71: warning: here-document at line 65 delimited by
end-of-file (wanted `PY')
IndentationError: unexpected indent
Process completed with exit code 1
Caught immediately on the post-merge release run on main,
manylinux_2_28_x86_64 / Python 3.11 (glibc 2.28-floor):
https://github.com/chopratejas/headroom/actions/runs/25361396712/job/74362427755
`python -c "<f-string>"` was the obvious next try but reintroduces
single-quote nesting (Python f-strings need quote chars; outer
`bash -ec '...'` cannot contain unescaped single quotes).
Fix: write the smoke script to `${RUNNER_TEMP}/smoke_import.py`
in a new "Stage smoke-import script" step (one heredoc at YAML
`run: |` level — uniform indent strip works fine). Linux job
mounts it via `-v ${RUNNER_TEMP}/smoke_import.py:/smoke_import.py:ro`
and runs `python /smoke_import.py`. macOS host runs the same
file directly. No quoting drift between paths.
Locally validated:
- actionlint clean
- 20/20 tests in test_release_workflows.py pass
- hand-execution of the heredoc + script roundtrip works
This is the second X1 follow-up after PR #387's shellcheck fix.
The original X1 design's gap: no PR-time release dry-run that
would have caught the heredoc on PR #387 itself. X2 (PR-time
dry-run) is the structural fix.
actionlint+shellcheck flagged the bash heredoc on the macOS host
step (release.yml:486) for SC2012 (use find instead of ls) and
SC2086 (quote variables to prevent globbing/word splitting). The
Linux container step had the same pattern but escaped detection
because actionlint can't reach into `docker run bash -ec '...'`.
Replace `ls .../headroom_ai-*-${py_tag}-${py_tag}-${arch_tag}.whl
| head -1` with `find ... -name "..." -print -quit` so the glob
expansion happens in find (handles non-alphanumeric filenames
correctly) and the variables sit inside a quoted -name argument
(no SC2086 trigger). Also replace the diagnostic `ls -la` calls
with portable find variants (-printf for GNU find on Linux,
-exec basename for BSD find on macOS).
Net behaviour identical: still picks one matching wheel (only one
exists per py_tag+arch_tag), still prints all wheels for diagnosis
on miss.
Issue #355 plus the three follow-on hotfixes (#384/#385/#386) all
share a pattern: the wheel is technically valid (clippy passes,
tests pass, auditwheel is happy, the static-symbol audit added in
#384 is happy) but FAILS at runtime on a customer's box because of
a dynamic-link symbol mismatch. None of our pre-publish gates
actually `import headroom._core` on a representative customer
environment. They only build it.
What X1 adds
------------
A `smoke-import-wheels` job that runs after `build-wheels` and
before `publish-pypi` / `publish-docker` / `create-release`.
Matrix (6 jobs in parallel, ~3 min wall-clock):
- `manylinux_2_28_x86_64` + Python 3.11 (the floor we promise)
- `ubuntu:22.04` (glibc 2.35) + Python 3.12 (issue #355's env)
- `ubuntu:20.04` (glibc 2.31) + Python 3.10 (older LTS)
- `manylinux_2_28_aarch64` + Python 3.11 (aarch64 floor)
- `ubuntu:22.04` arm64 + Python 3.12 (aarch64 customer env)
- `macos-14` host + Python 3.13 (Apple Silicon)
Each job downloads its arch's wheel artifact, installs the wheel
matching its Python version inside the container, and runs the
exact command the proxy's `_check_rust_core` runs at startup:
from headroom._core import hello as _rust_hello
If any matrix entry fails, `publish-pypi` / `publish-docker` /
`create-release` are blocked. The matrix tells us exactly which
customer environment combination breaks.
Regression test in tests/test_release_workflows.py:
`test_release_workflow_has_smoke_import_wheel_gate` pins the job's
existence, the required matrix entries, and — critically — the
gating wires (publish-pypi / publish-docker / create-release all
need-and-require-success on the smoke job). A future "this slow
CI step always passes anyway, drop it" refactor fails at PR time.
Companion tests `test_glibc_compat_shim_present_in_headroom_py`
and `test_release_workflow_audits_wheel_glibc_symbols` (added in
#384) cover the static-symbol gate; this PR is the dynamic-link
gate. Both are needed.
Issue #355: published headroom_ai-*-manylinux_2_28_*.whl fails to import on Ubuntu 22.04, Debian 11/12, Conda envs with libc < 2.38: 'ImportError: undefined symbol: __isoc23_strtoll'.
Root cause: ORT prebuilt artifacts (downloaded via fastembed's ort-download-binaries-rustls-tls feature) are compiled with gcc-14.2.1 on a glibc-2.38+ host and reference __isoc23_strtoll. Our manylinux build host has glibc 2.38 so the link succeeds; end users with older glibc don't.
Two-part fix: (1) crates/headroom-py/glibc_compat.c provides weak-alias definitions for __isoc23_strtol/strtoll/strtoul/strtoull delegating to the older strtol* family, compiled by build.rs on Linux/glibc only. The dynamic linker prefers glibc's strong symbol when present (>= 2.38) and falls back to ours when not (< 2.38). (2) scripts/audit_wheel_glibc_symbols.py is a release.yml gate that runs objdump -T on every Linux wheel and rejects any UND symbol whose required glibc version exceeds the wheel's manylinux floor.
Validated: the audit correctly rejects the actually-broken v0.20.26 wheel with a precise diagnostic. The shim itself is a tiny static link with zero runtime cost.
Regression tests in tests/test_release_workflows.py pin the shim's load-bearing pieces (.c file, build.rs trigger, [build-dependencies] cc dep) and the audit invocation in release.yml. Future drift fails at PR time, not in the next release.
This is the same bug class as PR #371 (rustls-everywhere). Each instance gets fixed; the audit gate now catches the *class* — any future static linkage that introduces a post-floor symbol gets blocked before publish.
PR #376 pinned the wheel matrix's `os:` from `ubuntu-latest` to
`ubuntu-24.04` (explicit pinning, no semantic change). It silently
disabled the sdist build, whose conditional was
if: matrix.os == 'ubuntu-latest' && matrix.target == 'x86_64-unknown-linux-gnu'
The literal `'ubuntu-latest'` no longer matched, sdist never built,
`release-assets/*.tar.gz` was empty, and `create-release` failed at
gh release upload release-assets/*.tar.gz --clobber
with "no matches found". v0.20.22's release didn't get its sdist on
the GitHub Release.
Fix: key the conditional on `matrix.target` only. sdist is
platform-independent so any single matrix row is fine; using `target`
decouples the sdist build from any future host-runner rename.
Regression test in tests/test_release_workflows.py:
`test_sdist_build_conditional_keyed_on_target_not_os` pins the
target-only form. A future "let's add `os` back for clarity" refactor
will fail at PR time, not 8 minutes into a release.
This is a follow-up to PR #379 (docker bake `name=`) — same root
cause class: PR #376's matrix-shape changes silently broke a
downstream conditional whose literal value was tied to the OLD
matrix shape. Both fixes are now in place + pinned by tests.
PR #376's per-arch fan-out correctly removed `bake-file-tags` from
the docker-build step (tags belong on the multi-arch manifest, not
on per-arch images). But that left bake without ANY reference for
the push target — no tags AND no explicit `name=` in the output
spec. Every release-time docker-build job failed with the
misleading message:
ERROR: tag is needed when pushing to registry
Buildx's actual constraint is "no tags AND no `name=` in the
output = no push target." Since push-by-digest discards tags
anyway, the fix is to specify `name=<registry>/<image>` directly
in the `*.output` spec. Tags are still applied later, only on the
multi-arch manifest by `docker-manifest`.
Regression test in tests/test_release_workflows.py:
`test_docker_per_arch_build_specifies_image_name_in_output`
pins the `name=` substring so a future "the labels block already
has the registry, surely buildx can figure it out" refactor will
fail at PR time rather than 2 minutes into release.
GitHub-hosted Linux arm64 runners (`ubuntu-24.04-arm`) went GA in Aug
2025 and are free for public repositories. Switching the aarch64
wheel + the multi-arch docker matrix off `ubuntu-latest`+QEMU onto
the native runner cuts wall-clock on both surfaces.
release.yml — build-wheels matrix
* `aarch64-unknown-linux-gnu`: `ubuntu-latest` → `ubuntu-24.04-arm`.
maturin-action still runs inside `quay.io/pypa/manylinux_2_28_aarch64`,
but the container now executes natively on an aarch64 kernel
instead of through QEMU emulation. Aarch64 wheel build drops
from ~50–60 min to ~10 min.
* `x86_64-unknown-linux-gnu`: `ubuntu-latest` → `ubuntu-24.04`
(pin the moving alias for reproducibility; no semantic change).
docker.yml — fan-out + manifest merge
* Pre-#377: one `docker-variant-tags` matrix job per variant on
`ubuntu-latest`, using bake's `platforms = [amd64, arm64]` with
QEMU for the arm64 leg. ~1h per variant, 8 variants.
* Post-#377: split into `docker-build` (variant × arch = 16
parallel jobs, each on its native runner, single-platform
push-by-digest) and `docker-manifest` (per variant, merges the
two arch digests into a multi-arch tagged manifest with
`docker buildx imagetools create`, signs the index manifest
with cosign). Wall-clock drops from ~1h per variant to ~10 min.
* `docker/setup-qemu-action` removed — there's no QEMU left.
* Per-(variant, arch) GHA cache scopes so the two arches don't
collide on cache keys.
* `promote-latest` rewired to depend on `docker-manifest`.
Behavior change: cosign now signs only the multi-arch index digest
per variant, not each per-platform image. `cosign verify <repo>:tag`
(the typical flow) is unchanged because cosign resolves the tag to
the index digest. Verifiers pinning a specific per-arch digest will
need to verify the index digest instead.
Regression tests in tests/test_release_workflows.py:
* `test_aarch64_wheel_uses_native_arm64_runner` — pins the
aarch64 row to `ubuntu-24.04-arm` (and the amd64 row to
`ubuntu-24.04`, not `-latest`), so a future "let me unify on
ubuntu-latest" refactor surfaces the QEMU regression at PR time.
* `test_docker_workflow_builds_on_native_arch_runners` — pins the
fan-out matrix's arch entries, asserts push-by-digest, asserts
`setup-qemu-action` is absent from non-comment lines, asserts
the manifest-merge job exists.
Verified:
* Both workflow files parse as valid YAML with the expected job
graph (`docker-build` → `docker-manifest` → `promote-latest`,
16 fan-out jobs, 8 manifest jobs).
* `docker buildx imagetools inspect <tag> --format '{{ json . }}'`
exposes the index digest at `.manifest.digest` (confirmed via
Docker's official reference).
* `ubuntu-24.04-arm` is the correct GitHub-hosted runner label
(GA 2025-08-07, free for public repos).
* `make ci-precheck-rust` and `make ci-precheck-python` both pass
locally; `tests/test_release_workflows.py` is 15/15 green
(13 existing + 2 new).
# Root cause of the wheel-build cascade
We have shipped 5 release-pipeline hot-fixes in 12 hours, each
addressing a different symptom of the same architectural problem:
1. PR #363 — npm artifact downloads + tried `yum openssl-devel`
2. PR #367 — vendored OpenSSL in `headroom-proxy` + dropped Intel mac
3. PR #369 — Debian-cross perl install (`perl` not `libipc-cmd-perl`)
4. PR #370 — moved `openssl/vendored` from headroom-proxy to headroom-py
5. (this PR) — ELIMINATE OpenSSL entirely
Each fix exposed a different missing system package or feature flag in
a different build surface (manylinux x86_64 vs aarch64-cross-Debian vs
macOS Intel vs e2e/wrap Dockerfile vs e2e/init Dockerfile vs main
Dockerfile vs devcontainer). We were playing whack-a-mole because every
Cargo dep change to the OpenSSL surface required matching system-package
updates in 6+ different Dockerfiles and workflows, and the PR-level CI
didn't exercise all of them.
# Why this PR is the structural fix
`fastembed` exposes clean rustls feature flags:
- `hf-hub-rustls-tls` (replaces default `hf-hub-native-tls`)
- `ort-download-binaries-rustls-tls` (replaces default `…native-tls`)
By disabling fastembed's default features and enabling the rustls
variants explicitly, we remove `native-tls` (and therefore `openssl-sys`,
`openssl`, `openssl-src`, perl modules, OpenSSL build-time deps,
vendored OpenSSL ~30s build cost) from the entire workspace dep tree.
Verified locally:
$ cargo tree -p headroom-py -i openssl-sys
error: package ID specification `openssl-sys` did not match any packages
$ cargo tree -p headroom-py -i native-tls
error: package ID specification `native-tls` did not match any packages
$ cargo build --release -p headroom-py
Finished `release` profile [optimized] target(s) in 25.57s
(Down from 1m+ with vendored OpenSSL.)
# Cleanups enabled by this change
- crates/headroom-py/Cargo.toml — dropped the `openssl/vendored`
workaround from PR #370.
- crates/headroom-proxy/Cargo.toml — same dep removed.
- e2e/wrap/Dockerfile — dropped `yum install openssl-devel pkgconfig
perl-IPC-Cmd`. Comment retained explaining why.
- e2e/init/Dockerfile — same.
- Dockerfile (main) — dropped `pkg-config libssl-dev` from apt-get.
- .devcontainer/Dockerfile — dropped `pkg-config libssl-dev`.
- .github/workflows/release.yml — removed the entire before-script-linux
block (perl install probe + multi-package-manager dispatch + fail-loud
assertion). No longer needed.
# Regression gate
Three new structural tests in tests/test_release_workflows.py:
- test_no_openssl_sys_in_wheel_build_tree — runs `cargo tree -p <crate>
-i openssl-sys` for headroom-py / headroom-proxy / headroom-core. If
openssl-sys reappears (a future native-tls enabler creeping in via a
new dep), this fails AT PR TIME with an actionable message.
- test_no_native_tls_in_wheel_build_tree — same shape, native-tls is
the proximate cause.
- test_fastembed_uses_rustls_features — checks the Cargo.toml so a
future "let me bump fastembed and forget the features" doesn't
silently re-introduce OpenSSL.
Plus two cleanup gates:
- test_dockerfiles_no_longer_install_openssl_devel
- test_release_yml_does_not_install_openssl_or_perl_for_wheels
All 13 release-workflow tests pass. `make ci-precheck` PASSED.
# What this teaches us about rollouts (per user's ultrathink ask)
The 5-fix cascade exposed three meta-problems:
1. PR checks don't block merges. PR #370 had docker-init-e2e,
docker-wrap-e2e, docker-native-e2e all FAILED yet got merged.
Branch protection should require these checks. Operator action
needed (cannot fix in code).
2. Local validation is misleading. `cargo build -p headroom-py` from
the workspace root used the workspace lockfile and looked green;
CI did fresh resolution against headroom-py's manifest alone where
the feature wasn't enabled. Lesson: verify structural invariants
with `cargo tree -e features` before trusting that a build "works."
3. 6+ build surfaces with independent system-dep state. Every Cargo
change required matching updates in 6 places. The structural answer
(this PR) is to NOT depend on system OpenSSL at all. Where structural
fixes are not possible, the answer is a single shared
scripts/install-rust-build-deps.sh — but with this PR there's
nothing left to install.
Previous wheel hot-fix (#367) introduced a NEW failure mode that the
F1-merge release run surfaced:
E: Unable to locate package libipc-cmd-perl
The aarch64-unknown-linux-gnu maturin-action target does NOT use the
AlmaLinux 8 manylinux_2_28 image — it uses a Debian/Ubuntu-based
cross-compile container. The previous hot-fix's apt branch installed
`libipc-cmd-perl` which is a deprecated alias and is no longer in the
default Debian/Ubuntu sources. The build failed before openssl-src
could even start.
# What the script actually needs to do
`IPC::Cmd` is a Perl core module since 5.10, so any working `perl`
install provides it. The fix:
1. **Probe first.** `perl -MIPC::Cmd -e 1` exits cleanly when the
module is already importable — skip the install entirely. Some
manylinux images already ship it; others don't.
2. **Cover every package manager.** dnf (modern RHEL family) → yum
(older RHEL) → apt-get (Debian/Ubuntu) → apk (Alpine/musllinux).
The maturin-action uses different containers per (target, manylinux)
combo and we don't get to pick.
3. **Use `perl` not `libipc-cmd-perl` on Debian.** The plain `perl`
meta-package pulls `perl-modules-*` which contains IPC::Cmd. Works
on every Debian/Ubuntu version we'll see; `libipc-cmd-perl` is
gone from default sources.
4. **Fail loud after install.** `perl -MIPC::Cmd -e 'print "loaded
OK"'` runs unconditionally at the end. If somehow the module is
STILL missing, we fail here — not 5 minutes later in the
openssl-src compile step where the error message is harder to
debug. Matches the project's "no silent fallback" rule.
# Tests
`test_build_wheels_installs_perl_ipc_cmd_for_vendored_openssl`
updated to gate the new shape:
- Asserts `perl -MIPC::Cmd -e 1` pre-probe is present
- Asserts dnf/yum/apt-get/apk branches all exist
- Asserts apt branch installs `perl` not `libipc-cmd-perl`
- Asserts `libipc-cmd-perl` does not appear on any non-comment line
- Asserts the final `perl -MIPC::Cmd` fail-loud assertion is present
All 11 release-workflow tests pass. `make ci-precheck` PASSED.
The previous hot-fix (#363) addressed npm artifact downloads and added
openssl-devel installs in the manylinux container, but the wheel build
still fails on three of four matrix entries with three distinct errors:
1. ubuntu-x86_64 with `manylinux: auto` resolved to manylinux2014
(CentOS 7 / OpenSSL 1.0.2k). `openssl-sys 0.9` requires OpenSSL
1.1.0+ — "different version of OpenSSL was found".
2. ubuntu-aarch64 cross-compiles via `aarch64-unknown-linux-gnu-gcc`
from an x86_64 manylinux container. The `yum install openssl-devel`
we added installs x86_64 headers; `/usr/aarch64-unknown-linux-gnu/
include/` has no OpenSSL — "openssl/opensslv.h: No such file or
directory".
3. macos-15-intel fails on `ort-sys` (transitive via the ML compression
backend), which has no prebuilt ONNX Runtime binaries for
`x86_64-apple-darwin`. Unrelated to OpenSSL; an upstream limitation.
Why the workspace pulls openssl-sys at all: `hf-hub` (transitive via
`fastembed`) hard-codes `native-tls` as a default feature. Cargo's
feature unification then enables openssl-sys for the whole workspace
despite our `reqwest`/`tokio-tungstenite`/`tokio-rustls` preferences.
# Fix 1: vendored OpenSSL
Add `openssl = { version = "0.10", features = ["vendored"] }` to
`crates/headroom-proxy/Cargo.toml`. The `vendored` feature compiles
OpenSSL from source as part of the cargo build — works on every target
uniformly. Local build verified: cargo now pulls
`openssl-src v300.6.0+3.6.2` and compiles it. ~30s extra one-time
build cost.
The `openssl/vendored` feature DEFEATS `OPENSSL_DIR`. We therefore
remove the previous hot-fix's "Install OpenSSL (macOS)" step that
exported `OPENSSL_DIR` — leaving it would silently regress to the
system-OpenSSL path that broke originally.
# Fix 2: pin manylinux floor to 2_28
Change x86_64-unknown-linux-gnu from `manylinux: auto` to
`manylinux: 2_28` (matching aarch64 + the e2e Dockerfiles). This
isn't strictly required with vendored OpenSSL — the floor is now
glibc 2.28 / AlmaLinux 8 which has modern toolchain — but it removes
the CentOS-7 surface entirely and matches our runtime container
target.
# Fix 3: drop x86_64-apple-darwin from the matrix
`ort-sys 2.0.0-rc.12` has no prebuilt ONNX Runtime binaries for that
target. Building ORT from source would add CMake + ~5 minutes per
build. Apple Silicon macOS (`aarch64-apple-darwin`) is fully covered;
Intel-mac users install from the platform-independent sdist this
matrix also produces.
Tracked as a follow-up: switch the ML backend to `ort-tract` or
upstream a request for x86_64 macOS prebuilts.
# before-script-linux: keep perl-IPC-Cmd, drop openssl-devel
OpenSSL's vendored `Configure` script needs `IPC::Cmd` (without it
the build fails with "Can't locate IPC/Cmd.pm"). System
openssl-devel is no longer needed.
# Tests
4 new regression tests gate this:
- `test_headroom_proxy_vendors_openssl`
- `test_build_wheels_installs_perl_ipc_cmd_for_vendored_openssl`
- `test_build_wheels_does_not_set_openssl_dir`
- `test_build_wheels_matrix_excludes_intel_macos`
Plus the previous 7. All 11 release-workflow tests pass.
`make ci-precheck` PASSED. Local `cargo build --release -p headroom-py`
green.
Three independent failures on the post-merge release run for PR #360,
all introduced by the single-wheel maturin refactor:
1. Linux wheel matrix (`ubuntu-x86_64`, `aarch64-unknown-linux-gnu`)
failed inside the manylinux container with:
Could not find openssl via pkg-config
The system library `openssl` required by crate `openssl-sys`
was not found.
`openssl-sys` is transitive via `fastembed` → `hf-hub` → `ureq`
→ `native-tls`. The e2e Dockerfiles (`e2e/wrap`, `e2e/init`) we
wrote for #360 install `openssl-devel` upfront, but the
`build-wheels` matrix uses `PyO3/maturin-action@v1` which spins
up its OWN manylinux container that does not inherit those
installs. Fix: add a `before-script-linux:` to the action with a
yum/apt-get conditional so it works on RHEL-family (manylinux2014,
manylinux_2_28) and Debian-family musllinux variants.
2. macOS x86_64 wheel build failed with `maturin` exit 1 from the
same `openssl-sys` lookup. The aarch64 macos-14 runner happens
to have `/opt/homebrew/opt/openssl@3` on `openssl-sys`'s default
discovery path; the Intel macos-15-intel runner uses
`/usr/local/Cellar` which is NOT on that path. Fix: add a
pre-maturin step that runs `brew install openssl@3` and exports
`OPENSSL_DIR` / `OPENSSL_LIB_DIR` / `OPENSSL_INCLUDE_DIR` /
`PKG_CONFIG_PATH`. The aarch64 runner happily picks up the
explicit env vars too — no regression.
3. `publish-npm` and `publish-github-packages` both fail with
"Artifact not found for name: dist". Both jobs `npm pack` + `npm
publish` directly from the checked-out source tree — they never
consume the Python `dist` artifact. The `Download dist artifact`
step was vestigial dead code carried over from a prior workflow
shape; the only reason it didn't fail before #360 is that the
pre-refactor `build` job DID upload a `dist` artifact. Post-#360,
`dist` is produced by `collect-dist` and neither publish job is
gated on it (by design — npm vs PyPI ecosystems publish
independently). Fix: remove the dead download step from both
jobs. Loose coupling is preserved; `create-release` still gates
the GitHub Release tag on all of build / build-wheels /
collect-dist / publish-* succeeding.
Why the PR-level CI didn't catch any of this: `release.yml` only
runs on `push: branches: [main]`. PR #360's CI exercised `ci.yml`
which has a separate `ci-build-wheels-on-pr` matrix that uses a
different setup. The release surface only fires post-merge.
Tests added (regression gates):
- `test_build_wheels_installs_openssl_devel_on_linux_via_before_script`
- `test_build_wheels_resolves_openssl_dir_explicitly_on_macos`
- `test_npm_publish_jobs_do_not_download_dist_artifact`
All 9 release-workflow tests pass. `make ci-precheck` PASSED.
Code-scanning alert #65 (CodeQL actions/missing-workflow-permissions,
CWE-275) flagged the new build-wheels job for not declaring an explicit
permissions block. While at it, audit the rest of release.yml — same
gap exists on detect-version, build, collect-dist, and publish-npm.
Each job gets `contents: read` (the minimal default) since none of them
push, write packages, or mutate releases through GITHUB_TOKEN. Existing
write-bearing jobs (publish-pypi: id-token, publish-github-packages:
packages, create-release: contents) keep their narrower scopes.
Two early failures on PR #360's first CI run:
1. validate (default/memory-stack) + validate-worktree failed at the
apt-get update step in .devcontainer/Dockerfile. The base image
mcr.microsoft.com/devcontainers/python:1-3.12-bookworm ships with
dl.yarnpkg.com configured as an apt source whose GPG key has expired:
Err:4 https://dl.yarnpkg.com/debian stable InRelease
The following signatures couldn't be verified because the public
key is not available: NO_PUBKEY 62D54FD4003F6525
apt-get returns exit 100, the whole RUN aborts before pkg-config /
libssl-dev install. The maturin refactor doesn't need yarn, so drop
/etc/apt/sources.list.d/yarn.list before apt-get update. The debian
main repo updates fine on its own.
2. workflow-validation (actionlint) failed parsing rust.yml step name
"Build wheel (single-wheel architecture: builds headroom-ai)" —
actionlint's YAML parser saw the unquoted colon inside the name as
a mapping. Quote the string.
Eliminates the dual-package architecture that was the root cause of #355.
`pip install headroom-ai` now produces ONE wheel containing both the Python
source (headroom/*.py) and the compiled Rust extension (headroom/_core.so).
No more separate `headroom-core-py` package, no more chicken-and-egg with
PyPI publication, no more wheelhouse / PIP_FIND_LINKS / composite-action
plumbing in CI.
This is the canonical pattern used by cryptography, polars, ruff,
pydantic-core, and other Rust-as-core Python packages. Honors the
"Rust as core engine" direction.
## What changed
- pyproject.toml: `[build-system]` swapped from hatchling to maturin.
`[tool.hatch.*]` deleted; `[tool.maturin]` added pointing at
`crates/headroom-py/Cargo.toml` for the cdylib. `python-source = "."`
picks up the root `headroom/` package directly (dashboard HTML
templates and other non-Python files included automatically).
- crates/headroom-py/pyproject.toml: deleted. The crate is no longer a
separate published package; its Cargo.toml stays as the cdylib build
target invoked via `[tool.maturin] manifest-path`.
- crates/headroom-py/python/: deleted (placeholder layout for the old
separate package).
## CI updates
- ci.yml: `test` / `test-extras` / `test-agno` jobs simplified — Rust
toolchain set up before `pip install -e .` (which now invokes maturin
via build-system). Removed the "build wheel + symlink .so" dance.
`build` job swapped from `python -m build` (hatch) to
`maturin build` + `maturin sdist`.
- release.yml: collapsed dual-package matrix into one. New `build-wheels`
matrix produces cross-platform wheels for cp310/11/12/13 ×
{linux x86_64, linux aarch64, macos x86_64, macos aarch64}. New
`collect-dist` aggregator merges artifacts. publish-pypi consumes the
merged dist.
- init-native-e2e.yml: dropped windows-latest from the matrix —
upstream `esaxx-rs` (/MT) and `ort-sys` (/MD) link with conflicting
MSVC C runtime libraries, so the Rust extension cannot build for
win_amd64 today. Tracked as a follow-up; not a blocker for Linux+macOS.
- headroom-e2e-setup: composite action now sets up Rust toolchain +
Swatinem/rust-cache before `pip install -e .[proxy]`.
- eval.yml, publish.yml, rust.yml: same pattern — rust toolchain before
install. rust.yml's wheels job builds from root pyproject.toml (no
more `-m crates/headroom-py/Cargo.toml`).
- e2e/init/Dockerfile, e2e/wrap/Dockerfile: install rust + maturin in
the build stage; copy `crates/` + workspace `Cargo.toml/lock` so the
install can build the extension. Dropped `HEADROOM_REQUIRE_RUST_CORE=false`
from wrap-e2e — the image now ships the full Rust core.
- Dockerfile (main): simplified — no more Layer 2/3 dance with
`headroom-core-py` install + symlink. Single `uv pip install` builds
+ installs everything.
- .devcontainer/Dockerfile: rust toolchain + libssl-dev + maturin
added so `uv sync` builds the extension inside the devcontainer.
## Lockfile + script
- uv.lock: regenerated. No `headroom-core-py` entries remain.
- scripts/build_rust_extension.sh: simplified from a symlink-into-tree
workaround to a thin wrapper around `pip install -e .`. The maturin
build-backend handles placement automatically.
## Local validation (all green on macOS aarch64)
1. Clean venv `pip install -e .` → `from headroom._core import …` works.
2. `maturin build --release` → 13.8 MB wheel, 336 files including
`headroom/_core.cpython-311-darwin.so` (32 MB cdylib) and
`headroom/dashboard/templates/dashboard.html`.
3. `pip install <wheel>` in fresh venv → import works.
4. Wheel contents verified via `unzip -l`.
5. `pytest tests/test_transforms/test_diff_compressor.py` — 29 passed.
6. `pytest tests/test_relevance.py` — 30 passed.
7. `cargo build --workspace` + `cargo test --workspace` — all green.
8. `make ci-precheck` — 176 Python tests + Rust + commitlint green.
## Migration notes
Users on `pip install headroom-ai` get the Rust core automatically
(linux + macos wheels). sdist installs require rust toolchain available
locally — pip will build via maturin.
Closes#355
Supersedes #357 (workarounds-based fix abandoned in favor of
architectural fix)
Eliminate P3-33 / P3-34. Wraps every per-block compression in
the live-zone dispatcher with two new gates:
1. Per-content-type byte thresholds — pinned as `const` at the top
of `live_zone.rs` so the table is grep-able and reviewable in
one place. No magic numbers anywhere in the dispatch logic; a
`threshold_for(ContentType)` helper returns the value. Below
threshold → no compressor invoked, recorded as
`BlockAction::BelowByteThreshold { content_type, byte_count,
threshold_bytes }`. Thresholds:
- JSON-array tool_results: 1 KiB
- Build / log output: 512 B
- Search-result blocks: 1 KiB
- Git-diff blocks: 1 KiB
- Source code: 2 KiB (pinned for the future
Rust code-compressor port)
- Plain text: 5 KiB (pinned for Kompress wiring)
- HTML: 5 KiB (no compressor today)
2. Tokenizer-validated rejection — the byte-length proxy
(`compressed_bytes >= original_bytes`) is replaced with a
token-count check using `headroom_core::tokenizer::get_tokenizer`.
The dispatcher creates one tokenizer per request (model-aware
via the new `model: &str` parameter to
`compress_anthropic_live_zone`) and counts both the original
and compressed text. When `compressed_tokens >= original_tokens`
the candidate is rejected and the original bytes are kept.
`BlockAction::Compressed` and `BlockAction::RejectedNotSmaller`
gain `original_tokens` and `compressed_tokens` fields so the
proxy can log token-savings (the currency that actually matters
for prompt cache + provider billing) instead of bytes.
The proxy `live_zone_anthropic.rs` extracts `body["model"]` (or
falls back to `DEFAULT_MODEL = "claude-3-5-sonnet-20241022"` when
the field is missing — the chars-per-token estimator is calibrated
for the Claude family at 3.5 cpt) and threads it through. The
`Compressed` outcome now reports token counts from the manifest,
not byte counts, so the existing
`tokens_before / tokens_after` plumbing is suddenly accurate.
Tests added:
- `live_zone_thresholds.rs::below_threshold_no_compression_attempted`
— 200 B JSON array → `BelowByteThreshold` and `NoChange`.
- `live_zone_thresholds.rs::above_threshold_compression_attempted`
— 10 KB JSON array → byte-threshold gate clears and a compressor
runs (either `Compressed` or `RejectedNotSmaller`).
- `live_zone_token_validation.rs::compressed_more_tokens_falls_back`
— pathological input must not produce `Compressed` with
`compressed_tokens >= original_tokens`.
- `live_zone_token_validation.rs::compressed_fewer_tokens_accepted`
— well-formed JSON array of dicts → `Compressed` with strict
token shrinkage.
- Property test `live_zone_compression_token_count_non_increasing`
— for any well-formed body generated by `proptest`, the
dispatcher's emitted body has token-count <= input's token-count.
Pins the central PR-B4 invariant: the dispatcher never inflates
tokens.
Existing 12 unit tests in `live_zone.rs` and 6 integration tests
in `tests/live_zone_dispatch.rs` updated for the new field shape
and the `model` parameter; all pass. The diff-routing test's
fixture grew to 1.3 KiB so it clears the new GitDiff threshold
gate, exercising the dispatch path rather than short-circuiting.
Per-PR-B4 plan: REALIGNMENT/04-phase-B-live-zone.md.
PR-A6 of the Phase A cache-safety lockdown. Eliminates P5-50 and preps
P0-6 (memory tool injection toggling).
Two cache-killer patterns the merge + tracker defeat:
1. Mid-session mutation: when memory was enabled the proxy did an
ad-hoc concat of `context-management-2025-06-27` onto the client
value (anthropic.py:1244-1248). The order varied with the client
value, breaking byte-stable headers across turns.
2. Token drop-out across turns: clients (Claude Code, Codex CLI) MAY
drop a beta token between turn N and turn N+1 even when the proxy
mutated turn N to add it. The cache hot zone is positional, so the
next turn's prefix bytes hash differently and the prefix-cache
read misses.
Changes
-------
`headroom/proxy/helpers.py`
* `merge_anthropic_beta` / `merge_openai_beta`: pure, deterministic,
order-preserving merge. Client tokens first (in their original
order), then Headroom-required tokens (in the order passed). Dedupe
is case-insensitive but preserves the original casing of the first
occurrence. No regex.
* `SessionBetaTracker`: bounded LRU keyed by (provider, session_id),
unioning client tokens with previously-seen tokens. OrderedDict
LRU; threading.RLock for thread safety (mirrors the
CompressionCache pattern from compression_cache.py).
* `get_session_beta_tracker` / `_reset_session_beta_tracker_for_test`
process-wide singleton with test reset.
* `log_beta_header_merge`: structured log per cache-affecting merge.
* Env-var knobs (NO HARDCODES):
- HEADROOM_BETA_HEADER_STICKY=enabled|disabled (default enabled).
- HEADROOM_BETA_TRACKER_MAX_SESSIONS (default 1000).
`headroom/proxy/handlers/anthropic.py`
* After `compute_session_id` (line ~744): record client
`anthropic-beta` against the session tracker, write the sticky
value back into `headers` if changed. Order matters: sticky-merge
FIRST so memory-injection has the canonical baseline.
* Memory-injection site (line ~1244): replace the ad-hoc concat with
`merge_anthropic_beta(headers["anthropic-beta"], required_tokens)`.
`headroom/proxy/handlers/openai.py`
* Chat-completions (line ~360): record/merge `openai-beta`.
* /v1/responses HTTP (line ~1213): compute `_responses_session_id`
and record/merge `openai-beta`.
* /v1/responses WS (line ~1711): replace the ad-hoc absent-only
inject with `merge_openai_beta(sticky, ["responses_websockets=
2026-02-06"])`. Replaces any case-variants of the existing key.
Tests
-----
`tests/test_anthropic_beta_session_sticky.py` (26 tests):
* Pure helper: empty inputs, only-client, only-headroom, ordering,
dedupe casing, deterministic memory-injection order, no-double-
inject when token already present.
* Tracker: sticky-on across turns even when client drops, casing
preservation, provider namespace independence, LRU eviction at
max_sessions, env-var validation (loud failures), thread safety
under 16-thread concurrent access, blank-input rejection.
`tests/test_openai_beta_session_sticky.py` (17 tests):
* Mirror of the anthropic suite for `OpenAI-Beta`.
* Plus WS-specific coverage: sticky-then-merge of
`responses_websockets=2026-02-06` against client baseline.
`tests/test_openai_codex_routing.py`
* Add `session_tracker_store` stub to `_DummyOpenAIHandler` so the
routing tests still exercise the responses HTTP handler now that
it computes a session_id for beta-merge.
Notes
-----
Build constraints honored:
* Configurable: HEADROOM_BETA_HEADER_STICKY,
HEADROOM_BETA_TRACKER_MAX_SESSIONS.
* No regex, no hardcodes (env-var bounds), no fallbacks (disabled
mode is operator opt-in for diagnostics, loud failures on invalid
values).
* Structured tracing log via `log_beta_header_merge`.
Acceptance:
* 43 new tests pass.
* `cargo test --workspace` green (no Rust changes).
* `make ci-precheck` green.
Adds an opt-in compression interceptor that buffers Anthropic
/v1/messages requests, runs IntelligentContextManager over the
messages array, and forwards the (possibly trimmed) body upstream.
All other paths, methods, and content-types stay on the original
streaming passthrough — so existing operators see zero change.
Behaviour gates ALL must be true to buffer + compress:
- --compression flag (or HEADROOM_PROXY_COMPRESSION=1)
- method == POST
- path == /v1/messages
- Content-Type: application/json
- ICM constructed successfully at startup
Falls through to streaming on any failure: parse, missing fields,
unknown model, body-too-large. Compression must never break a
request — that's the safety contract.
Model context windows come from a vendored LiteLLM snapshot at
crates/headroom-proxy/data/model_prices_and_context_window.json
parsed once into an OnceLock<HashMap>. Refresh via
scripts/refresh_model_limits.sh. Rationale documented inline:
hardcoded tables silently rot; LiteLLM is the canonical source
the entire LLM-tooling ecosystem relies on.
New tests:
- 16 unit tests across compression::{anthropic, icm, model_limits}
- 5 integration tests: off-passthrough, on-short-passthrough,
on-oversized-trim, on-non-json-skip, on-non-llm-path-skip
Verification:
- cargo test --workspace -> 884 passed, 0 failed
- cargo clippy --workspace -- -D warnings -> clean
- cargo fmt --check -> clean
The @1.95.0 git ref of dtolnay/rust-toolchain shipped action code
that errors on ubuntu-latest with:
failed to install component: 'clippy-preview-x86_64-unknown-linux-gnu',
detected conflict: 'bin/cargo-clippy'
The runner's pre-installed Rust ships cargo-clippy at $HOME/.cargo/bin,
and the older action code's rustup invocation hits a path conflict
when adding the clippy-preview component for 1.95.0.
The @stable ref of the action has the fix; pass toolchain: 1.95.0 as
input so the version stays pinned. rust-toolchain.toml continues to
be the source of truth for the version (used by cargo's
auto-detection); this keeps the action's install in sync.
Replaces PR1's lossless/lossy split with ReformatTransform (pack
denser, no info lost) and OffloadTransform (drop bytes, CCR-stash
original via required cache_key). With CCR every transform is
information-preserving end-to-end, so the lossless/lossy distinction
misnamed the architecture.
OffloadTransform carries a cheap, structural estimate_bloat() method
scoped to its domain — generic byte-redundancy heuristics miss domain
semantics. The orchestrator runs reformat phase + per-offload bloat
estimation in parallel via rayon::join + par_iter, then runs offload
iff bloat clears threshold OR reformat underwhelmed.
Transforms shipped:
REFORMATS (lossless):
- JsonMinifier: serde_json round-trip whitespace stripping.
- LogTemplate: Drain-inspired order-preserving template miner.
Collapses consecutive runs of same-template lines into
[Template Tn: ...] (Nx) + variant table. Win comes from emitting
the constant-token prefix once instead of N times. Lossless: every
original line reconstructible from template + variants.
OFFLOADS (drop bytes, stash original via CCR):
- LogOffload: wraps existing LogCompressor; bloat = repetition x
uniqueness_weight + dilution x priority_dilution_weight.
- DiffOffload: wraps existing DiffCompressor; bloat = context-to-
change ratio. Bug-fix-on-port — persists original under the
cache_key the parity-bound DiffCompressor mints (closes a leak).
- DiffNoise: drops lockfile hunks (Cargo.lock, package-lock.json,
yarn.lock, etc., suffix list configurable in TOML) and
whitespace-only hunks. Stashes original via CCR for retrieval.
Search offload exists but is not in default re-exports — modern
agents (Claude Code, Codex) use scoped rg/grep, the marginal value
didn't justify default registration. Reach via the explicit module
path if opting in.
JSON Offload is intentionally absent from this PR — already lives at
SmartCrusher; Phase 3g PR3 wraps it in the OffloadTransform contract.
Thresholds and weights live in config/pipeline.toml, embedded via
include_str!; PipelineConfig::from_toml_str loads runtime overrides.
98 new pipeline tests; full headroom-core suite (714) and workspace
tests green; cargo fmt clean. No regex, per project convention.
Ports `headroom.transforms.log_compressor` to Rust. The biggest-by-
impact remaining compressor port: build/test logs are where the
10-50x compression wins live.
* Stack-trace state machine: per-flavor dispatcher (Python Traceback,
JS, Java, Rust error, Go); each flavor has its own termination
rule. Python terminated on any blank line, dropping mid-trace
lines from chained-exception traces.
* Conservative dedupe: preserves message prefix (everything before
first `:` or `=`); only trailing region is tokenised. Python's
blanket normalisation collapsed segfaults at different addresses.
* Loud CCR failures: `tracing::warn!` + `logger.warning` instead of
bare `except: pass`.
* `LogLevel::FAIL` documented as cosmetic-equivalent to ERROR.
Same shape as search_compressor port. Rust `LogCompressor`
orchestrates format detect -> classify -> score -> select ->
format -> CCR. Inline static-table format detector (YAGNI),
aho-corasick level classifier with word-boundary post-filter
(`signals::keyword_detector` technique), hand-rolled per-flavor
stack-trace state machine. `signals::LineImportanceDetector` NOT
consumed -- log levels are structural, not prose-style importance.
`headroom.transforms.log_compressor` becomes a thin shim:
`compress()` delegates to Rust end-to-end; internal helpers
preserved for the existing 50-test surface. Two existing tests
updated for new dedupe semantics + new compress orchestration.
* 17 Rust unit tests
* 50 Python tests pass
* `make ci-precheck` clean
Establish `crates/headroom-core/src/signals/` as a top-level module
holding cross-cutting detection traits. Phase 3e.1 ports
`error_detection.py` to a `LineImportanceDetector` trait + a
`Tiered<T>` combinator + a single concrete `KeywordDetector` impl
backed by aho-corasick. Three traits at three granularities are
sketched (line / blob / item); only line-importance is implemented
today.
Two bug fixes from the Python source bake into both the Rust impl
and the Python regex shim:
1. `ERROR_KEYWORDS` listed `timeout|abort|denied|rejected` but
`ERROR_PATTERN` regex omitted them. Lines like `"Connection
timeout"` were silently neutral despite the keyword being canonical.
Both surfaces now flag them.
2. `SECURITY_KEYWORDS` carried `token`, which false-positived on
every reference to LLM tokens (`input_tokens`, `tokens_saved`, ...)
in our own product. Dropped from the security set.
The Python `error_detection.py` shim now reflects keyword data out of
Rust via `keyword_registry_snapshot()` and recompiles the legacy
`re.Pattern` objects on the fly. Existing callers (text_compressor,
search_compressor, intelligent_context) continue to import the same
names with no source changes; caller migration to the trait API
happens in their own port PRs.
The trait architecture is the seam where a future ML detector slots
in without touching `KeywordDetector` or any caller. The canonical
extension is documented in `signals/README.md` as a classifier head
on the existing `bge-small-en-v1.5` embedder loaded by
`relevance::EmbeddingScorer` -- 384-dim -> 4-class softmax,
~1.5 KB head, ~1 ms inference, no extra model file. Two alternatives
(distilled tinyBERT in ONNX, logistic regression on lexical
features) are kept open in case BGE-head underfits.
Per the no-silent-fallbacks rule: only `KeywordDetector` lands as a
concrete impl. No NoOp, no MockDetector, no stub-ML -- those will
arrive with their real implementations.
Phase 3g (Compression Pipeline Formalization, issue #315) is queued
as the cross-cutting follow-up that will make lossless-then-lossy-
then-CCR ordering an explicit, observable architecture rather than
implicit per-compressor logic. Trait shapes there will reuse the
signals primitive landed in this PR.
Re-lands two audit fixes that were marked "merged" on GitHub but never
reached main: squash-merging the parent stack changed its commit SHA,
which silently dropped the contents of the stacked PRs (#301, #305).
Single PR this time — no stacking risk.
What lands:
1. **`enable_ccr_marker` Rust gate** — new field on `SmartCrusherConfig`
(default `true`). `crush_array` checks it before emitting the
`<<ccr:HASH>>` marker text and the CCR store write. PyO3 surface +
parity-fixture tolerance updated; recorded fixtures predate the
field and inherit the `true` default.
2. **Python shim collapses both flags to the gate** — both
`ccr_config.enabled=False` and `ccr_config.inject_retrieval_marker
=False` now flip the Rust gate off. Storing a payload nothing in
the prompt can reference is pointless, and storing under
`enabled=False` would be a surprise side effect the user
explicitly opted out of.
3. **Custom `scorer` / `relevance_config` fails loud** — replaces the
prior WARNING-and-drop. Silently dropping a user-supplied scorer
is a textbook silent fallback. `NotImplementedError` instead.
Verified zero production callers pass these args; full plumbing
arrives with Stage-3c.2's relevance-crate Python bridge.
Tests:
- 2 new Rust unit tests in `crusher.rs::tests`
- 6 new Python tests in `test_smart_crusher_toin_attachment.py`
(3 CCR marker-knob behaviors + 3 scorer fail-loud)
- Removed `test_ccr_inject_marker_false_logs_warning` (the WARNING is
gone now that the flag is honored)
- `make ci-precheck` green; eval suite + observability tests run
twice consecutively to verify no TOIN file pollution leaks into the
regular+coverage double-run on Python 3.11
RUST_DEV.md audit table reflects both gaps closed.
The PR8 marker injection appends a sentinel object
{"_ccr_dropped": "<<ccr:HASH N_rows_offloaded>>"} to the kept-items
array on the lossy path so the LLM sees the retrieval pointer in the
prompt. Tests that iterate compressed arrays via subscript access
(e["level"], r["status"], i["labels"], m["text"]) hit KeyError on
the sentinel because it doesn't share the record schema.
Same root cause as the test_quality_retention fixes in PR8 -- these
integration tests were left out of that pass.
Ship a public helper headroom.transforms.smart_crusher.strip_ccr_sentinels
so tests can use it cleanly: `for e in strip_ccr_sentinels(entries):`
and production callers iterating compressed output get a single
canonical filter instead of inlining the _ccr_dropped check.
The 7 previously-failing tests in PR #292 CI now pass:
- langchain test_100_percent_errors_preserved_logs
- langchain test_errors_preserved_with_many_errors
- langchain test_search_results_with_query_term
- mcp test_all_log_errors_preserved
- mcp test_slack_significant_compression_with_content
- mcp test_database_error_status_preserved
- mcp test_github_bugs_partial_preservation
753 tests across the integration + transforms + retention suites pass
locally. Plugin manifests auto-bumped 0.13.3 -> 0.13.4 by the
sync-plugin-versions hook (unrelated to this fix).
The action was set to @stable, which installs whatever the latest
stable is (1.95.0 right now). Then maturin invokes cargo, which reads
rust-toolchain.toml and re-resolves to "1.95.0 + clippy + rustfmt".
rustup treats stable and 1.95.0 as distinct toolchain identities and
refuses the second install with:
failed to install component 'clippy-preview-x86_64-unknown-linux-gnu',
detected conflict: 'bin/cargo-clippy'
This was intermittent across the matrix (only test (3.10) tripped on
the most recent run; others got lucky on cache state). Pinning the
action ref to 1.95.0 makes both sides ask for the exact same toolchain
identity, so the second install is a no-op and the conflict can't fire.
Bump procedure stays the same: when rust-toolchain.toml's channel
changes, update these refs in lock-step.
Plugin manifests auto-bumped 0.11.0 -> 0.13.2 by sync-plugin-versions
hook (unrelated to the workflow fix).
Stage 3c.2 PR2. Adds an opt-in compaction stage that runs BEFORE the
existing lossy pipeline. When configured, it tries to losslessly
re-shape arrays of objects into a recursive Compaction IR and renders
that to bytes via a pluggable Formatter trait. When not configured
(default OSS), behavior is byte-equal with the pre-PR2 path — all 17
SmartCrusher parity fixtures stay green.
# What lands
- Recursive Compaction IR (`compaction/ir.rs`): Table / Buckets /
OpaqueRef / Untouched. CellValue can hold a nested Compaction so
multi-level cases (stringified-JSON inside cells, heterogeneous
arrays bucketed by discriminator, opaque blobs CCR-substituted)
share one tree shape.
- Cell classifier (`compaction/classifier.rs`): per-cell decision —
Scalar / JsonObject / JsonArray / StringifiedJson(parsed) /
Opaque(kind). Conservative: in doubt, return Scalar.
- TabularCompactor (`compaction/compactor.rs`): array → IR. Handles
uniform-nested flattening into dotted columns ("meta.region",
"meta.tier"), stringified-JSON parsing + recursion, opaque-blob
CCR-substitution (12-char SHA-256 prefix), and heterogeneous
bucketing by discriminator. Falls through to a sparse Table when
no clean discriminator exists, so we always do better than the
lossy path for object arrays.
- Formatter trait (`compaction/formatter.rs`) + two impls:
- JsonFormatter: structured JSON for debugging / programmatic use.
- CsvSchemaFormatter: [N]{col:type,col:type} declaration + CSV
rows. Steals TOON's row-count-and-shape declaration without
adopting TOON's bespoke escaping. CSV is the format LLMs are
strongest at — every model has seen millions of examples in
training. >30% smaller than raw JSON serialization on tabular
fixtures.
- Wiring (`crusher.rs`, `builder.rs`): SmartCrusher gains an optional
compaction stage. Builder methods with_compaction(stage) and
with_default_compaction() opt in. CrushArrayResult gets two new
fields (compacted, compaction_kind) populated only when the stage
runs. strategy_info becomes compaction kind when compaction won.
# Why this design
- Three-trait extension surface preserved. PR1 added Constraint /
Observer / Scorer; PR2 adds Formatter as the fourth pluggable
seam. Enterprise plug-ins land cleanly without forking core.
- Empty default builder rule held. SmartCrusherBuilder::new() still
produces a no-compaction crusher. with_default_compaction() is
the explicit OSS preset. No silent fallbacks.
- Recursive IR was the unlock. A flat table-of-scalars IR would have
collapsed the moment a cell held nested JSON. Making
CellValue::Nested hold another Compaction made stringified-JSON
parsing + heterogeneous bucketing + opaque substitution all share
one renderer pass.
- CCR substitution for opaque cells. Strings classified as
base64/HTML/long-opaque become structured markers keyed by 12-char
SHA-256 prefix. The full bytes round-trip via the CCR store (PyO3
bridge owns actual storage; this PR emits the marker and computes
the hash).
# Tests
- 60 new unit tests across IR / classifier / compactor / formatter /
wiring (448 total in headroom-core, was 388).
- 17/17 SmartCrusher parity fixtures byte-equal — default-config
path completely unchanged.
- 21/21 Python parity tests pass via PyO3 bridge.
- make ci-precheck green: ruff, mypy, cargo fmt/clippy/test
(1.95.0), commitlint.
# Deferred to follow-up PRs
- ToonFormatter (small; ship after eval harness compares formats)
- Diff/code detection in cells → routes to DiffCompressor /
CodeCompressor (coupled to ContentRouter Phase 4)
- Budget-aware row dropping (Constraint-respecting) when rendered
size exceeds budget
- Format A/B eval harness
- ContentRouter unification (Phase 4)
Modules: crates/headroom-core/src/transforms/smart_crusher/compaction/*, builder.rs, crusher.rs, mod.rs
The cosign signing step passed bake metadata via env var:
env:
BAKE_META: ${{ steps.bake.outputs.metadata }}
run: echo "$BAKE_META" | jq ...
For large bake targets (code-nonroot, runtime-code-nonroot) the
metadata JSON is large enough that combined argv+env at bash spawn
exceeds Linux ARG_MAX (~128 KiB on ubuntu-latest), so bash dies with
E2BIG before the script even runs.
Switch to writing metadata into a heredoc-backed temp file, then read
it via jq file input. Heredocs put the JSON in the script body itself,
which bash reads from a temp file (no ARG_MAX limit), bypassing the
env-size ceiling entirely.
Module: .github/workflows/docker.yml
Five gates broke on the 2026-04-27 push of the smart_crusher branch.
Each is fixed below; the second half adds a `make ci-precheck` target
(plus an installable git pre-push hook) so the same dance never happens
again.
Failures fixed:
1. cargo fmt — 22 files had formatting drift introduced over the
stage 3c.1 work. `cargo fmt --all` reformatted them; no semantic
changes. `cargo test --workspace` still green (388 + supporting).
2. wheels job (macOS x86_64) — `fastembed -> ort -> ort-sys` does not
publish prebuilt ONNX Runtime binaries for `x86_64-apple-darwin`.
Removed that target from `.github/workflows/rust.yml`'s wheels
matrix. Apple Silicon (`aarch64-apple-darwin`) covers macOS
distribution; Intel macOS users can build from source. The matrix
now has 2 targets: linux x86_64 + macOS aarch64.
3. test-extras (relevance.py) — `tests/test_relevance.py::TestSmartCrusherIntegration`
constructs a `SmartCrusher`, which hard-imports `headroom._core`
since the python implementation was retired in stage 3c.1b. The
test-extras job didn't build the rust extension. Added the same
`maturin build + symlink` block the main `test` job uses.
4. smoke-test (eval.yml) — same root cause:
`compression_only.evaluate_ccr_lossless` instantiates a SmartCrusher.
Same fix: build the rust extension before the smoke test runs.
5. commitlint — three rules tripped:
- `subject-case` rejects PascalCase identifiers in subjects, but
the project deliberately names classes (SmartCrusher, HfTokenizer,
ContentRouter, DiffCompressor) in commit subjects. Disabled.
- `footer-leading-blank` is a warning that the wagoid action turns
into a CI failure; lines like `Module: foo.rs` in our bodies
match the conventional footer pattern and trip it. Disabled.
- `type-enum` doesn't include `parity`, but the project ships
parity-test infrastructure as its own concern (separate from
`test:`); added `parity` to the allowed types.
Pre-push verification — the prevention half:
`make ci-precheck` runs all of the above CI gates locally:
- `ci-precheck-rust`: cargo fmt --check + clippy + test --workspace.
- `ci-precheck-python`: builds the rust extension via maturin, then
runs the smart_crusher-affected python test files (185 tests across
test_transforms/, test_relevance*, test_ccr, test_acceptance,
test_critical_fixes, test_quality_retention).
- `ci-precheck-commitlint`: `npx commitlint --from origin/main --to
HEAD` against the same config CI uses. Skipped silently if npx is
not on PATH (install Node 18+ to enable).
`make install-git-hooks` (or `scripts/install-git-hooks.sh`) installs
a git pre-push hook that runs `make ci-precheck` automatically.
Bypass with `--no-verify` only when truly needed.
When new CI gates land in `.github/workflows/`, mirror them into a
`make ci-precheck-*` target. The Makefile is the local mirror of the
CI configuration; keeping them in sync is a load-bearing invariant.
Verification: `make ci-precheck` runs green on this commit.
Code review (`/code-review` on commit `d219bee`) caught one critical
bug, two important parity gaps, and a few quality nits. Fixed all of
them; all 135 unit tests pass; diff_compressor parity harness
unaffected (27/27 still matched).
# Critical fix — `hash_field_name` truncation length
Rust truncated SHA-256 to **16** hex chars; Python uses **8** (per
`smart_crusher.py:177`: `hashlib.sha256(...).hexdigest()[:8]`). 16-char
hashes would never collide with TOIN's 8-char `preserve_fields`,
silently disabling the entire `use_feedback_hints` cache lookup path.
Fix: `hex[..8]` instead of `hex[..16]`. Three pinning tests re-verified
against actual Python reference output. Doc comment now warns
explicitly that the length must match Python or TOIN lookups silently miss.
# Important fix — `python_int_parse` mirrors Python's `int()` semantics
`statistics.rs::detect_sequential_pattern` previously called
`s.parse::<i64>()`. Python's `int()` differs in three ways that affect
realistic payloads:
- strips ASCII whitespace (Rust's `parse` rejects)
- accepts leading `+` (Rust accepts; same)
- accepts PEP 515 underscores like `"3_000"` (Rust rejects)
A field with `[" 1 ", " 2 ", " 3 ", "4", "5"]` would parse all five
in Python (sequential = True) but only one in Rust (`nums.len() < 5`
→ False). Silent parity break.
Fix: new private `python_int_parse` helper that strips whitespace,
handles underscore separators, and rejects edge cases Python rejects.
Six new tests pin the behavior.
# Important fix — `python_repr` for `item_matches_anchors`
Python compares anchors via `anchor in str(item).lower()`. We were
using `serde_json::to_string(&item).to_lowercase()`, which differs in
three ways that affect substring matching:
- quote chars (`'` vs `"`)
- bool/null literals (`True`/`False`/`None` vs `true`/`false`/`null`)
- spacing (`key: value, ...` vs `key:value,...`)
Anchor `"none"` would match Python form but not JSON. Inverse for
`"null"`. Real divergence.
Fix: new private `python_repr` walks `serde_json::Value` and emits
Python-equivalent form. Plus enable `serde_json/preserve_order` at
workspace level so `Value::Object` preserves JSON parse order
(matching Python `dict` since 3.7).
# Suggestion fixes
- Classifier comment for `[True, False, 1] -> MIXED_ARRAY` now walks
both Python and Rust paths step by step.
- `ArrayAnalysis::field_stats` doc notes the BTreeMap vs Python-dict
order nuance for the analyzer port to resolve.
- Added regression tests for "all unparseable strings", "single int
among strings", fractional-step sequential, and the email-typo
pattern.
# Build / test
- `cargo build -p headroom-core` clean.
- `cargo clippy -p headroom-core -- -D warnings` clean.
- 135 unit tests in `headroom-core`, all passing (was 55).
- `cargo run -p headroom-parity run` — diff_compressor 27/27 still matched.
Stage 3c.1 — like-for-like Rust port of `headroom/transforms/smart_crusher.py`.
This commit lays the foundation: module layout, configuration, foundational
data types, and the simpler helpers (classification, hashing, anchors,
basic statistics). Subsequent commits add the analyzer, crushers, plan
execution, and the orchestrator.
# What's in this commit
`crates/headroom-core/src/transforms/smart_crusher/`:
- `mod.rs` — module entry, public re-exports, port narrative.
- `classifier.rs` — `classify_array` / `ArrayType` (dict/string/number/
bool/nested/mixed/empty). Direct port of `_classify_array`.
- `config.rs` — `SmartCrusherConfig` with defaults pinned to Python
byte-for-byte.
- `hashing.rs` — `hash_field_name` (SHA-256 truncated to 16 hex chars),
matches `hashlib.sha256(name.encode()).hexdigest()[:16]` exactly.
- `statistics.rs` — `is_uuid_format`, `calculate_string_entropy`,
`detect_sequential_pattern` (with **BUG #2 fix** — see below).
- `anchors.rs` — `extract_query_anchors`, `item_matches_anchors`. Five
regex patterns ported via `std::sync::LazyLock`.
- `types.rs` — `CompressionStrategy`, `FieldStats`, `CrushabilityAnalysis`,
`ArrayAnalysis`, `CompressionPlan`, `CrushResult`. Field-by-field
mirror of the Python @dataclasses so the PyO3 bridge in 3c.1b can
reconstruct them without manual translators.
# Bug #2 fixed in this commit (Python fix lands later in same PR)
`smart_crusher.py:444-448` — `_detect_sequential_pattern` calls
`int(string_value)` and silently strips zero-padding, so padded string
IDs like `["001", "002", ..., "100"]` get misclassified as a sequential
numeric pattern. Fix: track whether each parsed numeric value
originated as a string. If EVERY parsed value was a string, refuse to
flag as sequential. Mixed numeric+string fields still detect
correctly because the unambiguous numerics dominate. Test:
`bug2_zero_padded_strings_no_longer_misclassified`.
# What's NOT in this commit (subsequent commits)
- `SmartAnalyzer` — `analyze_array`, `_analyze_field`, `_detect_change_points`,
`_detect_pattern`, `_detect_temporal_field`, `analyze_crushability`,
`_select_strategy`, `_estimate_reduction`.
- The five array crushers (`_crush_array`, `_crush_string_array`,
`_crush_number_array`, `_crush_mixed_array`, `_crush_object`).
- Planning (`_compute_k_split`, `_create_plan`, `_plan_*` family).
- Orchestration (`_prioritize_indices`, `_deduplicate_indices_by_content`,
`_fill_remaining_slots`).
- `SmartCrusher` orchestrator class itself.
- Parity harness fixtures.
- The remaining 3 Python bug fixes (#1, #3, #4) — landed alongside the
code paths they affect.
# Build / test
- `cargo build -p headroom-core` — clean.
- `cargo clippy -p headroom-core -- -D warnings` — clean.
- 55 new unit tests across the 6 new files, all passing.
Architectural improvements (lossless-first, unified saliency score,
structured CCR markers) are deferred to Stage 3c.2 — see design doc at
`~/Desktop/SmartCrusher-Architecture-Improvements.md`.
The previous version of this step did `python -c "import headroom._core"`
to find the wheel's installed `.so` path. That failed in CI:
ModuleNotFoundError: No module named 'headroom._core'
— exactly the chicken-and-egg this step exists to fix. The editable
install (`pip install -e .`) puts the in-tree `headroom/` source dir
ahead of site-packages on `sys.path`. So `import headroom` finds the
in-tree dir (which doesn't yet have the `.so`), then
`import headroom._core` fails to find the submodule. The symlink we're
about to create is what makes the import work — but we can't import
to discover the symlink target before creating it.
Locate the `.so` via filesystem instead: read site-packages from
`site.getsitepackages()[0]`, glob for `_core.cpython-*.so` under
`<site-packages>/headroom/`, and symlink that into the in-tree dir.
The smoke test (`from headroom._core import DiffCompressor`) runs
*after* the symlink and confirms end-to-end resolution.
Also added `set -euo pipefail` and a sanity check with `ls -la` of the
site-packages dir if the glob comes up empty, so future failures
diagnose themselves.
`maturin develop` requires a virtualenv (it errors with "Couldn't find a
virtualenv or conda environment"). CI's setup-python provides a bare
system Python without a venv, so the dev script's build path doesn't
work there.
This switches CI to build a release wheel via `maturin build` and
install it with `pip install --force-reinstall --no-deps`. Then symlink
the installed `.so` into the in-tree `headroom/` package so the
editable install resolves `import headroom._core` past the source-dir
shadowing of site-packages.
`scripts/build_rust_extension.sh` is unchanged — it stays optimized for
local dev (where there IS a venv).
The python `DiffCompressor` was retired in this PR's stage-3b commit;
the public class now delegates to `headroom._core` (built from
`crates/headroom-py`). Without the wheel installed in CI, every test
that constructs a `DiffCompressor` fails with `ModuleNotFoundError:
No module named 'headroom._core'`.
This adds a build step to the main `test` job in `ci.yml` that:
1. Installs the stable Rust toolchain (`dtolnay/rust-toolchain@stable`).
2. Caches the cargo registry and build output (`Swatinem/rust-cache@v2`).
3. Installs maturin.
4. Runs `scripts/build_rust_extension.sh`, which calls `maturin develop`
and symlinks the built `.so` into the in-tree `headroom/` package
so the editable install resolves `import headroom._core`.
Only the main `test` job needs this — `test-extras` and `test-agno`
run narrow subsets that don't construct `DiffCompressor`. The existing
`rust.yml` workflow continues to handle wheel builds for distribution
and `cargo test` for the Rust workspace.
