mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-10 14:27:00 -04:00
146 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Rob Francis
|
32ce99e4b4
|
fix(build): enable Intel macOS pip installs via ort-load-dynamic (#1538)
## Description
Fix `pip install headroom-ai` on Intel Mac (`x86_64-apple-darwin`).
Source installs failed because `ort-sys 2.0.0-rc.12` (transitive via
`fastembed`) does not ship prebuilt ONNX Runtime binaries for that
target, causing maturin/cargo to exit during the wheel build.
This PR mirrors the existing Windows fix: build the Rust core with
`ort-load-dynamic`, pin `ORT_DYLIB_PATH` to the pip `onnxruntime` native
library at import time, and publish Intel macOS wheels from CI.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `crates/headroom-core/Cargo.toml`: use `ort-load-dynamic` for
`x86_64-apple-darwin` instead of `ort-download-binaries-rustls-tls`.
- `headroom/_ort.py`: extend the ORT dylib pin hook to Intel macOS
(`darwin` + `x86_64`), resolving `libonnxruntime*.dylib` from the pip
`onnxruntime` package.
- `.github/workflows/release.yml` and `.github/workflows/rust.yml`: add
`macos-15-intel` / `x86_64-apple-darwin` wheel matrix entries.
- `tests/test_release_workflows.py` and
`tests/test_transforms/test_ort_dylib.py`: update/add coverage for the
new target and dylib pin behavior.
- `README.md`: note that prebuilt wheels are published for Intel macOS.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_transforms/test_ort_dylib.py \
tests/test_release_workflows.py::test_fastembed_uses_dynamic_ort_on_windows \
tests/test_release_workflows.py::test_build_wheels_matrix_includes_intel_macos_with_dynamic_ort -q
.......................... [100%]
10 passed in 0.18s
$ maturin build --release -o /tmp/headroom-dist
📦 Built wheel for abi3 Python ≥ 3.10 to /tmp/headroom-dist/headroom_ai-0.27.0-cp310-abi3-macosx_10_12_x86_64.whl
$ python3.11 -m venv /tmp/hr-venv && /tmp/hr-venv/bin/pip install .
Successfully built headroom-ai
Successfully installed headroom-ai-0.27.0
$ cd /tmp && /tmp/hr-venv/bin/python -c "import headroom; import headroom._core; print('ok')"
version 0.27.0
_core ok
```
## Real Behavior Proof
- Environment: macOS `x86_64-apple-darwin`, Python 3.11.5, Rust 1.95.0
- Exact command / steps: Reproduced the reported failure with `pip
install headroom-ai` (sdist build dies in `ort-sys` for
`x86_64-apple-darwin`); after this patch ran `maturin build --release`,
then `pip install .` in a clean venv, then `python -c "import
headroom._core"`.
- Observed result: Before fix, cargo/maturin exit 101 on missing ORT
prebuilts; after fix, wheel build succeeds and `headroom._core` imports
cleanly (`version 0.27.0`, `_core ok`).
- Not tested: `macos-15-intel` GitHub Actions wheel matrix row (will be
validated by CI after merge).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- ML features (magika detection, fastembed embeddings) still require
`onnxruntime` at runtime on Intel Mac. Users should install
`headroom-ai[proxy]` or `pip install onnxruntime`; `_ort.py` auto-pins
`ORT_DYLIB_PATH` when that package is present.
- Apple Silicon (`aarch64-apple-darwin`) behavior is unchanged: it
continues to bundle ORT via `ort-download-binaries-rustls-tls`.
- Lint/mypy not re-run locally in this pass; targeted pytest +
maturin/pip install proof covers the changed surface.
---------
Co-authored-by: Bor <you@example.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
Parideboy
|
728b33088b
|
fix(relevance): gate ONNX embedding backend behind AVX2 to avoid SIGILL (#1723) (#1765)
## Description Fixes the `SIGILL` / Illegal instruction crash in `headroom.compress` on CPUs without AVX2 (Docker / QEMU / older cloud VMs). The precompiled ONNX Runtime binary shipped by `ort-sys` (via fastembed's `ort-download-binaries*` feature) contains AVX2-family instructions on x86; running it on a non-AVX2 CPU traps with SIGILL — an uncatchable native fault that kills the whole host process. Magika detection was already guarded (#1162, landed after `v0.28.0`); the embedding relevance scorer shared the same `ort-sys` binary with no guard. This PR closes that remaining entry point and documents the requirement. Closes #1723 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add shared `onnx_cpu::onnx_runtime_supported_by_cpu()` helper (AVX2 check on x86/x86_64, `true` on other arches) as the single source of truth. - Route `magika_detector` through the shared helper (no behavior change). - Gate `EmbeddingScorer::try_new*` on the helper: unsupported CPU returns `Err` before touching ONNX, so callers fall back to BM25/stub instead of crashing. - Document the x86 AVX2 requirement + auto-fallback in the README. - Add offline tests (no network / no `RUN_FASTEMBED_TESTS`). ## Testing - [x] Unit tests pass (Rust: `cargo test -p headroom-core`) - [x] Linting passes (`cargo clippy -p headroom-core --all-targets`, `cargo fmt --check`) - [ ] Type checking passes (`mypy headroom`) — N/A, Rust-only change - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-core --lib relevance::embedding cargo test: 13 passed, 834 filtered out (1 suite, 0.00s) $ cargo test -p headroom-core --lib magika cargo test: 16 passed, 831 filtered out (1 suite, 0.16s) $ cargo clippy -p headroom-core --all-targets (no warnings, no errors) $ cargo fmt --check -p headroom-core (clean) $ cargo build --workspace cargo build (225 crates compiled) Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 57s ``` ## Real Behavior Proof - Environment: `headroom-core` workspace, Rust stable, x86_64 (AVX2-capable dev host). - Exact command / steps: added `onnx_guard_matches_cpu_features` and `try_new_errors_on_unsupported_cpu_instead_of_sigill` tests; ran the suites above. On a no-AVX2 host the guard makes `EmbeddingScorer::try_new()` return `Err(... "AVX2" ...)` instead of executing the AVX2 ONNX binary; callers fall back to BM25 relevance rather than crashing. - Observed result: guard returns `false` only when the CPU lacks AVX2; embedding + magika ONNX paths both short-circuit to non-ONNX fallbacks; no SIGILL. All suites green. - Not tested: end-to-end `pip install` run on a physically AVX2-less machine (dev host has AVX2); guard behavior is unit-tested via the shared `onnx_cpu` helper and mirrors the already-shipped magika guard (#1162). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable — N/A (release-please generates the changelog) ## Additional Notes Rust-only change, so the Python `pytest`/`ruff`/`mypy` items are N/A; equivalent Rust `cargo test`/`clippy`/`fmt` were run and pasted above. The fix is defense-in-depth parity with the existing magika AVX2 guard (#1162), applied to the second ONNX entry point (embedding relevance). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Abhay Singh
|
8cddf9b58e
|
fix(proxy/auth): match real Anthropic OAuth token prefix (sk-ant-oat) (#1672)
## Description
`classify_auth_mode` (in `headroom/proxy/auth_mode.py`) checks for
Anthropic
OAuth tokens with:
```python
if token.startswith("sk-ant-oat-"):
return AuthMode.OAUTH
if token.startswith("sk-ant-api") or token.startswith("sk-"):
return AuthMode.PAYG
```
But real Anthropic OAuth access tokens are **`sk-ant-oat01-...`** — a
version
number right after `oat`, **no dash**. So the `sk-ant-oat-` check never
matches a
real token; it falls through to the broad `sk-` rule and gets classified
**`PAYG`**.
That's exactly the misclassification the module is built to prevent: a
subscription/OAuth-bound request tagged `PAYG` gets the
aggressive-compression
policy — lossy compression, auto `cache_control`, `prompt_cache_key`
injection —
instead of the passthrough-prefer path OAuth is meant to get.
The existing tests didn't catch it because they use a synthetic
`sk-ant-oat-01-`
fixture (dashed) that happens to match the buggy prefix. Corroboration
that the
real shape is dash-less:
- `.gitguardian.yaml` fixture: `sk-ant-oat01-oauth-fixture`
- `tests/test_oauth_bearer_routing.py`: `sk-ant-oat01-xxx`
- the sibling helper `headroom/proxy/helpers.py` matches on `sk-ant-`
(no `oat-`)
## Fix
Match the dash-less `sk-ant-oat` prefix. It still matches the legacy
dashed
shape, and ordering relative to `sk-ant-api` / `sk-` is unchanged (OAuth
is
still checked first).
```python
if token.startswith("sk-ant-oat"):
return AuthMode.OAUTH
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/auth_mode.py`: match OAuth tokens on the dash-less
`sk-ant-oat` prefix.
- `tests/test_auth_mode.py`: add a regression test using the real
`sk-ant-oat01-...` format (the existing test keeps the legacy dashed
fixture, which still classifies correctly).
- `CHANGELOG.md`: Bug Fixes entry under Unreleased.
## Testing
- [x] New regression test added (`tests/test_auth_mode.py`)
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`)
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uv run ruff check headroom/proxy/auth_mode.py tests/test_auth_mode.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, headroom from this branch.
Importing `headroom` loads the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the classification
logic with a dependency-free script and left the full pytest to CI.
- Exact command / steps: replicated the Bearer-token branch of
`classify_auth_mode` in a standalone script (only stdlib, no `headroom`
import) and ran the real and legacy token shapes plus PAYG keys through
it.
- Observed result: the real `sk-ant-oat01-...` now classifies OAUTH (was
PAYG before the change); the legacy dashed fixture still classifies
OAUTH; `sk-ant-api*` / `sk-*` keys still classify PAYG:
```text
OK: sk-ant-oat01-... -> OAUTH (was PAYG before fix)
OK: sk-ant-oat-01-... -> OAUTH (legacy fixture still matches)
OK: sk-ant-api* / sk-* -> PAYG (unchanged)
AUTH LOGIC VERIFIED
```
- Not tested: a live proxied Anthropic OAuth request end-to-end (needs a
real subscription token); the classification is pure and covered by the
regression test. Full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- `crates/headroom-core/src/auth_mode.rs` carries the identical dashed
prefix (its Rust test matrix uses the same synthetic dashed fixture). I
scoped this PR to the Python runtime classifier since that's the
request-time path; happy to mirror the one-line fix in Rust in the same
PR or a follow-up — I just couldn't `cargo build` locally to verify, so
I left it out rather than push an unverified Rust edit.
- @JerrettDavis tagging you since you've been triaging these — small,
contained fix with a regression test if you have a moment.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Manmit Singh
|
e386c097d6
|
fix(detection): contain unidiff panic on orphaned +++ target line (#1548)
## Description `headroom._core.detect_content_type()` panics with `pyo3_runtime.PanicException: called Option::unwrap() on a None value` on any text containing a `+++ ` target line with no preceding `--- ` source line — e.g. `set -x` xtrace output or a partial `git diff` quoted out of context. The panic originates in the bundled `unidiff` 0.4.0 parser (`lib.rs:665`): on a target-file header it does `source_file.clone().unwrap()`, but `source_file` is still `None` when no source header was seen. The crate's only guard there checks `current_file`, not `source_file`, so it falls through and unwraps `None` instead of returning `Err`. Because detection runs inside a `ThreadPoolExecutor` worker on the Python side, the native panic surfaces as an uncaught `PanicException`, bypasses the compression error handling, and returns **HTTP 500** for the whole request. The failure is deterministic on payload content, so client retries fail until the offending text leaves the context window. `is_diff()` in `unidiff_detector.rs` is the single entry point that drives `PatchSet::parse`, so the fix is contained there: wrap the parse in `catch_unwind` and treat an unparseable fragment as "not a diff". This matches the workspace's deliberate no-`panic = "abort"` policy (Cargo.toml) of surviving bad input rather than taking the long-lived proxy down. Closes #1547 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `crates/headroom-core/src/transforms/unidiff_detector.rs`: contain any `unidiff` parser panic inside `is_diff()` via `catch_unwind`, returning `false` (not a diff) on panic. Added regression test `orphaned_target_line_does_not_panic`. - `CHANGELOG.md`: note under Unreleased → Fixed. ## Testing - [x] Unit tests pass (`cargo test -p headroom-core`) - [x] Linting passes (`cargo fmt --check`, `cargo clippy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output Before the fix (regression test reproduces the exact panic): ```text running 1 test test transforms::unidiff_detector::tests::orphaned_target_line_does_not_panic ... FAILED ---- transforms::unidiff_detector::tests::orphaned_target_line_does_not_panic stdout ---- thread '...' panicked at unidiff-0.4.0/src/lib.rs:665:54: called `Option::unwrap()` on a `None` value test result: FAILED. 0 passed; 1 failed; ... ``` After the fix: ```text running 15 tests test transforms::unidiff_detector::tests::orphaned_target_line_does_not_panic ... ok test transforms::unidiff_detector::tests::standard_git_diff_detected ... ok ... test result: ok. 15 passed; 0 failed; 0 ignored # whole transforms suite test result: ok. 700 passed; 0 failed; 0 ignored ``` ## Real Behavior Proof - Environment: macOS (arm64), Rust stable, `cargo test -p headroom-core`. - Exact command / steps: `cargo test -p headroom-core --lib unidiff_detector` then `cargo test -p headroom-core`. (1) Added a test calling `is_diff("+++ x")` / `detect_diff("+++ x")` and ran it → reproduced the panic at `unidiff-0.4.0/src/lib.rs:665:54` (output above), confirming the same crash path as the report. (2) Applied the `catch_unwind` containment in `is_diff()`. (3) Re-ran the test and the full transforms suite → all green (output above). - Observed result: the orphaned-`+++ ` input is now classified as "not a diff" (plain text) and returns normally instead of panicking. Real diffs (`standard_git_diff_detected`, `naked_hunk_without_git_header_detected`, multi-file, added/removed-only) still detect correctly, so the containment does not weaken detection. - Not tested: I exercised the Rust layer directly (the sole `unidiff` caller, which the `headroom._core.detect_content_type` binding routes through) rather than rebuilding the Python wheel; I did not run the live proxy against a real provider. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md |
||
|
DM
|
64783d8824
|
fix: skip Magika backend on x86 CPUs without AVX2 (#1162)
## Description Adds a narrow runtime AVX2 guard before initializing the Magika/ONNX Runtime detector on x86/x86_64. On x86/x86_64 CPUs without AVX2, Headroom falls back to existing non-Magika detection tiers instead of crashing during ONNX Runtime initialization. AVX2-capable systems retain existing behavior. Refs #1005 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Adds a Magika/ONNX Runtime CPU support guard before `Session::new()`. - Returns a normal Magika init error on x86/x86_64 hosts without AVX2, allowing the existing detection chain to fall through to non-Magika tiers. - Keeps AVX2-capable x86/x86_64 behavior unchanged. - Does not apply the x86-specific AVX2 gate on non-x86 targets. - Adds CPU-aware Rust tests and a short troubleshooting note. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-core --lib --locked 833 passed; 0 failed; 1 ignored $ cargo test --workspace --locked passed $ cargo clippy -p headroom-core --locked -- -D warnings clean ``` ## Real Behavior Proof - Environment: x86_64 Linux host with AVX but no AVX2 (Intel Xeon E5-2697 v2 on Proxmox), local build from this branch. - Exact command / steps: `python -X faulthandler -c 'from headroom._core import detect_content_type; print(detect_content_type("hello world"))'` - Observed result: before — process exited with `Fatal Python error: Illegal instruction`; after — command completed successfully returning `DetectionResult(content_type="text", ...)`, and full `cargo test -p headroom-core --lib --locked` passed with 833/0/1. - Not tested: generic no-AVX CPUs, alternate ONNX Runtime builds, non-x86 platforms. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes This partially addresses #1005 by handling one concrete native crash class: the Magika detector initializes ONNX Runtime through ort/ort-sys, whose precompiled runtime can contain AVX2-family instructions. On AVX-only x86_64 hosts, that initialization can SIGILL before Headroom can fall back. Scope: - This does not introduce generic no-AVX wheels. - This does not redesign Rust-core packaging. - This does not disable the Rust core globally. - This only prevents the Magika/ONNX detector tier from loading on x86/x86_64 CPUs where AVX2 is unavailable. - Non-Magika detection tiers continue to run. - On non-x86 targets, this x86-specific AVX2 gate is not applied. Changelog omitted: small native detector fallback fix with no public API change. Co-authored-by: AI Agent <ai-agent@homelab.internal> |
||
|
Tejas Chopra
|
5771a8020e
|
fix(deps): remediate dependency CVEs and publish SBOM (#1509)
## Description
Supply-chain hardening: takes the **shipped** dependency surface from
**26 known CVEs to 0**. `pip install headroom-ai[all]` now resolves with
no known vulnerabilities (verified with Anchore syft + grype). Also
publishes a checked-in SBOM package (`sbom/`) so any user — especially
pilots running their own security review — can verify what's inside and
that we track it.
This addresses the Dependabot alerts on `main` (9 high / 4 moderate / 7
low at time of writing).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
**Rust**
- `pyo3` 0.24 → 0.29 (GHSA-36hh-v3qg-5jq4 High, GHSA-chgr-c6px-7xpp
Med). Migrated `Python::allow_threads` → `Python::detach` (10 sites) and
added `from_py_object` to the `Clone`-deriving `#[pyclass]` types (both
required by the 0.25+ API).
- `pyo3-log` 0.12 → 0.13; `lru` 0.12 → 0.18 (GHSA-rhfx-m35p-ff5j).
**Python**
- `torch` → 2.12.1, `mem0ai` → 2.x.
- Floor-pinned transitive CVE deps via `[tool.uv]
constraint-dependencies`: `pygments>=2.20.0`,
`pydantic-settings>=2.14.2`, `gitpython>=3.1.50`, `langsmith>=0.9.0`.
- **Removed `benchmark` from the `[all]` aggregate** so the default
install is CVE-free. `lm-eval` is invoked as an external subprocess
(`python -m lm_eval`) and never imported, so it is not a true runtime
dep — it remains available via the opt-in `[benchmark]` extra. See
[Accepted Risks](#additional-notes).
**npm (build/test tooling — never shipped in the
wheel/container/published SDK)**
- `esbuild` override `>=0.28.1` in `sdk/typescript` + `plugins/openclaw`
(GHSA-g7r4-m6w7-qqqr).
- `docs/`: `@anthropic-ai/sdk` → `^0.106.0` (GHSA-p7fg-763f-g4gf),
`postcss` override to force Next.js's bundled copy ≥8.5.10
(GHSA-qx2v-qp2m-jg93); regenerated a stale `bun.lock` that carried a
**Critical** vitest/vite.
**CI**
- Pinned `pypa/gh-action-pypi-publish` `@release/v1` → `@v1.13.0`
(GHSA-vxmw-7h4f-hqxh) in `release.yml` + `publish.yml`.
**SBOM**
- New `sbom/` directory: CycloneDX 1.7 + SPDX 2.3 SBOMs, grype scan
evidence, 330-package license inventory, and a regeneration guide.
## Testing
- [ ] Unit tests pass (`pytest`) — N/A, no Python source changed
(deps/config only)
- [x] Linting passes — `cargo fmt --check` + `cargo clippy` clean on the
changed crate; 0 `.py` files changed so `ruff`/`mypy` scope is
unaffected
- [x] Type checking passes — `cargo check --workspace` (0 errors)
- [ ] New tests added — N/A (dependency bumps; covered by existing
suites)
- [x] Manual testing performed — see Real Behavior Proof
### Test Output
```text
# headroom-ai[all] product surface — the number that matters
$ grype sbom:sbom/headroom-sbom-all-extra.cdx.json
No vulnerabilities found
# full repo scan (universal lock incl. opt-in [benchmark] + dev)
$ grype sbom:sbom/headroom-sbom.cdx.json
NAME INSTALLED TYPE VULNERABILITY SEVERITY
sqlitedict 2.1.0 python GHSA-g4r7-86gm-pgqc High # [benchmark]-only, unpatchable, accepted
nltk 3.9.4 python GHSA-p4gq-832x-fm9v High # [benchmark]-only, unpatchable, accepted
# pyo3 0.29 migration — extension builds + imports + runs
$ cargo check --workspace
Finished `dev` profile [unoptimized + debuginfo] target(s)
$ maturin develop && python -c "from headroom._core import DiffCompressor, SmartCrusher; ..."
extension OK — detach + from_py_object paths exercised
# lru 0.18 — eviction path
$ cargo test -p headroom-proxy --lib drift
14 passed, 213 filtered out
# per-ecosystem npm audits
$ (cd sdk/typescript && npm audit) -> found 0 vulnerabilities
$ (cd plugins/openclaw && npm audit) -> found 0 vulnerabilities
$ (cd docs && npm audit && bun audit) -> found 0 vulnerabilities / No vulnerabilities found
```
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0, arm64), Python 3.12 `.venv`, Rust
1.95 toolchain, syft 1.46.0, grype 0.115.0, bun 1.3.14, maturin 1.13.3.
- Exact command / steps: (1) `uv export --extra all --no-dev
--no-emit-project | syft → grype` for the product surface; (2) `cargo
check --workspace` + `maturin develop` + extension import/compress smoke
test; (3) `cargo test -p headroom-proxy --lib drift`; (4) `cargo fmt
--check` + `cargo clippy -p headroom-py`; (5) `npm audit` in
sdk/openclaw/docs + `bun audit` in docs.
- Observed result: `headroom-ai[all]` resolution scans clean — "No
vulnerabilities found" (179 pkgs); full/prod SBOM shows only the 2
documented accepted CVEs; pyo3 0.29 extension imports and runs (detach +
from_py_object paths exercised); drift tests 14/14 pass; cargo fmt +
clippy clean; all npm/bun audits report 0.
- Not tested: full `pytest` suite (no Python source changed);
release-profile wheel build (used dev-profile `maturin develop` for the
import proof — the extension is semantically identical).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
(`sbom/README.md`)
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective — N/A
(dependency bumps; existing suites + scans cover it)
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md — N/A (Release Please
auto-generates from the conventional commit)
## Additional Notes
**Accepted risks (the 2 residual CVEs).** Both originate solely from the
EleutherAI `lm-evaluation-harness` under the **opt-in `[benchmark]`
extra**, which Headroom invokes as a subprocess (never imports):
- `sqlitedict` CVE-2024-35515 (High) — pickle deserialization; package
abandoned (last release 2021), **no upstream fix exists**.
- `nltk` CVE-2026-54293 (High) — path traversal in `nltk.data.load()`;
affects ≤3.9.4 (current latest), **no patched release**.
Neither is in `[all]`, the published wheel, or the container. They are
documented in `sbom/README.md` and will be picked up automatically once
upstream ships fixes.
**Release/CHANGELOG:** N/A items above are because this is a
dependency/security PR with no Python source changes; CHANGELOG is
Release-Please-managed via the conventional commit message.
|
||
|
Ali
|
0e6d922f88
|
feat(pricing): add DeepSeek V4 model pricing (deepseek-v4-flash, deepseek-v4-pro) (#1168)
## Description Adds pricing support for DeepSeek V4 models (`deepseek-v4-flash` and `deepseek-v4-pro`) when routing Headroom through `--anthropic-api-url https://api.deepseek.com/anthropic`. The vendored LiteLLM pricing database predates DeepSeek V4, so cost estimation silently returned `None` for these models. ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - **`headroom/pricing/deepseek_prices.py`** — New pricing data module with `ModelPricing` dataclass entries for both V4 models, following the pattern of `anthropic_prices.py` - **`headroom/pricing/__init__.py`** — Exports `DEEPSEEK_PRICES`, `get_deepseek_registry()`, `DEEPSEEK_LAST_UPDATED` - **`headroom/pricing/litellm_pricing.py`** — Runtime injection of DeepSeek V4 pricing into `litellm.model_cost`, plus `deepseek-` prefix added to `resolve_litellm_model()` provider prefix list - **`headroom/providers/anthropic.py`** — DeepSeek fallback in `_get_pricing()` when model starts with `deepseek-` and LiteLLM is unavailable - **`crates/headroom-proxy/data/model_prices_and_context_window.json`** — Vendored JSON entries (bare + provider-prefixed) for Rust-side context window lookups - **`tests/test_providers/test_deepseek.py`** — 20 tests across 3 test classes (pricing data, LiteLLM injection, Anthropic fallback) - **`tests/test_pricing.py`** — Added DeepSeek export validation alongside existing OpenAI/Anthropic assertions ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ``` ========================= 137 passed, 8 warnings in 8.47s ========================= ``` ## Real Behavior Proof - Environment: Windows 10, Python 3.12, litellm 1.60+ - Exact command / steps: `python -c "from headroom.proxy.cost import CostTracker; t = CostTracker(); print(t.estimate_cost('deepseek-v4-flash', input_tokens=1000000, output_tokens=1000000))"` - Observed result: `$0.4200` (0.14 input + 0.28 output per 1M tokens) - Not tested: Live DeepSeek API routing via `--anthropic-api-url` (requires API key and Docker deployment) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The 90% cache discount heuristic in `AnthropicProvider.estimate_cost()` (line 680) is a pre-existing pattern. DeepSeek V4 has much deeper cache discounts (98-99%), but the LiteLLM path currently falls through to the manual fallback which uses correct cached prices. A future improvement could prefer `cache_read_input_token_cost` from model info over the hardcoded `* 0.1` heuristic. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Nadia Ujovich
|
7c93c50c2c
|
Marker-free lossless_only mode + gate opaque-blob CCR markers behind enable_ccr_marker (#1129)
## Description `enable_ccr_marker` only gated the **row-drop sentinel** path. The **opaque-blob** path still emitted `<<ccr:HASH,kind,size>>` markers unconditionally whenever a string cell exceeded `opaque_min_bytes` (256), so **no configuration could produce a fully marker-free prompt**. Any `<<ccr:>>` marker is a promise that the full payload lives in the CCR store and must be fetched back via a retrieval tool call — there was no way to get compression without that round-trip dependency. **Rebased on #1130 (merged).** That PR fixed the opaque-blob gate at the classifier (`ClassifyConfig.emit_opaque_markers`, driven by `enable_ccr_marker`) and closed #1091. This branch originally carried its own equivalent gating commit; that commit is now **redundant and has been dropped** — `classifier.rs` here is identical to upstream. What remains is the **net-new** work that is **not** in #1130: - **Strict `lossless_only` mode** — keeps lossless tabular compaction, but routes every path that would need a CCR marker (row-drop sentinel **and** opaque-blob offload) to leave content uncompacted instead, so output is always marker-free **and** byte-recoverable. - **Python parity** — `lossless_only` exposed across both config dataclasses, a `SmartCrusher` kwarg, and a per-call `crush(..., lossless_only=)` override. - **`HEADROOM_LOSSLESS_ONLY` env var** — wires the mode through to the proxy runtime so real agents can use it. The #1130 opaque gate is consumed here through a single centralized helper (`opaque_markers_enabled() = enable_ccr_marker && !lossless_only`) used by **all four** `ClassifyConfig` construction sites. ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - **`feat(smart_crusher)`** — Add `lossless_only` (default `false`): keeps lossless tabular compaction but routes every marker-requiring path (row-drop sentinel + opaque-blob offload) to leave content uncompacted instead. Exposed across the Rust core, PyO3 bridge, both Python config dataclasses, a `SmartCrusher` kwarg, a per-call `crush(..., lossless_only=)` override, and `smart_crush_tool_output`. Includes a `debug_assert` documenting the load-bearing invariant (a `lossless_only` crusher must never reach the CCR store write). - **`refactor(smart_crusher)`** — Extract `SmartCrusherConfig::opaque_markers_enabled()` as the single source of truth for `enable_ccr_marker && !lossless_only`, consumed by **all four** `ClassifyConfig` sites: the compaction-stage builder, `with_compaction_format`, the top-level `process_string` path (Rust core), and the PyO3 `compact_document_json` document-compactor path. No site derives the gate inline anymore, so they cannot drift. - **`feat(proxy)`** — Expose the mode via `HEADROOM_LOSSLESS_ONLY`: `ContentRouterConfig.smart_crusher_lossless_only` → `_get_smart_crusher`; the proxy reads the env var and sets it on the live router config. Previously reachable only via the Python API, never through the proxy. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check` on changed files) - [ ] Type checking passes (`mypy headroom`) — not run (see Additional Notes) - [x] New tests added for new functionality - [x] Manual testing performed (proxy env-var seam, end-to-end — see Real Behavior Proof) ### Test Output ```text ### RUST (cargo test -p headroom-core --lib smart_crusher) test result: ok. 323 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out ### PYTEST (test_smart_crusher_bugs.py + test_agent_savings.py + test_smart_crusher_toin_attachment.py) 45 passed ### RUFF (changed files) All checks passed! ### FMT + CLIPPY (cargo fmt --check && cargo clippy --workspace --lib) clean — no warnings ``` New/affected tests: `enable_ccr_marker_false_suppresses_opaque_markers`, `lossless_only_leaves_array_uncompacted_instead_of_dropping`, `lossless_only_inlines_opaque_blobs_when_table_ships`, `lossless_only_never_writes_to_ccr_store` (Rust); `TestLosslessOnlyMode`, `test_router_lossless_only_flag_reaches_crusher`, `test_router_lossless_only_defaults_off` (Python). Coexists green with #1130's `long_string_stays_scalar_when_opaque_markers_disabled` (Rust) and `test_smart_crusher_toin_attachment.py` (Python). The Python `TestOpaqueMarkerGate` from the dropped gating commit was removed as redundant with #1130's coverage. ## Real Behavior Proof ### Proxy env-var seam — end-to-end (this revision) The one path with no automated coverage was `server.py` reading `HEADROOM_LOSSLESS_ONLY` from the environment and threading it into the live router config. Verified end-to-end by instantiating the **real** `HeadroomProxy`, pulling the `ContentRouter` out of its pipeline, and crushing a 50-row array with >256B opaque cells through the real Rust crusher: | | `HEADROOM_LOSSLESS_ONLY=1` | env unset (default) | |---|---|---| | `crusher._lossless_only` | **True** | **False** | | output contains `<<ccr:` | **No** (marker-free) | Yes (normal lossy) | | byte-recoverable (round-trips to original JSON) | **Yes** | No (rows offloaded) | This confirms the full chain `os.environ["HEADROOM_LOSSLESS_ONLY"]` → `server.py` → `ContentRouterConfig.smart_crusher_lossless_only` → `content_router.py` → `crusher_config.lossless_only` → Rust crusher. The default column proves strict mode genuinely changes behavior (not a no-op) and that the default path is unchanged. ### Prior live-traffic run - Environment: Headroom proxy in front of a real agent (Hermes) routed to NVIDIA NIM (OpenAI-compatible upstream). Isolated config dir; `OPENAI_TARGET_API_URL=https://integrate.api.nvidia.com/v1`, `HEADROOM_LOSSLESS_ONLY=1`. Confirmed with `ss` that all LLM traffic flowed agent → proxy → upstream with no direct bypass. - Exact command / steps: Start the proxy with `python -m headroom.proxy.server --host 127.0.0.1 --port 8787`; point the agent's LLM base_url at `http://127.0.0.1:8787/v1`; run a real chat plus a `search_files`-style task; read `/stats` and `/v1/retrieve/stats`; then toggle `HEADROOM_LOSSLESS_ONLY` and repeat for the comparison. - Observed result: With 150K+ tokens of real traffic processed, `lossless_only` kept the CCR store empty (`entry_count: 0`) and emitted zero markers. A synthetic before/after with opaque (>256B) cells produced 12 `<<ccr:>>` markers in default mode and 0 under `lossless_only`, with output round-tripping to the original JSON structure. - Not tested: A live `lossless_only`-vs-markers contrast on real agent traffic. The SmartCrusher offload path never engaged on this agent's tool outputs (`total_compressions: 0`; CCR store stayed at `entry_count: 0` even after a broad codebase search), and compression stayed marginal (~0.2–0.4%) in both modes. The agent's tool results don't match the crushable-array profile the offload paths target, so the marker path is never exercised in that integration. Why SmartCrusher barely engages with this agent's outputs is a separate integration question (output format / routing / size thresholds), out of scope for this change. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (config docstrings updated in-tree; no separate docs) - [x] My changes generate no new warnings - [x] I have added tests that prove my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable — N/A ## Additional Notes - Rebased on top of merged #1130; the now-redundant opaque-blob gating commit was dropped, so this PR is purely the `lossless_only` feature + proxy wiring on top of #1130's gate. - `mypy headroom` was not run in this environment; happy to add the result if CI requires it. - Default behavior is fully preserved: `enable_ccr_marker` defaults to `true`, `lossless_only` defaults to `false`, and `HEADROOM_LOSSLESS_ONLY` unset is a no-op. |
||
|
Zhenjia ZHOU
|
6c68ff4e9f
|
perf(compression): take large cold-start contexts off the synchronous kompress path (#1171) (#1298)
## Description On a cold-start large context, kompress (ModernBERT ONNX) runs **synchronously on the request thread** — ~200–300s for ~1M tokens. It blows the 30s compression budget, leaks a non-preemptible worker, and cascades (executor saturation → queue timeouts on healthy requests); on timeout the request is forwarded **uncompressed** after eating 30s. This adds four layered, **default-off, fail-open** mitigations so the request path is never blocked on ML compression. Closes #1171 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Phase 0 — size gate** (`HEADROOM_KOMPRESS_MAX_TOKENS`, default 50000): route oversized text away from ModernBERT (→ LogCompressor / TextCrusher / passthrough) at the single `_try_ml_compressor` boundary. - **Phase 1 — cooperative deadline** (`HEADROOM_COMPRESSION_DEADLINE_MS`, default 20000): any kompress run self-terminates at the next chunk boundary past the budget, keeping the unprocessed tail verbatim. - **Phase 2 — TextCrusher** (`HEADROOM_TEXT_CRUSHER`): a new **native Rust** extractive prose compressor in `crates/headroom-core/src/transforms/text_crusher/`, exposed via PyO3 as `headroom._core.TextCrusher` with a thin Python wrapper. It **reuses the shared `crate::relevance::BM25Scorer`** rather than reimplementing BM25, and ships record/replay parity fixtures (mirroring the SmartCrusher Rust-core + Python-shim pattern). - **Phase 3 — off-path compression** (`HEADROOM_BACKGROUND_COMPRESSION`): forward uncompressed immediately and compress in a per-process background drain; a byte-identical cache hit on a later turn means the request never blocks on ML. - Benchmark (`benchmarks/text_crusher_quality_eval.py`), CHANGELOG entry, and docstrings documenting the fail-open limits. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`, new modules) - [x] New tests added for new functionality - [ ] Manual testing performed (Phase 0/1 gate-fire + deadline observed on real traffic in earlier iterations; Phase 3 off-path is unit- + byte-identity-tested, not yet live-validated) ### Test Output ```text $ pytest tests/test_transforms/ tests/test_cache/ \ tests/test_proxy/test_background_compression.py tests/test_proxy/test_phase3_byte_identity.py -q 501 passed, 37 skipped in 40.33s $ cargo test -p headroom-core --lib text_crusher test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 834 filtered out $ ruff check <changed files> All checks passed! $ mypy headroom/proxy/background_compression.py headroom/transforms/text_crusher.py Success: no issues found in 2 source files ``` New coverage: size-gate incl. the strategy-dispatch funnel (KOMPRESS + TEXT); partial-run deadline (chunk-0 compressed + chunk-1 verbatim tail); BackgroundCompressor (dedup / queue-full / fail-open); Phase 3 byte-identity round-trip; TextCrusher unit + parity. ## Real Behavior Proof - Environment: macOS, local dev — `uv` venv, Rust `_core` built via `uv pip install -e .`. - Exact command / steps: the `pytest` / `cargo test` / `ruff` / `mypy` commands shown under Test Output; quality eval `python benchmarks/text_crusher_quality_eval.py /tmp/squad_dev.json`. - Observed result: 501 Python + 3 Rust tests pass; ruff + mypy clean on changed/new modules. Quality eval: TextCrusher keeps ~94% of buried SQuAD answers at 30% size vs ~36% truncate/random; self-contained speed run ~333k words in ~76ms (one O(n) pass) — sub-second where ModernBERT takes minutes (fast-vs-slow contrast, not a same-input run). - Not tested: Phase 3 off-path on live traffic; multi-worker (per-process by design — see Additional Notes). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - **All four features are off by default and fail-open** — with the env flags unset the paths are no-ops for realistic inputs; on any error the request is forwarded (compressed if possible, else verbatim), never dropped. A full background queue / duplicate key surfaces as `deferred:dropped`. - **Known limits (documented in `background_compression.py`):** Phase 3 is per-process, in-memory, and token-mode-only — these are **lost-savings, never lost-correctness**, and consistent with the project's existing per-process compression cache + sticky-session multi-worker model. The startup multi-worker warning now names off-path background compression. - Phase 2 reuses the existing BM25 scorer; reuse did not improve answer-retention over a Python prototype (query-awareness dominates) — its value is the Rust speed + repo-conventional Rust-core/Python-shim shape. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Parideboy
|
3ccdad6c67
|
Pin ORT dylib on Windows; init Python logging (#1010)
## Description On Windows, headroom's Rust core resolves `onnxruntime.dll` at runtime via `ort-load-dynamic`. Without an explicit `ORT_DYLIB_PATH`, the bare DLL search can land on `C:\Windows\System32\onnxruntime.dll`, the Windows ML OS component, and `Session::new()` can deadlock instead of returning an error. Since a hang is not an `Err`, the tiered fallback cannot engage until the proxy-level timeout fires. This PR pins `ORT_DYLIB_PATH` to the pip-installed `onnxruntime` DLL at import time, and wires Rust `tracing` events into Python logging so the proxy log surfaces these failures when they occur. Closes #928 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `headroom/_ort.py` with a Windows-only, idempotent `ensure_ort_dylib_pinned()` resolver that respects an existing `ORT_DYLIB_PATH`. - Call the pin from `headroom/__init__.py` before importing `_core` consumers. - Log the effective ORT dylib path from the content router startup path on Windows. - Enable Rust tracing-to-log compatibility and initialize `pyo3-log` in the `_core` module. - Add timeout diagnostics in the Magika detector with the effective `ORT_DYLIB_PATH`. - Document `ORT_DYLIB_PATH` and `HEADROOM_MAGIKA_INIT_TIMEOUT_SECS`. - Add unit coverage for the resolver behavior. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_transforms/test_ort_dylib.py -q`) - [x] Linting passes (`ruff check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py`) - [x] Formatting passes (`ruff format --check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_transforms/test_ort_dylib.py -q 7 passed in 0.19s $ ruff check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py All checks passed! $ ruff format --check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py 4 files already formatted $ cargo check -p headroom-py cargo: The term 'cargo' is not recognized as a name of a cmdlet, function, script file, or executable program. ``` ## Real Behavior Proof - Environment: Windows 11 24H2, Python 3.13, RTX 4080 - Exact command / steps: `python -c "import headroom; from headroom._core import detect_content_type as d; print(d(open('headroom/compress.py').read()).content_type)"` - Observed result: `source_code` in 301ms, clean exit, `Magika: ENABLED` in proxy log - Not tested: macOS/Linux manual runtime behavior; `_ort.py` is a no-op outside Windows, and CI covers cross-platform build/test behavior. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (N/A: repo uses release-please) ## Additional Notes The branch was rebased onto current `main` and the commit subject was updated to satisfy commitlint. Local Rust verification could not be run on this Windows machine because `cargo` is not installed; GitHub CI should be treated as the Rust build verification for the `pyo3-log` dependency and workspace lockfile changes. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
jichaowang02-lang
|
c7295cad1d
|
fix(ccr): store opaque blobs from lossless:table compaction (#1083) (#1182)
## Description
SmartCrusher's `lossless:table` compaction path emits opaque-blob CCR
markers
(`<<ccr:HASH,KIND,SIZE>>`) but never wrote the original payload to the
CCR
store. As a result `GET /v1/retrieve/{hash}` and the `headroom_retrieve`
tool
return **404** for those hashes. The opaque-*string* path
(`walker::emit_opaque_ccr_marker`) already stores its payload; the table
compactor diverged simply because no store was threaded into it.
Closes #1083
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `compaction/compactor.rs`: add `compact_with_store(items, cfg, store)`
and a
private `compact_inner`; thread `Option<&Arc<dyn CcrStore>>` through
`build_homogeneous_table` → `build_row` → `cell_from_value` and the
recursive
bucket/nested calls. In the `Opaque` branch, `store.put(&hash, payload)`
under
the **same** `hash_opaque` value that becomes the marker hash (mirrors
`walker::emit_opaque_ccr_marker`). Public `compact` is unchanged — it
delegates
with `None`.
- `compaction/mod.rs`: add `CompactionStage::run_with_store`; `run` is
unchanged.
- `crusher.rs`: the lossless branch now calls
`stage.run_with_store(items, self.ccr_store.as_ref())` instead of
`stage.run(items)`.
- Two new unit tests in `compactor.rs` (see below).
The IR (and therefore the rendered marker text) is identical whether or
not a
store is supplied — the store only gains the write that should already
have
happened, so existing output stays byte-for-byte the same.
## Testing
<!-- Check what you actually ran, then paste the real command output
below. -->
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
> Note: this change is in the Rust core (`crates/headroom-core`), so the
> Python-specific checks above are N/A. The Rust equivalents were run:
### Test Output
```text
$ cargo test -p headroom-core --lib compaction
test result: ok. 70 passed; 0 failed; 0 ignored; 0 measured; 766 filtered out; finished in 0.01s
$ cargo fmt -p headroom-core -- --check
# clean (exit 0)
```
New tests:
- `opaque_payload_is_stored_under_marker_hash` — after
`compact_with_store`, the
original blob is retrievable via `store.get(marker_hash)`, and the
stored key
equals `hash_opaque(payload)` (locks the key↔marker contract).
- `store_presence_does_not_change_the_ir` — `compact` and
`compact_with_store`
produce identical IR; only the store write is added.
(The full `cargo test -p headroom-core --lib` run has 18 pre-existing
failures,
all in `transforms::magika_detector` — they require the ONNX
runtime/model and
are unrelated to this change. All 70 compaction + crusher tests pass.)
## Real Behavior Proof
- Environment: Windows, Rust 1.95.0, `cargo test -p headroom-core` (no
live proxy).
- Exact command / steps: build a 2-item array with a long opaque-blob
field →
`compact_with_store(&items, &cfg, Some(&InMemoryCcrStore))` → read the
`OpaqueRef.ccr_hash` from the IR → `store.get(ccr_hash)`.
- Observed result: before the fix the store is empty (retrieval would
404);
after the fix `store.get(ccr_hash) == Some(original_payload)` and the
marker
hash is unchanged.
- Not tested: end-to-end through a running proxy / a real `GET
/v1/retrieve/{hash}`
HTTP round-trip. Verified at the unit level that the store now receives
the
payload under the exact marker hash, which is the write that was
missing.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Docs/CHANGELOG checklist items are N/A — this is an internal
correctness fix
with no user-facing API change.
- Scope is intentionally minimal: public `compact`/`run` signatures are
preserved (delegating with `None`), so all existing callers and the 68
in-crate compaction tests are unaffected. Only the lossless
`crush_array`
branch opts into the store-threading via `run_with_store`.
|
||
|
Zhenjia ZHOU
|
27d6f8e2a7
|
fix(smart-crusher): honor enable_ccr_marker on the opaque-blob path (#1130)
## Description Closes #1091. SmartCrusher's array compaction is lossless-first, but the **opaque-blob** substitution path emitted `<<ccr:HASH,string,KB>>` markers **unconditionally** — it did not honor `enable_ccr_marker` / `inject_retrieval_marker`, which gate only the lossy **row-drop** path. As the issue notes, the consequence was that *no configuration produced guaranteed-lossless, marker-free output*: any array with a single string cell over `opaque_min_bytes` (256B default) still emitted a CCR marker, forcing a retrieval round-trip for consumers that need verbatim output. **Root cause:** the row-drop path is gated (`crusher.rs` — `if dropped_count > 0 && self.config.enable_ccr_marker`), but opaque classification in `compaction/classifier.rs` keyed purely on byte length, with no reference to the flag, and both emit sites (`walker.rs`, `crusher.rs`) then produced a marker. **Fix:** thread the gate into classification. `ClassifyConfig` gains an `emit_opaque_markers` field (default `true`); when `false`, a long string is classified `Scalar` (kept verbatim) instead of `Opaque`, so no marker is emitted and nothing is written to the CCR store anywhere downstream. The flag is set from `enable_ccr_marker` at both `ClassifyConfig` construction sites in `crusher.rs`. > Design note: gating at the classifier (rather than at marker-emit time) is the single complete fix — it covers all three emit paths (walker inline-substitution, the crusher string path, and the compactor `OpaqueRef`→formatter path, which no longer has the original string by the time it formats). One consequence: with markers **off**, an array dominated by unique long-string cells now falls through to a conservative passthrough (`skip:unique_entities_no_signal`) instead of a lossy opaque table — still lossless and marker-free, which is the point of disabling markers. If you'd rather preserve structural table compaction with the blob inlined verbatim, that's a larger change at the emit + compactor layers; happy to take it that direction if preferred. Default behavior (`enable_ccr_marker=true`) is unchanged. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `crates/headroom-core/src/transforms/smart_crusher/compaction/classifier.rs`: add `emit_opaque_markers: bool` (default `true`) to `ClassifyConfig`; in `classify_cell`, keep long strings `Scalar` when it is `false`. New unit test `long_string_stays_scalar_when_opaque_markers_disabled`. - `crates/headroom-core/src/transforms/smart_crusher/crusher.rs`: set `classify.emit_opaque_markers = config.enable_ccr_marker` at both `ClassifyConfig` construction sites (the `CompactConfig` builder and the standalone string path). - `tests/test_smart_crusher_toin_attachment.py`: regression test pinning both directions — markers ON ⇒ opaque marker present (input really triggers the path); markers OFF ⇒ no marker, blob verbatim. ## Testing - [x] Unit tests pass (`pytest`) - [x] Rust tests pass (`cargo test`) - [x] Linting passes (`ruff check .`, `cargo fmt --check`, `cargo clippy -- -D warnings`) - [ ] Type checking (`mypy headroom`) — N/A (no headroom/ Python source changed) - [x] New tests added ### Test Output ```text # Rust $ cargo test -p headroom-core --lib smart_crusher test result: ok. 319 passed; 0 failed (incl. new: ...classifier::tests::long_string_stays_scalar_when_opaque_markers_disabled ... ok) $ cargo fmt --check && cargo clippy --workspace -- -D warnings ok # Python (after `uv pip install -e .` to rebuild the Rust core) $ pytest tests/test_smart_crusher_toin_attachment.py tests/test_transforms/ tests/test_ccr_row_drop_store_bridge.py 289 passed, 35 skipped # Full suite is green except the 5 pre-existing caplog logging-isolation # flakes that are unrelated to this change and fixed separately in #1117. ``` ## Real Behavior Proof - Environment: macOS, Python 3.13.3, Rust core rebuilt via `uv pip install -e .`. - Exact command / steps: crush a 60-row array whose rows carry a distinct >256B `blob` string, with `inject_retrieval_marker` ON then OFF. - Observed result: with `inject_retrieval_marker` OFF (after this fix) the crushed output contains NO `<<ccr:` marker and the original `sentinel5_…` blob survives verbatim; before the fix the same input still emitted `<<ccr:…,string,407B>>` (the bug); with markers ON behavior is unchanged. Concretely: - markers ON → `strategy=lossless:table`, output contains `<<ccr:…,string,407B>>` (blob replaced). - markers OFF (before fix) → `lossless:table` **still emitted `<<ccr:…>>`** (the bug). - markers OFF (after fix) → no `<<ccr:` marker, the original `sentinel5_…` blob present verbatim. - Not tested: behavior under CI's sharded jobs specifically; fix is deterministic and config-gated. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md — N/A 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Tejas Chopra
|
b7be3814f1
|
feat: compression extraction — Rust knob exposure, CCR hardening, traffic audits (#818)
## Description
A data-driven push for better compression savings without accuracy loss,
in four parts: expose and tune the Rust compressor knobs, harden the CCR
retrieval store, add traffic-audit tooling that sizes opportunities from
real transcripts, and introduce **read maturation** — a new,
live-validated mechanism that compresses Read outputs *before* they ever
enter the provider prefix cache.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
### 1. Rust compressor extraction
- Expose `lossless_min_savings_ratio` end-to-end and lower the default
0.30 → 0.15 (lockstep across Rust, PyO3, and both Python config classes)
so the lossless Table/CSV compaction path wins more often.
- Expose the `CompactConfig` heuristics (core-field fraction,
heterogeneity ratio, flatten cap, bucket bounds) through PyO3 + Python.
- `SearchCompressor` grouped-by-file output (`rg --heading` style — path
once per file instead of per match). Library default off; the proxy
enables it in token mode.
- Complete `factor_out_constants`: constant fields now emit once in a
`_constant_fields` sentinel with slim rows (defensive per-item value
match; default off).
- `ContentRouter` accepts a SmartCrusher config override and the
search-grouping knob.
### 2. CCR store hardening
- Session-scale TTL: 300s → 1800s (CCRConfig, CompressionEntry,
CompressionStore, Rust `DEFAULT_TTL` — lockstep).
- **SQLite is the default CCR backend** (`~/.headroom/ccr_store.db`,
WAL): survives proxy restarts and is shared across workers.
`HEADROOM_CCR_BACKEND=memory` opts out.
- Multi-worker safety: `busy_timeout`, and corruption detection narrowed
so transient `SQLITE_BUSY` errors can never trigger database deletion.
- Data-at-rest hygiene: `chmod 600` on db + sidecars, expired rows swept
at open.
- Retrieval-miss messages are actionable (re-read the file / re-run the
command).
### 3. Traffic audit tooling (measure before tuning)
- `headroom audit-reads`: sizes Read opportunities from local Claude
Code transcripts (read share, stale %, line-number overhead, context
residency, cache-death windows).
- `--simulate-maturation`: Mechanism B risk sizing (re-read rates,
never-touched-again share, quiesce coverage, at-risk edits).
- `--codex`: shell-read classifier for Codex transcripts (rtk-wrapper
aware, workdir resolution).
- Findings that shaped this PR (81 sessions): Reads are 67% of tool
bytes; median Read lingers 118 turns (~13x lifetime cost); a prototyped
repeat-Read dedup measured 0.1% and was **removed** rather than shipped
as dead code.
### 4. Read maturation (Mechanism B) — experimental, default OFF
- Activity-based: a fresh large Read is held **out** of the provider
cache (trailing breakpoint relocated before it), stays verbatim while
its file is active, and matures into a CCR-backed marker once the file
is quiet for `quiesce_turns` (default 5; `max_hold_turns` bounds busy
files).
- Only the final compressed form ever enters the cache — **no cached
byte is ever mutated**; matured markers replay byte-identically.
- Wired into the Anthropic handler behind `--read-maturation` /
`HEADROOM_READ_MATURATION=1`; session state rides on the prefix tracker;
advisory (can never fail a request).
- Live-validated against the Anthropic API: held content excluded from
cache_creation; after maturation the prior cached prefix still served —
the no-bust invariant holds end-to-end.
### 5. Rebase / CI fixups (this update)
- Rebased onto latest `main` (was 28 commits behind): picks up `ci: pass
CODECOV_TOKEN to coverage uploads (#968)`, which is what was turning the
4 test shards red — the tests themselves passed (1528) but the post-test
codecov upload exited non-zero on a protected branch.
- Resolved the duplicate `lossless_min_savings_ratio` that two
independent main/branch additions left in `SmartCrusherConfig` and the
Rust-config kwarg (import-time `SyntaxError` + mypy `no-redef`).
- Aligned CCR tests with the new defaults (SQLite backend, 1800s TTL)
across `test_ccr`, `test_adapter_hooks`, `test_compression_store`,
`test_proxy_ccr`, and the lossy row-drop bridge test.
## Testing
<!-- Check what you actually ran, then paste the real command output
below. -->
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_proxy_ccr.py tests/test_ccr.py tests/test_compression_store.py tests/test_adapter_hooks.py tests/test_ccr_row_drop_store_bridge.py -q
170 passed, 4 warnings in 42.49s
$ python -m pytest tests/test_audit_reads.py tests/test_audit_codex.py tests/test_read_maturation.py tests/test_transforms_content_router.py tests/test_smart_crusher_toin_attachment.py -q
83 passed
$ mypy headroom/
Success: no issues found in 365 source files
$ python -m compileall headroom/ -q
COMPILE-OK
# CI (run 27488990477, pre-rebase head): all 4 shards ran to completion —
# "1528 passed, 120 skipped, 4922 deselected"
# The red shards were the codecov upload step, not test failures; fixed by
# the #968 rebase above.
```
## Real Behavior Proof
- Environment: macOS (darwin), Python 3.12 venv; branch
`feat/compression-extraction` rebased onto `origin/main` (head
|
||
|
Yasser Sheikh
|
0dc2e1cb3f
|
feat(bedrock): cross-region + Converse compression; bundle proxy binary in images (#999)
## Description The native Bedrock path (Phase D) compresses + signs Anthropic-on-Bedrock requests, but two real-world cases slipped through, and the native binary that powers it was never shipped. This PR closes those gaps as a focused set of give-backs. Aligns with the Rust migration plan (see below). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Cross-region inference-profile detection** via a new `bedrock::vendor` module (`canonical_vendor()`), following the design proposed in #953: strip a known geo prefix (`eu.`/`us.`/`apac.`/`global.`) then match the canonical vendor. Geo-prefixed Anthropic profiles (`eu.anthropic.…`) now get live-zone compression instead of being silently skipped; geo-prefixed non-Anthropic vendors stay correctly excluded. - **Converse-body compression (two parts)**: 1. `run_anthropic_compression` no longer bails to passthrough when the body lacks an InvokeModel `anthropic_version` envelope; envelope re-emit stays gated on successful parse. 2. The **live-zone dispatcher now recognizes Bedrock Converse content blocks**. Converse blocks carry no `type` discriminator (the variant is the key: `{"text": …}` vs Anthropic's `{"type":"text","text":…}`), so real Converse user-message text was still passing through uncompressed. A typeless block whose `text` is a JSON string now routes through the same surgical text path. Anthropic blocks always carry `type`, so the Anthropic path is byte-for-byte unchanged; non-text Converse blocks (`{"image":…}`, `{"toolUse":…}`) stay unrecognized and no-op. - **Correct `/converse` upstream routing**: the non-streaming handler resolved the upstream action from a hard-coded `"invoke"`, so `/converse` requests were forwarded to Bedrock's `/invoke` endpoint. It now resolves the action from the inbound path (`extract_invoke_action`), mirroring the streaming handler's `extract_streaming_action`. SigV4 signs the same URL it forwards, so the signature stays consistent. - **`aws-config` `sso` feature**: SSO profiles now resolve through the default credential chain for SigV4 — the credential chain in `docs/bedrock.md` already promised SSO; this makes the code match. - **Ship the `headroom-proxy` binary in published images** (`Dockerfile`): built in the builder stage (`--locked`, with the cargo registry cache mounted at `CARGO_HOME`) and copied into both the debian and distroless runtime images. - **Docs** (`docs/bedrock.md`): document cross-region inference profiles and a "Running the proxy" section. AWS credentials mount at `/home/nonroot/.aws` (the default nonroot image home) where the SDK looks for `~/.aws`, with a note on the root-image alternative. ## Related issues - Closes #976 — ship the `headroom-proxy` binary in published images (this PR implements the exact fix proposed there). - Addresses the **cross-region inference-profile** half of #953 via its proposed `canonical_vendor()` design. Non-Anthropic vendor compression parity (Nova/GLM/MiniMax/ Kimi) is the natural follow-up — `bedrock::vendor` is the shared resolver it can build on. - Extends the native Bedrock InvokeModel compression requested in #734 (the Bedrock slice of #510) to cross-region profiles and Converse bodies. - Partially enables #181 (native, Python-free packaging): the native binary now ships in the images, though full Python-free distribution remains out of scope. ## Alignment with the Rust migration plan Per `docs/spec/022-rust-migration.md`, the migration is **proxy-first**: `headroom-proxy` is the deployable Rust artifact, native routes replace Python passthroughs one at a time (Stage 4 = provider expansion, Bedrock included), and the binary is meant to be "built, tested, and **released together with the Python package**." Two ways this PR advances that: - The binary-in-images change makes the codebase do what the spec already states (ship the artifact) — closing the gap that forced downstreams to build from source. - Hardening the native Bedrock route (cross-region, Converse routing + body compression) is exactly the Stage-4 provider-expansion work, keeping the native path at parity with real traffic so it can be the default rather than a passthrough. ## Testing - [x] Unit tests pass (`cargo test -p headroom-core -p headroom-proxy` — full suites, 0 failures) - [x] Linting passes (`cargo clippy -p headroom-core -p headroom-proxy --all-targets -- -D warnings`) - [x] Formatting passes (`cargo fmt -- --check`) - [x] New tests added — `bedrock::vendor` (foundation + inference-profile matching), `extract_invoke_action` + converse upstream URL, and live-zone Converse text-block routing (`block_has_string_text_field`, converse-vs-anthropic dispatch equivalence). - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-core -p headroom-proxy # all suites: ok, 0 failed $ cargo clippy -p headroom-core -p headroom-proxy --all-targets -- -D warnings # Finished, no warnings $ cargo fmt -- --check # clean # image validation (local, proxy/code extras): $ docker build --target runtime ... # debian: /usr/local/bin/headroom-proxy, --help OK $ docker build --target runtime-slim ... # distroless: binary links + --help OK ``` ## Real Behavior Proof - Environment: native Bedrock proxy against `bedrock-runtime.eu-west-2`, SSO profile, model `eu.anthropic.claude-haiku-4-5-20251001-v1:0`. - Exact command / steps: POST a large multi-turn Converse body to `/model/eu.anthropic.claude-haiku-4-5-20251001-v1:0/converse`; separately build the `runtime` + `runtime-slim` targets and run `/usr/local/bin/headroom-proxy --help`. - Observed result: before — `bedrock_compression_skipped` (geo-prefixed id not recognized), forwarded uncompressed to the wrong `/invoke` upstream; after — geo-prefixed id recognized, `/converse` forwarded to the `/converse` upstream, live-zone dispatcher compresses the Converse user-message text, measurable token savings. Images contain a runnable `headroom-proxy` in both variants. - Not tested: non-Anthropic vendor compression parity (#953 follow-up); Converse `toolResult` nested-text compression (follow-up — only top-level Converse text blocks compress today). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - An earlier revision flipped the EventStream `Accept` default (`*/*`/absent → passthrough); **dropped** — `*/*` is what most clients (incl. reqwest and the proxy's own metrics tests) send while expecting SSE, so forcing passthrough breaks the standard SSE path. - The binary build adds the native-proxy compile to the image build; happy to gate it behind a build arg if maintainers prefer it opt-in. - Addressed a Copilot review round: corrected the `/converse` upstream routing, the stale `run_anthropic_compression` comment, the Dockerfile cargo cache mount + `--locked`, and the nonroot AWS-credentials docs example. |
||
|
Serge ARADJ
|
60d952e857
|
Fix/magika new session hangs on windows (#928)
## Description Brief description of changes and motivation. Fixes #(issue number) ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Change 1 - Change 2 - Change 3 ## Testing Describe the tests you ran to verify your changes: - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ## Test Output ``` # Paste relevant test output here pytest -v tests/test_your_feature.py ``` ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes Any additional information that reviewers should know. <!-- headroom-maintainer-template-completion:start --> ## Description This PR prepares `Fix/magika new session hangs on windows` for review by documenting the intended change, validation evidence, and remaining merge-readiness context. Linked issues: None declared. ## Type of Change - [x] Bug fix - [ ] New feature - [ ] Documentation - [ ] Refactor - [ ] Tests only ## Changes Made - Commit: fix(magika): bound ONNX session init with configurable timeout to pre… - Commit: Merge branch 'main' into fix/magika-new-session-hangs-on-windows - Touches `crates/headroom-core/src/transforms/magika_detector.rs` - Touches `headroom/proxy/handlers/openai.py` ## Testing - [x] GitHub checks reviewed - [x] Metadata/template validation - [ ] Local functional testing ### Test Output ```text gh pr view 928 --repo chopratejas/headroom --json statusCheckRollup - CI / changes: SUCCESS - Init E2E / docker-init-e2e: SUCCESS - PR Governance / label: SUCCESS - Wrap E2E / docker-wrap-e2e: SUCCESS - rust / test (ubuntu): SUCCESS - CI / commitlint: SUCCESS - rust / wheels (x86_64-unknown-linux-gnu): SUCCESS - rust / wheels (aarch64-apple-darwin): SUCCESS - CI / lint: SUCCESS - rust / audit: SUCCESS - rust / parity (nightly, allowed to fail during Phase 0): SKIPPED - CI / build-wheel: SUCCESS ``` ## Real Behavior Proof - Environment: GitHub PR metadata and checks for `chopratejas/headroom` PR #928. - Exact command / steps: Reviewed PR title, commits, changed files, linked issues, labels, and check rollup; appended this maintainer template completion block without replacing the author's original description. - Observed result: PR body now contains all required governance sections, checked readiness fields, and a non-placeholder validation evidence block. - Not tested: This pass updated PR metadata only; code validation remains represented by the linked GitHub checks and any author-provided evidence above. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review <!-- headroom-maintainer-template-completion:end --> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Yasser Sheikh
|
b08ec15b0d
|
fix(proxy): add native Bedrock converse-stream route (#917)
## Description
Adds native Bedrock `POST /model/{model_id}/converse-stream` routing in
`headroom-proxy` by reusing the existing streaming handler and
preserving route-specific upstream action forwarding.
This addresses a gap where native Bedrock streaming support existed for
`invoke-with-response-stream` but not `converse-stream`, even though
both share the same EventStream transport and SSE translation path in
this proxy.
Fixes #919
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
## Changes Made
- Add route mount in `crates/headroom-proxy/src/proxy.rs`:
- `POST /model/:model_id/converse-stream` ->
`bedrock::invoke_streaming::handle_invoke_streaming`
- Update streaming handler URL construction in
`crates/headroom-proxy/src/bedrock/invoke_streaming.rs`:
- infer action from inbound path (`invoke-with-response-stream` or
`converse-stream`)
- build upstream URL with the resolved action
- return structured `400` for unsupported streaming action paths
- Add unit tests in
`crates/headroom-proxy/src/bedrock/invoke_streaming.rs`:
- action extraction coverage for both streaming paths
- upstream URL construction coverage for `converse-stream`
- Add integration coverage in
`crates/headroom-proxy/tests/integration_bedrock_streaming.rs`:
- `converse_stream_route_translates_to_sse`
- Add changelog entry under `Unreleased` bug fixes in `CHANGELOG.md`.
## Testing
- `cargo fmt --all`
- `cargo test -p headroom-proxy --test integration_bedrock_streaming --
--nocapture`
- `cargo test -p headroom-proxy --test integration_bedrock_metrics --
--nocapture`
## Real behavior proof
- **Setup tested on**
- macOS (darwin)
- Rust workspace local dev build
- `headroom-proxy` integration tests using wiremock upstream (no AWS
dependency)
- **Exact commands run after patch**
- `cargo test -p headroom-proxy --test integration_bedrock_streaming --
--nocapture`
- `cargo test -p headroom-proxy --test integration_bedrock_metrics --
--nocapture`
- **After-fix evidence + observed result**
- New integration test `converse_stream_route_translates_to_sse` passes.
- Streaming suite result: `10 passed; 0 failed`.
- Metrics suite result: `4 passed; 0 failed`.
- Logs show requests reaching `/model/.../converse-stream` and flowing
through Bedrock streaming path.
- **What I did not test**
- Live AWS Bedrock calls against real credentials/models.
- End-to-end CLI/runtime behavior outside Rust integration test harness.
|
||
|
Focused Instability
|
0632eba6c3
|
fix(policy): correct warm-cache penalty in net_mutation_gain to (S + dT) (#903)
Fixes #906. ## What Part of #904 (net-cost policy completion tracking). Follows up #856 / #857 with the corrected gain term raised in [this #856 comment](https://github.com/chopratejas/headroom/issues/856#issuecomment-4679706939) — prerequisite for P2 (pipeline consumption), which would otherwise wire in a formula that is always-pro-mutation by exactly `P_alive·(w−r)·ΔT`. ## Why the corrected form is right With a live cache, the ΔT tokens a mutation removes are **already cache-written** — keeping them costs only reads (`ΔT·r·R`), so a mutation cannot avoid a fresh write of them. Blending alive (`ΔT·r·R − (w−r)·S`) and dead (`ΔT·(w + r·(R−1))`, no suffix penalty) cases over `P_alive`: ``` gain = ΔT·(w + r·(R−1)) − P_alive·(w−r)·(S + ΔT) ``` Three independent confirmations: 1. **Direct cost check** (w=1.25, r=0.1, warm, ΔT=50K, S=10K, R=2): keeping costs 60K·0.1·2 = 12,000 in reads; mutating costs 10K·1.25 (suffix rewrite, the first of the R touches) + 10K·0.1 (remaining read) = 13,500 — mutation loses 1,500, matching the corrected gain of −1,500. The old form said +56,000. 2. **The issue's own anchors**: corrected break-even is exactly `R = 11.5·S/ΔT` → 2K/50K = 287.5 (~290, as the issue says) and 50K/10K = 2.3 — the spec text's anchor numbers can only be derived from the corrected penalty. The implemented form gave 276 and *negative*. 3. **Internal consistency**: `break_even_reads` already shipped with the ~11.5·S/ΔT shape; this PR reconciles `net_mutation_gain` with it (and drops break_even's stray −1 term). ## Behavior changes (formula is still dead code — nothing consumes it yet) - 50K-shave/10K-suffix/R=3 golden: +61,000 → **+3,500** (tight win, consistent with 2.3-read break-even). - 2K-shave/50K-suffix/R=10 golden: −53,200 → **−55,500**. - S=0 boundary: an edit of already-cached content with no suffix is profitable whenever ≥1 read remains (`gain = ΔT·r·R`), and exactly 0 at R=0 warm. Not-yet-cached (live-zone) content should bypass the formula — now documented on both implementations. Rust + Python goldens updated in lockstep: 13 Rust + 19 Python tests green. ## Next (separate PRs) - **P2**: flag-gated consumption (`HEADROOM_NET_COST_POLICY=1`) with decision telemetry. - **P3**: batch deep edits (reclaim threshold), idle-timer compaction near TTL lapse. Co-authored-by: integration-check <integration@local> |
||
|
Focused Instability
|
d5f58026e2
|
feat: net-cost cache mutation formula on CompressionPolicy (#856 P1) (#857)
Closes #856 **P1 of the #856 phased plan** — pure functions, zero behavior change. (Closing keyword links the issue; if P2 hasn't started when this merges, reopen #856 or it remains the design record for the P2/P3 follow-up PRs.) ## What Adds the break-even decision rule for deep (pre-cache-marker) edits to `CompressionPolicy`: ``` gain = ΔT · (w + r·(R−1)) − P_alive · (w − r) · S ``` - `net_mutation_gain()`, `should_mutate_deep()` (gain > 0), `break_even_reads()` (R = ((w−r)/r)·(S/ΔT−1) ≈ 11.5·S/ΔT) on the Rust struct (source of truth) and the Python hand-mirror, following the existing F2.1/F2.2 parity pattern. - `CACHE_WRITE_MULTIPLIER = 1.25` / `CACHE_READ_MULTIPLIER = 0.1` public constants (Anthropic 5-minute tier). - Inputs clamped (`expected_reads ≥ 0`, `p_alive ∈ [0,1]`); methods take `&self`/`self` so a follow-up can add per-mode margins. - The formula derives the existing Subscription live-zone policy as its S=0 special case rather than contradicting it. **No callers yet.** P2 (consuming this in `TransformPipeline` behind `HEADROOM_NET_COST_POLICY`, replacing the binary `live_zone_only` gate, with decision telemetry) is specified in #856 and awaits maintainer direction — this PR just lands the audited arithmetic both dispatchers will share. ## Tests Golden-value parity: 6 new Rust unit tests and 7 new Python tests assert the **identical scenario numbers** (loss −53 200 for a 2K shave under a 50K warm suffix at R=10; win +61 000 for a 50K shave under a 10K suffix at R=3; S=0 always profitable; P_alive=0 always profitable — the idle-timer window; clamping; break-even 276 reads for the 2K/50K anchor). A drift on either side trips the pair loudly, same contract as the existing field-map parity test. - `cargo test -p headroom-core --lib compression_policy`: 12 passed (6 existing + 6 new) - `pytest tests/test_compression_policy.py`: 17 passed (10 existing + 7 new) - `cargo fmt --check`, `cargo clippy -p headroom-core` clean; `ruff check` + `ruff format --check` clean ## Real behavior proof Not applicable in the runtime sense — this PR intentionally adds **no runtime behavior** (pure functions, no call sites). The arithmetic is validated against the research anchors above in both languages' test suites; live decision telemetry arrives with P2 where the formula first gates real traffic. ## Out of scope P2 (flag-gated pipeline consumption + telemetry), P3 (deep-edit batching, idle-timer compaction near TTL lapse), retiring the deprecated `volatile_token_threshold`/`max_lossy_ratio` fields — all tracked in #856. --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com> |
||
|
Focused Instability
|
06b2625b17
|
feat: gated Markdown-KV compaction formatter (serialization-aware output) (#859)
Closes #858. ## What Adds an opt-in **Markdown-KV** renderer to the lossless-first compaction stage, plus the plumbing to pick a compaction formatter by name. Default behavior is unchanged (`csv-schema`). Format-comprehension benchmarks show models retrieve values from Markdown-KV substantially more reliably than from CSV (~60.7% vs ~44.3%) — token-cheapest is not the same as most comprehensible. This makes the trade-off selectable per workload. ## How - **`MarkdownKvFormatter`** (`compaction/formatter.rs`): keeps the `[N]{cols}` declaration line, renders each row as a Markdown list item with `key: value` lines. - Missing cells omitted entirely (the KV advantage over positional CSV). - Strings ambiguous on a line (newlines, leading/trailing whitespace, empty) render JSON-quoted; everything else raw — commas and quotes need no escaping. - Nested cells inline compact JSON; opaque cells keep the fixed `<<ccr:HASH,KIND,SIZE>>` marker contract shared by all formatters. - **`CompactionStage::from_format_name`** maps `"csv-schema" | "json" | "markdown-kv"` to presets. - **Core**: `SmartCrusher::with_compaction_format(config, name)` — standard OSS composition with the named formatter. - **PyO3 bridge**: `SmartCrusher.with_compaction_format(config, format_name)` staticmethod; `ValueError` on unknown names (loud, no silent fallback). - **Python**: `SmartCrusher(compaction_format=...)` kwarg, falling back to the `HEADROOM_COMPACTION_FORMAT` env var, default `"csv-schema"`. ## Safety - **Default-off**: the default constructor path still calls the Rust `new()` constructor, so byte-parity coverage stays on the exact production codepath. A test asserts default output is byte-identical to an explicit `csv-schema` opt-in. - The existing `lossless_min_savings_ratio` gate (0.30) still applies. Markdown-KV repeats field names per row, so it clears the gate less often than CSV and falls through to the lossy path — we never inline a "lossless" rendering that isn't actually smaller. - CCR marker format unchanged across formatters; downstream retrieval pattern-matching keeps working. - No user/assistant content dropped — the formatter is a pure rendering of the same Compaction IR. ## Tests - Rust: 10 new unit tests in `compaction/formatter.rs` (table/buckets rendering, missing-cell omission, string quoting, CCR markers, drop summary, byte-size sanity vs raw JSON). `cargo test -p headroom-core`: 894 passed. Clippy + fmt clean. - Python: `tests/test_compaction_markdown_kv.py` (10 tests) — bridge rendering end-to-end, name→preset parity with the default constructor, kwarg/env knob precedence, loud failure on unknown names, default-output-unchanged guarantee. Existing smart_crusher suite: 38 passed. --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com> |
||
|
dependabot[bot]
|
4ff7b4426d
|
ci: bump pyo3 from 0.22.6 to 0.24.1 in the cargo group across 1 directory (#270)
Bumps the cargo group with 1 update in the / directory: [pyo3](https://github.com/pyo3/pyo3). Updates `pyo3` from 0.22.6 to 0.24.1 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pyo3/pyo3/releases">pyo3's releases</a>.</em></p> <blockquote> <h2>PyO3 0.24.1</h2> <p>This release is a security fix for the <code>PyString::from_object</code> method, which passed <code>&str</code> data to the Python C API without checking for a terminating nul byte. All historical PyO3 versions are affected, and we recommend you upgrade if you are using <code>PyString::from_object</code>. Thank you to <a href="https://github.com/vthib"><code>@vthib</code></a> for the report and <a href="https://github.com/Dr-Emann"><code>@Dr-Emann</code></a> for the fix. A RUSTSEC advisory will be published shortly.</p> <p>Aside from the security fix, this release contains a number of other non-breaking additions:</p> <ul> <li>An <code>abi3-py313</code> feature to support compiling with the Python 3.13 stable ABI.</li> <li><code>PyAnyMethods::getattr_opt</code> to get optional attributes without paying the cost of a Python exception when the attribute in question does not exist.</li> <li>Constructor for <code>PyInt::new</code>.</li> <li><code>with_critical_section2</code> for locking two objects at the same time on the free-threaded build.</li> <li>Fix for a PyO3 0.24.0 regression with <code>Option<&str></code> and <code>Option<&T></code> (where <code>T: PyClass</code>) function arguments no longer being permitted</li> </ul> <p>There are also a few other small bug fixes for edge cases, mostly related to compile errors from PyO3's macro code.</p> <p>Thank you to the following contributors for the improvements:</p> <p><a href="https://github.com/bschoenmaeckers"><code>@bschoenmaeckers</code></a> <a href="https://github.com/davidhewitt"><code>@davidhewitt</code></a> <a href="https://github.com/Dr-Emann"><code>@Dr-Emann</code></a> <a href="https://github.com/emmagordon"><code>@emmagordon</code></a> <a href="https://github.com/epontan"><code>@epontan</code></a> <a href="https://github.com/Icxolu"><code>@Icxolu</code></a> <a href="https://github.com/IvanIsCoding"><code>@IvanIsCoding</code></a> <a href="https://github.com/jelmer"><code>@jelmer</code></a> <a href="https://github.com/jonaspleyer"><code>@jonaspleyer</code></a> <a href="https://github.com/ngoldbaum"><code>@ngoldbaum</code></a> <a href="https://github.com/Owen-CH-Leung"><code>@Owen-CH-Leung</code></a> <a href="https://github.com/Tpt"><code>@Tpt</code></a> <a href="https://github.com/Trolldemorted"><code>@Trolldemorted</code></a> <a href="https://github.com/XuehaiPan"><code>@XuehaiPan</code></a></p> <h2>PyO3 0.24.0</h2> <p>This release is an incremental improvement of refinements and optimizations following the new APIs established in PyO3's last few releases.</p> <p>Support for <code>jiff</code> datetime conversions have been added, and also UUID conversions.</p> <p>The <code>FromPyObject</code> derive macro has gained new <code>#[pyo3(default = ...)]</code> and <code>#[pyo3(rename_all = ...)]</code> options, and the <code>IntoPyObject</code> derive macro has gained a new <code>#[pyo3(into_py_with = ...)]</code> option.</p> <p>PyO3 will now pass positional arguments to Python functions using the "vectorcall" protocol in many cases, which should be an optimization over the previous behaviour (of creating a Python tuple of positional arguments).</p> <p>Many methods on iterators of Python collections have been optimized.</p> <p>There are also many other incremental improvements, bug fixes and smaller features.</p> <p>Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following contributors' commits are included in this release:</p> <p><a href="https://github.com/0x676e67"><code>@0x676e67</code></a> <a href="https://github.com/alex"><code>@alex</code></a> <a href="https://github.com/arielb1"><code>@arielb1</code></a> <a href="https://github.com/bschoenmaeckers"><code>@bschoenmaeckers</code></a> <a href="https://github.com/davidhewitt"><code>@davidhewitt</code></a></p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md">pyo3's changelog</a>.</em></p> <blockquote> <h2>[0.24.1] - 2025-03-31</h2> <h3>Added</h3> <ul> <li>Add <code>abi3-py313</code> feature. <a href="https://redirect.github.com/PyO3/pyo3/pull/4969">#4969</a></li> <li>Add <code>PyAnyMethods::getattr_opt</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4978">#4978</a></li> <li>Add <code>PyInt::new</code> constructor for all supported number types (i32, u32, i64, u64, isize, usize). <a href="https://redirect.github.com/PyO3/pyo3/pull/4984">#4984</a></li> <li>Add <code>pyo3::sync::with_critical_section2</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4992">#4992</a></li> <li>Implement <code>PyCallArgs</code> for <code>Borrowed<'_, 'py, PyTuple></code>, <code>&Bound<'py, PyTuple></code>, and <code>&Py<PyTuple></code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/5013">#5013</a></li> </ul> <h3>Fixed</h3> <ul> <li>Fix <code>is_type_of</code> for native types not using same specialized check as <code>is_type_of_bound</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4981">#4981</a></li> <li>Fix <code>Probe</code> class naming issue with <code>#[pymethods]</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4988">#4988</a></li> <li>Fix compile failure with required <code>#[pyfunction]</code> arguments taking <code>Option<&str></code> and <code>Option<&T></code> (for <code>#[pyclass]</code> types). <a href="https://redirect.github.com/PyO3/pyo3/pull/5002">#5002</a></li> <li>Fix <code>PyString::from_object</code> causing of bounds reads with <code>encoding</code> and <code>errors</code> parameters which are not nul-terminated. <a href="https://redirect.github.com/PyO3/pyo3/pull/5008">#5008</a></li> <li>Fix compile error when additional options follow after <code>crate</code> for <code>#[pyfunction]</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/5015">#5015</a></li> </ul> <h2>[0.24.0] - 2025-03-09</h2> <h3>Packaging</h3> <ul> <li>Add supported CPython/PyPy versions to cargo package metadata. <a href="https://redirect.github.com/PyO3/pyo3/pull/4756">#4756</a></li> <li>Bump <code>target-lexicon</code> dependency to 0.13. <a href="https://redirect.github.com/PyO3/pyo3/pull/4822">#4822</a></li> <li>Add optional <code>jiff</code> dependency to add conversions for <code>jiff</code> datetime types. <a href="https://redirect.github.com/PyO3/pyo3/pull/4823">#4823</a></li> <li>Add optional <code>uuid</code> dependency to add conversions for <code>uuid::Uuid</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4864">#4864</a></li> <li>Bump minimum supported <code>inventory</code> version to 0.3.5. <a href="https://redirect.github.com/PyO3/pyo3/pull/4954">#4954</a></li> </ul> <h3>Added</h3> <ul> <li>Add <code>PyIterator::send</code> method to allow sending values into a python generator. <a href="https://redirect.github.com/PyO3/pyo3/pull/4746">#4746</a></li> <li>Add <code>PyCallArgs</code> trait for passing arguments into the Python calling protocol. This enabled using a faster calling convention for certain types, improving performance. <a href="https://redirect.github.com/PyO3/pyo3/pull/4768">#4768</a></li> <li>Add <code>#[pyo3(default = ...']</code> option for <code>#[derive(FromPyObject)]</code> to set a default value for extracted fields of named structs. <a href="https://redirect.github.com/PyO3/pyo3/pull/4829">#4829</a></li> <li>Add <code>#[pyo3(into_py_with = ...)]</code> option for <code>#[derive(IntoPyObject, IntoPyObjectRef)]</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4850">#4850</a></li> <li>Add FFI definitions <code>PyThreadState_GetFrame</code> and <code>PyFrame_GetBack</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4866">#4866</a></li> <li>Optimize <code>last</code> for <code>BoundListIterator</code>, <code>BoundTupleIterator</code> and <code>BorrowedTupleIterator</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4878">#4878</a></li> <li>Optimize <code>Iterator::count()</code> for <code>PyDict</code>, <code>PyList</code>, <code>PyTuple</code> & <code>PySet</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4878">#4878</a></li> <li>Optimize <code>nth</code>, <code>nth_back</code>, <code>advance_by</code> and <code>advance_back_by</code> for <code>BoundTupleIterator</code> <a href="https://redirect.github.com/PyO3/pyo3/pull/4897">#4897</a></li> <li>Add support for <code>types.GenericAlias</code> as <code>pyo3::types::PyGenericAlias</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4917">#4917</a></li> <li>Add <code>MutextExt</code> trait to help avoid deadlocks with the GIL while locking a <code>std::sync::Mutex</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4934">#4934</a></li> <li>Add <code>#[pyo3(rename_all = "...")]</code> option for <code>#[derive(FromPyObject)]</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4941">#4941</a></li> </ul> <h3>Changed</h3> <ul> <li>Optimize <code>nth</code>, <code>nth_back</code>, <code>advance_by</code> and <code>advance_back_by</code> for <code>BoundListIterator</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4810">#4810</a></li> <li>Use <code>DerefToPyAny</code> in blanket implementations of <code>From<Py<T>></code> and <code>From<Bound<'py, T>></code> for <code>PyObject</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4593">#4593</a></li> <li>Map <code>io::ErrorKind::IsADirectory</code>/<code>NotADirectory</code> to the corresponding Python exception on Rust 1.83+. <a href="https://redirect.github.com/PyO3/pyo3/pull/4747">#4747</a></li> <li><code>PyAnyMethods::call</code> and friends now require <code>PyCallArgs</code> for their positional arguments. <a href="https://redirect.github.com/PyO3/pyo3/pull/4768">#4768</a></li> <li>Expose FFI definitions for <code>PyObject_Vectorcall(Method)</code> on the stable abi on 3.12+. <a href="https://redirect.github.com/PyO3/pyo3/pull/4853">#4853</a></li> <li><code>#[pyo3(from_py_with = ...)]</code> now take a path rather than a string literal <a href="https://redirect.github.com/PyO3/pyo3/pull/4860">#4860</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
chopratejas
|
2a717a993e |
fix(observability): G3 remediation — bound cardinality + wire dead metrics
Phase G PR-G3 review identified 5 Critical + 4 High + 5 Medium
findings. This commit lands all 14 fixes plus the optional nits.
CRITICAL
* C1 (cardinality DoS): `service_tier` was read from inbound JSON
and used verbatim as a metric label. A malicious client could
blow up the metric vector unboundedly. Added bounded vocabulary
in `metric_names.rs::service_tier` ({auto, default, flex,
on_demand, priority, scale, other-sentinel}) + a `validate()`
helper. Both request-side (`handlers/responses.rs`) and
response-side (`proxy.rs` Responses arm) gate raw values through
it.
* C2 (dead metric): `proxy_passthrough_bytes_modified_total` had
no production emit site. Wired it in `proxy.rs` to fire when a
dispatcher arm returning `NoCompression`/`Passthrough` produces
a body of a different byte length (a true cache-poisoning
regression detector). The check runs BEFORE the PR-E4
prompt_cache_key injector so legitimate injector mutations do
not trip the alarm.
* C3 (Python/Rust boundary): `proxy_image_generation_call_log_redacted_total`
was a dead Rust counter — the redaction happens entirely in the
Python proxy's request_logger. Removed the Rust counter; moved
the metric to the Python proxy's `/metrics` exporter via the
existing `redactions_total()` module-level counter.
* C4 (Python/Rust boundary): `wrap_rtk_invocations_total` was a
dead Rust counter with no wrap-side bridge. Removed the Rust
counter; added new `headroom/cli/wrap_rtk_metrics.py` with
`record_rtk_invocation(tool, delta)` + `rtk_invocation_counts()`
primitives and surfaced them via the Python proxy's `/metrics`
exporter.
* C5 (dead metric): `proxy_compression_rejected_by_token_check_total`
had no production caller. Wired it in
`live_zone_anthropic.rs`, `live_zone_openai.rs`, and
`live_zone_responses.rs` to increment on every
`BlockAction::RejectedNotSmaller` block in the manifest. The
metric now reflects real "compressor ran but kept original"
cases.
HIGH
* H1 (per-strategy ratio garbage): `proxy_compression_ratio_by_strategy`
emitted the same aggregate ratio for every strategy in
`strategies_applied` when multiple strategies ran on one body.
Added `per_strategy_tokens: Vec<PerStrategyTokens>` to
`Outcome::Compressed`; per-strategy `(before, after)` is
accumulated from the manifest at the wrapper sites and emitted
one sample per strategy in `proxy.rs`. Empty vec → fallback to
one aggregate-labelled sample with a debug log (Phase E
normalization paths that don't track per-strategy tokens).
* H2 (aborted stream): cache_hit_rate observed on client
disconnects mid-stream. Added a gate: Anthropic only fires when
`state.status == MessageStop`, OpenAI Responses only when
`terminal_status().is_some()`. Extracted the gate into the
pure function `compute_anthropic_session_hit_rate(state)` so
the H2 contract is unit-testable independent of the shared
global registry.
* H3 (docs lie + alarm contract): docs claimed HELP/TYPE is
reachable on fresh boot, then contradicted itself. Force-zero
every counter / gauge MetricVec with an `__init__` sentinel
label on each scrape so HELP/TYPE + a zero row are visible from
boot. Histograms are NOT force-zeroed (a synthetic observe(0.0)
would pollute percentiles). PromQL queries in docs filter
`{... != "__init__"}` so the sentinel rows are excluded from
aggregations.
* H4 (crate-version dependency): pinned `prometheus = "=0.13.4"`
exactly (no caret) so a future minor bump cannot silently break
the H3 force-zero contract that relies on this crate's gather()
semantics. Added a clear "retest the alarm contract on bump"
paragraph in docs.
MEDIUM
* M1 (saturate on cached > input): OpenAI Chat + Responses cache-
hit-rate computed `non_cached = input.saturating_sub(cached)`,
silently clamping to 0 if `cached > input`. Per "no silent
fallbacks", log + skip the emit on this wire-format pathology.
* M2 (over-fire on non-image base64): Python redactor's "density
heuristic" over-fired on encrypted blobs / signed tokens /
minified JSON / tool outputs. Tightened: only redact strings
inside known image-bearing JSON paths (`data`, `url`,
`image_url`, `image`) OR strings starting with `data:image/`.
* M3 (NaN clamp): cache_hit_rate::observe used `f64::clamp(0,1)`
which returns NaN for NaN input; the `debug_assert!` was
compiled out in release. Added `is_finite()` guard with a
loud-log + skip before observe.
* M4 (PromQL median-only): added p95, p99, mean (sum/count), and
Phase H canary-gate query section to docs. Canary fails if ANY
of {p50, p95, p99, mean} regresses below the Python baseline.
* M5 (label byte vs char): the `<image:base64-redacted bytes=N>`
placeholder reported character count, not UTF-8 byte count.
Switched to `.encode('utf-8').__len__()` so the label is
honest for non-ASCII payloads (ASCII base64 still has byte ==
char so existing scrapes are unchanged).
OPTIONAL
* Removed dead `debug_assert_eq!(buffered.len(), buffered.len(),
...)` no-op in proxy.rs.
* Normalised `record_response_status` log level from `info` to
`debug` to match peer metric helpers.
Tests:
* Rust: 11 integration_metrics tests (was 6) + 9 cache_hit_rate
unit tests (was 4) + 2 compression_ratio (unchanged). New
coverage: service_tier known/unknown bucketing, C2 alarm wire,
H1 per-strategy ratio, H2 abort gate, M3 NaN/inf skip.
* Python: 27 tests (was 13). New coverage: M2 path-gated
redaction, M5 byte vs char label, wrap_rtk_metrics primitive
thread safety and validation.
`cargo fmt --check`, `cargo clippy --workspace -- -D warnings`,
`cargo test -p headroom-proxy --lib` (221 passed) and the
integration_metrics + integration_compression +
integration_volatile_detector + integration_cache_control +
integration_cache_drift + integration_responses +
integration_bedrock_metrics test files all green. Full
`cargo test --workspace` deferred — disk pressure during the
agent session left insufficient space for the linker to write
the full integration test artifacts; runs that did fit all
passed. `make ci-precheck` deferred for the same reason.
ruff check + ruff format + mypy headroom/proxy/request_logger.py
+ headroom/cli/wrap_rtk_metrics.py + headroom/proxy/prometheus_metrics.py
green.
|
||
|
chopratejas
|
5f264a5329 |
fix(observability): wire Phase G PR-G3 RTK + proxy metrics (H-blocker)
Phase H ("retire the Python proxy") needs cache-hit-rate parity
between the Rust and Python proxies during canary. This PR lands
the per-invocation RTK metrics and the proxy-side observability
surface that the canary gate depends on.
Rust observability:
- `proxy_cache_hit_rate_per_session{provider}` — histogram, emitted
per session at SSE state-machine close (Anthropic message_delta,
OpenAI Chat final usage chunk, OpenAI Responses response.completed).
The Phase H canary gate metric.
- `proxy_compression_ratio_by_strategy{strategy, content_type}` —
histogram; one sample per shrunk block.
- `proxy_compression_rejected_by_token_check_total{strategy}` —
counter for tokenizer-validated rejections.
- `proxy_passthrough_bytes_modified_total{path}` — counter (must
stay 0 outside compression hot path; alarmable via PromQL rate).
- `proxy_rate_limit_remaining_{requests,tokens,input_tokens,output_tokens}{provider}` —
gauges populated from anthropic-ratelimit-* / x-ratelimit-* headers.
- `proxy_service_tier_count_total{tier}` and
`proxy_response_status_count_total{status}` — counters for
Responses-API outcome telemetry.
- `proxy_image_generation_call_log_redacted_total` — counter.
- `wrap_rtk_invocations_total{tool}` and
`wrap_rtk_tokens_saved_per_session` — RTK metrics exposed via
the proxy's /metrics scrape so wrap-side tail can increment
through one observability surface.
All metric names and label keys live in a single
`observability/metric_names.rs` constants module per realignment
build-constraint "configurable". Bounded label vocabularies
(service_tier, response_status, provider) are defined alongside.
Python (P4-45):
- `headroom/proxy/request_logger.py` — base64-image payloads in
request/response logs over 1024 bytes are replaced with
`<image:base64-redacted bytes=N>` placeholders. Walks Anthropic
source.data and OpenAI data URLs. No regexes — substring +
density heuristic.
Tests:
- `crates/headroom-proxy/tests/integration_metrics.rs` — 6 tests
covering cache-hit-rate, compression-ratio, passthrough-bytes,
service-tier, response-status, and rate-limit-snapshot.
- `tests/test_image_log_redaction.py` — 13 tests for the Python
redaction helper.
- Existing tests: 1100+ Rust + 76 Python regression checks green.
Docs:
- `docs/observability.md` — metric catalogue + PromQL queries.
- `docs/rtk-architecture.md` — locks the wrap-CLI-only decision so
future contributors don't relitigate proxy-side RTK.
No silent fallbacks: zero-denominator cache-hit-rate logs and
skips rather than synthesising 0.0. Unparseable rate-limit headers
stay None rather than coerced to 0. Missing upstream JSON fields
log + skip emit rather than fabricating data.
|
||
|
Tejas Chopra
|
294df2b894
|
Merge pull request #403 from chopratejas/realign-F2_2-policy-tuning
fix(proxy): F2.2 — per-mode CompressionPolicy tuning fields |
||
|
Tejas Chopra
|
c83687798b | Fix Windows ORT builds and Docker signing retries | ||
|
Tejas Chopra
|
17ffae0cd8 |
fix: clear CI mypy + rust test failures introduced in eaf5980
compression_units.py:
- Replace dict-unpacking pattern with dataclasses.replace() so mypy can
type-check fields. The `**base` form forced mypy to infer
`dict[str, object]`, which doesn't satisfy the per-field types of
UnitCompressionResult (46 arg-type errors).
- Use `isinstance(candidates, Iterable)` for the transform-iteration
guard. The previous `iter()` call had a `# type: ignore[arg-type]`
that was misclassified — mypy actually emits `call-overload` here.
live_zone_thresholds.rs:
- Update the JsonArray threshold assertion from 1024 to 512 to match
the new constant.
|
||
|
Tejas Chopra
|
eaf5980b4a | fix: stabilize codex compression, stats, and proxy lifecycle | ||
|
Tejas Chopra
|
c6ecdc4299
|
Merge pull request #427 from mbachaud/fix/vertex-dead-code-and-body-limit
fix(vertex,bedrock): remove dead handle_raw_predict, honour X-Forwarded-Proto, cap Bedrock body size |
||
|
SwiftWing21
|
b784f400c1 |
fix(bedrock): use is_char_boundary loop instead of floor_char_boundary (MSRV 1.80)
floor_char_boundary was stabilised in Rust 1.91; headroom's MSRV is 1.80. Walk back from byte 64 manually — UTF-8 codepoints are at most 4 bytes so this loop runs at most 3 times in the worst case. |
||
|
SwiftWing21
|
28b2bacdf6 |
fix(vertex,bedrock): remove dead handle_raw_predict, honour X-Forwarded-Proto, cap Bedrock body size
Three related proxy hygiene fixes: #417 — delete handle_raw_predict from vertex/raw_predict.rs The dispatcher (handle_vertex_predict_dispatch in vertex/mod.rs) calls forward_vertex_request directly. handle_raw_predict was never wired into the router and is unreachable code. Deleting it removes 80 lines of dead logic and eliminates confusion for new contributors. #418 — honour X-Forwarded-Proto in forward_vertex_request build_forward_request_headers received a hardcoded literal 'http' for the forwarded protocol. Proxies deployed behind a TLS load balancer would emit X-Forwarded-Proto: http even for HTTPS upstream connections. Now reads the incoming X-Forwarded-Proto header and falls back to 'http' only when the header is absent. #416 — apply DefaultBodyLimit to Bedrock routes in proxy.rs Bedrock handlers use axum's Bytes extractor, which respects DefaultBodyLimit (default 2 MiB). All other routes buffer the body manually and apply config.max_body_bytes (default 100 MiB). Adding .layer(DefaultBodyLimit::max(state.config.max_body_bytes)) to the Bedrock router aligns the cap across providers. Closes #416, #417, #418 |
||
|
SwiftWing21
|
42ff8afa90 | fix(bedrock): apply rustfmt to header_value_preview tests | ||
|
SwiftWing21
|
82468e4e99 |
fix(bedrock): use floor_char_boundary to avoid UTF-8 slice panic in header preview
header_value_preview in eventstream_to_sse.rs used a raw byte-slice (&s[..64]) to truncate long header strings for log output. If byte index 64 landed inside a multi-byte codepoint (e.g. 63 ASCII chars followed by é or an emoji), Rust panics at runtime. Replace with floor_char_boundary(64) which returns the largest valid char boundary ≤ 64 without scanning the whole string. Two regression tests added: - truncates_at_char_boundary: 63 ASCII + é → must not panic, must end with … - exact_boundary_not_truncated: 64-byte ASCII string is returned unchanged Fixes #415 |
||
|
chopratejas
|
89f7b6c2dd |
fix: complete /v1/responses compression telemetry, multi-frame WS, frozen-count cap
Builds on PR #406 (HTTP /v1/responses PyO3) and PR #410 (WS first-frame PyO3) which landed the binding for `compress_openai_responses_live_zone`. This change closes the remaining gaps so every (provider × endpoint × auth-mode × streaming) combination compresses AND surfaces in the dashboard. Telemetry: extend the PyO3 binding return tuple from `(bytes, modified)` to `(bytes, modified, tokens_saved, transforms_applied)` by adding `CompressionManifest::tokens_saved()` and `transforms_applied()` accessors on the existing manifest. The Python proxy populates request-log telemetry from the binding output instead of recounting tokens. Updates the existing 2-tuple call sites in HTTP and WS first-frame, plus the unpacks in tests. WebSocket multi-frame compression: subscription Codex users keep a long-lived WS open and send multiple `response.create` events per session. PR #410 only compressed the first frame; subsequent frames went raw. Added `_maybe_compress_response_create_frame` closure inside `_client_to_upstream` that runs the same Rust dispatcher on every client→upstream `response.create` text frame, passes other event types (response.cancel, session.update, etc.) through unchanged, and accumulates `tokens_saved` / `transforms_applied` / `ws_frames_compressed` counters across the session. Pre-existing dashboard gap: `streaming.py` and `anthropic.py` write `RequestLog` entries; the non-streaming OpenAI HTTP and WS handlers did not. Result: /transformations/feed was invisible for every Codex turn and every Cline / OpenClaude / Aider turn. Added the same wiring in `handle_openai_chat` (non-streaming), `handle_openai_responses` (non-streaming HTTP), and `handle_openai_responses_ws` (session-end). All three populate `auth_mode` + `endpoint` tags so the dashboard can break compression activity down by client class (PAYG / OAuth / Subscription) and surface (`chat_completions` / `responses_http` / `responses_ws`). The WS metric record is now unconditional — was previously gated on `tokens_saved > 0`, so first-frame no-changes never registered. compute_frozen_count over-freeze for prose-format clients: `compute_frozen_count` walked until it found an unstable `tool_result` / `role: "tool"` block. Cline / OpenClaude / Aider — clients that embed tool calls as XML inside plain text — never produce such a boundary, so the function returned `len(messages)` and the pipeline froze 100% of messages including the brand-new user turn. Live zone empty → `Transform content_router: 16414 → 16414 tokens (saved 0)`. Reported on Discord 2026-05-07 with Cline+DeepSeek. Fix: cap at `max(0, len(messages) - 1)`. Updates 3 existing test assertions whose expected values encoded the old over-freeze. Adds 6 new prose-format invariant tests. CodeQL "clear-text logging of sensitive information" fix: `tests/e2e_real_compression.py` previously stored API keys in local variables in the same scope as diagnostic prints, which CodeQL flagged via data-flow analysis. Refactored to read keys from `os.environ` inside the request helper — the credentials never enter the runner's main scope, so the taint flow never reaches the print. End-to-end verification with real keys (.env): /v1/messages (PAYG, non-stream) tok 14109 → 969 saved 13140 /v1/messages (PAYG, stream) tok 14109 → 969 saved 13140 /v1/chat/completions (PAYG, non-stream) tok 18460 → 1374 saved 17086 /v1/chat/completions (PAYG, stream) tok 18460 → 1374 saved 17086 (cache_hit=100%) /v1/responses HTTP (PAYG, non-stream) bytes 50138 → 597 saved 18391 /v1/responses WS (frame 1) bytes 46429 → 488 saved 16791 /v1/responses WS (frame 2 multi) bytes 46429 → 488 saved 16791 /v1/responses WS (response.cancel) passthrough untouched Tests: cargo workspace + pytest (4846 pass, 0 fail), make ci-precheck passed, two E2E scripts (multi-turn HTTP, WS fake-upstream) all green. |
||
|
chopratejas
|
5b38cbf8a7 |
fix(transforms): F2.2 c2/3 — wire toin_read_only gate + extend policy_selected log
Wires the F2.2 ``toin_read_only`` field through the only consumer where it's load-bearing (TOIN write surface) and extends the proxy's structured ``policy_selected`` log event with all three F2.2 fields so the bake dashboard has per-mode observability. Wiring (gates only TOIN writes — compression itself still runs): - headroom/transforms/smart_crusher.py: capture kwargs["compression_policy"] onto self._runtime_compression_policy at the start of apply(). _record_to_toin returns early when the policy says toin_read_only=True. Direct crush() / crush_array_json() callers don't go through apply() and keep pre-F2.2 write-enabled behaviour (no auth context for non-proxy callers). - headroom/transforms/content_router.py: same one-liner in apply(), same gate in _record_to_toin. Mirrors the existing _runtime_target_ratio / _runtime_kompress_model pattern. Telemetry: - crates/headroom-proxy/src/proxy.rs: extend the policy_selected structured log with volatile_token_threshold, max_lossy_ratio, and toin_read_only. F2.2 bake telemetry can now observe all five fields on every request — load-bearing for the F2.2-followup tune decision since volatile_token_threshold and max_lossy_ratio are plumbed-but- unconsumed today and the log is the only signal that the values are flowing correctly. Plumbed-but-unconsumed (deliberate; flagged in PR body): - volatile_token_threshold — the volatile detector in cache_aligner.py is shape-based, not token-count-based; wiring it forces a detector refactor outside F2.2 scope. - max_lossy_ratio — distinct from the caller-driven target_ratio kwarg in content_router.py; gating lossy paths on a policy cap is F2.2- followup once telemetry decides whether to gate or just observe. Tests (tests/test_compression_policy_toin_gate.py): - 7 tests covering the gate. SmartCrusher tests skip when the headroom._core Rust wheel isn't installed (matches the existing test_smart_crusher_rust_parity.py pattern); the 3 ContentRouter tests exercise the gate without the Rust dependency. CI's ci-precheck-python target runs scripts/build_rust_extension.sh before pytest so all 7 will run in the gate. Refs: F2.1 (#400) |
||
|
chopratejas
|
797dc63da7 |
fix(core): F2.2 c1/3 — extend CompressionPolicy with three per-mode tuning fields
Adds three per-mode tuning fields to the F2.1 CompressionPolicy struct on both sides of the parity bridge: - volatile_token_threshold (u32 / int) — per-mode threshold below which content is treated as cache-stable. PAYG=128 (relaxed), Subscription=32 (strict). Plumbed but unconsumed in F2.2 — the volatile detector in cache_aligner.py is shape-based; wiring it is a follow-up. - max_lossy_ratio (f32 / float, [0.0, 1.0]) — per-mode upper bound on lossy compression aggressiveness. PAYG=0.45, Subscription=0.25. Plumbed but unconsumed in F2.2 — distinct from the caller-driven target_ratio kwarg in ContentRouter. - toin_read_only (bool) — TOIN learning gate. True = serve cached patterns but never write new observations from this request. PAYG/OAuth=false (network effect feeds on aggressive traffic), Subscription=true (consistency over learning). OAuth stays identical to PAYG across all five fields; the canary parity test (oauth_matches_payg_today) covers the full struct so a future divergence on any field trips the assertion just as loudly as a flag flip. Per-mode defaults are CONSERVATIVE pending F2.1 bake telemetry; F2.2- followup will tune. Per the realignment build constraints, the configuration IS the per-mode default — no separate env var per field. What's NOT in this commit: - TOIN gate wiring (next commit, c2/3) - policy_selected log extension (next commit, c2/3) Tests: - Rust: 6 unit tests in compression_policy::tests (3 new). OAuth=PAYG canary now compares ALL fields. - Python: 10 tests in tests/test_compression_policy.py. Hard-coded expected_fields set in TestRustParityFieldMap extended. Refs: F2.1 (#400) |
||
|
chopratejas
|
c48735d029 |
fix(core): expose compress_openai_responses_live_zone via PyO3 (hot-fix c1/2)
PR-C5 (May 3) retired the Python `/v1/responses` compression pipeline with the comment "Rust handles item-aware compression natively" — but the standalone `crates/headroom-proxy` binary that was supposed to do that compression is not deployed by the CLI today (`headroom proxy` and `headroom wrap codex` both run only the Python proxy via uvicorn). Result: every `/v1/responses` request since v0.20.16 has been forwarded uncompressed. Codex CLI is the flagship consumer of this endpoint; this is the regression users have been reporting. Closes Bug 1 of the Codex regression by exposing the existing `headroom_core::transforms::compress_openai_responses_live_zone` as a PyO3 binding so the Python proxy can call the live-zone dispatcher in-process. The `headroom._core` extension is already loaded at proxy startup (PR-A0 verifies), so adding one more callable is mechanical. Why PyO3 inline (Layer 1) vs originally-intended two-process chain (Layer 2): the inline call requires zero deployment changes — the wheel already ships `headroom._core`. Layer 2 (build + ship the standalone `headroom-proxy` binary, teach CLI to spawn both processes) is the right long-term move; Layer 1 restores v0.5.21 functional behaviour today. # Returns `(body, modified)`. On change → `(new_body_bytes, True)`; on passthrough → `(input_bytes, False)`. # Failure mode Never raises. The dispatcher's `LiveZoneError` cases (body not JSON, no input array) are passthrough conditions matching the Rust proxy's `compress_openai_responses_request` contract. # Tests 14 new tests in `tests/test_responses_pyo3_compression.py`: binding exposed, passthrough cases, every F1 AuthMode variant, empty-model default, no-raise on garbage bytes. |
||
|
chopratejas
|
0546795547 |
fix(proxy): rustfmt drift in live_zone_anthropic imports (F2.1 c2 followup)
c2 (
|
||
|
chopratejas
|
0fef428f1f |
fix(proxy): wire CompressionPolicy through handlers + flip default enabled (F2.1 c5/5)
Final commit of F2.1. Flips the CliArgs default for the auth-mode policy enforcement flag from Disabled to Enabled (matching the Rust + Python defaults), and wires resolve_policy() through the Python Anthropic and OpenAI chat handlers so the live TransformPipeline sees the correct CompressionPolicy on every request. Changes: - crates/headroom-proxy/src/config.rs: CliArgs default flips to AuthModePolicyEnforcement::Enabled. Config::for_test stays Disabled so the existing test corpus is unaffected. - headroom/proxy/handlers/anthropic.py: resolve_policy() called once per request just before pipeline.apply(); compression_policy= passed through to TransformPipeline. - headroom/proxy/handlers/openai.py: same pattern in both branches of the chat completion path. - headroom/transforms/compression_policy.py: added is_enforcement_enabled() reading HEADROOM_PROXY_AUTH_MODE_POLICY_ENFORCEMENT (matching Rust) and resolve_policy() — the single public entry point handlers call. Subscription-classified requests now skip CacheAligner, addressing the cache-instability complaints in #327 / #388. PAYG and OAuth remain on the aggressive path until F2.2 telemetry says otherwise. Refs: F2.1 |
||
|
chopratejas
|
027b203c23 |
fix(proxy): add auth_mode_policy_enforcement feature flag (F2.1 c3/6)
Phase F2.1, commit 3 of 6. Behind a default-disabled gate, no behaviour change until commit 6 flips the default. What lands: - New `AuthModePolicyEnforcement` clap-friendly enum (`Enabled`/`Disabled`) in `config.rs`. Pattern follows `CacheControlAutoFrozen` and `StripInternalHeaders`: ValueEnum derive, snake_case rename, env var `HEADROOM_PROXY_AUTH_MODE_POLICY_ENFORCEMENT`. - New `Config::auth_mode_policy_enforcement` field, wired through `from_cli` and `for_test`. `for_test` defaults to `Disabled` so every existing test stays green without per-test opt-out (F2.1's own integration tests opt-IN per case). - Proxy entry in `proxy.rs::proxy_request` reads the flag and gates the policy derivation: `Enabled` -> `CompressionPolicy::for_mode(auth_mode)` (the real per-mode value); `Disabled` -> forces `CompressionPolicy::for_mode(AuthMode::Payg)`. Either way the policy is stored in extensions; c4/6's dispatcher gate reads from there without re-checking the flag. - The `policy_selected` debug log now also emits the `enforcement` field so dashboards can split "policy is PAYG because mode is PAYG" from "policy is PAYG because the flag is off." Why a flag rather than landing the behaviour change directly: F2.1 ships in 6 commits. c1-c5 are wiring + Python parity; c6 is the single commit that flips behaviour for default users. Reviewers can land c1-c5 freely without worrying about subscription users seeing the new path before we have telemetry to validate it. An operator in a dogfood env can opt in early via the env var to generate that telemetry. Rollback story: flip env var back to `disabled` (instant if hot- reload available; else redeploy). c6/6 is the only commit that needs `git revert` to roll back if the default flip surfaces a regression. Note on conventional-commit prefix: this is a Rust-migration internal-phase commit; using `fix:` rather than `feat:` so semantic-release does not bump the package minor version on the phase plumbing. The user-visible behaviour change in c6/6 is the appropriate place for `feat:` if anywhere — and even there, arguably still `fix(proxy):` because the change addresses an existing user complaint (#327/#388 cache instability) rather than adding a brand-new capability. Verified: `cargo check -p headroom-proxy` clean. No tests modified because the default-disabled path is identical to current main behaviour - every existing test continues to assert what it asserted before. c4/6 adds the integration tests that exercise the enforcement-on path. |
||
|
chopratejas
|
948c8f2069 |
fix(proxy): plumb CompressionPolicy through proxy + dispatchers (F2.1 c2/6)
Phase F2.1, commit 2 of 6. No behaviour change — wiring only. Three things land: 1. **Proxy entry derives the policy alongside auth_mode** (proxy.rs). Both go into `req.extensions_mut()` so downstream stages can read either without re-classifying. New structured log event `policy_selected` fires once per request with auth_mode + live_zone_only + cache_aligner_enabled — gives F2.2 the bake-time data it'll need to tune. Pure addition; no existing log was removed. 2. **OpenAI live-zone dispatchers stop hard-coding `AuthMode::Payg`.** `compress_openai_chat_request` and `compress_openai_responses_request` already received `auth_mode` from the proxy caller (F1's plumbing), but the dispatcher invocation hard-coded `AuthMode::Payg` and ignored the parameter — c.f. the `_auth_mode` underscore-prefixed identifier in `compress_openai_chat_live_zone` upstream. Now the classified mode is forwarded via `auth_mode.into()` (uses the `From<>` impl added in c1/6 to bridge the two `AuthMode` enums). 3. **Anthropic live-zone dispatcher gets the same fix.** `compress_anthropic_request` was the closest to right — it threaded auth_mode all the way to `compress_anthropic_live_zone` already — but the inner call still had `AuthMode::Payg` hard-coded. Now it uses `auth_mode.into()` for symmetry and so the F2.1 dispatcher- level gate (added in c4/6) reads the real mode. The dispatcher is unchanged in F2.1: it still runs the same compression for every mode. The point of c2/6 is that *if* a future commit (c4/6 in this PR, or anything post-F2.1) gates on the mode-aware policy, the wiring is already in place. PAYG behaviour is byte-for-byte identical because every mode currently dispatches identically. Two tiny cleanups: drop the `AuthMode` import from the three live-zone dispatcher files now that the value flows in via `.into()` (the `RequestAuthMode` alias remains, since it's still in the function signatures). Behaviour-change risk in c2/6: zero. Verified by running the full `cargo test -p headroom-proxy` suite; existing tests pass without modification because they're already shape-compatible (tests pass `AuthMode::Payg` directly into the inner dispatcher; this commit only changes how the *outer* compress_* fns invoke that dispatcher). |
||
|
chopratejas
|
8376630062 |
fix(core): introduce CompressionPolicy struct + auth_mode mapping (F2.1 c1/6)
Phase F2.1, commit 1 of 6. Pure additive change — no call site reads
this struct yet, no behavior change in main.
What lands:
- New `headroom_core::compression_policy::CompressionPolicy` struct
with `live_zone_only: bool` and `cache_aligner_enabled: bool`.
- `CompressionPolicy::for_mode(AuthMode)` returns the F2.1 per-mode
values: PAYG and OAuth aggressive (live-zone-not-only,
cache-aligner on); Subscription live-zone-only with cache aligner
disabled. The OAuth=PAYG match is intentional in F2.1 — F2.2 will
diverge once telemetry collected during F2.1's main bake shows
what OAuth users need.
- `From<crate::auth_mode::AuthMode> for crate::transforms::live_zone::AuthMode`
to bridge the two `AuthMode` enums that exist in headroom-core
(one in F1's classifier, one in the live-zone dispatcher; differ
only by the dispatcher's `Unknown` sentinel for stored
recommendation rows). Keeps the cross-module call sites clean —
`mode.into()` instead of a hand-written match every time.
Per-mode values (F2.1 only — F2.2 will tune):
| Mode | live_zone_only | cache_aligner_enabled |
|--------------|----------------|-----------------------|
| Payg | false | true |
| OAuth | false | true (= PAYG) |
| Subscription | true | false |
3 unit tests covering each variant. `oauth_matches_payg_today`
fails on purpose if F2.2 (or any well-meaning future change)
diverges OAuth from PAYG without updating the test — the divergence
must be deliberate.
Why two flags and not more in F2.1: closing #327/#388 cache-
instability complaints requires exactly these two gates. Anything
more is F2.2 tuning that benefits from real bake-time telemetry,
and a smaller F2.1 lands faster + fewer regression sites.
Why a struct instead of `match auth_mode { ... }` everywhere:
Phase E already has two PAYG-only gates (cache_control auto-
placement, prompt_cache_key injection). Adding two more without
centralisation means four duplicated match arms. The struct
collapses them and gives F2.2 one place to add fields.
Next commit (c2/6) plumbs the policy through the Rust proxy +
plugs the `auth_mode` parameter the OpenAI dispatchers are still
missing today.
|
||
|
chopratejas
|
9112fed937 |
fix: PR-E2 recursive JSON Schema key sort (Phase E)
Recursively sort JSON Schema object keys inside each tool's schema so cache hits no longer depend on SDK-side serializer key-emission order (some sort, some preserve insertion, some hash-randomize). Wired into all three live-zone walkers, hooking the per-provider schema location: - Anthropic: `tool["input_schema"]` - OpenAI Chat: `tool["function"]["parameters"]` - OpenAI Responses: `tool["function"]["parameters"]` Same auth-mode gate as PR-E1 (PAYG only). NO marker check — the `cache_control` marker lives on the tool object itself, not inside the schema, so sorting schema keys never moves the marker. PR-E2 therefore runs even on tools that PR-E1 had to skip due to a present marker; the integration test pins this behaviour. Array semantics preserved: `oneOf`, `anyOf`, `allOf`, `prefixItems` and any other ordered JSON Schema array keep customer order; only object keys move. Idempotent — sorting an already-sorted schema yields byte-identical bytes (workspace `preserve_order` feature pins `serde_json::Map` emission to insertion order). Tests: unit tests for nested keys, oneOf preservation, deep nesting, and idempotency; integration tests boot the real proxy and assert PAYG -> sorted at every level, OAuth -> SHA-256 byte- equal, and PAYG-with-marker -> E1 skipped but E2 still runs. |
||
|
chopratejas
|
4a3b76bcc8 |
fix: PR-E1 tool array deterministic sort (Phase E)
Sort `tools[]` alphabetically by name on the way out so cache hits no
longer depend on the customer-side iteration order (commonly hash-
randomized via `set()` / `dict`). Mutates request bytes only when:
1. Auth mode is PAYG (`headroom_core::auth_mode::classify`).
2. No tool already carries a `cache_control` marker (reordering
would shift cache scope and silently void customer intent).
Every gate skip emits a structured `e1_skipped` event with `reason =
auth_mode | marker_present` so dashboards can see policy adoption.
Wired into all three live-zone walkers — Anthropic `/v1/messages`,
OpenAI `/v1/chat/completions`, OpenAI `/v1/responses` — plus the
Bedrock invoke + invoke-streaming entry points. Each passes
`auth_mode` (already pre-classified by Phase F PR-F1 middleware)
into the dispatcher so the gate evaluates without re-classifying.
Sort key uses `tool["name"]` (Anthropic) or `tool["function"]["name"]`
(OpenAI). Unnamed tools (rare; malformed inputs only) fall back to
MD5 of canonical-JSON serialization for a stable in-process key —
collision odds are astronomically small and `Vec::sort_by` is stable.
Tests: unit tests for sort + marker detection + idempotency + the
permutation property; integration tests boot the real proxy in front
of a wiremock upstream and assert PAYG -> sorted, OAuth/Subscription/
marker -> byte-equal passthrough (SHA-256).
|
||
|
chopratejas
|
17d6207bf5 |
fix(crusher): shim __libc_single_threaded for glibc < 2.32 + extend audit
PR #396's X2 dry-run caught a wheel-import failure on the manylinux_2_28 floor matrix entry (both x86_64 and aarch64). Same class as #355: ImportError: ... undefined symbol: __libc_single_threaded `__libc_single_threaded` is a single-byte char added in glibc 2.32. Newer libstdc++ (gcc 11+) reads it inside `__cxa_thread_atexit_impl` to elide locking on the single-threaded fast path. ORT prebuilt static archives compiled with gcc-14.2.1 against glibc-2.38+ headers bake in the reference. Users with glibc < 2.32 hit ImportError on `import headroom._core`. Latent since the ORT artifact bump that started using gcc 14. X1 is the gate that catches it at release time; X2 caught it at PR time — exactly as designed. Fix: 1. glibc_compat.c adds Section B: `char __libc_single_threaded = 0;` Setting to 0 (multi-threaded) is safe; libstdc++ takes the locked slow path. Setting to 1 would race in any multithreaded Rust wheel. 2. build.rs adds `-Wl,-u,__libc_single_threaded` so the shim's archive members are pulled regardless of scan order. 3. audit_wheel_glibc_symbols.py POST_FLOOR_SYMBOLS adds the new symbol — verified locally: the audit now rejects the failing PR #396 wheel with the right message. |
||
|
Tejas Chopra
|
2fc73d0f73
|
Merge pull request #380 from chopratejas/realign-E4-openai-prompt-cache-key
fix: PR-E4 OpenAI prompt_cache_key auto-injection (Phase E) |
||
|
chopratejas
|
820e66cae6 |
fix(ci): force-link glibc shim with -Wl,-u so aarch64 wheel includes it
PR #385's shim works on x86_64 wheel build but FAILS audit on aarch64 in run 25358313722: FAIL: headroom_ai-0.20.27-cp310-cp310-manylinux_2_28_aarch64.whl references symbols above its glibc floor: __isoc23_strtoll (no version tag, introduced in glibc 2.38) Diagnosis: cargo's link order on aarch64 happens to place our shim's static archive BEFORE the ORT prebuilt archives. When the linker scans our archive, no UND `__isoc23_*` exists yet (ORT hasn't been scanned), so our shim's `.o` is dropped (no symbol to satisfy). ORT scans next, registers UND, but our archive isn't rescanned. Result: `_core.so` still has UND `__isoc23_*` symbols and the audit rightly rejects the wheel. On x86_64 the order happened to be the opposite (ORT first → UND registered → our archive scans next → satisfies → pulled in). Order is implementation-defined and clearly arch-dependent. Fix: emit `cargo:rustc-link-arg=-Wl,-u,<sym>` for each `__isoc23_*` symbol in `build.rs`. `-u <sym>` (a.k.a. `--undefined`) tells the linker to treat the symbol as undefined at the START of linking, which forces any archive defining it to be scanned and its members pulled in regardless of relative archive order. Shim is now uniformly linked on both x86_64 and aarch64. Documented inline in `build.rs`. Standard workaround for the static-library-link-order problem when the consumer scans after the provider. |
||
|
chopratejas
|
6b15acc3b5 |
fix(ci): glibc shim — drop alias attribute, forward-declare strtol
PR #384 introduced glibc_compat.c using __attribute__((weak, alias("strtol"))) which fails to compile because GCC requires the alias TARGET to live in the same translation unit. strtol is in libc.so.6, not the .c file. Result: clippy fails on every Linux CI job for every PR + main: glibc_compat.c: error: '__isoc23_strtol' aliased to undefined symbol 'strtol' Two-line architectural change: (1) drop the alias attribute, give each __isoc23_* function a plain body that calls the older strtol family; (2) forward-declare the older prototypes ourselves instead of #include <stdlib.h>, otherwise GCC's __REDIRECT_NTH(strtol -> __isoc23_strtol) would silently rewrite our delegation into an infinite recursion. Symbol-resolution semantics unchanged: on glibc 2.38+, libc's strong __isoc23_strtoll preempts ours via global-scope-first lookup; on glibc < 2.38, ours wins. Either way the symbol resolves and import succeeds. Both traps documented inline in glibc_compat.c so a future refactor doesn't reintroduce them. PR #384's commit message overstated the validation: I tested the audit script against the broken wheel but did NOT compile the shim itself before merging. Adding the X1 smoke-import gate (separate PR) is what would have caught this. |
||
|
chopratejas
|
e2146724af |
fix: ship glibc 2.38 compat shim + wheel symbol audit (closes #355)
Issue #355: published headroom_ai-*-manylinux_2_28_*.whl fails to import on Ubuntu 22.04, Debian 11/12, Conda envs with libc < 2.38: 'ImportError: undefined symbol: __isoc23_strtoll'. Root cause: ORT prebuilt artifacts (downloaded via fastembed's ort-download-binaries-rustls-tls feature) are compiled with gcc-14.2.1 on a glibc-2.38+ host and reference __isoc23_strtoll. Our manylinux build host has glibc 2.38 so the link succeeds; end users with older glibc don't. Two-part fix: (1) crates/headroom-py/glibc_compat.c provides weak-alias definitions for __isoc23_strtol/strtoll/strtoul/strtoull delegating to the older strtol* family, compiled by build.rs on Linux/glibc only. The dynamic linker prefers glibc's strong symbol when present (>= 2.38) and falls back to ours when not (< 2.38). (2) scripts/audit_wheel_glibc_symbols.py is a release.yml gate that runs objdump -T on every Linux wheel and rejects any UND symbol whose required glibc version exceeds the wheel's manylinux floor. Validated: the audit correctly rejects the actually-broken v0.20.26 wheel with a precise diagnostic. The shim itself is a tiny static link with zero runtime cost. Regression tests in tests/test_release_workflows.py pin the shim's load-bearing pieces (.c file, build.rs trigger, [build-dependencies] cc dep) and the audit invocation in release.yml. Future drift fails at PR time, not in the next release. This is the same bug class as PR #371 (rustls-everywhere). Each instance gets fixed; the audit gate now catches the *class* — any future static linkage that introduces a post-floor symbol gets blocked before publish. |
||
|
chopratejas
|
573543fce7 |
fix: PR-E4 OpenAI prompt_cache_key auto-injection (Phase E)
OpenAI exposes `prompt_cache_key` to pin prefix-cache lookups to a tenant-stable identity (preventing org-wide cache collisions). Most clients don't set it. This PR auto-derives one from the request's structural prefix `(model, system, tools)` and injects it on PAYG OpenAI requests where the customer has not provided their own value. Universal safety contract: - Auth-mode gate: only AuthMode::Payg bodies are mutated. OAuth and Subscription requests pass through byte-equal (preserves Phase A passthrough invariant). Both gates emit `e4_skipped` events. - Customer-set values win: `prompt_cache_key` already present → skip injection. Empty strings count as absent. - Idempotent: same `(model, system, tools)` always derives the same key, so re-running yields identical bytes. Key derivation: `hex(sha256(model || sha256(system) || sha256(tools)))[..32]` — 128 bits of collision resistance, 32 hex chars on the wire. User/assistant message content is deliberately excluded (they vary per turn; including would defeat caching). Observability: every skip emits `e4_skipped` with a stable reason (`auth_mode` / `key_present` / `not_an_object`); every successful injection emits `e4_applied` with only the first 8 hex chars of the key (full key is identifying material — never logged). Hook point: `forward_http` in `crates/headroom-proxy/src/proxy.rs`, between the live-zone dispatcher's body decision and the upstream forward. Auth-mode is already classified at request entry. Affects pre-existing dispatcher byte-fidelity tests (chat completions, responses, responses streaming) — they previously asserted byte-equality with no auth header (default PAYG). Updated those tests to send an OAuth bearer so they keep their byte-equality intent independent of E4. The E4 byte-mutation behaviour has its own test matrix in `integration_e4_openai_cache_key.rs`. Files added: - `crates/headroom-proxy/src/cache_stabilization/openai_cache_key.rs` - `crates/headroom-proxy/tests/integration_e4_openai_cache_key.rs` Files modified: - `crates/headroom-proxy/src/cache_stabilization/mod.rs` — `pub mod openai_cache_key;` (only shared file with parallel E1/E2/E3/E6 PRs) - `crates/headroom-proxy/src/proxy.rs` — call site + helper - `crates/headroom-proxy/Cargo.toml` — promote sha2 to runtime dep - 3 integration test files — auth-mode plumbing for byte-equality invariants |
||
|
chopratejas
|
8672d5c326 |
fix: PR-E3 Anthropic cache_control auto-placement (Phase E)
Auto-place a single ephemeral cache_control marker on the last tool
definition for PAYG-classified Anthropic requests when the customer
has not placed any markers. Hand-rolled SDK callers and smaller
agents (Aider/Continue/curl) get prompt-cache hits without learning
Anthropic's marker API.
Safety contract:
1. Auth-mode (caller-side, F1 classify): PAYG only. OAuth and subscription requests pass through byte-equal — mutating their bytes risks looking like cache-evasion to upstream.
2. Customer-placement-wins: walks system (array form), messages[].content (array form), and tools[] top-level. Any pre-existing marker -> skip with reason=marker_present.
3. Idempotency: re-running on a body that already has our marker falls into gate (2).
First-ship policy: place ONE marker on the last tool. The system/message-history/4th slots are documented but require production telemetry to enable.
Bedrock invoke + invoke-streaming hard-code OAuth so AWS SigV4-signed requests never get auto-placed (Bedrock is an IAM channel, not PAYG).
Observability: tracing::info! event=e3_applied / event=e3_skipped (reason in {auth_mode, marker_present}) so dashboards can confirm the gates fire as designed.
Files added: cache_stabilization/anthropic_cache_control.rs (module + 15 unit tests); tests/integration_e3_anthropic_cache_control.rs (5 integration tests covering all three gates).
Files modified: cache_stabilization/mod.rs; compression/live_zone_anthropic.rs (new auth_mode parameter on compress_anthropic_request); proxy.rs; bedrock/invoke.rs; bedrock/invoke_streaming.rs.
|
||
|
chopratejas
|
c10a2195af |
fix(proxy): PR-D4 native Vertex publisher path + ADC bearer auth
Adds a Rust-native Vertex AI publisher route ahead of the LiteLLM
Python converter (which dropped `thinking`, `redacted_thinking`,
`document`, `image`, `server_tool_use`, `mcp_tool_use` block kinds —
the P4-37 / P4-38 bug). After this PR the Vertex `:rawPredict` and
`:streamRawPredict` calls survive byte-equal upstream and benefit
from the live-zone Anthropic dispatcher (PR-B-series) running over
the body — same behaviour as `/v1/messages`.
New module `crates/headroom-proxy/src/vertex/`:
- `mod.rs` — single dispatch handler at the
`/v1beta1/.../models/:model_action` route. Splits the trailing
`:<verb>` segment with `str::rsplit_once(':')` (no regex) and
flips an `attach_sse_tee` flag to dispatch to the streaming or
non-streaming arm. Both verbs share one axum route shape because
matchit can't distinguish two patterns that overlap on a
parameter.
- `envelope.rs` — `VertexEnvelope` parser. Confirms
`anthropic_version` present + `model` field absent (the two
fingerprints of the Vertex envelope vs `/v1/messages`).
- `adc.rs` — `TokenSource` trait + `GcpAdcTokenSource` (production,
`gcp_auth` 0.12) + `StaticTokenSource` (tests). Caches tokens
with a 60s refresh-ahead-of-expiry window. Emits structured
`event = "vertex_adc_token_refreshed"` per refresh.
- `raw_predict.rs` — POST handler + shared `forward_vertex_request`.
Buffers body, parses envelope, runs live-zone Anthropic
compression, fetches ADC bearer, attaches
`Authorization: Bearer <token>` (overwrites client-supplied
Authorization header), forwards. SSE telemetry tee for the
streaming verb reuses PR-C1's `AnthropicStreamState` directly
(Vertex streams plain SSE, unlike Bedrock's binary EventStream).
- `stream_raw_predict.rs` — module-level docs + alias to the
shared dispatcher (the streaming-vs-non-streaming difference is
one boolean flag inside the shared forwarder).
Modifications:
- `proxy.rs::build_app` — registers the single Vertex route.
- `proxy.rs::AppState` — new `vertex_token_source: Arc<dyn TokenSource>`
field. Production constructs `GcpAdcTokenSource` lazily (no GCP
call until first `bearer()`); tests inject `StaticTokenSource`
via the new `AppState::with_token_source` helper.
- `config.rs` — adds `--vertex-region` / `HEADROOM_PROXY_VERTEX_REGION`
(default `us-central1`, observability tag only — the upstream URL
is `--upstream`) and `--vertex-adc-scope` /
`HEADROOM_PROXY_VERTEX_ADC_SCOPE` (default `cloud-platform`).
- `Cargo.toml` (workspace + proxy) — adds `gcp_auth = "0.12"` and
`async-trait = "0.1"`.
- `tests/common/mod.rs` — `start_proxy_with_state` accepts both
config + state customizers; `install_static_token_source` helper
for tests.
`crates/headroom-proxy/tests/integration_vertex_raw_predict.rs` —
all five tests pass:
1. `native_envelope_round_trip_byte_equal` — Vertex-shape body
(with `anthropic_version`, no `model`) round-trips SHA-256
byte-equal upstream.
2. `adc_bearer_token_signed_correctly` — `Authorization: Bearer
<static-test-token>` reaches upstream verbatim and OVERWRITES a
client-supplied Authorization header.
3. `thinking_block_preserved` — request with `thinking` (incl.
signature) + `redacted_thinking` (incl. opaque `data`) blocks
round-trips byte-equal even with `LiveZone` compression mode
enabled. This is the P4-37 / P4-38 teeth.
4. `stream_raw_predict_sse_handled` — `:streamRawPredict` proxies
an Anthropic SSE response (full `message_start` →
`content_block_delta` → `message_stop` sequence) back to the
client without corruption; SSE content-type preserved end-to-end;
bearer attached.
5. (bonus, no-silent-fallback contract)
`adc_failure_returns_5xx_no_silent_forward` — when the token
source returns `Err`, the proxy returns 5xx and never reaches
upstream. Verifies the `event = "vertex_adc_fetch_failed"`
error path.
Workspace: `cargo test --workspace` green; `cargo clippy --workspace
-- -D warnings` clean; `make ci-precheck` passes.
- No silent fallbacks: ADC failure → structured 5xx, never an
unauthenticated forward.
- No hardcodes: every knob (region, ADC scope, upstream URL) is
CLI-flag + env-var configurable.
- No regexes: axum path parameters + `str::rsplit_once` only.
- Comprehensive structured logs: `event` field on every decision
point — `vertex_envelope_parsed`, `vertex_envelope_invalid`,
`vertex_compression_skipped`, `vertex_compression_applied`,
`vertex_adc_token_refreshed`, `vertex_adc_fetch_failed`,
`vertex_streaming_pipeline_active`, `vertex_sse_stream_closed`,
`vertex_forwarded`, `vertex_unknown_verb`, etc.
- Performant: no body clone; ADC token cached + refreshed
ahead-of-expiry, not fetched per request.
- Comprehensive tests: realistic Anthropic block content
(signature payload, redacted_thinking opaque blob) in
`thinking_block_preserved`.
The local `gcloud auth application-default print-access-token`
returns no credentials, so manual validation against a real Vertex
endpoint is not possible in this PR. Follow-up: the user runs
`gcloud auth application-default login` once and exercises a live
Vertex request — should be a no-code-change check.
PR-D1 (Bedrock native) is running concurrently and will land its
own envelope module at `crates/headroom-proxy/src/bedrock/envelope.rs`.
The two envelope modules are intentionally siblings (not a shared
trait) — the shapes differ (Bedrock has a different
`anthropic_version` value, no `model` field, AWS SigV4 instead of
GCP ADC), and a premature shared abstraction would obscure the
provider-specific contracts. Whichever PR merges second rebases
without conflict.
Retires P4-38 (and the Vertex parts of P4-39); marketplace BYOC
pitch (per project memory) gets one more native provider.
|