mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
66 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
dependabot[bot]
|
322425c43b
|
deps: bump sha2 from 0.10.9 to 0.11.0 (#2288)
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.10.9 to 0.11.0. <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
5731be7e68
|
deps: bump axum from 0.7.9 to 0.8.9 (#2966)
Bumps [axum](https://github.com/tokio-rs/axum) from 0.7.9 to 0.8.9. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/axum/releases">axum's releases</a>.</em></p> <blockquote> <h2>axum-v0.8.9</h2> <ul> <li><strong>added:</strong> <code>WebSocketUpgrade::{requested_protocols, set_selected_protocol}</code> for more flexible subprotocol selection (<a href="https://redirect.github.com/tokio-rs/axum/issues/3597">#3597</a>)</li> <li><strong>changed:</strong> Update minimum rust version to 1.80 (<a href="https://redirect.github.com/tokio-rs/axum/issues/3620">#3620</a>)</li> <li><strong>fixed:</strong> Set connect endpoint on correct field in MethodRouter (<a href="https://redirect.github.com/tokio-rs/axum/issues/3656">#3656</a>)</li> <li><strong>fixed:</strong> Return specific error message when multipart body limit is exceeded (<a href="https://redirect.github.com/tokio-rs/axum/issues/3611">#3611</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3597">#3597</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3597">tokio-rs/axum#3597</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3620">#3620</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3620">tokio-rs/axum#3620</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3656">#3656</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3656">tokio-rs/axum#3656</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3611">#3611</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3611">tokio-rs/axum#3611</a></p> <h2>axum v0.8.8</h2> <ul> <li>Clarify documentation for <code>Router::route_layer</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3567">#3567</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3567">#3567</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3567">tokio-rs/axum#3567</a></p> <h2>axum v0.8.7</h2> <ul> <li>Relax implicit <code>Send</code> / <code>Sync</code> bounds on <code>RouterAsService</code>, <code>RouterIntoService</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3555">#3555</a>)</li> <li>Make it easier to visually scan for default features (<a href="https://redirect.github.com/tokio-rs/axum/issues/3550">#3550</a>)</li> <li>Fix some documentation typos</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3550">#3550</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3550">tokio-rs/axum#3550</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3555">#3555</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3555">tokio-rs/axum#3555</a></p> <h2>axum v0.8.5</h2> <ul> <li><strong>fixed:</strong> Reject JSON request bodies with trailing characters after the JSON document (<a href="https://redirect.github.com/tokio-rs/axum/issues/3453">#3453</a>)</li> <li><strong>added:</strong> Implement <code>OptionalFromRequest</code> for <code>Multipart</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3220">#3220</a>)</li> <li><strong>added:</strong> Getter methods <code>Location::{status_code, location}</code></li> <li><strong>added:</strong> Support for writing arbitrary binary data into server-sent events (<a href="https://redirect.github.com/tokio-rs/axum/issues/3425">#3425</a>)]</li> <li><strong>added:</strong> <code>middleware::ResponseAxumBodyLayer</code> for mapping response body to <code>axum::body::Body</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3469">#3469</a>)</li> <li><strong>added:</strong> <code>impl FusedStream for WebSocket</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3443">#3443</a>)</li> <li><strong>changed:</strong> The <code>sse</code> module and <code>Sse</code> type no longer depend on the <code>tokio</code> feature (<a href="https://redirect.github.com/tokio-rs/axum/issues/3154">#3154</a>)</li> <li><strong>changed:</strong> If the location given to one of <code>Redirect</code>s constructors is not a valid header value, instead of panicking on construction, the <code>IntoResponse</code> impl now returns an HTTP 500, just like <code>Json</code> does when serialization fails (<a href="https://redirect.github.com/tokio-rs/axum/issues/3377">#3377</a>)</li> <li><strong>changed:</strong> Update minimum rust version to 1.78 (<a href="https://redirect.github.com/tokio-rs/axum/issues/3412">#3412</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3154">#3154</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3154">tokio-rs/axum#3154</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3220">#3220</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3220">tokio-rs/axum#3220</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3377">#3377</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3377">tokio-rs/axum#3377</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3412">#3412</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3412">tokio-rs/axum#3412</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3425">#3425</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3425">tokio-rs/axum#3425</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3443">#3443</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3443">tokio-rs/axum#3443</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3453">#3453</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3453">tokio-rs/axum#3453</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3469">#3469</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3469">tokio-rs/axum#3469</a></p> <h2>axum v0.8.4</h2> <ul> <li><strong>added:</strong> <code>Router::reset_fallback</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3320">#3320</a>)</li> <li><strong>added:</strong> <code>WebSocketUpgrade::selected_protocol</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3248">#3248</a>)</li> <li><strong>fixed:</strong> Panic location for overlapping method routes (<a href="https://redirect.github.com/tokio-rs/axum/issues/3319">#3319</a>)</li> <li><strong>fixed:</strong> Don't leak a tokio task when using <code>serve</code> without graceful shutdown (<a href="https://redirect.github.com/tokio-rs/axum/issues/3129">#3129</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
bbe901319d
|
deps: bump tokio-tungstenite from 0.24.0 to 0.30.0 (#2967)
Bumps [tokio-tungstenite](https://github.com/snapview/tokio-tungstenite) from 0.24.0 to 0.30.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/snapview/tokio-tungstenite/blob/master/CHANGELOG.md">tokio-tungstenite's changelog</a>.</em></p> <blockquote> <h1>0.30.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.30.0</code>. See <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md"><code>tungstenite</code> release</a>.</li> </ul> <h1>0.29.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.29.0</code>. See <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md"><code>tungstenite</code> release</a>.</li> </ul> <h1>0.28.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.28.0</code>. See <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md"><code>tungstenite</code> release</a>.</li> </ul> <h1>0.27.0</h1> <ul> <li>See <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md#0270">performance updates in <code>tungstenite-rs</code></a>.</li> </ul> <h1>0.26.2</h1> <ul> <li>Update <code>tungstenite</code>, see <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md#0262">changes here</a>.</li> </ul> <h1>0.26.1</h1> <ul> <li>Update <code>tungstenite</code> to address an issue that might cause UB in certain cases.</li> </ul> <h1>0.26.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.26.0</code> (<a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md#0260">breaking changes</a>).</li> </ul> <h1>0.25.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.25.0</code> (<a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md#0250">important updates!</a>).</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
JD Davis
|
3077ac81e8
|
feat: add deterministic runtime rollout controls (#1490)
## Description Establish one centrally resolved, observable, deterministic, versioned runtime rollout-control mechanism for Headroom. Runtime rollout controls which behaviors an already-built artifact may expose; it does not select or qualify a Headroom release/version. ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Bug fix (non-breaking change that fixes rollout enforcement regressions) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - Added `RolloutChannel`, `HEADROOM_ROLLOUT_CHANNEL`, `--rollout-channel`, and a versioned immutable `RolloutSnapshot` shared by Python configuration boundaries. - Added schema/policy versions, canonical registry and snapshot SHA-256 identities, per-feature decision reasons, disable precedence, unsafe qualification poisoning, strict CLI validation, and fail-closed environment handling. - Added `headroom rollout status --json`, Python `/stats.rollout`, and Rust `/rollout/status` runtime provenance. - Added equivalent Rust snapshot semantics and shared Python/Rust policy vectors while retaining language-specific feature registries. - Enforced rollout policy at alternate Python server composition roots so `HEADROOM_READ_MATURATION=1` cannot bypass its beta gate. - Preserved typed rollout snapshots across multi-worker serialization with schema, policy, registry, snapshot-digest, type, and feature-name validation. - Made loopback runtime output-shaper updates replace the immutable snapshot atomically for request readers, retain explicit request/disable provenance, preserve channel and kill-switch precedence, invalidate cached stats, and return the effective rollout decision. - Made `headroom learn --verbosity --apply` report a channel-blocked update instead of claiming the shaper is live. - Made explicit CLI feature flags fail loudly when their current channel blocks them. - Made persistent interceptor installation select canary automatically, or reject an explicitly insufficient channel unless the break-glass override is set. - Updated architecture, proxy, rollout, learn, and output-shaper documentation with required channels and hot-reload semantics. ## Testing - [x] Unit tests pass - [x] Linting passes (`ruff check .` and `ruff format --check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New regression tests added for every corrected behavior - [x] Rust tests and production-target Clippy pass - [x] Documentation build passes ### Test Output ```text Focused rollout coverage suite 57 passed; headroom.rollout + rollout CLI: 98% coverage Affected proxy/rollout/transform/governance suites 222 passed; 0 failed Final changed regression suites 100 passed; 0 failed Cross-module hot-reload isolation regression 6 passed; 0 failed cargo test -p headroom-core -p headroom-proxy --quiet headroom-core: 924 passed; 1 ignored headroom-proxy and integration suites: all passed cargo clippy -p headroom-core -p headroom-proxy --lib --bins -- -D warnings cargo fmt --all -- --check ruff check . ruff format --check . mypy headroom --ignore-missing-imports git diff --check All passed cd docs && npm run build Compiled successfully; 164 static pages generated ``` The unsharded Windows-only CI selection exposed unrelated baseline failures, principally the existing `sqlite:///C:\\...` URL parser producing an invalid `\\C:\\...` path. At commit ` |
||
|
Andrei Boldyrev
|
f6398a6476
|
fix(proxy): port session-sticky beta headers to the Rust proxy (#2381)
## Description The Python proxy protects prompt caches with `SessionBetaTracker` (PR-A6, `headroom/proxy/helpers.py`): interactive clients (Claude Code, Codex CLI) may drop an `anthropic-beta` / `openai-beta` token between turn N and turn N+1 of the same conversation, and since beta headers are part of the bytes that determine the upstream prefix-cache key, the drop rotates the key and the provider re-writes the whole prefix at the customer's cost. The tracker unions the client's tokens with everything previously seen for that `(provider, session)` and forwards the union — a documented operator contract (`docs/configuration.mdx`, "Session Beta Header Tracking"). The Rust proxy has no equivalent, and Phase H (#2258) deletes the tracker together with `helpers.py` and its test file (`tests/test_anthropic_beta_session_sticky.py`). None of the Phase A–G plans port it (Phase F consumes beta headers for auth-mode classification only), so the protection would silently not survive the migration — and the Phase-H gate "Cache-hit-rate parity with direct upstream confirmed" can't catch the loss, because re-injection makes proxied traffic *beat* direct upstream on cache hits; when the mechanism disappears, proxied traffic degrades *to* direct-upstream levels, which that comparison reads as parity. This PR ports the tracker semantics into the Rust proxy so the protection lives in the codebase Phase H keeps. Closes #2380 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) (New Rust functionality, but a parity port of already-shipped, already-documented Python behavior — the PR title uses `fix:` per `REALIGNMENT/INDEX.md`: "Commit prefix: `fix:` for Rust-migration phase commits".) ## Changes Made - **`cache_stabilization/beta_sticky.rs`** — the tracker: bounded LRU (1000 sessions, same sizing rationale and `# Panics` contract as the drift detector's capacity) keyed by `(provider, session)`, storing the per-session ordered token list. Union preserves first-seen order; dedup is case-insensitive with first-seen casing winning; lookups touch recency; overflow evicts the oldest — mirroring the Python tracker. The header-plumbing lives in the module too (`apply_sticky_betas`), so the merge is unit-testable without booting a proxy. - **`proxy.rs` wiring** — on the intercepted POST routes (`/v1/messages`, `/v1/chat/completions`, `/v1/responses`), right after the drift-detector observation, reusing the drift detector's `derive_session_key` output so both cache-stability subsystems agree on conversation identity. - **`config.rs`** — `--beta-header-sticky` / `HEADROOM_PROXY_BETA_HEADER_STICKY` (`enabled` default; `disabled` forwards the client value verbatim and keeps no state), mirroring the `StripInternalHeaders` flag pattern and the existing `HEADROOM_*` → `HEADROOM_PROXY_*` Python→Rust env pairing. Since the merge runs inside the compression interceptor, startup logs a warning when the flag is `enabled` while `--compression` is off, and both the CLI doc and the docs row state the dependency. - **`tests/integration_beta_header_sticky.rs`** — 9 end-to-end tests against a wiremock upstream asserting the headers/bytes the upstream actually receives; 21 unit tests port the behavioral contract from `tests/test_anthropic_beta_session_sticky.py` and cover the header-map plumbing. - **`docs/content/docs/configuration.mdx`** — one row for `HEADROOM_PROXY_BETA_HEADER_STICKY` next to the existing Python/Rust flag pairs. ## Testing - [x] Unit tests pass (`cargo test -p headroom-proxy`; Python side via `make ci-precheck-python` — `pytest` subset, 174 passed) - [x] Linting passes (`cargo clippy --all-targets` — 0 warnings; `cargo fmt --check` clean; Rust-only change, so `ruff`/`mypy` are covered by the untouched-Python `ci-precheck-python` build) - [ ] Type checking passes (`mypy headroom`) — N/A, no Python files touched - [x] New tests added for new functionality - [x] Manual testing performed (RED/GREEN before-and-after runs below) ### Test Output ```text $ cargo test -p headroom-proxy --lib beta_sticky test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 248 filtered out; finished in 0.03s $ cargo test -p headroom-proxy --test integration_beta_header_sticky test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s $ cargo test -p headroom-proxy # full crate: 37 suites, all ok $ cargo clippy -p headroom-proxy --all-targets # 0 warnings $ make ci-precheck-rust ci-precheck-python ci-precheck-commitlint # green ``` ## Real Behavior Proof - Environment: macOS arm64 (Darwin 24.6), `rustc 1.95.0`, real Rust proxy booted on an ephemeral port in front of a wiremock upstream (`tests/common::start_proxy_with`, `compression = true`). - Exact command / steps: two-turn conversation through the proxy — turn 1 `POST /v1/messages` with `anthropic-beta: context-management-2025-06-27,interleaved-thinking-2025-05-14`; turn 2, same conversation, client drops the second token. The wiremock responder captures the headers the upstream actually receives (`cargo test -p headroom-proxy --test integration_beta_header_sticky`). - Observed result: **before** the port (test written first, run against the unmodified proxy) the upstream sees the shrunken token set and the prefix-cache key rotates — ```text assertion `left == right` failed: turn 2 must re-inject the dropped token so the upstream prefix-cache key stays byte-stable left: Some("context-management-2025-06-27") right: Some("context-management-2025-06-27,interleaved-thinking-2025-05-14") ``` **After** the port the same scenario passes: the upstream receives the full union on turn 2, the internal `x-headroom-session-id` never crosses the upstream boundary, and the forwarded body is SHA-256-identical to what the client sent (asserted by `body_bytes_stay_byte_equal_while_header_is_rewritten`). - Not tested: live traffic against a real provider upstream (wiremock only); the WebSocket path and Bedrock/Vertex routes (out of scope — see Additional Notes). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A (proxy behavior; see Real Behavior Proof). ## Additional Notes Design decisions, and where I'd like reviewer judgment: 1. **Applies to all auth modes, like the Python handler.** The Phase-E module doctrine gates *body*-mutating normalizers on PAYG; this mechanism mutates headers only, and the Python source of truth applies it unconditionally — an auth-mode gate here would create a behavioral delta exactly where the PR's purpose is behavior preservation. It's also stealth-consistent by construction: the union only ever contains tokens this client itself sent (Headroom-added tokens are never recorded), `auth_mode.rs`'s own docs name "beta-header drift voids them" as the OAuth cache hazard (stickiness is the anti-drift), and F2's `CompressionPolicy` has no beta field — no gate is structurally expected. I've extended the `cache_stabilization/mod.rs` taxonomy with a third category ("re-echo client-sent state") to keep the module doctrine honest. Flagging explicitly since invariant #10 ("no beta drift") is subscription-critical: if you read it as "forward beta verbatim on Subscription", say so and I'll add the gate. 2. **One deliberate divergence from Python: sessions are keyed per conversation, not per `(model, system)` bucket.** The Python tracker keys on the store session id — explicit header, else a hash of model + leading system prompt — so a Claude Code session and every one of its subagents share one token union and cross-inherit tokens; two *different users* behind an org proxy with the same (model, system) do too. This port keys on the drift detector's conversation-aware key (#2301), so each conversation keeps its own union (pinned by `separate_conversations_do_not_leak_tokens`). That's the same conflation defect #2085/#2193/#2301 chased out of the other session-sticky subsystems, and it makes "the union only contains tokens this client sent" actually true — under the Python fallback key it isn't (cross-user union). Cost: Python's accidental cross-conversation repair is gone, and an OAuth access-token refresh mid-conversation re-keys the session (one turn forwards verbatim, then re-learns — fails safe). 3. **Repeated header lines are joined per RFC 9110 list semantics before recording.** A client sending two `anthropic-beta` lines gets both recorded; a later rewrite collapses to one line carrying the full set. (Reading only the first line — or Python's actual behavior, which keeps only the *last* line via its `dict(headers)` collapse — can shrink the upstream token set mid-conversation when a rewrite fires.) 4. **Scope: the three intercepted HTTP routes.** With the compression interceptor off the proxy is a strict byte-pipe (Phase-A invariant) — no header mutation, hence the startup warning. WebSocket keeps its behavior (Python's WS site keys on a per-connection UUID, so cross-turn accumulation is a near-no-op there; the Rust WS tunnel doesn't touch beta headers). Bedrock/Vertex are skipped by the same match that skips the drift detector (betas travel in the body as `anthropic_beta` on Bedrock). 5. **Log discipline**: `event=beta_header_merge` carries token *counts* only (beta tokens can carry experiment IDs; same privacy contract as Python's `log_beta_header_merge`, plus the drift detector's hashed session-key prefix instead of Python's raw session id). One deviation from Python's unconditional info: the no-op case logs at debug, matching the drift detector's silent-on-stable precedent — an info-level `beta_header_merge` always marks an actual cache-affecting rewrite. 6. **Capacity is a const (1000), not a flag** — following the drift-detector precedent rather than Python's `HEADROOM_BETA_TRACKER_MAX_SESSIONS` env var. Happy to make it configurable if you'd rather keep that operator knob. 7. **Fail-open everywhere**: non-ASCII client values are forwarded verbatim with nothing recorded; a poisoned tracker lock forwards the client value verbatim; an unencodable union (unreachable — every token came from a parsed header value) logs and forwards verbatim. The protection never delays or drops a request. |
||
|
dependabot[bot]
|
6448545a7f
|
deps: bump bytesize from 1.3.3 to 2.4.2 (#2286)
Bumps [bytesize](https://github.com/bytesize-rs/bytesize) from 1.3.3 to 2.4.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/bytesize-rs/bytesize/releases">bytesize's releases</a>.</em></p> <blockquote> <h2>bytesize: v2.4.2</h2> <ul> <li>Improve accuracy of parsing large non-decimal byte count strings.</li> </ul> <h2>bytesize: v2.4.1</h2> <ul> <li>Fix rounding error near power-of-unit boundaries.</li> </ul> <h2>bytesize: v2.4.0</h2> <ul> <li>Implement <code>Sum</code> for <code>ByteSize</code>.</li> <li>Minimum supported Rust version (MSRV) is now 1.85.</li> </ul> <h2>bytesize: v2.3.1</h2> <ul> <li>Fix unit truncation in error strings.</li> </ul> <h2>bytesize: v2.3.0</h2> <ul> <li>Add <code>Unit</code> enum.</li> <li>Add <code>UnitParseError</code> type.</li> </ul> <h2>bytesize: v2.2.0</h2> <ul> <li>Add <code>ByteSize::as_*()</code> methods to return equivalent sizes in KB, GiB, etc.</li> </ul> <h2>bytesize: v2.1.0</h2> <ul> <li>Support parsing and formatting exabytes (EB) & exbibytes (EiB).</li> <li>Migrate <code>serde</code> dependency to <code>serde_core</code>.</li> </ul> <h2>bytesize: v2.0.1</h2> <ul> <li>Add support for precision in <code>Display</code> implementations.</li> </ul> <h2>bytesize: v2.0.0</h2> <ul> <li>Add support for <code>no_std</code> targets.</li> <li>Use IEC (binary) format by default with <code>Display</code>.</li> <li>Use "kB" for SI unit.</li> <li>Add <code>Display</code> type for customizing printed format.</li> <li>Add <code>ByteSize::display()</code> method.</li> <li>Implement <code>Sub<ByteSize></code> for <code>ByteSize</code>.</li> <li>Implement <code>Sub<impl Into<u64>></code> for <code>ByteSize</code>.</li> <li>Implement <code>SubAssign<ByteSize></code> for <code>ByteSize</code>.</li> <li>Implement <code>SubAssign<impl Into<u64>></code> for <code>ByteSize</code>.</li> <li>Reject parsing non-unit characters after whitespace.</li> <li>Remove <code>ByteSize::to_string_as()</code> method.</li> <li>Remove top-level <code>to_string()</code> method.</li> <li>Remove top-level <code>B</code> constant.</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/bytesize-rs/bytesize/blob/master/CHANGELOG.md">bytesize's changelog</a>.</em></p> <blockquote> <h2>2.4.2</h2> <ul> <li>Improve accuracy of parsing large non-decimal byte count strings.</li> </ul> <h2>2.4.1</h2> <ul> <li>Fix rounding error near power-of-unit boundaries.</li> </ul> <h2>2.4.0</h2> <ul> <li>Implement <code>Sum</code> for <code>ByteSize</code>.</li> <li>Minimum supported Rust version (MSRV) is now 1.85.</li> </ul> <h2>2.3.1</h2> <ul> <li>Fix unit truncation in error strings.</li> </ul> <h2>2.3.0</h2> <ul> <li>Add <code>Unit</code> enum.</li> <li>Add <code>UnitParseError</code> type.</li> </ul> <h2>2.2.0</h2> <ul> <li>Add <code>ByteSize::as_*()</code> methods to return equivalent sizes in KB, GiB, etc.</li> </ul> <h2>2.1.0</h2> <ul> <li>Support parsing and formatting exabytes (EB) & exbibytes (EiB).</li> <li>Migrate <code>serde</code> dependency to <code>serde_core</code>.</li> </ul> <h2>2.0.1</h2> <ul> <li>Add support for precision in <code>Display</code> implementations.</li> </ul> <h2>v2.0.0</h2> <ul> <li>Add support for <code>no_std</code> targets.</li> <li>Use IEC (binary) format by default with <code>Display</code>.</li> <li>Use "kB" for SI unit.</li> <li>Add <code>Display</code> type for customizing printed format.</li> <li>Add <code>ByteSize::display()</code> method.</li> <li>Implement <code>Sub<ByteSize></code> for <code>ByteSize</code>.</li> <li>Implement <code>Sub<impl Into<u64>></code> for <code>ByteSize</code>.</li> <li>Implement <code>SubAssign<ByteSize></code> for <code>ByteSize</code>.</li> <li>Implement <code>SubAssign<impl Into<u64>></code> for <code>ByteSize</code>.</li> <li>Reject parsing non-unit characters after whitespace.</li> <li>Remove <code>ByteSize::to_string_as()</code> method.</li> <li>Remove top-level <code>to_string()</code> method.</li> <li>Remove top-level <code>B</code> constant.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Tejas Chopra
|
e0ce4b1d48
|
fix: remove rtk and lean-ctx CLI context tools (#2677)
## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt. |
||
|
Andrei Boldyrev
|
6744833afe
|
fix(proxy): key drift detector on conversations, not credentials; canonicalize drift hashes (#2301)
## Description
The Rust proxy's cache-bust drift detector
(`crates/headroom-proxy/src/cache_stabilization/drift_detector.rs`,
PR-E6) cannot currently tell drift from normal operation on interactive
agentic traffic, so it warns on nearly every turn and a real bust drowns
in the noise. Three compounding defects, all verified against live
Claude Code traffic:
1. `derive_session_key` stops at the credential hash — Claude Code sends
one OAuth bearer for every conversation, so all concurrent conversations
share one LRU slot and every conversation switch logs a false
`cache_drift_observed` (with `drift_dims` computed against the wrong
conversation's baseline).
2. The `early_messages` axis hashes the raw first-3-messages window, so
a lone conversation's normal growth (1 → 3 messages) and the client
relocating its `cache_control` breakpoint to the newest block both fire
a false `early_messages` drift at turn 2–3 of essentially every session.
3. `x-headroom-session-id` — the explicit session identity the Python
proxy honors everywhere session-sticky state exists — is ignored on the
Rust path.
This PR makes the detector's session identity conversation-scoped and
its comparison canonical, the same shape as the merged Python-side fix
for #2085 (`SessionTrackerStore.resolve_tracker` lineage resolution +
`_canonicalize_for_prefix_compare`):
- **`derive_session_key`**: honors `x-headroom-session-id` first
(hashed, like every other key input), then folds a conversation
discriminator into the credential/network arms: a 16-hex-char SHA-256
fingerprint of `(model, canonicalized first message)`. Provider prompt
caches are per-model, so a small-model sidecar call (title generation)
that reuses a conversation's opener stays a separate session instead of
false-drifting on `system`.
- **`canonicalize_for_hash`** on all axes and the discriminator: objects
rebuilt with sorted keys (this workspace enables serde_json
`preserve_order`, so a plain re-serialize would keep client wire order
and leave the hashes key-order sensitive) and `cache_control` stripped
outside opaque tool payloads (`input`/`arguments`/`json`/`input_schema`
— mirroring the Python canonicalizer's `_OPAQUE_PAYLOAD_KEYS`, so a user
field that happens to be *named* `cache_control` still counts as drift).
- **`early_messages`** becomes per-message hashes (`[Option<[u8; 32]>;
3]`) with a prefix-aware comparison: growing into the window is benign;
a settled message changing or disappearing under a stable session key is
still drift. `observe_drift` now gates the warning on drifted dimensions
rather than raw hash inequality.
True positives are preserved (`system`/`tools` changes, in-place history
rewrites under a pinned identity), and the detector remains a pure
observer — no forwarded byte changes, `does_not_mutate_input` still pins
that.
**Documented trade-off** (module doc + `conversation_discriminator`
doc): without the explicit header, a client that rewrites its first
message (history compaction, rolling-window truncation, Responses
chained mode) re-keys to a fresh session — the rewrite surfaces as
`cache_drift_first_request` rather than `cache_drift_observed` against
the old baseline. That is deliberate: the credential-keyed alternative
false-warned on every conversation switch, which buried those same
events anyway. `x-headroom-session-id` pins the identity and reports
rewrites as drift. Byte-identical openers on the same model under one
credential still conflate (rare; documented).
Closes #2300
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `derive_session_key`: `x-headroom-session-id` (hashed) wins;
credential/IP arms fold in `conversation_discriminator` — `(model,
canonicalized first message)`, 16 hex chars
- New `canonicalize_for_hash`: sorted-key object rebuild +
`cache_control` stripped outside `OPAQUE_PAYLOAD_KEYS`; applied to the
`system`/`tools`/`early_messages` axes and the discriminator
- `StructuralHash.early_messages`: `[u8; 32]` → `[Option<[u8; 32]>;
EARLY_MESSAGES_WINDOW]` per-message hashes;
`drift_dims`/`early_window_drifted` implement the prefix-aware rule;
`observe_drift` warns on non-empty dims instead of `!=`
- `conversation_messages` shape guard: bare-string message containers
only count for the Responses `input` sugar
- Docs: module header (canonicalization, trade-off, honest cost),
`conversation_discriminator` rationale + blind spots,
`DRIFT_DETECTOR_CAPACITY` cardinality note (per-conversation keys,
163-byte entry), `structural_hash_log_prefix` hex-length fix
- Tests: 13 new unit tests (conversation separation, turn-growth key
stability, explicit header priority, marker relocation + growth not
drift, rewrite/shrink still drift, per-model separation, key-order
neutrality, opaque-payload fields still count, Responses/Chat
discriminator shapes, string-container gating)
- `CHANGELOG.md`: Unreleased → Fixed entry
## Testing
- [x] Unit tests pass (`cargo test -p headroom-proxy` — full crate: lib
+ integration suites)
- [x] Linting passes (`cargo clippy -p headroom-proxy --all-targets` —
zero warnings; `cargo fmt --check` clean)
- [ ] Type checking passes (`mypy headroom`) — n/a, no Python files
touched
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ cargo test -p headroom-proxy --lib drift_detector
test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 221 filtered out
$ cargo test -p headroom-proxy
(all suites) test result: ok. 248 passed (lib) + integration suites, 0 failed
$ cargo clippy -p headroom-proxy --all-targets
(no warnings)
```
## Real Behavior Proof
- Environment: macOS 15 (arm64), rustc 1.95.0, repo @
|
||
|
JD Davis
|
f00833654f
|
fix(proxy): satisfy rustfmt import ordering (#2158)
## Summary Fixes the Rust workflow failure from https://github.com/headroomlabs-ai/headroom/actions/runs/29294598252/job/86965269576 by applying rustfmt's import ordering in `crates/headroom-proxy/src/proxy.rs`. ## Testing ```text cargo fmt --all -- --check # passed git diff --check # no output ``` |
||
|
Abhishek Mittal
|
52a024d28c
|
fix(proxy): strip [1m] model suffix before upstream forwarding (#2027)
## Description Scopes the `[1m]` context-window tier suffix sanitizer to Anthropic `/v1/messages` requests only (addresses PR #2027 review feedback). The original patch applied the rewrite to every buffered compressible endpoint, which would have silently mutated OpenAI Chat Completions and OpenAI Responses request model IDs. The `[1m]` marker is an Anthropic/Claude Code compatibility signal emitted by the Headroom CLI; the existing Python parity behavior (`sanitize_anthropic_model_id()`) is Anthropic-specific and must not leak onto OpenAI shapes. Refactors the helper into `compression::sanitize_anthropic_model_id_in_body`, drops the dead `sanitize_model_id` helper in `sse/anthropic.rs`, and adds 8 unit tests + 5 wiremock-backed integration tests that pin the scope. All 420 `headroom-proxy` tests pass; `cargo fmt` and `cargo clippy -D warnings` clean. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Move `sanitize_request_model_id` out of `proxy.rs` and into `compression::sanitize_anthropic_model_id_in_body` (Anthropic-specific name; private `trim_anthropic_model_id_suffix` helper for unit-testable pure behavior). - Gate the call site on `CompressibleEndpoint::AnthropicMessages` **after** classification. The OpenAI Chat Completions and OpenAI Responses arms get an explicit no-op match so the sanitizer cannot re-apply to those paths. - Drop the dead `sanitize_model_id` helper in `sse/anthropic.rs` (it was `#[allow(dead_code)]` with no callers). - 8 new unit tests in `compression/mod.rs`: trailing `[1m]` stripped, Claude-style suffix stripped, no-suffix passthrough (byte-equal), non-string model, missing `model` field, non-JSON body, `[1m]` mid-string, and the pure trim helper. - 5 new integration tests in `tests/integration_anthropic_model_sanitize.rs` that boot a real Rust proxy in front of a wiremock upstream. ## Testing - [x] Unit tests pass (`cargo test -p headroom-proxy` → 420 passed, 35 suites) - [x] Linting passes (`cargo clippy -p headroom-proxy --tests --all-features -- -D warnings` clean) - [x] Type checking passes (`cargo check -p headroom-proxy --tests --all-features` clean) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-proxy --test integration_anthropic_model_sanitize Compiling headroom-proxy v0.x.x Finished `test` profile [unoptimized + debuginfo] target(s) Running tests/integration_anthropic_model_sanitize.rs test anthropic_messages_strips_1m_suffix_glm ... ok test anthropic_messages_strips_1m_suffix_claude ... ok test anthropic_messages_passthrough_when_no_suffix ... ok test openai_chat_completions_passthrough_with_1m_model ... ok test openai_responses_passthrough_with_1m_model ... ok test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ``` ```text $ cargo test -p headroom-proxy test result: ok. 420 passed; 0 failed; 0 ignored; 0 measured; 235 filtered out finished in 10.93s ``` ```text $ cargo clippy -p headroom-proxy --tests --all-features -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) ``` ## Real Behavior Proof - **Environment:** macOS 14.x; `rustc` pinned via `rust-toolchain.toml`; `cargo` 1.x. No network access required (wiremock upstream). - **Exact command / steps:** 1. `cargo test -p headroom-proxy --test integration_anthropic_model_sanitize` — confirms `/v1/messages` strips `glm-5.2[1m]` and `claude-3-7-sonnet[1m]`; confirms `/v1/chat/completions` and `/v1/responses` leave the body byte-equal (SHA-256 asserted). 2. `cargo test -p headroom-proxy` — full suite green (420 passed). 3. `cargo clippy -p headroom-proxy --tests --all-features -- -D warnings` — clean. 4. `cargo fmt -p headroom-proxy --check` — clean. 5. Source inspection of `crates/headroom-proxy/src/proxy.rs` after the change: the call site is now in a `match endpoint` arm that explicitly returns `buffered` for the OpenAI variants, so the sanitizer cannot re-apply to those paths. - **Observed result:** all 5 new integration tests pass, all 420 crate tests pass, clippy and fmt clean. The OpenAI tests assert SHA-256 byte equality on a body whose `model` field ends in `[1m]`; if the sanitizer were to re-leak onto OpenAI shapes these would fail loudly with a length delta. - **Not tested:** a live Anthropic API call (would require real credentials and is not required to prove the byte-level scope fix). The Python proxy's `sanitize_anthropic_model_id()` is the documented parity reference (Python PR #1840, issue #1812). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (N/A — no user-facing docs change; the Python proxy's `sanitize_anthropic_model_id` is the parity reference cited in code comments) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (project uses git log + PR titles; this PR's title follows the conventional commit shape) ## Screenshots (if applicable) N/A — backend behavior, no UI change. ## Additional Notes - The OpenAI integration tests rely on a JWT-style `Authorization: Bearer` header to classify the request as `AuthMode::OAuth` and short-circuit the PR-E4 `prompt_cache_key` injector. This is the same control variable the existing `integration_chat_completions.rs` tests use to isolate dispatcher byte-fidelity from the E4 hook. Comments in each test explain the relationship. - The dead helper in `sse/anthropic.rs` is removed, so the diff is net negative on LoC for the SSE module. - The Python parity reference is `sanitize_anthropic_model_id()` (Python PR #1840, issue #1812); the function name and the call-site scope are the explicit parity contract. - Branch was rebased onto `upstream/main` (91 commits behind) before force-push to the fork; conflict-free rebase. The original PR commit and the fix are the only two commits on the PR. --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> Co-authored-by: Abhishek Mittal <abhishek.mittal@users.noreply.github.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
JD Davis
|
2c9eb7c5f1
|
feat(simulators): add provider simulator service (#2014)
## Description
Adds a Rust-only `headroom-simulators` workspace crate: a deterministic
local upstream simulator service for Headroom proxy and pipeline
validation. It supplies configurable stubs plus bottled provider-shaped
responses for supported provider/path surfaces without calling real
LLMs.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added `crates/headroom-simulators` Rust crate with library and
`headroom-simulators` binary.
- Added clean domain classification for supported surfaces: Anthropic
`/v1/messages`, OpenAI chat/responses/conversations, Bedrock
invoke/stream routes, Vertex raw/stream predict, health, and generic
fallback.
- Added JSON-configured stub matching by method, path, body substring,
and JSON pointer.
- Added bottled provider-shaped JSON, SSE, and Bedrock EventStream
responses for unconfigured requests.
- Added a container `Dockerfile` and README for local/GitHub Actions
usage.
- Added unit and HTTP integration tests for defaults, configured stubs,
SSE, Vertex, and Bedrock EventStream behavior.
- Added proxy-level simulator-backed E2E tests that run Headroom against
the simulator across Anthropic, OpenAI Chat, OpenAI Responses, OpenAI
Conversations, Bedrock invoke/converse/streaming, Vertex raw/stream
predict, and upstream health.
- Added simulator-backed provider error-path E2E coverage for OpenAI
429, Anthropic 529, Bedrock 502, and Vertex 503 responses flowing
through Headroom unchanged.
- Added Headroom-owned preflight error E2E coverage proving Bedrock
missing credentials and invalid Vertex envelopes stop inside the proxy
instead of silently falling through to the simulator/provider.
- Fixed direct Rust `headroom-core` binaries/tests on Windows so Magika
initializes ONNX Runtime via `ort::init_from` from an explicit pip
`onnxruntime` library path, with fail-fast fallback only when no safe
runtime is discoverable.
- Added a Rust CI `simulator-e2e` matrix for `ubuntu-latest`,
`macos-latest`, and `windows-latest` that runs `cargo test -p
headroom-proxy --test e2e_simulators`.
- Gated dynamic Magika `Path`/`PathBuf` imports to Windows and x86_64
macOS so Linux clippy does not see unused dynamic-ORT-only imports.
## Testing
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
cargo fmt --all -- --check
# passed
cargo clippy --workspace -- -D warnings
# passed
$env:ORT_DYLIB_PATH=$null
cargo test -p headroom-core transforms::magika_detector::tests:: --lib
# 17 passed, 0 failed; Magika initialized from discovered pip
onnxruntime DLL
$env:ORT_DYLIB_PATH=$null
cargo test --workspace
# passed
gitleaks protect --staged --no-banner --redact
# no leaks found
gitleaks git --log-opts="headroomlabs/main..HEAD" --no-banner --redact
# 5 commits scanned; no leaks found
## Real Behavior Proof
- **Environment:** Windows PowerShell, Rust toolchain `1.95.0`, clean
worktree from `headroomlabs/main` at `
|
||
|
dependabot[bot]
|
5229c98228
|
deps: bump prometheus from 0.13.4 to 0.14.0 (#1518)
Bumps [prometheus](https://github.com/tikv/rust-prometheus) from 0.13.4 to 0.14.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/tikv/rust-prometheus/blob/master/CHANGELOG.md">prometheus's changelog</a>.</em></p> <blockquote> <h2>0.14.0</h2> <ul> <li> <p>API change: Use <code>AsRef<str></code> for owned label values (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/537">#537</a>)</p> </li> <li> <p>Improvement: Hashing improvements (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/532">#532</a>)</p> </li> <li> <p>Dependency upgrade: Update <code>hyper</code> to 1.6 (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/524">#524</a>)</p> </li> <li> <p>Dependency upgrade: Update <code>procfs</code> to 0.17 (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/543">#543</a>)</p> </li> <li> <p>Dependency upgrade: Update <code>protobuf</code> to 3.7.2 for RUSTSEC-2024-0437 (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/541">#541</a>)</p> </li> <li> <p>Dependency upgrade: Update <code>thiserror</code> to 2.0 (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/534">#534</a>)</p> </li> <li> <p>Internal change: Fix LSP and Clippy warnings (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/540">#540</a>)</p> </li> <li> <p>Internal change: Bump MSRV to 1.81 (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/539">#539</a>)</p> </li> <li> <p>Documentation: Fix <code>register_histogram_vec_with_registry</code> docstring (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/528">#528</a>)</p> </li> <li> <p>Documentation: Fix typos in static-metric docstrings (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/479">#479</a>)</p> </li> <li> <p>Documentation: Add missing <code>protobuf</code> feature to README list (<a href="https://redirect.github.com/tikv/rust-prometheus/issues/531">#531</a>)</p> </li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
98f7f1c2a3
|
deps: bump tower-http from 0.6.11 to 0.7.0 (#1520)
Bumps [tower-http](https://github.com/tower-rs/tower-http) from 0.6.11 to 0.7.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tower-rs/tower-http/releases">tower-http's releases</a>.</em></p> <blockquote> <h2>tower-http-0.7.0</h2> <p><a href="https://github.com/tower-rs/tower-http/compare/tower-http-0.6.11...tower-http-0.7.0">Changes since 0.6.11</a></p> <h2>Added</h2> <ul> <li> <p><code>csrf</code>: add cross-site request forgery (CSRF) protection middleware, porting the cross-origin protection scheme introduced in Go 1.25 (<a href="https://redirect.github.com/tower-rs/tower-http/issues/699">#699</a>)</p> <pre lang="rust"><code>use tower::ServiceBuilder; use tower_http::csrf::CsrfLayer; <p>// Rejects cross-origin state-changing requests using <code>Sec-Fetch-Site</code>,<br /> // an <code>Origin</code> allow-list, and an <code>Origin</code>/<code>Host</code> fallback. No per-request<br /> // token state required.<br /> let layer = CsrfLayer::new().add_trusted_origin("<a href="https://example.com">https://example.com</a>")?;</p> <p>let service = ServiceBuilder::new().layer(layer).service_fn(handler);<br /> </code></pre></p> </li> <li> <p><code>timeout</code>: add <code>DeadlineBody</code> for non-resetting body timeouts, applied via the new <code>RequestBodyDeadlineLayer</code> and <code>ResponseBodyDeadlineLayer</code> (<a href="https://redirect.github.com/tower-rs/tower-http/issues/688">#688</a>)</p> <p>Unlike <code>TimeoutBody</code>, which resets its deadline on every frame, <code>DeadlineBody</code> caps the total time of a body transfer. A slow client trickling one byte at a time never trips an idle timeout but will trip a deadline.</p> <pre lang="rust"><code>use std::time::Duration; use tower::ServiceBuilder; use tower_http::timeout::RequestBodyDeadlineLayer; <p>// Abort the request body transfer after 30s total, regardless of how<br /> // frequently data arrives.<br /> let service = ServiceBuilder::new()<br /> .layer(RequestBodyDeadlineLayer::new(Duration::from_secs(30)))<br /> .service_fn(handler);<br /> </code></pre></p> </li> <li> <p><code>fs</code>: add strong <code>ETag</code> support to <code>ServeDir</code>, including <code>If-Match</code> and <code>If-None-Match</code> precondition handling per RFC 9110. <code>304 Not Modified</code> responses now carry the <code>ETag</code> and <code>Last-Modified</code> validators (<a href="https://redirect.github.com/tower-rs/tower-http/issues/691">#691</a>)</p> </li> <li> <p><code>fs</code>: add a <code>Backend</code> trait to make <code>ServeDir</code> work with non-filesystem sources (e.g. embedded assets or object storage). The default <code>TokioBackend</code> preserves existing behavior. Use <code>ServeDir::with_backend()</code> to plug in custom implementations (<a href="https://redirect.github.com/tower-rs/tower-http/issues/684">#684</a>)</p> <pre lang="rust"><code>use tower_http::services::fs::ServeDir; <p>// <code>MyBackend</code> implements <code>tower_http::services::fs::Backend</code>.<br /> // The default <code>ServeDir::new()</code> continues to use <code>TokioBackend</code> (local FS).<br /> let service = ServeDir::with_backend("assets", MyBackend::new());<br /> </code></pre></p> </li> <li> <p><code>fs</code>: add <code>html_as_default_extension</code> option to <code>ServeDir</code>, appending <code>.html</code> when the request path has no extension (<a href="https://redirect.github.com/tower-rs/tower-http/issues/519">#519</a>)</p> </li> <li> <p><code>fs</code>: add <code>redirect_path_prefix</code> option to <code>ServeDir</code>, prepending a prefix on trailing-slash redirects so the service can be mounted under a sub-path (<a href="https://redirect.github.com/tower-rs/tower-http/issues/486">#486</a>)</p> </li> <li> <p><code>validate-request</code>: add <code>ValidateRequestHeaderLayer::has_header_value()</code> to reject requests when a header does not have an expected value (<a href="https://redirect.github.com/tower-rs/tower-http/issues/360">#360</a>)</p> </li> <li> <p><code>body</code>: <code>UnsyncBoxBody::new()</code> constructor and <code>From<ServeFileSystemResponseBody></code> conversion to avoid double-boxing when combining <code>ServeDir</code> responses with other body types (<a href="https://redirect.github.com/tower-rs/tower-http/issues/537">#537</a>)</p> </li> <li> <p><code>limit</code>: implement <code>Default</code> for <code>limit::ResponseBody</code> when the wrapped body also implements <code>Default</code> (<a href="https://redirect.github.com/tower-rs/tower-http/issues/679">#679</a>)</p> </li> </ul> <h2>Changed</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Tejas Chopra
|
5771a8020e
|
fix(deps): remediate dependency CVEs and publish SBOM (#1509)
## Description
Supply-chain hardening: takes the **shipped** dependency surface from
**26 known CVEs to 0**. `pip install headroom-ai[all]` now resolves with
no known vulnerabilities (verified with Anchore syft + grype). Also
publishes a checked-in SBOM package (`sbom/`) so any user — especially
pilots running their own security review — can verify what's inside and
that we track it.
This addresses the Dependabot alerts on `main` (9 high / 4 moderate / 7
low at time of writing).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
**Rust**
- `pyo3` 0.24 → 0.29 (GHSA-36hh-v3qg-5jq4 High, GHSA-chgr-c6px-7xpp
Med). Migrated `Python::allow_threads` → `Python::detach` (10 sites) and
added `from_py_object` to the `Clone`-deriving `#[pyclass]` types (both
required by the 0.25+ API).
- `pyo3-log` 0.12 → 0.13; `lru` 0.12 → 0.18 (GHSA-rhfx-m35p-ff5j).
**Python**
- `torch` → 2.12.1, `mem0ai` → 2.x.
- Floor-pinned transitive CVE deps via `[tool.uv]
constraint-dependencies`: `pygments>=2.20.0`,
`pydantic-settings>=2.14.2`, `gitpython>=3.1.50`, `langsmith>=0.9.0`.
- **Removed `benchmark` from the `[all]` aggregate** so the default
install is CVE-free. `lm-eval` is invoked as an external subprocess
(`python -m lm_eval`) and never imported, so it is not a true runtime
dep — it remains available via the opt-in `[benchmark]` extra. See
[Accepted Risks](#additional-notes).
**npm (build/test tooling — never shipped in the
wheel/container/published SDK)**
- `esbuild` override `>=0.28.1` in `sdk/typescript` + `plugins/openclaw`
(GHSA-g7r4-m6w7-qqqr).
- `docs/`: `@anthropic-ai/sdk` → `^0.106.0` (GHSA-p7fg-763f-g4gf),
`postcss` override to force Next.js's bundled copy ≥8.5.10
(GHSA-qx2v-qp2m-jg93); regenerated a stale `bun.lock` that carried a
**Critical** vitest/vite.
**CI**
- Pinned `pypa/gh-action-pypi-publish` `@release/v1` → `@v1.13.0`
(GHSA-vxmw-7h4f-hqxh) in `release.yml` + `publish.yml`.
**SBOM**
- New `sbom/` directory: CycloneDX 1.7 + SPDX 2.3 SBOMs, grype scan
evidence, 330-package license inventory, and a regeneration guide.
## Testing
- [ ] Unit tests pass (`pytest`) — N/A, no Python source changed
(deps/config only)
- [x] Linting passes — `cargo fmt --check` + `cargo clippy` clean on the
changed crate; 0 `.py` files changed so `ruff`/`mypy` scope is
unaffected
- [x] Type checking passes — `cargo check --workspace` (0 errors)
- [ ] New tests added — N/A (dependency bumps; covered by existing
suites)
- [x] Manual testing performed — see Real Behavior Proof
### Test Output
```text
# headroom-ai[all] product surface — the number that matters
$ grype sbom:sbom/headroom-sbom-all-extra.cdx.json
No vulnerabilities found
# full repo scan (universal lock incl. opt-in [benchmark] + dev)
$ grype sbom:sbom/headroom-sbom.cdx.json
NAME INSTALLED TYPE VULNERABILITY SEVERITY
sqlitedict 2.1.0 python GHSA-g4r7-86gm-pgqc High # [benchmark]-only, unpatchable, accepted
nltk 3.9.4 python GHSA-p4gq-832x-fm9v High # [benchmark]-only, unpatchable, accepted
# pyo3 0.29 migration — extension builds + imports + runs
$ cargo check --workspace
Finished `dev` profile [unoptimized + debuginfo] target(s)
$ maturin develop && python -c "from headroom._core import DiffCompressor, SmartCrusher; ..."
extension OK — detach + from_py_object paths exercised
# lru 0.18 — eviction path
$ cargo test -p headroom-proxy --lib drift
14 passed, 213 filtered out
# per-ecosystem npm audits
$ (cd sdk/typescript && npm audit) -> found 0 vulnerabilities
$ (cd plugins/openclaw && npm audit) -> found 0 vulnerabilities
$ (cd docs && npm audit && bun audit) -> found 0 vulnerabilities / No vulnerabilities found
```
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0, arm64), Python 3.12 `.venv`, Rust
1.95 toolchain, syft 1.46.0, grype 0.115.0, bun 1.3.14, maturin 1.13.3.
- Exact command / steps: (1) `uv export --extra all --no-dev
--no-emit-project | syft → grype` for the product surface; (2) `cargo
check --workspace` + `maturin develop` + extension import/compress smoke
test; (3) `cargo test -p headroom-proxy --lib drift`; (4) `cargo fmt
--check` + `cargo clippy -p headroom-py`; (5) `npm audit` in
sdk/openclaw/docs + `bun audit` in docs.
- Observed result: `headroom-ai[all]` resolution scans clean — "No
vulnerabilities found" (179 pkgs); full/prod SBOM shows only the 2
documented accepted CVEs; pyo3 0.29 extension imports and runs (detach +
from_py_object paths exercised); drift tests 14/14 pass; cargo fmt +
clippy clean; all npm/bun audits report 0.
- Not tested: full `pytest` suite (no Python source changed);
release-profile wheel build (used dev-profile `maturin develop` for the
import proof — the extension is semantically identical).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
(`sbom/README.md`)
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective — N/A
(dependency bumps; existing suites + scans cover it)
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md — N/A (Release Please
auto-generates from the conventional commit)
## Additional Notes
**Accepted risks (the 2 residual CVEs).** Both originate solely from the
EleutherAI `lm-evaluation-harness` under the **opt-in `[benchmark]`
extra**, which Headroom invokes as a subprocess (never imports):
- `sqlitedict` CVE-2024-35515 (High) — pickle deserialization; package
abandoned (last release 2021), **no upstream fix exists**.
- `nltk` CVE-2026-54293 (High) — path traversal in `nltk.data.load()`;
affects ≤3.9.4 (current latest), **no patched release**.
Neither is in `[all]`, the published wheel, or the container. They are
documented in `sbom/README.md` and will be picked up automatically once
upstream ships fixes.
**Release/CHANGELOG:** N/A items above are because this is a
dependency/security PR with no Python source changes; CHANGELOG is
Release-Please-managed via the conventional commit message.
|
||
|
Ali
|
0e6d922f88
|
feat(pricing): add DeepSeek V4 model pricing (deepseek-v4-flash, deepseek-v4-pro) (#1168)
## Description Adds pricing support for DeepSeek V4 models (`deepseek-v4-flash` and `deepseek-v4-pro`) when routing Headroom through `--anthropic-api-url https://api.deepseek.com/anthropic`. The vendored LiteLLM pricing database predates DeepSeek V4, so cost estimation silently returned `None` for these models. ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - **`headroom/pricing/deepseek_prices.py`** — New pricing data module with `ModelPricing` dataclass entries for both V4 models, following the pattern of `anthropic_prices.py` - **`headroom/pricing/__init__.py`** — Exports `DEEPSEEK_PRICES`, `get_deepseek_registry()`, `DEEPSEEK_LAST_UPDATED` - **`headroom/pricing/litellm_pricing.py`** — Runtime injection of DeepSeek V4 pricing into `litellm.model_cost`, plus `deepseek-` prefix added to `resolve_litellm_model()` provider prefix list - **`headroom/providers/anthropic.py`** — DeepSeek fallback in `_get_pricing()` when model starts with `deepseek-` and LiteLLM is unavailable - **`crates/headroom-proxy/data/model_prices_and_context_window.json`** — Vendored JSON entries (bare + provider-prefixed) for Rust-side context window lookups - **`tests/test_providers/test_deepseek.py`** — 20 tests across 3 test classes (pricing data, LiteLLM injection, Anthropic fallback) - **`tests/test_pricing.py`** — Added DeepSeek export validation alongside existing OpenAI/Anthropic assertions ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ``` ========================= 137 passed, 8 warnings in 8.47s ========================= ``` ## Real Behavior Proof - Environment: Windows 10, Python 3.12, litellm 1.60+ - Exact command / steps: `python -c "from headroom.proxy.cost import CostTracker; t = CostTracker(); print(t.estimate_cost('deepseek-v4-flash', input_tokens=1000000, output_tokens=1000000))"` - Observed result: `$0.4200` (0.14 input + 0.28 output per 1M tokens) - Not tested: Live DeepSeek API routing via `--anthropic-api-url` (requires API key and Docker deployment) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The 90% cache discount heuristic in `AnthropicProvider.estimate_cost()` (line 680) is a pre-existing pattern. DeepSeek V4 has much deeper cache discounts (98-99%), but the LiteLLM path currently falls through to the manual fallback which uses correct cached prices. A future improvement could prefer `cache_read_input_token_cost` from model info over the hardcoded `* 0.1` heuristic. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Yasser Sheikh
|
0dc2e1cb3f
|
feat(bedrock): cross-region + Converse compression; bundle proxy binary in images (#999)
## Description The native Bedrock path (Phase D) compresses + signs Anthropic-on-Bedrock requests, but two real-world cases slipped through, and the native binary that powers it was never shipped. This PR closes those gaps as a focused set of give-backs. Aligns with the Rust migration plan (see below). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Cross-region inference-profile detection** via a new `bedrock::vendor` module (`canonical_vendor()`), following the design proposed in #953: strip a known geo prefix (`eu.`/`us.`/`apac.`/`global.`) then match the canonical vendor. Geo-prefixed Anthropic profiles (`eu.anthropic.…`) now get live-zone compression instead of being silently skipped; geo-prefixed non-Anthropic vendors stay correctly excluded. - **Converse-body compression (two parts)**: 1. `run_anthropic_compression` no longer bails to passthrough when the body lacks an InvokeModel `anthropic_version` envelope; envelope re-emit stays gated on successful parse. 2. The **live-zone dispatcher now recognizes Bedrock Converse content blocks**. Converse blocks carry no `type` discriminator (the variant is the key: `{"text": …}` vs Anthropic's `{"type":"text","text":…}`), so real Converse user-message text was still passing through uncompressed. A typeless block whose `text` is a JSON string now routes through the same surgical text path. Anthropic blocks always carry `type`, so the Anthropic path is byte-for-byte unchanged; non-text Converse blocks (`{"image":…}`, `{"toolUse":…}`) stay unrecognized and no-op. - **Correct `/converse` upstream routing**: the non-streaming handler resolved the upstream action from a hard-coded `"invoke"`, so `/converse` requests were forwarded to Bedrock's `/invoke` endpoint. It now resolves the action from the inbound path (`extract_invoke_action`), mirroring the streaming handler's `extract_streaming_action`. SigV4 signs the same URL it forwards, so the signature stays consistent. - **`aws-config` `sso` feature**: SSO profiles now resolve through the default credential chain for SigV4 — the credential chain in `docs/bedrock.md` already promised SSO; this makes the code match. - **Ship the `headroom-proxy` binary in published images** (`Dockerfile`): built in the builder stage (`--locked`, with the cargo registry cache mounted at `CARGO_HOME`) and copied into both the debian and distroless runtime images. - **Docs** (`docs/bedrock.md`): document cross-region inference profiles and a "Running the proxy" section. AWS credentials mount at `/home/nonroot/.aws` (the default nonroot image home) where the SDK looks for `~/.aws`, with a note on the root-image alternative. ## Related issues - Closes #976 — ship the `headroom-proxy` binary in published images (this PR implements the exact fix proposed there). - Addresses the **cross-region inference-profile** half of #953 via its proposed `canonical_vendor()` design. Non-Anthropic vendor compression parity (Nova/GLM/MiniMax/ Kimi) is the natural follow-up — `bedrock::vendor` is the shared resolver it can build on. - Extends the native Bedrock InvokeModel compression requested in #734 (the Bedrock slice of #510) to cross-region profiles and Converse bodies. - Partially enables #181 (native, Python-free packaging): the native binary now ships in the images, though full Python-free distribution remains out of scope. ## Alignment with the Rust migration plan Per `docs/spec/022-rust-migration.md`, the migration is **proxy-first**: `headroom-proxy` is the deployable Rust artifact, native routes replace Python passthroughs one at a time (Stage 4 = provider expansion, Bedrock included), and the binary is meant to be "built, tested, and **released together with the Python package**." Two ways this PR advances that: - The binary-in-images change makes the codebase do what the spec already states (ship the artifact) — closing the gap that forced downstreams to build from source. - Hardening the native Bedrock route (cross-region, Converse routing + body compression) is exactly the Stage-4 provider-expansion work, keeping the native path at parity with real traffic so it can be the default rather than a passthrough. ## Testing - [x] Unit tests pass (`cargo test -p headroom-core -p headroom-proxy` — full suites, 0 failures) - [x] Linting passes (`cargo clippy -p headroom-core -p headroom-proxy --all-targets -- -D warnings`) - [x] Formatting passes (`cargo fmt -- --check`) - [x] New tests added — `bedrock::vendor` (foundation + inference-profile matching), `extract_invoke_action` + converse upstream URL, and live-zone Converse text-block routing (`block_has_string_text_field`, converse-vs-anthropic dispatch equivalence). - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-core -p headroom-proxy # all suites: ok, 0 failed $ cargo clippy -p headroom-core -p headroom-proxy --all-targets -- -D warnings # Finished, no warnings $ cargo fmt -- --check # clean # image validation (local, proxy/code extras): $ docker build --target runtime ... # debian: /usr/local/bin/headroom-proxy, --help OK $ docker build --target runtime-slim ... # distroless: binary links + --help OK ``` ## Real Behavior Proof - Environment: native Bedrock proxy against `bedrock-runtime.eu-west-2`, SSO profile, model `eu.anthropic.claude-haiku-4-5-20251001-v1:0`. - Exact command / steps: POST a large multi-turn Converse body to `/model/eu.anthropic.claude-haiku-4-5-20251001-v1:0/converse`; separately build the `runtime` + `runtime-slim` targets and run `/usr/local/bin/headroom-proxy --help`. - Observed result: before — `bedrock_compression_skipped` (geo-prefixed id not recognized), forwarded uncompressed to the wrong `/invoke` upstream; after — geo-prefixed id recognized, `/converse` forwarded to the `/converse` upstream, live-zone dispatcher compresses the Converse user-message text, measurable token savings. Images contain a runnable `headroom-proxy` in both variants. - Not tested: non-Anthropic vendor compression parity (#953 follow-up); Converse `toolResult` nested-text compression (follow-up — only top-level Converse text blocks compress today). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - An earlier revision flipped the EventStream `Accept` default (`*/*`/absent → passthrough); **dropped** — `*/*` is what most clients (incl. reqwest and the proxy's own metrics tests) send while expecting SSE, so forcing passthrough breaks the standard SSE path. - The binary build adds the native-proxy compile to the image build; happy to gate it behind a build arg if maintainers prefer it opt-in. - Addressed a Copilot review round: corrected the `/converse` upstream routing, the stale `run_anthropic_compression` comment, the Dockerfile cargo cache mount + `--locked`, and the nonroot AWS-credentials docs example. |
||
|
Yasser Sheikh
|
b08ec15b0d
|
fix(proxy): add native Bedrock converse-stream route (#917)
## Description
Adds native Bedrock `POST /model/{model_id}/converse-stream` routing in
`headroom-proxy` by reusing the existing streaming handler and
preserving route-specific upstream action forwarding.
This addresses a gap where native Bedrock streaming support existed for
`invoke-with-response-stream` but not `converse-stream`, even though
both share the same EventStream transport and SSE translation path in
this proxy.
Fixes #919
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
## Changes Made
- Add route mount in `crates/headroom-proxy/src/proxy.rs`:
- `POST /model/:model_id/converse-stream` ->
`bedrock::invoke_streaming::handle_invoke_streaming`
- Update streaming handler URL construction in
`crates/headroom-proxy/src/bedrock/invoke_streaming.rs`:
- infer action from inbound path (`invoke-with-response-stream` or
`converse-stream`)
- build upstream URL with the resolved action
- return structured `400` for unsupported streaming action paths
- Add unit tests in
`crates/headroom-proxy/src/bedrock/invoke_streaming.rs`:
- action extraction coverage for both streaming paths
- upstream URL construction coverage for `converse-stream`
- Add integration coverage in
`crates/headroom-proxy/tests/integration_bedrock_streaming.rs`:
- `converse_stream_route_translates_to_sse`
- Add changelog entry under `Unreleased` bug fixes in `CHANGELOG.md`.
## Testing
- `cargo fmt --all`
- `cargo test -p headroom-proxy --test integration_bedrock_streaming --
--nocapture`
- `cargo test -p headroom-proxy --test integration_bedrock_metrics --
--nocapture`
## Real behavior proof
- **Setup tested on**
- macOS (darwin)
- Rust workspace local dev build
- `headroom-proxy` integration tests using wiremock upstream (no AWS
dependency)
- **Exact commands run after patch**
- `cargo test -p headroom-proxy --test integration_bedrock_streaming --
--nocapture`
- `cargo test -p headroom-proxy --test integration_bedrock_metrics --
--nocapture`
- **After-fix evidence + observed result**
- New integration test `converse_stream_route_translates_to_sse` passes.
- Streaming suite result: `10 passed; 0 failed`.
- Metrics suite result: `4 passed; 0 failed`.
- Logs show requests reaching `/model/.../converse-stream` and flowing
through Bedrock streaming path.
- **What I did not test**
- Live AWS Bedrock calls against real credentials/models.
- End-to-end CLI/runtime behavior outside Rust integration test harness.
|
||
|
chopratejas
|
2a717a993e |
fix(observability): G3 remediation — bound cardinality + wire dead metrics
Phase G PR-G3 review identified 5 Critical + 4 High + 5 Medium
findings. This commit lands all 14 fixes plus the optional nits.
CRITICAL
* C1 (cardinality DoS): `service_tier` was read from inbound JSON
and used verbatim as a metric label. A malicious client could
blow up the metric vector unboundedly. Added bounded vocabulary
in `metric_names.rs::service_tier` ({auto, default, flex,
on_demand, priority, scale, other-sentinel}) + a `validate()`
helper. Both request-side (`handlers/responses.rs`) and
response-side (`proxy.rs` Responses arm) gate raw values through
it.
* C2 (dead metric): `proxy_passthrough_bytes_modified_total` had
no production emit site. Wired it in `proxy.rs` to fire when a
dispatcher arm returning `NoCompression`/`Passthrough` produces
a body of a different byte length (a true cache-poisoning
regression detector). The check runs BEFORE the PR-E4
prompt_cache_key injector so legitimate injector mutations do
not trip the alarm.
* C3 (Python/Rust boundary): `proxy_image_generation_call_log_redacted_total`
was a dead Rust counter — the redaction happens entirely in the
Python proxy's request_logger. Removed the Rust counter; moved
the metric to the Python proxy's `/metrics` exporter via the
existing `redactions_total()` module-level counter.
* C4 (Python/Rust boundary): `wrap_rtk_invocations_total` was a
dead Rust counter with no wrap-side bridge. Removed the Rust
counter; added new `headroom/cli/wrap_rtk_metrics.py` with
`record_rtk_invocation(tool, delta)` + `rtk_invocation_counts()`
primitives and surfaced them via the Python proxy's `/metrics`
exporter.
* C5 (dead metric): `proxy_compression_rejected_by_token_check_total`
had no production caller. Wired it in
`live_zone_anthropic.rs`, `live_zone_openai.rs`, and
`live_zone_responses.rs` to increment on every
`BlockAction::RejectedNotSmaller` block in the manifest. The
metric now reflects real "compressor ran but kept original"
cases.
HIGH
* H1 (per-strategy ratio garbage): `proxy_compression_ratio_by_strategy`
emitted the same aggregate ratio for every strategy in
`strategies_applied` when multiple strategies ran on one body.
Added `per_strategy_tokens: Vec<PerStrategyTokens>` to
`Outcome::Compressed`; per-strategy `(before, after)` is
accumulated from the manifest at the wrapper sites and emitted
one sample per strategy in `proxy.rs`. Empty vec → fallback to
one aggregate-labelled sample with a debug log (Phase E
normalization paths that don't track per-strategy tokens).
* H2 (aborted stream): cache_hit_rate observed on client
disconnects mid-stream. Added a gate: Anthropic only fires when
`state.status == MessageStop`, OpenAI Responses only when
`terminal_status().is_some()`. Extracted the gate into the
pure function `compute_anthropic_session_hit_rate(state)` so
the H2 contract is unit-testable independent of the shared
global registry.
* H3 (docs lie + alarm contract): docs claimed HELP/TYPE is
reachable on fresh boot, then contradicted itself. Force-zero
every counter / gauge MetricVec with an `__init__` sentinel
label on each scrape so HELP/TYPE + a zero row are visible from
boot. Histograms are NOT force-zeroed (a synthetic observe(0.0)
would pollute percentiles). PromQL queries in docs filter
`{... != "__init__"}` so the sentinel rows are excluded from
aggregations.
* H4 (crate-version dependency): pinned `prometheus = "=0.13.4"`
exactly (no caret) so a future minor bump cannot silently break
the H3 force-zero contract that relies on this crate's gather()
semantics. Added a clear "retest the alarm contract on bump"
paragraph in docs.
MEDIUM
* M1 (saturate on cached > input): OpenAI Chat + Responses cache-
hit-rate computed `non_cached = input.saturating_sub(cached)`,
silently clamping to 0 if `cached > input`. Per "no silent
fallbacks", log + skip the emit on this wire-format pathology.
* M2 (over-fire on non-image base64): Python redactor's "density
heuristic" over-fired on encrypted blobs / signed tokens /
minified JSON / tool outputs. Tightened: only redact strings
inside known image-bearing JSON paths (`data`, `url`,
`image_url`, `image`) OR strings starting with `data:image/`.
* M3 (NaN clamp): cache_hit_rate::observe used `f64::clamp(0,1)`
which returns NaN for NaN input; the `debug_assert!` was
compiled out in release. Added `is_finite()` guard with a
loud-log + skip before observe.
* M4 (PromQL median-only): added p95, p99, mean (sum/count), and
Phase H canary-gate query section to docs. Canary fails if ANY
of {p50, p95, p99, mean} regresses below the Python baseline.
* M5 (label byte vs char): the `<image:base64-redacted bytes=N>`
placeholder reported character count, not UTF-8 byte count.
Switched to `.encode('utf-8').__len__()` so the label is
honest for non-ASCII payloads (ASCII base64 still has byte ==
char so existing scrapes are unchanged).
OPTIONAL
* Removed dead `debug_assert_eq!(buffered.len(), buffered.len(),
...)` no-op in proxy.rs.
* Normalised `record_response_status` log level from `info` to
`debug` to match peer metric helpers.
Tests:
* Rust: 11 integration_metrics tests (was 6) + 9 cache_hit_rate
unit tests (was 4) + 2 compression_ratio (unchanged). New
coverage: service_tier known/unknown bucketing, C2 alarm wire,
H1 per-strategy ratio, H2 abort gate, M3 NaN/inf skip.
* Python: 27 tests (was 13). New coverage: M2 path-gated
redaction, M5 byte vs char label, wrap_rtk_metrics primitive
thread safety and validation.
`cargo fmt --check`, `cargo clippy --workspace -- -D warnings`,
`cargo test -p headroom-proxy --lib` (221 passed) and the
integration_metrics + integration_compression +
integration_volatile_detector + integration_cache_control +
integration_cache_drift + integration_responses +
integration_bedrock_metrics test files all green. Full
`cargo test --workspace` deferred — disk pressure during the
agent session left insufficient space for the linker to write
the full integration test artifacts; runs that did fit all
passed. `make ci-precheck` deferred for the same reason.
ruff check + ruff format + mypy headroom/proxy/request_logger.py
+ headroom/cli/wrap_rtk_metrics.py + headroom/proxy/prometheus_metrics.py
green.
|
||
|
chopratejas
|
5f264a5329 |
fix(observability): wire Phase G PR-G3 RTK + proxy metrics (H-blocker)
Phase H ("retire the Python proxy") needs cache-hit-rate parity
between the Rust and Python proxies during canary. This PR lands
the per-invocation RTK metrics and the proxy-side observability
surface that the canary gate depends on.
Rust observability:
- `proxy_cache_hit_rate_per_session{provider}` — histogram, emitted
per session at SSE state-machine close (Anthropic message_delta,
OpenAI Chat final usage chunk, OpenAI Responses response.completed).
The Phase H canary gate metric.
- `proxy_compression_ratio_by_strategy{strategy, content_type}` —
histogram; one sample per shrunk block.
- `proxy_compression_rejected_by_token_check_total{strategy}` —
counter for tokenizer-validated rejections.
- `proxy_passthrough_bytes_modified_total{path}` — counter (must
stay 0 outside compression hot path; alarmable via PromQL rate).
- `proxy_rate_limit_remaining_{requests,tokens,input_tokens,output_tokens}{provider}` —
gauges populated from anthropic-ratelimit-* / x-ratelimit-* headers.
- `proxy_service_tier_count_total{tier}` and
`proxy_response_status_count_total{status}` — counters for
Responses-API outcome telemetry.
- `proxy_image_generation_call_log_redacted_total` — counter.
- `wrap_rtk_invocations_total{tool}` and
`wrap_rtk_tokens_saved_per_session` — RTK metrics exposed via
the proxy's /metrics scrape so wrap-side tail can increment
through one observability surface.
All metric names and label keys live in a single
`observability/metric_names.rs` constants module per realignment
build-constraint "configurable". Bounded label vocabularies
(service_tier, response_status, provider) are defined alongside.
Python (P4-45):
- `headroom/proxy/request_logger.py` — base64-image payloads in
request/response logs over 1024 bytes are replaced with
`<image:base64-redacted bytes=N>` placeholders. Walks Anthropic
source.data and OpenAI data URLs. No regexes — substring +
density heuristic.
Tests:
- `crates/headroom-proxy/tests/integration_metrics.rs` — 6 tests
covering cache-hit-rate, compression-ratio, passthrough-bytes,
service-tier, response-status, and rate-limit-snapshot.
- `tests/test_image_log_redaction.py` — 13 tests for the Python
redaction helper.
- Existing tests: 1100+ Rust + 76 Python regression checks green.
Docs:
- `docs/observability.md` — metric catalogue + PromQL queries.
- `docs/rtk-architecture.md` — locks the wrap-CLI-only decision so
future contributors don't relitigate proxy-side RTK.
No silent fallbacks: zero-denominator cache-hit-rate logs and
skips rather than synthesising 0.0. Unparseable rate-limit headers
stay None rather than coerced to 0. Missing upstream JSON fields
log + skip emit rather than fabricating data.
|
||
|
Tejas Chopra
|
294df2b894
|
Merge pull request #403 from chopratejas/realign-F2_2-policy-tuning
fix(proxy): F2.2 — per-mode CompressionPolicy tuning fields |
||
|
Tejas Chopra
|
eaf5980b4a | fix: stabilize codex compression, stats, and proxy lifecycle | ||
|
Tejas Chopra
|
c6ecdc4299
|
Merge pull request #427 from mbachaud/fix/vertex-dead-code-and-body-limit
fix(vertex,bedrock): remove dead handle_raw_predict, honour X-Forwarded-Proto, cap Bedrock body size |
||
|
SwiftWing21
|
b784f400c1 |
fix(bedrock): use is_char_boundary loop instead of floor_char_boundary (MSRV 1.80)
floor_char_boundary was stabilised in Rust 1.91; headroom's MSRV is 1.80. Walk back from byte 64 manually — UTF-8 codepoints are at most 4 bytes so this loop runs at most 3 times in the worst case. |
||
|
SwiftWing21
|
28b2bacdf6 |
fix(vertex,bedrock): remove dead handle_raw_predict, honour X-Forwarded-Proto, cap Bedrock body size
Three related proxy hygiene fixes: #417 — delete handle_raw_predict from vertex/raw_predict.rs The dispatcher (handle_vertex_predict_dispatch in vertex/mod.rs) calls forward_vertex_request directly. handle_raw_predict was never wired into the router and is unreachable code. Deleting it removes 80 lines of dead logic and eliminates confusion for new contributors. #418 — honour X-Forwarded-Proto in forward_vertex_request build_forward_request_headers received a hardcoded literal 'http' for the forwarded protocol. Proxies deployed behind a TLS load balancer would emit X-Forwarded-Proto: http even for HTTPS upstream connections. Now reads the incoming X-Forwarded-Proto header and falls back to 'http' only when the header is absent. #416 — apply DefaultBodyLimit to Bedrock routes in proxy.rs Bedrock handlers use axum's Bytes extractor, which respects DefaultBodyLimit (default 2 MiB). All other routes buffer the body manually and apply config.max_body_bytes (default 100 MiB). Adding .layer(DefaultBodyLimit::max(state.config.max_body_bytes)) to the Bedrock router aligns the cap across providers. Closes #416, #417, #418 |
||
|
SwiftWing21
|
42ff8afa90 | fix(bedrock): apply rustfmt to header_value_preview tests | ||
|
SwiftWing21
|
82468e4e99 |
fix(bedrock): use floor_char_boundary to avoid UTF-8 slice panic in header preview
header_value_preview in eventstream_to_sse.rs used a raw byte-slice (&s[..64]) to truncate long header strings for log output. If byte index 64 landed inside a multi-byte codepoint (e.g. 63 ASCII chars followed by é or an emoji), Rust panics at runtime. Replace with floor_char_boundary(64) which returns the largest valid char boundary ≤ 64 without scanning the whole string. Two regression tests added: - truncates_at_char_boundary: 63 ASCII + é → must not panic, must end with … - exact_boundary_not_truncated: 64-byte ASCII string is returned unchanged Fixes #415 |
||
|
chopratejas
|
5b38cbf8a7 |
fix(transforms): F2.2 c2/3 — wire toin_read_only gate + extend policy_selected log
Wires the F2.2 ``toin_read_only`` field through the only consumer where it's load-bearing (TOIN write surface) and extends the proxy's structured ``policy_selected`` log event with all three F2.2 fields so the bake dashboard has per-mode observability. Wiring (gates only TOIN writes — compression itself still runs): - headroom/transforms/smart_crusher.py: capture kwargs["compression_policy"] onto self._runtime_compression_policy at the start of apply(). _record_to_toin returns early when the policy says toin_read_only=True. Direct crush() / crush_array_json() callers don't go through apply() and keep pre-F2.2 write-enabled behaviour (no auth context for non-proxy callers). - headroom/transforms/content_router.py: same one-liner in apply(), same gate in _record_to_toin. Mirrors the existing _runtime_target_ratio / _runtime_kompress_model pattern. Telemetry: - crates/headroom-proxy/src/proxy.rs: extend the policy_selected structured log with volatile_token_threshold, max_lossy_ratio, and toin_read_only. F2.2 bake telemetry can now observe all five fields on every request — load-bearing for the F2.2-followup tune decision since volatile_token_threshold and max_lossy_ratio are plumbed-but- unconsumed today and the log is the only signal that the values are flowing correctly. Plumbed-but-unconsumed (deliberate; flagged in PR body): - volatile_token_threshold — the volatile detector in cache_aligner.py is shape-based, not token-count-based; wiring it forces a detector refactor outside F2.2 scope. - max_lossy_ratio — distinct from the caller-driven target_ratio kwarg in content_router.py; gating lossy paths on a policy cap is F2.2- followup once telemetry decides whether to gate or just observe. Tests (tests/test_compression_policy_toin_gate.py): - 7 tests covering the gate. SmartCrusher tests skip when the headroom._core Rust wheel isn't installed (matches the existing test_smart_crusher_rust_parity.py pattern); the 3 ContentRouter tests exercise the gate without the Rust dependency. CI's ci-precheck-python target runs scripts/build_rust_extension.sh before pytest so all 7 will run in the gate. Refs: F2.1 (#400) |
||
|
chopratejas
|
0546795547 |
fix(proxy): rustfmt drift in live_zone_anthropic imports (F2.1 c2 followup)
c2 (
|
||
|
chopratejas
|
0fef428f1f |
fix(proxy): wire CompressionPolicy through handlers + flip default enabled (F2.1 c5/5)
Final commit of F2.1. Flips the CliArgs default for the auth-mode policy enforcement flag from Disabled to Enabled (matching the Rust + Python defaults), and wires resolve_policy() through the Python Anthropic and OpenAI chat handlers so the live TransformPipeline sees the correct CompressionPolicy on every request. Changes: - crates/headroom-proxy/src/config.rs: CliArgs default flips to AuthModePolicyEnforcement::Enabled. Config::for_test stays Disabled so the existing test corpus is unaffected. - headroom/proxy/handlers/anthropic.py: resolve_policy() called once per request just before pipeline.apply(); compression_policy= passed through to TransformPipeline. - headroom/proxy/handlers/openai.py: same pattern in both branches of the chat completion path. - headroom/transforms/compression_policy.py: added is_enforcement_enabled() reading HEADROOM_PROXY_AUTH_MODE_POLICY_ENFORCEMENT (matching Rust) and resolve_policy() — the single public entry point handlers call. Subscription-classified requests now skip CacheAligner, addressing the cache-instability complaints in #327 / #388. PAYG and OAuth remain on the aggressive path until F2.2 telemetry says otherwise. Refs: F2.1 |
||
|
chopratejas
|
027b203c23 |
fix(proxy): add auth_mode_policy_enforcement feature flag (F2.1 c3/6)
Phase F2.1, commit 3 of 6. Behind a default-disabled gate, no behaviour change until commit 6 flips the default. What lands: - New `AuthModePolicyEnforcement` clap-friendly enum (`Enabled`/`Disabled`) in `config.rs`. Pattern follows `CacheControlAutoFrozen` and `StripInternalHeaders`: ValueEnum derive, snake_case rename, env var `HEADROOM_PROXY_AUTH_MODE_POLICY_ENFORCEMENT`. - New `Config::auth_mode_policy_enforcement` field, wired through `from_cli` and `for_test`. `for_test` defaults to `Disabled` so every existing test stays green without per-test opt-out (F2.1's own integration tests opt-IN per case). - Proxy entry in `proxy.rs::proxy_request` reads the flag and gates the policy derivation: `Enabled` -> `CompressionPolicy::for_mode(auth_mode)` (the real per-mode value); `Disabled` -> forces `CompressionPolicy::for_mode(AuthMode::Payg)`. Either way the policy is stored in extensions; c4/6's dispatcher gate reads from there without re-checking the flag. - The `policy_selected` debug log now also emits the `enforcement` field so dashboards can split "policy is PAYG because mode is PAYG" from "policy is PAYG because the flag is off." Why a flag rather than landing the behaviour change directly: F2.1 ships in 6 commits. c1-c5 are wiring + Python parity; c6 is the single commit that flips behaviour for default users. Reviewers can land c1-c5 freely without worrying about subscription users seeing the new path before we have telemetry to validate it. An operator in a dogfood env can opt in early via the env var to generate that telemetry. Rollback story: flip env var back to `disabled` (instant if hot- reload available; else redeploy). c6/6 is the only commit that needs `git revert` to roll back if the default flip surfaces a regression. Note on conventional-commit prefix: this is a Rust-migration internal-phase commit; using `fix:` rather than `feat:` so semantic-release does not bump the package minor version on the phase plumbing. The user-visible behaviour change in c6/6 is the appropriate place for `feat:` if anywhere — and even there, arguably still `fix(proxy):` because the change addresses an existing user complaint (#327/#388 cache instability) rather than adding a brand-new capability. Verified: `cargo check -p headroom-proxy` clean. No tests modified because the default-disabled path is identical to current main behaviour - every existing test continues to assert what it asserted before. c4/6 adds the integration tests that exercise the enforcement-on path. |
||
|
chopratejas
|
948c8f2069 |
fix(proxy): plumb CompressionPolicy through proxy + dispatchers (F2.1 c2/6)
Phase F2.1, commit 2 of 6. No behaviour change — wiring only. Three things land: 1. **Proxy entry derives the policy alongside auth_mode** (proxy.rs). Both go into `req.extensions_mut()` so downstream stages can read either without re-classifying. New structured log event `policy_selected` fires once per request with auth_mode + live_zone_only + cache_aligner_enabled — gives F2.2 the bake-time data it'll need to tune. Pure addition; no existing log was removed. 2. **OpenAI live-zone dispatchers stop hard-coding `AuthMode::Payg`.** `compress_openai_chat_request` and `compress_openai_responses_request` already received `auth_mode` from the proxy caller (F1's plumbing), but the dispatcher invocation hard-coded `AuthMode::Payg` and ignored the parameter — c.f. the `_auth_mode` underscore-prefixed identifier in `compress_openai_chat_live_zone` upstream. Now the classified mode is forwarded via `auth_mode.into()` (uses the `From<>` impl added in c1/6 to bridge the two `AuthMode` enums). 3. **Anthropic live-zone dispatcher gets the same fix.** `compress_anthropic_request` was the closest to right — it threaded auth_mode all the way to `compress_anthropic_live_zone` already — but the inner call still had `AuthMode::Payg` hard-coded. Now it uses `auth_mode.into()` for symmetry and so the F2.1 dispatcher- level gate (added in c4/6) reads the real mode. The dispatcher is unchanged in F2.1: it still runs the same compression for every mode. The point of c2/6 is that *if* a future commit (c4/6 in this PR, or anything post-F2.1) gates on the mode-aware policy, the wiring is already in place. PAYG behaviour is byte-for-byte identical because every mode currently dispatches identically. Two tiny cleanups: drop the `AuthMode` import from the three live-zone dispatcher files now that the value flows in via `.into()` (the `RequestAuthMode` alias remains, since it's still in the function signatures). Behaviour-change risk in c2/6: zero. Verified by running the full `cargo test -p headroom-proxy` suite; existing tests pass without modification because they're already shape-compatible (tests pass `AuthMode::Payg` directly into the inner dispatcher; this commit only changes how the *outer* compress_* fns invoke that dispatcher). |
||
|
chopratejas
|
9112fed937 |
fix: PR-E2 recursive JSON Schema key sort (Phase E)
Recursively sort JSON Schema object keys inside each tool's schema so cache hits no longer depend on SDK-side serializer key-emission order (some sort, some preserve insertion, some hash-randomize). Wired into all three live-zone walkers, hooking the per-provider schema location: - Anthropic: `tool["input_schema"]` - OpenAI Chat: `tool["function"]["parameters"]` - OpenAI Responses: `tool["function"]["parameters"]` Same auth-mode gate as PR-E1 (PAYG only). NO marker check — the `cache_control` marker lives on the tool object itself, not inside the schema, so sorting schema keys never moves the marker. PR-E2 therefore runs even on tools that PR-E1 had to skip due to a present marker; the integration test pins this behaviour. Array semantics preserved: `oneOf`, `anyOf`, `allOf`, `prefixItems` and any other ordered JSON Schema array keep customer order; only object keys move. Idempotent — sorting an already-sorted schema yields byte-identical bytes (workspace `preserve_order` feature pins `serde_json::Map` emission to insertion order). Tests: unit tests for nested keys, oneOf preservation, deep nesting, and idempotency; integration tests boot the real proxy and assert PAYG -> sorted at every level, OAuth -> SHA-256 byte- equal, and PAYG-with-marker -> E1 skipped but E2 still runs. |
||
|
chopratejas
|
4a3b76bcc8 |
fix: PR-E1 tool array deterministic sort (Phase E)
Sort `tools[]` alphabetically by name on the way out so cache hits no
longer depend on the customer-side iteration order (commonly hash-
randomized via `set()` / `dict`). Mutates request bytes only when:
1. Auth mode is PAYG (`headroom_core::auth_mode::classify`).
2. No tool already carries a `cache_control` marker (reordering
would shift cache scope and silently void customer intent).
Every gate skip emits a structured `e1_skipped` event with `reason =
auth_mode | marker_present` so dashboards can see policy adoption.
Wired into all three live-zone walkers — Anthropic `/v1/messages`,
OpenAI `/v1/chat/completions`, OpenAI `/v1/responses` — plus the
Bedrock invoke + invoke-streaming entry points. Each passes
`auth_mode` (already pre-classified by Phase F PR-F1 middleware)
into the dispatcher so the gate evaluates without re-classifying.
Sort key uses `tool["name"]` (Anthropic) or `tool["function"]["name"]`
(OpenAI). Unnamed tools (rare; malformed inputs only) fall back to
MD5 of canonical-JSON serialization for a stable in-process key —
collision odds are astronomically small and `Vec::sort_by` is stable.
Tests: unit tests for sort + marker detection + idempotency + the
permutation property; integration tests boot the real proxy in front
of a wiremock upstream and assert PAYG -> sorted, OAuth/Subscription/
marker -> byte-equal passthrough (SHA-256).
|
||
|
chopratejas
|
573543fce7 |
fix: PR-E4 OpenAI prompt_cache_key auto-injection (Phase E)
OpenAI exposes `prompt_cache_key` to pin prefix-cache lookups to a tenant-stable identity (preventing org-wide cache collisions). Most clients don't set it. This PR auto-derives one from the request's structural prefix `(model, system, tools)` and injects it on PAYG OpenAI requests where the customer has not provided their own value. Universal safety contract: - Auth-mode gate: only AuthMode::Payg bodies are mutated. OAuth and Subscription requests pass through byte-equal (preserves Phase A passthrough invariant). Both gates emit `e4_skipped` events. - Customer-set values win: `prompt_cache_key` already present → skip injection. Empty strings count as absent. - Idempotent: same `(model, system, tools)` always derives the same key, so re-running yields identical bytes. Key derivation: `hex(sha256(model || sha256(system) || sha256(tools)))[..32]` — 128 bits of collision resistance, 32 hex chars on the wire. User/assistant message content is deliberately excluded (they vary per turn; including would defeat caching). Observability: every skip emits `e4_skipped` with a stable reason (`auth_mode` / `key_present` / `not_an_object`); every successful injection emits `e4_applied` with only the first 8 hex chars of the key (full key is identifying material — never logged). Hook point: `forward_http` in `crates/headroom-proxy/src/proxy.rs`, between the live-zone dispatcher's body decision and the upstream forward. Auth-mode is already classified at request entry. Affects pre-existing dispatcher byte-fidelity tests (chat completions, responses, responses streaming) — they previously asserted byte-equality with no auth header (default PAYG). Updated those tests to send an OAuth bearer so they keep their byte-equality intent independent of E4. The E4 byte-mutation behaviour has its own test matrix in `integration_e4_openai_cache_key.rs`. Files added: - `crates/headroom-proxy/src/cache_stabilization/openai_cache_key.rs` - `crates/headroom-proxy/tests/integration_e4_openai_cache_key.rs` Files modified: - `crates/headroom-proxy/src/cache_stabilization/mod.rs` — `pub mod openai_cache_key;` (only shared file with parallel E1/E2/E3/E6 PRs) - `crates/headroom-proxy/src/proxy.rs` — call site + helper - `crates/headroom-proxy/Cargo.toml` — promote sha2 to runtime dep - 3 integration test files — auth-mode plumbing for byte-equality invariants |
||
|
chopratejas
|
8672d5c326 |
fix: PR-E3 Anthropic cache_control auto-placement (Phase E)
Auto-place a single ephemeral cache_control marker on the last tool
definition for PAYG-classified Anthropic requests when the customer
has not placed any markers. Hand-rolled SDK callers and smaller
agents (Aider/Continue/curl) get prompt-cache hits without learning
Anthropic's marker API.
Safety contract:
1. Auth-mode (caller-side, F1 classify): PAYG only. OAuth and subscription requests pass through byte-equal — mutating their bytes risks looking like cache-evasion to upstream.
2. Customer-placement-wins: walks system (array form), messages[].content (array form), and tools[] top-level. Any pre-existing marker -> skip with reason=marker_present.
3. Idempotency: re-running on a body that already has our marker falls into gate (2).
First-ship policy: place ONE marker on the last tool. The system/message-history/4th slots are documented but require production telemetry to enable.
Bedrock invoke + invoke-streaming hard-code OAuth so AWS SigV4-signed requests never get auto-placed (Bedrock is an IAM channel, not PAYG).
Observability: tracing::info! event=e3_applied / event=e3_skipped (reason in {auth_mode, marker_present}) so dashboards can confirm the gates fire as designed.
Files added: cache_stabilization/anthropic_cache_control.rs (module + 15 unit tests); tests/integration_e3_anthropic_cache_control.rs (5 integration tests covering all three gates).
Files modified: cache_stabilization/mod.rs; compression/live_zone_anthropic.rs (new auth_mode parameter on compress_anthropic_request); proxy.rs; bedrock/invoke.rs; bedrock/invoke_streaming.rs.
|
||
|
chopratejas
|
c10a2195af |
fix(proxy): PR-D4 native Vertex publisher path + ADC bearer auth
Adds a Rust-native Vertex AI publisher route ahead of the LiteLLM
Python converter (which dropped `thinking`, `redacted_thinking`,
`document`, `image`, `server_tool_use`, `mcp_tool_use` block kinds —
the P4-37 / P4-38 bug). After this PR the Vertex `:rawPredict` and
`:streamRawPredict` calls survive byte-equal upstream and benefit
from the live-zone Anthropic dispatcher (PR-B-series) running over
the body — same behaviour as `/v1/messages`.
New module `crates/headroom-proxy/src/vertex/`:
- `mod.rs` — single dispatch handler at the
`/v1beta1/.../models/:model_action` route. Splits the trailing
`:<verb>` segment with `str::rsplit_once(':')` (no regex) and
flips an `attach_sse_tee` flag to dispatch to the streaming or
non-streaming arm. Both verbs share one axum route shape because
matchit can't distinguish two patterns that overlap on a
parameter.
- `envelope.rs` — `VertexEnvelope` parser. Confirms
`anthropic_version` present + `model` field absent (the two
fingerprints of the Vertex envelope vs `/v1/messages`).
- `adc.rs` — `TokenSource` trait + `GcpAdcTokenSource` (production,
`gcp_auth` 0.12) + `StaticTokenSource` (tests). Caches tokens
with a 60s refresh-ahead-of-expiry window. Emits structured
`event = "vertex_adc_token_refreshed"` per refresh.
- `raw_predict.rs` — POST handler + shared `forward_vertex_request`.
Buffers body, parses envelope, runs live-zone Anthropic
compression, fetches ADC bearer, attaches
`Authorization: Bearer <token>` (overwrites client-supplied
Authorization header), forwards. SSE telemetry tee for the
streaming verb reuses PR-C1's `AnthropicStreamState` directly
(Vertex streams plain SSE, unlike Bedrock's binary EventStream).
- `stream_raw_predict.rs` — module-level docs + alias to the
shared dispatcher (the streaming-vs-non-streaming difference is
one boolean flag inside the shared forwarder).
Modifications:
- `proxy.rs::build_app` — registers the single Vertex route.
- `proxy.rs::AppState` — new `vertex_token_source: Arc<dyn TokenSource>`
field. Production constructs `GcpAdcTokenSource` lazily (no GCP
call until first `bearer()`); tests inject `StaticTokenSource`
via the new `AppState::with_token_source` helper.
- `config.rs` — adds `--vertex-region` / `HEADROOM_PROXY_VERTEX_REGION`
(default `us-central1`, observability tag only — the upstream URL
is `--upstream`) and `--vertex-adc-scope` /
`HEADROOM_PROXY_VERTEX_ADC_SCOPE` (default `cloud-platform`).
- `Cargo.toml` (workspace + proxy) — adds `gcp_auth = "0.12"` and
`async-trait = "0.1"`.
- `tests/common/mod.rs` — `start_proxy_with_state` accepts both
config + state customizers; `install_static_token_source` helper
for tests.
`crates/headroom-proxy/tests/integration_vertex_raw_predict.rs` —
all five tests pass:
1. `native_envelope_round_trip_byte_equal` — Vertex-shape body
(with `anthropic_version`, no `model`) round-trips SHA-256
byte-equal upstream.
2. `adc_bearer_token_signed_correctly` — `Authorization: Bearer
<static-test-token>` reaches upstream verbatim and OVERWRITES a
client-supplied Authorization header.
3. `thinking_block_preserved` — request with `thinking` (incl.
signature) + `redacted_thinking` (incl. opaque `data`) blocks
round-trips byte-equal even with `LiveZone` compression mode
enabled. This is the P4-37 / P4-38 teeth.
4. `stream_raw_predict_sse_handled` — `:streamRawPredict` proxies
an Anthropic SSE response (full `message_start` →
`content_block_delta` → `message_stop` sequence) back to the
client without corruption; SSE content-type preserved end-to-end;
bearer attached.
5. (bonus, no-silent-fallback contract)
`adc_failure_returns_5xx_no_silent_forward` — when the token
source returns `Err`, the proxy returns 5xx and never reaches
upstream. Verifies the `event = "vertex_adc_fetch_failed"`
error path.
Workspace: `cargo test --workspace` green; `cargo clippy --workspace
-- -D warnings` clean; `make ci-precheck` passes.
- No silent fallbacks: ADC failure → structured 5xx, never an
unauthenticated forward.
- No hardcodes: every knob (region, ADC scope, upstream URL) is
CLI-flag + env-var configurable.
- No regexes: axum path parameters + `str::rsplit_once` only.
- Comprehensive structured logs: `event` field on every decision
point — `vertex_envelope_parsed`, `vertex_envelope_invalid`,
`vertex_compression_skipped`, `vertex_compression_applied`,
`vertex_adc_token_refreshed`, `vertex_adc_fetch_failed`,
`vertex_streaming_pipeline_active`, `vertex_sse_stream_closed`,
`vertex_forwarded`, `vertex_unknown_verb`, etc.
- Performant: no body clone; ADC token cached + refreshed
ahead-of-expiry, not fetched per request.
- Comprehensive tests: realistic Anthropic block content
(signature payload, redacted_thinking opaque blob) in
`thinking_block_preserved`.
The local `gcloud auth application-default print-access-token`
returns no credentials, so manual validation against a real Vertex
endpoint is not possible in this PR. Follow-up: the user runs
`gcloud auth application-default login` once and exercises a live
Vertex request — should be a no-code-change check.
PR-D1 (Bedrock native) is running concurrently and will land its
own envelope module at `crates/headroom-proxy/src/bedrock/envelope.rs`.
The two envelope modules are intentionally siblings (not a shared
trait) — the shapes differ (Bedrock has a different
`anthropic_version` value, no `model` field, AWS SigV4 instead of
GCP ADC), and a premature shared abstraction would obscure the
provider-specific contracts. Whichever PR merges second rebases
without conflict.
Retires P4-38 (and the Vertex parts of P4-39); marketplace BYOC
pitch (per project memory) gets one more native provider.
|
||
|
chopratejas
|
ce37940d17 |
fix: PR-E6 cache-bust drift detector telemetry (Phase E)
Per-session SHA-256 fingerprint of the cache hot zone (system / tools / first 3 messages) with structured-log emission on drift. Detector is read-only: never mutates request bytes, preserves the Phase A passthrough invariant. Surfaces invisible cache busts (system prompt edited mid-session, tools reshuffled, early message changed) without rewriting them. * crates/headroom-proxy/src/cache_stabilization/drift_detector.rs: StructuralHash (system, tools, early_messages digests), compute_structural_hash, observe_drift, derive_session_key, DriftState (LRU bounded to 1000 sessions in production). * Session keys derive from Authorization / x-api-key / client IP / (IP, user-agent). Bearer tokens and API keys are SHA-256 hashed before they ever reach the log line; the raw secret is never logged. * Wired into forward_http after the body is buffered, before the compression dispatcher runs. Skips paths whose wire shape is not Anthropic / OpenAI Chat / OpenAI Responses. * AppState gains drift_state: DriftState. Bedrock unit-test literal-construction sites updated. * 14 unit tests + 1 integration test covering first-request, no-drift, per-dimension drift, multi-dim drift, LRU eviction, non-mutation invariant, and bearer-token-never-logged. Adds lru = "0.12" and promotes sha2 = "0.10" to a normal dependency on headroom-proxy. |
||
|
chopratejas
|
d8aae382b0 |
fix: PR-E5 volatile-content detector + customer warning (Phase E)
Adds an observation-only detector that scans inbound LLM request
bodies for content that busts prompt-cache hits and emits one
structured WARN log per finding. Strictly read-only — never
mutates the request, the Phase A bytes-in==bytes-out invariant
still holds.
Detection patterns (no regex, per Realignment build constraints):
- ISO-8601 timestamps (byte-position check at 4=- 7=- 10=T 13=: 16=:)
- UUID v4 (36 chars, hyphens at 8/13/18/23, version nibble 4 at pos 14,
RFC 4122 §4.4 variant nibble at pos 19)
- ID-named JSON keys (request_id / trace_id / session_id /
correlation_id) with non-empty values
Scope:
- Anthropic body shape: system, messages[].content (string or blocks),
tools[].description, tools[].input_schema (recursive).
- OpenAI body shape: messages[].content, tools[].function.description,
tools[].function.parameters (recursive).
- Other paths (Bedrock / Vertex etc.) deferred to Phase E follow-up.
Findings capped at 10 per request; sample truncated to 80 bytes
(UTF-8 boundary safe) to avoid logging bulk customer data.
Files:
- New crates/headroom-proxy/src/cache_stabilization/{mod,volatile_detector}.rs
- crates/headroom-proxy/src/lib.rs: pub mod cache_stabilization
- crates/headroom-proxy/src/proxy.rs: detector hook in the
buffered-body branch, before the compression dispatcher
- New tests/integration_volatile_detector.rs (subscriber capture
asserts the WARN line + byte-equal upstream body)
12 unit tests (timestamp, UUID, ID-field nesting, stable content,
cap, non-mutation, ApiKind shape isolation, empty-ID-value guard,
RFC 3339 space separator, non-v4 UUID rejection, UTF-8 truncate
safety, endpoint mapping) and 1 integration test (warn emitted
+ upstream body byte-equal). cargo fmt / clippy --workspace -D
warnings / workspace test all pass; full make ci-precheck green.
|
||
|
chopratejas
|
90ef66213d |
fix(proxy): PR-D3 Bedrock observability + auth-mode integration
Phase D close. Adds the operator-facing observability surface that
PRs D1 (native invoke) and D2 (streaming EventStream) deferred, and
wires the Phase F PR-F1 auth-mode classifier into the Bedrock route
so downstream cache/compression policy gates have something to read.
Changes
-------
* New `bedrock::auth_mode_layer` middleware. Classifies every
inbound Bedrock request via F1's `classify`, coerces the result
to `AuthMode::OAuth` per the Bedrock policy matrix (SigV4 IAM is
OAuth-equivalent), and stores the resolved value in
`request.extensions()` so PR-F2/F3 can read it without
re-classifying. Mismatches are logged at WARN with
`event=bedrock_auth_mode_unexpected` — no silent coercion.
* New `observability` module with three Prometheus families:
- `bedrock_invoke_count_total{model, region, auth_mode}` (counter)
- `bedrock_invoke_latency_seconds{model, region}` (histogram)
- `bedrock_eventstream_message_count_total{model, region, event_type}`
(counter)
Registered lazily via `OnceLock` so per-request work is just
`inc_with_label_values` / `observe`. Latency observed via an
RAII `LatencyGuard` so every error path is instrumented; a
future regression that adds a new return path can't drop the
observation.
* New `GET /metrics` endpoint serves the registry in Prometheus
text format. Mounted unconditionally — no feature flag gate — so
scrape works regardless of which provider routes are mounted.
* Bedrock invoke + invoke-streaming handlers now extract
`Extension<AuthMode>`, log it in their entry breadcrumbs
(`event=bedrock_invoke_received`, `event=bedrock_invoke_streaming_received`),
and pass `model`/`region` into `translate_stream` so per-message
metrics carry the right labels.
* Operator docs at `docs/bedrock.md`: AWS credential chain,
region/endpoint config, supported model IDs (`anthropic.*`
literal-match — no regexes), compression behaviour, sample
PromQL queries, structured-log correlation, rollback path.
Tests added (6, all green)
--------------------------
Auth-mode (`integration_bedrock_authmode.rs`):
1. `bedrock_classified_as_oauth` — empty headers → OAuth in
extensions.
2. `oauth_policy_passthrough_prefer` — body byte-equal upstream;
no auto cache_control / prompt_cache_key injected.
Metrics (`integration_bedrock_metrics.rs`):
3. `metrics_increment_per_invoke` — 3 invokes → counter=3 with
correct labels.
4. `metrics_observe_latency` — 1 invoke → histogram count=1,
sum>0.
5. `eventstream_metrics_per_message_type` — 5 chunks → counter=5
with `event_type=chunk`.
6. `metrics_endpoint_serves_scrape` — `/metrics` returns 200,
`text/plain`, all three metric families' HELP/TYPE lines
present.
Each metrics test owns a unique (model, region) tuple so the
global `prometheus` registry — shared across parallel tests in
the same binary — gives each test isolated label rows. Without
isolation, parallel tests cross-contaminate counters.
Constraints honoured
--------------------
* No silent fallbacks — auth-mode coercion is logged at WARN.
* No hardcodes — region from `--bedrock-region`, model from axum
path parameter.
* No regexes — vendor prefix is literal `anthropic.`.
* Comprehensive structured logs — every metric increment paired
with `tracing::debug!` carrying the same labels for incident
correlation.
* Performant — `OnceLock`-cached descriptors, RAII guard, total
D3 overhead well under 1us per request.
* Cardinality bounded — labels driven by config + bounded enums,
never by user-controlled bytes.
Live cloud validation deferred
------------------------------
The wiremock-backed integration tests are the canonical correctness
gate for D3. A real Bedrock smoke test requires `bedrock:InvokeModel`
permissions in the developer's AWS account and is documented in
`docs/bedrock.md` — both D1 and D2 hit sandbox permission issues
trying this path; D3 follows the same convention.
Stacked on
----------
PR #364 (D1 native invoke), PR #365 (D2 streaming EventStream),
PR #366 (F1 classifier helper). Merge those first; this PR will be
rebased onto main once they land.
|
||
|
chopratejas
|
6f2c0a8400 |
fix(ci): rustls-everywhere — eliminate openssl-sys from build tree
# Root cause of the wheel-build cascade We have shipped 5 release-pipeline hot-fixes in 12 hours, each addressing a different symptom of the same architectural problem: 1. PR #363 — npm artifact downloads + tried `yum openssl-devel` 2. PR #367 — vendored OpenSSL in `headroom-proxy` + dropped Intel mac 3. PR #369 — Debian-cross perl install (`perl` not `libipc-cmd-perl`) 4. PR #370 — moved `openssl/vendored` from headroom-proxy to headroom-py 5. (this PR) — ELIMINATE OpenSSL entirely Each fix exposed a different missing system package or feature flag in a different build surface (manylinux x86_64 vs aarch64-cross-Debian vs macOS Intel vs e2e/wrap Dockerfile vs e2e/init Dockerfile vs main Dockerfile vs devcontainer). We were playing whack-a-mole because every Cargo dep change to the OpenSSL surface required matching system-package updates in 6+ different Dockerfiles and workflows, and the PR-level CI didn't exercise all of them. # Why this PR is the structural fix `fastembed` exposes clean rustls feature flags: - `hf-hub-rustls-tls` (replaces default `hf-hub-native-tls`) - `ort-download-binaries-rustls-tls` (replaces default `…native-tls`) By disabling fastembed's default features and enabling the rustls variants explicitly, we remove `native-tls` (and therefore `openssl-sys`, `openssl`, `openssl-src`, perl modules, OpenSSL build-time deps, vendored OpenSSL ~30s build cost) from the entire workspace dep tree. Verified locally: $ cargo tree -p headroom-py -i openssl-sys error: package ID specification `openssl-sys` did not match any packages $ cargo tree -p headroom-py -i native-tls error: package ID specification `native-tls` did not match any packages $ cargo build --release -p headroom-py Finished `release` profile [optimized] target(s) in 25.57s (Down from 1m+ with vendored OpenSSL.) # Cleanups enabled by this change - crates/headroom-py/Cargo.toml — dropped the `openssl/vendored` workaround from PR #370. - crates/headroom-proxy/Cargo.toml — same dep removed. - e2e/wrap/Dockerfile — dropped `yum install openssl-devel pkgconfig perl-IPC-Cmd`. Comment retained explaining why. - e2e/init/Dockerfile — same. - Dockerfile (main) — dropped `pkg-config libssl-dev` from apt-get. - .devcontainer/Dockerfile — dropped `pkg-config libssl-dev`. - .github/workflows/release.yml — removed the entire before-script-linux block (perl install probe + multi-package-manager dispatch + fail-loud assertion). No longer needed. # Regression gate Three new structural tests in tests/test_release_workflows.py: - test_no_openssl_sys_in_wheel_build_tree — runs `cargo tree -p <crate> -i openssl-sys` for headroom-py / headroom-proxy / headroom-core. If openssl-sys reappears (a future native-tls enabler creeping in via a new dep), this fails AT PR TIME with an actionable message. - test_no_native_tls_in_wheel_build_tree — same shape, native-tls is the proximate cause. - test_fastembed_uses_rustls_features — checks the Cargo.toml so a future "let me bump fastembed and forget the features" doesn't silently re-introduce OpenSSL. Plus two cleanup gates: - test_dockerfiles_no_longer_install_openssl_devel - test_release_yml_does_not_install_openssl_or_perl_for_wheels All 13 release-workflow tests pass. `make ci-precheck` PASSED. # What this teaches us about rollouts (per user's ultrathink ask) The 5-fix cascade exposed three meta-problems: 1. PR checks don't block merges. PR #370 had docker-init-e2e, docker-wrap-e2e, docker-native-e2e all FAILED yet got merged. Branch protection should require these checks. Operator action needed (cannot fix in code). 2. Local validation is misleading. `cargo build -p headroom-py` from the workspace root used the workspace lockfile and looked green; CI did fresh resolution against headroom-py's manifest alone where the feature wasn't enabled. Lesson: verify structural invariants with `cargo tree -e features` before trusting that a build "works." 3. 6+ build surfaces with independent system-dep state. Every Cargo change required matching updates in 6 places. The structural answer (this PR) is to NOT depend on system OpenSSL at all. Where structural fixes are not possible, the answer is a single shared scripts/install-rust-build-deps.sh — but with this PR there's nothing left to install. |
||
|
Tejas Chopra
|
fb25a26180
|
Merge pull request #366 from chopratejas/realign-F1-classify-auth-mode
fix: PR-F1 classify_auth_mode helper (Phase F kickoff) |
||
|
chopratejas
|
1314842b19 |
fix(ci): vendor OpenSSL via cargo + drop x86_64 macOS from wheel matrix
The previous hot-fix (#363) addressed npm artifact downloads and added openssl-devel installs in the manylinux container, but the wheel build still fails on three of four matrix entries with three distinct errors: 1. ubuntu-x86_64 with `manylinux: auto` resolved to manylinux2014 (CentOS 7 / OpenSSL 1.0.2k). `openssl-sys 0.9` requires OpenSSL 1.1.0+ — "different version of OpenSSL was found". 2. ubuntu-aarch64 cross-compiles via `aarch64-unknown-linux-gnu-gcc` from an x86_64 manylinux container. The `yum install openssl-devel` we added installs x86_64 headers; `/usr/aarch64-unknown-linux-gnu/ include/` has no OpenSSL — "openssl/opensslv.h: No such file or directory". 3. macos-15-intel fails on `ort-sys` (transitive via the ML compression backend), which has no prebuilt ONNX Runtime binaries for `x86_64-apple-darwin`. Unrelated to OpenSSL; an upstream limitation. Why the workspace pulls openssl-sys at all: `hf-hub` (transitive via `fastembed`) hard-codes `native-tls` as a default feature. Cargo's feature unification then enables openssl-sys for the whole workspace despite our `reqwest`/`tokio-tungstenite`/`tokio-rustls` preferences. # Fix 1: vendored OpenSSL Add `openssl = { version = "0.10", features = ["vendored"] }` to `crates/headroom-proxy/Cargo.toml`. The `vendored` feature compiles OpenSSL from source as part of the cargo build — works on every target uniformly. Local build verified: cargo now pulls `openssl-src v300.6.0+3.6.2` and compiles it. ~30s extra one-time build cost. The `openssl/vendored` feature DEFEATS `OPENSSL_DIR`. We therefore remove the previous hot-fix's "Install OpenSSL (macOS)" step that exported `OPENSSL_DIR` — leaving it would silently regress to the system-OpenSSL path that broke originally. # Fix 2: pin manylinux floor to 2_28 Change x86_64-unknown-linux-gnu from `manylinux: auto` to `manylinux: 2_28` (matching aarch64 + the e2e Dockerfiles). This isn't strictly required with vendored OpenSSL — the floor is now glibc 2.28 / AlmaLinux 8 which has modern toolchain — but it removes the CentOS-7 surface entirely and matches our runtime container target. # Fix 3: drop x86_64-apple-darwin from the matrix `ort-sys 2.0.0-rc.12` has no prebuilt ONNX Runtime binaries for that target. Building ORT from source would add CMake + ~5 minutes per build. Apple Silicon macOS (`aarch64-apple-darwin`) is fully covered; Intel-mac users install from the platform-independent sdist this matrix also produces. Tracked as a follow-up: switch the ML backend to `ort-tract` or upstream a request for x86_64 macOS prebuilts. # before-script-linux: keep perl-IPC-Cmd, drop openssl-devel OpenSSL's vendored `Configure` script needs `IPC::Cmd` (without it the build fails with "Can't locate IPC/Cmd.pm"). System openssl-devel is no longer needed. # Tests 4 new regression tests gate this: - `test_headroom_proxy_vendors_openssl` - `test_build_wheels_installs_perl_ipc_cmd_for_vendored_openssl` - `test_build_wheels_does_not_set_openssl_dir` - `test_build_wheels_matrix_excludes_intel_macos` Plus the previous 7. All 11 release-workflow tests pass. `make ci-precheck` PASSED. Local `cargo build --release -p headroom-py` green. |
||
|
chopratejas
|
ca9de93cfc |
fix: PR-F1 classify_auth_mode helper (Phase F kickoff)
Add the classify_auth_mode helper that maps inbound request headers to one of three auth modes — Payg / OAuth / Subscription — at request entry. The mode is the first-class policy axis Phase F's remaining PRs (F2 cache+lossy gates, F3 TOIN per-tenant aggregation, F4 X-Forwarded-* skip) gate behavior on. Detection rules (most-specific signal wins): - Subscription UA prefix in user-agent → Subscription - Bearer sk-ant-oat-* → OAuth (Claude Pro/Max) - Bearer sk-ant-api* / Bearer sk-* → Payg - Bearer <jwt> (3 dot-segments) → OAuth (Codex/Cursor/Copilot) - Authorization present but not Bearer (AWS SigV4) → OAuth (Bedrock) - x-api-key / x-goog-api-key → Payg - Default → Payg Hard constraints met: pure function, no regex, no silent fallback (non-UTF-8 headers warn! and fall through), no hardcoded list (UA prefixes in module-scope const ready to swap for config in a follow-up). Files: - crates/headroom-core/src/auth_mode.rs (new) — Rust impl - crates/headroom-core/tests/auth_mode.rs (new) — 14 unit + 1 perf - crates/headroom-core/benches/auth_mode.rs (new) — Criterion bench - crates/headroom-core/Cargo.toml — add http dep + bench entry - crates/headroom-core/src/lib.rs — pub mod auth_mode - crates/headroom-proxy/src/proxy.rs — classify at request entry, store in extensions, log event=auth_mode_classified - headroom/proxy/auth_mode.py (new) — Python port (parity) - headroom/proxy/handlers/anthropic.py — wire into messages handler - headroom/proxy/handlers/openai.py — wire into chat + responses - tests/test_auth_mode.py (new) — 23 Python parity tests - docs/auth-modes.md (new) — detection rules + how-to-extend Tests: 15 Rust + 23 Python all green. cargo fmt + clippy + workspace tests + ci-precheck all green. Performance (criterion, M-series): - auth_mode/classify/empty: 68 ns - auth_mode/classify/payg_anthropic_api_key: 75 ns - auth_mode/classify/oauth_jwt: 182 ns - auth_mode/classify/subscription_claude_code: 81 ns All paths well under the <10us budget (~50-150x headroom). Refs: REALIGNMENT/08-phase-F-auth-mode.md PR-F1. |
||
|
chopratejas
|
66426e7b75 |
fix(proxy): PR-D2 Bedrock streaming via binary EventStream
Add the Phase D PR-D2 streaming counterpart to PR-D1's native
Bedrock InvokeModel route.
Bedrock's `/model/{id}/invoke-with-response-stream` returns
`application/vnd.amazon.eventstream` — a binary, length-prefixed,
CRC32-checksummed framing format. This PR adds an incremental
parser, an SSE translator, and the streaming POST handler.
Components:
- `bedrock/eventstream.rs` — stateful incremental EventStream
parser. Validates prelude + message CRC32 (configurable via
`--bedrock-validate-eventstream-crc`, default on). Returns
structured `ParseError` on every malformed-bytes path; never
panics. Supports all 10 AWS header value types; bytes-typed
values surfaced via `HeaderValue::Bytes`, strings via
`HeaderValue::String`.
- `bedrock/eventstream_to_sse.rs` — translator. Picks output mode
per `Accept` header: `application/vnd.amazon.eventstream` →
byte-equal passthrough; everything else (default) → SSE
translation. Each `chunk` payload becomes a canonical Anthropic
`event: <type>\ndata: <json>\n\n` SSE frame so existing
`AnthropicStreamState` telemetry runs unchanged.
- `bedrock/invoke_streaming.rs` — POST handler. Reuses D1's
`BedrockEnvelope`, live-zone compression, SigV4 signing.
Tees translated SSE frames into `AnthropicStreamState` via the
same bounded-mpsc tee pattern as `/v1/messages` — byte path
never blocks on parser readiness.
Config:
- New `--bedrock-validate-eventstream-crc` / env
`HEADROOM_PROXY_BEDROCK_VALIDATE_EVENTSTREAM_CRC` flag, default
on. Disabling logs a warn at app-build time.
Routing:
- `proxy.rs::build_app` mounts
`POST /model/:model_id/invoke-with-response-stream` only when
`enable_bedrock_native` is on (matches D1).
Failure modes (all loud; no silent fallbacks):
- CRC mismatch → `event=bedrock_eventstream_crc_mismatch` warn,
closes the stream with an SSE error frame.
- Parse error → `event=bedrock_eventstream_parse_failed` warn +
SSE error frame.
- `:message-type == exception` → `event=bedrock_eventstream_upstream_exception`
warn + SSE error frame.
- Unknown `:event-type` →
`event=bedrock_eventstream_unknown_event_type` warn, skipped.
- Missing creds / SigV4 fail → 5xx, identical to D1.
Tests added (12 total):
- 4 parser unit-style integration: byte-equal round trip, drip-feed
one-byte-at-a-time, CRC mismatch surfaces structured error,
validation-off accepts corrupt.
- 3 end-to-end: `eventstream_translated_to_sse`,
`usage_extracted_from_translated_stream`,
`client_can_choose_eventstream_or_sse`.
- 2 property tests via `proptest`: random bytes never panic the
parser (1024 cases each: bulk + drip-feed).
- 3 trivial smoke tests in unit modules
(`eventstream::tests::*`, `eventstream_to_sse::tests::*`).
Manual cloud validation:
- Not exercised — running `aws bedrock-runtime invoke-model-with-
response-stream` against the proxy in the sandbox would require
AWS API access this environment does not have. The wiremock-
served binary EventStream + property tests cover the parser
semantics and CRC validation rigorously.
Stacked on PR-D1 (#364). Will be rebased onto main once D1 lands.
|
||
|
chopratejas
|
f2d4fe39cb |
fix(proxy): PR-D1 native Bedrock InvokeModel route + SigV4
Phase D PR-D1 lands the first native Rust path for AWS Bedrock,
replacing the lossy LiteLLM Python converter for Anthropic-on-Bedrock
non-streaming requests. Eliminates part of P4-37 and P4-39.
What landed
-----------
- New crates/headroom-proxy/src/bedrock/ module:
- envelope.rs: parses the {"anthropic_version": "...", ...}
Bedrock body shape; re-emits with anthropic_version preserved
as the first key (relies on serde_json preserve_order).
- sigv4.rs: AWS SigV4 signer wrapping the aws-sigv4 crate.
Forces PayloadChecksumKind::XAmzSha256 so x-amz-content-sha256
is in the canonical request, hashed over the post-compression
body bytes (the bytes that actually hit Bedrock). No silent
fallback: signing failures return 5xx with
event=bedrock_sigv4_failed.
- invoke.rs: POST handler for /model/{model_id}/invoke
(and /converse - same wire shape for anthropic.claude-*).
Detects Anthropic vendor via literal starts_with("anthropic.")
(no regex per project rule), routes Anthropic-shape bodies
through the existing compress_anthropic_request live-zone
dispatcher, then signs and forwards to the configured Bedrock
endpoint.
- Modified:
- proxy.rs: routes /model/:model_id/invoke and
/model/:model_id/converse when enable_bedrock_native is on
(default). Adds bedrock_credentials: Option<Arc<Credentials>>
to AppState.
- config.rs: new flags --bedrock-region (default us-east-1,
env HEADROOM_PROXY_BEDROCK_REGION), --bedrock-endpoint
(operator override for FIPS/VPC/test setups),
--enable-bedrock-native (default true), --aws-profile.
- main.rs: resolves AWS credentials at startup via
aws_config::defaults(BehaviorVersion::latest()). Failure logs
event=bedrock_credentials_unavailable at WARN; the handler
refuses to forward unsigned (event=bedrock_credentials_missing).
- Cargo.toml: workspace deps aws-sigv4, aws-config,
aws-credential-types, aws-smithy-runtime-api.
Tests
-----
8 integration tests under
crates/headroom-proxy/tests/integration_bedrock_invoke.rs:
1. native_envelope_round_trip_byte_equal
2. sigv4_signed_correctly_after_compression - confirms
authorization is SigV4-shape and x-amz-content-sha256
matches sha256(body received by upstream).
3. thinking_block_preserved_through_bedrock
4. redacted_thinking_preserved
5. document_block_preserved
6. tool_result_array_with_image_preserved
7. stop_sequence_null_only_when_present - pins that the proxy
does NOT inject stop_sequence: null (P4-37 hardcode).
8. tool_use_input_byte_equal_preserves_key_order
All eight pass. Full workspace test run is green; clippy + fmt
clean. make ci-precheck (rust + python + commitlint) passes
locally.
Build constraints honoured
--------------------------
- No silent fallbacks: missing creds / signing failures return
5xx with structured event=... log; no path ever forwards
unsigned.
- No hardcodes: region, endpoint, profile, enable-flag all
configurable via CLI + env.
- No regexes: vendor detection is str::starts_with.
- Comprehensive structured logs: event=bedrock_invoke_received,
bedrock_envelope_parsed, bedrock_compression_skipped,
bedrock_credentials_missing, sigv4_signed,
bedrock_invoke_forwarded, etc.
- Performant: body buffered once, passed by &[u8] to signer
(zero-copy), Bytes::clone only for ownership transfer to
reqwest. Sign exactly once per request.
- Elegant: 4 small focused modules mirror handlers/ + sse/.
- Tests use realistic Anthropic block content (real thinking,
redacted_thinking, document, base64 image fixtures).
Streaming (PR-D2) and observability (PR-D3) follow.
|
||
|
chopratejas
|
866d346bd0 |
fix: C4 — /v1/responses streaming + Conversations API in Rust
Phase C PR-C4 of the Realignment. Completes the Responses surface
in the Rust proxy and lays the Conversations passthrough rails.
# /v1/responses streaming pipeline
C3 left an INFO-WARN breadcrumb (`responses_streaming_passthrough_until_c4`)
on every SSE-Accept request because the byte-level SSE framer +
ResponseState machine wired by C1 was not formally tied into the
endpoint's contract. C4 closes the loop:
- Replaces the C3 fallback warn with a structured-log INFO event
`event = "responses_streaming_pipeline_active"`. The bytes flow
unchanged (compression on streamed response output is OUT of scope
per live-zone-only contract); telemetry runs in the parallel task.
- New emergency-rollback toggle `--enable-responses-streaming`
(env: `HEADROOM_PROXY_ENABLE_RESPONSES_STREAMING`, default `true`).
When `false`, the OpenAI-Responses SSE state machine is skipped
and `event = "responses_streaming_state_machine_skipped"` is
emitted; bytes still pass through. Other providers' state
machines are unaffected.
- Request-side compression (the C3 live-zone dispatcher) continues
to run on streaming requests — `should_intercept` does not gate on
Accept, so SSE clients get the same body compression as
non-streaming clients.
# Conversations API surface (`/v1/conversations*`)
New `crates/headroom-proxy/src/handlers/conversations.rs` mounts
explicit axum routes (no regex per build constraints) for OpenAI's
stateful thread surface:
- `POST /v1/conversations`
- `GET /v1/conversations/{id}`
- `POST /v1/conversations/{id}` (metadata update)
- `DELETE /v1/conversations/{id}`
- `POST /v1/conversations/{id}/items`
- `GET /v1/conversations/{id}/items`
- `GET /v1/conversations/{id}/items/{item_id}`
- `DELETE /v1/conversations/{id}/items/{item_id}`
For PR-C4 each handler is passthrough-with-instrumentation: forward
upstream byte-equal via `forward_http`, emit
`event = "conversations_passthrough_pr_c4"` with route name +
extracted IDs. Compression of stored conversation items is C5+/B-phase
territory and explicitly NOT done here. Bodies are streamed (no
buffering) — `is_compressible_path` does not match
`/v1/conversations*` so the compression gate's else-branch streams
the body via `reqwest::Body::wrap_stream`.
New toggle `--enable-conversations-passthrough` (env:
`HEADROOM_PROXY_ENABLE_CONVERSATIONS_PASSTHROUGH`, default `true`).
When `false`, the per-route handlers are NOT mounted — requests
still reach upstream via the catch-all (no per-route logs); a
single WARN at app-build time confirms the rollback.
# Tests added
- `tests/integration_responses_streaming.rs` (4 tests):
request bytes byte-equal upstream on streaming; client receives
bytes that round-trip through the same SseFramer + ResponseState
the proxy spawns; rollback path still passes bytes;
below-threshold streaming request round-trips byte-equal.
- `tests/integration_conversations.rs` (10 tests):
every CRUD endpoint passthrough byte-equal through wiremock;
4xx upstream errors surface verbatim (no swallowing);
passthrough disabled still falls through to catch-all byte-equal.
- `tests/sse_openai_responses.rs` (+2 tests):
`chunk_boundary_invariance_pr_c4` — every single-byte split point
produces the same final state (cache-safety streaming property);
`minimal_upstream_response_pr_c4` — empty `[DONE]` upstream never
panics the state machine.
# Structured-log events introduced
- `responses_streaming_pipeline_active` (INFO) — replaces the C3 WARN
- `responses_streaming_pipeline_disabled` (WARN, only when toggle off)
- `responses_streaming_state_machine_skipped` (INFO, in proxy.rs)
- `conversations_passthrough_pr_c4` (INFO, per request)
- `conversations_passthrough_disabled` (WARN, app-build time)
# Config keys introduced
- `enable_responses_streaming: bool` (default `true`)
- `enable_conversations_passthrough: bool` (default `true`)
# Verification
- `cargo build --workspace --all-features`: green
- `cargo test --workspace --all-features`: 793 + 14 + ... all green
- `cargo clippy --workspace --all-features -- -D warnings`: zero
- `cargo fmt --all --check`: green
- `make ci-precheck-rust`: green
- `make ci-precheck-python` (against main repo .venv): 176 passed
No Python files modified; Conversations compression deferred to C5+
and follow-up B-phase work that retires
`headroom/proxy/responses_converter.py`.
|
||
|
chopratejas
|
57c3e38cb3 |
fix: C3 — /v1/responses Rust HTTP handler + per-item-type passthrough
Ports the OpenAI Responses API request path to Rust with first-class per-item-type handling, replacing the fragile `headroom/proxy/responses_converter.py` shim that flattens Responses-shape items into Chat-Completions-shape (and silently breaks every time OpenAI lands a new item type). What lands: - New `compress_openai_responses_live_zone` dispatcher in `headroom-core` (sibling of the Anthropic / Chat Completions ones) that walks the `input` array (with `messages` accepted as legacy alias) and identifies the latest of each compressible kind: `function_call_output`, `local_shell_call_output`, `apply_patch_call_output`, plus the latest user-role `message`. Earlier *_output items are FROZEN (cached prefix). All other item types pass through verbatim via byte-range surgery. - New 2 KiB output-item floor for `*_output` items (per spec PR-C3 line 167) on top of the existing per-content-type byte thresholds. - `crates/headroom-proxy/src/responses_items.rs` exposes a typed `ResponseItem<'a>` enum for telemetry / decision-making, paired with a `ClassifiedItem` two-pass classifier that keeps the original `&RawValue` slice so byte fidelity is preserved independently of the typed view. `Cow<'a, str>` on string fields handles both borrowed (no-escape) and owned (escape-bearing) JSON values without allocation on the common path. - `crates/headroom-proxy/src/compression/live_zone_responses.rs` proxy-side dispatcher mirrors the Chat Completions shape (same `Outcome` / `Passthrough` arms, same structured logs, same manifest aggregation). Logs unknown `type` values at warn (`event = responses_unknown_item_type`) and never strips them. - `crates/headroom-proxy/src/handlers/responses.rs` POST handler buffers the body and re-injects via `forward_http`. Detects `Accept: text/event-stream` and emits `event = responses_streaming_passthrough_until_c4` so we can measure the volume before C4 wires the streaming state machine. - Image-generation log redaction: `image_generation_call` items are logged with byte size only (no `image_data` in the log path). The upstream-bound bytes are NOT mutated — redaction is log-channel only, per spec. - `CompressibleEndpoint::OpenAiResponses` variant; route `POST /v1/responses` wired in `proxy.rs`. Tests added (~40 across the workspace): - 9 core dispatcher unit tests in `live_zone.rs` - 4 proxy dispatcher unit tests + 5 handler unit tests + 5 typed-enum unit tests - 16 integration tests in `tests/integration_responses.rs` covering: V4A patch byte-equality, argv-array preservation, Codex `phase=commentary`/`final_answer`, compaction passthrough, reasoning passthrough, function_call.arguments string preservation, call_id-vs-id distinction, 2 KiB output-item floor (below + above), local_shell output compression, MCP / computer-use / image generation passthrough, unknown-type warn-and-preserve, and a representative round-trip with reasoning + function_call + local_shell + apply_patch + custom items. Per-PR-C3 plan: REALIGNMENT/05-phase-C-rust-proxy.md. |
||
|
chopratejas
|
fe00dc006a |
fix: C2 — /v1/chat/completions Rust handler + OpenAI live-zone
Adds a POST handler for /v1/chat/completions and a sibling live-zone dispatcher for the OpenAI Chat Completions request shape. Same compressor backend as Anthropic (SmartCrusher / LogCompressor / SearchCompressor / DiffCompressor), same per-content-type byte thresholds, same tokenizer-validated rejection gate, same byte-range surgery for cache-stable rewrite. Live zone for Chat Completions: the latest role=tool message's content AND the latest role=user message's text content. Earlier tool/user messages are part of the cache hot zone; never touched. tools[] and tool_choice are never read or rewritten — they round-trip byte-equal as a side effect of byte-range surgery. Behaviours: - n > 1 → passthrough (multiple completions imply non-determinism; the proxy gate skips dispatch and forwards original bytes). - stream: true → pass through to forward_http's existing C1 SSE parser tee (ChunkState). - tool_choice change → never mutated. - mode == Off → passthrough with structured 'mode_off' log. - Body not JSON / no messages → passthrough; the dispatcher logs the decision and forwards original bytes. The handler is wired as an explicit POST route on /v1/chat/completions, buffers the body into Bytes, and re-injects it into the shared forward_http function. forward_http's compression gate now classifies the path (AnthropicMessages vs OpenAiChatCompletions) and dispatches to the right module (compress_anthropic_request / compress_openai_chat_request). Single forwarding code path keeps SSE telemetry, header stripping, and request-id plumbing single-source. Files added: - crates/headroom-proxy/src/handlers/chat_completions.rs - crates/headroom-proxy/src/handlers/mod.rs - crates/headroom-proxy/src/compression/live_zone_openai.rs - crates/headroom-proxy/tests/integration_chat_completions.rs Files modified: - crates/headroom-core/src/transforms/live_zone.rs (+compress_openai_chat_live_zone, +helpers) - crates/headroom-core/src/transforms/mod.rs (re-export) - crates/headroom-proxy/src/compression/mod.rs (+CompressibleEndpoint classification, expose live_zone_openai) - crates/headroom-proxy/src/lib.rs (expose handlers module) - crates/headroom-proxy/src/proxy.rs (route + dispatch) Tests: - 7 integration tests in tests/integration_chat_completions.rs covering passthrough byte-equality, tool message compression (≥40% reduction on 1500-row JSON-array fodder), n>1 passthrough, stream_options round-trip, tool_choice non-mutation, refusal delta handling via ChunkState, and tool_call argument accumulation across three streaming chunks. - Unit tests on compress_openai_chat_live_zone (6) and compress_openai_chat_request (7) cover the dispatcher and proxy shim independently. Workspace test count: 953 (after C1) → 975. fmt clean. clippy --all-targets --all-features -D warnings clean. make ci-precheck PASSED. Plugin marketplace versions auto-bumped by the sync-plugin-versions pre-commit hook. Per-PR-C2 plan: REALIGNMENT/05-phase-C-rust-proxy.md. |
||
|
chopratejas
|
ddc6f6ceb0 |
fix: C1 — byte-level SSE parser + state machines
Foundation of Phase C. Delivers:
* Byte-level SSE framing (bytes::Bytes / BytesMut) with UTF-8
decoded only at \n\n event boundaries — no per-chunk decode,
no errors=ignore data loss across TCP reads.
* Three provider state machines:
- Anthropic: blocks keyed by index, all delta types
(text/thinking/input_json/citations/signature) preserved
byte-equal.
- OpenAI Chat: ToolCallState concatenation, refusal field,
include_usage final chunk handling.
- OpenAI Responses: items keyed by id (not position) for
out-of-order completion; full event coverage.
* State machine runs in parallel with byte-passthrough via a
tokio::spawn task fed by a bounded mpsc — clients see raw
bytes immediately; telemetry populates without blocking.
Retires P1-8, P1-9, P1-14, P1-15, P1-17, P4-48 in the Rust path
(Python A8 hotfix preserved as fallback until Phase H).
Per-PR-C1 plan: REALIGNMENT/05-phase-C-rust-proxy.md.
|
||
|
chopratejas
|
b3b3feff6f |
fix: B4 — token validation gate + per-content-type byte thresholds
Eliminate P3-33 / P3-34. Wraps every per-block compression in
the live-zone dispatcher with two new gates:
1. Per-content-type byte thresholds — pinned as `const` at the top
of `live_zone.rs` so the table is grep-able and reviewable in
one place. No magic numbers anywhere in the dispatch logic; a
`threshold_for(ContentType)` helper returns the value. Below
threshold → no compressor invoked, recorded as
`BlockAction::BelowByteThreshold { content_type, byte_count,
threshold_bytes }`. Thresholds:
- JSON-array tool_results: 1 KiB
- Build / log output: 512 B
- Search-result blocks: 1 KiB
- Git-diff blocks: 1 KiB
- Source code: 2 KiB (pinned for the future
Rust code-compressor port)
- Plain text: 5 KiB (pinned for Kompress wiring)
- HTML: 5 KiB (no compressor today)
2. Tokenizer-validated rejection — the byte-length proxy
(`compressed_bytes >= original_bytes`) is replaced with a
token-count check using `headroom_core::tokenizer::get_tokenizer`.
The dispatcher creates one tokenizer per request (model-aware
via the new `model: &str` parameter to
`compress_anthropic_live_zone`) and counts both the original
and compressed text. When `compressed_tokens >= original_tokens`
the candidate is rejected and the original bytes are kept.
`BlockAction::Compressed` and `BlockAction::RejectedNotSmaller`
gain `original_tokens` and `compressed_tokens` fields so the
proxy can log token-savings (the currency that actually matters
for prompt cache + provider billing) instead of bytes.
The proxy `live_zone_anthropic.rs` extracts `body["model"]` (or
falls back to `DEFAULT_MODEL = "claude-3-5-sonnet-20241022"` when
the field is missing — the chars-per-token estimator is calibrated
for the Claude family at 3.5 cpt) and threads it through. The
`Compressed` outcome now reports token counts from the manifest,
not byte counts, so the existing
`tokens_before / tokens_after` plumbing is suddenly accurate.
Tests added:
- `live_zone_thresholds.rs::below_threshold_no_compression_attempted`
— 200 B JSON array → `BelowByteThreshold` and `NoChange`.
- `live_zone_thresholds.rs::above_threshold_compression_attempted`
— 10 KB JSON array → byte-threshold gate clears and a compressor
runs (either `Compressed` or `RejectedNotSmaller`).
- `live_zone_token_validation.rs::compressed_more_tokens_falls_back`
— pathological input must not produce `Compressed` with
`compressed_tokens >= original_tokens`.
- `live_zone_token_validation.rs::compressed_fewer_tokens_accepted`
— well-formed JSON array of dicts → `Compressed` with strict
token shrinkage.
- Property test `live_zone_compression_token_count_non_increasing`
— for any well-formed body generated by `proptest`, the
dispatcher's emitted body has token-count <= input's token-count.
Pins the central PR-B4 invariant: the dispatcher never inflates
tokens.
Existing 12 unit tests in `live_zone.rs` and 6 integration tests
in `tests/live_zone_dispatch.rs` updated for the new field shape
and the `model` parameter; all pass. The diff-routing test's
fixture grew to 1.3 KiB so it clears the new GitDiff threshold
gate, exercising the dispatch path rather than short-circuiting.
Per-PR-B4 plan: REALIGNMENT/04-phase-B-live-zone.md.
|