mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
33 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
dependabot[bot]
|
5731be7e68
|
deps: bump axum from 0.7.9 to 0.8.9 (#2966)
Bumps [axum](https://github.com/tokio-rs/axum) from 0.7.9 to 0.8.9. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/axum/releases">axum's releases</a>.</em></p> <blockquote> <h2>axum-v0.8.9</h2> <ul> <li><strong>added:</strong> <code>WebSocketUpgrade::{requested_protocols, set_selected_protocol}</code> for more flexible subprotocol selection (<a href="https://redirect.github.com/tokio-rs/axum/issues/3597">#3597</a>)</li> <li><strong>changed:</strong> Update minimum rust version to 1.80 (<a href="https://redirect.github.com/tokio-rs/axum/issues/3620">#3620</a>)</li> <li><strong>fixed:</strong> Set connect endpoint on correct field in MethodRouter (<a href="https://redirect.github.com/tokio-rs/axum/issues/3656">#3656</a>)</li> <li><strong>fixed:</strong> Return specific error message when multipart body limit is exceeded (<a href="https://redirect.github.com/tokio-rs/axum/issues/3611">#3611</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3597">#3597</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3597">tokio-rs/axum#3597</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3620">#3620</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3620">tokio-rs/axum#3620</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3656">#3656</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3656">tokio-rs/axum#3656</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3611">#3611</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3611">tokio-rs/axum#3611</a></p> <h2>axum v0.8.8</h2> <ul> <li>Clarify documentation for <code>Router::route_layer</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3567">#3567</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3567">#3567</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3567">tokio-rs/axum#3567</a></p> <h2>axum v0.8.7</h2> <ul> <li>Relax implicit <code>Send</code> / <code>Sync</code> bounds on <code>RouterAsService</code>, <code>RouterIntoService</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3555">#3555</a>)</li> <li>Make it easier to visually scan for default features (<a href="https://redirect.github.com/tokio-rs/axum/issues/3550">#3550</a>)</li> <li>Fix some documentation typos</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3550">#3550</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3550">tokio-rs/axum#3550</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3555">#3555</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3555">tokio-rs/axum#3555</a></p> <h2>axum v0.8.5</h2> <ul> <li><strong>fixed:</strong> Reject JSON request bodies with trailing characters after the JSON document (<a href="https://redirect.github.com/tokio-rs/axum/issues/3453">#3453</a>)</li> <li><strong>added:</strong> Implement <code>OptionalFromRequest</code> for <code>Multipart</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3220">#3220</a>)</li> <li><strong>added:</strong> Getter methods <code>Location::{status_code, location}</code></li> <li><strong>added:</strong> Support for writing arbitrary binary data into server-sent events (<a href="https://redirect.github.com/tokio-rs/axum/issues/3425">#3425</a>)]</li> <li><strong>added:</strong> <code>middleware::ResponseAxumBodyLayer</code> for mapping response body to <code>axum::body::Body</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3469">#3469</a>)</li> <li><strong>added:</strong> <code>impl FusedStream for WebSocket</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3443">#3443</a>)</li> <li><strong>changed:</strong> The <code>sse</code> module and <code>Sse</code> type no longer depend on the <code>tokio</code> feature (<a href="https://redirect.github.com/tokio-rs/axum/issues/3154">#3154</a>)</li> <li><strong>changed:</strong> If the location given to one of <code>Redirect</code>s constructors is not a valid header value, instead of panicking on construction, the <code>IntoResponse</code> impl now returns an HTTP 500, just like <code>Json</code> does when serialization fails (<a href="https://redirect.github.com/tokio-rs/axum/issues/3377">#3377</a>)</li> <li><strong>changed:</strong> Update minimum rust version to 1.78 (<a href="https://redirect.github.com/tokio-rs/axum/issues/3412">#3412</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3154">#3154</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3154">tokio-rs/axum#3154</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3220">#3220</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3220">tokio-rs/axum#3220</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3377">#3377</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3377">tokio-rs/axum#3377</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3412">#3412</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3412">tokio-rs/axum#3412</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3425">#3425</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3425">tokio-rs/axum#3425</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3443">#3443</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3443">tokio-rs/axum#3443</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3453">#3453</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3453">tokio-rs/axum#3453</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3469">#3469</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3469">tokio-rs/axum#3469</a></p> <h2>axum v0.8.4</h2> <ul> <li><strong>added:</strong> <code>Router::reset_fallback</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3320">#3320</a>)</li> <li><strong>added:</strong> <code>WebSocketUpgrade::selected_protocol</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3248">#3248</a>)</li> <li><strong>fixed:</strong> Panic location for overlapping method routes (<a href="https://redirect.github.com/tokio-rs/axum/issues/3319">#3319</a>)</li> <li><strong>fixed:</strong> Don't leak a tokio task when using <code>serve</code> without graceful shutdown (<a href="https://redirect.github.com/tokio-rs/axum/issues/3129">#3129</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
bbe901319d
|
deps: bump tokio-tungstenite from 0.24.0 to 0.30.0 (#2967)
Bumps [tokio-tungstenite](https://github.com/snapview/tokio-tungstenite) from 0.24.0 to 0.30.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/snapview/tokio-tungstenite/blob/master/CHANGELOG.md">tokio-tungstenite's changelog</a>.</em></p> <blockquote> <h1>0.30.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.30.0</code>. See <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md"><code>tungstenite</code> release</a>.</li> </ul> <h1>0.29.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.29.0</code>. See <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md"><code>tungstenite</code> release</a>.</li> </ul> <h1>0.28.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.28.0</code>. See <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md"><code>tungstenite</code> release</a>.</li> </ul> <h1>0.27.0</h1> <ul> <li>See <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md#0270">performance updates in <code>tungstenite-rs</code></a>.</li> </ul> <h1>0.26.2</h1> <ul> <li>Update <code>tungstenite</code>, see <a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md#0262">changes here</a>.</li> </ul> <h1>0.26.1</h1> <ul> <li>Update <code>tungstenite</code> to address an issue that might cause UB in certain cases.</li> </ul> <h1>0.26.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.26.0</code> (<a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md#0260">breaking changes</a>).</li> </ul> <h1>0.25.0</h1> <ul> <li>Update <code>tungstenite</code> to <code>0.25.0</code> (<a href="https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md#0250">important updates!</a>).</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Andrei Boldyrev
|
f6398a6476
|
fix(proxy): port session-sticky beta headers to the Rust proxy (#2381)
## Description The Python proxy protects prompt caches with `SessionBetaTracker` (PR-A6, `headroom/proxy/helpers.py`): interactive clients (Claude Code, Codex CLI) may drop an `anthropic-beta` / `openai-beta` token between turn N and turn N+1 of the same conversation, and since beta headers are part of the bytes that determine the upstream prefix-cache key, the drop rotates the key and the provider re-writes the whole prefix at the customer's cost. The tracker unions the client's tokens with everything previously seen for that `(provider, session)` and forwards the union — a documented operator contract (`docs/configuration.mdx`, "Session Beta Header Tracking"). The Rust proxy has no equivalent, and Phase H (#2258) deletes the tracker together with `helpers.py` and its test file (`tests/test_anthropic_beta_session_sticky.py`). None of the Phase A–G plans port it (Phase F consumes beta headers for auth-mode classification only), so the protection would silently not survive the migration — and the Phase-H gate "Cache-hit-rate parity with direct upstream confirmed" can't catch the loss, because re-injection makes proxied traffic *beat* direct upstream on cache hits; when the mechanism disappears, proxied traffic degrades *to* direct-upstream levels, which that comparison reads as parity. This PR ports the tracker semantics into the Rust proxy so the protection lives in the codebase Phase H keeps. Closes #2380 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) (New Rust functionality, but a parity port of already-shipped, already-documented Python behavior — the PR title uses `fix:` per `REALIGNMENT/INDEX.md`: "Commit prefix: `fix:` for Rust-migration phase commits".) ## Changes Made - **`cache_stabilization/beta_sticky.rs`** — the tracker: bounded LRU (1000 sessions, same sizing rationale and `# Panics` contract as the drift detector's capacity) keyed by `(provider, session)`, storing the per-session ordered token list. Union preserves first-seen order; dedup is case-insensitive with first-seen casing winning; lookups touch recency; overflow evicts the oldest — mirroring the Python tracker. The header-plumbing lives in the module too (`apply_sticky_betas`), so the merge is unit-testable without booting a proxy. - **`proxy.rs` wiring** — on the intercepted POST routes (`/v1/messages`, `/v1/chat/completions`, `/v1/responses`), right after the drift-detector observation, reusing the drift detector's `derive_session_key` output so both cache-stability subsystems agree on conversation identity. - **`config.rs`** — `--beta-header-sticky` / `HEADROOM_PROXY_BETA_HEADER_STICKY` (`enabled` default; `disabled` forwards the client value verbatim and keeps no state), mirroring the `StripInternalHeaders` flag pattern and the existing `HEADROOM_*` → `HEADROOM_PROXY_*` Python→Rust env pairing. Since the merge runs inside the compression interceptor, startup logs a warning when the flag is `enabled` while `--compression` is off, and both the CLI doc and the docs row state the dependency. - **`tests/integration_beta_header_sticky.rs`** — 9 end-to-end tests against a wiremock upstream asserting the headers/bytes the upstream actually receives; 21 unit tests port the behavioral contract from `tests/test_anthropic_beta_session_sticky.py` and cover the header-map plumbing. - **`docs/content/docs/configuration.mdx`** — one row for `HEADROOM_PROXY_BETA_HEADER_STICKY` next to the existing Python/Rust flag pairs. ## Testing - [x] Unit tests pass (`cargo test -p headroom-proxy`; Python side via `make ci-precheck-python` — `pytest` subset, 174 passed) - [x] Linting passes (`cargo clippy --all-targets` — 0 warnings; `cargo fmt --check` clean; Rust-only change, so `ruff`/`mypy` are covered by the untouched-Python `ci-precheck-python` build) - [ ] Type checking passes (`mypy headroom`) — N/A, no Python files touched - [x] New tests added for new functionality - [x] Manual testing performed (RED/GREEN before-and-after runs below) ### Test Output ```text $ cargo test -p headroom-proxy --lib beta_sticky test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 248 filtered out; finished in 0.03s $ cargo test -p headroom-proxy --test integration_beta_header_sticky test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s $ cargo test -p headroom-proxy # full crate: 37 suites, all ok $ cargo clippy -p headroom-proxy --all-targets # 0 warnings $ make ci-precheck-rust ci-precheck-python ci-precheck-commitlint # green ``` ## Real Behavior Proof - Environment: macOS arm64 (Darwin 24.6), `rustc 1.95.0`, real Rust proxy booted on an ephemeral port in front of a wiremock upstream (`tests/common::start_proxy_with`, `compression = true`). - Exact command / steps: two-turn conversation through the proxy — turn 1 `POST /v1/messages` with `anthropic-beta: context-management-2025-06-27,interleaved-thinking-2025-05-14`; turn 2, same conversation, client drops the second token. The wiremock responder captures the headers the upstream actually receives (`cargo test -p headroom-proxy --test integration_beta_header_sticky`). - Observed result: **before** the port (test written first, run against the unmodified proxy) the upstream sees the shrunken token set and the prefix-cache key rotates — ```text assertion `left == right` failed: turn 2 must re-inject the dropped token so the upstream prefix-cache key stays byte-stable left: Some("context-management-2025-06-27") right: Some("context-management-2025-06-27,interleaved-thinking-2025-05-14") ``` **After** the port the same scenario passes: the upstream receives the full union on turn 2, the internal `x-headroom-session-id` never crosses the upstream boundary, and the forwarded body is SHA-256-identical to what the client sent (asserted by `body_bytes_stay_byte_equal_while_header_is_rewritten`). - Not tested: live traffic against a real provider upstream (wiremock only); the WebSocket path and Bedrock/Vertex routes (out of scope — see Additional Notes). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A (proxy behavior; see Real Behavior Proof). ## Additional Notes Design decisions, and where I'd like reviewer judgment: 1. **Applies to all auth modes, like the Python handler.** The Phase-E module doctrine gates *body*-mutating normalizers on PAYG; this mechanism mutates headers only, and the Python source of truth applies it unconditionally — an auth-mode gate here would create a behavioral delta exactly where the PR's purpose is behavior preservation. It's also stealth-consistent by construction: the union only ever contains tokens this client itself sent (Headroom-added tokens are never recorded), `auth_mode.rs`'s own docs name "beta-header drift voids them" as the OAuth cache hazard (stickiness is the anti-drift), and F2's `CompressionPolicy` has no beta field — no gate is structurally expected. I've extended the `cache_stabilization/mod.rs` taxonomy with a third category ("re-echo client-sent state") to keep the module doctrine honest. Flagging explicitly since invariant #10 ("no beta drift") is subscription-critical: if you read it as "forward beta verbatim on Subscription", say so and I'll add the gate. 2. **One deliberate divergence from Python: sessions are keyed per conversation, not per `(model, system)` bucket.** The Python tracker keys on the store session id — explicit header, else a hash of model + leading system prompt — so a Claude Code session and every one of its subagents share one token union and cross-inherit tokens; two *different users* behind an org proxy with the same (model, system) do too. This port keys on the drift detector's conversation-aware key (#2301), so each conversation keeps its own union (pinned by `separate_conversations_do_not_leak_tokens`). That's the same conflation defect #2085/#2193/#2301 chased out of the other session-sticky subsystems, and it makes "the union only contains tokens this client sent" actually true — under the Python fallback key it isn't (cross-user union). Cost: Python's accidental cross-conversation repair is gone, and an OAuth access-token refresh mid-conversation re-keys the session (one turn forwards verbatim, then re-learns — fails safe). 3. **Repeated header lines are joined per RFC 9110 list semantics before recording.** A client sending two `anthropic-beta` lines gets both recorded; a later rewrite collapses to one line carrying the full set. (Reading only the first line — or Python's actual behavior, which keeps only the *last* line via its `dict(headers)` collapse — can shrink the upstream token set mid-conversation when a rewrite fires.) 4. **Scope: the three intercepted HTTP routes.** With the compression interceptor off the proxy is a strict byte-pipe (Phase-A invariant) — no header mutation, hence the startup warning. WebSocket keeps its behavior (Python's WS site keys on a per-connection UUID, so cross-turn accumulation is a near-no-op there; the Rust WS tunnel doesn't touch beta headers). Bedrock/Vertex are skipped by the same match that skips the drift detector (betas travel in the body as `anthropic_beta` on Bedrock). 5. **Log discipline**: `event=beta_header_merge` carries token *counts* only (beta tokens can carry experiment IDs; same privacy contract as Python's `log_beta_header_merge`, plus the drift detector's hashed session-key prefix instead of Python's raw session id). One deviation from Python's unconditional info: the no-op case logs at debug, matching the drift detector's silent-on-stable precedent — an info-level `beta_header_merge` always marks an actual cache-affecting rewrite. 6. **Capacity is a const (1000), not a flag** — following the drift-detector precedent rather than Python's `HEADROOM_BETA_TRACKER_MAX_SESSIONS` env var. Happy to make it configurable if you'd rather keep that operator knob. 7. **Fail-open everywhere**: non-ASCII client values are forwarded verbatim with nothing recorded; a poisoned tracker lock forwards the client value verbatim; an unencodable union (unreachable — every token came from a parsed header value) logs and forwards verbatim. The protection never delays or drops a request. |
||
|
Abhishek Mittal
|
52a024d28c
|
fix(proxy): strip [1m] model suffix before upstream forwarding (#2027)
## Description Scopes the `[1m]` context-window tier suffix sanitizer to Anthropic `/v1/messages` requests only (addresses PR #2027 review feedback). The original patch applied the rewrite to every buffered compressible endpoint, which would have silently mutated OpenAI Chat Completions and OpenAI Responses request model IDs. The `[1m]` marker is an Anthropic/Claude Code compatibility signal emitted by the Headroom CLI; the existing Python parity behavior (`sanitize_anthropic_model_id()`) is Anthropic-specific and must not leak onto OpenAI shapes. Refactors the helper into `compression::sanitize_anthropic_model_id_in_body`, drops the dead `sanitize_model_id` helper in `sse/anthropic.rs`, and adds 8 unit tests + 5 wiremock-backed integration tests that pin the scope. All 420 `headroom-proxy` tests pass; `cargo fmt` and `cargo clippy -D warnings` clean. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Move `sanitize_request_model_id` out of `proxy.rs` and into `compression::sanitize_anthropic_model_id_in_body` (Anthropic-specific name; private `trim_anthropic_model_id_suffix` helper for unit-testable pure behavior). - Gate the call site on `CompressibleEndpoint::AnthropicMessages` **after** classification. The OpenAI Chat Completions and OpenAI Responses arms get an explicit no-op match so the sanitizer cannot re-apply to those paths. - Drop the dead `sanitize_model_id` helper in `sse/anthropic.rs` (it was `#[allow(dead_code)]` with no callers). - 8 new unit tests in `compression/mod.rs`: trailing `[1m]` stripped, Claude-style suffix stripped, no-suffix passthrough (byte-equal), non-string model, missing `model` field, non-JSON body, `[1m]` mid-string, and the pure trim helper. - 5 new integration tests in `tests/integration_anthropic_model_sanitize.rs` that boot a real Rust proxy in front of a wiremock upstream. ## Testing - [x] Unit tests pass (`cargo test -p headroom-proxy` → 420 passed, 35 suites) - [x] Linting passes (`cargo clippy -p headroom-proxy --tests --all-features -- -D warnings` clean) - [x] Type checking passes (`cargo check -p headroom-proxy --tests --all-features` clean) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-proxy --test integration_anthropic_model_sanitize Compiling headroom-proxy v0.x.x Finished `test` profile [unoptimized + debuginfo] target(s) Running tests/integration_anthropic_model_sanitize.rs test anthropic_messages_strips_1m_suffix_glm ... ok test anthropic_messages_strips_1m_suffix_claude ... ok test anthropic_messages_passthrough_when_no_suffix ... ok test openai_chat_completions_passthrough_with_1m_model ... ok test openai_responses_passthrough_with_1m_model ... ok test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out ``` ```text $ cargo test -p headroom-proxy test result: ok. 420 passed; 0 failed; 0 ignored; 0 measured; 235 filtered out finished in 10.93s ``` ```text $ cargo clippy -p headroom-proxy --tests --all-features -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) ``` ## Real Behavior Proof - **Environment:** macOS 14.x; `rustc` pinned via `rust-toolchain.toml`; `cargo` 1.x. No network access required (wiremock upstream). - **Exact command / steps:** 1. `cargo test -p headroom-proxy --test integration_anthropic_model_sanitize` — confirms `/v1/messages` strips `glm-5.2[1m]` and `claude-3-7-sonnet[1m]`; confirms `/v1/chat/completions` and `/v1/responses` leave the body byte-equal (SHA-256 asserted). 2. `cargo test -p headroom-proxy` — full suite green (420 passed). 3. `cargo clippy -p headroom-proxy --tests --all-features -- -D warnings` — clean. 4. `cargo fmt -p headroom-proxy --check` — clean. 5. Source inspection of `crates/headroom-proxy/src/proxy.rs` after the change: the call site is now in a `match endpoint` arm that explicitly returns `buffered` for the OpenAI variants, so the sanitizer cannot re-apply to those paths. - **Observed result:** all 5 new integration tests pass, all 420 crate tests pass, clippy and fmt clean. The OpenAI tests assert SHA-256 byte equality on a body whose `model` field ends in `[1m]`; if the sanitizer were to re-leak onto OpenAI shapes these would fail loudly with a length delta. - **Not tested:** a live Anthropic API call (would require real credentials and is not required to prove the byte-level scope fix). The Python proxy's `sanitize_anthropic_model_id()` is the documented parity reference (Python PR #1840, issue #1812). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (N/A — no user-facing docs change; the Python proxy's `sanitize_anthropic_model_id` is the parity reference cited in code comments) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (project uses git log + PR titles; this PR's title follows the conventional commit shape) ## Screenshots (if applicable) N/A — backend behavior, no UI change. ## Additional Notes - The OpenAI integration tests rely on a JWT-style `Authorization: Bearer` header to classify the request as `AuthMode::OAuth` and short-circuit the PR-E4 `prompt_cache_key` injector. This is the same control variable the existing `integration_chat_completions.rs` tests use to isolate dispatcher byte-fidelity from the E4 hook. Comments in each test explain the relationship. - The dead helper in `sse/anthropic.rs` is removed, so the diff is net negative on LoC for the SSE module. - The Python parity reference is `sanitize_anthropic_model_id()` (Python PR #1840, issue #1812); the function name and the call-site scope are the explicit parity contract. - Branch was rebased onto `upstream/main` (91 commits behind) before force-push to the fork; conflict-free rebase. The original PR commit and the fix are the only two commits on the PR. --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> Co-authored-by: Abhishek Mittal <abhishek.mittal@users.noreply.github.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
JD Davis
|
2c9eb7c5f1
|
feat(simulators): add provider simulator service (#2014)
## Description
Adds a Rust-only `headroom-simulators` workspace crate: a deterministic
local upstream simulator service for Headroom proxy and pipeline
validation. It supplies configurable stubs plus bottled provider-shaped
responses for supported provider/path surfaces without calling real
LLMs.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added `crates/headroom-simulators` Rust crate with library and
`headroom-simulators` binary.
- Added clean domain classification for supported surfaces: Anthropic
`/v1/messages`, OpenAI chat/responses/conversations, Bedrock
invoke/stream routes, Vertex raw/stream predict, health, and generic
fallback.
- Added JSON-configured stub matching by method, path, body substring,
and JSON pointer.
- Added bottled provider-shaped JSON, SSE, and Bedrock EventStream
responses for unconfigured requests.
- Added a container `Dockerfile` and README for local/GitHub Actions
usage.
- Added unit and HTTP integration tests for defaults, configured stubs,
SSE, Vertex, and Bedrock EventStream behavior.
- Added proxy-level simulator-backed E2E tests that run Headroom against
the simulator across Anthropic, OpenAI Chat, OpenAI Responses, OpenAI
Conversations, Bedrock invoke/converse/streaming, Vertex raw/stream
predict, and upstream health.
- Added simulator-backed provider error-path E2E coverage for OpenAI
429, Anthropic 529, Bedrock 502, and Vertex 503 responses flowing
through Headroom unchanged.
- Added Headroom-owned preflight error E2E coverage proving Bedrock
missing credentials and invalid Vertex envelopes stop inside the proxy
instead of silently falling through to the simulator/provider.
- Fixed direct Rust `headroom-core` binaries/tests on Windows so Magika
initializes ONNX Runtime via `ort::init_from` from an explicit pip
`onnxruntime` library path, with fail-fast fallback only when no safe
runtime is discoverable.
- Added a Rust CI `simulator-e2e` matrix for `ubuntu-latest`,
`macos-latest`, and `windows-latest` that runs `cargo test -p
headroom-proxy --test e2e_simulators`.
- Gated dynamic Magika `Path`/`PathBuf` imports to Windows and x86_64
macOS so Linux clippy does not see unused dynamic-ORT-only imports.
## Testing
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
cargo fmt --all -- --check
# passed
cargo clippy --workspace -- -D warnings
# passed
$env:ORT_DYLIB_PATH=$null
cargo test -p headroom-core transforms::magika_detector::tests:: --lib
# 17 passed, 0 failed; Magika initialized from discovered pip
onnxruntime DLL
$env:ORT_DYLIB_PATH=$null
cargo test --workspace
# passed
gitleaks protect --staged --no-banner --redact
# no leaks found
gitleaks git --log-opts="headroomlabs/main..HEAD" --no-banner --redact
# 5 commits scanned; no leaks found
## Real Behavior Proof
- **Environment:** Windows PowerShell, Rust toolchain `1.95.0`, clean
worktree from `headroomlabs/main` at `
|
||
|
Yasser Sheikh
|
b08ec15b0d
|
fix(proxy): add native Bedrock converse-stream route (#917)
## Description
Adds native Bedrock `POST /model/{model_id}/converse-stream` routing in
`headroom-proxy` by reusing the existing streaming handler and
preserving route-specific upstream action forwarding.
This addresses a gap where native Bedrock streaming support existed for
`invoke-with-response-stream` but not `converse-stream`, even though
both share the same EventStream transport and SSE translation path in
this proxy.
Fixes #919
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
## Changes Made
- Add route mount in `crates/headroom-proxy/src/proxy.rs`:
- `POST /model/:model_id/converse-stream` ->
`bedrock::invoke_streaming::handle_invoke_streaming`
- Update streaming handler URL construction in
`crates/headroom-proxy/src/bedrock/invoke_streaming.rs`:
- infer action from inbound path (`invoke-with-response-stream` or
`converse-stream`)
- build upstream URL with the resolved action
- return structured `400` for unsupported streaming action paths
- Add unit tests in
`crates/headroom-proxy/src/bedrock/invoke_streaming.rs`:
- action extraction coverage for both streaming paths
- upstream URL construction coverage for `converse-stream`
- Add integration coverage in
`crates/headroom-proxy/tests/integration_bedrock_streaming.rs`:
- `converse_stream_route_translates_to_sse`
- Add changelog entry under `Unreleased` bug fixes in `CHANGELOG.md`.
## Testing
- `cargo fmt --all`
- `cargo test -p headroom-proxy --test integration_bedrock_streaming --
--nocapture`
- `cargo test -p headroom-proxy --test integration_bedrock_metrics --
--nocapture`
## Real behavior proof
- **Setup tested on**
- macOS (darwin)
- Rust workspace local dev build
- `headroom-proxy` integration tests using wiremock upstream (no AWS
dependency)
- **Exact commands run after patch**
- `cargo test -p headroom-proxy --test integration_bedrock_streaming --
--nocapture`
- `cargo test -p headroom-proxy --test integration_bedrock_metrics --
--nocapture`
- **After-fix evidence + observed result**
- New integration test `converse_stream_route_translates_to_sse` passes.
- Streaming suite result: `10 passed; 0 failed`.
- Metrics suite result: `4 passed; 0 failed`.
- Logs show requests reaching `/model/.../converse-stream` and flowing
through Bedrock streaming path.
- **What I did not test**
- Live AWS Bedrock calls against real credentials/models.
- End-to-end CLI/runtime behavior outside Rust integration test harness.
|
||
|
chopratejas
|
2a717a993e |
fix(observability): G3 remediation — bound cardinality + wire dead metrics
Phase G PR-G3 review identified 5 Critical + 4 High + 5 Medium
findings. This commit lands all 14 fixes plus the optional nits.
CRITICAL
* C1 (cardinality DoS): `service_tier` was read from inbound JSON
and used verbatim as a metric label. A malicious client could
blow up the metric vector unboundedly. Added bounded vocabulary
in `metric_names.rs::service_tier` ({auto, default, flex,
on_demand, priority, scale, other-sentinel}) + a `validate()`
helper. Both request-side (`handlers/responses.rs`) and
response-side (`proxy.rs` Responses arm) gate raw values through
it.
* C2 (dead metric): `proxy_passthrough_bytes_modified_total` had
no production emit site. Wired it in `proxy.rs` to fire when a
dispatcher arm returning `NoCompression`/`Passthrough` produces
a body of a different byte length (a true cache-poisoning
regression detector). The check runs BEFORE the PR-E4
prompt_cache_key injector so legitimate injector mutations do
not trip the alarm.
* C3 (Python/Rust boundary): `proxy_image_generation_call_log_redacted_total`
was a dead Rust counter — the redaction happens entirely in the
Python proxy's request_logger. Removed the Rust counter; moved
the metric to the Python proxy's `/metrics` exporter via the
existing `redactions_total()` module-level counter.
* C4 (Python/Rust boundary): `wrap_rtk_invocations_total` was a
dead Rust counter with no wrap-side bridge. Removed the Rust
counter; added new `headroom/cli/wrap_rtk_metrics.py` with
`record_rtk_invocation(tool, delta)` + `rtk_invocation_counts()`
primitives and surfaced them via the Python proxy's `/metrics`
exporter.
* C5 (dead metric): `proxy_compression_rejected_by_token_check_total`
had no production caller. Wired it in
`live_zone_anthropic.rs`, `live_zone_openai.rs`, and
`live_zone_responses.rs` to increment on every
`BlockAction::RejectedNotSmaller` block in the manifest. The
metric now reflects real "compressor ran but kept original"
cases.
HIGH
* H1 (per-strategy ratio garbage): `proxy_compression_ratio_by_strategy`
emitted the same aggregate ratio for every strategy in
`strategies_applied` when multiple strategies ran on one body.
Added `per_strategy_tokens: Vec<PerStrategyTokens>` to
`Outcome::Compressed`; per-strategy `(before, after)` is
accumulated from the manifest at the wrapper sites and emitted
one sample per strategy in `proxy.rs`. Empty vec → fallback to
one aggregate-labelled sample with a debug log (Phase E
normalization paths that don't track per-strategy tokens).
* H2 (aborted stream): cache_hit_rate observed on client
disconnects mid-stream. Added a gate: Anthropic only fires when
`state.status == MessageStop`, OpenAI Responses only when
`terminal_status().is_some()`. Extracted the gate into the
pure function `compute_anthropic_session_hit_rate(state)` so
the H2 contract is unit-testable independent of the shared
global registry.
* H3 (docs lie + alarm contract): docs claimed HELP/TYPE is
reachable on fresh boot, then contradicted itself. Force-zero
every counter / gauge MetricVec with an `__init__` sentinel
label on each scrape so HELP/TYPE + a zero row are visible from
boot. Histograms are NOT force-zeroed (a synthetic observe(0.0)
would pollute percentiles). PromQL queries in docs filter
`{... != "__init__"}` so the sentinel rows are excluded from
aggregations.
* H4 (crate-version dependency): pinned `prometheus = "=0.13.4"`
exactly (no caret) so a future minor bump cannot silently break
the H3 force-zero contract that relies on this crate's gather()
semantics. Added a clear "retest the alarm contract on bump"
paragraph in docs.
MEDIUM
* M1 (saturate on cached > input): OpenAI Chat + Responses cache-
hit-rate computed `non_cached = input.saturating_sub(cached)`,
silently clamping to 0 if `cached > input`. Per "no silent
fallbacks", log + skip the emit on this wire-format pathology.
* M2 (over-fire on non-image base64): Python redactor's "density
heuristic" over-fired on encrypted blobs / signed tokens /
minified JSON / tool outputs. Tightened: only redact strings
inside known image-bearing JSON paths (`data`, `url`,
`image_url`, `image`) OR strings starting with `data:image/`.
* M3 (NaN clamp): cache_hit_rate::observe used `f64::clamp(0,1)`
which returns NaN for NaN input; the `debug_assert!` was
compiled out in release. Added `is_finite()` guard with a
loud-log + skip before observe.
* M4 (PromQL median-only): added p95, p99, mean (sum/count), and
Phase H canary-gate query section to docs. Canary fails if ANY
of {p50, p95, p99, mean} regresses below the Python baseline.
* M5 (label byte vs char): the `<image:base64-redacted bytes=N>`
placeholder reported character count, not UTF-8 byte count.
Switched to `.encode('utf-8').__len__()` so the label is
honest for non-ASCII payloads (ASCII base64 still has byte ==
char so existing scrapes are unchanged).
OPTIONAL
* Removed dead `debug_assert_eq!(buffered.len(), buffered.len(),
...)` no-op in proxy.rs.
* Normalised `record_response_status` log level from `info` to
`debug` to match peer metric helpers.
Tests:
* Rust: 11 integration_metrics tests (was 6) + 9 cache_hit_rate
unit tests (was 4) + 2 compression_ratio (unchanged). New
coverage: service_tier known/unknown bucketing, C2 alarm wire,
H1 per-strategy ratio, H2 abort gate, M3 NaN/inf skip.
* Python: 27 tests (was 13). New coverage: M2 path-gated
redaction, M5 byte vs char label, wrap_rtk_metrics primitive
thread safety and validation.
`cargo fmt --check`, `cargo clippy --workspace -- -D warnings`,
`cargo test -p headroom-proxy --lib` (221 passed) and the
integration_metrics + integration_compression +
integration_volatile_detector + integration_cache_control +
integration_cache_drift + integration_responses +
integration_bedrock_metrics test files all green. Full
`cargo test --workspace` deferred — disk pressure during the
agent session left insufficient space for the linker to write
the full integration test artifacts; runs that did fit all
passed. `make ci-precheck` deferred for the same reason.
ruff check + ruff format + mypy headroom/proxy/request_logger.py
+ headroom/cli/wrap_rtk_metrics.py + headroom/proxy/prometheus_metrics.py
green.
|
||
|
chopratejas
|
5f264a5329 |
fix(observability): wire Phase G PR-G3 RTK + proxy metrics (H-blocker)
Phase H ("retire the Python proxy") needs cache-hit-rate parity
between the Rust and Python proxies during canary. This PR lands
the per-invocation RTK metrics and the proxy-side observability
surface that the canary gate depends on.
Rust observability:
- `proxy_cache_hit_rate_per_session{provider}` — histogram, emitted
per session at SSE state-machine close (Anthropic message_delta,
OpenAI Chat final usage chunk, OpenAI Responses response.completed).
The Phase H canary gate metric.
- `proxy_compression_ratio_by_strategy{strategy, content_type}` —
histogram; one sample per shrunk block.
- `proxy_compression_rejected_by_token_check_total{strategy}` —
counter for tokenizer-validated rejections.
- `proxy_passthrough_bytes_modified_total{path}` — counter (must
stay 0 outside compression hot path; alarmable via PromQL rate).
- `proxy_rate_limit_remaining_{requests,tokens,input_tokens,output_tokens}{provider}` —
gauges populated from anthropic-ratelimit-* / x-ratelimit-* headers.
- `proxy_service_tier_count_total{tier}` and
`proxy_response_status_count_total{status}` — counters for
Responses-API outcome telemetry.
- `proxy_image_generation_call_log_redacted_total` — counter.
- `wrap_rtk_invocations_total{tool}` and
`wrap_rtk_tokens_saved_per_session` — RTK metrics exposed via
the proxy's /metrics scrape so wrap-side tail can increment
through one observability surface.
All metric names and label keys live in a single
`observability/metric_names.rs` constants module per realignment
build-constraint "configurable". Bounded label vocabularies
(service_tier, response_status, provider) are defined alongside.
Python (P4-45):
- `headroom/proxy/request_logger.py` — base64-image payloads in
request/response logs over 1024 bytes are replaced with
`<image:base64-redacted bytes=N>` placeholders. Walks Anthropic
source.data and OpenAI data URLs. No regexes — substring +
density heuristic.
Tests:
- `crates/headroom-proxy/tests/integration_metrics.rs` — 6 tests
covering cache-hit-rate, compression-ratio, passthrough-bytes,
service-tier, response-status, and rate-limit-snapshot.
- `tests/test_image_log_redaction.py` — 13 tests for the Python
redaction helper.
- Existing tests: 1100+ Rust + 76 Python regression checks green.
Docs:
- `docs/observability.md` — metric catalogue + PromQL queries.
- `docs/rtk-architecture.md` — locks the wrap-CLI-only decision so
future contributors don't relitigate proxy-side RTK.
No silent fallbacks: zero-denominator cache-hit-rate logs and
skips rather than synthesising 0.0. Unparseable rate-limit headers
stay None rather than coerced to 0. Missing upstream JSON fields
log + skip emit rather than fabricating data.
|
||
|
Tejas Chopra
|
eaf5980b4a | fix: stabilize codex compression, stats, and proxy lifecycle | ||
|
chopratejas
|
9112fed937 |
fix: PR-E2 recursive JSON Schema key sort (Phase E)
Recursively sort JSON Schema object keys inside each tool's schema so cache hits no longer depend on SDK-side serializer key-emission order (some sort, some preserve insertion, some hash-randomize). Wired into all three live-zone walkers, hooking the per-provider schema location: - Anthropic: `tool["input_schema"]` - OpenAI Chat: `tool["function"]["parameters"]` - OpenAI Responses: `tool["function"]["parameters"]` Same auth-mode gate as PR-E1 (PAYG only). NO marker check — the `cache_control` marker lives on the tool object itself, not inside the schema, so sorting schema keys never moves the marker. PR-E2 therefore runs even on tools that PR-E1 had to skip due to a present marker; the integration test pins this behaviour. Array semantics preserved: `oneOf`, `anyOf`, `allOf`, `prefixItems` and any other ordered JSON Schema array keep customer order; only object keys move. Idempotent — sorting an already-sorted schema yields byte-identical bytes (workspace `preserve_order` feature pins `serde_json::Map` emission to insertion order). Tests: unit tests for nested keys, oneOf preservation, deep nesting, and idempotency; integration tests boot the real proxy and assert PAYG -> sorted at every level, OAuth -> SHA-256 byte- equal, and PAYG-with-marker -> E1 skipped but E2 still runs. |
||
|
chopratejas
|
4a3b76bcc8 |
fix: PR-E1 tool array deterministic sort (Phase E)
Sort `tools[]` alphabetically by name on the way out so cache hits no
longer depend on the customer-side iteration order (commonly hash-
randomized via `set()` / `dict`). Mutates request bytes only when:
1. Auth mode is PAYG (`headroom_core::auth_mode::classify`).
2. No tool already carries a `cache_control` marker (reordering
would shift cache scope and silently void customer intent).
Every gate skip emits a structured `e1_skipped` event with `reason =
auth_mode | marker_present` so dashboards can see policy adoption.
Wired into all three live-zone walkers — Anthropic `/v1/messages`,
OpenAI `/v1/chat/completions`, OpenAI `/v1/responses` — plus the
Bedrock invoke + invoke-streaming entry points. Each passes
`auth_mode` (already pre-classified by Phase F PR-F1 middleware)
into the dispatcher so the gate evaluates without re-classifying.
Sort key uses `tool["name"]` (Anthropic) or `tool["function"]["name"]`
(OpenAI). Unnamed tools (rare; malformed inputs only) fall back to
MD5 of canonical-JSON serialization for a stable in-process key —
collision odds are astronomically small and `Vec::sort_by` is stable.
Tests: unit tests for sort + marker detection + idempotency + the
permutation property; integration tests boot the real proxy in front
of a wiremock upstream and assert PAYG -> sorted, OAuth/Subscription/
marker -> byte-equal passthrough (SHA-256).
|
||
|
chopratejas
|
573543fce7 |
fix: PR-E4 OpenAI prompt_cache_key auto-injection (Phase E)
OpenAI exposes `prompt_cache_key` to pin prefix-cache lookups to a tenant-stable identity (preventing org-wide cache collisions). Most clients don't set it. This PR auto-derives one from the request's structural prefix `(model, system, tools)` and injects it on PAYG OpenAI requests where the customer has not provided their own value. Universal safety contract: - Auth-mode gate: only AuthMode::Payg bodies are mutated. OAuth and Subscription requests pass through byte-equal (preserves Phase A passthrough invariant). Both gates emit `e4_skipped` events. - Customer-set values win: `prompt_cache_key` already present → skip injection. Empty strings count as absent. - Idempotent: same `(model, system, tools)` always derives the same key, so re-running yields identical bytes. Key derivation: `hex(sha256(model || sha256(system) || sha256(tools)))[..32]` — 128 bits of collision resistance, 32 hex chars on the wire. User/assistant message content is deliberately excluded (they vary per turn; including would defeat caching). Observability: every skip emits `e4_skipped` with a stable reason (`auth_mode` / `key_present` / `not_an_object`); every successful injection emits `e4_applied` with only the first 8 hex chars of the key (full key is identifying material — never logged). Hook point: `forward_http` in `crates/headroom-proxy/src/proxy.rs`, between the live-zone dispatcher's body decision and the upstream forward. Auth-mode is already classified at request entry. Affects pre-existing dispatcher byte-fidelity tests (chat completions, responses, responses streaming) — they previously asserted byte-equality with no auth header (default PAYG). Updated those tests to send an OAuth bearer so they keep their byte-equality intent independent of E4. The E4 byte-mutation behaviour has its own test matrix in `integration_e4_openai_cache_key.rs`. Files added: - `crates/headroom-proxy/src/cache_stabilization/openai_cache_key.rs` - `crates/headroom-proxy/tests/integration_e4_openai_cache_key.rs` Files modified: - `crates/headroom-proxy/src/cache_stabilization/mod.rs` — `pub mod openai_cache_key;` (only shared file with parallel E1/E2/E3/E6 PRs) - `crates/headroom-proxy/src/proxy.rs` — call site + helper - `crates/headroom-proxy/Cargo.toml` — promote sha2 to runtime dep - 3 integration test files — auth-mode plumbing for byte-equality invariants |
||
|
chopratejas
|
8672d5c326 |
fix: PR-E3 Anthropic cache_control auto-placement (Phase E)
Auto-place a single ephemeral cache_control marker on the last tool
definition for PAYG-classified Anthropic requests when the customer
has not placed any markers. Hand-rolled SDK callers and smaller
agents (Aider/Continue/curl) get prompt-cache hits without learning
Anthropic's marker API.
Safety contract:
1. Auth-mode (caller-side, F1 classify): PAYG only. OAuth and subscription requests pass through byte-equal — mutating their bytes risks looking like cache-evasion to upstream.
2. Customer-placement-wins: walks system (array form), messages[].content (array form), and tools[] top-level. Any pre-existing marker -> skip with reason=marker_present.
3. Idempotency: re-running on a body that already has our marker falls into gate (2).
First-ship policy: place ONE marker on the last tool. The system/message-history/4th slots are documented but require production telemetry to enable.
Bedrock invoke + invoke-streaming hard-code OAuth so AWS SigV4-signed requests never get auto-placed (Bedrock is an IAM channel, not PAYG).
Observability: tracing::info! event=e3_applied / event=e3_skipped (reason in {auth_mode, marker_present}) so dashboards can confirm the gates fire as designed.
Files added: cache_stabilization/anthropic_cache_control.rs (module + 15 unit tests); tests/integration_e3_anthropic_cache_control.rs (5 integration tests covering all three gates).
Files modified: cache_stabilization/mod.rs; compression/live_zone_anthropic.rs (new auth_mode parameter on compress_anthropic_request); proxy.rs; bedrock/invoke.rs; bedrock/invoke_streaming.rs.
|
||
|
chopratejas
|
c10a2195af |
fix(proxy): PR-D4 native Vertex publisher path + ADC bearer auth
Adds a Rust-native Vertex AI publisher route ahead of the LiteLLM
Python converter (which dropped `thinking`, `redacted_thinking`,
`document`, `image`, `server_tool_use`, `mcp_tool_use` block kinds —
the P4-37 / P4-38 bug). After this PR the Vertex `:rawPredict` and
`:streamRawPredict` calls survive byte-equal upstream and benefit
from the live-zone Anthropic dispatcher (PR-B-series) running over
the body — same behaviour as `/v1/messages`.
New module `crates/headroom-proxy/src/vertex/`:
- `mod.rs` — single dispatch handler at the
`/v1beta1/.../models/:model_action` route. Splits the trailing
`:<verb>` segment with `str::rsplit_once(':')` (no regex) and
flips an `attach_sse_tee` flag to dispatch to the streaming or
non-streaming arm. Both verbs share one axum route shape because
matchit can't distinguish two patterns that overlap on a
parameter.
- `envelope.rs` — `VertexEnvelope` parser. Confirms
`anthropic_version` present + `model` field absent (the two
fingerprints of the Vertex envelope vs `/v1/messages`).
- `adc.rs` — `TokenSource` trait + `GcpAdcTokenSource` (production,
`gcp_auth` 0.12) + `StaticTokenSource` (tests). Caches tokens
with a 60s refresh-ahead-of-expiry window. Emits structured
`event = "vertex_adc_token_refreshed"` per refresh.
- `raw_predict.rs` — POST handler + shared `forward_vertex_request`.
Buffers body, parses envelope, runs live-zone Anthropic
compression, fetches ADC bearer, attaches
`Authorization: Bearer <token>` (overwrites client-supplied
Authorization header), forwards. SSE telemetry tee for the
streaming verb reuses PR-C1's `AnthropicStreamState` directly
(Vertex streams plain SSE, unlike Bedrock's binary EventStream).
- `stream_raw_predict.rs` — module-level docs + alias to the
shared dispatcher (the streaming-vs-non-streaming difference is
one boolean flag inside the shared forwarder).
Modifications:
- `proxy.rs::build_app` — registers the single Vertex route.
- `proxy.rs::AppState` — new `vertex_token_source: Arc<dyn TokenSource>`
field. Production constructs `GcpAdcTokenSource` lazily (no GCP
call until first `bearer()`); tests inject `StaticTokenSource`
via the new `AppState::with_token_source` helper.
- `config.rs` — adds `--vertex-region` / `HEADROOM_PROXY_VERTEX_REGION`
(default `us-central1`, observability tag only — the upstream URL
is `--upstream`) and `--vertex-adc-scope` /
`HEADROOM_PROXY_VERTEX_ADC_SCOPE` (default `cloud-platform`).
- `Cargo.toml` (workspace + proxy) — adds `gcp_auth = "0.12"` and
`async-trait = "0.1"`.
- `tests/common/mod.rs` — `start_proxy_with_state` accepts both
config + state customizers; `install_static_token_source` helper
for tests.
`crates/headroom-proxy/tests/integration_vertex_raw_predict.rs` —
all five tests pass:
1. `native_envelope_round_trip_byte_equal` — Vertex-shape body
(with `anthropic_version`, no `model`) round-trips SHA-256
byte-equal upstream.
2. `adc_bearer_token_signed_correctly` — `Authorization: Bearer
<static-test-token>` reaches upstream verbatim and OVERWRITES a
client-supplied Authorization header.
3. `thinking_block_preserved` — request with `thinking` (incl.
signature) + `redacted_thinking` (incl. opaque `data`) blocks
round-trips byte-equal even with `LiveZone` compression mode
enabled. This is the P4-37 / P4-38 teeth.
4. `stream_raw_predict_sse_handled` — `:streamRawPredict` proxies
an Anthropic SSE response (full `message_start` →
`content_block_delta` → `message_stop` sequence) back to the
client without corruption; SSE content-type preserved end-to-end;
bearer attached.
5. (bonus, no-silent-fallback contract)
`adc_failure_returns_5xx_no_silent_forward` — when the token
source returns `Err`, the proxy returns 5xx and never reaches
upstream. Verifies the `event = "vertex_adc_fetch_failed"`
error path.
Workspace: `cargo test --workspace` green; `cargo clippy --workspace
-- -D warnings` clean; `make ci-precheck` passes.
- No silent fallbacks: ADC failure → structured 5xx, never an
unauthenticated forward.
- No hardcodes: every knob (region, ADC scope, upstream URL) is
CLI-flag + env-var configurable.
- No regexes: axum path parameters + `str::rsplit_once` only.
- Comprehensive structured logs: `event` field on every decision
point — `vertex_envelope_parsed`, `vertex_envelope_invalid`,
`vertex_compression_skipped`, `vertex_compression_applied`,
`vertex_adc_token_refreshed`, `vertex_adc_fetch_failed`,
`vertex_streaming_pipeline_active`, `vertex_sse_stream_closed`,
`vertex_forwarded`, `vertex_unknown_verb`, etc.
- Performant: no body clone; ADC token cached + refreshed
ahead-of-expiry, not fetched per request.
- Comprehensive tests: realistic Anthropic block content
(signature payload, redacted_thinking opaque blob) in
`thinking_block_preserved`.
The local `gcloud auth application-default print-access-token`
returns no credentials, so manual validation against a real Vertex
endpoint is not possible in this PR. Follow-up: the user runs
`gcloud auth application-default login` once and exercises a live
Vertex request — should be a no-code-change check.
PR-D1 (Bedrock native) is running concurrently and will land its
own envelope module at `crates/headroom-proxy/src/bedrock/envelope.rs`.
The two envelope modules are intentionally siblings (not a shared
trait) — the shapes differ (Bedrock has a different
`anthropic_version` value, no `model` field, AWS SigV4 instead of
GCP ADC), and a premature shared abstraction would obscure the
provider-specific contracts. Whichever PR merges second rebases
without conflict.
Retires P4-38 (and the Vertex parts of P4-39); marketplace BYOC
pitch (per project memory) gets one more native provider.
|
||
|
chopratejas
|
ce37940d17 |
fix: PR-E6 cache-bust drift detector telemetry (Phase E)
Per-session SHA-256 fingerprint of the cache hot zone (system / tools / first 3 messages) with structured-log emission on drift. Detector is read-only: never mutates request bytes, preserves the Phase A passthrough invariant. Surfaces invisible cache busts (system prompt edited mid-session, tools reshuffled, early message changed) without rewriting them. * crates/headroom-proxy/src/cache_stabilization/drift_detector.rs: StructuralHash (system, tools, early_messages digests), compute_structural_hash, observe_drift, derive_session_key, DriftState (LRU bounded to 1000 sessions in production). * Session keys derive from Authorization / x-api-key / client IP / (IP, user-agent). Bearer tokens and API keys are SHA-256 hashed before they ever reach the log line; the raw secret is never logged. * Wired into forward_http after the body is buffered, before the compression dispatcher runs. Skips paths whose wire shape is not Anthropic / OpenAI Chat / OpenAI Responses. * AppState gains drift_state: DriftState. Bedrock unit-test literal-construction sites updated. * 14 unit tests + 1 integration test covering first-request, no-drift, per-dimension drift, multi-dim drift, LRU eviction, non-mutation invariant, and bearer-token-never-logged. Adds lru = "0.12" and promotes sha2 = "0.10" to a normal dependency on headroom-proxy. |
||
|
chopratejas
|
d8aae382b0 |
fix: PR-E5 volatile-content detector + customer warning (Phase E)
Adds an observation-only detector that scans inbound LLM request
bodies for content that busts prompt-cache hits and emits one
structured WARN log per finding. Strictly read-only — never
mutates the request, the Phase A bytes-in==bytes-out invariant
still holds.
Detection patterns (no regex, per Realignment build constraints):
- ISO-8601 timestamps (byte-position check at 4=- 7=- 10=T 13=: 16=:)
- UUID v4 (36 chars, hyphens at 8/13/18/23, version nibble 4 at pos 14,
RFC 4122 §4.4 variant nibble at pos 19)
- ID-named JSON keys (request_id / trace_id / session_id /
correlation_id) with non-empty values
Scope:
- Anthropic body shape: system, messages[].content (string or blocks),
tools[].description, tools[].input_schema (recursive).
- OpenAI body shape: messages[].content, tools[].function.description,
tools[].function.parameters (recursive).
- Other paths (Bedrock / Vertex etc.) deferred to Phase E follow-up.
Findings capped at 10 per request; sample truncated to 80 bytes
(UTF-8 boundary safe) to avoid logging bulk customer data.
Files:
- New crates/headroom-proxy/src/cache_stabilization/{mod,volatile_detector}.rs
- crates/headroom-proxy/src/lib.rs: pub mod cache_stabilization
- crates/headroom-proxy/src/proxy.rs: detector hook in the
buffered-body branch, before the compression dispatcher
- New tests/integration_volatile_detector.rs (subscriber capture
asserts the WARN line + byte-equal upstream body)
12 unit tests (timestamp, UUID, ID-field nesting, stable content,
cap, non-mutation, ApiKind shape isolation, empty-ID-value guard,
RFC 3339 space separator, non-v4 UUID rejection, UTF-8 truncate
safety, endpoint mapping) and 1 integration test (warn emitted
+ upstream body byte-equal). cargo fmt / clippy --workspace -D
warnings / workspace test all pass; full make ci-precheck green.
|
||
|
chopratejas
|
90ef66213d |
fix(proxy): PR-D3 Bedrock observability + auth-mode integration
Phase D close. Adds the operator-facing observability surface that
PRs D1 (native invoke) and D2 (streaming EventStream) deferred, and
wires the Phase F PR-F1 auth-mode classifier into the Bedrock route
so downstream cache/compression policy gates have something to read.
Changes
-------
* New `bedrock::auth_mode_layer` middleware. Classifies every
inbound Bedrock request via F1's `classify`, coerces the result
to `AuthMode::OAuth` per the Bedrock policy matrix (SigV4 IAM is
OAuth-equivalent), and stores the resolved value in
`request.extensions()` so PR-F2/F3 can read it without
re-classifying. Mismatches are logged at WARN with
`event=bedrock_auth_mode_unexpected` — no silent coercion.
* New `observability` module with three Prometheus families:
- `bedrock_invoke_count_total{model, region, auth_mode}` (counter)
- `bedrock_invoke_latency_seconds{model, region}` (histogram)
- `bedrock_eventstream_message_count_total{model, region, event_type}`
(counter)
Registered lazily via `OnceLock` so per-request work is just
`inc_with_label_values` / `observe`. Latency observed via an
RAII `LatencyGuard` so every error path is instrumented; a
future regression that adds a new return path can't drop the
observation.
* New `GET /metrics` endpoint serves the registry in Prometheus
text format. Mounted unconditionally — no feature flag gate — so
scrape works regardless of which provider routes are mounted.
* Bedrock invoke + invoke-streaming handlers now extract
`Extension<AuthMode>`, log it in their entry breadcrumbs
(`event=bedrock_invoke_received`, `event=bedrock_invoke_streaming_received`),
and pass `model`/`region` into `translate_stream` so per-message
metrics carry the right labels.
* Operator docs at `docs/bedrock.md`: AWS credential chain,
region/endpoint config, supported model IDs (`anthropic.*`
literal-match — no regexes), compression behaviour, sample
PromQL queries, structured-log correlation, rollback path.
Tests added (6, all green)
--------------------------
Auth-mode (`integration_bedrock_authmode.rs`):
1. `bedrock_classified_as_oauth` — empty headers → OAuth in
extensions.
2. `oauth_policy_passthrough_prefer` — body byte-equal upstream;
no auto cache_control / prompt_cache_key injected.
Metrics (`integration_bedrock_metrics.rs`):
3. `metrics_increment_per_invoke` — 3 invokes → counter=3 with
correct labels.
4. `metrics_observe_latency` — 1 invoke → histogram count=1,
sum>0.
5. `eventstream_metrics_per_message_type` — 5 chunks → counter=5
with `event_type=chunk`.
6. `metrics_endpoint_serves_scrape` — `/metrics` returns 200,
`text/plain`, all three metric families' HELP/TYPE lines
present.
Each metrics test owns a unique (model, region) tuple so the
global `prometheus` registry — shared across parallel tests in
the same binary — gives each test isolated label rows. Without
isolation, parallel tests cross-contaminate counters.
Constraints honoured
--------------------
* No silent fallbacks — auth-mode coercion is logged at WARN.
* No hardcodes — region from `--bedrock-region`, model from axum
path parameter.
* No regexes — vendor prefix is literal `anthropic.`.
* Comprehensive structured logs — every metric increment paired
with `tracing::debug!` carrying the same labels for incident
correlation.
* Performant — `OnceLock`-cached descriptors, RAII guard, total
D3 overhead well under 1us per request.
* Cardinality bounded — labels driven by config + bounded enums,
never by user-controlled bytes.
Live cloud validation deferred
------------------------------
The wiremock-backed integration tests are the canonical correctness
gate for D3. A real Bedrock smoke test requires `bedrock:InvokeModel`
permissions in the developer's AWS account and is documented in
`docs/bedrock.md` — both D1 and D2 hit sandbox permission issues
trying this path; D3 follows the same convention.
Stacked on
----------
PR #364 (D1 native invoke), PR #365 (D2 streaming EventStream),
PR #366 (F1 classifier helper). Merge those first; this PR will be
rebased onto main once they land.
|
||
|
chopratejas
|
66426e7b75 |
fix(proxy): PR-D2 Bedrock streaming via binary EventStream
Add the Phase D PR-D2 streaming counterpart to PR-D1's native
Bedrock InvokeModel route.
Bedrock's `/model/{id}/invoke-with-response-stream` returns
`application/vnd.amazon.eventstream` — a binary, length-prefixed,
CRC32-checksummed framing format. This PR adds an incremental
parser, an SSE translator, and the streaming POST handler.
Components:
- `bedrock/eventstream.rs` — stateful incremental EventStream
parser. Validates prelude + message CRC32 (configurable via
`--bedrock-validate-eventstream-crc`, default on). Returns
structured `ParseError` on every malformed-bytes path; never
panics. Supports all 10 AWS header value types; bytes-typed
values surfaced via `HeaderValue::Bytes`, strings via
`HeaderValue::String`.
- `bedrock/eventstream_to_sse.rs` — translator. Picks output mode
per `Accept` header: `application/vnd.amazon.eventstream` →
byte-equal passthrough; everything else (default) → SSE
translation. Each `chunk` payload becomes a canonical Anthropic
`event: <type>\ndata: <json>\n\n` SSE frame so existing
`AnthropicStreamState` telemetry runs unchanged.
- `bedrock/invoke_streaming.rs` — POST handler. Reuses D1's
`BedrockEnvelope`, live-zone compression, SigV4 signing.
Tees translated SSE frames into `AnthropicStreamState` via the
same bounded-mpsc tee pattern as `/v1/messages` — byte path
never blocks on parser readiness.
Config:
- New `--bedrock-validate-eventstream-crc` / env
`HEADROOM_PROXY_BEDROCK_VALIDATE_EVENTSTREAM_CRC` flag, default
on. Disabling logs a warn at app-build time.
Routing:
- `proxy.rs::build_app` mounts
`POST /model/:model_id/invoke-with-response-stream` only when
`enable_bedrock_native` is on (matches D1).
Failure modes (all loud; no silent fallbacks):
- CRC mismatch → `event=bedrock_eventstream_crc_mismatch` warn,
closes the stream with an SSE error frame.
- Parse error → `event=bedrock_eventstream_parse_failed` warn +
SSE error frame.
- `:message-type == exception` → `event=bedrock_eventstream_upstream_exception`
warn + SSE error frame.
- Unknown `:event-type` →
`event=bedrock_eventstream_unknown_event_type` warn, skipped.
- Missing creds / SigV4 fail → 5xx, identical to D1.
Tests added (12 total):
- 4 parser unit-style integration: byte-equal round trip, drip-feed
one-byte-at-a-time, CRC mismatch surfaces structured error,
validation-off accepts corrupt.
- 3 end-to-end: `eventstream_translated_to_sse`,
`usage_extracted_from_translated_stream`,
`client_can_choose_eventstream_or_sse`.
- 2 property tests via `proptest`: random bytes never panic the
parser (1024 cases each: bulk + drip-feed).
- 3 trivial smoke tests in unit modules
(`eventstream::tests::*`, `eventstream_to_sse::tests::*`).
Manual cloud validation:
- Not exercised — running `aws bedrock-runtime invoke-model-with-
response-stream` against the proxy in the sandbox would require
AWS API access this environment does not have. The wiremock-
served binary EventStream + property tests cover the parser
semantics and CRC validation rigorously.
Stacked on PR-D1 (#364). Will be rebased onto main once D1 lands.
|
||
|
chopratejas
|
f2d4fe39cb |
fix(proxy): PR-D1 native Bedrock InvokeModel route + SigV4
Phase D PR-D1 lands the first native Rust path for AWS Bedrock,
replacing the lossy LiteLLM Python converter for Anthropic-on-Bedrock
non-streaming requests. Eliminates part of P4-37 and P4-39.
What landed
-----------
- New crates/headroom-proxy/src/bedrock/ module:
- envelope.rs: parses the {"anthropic_version": "...", ...}
Bedrock body shape; re-emits with anthropic_version preserved
as the first key (relies on serde_json preserve_order).
- sigv4.rs: AWS SigV4 signer wrapping the aws-sigv4 crate.
Forces PayloadChecksumKind::XAmzSha256 so x-amz-content-sha256
is in the canonical request, hashed over the post-compression
body bytes (the bytes that actually hit Bedrock). No silent
fallback: signing failures return 5xx with
event=bedrock_sigv4_failed.
- invoke.rs: POST handler for /model/{model_id}/invoke
(and /converse - same wire shape for anthropic.claude-*).
Detects Anthropic vendor via literal starts_with("anthropic.")
(no regex per project rule), routes Anthropic-shape bodies
through the existing compress_anthropic_request live-zone
dispatcher, then signs and forwards to the configured Bedrock
endpoint.
- Modified:
- proxy.rs: routes /model/:model_id/invoke and
/model/:model_id/converse when enable_bedrock_native is on
(default). Adds bedrock_credentials: Option<Arc<Credentials>>
to AppState.
- config.rs: new flags --bedrock-region (default us-east-1,
env HEADROOM_PROXY_BEDROCK_REGION), --bedrock-endpoint
(operator override for FIPS/VPC/test setups),
--enable-bedrock-native (default true), --aws-profile.
- main.rs: resolves AWS credentials at startup via
aws_config::defaults(BehaviorVersion::latest()). Failure logs
event=bedrock_credentials_unavailable at WARN; the handler
refuses to forward unsigned (event=bedrock_credentials_missing).
- Cargo.toml: workspace deps aws-sigv4, aws-config,
aws-credential-types, aws-smithy-runtime-api.
Tests
-----
8 integration tests under
crates/headroom-proxy/tests/integration_bedrock_invoke.rs:
1. native_envelope_round_trip_byte_equal
2. sigv4_signed_correctly_after_compression - confirms
authorization is SigV4-shape and x-amz-content-sha256
matches sha256(body received by upstream).
3. thinking_block_preserved_through_bedrock
4. redacted_thinking_preserved
5. document_block_preserved
6. tool_result_array_with_image_preserved
7. stop_sequence_null_only_when_present - pins that the proxy
does NOT inject stop_sequence: null (P4-37 hardcode).
8. tool_use_input_byte_equal_preserves_key_order
All eight pass. Full workspace test run is green; clippy + fmt
clean. make ci-precheck (rust + python + commitlint) passes
locally.
Build constraints honoured
--------------------------
- No silent fallbacks: missing creds / signing failures return
5xx with structured event=... log; no path ever forwards
unsigned.
- No hardcodes: region, endpoint, profile, enable-flag all
configurable via CLI + env.
- No regexes: vendor detection is str::starts_with.
- Comprehensive structured logs: event=bedrock_invoke_received,
bedrock_envelope_parsed, bedrock_compression_skipped,
bedrock_credentials_missing, sigv4_signed,
bedrock_invoke_forwarded, etc.
- Performant: body buffered once, passed by &[u8] to signer
(zero-copy), Bytes::clone only for ownership transfer to
reqwest. Sign exactly once per request.
- Elegant: 4 small focused modules mirror handlers/ + sse/.
- Tests use realistic Anthropic block content (real thinking,
redacted_thinking, document, base64 image fixtures).
Streaming (PR-D2) and observability (PR-D3) follow.
|
||
|
chopratejas
|
866d346bd0 |
fix: C4 — /v1/responses streaming + Conversations API in Rust
Phase C PR-C4 of the Realignment. Completes the Responses surface
in the Rust proxy and lays the Conversations passthrough rails.
# /v1/responses streaming pipeline
C3 left an INFO-WARN breadcrumb (`responses_streaming_passthrough_until_c4`)
on every SSE-Accept request because the byte-level SSE framer +
ResponseState machine wired by C1 was not formally tied into the
endpoint's contract. C4 closes the loop:
- Replaces the C3 fallback warn with a structured-log INFO event
`event = "responses_streaming_pipeline_active"`. The bytes flow
unchanged (compression on streamed response output is OUT of scope
per live-zone-only contract); telemetry runs in the parallel task.
- New emergency-rollback toggle `--enable-responses-streaming`
(env: `HEADROOM_PROXY_ENABLE_RESPONSES_STREAMING`, default `true`).
When `false`, the OpenAI-Responses SSE state machine is skipped
and `event = "responses_streaming_state_machine_skipped"` is
emitted; bytes still pass through. Other providers' state
machines are unaffected.
- Request-side compression (the C3 live-zone dispatcher) continues
to run on streaming requests — `should_intercept` does not gate on
Accept, so SSE clients get the same body compression as
non-streaming clients.
# Conversations API surface (`/v1/conversations*`)
New `crates/headroom-proxy/src/handlers/conversations.rs` mounts
explicit axum routes (no regex per build constraints) for OpenAI's
stateful thread surface:
- `POST /v1/conversations`
- `GET /v1/conversations/{id}`
- `POST /v1/conversations/{id}` (metadata update)
- `DELETE /v1/conversations/{id}`
- `POST /v1/conversations/{id}/items`
- `GET /v1/conversations/{id}/items`
- `GET /v1/conversations/{id}/items/{item_id}`
- `DELETE /v1/conversations/{id}/items/{item_id}`
For PR-C4 each handler is passthrough-with-instrumentation: forward
upstream byte-equal via `forward_http`, emit
`event = "conversations_passthrough_pr_c4"` with route name +
extracted IDs. Compression of stored conversation items is C5+/B-phase
territory and explicitly NOT done here. Bodies are streamed (no
buffering) — `is_compressible_path` does not match
`/v1/conversations*` so the compression gate's else-branch streams
the body via `reqwest::Body::wrap_stream`.
New toggle `--enable-conversations-passthrough` (env:
`HEADROOM_PROXY_ENABLE_CONVERSATIONS_PASSTHROUGH`, default `true`).
When `false`, the per-route handlers are NOT mounted — requests
still reach upstream via the catch-all (no per-route logs); a
single WARN at app-build time confirms the rollback.
# Tests added
- `tests/integration_responses_streaming.rs` (4 tests):
request bytes byte-equal upstream on streaming; client receives
bytes that round-trip through the same SseFramer + ResponseState
the proxy spawns; rollback path still passes bytes;
below-threshold streaming request round-trips byte-equal.
- `tests/integration_conversations.rs` (10 tests):
every CRUD endpoint passthrough byte-equal through wiremock;
4xx upstream errors surface verbatim (no swallowing);
passthrough disabled still falls through to catch-all byte-equal.
- `tests/sse_openai_responses.rs` (+2 tests):
`chunk_boundary_invariance_pr_c4` — every single-byte split point
produces the same final state (cache-safety streaming property);
`minimal_upstream_response_pr_c4` — empty `[DONE]` upstream never
panics the state machine.
# Structured-log events introduced
- `responses_streaming_pipeline_active` (INFO) — replaces the C3 WARN
- `responses_streaming_pipeline_disabled` (WARN, only when toggle off)
- `responses_streaming_state_machine_skipped` (INFO, in proxy.rs)
- `conversations_passthrough_pr_c4` (INFO, per request)
- `conversations_passthrough_disabled` (WARN, app-build time)
# Config keys introduced
- `enable_responses_streaming: bool` (default `true`)
- `enable_conversations_passthrough: bool` (default `true`)
# Verification
- `cargo build --workspace --all-features`: green
- `cargo test --workspace --all-features`: 793 + 14 + ... all green
- `cargo clippy --workspace --all-features -- -D warnings`: zero
- `cargo fmt --all --check`: green
- `make ci-precheck-rust`: green
- `make ci-precheck-python` (against main repo .venv): 176 passed
No Python files modified; Conversations compression deferred to C5+
and follow-up B-phase work that retires
`headroom/proxy/responses_converter.py`.
|
||
|
chopratejas
|
57c3e38cb3 |
fix: C3 — /v1/responses Rust HTTP handler + per-item-type passthrough
Ports the OpenAI Responses API request path to Rust with first-class per-item-type handling, replacing the fragile `headroom/proxy/responses_converter.py` shim that flattens Responses-shape items into Chat-Completions-shape (and silently breaks every time OpenAI lands a new item type). What lands: - New `compress_openai_responses_live_zone` dispatcher in `headroom-core` (sibling of the Anthropic / Chat Completions ones) that walks the `input` array (with `messages` accepted as legacy alias) and identifies the latest of each compressible kind: `function_call_output`, `local_shell_call_output`, `apply_patch_call_output`, plus the latest user-role `message`. Earlier *_output items are FROZEN (cached prefix). All other item types pass through verbatim via byte-range surgery. - New 2 KiB output-item floor for `*_output` items (per spec PR-C3 line 167) on top of the existing per-content-type byte thresholds. - `crates/headroom-proxy/src/responses_items.rs` exposes a typed `ResponseItem<'a>` enum for telemetry / decision-making, paired with a `ClassifiedItem` two-pass classifier that keeps the original `&RawValue` slice so byte fidelity is preserved independently of the typed view. `Cow<'a, str>` on string fields handles both borrowed (no-escape) and owned (escape-bearing) JSON values without allocation on the common path. - `crates/headroom-proxy/src/compression/live_zone_responses.rs` proxy-side dispatcher mirrors the Chat Completions shape (same `Outcome` / `Passthrough` arms, same structured logs, same manifest aggregation). Logs unknown `type` values at warn (`event = responses_unknown_item_type`) and never strips them. - `crates/headroom-proxy/src/handlers/responses.rs` POST handler buffers the body and re-injects via `forward_http`. Detects `Accept: text/event-stream` and emits `event = responses_streaming_passthrough_until_c4` so we can measure the volume before C4 wires the streaming state machine. - Image-generation log redaction: `image_generation_call` items are logged with byte size only (no `image_data` in the log path). The upstream-bound bytes are NOT mutated — redaction is log-channel only, per spec. - `CompressibleEndpoint::OpenAiResponses` variant; route `POST /v1/responses` wired in `proxy.rs`. Tests added (~40 across the workspace): - 9 core dispatcher unit tests in `live_zone.rs` - 4 proxy dispatcher unit tests + 5 handler unit tests + 5 typed-enum unit tests - 16 integration tests in `tests/integration_responses.rs` covering: V4A patch byte-equality, argv-array preservation, Codex `phase=commentary`/`final_answer`, compaction passthrough, reasoning passthrough, function_call.arguments string preservation, call_id-vs-id distinction, 2 KiB output-item floor (below + above), local_shell output compression, MCP / computer-use / image generation passthrough, unknown-type warn-and-preserve, and a representative round-trip with reasoning + function_call + local_shell + apply_patch + custom items. Per-PR-C3 plan: REALIGNMENT/05-phase-C-rust-proxy.md. |
||
|
chopratejas
|
fe00dc006a |
fix: C2 — /v1/chat/completions Rust handler + OpenAI live-zone
Adds a POST handler for /v1/chat/completions and a sibling live-zone dispatcher for the OpenAI Chat Completions request shape. Same compressor backend as Anthropic (SmartCrusher / LogCompressor / SearchCompressor / DiffCompressor), same per-content-type byte thresholds, same tokenizer-validated rejection gate, same byte-range surgery for cache-stable rewrite. Live zone for Chat Completions: the latest role=tool message's content AND the latest role=user message's text content. Earlier tool/user messages are part of the cache hot zone; never touched. tools[] and tool_choice are never read or rewritten — they round-trip byte-equal as a side effect of byte-range surgery. Behaviours: - n > 1 → passthrough (multiple completions imply non-determinism; the proxy gate skips dispatch and forwards original bytes). - stream: true → pass through to forward_http's existing C1 SSE parser tee (ChunkState). - tool_choice change → never mutated. - mode == Off → passthrough with structured 'mode_off' log. - Body not JSON / no messages → passthrough; the dispatcher logs the decision and forwards original bytes. The handler is wired as an explicit POST route on /v1/chat/completions, buffers the body into Bytes, and re-injects it into the shared forward_http function. forward_http's compression gate now classifies the path (AnthropicMessages vs OpenAiChatCompletions) and dispatches to the right module (compress_anthropic_request / compress_openai_chat_request). Single forwarding code path keeps SSE telemetry, header stripping, and request-id plumbing single-source. Files added: - crates/headroom-proxy/src/handlers/chat_completions.rs - crates/headroom-proxy/src/handlers/mod.rs - crates/headroom-proxy/src/compression/live_zone_openai.rs - crates/headroom-proxy/tests/integration_chat_completions.rs Files modified: - crates/headroom-core/src/transforms/live_zone.rs (+compress_openai_chat_live_zone, +helpers) - crates/headroom-core/src/transforms/mod.rs (re-export) - crates/headroom-proxy/src/compression/mod.rs (+CompressibleEndpoint classification, expose live_zone_openai) - crates/headroom-proxy/src/lib.rs (expose handlers module) - crates/headroom-proxy/src/proxy.rs (route + dispatch) Tests: - 7 integration tests in tests/integration_chat_completions.rs covering passthrough byte-equality, tool message compression (≥40% reduction on 1500-row JSON-array fodder), n>1 passthrough, stream_options round-trip, tool_choice non-mutation, refusal delta handling via ChunkState, and tool_call argument accumulation across three streaming chunks. - Unit tests on compress_openai_chat_live_zone (6) and compress_openai_chat_request (7) cover the dispatcher and proxy shim independently. Workspace test count: 953 (after C1) → 975. fmt clean. clippy --all-targets --all-features -D warnings clean. make ci-precheck PASSED. Plugin marketplace versions auto-bumped by the sync-plugin-versions pre-commit hook. Per-PR-C2 plan: REALIGNMENT/05-phase-C-rust-proxy.md. |
||
|
chopratejas
|
ddc6f6ceb0 |
fix: C1 — byte-level SSE parser + state machines
Foundation of Phase C. Delivers:
* Byte-level SSE framing (bytes::Bytes / BytesMut) with UTF-8
decoded only at \n\n event boundaries — no per-chunk decode,
no errors=ignore data loss across TCP reads.
* Three provider state machines:
- Anthropic: blocks keyed by index, all delta types
(text/thinking/input_json/citations/signature) preserved
byte-equal.
- OpenAI Chat: ToolCallState concatenation, refusal field,
include_usage final chunk handling.
- OpenAI Responses: items keyed by id (not position) for
out-of-order completion; full event coverage.
* State machine runs in parallel with byte-passthrough via a
tokio::spawn task fed by a bounded mpsc — clients see raw
bytes immediately; telemetry populates without blocking.
Retires P1-8, P1-9, P1-14, P1-15, P1-17, P4-48 in the Rust path
(Python A8 hotfix preserved as fallback until Phase H).
Per-PR-C1 plan: REALIGNMENT/05-phase-C-rust-proxy.md.
|
||
|
chopratejas
|
2b55050a65 |
fix: B3 — wire type-aware compressors into live-zone dispatcher
Phase B step 3: replace PR-B2's no-op dispatcher with real per-block
compression. SmartCrusher / LogCompressor / SearchCompressor /
DiffCompressor are wired behind content-type detection. SourceCode
and PlainText remain no-op for now (Rust code-compressor port and
Kompress prose compressor land in follow-up work; they're explicit
TODOs in `dispatch_compressor`).
# What's wired
For each block in the latest user message (live zone):
| Detected type | Compressor | Strategy tag |
|---------------|------------------|--------------------|
| `JsonArray` | SmartCrusher | `smart_crusher` |
| `BuildOutput` | LogCompressor | `log_compressor` |
| `SearchResults` | SearchCompressor | `search_compressor` |
| `GitDiff` | DiffCompressor | `diff_compressor` |
| `SourceCode` | (no-op, Rust port pending) | |
| `PlainText` | (no-op, PR-B4 wires Kompress) | |
| `Html` | (no-op, no compressor) | |
Anthropic-specific block types (`tool_use`, `thinking`,
`redacted_thinking`, `compaction`) stay tagged `BlockAction::Excluded`
so they remain in the cache hot zone even when they appear in the
live-zone message.
# Cache-safety invariant — byte-range surgery
The PR replaces "deserialize → mutate → serialize" with byte-range
surgery: the dispatcher uses `serde_json::value::RawValue` borrowed
slices and pointer arithmetic to recover each block's exact byte
offset in the input buffer, then splices replacement bytes
in-place. Bytes outside any rewritten range are *literally copied*
from the input, never re-serialized.
The new integration test
`crates/headroom-core/tests/live_zone_dispatch.rs::byte_fidelity_outside_compressed_block`
pins this in CI: SHA-256 of `body[..block_start]` and
`body[block_end..]` must equal the input's, AND the block must
shrink by >2× on a 50 KB JSON-array tool_result.
# Provider scope (Phase B is Anthropic-only)
The entry point is renamed `compress_live_zone` →
`compress_anthropic_live_zone` to make scope explicit. OpenAI Chat
Completions, OpenAI Responses, and Google Gemini each need their
own dispatcher because the request shapes diverge: OpenAI puts
tool results in `role: "tool"` messages (not nested in user),
Responses uses `input` with `function_call_output` items, Gemini
uses `contents`/`parts`/`function_response`. Phase C
(`REALIGNMENT/05-phase-C-rust-proxy.md`) introduces those
dispatchers; they share `LiveZoneOutcome`, `BlockAction`,
`CompressionManifest` and the per-content-type compressor backend
from this module.
# BlockAction taxonomy (replacing PR-B2's `NoOpSkeleton`)
- `Compressed { strategy, original_bytes, compressed_bytes }` —
compressor ran and produced strictly smaller output; spliced in.
- `RejectedNotSmaller { strategy, original_bytes, compressed_bytes }`
— compressor ran but didn't shrink; original kept. PR-B4 swaps
this byte-length proxy for a tokenizer-validated count.
- `CompressorError { strategy, error }` — compressor failed loudly.
Per project memory `feedback_no_silent_fallbacks.md`, surfaced in
the manifest; proxy logs warn-level and forwards original bytes
for that block; other blocks in the same body still compress.
- `NoCompressionApplied { content_type }` — content type has no
applicable compressor (PlainText, SourceCode, Html, Image,
Unknown). Replaces PR-B2's `NoOpSkeleton` as the default.
- `Excluded { reason }` — block intentionally outside live zone
(HotZoneBlockType, BelowFrozenFloor, AboveLiveZone).
# Sequential per-block dispatch (parallelism deferred)
Per-block compression is sequential in B3. Most requests have 1-3
blocks in the latest user message; the rayon/spawn_blocking
overhead approaches the savings below ~4 blocks. PR-B4 will add
async coordination per block (since token validation needs an
async hop anyway) — that's the natural place to add parallelism
guarded by a benchmark-driven threshold.
# Observability
The proxy log line gains the new fields when bytes are rewritten:
- `decision="compressed"`, `reason="live_zone_blocks_rewritten"`
- `body_bytes_in`, `body_bytes_out`, `bytes_freed`
- `live_zone_strategies` (Vec of unique strategy tags)
- `live_zone_block_original_bytes`, `live_zone_block_compressed_bytes`
The PR-B2 `decision="no_change"` arm is preserved with
`reason="no_block_compressed"`.
# Files
- `crates/headroom-core/src/transforms/live_zone.rs` (≈1100 LOC,
+900 from B2): byte-range surgery; `dispatch_compressor` switch;
`OnceLock` singletons for SmartCrusher / Log / Search / Diff;
expanded `BlockAction` enum.
- `crates/headroom-proxy/src/compression/live_zone_anthropic.rs`:
translates `LiveZoneOutcome::Modified` → `Outcome::Compressed`
with aggregated manifest counters.
- `crates/headroom-core/tests/live_zone_dispatch.rs` (NEW):
routing tests + 50 KB byte-fidelity invariant test.
- `crates/headroom-proxy/tests/integration_compression.rs`: log
contract updated to `reason="no_block_compressed"`.
# Acceptance
- `cargo build --workspace` + `clippy` + `fmt` green.
- `cargo test --workspace --exclude headroom-py`: 881 passed.
- 6 new integration tests in `live_zone_dispatch.rs`:
json/log/diff routing, source-code no-op, unknown no-op,
byte-fidelity (50 KB → >2× reduction with byte-equal envelope).
- Existing 12 unit tests in `live_zone.rs` still pass.
Per-PR-B3 plan: REALIGNMENT/04-phase-B-live-zone.md.
|
||
|
chopratejas
|
e190544c77 |
fix: B2 — live-zone block dispatcher skeleton
Phase B step 2 of the live-zone-only realignment. Replaces PR-A1's
unconditional "passthrough" stub with a real dispatcher that
inspects the Anthropic /v1/messages body, identifies the live zone
(latest user message at index >= frozen_message_count), and routes
each block to a per-type compressor. PR-B2 wires every per-type
compressor to a no-op, so the dispatcher returns
LiveZoneOutcome::NoChange on every call — bytes-in == bytes-out.
PR-B3+ replaces the no-ops with SmartCrusher, Log, Search, Diff,
and Code compressors.
Adds:
- crates/headroom-core/src/transforms/live_zone.rs — public API:
- `compress_live_zone(body, frozen_message_count, AuthMode)`
- `LiveZoneOutcome::{NoChange, Modified}`
- `CompressionManifest` with per-block outcomes (message_index,
block_index, block_type, BlockAction).
- `BlockAction::{NoOpSkeleton, Excluded { reason }}`. The
HOT_ZONE_BLOCK_TYPES list (`tool_use`, `thinking`,
`redacted_thinking`, `compaction`) excludes blocks even when
they appear in the latest user message.
- `AuthMode::{Payg, OAuth, Subscription}` — accepted but unused
in B2; PR-F2 wires the auth-mode gate.
- 12 unit tests pin: empty messages, no messages field, invalid
JSON, latest user message selection, frozen_count respect,
hot-zone block exclusion, string-shaped content, no user msg
in live zone, AuthMode no-op, NoChange contract, manifest
counters, frozen-count clamping.
- crates/headroom-proxy/src/compression/live_zone_anthropic.rs —
new entry point. `compress_anthropic_request` parses the body,
resolves frozen_count via `resolve_frozen_count` (PR-A4 helper),
dispatches via `compress_live_zone`, and returns
`Outcome::NoCompression` on PR-B2 success / `Outcome::Passthrough
{ reason: NotJson | NoMessages | ModeOff }` on body-shape /
policy issues. Six unit tests pin: mode_off short-circuit, no
messages field, invalid JSON, valid body NoCompression,
empty body, cache_control disabled.
Modifies:
- compression/mod.rs — re-exports `compress_anthropic_request` from
`live_zone_anthropic` instead of `anthropic`. The old anthropic
module is reduced to the `resolve_frozen_count` helper only
(not deleted, because its CacheControlAutoFrozen-policy gate is
reused).
- proxy.rs — passes `state.config.cache_control_auto_frozen` into
the dispatcher. Drops the obsolete "live_zone reserved for
Phase B" warning that PR-A1 emitted on every request.
- compression/anthropic.rs — pruned to the resolve_frozen_count
helper plus its tests. The PR-A1 passthrough stub
`compress_anthropic_request` is gone (live_zone_anthropic owns
the name now).
- config.rs — `compression_mode` doc updated to reflect the wired
dispatcher (no longer "reserved for Phase B").
- tests/integration_compression.rs — `compression_decision_logged`
pins the new log contract (`decision="no_change"`,
`reason="no_op_skeleton_pr_b2"`, plus manifest fields
`frozen_message_count`, `messages_total`, `live_zone_blocks`).
Asserts the obsolete Phase A warning is NOT emitted.
- proxy.rs no longer imports CompressionMode (only used inside the
retired warning).
Benchmark cleanup (B1 leftovers that surfaced now):
- benchmarks/proxy_mode_benchmark.py + claude_session_mode_benchmark.py:
drop `intelligent_context=False` arg from ProxyConfig (the field
was retired in B1; tests/test_proxy_mode_benchmark.py and
tests/test_claude_session_mode_benchmark.py imported these
factories and started failing).
- benchmarks/bench_transforms.py: delete TestRollingWindowBenchmarks
class; rewire TestTransformPipelineBenchmarks fixture without
RollingWindow.
- benchmarks/conftest.py: drop rolling_window_config fixture.
- benchmarks/run_benchmarks.py: drop the `window` suite + table
rows referencing RollingWindow.
Cache-safety invariant:
- PR-B2 dispatcher never mutates body bytes (no-op skeleton). The
proxy forwards the original buffered bytes byte-equal. Phase A's
SHA-256 fixtures pin this.
- `passthrough_mode_live_zone_currently_passthrough_byte_equal_sha256`
retitled comment to reflect the dispatcher being live but
no-op.
Acceptance:
- cargo build --workspace + clippy + fmt: green.
- cargo test --workspace --exclude headroom-py: all green
(777 + 12 new live_zone + 6 new live_zone_anthropic tests).
- pytest: 4678 passed, 240 skipped, 0 failed.
- Anthropic decision log includes manifest fields per the
observability contract documented in
REALIGNMENT/02-architecture.md.
Per-PR-B2 plan: REALIGNMENT/04-phase-B-live-zone.md.
|
||
|
chopratejas
|
148ded392a |
fix: A8 — SSE delta arms, UTF-8 buffer, phase preservation, request-id, 413
Eliminates the Python wire-format hotfix bugs gated on Phase A's
lockdown so the proxy is safe through Phase H's Python retirement.
Bugs retired:
- P0-7 / P4-44: Codex `phase` field is now explicitly preserved
through the Responses-API ↔ Chat-Completions round-trip; multi
text-part rebuild collapses to a single text part (no more
content doubling).
- P1-8: Bytes-level SSE event splitter
`parse_sse_events_from_byte_buffer`; emoji/CJK split across
chunks survive intact. Buffer is `bytearray`; UTF-8 decode happens
only AFTER the `\n\n` event terminator is located in bytes.
Invalid UTF-8 in a *complete* event raises (operator-visible
diagnostic, not silent corruption).
- P1-9: `_parse_sse_to_response` handles all delta types per
Anthropic guide §5.1: `thinking_delta`, `signature_delta`,
`citations_delta`. Block map keyed by `index` so out-of-order
events reconstruct correctly. `redacted_thinking.data` preserved.
- P4-47: Unknown Responses-API item types now log a structured
`unknown_responses_item_type` warning so operators see new
Codex item types in flight before they break.
- P5-57: Rust proxy captures upstream `request-id` (Anthropic) and
`x-request-id` (OpenAI); surfaced as `headroom-upstream-request-id`
on the response and as a tracing span field. Distinct from the
proxy's own `x-request-id`.
- P5-59: Body-too-large now returns 413 (was 400). Pre-checks
`Content-Length` and rejects without consuming the body when
present; chunked uploads still buffer-then-fail with 413.
Configurability (no hardcodes):
- HEADROOM_SSE_BUFFER_MAX_BYTES (default 1 MiB) — per-event cap.
- HEADROOM_PROXY_BODY_TOO_LARGE_STATUS (default 413) — operator
override for body-too-large status.
A7 follow-up: `_DummyAnthropicHandler._retry_request` accepts the
A3 byte-faithful kwargs (`original_body_bytes`, `body_mutated`,
`mutation_reasons`, `request_id`, `forwarder_name`, `path_for_log`)
so the existing 20 backpressure tests stay green against the real
handler signature.
The project-wide grep
git grep 'errors="ignore"\|errors="replace"' headroom/proxy/handlers/ headroom/ccr/
returns nothing; the single remaining lossy-decode site (response-
body diagnostics, not SSE) routes through `safe_decode_for_logging`
in `headroom/proxy/helpers.py`.
Tests:
- tests/test_sse_thinking_blocks.py (4 tests)
- tests/test_sse_utf8_split.py (3 tests)
- tests/test_proxy_responses_phase_preservation.py (4 tests)
- crates/headroom-proxy/tests/integration_request_id.rs (2 tests)
- crates/headroom-proxy/tests/integration_body_size.rs (2 tests)
|
||
|
chopratejas
|
2e874c5e3e |
fix: A5 — strip x-headroom-* from upstream-bound headers (P5-49)
Eliminate P5-49: every Python forwarder and the Rust transparent proxy
now drop internal `x-headroom-*` request headers (`x-headroom-bypass`,
`x-headroom-mode`, `x-headroom-user-id`, `x-headroom-stack`,
`x-headroom-base-url`) before the upstream call. Stops fingerprinting
of the proxy by subscription-revocation enforcers and prevents leakage
of internal user-id / stack / base-url internals to whichever vendor
terminates the request.
Python:
- `_strip_internal_headers(headers)` in `headroom/proxy/helpers.py`
returns a NEW dict with `x-headroom-*` keys removed (case-insensitive
prefix match, no regex). Pure function. Operator opt-in
`HEADROOM_STRIP_INTERNAL_HEADERS=disabled` keeps internal headers in
the upstream-bound dict for diagnostic shadow tracing — explicit, not
a fallback.
- Strip applied at every handler entry capture in `anthropic.py`,
`openai.py`, `batch.py`, `gemini.py` (chat completions, responses,
WebSocket handshake, Copilot passthrough, batch passthroughs, Gemini
generate / stream / countTokens / cloudcode-assist, Anthropic
passthrough + batch results). Inbound reads of x-headroom (bypass
gating, memory user-id) migrated to `request.headers.get(...)` so
they continue working off the original dict.
- `log_outbound_headers` emits `event=outbound_headers forwarder=...
stripped_count=N request_id=...` per call. Never logs header values.
Rust (crates/headroom-proxy):
- `strip_internal_headers(&mut HeaderMap)` and `is_internal_header`
helpers in `src/headers.rs`. `build_forward_request_headers` accepts
a `strip_internal: bool` so the same path serves HTTP and WebSocket.
- `Config::strip_internal_headers: StripInternalHeaders` driven by CLI
flag `--strip-internal-headers` and env var
`HEADROOM_PROXY_STRIP_INTERNAL_HEADERS` (default `enabled`).
- `proxy.rs` and `websocket.rs` call `build_forward_request_headers`
with the resolved policy; structured `tracing::info!` /
`tracing::warn!` line per request describes the strip decision.
Tests: 24 Python (`tests/test_header_isolation.py`) + 4 Rust
integration (`crates/headroom-proxy/tests/integration_headers.rs`) +
4 Rust unit tests in `headers.rs`. Covers every named header
(`bypass`, `mode`, `user-id`, `stack`, `base-url`), case-insensitive
prefix matching, legitimate-headers passthrough, the `disabled`
operator-opt-in mode, and that the inbound bypass-gating read path
is unaffected by the strip.
Acceptance: targeted `pytest -x` suite green (87 tests across
test_header_isolation, test_proxy_byte_faithful_forwarding,
test_proxy_anthropic_cache_stability, test_proxy_system_prompt_immutable,
test_proxy_openai_cache_stability, test_proxy_pipeline_lifecycle).
`cargo test -p headroom-proxy` green (23 tests across all integrations
plus 7 lib unit tests). `cargo clippy -p headroom-proxy -- -D warnings`
clean. `cargo fmt --all -- --check` clean. `cargo test --workspace`
green (~900 tests total).
Per realignment build constraints: configurable (env + CLI), no
hardcodes, no regex (pure `.lower().starts_with()` match), no silent
fallbacks (`disabled` is loud operator opt-in), structured logs
(`event=outbound_headers`).
Remaining `x-headroom-` references in `headroom/proxy/handlers/` are
inbound-read sites only: `request.headers.get("x-headroom-bypass")` /
`x-headroom-mode` for behavior gating, `request.headers.get
("x-headroom-user-id")` for memory user-id resolution, and `ws_headers
.get(...)` on the WebSocket inbound path. Response-side `X-Headroom-*`
injection (e.g. `x-headroom-tokens-saved`) is unrelated to upstream
forwarding and untouched.
|
||
|
chopratejas
|
3f99128236 |
fix(rust): A4 — honor cache_control markers; serde_json arbitrary_precision + raw_value
PR-A4 of the Realignment Phase A lockdown
(REALIGNMENT/03-phase-A-lockdown.md). Eliminates P0-3 (Rust proxy
ignores customer cache_control markers) and P0-5 (numeric precision
lost via serde_json::Value round-trip) at the library level; Phase B
PR-B2 wires the helper into the live-zone block dispatcher.
Cargo.toml — add `arbitrary_precision` and `raw_value` to
`serde_json` workspace features. `arbitrary_precision` keeps `1.0`
from collapsing to `1` and preserves >2^53 integers; `raw_value`
exposes `&RawValue` so PR-B2 can forward unmodified `messages[*]`
entries as exact byte copies.
crates/headroom-core/src/cache_control.rs (new) — `compute_frozen_count`
walks `messages[i].content[*].cache_control` via serde_json
accessors only (no regex) and returns the smallest N such that
`messages[i]` is frozen for every i < N. Markers in `system` or
`tools[*]` log at debug! but never bump the floor (those fields are
unconditionally cache-hot per invariant I2). TTL ordering violations
(5m before 1h, guide §2.19) emit `tracing::warn!` but the function
computes the correct count regardless — the customer's request, not
ours to reject.
crates/headroom-core/src/lib.rs — re-export `compute_frozen_count` at
crate root so the proxy crate has a stable import path.
crates/headroom-proxy/src/compression/anthropic.rs — add
`resolve_frozen_count` thin wrapper that consults the
`cache_control_auto_frozen` config flag. When `disabled`, returns 0
regardless of body content (operator opt-out for benchmarking).
crates/headroom-proxy/src/config.rs — add `CacheControlAutoFrozen`
enum and the matching CLI flag `--cache-control-auto-frozen` /
env var `HEADROOM_PROXY_CACHE_CONTROL_AUTO_FROZEN`. Default is
`enabled`. Documented in the doc comments.
Tests
- crates/headroom-core/src/cache_control.rs (inline): 11 unit tests
covering marker detection, system/tools negative cases, ordering
state machine, defensive (missing fields, non-array messages,
non-object content blocks).
- crates/headroom-core/tests/cache_control.rs: 11 unit + 3 property
tests (monotonic non-decrease as markers are added; system/tools
markers don't change count; empty messages → 0).
- crates/headroom-proxy/tests/integration_cache_control.rs: 8 tests
exercising the proxy wrapper (configurability gate; tracing
capture for the 5m-before-1h warn path).
Acceptance gates: `cargo build --workspace`, `cargo test --workspace`
(33 new tests green), `cargo clippy --workspace -- -D warnings`,
`cargo fmt --all --check` all clean. No new `regex::` imports;
`git grep -n 'regex::' crates/{headroom-core/src/cache_control.rs,
headroom-core/tests/cache_control.rs, headroom-proxy/tests/
integration_cache_control.rs}` empty.
Honors the realignment build constraints: configurable (CLI + env),
no hardcodes (TTL strings live as const), no regex (serde_json
accessor walk), no fallbacks (one impl), structured logging
(debug!/warn! with field/index/ttl/rule context), tests
comprehensive (unit + property + integration + tracing capture).
|
||
|
chopratejas
|
a974bb153a |
fix(rust): PR-A1 — make /v1/messages compression a passthrough
Stop calling IntelligentContextManager from the Rust proxy on
/v1/messages. The proxy is now a byte-faithful passthrough on this
endpoint. Eliminates the C1+C2+C3+C4 cache-killer cluster (P0-3,
P0-4, P0-5, P1-13) by not running ICM with `frozen_message_count: 0`
hardcoded — Phase B PR-B2 brings live-zone-only compression back.
Per REALIGNMENT/03-phase-A-lockdown.md.
Changes:
- Add `--compression-mode {off,live_zone}` flag and
`HEADROOM_PROXY_COMPRESSION_MODE` env var. Default `off`. Both
modes passthrough in PR-A1; `live_zone` warns loudly because
Phase B isn't implemented yet (no silent fallback).
- Replace `compress_anthropic_request` body with a passthrough
stub that emits a structured `tracing::info!` decision log line
(request_id, path, method, compression_mode, decision,
reason="phase_a_lockdown", body_bytes) and returns
`Outcome::NoCompression`. Function signature preserved so
Phase B PR-B2 is a pure body swap.
- Delete `compression/icm.rs` (per the realignment plan: ICM
modules in headroom-core are deleted in PR-B1).
- Drop the `Arc<IntelligentContextManager>` field from `AppState`
— no longer used.
- Add request-entry `tracing::debug!` with auth_mode_placeholder
("unknown" until Phase F PR-F1 wires the auth-mode classifier).
- Add `debug_assert!` on the NoCompression branch that the
buffered bytes length is stable, locking in Phase A's
cache-safety invariant at the call site.
- Tighten existing tests from `len()` equality to SHA-256 byte
equality. Rename `compression_on_oversized_body_trims_messages`
→ `compression_on_long_body_passes_through_in_phase_a` and
flip the assertion to byte-equal.
- Add new tests: passthrough_mode_off_byte_equal_sha256,
passthrough_mode_live_zone_currently_passthrough_byte_equal_sha256,
passthrough_preserves_numeric_precision (literal-byte body so
serde_json's f64 quantization can't mask a regression),
passthrough_preserves_cache_control_markers,
passthrough_preserves_thinking_signature,
passthrough_preserves_redacted_thinking_data,
passthrough_recorded_fixture_byte_equal_sha256,
tracing_capture::compression_decision_logged.
- Add fixture
`crates/headroom-proxy/tests/fixtures/anthropic_messages_request_real.json`
with system block list + cache_control markers, tools with
nested JSON Schema, messages containing text + thinking +
signature + tool_use + tool_result + image, non-ASCII content,
large numbers. Used as the canonical SHA-256 round-trip gate.
Constraints honored: configurable (compression_mode is the only
new knob), no hardcoded thresholds, no regex usage, no silent
fallbacks (live_zone-not-implemented warns), structured tracing
on every cache-affecting decision, comprehensive tests.
Acceptance criteria from PR-A1 spec:
- `cargo build --workspace` clean
- `cargo test --workspace` green (886 tests pass)
- `cargo clippy --workspace -- -D warnings` clean
- `cargo fmt --all --check` clean
- `make ci-precheck` green
- New SHA-256 byte-equality tests pass against the recorded fixture
- `tracing::info!` decision-log line is observable
- `--compression-mode` CLI + env var work
- No regex import added
|
||
|
chopratejas
|
fa5fbfabf4 |
fix(rust): wire ICM compressor into Rust proxy on /v1/messages
Adds an opt-in compression interceptor that buffers Anthropic
/v1/messages requests, runs IntelligentContextManager over the
messages array, and forwards the (possibly trimmed) body upstream.
All other paths, methods, and content-types stay on the original
streaming passthrough — so existing operators see zero change.
Behaviour gates ALL must be true to buffer + compress:
- --compression flag (or HEADROOM_PROXY_COMPRESSION=1)
- method == POST
- path == /v1/messages
- Content-Type: application/json
- ICM constructed successfully at startup
Falls through to streaming on any failure: parse, missing fields,
unknown model, body-too-large. Compression must never break a
request — that's the safety contract.
Model context windows come from a vendored LiteLLM snapshot at
crates/headroom-proxy/data/model_prices_and_context_window.json
parsed once into an OnceLock<HashMap>. Refresh via
scripts/refresh_model_limits.sh. Rationale documented inline:
hardcoded tables silently rot; LiteLLM is the canonical source
the entire LLM-tooling ecosystem relies on.
New tests:
- 16 unit tests across compression::{anthropic, icm, model_limits}
- 5 integration tests: off-passthrough, on-short-passthrough,
on-oversized-trim, on-non-json-skip, on-non-llm-path-skip
Verification:
- cargo test --workspace -> 884 passed, 0 failed
- cargo clippy --workspace -- -D warnings -> clean
- cargo fmt --check -> clean
|
||
|
chopratejas
|
0144cfba51 |
ci: fix cargo fmt + maturin action invocation
cargo fmt --check failed in CI: import order in proxy.rs (cfg(test) attributes before/after non-attr imports) and a few line-wrapping nits in e2e_real.rs. Ran cargo fmt --all to fix. maturin-action@v1 does not have a 'manifest-path' input — the action warned 'Unexpected input(s) manifest-path' and proceeded to invoke maturin from the repo root, which sees the workspace Cargo.toml with no [package] section and bails. Move -m crates/headroom-py/Cargo.toml back inside the 'args' string. |
||
|
chopratejas
|
15877fb63f |
ci: workflow permissions + comprehensive e2e tests (phase-1)
CodeQL alert #61 (CWE-275, actions/missing-workflow-permissions): add explicit `permissions: contents: read` to the rust workflow root. Defaults the GITHUB_TOKEN to read-only across all jobs, so even if the repo policy changes, this workflow stays at least-privilege. No job in this workflow needs write — wheels/audit/parity all read-only. Add real end-to-end test suite at tests/e2e_real.rs gated behind HEADROOM_E2E=1. Spawns the actual Python Headroom proxy as a subprocess, runs the Rust proxy in-process in front of it, and exercises: - health endpoints across the full chain - Anthropic non-streaming (real API call) - Anthropic streaming SSE (real API call) with chunk-level validation - OpenAI non-streaming (real API call) - X-Request-Id generation and pass-through Adds tokio-process feature for Command/Child usage. Loads .env at the repo root for API keys (does not log values). Tests skip cleanly when HEADROOM_E2E is unset, so cargo test stays fast. |
||
|
chopratejas
|
bcc2ad810a |
test(rust): integration tests + wiremock harness (phase-1)
15 integration tests across five suites that spin up the proxy on an ephemeral port pointed at a per-test mock upstream: - integration_http: all 7 methods round-trip with body, status passthrough for 404/500/502, query strings preserved, 1MB POST streams through. - integration_sse: a 10-event in-process hyper SSE upstream emits at 50ms cadence; chunks reach the client with max gap < 500ms (loose CI bound) and a client disconnect propagates to the upstream within 2s. - integration_ws: 5 text + 5 binary messages echo through a tungstenite upstream byte-equal; client-initiated close propagates. - integration_headers: hop-by-hop strip both directions, X-Forwarded-* injection, X-Forwarded-For appends to existing value, multi-valued response headers preserved. - integration_body: 5MB POST round-trips byte-equal; streaming response yields first byte before the upstream finishes sending. - integration_health: own /healthz always 200; /healthz/upstream is 200 when upstream healthy and 503 when down. The Sec-WebSocket-Protocol forwarding is exercised implicitly by the WS tests via tungstenite handshake. The harness lives at tests/common/mod.rs and is shared by every integration suite. |