Builds on PR #406 (HTTP /v1/responses PyO3) and PR #410 (WS first-frame
PyO3) which landed the binding for `compress_openai_responses_live_zone`.
This change closes the remaining gaps so every (provider × endpoint ×
auth-mode × streaming) combination compresses AND surfaces in the
dashboard.
Telemetry: extend the PyO3 binding return tuple from `(bytes, modified)`
to `(bytes, modified, tokens_saved, transforms_applied)` by adding
`CompressionManifest::tokens_saved()` and `transforms_applied()`
accessors on the existing manifest. The Python proxy populates
request-log telemetry from the binding output instead of recounting
tokens. Updates the existing 2-tuple call sites in HTTP and WS
first-frame, plus the unpacks in tests.
WebSocket multi-frame compression: subscription Codex users keep a
long-lived WS open and send multiple `response.create` events per
session. PR #410 only compressed the first frame; subsequent frames
went raw. Added `_maybe_compress_response_create_frame` closure inside
`_client_to_upstream` that runs the same Rust dispatcher on every
client→upstream `response.create` text frame, passes other event
types (response.cancel, session.update, etc.) through unchanged, and
accumulates `tokens_saved` / `transforms_applied` /
`ws_frames_compressed` counters across the session.
Pre-existing dashboard gap: `streaming.py` and `anthropic.py` write
`RequestLog` entries; the non-streaming OpenAI HTTP and WS handlers
did not. Result: /transformations/feed was invisible for every Codex
turn and every Cline / OpenClaude / Aider turn. Added the same wiring
in `handle_openai_chat` (non-streaming), `handle_openai_responses`
(non-streaming HTTP), and `handle_openai_responses_ws` (session-end).
All three populate `auth_mode` + `endpoint` tags so the dashboard can
break compression activity down by client class (PAYG / OAuth /
Subscription) and surface (`chat_completions` / `responses_http` /
`responses_ws`). The WS metric record is now unconditional — was
previously gated on `tokens_saved > 0`, so first-frame no-changes
never registered.
compute_frozen_count over-freeze for prose-format clients:
`compute_frozen_count` walked until it found an unstable
`tool_result` / `role: "tool"` block. Cline / OpenClaude / Aider —
clients that embed tool calls as XML inside plain text — never
produce such a boundary, so the function returned `len(messages)` and
the pipeline froze 100% of messages including the brand-new user
turn. Live zone empty → `Transform content_router: 16414 → 16414
tokens (saved 0)`. Reported on Discord 2026-05-07 with Cline+DeepSeek.
Fix: cap at `max(0, len(messages) - 1)`. Updates 3 existing test
assertions whose expected values encoded the old over-freeze. Adds 6
new prose-format invariant tests.
CodeQL "clear-text logging of sensitive information" fix:
`tests/e2e_real_compression.py` previously stored API keys in local
variables in the same scope as diagnostic prints, which CodeQL flagged
via data-flow analysis. Refactored to read keys from `os.environ`
inside the request helper — the credentials never enter the runner's
main scope, so the taint flow never reaches the print.
End-to-end verification with real keys (.env):
/v1/messages (PAYG, non-stream) tok 14109 → 969 saved 13140
/v1/messages (PAYG, stream) tok 14109 → 969 saved 13140
/v1/chat/completions (PAYG, non-stream) tok 18460 → 1374 saved 17086
/v1/chat/completions (PAYG, stream) tok 18460 → 1374 saved 17086 (cache_hit=100%)
/v1/responses HTTP (PAYG, non-stream) bytes 50138 → 597 saved 18391
/v1/responses WS (frame 1) bytes 46429 → 488 saved 16791
/v1/responses WS (frame 2 multi) bytes 46429 → 488 saved 16791
/v1/responses WS (response.cancel) passthrough untouched
Tests: cargo workspace + pytest (4846 pass, 0 fail), make ci-precheck
passed, two E2E scripts (multi-turn HTTP, WS fake-upstream) all green.
PR-C5 (May 3) retired the Python `/v1/responses` compression pipeline
with the comment "Rust handles item-aware compression natively" — but
the standalone `crates/headroom-proxy` binary that was supposed to do
that compression is not deployed by the CLI today (`headroom proxy`
and `headroom wrap codex` both run only the Python proxy via uvicorn).
Result: every `/v1/responses` request since v0.20.16 has been
forwarded uncompressed. Codex CLI is the flagship consumer of this
endpoint; this is the regression users have been reporting.
Closes Bug 1 of the Codex regression by exposing the existing
`headroom_core::transforms::compress_openai_responses_live_zone` as
a PyO3 binding so the Python proxy can call the live-zone dispatcher
in-process. The `headroom._core` extension is already loaded at
proxy startup (PR-A0 verifies), so adding one more callable is
mechanical.
Why PyO3 inline (Layer 1) vs originally-intended two-process chain
(Layer 2): the inline call requires zero deployment changes — the
wheel already ships `headroom._core`. Layer 2 (build + ship the
standalone `headroom-proxy` binary, teach CLI to spawn both
processes) is the right long-term move; Layer 1 restores v0.5.21
functional behaviour today.
# Returns
`(body, modified)`. On change → `(new_body_bytes, True)`; on
passthrough → `(input_bytes, False)`.
# Failure mode
Never raises. The dispatcher's `LiveZoneError` cases (body not JSON,
no input array) are passthrough conditions matching the Rust proxy's
`compress_openai_responses_request` contract.
# Tests
14 new tests in `tests/test_responses_pyo3_compression.py`:
binding exposed, passthrough cases, every F1 AuthMode variant,
empty-model default, no-raise on garbage bytes.
