Builds on PR #406 (HTTP /v1/responses PyO3) and PR #410 (WS first-frame
PyO3) which landed the binding for `compress_openai_responses_live_zone`.
This change closes the remaining gaps so every (provider × endpoint ×
auth-mode × streaming) combination compresses AND surfaces in the
dashboard.
Telemetry: extend the PyO3 binding return tuple from `(bytes, modified)`
to `(bytes, modified, tokens_saved, transforms_applied)` by adding
`CompressionManifest::tokens_saved()` and `transforms_applied()`
accessors on the existing manifest. The Python proxy populates
request-log telemetry from the binding output instead of recounting
tokens. Updates the existing 2-tuple call sites in HTTP and WS
first-frame, plus the unpacks in tests.
WebSocket multi-frame compression: subscription Codex users keep a
long-lived WS open and send multiple `response.create` events per
session. PR #410 only compressed the first frame; subsequent frames
went raw. Added `_maybe_compress_response_create_frame` closure inside
`_client_to_upstream` that runs the same Rust dispatcher on every
client→upstream `response.create` text frame, passes other event
types (response.cancel, session.update, etc.) through unchanged, and
accumulates `tokens_saved` / `transforms_applied` /
`ws_frames_compressed` counters across the session.
Pre-existing dashboard gap: `streaming.py` and `anthropic.py` write
`RequestLog` entries; the non-streaming OpenAI HTTP and WS handlers
did not. Result: /transformations/feed was invisible for every Codex
turn and every Cline / OpenClaude / Aider turn. Added the same wiring
in `handle_openai_chat` (non-streaming), `handle_openai_responses`
(non-streaming HTTP), and `handle_openai_responses_ws` (session-end).
All three populate `auth_mode` + `endpoint` tags so the dashboard can
break compression activity down by client class (PAYG / OAuth /
Subscription) and surface (`chat_completions` / `responses_http` /
`responses_ws`). The WS metric record is now unconditional — was
previously gated on `tokens_saved > 0`, so first-frame no-changes
never registered.
compute_frozen_count over-freeze for prose-format clients:
`compute_frozen_count` walked until it found an unstable
`tool_result` / `role: "tool"` block. Cline / OpenClaude / Aider —
clients that embed tool calls as XML inside plain text — never
produce such a boundary, so the function returned `len(messages)` and
the pipeline froze 100% of messages including the brand-new user
turn. Live zone empty → `Transform content_router: 16414 → 16414
tokens (saved 0)`. Reported on Discord 2026-05-07 with Cline+DeepSeek.
Fix: cap at `max(0, len(messages) - 1)`. Updates 3 existing test
assertions whose expected values encoded the old over-freeze. Adds 6
new prose-format invariant tests.
CodeQL "clear-text logging of sensitive information" fix:
`tests/e2e_real_compression.py` previously stored API keys in local
variables in the same scope as diagnostic prints, which CodeQL flagged
via data-flow analysis. Refactored to read keys from `os.environ`
inside the request helper — the credentials never enter the runner's
main scope, so the taint flow never reaches the print.
End-to-end verification with real keys (.env):
/v1/messages (PAYG, non-stream) tok 14109 → 969 saved 13140
/v1/messages (PAYG, stream) tok 14109 → 969 saved 13140
/v1/chat/completions (PAYG, non-stream) tok 18460 → 1374 saved 17086
/v1/chat/completions (PAYG, stream) tok 18460 → 1374 saved 17086 (cache_hit=100%)
/v1/responses HTTP (PAYG, non-stream) bytes 50138 → 597 saved 18391
/v1/responses WS (frame 1) bytes 46429 → 488 saved 16791
/v1/responses WS (frame 2 multi) bytes 46429 → 488 saved 16791
/v1/responses WS (response.cancel) passthrough untouched
Tests: cargo workspace + pytest (4846 pass, 0 fail), make ci-precheck
passed, two E2E scripts (multi-turn HTTP, WS fake-upstream) all green.
PR-C5 retired Python compression on the WebSocket /v1/responses path
expecting the standalone Rust proxy binary to take over. That binary
isn't deployed by the CLI today (`headroom proxy` runs only the Python
proxy via uvicorn). PR #406 closed the equivalent gap on the HTTP path;
this commit closes the gap on WebSocket.
Subscription users matter most here. The PR #409 reviewer confirmed
empirically that ChatGPT-subscription Codex CLI defaults to WebSocket
transport for /v1/responses. After PR #409 the routing reaches Headroom;
before this commit, every byte was forwarded uncompressed.
# What this does
In handle_openai_responses_ws, after memory injection finalises the
first-frame body, we re-parse first_msg_raw, detect the wrap shape
(Codex sends either {"type": "response.create", "response": {...}} or
the payload directly), call the PyO3 binding from PR #406 on the
inner payload, and re-wrap on modified return.
The compression engine runs in Rust — the binding exposes it inline
so the Python WS handler can call it without a process chain.
# Failure mode
Wrapped in try/except — passthrough on any unexpected condition.
# Subsequent client→upstream frames
Out of scope. Multi-frame compression is a separate follow-up.
# Tests
26 new tests in tests/test_responses_ws_pyo3_compression.py pinning
the body-shape contract: wrapped + unwrapped envelopes, garbage shapes
(no exception leak), every F1 AuthMode accepted. Existing WS
lifecycle + timings tests still pass.
Refs: PR #406 (HTTP path), PR-C5 (the retirement that needs closing)
