## Description
Closes#2053
Modern Claude Code sends `tool_result` content as a list of typed blocks
(`[{"type": "text", "text": "..."}]`) instead of a plain string.
`_extract_tool_result_content` and `_swap_tool_result_content` in
`compression_cache.py` only handled the plain string case, so every
tool_result was skipped before compression — zero savings for all
`headroom wrap claude` users.
Fix: extract text from list-of-blocks content in
`_extract_tool_result_content`, and collapse the list to a single text
block in `_swap_tool_result_content` when replacing with compressed
content.
## Type of Change
- [x] Bug fix (non-breaking)
- [ ] New feature (non-breaking)
- [ ] Breaking change
- [ ] Documentation update
## Changes Made
- `headroom/cache/compression_cache.py`:
- Added `_extract_text_from_blocks()` helper to extract joined text from
Anthropic list-of-blocks format
- Updated `_extract_tool_result_content()` to handle list content in
both Anthropic tool_result blocks and OpenAI role=tool messages
- Updated `_swap_tool_result_content()` to collapse list-of-blocks to a
single text block when replacing content (preserves structure, prevents
multi-block join mismatch)
- `tests/test_token_headroom_mode.py`: Added 12 tests covering Anthropic
list-of-blocks, OpenAI list, mixed blocks, empty list, non-mutation,
missing-text-block fallback, and non-tool messages
## Testing
- [x] Existing tests pass
- [x] New tests cover the fix
- [x] PBT round-trip property verified (6 properties × 850+ random
examples)
- [x] Adversarial edge case tests pass (50 cases across 3 functions)
```
tests/test_token_headroom_mode.py ....................... 32 passed (0.35s)
tests/test_transforms_content_router.py ................. 37 passed (1.08s)
tests/test_backend_bugs.py ............................... 37 passed (4.86s)
```
## Real Behavior Proof
- Environment: headroom main (upstream/main at time of PR), uv-managed
Python 3.12
- Exact command / steps:
1. `uv run python -m pytest tests/test_token_headroom_mode.py -x -q
--no-header`
2. `uv run python /tmp/pbt_tool_result_content.py` (PBT round-trip, 200
examples each property)
3. `uv run python /tmp/adversarial_lens_security.py` (50 edge case
checks)
4. Design scan: grep'd codebase for `isinstance(content, str)` near
tool_result — 3 sibling functions (`_to_text`, `_block_text`) already
handle list-of-blocks
- Observed result: All 106 tests pass. PBT confirmed extract+swap
round-trip invariant holds for 850+ random inputs. 50 adversarial edge
cases (null/None/nested/unicode/100K chars/1000 blocks) produce no
crashes or wrong output.
- Not tested: live proxy with real Claude Code traffic (the
ContentRouter already handles list content correctly since v0.32.0,
confirmed by code audit)
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
---------
Co-authored-by: lennney <lennney@users.noreply.github.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
Builds on PR #406 (HTTP /v1/responses PyO3) and PR #410 (WS first-frame
PyO3) which landed the binding for `compress_openai_responses_live_zone`.
This change closes the remaining gaps so every (provider × endpoint ×
auth-mode × streaming) combination compresses AND surfaces in the
dashboard.
Telemetry: extend the PyO3 binding return tuple from `(bytes, modified)`
to `(bytes, modified, tokens_saved, transforms_applied)` by adding
`CompressionManifest::tokens_saved()` and `transforms_applied()`
accessors on the existing manifest. The Python proxy populates
request-log telemetry from the binding output instead of recounting
tokens. Updates the existing 2-tuple call sites in HTTP and WS
first-frame, plus the unpacks in tests.
WebSocket multi-frame compression: subscription Codex users keep a
long-lived WS open and send multiple `response.create` events per
session. PR #410 only compressed the first frame; subsequent frames
went raw. Added `_maybe_compress_response_create_frame` closure inside
`_client_to_upstream` that runs the same Rust dispatcher on every
client→upstream `response.create` text frame, passes other event
types (response.cancel, session.update, etc.) through unchanged, and
accumulates `tokens_saved` / `transforms_applied` /
`ws_frames_compressed` counters across the session.
Pre-existing dashboard gap: `streaming.py` and `anthropic.py` write
`RequestLog` entries; the non-streaming OpenAI HTTP and WS handlers
did not. Result: /transformations/feed was invisible for every Codex
turn and every Cline / OpenClaude / Aider turn. Added the same wiring
in `handle_openai_chat` (non-streaming), `handle_openai_responses`
(non-streaming HTTP), and `handle_openai_responses_ws` (session-end).
All three populate `auth_mode` + `endpoint` tags so the dashboard can
break compression activity down by client class (PAYG / OAuth /
Subscription) and surface (`chat_completions` / `responses_http` /
`responses_ws`). The WS metric record is now unconditional — was
previously gated on `tokens_saved > 0`, so first-frame no-changes
never registered.
compute_frozen_count over-freeze for prose-format clients:
`compute_frozen_count` walked until it found an unstable
`tool_result` / `role: "tool"` block. Cline / OpenClaude / Aider —
clients that embed tool calls as XML inside plain text — never
produce such a boundary, so the function returned `len(messages)` and
the pipeline froze 100% of messages including the brand-new user
turn. Live zone empty → `Transform content_router: 16414 → 16414
tokens (saved 0)`. Reported on Discord 2026-05-07 with Cline+DeepSeek.
Fix: cap at `max(0, len(messages) - 1)`. Updates 3 existing test
assertions whose expected values encoded the old over-freeze. Adds 6
new prose-format invariant tests.
CodeQL "clear-text logging of sensitive information" fix:
`tests/e2e_real_compression.py` previously stored API keys in local
variables in the same scope as diagnostic prints, which CodeQL flagged
via data-flow analysis. Refactored to read keys from `os.environ`
inside the request helper — the credentials never enter the runner's
main scope, so the taint flow never reaches the print.
End-to-end verification with real keys (.env):
/v1/messages (PAYG, non-stream) tok 14109 → 969 saved 13140
/v1/messages (PAYG, stream) tok 14109 → 969 saved 13140
/v1/chat/completions (PAYG, non-stream) tok 18460 → 1374 saved 17086
/v1/chat/completions (PAYG, stream) tok 18460 → 1374 saved 17086 (cache_hit=100%)
/v1/responses HTTP (PAYG, non-stream) bytes 50138 → 597 saved 18391
/v1/responses WS (frame 1) bytes 46429 → 488 saved 16791
/v1/responses WS (frame 2 multi) bytes 46429 → 488 saved 16791
/v1/responses WS (response.cancel) passthrough untouched
Tests: cargo workspace + pytest (4846 pass, 0 fail), make ci-precheck
passed, two E2E scripts (multi-turn HTTP, WS fake-upstream) all green.
