## Description
Since v0.31.0 shipped cache mode as the default (68676daa), users report
the dashboard showing "Optimization ENABLED" while every request
forwards with 0 tokens saved — Before Compression == After Compression
even on 100k-token requests (#2357).
Root cause: in the cache-mode branch of `handle_anthropic_messages`,
when `_extract_cache_stable_delta` returns `None` the handler silently
sets `optimized_messages = messages`. That single fall-through covers
two very different cases:
1. **Session cold start** — no previous turn recorded for the session.
Every fresh proxy session, including a resumed 100k-token Claude Code
transcript, was forwarded raw. Until session identity stabilized (#2193
helped), this could be *every* turn, i.e. compression literally never
ran.
2. **Mid-session prefix mismatch** — the client rewrote history. This
passthrough is intentional (replaying a rewritten transcript risks
per-turn cache busts) but was invisible: no tag, no log, so
`optimize:true` + 0 savings looked like a broken product.
This PR: (1) cold starts now run the same full-message compression as
non-cache modes — there is no provider cache prefix to protect yet, and
the compressed output is recorded as the forwarded messages so later
turns replay it byte-identically through the existing stable-delta path
(append-only cache safety preserved); (2) the mismatch passthrough is
kept but tagged `passthrough_reason=cache_mode_prefix_mismatch` and
logged, mirroring the existing `pre_upstream_backpressure` inline-tag
pattern.
Fixes#2357
## Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update
- [ ] Refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py` (cache-mode branch only): split
the `delta is None` fall-through. When
`prefix_tracker.get_last_original_messages()` is empty (cold start), run
the full pipeline via `_run_compression_in_executor` (same call shape as
the non-cache branch) and append a `cache_mode:cold_start_full`
transform marker. When a previous turn exists but the delta is `None`
(prefix mismatch), keep the conservative passthrough but set
`tags["passthrough_reason"] = "cache_mode_prefix_mismatch"` and log it.
`CompressionDecision` untouched — it is the frozen pre-pipeline gate;
inline tags are the established mechanism for mid-pipeline passthrough
reasons.
- `tests/test_cache_mode_cold_start.py` (new): handler-level tests using
the same dummy-handler harness as `tests/test_cold_start_fast_pass.py`,
with `mode="cache"`. Cold start → pipeline invoked once, compressed form
forwarded upstream, no passthrough tag. Prefix mismatch → pipeline not
invoked, original bytes forwarded unmodified, outcome tags carry
`passthrough_reason=cache_mode_prefix_mismatch`.
## Testing
- [x] Unit tests pass locally
- [x] Lint/format/type checks pass locally
```
$ python -m pytest tests/test_cache_mode_cold_start.py tests/test_cache_mode_delta_marker.py tests/test_cache_prefix_overlay.py tests/test_cache/test_prefix_tracker.py tests/test_token_headroom_mode.py tests/test_cold_start_fast_pass.py tests/test_anthropic_pre_upstream_backpressure.py tests/test_handler_outcome_tag_invariant.py -q
146 passed
$ ruff check headroom/proxy/handlers/anthropic.py tests/test_cache_mode_cold_start.py
All checks passed!
$ ruff format --check headroom/proxy/handlers/anthropic.py tests/test_cache_mode_cold_start.py
2 files already formatted
$ mypy headroom --ignore-missing-imports
(no error lines)
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.13.11, this branch; real
`AnthropicHandlerMixin.handle_anthropic_messages` driven end-to-end
through a FastAPI `Request` with upstream stubbed at `_retry_request`
(harness identical to the existing
`tests/test_cold_start_fast_pass.py`), `mode="cache"`.
- Exact command / steps: `python -m pytest
tests/test_cache_mode_cold_start.py -q`, plus a manual run of the
mismatch scenario with `logging.basicConfig(level=INFO)`.
- Observed result: Cold start: `anthropic_pipeline.apply` invoked once
and the forwarded upstream body contains the compressed tool_result
content (previously: forwarded raw with zero pipeline invocations).
Mismatch: bytes forwarded unmodified, and the proxy log now emits
`[req-...] Compression skipped: reason=cache_mode_prefix_mismatch` with
the same reason present in `RequestOutcome.tags["passthrough_reason"]`
(previously: nothing).
- Not tested: a live multi-turn session against the real Anthropic API
measuring `cache_read_input_tokens` across turns (the byte-identical
replay contract the cold-start path relies on is the same one exercised
by the existing stable-delta tests in
`tests/test_cache_mode_delta_marker.py` and
`tests/test_cache_prefix_overlay.py`, all green).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
Parideboy