mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
c16be9bbbe
## Description A Claude Code session that reads a large tool result through `headroom proxy` can fail on turn 2 with Anthropic's `400 prompt_too_long`. The reporter's controlled comparison completed eight turns through 0.33.0 with 187,986 input tokens, while 0.35.0 failed after five requests with 753,077 input tokens. The local regression uses an actual prior optimized request to populate tracker state, then a decision-false bypass turn with Claude-shaped tool-result content. The old unconditional replay path substitutes the compressed prefix; the eligibility gate preserves the client's outbound body without claiming a live provider reproduction. The Anthropic `/v1/messages` route computes whether a request should be compressed, but cached-prefix replay currently runs outside that decision. The replay helper also derives its prefix length from the original message list and applies that index to the optimized list without proving the two lists still align. A stale forwarded prefix can therefore be grafted onto the wrong positions and enlarge later requests. This change limits replay to requests whose existing compression decision permits it and whose pre-upstream backpressure path is inactive. It also makes `overlay_cached_prefix()` decline misaligned or inflating candidates while preserving normal append-only replay. Reported by @itsumonotakumi, whose controlled comparison isolated the failure from compression, headers, one-request serialization, memory, code graph, and CCR. Closes #3026 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Gate Anthropic cached-prefix replay on the existing `CompressionDecision.should_compress` result and the existing pre-upstream backpressure state. - Require positional alignment between optimized and original message arrays before replay. - Reject replay candidates that would serialize larger than the current optimized messages. - Add focused handler coverage for the decision-false tool-result regression, bypass and backpressure paths, and outbound optimize-on preservation. - Add direct unit coverage for positional mismatch, no-inflation, and JSON sizing-failure bailouts. - Update the moved-cache-control and pure-block-append regression fixtures to keep the no-inflation contract explicit. - Run the unchanged OpenAI cache-stability preservation proof; no OpenAI production code was edited. ## Testing - [x] Unit tests pass (153 focused proxy, helper, cache-control, block-append, cross-turn, byte-faithful, Anthropic, OpenAI, and backpressure tests) - [x] Linting passes (Ruff check and format validation on the seven changed repository files) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed with the in-process proxy and local stub upstream ### Test Output ```text python -m pytest tests/test_proxy/test_anthropic_no_optimize_history_passthrough.py tests/test_cache_prefix_overlay.py tests/test_cross_turn_cache_safety.py tests/test_cache_control_move_bust.py tests/test_proxy_byte_faithful_forwarding.py tests/test_proxy_anthropic_cache_stability.py tests/test_anthropic_pre_upstream_backpressure.py -q python -m pytest tests/test_proxy_openai_cache_stability.py -q python -m pytest tests/test_issue_2671_block_growth_cache.py::test_pure_append_replays_forwarded_blocks_and_advances_breakpoint -q 153 passed across focused invocations, exit code 0 optimize_off turn2_message_count=3 marker_count=1 outbound_compact_utf8_bytes=2293 client_compact_utf8_bytes=2293 optimize_on turn2_message_count=3 client_message_count=3 marker_count=0 outbound_compact_utf8_bytes=171 client_compact_utf8_bytes=182 python -m ruff check headroom/proxy/handlers/anthropic.py headroom/cache/prefix_tracker.py tests/test_proxy/test_anthropic_no_optimize_history_passthrough.py tests/test_cache_prefix_overlay.py tests/test_cache_control_move_bust.py tests/test_issue_2671_block_growth_cache.py tests/test_proxy_openai_cache_stability.py All checks passed!, exit code 0 python -m ruff format headroom/proxy/handlers/anthropic.py headroom/cache/prefix_tracker.py tests/test_proxy/test_anthropic_no_optimize_history_passthrough.py tests/test_cache_prefix_overlay.py tests/test_cache_control_move_bust.py tests/test_issue_2671_block_growth_cache.py tests/test_proxy_openai_cache_stability.py --check 7 files already formatted, exit code 0 git diff --check clean, exit code 0 ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12 via `uv`, real Headroom proxy app with a local stub Anthropic upstream - Exact command / steps: send an actual optimize-on first request through the in-process proxy with a deterministic production-pipeline seam, then send a decision-false bypass turn containing a large Claude-shaped `tool_result` with moved `cache_control`; separately send an aligned optimize-on turn with a new suffix - Observed result: the exact base checkout fails with `AssertionError: assert 'compressed-tool-result' == 'large-tool-result-marker ...'`; the guarded path passes with the client marker present once and outbound compact JSON no larger than the client body. The optimize-on preservation run records `optimize_on turn2_message_count=3 client_message_count=3 marker_count=0 outbound_compact_utf8_bytes=171 client_compact_utf8_bytes=182`, proving the actual compressed prefix is outbound before the new suffix without turn-2 growth. - Not tested: live Claude Code session against api.anthropic.com on this host ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: N/A. - Stable/default behavior changed: cached-prefix replay now follows the existing compression and backpressure decision and rejects misaligned or inflating candidates. - Kill switch / disable path: no new switch; the existing optimize and bypass controls remain available. - Unsafe override required: none. - Qualification impact: none. - Rollback path: revert the implementation commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes ## Additional Notes `CHANGELOG.md` is not modified because Headroom's release automation generates it from conventional commits. This change does not add a context-limit guard or alter compression, streaming tracker provenance, outbound-body selection, OpenAI behavior, or provider limits. Local tests prove request-body ownership and replay bounds. The reporter's live Claude Code completion and Anthropic token acceptance remain external to this local proof. |
||
|---|---|---|
| .. | ||
| test_anthropic_buffered_timeout.py | ||
| test_anthropic_ccr_deferred_injection.py | ||
| test_anthropic_ccr_raise.py | ||
| test_anthropic_no_optimize_history_passthrough.py | ||
| test_anthropic_recount_and_reparse_safety.py | ||
| test_anthropic_streaming_ccr_retrieve.py | ||
| test_anthropic_upstream_header.py | ||
| test_background_compression.py | ||
| test_bedrock_passthrough.py | ||
| test_bedrock_sse_ping.py | ||
| test_cc_switch_reconciler.py | ||
| test_ccr_frozen_prefix_coupling.py | ||
| test_compression_failure_action.py | ||
| test_compression_timeout_config.py | ||
| test_compute_turn_id.py | ||
| test_gemini_savings_profile.py | ||
| test_header_safe_transforms.py | ||
| test_mcp_stats_aggregation.py | ||
| test_model_router.py | ||
| test_model_router_wiring.py | ||
| test_openai_backend_path.py | ||
| test_openai_chat_ccr_injection.py | ||
| test_openai_chat_savings_profile.py | ||
| test_openai_responses_ccr.py | ||
| test_openai_stream_usage_option.py | ||
| test_openai_transport_path_prefix.py | ||
| test_openai_upstream_header.py | ||
| test_phase3_byte_identity.py | ||
| test_request_logger.py | ||
| test_settings_fresh_process_precedence.py | ||
| test_settings_store.py | ||
| test_tool_search_repair_after_turn_hooks.py | ||
| test_transformations_feed.py | ||