mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
248ae0f3e0
The freeze path (both providers) emits the agent's ORIGINAL bytes for a frozen message, but the provider cached whatever we FORWARDED last turn (the compressed form). Forwarding original then mismatches the cached prefix and busts it from that point — re-creating the whole suffix. Measured on a real SWE-bench run: 100% of attributed misses were prefix_change, ~56% of ALL cache-writes were bust-induced (2.8M tokens), driving cache_create +150% and cost +41% vs baseline. Cache mode already avoided this via _extract_cache_stable_delta (replay the previously-forwarded prefix, compress only the delta). Token mode called apply(frozen_count) directly, which forwards original for the frozen region. Fix: add a shared, provider-agnostic overlay_cached_prefix() that replays the previously-forwarded (cached, compressed) prefix byte-identical, append-only guarded and idempotent, and apply it in BOTH the Anthropic and OpenAI handlers right before forwarding. This makes freezing byte-identical in every mode, so the only remaining difference between "token" and "cache" mode is how large a mutable (still-compressible) tail each leaves — not whether the frozen prefix busts the cache. Tests: - test_cache_prefix_overlay.py: the helper (replay, append-only guard, idempotence). - test_cross_turn_cache_safety.py: the invariant that was missing — drive the REAL tracker + freeze + overlay over multiple append-only turns against a simulated provider prefix cache and assert the forwarded prefix stays byte-identical turn-over-turn. Load-bearing: it fails (detects the bust) without the overlay. ## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> |
||
|---|---|---|
| .. | ||
| test_anthropic_buffered_timeout.py | ||
| test_anthropic_ccr_deferred_injection.py | ||
| test_anthropic_ccr_raise.py | ||
| test_anthropic_streaming_ccr_retrieve.py | ||
| test_background_compression.py | ||
| test_bedrock_passthrough.py | ||
| test_cc_switch_reconciler.py | ||
| test_ccr_frozen_prefix_coupling.py | ||
| test_compression_failure_action.py | ||
| test_compression_timeout_config.py | ||
| test_compute_turn_id.py | ||
| test_header_safe_transforms.py | ||
| test_mcp_stats_aggregation.py | ||
| test_openai_backend_path.py | ||
| test_openai_transport_path_prefix.py | ||
| test_openai_upstream_header.py | ||
| test_phase3_byte_identity.py | ||
| test_request_logger.py | ||
| test_transformations_feed.py | ||