mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
20 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Ashish Patel
|
41dab2d099
|
fix(ccr): verify a scanned marker's hash before advertising it (#2908)
## Description
`CCRToolInjector.scan_for_markers()` decides whether a compression
marker is Headroom's own by *shape* alone — any bracket marker carrying
a 24-hex hash counts, per the generic fallback pattern
(`\[.*?compressed.*?hash=([a-f0-9]{24})\]`). Other context tools emit
exactly that shape. Once a foreign hash is scanned,
`has_compressed_content` flips true and the retrieve tool + "Available
hashes" system instruction get injected for a hash this proxy never
stored — the model calls `headroom_retrieve`, gets a guaranteed miss,
and re-does work it already had. Two wasted turns per adopted foreign
hash.
Closes #2836
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/ccr/tool_injection.py`: added
`CCRToolInjector.verify_ownership()` — filters `detected_hashes` down to
hashes the compression store actually recognizes, via the same
`store.exists()` check the retrieve endpoint itself performs. Added a
small `_HashOwnershipStore` Protocol (structural typing, not a hard
dependency on the concrete `CompressionStore` class) and a
`compression_store` constructor field for dependency injection/testing.
`scan_for_markers()` itself is untouched — kept store-independent (pure
regex) rather than baking the check into the scan loop, since that
approach broke 24 existing tests that correctly test "does this shape
match" in isolation.
- `headroom/proxy/handlers/anthropic.py`,
`headroom/proxy/handlers/openai.py`: call `injector.verify_ownership()`
right after `scan_for_markers()` — the two real per-request call sites.
- `verify_ownership()` is also called inside `process_request()` (the
convenience wrapper `batch.py`'s Google path uses), so that path is
covered without a separate call site edit.
- `tests/test_ccr_tool_injection.py`: new `TestVerifyOwnership` class (6
tests) — the exact issue repro, a real-hash-survives case, mixed
own/foreign hashes, explicit store override, store-exception safety
(must not raise), and no-op-on-empty-hashes.
- `tests/test_proxy_anthropic_cache_stability.py`: 3 pre-existing tests
needed updating for the new (correct) behavior — two `_FakeInjector`
test doubles needed a `verify_ownership()` stub added, and one real
end-to-end test needed a genuine store entry seeded (via
`explicit_hash`) for the hash its hand-typed marker references, instead
of asserting on an unverified shape-only match.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — not run locally, will
confirm via CI
- [x] New tests added for new functionality
- [x] Manual testing performed (see Real Behavior Proof)
### Test Output
```text
$ .venv/Scripts/python -m pytest tests/test_ccr_marker_policy.py tests/test_ccr_tool_always_on.py tests/test_ccr_tool_injection.py tests/test_proxy/test_ccr_frozen_prefix_coupling.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_handlers_batch.py tests/test_proxy_handler_helpers.py -q
151 passed in 15.87s
$ .venv/Scripts/python -m pytest tests/test_proxy_ccr.py tests/test_proxy_openai_responses_stream_ccr.py tests/test_anthropic_ccr_workspace_unbound.py tests/test_compression_store.py tests/test_no_ccr_lossy.py tests/test_proxy_handlers_batch.py -q
121 passed in 20.73s
$ .venv/Scripts/ruff check . && .venv/Scripts/ruff format --check . # touched files only
All checks passed / already formatted
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.14.5, local venv
- Exact command / steps: ran the issue's exact 3-line repro
(`CCRToolInjector.scan_for_markers()` on the foreign marker text, then
`verify_ownership()`) before and after the fix; separately verified a
genuinely-Headroom-stored hash (via `store.store(...,
explicit_hash=...)`) still survives verification and still drives
injection
- Observed result: before the fix (scan only, no verify step exists yet)
`has_compressed_content` is `True` for the foreign marker — matches the
bug report exactly. After adding `verify_ownership()`: foreign marker →
`detected_hashes == []`, `has_compressed_content is False`; real stored
hash → `detected_hashes == [real_hash]`, `has_compressed_content is
True`.
- Not tested: have not driven this through a live two-context-tool proxy
session (e.g. Headroom alongside another CCR-shaped tool in the same
conversation) — verified at the unit/integration level (the exact repro
plus the real proxy handler call sites via
`test_proxy_anthropic_cache_stability.py`'s end-to-end `TestClient`
tests), not via a live multi-tool session.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A —
internal CCR safety behavior, no user-facing docs reference the
marker-adoption mechanism)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable (release-please
generates this automatically from commit messages)
## Additional Notes
Design note on why `verify_ownership()` is a separate step rather than
baked into `scan_for_markers()`: my first attempt did exactly that and
broke 24 tests across `test_ccr_tool_injection.py`,
`test_ccr_marker_policy.py`, `test_proxy_anthropic_cache_stability.py`,
and `test_proxy_handler_helpers.py` — all of them legitimately testing
"does the regex detect this marker shape" independent of any store
state. Keeping the scan pure and adding an explicit, separately-testable
verification step kept that test surface intact while still closing the
real gap at the three places that actually decide whether to advertise
the retrieve tool.
|
||
|
Parideboy
|
aaeba0a319
|
fix(proxy): compress cache-mode cold starts and tag prefix-mismatch passthrough (#2365)
## Description
Since v0.31.0 shipped cache mode as the default (
|
||
|
Tejas Chopra
|
1a04c957f5
|
fix(cache): stabilize Anthropic block-growing lineages (#2917)
## Description Fixes the remaining Anthropic prompt-cache failure in #2671 and the newly reported parallel-tool-profile variant. The production failure has three connected parts: 1. `SessionTrackerStore.resolve_tracker` only recognized whole-message prefixes. A caller that grows or regenerates blocks inside one message therefore received a fresh tracker every turn, so previous forwarded state was always empty and breakpoint relocation could never run. 2. `normalize_message_cache_control` always moved the message breakpoint to the newest block. That is correct for a pure block append, but a message that rewrites its tail can never match the prior newest-block write and repeatedly rewrites the full message prefix. 3. Parallel Anthropic sub-calls can carry identical messages but different tools. Because tools precede messages in the provider cache key, sharing one frozen-prefix tracker across those calls cross-contaminates cache state even when message lineage is identical. This PR deliberately combines the valid parts of #2699 and #2702, fixes the discriminator between their two shapes, and adds cache-key affinity for the second pattern reported on #2671. In particular, a pure append is identified by `stable_prefix_blocks == previous_block_count`; rewritten-tail relocation is only possible when `stable_prefix_blocks < previous_block_count`. This prevents a pure append from being pinned to an old boundary. Closes #2671. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added one canonical history classifier with distinct exact, whole-message append, pure block append, rewritten-tail, and diverged outcomes. - Kept pure block appends on newest-block breakpoint placement so each request reads the old prefix and writes only its appended blocks. - Added block-level replay of the prior forwarded bytes for pure appends; the whole-message delta path explicitly refuses this shape so it cannot silently discard appended blocks. - Kept a rewritten-tail request on its existing tracker and anchored its breakpoint to the end of the byte-stable leading run. - Made rewritten-tail matching conservative: one changed message, unchanged message count, no shrink, at least 8 stable leading blocks covering at least half of old and new content, and a fixed suffix of at least 2 blocks. - Required a unique best rewritten-tail lineage match. Ambiguity creates a fresh lineage instead of making sibling sub-calls ping-pong one tracker. - Added a stable affinity fingerprint over model, deterministically forwarded tools, tool choice, thinking, and output configuration. Different provider cache-key profiles cannot share frozen-prefix state. - Snapshotted previous original/forwarded messages once in the Anthropic handler and reused that exact state for delta extraction, replay, and breakpoint placement. - Added `HEADROOM_STABLE_BOUNDARY_BREAKPOINT=0` as a rollback switch for rewritten-tail relocation. The canonical projection is used only for comparison. Replayed content always comes from the exact previously forwarded bytes or the current raw/optimized tail; canonicalized data is never reconstructed into an upstream request. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check` on changed files) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_issue_2671_block_growth_cache.py -q 12 passed in 0.10s $ python -m pytest tests/test_cache -q 253 passed, 3 skipped in 1.94s $ python -m pytest <Anthropic handler/proxy regression set> -q 134 passed, 1 warning in 9.42s $ ruff format --check <changed files> 4 files already formatted $ ruff check <changed files> All checks passed! $ git diff --check # clean ``` The Anthropic regression set covers beta stickiness, CCR injection, compaction transforms, pre-upstream backpressure, streaming reconstruction, upstream headers, model sanitization, diagnostics, and cache stability. ## Real Behavior Proof - Environment: macOS, Python 3.12.13, real FastAPI Anthropic handler with a local upstream stub plus a deterministic provider-cache oracle. - Exact steps: send a cold 35-block aggregate message, then three requests that preserve a 30-block prefix and fixed two-block suffix while regenerating a growing middle tail. Resolve the real session tracker, normalize the real handler body, record the response, and repeat. - Observed result: handler breakpoint indices are `34 -> 29 -> 29`; the cache oracle transitions from a cold 35-block write to establishing the 30-block stable boundary, then produces `(read=30, write=0)` on subsequent rewritten-tail turns. A separate pure-append sequence produces `(0,30) -> (30,4) -> (34,4) -> (38,5)`, proving its breakpoint continues to advance. - Also observed: identical message histories with different tool schemas resolve to distinct trackers in the real handler path. - Not tested: a live Anthropic billing soak, the complete repository test suite, or mypy. #2702 contains earlier live production measurements for the rewritten-tail mechanism; this PR adds the pure-append correction, affinity isolation, and broader regression model. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding documentation updates where applicable (internal behavior is documented in code; no user-facing surface changed) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing relevant unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from the Conventional Commit PR title ## Additional Notes - Consolidates the complementary approaches in #2699 and #2702. Credit to @axisrow and @nangsontay for the traces, root-cause work, and live validation that made the two production shapes distinguishable. - The 20-block minimum for relocation mirrors the provider lookup-window risk boundary and keeps short ordinary messages on the established newest-block behavior. - Disabling stable-boundary relocation does not disable improved lineage resolution or tool-profile isolation; it restores only the previous breakpoint placement. Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> |
||
|
nangsontay
|
08fce29b47
|
fix(proxy): stop toggling headroom_retrieve in the Anthropic tools array (#2672)
## Description `should_inject_ccr_tool` deferred CCR tool injection whenever `frozen_message_count > 0`. Because `tools` is the head of Anthropic's cache key, that dropped a tool which was already inside the provider-cached prefix and invalidated the whole prefix — in both directions (`0 → >0` removes it; `>0 → 0` on proxy restart, `/model` switch, lineage eviction or TTL lapse adds it back). On three days of local proxy logs the turns that flipped injection state carried **44.7% of all cache-write tokens at a 52.0% hit rate**, against 98.1% for non-flipping turns. The log signature is `cache_read` alternating between two values exactly 172 tokens apart — the 464-byte tool definition. This deletes the gate and calls `apply_session_sticky_ccr_tool` directly, which is **what `openai.py` already does** — the two handlers now have the same shape. Net −61 production lines, no new state, no new config flag. Fixes defect 1 of #2671. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Performance improvement ## Changes Made - `headroom/proxy/ccr_marker_policy.py` — deleted the `should_inject_ccr_tool` gate; `apply_session_sticky_ccr_tool` is now the single decision point. - `headroom/proxy/handlers/anthropic.py` — calls `apply_session_sticky_ccr_tool` directly, matching `openai.py`. - `headroom/proxy/helpers.py` — dropped the now-unused gate plumbing. - `tests/test_proxy_anthropic_cache_stability.py` — new test asserting the forwarded `tools` array is byte-identical across a `frozen 0 → >0` transition. - `tests/test_ccr_marker_policy.py` — removed the three unit tests that pinned the deleted decision (they encoded the defect). - `tests/test_proxy/test_ccr_frozen_prefix_coupling.py` — same unredeemable-marker intent, re-pinned at the sticky helper. - `tests/test_proxy/test_anthropic_ccr_deferred_injection.py` — autouse reset fixture for the process-global `SessionCcrTracker` (separate commit). - Formatting-only follow-up commit applying `ruff format` (pinned 0.15.17) to the two test files above. ### Why deleting the gate is safe `apply_session_sticky_ccr_tool` already holds the correct rule. Its four branches, in order: | # | condition | action | |---|---|---| | 1 | tool already in the incoming tool list (client/MCP pre-registered) | skip; the client's bytes win | | 2 | `session_id is None` (WS / pre-session) | per-turn flag drives it verbatim | | 3 | session has done CCR | always inject the recorded golden bytes | | 4 | fresh session, no compression this turn | **skip** | Branch 4 is the safety property: a session that has never compressed still gets no tool, so removing the gate cannot start injecting into non-CCR conversations. Branch 3 is what the gate was starving. `has_new_ccr_markers` still gates first-time injection, so markers replayed from the previously-forwarded prefix cannot trigger one. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed The three deleted unit tests encoded the defect. Coverage moves to the property that actually matters and was previously untested: **the forwarded `tools` array must be byte-identical across a `frozen 0 → >0` transition.** That test asserts on the forwarded request body rather than on a policy function's return value; unit-testing the old policy in isolation is exactly what let a wrong-but-self-consistent decision pass. Verified failing on `upstream/main` with an assertion on the missing tool (not an `ImportError`, so it fails for the right reason). Full suite: same pre-existing unrelated failures as `upstream/main`, **zero new** (verified by running the whole suite on both revisions and diffing the failure sets). ### Test Output ```text $ uv run pytest tests/test_ccr_marker_policy.py \ tests/test_proxy/test_anthropic_ccr_deferred_injection.py \ tests/test_proxy/test_ccr_frozen_prefix_coupling.py \ tests/test_proxy_anthropic_cache_stability.py -q collected 48 items tests/test_ccr_marker_policy.py ..... [ 10%] tests/test_proxy/test_anthropic_ccr_deferred_injection.py .............. [ 39%] . [ 41%] tests/test_proxy/test_ccr_frozen_prefix_coupling.py .. [ 45%] tests/test_proxy_anthropic_cache_stability.py .......................... [100%] ======================= 48 passed, 2 warnings in 13.59s ======================== $ ruff check . All checks passed! $ ruff format --check . 1349 files already formatted ``` ## Real Behavior Proof - Environment: local macOS proxy serving live Claude Code traffic to the Anthropic API; baseline = 3 days of proxy logs on `upstream/main`, after = 5.5 hours with this change live. - Exact command / steps: ran the proxy with this branch built in, drove normal Claude Code sessions through it (including `/model` switches and proxy restarts, the two events that used to flip injection state), then parsed 235 real turns from the proxy logs with the same parser used for the baseline in #2671. - Observed result: flip turns fell from 177 (44.7% of all cache write) to 2 (1.7%); steady-state write share 1.192% → 0.867%; aggregate hit rate 86.75% → 89.21%; main conversation warm hit rate 98.1% → 97.70% (n=149). The 2 remaining "flips" have `cache_read == 0` — cold starts that the bucketing counts as a state change, not real flips. | metric | baseline | after | |---|---|---| | flip turns | 177, carrying 44.7% of all cache write | **2**, carrying **1.7%** | | main conv, warm | 98.1% | **97.70%** (n=149) | | steady-state write share | 1.192% | **0.867%** | | aggregate | 86.75% | **89.21%** | - Not tested: `mypy headroom` was not run locally for this body; the OpenAI handler path (unchanged by this PR); tracker state loss mid-session (see note below); and defect 2 of #2671 (the sub-call breakpoint), which is untouched and is now 54.9% of remaining cache write — that is why aggregate stays just under 90%. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes **Pre-existing and unchanged here:** if the tracker loses state mid-session while the transcript still carries markers, branch 4 returns no tool and those markers are unredeemable. `upstream/main` has no recovery for that; this PR neither creates nor fixes it. See my comment on #2500, which adds a recovery path for the related dangling-reference case. **N/A checklist items:** no documentation changes — this removes an internal policy function with no user-facing surface. `mypy headroom` left unchecked because it was not run for this body; CI covers it. **Merge-order conflict with #2500 (please read before landing either):** this PR *deletes* `should_inject_ccr_tool`, which is the exact function #2500 extends with `transcript_requires_tool`. Whichever lands second needs a semantic rebase, not just a textual one — git will not flag it. If this PR lands first, #2500's recovery path should re-target `apply_session_sticky_ccr_tool` (the sticky helper now owns the decision alone) or the handler call site in `handlers/anthropic.py`. If #2500 lands first, the gate deletion here still applies but the `transcript_requires_tool` override needs to move with it. Happy to do the rebase either way — say which order you prefer. |
||
|
Tejas Chopra
|
7c2f0ea079
|
feat(cache): provider-agnostic cache-mode delta + cc-agnostic prefix comparison (#1868)
## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> |
||
|
Vinay Gupta
|
a9322477e3
|
fix: preserve anthropic passthrough tool order (#1427)
## Description Preserves Anthropic `tools` order when Headroom is forwarding a passthrough/no-optimize request. This fixes a Claude Code style `tool_result` continuation failure against stricter Anthropic-compatible upstreams that treat the client's original tool ordering as part of the conversation state. Closes #1417 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Preserve client-provided Anthropic `tools` order when `optimize=False` or the request is explicitly in Headroom passthrough/bypass mode. - Keep deterministic tool sorting for optimized requests where Headroom may rewrite the body for cache stability. - Avoid sorting batch-request tools before the no-optimize passthrough branch. - Add regression coverage for the Anthropic HTTP path to prove no-optimize forwarding keeps `Read`, then `Bash` tool order. - Update existing cache-stability and byte-faithful forwarding tests so no-optimize/passthrough expects preserved client order while optimized mode still proves deterministic sorting. ## Testing - [x] Focused unit tests pass (`pytest` on touched proxy test files) - [x] Linting passes (`ruff check` and `ruff format --check` on touched files) - [x] Type checking passes (`mypy headroom`) - [x] New regression tests added - [x] Manual testing performed ### Test Output ```text $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with pytest --with pytest-asyncio --with anyio --with 'httpx[http2]' --with fastapi --with pydantic --with tiktoken --with click --with rich --with opentelemetry-api --with opentelemetry-sdk --with zstandard --with openai --with mcp --with uvicorn pytest tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py -q ============================= test session starts ============================== platform darwin -- Python 3.13.11, pytest-9.1.1, pluggy-1.6.0 rootdir: /Users/vinaygupta/Desktop/git/headroom-fix-anthropic-tool-order configfile: pyproject.toml plugins: anyio-4.14.1, asyncio-1.4.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 87 items tests/test_proxy_handler_helpers.py .......................... [ 29%] tests/test_anthropic_stage_timings.py .... [ 34%] tests/test_proxy_anthropic_cache_stability.py ......................... [ 63%] tests/test_proxy_byte_faithful_forwarding.py ........................... [ 94%] ..... [100%] =============================== warnings summary =============================== .../fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. ======================== 87 passed, 1 warning in 5.13s ========================= $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py 5 files already formatted $ uv run --no-project --python /opt/homebrew/opt/python@3.13/bin/python3.13 --with ruff==0.15.17 ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_handler_helpers.py tests/test_anthropic_stage_timings.py tests/test_proxy_anthropic_cache_stability.py tests/test_proxy_byte_faithful_forwarding.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS, Python 3.13.11, local fake Anthropic-compatible upstream, local Headroom proxy launched with `--no-optimize --no-cache --no-rate-limit --stateless`. - Exact command / steps: ran a local reproduction harness that starts a fake `/v1/messages` upstream and Headroom proxy, then sends a Claude Code style two-turn flow: first assistant `Bash` `tool_use`, then user `tool_result`. - Observed result: after this patch, both direct and proxied flows returned `200` for `first_tool_use` and `second_tool_result`. The fake upstream log showed the proxied `tools` array remained `["Read", "Bash"]` on both turns. ```text DIRECT first_tool_use: 200 second_tool_result: 200 PROXIED first_tool_use: 200 second_tool_result: 200 UPSTREAM REQUEST LOG proxied first turn tools: ["Read", "Bash"] proxied tool_result turn tools: ["Read", "Bash"] ``` - Not tested: full `pytest`, full-repo `ruff check .`, `mypy headroom`, or a live third-party Anthropic-compatible provider. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - This PR intentionally does not add documentation because it fixes passthrough behavior rather than introducing a new user-facing option. - The code-comment checklist item is left unchecked because the change is covered by a small helper docstring and regression tests; no extra inline comments seemed necessary. - `CHANGELOG.md` is left unchanged because this is a narrowly scoped bug fix. - Local pytest collection for these proxy tests required a local `headroom._core` extension symlink, which was removed before committing. |
||
|
weijie_chen
|
b4682d6f91
|
fix(proxy): honor force_kompress routing profile (#996)
## Description Honor the proxy savings profile's `force_kompress` setting all the way through the Anthropic proxy path. `HEADROOM_SAVINGS_PROFILE=agent-90` already resolves to `force_kompress=True`, but `ContentRouter` still paid for the full auto-detection path before selecting Kompress. On long Claude Code / tool-output conversations this can hang inside the detection/router path before any `Transform content_router` line is emitted. This change makes the forced-Kompress path skip unused strategy detection during compression, while still preserving recent-code protection via the lightweight regex detector. This also passes `proxy_pipeline_kwargs(self.config)` through Anthropic batch requests so batch traffic receives the same savings-profile knobs as normal Anthropic messages. Refs #946 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Skip `is_mixed_content()` / `_detect_content()` when runtime `force_kompress` is set and route directly to `CompressionStrategy.KOMPRESS`. - Keep forced-Kompress recent-code protection, but use `_regex_detect_content_type()` instead of the full router detection chain. - Read `_runtime_force_kompress` defensively in `ContentRouter.apply()` so regular `ContentRouter()` instances keep the normal content-detection path. - Pass proxy savings-profile kwargs into Anthropic batch compression. - Add regression tests for forced-Kompress routing, normal routing, recent-code protection, and Anthropic batch profile propagation. - Update `CHANGELOG.md`. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/transforms/content_router.py tests/test_transforms_content_router.py All checks passed! $ ruff format --check headroom/transforms/content_router.py tests/test_transforms_content_router.py 2 files already formatted $ pytest tests/test_transforms_content_router.py::test_force_kompress_bypasses_content_detection \ tests/test_transforms_content_router.py::test_normal_compress_path_still_uses_content_detection \ tests/test_transforms_content_router.py::test_force_kompress_apply_uses_lightweight_detection \ tests/test_transforms_content_router.py::test_force_kompress_apply_lightweight_detection_protects_recent_code \ tests/test_proxy_anthropic_cache_stability.py::test_batch_optimization_passes_savings_profile_kwargs \ tests/test_bundled_tools_savings.py -q ============================= test session starts ============================= platform win32 -- Python 3.13.1, pytest-9.0.3, pluggy-1.6.0 rootdir: E:\work\code\third-party\headroom configfile: pyproject.toml plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0, cov-7.0.0, timeout-2.4.0 collected 11 items tests\test_transforms_content_router.py .... [ 36%] tests\test_proxy_anthropic_cache_stability.py . [ 45%] tests\test_bundled_tools_savings.py ....ss [100%] ======================== 9 passed, 2 skipped in 9.77s ========================= ``` Full-suite attempt status on Windows / Python 3.13 after installing missing local test dependencies and bundled tools (`fastembed`, `socksio`, `pytest-timeout`, `difft`, `scc`, cached HF models with offline env vars): ```text tests/test_adapter_hooks.py: 29 passed, 2 failed - sqlite:///C:\... and jsonl:///C:\... URLs are parsed into invalid \C:\... paths on Windows. tests/test_cache/test_client_integration.py: 16 failed - Same Windows URL path parsing issue. tests/test_cli/test_wrap_helpers.py: 29 passed, then KeyboardInterrupt during read/cleanup. tests/test_memory tests/test_storage: - Collection/run receives KeyboardInterrupt in this Windows environment. ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.1, `headroom-ai` v0.25.0, Anthropic proxy on `127.0.0.1:8787`, Kompress ONNX backend, `HEADROOM_SAVINGS_PROFILE=agent-90`, `HEADROOM_COMPRESS_USER_MESSAGES=1`, `HEADROOM_MIN_TOKENS=120`. - Exact command / steps: started the proxy with the local launcher, sent a long `/v1/messages` request with a fake upstream token, and inspected `/livez`, `/stats?include_config=true`, and `~/.headroom/logs/proxy.log`. - Observed result: request returned promptly with the expected upstream auth failure after local compression, and logs showed the compression ran before forwarding: ```text /livez healthy /v1/messages completed in ~3005ms with expected upstream 401 Transform content_router: 1900 -> 193 tokens (saved 1707) [1328.4ms] Pipeline complete: 1907 -> 200 tokens (saved 1707, 89.5% reduction) UPSTREAM_ERROR ... compressed=yes transforms=['router:kompress:0.06'] original_tokens=1886 optimized_tokens=119 PERF ... tok_before=1886 tok_after=119 tok_saved=1767 transforms=router:kompress:0.06 /stats tokens.saved = 1767 /stats compressions_by_strategy = {"kompress": 1} ``` - Not tested: full upstream CI matrix, full `uv run pytest`, full `ruff check .`, `mypy headroom`, real Anthropic success response with a valid upstream token, and Anthropic batch against the live upstream. The Anthropic batch change is covered by a local handler regression test. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes This PR is ready for human review. The patch is scoped to the forced-Kompress profile path and does not change the default auto-routing behavior when `force_kompress` is false. The latest `PR Governance / template` check passes after the readiness checkbox update. A later `PR Governance / label` run currently fails while trying to execute `.github/scripts/pr-health-labels.py` from the base checkout; that file is missing on the checked-out base ref, so this appears to be a governance workflow issue rather than a PR-template/content failure in this branch. |
||
|
chopratejas
|
2fb905fdb0 |
fix: integrate B6+B7 — fix cross-test contamination + injector mock parity
Two follow-ups surfaced when B6 and B7 were merged onto the megamerge branch and the full suite ran: 1. tests/test_proxy_anthropic_cache_stability.py PR-B7 added `injector.scan_for_markers(optimized_messages)` to the Anthropic handler so the always-on tool-registration logic can see detected hashes for the current request. The two pre-existing `_FakeInjector` mocks (`test_ccr_system_instruction_injection_disabled_*` and `test_ccr_tool_injection_disabled_*`) didn't implement that method. Added a no-op `scan_for_markers` returning [] to both mocks — matches the real injector's contract for the not-yet-compressed request shape these tests exercise. 2. tests/test_memory_tool_mode.py::test_tool_mode_skip_emits_structured_log The B6 caplog assertion passed in isolation but failed in the full suite. Root cause: when an earlier test triggers proxy startup, `_setup_file_logging` flips `headroom.propagate=False` and attaches a RotatingFileHandler to the headroom logger. caplog captures via propagation to root, so log records stop reaching it. The conftest autouse fixture that resets `propagate=True` before every test gets shadowed by fixture-ordering edge cases. Principled fix: attach `caplog.handler` directly to `headroom.proxy.memory_handler` for the duration of the test so the capture is independent of propagation state. Restore the original level + remove the handler in `finally` to keep the test hermetic. Both B6 and B7 cherry-picks themselves are unmodified. This commit only adjusts test harness code so the pre-existing mocks/capture stay consistent with the new code paths. |
||
|
chopratejas
|
f0dcc02775 |
fix: A3 — byte-faithful Python forwarders; serialize canonical only when mutated
Eliminates P0-2 universally. Every Python forwarder (server.py
`_retry_request`, handlers/streaming.py `_stream_response`,
handlers/openai.py `_ws_http_fallback`, handlers/batch.py `_batch_passthrough`
+ batch-create + Google batch passthrough, handlers/anthropic.py CCR
continuation + batch endpoint) now switches from `httpx ... json=body` to
`httpx ... content=raw_bytes`. The default httpx JSON encoder was
re-serializing every request with `, `/`: ` separators and `\\uXXXX` ASCII
escapes — collapsing Anthropic prompt-cache hit-rate.
Forwarder strategy:
- unmutated body → forward `await request.body()` verbatim;
- mutated body → re-serialize once via the new
`serialize_body_canonical(body) -> bytes` helper (compact separators,
`ensure_ascii=False`, dict insertion order preserved).
`HEADROOM_PROXY_PYTHON_FORWARDER_MODE` env var configures the mode:
- `byte_faithful` (default) — the new behavior;
- `legacy_json_kwarg` — explicit operator opt-in for emergency rollback.
Documented in `docs/content/docs/configuration.mdx`. NOT a fallback —
unknown values raise loudly per build constraint #4.
`BodyMutationTracker` accompanies each request through the handler so
transform sites mark the tracker (`memory_injection`,
`image_compression`, `compression_*`, `batch_compression`,
`ccr_continuation`, etc.). At forwarder dispatch we additionally compare
the final body dict against the parsed original bytes as a structural
safety net — any silent mutation we missed still triggers canonical
re-serialization.
A2 follow-up: `handlers/openai.py:534-540` (Chat Completions memory
injection) was prepending a system message; replaced with
`append_text_to_latest_user_chat_message`, the OpenAI Chat Completions
analog of `_append_context_to_latest_non_frozen_user_turn`. The cache
hot zone (system messages) is now sacrosanct on /v1/chat/completions
too. Honors `HEADROOM_MEMORY_INJECTION_MODE=disabled`.
Structured logging: every forwarder emits an `event=outbound_request`
log line with `forwarder`, `path`, `body_bytes`, `body_mutated`,
`mutation_reasons`, `source` (passthrough|canonical|legacy),
`request_id`. Never logs Authorization or full body.
`_read_request_json` factored to share `_read_request_body_bytes` with
new `read_request_json_with_bytes` so the anthropic handler can capture
both the parsed dict and the original (decompressed) bytes.
Tests:
- `tests/test_proxy_byte_faithful_forwarding.py` (28 tests):
SHA-256 byte-equality on /v1/messages and streaming, unicode
preservation, numeric precision, mutation-tracker invariants,
canonical-serializer properties, legacy-mode rollback, OpenAI
Chat memory routing.
- Existing test mocks updated to accept the new `**kwargs` on
`_retry_request` (no behavior change).
- `tests/test_proxy_handlers_batch.py` updated to read the captured
`content=` bytes (formerly `json=`).
- One A2 test corrected (`test_anthropic_tool_sort_and_context_append_helpers`)
to match the live-zone-tail semantics introduced by A2.
Constraints satisfied: configurable env var; no new regex / hardcodes;
no silent fallback (`legacy_json_kwarg` is operator opt-in);
performant (`prepare_outbound_body_bytes` is O(1) for passthrough);
elegant single-responsibility helpers; structured tracing logs.
|
||
|
chopratejas
|
521fbbeabd | style: apply ruff format to test_proxy_anthropic_cache_stability lambdas | ||
|
chopratejas
|
35eaf8de7f |
fix(proxy): remove content-keyed TTL walker that conflated content with positional cache (#327)
The Anthropic token-mode handler walked past prefix_tracker.frozen_message_count
whenever an upcoming tool_result's content-hash matched comp_cache._stable_hashes
or should_defer_compression returned True. That conflated content equality with
positional cache membership.
Anthropic's prefix cache is POSITIONAL: bytes 0..K cached, anything past K is
fresh. _stable_hashes is content-keyed and grows unbounded. In long Claude Code
sessions where tool_result content rhymes across turns (repeated system prompts,
repeated file reads, repeated tool descriptions), the walker advanced
frozen_message_count to len(messages) on every turn and the pipeline produced
transforms_applied=[] on 73% of requests in user SvenMeyer's reported session
(headroom-stats-2026-05-01.json: 74 of 101 eligible requests "prefix_frozen") —
even after the prior fix in
|
||
|
SwiftWing21
|
429ae0095b |
fix(proxy): guard CCR tool injection against frozen prefix to preserve cache
The Anthropic handler's CCR injector path applied a frozen_message_count guard to system instruction injection but not to tool injection. When Kompress fired for the first time in a session, the tools array was mutated unconditionally, invalidating Anthropic's prefix cache and dropping cache_read_input_tokens to zero on calls where ~48K tokens were previously being cached. Mirror the existing inject_system_instructions guard for inject_tool: when frozen_message_count > 0, defer tool injection so the warm prefix stays intact. Adds test_ccr_tool_injection_disabled_when_prefix_frozen as a direct companion to the existing test_ccr_system_instruction_injection_ disabled_when_prefix_frozen. Fixes #294 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
chopratejas
|
a9aa66e4c6 |
Fix 19 test failures: missing security attr, nosec inside f-strings, stale test mock
- Initialize self.security = None on HeadroomProxy (enterprise plugin hook) - Move '# nosec B608' comments outside f-strings in sqlite.py and fts5.py (SQLite doesn't understand # as comment, causing OperationalError) - Add mark_stable_from_messages to _FakeCompressionCache test mocks - Update token mode freeze assertion to match cache-aware behavior |
||
|
JerrettDavis
|
6f701033a1 |
test: align anthropic cache stability fixtures
Sync the Anthropic cache stability test double with the prefix tracker contract used by the handler. Format the benchmark scripts that were failing ruff format --check in CI. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
JerrettDavis
|
5ffa77f4a3 | Harden anthropic cache-mode replay stability | ||
|
JerrettDavis
|
96fd3d9652 | Add Anthropic cache-mode delta replay | ||
|
JerrettDavis
|
f2a32f9721 | Harden Anthropic cache mode stability | ||
|
JerrettDavis
|
83b730f2b9 | Fix CI lint/format failures after proxy mode hardening | ||
|
JerrettDavis
|
54419ad8b8 | Rebrand proxy modes to token/cache and harden cache-mode stability | ||
|
JerrettDavis
|
7d02829f02 | Harden Anthropic prefix cache stability across proxy and batch paths |