mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
fix(proxy): stop toggling headroom_retrieve in the Anthropic tools array (#2672)
## Description `should_inject_ccr_tool` deferred CCR tool injection whenever `frozen_message_count > 0`. Because `tools` is the head of Anthropic's cache key, that dropped a tool which was already inside the provider-cached prefix and invalidated the whole prefix — in both directions (`0 → >0` removes it; `>0 → 0` on proxy restart, `/model` switch, lineage eviction or TTL lapse adds it back). On three days of local proxy logs the turns that flipped injection state carried **44.7% of all cache-write tokens at a 52.0% hit rate**, against 98.1% for non-flipping turns. The log signature is `cache_read` alternating between two values exactly 172 tokens apart — the 464-byte tool definition. This deletes the gate and calls `apply_session_sticky_ccr_tool` directly, which is **what `openai.py` already does** — the two handlers now have the same shape. Net −61 production lines, no new state, no new config flag. Fixes defect 1 of #2671. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Performance improvement ## Changes Made - `headroom/proxy/ccr_marker_policy.py` — deleted the `should_inject_ccr_tool` gate; `apply_session_sticky_ccr_tool` is now the single decision point. - `headroom/proxy/handlers/anthropic.py` — calls `apply_session_sticky_ccr_tool` directly, matching `openai.py`. - `headroom/proxy/helpers.py` — dropped the now-unused gate plumbing. - `tests/test_proxy_anthropic_cache_stability.py` — new test asserting the forwarded `tools` array is byte-identical across a `frozen 0 → >0` transition. - `tests/test_ccr_marker_policy.py` — removed the three unit tests that pinned the deleted decision (they encoded the defect). - `tests/test_proxy/test_ccr_frozen_prefix_coupling.py` — same unredeemable-marker intent, re-pinned at the sticky helper. - `tests/test_proxy/test_anthropic_ccr_deferred_injection.py` — autouse reset fixture for the process-global `SessionCcrTracker` (separate commit). - Formatting-only follow-up commit applying `ruff format` (pinned 0.15.17) to the two test files above. ### Why deleting the gate is safe `apply_session_sticky_ccr_tool` already holds the correct rule. Its four branches, in order: | # | condition | action | |---|---|---| | 1 | tool already in the incoming tool list (client/MCP pre-registered) | skip; the client's bytes win | | 2 | `session_id is None` (WS / pre-session) | per-turn flag drives it verbatim | | 3 | session has done CCR | always inject the recorded golden bytes | | 4 | fresh session, no compression this turn | **skip** | Branch 4 is the safety property: a session that has never compressed still gets no tool, so removing the gate cannot start injecting into non-CCR conversations. Branch 3 is what the gate was starving. `has_new_ccr_markers` still gates first-time injection, so markers replayed from the previously-forwarded prefix cannot trigger one. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed The three deleted unit tests encoded the defect. Coverage moves to the property that actually matters and was previously untested: **the forwarded `tools` array must be byte-identical across a `frozen 0 → >0` transition.** That test asserts on the forwarded request body rather than on a policy function's return value; unit-testing the old policy in isolation is exactly what let a wrong-but-self-consistent decision pass. Verified failing on `upstream/main` with an assertion on the missing tool (not an `ImportError`, so it fails for the right reason). Full suite: same pre-existing unrelated failures as `upstream/main`, **zero new** (verified by running the whole suite on both revisions and diffing the failure sets). ### Test Output ```text $ uv run pytest tests/test_ccr_marker_policy.py \ tests/test_proxy/test_anthropic_ccr_deferred_injection.py \ tests/test_proxy/test_ccr_frozen_prefix_coupling.py \ tests/test_proxy_anthropic_cache_stability.py -q collected 48 items tests/test_ccr_marker_policy.py ..... [ 10%] tests/test_proxy/test_anthropic_ccr_deferred_injection.py .............. [ 39%] . [ 41%] tests/test_proxy/test_ccr_frozen_prefix_coupling.py .. [ 45%] tests/test_proxy_anthropic_cache_stability.py .......................... [100%] ======================= 48 passed, 2 warnings in 13.59s ======================== $ ruff check . All checks passed! $ ruff format --check . 1349 files already formatted ``` ## Real Behavior Proof - Environment: local macOS proxy serving live Claude Code traffic to the Anthropic API; baseline = 3 days of proxy logs on `upstream/main`, after = 5.5 hours with this change live. - Exact command / steps: ran the proxy with this branch built in, drove normal Claude Code sessions through it (including `/model` switches and proxy restarts, the two events that used to flip injection state), then parsed 235 real turns from the proxy logs with the same parser used for the baseline in #2671. - Observed result: flip turns fell from 177 (44.7% of all cache write) to 2 (1.7%); steady-state write share 1.192% → 0.867%; aggregate hit rate 86.75% → 89.21%; main conversation warm hit rate 98.1% → 97.70% (n=149). The 2 remaining "flips" have `cache_read == 0` — cold starts that the bucketing counts as a state change, not real flips. | metric | baseline | after | |---|---|---| | flip turns | 177, carrying 44.7% of all cache write | **2**, carrying **1.7%** | | main conv, warm | 98.1% | **97.70%** (n=149) | | steady-state write share | 1.192% | **0.867%** | | aggregate | 86.75% | **89.21%** | - Not tested: `mypy headroom` was not run locally for this body; the OpenAI handler path (unchanged by this PR); tracker state loss mid-session (see note below); and defect 2 of #2671 (the sub-call breakpoint), which is untouched and is now 54.9% of remaining cache write — that is why aggregate stays just under 90%. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes **Pre-existing and unchanged here:** if the tracker loses state mid-session while the transcript still carries markers, branch 4 returns no tool and those markers are unredeemable. `upstream/main` has no recovery for that; this PR neither creates nor fixes it. See my comment on #2500, which adds a recovery path for the related dangling-reference case. **N/A checklist items:** no documentation changes — this removes an internal policy function with no user-facing surface. `mypy headroom` left unchecked because it was not run for this body; CI covers it. **Merge-order conflict with #2500 (please read before landing either):** this PR *deletes* `should_inject_ccr_tool`, which is the exact function #2500 extends with `transcript_requires_tool`. Whichever lands second needs a semantic rebase, not just a textual one — git will not flag it. If this PR lands first, #2500's recovery path should re-target `apply_session_sticky_ccr_tool` (the sticky helper now owns the decision alone) or the handler call site in `handlers/anthropic.py`. If #2500 lands first, the gate deletion here still applies but the `transcript_requires_tool` override needs to move with it. Happy to do the rebase either way — say which order you prefer.
This commit is contained in:
nangsontay
GitHub
parent
0221e7f240
commit
08fce29b47
7 changed files with 193 additions and 186 deletions
|
|
@ -1,4 +1,8 @@
|
|||
"""CCR marker freshness and retrieval-tool injection policy."""
|
||||
"""CCR marker freshness policy.
|
||||
|
||||
Retrieval-tool injection is decided by ``apply_session_sticky_ccr_tool`` in
|
||||
``headroom.proxy.helpers``, from what the session has actually forwarded.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
|
@ -28,18 +32,3 @@ def has_new_ccr_markers(
|
|||
)
|
||||
previous.scan_for_markers(previous_forwarded_messages)
|
||||
return bool(current - set(previous.detected_hashes))
|
||||
|
||||
|
||||
def should_inject_ccr_tool(
|
||||
*,
|
||||
configured_inject_tool: bool,
|
||||
frozen_message_count: int,
|
||||
has_compressed_content: bool,
|
||||
) -> tuple[bool, bool]:
|
||||
"""Decide whether the CCR retrieval tool must be injected this turn."""
|
||||
|
||||
inject_tool = configured_inject_tool
|
||||
if inject_tool and frozen_message_count > 0:
|
||||
inject_tool = False
|
||||
is_marker_override = not inject_tool and has_compressed_content
|
||||
return (inject_tool or is_marker_override), is_marker_override
|
||||
|
|
|
|||
|
|
@ -1845,11 +1845,6 @@ class AnthropicHandlerMixin:
|
|||
)
|
||||
inject_system_instructions = False
|
||||
configured_inject_tool = self.config.ccr_inject_tool
|
||||
if configured_inject_tool and frozen_message_count > 0:
|
||||
logger.info(
|
||||
f"[{request_id}] CCR: deferring tool injection "
|
||||
f"(frozen_message_count={frozen_message_count}) to preserve cache"
|
||||
)
|
||||
# Scan for compression markers + maybe inject system instructions.
|
||||
# Tool-list injection is handled separately via the sticky helper.
|
||||
injector = CCRToolInjector(
|
||||
|
|
@ -1865,47 +1860,34 @@ class AnthropicHandlerMixin:
|
|||
# retrieval tool once a session has done CCR, regardless
|
||||
# of whether THIS turn produced compressed content.
|
||||
#
|
||||
# #1006: if tool injection was deferred (frozen prefix) but
|
||||
# compression just emitted NEW markers this turn, override the
|
||||
# deferral — the agent has no other way to redeem those markers.
|
||||
# The cache miss on this one request is preferable to silent
|
||||
# data loss. If the session has already done CCR the tool is
|
||||
# already in the client's tool list, so sticky replay is a
|
||||
# no-op and the cache is unaffected.
|
||||
# ponytail: ceiling is one extra cache miss on the first CCR
|
||||
# turn in a frozen-prefix session.
|
||||
from headroom.proxy.helpers import (
|
||||
has_new_ccr_markers,
|
||||
should_inject_ccr_tool,
|
||||
)
|
||||
|
||||
# #1850: only markers NEW this turn justify overriding the
|
||||
# injection deferral (#1006). Markers replayed from the
|
||||
# previously-forwarded prefix (overlay_cached_prefix) are
|
||||
# historical — counting them would re-inject the tool on every
|
||||
# frozen turn and bust the *tools* cache segment, undoing the
|
||||
# overlay's messages-prefix cache-safety.
|
||||
has_new_compressed_content = has_new_ccr_markers(
|
||||
current_detected_hashes=injector.detected_hashes,
|
||||
previous_forwarded_messages=prefix_tracker.get_last_forwarded_messages(),
|
||||
provider="anthropic",
|
||||
)
|
||||
|
||||
should_inject, is_marker_override = should_inject_ccr_tool(
|
||||
configured_inject_tool=configured_inject_tool,
|
||||
frozen_message_count=frozen_message_count,
|
||||
has_compressed_content=has_new_compressed_content,
|
||||
)
|
||||
if should_inject:
|
||||
if is_marker_override:
|
||||
logger.info(
|
||||
f"[{request_id}] CCR: overriding injection deferral — "
|
||||
f"new markers emitted but headroom_retrieve unavailable "
|
||||
f"(frozen_message_count={frozen_message_count}); injecting to "
|
||||
"prevent unredeemable markers (#1006)"
|
||||
)
|
||||
from headroom.proxy.helpers import apply_session_sticky_ccr_tool
|
||||
# Injection is deliberately NOT gated on
|
||||
# ``frozen_message_count``. That counter answers "is the
|
||||
# prefix warm?", but the decision needs "does the established
|
||||
# prefix already contain the tool?" — which only
|
||||
# ``SessionCcrTracker`` knows. Gating on the counter dropped a
|
||||
# tool that was already inside the provider-cached prefix, and
|
||||
# ``tools`` is the head of Anthropic's cache key, so every
|
||||
# toggle invalidated the entire prefix in both directions.
|
||||
# ``apply_session_sticky_ccr_tool`` carries the correct rule:
|
||||
# a session that has never compressed still gets no tool, so
|
||||
# dropping the gate cannot start injecting into non-CCR
|
||||
# conversations.
|
||||
if configured_inject_tool:
|
||||
from headroom.proxy.helpers import (
|
||||
apply_session_sticky_ccr_tool,
|
||||
has_new_ccr_markers,
|
||||
)
|
||||
|
||||
# #1850: markers replayed from the previously-forwarded
|
||||
# prefix (overlay_cached_prefix) are historical; only
|
||||
# markers NEW this turn may drive a first-time injection,
|
||||
# else a replayed marker injects the tool into a session
|
||||
# that never actually compressed.
|
||||
has_new_compressed_content = has_new_ccr_markers(
|
||||
current_detected_hashes=injector.detected_hashes,
|
||||
previous_forwarded_messages=prefix_tracker.get_last_forwarded_messages(),
|
||||
provider="anthropic",
|
||||
)
|
||||
tools, ccr_tool_injected = apply_session_sticky_ccr_tool(
|
||||
provider="anthropic",
|
||||
session_id=session_id,
|
||||
|
|
|
|||
|
|
@ -73,9 +73,6 @@ from headroom.proxy.ccr_golden_policy import (
|
|||
from headroom.proxy.ccr_marker_policy import (
|
||||
has_new_ccr_markers as _has_new_ccr_markers,
|
||||
)
|
||||
from headroom.proxy.ccr_marker_policy import (
|
||||
should_inject_ccr_tool as _should_inject_ccr_tool,
|
||||
)
|
||||
from headroom.proxy.ccr_session_tracker import SessionCcrTracker as _SessionCcrTracker
|
||||
from headroom.proxy.internal_header_policy import (
|
||||
INTERNAL_HEADER_PREFIX,
|
||||
|
|
@ -1715,35 +1712,6 @@ def has_new_ccr_markers(
|
|||
)
|
||||
|
||||
|
||||
def should_inject_ccr_tool(
|
||||
*,
|
||||
configured_inject_tool: bool,
|
||||
frozen_message_count: int,
|
||||
has_compressed_content: bool,
|
||||
) -> tuple[bool, bool]:
|
||||
"""Decide whether the ``headroom_retrieve`` tool must be injected this turn.
|
||||
|
||||
This is the decision the Anthropic handler used to inline. It is extracted
|
||||
so the #1006 regression can be pinned at the decision point itself.
|
||||
|
||||
Tool injection is normally deferred when there is a frozen message prefix
|
||||
(``frozen_message_count > 0``) to preserve the prompt cache. But if
|
||||
compression emitted fresh markers this turn, deferring would hand the agent
|
||||
a ``<<ccr:hash>>`` marker with no tool to redeem it — silent data loss. In
|
||||
that case we override the deferral and inject anyway (one cache miss is
|
||||
cheaper than dropped content).
|
||||
|
||||
Returns ``(should_inject, is_marker_override)``. ``is_marker_override`` is
|
||||
True only when injection happens *because* of new markers despite a deferral,
|
||||
so the caller can log the override distinctly.
|
||||
"""
|
||||
return _should_inject_ccr_tool(
|
||||
configured_inject_tool=configured_inject_tool,
|
||||
frozen_message_count=frozen_message_count,
|
||||
has_compressed_content=has_compressed_content,
|
||||
)
|
||||
|
||||
|
||||
def apply_session_sticky_ccr_tool(
|
||||
*,
|
||||
provider: Literal["anthropic", "openai", "google"],
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
from __future__ import annotations
|
||||
|
||||
from headroom.ccr.tool_injection import CCRToolInjector
|
||||
from headroom.proxy.ccr_marker_policy import has_new_ccr_markers, should_inject_ccr_tool
|
||||
from headroom.proxy.ccr_marker_policy import has_new_ccr_markers
|
||||
|
||||
|
||||
def _hashes(*contents: str) -> list[str]:
|
||||
|
|
@ -70,27 +70,3 @@ def test_has_new_ccr_markers_returns_false_without_current_hashes() -> None:
|
|||
)
|
||||
is False
|
||||
)
|
||||
|
||||
|
||||
def test_should_inject_ccr_tool_overrides_frozen_prefix_deferral_for_markers() -> None:
|
||||
assert should_inject_ccr_tool(
|
||||
configured_inject_tool=True,
|
||||
frozen_message_count=3,
|
||||
has_compressed_content=True,
|
||||
) == (True, True)
|
||||
|
||||
|
||||
def test_should_inject_ccr_tool_defers_frozen_prefix_without_markers() -> None:
|
||||
assert should_inject_ccr_tool(
|
||||
configured_inject_tool=True,
|
||||
frozen_message_count=3,
|
||||
has_compressed_content=False,
|
||||
) == (False, False)
|
||||
|
||||
|
||||
def test_should_inject_ccr_tool_injects_configured_tool_without_frozen_prefix() -> None:
|
||||
assert should_inject_ccr_tool(
|
||||
configured_inject_tool=True,
|
||||
frozen_message_count=0,
|
||||
has_compressed_content=False,
|
||||
) == (True, False)
|
||||
|
|
|
|||
|
|
@ -9,11 +9,27 @@ pytest.importorskip("fastapi")
|
|||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from headroom.proxy.helpers import _reset_session_ccr_tracker_for_test
|
||||
from headroom.proxy.server import ProxyConfig, create_app
|
||||
|
||||
_RAW_TRANSCRIPT = "\n".join(f"row {idx}: payload payload payload" for idx in range(80))
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _reset_ccr_tracker():
|
||||
"""Isolate the process-global ``SessionCcrTracker`` between tests.
|
||||
|
||||
Several tests here share ``session_id="stable-session"``, and the tracker's
|
||||
``has_done_ccr`` flag is monotonic per session. Without this, a test that
|
||||
injects the tool leaves the flag set and the next test sees a sticky replay
|
||||
it never set up — order-dependent, and only visible in file order, not when
|
||||
run alone. Mirrors the fixture in ``tests/test_ccr_tool_always_on.py``.
|
||||
"""
|
||||
_reset_session_ccr_tracker_for_test()
|
||||
yield
|
||||
_reset_session_ccr_tracker_for_test()
|
||||
|
||||
|
||||
class _FakePrefixTracker:
|
||||
def __init__(self, frozen_count: int):
|
||||
self._frozen_count = frozen_count
|
||||
|
|
|
|||
|
|
@ -1,13 +1,21 @@
|
|||
"""Regression test for #1006: the proxy must not emit unredeemable CCR markers.
|
||||
|
||||
When frozen_message_count > 0, the old code deferred headroom_retrieve tool
|
||||
injection unconditionally — even if compression just emitted NEW <<ccr:hash>>
|
||||
markers the agent has no tool to redeem.
|
||||
If compression emits a fresh ``<<ccr:hash>>`` marker, the forwarded request must
|
||||
also carry ``headroom_retrieve`` — a marker the agent has no tool to redeem is
|
||||
silent data loss.
|
||||
|
||||
The fix: if new markers were emitted this turn, override the deferral and inject
|
||||
the tool (one cache miss is acceptable; silent data loss is not). That decision
|
||||
lives in ``should_inject_ccr_tool``, which the Anthropic handler calls; this test
|
||||
pins the decision at that function so removing the override would fail here.
|
||||
This used to be enforced by an override *inside* a ``frozen_message_count``
|
||||
deferral gate. That gate is gone: deferring on the freeze counter dropped a tool
|
||||
that was already inside the provider-cached prefix, and ``tools`` is the head of
|
||||
Anthropic's cache key, so every toggle invalidated the whole prefix.
|
||||
``apply_session_sticky_ccr_tool`` now decides alone, from what the session has
|
||||
actually forwarded. #1006 is therefore pinned here, at that helper, and the
|
||||
turn-over-turn cache property is pinned in
|
||||
``tests/test_proxy_anthropic_cache_stability.py``.
|
||||
|
||||
These cases deliberately drive a real ``CCRToolInjector`` marker scan rather than
|
||||
passing a hand-set boolean, so the marker -> flag -> tool chain stays covered end
|
||||
to end.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
@ -15,50 +23,13 @@ from __future__ import annotations
|
|||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from headroom.ccr.tool_injection import CCR_TOOL_NAME, CCRToolInjector
|
||||
from headroom.proxy.helpers import (
|
||||
apply_session_sticky_ccr_tool,
|
||||
should_inject_ccr_tool,
|
||||
)
|
||||
from headroom.proxy.helpers import apply_session_sticky_ccr_tool
|
||||
|
||||
|
||||
class TestShouldInjectCCRTool:
|
||||
"""The decision the handler used to inline. This is where #1006 lived."""
|
||||
class TestMarkersImplyRedeemableTool:
|
||||
"""A marker emitted this turn must arrive with the tool that redeems it."""
|
||||
|
||||
def test_overrides_deferral_when_markers_emitted(self):
|
||||
"""Frozen prefix would normally defer, but fresh markers force injection."""
|
||||
should_inject, is_override = should_inject_ccr_tool(
|
||||
configured_inject_tool=True,
|
||||
frozen_message_count=3,
|
||||
has_compressed_content=True,
|
||||
)
|
||||
assert should_inject, "must inject to keep markers redeemable (#1006)"
|
||||
assert is_override, "this is the deferral override path"
|
||||
|
||||
def test_defers_when_no_markers(self):
|
||||
"""Frozen prefix with no new markers stays deferred — no spurious tool."""
|
||||
should_inject, is_override = should_inject_ccr_tool(
|
||||
configured_inject_tool=True,
|
||||
frozen_message_count=3,
|
||||
has_compressed_content=False,
|
||||
)
|
||||
assert not should_inject
|
||||
assert not is_override
|
||||
|
||||
def test_injects_normally_without_frozen_prefix(self):
|
||||
"""No frozen prefix → inject as configured, not via the override path."""
|
||||
should_inject, is_override = should_inject_ccr_tool(
|
||||
configured_inject_tool=True,
|
||||
frozen_message_count=0,
|
||||
has_compressed_content=False,
|
||||
)
|
||||
assert should_inject
|
||||
assert not is_override
|
||||
|
||||
|
||||
class TestCCRInjectionEndToEnd:
|
||||
"""The decision feeds apply_session_sticky_ccr_tool; assert the tool lands."""
|
||||
|
||||
def test_marker_in_frozen_prefix_yields_injected_tool(self):
|
||||
def test_fresh_marker_yields_injected_tool(self):
|
||||
# Injector detects a fresh marker, i.e. compression ran this turn.
|
||||
injector = CCRToolInjector(provider="anthropic")
|
||||
injector.scan_for_markers(
|
||||
|
|
@ -77,14 +48,6 @@ class TestCCRInjectionEndToEnd:
|
|||
)
|
||||
assert injector.has_compressed_content, "test setup: injector should detect marker"
|
||||
|
||||
# Drive the real decision the handler makes under a frozen prefix.
|
||||
should_inject, _ = should_inject_ccr_tool(
|
||||
configured_inject_tool=True,
|
||||
frozen_message_count=3,
|
||||
has_compressed_content=injector.has_compressed_content,
|
||||
)
|
||||
assert should_inject
|
||||
|
||||
with patch("headroom.proxy.helpers.get_session_ccr_tracker") as mock_tracker_fn:
|
||||
mock_tracker = MagicMock()
|
||||
mock_tracker.has_done_ccr.return_value = False # first CCR ever
|
||||
|
|
@ -101,22 +64,19 @@ class TestCCRInjectionEndToEnd:
|
|||
|
||||
tool_names = [t.get("name") for t in tools_out]
|
||||
assert CCR_TOOL_NAME in tool_names, (
|
||||
f"headroom_retrieve not injected when markers emitted and prefix frozen (#1006). "
|
||||
f"tools={tool_names}"
|
||||
f"headroom_retrieve not injected when markers were emitted (#1006). tools={tool_names}"
|
||||
)
|
||||
|
||||
def test_no_marker_in_frozen_prefix_skips_tool(self):
|
||||
def test_no_marker_on_session_that_never_compressed_skips_tool(self):
|
||||
"""The property that makes dropping the freeze gate safe.
|
||||
|
||||
A session with no markers and no CCR history still gets no tool, so
|
||||
removing the gate cannot start injecting into non-CCR conversations.
|
||||
"""
|
||||
injector = CCRToolInjector(provider="anthropic")
|
||||
injector.scan_for_markers([{"role": "user", "content": "hello"}])
|
||||
assert not injector.has_compressed_content, "test setup: no markers expected"
|
||||
|
||||
should_inject, _ = should_inject_ccr_tool(
|
||||
configured_inject_tool=True,
|
||||
frozen_message_count=3,
|
||||
has_compressed_content=injector.has_compressed_content,
|
||||
)
|
||||
assert not should_inject, "no markers → no forced injection"
|
||||
|
||||
with patch("headroom.proxy.helpers.get_session_ccr_tracker") as mock_tracker_fn:
|
||||
mock_tracker = MagicMock()
|
||||
mock_tracker.has_done_ccr.return_value = False
|
||||
|
|
@ -128,10 +88,10 @@ class TestCCRInjectionEndToEnd:
|
|||
session_id="session-frozen-no-markers",
|
||||
request_id="req-test-2",
|
||||
existing_tools=[],
|
||||
has_compressed_content_this_turn=False,
|
||||
has_compressed_content_this_turn=injector.has_compressed_content,
|
||||
)
|
||||
|
||||
tool_names = [t.get("name") for t in tools_out]
|
||||
assert CCR_TOOL_NAME not in tool_names, (
|
||||
"headroom_retrieve should NOT be injected when no markers and frozen prefix"
|
||||
"headroom_retrieve should NOT be injected for a session that never compressed"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -607,6 +607,122 @@ def test_ccr_tool_injection_disabled_when_prefix_frozen(monkeypatch) -> None:
|
|||
assert captured["inject_tool"] is False
|
||||
|
||||
|
||||
def test_ccr_tool_stays_in_forwarded_tools_across_frozen_transition() -> None:
|
||||
"""``tools`` identity must survive the ``frozen 0 -> >0`` transition.
|
||||
|
||||
``tools`` is the head of Anthropic's cache key, so adding or removing
|
||||
``headroom_retrieve`` between turns invalidates 100% of the provider-cached
|
||||
prefix — in both directions. Turn 1 (cold prefix, fresh markers) injects the
|
||||
tool; turn 2 (warm prefix, no *new* markers) must forward the same bytes
|
||||
rather than dropping it.
|
||||
|
||||
Asserts on the forwarded request body, not on a policy function's return
|
||||
value: unit-testing the old policy in isolation is exactly what let a
|
||||
wrong-but-self-consistent decision pass.
|
||||
"""
|
||||
from headroom.ccr.tool_injection import CCR_TOOL_NAME
|
||||
from headroom.proxy.helpers import (
|
||||
_reset_session_ccr_tracker_for_test,
|
||||
serialize_tool_definition_canonical,
|
||||
)
|
||||
|
||||
marker_message = {
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": "toolu_bash_x",
|
||||
"content": (
|
||||
"[50 items compressed to 5. Retrieve more: hash=abc123def456abc123def456]"
|
||||
),
|
||||
}
|
||||
],
|
||||
}
|
||||
forwarded: list[dict] = []
|
||||
|
||||
_reset_session_ccr_tracker_for_test()
|
||||
try:
|
||||
with _make_proxy_client() as client:
|
||||
proxy = client.app.state.proxy
|
||||
proxy.config.optimize = False
|
||||
proxy.config.image_optimize = False
|
||||
proxy.config.ccr_inject_tool = True
|
||||
proxy.config.ccr_inject_system_instructions = False
|
||||
|
||||
fake_tracker = _FakePrefixTracker(frozen_count=0)
|
||||
proxy.session_tracker_store.compute_session_id = lambda request, model, messages: (
|
||||
"frozen-transition-session"
|
||||
)
|
||||
proxy.session_tracker_store.get_or_create = lambda session_id, provider: fake_tracker
|
||||
|
||||
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001
|
||||
forwarded.append(body)
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "msg_frozen_transition",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"content": [{"type": "text", "text": "ok"}],
|
||||
"usage": {
|
||||
"input_tokens": 20,
|
||||
"output_tokens": 3,
|
||||
"cache_read_input_tokens": 0,
|
||||
"cache_creation_input_tokens": 0,
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
proxy._retry_request = _fake_retry
|
||||
|
||||
def _post():
|
||||
return client.post(
|
||||
"/v1/messages",
|
||||
headers={"x-api-key": "test-key", "anthropic-version": "2023-06-01"},
|
||||
json={
|
||||
"model": "claude-sonnet-4-6",
|
||||
"max_tokens": 64,
|
||||
"messages": [marker_message],
|
||||
},
|
||||
)
|
||||
|
||||
# Turn 1 — cold prefix, marker is new: first-time injection.
|
||||
assert _post().status_code == 200
|
||||
|
||||
# Turn 2 — the provider cached turn 1's prefix (tool included), and
|
||||
# the marker is now historical rather than new. Seed both facts
|
||||
# explicitly instead of relying on ``update_from_response`` plumbing:
|
||||
# if the marker still counted as new, the old code would have
|
||||
# injected via its override path and this test would pass against
|
||||
# the defect.
|
||||
fake_tracker._frozen_count = 3
|
||||
fake_tracker._last_forwarded_messages = [marker_message]
|
||||
|
||||
assert _post().status_code == 200
|
||||
finally:
|
||||
_reset_session_ccr_tracker_for_test()
|
||||
|
||||
assert len(forwarded) == 2, "expected exactly two forwarded requests"
|
||||
|
||||
def _ccr_tools(body: dict) -> list[dict]:
|
||||
return [t for t in (body.get("tools") or []) if t.get("name") == CCR_TOOL_NAME]
|
||||
|
||||
turn1 = _ccr_tools(forwarded[0])
|
||||
turn2 = _ccr_tools(forwarded[1])
|
||||
|
||||
assert turn1, "test setup: turn 1 should inject headroom_retrieve on fresh markers"
|
||||
assert turn2, (
|
||||
"headroom_retrieve was dropped from the forwarded tools array once the "
|
||||
"prefix went warm — that removes a tool already inside the cached prefix "
|
||||
"and busts 100% of it"
|
||||
)
|
||||
# Byte-identity, not ``==``: a re-serialized definition with a different key
|
||||
# order compares equal as a dict but busts the cache just as hard.
|
||||
assert serialize_tool_definition_canonical(turn1[0]) == serialize_tool_definition_canonical(
|
||||
turn2[0]
|
||||
), "headroom_retrieve was re-serialized rather than replayed byte-for-byte"
|
||||
|
||||
|
||||
def test_previous_turns_always_frozen_only_final_turn_mutable() -> None:
|
||||
captured = {}
|
||||
with _make_proxy_client() as client:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue