From 7de35739c61bed385dd078aee1b36865938c486d Mon Sep 17 00:00:00 2001 From: Abhay Singh Date: Fri, 14 Aug 2026 01:35:07 +0530 Subject: [PATCH] fix(proxy/anthropic): repair headroom_retrieve history references the tools array cannot support (#2876) ## Description #2805 / #2807 established the mechanism: Claude Code replays one transcript across requests that carry different `tools` arrays, and Anthropic validates every history reference against the array of the request at hand. #2807 fixed it for tool-search blocks by repairing history (`strip_unsupported_tool_search_blocks`) rather than trying to predict the client's tool set. The same mechanism applies to CCR's `headroom_retrieve`, and it is tool-agnostic. A passthrough side-request (the prompt-type Stop hook evaluator, `/compact`) that the proxy forwards without declaring `headroom_retrieve` still carries a historical `tool_use` naming it, and Anthropic 400s on the dangling reference. The injection-side fixes (#2766 / #2533) decide *when to re-declare the tool*; this makes the 400 *structurally impossible* where the tool is intentionally absent. It is belt-and-braces with them, not a replacement. The fix adds the symmetric repair next to #2807's. When the outbound `tools` array does not declare `headroom_retrieve`, it replaces each `headroom_retrieve` `tool_use` and its paired `tool_result` with a text block, so no dangling reference survives. It **neutralizes** (replaces in place) rather than **drops**, which is the one deliberate difference from #2807: CCR's `tool_use` lives in an assistant turn and its `tool_result` in the next user turn, i.e. two different messages. Dropping a whole message could leave two same-role messages adjacent and break Anthropic's strict user/assistant alternation, turning one 400 into another. Replacing blocks in place keeps every message and role intact, and preserves the retrieved text the model already saw. #2807's server-tool blocks both live in the same assistant turn, so dropping was safe there. It runs after CCR tool injection, so on the main loop -- where the tool IS injected (a present marker) -- it neutralizes nothing and the prompt-cache prefix is untouched, mirroring #2807's placement and sequencing. Fixes #2814 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/helpers.py`: added `strip_unsupported_ccr_retrieve_blocks(messages, tools)` (and a small `_ccr_result_as_text` helper). No-ops (returning the original object by identity) when `headroom_retrieve` is declared or no such history exists; otherwise neutralizes the `tool_use` and its paired `tool_result` to text. - `headroom/proxy/handlers/anthropic.py`: call the repair right after the tool-search history repair (which is after CCR tool injection), guarded on it actually changing anything, tagged `router:ccr_retrieve_repair:Nblocks`. - `tests/test_ccr_retrieve_history_repair.py`: 5 unit tests (no-op when declared, no-op without retrieve history, neutralize + preserve result text + keep alternation, leave foreign tool_use untouched, placeholder when the result has no text). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text tests/test_ccr_retrieve_history_repair.py 5 passed # Broader CCR / tool-search / handler suites (unchanged behavior): tests/test_ccr_retrieve_history_repair.py tests/test_proxy/test_anthropic_ccr_deferred_injection.py tests/test_issue_746_tool_search.py 71 passed # uvx ruff@0.15.17 check -> All checks passed! # uvx mypy@1.20.2 headroom/proxy/helpers.py -> Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1, ruff 0.15.17 and mypy 1.20.2 via uvx. - Exact command / steps: confirmed the injection point (`apply_session_sticky_ccr_tool`) and the tool-search repair placement in `handlers/anthropic.py`, confirmed `body["tools"]` reflects the CCR injection before the repair call site (`body["tools"] = tools` is written well upstream and the adjacent tool-search repair already relies on it), then drove the helper over a transcript with a `headroom_retrieve` tool_use + paired tool_result: with the tool declared it returns the original object unchanged; with the tool absent it neutralizes both blocks, preserves the result text, and keeps the message roles/count identical. - Observed result: a forwarded request that would 400 with "Tool reference 'headroom_retrieve' not found in available tools" now carries text blocks in place of the retrieve `tool_use`/`tool_result`, so there is no reference for Anthropic to reject, and user/assistant alternation is preserved. The main loop (tool present) is a no-op. - Not tested: a live multi-turn Claude Code session hitting a Stop-hook/`/compact` side-request against a real provider (no live provider here). The repair is a pure function verified directly over the exact block shapes Anthropic validates, and it mirrors the already-merged tool-search repair's mechanism and wiring. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md`: it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes The issue reporter noted their own logs show the tool-search variant of this 400 (61 across 11 days) but zero `headroom_retrieve` occurrences, because they run `HEADROOM_LOSSLESS=1` which disables CCR entirely. This PR fixes the CCR variant of the same, proven, tool-agnostic mechanism rather than a fresh CCR repro. The neutralize-vs-drop choice is the one place I departed from #2807, for the alternation reason above; if you would rather it drop (accepting the alternation handling that implies), I am happy to switch it. --------- Co-authored-by: Jerrett Davis --- headroom/proxy/handlers/anthropic.py | 22 ++++ headroom/proxy/helpers.py | 120 ++++++++++++++++++ tests/test_ccr_retrieve_history_repair.py | 119 +++++++++++++++++ ...est_tool_search_repair_after_turn_hooks.py | 69 +++++++++- 4 files changed, 327 insertions(+), 3 deletions(-) create mode 100644 tests/test_ccr_retrieve_history_repair.py diff --git a/headroom/proxy/handlers/anthropic.py b/headroom/proxy/handlers/anthropic.py index 7afc106c6..4abb25b41 100644 --- a/headroom/proxy/handlers/anthropic.py +++ b/headroom/proxy/handlers/anthropic.py @@ -2737,6 +2737,28 @@ class AnthropicHandlerMixin: _ts_stripped, ) + # CCR retrieve history repair (#2814). Run beside the tool-search + # repair and after both normal CCR injection and turn hooks, so it + # validates against the final outbound tools array. A side-request + # that does not declare headroom_retrieve cannot retain historical + # tool_use/tool_result references that Anthropic would reject. + from headroom.proxy.helpers import strip_unsupported_ccr_retrieve_blocks + + _ccr_repaired, _ccr_neutralized = strip_unsupported_ccr_retrieve_blocks( + body.get("messages"), body.get("tools") + ) + if _ccr_neutralized: + body["messages"] = _ccr_repaired + optimized_messages = _ccr_repaired + body_mutation_tracker.mark_mutated("ccr_retrieve_history_repair") + transforms_applied.append(f"router:ccr_retrieve_repair:{_ccr_neutralized}blocks") + logger.info( + "[%s] CCR: neutralized %d headroom_retrieve history block(s) " + "(tools array does not declare the tool this turn)", + request_id, + _ccr_neutralized, + ) + # Consistency: report tok_before/tok_after with ONE tokenizer. The pipeline # and the handler use different token estimators, and cache-mode branches # can leave original_tokens (handler, line ~1049) and optimized_tokens diff --git a/headroom/proxy/helpers.py b/headroom/proxy/helpers.py index 6ea06e19d..02a2f3c77 100644 --- a/headroom/proxy/helpers.py +++ b/headroom/proxy/helpers.py @@ -3027,6 +3027,126 @@ def strip_unsupported_tool_search_blocks(messages: Any, tools: Any) -> tuple[Any return (out, removed) if changed else (messages, 0) +def _ccr_result_as_text(block: dict[str, Any]) -> str: + """Flatten a ``tool_result`` block's content to plain text, preserving what + the model already saw. Falls back to a short placeholder when there is no + textual content to keep.""" + content = block.get("content") + if isinstance(content, str) and content.strip(): + return content + if isinstance(content, list): + parts = [ + b.get("text", "") + for b in content + if isinstance(b, dict) and b.get("type") == "text" and b.get("text") + ] + joined = "\n".join(part for part in parts if part) + if joined.strip(): + return joined + return "[headroom_retrieve result omitted]" + + +def strip_unsupported_ccr_retrieve_blocks(messages: Any, tools: Any) -> tuple[Any, int]: + """Neutralize ``headroom_retrieve`` history references the outbound ``tools`` + array cannot support. + + Claude Code replays one transcript across requests that carry different + ``tools`` arrays, and Anthropic validates every history ``tool_use`` against + the array of the request at hand. A passthrough side-request (the prompt-type + Stop hook evaluator, ``/compact``) that the proxy forwards without declaring + ``headroom_retrieve`` then 400s on a historical ``tool_use`` that names it -- + the CCR sibling of the tool-search history repair (#2814 / #2807). This is + belt-and-braces with the injection-side fixes: they keep the tool available + where it belongs; this makes the 400 structurally impossible where it cannot. + + When the request does NOT declare ``headroom_retrieve``, replace each + ``headroom_retrieve`` ``tool_use`` block and its paired ``tool_result`` with a + text block, so no dangling reference survives. Neutralize rather than drop: + CCR's ``tool_use`` (an assistant turn) and its ``tool_result`` (the next user + turn) live in DIFFERENT messages, so removing a message could leave two + same-role messages adjacent and break Anthropic's user/assistant alternation. + Replacing blocks in place keeps every message and role intact, and preserves + the retrieved text the model already saw. + + Returns ``(messages, blocks_neutralized)``, and the ORIGINAL ``messages`` + object when nothing changed -- callers rely on identity to skip the write-back. + """ + from headroom.ccr.tool_injection import CCR_TOOL_NAME + + if not isinstance(messages, list): + return messages, 0 + + tool_list = tools if isinstance(tools, list) else [] + available = {str(t["name"]) for t in tool_list if isinstance(t, dict) and t.get("name")} + # The tool is declared this turn (e.g. the main loop, or sticky re-injection), + # so its history references resolve. Nothing to repair. + if CCR_TOOL_NAME in available: + return messages, 0 + + # First pass: collect the ids of headroom_retrieve tool_use blocks so their + # paired tool_result blocks (in a later user turn) can be matched. + retrieve_ids: set[str] = set() + for message in messages: + content = message.get("content") if isinstance(message, dict) else None + if not isinstance(content, list): + continue + for block in content: + if ( + isinstance(block, dict) + and block.get("type") == "tool_use" + and block.get("name") == CCR_TOOL_NAME + ): + use_id = block.get("id") + if use_id: + retrieve_ids.add(str(use_id)) + + if not retrieve_ids: + return messages, 0 + + out: list[Any] = [] + neutralized = 0 + changed = False + for message in messages: + content = message.get("content") if isinstance(message, dict) else None + if not isinstance(content, list): + out.append(message) + continue + + new_content: list[Any] = [] + touched = False + for block in content: + if isinstance(block, dict): + if block.get("type") == "tool_use" and block.get("name") == CCR_TOOL_NAME: + new_content.append( + { + "type": "text", + "text": "[headroom_retrieve call omitted: tool not available this turn]", + } + ) + neutralized += 1 + touched = True + continue + if ( + block.get("type") == "tool_result" + and str(block.get("tool_use_id", "")) in retrieve_ids + ): + new_content.append({"type": "text", "text": _ccr_result_as_text(block)}) + neutralized += 1 + touched = True + continue + new_content.append(block) + + if touched: + changed = True + repaired = dict(message) + repaired["content"] = new_content + out.append(repaired) + else: + out.append(message) + + return (out, neutralized) if changed else (messages, 0) + + # --------------------------------------------------------------------------- # Server-side Tool Search injection — OpenAI Responses API (gpt-5.4+). # diff --git a/tests/test_ccr_retrieve_history_repair.py b/tests/test_ccr_retrieve_history_repair.py new file mode 100644 index 000000000..bbd840d95 --- /dev/null +++ b/tests/test_ccr_retrieve_history_repair.py @@ -0,0 +1,119 @@ +"""Regression tests for the CCR headroom_retrieve history repair (#2814). + +A passthrough side-request (prompt-type Stop hook evaluator, /compact) replays a +transcript whose history contains a ``headroom_retrieve`` tool_use, but the +request's own ``tools`` array does not declare that tool. Anthropic 400s on the +dangling reference. The repair neutralizes those history blocks so the 400 is +structurally impossible, without breaking user/assistant alternation. +""" + +from __future__ import annotations + +from headroom.ccr.tool_injection import CCR_TOOL_NAME +from headroom.proxy.helpers import strip_unsupported_ccr_retrieve_blocks + + +def _transcript_with_retrieve() -> list[dict]: + return [ + {"role": "user", "content": [{"type": "text", "text": "read config"}]}, + { + "role": "assistant", + "content": [ + {"type": "text", "text": "Let me expand that."}, + { + "type": "tool_use", + "id": "toolu_ccr_1", + "name": CCR_TOOL_NAME, + "input": {"hash": "abc123def456abc123def456"}, + }, + ], + }, + { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "toolu_ccr_1", + "content": "the full expanded config text", + } + ], + }, + {"role": "assistant", "content": [{"type": "text", "text": "Done."}]}, + ] + + +def test_noop_when_tool_is_declared() -> None: + messages = _transcript_with_retrieve() + tools = [{"name": CCR_TOOL_NAME}] + out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools) + assert n == 0 + assert out is messages # identity: caller skips the write-back + + +def test_noop_when_no_retrieve_history() -> None: + messages = [ + {"role": "user", "content": [{"type": "text", "text": "hi"}]}, + {"role": "assistant", "content": [{"type": "text", "text": "hello"}]}, + ] + out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[]) + assert n == 0 + assert out is messages + + +def test_neutralizes_retrieve_reference_when_tool_absent() -> None: + messages = _transcript_with_retrieve() + out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[{"name": "Bash"}]) + + assert n == 2 # tool_use + its paired tool_result + # No dangling references survive anywhere. + for message in out: + for block in message["content"]: + assert block.get("type") != "tool_use" or block.get("name") != CCR_TOOL_NAME + assert not ( + block.get("type") == "tool_result" and block.get("tool_use_id") == "toolu_ccr_1" + ) + + # Message count and roles are unchanged (alternation intact). + assert [m["role"] for m in out] == [m["role"] for m in messages] + # The assistant's own text survives; the tool_use became a text block. + assistant = out[1]["content"] + assert assistant[0] == {"type": "text", "text": "Let me expand that."} + assert assistant[1]["type"] == "text" + # The retrieved result text is preserved, not dropped. + assert out[2]["content"][0] == { + "type": "text", + "text": "the full expanded config text", + } + + +def test_leaves_foreign_tool_use_untouched() -> None: + messages = [ + { + "role": "assistant", + "content": [ + {"type": "tool_use", "id": "toolu_bash_1", "name": "Bash", "input": {"cmd": "ls"}} + ], + }, + { + "role": "user", + "content": [ + {"type": "tool_result", "tool_use_id": "toolu_bash_1", "content": "file.txt"} + ], + }, + ] + out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[{"name": "Bash"}]) + assert n == 0 + assert out is messages + + +def test_result_falls_back_to_placeholder_without_text() -> None: + messages = [ + { + "role": "assistant", + "content": [{"type": "tool_use", "id": "t1", "name": CCR_TOOL_NAME, "input": {}}], + }, + {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "t1", "content": []}]}, + ] + out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[]) + assert n == 2 + assert out[1]["content"][0] == {"type": "text", "text": "[headroom_retrieve result omitted]"} diff --git a/tests/test_proxy/test_tool_search_repair_after_turn_hooks.py b/tests/test_proxy/test_tool_search_repair_after_turn_hooks.py index 30d495760..9d635fc1b 100644 --- a/tests/test_proxy/test_tool_search_repair_after_turn_hooks.py +++ b/tests/test_proxy/test_tool_search_repair_after_turn_hooks.py @@ -21,12 +21,18 @@ pytest.importorskip("fastapi") from fastapi.testclient import TestClient +from headroom.ccr.tool_injection import CCR_TOOL_NAME from headroom.proxy.server import ProxyConfig, create_app from headroom.proxy.turn_hooks import clear_turn_hooks, register_turn_hook _SEARCH_TOOL = {"type": "tool_search_tool_20250917", "name": "tool_search"} _GREP = {"name": "Grep", "description": "search files", "input_schema": {"type": "object"}} _READ = {"name": "Read", "description": "read a file", "input_schema": {"type": "object"}} +_CCR_TOOL = { + "name": CCR_TOOL_NAME, + "description": "retrieve compressed content", + "input_schema": {"type": "object"}, +} # A transcript that already carries a resolved tool-search round trip for `Grep`. _POISONED_MESSAGES = [ @@ -53,6 +59,31 @@ _POISONED_MESSAGES = [ {"role": "user", "content": "now use it"}, ] +_CCR_POISONED_MESSAGES = [ + {"role": "user", "content": "expand the saved result"}, + { + "role": "assistant", + "content": [ + { + "type": "tool_use", + "id": "toolu_ccr_1", + "name": CCR_TOOL_NAME, + "input": {"hash": "abc123def456abc123def456"}, + } + ], + }, + { + "role": "user", + "content": [ + { + "type": "tool_result", + "tool_use_id": "toolu_ccr_1", + "content": "the expanded result", + } + ], + }, +] + @pytest.fixture(autouse=True) def _clean_registry(): @@ -77,7 +108,12 @@ class _InertHook: return None -def _run(hook) -> dict: # noqa: ANN001 +def _run( + hook, # noqa: ANN001 + *, + messages: list[dict] = _POISONED_MESSAGES, + tools: list[dict] | None = None, +) -> dict: """POST a poisoned transcript through the handler, return the forwarded body.""" captured: dict[str, object] = {} register_turn_hook(hook) @@ -123,8 +159,8 @@ def _run(hook) -> dict: # noqa: ANN001 json={ "model": "claude-sonnet-4-6", "max_tokens": 64, - "messages": _POISONED_MESSAGES, - "tools": [_SEARCH_TOOL, _GREP, _READ], + "messages": messages, + "tools": tools or [_SEARCH_TOOL, _GREP, _READ], }, ) assert response.status_code == 200 @@ -174,3 +210,30 @@ def test_repair_leaves_resolvable_history_alone_when_the_hook_keeps_the_tool() - assert _referenced_tool_names(forwarded) == ["Grep"] assert "tool_search_tool_result" in _block_types(forwarded) + + +def test_ccr_repair_sees_the_tools_array_the_hook_left_behind() -> None: + """CCR repair also runs after a hook removes headroom_retrieve.""" + forwarded = _run( + _DropToolHook(CCR_TOOL_NAME), + messages=_CCR_POISONED_MESSAGES, + tools=[_CCR_TOOL, _READ], + ) + + assert CCR_TOOL_NAME not in [t.get("name") for t in forwarded["tools"]] + assert "tool_use" not in _block_types(forwarded) + assert "tool_result" not in _block_types(forwarded) + assert forwarded["messages"][2]["content"] == [{"type": "text", "text": "the expanded result"}] + + +def test_ccr_repair_leaves_resolvable_history_when_hook_keeps_tool() -> None: + """CCR history remains structured when the final tools array declares it.""" + forwarded = _run( + _InertHook(), + messages=_CCR_POISONED_MESSAGES, + tools=[_CCR_TOOL, _READ], + ) + + assert CCR_TOOL_NAME in [t.get("name") for t in forwarded["tools"]] + assert "tool_use" in _block_types(forwarded) + assert "tool_result" in _block_types(forwarded)