mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
## Description #2805 / #2807 established the mechanism: Claude Code replays one transcript across requests that carry different `tools` arrays, and Anthropic validates every history reference against the array of the request at hand. #2807 fixed it for tool-search blocks by repairing history (`strip_unsupported_tool_search_blocks`) rather than trying to predict the client's tool set. The same mechanism applies to CCR's `headroom_retrieve`, and it is tool-agnostic. A passthrough side-request (the prompt-type Stop hook evaluator, `/compact`) that the proxy forwards without declaring `headroom_retrieve` still carries a historical `tool_use` naming it, and Anthropic 400s on the dangling reference. The injection-side fixes (#2766 / #2533) decide *when to re-declare the tool*; this makes the 400 *structurally impossible* where the tool is intentionally absent. It is belt-and-braces with them, not a replacement. The fix adds the symmetric repair next to #2807's. When the outbound `tools` array does not declare `headroom_retrieve`, it replaces each `headroom_retrieve` `tool_use` and its paired `tool_result` with a text block, so no dangling reference survives. It **neutralizes** (replaces in place) rather than **drops**, which is the one deliberate difference from #2807: CCR's `tool_use` lives in an assistant turn and its `tool_result` in the next user turn, i.e. two different messages. Dropping a whole message could leave two same-role messages adjacent and break Anthropic's strict user/assistant alternation, turning one 400 into another. Replacing blocks in place keeps every message and role intact, and preserves the retrieved text the model already saw. #2807's server-tool blocks both live in the same assistant turn, so dropping was safe there. It runs after CCR tool injection, so on the main loop -- where the tool IS injected (a present marker) -- it neutralizes nothing and the prompt-cache prefix is untouched, mirroring #2807's placement and sequencing. Fixes #2814 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/helpers.py`: added `strip_unsupported_ccr_retrieve_blocks(messages, tools)` (and a small `_ccr_result_as_text` helper). No-ops (returning the original object by identity) when `headroom_retrieve` is declared or no such history exists; otherwise neutralizes the `tool_use` and its paired `tool_result` to text. - `headroom/proxy/handlers/anthropic.py`: call the repair right after the tool-search history repair (which is after CCR tool injection), guarded on it actually changing anything, tagged `router:ccr_retrieve_repair:Nblocks`. - `tests/test_ccr_retrieve_history_repair.py`: 5 unit tests (no-op when declared, no-op without retrieve history, neutralize + preserve result text + keep alternation, leave foreign tool_use untouched, placeholder when the result has no text). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text tests/test_ccr_retrieve_history_repair.py 5 passed # Broader CCR / tool-search / handler suites (unchanged behavior): tests/test_ccr_retrieve_history_repair.py tests/test_proxy/test_anthropic_ccr_deferred_injection.py tests/test_issue_746_tool_search.py 71 passed # uvx ruff@0.15.17 check -> All checks passed! # uvx mypy@1.20.2 headroom/proxy/helpers.py -> Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1, ruff 0.15.17 and mypy 1.20.2 via uvx. - Exact command / steps: confirmed the injection point (`apply_session_sticky_ccr_tool`) and the tool-search repair placement in `handlers/anthropic.py`, confirmed `body["tools"]` reflects the CCR injection before the repair call site (`body["tools"] = tools` is written well upstream and the adjacent tool-search repair already relies on it), then drove the helper over a transcript with a `headroom_retrieve` tool_use + paired tool_result: with the tool declared it returns the original object unchanged; with the tool absent it neutralizes both blocks, preserves the result text, and keeps the message roles/count identical. - Observed result: a forwarded request that would 400 with "Tool reference 'headroom_retrieve' not found in available tools" now carries text blocks in place of the retrieve `tool_use`/`tool_result`, so there is no reference for Anthropic to reject, and user/assistant alternation is preserved. The main loop (tool present) is a no-op. - Not tested: a live multi-turn Claude Code session hitting a Stop-hook/`/compact` side-request against a real provider (no live provider here). The repair is a pure function verified directly over the exact block shapes Anthropic validates, and it mirrors the already-merged tool-search repair's mechanism and wiring. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md`: it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes The issue reporter noted their own logs show the tool-search variant of this 400 (61 across 11 days) but zero `headroom_retrieve` occurrences, because they run `HEADROOM_LOSSLESS=1` which disables CCR entirely. This PR fixes the CCR variant of the same, proven, tool-agnostic mechanism rather than a fresh CCR repro. The neutralize-vs-drop choice is the one place I departed from #2807, for the alternation reason above; if you would rather it drop (accepting the alternation handling that implies), I am happy to switch it. --------- Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
119 lines
4.3 KiB
Python
119 lines
4.3 KiB
Python
"""Regression tests for the CCR headroom_retrieve history repair (#2814).
|
|
|
|
A passthrough side-request (prompt-type Stop hook evaluator, /compact) replays a
|
|
transcript whose history contains a ``headroom_retrieve`` tool_use, but the
|
|
request's own ``tools`` array does not declare that tool. Anthropic 400s on the
|
|
dangling reference. The repair neutralizes those history blocks so the 400 is
|
|
structurally impossible, without breaking user/assistant alternation.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from headroom.ccr.tool_injection import CCR_TOOL_NAME
|
|
from headroom.proxy.helpers import strip_unsupported_ccr_retrieve_blocks
|
|
|
|
|
|
def _transcript_with_retrieve() -> list[dict]:
|
|
return [
|
|
{"role": "user", "content": [{"type": "text", "text": "read config"}]},
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{"type": "text", "text": "Let me expand that."},
|
|
{
|
|
"type": "tool_use",
|
|
"id": "toolu_ccr_1",
|
|
"name": CCR_TOOL_NAME,
|
|
"input": {"hash": "abc123def456abc123def456"},
|
|
},
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{
|
|
"type": "tool_result",
|
|
"tool_use_id": "toolu_ccr_1",
|
|
"content": "the full expanded config text",
|
|
}
|
|
],
|
|
},
|
|
{"role": "assistant", "content": [{"type": "text", "text": "Done."}]},
|
|
]
|
|
|
|
|
|
def test_noop_when_tool_is_declared() -> None:
|
|
messages = _transcript_with_retrieve()
|
|
tools = [{"name": CCR_TOOL_NAME}]
|
|
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools)
|
|
assert n == 0
|
|
assert out is messages # identity: caller skips the write-back
|
|
|
|
|
|
def test_noop_when_no_retrieve_history() -> None:
|
|
messages = [
|
|
{"role": "user", "content": [{"type": "text", "text": "hi"}]},
|
|
{"role": "assistant", "content": [{"type": "text", "text": "hello"}]},
|
|
]
|
|
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[])
|
|
assert n == 0
|
|
assert out is messages
|
|
|
|
|
|
def test_neutralizes_retrieve_reference_when_tool_absent() -> None:
|
|
messages = _transcript_with_retrieve()
|
|
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[{"name": "Bash"}])
|
|
|
|
assert n == 2 # tool_use + its paired tool_result
|
|
# No dangling references survive anywhere.
|
|
for message in out:
|
|
for block in message["content"]:
|
|
assert block.get("type") != "tool_use" or block.get("name") != CCR_TOOL_NAME
|
|
assert not (
|
|
block.get("type") == "tool_result" and block.get("tool_use_id") == "toolu_ccr_1"
|
|
)
|
|
|
|
# Message count and roles are unchanged (alternation intact).
|
|
assert [m["role"] for m in out] == [m["role"] for m in messages]
|
|
# The assistant's own text survives; the tool_use became a text block.
|
|
assistant = out[1]["content"]
|
|
assert assistant[0] == {"type": "text", "text": "Let me expand that."}
|
|
assert assistant[1]["type"] == "text"
|
|
# The retrieved result text is preserved, not dropped.
|
|
assert out[2]["content"][0] == {
|
|
"type": "text",
|
|
"text": "the full expanded config text",
|
|
}
|
|
|
|
|
|
def test_leaves_foreign_tool_use_untouched() -> None:
|
|
messages = [
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{"type": "tool_use", "id": "toolu_bash_1", "name": "Bash", "input": {"cmd": "ls"}}
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{"type": "tool_result", "tool_use_id": "toolu_bash_1", "content": "file.txt"}
|
|
],
|
|
},
|
|
]
|
|
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[{"name": "Bash"}])
|
|
assert n == 0
|
|
assert out is messages
|
|
|
|
|
|
def test_result_falls_back_to_placeholder_without_text() -> None:
|
|
messages = [
|
|
{
|
|
"role": "assistant",
|
|
"content": [{"type": "tool_use", "id": "t1", "name": CCR_TOOL_NAME, "input": {}}],
|
|
},
|
|
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "t1", "content": []}]},
|
|
]
|
|
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[])
|
|
assert n == 2
|
|
assert out[1]["content"][0] == {"type": "text", "text": "[headroom_retrieve result omitted]"}
|