headroom/tests/test_ccr_retrieve_history_repair.py
Abhay Singh 7de35739c6
fix(proxy/anthropic): repair headroom_retrieve history references the tools array cannot support (#2876)
## Description

#2805 / #2807 established the mechanism: Claude Code replays one
transcript across requests that carry different `tools` arrays, and
Anthropic validates every history reference against the array of the
request at hand. #2807 fixed it for tool-search blocks by repairing
history (`strip_unsupported_tool_search_blocks`) rather than trying to
predict the client's tool set.

The same mechanism applies to CCR's `headroom_retrieve`, and it is
tool-agnostic. A passthrough side-request (the prompt-type Stop hook
evaluator, `/compact`) that the proxy forwards without declaring
`headroom_retrieve` still carries a historical `tool_use` naming it, and
Anthropic 400s on the dangling reference. The injection-side fixes
(#2766 / #2533) decide *when to re-declare the tool*; this makes the 400
*structurally impossible* where the tool is intentionally absent. It is
belt-and-braces with them, not a replacement.

The fix adds the symmetric repair next to #2807's. When the outbound
`tools` array does not declare `headroom_retrieve`, it replaces each
`headroom_retrieve` `tool_use` and its paired `tool_result` with a text
block, so no dangling reference survives.

It **neutralizes** (replaces in place) rather than **drops**, which is
the one deliberate difference from #2807: CCR's `tool_use` lives in an
assistant turn and its `tool_result` in the next user turn, i.e. two
different messages. Dropping a whole message could leave two same-role
messages adjacent and break Anthropic's strict user/assistant
alternation, turning one 400 into another. Replacing blocks in place
keeps every message and role intact, and preserves the retrieved text
the model already saw. #2807's server-tool blocks both live in the same
assistant turn, so dropping was safe there.

It runs after CCR tool injection, so on the main loop -- where the tool
IS injected (a present marker) -- it neutralizes nothing and the
prompt-cache prefix is untouched, mirroring #2807's placement and
sequencing.

Fixes #2814

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/helpers.py`: added
`strip_unsupported_ccr_retrieve_blocks(messages, tools)` (and a small
`_ccr_result_as_text` helper). No-ops (returning the original object by
identity) when `headroom_retrieve` is declared or no such history
exists; otherwise neutralizes the `tool_use` and its paired
`tool_result` to text.
- `headroom/proxy/handlers/anthropic.py`: call the repair right after
the tool-search history repair (which is after CCR tool injection),
guarded on it actually changing anything, tagged
`router:ccr_retrieve_repair:Nblocks`.
- `tests/test_ccr_retrieve_history_repair.py`: 5 unit tests (no-op when
declared, no-op without retrieve history, neutralize + preserve result
text + keep alternation, leave foreign tool_use untouched, placeholder
when the result has no text).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
tests/test_ccr_retrieve_history_repair.py  5 passed

# Broader CCR / tool-search / handler suites (unchanged behavior):
tests/test_ccr_retrieve_history_repair.py tests/test_proxy/test_anthropic_ccr_deferred_injection.py
tests/test_issue_746_tool_search.py                                      71 passed

# uvx ruff@0.15.17 check  -> All checks passed!
# uvx mypy@1.20.2 headroom/proxy/helpers.py -> Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.17 and mypy 1.20.2 via uvx.
- Exact command / steps: confirmed the injection point
(`apply_session_sticky_ccr_tool`) and the tool-search repair placement
in `handlers/anthropic.py`, confirmed `body["tools"]` reflects the CCR
injection before the repair call site (`body["tools"] = tools` is
written well upstream and the adjacent tool-search repair already relies
on it), then drove the helper over a transcript with a
`headroom_retrieve` tool_use + paired tool_result: with the tool
declared it returns the original object unchanged; with the tool absent
it neutralizes both blocks, preserves the result text, and keeps the
message roles/count identical.
- Observed result: a forwarded request that would 400 with "Tool
reference 'headroom_retrieve' not found in available tools" now carries
text blocks in place of the retrieve `tool_use`/`tool_result`, so there
is no reference for Anthropic to reject, and user/assistant alternation
is preserved. The main loop (tool present) is a no-op.
- Not tested: a live multi-turn Claude Code session hitting a
Stop-hook/`/compact` side-request against a real provider (no live
provider here). The repair is a pure function verified directly over the
exact block shapes Anthropic validates, and it mirrors the
already-merged tool-search repair's mechanism and wiring.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

The issue reporter noted their own logs show the tool-search variant of
this 400 (61 across 11 days) but zero `headroom_retrieve` occurrences,
because they run `HEADROOM_LOSSLESS=1` which disables CCR entirely. This
PR fixes the CCR variant of the same, proven, tool-agnostic mechanism
rather than a fresh CCR repro. The neutralize-vs-drop choice is the one
place I departed from #2807, for the alternation reason above; if you
would rather it drop (accepting the alternation handling that implies),
I am happy to switch it.

---------

Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
2026-08-13 15:05:07 -05:00

119 lines
4.3 KiB
Python

"""Regression tests for the CCR headroom_retrieve history repair (#2814).
A passthrough side-request (prompt-type Stop hook evaluator, /compact) replays a
transcript whose history contains a ``headroom_retrieve`` tool_use, but the
request's own ``tools`` array does not declare that tool. Anthropic 400s on the
dangling reference. The repair neutralizes those history blocks so the 400 is
structurally impossible, without breaking user/assistant alternation.
"""
from __future__ import annotations
from headroom.ccr.tool_injection import CCR_TOOL_NAME
from headroom.proxy.helpers import strip_unsupported_ccr_retrieve_blocks
def _transcript_with_retrieve() -> list[dict]:
return [
{"role": "user", "content": [{"type": "text", "text": "read config"}]},
{
"role": "assistant",
"content": [
{"type": "text", "text": "Let me expand that."},
{
"type": "tool_use",
"id": "toolu_ccr_1",
"name": CCR_TOOL_NAME,
"input": {"hash": "abc123def456abc123def456"},
},
],
},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_ccr_1",
"content": "the full expanded config text",
}
],
},
{"role": "assistant", "content": [{"type": "text", "text": "Done."}]},
]
def test_noop_when_tool_is_declared() -> None:
messages = _transcript_with_retrieve()
tools = [{"name": CCR_TOOL_NAME}]
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools)
assert n == 0
assert out is messages # identity: caller skips the write-back
def test_noop_when_no_retrieve_history() -> None:
messages = [
{"role": "user", "content": [{"type": "text", "text": "hi"}]},
{"role": "assistant", "content": [{"type": "text", "text": "hello"}]},
]
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[])
assert n == 0
assert out is messages
def test_neutralizes_retrieve_reference_when_tool_absent() -> None:
messages = _transcript_with_retrieve()
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[{"name": "Bash"}])
assert n == 2 # tool_use + its paired tool_result
# No dangling references survive anywhere.
for message in out:
for block in message["content"]:
assert block.get("type") != "tool_use" or block.get("name") != CCR_TOOL_NAME
assert not (
block.get("type") == "tool_result" and block.get("tool_use_id") == "toolu_ccr_1"
)
# Message count and roles are unchanged (alternation intact).
assert [m["role"] for m in out] == [m["role"] for m in messages]
# The assistant's own text survives; the tool_use became a text block.
assistant = out[1]["content"]
assert assistant[0] == {"type": "text", "text": "Let me expand that."}
assert assistant[1]["type"] == "text"
# The retrieved result text is preserved, not dropped.
assert out[2]["content"][0] == {
"type": "text",
"text": "the full expanded config text",
}
def test_leaves_foreign_tool_use_untouched() -> None:
messages = [
{
"role": "assistant",
"content": [
{"type": "tool_use", "id": "toolu_bash_1", "name": "Bash", "input": {"cmd": "ls"}}
],
},
{
"role": "user",
"content": [
{"type": "tool_result", "tool_use_id": "toolu_bash_1", "content": "file.txt"}
],
},
]
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[{"name": "Bash"}])
assert n == 0
assert out is messages
def test_result_falls_back_to_placeholder_without_text() -> None:
messages = [
{
"role": "assistant",
"content": [{"type": "tool_use", "id": "t1", "name": CCR_TOOL_NAME, "input": {}}],
},
{"role": "user", "content": [{"type": "tool_result", "tool_use_id": "t1", "content": []}]},
]
out, n = strip_unsupported_ccr_retrieve_blocks(messages, tools=[])
assert n == 2
assert out[1]["content"][0] == {"type": "text", "text": "[headroom_retrieve result omitted]"}