mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
## Description Three compression consumers compare tool names against the bare literal `headroom_retrieve`, so the qualified forms MCP clients actually send (`mcp__Headroom__headroom_retrieve`, `mcp_Headroom_headroom_retrieve`) slip past the guard and get recompressed. `SmartCrusher.apply` has the bare comparison at both its OpenAI `role=tool` site and its Anthropic `tool_result` block site; the LangGraph compressor and the Strands hook have no tool-name check at all. Recompressing already-retrieved CCR content mints a new `<<ccr:hash>>` marker the agent cannot redeem. `headroom.config.is_tool_excluded` already owns alias resolution, including the MCP wrapper forms. This routes all three consumers through it instead of adding a second name matcher. Closes #2656. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `SmartCrusher.apply` routes both its `role=tool` and its Anthropic `tool_result` guards through `is_tool_excluded` - `_should_skip` in the LangGraph compressor takes the tool name and skips excluded tools; tool-call names are indexed by id so a `ToolMessage` without a copied `name` is still classifiable - `_should_skip_compression` in the Strands hook takes the tool name and skips excluded tools, recording `tool_excluded` - regressions for the qualified and bare names across all three consumers, the Anthropic block shape, the MCP wrapper entry point, and a near-match name that must still compress - a LangGraph regression for incomplete tool-call metadata that continues to a later qualified call ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output `pytest tests/test_smart_crusher.py tests/integrations/test_langgraph.py tests/integrations/test_strands tests/test_transforms/test_smart_crusher_ccr_retrieve_exemption.py -q` ```text tests\test_smart_crusher.py ............ [ 10%] tests\integrations\test_langgraph.py ..... [ 15%] tests\integrations\test_strands\test_ccr_exclusion.py ..... [ 19%] tests\integrations\test_strands\test_hooks.py sssssssss [ 27%] tests\integrations\test_strands\test_hooks_unit.py ssssssssssssssssssssssssssssssssss [ 57%] tests\integrations\test_strands\test_model.py ssssssssssssssss [ 71%] tests\integrations\test_strands\test_model_unit.py sssssssssssssssssssssssssss [ 95%] tests\test_transforms\test_smart_crusher_ccr_retrieve_exemption.py ..... [100%] 28 passed, 86 skipped in the focused invariant suite ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.13, `headroom._core` built - Exact command / steps: `uv run pytest tests/test_smart_crusher.py tests/integrations/test_langgraph.py tests/integrations/test_strands -q`, and the same suite run against the pre-change implementation with the new tests in place - Observed result: before the change, five regressions fail. `SmartCrusher` returns non-byte-identical content for a `mcp__Headroom__headroom_retrieve` result, the LangGraph compressor replaces the message content, and the Strands hook returns `"compressed"` in place of the tool output. After the change all three preserve the content byte-for-byte, incomplete LangGraph tool-call metadata is ignored while the later qualified call remains indexed, the Strands hook records `tool_excluded` and never calls the crusher, and `HeadroomMCPCompressor.compress` returns the payload unchanged. `mcp__Headroom__headroom_retrieve_extra` still compresses in all three, and the Kompress and ContentRouter suites are unchanged. - Not tested: the optional Strands package, so the additions to `tests/integrations/test_strands/test_hooks_unit.py` skip locally ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes One deliberate divergence from the issue: the suggested snippet passes `DEFAULT_VERBATIM_EXCLUDE_TOOLS` to `is_tool_excluded`, but that constant holds only `WebSearch`, `WebFetch`, `web_search`, `web_fetch`. Applied literally it would drop `headroom_retrieve` from the comparison entirely and delete the #1077 guard these two SmartCrusher sites exist to enforce. This passes `(CCR_TOOL_NAME,)` so each guard keeps doing the one thing it documents. If you'd rather these paths also honor the verbatim-exclude set, the tuple can become `(CCR_TOOL_NAME, *DEFAULT_VERBATIM_EXCLUDE_TOOLS)` — the CCR name has to stay in it either way. Adjacent work: PR #2654 covers `ContentRouter` only.
141 lines
4.6 KiB
Python
141 lines
4.6 KiB
Python
"""Regression tests for qualified CCR retrieval tool names."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
|
|
import pytest
|
|
|
|
from headroom import OpenAIProvider, Tokenizer
|
|
from headroom.ccr.tool_injection import CCR_TOOL_NAME
|
|
from headroom.config import SmartCrusherConfig
|
|
|
|
try:
|
|
from headroom._core import SmartCrusher as _RustSmartCrusher # noqa: F401
|
|
except ImportError:
|
|
pytest.skip("headroom._core not built", allow_module_level=True)
|
|
|
|
from headroom.transforms.smart_crusher import SmartCrusher
|
|
|
|
|
|
def _big_content() -> str:
|
|
return json.dumps([{"id": i, "value": "x" * 20} for i in range(60)])
|
|
|
|
|
|
def _apply_for_tool(tool_name: str):
|
|
messages = [
|
|
{
|
|
"role": "assistant",
|
|
"tool_calls": [
|
|
{
|
|
"id": "call_1",
|
|
"function": {"name": tool_name, "arguments": "{}"},
|
|
}
|
|
],
|
|
},
|
|
{"role": "tool", "tool_call_id": "call_1", "content": _big_content()},
|
|
]
|
|
tokenizer = Tokenizer(OpenAIProvider().get_token_counter("gpt-4o"), "gpt-4o")
|
|
result = SmartCrusher(config=SmartCrusherConfig(min_tokens_to_crush=0)).apply(
|
|
messages, tokenizer
|
|
)
|
|
return messages[1]["content"], result
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"tool_name",
|
|
["mcp__Headroom__headroom_retrieve", "mcp_Headroom_headroom_retrieve"],
|
|
)
|
|
def test_qualified_ccr_retrieval_result_is_preserved(tool_name: str) -> None:
|
|
original, result = _apply_for_tool(tool_name)
|
|
|
|
assert result.messages[1]["content"] == original
|
|
assert not any("smart_crush" in transform for transform in result.transforms_applied)
|
|
|
|
|
|
def test_near_match_ccr_tool_name_still_compresses() -> None:
|
|
original, result = _apply_for_tool("mcp__Headroom__headroom_retrieve_extra")
|
|
|
|
assert result.messages[1]["content"] != original or result.tokens_after < result.tokens_before
|
|
|
|
|
|
def test_bare_ccr_tool_name_remains_preserved() -> None:
|
|
original, result = _apply_for_tool(CCR_TOOL_NAME)
|
|
|
|
assert result.messages[1]["content"] == original
|
|
|
|
|
|
def _apply_anthropic_for_tool(tool_name: str):
|
|
"""Anthropic block shape: tool_use in the assistant turn, tool_result in the user turn."""
|
|
messages = [
|
|
{
|
|
"role": "assistant",
|
|
"content": [
|
|
{"type": "tool_use", "id": "tu_1", "name": tool_name, "input": {}},
|
|
],
|
|
},
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{"type": "tool_result", "tool_use_id": "tu_1", "content": _big_content()},
|
|
],
|
|
},
|
|
]
|
|
tokenizer = Tokenizer(OpenAIProvider().get_token_counter("gpt-4o"), "gpt-4o")
|
|
result = SmartCrusher(config=SmartCrusherConfig(min_tokens_to_crush=0)).apply(
|
|
messages, tokenizer
|
|
)
|
|
return messages[1]["content"][0]["content"], result
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"tool_name",
|
|
[
|
|
"mcp__Headroom__headroom_retrieve",
|
|
"mcp_Headroom_headroom_retrieve",
|
|
CCR_TOOL_NAME,
|
|
],
|
|
)
|
|
def test_qualified_ccr_tool_result_block_is_preserved(tool_name: str) -> None:
|
|
original, result = _apply_anthropic_for_tool(tool_name)
|
|
|
|
assert result.messages[1]["content"][0]["content"] == original
|
|
assert not any("smart" in transform for transform in result.transforms_applied)
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"tool_name",
|
|
["mcp__Headroom__headroom_retrieve", "mcp_Headroom_headroom_retrieve", CCR_TOOL_NAME],
|
|
)
|
|
def test_mcp_compressor_preserves_qualified_ccr_output(tool_name: str) -> None:
|
|
"""`HeadroomMCPCompressor.compress` is the production entry point issue #2656 names.
|
|
|
|
It drives `SmartCrusher.apply` with a `role=tool` message, so the guard has to
|
|
hold through that wrapper and not only on a directly built message list.
|
|
"""
|
|
from headroom.integrations.mcp.server import HeadroomMCPCompressor
|
|
|
|
content = json.dumps({"results": [{"id": i, "value": "x" * 40} for i in range(80)]})
|
|
result = HeadroomMCPCompressor().compress(content, tool_name=tool_name)
|
|
|
|
assert result.compressed_content == content
|
|
|
|
|
|
def test_mcp_compressor_still_compresses_a_near_match_name() -> None:
|
|
from headroom.integrations.mcp.server import HeadroomMCPCompressor
|
|
|
|
content = json.dumps({"results": [{"id": i, "value": "x" * 40} for i in range(80)]})
|
|
result = HeadroomMCPCompressor().compress(
|
|
content, tool_name="mcp__Headroom__headroom_retrieve_extra"
|
|
)
|
|
|
|
assert result.compressed_content != content
|
|
|
|
|
|
def test_near_match_ccr_tool_result_block_still_compresses() -> None:
|
|
original, result = _apply_anthropic_for_tool("mcp__Headroom__headroom_retrieve_extra")
|
|
|
|
assert (
|
|
result.messages[1]["content"][0]["content"] != original
|
|
or result.tokens_after < result.tokens_before
|
|
)
|