fix(proxy): keep core tools and the client's ToolSearch resident for PascalCase clients (#2647)

## Description

`_TOOL_SEARCH_CORE_TOOLS` is spelled in lowercase, but the membership
test compared
the raw tool name, so the core-tool exemption never fired for clients
that send
PascalCase names. For Claude Code (`Bash`, `Read`, `Edit`, `ToolSearch`)
**every**
tool in the request body was deferred.

The damaging part is that Claude Code's own `ToolSearch` was deferred.
It is the
schema fetcher for tools the client keeps in its local registry and
never sends in
the body — `TaskCreate`, `TaskUpdate`, `TaskList`, `WebFetch`,
`EnterPlanMode`,
`Monitor`, `LSP`, `Cron*`, `SendMessage`. Hiding it makes all of them
permanently
uncallable: advertised to the model in a `<system-reminder>`, but no
search can
return their schemas, because the injected `tool_search_tool_regex` only
indexes
what is in the request body.

Closes #2646

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Compare tool names against the core set case-insensitively in
`inject_tool_search_deferral` (`helpers.py`).
- Add `"toolsearch"` to `_TOOL_SEARCH_CORE_TOOLS` so a client's own
schema-fetch tool is never deferred.
- Apply the same case-insensitive comparison to
`inject_tool_search_deferral_openai`, which had the identical
exact-match bug (including against `_OPENAI_TOOL_SEARCH_RESIDENT_NAMES =
{"terminal"}`).
- Add 3 tests on the Anthropic path and 1 on the OpenAI path.

Both source changes are required: case-folding alone does not help
`ToolSearch`
(it was not in the set), and adding it alone does not help
`Bash`/`Read`/`Edit`.

**The token saving is unchanged** — MCP tools are still deferred. This
is not a
request to disable the feature.

Beyond the stranded tools, the old behaviour also meant (a) routine
`Bash`/`Read`/`Edit` loops each paid a search round-trip, the exact cost
the core
set exists to avoid, and (b) zero resident *real* tools remained,
silently
violating the invariant documented on `inject_tool_search_deferral` —
the injected
search tool is typed and does not satisfy it — which risks an upstream
400. The
existing assertion for that invariant passes today only because its
fixture uses
lowercase names.

## Testing

- [x] Unit tests pass (`pytest`) — the two affected files; see scope
note below
- [ ] Linting passes (`ruff check .`) — see note
- [ ] Type checking passes (`mypy headroom`) — could not run, see note
- [x] New tests added for new functionality
- [x] Manual testing performed

`ruff check .` reports 4 findings repo-wide, **all pre-existing and
unrelated**
(`plugins/headroom-oauth2/`), confirmed identical on unmodified `main`.
Zero
findings in the three files this PR touches, and `ruff format --check`
is clean on
all three. Left unchecked because the repo-wide command does not exit 0.

`mypy headroom` could not run in my environment (numpy stubs error out
under the
resolved Python version before checking begins). Not attempted further —
CI should
be the authority.

### Test Output

```text
$ python -m pytest tests/test_issue_746_tool_search.py tests/test_openai_tool_search_deferral.py -q
65 passed, 1 warning in 0.70s

# Baseline on those two files before this PR: 62 (36 + 26).
# The 3 new Anthropic tests + 1 new OpenAI test bring it to 65.

# Red before the source change (tests written first):
tests/test_issue_746_tool_search.py::test_core_tools_match_case_insensitively FAILED
    AssertionError: Bash
    assert True is None
    where {'name': 'Bash', ..., 'defer_loading': True}.get('defer_loading')
tests/test_issue_746_tool_search.py::test_client_tool_search_tool_is_never_deferred FAILED
    AssertionError: assert True is None
    where {'name': 'ToolSearch', ..., 'defer_loading': True}.get('defer_loading')
tests/test_issue_746_tool_search.py::test_resident_real_tool_survives_pascal_case_surface FAILED
    assert any(not t.get("type") and not t.get("defer_loading") for t in out)
    assert False
3 failed, 36 deselected

$ python -m ruff check headroom/proxy/helpers.py tests/test_issue_746_tool_search.py tests/test_openai_tool_search_deferral.py
All checks passed!

$ python -m ruff format --check <same three files>
3 files already formatted
```

## Real Behavior Proof

- Environment: headroom 0.32.1 installed / 0.32.0 source, Python 3.13,
macOS 15 (Darwin 25.5.0), Claude Code 2.1.220 with
`ENABLE_TOOL_SEARCH=true` and
`ANTHROPIC_BASE_URL=http://localhost:8787`, first-party Anthropic
upstream, `HEADROOM_TOOL_SEARCH` truthy
- Exact command / steps: build a Claude Code tool surface and pass it
through the injector — `names =
["Bash","Read","Write","Edit","Glob","Grep","ToolSearch"] +
[f"mcp__srv__t{i}" for i in range(12)]`, `tools = [{"name": n,
"description": n, "input_schema": {}} for n in names]`, then
`inject_tool_search_deferral(tools)` and print which entries carry
`defer_loading`
- Observed result: before the fix `resident real tools: []` with
`ToolSearch deferred: True` (every built-in deferred). After the fix
`resident real tools:
['Bash','Edit','Glob','Grep','Read','ToolSearch','Write']` with all 12
`mcp__srv__t*` still deferred, so the saving is retained. This matches a
live session: the proxy logged
`router:tool_search_deferral:25tools:22182tok ... client=claude-code`
and `tool_search_tool_regex` could resolve only `mcp__*` tools —
`TaskCreate`/`WebFetch`/`EnterPlanMode` returned no match until
`ToolSearch` was recovered by regex-searching for it and then calling
`select:TaskCreate,...`
- Not tested: the full pytest suite (164 modules fail collection with
`ModuleNotFoundError: No module named 'headroom._core'` because my
environment imports the package via `PYTHONPATH` without building the
Rust extension; identical failure confirmed on unmodified `main`, so it
is environmental). `mypy headroom` not runnable here. No end-to-end run
against a live upstream through a rebuilt proxy — verification is at the
function boundary plus the live-session log evidence above. The OpenAI
Responses path is covered by unit test only, not exercised against a
real gpt-5.4+ deployment.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- **Documentation**: N/A — no user-facing surface changes; behaviour
returns to what the existing comments and docstring already describe.
- **"New and existing unit tests pass locally"**: left unchecked
deliberately. The tests covering the changed symbols pass (65), but I
cannot run the whole suite locally without the compiled
`headroom._core`. Not claiming more than I verified.
- **Scope**: the OpenAI-path fix rides along because it is the identical
three-line comparison bug in the sibling function. Happy to split it
into its own PR if you would rather keep this Anthropic-only.
- **Deliberately not done**: I did not add a `client != "claude-code"`
gate at `handlers/anthropic.py`, even though the feature's own comment
block scopes it to non-Claude-Code clients and `client=claude-code` is
already known there (it appears in the `transforms=` log line). Gating
there would forfeit the ~22k tokens/request currently saved on Claude
Code's eagerly-shipped MCP schemas; keeping the meta-tool resident
preserves both the saving and reachability. Flagging in case you would
prefer to gate as well.
- **Adjacent blind spot, out of scope**:
`claude_code_tool_search_inactive` already checks both the tools array
*and* the `anthropic-beta` header, but the injector's early-return guard
checks only the array. That is why a plain-function `ToolSearch` slips
past it and the injection runs on a client that is already deferring.

Co-authored-by: Fabien Culpo <fabien.culpo@dawex.com>
This commit is contained in:
Fabien Culpo 2026-07-29 18:06:51 +02:00 committed by GitHub
parent 1588f5e041
commit 1d29738818
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 78 additions and 4 deletions

View file

@ -2857,6 +2857,11 @@ _TOOL_SEARCH_CORE_TOOLS = frozenset(
"webfetch",
"question",
"skill",
# A client's own tool-search/schema-fetch tool (Claude Code's ``ToolSearch``).
# It resolves tools the client keeps in its local registry and never puts in
# the request body (TaskCreate, WebFetch, …), so deferring it hides the only
# tool that can load them and they become permanently unreachable.
"toolsearch",
}
)
_TOOL_SEARCH_DEFAULT_TYPE = "tool_search_tool_regex_20251119"
@ -2900,8 +2905,18 @@ def inject_tool_search_deferral(
last_resident_real: dict[str, Any] | None = None
resident_has_cache_control = False
# Clients disagree on casing for the same tool: Claude Code sends ``Bash`` /
# ``ToolSearch`` where opencode sends ``bash``. Compare case-insensitively so
# the exemption applies to both — an exact match silently deferred *every*
# tool for PascalCase clients, including their own tool-search tool.
core_lower = {name.lower() for name in core_tools}
for tool in tools:
if not isinstance(tool, dict) or tool.get("type") or tool.get("name") in core_tools:
if (
not isinstance(tool, dict)
or tool.get("type")
or str(tool.get("name") or "").lower() in core_lower
):
# Non-dict, server/typed tools (web_search, computer, …), and core
# tools stay resident and unchanged.
out.append(tool)
@ -3011,6 +3026,11 @@ def inject_tool_search_deferral_openai(
out: list[Any] = [{"type": _OPENAI_TOOL_SEARCH_TYPE}]
deferred = 0
# Case-insensitive for the same reason as the Anthropic path above: the
# resident-name sets are lowercase, clients are not required to be.
resident_lower = {name.lower() for name in core_tools} | {
name.lower() for name in _OPENAI_TOOL_SEARCH_RESIDENT_NAMES
}
for tool in tools:
if not isinstance(tool, dict):
out.append(tool)
@ -3020,9 +3040,7 @@ def inject_tool_search_deferral_openai(
# trained to search namespaces / MCP servers). Everything else — core
# coding tools and other hosted tools — stays resident.
deferrable = (
ttype == "function"
and tool.get("name") not in core_tools
and tool.get("name") not in _OPENAI_TOOL_SEARCH_RESIDENT_NAMES
ttype == "function" and str(tool.get("name") or "").lower() not in resident_lower
) or ttype == "mcp"
if deferrable and not tool.get("defer_loading"):
new_tool = dict(tool)

View file

@ -257,3 +257,45 @@ def test_non_dict_and_typed_tools_stay_resident() -> None:
out = inject_tool_search_deferral(tools)
typed = [t for t in out if t.get("type") == "web_search_20250305"]
assert len(typed) == 1 and typed[0].get("defer_loading") is None
# ---------------------------------------------------------------------------
# PascalCase clients (Claude Code). The core-tool exemption is spelled in
# lowercase, so an exact-match comparison never fired for Claude Code: every
# tool was deferred, including Claude Code's own ``ToolSearch``.
# ---------------------------------------------------------------------------
def _claude_code_tools() -> list[dict]:
"""Claude Code's surface: PascalCase built-ins, its ToolSearch, MCP tools."""
names = ["Bash", "Read", "Write", "Edit", "Glob", "Grep", "ToolSearch"] + [
f"mcp__srv__t{i}" for i in range(12)
]
return [{"name": n, "description": n, "input_schema": {}} for n in names]
def test_core_tools_match_case_insensitively() -> None:
# Without a case-insensitive match, routine edit/read/run loops each pay a
# search round-trip — the exact thing _TOOL_SEARCH_CORE_TOOLS exists to avoid.
out = inject_tool_search_deferral(_claude_code_tools())
by_name = {t.get("name"): t for t in out if "name" in t}
for name in ("Bash", "Read", "Write", "Edit", "Glob", "Grep"):
assert by_name[name].get("defer_loading") is None, name
# MCP tools are still deferred — the token saving is preserved.
assert by_name["mcp__srv__t0"].get("defer_loading") is True
def test_client_tool_search_tool_is_never_deferred() -> None:
# ToolSearch is the client's own schema fetcher for tools that never appear
# in the request body (TaskCreate, WebFetch, …). Deferring it hides the only
# tool that can load them, so they become permanently unreachable.
out = inject_tool_search_deferral(_claude_code_tools())
by_name = {t.get("name"): t for t in out if "name" in t}
assert by_name["ToolSearch"].get("defer_loading") is None
def test_resident_real_tool_survives_pascal_case_surface() -> None:
# The injected search tool is typed and does not satisfy the invariant on its
# own; Anthropic 400s when every real tool is deferred.
out = inject_tool_search_deferral(_claude_code_tools())
assert any(not t.get("type") and not t.get("defer_loading") for t in out)

View file

@ -143,3 +143,17 @@ def test_noop_when_nothing_deferrable():
def test_noop_for_non_list():
assert inject_tool_search_deferral_openai(None, "gpt-5.5") is None
def test_resident_names_match_case_insensitively():
# The resident-name sets are lowercase; clients are not required to be. An
# exact match deferred every tool for a PascalCase client, including its own
# tool-search tool. Mirrors the Anthropic-side fix.
tools = [_fn(n) for n in ("Bash", "Read", "Edit", "Terminal", "ToolSearch")] + [
_fn(f"slack_{i}") for i in range(10)
]
out = inject_tool_search_deferral_openai(tools, "gpt-5.5")
by_name = {t.get("name"): t for t in out if "name" in t}
for name in ("Bash", "Read", "Edit", "Terminal", "ToolSearch"):
assert by_name[name].get("defer_loading") is None, name
assert by_name["slack_0"].get("defer_loading") is True