1 commit

Author	SHA1	Message	Date
Parideboy	b6f9877c78	fix(tokenizer): coerce non-string tool_call fields before counting (#2801) ... ## Description `/v1/compress` returned HTTP 503 with an unhandled `TypeError` when a message carried a `tool_calls[].function.arguments` value that was not a string. `arguments` is a JSON *string* per the OpenAI spec, but OpenAI-compatible upstreams do emit `None` or a raw object there, and every token counter passed the value straight to `tiktoken.encode()`. Because the malformed message persists in conversation history, the failure was sticky: every later request replaying that history failed too, regardless of destination provider. Reported in #2782. The exact repro in that issue (`arguments: null`) no longer raises — `count_text` grew a falsy guard since 0.33.0 — but the root cause is still live for any *truthy* non-string, which I reproduced against all four counters on `main` before the fix. ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation update - [ ] Refactor / internal cleanup ## Changes Made - `headroom/tokenizers/base.py`: new `coerce_countable_text()`. Strings pass through untouched, `None` counts as nothing, dict/list/tuple are JSON-serialized, anything else falls back to `str()`. The serialized form is capped at 200K chars so a malformed upstream can't turn a token *estimate* into a multi-megabyte encode. - Applied at the tool-call field sites (`function.name`, `function.arguments`, `id`, and the legacy `function_call`) in `tokenizers/base.py`, `tokenizers/tiktoken_counter.py`, `providers/openai.py`, `providers/openai_compatible.py`, `providers/anthropic.py`. - Guarded `{"function": null}` / `{"id": null}`, which reach the same encode path. - New test file `tests/test_tool_call_arguments_not_a_string.py` (11 cases). Serializing dicts rather than the one-liner suggested in the issue (`str(func.get("arguments") or "")`) is deliberate: `str()` on a dict yields Python repr with single quotes, which is not what the upstream would have billed, and it is unbounded. ## Testing - [x] Existing tests pass - [x] New tests added for the fix - [ ] Manual testing performed New tests: ``` $ python -m pytest tests/test_tool_call_arguments_not_a_string.py -q tests\test_tool_call_arguments_not_a_string.py ........... [100%] ============================= 11 passed in 0.40s ============================== ``` Surrounding tokenizer/provider suites, unchanged: ``` $ python -m pytest tests/test_tool_call_arguments_not_a_string.py tests/test_tokenizer.py \ tests/test_tokenizers.py tests/test_tokenizers \ tests/test_provider_counter_content_blocks.py tests/test_provider_tokenizer_one_ruler.py -q tests\test_provider_counter_content_blocks.py ............ [ 91%] tests\test_provider_tokenizer_one_ruler.py ......... [100%] ======================= 91 passed, 14 skipped in 1.50s ======================== ``` Lint/format on the touched files: ``` $ ruff check <touched files> && ruff format --check <touched files> All checks passed! 6 files already formatted ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.11, pytest 9.1.1, repo at `upstream/main` (d0a86d40) with this branch applied; `headroom._core` built locally. - Exact command / steps: ran the same script before and after the change, driving the four counters directly (the crash site the proxy 503 unwinds to): ``` python -c " from headroom.providers.openai import OpenAITokenCounter from headroom.providers.openai_compatible import OpenAICompatibleTokenCounter from headroom.providers.anthropic import AnthropicTokenCounter from headroom.tokenizers.tiktoken_counter import TiktokenCounter def mk(a): return [{'role':'assistant','content':None,'tool_calls':[ {'id':'c1','type':'function','function':{'name':'read_file','arguments':a}}]}] for name,c in [('openai',OpenAITokenCounter('gpt-4o')), ('compat',OpenAICompatibleTokenCounter('gpt-4o')), ('anthropic',AnthropicTokenCounter('claude-sonnet-4')), ('tiktoken',TiktokenCounter('gpt-4o'))]: for a in [None, {'path':'x'}, 5]: try: print(name, repr(a), c.count_messages(mk(a))) except Exception as e: print(name, repr(a), 'ERR', type(e).__name__, e) " ``` - Observed result: before the change, all four counters raised on every truthy non-string; after, all return finite counts and an object `arguments` prices within 5 tokens of its JSON string form. ``` BEFORE openai None 21 openai {'path': 'x'} ERR TypeError expected string or buffer openai 5 ERR TypeError expected string or buffer compat {'path': 'x'} ERR TypeError expected string or buffer anthropic {'path': 'x'} ERR TypeError expected string or buffer tiktoken {'path': 'x'} ERR TypeError expected string or buffer AFTER openai None 21 {'path': 'x'} 27 5 22 '{"path":"x"}' 26 compat None 20 {'path': 'x'} 26 5 21 '{"path":"x"}' 25 anthropic None 9 {'path': 'x'} 15 5 10 '{"path":"x"}' 14 tiktoken None 14 {'path': 'x'} 20 5 15 '{"path":"x"}' 19 ``` - Not tested: I did not exercise a live `headroom proxy --mode cache` + `curl /v1/compress` round trip, nor a real OpenAI-compatible upstream that emits object `arguments`. The proxy path was verified only down to the counters that its traceback terminates in, plus the automated tests above. Also untested: `providers/google.py`, `cohere.py`, `litellm.py`, which have no tool-call counting branch and so were left alone. ## Review Readiness - [x] I have performed a self-review - [x] I have commented my code where the reasoning is not obvious - [x] This PR is ready for human review Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-08-05 08:32:14 -07:00