mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
2b5ee7cde8
## Description
The direct (non-backend) Anthropic buffered `/v1/messages` path reads
token counts from the response usage to record metrics and update the
prefix tracker:
```python
usage = resp_json.get("usage", {})
output_tokens = usage.get("output_tokens", 0)
cr_tokens = usage.get("cache_read_input_tokens", 0)
cw_tokens = usage.get("cache_creation_input_tokens", 0)
...
uncached_input_tokens = usage.get("input_tokens", 0)
```
`.get(key, default)` only falls back when the key is **absent**. When a
key is present with a **null** value, `.get` returns `None`. The direct
Anthropic API always sends integer usage, but this same handler serves
any Anthropic-compatible upstream reached through a custom
`ANTHROPIC_TARGET_API_URL` gateway (the scenario `install apply` now
supports), and such a gateway can emit null counts on a stopped or empty
turn.
Those `None`s then reach `max(0, expected_cached - cr_tokens)` in the
cache-bust block and the int-typed `RequestOutcome` / metrics recorder,
so a single such response raises an uncaught `TypeError` and 502s the
request. This is the same class as the Gemini crash fixed in #2347 and
the OpenAI chat path.
## Fix
Coerce the four counts with `int(... or 0)` at the direct-path
usage-extraction site, matching `_extract_anthropic_cache_ttl_metrics`
(which already guards its TTL buckets this way) and the Gemini fix. A
normal integer usage is unchanged; only a null (or absent) value now
becomes 0.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py`: `int(... or 0)`-guard
`output_tokens` / `cache_read_input_tokens` /
`cache_creation_input_tokens` / `input_tokens` at the direct
buffered-path usage-extraction site.
- `tests/test_proxy/test_anthropic_buffered_timeout.py`: regression
driving a buffered `/v1/messages` request whose upstream usage reports
null counts, asserting a 200 instead of a 502.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_proxy/test_anthropic_buffered_timeout.py -q
# all pass
# with the fix reverted, the new test fails (the null-usage response 502s):
$ git stash push -- headroom/proxy/handlers/anthropic.py
$ python -m pytest "tests/test_proxy/test_anthropic_buffered_timeout.py::test_anthropic_messages_buffered_survives_null_usage_counts" -q
1 failed (TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType')
$ uvx ruff@0.15.17 check headroom/proxy/handlers/anthropic.py tests/test_proxy/test_anthropic_buffered_timeout.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: ran the new FastAPI `TestClient` regression,
which drives the real direct buffered `/v1/messages` handler with
`proxy._retry_request` returning a 200 whose `usage` has null
`input_tokens` / `output_tokens` / `cache_read_input_tokens` /
`cache_creation_input_tokens`; then reverted only `anthropic.py` and
re-ran.
- Observed result: with the fix the request returns 200; with the fix
reverted the same request 502s with `TypeError: unsupported operand
type(s) for +: 'NoneType' and 'NoneType'`. Ran against the actual
handler via the app.
- Not tested: a live third-party Anthropic-compatible gateway emitting
null usage.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|---|---|---|
| .. | ||
| test_anthropic_buffered_timeout.py | ||
| test_anthropic_ccr_deferred_injection.py | ||
| test_anthropic_ccr_raise.py | ||
| test_anthropic_recount_and_reparse_safety.py | ||
| test_anthropic_streaming_ccr_retrieve.py | ||
| test_anthropic_upstream_header.py | ||
| test_background_compression.py | ||
| test_bedrock_passthrough.py | ||
| test_bedrock_sse_ping.py | ||
| test_cc_switch_reconciler.py | ||
| test_ccr_frozen_prefix_coupling.py | ||
| test_compression_failure_action.py | ||
| test_compression_timeout_config.py | ||
| test_compute_turn_id.py | ||
| test_gemini_savings_profile.py | ||
| test_header_safe_transforms.py | ||
| test_mcp_stats_aggregation.py | ||
| test_model_router.py | ||
| test_model_router_wiring.py | ||
| test_openai_backend_path.py | ||
| test_openai_chat_ccr_injection.py | ||
| test_openai_chat_savings_profile.py | ||
| test_openai_responses_ccr.py | ||
| test_openai_stream_usage_option.py | ||
| test_openai_transport_path_prefix.py | ||
| test_openai_upstream_header.py | ||
| test_phase3_byte_identity.py | ||
| test_request_logger.py | ||
| test_settings_fresh_process_precedence.py | ||
| test_settings_store.py | ||
| test_tool_search_repair_after_turn_hooks.py | ||
| test_transformations_feed.py | ||