Two coupled defects on Anthropic's 1M-context tier: Headroom
**under-budgeted** those sessions and **under-priced** them by ~2x. The
second gets worse once the first is fixed, so they ship together.
---
# Part 1 — `[1m]` was lost before the context budget was sized
`sanitize_anthropic_model_id()` strips a trailing `[1m]`, which is
correct for the wire — upstream Anthropic rejects the suffix, and #2027
added the strip for exactly that reason.
But `[1m]` is not only an ANSI artifact. Claude Code appends it to a
model id to request the **1M context tier**, and only sends the
`context-1m` beta header when it is present (#1158 — what `headroom wrap
claude --1m` sets up).
`get_context_limit()` sanitized *before* resolving, so the tier was gone
by lookup time:
```python
provider.get_context_limit("claude-sonnet-4-5[1m]") # 200_000 ← real window is 1M
```
The request still reached Anthropic correctly and still got a 1M window
— the beta header goes through untouched. What broke is our **budget**:
Headroom sized a 1M session at 200K and began compacting at a fifth of
the available room.
Models whose base is already 1M (`claude-opus-5`, `claude-sonnet-5`)
resolved to 1M either way, which is why this went unnoticed. It bites
the Sonnet 4 / 4.5 family — the models `[1m]` exists for.
**Fix:** read the tier off the id *before* sanitizing; raise the
resolved limit to at least 1M. `max()` rather than assignment, so a base
wider than 1M keeps its own window. Detection is deliberately narrower
than the sanitizer — only a literal `[1m]`; `[0m]`, `[1;32m]` and real
`ESC[` sequences still strip without promoting.
| model id | wire id (unchanged) | limit before | limit after |
|---|---|---|---|
| `claude-sonnet-4-5` | `claude-sonnet-4-5` | 200K | 200K |
| `claude-sonnet-4-5[1m]` | `claude-sonnet-4-5` | **200K** | **1M** |
| `claude-opus-5[1m]` | `claude-opus-5` | 1M | 1M |
| `claude-sonnet-4-5[0m]` | `claude-sonnet-4-5` | 200K | 200K |
| `ESC[1m claude-sonnet-4-5 ESC[0m` | `claude-sonnet-4-5` | 200K | 200K
|
The wire id is unchanged in every case, so #2027 holds — guarded by a
regression test.
---
# Part 2 — the pricing that reports those sessions was wrong
### 2a. The LiteLLM cost path was dead in every provider
`litellm.completion_cost()` no longer accepts `prompt_tokens` /
`completion_tokens`. Every call raised `TypeError`:
```
TypeError: completion_cost() got an unexpected keyword argument 'prompt_tokens'
```
All five providers — `anthropic`, `openai`, `google`, `cohere`,
`litellm` — caught it with a bare `except` and silently fell through to
their hand-maintained tables. The "up-to-date pricing from LiteLLM" the
docstrings promise **has not run at all**. Anthropic additionally passed
`input_tokens - cached_tokens`, the wrong convention (LiteLLM expects
the cache-inclusive total), which would also have suppressed the
long-context threshold even had the call worked.
Replaced with `litellm.cost_per_token()` behind one shared helper,
`pricing.litellm_pricing.estimate_cost_from_tokens()`, which reuses the
existing gateway-alias candidate chain and returns `None` (not an
exception) when LiteLLM can't price a model.
### 2b. Neither path applied Anthropic's long-context premium
On the Sonnet 4 / 4.5 family a prompt over 200K re-prices the **whole**
request — input 2×, output 1.5×, cache 2× — not just the tokens past the
threshold. Rates confirmed from LiteLLM's `*_above_200k_tokens` fields.
| request (`claude-sonnet-4-5`) | reported before | true | error |
|---|---|---|---|
| 100K in / 5K out | $0.3750 | $0.3750 | — |
| 300K in / 5K out | $0.9750 | **$1.9125** | −49% |
| 300K in (150K cached) / 5K out | $0.5700 | **$1.1025** | −48% |
LiteLLM applies this itself once the call works. The manual fallback
needed `_apply_long_context_premium()` — the LiteLLM dependency is gated
`python_version < '3.14'`, so on 3.14 the fallback is the *only* path.
**Both paths now agree to four decimal places on every case under
test.**
---
## What I checked and did *not* change
The fork report that prompted this claimed the Anthropic tables were
materially stale ("Opus 4.x priced wrong"). **That does not hold.** I
audited every entry against LiteLLM's vendored table:
- **Anthropic** — every model LiteLLM knows matches exactly, Opus 4.x
included.
- **OpenAI** — all 17 entries match; the two that don't resolve are
retired models.
The defect was the mechanism, not the numbers, so the rate cards are
untouched.
One thing the repaired path fixes for free: OpenAI's cached-input
discount is **50% on gpt-4o, 75% on gpt-4.1, 90% on gpt-5**, but the
manual path applies a flat 50% estimate. With LiteLLM live, real
per-model rates are used. The flat estimate remains only as the offline
fallback.
## Scope
**No Rust change needed.**
`crates/headroom-proxy/src/compression/model_limits.rs` resolves context
windows but has **no in-tree callers**; the Rust `[1m]` handling is
wire-body sanitization only, correct as-is, and its integration tests
assert behavior this PR does not touch.
**Judgment call worth a reviewer's eye:** the `[1m]` marker is honored
for *any* model, including ones with no 1M tier
(`claude-haiku-4-5-20251001[1m]` → 1M). Gating on an allowlist would be
more precise but reintroduces a hand-maintained table that rots — the
failure mode `model_limits.rs` already documents against. Since `[1m]`
is set by our own wrapper and Claude Code's opt-in, honoring it seemed
the better default. Happy to tighten.
## Tests
- `TestContext1MSuffix` — detection, the 200K→1M promotion, the `max()`
floor, ANSI non-promotion, and the wire-id guard for #2027.
- `TestLongContextPricing` — the premium on both paths (parametrized),
threshold boundary (200,000 vs 200,001), an untiered model charged no
premium, and the two halves meeting: a `[1m]` request gets both the 1M
window and the premium rate.
- `TestLiteLLMCostHelper` — unknown model returns `None`, a known model
prices correctly, and `input_tokens` is cache-inclusive.
Two existing tests were updated, both pinned to the broken behavior:
- `test_estimate_cost_basic` probed a "per 1M" rate by sending exactly
1M tokens, which now crosses the 200K threshold. Re-probed at 100K.
(Worth knowing: `claude-3-5-sonnet-20241022` is retired and no longer in
LiteLLM, so the alias chain resolves it to `claude-sonnet-4-20250514`
and it inherits that model's tier. Harmless — a 200K-window model can't
exceed 200K in reality — but it explains the number.)
- `test_litellm_provider_info_and_cost_fallbacks` monkeypatched
`litellm.completion_cost`; repointed at the new helper seam.
```
ruff check / ruff format / mypy — clean across all six changed source files
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
## Description
Each token counter in `headroom/providers/` had grown its own shortened
content-block walker, handling only the shapes its provider was expected
to send. Everything else fell through and contributed **nothing**.
Measured on one 6,800-char block, via `count_messages` of a single-block
message — so 7–8 is message overhead alone:
| block type | OpenAI ctr | Anthropic ctr |
|---|---|---|
| `text` (control) | 3409 | 3748 |
| `tool_result` | **8** | 3748 |
| `thinking` | **8** | **7** |
| `document` | **8** | **7** |
| `mcp_tool_result` | **8** | **7** |
| `output_text` | **8** | **7** |
| `refusal` | **8** | **7** |
Two things make this worse than a coverage gap:
1. **Each counter zeroed blocks from its own provider.** `output_text`
and `refusal` are OpenAI Responses shapes; `thinking` and `document` are
Anthropic's.
2. **These are the counters the live pipelines use.** `proxy/server.py`
builds them with `AnthropicProvider` / `OpenAIProvider`, so this is the
main request path — not an edge case. #2743 fixed this for
`/v1/compress` only, by routing that route to the registry tokenizers,
whose `BaseTokenizer` walker is complete.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
Rather than add a **fifth** partial walker, the counters now delegate to
the audited one:
- `tokenizers/base.py` — new `count_content_blocks(parts,
count_text_fn)` plus a thin `_DelegatingBlockCounter` adapter, since the
provider counters are not `BaseTokenizer` subclasses. `BaseTokenizer`
itself is untouched.
- `providers/openai.py`, `providers/anthropic.py`,
`providers/openai_compatible.py` — list-content branches delegate.
**Why delegate instead of adding a `count_text(str(block))` catch-all:**
that would serialize a base64 blob and price it as text.
`tiktoken_counter.py` already documents the failure — a 1MB image
becomes ~330K phantom tokens. The shared walker gives media a
pixel/byte-based estimate.
**Scope:** the three counters that accumulate token counts. `google.py`
and `cohere.py` extract a *text string* first and count that, so the
same defect there needs a differently-shaped fix — left as a follow-up.
## Testing
- [x] Unit tests pass
- [x] Linting passes (`ruff check` + `format --check`, pinned 0.15.17)
- [x] New tests added
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_provider_counter_content_blocks.py -q
12 passed in 0.52s
$ uvx ruff@0.15.17 check headroom/ tests/... --exclude headroom/dashboard/templates
All checks passed!
```
**After the fix**, every shape lands within ~1% of the equivalent plain
text, and media stays bounded:
```text
block OpenAI Anthropic
text (control) 3409 3748
tool_result 3409 3748
thinking 3419 3759
document 3423 3763
mcp_tool_result 3422 3762
output_text 3420 3760
refusal 3421 3761
image b64 200KB 1608 1607 <- pixel estimate, not ~50K as text
```
## Real Behavior Proof — including a regression I caught
**This change flipped an existing test**, and I only found it because
every suite was run against clean `upstream/main` in the same
environment with the failure sets diffed:
```text
before the test rewrite:
upstream/main : 1 failed, 104 passed
this branch : 2 failed, 103 passed <- regression
diff : + test_openai_compatible_token_counter_ignores_unhandled_content_shapes
```
That test asserted `content: [{"type": "image"}, 123] == 8` — i.e. it
**pinned the defect**, that unhandled shapes contribute nothing.
Rewritten as `..._prices_declared_media`: a declared image is now priced
(1608) while a bare int is still correctly ignored (8), with the
rationale in the docstring.
```text
after the rewrite:
upstream/main : 1 failed, 104 passed, 10 skipped, 25 errors
this branch : 1 failed, 104 passed, 10 skipped, 25 errors
failure sets : IDENTICAL
```
- **Pre-existing, not from this change:** the 1 failure and all 25
errors. The errors are all in
`test_compress_route_tokenizer_by_model.py`, whose loopback `TestClient`
fixture this throwaway env cannot satisfy.
- **Environment note:** `content_router` and several suites need the
compiled `headroom._core`, which isn't in a fresh worktree (gitignored,
built in-place). I copied the built `.so` in to run these and removed it
before committing.
- **Not tested:** no live provider call, so the *absolute* accuracy of
the 1600 image estimate against a real Anthropic/OpenAI bill is
unverified — it is the value `BaseTokenizer` already used, and this PR
only changes which blocks reach it.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] I did **not** edit `CHANGELOG.md`
## Related
Fourth PR from one tokenizer-consistency audit: #2757 (litellm total
prompt / `--budget`), #2758 (HuggingFace chat templates, `gpt-5`,
gateway-wrapped names), #2759 (router token units). Plus #2756, which
splits the local/provider token scales in `RequestOutcome`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The tokenizer registry routed deepseek-v4-pro, deepseek-v4-flash,
deepseek-chat, deepseek-reasoner, and other modern DeepSeek models
to the 2023-era deepseek-llm-7b-base tokenizer via prefix fallback.
This caused token counts off by 30-50%, broken context-limit detection
(V4-Pro supports 1M but got 32K), and inaccurate savings reports.
## Fix
3 files, +43/-2:
- **huggingface.py**: 16 new MODEL_TO_TOKENIZER entries with verified
HuggingFace IDs (deepseek-ai/DeepSeek-V4-Pro, V4-Flash, V3.2,
V3-0324, R1, R1-0528, Reasoner, Chat, Coder-V2, etc.)
- **openai_compatible.py**: 17 new _DEFAULT_CONTEXT_LIMITS entries
(V4-Pro/Flash -> 1M, R1/Reasoner -> 131K, V3 -> 128K, etc.)
- **openai.py**: 8 new _CONTEXT_LIMITS entries for LiteLLM-fallback.
Existing mappings untouched (backward compatible).
## Real behavior proof
- **Setup**: Windows 11, Python 3.13.14, headroom-ai 0.2.15 wheel +
source checkout at v0.24.0. No Rust extension built (headroom._core
unavailable). Touched files are at parity with v0.24.0.
- **Steps after patch**:
```
python3 -c "
from headroom.tokenizers.huggingface import get_tokenizer_name
for m in
['deepseek-v4-pro','deepseek-chat','deepseek-reasoner','deepseek-v4-flash']:
print(f'{m} -> {get_tokenizer_name(m)}')
from headroom.tokenizers.registry import get_tokenizer
for m in ['deepseek-v4-pro','deepseek-chat','deepseek-reasoner']:
print(f'{m}: {get_tokenizer(m)}')
"
```
- **Observed result**:
```
deepseek-v4-pro -> deepseek-ai/DeepSeek-V4-Pro
deepseek-v4-flash -> deepseek-ai/DeepSeek-V4-Flash
deepseek-chat -> deepseek-ai/DeepSeek-V3
deepseek-reasoner -> deepseek-ai/DeepSeek-R1
```
Previously ALL resolved to deepseek-ai/deepseek-llm-7b-base.
TokenizerRegistry routes correctly. Context limits verified
(1M / 131K / 128K). compress() import smoke-tested OK.
- **Not tested**: full proxy e2e with a live DeepSeek API key
(no available key). HuggingFace AutoTokenizer download confirmed
for V4-Pro/V3/R1 but produced GBK decode errors from hf_hub on
this zh-CN Windows locale during config fetch -- a separate
huggingface_hub issue unrelated to this change.
<!-- headroom-maintainer-template-completion:start -->
## Description
This PR prepares `fix: add DeepSeek V4/V3.2/R1 tokenizer mappings and
context limits` for review by documenting the intended change,
validation evidence, and remaining merge-readiness context.
Linked issues: None declared.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Documentation
- [ ] Refactor
- [ ] Tests only
## Changes Made
- Commit: fix: add DeepSeek V4/V3.2/R1 tokenizer mappings and context
limits
- Touches `headroom/providers/openai.py`
- Touches `headroom/providers/openai_compatible.py`
- Touches `headroom/tokenizers/huggingface.py`
## Testing
- [x] GitHub checks reviewed
- [x] Metadata/template validation
- [ ] Local functional testing
### Test Output
```text
gh pr view 912 --repo chopratejas/headroom --json statusCheckRollup
- PR Governance / label: SUCCESS
- external / GitGuardian Security Checks: SUCCESS
```
## Real Behavior Proof
- Environment: GitHub PR metadata and checks for `chopratejas/headroom`
PR #912.
- Exact command / steps: Reviewed PR title, commits, changed files,
linked issues, labels, and check rollup; appended this maintainer
template completion block without replacing the author's original
description.
- Observed result: PR body now contains all required governance
sections, checked readiness fields, and a non-placeholder validation
evidence block.
- Not tested: This pass updated PR metadata only; code validation
remains represented by the linked GitHub checks and any author-provided
evidence above.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
<!-- headroom-maintainer-template-completion:end -->
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
## Description
Update `_DEFAULT_CONTEXT_LIMITS` so DeepSeek V3/V4 use their actual 1M
(1,048,576) context window instead of the outdated 128K. The hardcoded
128K causes Headroom to trigger compression far too early, defeating the
purpose of using a long-context model.
Closes#1038
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Update `deepseek` default from 32,768 to 1,048,576 (V3/V4 family
default)
- Update `deepseek-v3` from 128,000 to 1,048,576
- Update `deepseek-coder` from 16,384 to 128,000 (Coder V2+)
- Add `deepseek-v4` entry at 1,048,576
- `deepseek-v2` stays at 128,000 (unchanged)
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_providers/test_universal.py -v -x
37 passed, 3 skipped in 38.08s
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.11, headroom main (6904d47)
- Exact command / steps: python -m pytest
tests/test_providers/test_universal.py::TestOpenAICompatibleProvider::test_get_context_limit_deepseek_v3_is_1m
-v
- Observed result: PASSED - deepseek-v3 returns 1048576, deepseek-v4
returns 1048576, deepseek returns 1048576, deepseek-v2 returns 128000
- Not tested: no manual proxy testing with a live DeepSeek endpoint
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
The context limit values are based on the DeepSeek V3 technical report
and API documentation, which specify a 1M-token context window. DeepSeek
Coder V2+ also supports 128K, up from the original Coder's 16K.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
