mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
2471 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tejas Chopra
|
f624d3a00a
|
perf(proxy): bound upstream calls and hot-path costs (#2852)
Seven commits from one week of load testing: one hang, two request-path correctness fixes, and four hot-path costs that only show up in production. ## Reliability **Bound every upstream call.** The litellm backend had no timeout at all, so a request the upstream never answered blocked its caller forever. Observed under load on 2026-08-07: four agent workers on ESTABLISHED connections for 36+ minutes while `/readyz` answered in 0.11s. No error, no retry, no log line — indistinguishable from slow work, which is the worst shape a failure can take. A float rather than an `httpx.Timeout`, deliberately: litellm expands a float across all four httpx phases, so on a streaming call it becomes the maximum gap *between chunks*, not a cap on total generation. A long answer streaming steadily is never cut off; a stalled one dies. Default 600s via `HEADROOM_UPSTREAM_TIMEOUT`; 0, negative, and junk fall back to the default rather than meaning "no timeout". **Keep the consistency re-count off the event loop.** It ran `tokenizer.count_messages` twice directly on the loop. Since Claude counting moved to a real BPE that is CPU-bound work stalling every other in-flight request — ~1s on a 2.3 MB body, with `/healthz` gaps tracking body size. Offloaded via `asyncio.to_thread` on the same tokenizer instance, so reported values are unchanged. (#2810) **Survive a re-parse MemoryError.** `MemoryError` is not a `ValueError`, so on 1M-context payloads the byte-faithful forwarder's verification re-parse escaped the handler and aborted an otherwise-fine request — 14 aborts across 8 days of reporter logs. (#2768) ## Performance All four are measured, not guessed. Each degrades with something a short benchmark does not vary: uptime, content shape, or process age. | fix | before | after | |---|---|---| | Cost-record walk per request (at 100k records) | 13.6 ms | bounded by model count | | JSON-block scan, JS-style object logs (1200 lines) | 4643 ms | 183 ms | | JSON-block scan, truncated JSONL | 3737 ms | 116 ms | | Lazy imports inside user requests | multi-second | paid at startup | | `count_text` (80% of local CPU) | — | memoised | Two worth calling out: - **The cost walk degrades with proxy *uptime*, not load.** A freshly started proxy pays ~0.01 ms; a month-old one pays 4–13 ms on every request, on the event loop, holding the metrics lock. Deliberately not a TTL cache over `stats()`: those values feed `check_budget()` when `--budget` is set, and a stale reading under-enforces the budget. The fix is to stop computing what the caller discards. - **The JSON-block memo is built only *after* a scan fails to balance.** That ordering is load-bearing, not an optimisation — caching from the start made pretty-printed JSON ~2x slower, since content that balances on the first scan has nothing to reuse and just pays the per-line dict traffic. Still a constant-factor fix, not an asymptotic one. ## Tests +1202 lines, 20 files. Each fix is pinned by a test that fails on the unmodified code: the re-count test asserts no `count_messages` pass runs with a live event loop in its thread; the re-parse test drives a `MemoryError` through the real request path and expects a 200; `totals()` equality with `stats()` is asserted across model counts, request volumes, and both pricing branches. The timeout test is structural rather than a mock — the failure mode is a dispatch path someone adds later without a guard, which mocking the existing four cannot catch. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tejas Chopra
|
e0870ef931
|
feat(beacon): hourly R2 compaction, per-strategy savings, and a stack that reports (#2853)
Three beacon changes bundled because they are one story: the corpus got
too
slow to query, and then too coarse to answer the question it was
collected for.
## 1. Hourly compaction (`deploy/beacon`)
The beacon writes one ~1 KB object per heartbeat — **64,987 on
2026-08-06** and
climbing. A full analysis `pull` was ~100k HTTPS round trips for 95 MB:
minutes
of pure per-object latency. Listing the bucket alone took 88 seconds.
Moving the query server-side does not help — R2 SQL reads only Iceberg
tables,
and a pile of tiny files is the pathological case for every query
engine.
Compaction is the fix, and it is Iceberg's own answer to the same
problem.
An hourly cron collapses each **complete** hour of `sessions/` into one
`rollup/dt=…/hh=…/data.ndjson`, keeping the highest-`seq` heartbeat per
`(install, session)`.
| measured on `dt=2026-08-06/hh=14` | before | after |
|---|---|---|
| objects | 3,938 | **1** |
| rows | 3,938 | **1,061** |
| analysis `pull` | minutes | **seconds** |
Hourly rather than daily because every R2 binding call is a subrequest:
a day
is ~65k, an hour is ~4k. Newest-first, so a backlog drains from the
present
backwards and live data never starves behind it; a failing hour is
logged and
skipped rather than blocking every older hour behind it. **Raw objects
are
never deleted**, so any rollup is rebuildable by deleting it.
Backfill runs to the **oldest surviving raw day**, not a fixed window. A
fixed
lookback strands everything older than it the moment analysis stops
reading
`sessions/`: the raw objects are still there, but nothing would ever
compact
them, so they disappear from every report. `oldestRawDay()` finds that
floor in
one delimited LIST, and the rollup listing starts from it — so the work
is
bounded by retention rather than by total history.
Three failure modes the tests pin down, because each one is silent:
- A **failed `get`** is transient, so the hour throws and writes
nothing. A
rollup is built once and trusted forever, so a short read would quietly
become the permanent record.
- A **corrupt record** loses only itself. This Worker wrote that content
with
`JSON.stringify`; it will never become valid, so blocking on it strands
the
hour instead of the record.
- An **empty hour** writes an `empty` marker. Without one the hour stays
"missing" and is re-listed on every run forever.
`test-rollup.mjs` asserts the Worker's dedup picks exactly the same rows
as the
analysis-side `QUALIFY`. If those two ever disagree the reports go
quietly
wrong rather than loudly broken, which is why that check exists. Its
stub
paginates at 3 keys so the list cursor loop — load-bearing at the real
~4,000
objects/hour — runs in every case.
## 2. Per-strategy savings (`compression.by_strategy`)
`compression.transforms` counts *invocations*, which cannot distinguish
a
compressor that saved 60% from one that ran constantly and saved
nothing. The
fleet's top transform by count contributes an unknown share of
`tokens.saved`.
It is worse than that in practice. Transform labels are slugged with
`split(":", 1)[0]`, so every `router:<strategy>:<detail>` label
collapses into a
single `router` bucket. On 2026-08-08 that bucket held **19.1 M of the
day's
transform counts, across 8,528 of 9,616 sessions** — the compressors
that do
most of the work are indistinguishable from each other, by name as well
as by
yield:
| transform | n | sessions |
|---|---|---|
| `router` | 19,108,118 | 8,528 |
| `anthropic` | 688,124 | 4,734 |
| `output_shaper` | 548,017 | 1,120 |
No question about which strategy is earning its keep can be answered
from that,
which is what this field is for.
The measurement already existed. `PrometheusMetrics.record_compression`
is the
configured `CompressionObserver` and already accumulates
`tokens_saved_by_strategy` on the hot path — the numbers just never left
the
process. This forwards from that one chokepoint rather than adding a
second
observer and a second measurement pass. The paths that have **no**
observer
configured (MCP server, LangGraph, Strands hooks, the transform
pipeline) get
`BeaconCompressionObserver` passed directly.
Compression runs on the executor thread *before* that request's outcome
reaches
`record()`, so events are **staged** into module state and drained by
the next
outcome. Staging is what makes two things true at once:
- The first turn of a session still reports its numbers — otherwise
every
session's opening turn, and any session short enough to be one turn,
would
report nothing.
- A compression event **never opens a session**. An abandoned request
would
otherwise emit a phantom `turns=0` row with all-zero tokens, inflating
fleet
session and install counts.
Staging takes a dedicated mutex the request path never touches, so the
fan-out
stays off the aggregator's lock and `record_compression` keeps its
"synchronous + lock-free" contract.
```json
"by_strategy": [
{"strategy": "code_aware", "n": 1, "tokens_in": 800, "tokens_out": 800},
{"strategy": "smart_crusher", "n": 2, "tokens_in": 1500, "tokens_out": 700}
]
```
A **list of records, sorted by strategy** — not an object keyed by
strategy.
Keyed shapes change type as keys accumulate: DuckDB infers a STRUCT
under ~24
keys and a MAP over it, so the analysis query breaks on the day the
fleet
picks up a 25th strategy. Sorted so heartbeats are byte-comparable.
**These do not sum to `tokens.saved`,** and the field comment says so:
strategies compose (the router routes, a strategy runs inside it) so the
same
text is measured more than once. A row means "of what this strategy was
handed,
it removed this much" — a per-strategy yield, not a share of the total.
A
strategy that saved nothing still appears; dropping it would make every
strategy look effective.
## 3. `headroom.stack`
`resource_attributes()` was called with no arguments at its one call
site, so
`headroom.stack` was absent from **all 24,040 sessions** in the corpus
while
`detect_stack` sat unused — dead code on both ends of a wire nobody
connected.
The fleet was unsegmentable by agent, which is the question the corpus
is asked
most often.
Environment detection alone is not enough. It answers `wrap_claude` only
under
`headroom wrap`; every install that points an agent at a persistent
proxy — the
common deployment — reports `proxy`, which segments nothing. The
per-request
`X-Headroom-Stack` slugs are the only signal that names the harness
there, so
`record_stack()` stages them the same way and feeds `detect_stack`'s
`by_stack`
branch:
```
9x wrap_claude, 1x wrap_cursor -> wrap_claude (dominant harness wins)
5x wrap_claude, 5x wrap_cursor -> mixed
no per-request signal -> proxy (environment fallback)
junk slug -> dropped before staging
```
## Privacy
The strategy string is slugged through the same `_safe_slug` as skip
reasons
and capped at `MAX_STRATEGIES`, because the observer protocol takes a
free
string and an extension could otherwise invent keys per request. Stack
slugs
are normalized and capped the same way.
**Deliberately not collected:** the tool names in
`smart_crush:<count>:<names>`.
Those are user-defined MCP identifiers and can name internal tooling
(`acme_deploy_prod`). They stay stripped by the existing `split(":",
1)[0]`,
and this PR does not widen it. No new key was needed in `worker.js` —
`by_strategy` nests under the already-allowlisted `compression`, and
`headroom.stack` was already in `ALLOWED_RESOURCE`.
Nothing here deletes or rewrites existing data: the Worker only ever
writes,
`sessions/` is never pruned by it, and readers merge old and new shapes
with
`union_by_name`, so pre-change heartbeats keep reading with the new
fields null.
## Verification
- `python -m headroom.telemetry.session` — self-check covers staging,
the
no-phantom-session case, drain exhaustiveness, a 0%-yield strategy
staying
visible, the cardinality cap, slug safety, and dominant/mixed/junk stack
resolution
- `node test-rollup.mjs /tmp/hr` — 3,938 real corpus objects → 1,061
sessions
in 1 object, plus the pagination, partial-read, corrupt-record,
empty-hour
and `oldestRawDay` cases
- 51 passing in `test_compression_observability`,
`test_prometheus_obs_counters`,
`test_telemetry_context`, `test_compression_strategy_outcomes`
- Consumer side exists and is checked: `beacon.sh by_strategy` in
headroom-beacon-stats reads the field end to end (sessions, installs,
invocations, tokens in/out, yield %), verified against a synthetic
parquet for
the cases that matter — aggregation across installs, a 0%-yield strategy
staying visible, pre-field sessions dropping out rather than erroring.
It is
guarded on the column, so it prints an instruction instead of a binder
error
until a release carrying this PR reaches the fleet.
- Deployed and running against the live corpus on the `5 * * * *`
trigger
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Tejas Chopra
|
675d13f08d
|
fix(proxy/openai): run response hooks on Responses, and bill their re-drives (#2872)
The Responses path runs `run_request_hooks` but never `run_response_hooks` — only `handle_openai_chat` does. So a turn hook can shrink a Responses turn and then never be asked to resolve what the model did about it: the model's injected tool call goes straight to a client that has no such tool. That asymmetry is why tool-belt deferral has to be disabled wholesale on the Responses API, which is the surface Codex uses. ## 1. Wire the response side Mirrors the chat-completions block. **Buffered path only**, for the same reason CCR already forces `stream:false` when it needs to intercept: you cannot re-drive a turn whose bytes are already flowing. ## 2. Honour `stream_safe_only` on the Responses request path It was the one hook call site that ignored the flag. A re-driving hook would run its shrink on a streamed turn and then have no response side to finish it — latent until (1) lands, live afterwards. `stream` is not a parameter of `_compress_openai_responses_payload`, but the payload it is compressing carries the flag. It is read **before** CCR may force `stream:false` further down, so this is the client's request rather than the effective one — conservative in the safe direction: at worst a CCR-buffered turn misses a saving, never a stranded tool call. Fold-only hooks that declare `stream_safe = True` are unaffected. ## 3. Bill what the re-drives cost Both handlers read usage from the **final** upstream response, so every intermediate call a hook made was free as far as Headroom was concerned. For a token-saving feature that is not a rounding error. A tool-search reload is a whole extra model call; counting only the last one lets the feature hide its own overhead behind the saving it is claiming, and the numbers come out better than the truth. `TurnHookUsage` accumulates input/output/cached across re-drives; both HTTP paths fold it into their totals. The two surfaces report the same three quantities under different names (`prompt_tokens` vs `input_tokens`), so the key pair is passed in. Expect measured cost to go **up** and savings percentage to go **down** on any deployment running a re-driving hook. That is the correction, not a regression. ## Also: restore the body after the hooks A re-drive rewrites `body[input]` / `body[messages]` / `body[tools]` so the next upstream call carries the hook's turn. Everything downstream — CCR's `_responses_input_to_items(body["input"])`, usage accounting, observability — is describing the request the *client* made, not the proxy's internal detour. Without the restore, a turn that both reloaded a tool and hit CCR retrieval hands CCR the proxy's synthetic items. The chat path had the same leak (`body["messages"]` stayed rewritten); both are fixed the same way. ## Known gap A re-drive on the custom backend path (`send_openai_message`) is still not folded into that request's accounting — its usage is recorded elsewhere. Commented at the call site rather than silently skipped. ## Blast radius **Inert unless a turn hook is registered**, so no behaviour change for a stock OSS proxy. `TurnHookUsage` starts at zero and stays there on every path that does not re-drive. ## Verification - `tests/test_turn_hook_usage.py` — 5 new tests: per-surface key names, accumulation across rounds, negative counts floored not subtracted, and that an unreadable shape still counts the call (a silent zero there looks exactly like "the hook cost nothing") - 434 passing across `turn_hook`, `extension`, `tool_search`, `responses` and `openai_chat` suites - `ruff check` + `ruff format` clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Romulo Reis
|
91d6bf33cd
|
perf(subscription): skip transcripts older than the window in compute_window_tokens (#2861)
## Problem
`compute_window_tokens()` walks **every** `.jsonl` under
`~/.claude/projects` and runs
`json.loads()` on **every line**, only to discard the entries that fall
outside
`[start_ts, end_ts)`. `subscription/tracker._poll_loop` calls it every
**300 s**, so the
cost is paid continuously and grows with the user's history.
On one long-running install this meant **1,973 files / 1.1 GB / 261,003
lines re-parsed
every 5 minutes** — about 316 GB of JSON parsing per day.
The user-visible symptom is worse than the CPU bill: the poll pins **100
% CPU with zero
open connections** for ~12 s. That is exactly the signature external
watchdogs use to
detect a runaway loop, so the proxy kept being **restarted while it was
doing scheduled
work** (13 restarts / 13.5 CPU-hours on that host before we traced it
with `py-spy`).
Stack captured during one of those episodes:
```
raw_decode (json/decoder.py:356)
decode (json/decoder.py:337)
loads (json/__init__.py:346)
compute_window_tokens (headroom/subscription/session_tracking.py:127)
_compute_window_tokens_for_snapshot (headroom/subscription/tracker.py:872)
_maybe_poll (headroom/subscription/tracker.py:731)
_poll_loop (headroom/subscription/tracker.py:693)
```
## Fix
Transcripts are append-only and chronological, so a file whose `mtime`
predates the window
start cannot contain an entry inside the window. One guard before
opening the file:
```python
try:
if path.stat().st_mtime < start_ts:
continue
except OSError:
continue
```
## Measurement
Same install, same 5 h window, before vs after:
| | files read | lines parsed | time | result |
|---|---|---|---|---|
| before | 1,973 | 261,003 | **12.1 s** | `weighted_token_equivalent =
741388.0` |
| after | 14 (1,959 skipped) | 4,246 | **0.39 s** |
`weighted_token_equivalent = 741388.0` |
**Identical result, 31× faster.** In production the process CPU peak
over a full poll cycle
dropped from 100 % to 10 %.
## Notes
- Behaviour is unchanged: the guard only skips files that provably
cannot contribute.
- A further optimisation (not included here, to keep the change minimal)
is to read active
transcripts backwards and stop at the first entry older than `start_ts`.
The `mtime`
guard already removes ~99 % of the cost.
- Reproduced on 0.25.0, 0.27.0 and confirmed present in current `main`.
Co-authored-by: romulomorgan <oi@ialucas.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
Rod Boev
|
7f6950be34
|
fix(anthropic): strip first-party tool search on custom upstreams (#2539)
## Description Third-party Anthropic-compatible upstreams can reject Headroom-routed Claude requests before generation starts because the forwarded `tools[]` array still contains the first-party Anthropic server tool type `tool_search_tool_regex_20251119`. That path is valid when the upstream really is Anthropic, but DeepSeek-style Anthropic-compatible gateways reject it with a 400 and never reach model execution. This change strips first-party Anthropic `tool_search_tool_*` entries only when Headroom forwards an Anthropic-wire request to a third-party upstream selected through `anthropic_api_url`. Direct Anthropic behavior stays intact, and unrelated typed or untyped tools keep their existing forwarding contract. Closes #2526. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - add a narrow Anthropic helper that strips first-party `tool_search_tool_*` entries from client-supplied tool lists when the outbound target is a third-party Anthropic-compatible upstream - wire the sanitizer into the Anthropic handler's third-party forwarding path without changing the first-party `HEADROOM_TOOL_SEARCH` injector branch - add focused helper coverage for third-party stripping, first-party preservation, and typed-tool negative space - add a production-path regression through `handle_anthropic_messages()` that captures the custom-upstream request body and verifies the sanitizer wiring ## Testing - [x] Unit tests pass (`uv run pytest tests/test_issue_746_tool_search.py tests/test_anthropic_stage_timings.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py tests/test_issue_746_tool_search.py tests/test_anthropic_stage_timings.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_issue_746_tool_search.py tests/test_anthropic_stage_timings.py -q 50 passed in 0.72s uv run ruff check headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py tests/test_issue_746_tool_search.py tests/test_anthropic_stage_timings.py All checks passed! uv run ruff format headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py tests/test_issue_746_tool_search.py tests/test_anthropic_stage_timings.py --check 4 files already formatted git diff --check (no output) ``` ## Real Behavior Proof - Environment: focused Headroom worktree with Anthropic-wire regression tests - Exact command / steps: use the issue reproduction at https://github.com/headroomlabs-ai/headroom/issues/2526, then run the focused helper and handler tests; the handler regression calls `handle_anthropic_messages()` with a DeepSeek-compatible upstream and captures the outbound request body - Observed result: the base repro printed `FAIL issue2526 third-party sanitize -> [{'type': 'tool_search_tool_regex_20251119', 'name': 'tool_search_tool_regex'}, {'name': 'Bash', 'description': 'run a command', 'input_schema': {}}]`, while the head repro printed `PASS issue2526 third-party sanitize -> [{'name': 'Bash', 'description': 'run a command', 'input_schema': {}}]`; the handler-level test captured the same removal while preserving `Bash` and `web_search_20250305`, and the combined focused run passed 50 tests - Not tested: live DeepSeek account on this host ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A - proxy forwarding change only. ## Additional Notes - `CHANGELOG.md` stays untouched because Headroom's release automation generates it from conventional commits. - The narrow slice strips only first-party Anthropic server tool-search entries on third-party Anthropic-compatible upstreams. It does not invent or translate third-party search-tool semantics. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net> |
||
|
Abhay Singh
|
5c561bd913
|
fix(onnx): stop ONNX thread pools from spinning idle cores (#2495) (#2540)
## Description Fixes #2495 (tokensave / the proxy using ~100% of all cores). ONNX Runtime's intra-op (and inter-op) thread pools **spin-wait on every core between inferences** by default. Headroom is a long-lived process that keeps ONNX models loaded — the kompress code compressor ("tokensave"), the image technique/SigLIP routers, and the memory embedder — so once a model is loaded, its idle thread pool keeps every core busy even when no compression is running. That matches the report exactly: CPU climbs to ~100% of all cores "after a period of time" and the whole machine slows down, with no obvious trigger. `create_cpu_session_options` (the shared factory every CPU ONNX session goes through) configured threads and the memory arena but never touched spinning, so ORT's default (spin enabled) was in effect everywhere. ## Fix Disable intra-op and inter-op thread spinning in `create_cpu_session_options` so idle ORT threads block instead of spin-waiting. This applies to every ONNX session built through the factory (kompress + the image routers). It: - is **best-effort per key** (wrapped in try/except) so an older ORT build that doesn't recognize a config key still creates a session; - is **overridable** via `HEADROOM_ONNX_ALLOW_SPINNING=1` for a dedicated/batch box that wants ORT's peak-throughput spinning; - does not change active-inference throughput meaningfully — blocking threads wake on new work with only microsecond-scale latency, which is the recommended setting for a server/proxy with idle periods. The memory embedder already builds its own options with `intra_op_num_threads=1`; this change is orthogonal and additionally quiets its idle spinning if it were ever routed through the factory. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/onnx_runtime.py`: add `ONNX_ALLOW_SPINNING_ENV` + `onnx_thread_spinning_enabled()`; disable `session.intra_op.allow_spinning` / `session.inter_op.allow_spinning` in `create_cpu_session_options` unless spinning is explicitly re-enabled. - `tests/test_onnx_runtime.py`: spinning is disabled by default (both keys), `HEADROOM_ONNX_ALLOW_SPINNING=1` re-enables it, an explicit `0` disables it, and a config key an older ORT rejects doesn't break session creation. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_onnx_runtime.py -q 11 passed # with the fix reverted the new symbols don't exist, so the spinning tests # fail at import — the pre-fix factory left ORT's spinning at its (enabled) default $ uvx ruff@0.15.17 check headroom/onnx_runtime.py tests/test_onnx_runtime.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/onnx_runtime.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`, onnxruntime 1.23.2 installed), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: built a real `onnxruntime.SessionOptions` via `create_cpu_session_options(ort)` and read back `session.intra_op.allow_spinning` / `session.inter_op.allow_spinning`; repeated with `HEADROOM_ONNX_ALLOW_SPINNING=1`. - Observed result: by default both keys read back `"0"` (spinning disabled); with `HEADROOM_ONNX_ALLOW_SPINNING=1` neither key is set (ORT's default spinning restored). Against a real ORT the pre-fix factory set neither key, so ORT's default (spinning enabled) applied — the idle all-cores burn. Ran against the actual module and real onnxruntime. - Not tested: a live multi-hour VS Code + Claude session measuring CPU before/after (the spinning-disable is the documented ORT remedy for idle-CPU in a long-lived process; the config change itself is verified end to end against real ORT). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable --------- Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net> |
||
|
Rod Boev
|
54ea28d983
|
fix(openai): skip Responses tool-search deferral for clients that cannot execute it (#2696)
## Description OpenCode rejects the proxy-injected Responses `tool_search` tool because it resolves tool calls against its local registry. This PR now uses the shared client policy from current `main` and leaves OpenCode tools resident, alongside the existing Codex exclusion. Other clients retain tool-search deferral. Closes #2660. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] Documentation update ## Changes Made - Add `opencode` to the shared exact-match unsupported-client set in `headroom.proxy.helpers`. - Carry the already-classified `client` through native HTTP, WebSocket, and custom-base Responses paths. - Preserve `main`'s compatibility loop, which retries only exact unsupported `client` or `timing` keyword errors and re-raises internal `TypeError`s. - Add focused helper, compressor, HTTP, passthrough, and WebSocket coverage. ## Testing - [x] Unit tests pass - [x] Ruff check and format pass - [x] New tests added - [ ] Live OpenCode session tested ```text uv run --extra dev pytest tests/test_openai_tool_search_deferral.py tests/test_proxy_openai.py -q 57 passed uv run --extra dev ruff check headroom/proxy/handlers/openai.py headroom/proxy/helpers.py tests/test_openai_tool_search_deferral.py tests/test_proxy_openai.py All checks passed ``` ## Real Behavior Proof The focused route tests classify OpenCode from both `User-Agent` and `X-Client`, verify its tools remain untouched, and verify the decision reaches all three Responses ingresses. Supported clients continue to receive deferral. Codex remains excluded by the policy already on `main`. Not tested: a live OpenCode instance; the incompatibility itself remains based on the reporter's reproduction in #2660. ## Review Readiness - [x] Updated from current upstream `main` - [x] Merge conflicts resolved - [x] Focused tests pass locally - [x] Ready for human review ## Additional Notes No user configuration or documentation change is required. Vercel authorization failures are external integration noise, not a source check. --------- Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net> |
||
|
Abhay Singh
|
c49be269a1
|
fix(wrap): stop the launch cwd from shadowing the installed package in the proxy subprocess (#2843)
## Description
`headroom wrap` starts the proxy via `_start_proxy`, which builds `cmd =
[sys.executable, "-m", "headroom.cli", "proxy", ...]`. A `python -m
<module>` invocation prepends the launch cwd to `sys.path`. So when
`wrap` is run from a directory that contains a `headroom/` folder (most
commonly a clone of this very repo, whose package lives at
`<repo-root>/headroom/`), that raw source tree shadows the installed
wheel in site-packages. The source tree has no compiled `headroom._core`
(the maturin extension only exists in the built wheel), so the proxy
dies with:
```text
Error: Proxy dependencies not installed. Run: pip install headroom-ai[proxy]
Details: No module named 'headroom._core'
```
`wrap` then falls back to launching the client unwrapped, and the "not
installed" hint is misleading: the dependency is installed, it is being
shadowed by cwd.
The fix sets `PYTHONSAFEPATH=1` in the proxy subprocess env. That
disables the cwd/script-dir prepend to `sys.path` (Python 3.11+, and a
harmless no-op on 3.10, so it never breaks the supported floor), which
is exactly what the issue reporter confirmed resolves it:
```console
$ PYTHONSAFEPATH=1 python -c "import headroom._core; print('OK')" # -> OK
```
The proxy is still launched as `-m headroom.cli`, so nothing about the
invocation changes except that it now always resolves the installed
package.
Fixes #2793
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/wrap.py` (`_start_proxy`): set
`proxy_env["PYTHONSAFEPATH"] = "1"` alongside the existing
`PYTHONIOENCODING`, with a comment explaining the cwd-shadow failure
mode.
- `tests/test_cli/test_wrap_claude_vertex_proxy_env.py`: added
`test_start_proxy_sets_pythonsafepath_to_avoid_cwd_shadow`, which drives
`_start_proxy` with a faked `subprocess.Popen` and asserts the
subprocess env carries `PYTHONSAFEPATH=1` while still launching `-m
headroom.cli proxy`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
# Fail-before (source fix stashed, new test kept):
tests/test_cli/test_wrap_claude_vertex_proxy_env.py::test_start_proxy_sets_pythonsafepath_to_avoid_cwd_shadow FAILED
assert captured["kwargs"]["env"]["PYTHONSAFEPATH"] == "1"
KeyError: 'PYTHONSAFEPATH'
# Pass-after (fix applied):
tests/test_cli/test_wrap_claude_vertex_proxy_env.py 18 passed
# Broader wrap suites:
tests/test_cli/test_wrap_claude_vertex_proxy_env.py tests/test_cli_proxy_env.py tests/test_cli/test_wrap_persistent.py
121 passed, 1 skipped
# uvx ruff@0.15.17 check -> All checks passed!
# uvx mypy@1.20.2 headroom/cli/wrap.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.17 and mypy 1.20.2 via uvx.
- Exact command / steps: confirmed `_start_proxy` builds
`[sys.executable, "-m", "headroom.cli", "proxy", ...]` and constructs
the subprocess env as `proxy_env`, reproduced the shadowing behaviour in
the reporter's terms (`python -m` prepends cwd; a cwd `headroom/`
without `_core` shadows the wheel), fail-before with `git stash push
headroom/cli/wrap.py` and `python -m pytest ... -k pythonsafepath` (the
env lacks the key), then pass-after with `git stash pop` and rerunning
the file (18 passed) plus the broader wrap suites (121 passed, 1
skipped).
- Observed result: the proxy subprocess env now carries
`PYTHONSAFEPATH=1`, which disables the cwd prepend, so `import
headroom._core` resolves the installed wheel instead of a shadowing
local `headroom/` source tree. The proxy command is unchanged otherwise.
- Not tested: an end-to-end `cd <repo-checkout> && headroom wrap claude`
against a real installed wheel (this environment is a source checkout
without a separate installed wheel to shadow). The behaviour is verified
through the spawn env the subprocess inherits, and `PYTHONSAFEPATH` is
the documented, reporter-confirmed switch for this exact failure mode.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
Scoped to the proxy launch, which is the reported, high-impact path (its
failure makes `wrap` fall back to unwrapped). `wrap` spawns one other
`python -m headroom.*` subprocess (the memory-sync helper in the Claude
flow) that shares the same root cause; it is a lower-severity,
unreported path and is left for a follow-up rather than widening this
diff. The misleading "pip install headroom-ai[proxy]" message the
reporter also flagged is a separate error-text concern and is likewise
out of scope here.
|
||
|
Abhay Singh
|
3488f8d4b5
|
fix(install): use --userns=keep-id under Podman so bind-mount writes don't fail (#2846)
## Description
`build_runtime_command` unconditionally adds `--user <uid>:<gid>` on
non-Windows hosts:
```python
# headroom/install/runtime.py
if not _is_windows():
getuid = getattr(os, "getuid", None)
getgid = getattr(os, "getgid", None)
if callable(getuid) and callable(getgid):
command.extend(["--user", f"{getuid()}:{getgid()}"])
```
That is correct for Docker, where container UIDs equal host UIDs, but
wrong for rootless Podman, where the host user is already mapped to
container UID 0 and the `/etc/subuid` range is mapped to container UIDs
1 and above. Passing `--user $(id -u):$(id -g)` therefore selects a
container UID backed by a subordinate host UID that owns nothing. The
bind-mounted `~/.headroom` appears inside the container as `root:root`
and is unwritable, so every write fails:
```text
PermissionError: [Errno 13] Permission denied: '/tmp/headroom-home/.headroom/memories'
event=proxy_inbound_request_aborted path=/v1/messages reason=PermissionError
```
The proxy still starts and reports healthy, so the failure only surfaces
once a request touches a write path. As the reporter confirmed,
`--userns=keep-id` (or omitting `--user`) fixes it.
The fix detects Podman and uses `--userns=keep-id` instead of `--user`,
which maps the host user to the same UID inside the container and keeps
the bind mounts writable. Docker still gets `--user`, unchanged.
Detection is subprocess-free: it resolves the `docker` binary and checks
its real name for the common `docker -> podman` symlink shim (e.g. NixOS
`/run/current-system/sw/bin/docker -> podman`), with an explicit
`HEADROOM_CONTAINER_RUNTIME` (`podman` / `docker`) override for setups
the symlink heuristic cannot see, such as a wrapper script.
Fixes #2804
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/install/runtime.py`: added `_container_runtime_is_podman()`
(env override, then a `docker`-binary realpath basename check, no
subprocess). In `build_runtime_command`, when Podman is detected the
command uses `--userns=keep-id` instead of `--user <uid>:<gid>`.
- `tests/test_install/test_runtime.py`: pinned the existing docker test
to the Docker path via `HEADROOM_CONTAINER_RUNTIME=docker` and asserted
`--userns=keep-id` is absent there; added
`test_build_runtime_command_podman_uses_keep_id_not_user` asserting the
Podman path drops `--user` and adds `--userns=keep-id`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
# Fail-before (source fix stashed, new test kept):
tests/test_install/test_runtime.py::test_build_runtime_command_podman_uses_keep_id_not_user FAILED
assert "--userns=keep-id" in command
AssertionError: assert '--userns=keep-id' in ['docker', 'run', '--rm', ...]
# Pass-after (fix applied):
tests/test_install/test_runtime.py 26 passed
# Broader install suite (excluding the pre-existing env-specific PowerShell installer test):
tests/test_install/ 142 passed, 1 skipped
# uvx ruff@0.15.17 check -> All checks passed!
# uvx mypy@1.20.2 headroom/install/runtime.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.17 and mypy 1.20.2 via uvx.
- Exact command / steps: confirmed `build_runtime_command` adds `--user`
unconditionally on non-Windows, then drove both runtimes
deterministically via the `HEADROOM_CONTAINER_RUNTIME` override.
Fail-before with `git stash push headroom/install/runtime.py` and
`python -m pytest tests/test_install/test_runtime.py -k
podman_uses_keep_id` (the command still carries `--user`, no keep-id),
pass-after with `git stash pop` and rerunning the file (26 passed).
- Observed result: with Podman detected the docker command now contains
`--userns=keep-id` and no `--user`/`1000:1001`, matching the
`--userns=keep-id` invocation the reporter verified writes successfully;
with Docker it is unchanged (`--user 1000:1001`, no keep-id).
- Not tested: a live rootless-Podman deployment writing to a bind mount
(no Podman in this environment). The command construction is verified
directly, and `--userns=keep-id` is the documented, reporter-confirmed
switch for the rootless-Podman ID-mapping.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
Detection is intentionally subprocess-free and conservative: it only
diverges from today's behavior when the `docker` binary literally
resolves to a `podman`-named target, or when
`HEADROOM_CONTAINER_RUNTIME` is set. Real Docker installs are untouched.
The override also gives a clean escape hatch in both directions if a
given host's symlink layout hides the runtime. This is the `--user` half
of the persistent-docker + Podman issues; the separate host-memory-path
problem (#2803) is addressed in its own PR.
|
||
|
Abhay Singh
|
14c4c9d5b7
|
fix(install): stop baking the host memory DB path into a container deployment (#2845)
## Description `headroom deploy --memory` on the `persistent-docker` preset can never become ready. The planner resolves the memory DB path against the **host** home and appends it verbatim to `proxy_args`: ```python # headroom/install/planner.py proxy_args.extend(["--memory", "--memory-db-path", str(_paths.memory_db_path())]) # -> --memory-db-path /home/<user>/.headroom/memory.db ``` The docker runtime passes everything after the leading `--host` pair through unchanged, and the container's `HOME` is `/tmp/headroom-home` with the host's `~/.headroom` bind-mounted at `/tmp/headroom-home/.headroom`. The host path `/home/<user>/.headroom/memory.db` does not exist inside the container, so SQLite cannot open the DB: ```text Memory: backend initialization failed (startup continues): unable to open database file ``` `/health` then reports `memory.ready = false`, `/readyz` stays 503 for the full `wait_ready` window, and `_start_deployment` times out and rolls back, so the failure presents as "did not become ready" rather than a path bug. The same applies on macOS with `/Users/<user>/...`. The fix omits `--memory-db-path` for a container (docker) runtime. When the flag is absent the proxy resolves the DB under its own cwd (`.headroom/memory.db`), and the container's workdir is `/tmp/headroom-home` (the bind mount), so the DB lands in exactly the same host file the explicit path intended. The host (python) runtime still passes the resolved host path, which is correct there. Fixes #2803 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/install/planner.py` (`build_manifest`): append `--memory` always, but add `--memory-db-path <host path>` only when `runtime_kind != RuntimeKind.DOCKER.value`. Imported `RuntimeKind` from `.models`. - `tests/test_install/test_planner.py`: extended `test_build_manifest_for_persistent_docker_sets_expected_defaults` to assert `--memory-db-path` is absent for the docker runtime, and added `test_build_manifest_python_runtime_keeps_explicit_memory_db_path` asserting it is still present for the python runtime. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Fail-before (source fix stashed, updated tests kept): tests/test_install/test_planner.py::test_build_manifest_for_persistent_docker_sets_expected_defaults FAILED assert "--memory-db-path" not in manifest.proxy_args AssertionError: assert '--memory-db-path' not in ['--host', '127.0.0.1', ...] # Pass-after (fix applied): tests/test_install/test_planner.py 19 passed # Broader install suites: tests/test_install/ 141 passed, 1 skipped, 2 unrelated pre-existing/flaky failures # - test_native_installers.py::test_powershell_native_installer_supports_persistent_docker_lifecycle # runs scripts/install.ps1 and fails identically on clean main (environment-specific). # - test_runtime.py::test_runtime_status_survives_winerror87_systemerror passes in isolation # and in its own file; it only failed under cross-file ordering in the broad run, and is # untouched by this diff (planner.py only). # uvx ruff@0.15.17 check -> All checks passed! # uvx mypy@1.20.2 headroom/install/planner.py -> Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1, ruff 0.15.17 and mypy 1.20.2 via uvx. - Exact command / steps: traced the path from `planner.py` (`--memory-db-path str(_paths.memory_db_path())`, host home) through `runtime.py` (`build_runtime_command` passes `proxy_args[_PROXY_ARGS_HOST_PAIR_LEN:]` through, container HOME `/tmp/headroom-home`, `~/.headroom` bind-mounted) and confirmed via `server.py` that an empty `memory_db_path` resolves to `Path.cwd()/.headroom/memory.db` (the container workdir, hence the mount). Fail-before with `git stash push headroom/install/planner.py` and `python -m pytest tests/test_install/test_planner.py -k persistent_docker` (host path present in proxy_args), pass-after with `git stash pop` and rerunning (19 passed). - Observed result: for the docker runtime, `manifest.proxy_args` now carries `--memory` without `--memory-db-path`, so the container resolves the DB to `/tmp/headroom-home/.headroom/memory.db` (the bind mount to host `~/.headroom/memory.db`) and can open it, instead of receiving a nonexistent host path. The python runtime still carries the explicit host path. - Not tested: a live `headroom deploy --memory` against a running Docker daemon (no container runtime in this environment). The manifest construction is verified directly, and the container-side resolution it relies on is existing server behavior (`empty memory_db_path -> cwd/.headroom/memory.db`) confirmed by reading `server.py`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md`: it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes The DB persistence location is unchanged: both the old host path and the new container-cwd resolution point at the host's `~/.headroom/memory.db` (directly on the host, or through the bind mount inside the container), so existing memory DBs are picked up either way. This is the memory-path half of the persistent-docker issues; the separate rootless-Podman `--user` bind-mount problem (#2804) is left for its own fix. |
||
|
Abhay Singh
|
3808f60ca6
|
fix(proxy/anthropic): inject headroom_retrieve whenever a CCR marker is present, not only for new markers (#2848)
## Description
On a frozen-prefix turn that replays an existing `<<ccr:hash>>` marker,
the proxy did not inject the `headroom_retrieve` tool, so the agent held
a marker it could not redeem. When it tried, the Anthropic API rejected
the whole request:
```text
API Error: 400 Tool reference 'headroom_retrieve' not found in available tools
```
This was a frequent, user-visible failure in Claude Code.
### Root cause
The sticky tool-injection gate in `handlers/anthropic.py` was driven by
`has_new_ccr_markers(...)` -- markers created THIS turn only:
```python
has_new_compressed_content = has_new_ccr_markers(
current_detected_hashes=injector.detected_hashes,
previous_forwarded_messages=prefix_tracker.get_last_forwarded_messages(),
provider="anthropic",
)
tools, ccr_tool_injected = apply_session_sticky_ccr_tool(
...,
has_compressed_content_this_turn=has_new_compressed_content,
)
```
`apply_session_sticky_ccr_tool` returns early with `decision="skip"` for
a session it considers fresh when `not
has_compressed_content_this_turn`. A marker replayed from the frozen
prefix is "historical" (already in `previous_forwarded_messages`), so
`has_new_ccr_markers` returns `False`, and on a fresh session the tool
is skipped even though the request carries a redeemable marker. The
`SessionCcrTracker` is per-process, so every proxy restart makes live
sessions look fresh again and re-arms the failure mid-conversation.
Anything that instructs the model to retrieve later (a project
instruction saying "call `headroom_retrieve` with the hash before
asserting an exact value") lands on this path by construction.
### Fix
Drive the gate from `injector.has_compressed_content` -- whether the
forwarded request carries ANY CCR marker, new or replayed -- instead of
new-markers-only. `#1850` narrowed the first-time gate to new markers to
avoid arming a session that never compressed, but a present marker means
the session HAS compressed, and a replayed marker is exactly as
unredeemable as a fresh one. Since a new marker is also a present
marker, `has_new_compressed_content or injector.has_compressed_content`
collapses to `injector.has_compressed_content`, so the now-redundant
`has_new_ccr_markers` call is removed.
The cache argument cuts in favor of this: toggling the tool in and out
of the tools array between turns is what busts the tools cache segment.
Injecting consistently whenever markers exist is the cache-stable
option, and it removes a hard 400 in exchange for at most one cache
miss. The frozen message prefix is still replayed byte-identical, so the
prompt-cache prefix is unaffected; only the tools array gains a stable
entry.
Fixes #2766
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py`: the sticky CCR tool-injection
gate now passes
`has_compressed_content_this_turn=injector.has_compressed_content` (any
marker present) instead of the new-markers-only signal, and the
now-redundant `has_new_ccr_markers` computation/import is dropped.
- `tests/test_proxy/test_anthropic_ccr_deferred_injection.py`: the two
tests that encoded the superseded `#1850` behavior (a replayed
historical marker forwarded WITHOUT the tool) now assert the tool IS
injected, with updated rationale. One was renamed from
`..._when_tool_injection_is_deferred` to
`..._and_injects_retrieve_tool`. The byte-identical message-prefix
replay assertions are unchanged.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
# Fail-before (source fix stashed, updated tests kept):
tests/test_proxy/test_anthropic_ccr_deferred_injection.py
::test_cache_mode_compresses_delta_but_replays_cached_prefix_when_markers_are_historical FAILED
::test_cache_mode_exact_prefix_replay_forwards_cached_compressed_prefix_and_injects_retrieve_tool FAILED
assert [tool["name"] for tool in forwarded["tools"]] == ["headroom_retrieve"]
KeyError: 'tools'
# Pass-after (fix applied):
tests/test_proxy/test_anthropic_ccr_deferred_injection.py 15 passed
# Broader CCR suites:
tests/test_proxy/test_ccr_frozen_prefix_coupling.py tests/test_ccr_tool_always_on.py
tests/test_ccr_session_tracker.py tests/test_ccr_tool_injection.py 61 passed
tests/test_ccr_marker_policy.py tests/test_anthropic_ccr_workspace_unbound.py
tests/test_ccr_tool_calls.py tests/test_corrupt_golden_bytes_recovery.py 21 passed
# uvx ruff@0.15.17 check -> All checks passed!
# uvx mypy@1.20.2 headroom/proxy/handlers/anthropic.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.17 and mypy 1.20.2 via uvx.
- Exact command / steps: traced the gate (`has_new_ccr_markers` ->
`apply_session_sticky_ccr_tool` fresh-session `skip`) and confirmed
`injector.has_compressed_content` reflects any marker present in the
forwarded messages (`len(_detected_hashes) > 0` after
`scan_for_markers`). Reproduced the exact bug in the handler harness: a
cache-mode frozen replay where `fake_tracker._last_forwarded_messages`
already holds the marker (so `has_new` is `False`) on a session the
reset tracker considers fresh, with the marker forwarded to upstream.
Fail-before with `git stash push headroom/proxy/handlers/anthropic.py`
and rerunning the two replay tests (the forwarded body has no `tools`),
pass-after with `git stash pop` (the body carries `headroom_retrieve`).
- Observed result: on a replayed-marker turn the forwarded request now
includes `"tools": [{"name": "headroom_retrieve", ...}]`, so the agent
can redeem the hash and Anthropic no longer 400s. The frozen message
prefix is still replayed byte-identical (`forwarded["messages"]`
unchanged). Sessions that never compressed still get no tool (no marker
-> `has_compressed_content` is `False`).
- Not tested: a live multi-turn Claude Code session across a real proxy
restart (no live provider here). The gate is exercised end-to-end
through the handler via the TestClient harness, reproducing the
historical-marker-on-fresh-session desync the issue describes.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
This deliberately reworks the `#1850` deferral for historical markers,
so it changes two tests that encoded "tool absent on frozen replay."
That behavior was the source of the 400: a marker in the prompt with no
tool to redeem it is a hard failure, whereas a re-injected tool is a
stable, cheap entry in the tools array. The reporter validated the same
change locally (33 requests, 0 errors, 0 `skip`). Scope is the Anthropic
interactive path where the bug was reported; the stateless batch path (a
separate `CCRToolInjector.process_request` gated on `tokens_saved > 0`)
is unchanged.
|
||
|
Abhay Singh
|
1f5fefffd3
|
fix(memory): bound the TrafficLearner pending-pattern accumulator (memory leak) (#2579)
## Description
`TrafficLearner` (the memory/learning subsystem that accumulates
patterns from proxy traffic) has an unbounded in-memory accumulator.
`_pattern_counts` maps `content_hash -> (pattern, count)`. A pattern is
added on first sighting, its count is bumped on each re-sighting, and it
is **removed only when it reaches `min_evidence`** (default 5), at which
point it is promoted and its hash moves to `_saved_hashes`:
```python
if h in self._pattern_counts:
existing, count = self._pattern_counts[h]
count += 1
self._pattern_counts[h] = (existing, count)
else:
self._pattern_counts[h] = (pattern, 1)
return # first sighting — wait for more evidence
...
if count >= self._min_evidence:
del self._pattern_counts[h] # only removal path
self._saved_hashes.add(h)
if len(self._saved_hashes) > self._dedup_window: # sibling IS trimmed
self._saved_hashes.pop()
```
A pattern seen **once but never corroborated** — the common case for
one-off traffic (a unique error string, an ad-hoc shell command, a
distinct file path) — never reaches `min_evidence`, so it is **never
removed**. Over a long-lived proxy processing varied traffic,
`_pattern_counts` grows without bound and RSS climbs. The sibling
`_saved_hashes` is explicitly trimmed to `dedup_window` ("prevent
unbounded growth"); `_pattern_counts` was missed.
Reproduced directly: feeding 500 distinct one-off patterns leaves 500
entries in `_pattern_counts` (one per pattern, forever).
## Fix
Make `_pattern_counts` an LRU-ordered `OrderedDict` capped at a new
`max_pending_patterns` (default 2048):
- On each corroboration, `move_to_end(h)` so an actively-accumulating
pattern stays "fresh" and is never evicted before it can be promoted.
- On a first sighting when the accumulator is full, evict the
least-recently-corroborated pending entry (`popitem(last=False)`).
Evicting a stale one-off is safe: if it recurs it simply restarts
accumulation (delayed promotion at worst) — the same tradeoff
`_saved_hashes` already makes. Promotion at `min_evidence` is unchanged,
and the cap (2048) is generous enough that any pattern receiving repeat
sightings within a normal window reaches `min_evidence=5` long before
eviction.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
## Changes Made
- `headroom/memory/traffic_learner.py`: `_pattern_counts` becomes a
capped LRU `OrderedDict`; add `max_pending_patterns` (default 2048);
`move_to_end` on corroboration and evict-oldest on overflow.
- `tests/test_memory/test_traffic_learner.py`: a regression that 500
one-off patterns keep the accumulator at its cap, and one that a
corroborated pattern still promotes into `_saved_hashes` (both sync via
`asyncio.run` so they run without the pytest-asyncio plugin).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_memory/test_traffic_learner.py -q
35 failed, 109 passed
# the 35 failures are pre-existing @pytest.mark.asyncio tests that need
# pytest-asyncio (not configured in this environment); they fail identically
# on clean main (35 failed, 107 passed) and pass in CI. My two new tests are
# synchronous and pass; they add +2 passing with no new failures.
# with the fix reverted, test_pending_accumulator_is_bounded fails
# (the accumulator holds all 500 one-off patterns)
$ uvx ruff@0.15.17 check headroom/memory/traffic_learner.py tests/test_memory/test_traffic_learner.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/memory/traffic_learner.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: built a `TrafficLearner(backend=None,
min_evidence=5, max_pending_patterns=8)` and drove `_accumulate` with
500 distinct one-off `ExtractedPattern`s; separately corroborated one
pattern to `min_evidence`; then reverted the source and re-ran.
- Observed result: with the fix `len(_pattern_counts)` stays at the cap
(8) after 500 one-offs, the corroborated pattern is removed from pending
and present in `_saved_hashes`, and an actively-bumped pattern survives
LRU eviction; with the fix reverted the accumulator holds all 500
one-off entries (the unbounded leak). Ran against the actual module.
- Not tested: a live multi-day proxy run measuring RSS (the leak is
inferred from the removed unbounded-growth path; the accumulator bound
is verified directly).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Abhay Singh
|
b97c7c6e99
|
fix(proxy/gemini): keep streaming-parity baseline so eligible_pct can't exceed 100 (#2824)
## Description The non-streaming Gemini `generateContent` finalizer builds its `RequestOutcome` with `optimized_tokens` set to Gemini's own `promptTokenCount` (the provider's tokenizer scale, which correctly feeds billing and the dashboard), while `original_tokens` stays a local estimator count. Those two are on different rulers. Every delta the beacon derives from the pair is a same-ruler difference: `tokens_saved`, `tokens_inflated`, `attempted_input_tokens`, and the beacon's `eligible_pct` / `yield_pct`. When Gemini counts the forwarded prompt higher than our local estimator does, `attempted_input_tokens` (which is `optimized_tokens + tokens_saved`) exceeds the local `original_tokens`, and the request ships a structurally-impossible `eligible_pct > 100` plus a phantom `tokens_inflated`. This is the exact class of bug #2756 removed, on a path #2756 did not touch: it fixed the non-streaming OpenAI handler, and the streaming finalizer (`_finalize_stream_response`) already guards against it by lifting the baseline onto the provider scale. The non-streaming Gemini path had neither treatment. The fix mirrors the streaming finalizer's already-tested handling: when a provider count is present, lift the baseline to `max(original_tokens, promptTokenCount + tokens_saved)` so `attempted_input_tokens <= original_tokens` holds and `tokens_inflated` collapses to 0. It is guarded on a present count, so a null or absent `promptTokenCount` leaves the local baseline untouched and the existing zero-usage preservation test still holds. `optimized_tokens` still carries the provider count, so billing and the dashboard are unchanged. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/gemini.py` (`handle_gemini_request`, non-streaming `generateContent` branch): compute `effective_original_tokens = max(original_tokens, total_input_tokens + tokens_saved)` when `total_input_tokens > 0` (else keep `original_tokens`), and pass it as the outcome's `original_tokens`. Mirrors the streaming finalizer's provider-usage handling. - `tests/test_proxy/test_gemini_savings_profile.py`: added `test_gemini_provider_count_above_local_estimate_does_not_inflate_eligible`, which drives a request where Gemini's `promptTokenCount` (150) exceeds the local post-compression count (80), and asserts `attempted_input_tokens <= original_tokens`, `tokens_inflated == 0`, the provider count is still carried in `optimized_tokens`, and the baseline is lifted to 170. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Fail-before (source fix stashed, new test kept): tests/test_proxy/test_gemini_savings_profile.py::test_gemini_provider_count_above_local_estimate_does_not_inflate_eligible FAILED assert outcome.attempted_input_tokens <= outcome.original_tokens AssertionError: assert 170 <= 100 # Pass-after (fix applied): tests/test_proxy/test_gemini_savings_profile.py::test_gemini_provider_count_above_local_estimate_does_not_inflate_eligible PASSED # Full file + related outcome suites: tests/test_proxy/test_gemini_savings_profile.py tests/test_proxy_gemini_native_integration.py tests/test_request_outcome.py tests/test_outcome_token_scale.py 47 passed, 18 skipped # uvx ruff@0.15.17 check -> All checks passed! # uvx mypy@1.20.2 headroom/proxy/handlers/gemini.py -> Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, project venv (litellm installed), pytest 9.1.1 with pytest-asyncio 1.4.0 (asyncio_mode=auto per pyproject), ruff 0.15.17 and mypy 1.20.2 via uvx. - Exact command / steps: confirmed the streaming sibling already lifts the baseline (`_finalize_stream_response` in `headroom/proxy/handlers/streaming.py` sets `effective_original_tokens = max(original_tokens, provider_input_tokens + tokens_saved)` for openai/gemini), then fail-before with `git stash push headroom/proxy/handlers/gemini.py` and `python -m pytest tests/test_proxy/test_gemini_savings_profile.py -k inflate_eligible` (the assertion fails with `170 <= 100`, i.e. eligible_pct 170%), then pass-after with `git stash pop` and rerunning (passes), then the full file plus the outcome suites (47 passed, 18 skipped). - Observed result: with Gemini reporting `promptTokenCount=150` against a local post-compression count of 80 (saved 20), the outcome now reports `original_tokens=170`, `attempted_input_tokens=170` (so `eligible_pct <= 100`) and `tokens_inflated=0`, while `optimized_tokens` stays 150 so billing and the dashboard are unchanged. Before the fix the same request reported `original_tokens=100`, `attempted_input_tokens=170` (eligible_pct 170%) and `tokens_inflated=50`. - Not tested: a live streamed call to real Gemini/Vertex (no provider credentials in this environment). The provider-count-above-local case is reproduced with a mock response mirroring Gemini's `usageMetadata` shape, and the baseline-lift it mirrors is existing, tested code on the streaming path. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md`: it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Docs and manual testing are N/A: this aligns the non-streaming Gemini finalizer with the already-correct streaming finalizer, no API surface change. The baseline lift is guarded on a present provider count, so the existing zero-usage preservation test (`test_gemini_zero_usage_prompt_count_is_preserved`) is unaffected: a null or zero `promptTokenCount` keeps the local baseline and leaves `optimized_tokens` at 0. |
||
|
JD Davis
|
01161fe019
|
test(openclaw): match inherited PATH shell check (#2821)
## Description
Fix the OpenClaw test failure on `main` by aligning its PATH-launcher
expectation with the intentionally shipped `sh -c` behavior from #1459.
The non-login shell preserves the PATH inherited from the OpenClaw
process. Changing production code back to `sh -lc` would risk a login
shell resetting that PATH and would undo the compatibility fix. This PR
therefore corrects only the stale assertion; runtime behavior and
defaults do not change.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Expect `sh -c` for the non-Windows lightweight `command -v headroom`
check.
- Preserve the existing Windows `where.exe` behavior and all launcher
behavior.
## Testing
- [x] Unit tests pass (`npm test`)
- [x] Linting passes (`npm run typecheck`)
- [x] Type checking passes (`npm run typecheck`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ npm test
Test Files 6 passed (6)
Tests 75 passed (75)
$ npm run typecheck
> tsc --noEmit
$ npm run build
ESM Build success
DTS Build success
$ npm ci
found 0 vulnerabilities
```
## Real Behavior Proof
- Environment: macOS, Node/npm, clean install from `origin/main` at
`
|
||
|
Ashish Patel
|
4ec416df88
|
fix(proxy): stop discarding compressed Codex WS later-frame payloads (#2823)
## Description
`headroom perf` reports 0 tokens saved for Codex CLI sessions despite
real traffic being processed (confirmed via the reporter's live proxy
stats in the issue). Root cause: a misplaced `return` statement in the
Codex WS later-frame compression path silently discards every compressed
payload and skips all token/savings bookkeeping for it.
Closes #2819
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/openai.py`
(`_maybe_compress_response_create_frame`): PR #1579 (2026-07-16) moved a
`return (raw_after_store, ...)` statement to the same indentation as the
enclosing `except Exception:` block instead of inside it. That made the
`return` fire **unconditionally** after every later (2nd+)
`response.create` frame in a Codex WS session — success or failure —
always forwarding the original pre-compression frame upstream and
skipping the entire success-path code below it (correct
rewritten-payload return, `tokens_saved`,
`attempted_input_tokens_total`, `ws_frames_compressed`). Fixed by moving
the `return` back inside the `except` block, restoring the success path.
- `tests/test_openai_codex_ws_lifecycle.py`: new regression test
`test_ws_later_frame_compression_is_actually_forwarded` — mocks the
compressor to report `modified=True` with a distinct rewritten payload
on a later frame, asserts the rewritten payload (not the original) is
what's actually sent upstream.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`) — not run locally, will confirm
via CI
- [ ] Type checking passes (`mypy headroom`) — not run locally, will
confirm via CI
- [x] New tests added for new functionality
- [x] Manual testing performed (see Real Behavior Proof)
### Test Output
```text
$ .venv/Scripts/python -m pytest tests/test_openai_codex_ws_lifecycle.py tests/test_codex_ws_per_frame_memory.py tests/test_codex_ws_savings_deferral.py tests/test_openai_codex_ws_timings.py -q
............................................... 48 passed in 5.34s
$ .venv/Scripts/python -m pytest tests/ -k "openai or codex" -q (wider sweep, unrelated dirs excluded)
968 passed, 3 failed, 77 skipped, 2 errors in 483.21s
```
The 3 failures
(`test_client_integration.py::test_auto_detect_openai_optimizer`,
`test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4]`,
`test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline`)
reproduce identically on a clean, unmodified `main` — confirmed by
stashing this PR's changes and re-running. They're local-environment
issues (a live litellm 503, and this dev box's tool registry missing a
`Bash` entry), not caused by this change.
## Real Behavior Proof
- Environment: Windows 11, Python 3.14.5, local venv, `headroom._core`
rebuilt via `maturin develop --release` against current `main` to rule
out stale-build noise
- Exact command / steps: (1) `git blame` on the buggy block traced the
misplaced `return` to commit `
|
||
|
Tejas Chopra
|
53af90d68c
|
perf: cut hot-path latency 27% (token-count memo, startup preloads, JSON scan memo) (#2838)
## Description
Four independent latency fixes on the request hot path, found by
profiling and each measured in isolation. No behaviour changes: every
commit is either a memo of a pure function, work moved to startup, or
work that was computed and discarded.
**End to end: 287ms → 210ms (−27%) on a 68k-token mixed payload, with
byte-identical output** (68,514 → 48,725 tokens both before and after).
Plus one-off costs removed that don't show in steady-state numbers:
~4.9s of lazy imports that were firing *inside* user requests, and
~750ms of HuggingFace round-trips per process start.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
**1. Memoise `count_text` (`ac369277`)** — tiktoken's `CoreBPE.encode`
was 0.243s of a 0.30s profiled request. It dominates because the same
string is counted repeatedly: a 103KB payload drove 600KB of encoding,
~6x the content, across six call sites (`tokenizers/base.py:196`,
`content_router.py:4704` and `:5474`, `parser.py:185/192/298`). 35% of
encode calls and 22% of encoded characters were an exact repeat *within
one request*.
`count_text` is a pure function of its text, so replaying a stored count
returns the same integer. That is the whole safety argument, and it is
what makes this safe at the sites whose count feeds a routing decision
(`context_pressure` → `min_ratio`) rather than a log line — an
*estimate* there would change which blocks compress; a memo cannot.
Keyed on the text itself, not a hash: a collision would hand back a
wrong count for real content and silently change compression. The cost
is holding the strings, so entries and total characters are both capped.
Clear-on-full rather than LRU eviction — the pipeline runs on a thread
pool, `dict` get/set/clear are atomic under the GIL but
`OrderedDict.move_to_end` is not.
**2. Preload what was importing mid-request (`2921a15b`)** — `litellm`
(2.9–3.8s) was imported lazily *on the event loop* during the first
request: `emit_request_outcome` → `record_request` →
`_estimate_compression_savings_usd` calls the loader before its own
`tokens_saved <= 0` early return, so even a request that saved nothing
paid it. `trafilatura` (978ms, pulling `htmldate` → `dateparser` and its
timezone tables) is the most expensive lazy import in the transform tree
— every other compressor module is 1–20ms — and fires on the first
request carrying an HTML-ish or mixed-content block. The TOIN singleton
reads ~5MB of learned patterns on construction (~150ms); a stale comment
claimed the SmartCrusher preload covered it, and it does not.
All three now load in `_eager_preload_transforms`, which already runs
under `asyncio.to_thread` and so cannot delay the port bind.
Same commit, two Kompress cold-path fixes: `_load_modernbert_tokenizer`
always used `local_files_only=False`, which makes transformers
re-validate against the Hub on every load — a tree listing plus a HEAD
per file — even when fully cached (~900ms warm-cache vs ~150ms
local-only). And `ensure_background_download` re-spawned a
finished-or-failed thread on the next call, so an unreachable Hub meant
one fresh download thread *per request* for the life of the process,
each importing transformers and holding the GIL against the event loop.
Consecutive failures now back off; success clears it, so the happy path
and the transient-failure path are unchanged.
**3. Memoise the JSON-block scan (`039c9735`)** —
`_has_valid_json_block_with_text` tries every `{`/`[`-leading line as a
possible block start. When a candidate never balances,
`_extract_json_block` scans character-by-character to the end of the
content and returns nothing — then the next candidate does it again.
Quadratic, on the request path, growing exactly 4x per doubling.
**4. `CostTracker.totals()` (`286b97e4`)** —
`_current_savings_tracker_totals` called `stats()` once per request and
read two of its fields. Building the rest includes
`period_cost_breakdown()`, which walks up to 100k cost records over 31
days, on the event loop, holding the metrics lock. It degrades with
proxy **uptime**, not load, which is why no short benchmark would
surface it.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ ruff check headroom/ tests/ --exclude headroom/dashboard/templates
All checks passed!
$ mypy --python-version 3.12 headroom/
Found 1 error in 1 file (checked 515 source files)
headroom/release_version.py:235: error: Name "tomllib" already defined (by an import)
# pre-existing on main, in a file this PR does not touch — verified by
# running the same command on a clean main checkout.
$ python -m pytest tests/test_token_count_cache.py tests/test_mixed_content_scan_cache.py \
tests/test_kompress_download_backoff.py tests/test_cost_tracker_totals.py -q
306 passed
$ python -m pytest tests/ -q -k "token or tokenizer or count or estimator or provider"
1303 passed, 105 skipped in 423.42s
$ python -m pytest tests/ -q -k "cost or budget or metrics or savings or stats"
683 passed, 127 skipped, 1 failed
# tests/test_proxy_memory_integration.py::TestMemoryStats::test_health_endpoint_works_with_memory
# Order-dependent and pre-existing: it SKIPS in isolation, and fails identically
# on a clean main checkout under the same -k selection (681 passed, 1 failed).
```
## Real Behavior Proof
- **Environment:** macOS, Python 3.12.6, local CPU, remote Kompress
disabled. Profiled with `cProfile` on `anthropic_pipeline.apply`.
- **Exact command / steps:** a 68k-token payload of four `tool_result`
blocks (900-item pretty JSON, 60KB of Python source, 500 lines of
JS-style object logs, 500 plain log lines), six reps, **content unique
per rep so every run is router-cache-cold**, run on this branch and on
main in alternation.
- **Observed result:**
| | median | min | tokens |
|---|---|---|---|
| main | 287ms | 286ms | 68,514 → 48,725 |
| this branch | 210ms | 208ms | 68,514 → 48,725 |
Per-change, measured in isolation:
| change | before | after |
|---|---|---|
| `count_text` memo | — | −25% pipeline wall; 44% of counted chars from
cache on new content, 100% when history repeats |
| litellm / trafilatura / TOIN | 3829 / 978 / 150ms mid-request | at
startup, off the event loop |
| Kompress tokenizer | ~900ms | ~150ms |
| JS-style object logs (1200 lines) | 4643ms | 183ms |
| truncated JSONL (1200 lines) | 3737ms | 116ms |
| `cost_tracker` per request | 2.8ms @20k records, 13.6ms @100k | loop
over models, not records |
Output equality: 18/18 payloads byte-identical on `tokens_before`,
`tokens_after` and a sha256 of the resulting messages, with the memo
forced on vs off.
- **Not tested:** Windows and Linux (the ORT dylib and CPU-arena paths
differ); multi-worker deployments; a proxy with a genuinely large live
cost ledger (the 100k figure is from a synthetic ledger); real
HTML-heavy traffic through the preloaded trafilatura path.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
**Docs:** N/A — no user-facing surface changes. The reasoning lives in
the code, at the sites where someone debugging would look.
**A regression I introduced and caught.** The scan memo initially made
pretty-printed JSON ~2x **slower**: content that balances on the first
scan has nothing to reuse and just pays the per-line dict traffic. The
cache is now built only *after* a scan has run to the end without
balancing, which is the actual signal that later candidates will re-walk
the same tail. Every shape now improves and none regress:
```
before after
js object logs 4642.9ms 182.7ms 25x
JSONL truncated 3736.8ms 115.9ms 32x
pretty JSON 5.6ms 3.5ms
JSONL valid 5.6ms 3.2ms
plain logs 1.0ms 0.6ms
python source 1.0ms 0.5ms
markdown prose 0.9ms 0.5ms
```
Worth stating plainly: had I only benchmarked the shape I was fixing,
I'd have shipped a win on rare content and a loss on the common case.
**The scan fix is constant-factor, not asymptotic.** The walk over
remaining lines is still O(candidates × lines), so 3200 lines of the
pathological shape is still ~1.4s. The tests assert scan-call counts
rather than implying linearity. True linearity needs a prefix-sum
rewrite with a string-state fallback; that seemed like the wrong risk
for this PR.
**How the parser change is proven safe.** `_extract_json_block` is a
parser, so golden values would only encode whatever the new code does.
Instead the pre-memo implementation is kept verbatim in the test file as
an oracle, and every candidate index of a 139-document corpus — escapes,
unterminated strings, delimiters inside strings, code fences, truncated
JSON, randomised mixtures — is asserted equal, with a cold cache, with
the shared cache the real callers use, and replayed.
**Measurement trap, for anyone re-running these numbers.** Give each arm
its own content. Reusing one payload across arms lets the second arm hit
the router's result cache, which reads as a speedup having nothing to do
with the change under test. I hit this twice while working on it: it
manufactured a fake "INFO logging costs 21.8%" finding (real answer:
0.3%) and it *understated* the memo win.
**Deliberately not in this PR:**
- **ONNX thread tuning** — measured zero gain, and
`intra_op_num_threads` is not bitwise-safe (1.6e-05 score drift from
float reduction order), so it would trade an output risk for nothing.
- **`str(content)` on block lists** counts a base64 image at 210,775
tokens instead of 1,604 (131x), pinning `context_pressure` to 1.0 and
forcing the most aggressive `min_ratio` on any conversation containing
an image. Real bug, but fixing it changes compression output — needs its
own reviewed behaviour-change PR.
- **`chunk_words=350` against the tokenizer's 512-token limit** silently
drops roughly a third of every full chunk (measured: 240/240 words kept
in the first 240, 15/110 in the tail). That is data loss rather than
latency, it changes every output, and correcting it costs ~1.3x latency.
Filing separately.
- **Telemetry off the request thread** — the TOIN auto-save is a 236ms
inline stall every 600s and the waste-signal re-parse is ~50ms/request
that is invisible in `pipeline_total` (computed before it). Both want
deferral rather than removal, which is a larger change than belongs
here.
|
||
|
Tejas Chopra
|
564e0a8d0f
|
fix(deps): bump h2 to 4.4.1 for CVE-2026-71554 (#2839)
## Description
`pip-audit` is currently red on every open PR. Not because of anything
in those branches — `uv.lock` pins `h2` at 4.3.0, and CVE-2026-71554 was
published against `h2 <=4.4.0`.
> h2 <=4.4.0 accepts request header blocks containing more than one Host
header, and forwards every Host header to the consuming application.
Where the consumer downgrades HTTP/2 to HTTP/1.1, the resulting request
carries two Host header lines, which is a request smuggling primitive
(CWE-444).
Fixed in 4.4.1.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `uv lock --upgrade-package h2`, which moves exactly two packages:
```
h2 4.3.0 -> 4.4.1
hpack 4.1.0 -> 4.2.0
```
`h2` arrives transitively via `httpx[http2]`, and the constraint in
`pyproject.toml` is already wide enough (`>=3,<5`), so only the lock
needed to move — no source or `pyproject.toml` change.
`requirements-prod.txt` is not checked in; the audit workflow exports it
from `uv.lock` at run time, so the lock bump is the entire fix.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
### Test Output
Reproduced the CI gate locally with the exact command from
`.github/workflows/security.yml`:
```text
$ uv export --frozen --no-dev --no-emit-project --no-hashes \
--extra all --format requirements-txt > requirements-prod.txt
$ grep -E '^(h2|hpack)==' requirements-prod.txt
h2==4.4.1
hpack==4.2.0
$ pip-audit -r requirements-prod.txt
No known vulnerabilities found
```
Before this change, the same command reported:
```text
Name | Version | ID | Fix Versions
h2 | 4.3.0 | CVE-2026-71554 | 4.4.1
Found 1 known vulnerability in 1 package
```
## Real Behavior Proof
- **Environment:** macOS, uv 0.9.x, Python 3.12.6.
- **Exact command / steps:** `uv lock --upgrade-package h2 --dry-run` to
confirm the blast radius, then the real lock, then the workflow's own
export + `pip-audit` invocation.
- **Observed result:** resolution touches only `h2` and `hpack`; 269
packages resolved with no other version movement. `pip-audit` goes from
1 known vulnerability to none.
- **Not tested:** HTTP/2 traffic against a live upstream. `h2` 4.4.1 is
a patch release on a library used transitively by `httpx`; Headroom does
not import `h2` directly (`grep -rn "import h2" headroom/` is empty), so
the exposure is whatever `httpx[http2]` does with it.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
N/A items above: no code changed, so ruff/mypy/new tests do not apply —
the verification that matters is the audit output, which is quoted in
full.
**Why this is standalone.** It surfaced while fixing CI on #2838, but it
is not caused by that branch and it blocks #2832 identically. Landing it
separately unblocks the gate for every open PR at once and keeps a
supply-chain bump out of an unrelated change.
**One unrelated warning the resolver prints**, noted so it is not
mistaken for a side effect of this PR:
```
warning: `pypdfium2==5.12.0` is yanked (reason: "Setup blunder breaking some
bindgen codepaths ... Wheels are valid and effectively identical to 5.12.1")
```
That predates this change and is not touched by it. Worth its own bump,
but not here.
|
||
|
Tejas Chopra
|
7940c05ebf
|
feat(beacon): allowlist the routing summary key (#2818)
One line in the receiver's allowlist. No client change; the proxy's own
payload is untouched.
## Why
A routing extension sees things the proxy alone cannot, and they are all
measurements rather than opinions:
- **Empirical `min_cacheable` per provider.** Fireworks, Together and
DeepInfra publish no minimum and litellm carries no value for them, so a
router has to guess. But the number is directly observable — send prefix
length L, see whether the repeat reports cached tokens. Across enough
installs the step function falls out.
- **TTL survival.** Currently modelled as a constant.
- **Conversation length distribution.** The horizon is the only free
parameter in a cache-aware cost model, and it decides the answer: at a
900-token prefix, 1 remaining turn and 20 remaining turns route to
different models.
- **Predicted vs actual cache hits.** Every response carries
`cache_read_input_tokens`. Comparing it to what was predicted is the
only way to find out when the cost model is lying.
## What lands here
`'routing'` added to `ALLOWED_KEYS`, and the comment above the list
corrected — it claimed the set mirrors `_Session.payload()`, which is no
longer the whole story now that an extension can emit its own event
carrying one of these keys.
The ordering constraint is the reason this is its own PR: **allowlisting
is a write-side gate**, so anything sent before the key exists is
dropped and unrecoverable. This has to be deployed before any client
starts emitting it, not alongside.
## Shape of the block
Same rule as every other key — counters and model ids, no free text:
```json
"routing": {
"harness": "claude-code",
"decisions": 47, "would_change": 12, "enforced": 9, "holdout": 3,
"at_free_boundary": 4, "cross_protocol": 0,
"picked": {"claude-haiku-4-5": 12, "claude-opus-5": 35},
"requested": {"claude-opus-5": 47},
"mean_prefix_tokens": 7514,
"measured_cost": 0.0236, "modelled_cost": 0.0376,
"cache_read_tokens": 3200, "cache_write_tokens": 0,
"predicted_hits": 4, "actual_hits": 4
}
```
`measured_cost` comes from the provider's own usage; `modelled_cost`
from the router's cost function. They stay separate because the
difference is the only thing that means anything.
The extension's `reason` string is deliberately absent. It is
code-generated, so it carries no user content, but it is unbounded — it
stays out rather than being reasoned about.
`holdout` is the count of turns deliberately left unrouted as a control.
Without it the rest is observational: once a router is acting on every
request, the corpus is entirely that router's own policy.
`sample-event.json` is unchanged on purpose — it mirrors
`_Session.payload()`, which does not produce this key, and adding it
there would suggest the proxy emits it.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Patrick A
|
17cdb185bc
|
fix(proxy): graceful shutdown and reliable Ctrl+C exit (#621)
## Problems
### 1. Noisy CancelledError traceback on Ctrl+C
Every Ctrl+C produced one or more "Exception in ASGI application" ERROR
log entries with a CancelledError traceback:
```
ERROR: Exception in ASGI application
Traceback (most recent call last):
...
File "uvicorn/protocols/http/h11_impl.py", line 410, in run_asgi
result = await app(...)
...
asyncio.exceptions.CancelledError
```
### 2. Inconsistent / hung shutdown in multi-worker mode (`--workers 8`)
Workers blocked in a C-extension call (hnswlib, tree-sitter, ONNX
inference) could prevent Ctrl+C from completing because
`timeout_graceful_shutdown` defaulted to `None` (wait forever).
---
## Root causes
**Root cause A (CancelledError noise)**
uvicorn 0.40.0's `h11_impl.run_asgi()` (line 413) catches
`BaseException` — not just `Exception` — so `asyncio.CancelledError`
raised on every in-flight request at shutdown is unconditionally logged
as `ERROR: Exception in ASGI application`. This is expected behaviour
during shutdown, not a bug.
**Root cause B (hung multi-worker shutdown)**
`uvicorn.run()` was called without `timeout_graceful_shutdown`, which
defaults to `None`. This means the supervisor waits indefinitely for
in-flight requests to drain. A single request blocked in a C-extension
(e.g. hnswlib nearest-neighbour search, tree-sitter parse, ONNX
inference) prevents the whole process group from exiting.
**Root cause C (hung single-worker shutdown — lifespan unbounded
awaits)**
The lifespan `finally` block contained unbounded `await` calls to
`_beacon.stop()`, `proxy.usage_reporter.stop()`,
`proxy.traffic_learner.stop()`, and `proxy.shutdown()`. uvicorn's
`lifespan.shutdown()` calls `await self.shutdown_event.wait()` with no
timeout — that event is only set once the lifespan `finally` block
returns. Any of these awaits hanging (e.g. a reporter making a network
call) therefore requires a second Ctrl+C to force-exit.
---
## Changes
### `headroom/proxy/server.py`
1. **`_SuppressCancelledErrorFilter`** (new class, ~10 lines): a
`logging.Filter` that returns `False` for ERROR records on
`uvicorn.error` whose `exc_info[0]` is a subclass of
`asyncio.CancelledError`. Installed on
`logging.getLogger("uvicorn.error")` at the start of `run_server()`.
2. **`timeout_graceful_shutdown=10`** added to `uvicorn.run()`: forces
cancellation of any tasks still running 10 seconds after the shutdown
signal, ensuring workers blocked in C-extensions are reaped promptly.
3. **Bounded awaits in lifespan `finally` block**: a local `_timed(coro,
label, timeout)` helper wraps each shutdown step with
`asyncio.wait_for()`. Timeouts: beacon.stop 3s, usage_reporter.stop 3s,
traffic_learner.stop 3s, proxy.shutdown 5s. Each step logs a warning on
timeout/error and continues — the teardown path is now deterministic and
completes within ~15s on a single Ctrl+C.
4. **Shutdown log message** in the lifespan `finally` block:
`event=proxy_shutdown reason=signal pid=<n>` is logged as the first
action on teardown.
### `tests/test_graceful_shutdown.py` (new)
9 tests:
- 6 unit tests for `_SuppressCancelledErrorFilter` (suppresses
CancelledError at ERROR level, passes through WARNING-level
CancelledError, passes through other exceptions, handles
`exc_info=None`, handles `(None,None,None)` tuple, suppresses
subclasses)
- 1 integration test: `run_server()` installs the filter on
`uvicorn.error`
- 1 integration test: `run_server()` passes
`timeout_graceful_shutdown=10` to `uvicorn.run()`
- 1 integration test: lifespan emits `event=proxy_shutdown` on teardown
---
## Files changed
- `headroom/proxy/server.py` — filter class, bounded lifespan awaits,
graceful shutdown timeout
- `tests/test_graceful_shutdown.py` (new) — 9 tests
- `uv.lock` — dependency lockfile updated (routine sync, no dependency
changes)
- `CHANGELOG.md` — changelog entry
---
## How to verify
1. Start the proxy: `headroom proxy --port 8787 --workers 8 --memory
--code-aware ...`
2. Press Ctrl+C
3. Before: ERROR traceback for each in-flight request; second Ctrl+C
sometimes required
4. After: clean `event=proxy_shutdown reason=signal pid=...` log, then
process exits within ~15s regardless of stuck C-extensions or slow
reporters
---------
Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
|
||
|
Tejas Chopra
|
2954e37048
|
fix(beacon): split session failures by status code (#2815)
## Description
The session beacon reports `failures` as a single count, incremented
whenever a turn ends `>= 500` (`headroom/telemetry/session.py`). Across
the current corpus that reads **3,969 failures on 595,445 turns
(0.67%)** — and the number cannot answer the only question anyone asks
of it: an Anthropic `529` is the provider shedding load and there is
nothing to fix; a `500` is usually ours. Today the two are
indistinguishable, so diagnosis falls back to inference from time-of-day
curves and per-install concentration.
This counts the status alongside the total.
```json
"failures": 3,
"failure_statuses": {"529": 2, "500": 1}
```
Motivating investigation on the live corpus (0.67% of turns, 6% of
sessions, 63% of all failures from 48 installs, a 2.5% plateau at 08–11
UTC decaying to 0.03% during the fleet's busiest hour) strongly suggests
provider-side 529 after retry exhaustion — but "strongly suggests" is
exactly the gap this field closes.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **`headroom/telemetry/session.py`** — `_Session.failure_statuses`,
incremented next to `failures` in `record_outcome`. Keys are the bare
status string for the 5xx range, `"other"` beyond it. Emitted as a
sibling of `failures` in `payload()`.
- **`deploy/beacon/worker.js`** — `failure_statuses` added to
`ALLOWED_KEYS`. Without this the ingest allowlist silently drops it.
- **`deploy/beacon/sample-event.json`** — sample carries the new key in
OTLP `kvlistValue` form.
### Why no slug bounding
`skips` runs values through `_safe_slug` because they arrive as free
strings. A status code is an `int` the proxy itself produced; the `500
<= status < 600` check is what keeps a garbage value from inventing map
keys. Nothing here is user-derived, so the field stays content-free.
### Why `schema_version` stays 1
Additive, matching the precedent set by #2796, which added
`tokens.tool_saved` and the two `all_layers_*` rates without a bump.
Bumping signals a break to consumers when nothing about older rows
becomes invalid.
## Testing
- [x] Unit tests pass (`pytest`) — the module's own self-check, extended
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — see note
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m headroom.telemetry.session
ok
$ ruff check headroom/telemetry/session.py
All checks passed!
$ ruff format --check headroom/telemetry/session.py
1 file already formatted
$ mypy --python-version 3.12 headroom/telemetry/session.py
Success: no issues found in 1 source file
# --python-version 3.12 only to skip a pre-existing numpy-stub syntax error the
# repo's python_version = "3.10" triggers locally; unrelated to this diff.
$ node --check deploy/beacon/worker.js # ok
$ python -c "import json; json.load(open('deploy/beacon/sample-event.json'))" # parses
```
The self-check in `headroom/telemetry/session.py` now records two 529s
and one 500 and asserts both the total and the split:
```python
assert emitted[-1]["failures"] == 3
assert emitted[-1]["failure_statuses"] == {"529": 2, "500": 1}
```
plus `assert event["failure_statuses"] == {}` on the clean-session path.
## Real Behavior Proof
- **Environment:** macOS 25.4.0, Python 3.12 venv, this branch.
- **Exact command / steps:** drive `SessionAggregator` with three
failing outcomes and encode the payload through the same `_any_value`
the wire uses.
```text
payload: 3 {'529': 2, '500': 1}
otlp : {"kvlistValue": {"values": [{"key": "529", "value": {"intValue": "2"}},
{"key": "500", "value": {"intValue": "1"}}]}}
```
The OTLP form matches `deploy/beacon/sample-event.json` byte-for-byte in
shape, and `unwrap()` in `worker.js` turns `kvlistValue` back into a
plain object, so it lands in R2 as `{"529": 2, "500": 1}` — the same
shape as `skips`, which DuckDB reads as `MAP(VARCHAR, BIGINT)`.
- **Observed result:** as above. Verified against the live corpus that
schema evolution here is already routine — 3,836 of 3,884 existing rows
have `rates.all_layers_saved_pct = NULL` from #2796 landing mid-corpus,
and every report still runs.
- **Not tested:** the deployed Worker (no staging R2 binding locally);
`node --check` covers syntax only. The allowlist addition is one array
entry consumed by the existing `pick()`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Screenshots (if applicable)
N/A — wire-format change, covered by the output above.
## Additional Notes
**Deploy order matters.** The Worker allowlist drops unknown keys, so
`deploy/beacon/worker.js` must be deployed *before* a client release
that emits the field — otherwise it is discarded at the door. No
corruption either way, just missing data until the Worker catches up.
**Old data is unaffected.** R2 objects are immutable NDJSON written per
request; nothing rewrites history. The corpus reader already passes
`union_by_name = true`, which fills the column with NULL for rows
written before this ships.
|
||
|
Tejas Chopra
|
c07da992dd
|
Per-request backend selection for routing extensions (#2809)
## The gap
Headroom picks its egress backend **once**, at startup:
`create_proxy_backend` returns a single `Backend` (or `None` for the
direct Anthropic path) and every request goes through it. That is the
right shape for *"run this whole proxy against Bedrock instead of
Anthropic"* and the wrong shape for *"this request is cheaper on a
different provider than the last one."*
`ModelRouter` already lets an extension change `body["model"]` per
request — but only within the protocol the request arrived in, because a
model id alone cannot move a request to another provider.
So an extension can currently **decide** something Headroom has no way
to **carry out**. This adds the missing half.
## The seam
An extension publishes a decision on the request state:
```python
request.state.headroom_route = SimpleNamespace(
model="moonshot/kimi-k2", # required
provider="moonshot", # optional; inferred from the model id if absent
reason="cheaper at this prefix length",
)
```
Headroom resolves a `LiteLLMBackend` for that provider — which is where
translation already lives — and serves **that one request** from it.
Nothing in core names any particular extension; the field is duck-typed,
so an extension does not import Headroom to talk to Headroom.
## Absent means unchanged
This is the property the tests are built around, and the reason this
should be safe to merge.
With nothing published, every path is what it was before. Advice that is
**absent, malformed, names an unknown provider, names a native provider,
or fails to build** all resolve to `self.anthropic_backend` — including
when that is `None`, which is the direct-API path and must survive. A
routing preference can never take traffic down.
## Coverage
| path | |
|---|---|
| `/v1/messages` | non-streaming + streaming |
| `/v1/chat/completions` | non-streaming + streaming |
| Responses API | untouched — does not use the backend abstraction |
Streaming is the one that matters. The resolver rewrites
`body["model"]`, so had `_stream_response_bedrock` kept reading
`self.anthropic_backend`, every streamed routed request would have sent
a foreign model id to Anthropic. Both streaming helpers now take an
optional `backend`, defaulting to the configured one.
## Details worth review
- **Validate the provider name before building.** `LiteLLMBackend`
accepts *any* provider string — the registry falls through to a generic
pass-through config — so a typo silently builds a backend that only
fails later, at request time, with an error pointing nowhere near the
typo. `_known_provider()` checks against `litellm.provider_list` first.
- **Cache per provider, and cache the failures too**, or a broken
provider name costs a construction attempt on every request. (Bedrock
construction calls out to AWS to enumerate inference profiles — it is
not free.)
- **`backend_owns_translation` now asks the per-request backend.** It
decides whether Headroom or the backend owns the `max_tokens` /
`max_completion_tokens` spelling; asking `self.anthropic_backend` would
answer "Headroom does" for a request about to be served by a translating
backend that does.
- **`_route_resolver` lives in `route_advice.py`, not on a handler
mixin.** Two mixins need it, and reaching across sibling mixins only
works by accident of how `HeadroomProxy` composes them.
## Tests
`tests/test_route_advice.py` — 20 tests, most of them asserting the
absent-means-unchanged property from a different angle.
Local runs: 20/20 on the new file; **1102 passed, 1 failed** on `-k
"openai or chat_completions or ccr"`, and **414 passed, 0 failed** on
`-k "stream or bedrock or route_advice"`. The single failure is
`test_realignment_live_multi_turn::test_ccr_marker_round_trip_live`,
which fails identically on this branch's merge-base — verified by
checking out `59314cff~1` and re-running it.
Note for anyone reproducing: `pytest-asyncio` is a declared dev
dependency but was missing from my venv, which made every `async def
test_` in the repo fail. Worth checking before diagnosing a large
failure count.
## Docs
`docs/content/docs/pipeline-extensions.mdx` gains a section on the
contract, next to the existing `x-headroom-base-url` one.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Tejas Chopra
|
0237cbffbb
|
fix(proxy): enable tool search by default and repair poisoned transcripts (#2807)
## Description Server-side tool search poisons the Claude Code transcript: once the proxy injects deferral and the model runs one search, Anthropic's `server_tool_use` + `tool_search_tool_result` pair lives in the message history forever. Upstream validates **every `tool_reference` in that history against the *current* request's `tools` array** — and Claude Code replays one transcript across requests with wildly different tools arrays (main loop: hundreds of tools; prompt-type Stop hook evaluator, `/compact`, other side-requests: a handful). Every one of those side-requests 400s with `Tool reference 'X' not found in available tools`. This PR keeps tool search **on** — it's the whole point of the feature, and the default `coding` savings profile already turned it on at proxy startup — and instead repairs the transcript per request, statelessly. The issue author's preferred fix (never inject for Claude Code clients) would disable the feature for its main audience. A session-sticky approach was also considered and rejected: it needs session state, it can't re-add ~500 tool definitions to a 5-tool side-request without erasing the savings, and it can't heal transcripts already poisoned before the upgrade. Closes #2805 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **`headroom/proxy/helpers.py`** — new `strip_unsupported_tool_search_blocks(messages, tools)`. Builds the set of names this request can resolve, drops any `tool_search_tool_result` whose `tool_reference` entries aren't all resolvable (or when no search tool is present at all), and drops the paired `server_tool_use` by `tool_use_id`. Other server tools (`web_search`, code execution) are untouched. Turns left with zero content blocks are removed rather than forwarded empty. Copy-on-write: returns the **original** `messages` object by identity when nothing was removed. - **`headroom/proxy/handlers/anthropic.py`** — runs the repair right after the injection block, so the tool just injected counts as present and the main loop is a no-op with a byte-identical prefix. Deliberately **not** gated on `HEADROOM_TOOL_SEARCH`, so transcripts poisoned before an upgrade (or before someone sets the flag to `0`) still recover. Logs and tags `router:tool_search_repair:Nblocks` when it fires. - **`headroom/proxy/handlers/anthropic.py`** — `HEADROOM_TOOL_SEARCH` now defaults to `1`. This matches the posture `seed_proxy_env_defaults()` already established for the default `coding` profile; the flip only affects entry points that never seeded. - **`docs/content/docs/proxy.mdx`** — documents on-by-default plus `HEADROOM_TOOL_SEARCH=0` as the opt-out. - **`tests/test_issue_746_tool_search.py`** — 6 tests covering the repair. ### Answering the issue's open question > we could not determine what enables it — `/proc/<pid>/environ` shows no `HEADROOM_TOOL_SEARCH` `seed_proxy_env_defaults()` calls `os.environ.setdefault("HEADROOM_TOOL_SEARCH", "1")` at proxy startup because the default savings profile is `coding`, which has `tool_search=True` (`headroom/agent_savings.py`). In-process mutation of `os.environ` never appears in the process's environ snapshot, which is why the flag looked unset. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_issue_746_tool_search.py -q 45 passed, 1 warning in 1.56s $ python -m pytest tests/test_*anthropic*.py tests/test_*tool*.py -q 4 failed, 459 passed, 2 skipped, 7 warnings in 27.40s # the 4 failures are in tests/test_bedrock_tool_result_cache_and_streaming_stats.py # and reproduce identically on this branch's merge-base with the changes stashed: # 4 failed, 9 passed, 5 warnings in 3.02s $ ruff check headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py tests/test_issue_746_tool_search.py All checks passed! $ ruff format --check <same three files> 3 files already formatted $ mypy --python-version 3.12 headroom/proxy/helpers.py headroom/proxy/handlers/anthropic.py Success: no issues found in 2 source files # --python-version 3.12 only to skip a pre-existing numpy-stub syntax error that # the repo's python_version = "3.10" setting triggers on this machine. ``` New tests: | Test | Asserts | |---|---| | `test_repair_drops_blocks_the_hook_evaluator_cannot_resolve` | small tools array → both blocks dropped, surrounding assistant text survives | | `test_repair_is_noop_on_the_main_loop` | search tool + referenced tool present → `removed == 0` and `messages is transcript` (prefix cache untouched) | | `test_repair_drops_a_turn_left_with_no_blocks` | a turn that was *only* the search round-trip is removed, not forwarded empty | | `test_repair_leaves_other_server_tools_alone` | `web_search` `server_tool_use` blocks survive | | `test_repair_is_idempotent` | second pass over a repaired transcript removes nothing | | `test_repair_strips_search_history_when_only_the_tool_is_missing` | references resolvable but no search tool in the array → still stripped | ## Real Behavior Proof - **Environment:** macOS 25.4.0, Python 3.12 venv, live `api.anthropic.com`, `claude-sonnet-4-6`, local proxy on `127.0.0.1:8799` built from this branch. - **Exact command / steps:** one request body — a poisoned transcript (`server_tool_use` + `tool_search_tool_result` referencing `AskUserQuestion`) with a **1-tool** `tools` array (`Read`), exactly the shape a Claude Code side-request replays — sent twice: once straight to `https://api.anthropic.com`, once to the proxy. ```text $ python /tmp/hr-2805-repro.py https://api.anthropic.com HTTP 400 {"type": "invalid_request_error", "message": "Tool reference 'AskUserQuestion' not found in available tools"} $ python /tmp/hr-2805-repro.py http://127.0.0.1:8799 HTTP 200 content: [{"type": "text", "text": "OK"}] ``` - **Observed result:** the exact 400 from the issue reproduces against upstream; the identical body through the proxy returns 200. The proxy's savings event for that request records `before: 133, after: 32, saved: 101` tokens — the two dropped blocks. The one-tool array is below `_TOOL_SEARCH_MIN_TOOLS = 12`, so no injection ran; the repair alone is what made the request valid. - **Not tested:** a full end-to-end Claude Code session with a real Stop hook (the synthetic replay above is the same request shape the hook evaluator produces); non-Anthropic providers, which don't have server-side tool search. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Screenshots (if applicable) N/A — proxy-side behavior, covered by the command output above. ## Additional Notes - **Cache cost is zero on the hot path.** The repair only rewrites requests whose transcripts reference tools they don't carry — request families that were 400ing anyway. The main loop takes the identity path and its prefix stays byte-identical. - **Out of scope, spotted while here:** `run-all-plugins.sh` exports `HEADROOM_TOOL_SEARCH_MIN_TOOLS=5`, but nothing in Python reads it — `_TOOL_SEARCH_MIN_TOOLS` is a hardcoded `12`. Worth a follow-up. |
||
|
Tejas Chopra
|
303e0522c4
|
fix(opencode): don't preload a missing transport shim into child processes (#2806)
## Description `headroom wrap opencode` broke third-party MCP servers in pip/wheel installs. The wrap transport plugin appended `NODE_OPTIONS=--import=<plugin dir>/../hook-shim/handler.js` to its own env (and injected it into every child it spawns), but that path only resolves in a repo checkout. Wheel installs load the standalone bundle from `headroom/providers/opencode/_dist/`, which has no `hook-shim/` sibling — the shim lives under `plugins/` and maturin only ships files under `headroom/` (pyproject.toml `python-source`/package-dir behavior). Every Node child then aborted with `ERR_MODULE_NOT_FOUND` before executing a line, including OpenCode's stdio MCP servers. OpenCode reports that as `<server> MCP error -32000: Connection closed`. Headroom's own MCP server is a Python process, so it stayed connected — which is why the breakage looked selective, and why nothing appeared in the proxy logs (the failure is entirely inside OpenCode's child process). Docker and `--no-proxy` are incidental: the plugin installs the transport on load in every wrap mode. Fix: resolve the shim only when it exists on disk, and skip the `NODE_OPTIONS` mutation otherwise. Children go direct instead of dying. Checkout builds still get child-process transport hooking, unchanged. Closes #2798 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `plugins/opencode/src/transport.ts`: `shimImportSpecifier()` returns `string | undefined`, gated on `fs.existsSync`; `installProcessEnv()` and `withShimEnv()` leave `NODE_OPTIONS` untouched when the shim is absent. - `plugins/opencode/src/transport.test.ts`: new regression test — with the shim missing, the parent's `NODE_OPTIONS` is unmodified and a spawned `npx -y firecrawl-mcp` receives no `--import`. - `headroom/providers/opencode/_dist/entry.opencode.js`: regenerated via `npm run build:standalone` (the bundle that wheel installs actually load). ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed No Python source changed, so `pytest` / `ruff` / `mypy` are N/A here; the TypeScript equivalents were run instead. ### Test Output ```text $ npm run typecheck > tsc --noEmit (no output) $ npm test RUN v4.1.9 /private/tmp/hr-pr-2798/plugins/opencode Test Files 2 passed (2) Tests 14 passed (14) Duration 416ms # The new test is not vacuous — reverting the guard to `return shim.href` reddens it: $ npx vitest run -t "#2798" Test Files 1 failed | 1 skipped (2) Tests 1 failed | 13 skipped (14) ``` ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), Node v24, Bun present; both bundles loaded directly from disk. - Exact command / steps: load each built bundle, invoke the default plugin export, print `process.env.NODE_OPTIONS`, then `spawnSync(process.execPath, ["-e", "console.log('mcp server handshake ok')"])` — the same way OpenCode launches a stdio MCP server. ```text ### BEFORE (wheel layout, shim missing) ### NODE_OPTIONS: "--import=file:///…/headroom/providers/opencode/hook-shim/handler.js" child: Error [ERR_MODULE_NOT_FOUND]: Cannot find module '…/headroom/providers/opencode/hook-shim/handler.js' <-- becomes MCP -32000 ### AFTER — wheel layout (headroom/providers/opencode/_dist/) ### NODE_OPTIONS after plugin load: undefined child status: 0 | stdout: mcp server handshake ok ### AFTER — checkout layout (plugins/opencode/dist/, shim present) ### NODE_OPTIONS after plugin load: "--import=file:///…/plugins/opencode/hook-shim/handler.js" child status: 0 | stdout: mcp server handshake ok ``` - Observed result: wheel installs no longer poison child env, so Node MCP servers start; checkout builds keep the preload and still start children cleanly. - Not tested: no reproduction against a live `opencode` + codegraph/firecrawl session on Ubuntu (no OpenCode install on this machine); the child-process failure was reproduced directly instead, which is the exact mechanism behind the reported `-32000`. Docker proxy path not re-tested — it is unrelated to the fix. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Docs unchanged: this is an internal packaging/runtime bug with no documented behavior attached. Follow-up (deliberately not in this PR): wheel installs now lose child-process transport hooking rather than crashing — the same coverage they effectively had, since the preload never once loaded from a wheel. Restoring it means a standalone shim build emitted into `_dist/` plus exporting `installHeadroomTransport` from that bundle; `hook-shim/handler.js` also imports `../dist/index.js`, which does not exist in the wheel layout, so copying the file alone would not be enough. Worth doing only if something needs a subprocess's LLM traffic proxied. |
||
|
Tejas Chopra
|
6ec3e3478a
|
feat(cli,pricing): add CLI extension seam and prompt-cache TTL pricing (#2802)
## Description
Two small, independent additions. Both exist because an out-of-tree
package needed
them and neither had a home in the current API.
1. **`headroom.cli_extension`** — an entry-point group so a package can
add a
`headroom` subcommand. `headroom.proxy_extension` requires a running
FastAPI
app, so it cannot carry a read-only CLI tool, and `_register_commands()`
was a
hardcoded import list with no discovery.
2. **`headroom/pricing/cache_ttl.py`** — the prompt-cache TTL price
structure
(read `0.10x`, 5m write `1.25x`, 1h write `2.00x` of base input) plus
the
break-even share above which the 1h TTL is cheaper. `ModelPricing`
carries
`cached_input_per_1m` for reads but has no field for the *write* side.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/extensions.py` (new) — `register_all(main)` discovers
the
`headroom.cli_extension` group. Contract: `register(main: click.Group)
-> None`.
Invoked from `_register_commands()` **last**, so built-ins are already
attached.
- Deliberately **not** opt-in gated, unlike proxy extensions: installing
the
package is the opt-in, because adding a subcommand cannot silently
change what
an existing command does. What *would* be a silent change is shadowing a
built-in, so that is detected and rolled back — a stale plugin can never
quietly
take over `headroom proxy`. Load failures and partial registrations roll
back
too, and one bad plugin never blocks another.
- `headroom/pricing/cache_ttl.py` (new) — `CACHE_READ_MULTIPLIER`,
`CACHE_WRITE_MULTIPLIERS`, `cache_write_multiplier()`,
`cache_rates_per_1m()`,
`ttl_breakeven_share()`. Ratios are derived from base input rather than
transcribed into a per-model table that would triple its columns and
drift.
- `cache_write_multiplier()` raises on an unknown TTL rather than
falling back to
the cheaper 5m rate, which would understate cost.
- `headroom/pricing/litellm_pricing.py` — three additive optional fields
on
`LiteLLMModelPricing` exposing LiteLLM's own
`cache_read_input_token_cost`,
`cache_creation_input_token_cost` and
`cache_creation_input_token_cost_above_1hr`
(present for 212 and 123 models respectively). Published rates should
win over
derived ones. All default to `None`, and `None` means "not published" —
distinct
from `0.0` meaning "free" — so every existing caller is unaffected.
- `headroom/pricing/__init__.py` — re-exports.
### Why `ttl_breakeven_share()` exists
The TTL trade has two terms and both must be counted: moving to 1h turns
idle-gap
rewrites into cheap reads **and** raises the price of every write that
still
happens. Modelling only the recovery overstates the saving. On a real
531-transcript corpus that error was **1.9x** — $1,021 claimed against
$538 real.
`test_write_premium_is_not_forgotten` pins those exact figures so the
mistake
cannot be reintroduced quietly.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`) — see note below
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_cli_extension_seam.py tests/test_pricing_cache_ttl.py tests/test_pricing_from_litellm.py -q
tests/test_cli_extension_seam.py ....... [ 25%]
tests/test_pricing_cache_ttl.py .......... [ 62%]
tests/test_pricing_from_litellm.py .......... [100%]
======================== 27 passed, 1 warning in 3.54s =========================
$ .venv/bin/ruff check headroom/cli/extensions.py headroom/cli/main.py headroom/pricing/ tests/test_cli_extension_seam.py tests/test_pricing_cache_ttl.py
All checks passed!
$ .venv/bin/mypy --python-version 3.12 headroom/cli/extensions.py headroom/pricing/cache_ttl.py headroom/pricing/litellm_pricing.py
Success: no issues found in 3 source files
```
`tests/test_pricing_from_litellm.py` is the **pre-existing** pricing
suite, included
to show the `LiteLLMModelPricing` change is non-breaking.
**mypy note.** With the repo's configured `python_version = "3.10"`,
mypy fails on
numpy's own stubs for any file that transitively reaches numpy:
```text
$ .venv/bin/mypy headroom/pricing/cache_ttl.py
.venv/lib/python3.12/site-packages/numpy/__init__.pyi:737: error: Type statement is only supported in Python 3.12 and greater [syntax]
Found 1 error in 1 file (errors prevented further checking)
```
This is pre-existing and unrelated — untouched files reproduce it
identically
(`mypy headroom/cli/doctor.py`, `mypy headroom/pricing/registry.py`).
Hence the
`--python-version 3.12` run above, which matches the interpreter
actually in use.
Worth fixing separately; not addressed here.
## Real Behavior Proof
- **Environment:** macOS 15 (darwin 25.4.0), Python 3.12.6,
`headroom-ai` 0.34.0
working tree, branch off `upstream/main`.
- **Exact command / steps:**
1. Built a separate out-of-tree package declaring
`[project.entry-points."headroom.cli_extension"] fleet =
"headroom_fleet.cli:register"`.
2. `pip install --no-deps headroom_fleet-0.1.0-py3-none-any.whl`
3. `headroom econ --help`
- **Observed result:** the subcommand registers with no configuration
and appears
in `headroom --help`:
```text
$ python -c "import importlib.metadata as m; print([e.name+' ->
'+e.value for e in m.entry_points(group='headroom.cli_extension')])"
['fleet -> headroom_fleet.cli:register']
$ headroom --help | grep econ
econ Report where local AI-coding token spend goes, and what...
$ headroom econ --help
Usage: headroom econ [OPTIONS] [COMMAND] [ARGS]...
Commands:
fix Write the recommended cache-TTL and compaction settings.
unfix Restore every setting ``econ fix`` changed, exactly as it was.
```
`headroom --help` and every built-in still work with the plugin
installed and
after it is uninstalled. Verified per-model cache rates resolve from
LiteLLM for
`claude-opus-4-8`, `claude-opus-5`, `claude-sonnet-5` and
`claude-haiku-4-5-20251001` — all exactly `1.250x` / `2.000x` / `0.100x`
of base
input, matching the derived fallback.
- **Not tested:** Windows; a plugin that raises at *import* time rather
than in
`register()` (covered by unit test with a stubbed entry point, not a
real
package); the `--python-version 3.10` mypy path, which is blocked by the
pre-existing numpy stub issue above.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
## Note for reviewers
This PR contains **only** the two additions above. Unrelated
`plugins/opencode/src/transport.ts` changes in my working tree are
deliberately
excluded and will follow separately.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
JD Davis
|
64e203931b
|
fix(deps): enforce audited transitive dependency floors (#2791)
## Description Enforces patched minimum versions for the vulnerable transitive `aiohttp` and `cryptography` dependencies so future lockfile refreshes cannot reintroduce the pip-audit failures affecting open pull requests. Related to the shared Security / pip-audit failures across open PRs. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Enforces `aiohttp>=3.14.3` for PYSEC-2026-3545/3546/3547. - Enforces `cryptography>=50.0.0` for PYSEC-2026-3552/3553/3554. - Synchronizes the project version recorded in `uv.lock` with `pyproject.toml`. ## Testing - [x] Dependency audit passes (`pip-audit`) - [x] Lockfile validation passes (`uv lock --check`) - [ ] Unit tests pass (`pytest`) - [ ] Type checking passes (`mypy headroom`) - [x] Manual verification performed ### Test Output ```text $ uv lock --check Resolved 269 packages $ uv export --frozen --no-dev --no-emit-project --no-hashes --extra all --format requirements-txt | uvx --python 3.12 pip-audit -r /dev/stdin No known vulnerabilities found ``` ## Real Behavior Proof - Environment: Local macOS worktree using CPython 3.12.13 and the frozen production dependency export. - Exact command / steps: Validated the lockfile, exported every production dependency with the `all` extra, and audited that exact export with pip-audit. - Observed result: The lockfile resolved successfully and pip-audit reported no known vulnerabilities. - Not tested: Publishing or deployment; the refreshed GitHub CI suite covers builds, wheels, containers, security scans, and platform tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my changes - [x] No explanatory code comments are required beyond the PYSEC constraint annotations - [x] Documentation changes are not required for transitive security floors - [x] My changes generate no new local warnings - [x] The dependency audit proves the security fix is effective - [ ] Full repository tests are delegated to GitHub CI - [x] I did not edit `CHANGELOG.md`; release-please owns it ## Screenshots (if applicable) N/A — dependency metadata only. ## Additional Notes The earlier Docker-native failure was a transient Docker Hub HTTP 502 while resolving `python:3.13-slim`; the build did not reach project code. A fresh CI suite is running on the current head. Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net> |
||
|
JD Davis
|
f236ef2e31
|
test(ccr): cross SQLite max lifetime boundary (#2794)
## Description
Fixes the failing Rust test on `main` after #2669 made SQLite CCR
entries valid at the exact TTL boundary. The integration test waited
only 3.3 seconds for a three-second ceiling; unix-second truncation can
represent that as exactly three seconds, so the entry is correctly still
valid. The test now crosses a guaranteed four-second elapsed boundary.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Extend the max-lifetime test's access loop from four to five 700 ms
gaps.
- Document why four gaps can land on the valid equality boundary and why
five are deterministic.
- Leave production SQLite TTL behavior and defaults unchanged.
## Testing
- [x] Unit tests pass (`cargo test -p headroom-core --test
ccr_backends`)
- [x] Formatting passes (`cargo fmt --all -- --check`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
cargo test -p headroom-core --test ccr_backends
test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
5 consecutive repetitions of the previously failing test:
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 11 filtered out
```
## Real Behavior Proof
- Environment: macOS, Rust workspace at
`
|
||
|
Tejas Chopra
|
e9a24f3ec1
|
fix(beacon): report all-layers savings, not context-compression only (#2796)
## Description
`rates.saved_pct` and `rates.yield_pct` in the beacon payload divide
`tokens_saved` by `original` / `attempted`. Tool-schema deferral never
lands in either denominator — `outcome.py` says so explicitly, and
`tokens.tool_saved` exists precisely because of it — so every beacon
rate silently reports context compression only.
On a tool-heavy fleet that is not a rounding difference. Across the
first 516 sessions in the corpus the beacon reads **2.80%** where the
dashboard headline for the same traffic reads **12.82%**: 157.6M context
tokens vs 803.9M all-layers, with 646.3M of tool-schema deferral missing
from the ratio.
`headroom/proxy/server.py` already resolved this for the dashboard in
#2737 — `savings_percent` is `all_layers_saved / (input +
all_layers_saved)` and `active_savings_percent` puts tool savings on
both sides of the ratio. The beacon was never brought along. This does
that.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/telemetry/session.py`: add `rates.all_layers_saved_pct` and
`rates.all_layers_yield_pct`, computed the way `server.py` builds
`savings_percent` / `active_savings_percent` — tool savings added to
**both** sides, since deferred schemas were attempted work that
succeeded whole.
- `headroom/telemetry/session.py`: extend the `demo()` self-test to
assert both new rates against the existing tool-heavy fixture.
- `deploy/beacon/query.sh`: add an `all_layers_pct` column to the fleet
summary, so the reader stops showing the understated number too.
**Kept alongside `saved_pct` rather than folded into it.** Every row
already in the corpus means context-only under that name; redefining it
would make old and new rows non-comparable with no field to tell them
apart.
**`SCHEMA_VERSION` deliberately stays at 1.** The change is purely
additive, nothing reads the field, and the query uses `union_by_name =
true`, so old and new rows mix cleanly. Happy to bump it if maintainers
want the marker.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m headroom.telemetry.session
ok
$ python -m pytest tests/test_savings_tool_search_aggregation.py tests/test_outcome_dual_ruler_funnel.py -q
FAILED tests/test_outcome_dual_ruler_funnel.py::test_ledger_delta_stays_on_the_local_ruler
FAILED tests/test_outcome_dual_ruler_funnel.py::test_ledger_falls_back_to_billed_when_local_omitted
2 failed, 5 passed, 3 warnings in 4.42s
$ ruff check headroom/
All checks passed!
$ mypy --python-version 3.12 headroom/telemetry/session.py
Success: no issues found in 1 source file
```
The two `test_outcome_dual_ruler_funnel.py` failures are **pre-existing
on `main`, not caused by this PR** — verified by `git stash`-ing the
change and re-running:
```text
$ git stash && python -m pytest tests/test_outcome_dual_ruler_funnel.py -q
FAILED tests/test_outcome_dual_ruler_funnel.py::test_ledger_delta_stays_on_the_local_ruler
FAILED tests/test_outcome_dual_ruler_funnel.py::test_ledger_falls_back_to_billed_when_local_omitted
2 failed, 3 passed, 3 warnings in 3.71s
```
## Real Behavior Proof
- **Environment:** macOS 25.4.0 arm64, Python 3.12.6, repo `.venv`,
branch rebased on `upstream/main` @ `
|
||
|
Parideboy
|
b6f9877c78
|
fix(tokenizer): coerce non-string tool_call fields before counting (#2801)
## Description
`/v1/compress` returned HTTP 503 with an unhandled `TypeError` when a
message carried a `tool_calls[].function.arguments` value that was not a
string. `arguments` is a JSON *string* per the OpenAI spec, but
OpenAI-compatible upstreams do emit `None` or a raw object there, and
every token counter passed the value straight to `tiktoken.encode()`.
Because the malformed message persists in conversation history, the
failure was sticky: every later request replaying that history failed
too, regardless of destination provider.
Reported in #2782. The exact repro in that issue (`arguments: null`) no
longer raises — `count_text` grew a falsy guard since 0.33.0 — but the
root cause is still live for any *truthy* non-string, which I reproduced
against all four counters on `main` before the fix.
## Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update
- [ ] Refactor / internal cleanup
## Changes Made
- `headroom/tokenizers/base.py`: new `coerce_countable_text()`. Strings
pass through untouched, `None` counts as nothing, dict/list/tuple are
JSON-serialized, anything else falls back to `str()`. The serialized
form is capped at 200K chars so a malformed upstream can't turn a token
*estimate* into a multi-megabyte encode.
- Applied at the tool-call field sites (`function.name`,
`function.arguments`, `id`, and the legacy `function_call`) in
`tokenizers/base.py`, `tokenizers/tiktoken_counter.py`,
`providers/openai.py`, `providers/openai_compatible.py`,
`providers/anthropic.py`.
- Guarded `{"function": null}` / `{"id": null}`, which reach the same
encode path.
- New test file `tests/test_tool_call_arguments_not_a_string.py` (11
cases).
Serializing dicts rather than the one-liner suggested in the issue
(`str(func.get("arguments") or "")`) is deliberate: `str()` on a dict
yields Python repr with single quotes, which is not what the upstream
would have billed, and it is unbounded.
## Testing
- [x] Existing tests pass
- [x] New tests added for the fix
- [ ] Manual testing performed
New tests:
```
$ python -m pytest tests/test_tool_call_arguments_not_a_string.py -q
tests\test_tool_call_arguments_not_a_string.py ........... [100%]
============================= 11 passed in 0.40s ==============================
```
Surrounding tokenizer/provider suites, unchanged:
```
$ python -m pytest tests/test_tool_call_arguments_not_a_string.py tests/test_tokenizer.py \
tests/test_tokenizers.py tests/test_tokenizers \
tests/test_provider_counter_content_blocks.py tests/test_provider_tokenizer_one_ruler.py -q
tests\test_provider_counter_content_blocks.py ............ [ 91%]
tests\test_provider_tokenizer_one_ruler.py ......... [100%]
======================= 91 passed, 14 skipped in 1.50s ========================
```
Lint/format on the touched files:
```
$ ruff check <touched files> && ruff format --check <touched files>
All checks passed!
6 files already formatted
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.13.11, pytest 9.1.1, repo at
`upstream/main` (
|
||
|
Agistaris
|
d0a86d409f
|
fix(ccr): preserve exact SQLite TTL boundary (#2669)
## Description SQLite CCR timestamps have whole-second resolution. Expiring a row when `last_accessed + ttl == now` or `created_at + max_lifetime == now` can shorten the configured lifetime by almost one second. This change keeps entries valid at the exact boundary and expires them one second later. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Use strict expiration predicates for idle TTL and maximum lifetime. - Keep lookup predicates valid at the exact boundary. - Add deterministic fixed-time tests for both boundaries. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text running 3 tests test ccr::backends::sqlite::tests::exact_max_lifetime_boundary_is_still_valid ... ok test ccr::backends::sqlite::tests::exact_idle_ttl_boundary_is_still_valid ... ok test transforms::code_compressor::tests::english_exact_token_match_unchanged ... ok test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 911 filtered out ``` ## Real Behavior Proof - Environment: Linux x86_64, repository Rust toolchain. - Exact command / steps: `cargo test -p headroom-core exact_` - Observed result: Both SQLite boundary tests returned the stored payload at the exact configured boundary and removed it one second later. The focused command passed 3/3 selected tests, including one unrelated existing exact-token test. - Not tested: Live provider or model traffic; the change is isolated to the deterministic Rust SQLite backend. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [x] I have commented the boundary behavior - [ ] I have made corresponding documentation changes (not applicable; behavior and tests are local to the backend) - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing relevant tests pass locally with my changes - [x] I did not edit `CHANGELOG.md` ## Screenshots (if applicable) Not applicable. ## Additional Notes Rust formatting and `headroom-core` Clippy pass. A broader package run passed 994 tests with three ignored; two unrelated ONNX parity tests were excluded after reproducing their pre-existing futex stall. |
||
|
pgjh
|
a97b82413b
|
fix(proxy): unwrap Hermes tool_call bridge in tool name map (#2717)
## Description
Hermes Agent (NousResearch/hermes-agent) loads on-demand ("deferred")
tools via a `tool_search` → `tool_describe` → `tool_call` indirection.
On the wire, the emitted tool call is named **`tool_call`**, and the
REAL tool name lives inside the arguments payload:
```json
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "tool_call",
"arguments": "{\"name\": \"read_file\", \"arguments\": {\"path\": \"/etc/hostname\"}}"
}
}
```
`ContentRouter._build_tool_name_map` only reads
`tool_calls[].function.name`, so it maps `tool_call` → `"tool_call"`
instead of the real tool name.
**Consequence**: `HEADROOM_EXCLUDE_TOOLS` /
`HEADROOM_PROTECT_TOOL_RESULTS` silently no-op for **ALL** deferred
tools (`read_file`, `write_file`, `search_files`, `mcp__*`,
`headroom_retrieve`, etc.). Their outputs get lossy-compressed even when
explicitly whitelisted, and `headroom_retrieve` falls into an endless
re-compression loop (`<<ccr:hash>>` → retrieve → re-compress →
`original_tokens: 0`).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
Add an `unwrap_tool_call_name(name, arguments)` helper in `config.py`
(beside the existing `_tool_name_aliases`) and apply it at the **3
sites** where the tool_call_id → tool_name map is built:
1. **OpenAI chat path** — `content_router.py` `_build_tool_name_map`
(`tool_calls[].function`)
2. **Anthropic path** — `content_router.py` `_build_tool_name_map`
(`tool_use` blocks)
3. **Responses API path** — `openai.py`
`_compress_openai_responses_live_text_units_with_router`
(`function_call` items)
The helper:
- Passes non-wrapper names through unchanged
- Parses the arguments payload (JSON string or dict) and extracts the
inner `name`
- Fails open (returns the wrapper name) on malformed/unparseable
payloads — safe for all other clients
After unwrap, `is_tool_excluded()`/protect-list matching sees the real
tool name, so whitelists work for deferred tools exactly as they already
did for classic tools.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] New tests added for new functionality
- [x] Manual testing performed
`tests/test_hermes_tool_call_unwrap.py` — **14 tests**, all pass:
- Helper unit tests: passthrough, None/bad-JSON/missing-name fail-open,
unwrap (web_search, read_file, mcp__*, dict-args form)
- Whitelist activation: unwrapped name + `is_tool_excluded` against
`DEFAULT_EXCLUDE_TOOLS`
- Integration: `_build_tool_name_map` with OpenAI-format `tool_call`
wrapper and Anthropic-format `tool_use` wrapper
- Regression guard: documents that `tool_call` itself is NOT in
`DEFAULT_EXCLUDE_TOOLS` (the pre-fix failure mode)
### Test Output
```text
$ uv run pytest tests/test_hermes_tool_call_unwrap.py tests/test_content_router_exclude_tools.py tests/test_config.py -q
56 passed in 1.24s
$ uv run ruff check .
All checks passed!
$ uv run ruff format --check .
All checks passed!
```
## Real Behavior Proof
- Environment: Ubuntu 24.04 (kernel 7.0.0-28), Python 3.11.15, headroom
built from source at `v0.33.0-5-g6d5516dc` via `uv sync` (maturin), Rust
1.95.0 toolchain. Headroom proxy 0.34.0-dev running as a systemd user
service, `HEADROOM_PROTECT_TOOL_RESULTS=read_file,headroom_retrieve`,
mode=token. Upstream: local new-api-compatible gateway (deepseek-v4-pro
/ GLM-5.2).
- Exact command / steps: Client = Hermes Agent (fresh session via
`hermes chat -q --provider newapi2`, traffic routed through the headroom
proxy at 127.0.0.1:8788). Trigger a deferred `read_file` tool call and
observe `/stats` → `recent_requests`.
- Observed result: After fix, live traffic evidence shows request
`hr_1785683282_000016` (GLM-5.2 session) with `transforms_applied:
["router:excluded:tool", "openai:chat:tool_schema_compaction"]` — the
deferred `read_file` tool was recognized and excluded (whitelist hit),
21918 → 17218 input tokens. Before the fix this request showed no
`router:excluded:tool` for deferred tools — they were compressed. E2E
unit-level proof (`test_build_tool_name_map_exclusion_after_unwrap` +
standalone pipeline run): a `tool_call`-wrapped `read_file` output
(~9KB, 100 lines) passed through `ContentRouter.apply()` **verbatim**
(byte-identical), while a non-whitelisted tool (Bash) in the same
pipeline was compressed — proving the whitelist now works and the
pipeline still compresses normally.
- Not tested: Anthropic native clients (Claude Code) and Responses-API
clients (Codex) end-to-end — the patch sites for those paths are covered
by unit tests only. No new dependencies; no public API changes.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
No documentation changes needed (no public API change; behavior is
internal to the proxy tool-name map). Follow-up: end-to-end verification
with Anthropic/Responses-API clients once maintainers can run the proxy
CI on those paths.
Co-authored-by: pgjh <pgjh@users.noreply.github.com>
|
||
|
dependabot[bot]
|
267c2bdcb5
|
deps: bump postcss from 8.5.19 to 8.5.25 in /sdk/typescript (#2747)
Bumps [postcss](https://github.com/postcss/postcss) from 8.5.19 to 8.5.25. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/postcss/postcss/releases">postcss's releases</a>.</em></p> <blockquote> <h2>8.5.25</h2> <ul> <li>Fixed 8.5.17 visitor regression.</li> <li>Fixed <code>list.split()</code> for non-string values (by <a href="https://github.com/amir-rezaei"><code>@amir-rezaei</code></a>).</li> </ul> <h2>8.5.24</h2> <ul> <li>Preserve the BOM after the processing (by <a href="https://github.com/hdimer"><code>@hdimer</code></a>).</li> </ul> <h2>8.5.23</h2> <ul> <li>Do not load source map without <code>opts.from</code> for security reasons.</li> </ul> <h2>8.5.22</h2> <ul> <li>Fixed custom property losing semicolon before a comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> </ul> <h2>8.5.21</h2> <ul> <li>Fixed childless at-rule losing semicolon before comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed docs (by <a href="https://github.com/isker"><code>@isker</code></a>).</li> </ul> <h2>8.5.20</h2> <ul> <li>Fixed missing space if <code>AtRule#params</code> is set after (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed mixing AST error on warnings (by <a href="https://github.com/MahinAnowar"><code>@MahinAnowar</code></a>).</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/postcss/postcss/blob/main/CHANGELOG.md">postcss's changelog</a>.</em></p> <blockquote> <h2>8.5.25</h2> <ul> <li>Fixed 8.5.17 visitor regression.</li> <li>Fixed <code>list.split()</code> for non-string values (by <a href="https://github.com/amir-rezaei"><code>@amir-rezaei</code></a>).</li> </ul> <h2>8.5.24</h2> <ul> <li>Preserve the BOM after the processing (by <a href="https://github.com/hdimer"><code>@hdimer</code></a>).</li> </ul> <h2>8.5.23</h2> <ul> <li>Do not load source map without <code>opts.from</code> for security reasons.</li> </ul> <h2>8.5.22</h2> <ul> <li>Fixed custom property losing semicolon before a comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> </ul> <h2>8.5.21</h2> <ul> <li>Fixed childless at-rule losing semicolon before comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed docs (by <a href="https://github.com/isker"><code>@isker</code></a>).</li> </ul> <h2>8.5.20</h2> <ul> <li>Fixed missing space if <code>AtRule#params</code> is set after (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed mixing AST error on warnings (by <a href="https://github.com/MahinAnowar"><code>@MahinAnowar</code></a>).</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
ff4e0167bb
|
deps: bump postcss from 8.5.19 to 8.5.25 in /plugins/opencode (#2748)
Bumps [postcss](https://github.com/postcss/postcss) from 8.5.19 to 8.5.25. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/postcss/postcss/releases">postcss's releases</a>.</em></p> <blockquote> <h2>8.5.25</h2> <ul> <li>Fixed 8.5.17 visitor regression.</li> <li>Fixed <code>list.split()</code> for non-string values (by <a href="https://github.com/amir-rezaei"><code>@amir-rezaei</code></a>).</li> </ul> <h2>8.5.24</h2> <ul> <li>Preserve the BOM after the processing (by <a href="https://github.com/hdimer"><code>@hdimer</code></a>).</li> </ul> <h2>8.5.23</h2> <ul> <li>Do not load source map without <code>opts.from</code> for security reasons.</li> </ul> <h2>8.5.22</h2> <ul> <li>Fixed custom property losing semicolon before a comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> </ul> <h2>8.5.21</h2> <ul> <li>Fixed childless at-rule losing semicolon before comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed docs (by <a href="https://github.com/isker"><code>@isker</code></a>).</li> </ul> <h2>8.5.20</h2> <ul> <li>Fixed missing space if <code>AtRule#params</code> is set after (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed mixing AST error on warnings (by <a href="https://github.com/MahinAnowar"><code>@MahinAnowar</code></a>).</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/postcss/postcss/blob/main/CHANGELOG.md">postcss's changelog</a>.</em></p> <blockquote> <h2>8.5.25</h2> <ul> <li>Fixed 8.5.17 visitor regression.</li> <li>Fixed <code>list.split()</code> for non-string values (by <a href="https://github.com/amir-rezaei"><code>@amir-rezaei</code></a>).</li> </ul> <h2>8.5.24</h2> <ul> <li>Preserve the BOM after the processing (by <a href="https://github.com/hdimer"><code>@hdimer</code></a>).</li> </ul> <h2>8.5.23</h2> <ul> <li>Do not load source map without <code>opts.from</code> for security reasons.</li> </ul> <h2>8.5.22</h2> <ul> <li>Fixed custom property losing semicolon before a comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> </ul> <h2>8.5.21</h2> <ul> <li>Fixed childless at-rule losing semicolon before comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed docs (by <a href="https://github.com/isker"><code>@isker</code></a>).</li> </ul> <h2>8.5.20</h2> <ul> <li>Fixed missing space if <code>AtRule#params</code> is set after (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed mixing AST error on warnings (by <a href="https://github.com/MahinAnowar"><code>@MahinAnowar</code></a>).</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
cd60ee9ae8
|
deps: bump postcss from 8.5.19 to 8.5.25 in /plugins/openclaw (#2749)
Bumps [postcss](https://github.com/postcss/postcss) from 8.5.19 to 8.5.25. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/postcss/postcss/releases">postcss's releases</a>.</em></p> <blockquote> <h2>8.5.25</h2> <ul> <li>Fixed 8.5.17 visitor regression.</li> <li>Fixed <code>list.split()</code> for non-string values (by <a href="https://github.com/amir-rezaei"><code>@amir-rezaei</code></a>).</li> </ul> <h2>8.5.24</h2> <ul> <li>Preserve the BOM after the processing (by <a href="https://github.com/hdimer"><code>@hdimer</code></a>).</li> </ul> <h2>8.5.23</h2> <ul> <li>Do not load source map without <code>opts.from</code> for security reasons.</li> </ul> <h2>8.5.22</h2> <ul> <li>Fixed custom property losing semicolon before a comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> </ul> <h2>8.5.21</h2> <ul> <li>Fixed childless at-rule losing semicolon before comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed docs (by <a href="https://github.com/isker"><code>@isker</code></a>).</li> </ul> <h2>8.5.20</h2> <ul> <li>Fixed missing space if <code>AtRule#params</code> is set after (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed mixing AST error on warnings (by <a href="https://github.com/MahinAnowar"><code>@MahinAnowar</code></a>).</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/postcss/postcss/blob/main/CHANGELOG.md">postcss's changelog</a>.</em></p> <blockquote> <h2>8.5.25</h2> <ul> <li>Fixed 8.5.17 visitor regression.</li> <li>Fixed <code>list.split()</code> for non-string values (by <a href="https://github.com/amir-rezaei"><code>@amir-rezaei</code></a>).</li> </ul> <h2>8.5.24</h2> <ul> <li>Preserve the BOM after the processing (by <a href="https://github.com/hdimer"><code>@hdimer</code></a>).</li> </ul> <h2>8.5.23</h2> <ul> <li>Do not load source map without <code>opts.from</code> for security reasons.</li> </ul> <h2>8.5.22</h2> <ul> <li>Fixed custom property losing semicolon before a comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> </ul> <h2>8.5.21</h2> <ul> <li>Fixed childless at-rule losing semicolon before comment (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed docs (by <a href="https://github.com/isker"><code>@isker</code></a>).</li> </ul> <h2>8.5.20</h2> <ul> <li>Fixed missing space if <code>AtRule#params</code> is set after (by <a href="https://github.com/sarathfrancis90"><code>@sarathfrancis90</code></a>).</li> <li>Fixed mixing AST error on warnings (by <a href="https://github.com/MahinAnowar"><code>@MahinAnowar</code></a>).</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
0fd0b996a4
|
deps: bump next from 16.2.10 to 16.3.0 in /docs (#2750)
Bumps [next](https://github.com/vercel/next.js) from 16.2.10 to 16.3.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/vercel/next.js/releases">next's releases</a>.</em></p> <blockquote> <h2>v16.3.0</h2> <h3>Core Changes</h3> <ul> <li>Update vendored lodash to 4.17.23 to fix CVE-2025-13465: <a href="https://redirect.github.com/vercel/next.js/issues/91558">#91558</a></li> <li>Fix invalid HTML response for route-level RSC requests in deployment adapter: <a href="https://redirect.github.com/vercel/next.js/issues/91541">#91541</a></li> <li>Normalize encoded dynamic placeholders in app routes: <a href="https://redirect.github.com/vercel/next.js/issues/91603">#91603</a></li> <li>Fix(pages-router): restore Content-Length and ETag for /_next/data/ JSON responses: <a href="https://redirect.github.com/vercel/next.js/issues/90304">#90304</a></li> <li>Update tokio from 1.43.0 to 1.47.3: <a href="https://redirect.github.com/vercel/next.js/issues/90945">#90945</a></li> <li>[turbopack] Simplify snapshotting logic: <a href="https://redirect.github.com/vercel/next.js/issues/91178">#91178</a></li> <li>Turbopack: enable server HMR for app route handlers: <a href="https://redirect.github.com/vercel/next.js/issues/91466">#91466</a></li> <li>turbo-tasks-backend: batch find_and_schedule_dirty using for_each_task_meta: <a href="https://redirect.github.com/vercel/next.js/issues/91497">#91497</a></li> <li>[turbopack] Use bail! instead of panic! for duplicate module ident error: <a href="https://redirect.github.com/vercel/next.js/issues/91636">#91636</a></li> <li>Skip loadBindings() Lightning CSS check during next start: <a href="https://redirect.github.com/vercel/next.js/issues/91538">#91538</a></li> <li>turbo-tasks-backend: batch schedule dirty tasks in aggregation_update: <a href="https://redirect.github.com/vercel/next.js/issues/91461">#91461</a></li> <li>Turbopack: Add importModule() support to webpack loaders: <a href="https://redirect.github.com/vercel/next.js/issues/89630">#89630</a></li> <li>turbo-persistence: fix mmap page alignment and improve error context in MetaFile::open_internal: <a href="https://redirect.github.com/vercel/next.js/issues/91640">#91640</a></li> <li>turbopack-css: demote recoverable CSS parse warnings to Warning severity: <a href="https://redirect.github.com/vercel/next.js/issues/91524">#91524</a></li> <li>feat(node-streams): add config flag, define-env, and env precedence test: <a href="https://redirect.github.com/vercel/next.js/issues/90427">#90427</a></li> <li>Rename /_next/webpack-hmr to /_next/hmr: <a href="https://redirect.github.com/vercel/next.js/issues/91415">#91415</a></li> <li>Add per-slot error attribution for instant validation using slot markers and config depth preference: <a href="https://redirect.github.com/vercel/next.js/issues/91610">#91610</a></li> <li>Handle encoded params further: <a href="https://redirect.github.com/vercel/next.js/issues/91627">#91627</a></li> <li>[turbopack] Respect <code>{eval:true}</code> in worker_threads constructors: <a href="https://redirect.github.com/vercel/next.js/issues/91666">#91666</a></li> <li>Fix missing route in otel spans without base-server: <a href="https://redirect.github.com/vercel/next.js/issues/91665">#91665</a></li> <li>[turbopack] Optimize compaction cpu usage: <a href="https://redirect.github.com/vercel/next.js/issues/91468">#91468</a></li> <li>Fix layout segment optimization: move app-page imports to server-utility transition: <a href="https://redirect.github.com/vercel/next.js/issues/91701">#91701</a></li> <li>Fix server actions in standalone mode with <code>cacheComponents</code>: <a href="https://redirect.github.com/vercel/next.js/issues/91711">#91711</a></li> <li>turbo-persistence: remove Unmergeable mmap advice: <a href="https://redirect.github.com/vercel/next.js/issues/91713">#91713</a></li> <li>turbopack: move "compact database" tracing span to backend layer: <a href="https://redirect.github.com/vercel/next.js/issues/91693">#91693</a></li> <li>Turbopack: lazy require metadata and handle TLA: <a href="https://redirect.github.com/vercel/next.js/issues/91705">#91705</a></li> <li>Fix adapter outputs for dynamic metadata routes: <a href="https://redirect.github.com/vercel/next.js/issues/91680">#91680</a></li> <li>Turbopack: fix webpack loader runner layer: <a href="https://redirect.github.com/vercel/next.js/issues/91727">#91727</a></li> <li>[turbopack] Remove incorrect debug_assert in try_read_task_cell: <a href="https://redirect.github.com/vercel/next.js/issues/91699">#91699</a></li> <li>Add module count field to module graph tracing spans: <a href="https://redirect.github.com/vercel/next.js/issues/91697">#91697</a></li> <li>turbopack-cli: add --persistent-caching flag for filesystem-backed cache: <a href="https://redirect.github.com/vercel/next.js/issues/91657">#91657</a></li> <li>Turbopack: pull in updated vercel/nft tests: <a href="https://redirect.github.com/vercel/next.js/issues/91651">#91651</a></li> <li>[turbopack] Improve regressed build speed on cross-compiled MUSL: <a href="https://redirect.github.com/vercel/next.js/issues/91477">#91477</a></li> <li>[Segment Bundling] [Scaffolding] Ensure inlining hint correctness: <a href="https://redirect.github.com/vercel/next.js/issues/91320">#91320</a></li> <li>[Segment Bundling] [Scaffolding] Track which segments can be omitted from prefetch: <a href="https://redirect.github.com/vercel/next.js/issues/91438">#91438</a></li> <li>Avoid deprecated TS node10 moduleResolution defaults: <a href="https://redirect.github.com/vercel/next.js/issues/91847">#91847</a></li> <li>[turbopack] Rebuild the docker build scripts: <a href="https://redirect.github.com/vercel/next.js/issues/91799">#91799</a></li> <li>Fix TS6 baseUrl deprecation for extended tsconfig: <a href="https://redirect.github.com/vercel/next.js/issues/91855">#91855</a></li> <li>Add <code>next internal post-build</code> CLI command for Turbopack database compaction: <a href="https://redirect.github.com/vercel/next.js/issues/91336">#91336</a></li> <li>Turbopack: Define <code>Effect</code> as a trait instead of a closure: <a href="https://redirect.github.com/vercel/next.js/issues/89080">#89080</a></li> <li>Turbopack: Implement TraceRawVcs and NonLocalValue correctly for Effects: <a href="https://redirect.github.com/vercel/next.js/issues/89133">#89133</a></li> <li>turbo-tasks-backend: improve print_cache_item_size instrumentation: <a href="https://redirect.github.com/vercel/next.js/issues/91742">#91742</a></li> <li>Turbopack: switch from base40 to base38 hash encoding (remove ~ and . from charset): <a href="https://redirect.github.com/vercel/next.js/issues/91832">#91832</a></li> <li>Use charCodeAt for normalizePathTrailingSlash: <a href="https://redirect.github.com/vercel/next.js/issues/91380">#91380</a></li> <li>Turbopack: Only patch lockfile when bindings fails to load: <a href="https://redirect.github.com/vercel/next.js/issues/91379">#91379</a></li> <li>[create-next-app] Skip interactive prompts when CLI flags are provided: <a href="https://redirect.github.com/vercel/next.js/issues/91840">#91840</a></li> <li>[devtools] Make instant navs panel draggable: <a href="https://redirect.github.com/vercel/next.js/issues/91914">#91914</a></li> <li>[Segment Bundling] Bundle static prefetches based on size: <a href="https://redirect.github.com/vercel/next.js/issues/91439">#91439</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
56ee57be98
|
deps: bump brace-expansion from 5.0.7 to 5.0.9 in /docs (#2751)
Bumps [brace-expansion](https://github.com/juliangruber/brace-expansion) from 5.0.7 to 5.0.9. <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Lucas Santos
|
3107994aed
|
fix(litellm): add async_post_call_success_hook to HeadroomCallback (#1322)
## Description Pointing a litellm proxy at the Headroom callback blows up on the post-call success path: ``` type object 'HeadroomCallback' has no attribute 'async_post_call_success_hook' ``` litellm's logging contract calls `async_post_call_success_hook` after a successful response, and `HeadroomCallback` simply doesn't have it. We implement `async_pre_call_hook`, `async_success_handler` and `async_failure_handler`, but not this one, so litellm hits an `AttributeError` instead of a no-op and the whole request fails. This adds the missing `async_post_call_success_hook(self, data, user_api_key_dict, response)` matching litellm's signature. It returns `response` unchanged, the token accounting already lives in `async_success_handler` so there's nothing to do here except not crash. A few notes: 1. I did not make `HeadroomCallback` inherit litellm's `CustomLogger`, on purpose. The class keeps litellm as an optional dependency, so it stays a plain class and just provides the hooks litellm looks up by name. 2. It's a pass-through, so it's safe regardless of what the response contains. Closes #1114 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/integrations/litellm_callback.py`: add `async_post_call_success_hook` to `HeadroomCallback`, returning the response unchanged; update the class docstring to list the full set of litellm hooks. - `tests/test_integrations/test_litellm_callback.py`: new tests that the method exists, is a coroutine, and returns the response untouched; build the module path with `pathlib` instead of a fragile `__file__.replace`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --extra dev python -m pytest tests/test_integrations/test_litellm_callback.py -q 3 passed ruff: All checks passed! mypy: Success: no issues found ``` ## Real Behavior Proof - Environment: macOS, Python 3.13, this branch. - Exact command / steps: `uv run --extra dev python -m pytest tests/test_integrations/test_litellm_callback.py -q`. The tests import the callback module directly and resolve `async_post_call_success_hook` by name, the same way litellm does, then await it with a sentinel response. - Observed result: 3 passed. The hook exists, is a coroutine, and returns the exact response object it was given. Before the fix, resolving the attribute raised `AttributeError`. - Not tested: I did not stand up a full litellm proxy end to end. The fix is the missing hook method, which the unit tests cover. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes --------- Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Focused Instability
|
4dab254d52
|
fix: emit SSE ping before message_start on Bedrock streaming path (issue #902) (#1080)
## Description Closes #902 Mid-turn user interjections (steering) silently dropped through the Bedrock streaming path. The _stream_response_bedrock code path reconstructs Anthropic SSE events from parsed StreamEvent objects instead of passing raw bytes through, so SSE-level ping keepalives are never forwarded to Claude Code. Claude Code relies on ping events to arm its mid-turn steering / interruptible state; without them, queued interjections are discarded instead of sent. Root cause (confirmed): - Standard direct-Anthropic path does a raw yield-chunk passthrough — pings flow unchanged. - Bedrock path (_stream_response_bedrock.generate()) reconstructs events from litellm/anyllm stream_message() output, which only yields semantic events (message_start, content_block_*, message_delta, message_stop, error). No pings, ever. Fix: emit a synthetic 'event: ping / data: {}' at stream start (before the first message_start) so downstream clients see the same ping-then-content cadence as a real Anthropic stream. Note: periodic pings for very long responses (>~25s) may be needed if steering disarms on a timer. This commit arms it at turn start; follow-up if reporters confirm steering still drops on long turns. The causal link (ping → steering) is the reporter's hypothesis from hands-on debugging. The observable defect (zero pings in stream) is confirmed and fixed. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - headroom/proxy/handlers/streaming.py: yield ping event before the event loop in _stream_response_bedrock.generate() - tests/test_proxy/test_bedrock_sse_ping.py: 3 new tests asserting ping appears before message_start ## Testing - [x] Unit tests pass (pytest) - [x] Linting passes (ruff check .) - [x] New tests added for new functionality ### Test Output ``` tests/test_proxy/test_bedrock_sse_ping.py::test_bedrock_stream_emits_ping_before_message_start PASSED tests/test_proxy/test_bedrock_sse_ping.py::test_bedrock_stream_ping_has_empty_data PASSED tests/test_proxy/test_bedrock_sse_ping.py::test_bedrock_stream_contains_message_stop PASSED tests/test_backend_streaming_cache_metrics.py (4 tests) PASSED 7 passed in 4.41s ``` ## Real Behavior Proof - Environment: macOS, Python 3.11, headroom unit tests - Exact command / steps: pytest tests/test_proxy/test_bedrock_sse_ping.py -v - Observed result: 3 new tests pass; ping appears before message_start in Bedrock stream - Not tested: end-to-end against live Bedrock + Claude Code (no Bedrock credentials available) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] Code follows project style guidelines - [x] Code is commented where non-obvious - [x] No new warnings - [x] Tests added and passing ## Additional Notes The Rust proxy files mentioned in the issue (sse/framing.rs, sse/anthropic.rs) are NOT part of this fix. Those drops are in a telemetry-only tee task that never affects the client byte path — the Rust proxy does a raw bytes passthrough for all responses. The defect is Python-only, confined to the Bedrock path. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Tejas Chopra
|
9fd5ae3d53
|
chore: release main (#2679)
🤖 I have created a release *beep* *boop* --- <details><summary>0.34.0</summary> ## [0.34.0](https://github.com/headroomlabs-ai/headroom/compare/v0.33.0...v0.34.0) (2026-08-05) ### Features * **claude:** support Claude Code in VS Code ([#2752](https://github.com/headroomlabs-ai/headroom/issues/2752)) ([ |
||
|
Tejas Chopra
|
702a076fc1
|
docs(readme): surface the Serena opt-out in the wrap quickstart (#2790)
## Description
The README quickstart announces that `headroom wrap` installs Serena,
then stops — it offers no way to decline. The opt-out exists
(`--code-memory none`) but was documented only on the docs site at
`docs/content/docs/proxy.mdx:78`, which a README reader never reaches.
In #2783 a user followed the quickstart verbatim, hit a Serena startup
failure, and found the flag via `headroom wrap claude --help` only
afterwards, while recovering the machine. Their words: *"a first-time
user following the documented `headroom wrap claude` command has no
signal that this failure mode exists or how to avoid it."*
One line at the point where the install is announced closes that gap.
Closes #2788
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `README.md` — one sentence after the existing wrap paragraph:
> Serena is registered at **user scope** (for Claude Code, in
`~/.claude.json`), so it stays available in your other projects until
you run `headroom unwrap`. To skip it entirely, wrap with `--code-memory
none`.
Two facts, both of which the #2783 reporter needed and could not find:
the registration is user-scoped (so Serena shows up in projects that
were never wrapped, until `unwrap`), and there is a flag to skip it.
`docs/content/docs/quickstart.mdx` was checked for the same gap and has
no equivalent wrap section — its only `wrap` mention is `vscode-claude`
— so there is nothing to mirror. `proxy.mdx` already covers the flag and
is unchanged.
## Testing
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
A two-line Markdown addition to `README.md`. No Python, no config, no
build inputs — pytest / ruff / mypy have nothing to cover. What was
worth checking is that the sentence is *true*, so I verified both claims
against the CLI on `main` rather than against the issue text.
### Test Output
```text
$ grep -n '"--code-memory"' -A 2 headroom/cli/wrap.py
924: "--code-memory",
925- type=click.Choice([_CODE_MEMORY_SERENA, _CODE_MEMORY_NONE]),
926- default=None,
$ headroom wrap claude --help | grep -A3 code-memory
headroom wrap claude --code-memory none # No code-memory MCP
...
--code-memory [serena|none] Code-memory MCP to register: 'serena' (default)
or 'none'. Also set by HEADROOM_CODE_MEMORY.
Replaces --serena/--no-serena.
```
## Real Behavior Proof
- **Environment:** macOS (darwin 25.4.0), repo `.venv`, branch cut from
`upstream/main` @ `
|
||
|
Tejas Chopra
|
6b63b623e0
|
docs(metrics): document OTLP metric export and Dynatrace ingest (#2785)
## Description
The proxy can already push its counters to any OTLP/HTTP endpoint via
`HEADROOM_OTEL_METRICS_*`, but the docs site only surfaced this as a
single row in the proxy env table (`proxy.mdx:287`). The endpoint,
header, service-name, and resource-attribute variables were documented
only in `wiki/metrics.md` — so an operator reading the Vercel docs had
no way to wire Headroom into their existing observability stack.
This adds that section, plus a Dynatrace subsection, because Dynatrace
has a silent failure mode that costs an afternoon to diagnose.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `docs/content/docs/metrics.mdx` — new `### OpenTelemetry (OTLP)
Export` section after the Prometheus section: the
`headroom-ai[proxy,otel]` install, all seven `HEADROOM_OTEL_*` variables
in a table, the exported counter names (`headroom.proxy.tokens.saved` et
al.), the `curl /stats | jq .otel` verification, and the note that an
app-managed global meter provider is recorded into automatically.
- `docs/content/docs/metrics.mdx` — new `### Dynatrace` subsection:
copy-paste env block, `metrics.ingest` token scope, a `warn` Callout on
the delta-temporality requirement, the ActiveGate URL variant, the
Collector + `cumulativetodelta` alternative, and one paragraph
explaining that trace export needs `opentelemetry-instrument`
(Headroom's self-configured tracing targets Langfuse only).
- `docs/content/docs/proxy.mdx` — the `HEADROOM_OTEL_METRICS_ENABLED`
row now links to `/docs/metrics#opentelemetry-otlp-export`.
No code, config, or nav changes — the Observability nav slot already
points at `metrics.mdx`.
## Testing
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
Docs-only change: no Python touched, so pytest/ruff/mypy have nothing to
cover here. `next build` was **not** run — `docs/node_modules` is absent
in this checkout, which would require a full `npm install`; Vercel's
preview build is the real gate. In its place I verified the MDX cannot
break the build by parsing for the two things that actually fail MDX v3
— unbalanced JSX and bare `<`/`{` in prose.
### Test Output
```text
$ python - <<'PY' # strip fenced + inline code, then scan prose for MDX hazards
...
PY
hazards: [(80, '<Tabs groupId="lang" items={[\'TypeScript\', \'Python\']}>'),
(125, '<Tabs groupId="lang" items={[\'Python\', \'Proxy\']}>')]
Callout balance: 1 open / 1 close
```
Both flagged lines are pre-existing `<Tabs>` JSX expressions, untouched
by this PR. The added prose introduces no bare `<` or `{` (every
`<env-id>` / `<activegate>` placeholder sits inside a code fence or
inline backticks). `type="warn"` is already used on three other pages,
and the anchor `#opentelemetry-otlp-export` matches the GitHub-slugger
form of the new heading.
## Real Behavior Proof
- **Environment:** macOS (darwin 25.4.0), repo `.venv`,
`opentelemetry-sdk` 1.44.0, `opentelemetry-exporter-otlp-proto-http`,
headroom @
|
||
|
Tejas Chopra
|
3c10e8ff00
|
chore(docs): one documentation site, not two (#2784)
## Description The repo published **two** documentation sites from two source trees: ``` docs/ -> Next.js/Fumadocs -> headroom-docs.vercel.app <- canonical wiki/ -> MkDocs -> gh-pages branch -> github.io/headroom <- orphan ``` The Vercel site is what the README badge and **every** README deep link point at, and what `pyproject.toml` names as both `Homepage` and `Documentation`. The Pages site is referenced from **nowhere** in the repo — not README, not `pyproject`, not `CLAUDE.md`, not any docs page. I grepped for `github.io` and `gh-pages` across all of them and got zero hits. So it was costing work and causing breakage while nobody was reading it: - **Every documented change had to be written twice.** This session I wrote the same configuration content into `docs/content/docs/configuration.mdx` *and* `wiki/configuration.md`. That's the tax, and it compounds silently — the two drift and no one notices which is stale. - **It broke the Vercel deployment.** Each Pages deploy runs `mkdocs gh-deploy --force`, force-pushing `gh-pages`. Vercel's Git integration then tries to build that branch with Root Directory `docs`, which fails: *"The specified Root Directory `docs` does not exist"* — because `gh-pages` holds only the rendered site (`.nojekyll`, `404.html`, …). Timing was exact: ```text 23:06:44 main |
||
|
Tejas Chopra
|
a9a2fbd74f
|
ci(docs): deploy Pages on wiki changes, drop the never-working Vercel job (#2746)
## Description
Two independent bugs in `.github/workflows/docs.yml`.
**1. Pages went stale because the push filter watched the wrong
directory.**
`mkdocs.yml` sets `docs_dir: wiki`, but the push filter listed `docs/**`
and not `wiki/**`. So a merge touching only `wiki/` never triggered the
workflow and the published Pages site silently went stale, while a
`docs/**`-only change triggered a Pages rebuild whose sources mkdocs
doesn't even read.
Push now filters on `wiki/**` + `mkdocs.yml`. **Pull requests keep
`docs/**`**, so `validate-nextjs` still catches a broken MDX change
before merge.
**2. `deploy-vercel` never worked — it wasn't a working path that went
stale.**
It ran `npx vercel deploy --prod --token=${{ secrets.VERCEL_TOKEN }}`,
but those secrets don't exist:
```text
$ gh api repos/headroomlabs-ai/headroom/actions/secrets --jq '.secrets[].name' | grep -i vercel
(nothing)
$ gh api orgs/headroomlabs-ai/actions/secrets --jq '.secrets[].name' | grep -i vercel
(nothing)
```
So it invoked the CLI with an empty `--token=` and exited 1, every time:
```text
30 failed runs on main, 2026-07-14 .. 2026-08-04
most recent:
30874173333
|
||
|
Tejas Chopra
|
04e1517ede
|
fix(telemetry): stop mixing tokenizer scales in RequestOutcome, and fix the overhead framing (#2756)
## Description
Two defects found by reading real beacon payloads, not by inspection.
### 1. `eligible_pct: 120`
A gpt-4o-mini session shipped this:
```json
"tokens": {"original": 10, "attempted": 12, "input": 12, "saved": 0},
"rates": {"eligible_pct": 120}
```
120% is structurally impossible — you cannot attempt to compress more
than arrived. And nothing had grown.
`original_tokens` is our **local tokenizer** count. `optimized_tokens`
on the OpenAI path carried the **provider's** `usage.prompt_tokens`. Our
estimator undercounted gpt-4o-mini by 2 tokens on a 10-token request,
and every quantity derived from that pair inherited the mismatch:
- `attempted_input_tokens = optimized + saved` → 12, exceeding
`original` → `eligible_pct` 120, `yield_pct` contaminated
- `tokens_inflated` (added in #2708) → reported **2 tokens of phantom
growth**
`optimized_tokens` was dual-purpose by design — *"post-compression bytes
actually forwarded, for `input_tokens` and `tok_after`"*. Billing wants
the provider's count; deltas need the same ruler as `original_tokens`.
Those are different jobs sharing one field.
This is the same class of bug as #2743, on the request path instead of
`/v1/compress`, and it is the exact false positive I flagged as
theoretical when reviewing #2708 — where I measured the margin on a real
722-request log as **exactly zero**, so any provider counting above our
estimator would flip it. gpt-4o-mini does, and it is now in production
telemetry.
### 2. `overhead_pct` was documented as wall-clock, and is not
A 1393-turn session shipped `latency_ms_total: 16396565.8` against
`duration_s: 8904` — **4.55h of latency inside a 2.47h session, 1.84×
over wall clock.**
Both terms are sums over turns, and turns run concurrently (parallel
tool calls, subagents, several clients per proxy), so each sum
over-counts elapsed time. `overhead_pct` is still a meaningful
latency-weighted per-request share, but the comment claiming *"what
fraction of wall-clock did Headroom itself add?"* was wrong, and
shipping `latency_ms_total` beside `duration_s` invites a comparison
that yields nonsense.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
**Scale split (`outcome.py`, `handlers/openai.py`)**
- `optimized_tokens` is now always the **local** count — same tokenizer
as `original_tokens`, so every delta built from the pair is coherent.
- New optional `provider_input_tokens` carries the provider's own count.
Defaults to `0`, so the other emit sites need no change.
- Cost and volume totals read `provider_input_tokens or
optimized_tokens`, so **billing is unchanged** wherever a provider
reports usage, and falls back exactly as before where it doesn't.
- Removes a band-aid: one of the three OpenAI sites already computed
`effective_original_tokens = max(original_tokens, optimized + saved)`,
inflating `original` upward so `attempted` could not exceed it. That hid
the symptom at one site while the other two shipped the impossible
ratio.
**Overhead framing (`telemetry/session.py`)**
- Corrects the wall-clock comment on `overhead_pct` and states what it
actually is.
- Documents that `latency_ms_total` is a sum of per-request durations,
not elapsed time, and why it can exceed `session.duration_s`.
- Adds `overhead_ms_per_turn` and `latency_ms_per_turn` — unambiguous
under concurrency and comparable across sessions.
- Field names kept for schema-v1 consumers; **additive only**, no schema
bump.
## Testing
- [x] Unit tests pass (new file)
- [x] Linting passes (`ruff check` + `format --check`, pinned 0.15.17)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/outcome.py headroom/proxy/handlers/openai.py headroom/telemetry/session.py
All checks passed!
$ uvx ruff@0.15.17 format --check <same three>
3 files already formatted
$ pytest tests/test_outcome_token_scale.py -q
5 passed in 0.24s
```
## Real Behavior Proof
- **Environment:** macOS 26.4 arm64, isolated git worktree off
`upstream/main`, throwaway venv (see caveat).
Replayed both reported payloads through the corrected arithmetic:
```text
ENTRY 1 (gpt-4o-mini, the eligible_pct:120 case)
BEFORE: original=10 (local) optimized=12 (PROVIDER) saved=0
attempted = 12+0 = 12 eligible_pct = 120.0 <- impossible
tok_inflated = max(0, 12-10) = 2 <- phantom
AFTER : original=10 (local) optimized=10 (local) provider_input=12
attempted = 10+0 = 10 eligible_pct = 100.0
tok_inflated = 0
billed_input = 12 -> cost/cache math unchanged
ENTRY 2 (1393 turns, the overhead case)
latency_ms_total = 16397s vs duration 8904s -> 1.84x wall clock
NEW overhead_ms_per_turn = 333.5 latency_ms_per_turn = 11770.7
wall clock per turn = 6392.0 -> per-turn latency exceeds it => concurrency
```
Genuine post-compression growth still surfaces: `55,161 → 57,845`
reports `tokens_inflated = 2,684` (pinned as a test), so the fix doesn't
mute what #2708 exists to show.
- **Not tested locally beyond the new file.** The repo venv currently
has no `pytest`, no `ruff`, and no compiled `headroom._core`, so I used
a throwaway venv. The outcome/telemetry suites fail there for missing
deps — `click` (12/12), then `headroom._core` (10/10) — with **zero
assertion failures**, and an **identical failure set on this branch and
on clean `upstream/main`** in the same env. So they are
environment-only, not regressions. CI on this PR is the authoritative
signal for the full suite.
- Unrelated: `Wrap E2E / docker-wrap-e2e` is currently red on `main`
from a quay.io CDN `tls: internal error` pulling the manylinux base
image — infrastructure, transient, and green on the three prior runs.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] I did **not** edit `CHANGELOG.md`
## Follow-ups not in this PR
- **`cache_write` and `uncached` double-count** on inferred-cache
providers. The same beacon entry shows `input: 12, cache_read: 0,
cache_write: 12, uncached: 12` — both fields describe the identical 12
tokens, because `_infer_openai_cache_write_tokens` and
`uncached_input_tokens` are computed the same way (`input −
cache_read`). Any consumer summing them gets 2×. The payload also
carries no `cache_inferred` flag, so a reader can't distinguish an
inferred write from an Anthropic-reported one.
- **No skip reason for compression itself.** That entry records
`memory_skip:no_handler` but nothing explains why compression didn't
fire; "below the size floor" and "compressor failed" are
indistinguishable — the same ambiguity #2708 just removed for inflation.
- **`eligible_pct` can still legitimately exceed 100** when a request
genuinely grows after compression (memory injection, proactive
expansion), since `attempted = optimized + saved` uses the forwarded
size. Left alone deliberately: clamping would hide real inflation, and
`tokens_inflated` now expresses it properly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Tejas Chopra
|
0e1d6bfa79
|
refactor(pricing): make LiteLLM the source of truth, not the hardcoded table (#2779)
## Description > **Stacked on #2777.** That PR corrects the built-in table's *values*; this one stops the table being *authoritative*. Both are wanted — the fallback should be right **and** not in charge. Merge #2777 first. You asked whether the token/savings code could be simpler and whether the hardcoding could go. This is the hardcoding half, and the encouraging finding is that **almost none of it needed writing** — the infrastructure already existed and was simply unused. `headroom/pricing/litellm_pricing.py` (300 lines, LiteLLM-backed, with an `ImportError` fallback and gateway-prefix handling) has been in the tree the whole time. `ModelInfo`'s own docstring says: > *"Pricing is fetched dynamically from LiteLLM's database. Use `ModelRegistry.estimate_cost()` to get current pricing."* Yet **zero of the four providers called it** (`grep -c litellm_pricing` → openai 0, anthropic 0, google 0, cohere 0). Each kept a parallel hardcoded table. `_get_pricing` had no LiteLLM lookup at all, unlike `get_context_limit` — which is precisely how it went ~18 months stale and priced `gpt-4.1-nano` **300× over**. ## Changes Made **1. Resolution order now mirrors `get_context_limit`**, so limits and prices can't disagree: ``` explicit user config -> LiteLLM -> built-in table -> family -> unknown default ``` Config beats LiteLLM because a configured price is a decision, not a guess. The table stays because it must: the `litellm` dependency is gated `python_version < '3.14'`, and LiteLLM doesn't know every model. It just isn't in charge, so its drift only reaches installs with no LiteLLM. **2. Gateway-routed names now resolve at all.** `litellm.model_cost` keys the *unwrapped* form, so `bedrock/anthropic.claude-...` missed every candidate and silently took the $2.50/$10.00 unknown default: | model | before | after | |---|---|---| | `bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0` | $2.50 / $10.00 | **$3.00 / $15.00** | | `bedrock/us.anthropic.claude-3-5-sonnet-...-v2:0` | $2.50 / $10.00 | **$3.00 / $15.00** | | `vertex_ai/claude-sonnet-4-5` | $2.50 / $10.00 | **$3.00 / $15.00** | | `groq/llama-3.3-70b-versatile` | $2.50 / $10.00 | **$0.59 / $0.79** | | `gemini-2.5-flash` | $2.50 / $10.00 | **$0.30 / $2.50** | | `deepseek-chat` | $2.50 / $10.00 | **$0.28 / $0.42** | `pricing_lookup_candidates` only ever *prepended* provider prefixes. It now also tries progressively unwrapped forms, derived by splitting on `/` — deliberately **not** another hardcoded gateway-prefix list. A wrong guess costs nothing: each candidate is an exact dict lookup, so it just misses. **3. The staleness warning became meaningful.** It fires only when the fallback table is actually used. Before, it was unconditional — and with `_PRICING_LAST_UPDATED = 2025-01-14` against a 60-day window it had been firing for ~18 months, which trains people to ignore it. **4. `pricing_per_1m` rounds to 6dp.** LiteLLM stores cost *per token*, so `× 1e6` leaves float noise ($0.4/1M arrives as `0.39999999999999997`). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Code refactoring ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check` + `ruff format`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_pricing_from_litellm.py -q 10 passed ``` Full pricing / cost / provider / models / savings / reporting / tokenizer set: ```text $ pytest tests/test_*{pricing,cost,provider,models,savings,utils,reporting,token}*.py -q 3 failed, 1026 passed, 38 skipped pre-existing on main (all three in my recorded baseline): test_bundled_tools_savings.py::test_compressed_payload_preserves_answer_anthropic test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4] (needs transformers) test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4] (needs transformers) ``` Deferring the full sharded run to CI — no maturin/Rust core locally. ### One test of mine changed, and why Three cases in #2777's `test_openai_pricing_resolution.py` failed on exact equality once prices started coming from LiteLLM — `gpt-4.1-mini` arrived as `0.39999999999999997` rather than `0.4`. The values were right; binary floating point isn't exact. Switched those to `pytest.approx(..., abs=0.001)` — money compared to the cent — which passes whether the number comes from LiteLLM or the literal table. ## Deliberately NOT in this PR - **Encodings stay hardcoded.** LiteLLM carries no tiktoken encoding data, and `_lookup_encoding_name`'s `None` return is load-bearing (it's the "not an OpenAI model" signal from #2761). Encodings also track tokenizer generations, not monthly price changes — they aren't the drift problem. - **Anthropic / Cohere / Google providers.** Same shape, same fix, but Anthropic's pricing is a `{input, output, cached_input}` dict rather than a tuple, and its matcher is worse (`if model in known_model or known_model in model` — bidirectional substring). Worth its own PR rather than tripling this diff. - **Context limits.** Already LiteLLM-first; the layering there was correct all along. - `accounts/fireworks/models/kimi-k2` still falls back — LiteLLM genuinely has no entry. The file already shows the pattern for filling such gaps (`_register_minimax_pricing`, `_inject_deepseek_pricing`) if we want it. |
||
|
Tejas Chopra
|
f03cc6d88b
|
fix(router): stop counting an image's base64 payload as suffix tokens (#2778)
## Description
`_netcost_message_tokens` walked block-list content itself and fell back
to `str(block)` for anything that wasn't `text` or `tool_result` — on
the stated assumption that such blocks *"rarely dominate a suffix"*. An
`image` block is the exception that breaks it: `str()` embeds the whole
base64 payload.
```text
counted real over
512x512 PNG 20,034 349 57x
1092x1092 screenshot 100,034 1,589 63x
1568x1568 233,367 1,600 146x
```
**Why this changes behaviour, not just a number.** S is the cache-bust
cost — the tokens re-written if message *j* is mutated. `apply()` builds
it as a running suffix sum:
```python
for j in range(num_messages - 1, -1, -1):
netcost_suffix_tokens[j] = netcost_suffix_tokens[j + 1] + _netcost_message_tokens(...)
```
So one image inflates S for **every message before it**, and the
break-even gate then declines to compress any of them. A single
screenshot could switch off net-cost-gated compression for the whole
earlier conversation — and screenshots are routine in agent sessions.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
Delegate block-list content to `tokenizers.base.count_content_blocks`,
deleting the local walk. That counter already guards exactly this case —
its comment reads *"1MB image = ~330K fake tokens without this"* — so
this walk simply predated it.
Beyond the raw fix, this removes a **second pricing rule**: the gate now
values images the same way the tokenizer that computes
`tokens_before`/`tokens_after` does (a flat 1600, "max after
auto-resize"). Pricing images one way for the gate and another for the
savings math is the same class of problem as #2761.
Verified byte-identical on the shapes the old walk handled correctly:
```text
old walk canonical
text only 101 101
tool_result str 81 81
tool_result list 61 61
image only 100,034 1,600
mixed 100,036 1,602
```
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check` + `ruff format`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
### Test Output
```text
$ pytest tests/test_netcost_suffix_image_tokens.py -q
8 passed
$ git stash push headroom/ && pytest tests/test_netcost_suffix_image_tokens.py -q
4 failed, 4 passed
# the 4 failures are the payload-scaling assertions; the 4 passes are the
# text/tool_result/string shapes, included to prove delegation is behaviour-preserving
```
All netcost + content-router suites:
```text
$ pytest tests/test_netcost_gate.py tests/test_content_router_*.py \
tests/test_transforms_content_router.py tests/test_netcost_suffix_image_tokens.py -q
126 passed
```
```text
$ ruff check headroom/transforms/content_router.py tests/... All checks passed!
$ mypy headroom/transforms/content_router.py no new errors
```
Deferring the full suite to CI — no maturin/Rust core in this
environment.
## One existing test rewritten — please look at this bit
`test_netcost_gate.py::TestNetCostHelpers::test_message_tokens_block_list_beats_repr`
fails under the fix, and I want to be explicit that I changed a test
rather than bury it.
It built its image block as `{"type": "image", "source": {"data": "x" *
500}}`. A 500-char stub is **cheaper than a single image's real token
cost**, so `str()` over it looked harmless (~130 tokens) and its
assertion `abs(helper - text_only) < text_only * 0.5` held. That
unrepresentative fixture is precisely why the payload-scaling bug
survived — the test named "beats repr" was passing on the one payload
size where repr happens not to be catastrophic.
Rewritten to use a realistic 200KB payload and to assert what actually
matters:
```python
assert helper >= text_only # text still counted in full
assert helper - text_only <= 2000 # image cost is bounded, not payload-scaled
assert helper < count_text(str(content)) / 10 # ...and far below repr
```
I checked this both ways, so it is a real test and not a rubber stamp:
```text
old test + fixed code -> FAILS (it was pinning the defect)
new test + main -> FAILS (it catches the real bug)
new test + fixed code -> passes
```
## Known limitation
The canonical estimate is a flat 1600 per image regardless of
dimensions, so a small icon is now over-charged (~1600 vs ~13 real)
where repr would have charged ~200. I kept the flat constant
deliberately: it is the value every other counter in the codebase uses,
and introducing a third rule here to shave small-icon cost would
recreate the inconsistency this PR removes. The error is bounded at 1600
tokens and biases the gate conservative, versus an unbounded 100K+ error
before.
|
||
|
Tejas Chopra
|
fc4680b37a
|
fix(tokenizers): resolve gpt-5 and mixed-case model names to the right encoding (#2776)
## Description Two defects in `get_encoding_for_model`, both reachable through the normal `get_tokenizer()` path. **1. `gpt-5` had no prefix entry.** It fell through to `DEFAULT_ENCODING` (`cl100k_base`) instead of `o200k_base`. Same class as the `o4` gap already patched in that tuple. cl100k emits ~33% more tokens than o200k on CJK, so every gpt-5 count was inflated there: ```text CJK sample (30x repeated sentence) o200k_base (correct) 450 tokens cl100k_base (actual) 600 tokens +33.3% ``` Note #2758 taught the *registry* that `gpt-5` → the tiktoken backend; this is the next hop, where that backend picks its *encoding*. So gpt-5 got the right tokenizer family and the wrong encoding inside it. **2. Resolution was case-sensitive.** `TokenizerRegistry.get` lowercases only its **cache key**, then constructs the counter from the caller's original string (`_create_tokenizer(model, backend)`). An uppercase deployment name — routine on Azure, where the deployment name is user-chosen — arrived verbatim, matched no prefix, and took the default encoding. The cache makes this one genuinely unpleasant: key lowercased, construction not, so **the encoding a model receives depends on the casing of whichever request warmed the cache first**, and can differ across restarts. ```text cold cache, uppercase resolved first: GPT-4o -> cl100k_base CJK=600 WRONG GPT-4.1 -> cl100k_base CJK=600 WRONG Gpt-4O-Mini -> cl100k_base CJK=600 WRONG gpt-4o -> o200k_base CJK=450 ok ``` I nearly filed this as "not reachable" — my first check ran the lowercase spelling first, which populated the shared lowercased cache key and masked it completely. The tests call `clear_cache()` so the uppercase spelling resolves cold, which is the failing order. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `("gpt-5", "o200k_base")` to the ordered prefix tuple. - `get_encoding_for_model` now lowercases its input. The lowercasing is deliberately scoped to this function rather than the registry: every `MODEL_TO_ENCODING` key is already lowercase (asserted), so it is safe here — whereas lowercasing in `TokenizerRegistry` would break HuggingFace repo ids, which *are* case-sensitive (`Qwen/Qwen3-Coder`). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check` + `ruff format`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_tokenizer_encoding_resolution.py -q 21 passed $ git stash push headroom/ && pytest tests/test_tokenizer_encoding_resolution.py -q # gpt-5 family, every uppercase case, and both cache-order tests fail ``` Targeted run across the tokenizer/pricing/provider suites, including `test_evals_cjk_tokenization.py` since CJK is the affected content type: ```text $ pytest tests/test_utils.py tests/test_reporting.py tests/test_cost_pricing_warning_dedup.py \ tests/test_pricing.py tests/test_pricing_litellm.py tests/test_provider_model_fallback.py \ tests/test_models.py tests/test_savings_ledger.py tests/test_tokenizers.py \ tests/test_tokenizer.py tests/test_tokenizer_selection_coverage.py \ tests/test_provider_tokenizer_one_ruler.py tests/test_openai_model_table_resolution.py \ tests/test_evals_cjk_tokenization.py -q 233 passed, 16 skipped ``` ```text $ ruff check <changed files> All checks passed! $ mypy headroom/tokenizers/tiktoken_counter.py # only pre-existing release_version.py tomllib redef, present on main ``` Deferring the full suite to CI — this environment has no maturin/Rust core, so the native-dependent shards can't run locally. ## Real Behavior Proof - **Environment:** macOS, Python 3.13.7, isolated worktree at `upstream/main` (`0cb72f45`). - **Exact command / steps:** `TokenizerRegistry.clear_cache()`, then resolve each spelling cold and count a CJK sample. - **Observed result:** ```text before after gpt-5 cl100k_base CJK=600 o200k_base CJK=450 gpt-5-mini cl100k_base CJK=600 o200k_base CJK=450 GPT-4o cl100k_base CJK=600 o200k_base CJK=450 GPT-4.1 cl100k_base CJK=600 o200k_base CJK=450 Gpt-4O-Mini cl100k_base CJK=600 o200k_base CJK=450 GPT-4 cl100k_base CJK=600 cl100k_base CJK=600 (unchanged, correct) gpt-4o o200k_base CJK=450 o200k_base CJK=450 (unchanged) gpt-3.5-turbo cl100k_base CJK=600 cl100k_base CJK=600 (unchanged) ``` `GPT-4` was previously "correct" only by accident — it missed every prefix and landed on `DEFAULT_ENCODING`, which happens to be `cl100k_base`. It is now correct by resolution. |
||
|
Tejas Chopra
|
0cb72f45b2
|
fix(providers): stop a shorter model family shadowing a longer one (#2762)
## Description > **Stacked on #2761** — that PR splits `_lookup_encoding_name` out of `_get_encoding_name_for_model`, which this one builds on. Please merge #2761 first; the diff here will shrink to just this commit afterwards. `_MODEL_ENCODINGS` and `_CONTEXT_LIMITS` are matched by prefix, iterating in **plain dict order** — so the first *inserted* prefix wins rather than the most specific one. `gpt-4.1` matched the `gpt-4` entry: | model | resolved | actual | | |---|---|---|---| | `gpt-4.1` | 8192 | 1,047,576 | **128× under** | | `gpt-4.1-mini` | 8192 | 1,047,576 | **128× under** | | `gpt-4.1-nano` | 8192 | 1,047,576 | **128× under** | | `gpt-4-32k-0613` | 8192 | 32,768 | 4× under | | `gpt-5` / `-mini` / `-nano` | 128,000 | 400,000 | fell to unknown-model default | | `o4-mini` | 128,000 | 200,000 | fell to unknown-model default | A 128× under-estimate matters because the context limit is what tells the proxy how much headroom is left: it treats a 1M-context model as nearly full and compresses accordingly. The same shadowing picked the **wrong encoding** — `gpt-4.1` got `cl100k_base` instead of `o200k_base`. Measured cost of that: ```text cl100k o200k error python code 420 420 +0.0% json blob 555 555 +0.0% logs 580 580 +0.0% english 201 201 +0.0% CJK 600 450 +33.3% ``` So the encoding half is narrow but real — it only bites CJK content, which the repo already treats as a case worth testing (`tests/test_evals_cjk_tokenization.py`). **Scope honestly:** `get_context_limit` consults LiteLLM *before* this table, so the limit half only surfaces where LiteLLM is absent or does not know the model. That is not hypothetical — the `litellm` dependency carries a `python_version < '3.14'` marker (`pyproject.toml:56`), so **any install on Python 3.14+ has no LiteLLM** and this table is load-bearing. The encoding half never had a LiteLLM fallback and was always wrong. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Both prefix loops now iterate `sorted(..., key=len, reverse=True)` — longest prefix wins. This is the root-cause fix: it also protects the *next* model added to these tables. - Added the missing families: `gpt-4.1` (+`-mini`/`-nano`), `gpt-5` (+`-mini`/`-nano`), `o4-mini` to both tables. - Left `supports_model`'s prefix loop alone — it only returns a bool, so order cannot change its answer. Not touched: `_PRICING`. `gpt-4.1`/`gpt-5` also fall through to the GPT-4o pricing tier, which skews cost reporting, but that is a separate concern with its own verification burden (published rates, staleness window) and does not belong in a tokenizer-correctness fix. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check` + `ruff format`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output 12 of 24 new cases fail without the fix; the 12 that pass are the "must not regress" rows (`gpt-4`, `gpt-4-turbo`, `gpt-4o`, `o3`, `gpt-3.5-turbo`) — included precisely so the longest-prefix change can't quietly move them: ```text $ git stash push headroom/providers/openai.py && pytest tests/test_openai_model_table_resolution.py -q FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-4.1-mini-1047576] FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-4.1-nano-1047576] FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-4.1-2025-04-14-1047576] FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-4-32k-0613-32768] FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-5-400000] FAILED ...::test_context_limit_prefers_the_most_specific_prefix[gpt-5-mini-400000] FAILED ...::test_context_limit_prefers_the_most_specific_prefix[o4-mini-200000] FAILED ...::test_encoding_prefers_the_most_specific_prefix[gpt-4.1-o200k_base] FAILED ...::test_encoding_prefers_the_most_specific_prefix[gpt-4.1-mini-o200k_base] FAILED ...::test_encoding_prefers_the_most_specific_prefix[gpt-4.1-2025-04-14-o200k_base] FAILED ...::test_cjk_is_not_over_counted_for_gpt_41 12 failed, 12 passed in 0.70s $ git stash pop && pytest tests/test_openai_model_table_resolution.py -q 24 passed in 0.43s ``` Regression check — 119 suites touching openai / cost / savings / token / compress / outcome / budget, this branch vs clean `main` in the same environment, comparing failure *sets*: ```text branch : 5 failed, 1486 passed, 86 skipped in 111.01s main : 5 failed, 1453 passed, 86 skipped in 132.07s NEW failures introduced: (none) pre-existing on both: test_bundled_tools_savings.py::test_compressed_payload_preserves_answer_anthropic test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4] (needs `transformers`) test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4] (needs `transformers`) test_image_compressor_singleton_reuse.py::test_onnx_router_is_built_once_and_cached test_openai_streaming_backend.py::...test_litellm_vertex_streaming_preserves_max_tokens_and_vendor_fields ``` ```text $ ruff check headroom/providers/openai.py tests/test_openai_model_table_resolution.py All checks passed! $ mypy headroom/providers/openai.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - **Environment:** macOS, Python 3.13.7, isolated worktree at `upstream/main` (`ad56dd38`), no `litellm` installed (matching a Python 3.14+ install, where the dep marker excludes it). - **Exact command / steps:** resolve context limit + encoding for each model against published OpenAI values, before and after. - **Observed result:** ```text before after model limit enc model limit enc gpt-4.1 8192 cl100k gpt-4.1 1047576 o200k_base gpt-4.1-mini 8192 cl100k gpt-4.1-mini 1047576 o200k_base gpt-4.1-nano 8192 cl100k gpt-4.1-nano 1047576 o200k_base gpt-4.1-2025-04-14 8192 cl100k gpt-4.1-2025-04-14 1047576 o200k_base gpt-4-32k-0613 8192 cl100k gpt-4-32k-0613 32768 cl100k_base gpt-5 128000 o200k gpt-5 400000 o200k_base o4-mini 128000 o200k o4-mini 200000 o200k_base unchanged: gpt-4=8192/cl100k, gpt-4-turbo=128000/cl100k, gpt-4o=128000/o200k, o3=200000, gpt-3.5-turbo=16385/cl100k ``` |
||
|
Tejas Chopra
|
cd92ed52ff
|
fix(providers): give every model exactly one tokenizer (#2761)
## Description `/v1/chat/completions` is a multi-provider passthrough, but `OpenAIProvider` handed **any** unrecognized model a guessed `o200k_base` encoding. Kimi through Fireworks — a documented Headroom configuration — counted **~19% low**. This is not just an accuracy nit, because **two resolvers race on the same request**: - handlers count via the tokenizer registry (`count_tokens_offloaded` → `get_tokenizer(model)`) - `TransformPipeline` counts via `provider.get_token_counter(model)`, because the proxy builds its pipelines with `provider=self.openai_provider` (`server.py:953-958`) `tokens_saved = original_tokens - optimized_tokens`, and in token mode those two operands come from *different* resolvers (`handlers/openai.py:3150-3155` keeps the handler's `original_tokens` and takes the pipeline's `optimized_tokens`). When the rulers disagree the subtraction is noise — it can invent savings on an untouched request, or trip the `optimization inflated tokens` revert guard at `handlers/openai.py:3219` and throw away real compression. Measured on `main` before this change, same 2-message payload: | model | registry (handler) | provider (pipeline) | gap | |---|---|---|---| | `moonshotai/kimi-k2` | 686 | 554 | **19.2%** | | `accounts/fireworks/models/kimi-k2-instruct` | 686 | 554 | **19.2%** | | `gemini-2.5-pro` | 534 | 554 | 3.7% | | `command-r-plus` | 534 | 554 | 3.7% | | `mistral-large-latest` | 563 | 554 | 1.6% | | `gpt-4o` / `claude-sonnet-4-6` | 552 | 554 | 0.4% | The provider returned **554 for every model** — it was model-blind. This follows the precedent already documented in `tests/test_compress_route_tokenizer_by_model.py`: pinning one provider's counter for a multi-model route is the bug, and the registry is the canonical resolver (every registry tokenizer derives from `BaseTokenizer`, whose `_count_content_parts` ends in a serialize-and-count catch-all). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `_get_encoding_name_for_model` split into `_lookup_encoding_name` (returns `None` when nothing claims the model) plus the original fallback wrapper, so callers can distinguish "OpenAI model" from "guessed". - `OpenAIProvider.get_token_counter` defers to `get_tokenizer(model)` when no real tiktoken encoding claims the model. - Per-message overhead `4` → `3`. OpenAI's counting guide uses `tokens_per_message = 3` for every model since `gpt-3.5-turbo-0613`; only the retired `gpt-3.5-turbo-0301` used 4. Staying on 4 over-counted every message by one token *and* disagreed with the registry, so a 100-message conversation drifted by 100 tokens depending on who counted it. - `_token_counters` annotation widened to `dict[str, TokenCounter]`. Preserved deliberately: explicit `model -> encoding` mappings (custom config / `HEADROOM_MODEL_LIMITS`) still win, and genuine OpenAI models still use `OpenAITokenCounter`. Both are pinned by tests. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check` + `ruff format`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output New tests fail on `main` and pass here — 7 of 9 fail without the fix (the 2 that pass are the invariants the fix must not break): ```text $ git stash push headroom/providers/openai.py && pytest tests/test_provider_tokenizer_one_ruler.py -q FAILED ...::test_non_openai_models_resolve_to_the_registry_tokenizer[moonshotai/kimi-k2] FAILED ...::test_non_openai_models_resolve_to_the_registry_tokenizer[accounts/fireworks/models/kimi-k2-instruct] FAILED ...::test_non_openai_models_resolve_to_the_registry_tokenizer[gemini-2.5-pro] FAILED ...::test_non_openai_models_resolve_to_the_registry_tokenizer[command-r-plus] FAILED ...::test_non_openai_models_resolve_to_the_registry_tokenizer[claude-sonnet-4-6] FAILED ...::test_kimi_is_not_counted_with_an_openai_encoding FAILED ...::test_per_message_overhead_matches_openai_and_the_registry 7 failed, 2 passed in 0.49s $ git stash pop && pytest tests/test_provider_tokenizer_one_ruler.py -q 9 passed in 0.51s ``` Regression check — 119 suites touching openai / cost / savings / token / compress / outcome / budget, run on this branch and on clean `main` **in the same environment**, comparing failure *sets*: ```text branch : 5 failed, 1453 passed, 86 skipped in 97.60s main : 5 failed, 1453 passed, 86 skipped in 132.07s NEW failures introduced by fix: (none) pre-existing on both: test_bundled_tools_savings.py::test_compressed_payload_preserves_answer_anthropic test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4] (needs `transformers`) test_compress_route_tokenizer_by_model.py::...[deepseek/deepseek-v4] (needs `transformers`) test_image_compressor_singleton_reuse.py::test_onnx_router_is_built_once_and_cached test_openai_streaming_backend.py::...test_litellm_vertex_streaming_preserves_max_tokens_and_vendor_fields ``` ```text $ ruff check headroom/providers/openai.py tests/test_provider_tokenizer_one_ruler.py All checks passed! $ mypy headroom/providers/openai.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - **Environment:** macOS, Python 3.13.7, isolated worktree at `upstream/main` (`ad56dd38`), in-place `headroom/_core.abi3.so` copied in. - **Exact command / steps:** resolve both tokenizers for the same 2-message payload and compare, before and after. - **Observed result:** ```text before (main) gpt-4o registry=552 provider=554 DIVERGES 2 moonshotai/kimi-k2 registry=686 provider=554 DIVERGES 132 (19.2%) gemini-2.5-pro registry=534 provider=554 DIVERGES 20 (3.7%) command-r-plus registry=534 provider=554 DIVERGES 20 (3.7%) after (this branch) gpt-4o registry=552 provider=552 AGREE gpt-5 registry=552 provider=552 AGREE o4-mini registry=552 provider=552 AGREE claude-sonnet-4-6 registry=552 provider=552 AGREE moonshotai/kimi-k2 registry=686 provider=686 AGREE gemini-2.5-pro registry=534 provider=534 AGREE command-r-plus registry=534 provider=534 AGREE ``` ## Known remaining gap (deliberately not in this PR) Plain and `name`-bearing messages now agree exactly, but **tool-call accounting still differs** on genuine OpenAI models: ```text tool msg (tool_call_id) registry=11 provider=13 delta +2 assistant tool_calls registry=14 provider=21 delta +7 ``` `OpenAITokenCounter` adds flat guesses (`+10` per tool call, `+2` per `tool_call_id`); the registry serializes the real structure and counts it. I believe the registry is closer to what the model actually sees, but I could not ground-truth it — there are no recorded `usage.prompt_tokens` fixtures in `tests/parity/`, and I did not want to shift everyone's tool-heavy numbers on a hunch. Tool-heavy agent traffic is the dominant Headroom workload, so this deserves its own PR with a real API capture to compare against. Filing separately. |
||
|
Tejas Chopra
|
ad56dd382b
|
fix(router): compare token quantities in one unit (#2759)
## Description Two places compared a token quantity against something measured in a **different unit**. Both changed compression **behaviour**, not just reporting — which is the worse class. ### 1. The CONFIG branch put a word count in a token ratio `compressed_tokens = len(compressed.split())` was divided by `original_tokens`, which comes from `_estimate_tokens(content)`. Words run ~2.8× fewer than estimator tokens on config text, so a compressor that returned its input **byte-identically** scored ~0.36. `min_ratio` is 1.0 — accept any real shrink — so the router **accepted the no-op**: cached the result, pinned a frozen "compress" verdict, emitted a `router:config_compressor` label into `transforms_applied`, and recorded a fabricated saving to TOIN. ```text mkdocs.yml, compressor returns its input unchanged denominator (_estimate_tokens) = 936 OLD numerator len(split()) = 334 -> ratio 0.357 claims 64% saved ACCEPTED NEW numerator (_estimate_tokens) = 936 -> ratio 1.000 correctly rejected ``` The sibling TABULAR branch already used `_estimate_tokens`; CONFIG was the outlier. ### 2. The Kompress size gate tested a token cap in chars/4 `len(text_to_compress) > self._kompress_max_tokens * 4` under-counts anything denser than 4 chars/token, and compact JSON runs ~3.2. Against the 50,000-token default there is a band where an oversized payload passes: ```text records chars old_gate(len/4) new_gate(tokens) 2700 119,281 False False 4000 177,781 False True <- 44,445 vs 55,557 tokens, 11% over 4400 195,781 False True <- 48,945 vs 61,182 tokens, 22% over 5000 222,781 True True ``` Those payloads entered ONNX inference — exactly the >30s non-preemptible worker stall the gate exists to prevent (#1171). Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `content_router.py` CONFIG branch — `compressed_tokens` now from `_estimate_tokens(compressed)`, matching its denominator and every sibling branch. - `content_router.py` Kompress gate — compared with `_estimate_tokens`, the unit the cap is actually expressed in. The extra O(n) char scan is negligible against the inference it guards. ## Testing - [x] Unit tests pass - [x] Linting passes (`ruff check` + `format --check`, pinned 0.15.17) - [x] New tests added - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_content_router_token_units.py -q 4 passed in 0.33s $ uvx ruff@0.15.17 check headroom/transforms/content_router.py tests/test_content_router_token_units.py All checks passed! ``` **Regression check against clean `upstream/main` in the same environment:** ```text tests/test_transforms/ + test_transforms_content_router.py + kompress suites upstream/main : 2 failed, 468 passed, 78 skipped this branch : 2 failed, 468 passed, 78 skipped failure sets : identical ``` The 2 failures are `test_kompress_failsafe`'s artifact-selection tests, which need a real `onnxruntime` this throwaway env lacks. Unrelated to this change. The 4 new tests pin the unit contract *and* the bounds of the disagreement band — including the cases where both formulations agree, so the band is demonstrated rather than assumed. ## Real Behavior Proof - **Environment:** macOS 26.4 arm64, isolated worktree off `upstream/main`. `content_router` needs the compiled `headroom._core`, which isn't in a fresh worktree (gitignored, built in-place); I copied the built `.so` in to run these, then removed it before committing. - **Observed:** both tables above are from running the real `_estimate_tokens` against the real thresholds, not reconstructed arithmetic. - **Not tested:** no live ONNX inference — the >30s stall the gate prevents is cited from #1171, not reproduced. The CONFIG no-op was demonstrated at the ratio level rather than by driving a stubbed compressor through `apply()`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] I did **not** edit `CHANGELOG.md` ## Related Third of three PRs from one tokenizer-consistency audit — see #2757 (litellm total-prompt / `--budget`) and #2758 (HuggingFace chat templates, `gpt-5`, gateway-wrapped names). Separate subsystems, separate risk. Known remaining from the same audit, not in any of the three: `_netcost_message_tokens` pricing an image by Python `repr` (34× over-count, flag-gated), three transforms reporting via `count_text(str(content))` where the pipeline uses `count_messages` (19% apart in one log file), `frozen_message_count` walking a chars/3.5 estimate against provider-reported cached tokens, and `target_ratio` honoured in words while documented as tokens. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Tejas Chopra
|
a033ac4176
|
fix(cost): send litellm the total prompt so --budget stops seeing $0 (#2757)
## Description **`--budget` has been silently inert on any cache-warm request** — which is the normal case in an agent session. This is a disabled control, not a metrics bug. `record_tokens` passed only the **uncached slice** as litellm's `prompt_tokens`. Measured, `litellm.cost_per_token` charges: ``` (prompt_tokens - cache_read - cache_creation) * input_rate + cache_read * read_rate + cache_creation * write_rate ``` So `prompt_tokens` is the **whole** prompt and litellm removes the cached parts itself. Handing it the uncached slice drives the input term **negative** as soon as anything is cached. `estimate_cost` ends with `float(total) if total > 0 else None`, so it returned `None`, no `CostEntry` was appended, and `check_budget()` saw **$0**. | model | 100k prompt, 80k cached — old call | booked | | --- | --- | --- | | `gpt-5` | **-$0.065000** | None | | `gpt-4o-mini` | **-$0.003000** | None | | `claude-sonnet-4-5` | **-$0.156000** | None | Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made Fixing the total alone would **over-charge OpenAI**, because two bugs are entangled here. OpenAI exposes no cache-write counter, so `_infer_openai_cache_write_tokens` uses the uncached portion as a write proxy. At two of the four inference sites (`openai.py:4304`, `openai.py:5413`) `uncached_input_tokens` is derived by subtracting **only** `cache_read`, so `cache_write_tokens` and `uncached_tokens` are **the same tokens**. Summing all three would double-count the prompt and charge a write premium OpenAI does not have. (The other two sites — `openai.py:3969` and `streaming.py:2024` — subtract both, so their buckets are genuinely disjoint; those are left alone.) - `cost.py` — `record_tokens` now passes `uncached + cache_read + cache_write` as the prompt total. - `cost.py` — new `cache_inferred: bool = False` parameter. When set, the inferred write is excluded from **both** the prompt total and the write premium. The default preserves behaviour for every provider that reports disjoint buckets. - `outcome.py` — plumbs `outcome.cache_inferred` through. The field already existed on `RequestOutcome` for the dashboard; it just never reached cost. - `handlers/openai.py` — sets `cache_inferred=True` at the two outcome sites whose buckets genuinely duplicate. ## Testing - [x] Unit tests pass (new file) - [x] Linting passes (`ruff check` + `format --check`, pinned 0.15.17) - [x] New tests added - [x] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/proxy/cost.py headroom/proxy/outcome.py headroom/proxy/handlers/openai.py All checks passed! $ uvx ruff@0.15.17 format --check <same three> 3 files already formatted $ pytest tests/test_cost_budget_total_prompt.py -q 5 passed in 0.26s ``` The 5 new tests assert on the **arguments handed to `estimate_cost`** rather than on dollar values, so they pin the contract that broke without depending on litellm's pricing tables — or on litellm being installed at all. ## Real Behavior Proof - **Environment:** macOS 26.4 arm64, isolated git worktree off `upstream/main`. **litellm's actual formula**, established by probe rather than assumed: ```text rates (claude-sonnet-4-5): input=3e-06 read=3e-07 write=3.75e-06 total=100k, 80k read, 5k write -> $0.087750 hypothesis (p-r)*ir + r*rr + w*wr = $0.102750 x hypothesis (p-r-w)*ir + r*rr + w*wr = $0.087750 <- matches ``` **Before → after:** ```text Anthropic, disjoint: uncached=900 read=48000 write=1500 OLD prompt=900 raw=-0.125775 booked=None <-- BUDGET BLIND NEW prompt=50400 raw= 0.022725 booked=0.022725 OpenAI, inferred write == uncached: uncached=20000 read=80000 write=20000 OLD prompt=20000 raw=-0.090000 booked=None <-- BUDGET BLIND NEW prompt=100000, write excluded raw= 0.035000 booked=0.035 truth=0.035000 ``` The OpenAI "after" equals the hand-computed truth `20,000*input + 80,000*read_rate` exactly. - **Not fully tested locally.** The cost/outcome suites show an **identical 10-failure set** on this branch and on clean `upstream/main` in the same throwaway env — all `ModuleNotFoundError: headroom._core`, the compiled Rust extension this machine cannot currently build. So they are environment-only, not regressions. One of them, `test_funnel_passes_canonical_record_tokens_shape`, covers the `record_tokens` call shape this PR changes, so **CI is the authoritative check for that one**. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] I did **not** edit `CHANGELOG.md` ## Context Found during a tokenizer-consistency audit that also turned up: HuggingFace-routed models counting a 6,000-char message as **2 tokens**, `gpt-5`/`o4-mini` falling to a char estimator, and the Kompress size gate missing its own cap by 24%. Those are separate PRs — different subsystems, different risk. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |