mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
29 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Abhay Singh
|
38306a331c
|
fix(proxy/gemini): thread savings-profile kwargs into apply() (#1994)
## Description
The native Gemini / Vertex-google compression handlers build the
pipeline call like this
(`headroom/proxy/handlers/gemini.py`, three sites —
`handle_gemini_generate_content` ~L487,
`handle_google_cloudcode_stream` ~L843, `handle_gemini_count_tokens`
~L1104):
```python
result = await self._run_compression_in_executor(
lambda: self.openai_pipeline.apply(
messages=messages,
model=model,
model_limit=context_limit,
context=extract_user_query(messages),
waste_messages=waste_messages,
), # <-- no **proxy_pipeline_kwargs(self.config)
...
)
```
Every other live compression path threads
`proxy_pipeline_kwargs(self.config)` into `apply()`
— `handlers/openai.py` (chat + responses), `handlers/anthropic.py`, and
the `/v1/compress`
endpoint. The Gemini handler never imports or calls it, so the savings
profile and the
ProxyConfig compression knobs never reach the pipeline for Gemini/Vertex
requests.
The proxy pipeline is built with only `transforms` + `provider`
(`server.py`), and
`ContentRouter` reads the accuracy-sensitive knobs per-call from
`**kwargs`. With the kwargs
missing, Gemini falls back to router defaults instead of the
profile/config values:
- `min_tokens_to_compress` → hardcoded **50** instead of the
coding-profile **25** / `config.min_tokens_to_crush`
- `protect_recent` → router default instead of the profile / configured
value
- `target_ratio` → **None** instead of the CLI default **0.4** used
everywhere else
- `max_items_after_crush`, `smart_crusher_with_compaction`,
`force_kompress` → router defaults
So Gemini/Vertex requests compress with a materially different (and
inconsistent) posture than
Claude/Codex/Cursor, and any user-tuned `HEADROOM_SAVINGS_PROFILE` /
`HEADROOM_TARGET_RATIO` / `HEADROOM_MIN_TOKENS` /
`HEADROOM_PROTECT_RECENT` is ignored on this
path.
This is the exact bug **#1534** fixed for the OpenAI chat path.
Closes: no issue filed — found while auditing profile/config threading
across the provider handlers.
## Fix
Import `proxy_pipeline_kwargs` (as `openai.py`/`anthropic.py` do) and
add
`**proxy_pipeline_kwargs(self.config)` to all three Gemini
`openai_pipeline.apply(...)` calls.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/handlers/gemini.py`: import `proxy_pipeline_kwargs`;
thread `**proxy_pipeline_kwargs(self.config)` into the three
`openai_pipeline.apply(...)` call sites (generateContent, Cloud Code
stream, countTokens).
- `tests/test_proxy/test_gemini_savings_profile.py`: drive the native
`/v1beta/models/{model}:generateContent` route with
`savings_profile="agent-90"` and assert the profile knobs
(`compress_user_messages`, `target_ratio`, `min_tokens_to_compress`,
`compress_system_messages`) reach `apply()`.
## Testing
- [x] New regression test added
(`tests/test_proxy/test_gemini_savings_profile.py`), mirroring the #1534
chat-path test
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`) — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/gemini.py tests/test_proxy/test_gemini_savings_profile.py
All checks passed!
$ uvx ruff@0.15.17 format --check headroom/proxy/handlers/gemini.py tests/test_proxy/test_gemini_savings_profile.py
2 files already formatted
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the kwargs threading
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: mechanically reproduced the call site —
`apply(**base)` (old) vs `apply(**base,
**proxy_pipeline_kwargs(config))` (new) — and captured the kwargs each
produced.
- Observed result: the old call omits the profile knobs entirely; the
new call threads them:
```text
OLD gemini apply() kwargs: ['context', 'messages', 'model', 'model_limit', 'waste_messages']
-> profile knobs DROPPED (savings profile / config ignored)
NEW gemini apply() kwargs: ['compress_system_messages', 'compress_user_messages', 'context',
'max_items_after_crush', 'messages', 'min_tokens_to_compress', 'model', 'model_limit',
'protect_recent', 'target_ratio', 'waste_messages']
-> profile knobs THREADED: target_ratio=0.10, min_tokens_to_compress=120, compress_user/system=True
GEMINI KWARGS THREADING VERIFIED
```
- Not tested: a full request through a live Gemini/Vertex upstream
(needs the heavy stack + a key). The new test drives the native route
with a mocked upstream and asserts the kwargs reach `apply()`. Full
local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No new dependencies; one import plus three call-site kwargs and a
test.
- @JerrettDavis tagging you — this is the Gemini sibling of the #1534
chat-path fix; the profile/config knobs are currently ignored for the
whole Gemini/Vertex surface, so it may be worth a look when you have a
moment.
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
Adryan Eka Vandra
|
bb2acf700a
|
fix(proxy): honor x-headroom-base-url on /v1/messages route (#1763)
## Description The Anthropic Messages route (`POST /v1/messages`) ignored the `x-headroom-base-url` per-request upstream override and unconditionally forwarded to `api.anthropic.com`. `handle_anthropic_messages` already accepts `upstream_base_url` (it builds the upstream URL via `build_copilot_upstream_url`), but the route never passed it. Clients that speak the Anthropic Messages wire format while authenticating against a non-Anthropic gateway (e.g. OpenCode Zen's "Go" tier) were forwarded to the real Anthropic API, which rejected the gateway key with `401 invalid x-api-key`. The route now reads and trims `x-headroom-base-url` and passes it through as `upstream_base_url`, mirroring the OpenAI-compatible routes and the generic passthrough route (`proxy_routes.py:996`). Closes #1760 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/providers/proxy_routes.py`: the `/v1/messages` route reads `x-headroom-base-url`; when present it strips whitespace and a trailing slash and passes the value as `upstream_base_url` to `handle_anthropic_messages`. Absent or whitespace-only headers keep the previous default (`api.anthropic.com`). - `tests/test_proxy/test_anthropic_upstream_header.py`: new test module pinning the route contract (header present, absent, empty, whitespace-only, trimming + trailing-slash stripping). - `docs/content/docs/configuration.mdx`: new "Proxy upstream override (`x-headroom-base-url`)" subsection under Per-Request Overrides documenting the header across the OpenAI, Anthropic Messages, and passthrough routes. - `CHANGELOG.md`: `Unreleased > Fixed` entry for the `/v1/messages` override. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_proxy/ -k "anthropic or passthrough or bedrock" collected 140 items / 91 deselected / 49 selected tests/test_proxy/test_anthropic_upstream_header.py .... [ 65%] ... 49 passed, 91 deselected, 1 warning in 79.68s $ ruff check headroom/providers/proxy_routes.py tests/test_proxy/test_anthropic_upstream_header.py All checks passed! $ mypy headroom/providers/proxy_routes.py Success: no issues found in 1 source file ``` ## Real Behavior Proof Ran the actual `headroom proxy` against a local mock upstream (a tiny HTTP server on `127.0.0.1:9911` that logs the path it receives) to reproduce the issue's before/after. - Environment: local, macOS, Python 3.12; ran `headroom proxy --port 8799` against a local mock upstream (a tiny HTTP server on `127.0.0.1:9911` that logs the path it receives). - Exact command / steps: started the proxy and the mock upstream, then sent one `POST /v1/messages` **with** the override header and one **without** it (negative control), using these two `curl` commands. ```bash # WITH the override header — expect routing to the mock at 127.0.0.1:9911 curl http://127.0.0.1:8799/v1/messages \ -H "content-type: application/json" -H "anthropic-version: 2023-06-01" \ -H "x-headroom-base-url: http://127.0.0.1:9911" \ -H "x-api-key: zen-test-key" \ -d '{"model":"glm-5.2","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}' # WITHOUT the override header — expect routing to the real api.anthropic.com curl http://127.0.0.1:8799/v1/messages \ -H "content-type: application/json" -H "anthropic-version: 2023-06-01" \ -H "x-api-key: sk-ant-fake" \ -d '{"model":"claude-3-5-sonnet-20241022","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}' ``` - Observed result: with the header, the mock upstream logged `HIT path=/v1/messages x-api-key=zen-test-key` and the proxy returned `HTTP 200`, confirming the request was routed to `<x-headroom-base-url>/v1/messages` carrying the gateway key. Without the header, the request went to the real `api.anthropic.com` (returned `HTTP 401` with a genuine `request_id` and `{"type":"authentication_error","message":"invalid x-api-key"}`) and the mock received no additional hit — matching the pre-fix behavior in the issue. Also verified by TDD: the two override unit cases failed before the route change (`assert None == 'https://opencode.ai/zen/go'`) and passed after it; all 4 new cases and 49 related proxy tests are green. - Not tested: a request against the real OpenCode Zen gateway (no credentials); the gateway path is verified with a local mock upstream instead. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Manual testing against the real OpenCode Zen gateway is N/A (no credentials); a local mock upstream is used instead to prove the routing (see Real Behavior Proof). - Scope is limited to `/v1/messages`. The related `/v1/messages/count_tokens` route uses a fixed passthrough target and is out of scope for this issue. |
||
|
Rod Boev
|
62cd3072a2
|
feat(ccr): wire retrieve-tool interception into OpenAI Responses handler (#1898)
## Description Refs #1877. `handle_openai_responses` (the `/v1/responses` HTTP handler) had zero CCR / `headroom_retrieve` wiring, so a retrieve `function_call` in a Responses API reply passed straight through to the client instead of being resolved server-side, unlike the parallel chat-completions backend path (`handle_openai_chat`, ~2775-2848), which already intercepts `headroom_retrieve` tool calls via `ccr_response_handler.has_ccr_tool_calls()` / `handle_response()`. This PR is scoped to the core interception gap only. The issue's proposals A (egress scrubber) and B/C (event-level SSE parsing/splicing for true mid-stream interception) are out of scope here; the streaming case is instead handled by forcing a buffered (non-streaming) upstream call when `headroom_retrieve` is offered, matching the existing buffered-CCR pattern in the Anthropic handler. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/ccr/response_handler.py`: added an `"openai_responses"` provider branch to `CCRResponseHandler`; `_extract_tool_calls` reads flat `function_call` items from the top-level `output[]` array, tool-call IDs key off `call_id`, and `_extract_assistant_message` / `_create_tool_result_message` return sentinel-keyed item lists that `handle_response()` extends into the running item history. - `headroom/ccr/tool_injection.py`: added a `parse_tool_call` branch for `"openai_responses"` where name and arguments are flat on the item. - `headroom/proxy/handlers/openai.py`: detects non-streaming `headroom_retrieve` function calls and runs `ccr_response_handler.handle_response()` with a stateless continuation that resends the full `input[]` item history. - `headroom/proxy/handlers/openai.py`: forces `stream:true` requests with `headroom_retrieve` available through a buffered `stream:false` upstream call, resolves retrieval server-side, then reconstructs a minimal Responses SSE stream for the client. - `headroom/proxy/handlers/openai.py`: treats `ccr_response_handler` as optional on `OpenAIHandlerMixin` consumers, so handlers without CCR support keep the existing Codex routing, streaming, header stripping, memory timeout, and compression fail-open behavior. - Non-CCR streaming requests are unaffected; they still go through `_stream_response()` as before. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_ccr_response_handler_openai_responses.py -q`) - [x] Integration tests pass (`uv run pytest tests/test_proxy/test_openai_responses_ccr.py -q`) - [x] Regression tests pass (`uv run pytest tests/test_openai_codex_routing.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_routing.py tests/test_proxy/test_openai_responses_ccr.py tests/test_ccr_response_handler_openai_responses.py`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_openai_codex_routing.py -q tests\test_openai_codex_routing.py .................... [100%] 20 passed in 0.51s $ uv run pytest tests/test_proxy/test_openai_responses_ccr.py tests/test_ccr_response_handler_openai_responses.py -q tests\test_proxy\test_openai_responses_ccr.py .... [ 25%] tests\test_ccr_response_handler_openai_responses.py ............ [100%] 16 passed, 1 warning in 27.51s $ uv run ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_routing.py tests/test_proxy/test_openai_responses_ccr.py tests/test_ccr_response_handler_openai_responses.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, Python 3.12 via uv-managed venv, local PR worktree on branch `pr/1877-ccr-responses-interception`, no live LLM provider needed because the tests stub upstream HTTP and CCR continuation behavior. - Exact command / steps: Ran `uv run pytest tests/test_openai_codex_routing.py -q` to reproduce the CI-failing Codex routing surface after the optional-handler fix; ran `uv run pytest tests/test_proxy/test_openai_responses_ccr.py tests/test_ccr_response_handler_openai_responses.py -q` to cover the positive Responses CCR interception path; ran targeted Ruff on the touched handler and related tests. - Observed result: Codex routing tests that previously failed with `AttributeError: '_DummyOpenAIHandler' object has no attribute 'ccr_response_handler'` now pass; Responses CCR still detects and resolves `headroom_retrieve` when a real proxy installs `ccr_response_handler`; non-CCR streaming requests still route through `_stream_response()`. - Not tested: true event-level mid-stream Responses SSE splicing and client-bound egress marker scrubbing are out of scope for this PR and remain future work from issue #1877's broader proposals. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes No documentation or changelog update was made because this is internal proxy CCR behavior, not a user-facing command or configuration change. |
||
|
Tejas Chopra
|
7c2f0ea079
|
feat(cache): provider-agnostic cache-mode delta + cc-agnostic prefix comparison (#1868)
## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> |
||
|
Vinay Gupta
|
f18c6bd896
|
fix(codex): OpenCode Zen telemetry attribution (#1648)
## Description Fixes #1602. OpenCode Zen custom-base requests can reach Headroom through the generic passthrough path, but that route was not supplying endpoint/provider metadata for Zen chat completions. This made forwarded Zen traffic invisible in dashboard provider, usage, and token telemetry. Closes #1602 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a narrow OpenCode Zen custom-base classifier for `POST /zen/v1/chat/completions` on `opencode.ai` and `www.opencode.ai`. - Passed `endpoint_name="chat/completions"` and `provider="zen"` into catch-all passthrough telemetry for matching Zen traffic. - Attributed normalized OpenCode transport traffic (`/v1/chat/completions` with `x-headroom-original-path: /zen/v1/chat/completions`) to `zen` for request outcomes while keeping the OpenAI parser path unchanged. - Added coverage for direct catch-all routing, normalized original-path routing, token usage outcome recording, and false-positive paths like `/mcp/v1/chat/completions`, `/npm/v1/chat/completions`, and `/context7/v1/chat/completions`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ rtk pytest tests/test_custom_base_passthrough_telemetry.py -q Pytest: 4 passed $ rtk uvx --from ruff==0.15.17 ruff check headroom/proxy/handlers/openai.py headroom/providers/proxy_routes.py tests/test_custom_base_passthrough_telemetry.py tests/test_provider_proxy_routes.py tests/test_proxy/test_openai_transport_path_prefix.py All checks passed! $ rtk uvx --from ruff==0.15.17 ruff format --check headroom/proxy/handlers/openai.py headroom/providers/proxy_routes.py tests/test_custom_base_passthrough_telemetry.py tests/test_provider_proxy_routes.py tests/test_proxy/test_openai_transport_path_prefix.py 5 files already formatted $ rtk /Library/Frameworks/Python.framework/Versions/3.13/bin/python3 -m py_compile headroom/proxy/handlers/openai.py headroom/providers/proxy_routes.py tests/test_custom_base_passthrough_telemetry.py tests/test_provider_proxy_routes.py tests/test_proxy/test_openai_transport_path_prefix.py # passed $ rtk git diff --check # passed ``` GitHub Actions also passed after the final push, including CI, Docker native/wrap/init E2E, security, lint, and PR governance. ## Real Behavior Proof - Environment: local worktree on macOS plus GitHub Actions for PR #1648. - Exact command / steps: ran focused pytest coverage for Zen passthrough telemetry, Ruff check/format validation on touched files, Python compile validation, `git diff --check`, and waited for the full GitHub Actions rollup. - Observed result: Zen custom-base chat completions now record request outcomes as provider `zen` with endpoint `chat/completions`; false-positive OpenCode paths remain unattributed to Zen; GitHub checks are green. - Not tested: full local test suite did not collect in this worktree because the native `headroom._core` extension is not installed. `rtk npm --prefix plugins/opencode test` is also blocked locally because `vitest` is not installed in `plugins/opencode/node_modules`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes The documentation and CHANGELOG checklist items are not applicable for this narrow telemetry bug fix. No new comments were added because the code path is covered by narrowly named helper/test cases. |
||
|
Abhay Singh
|
7ff842da17
|
fix(proxy/openai): thread savings-profile kwargs into chat completions (#1606)
## Description OpenAI-compatible `/v1/chat/completions` requests didn't receive the same proxy savings/profile kwargs as the other compression paths. The live chat handler (`handle_openai_chat` in `headroom/proxy/handlers/openai.py`) called `openai_pipeline.apply()` with only `model_limit` / `context` / `frozen_message_count` / `biases` / `compression_policy` — it never passed `proxy_pipeline_kwargs(self.config)`. So when the proxy runs with `HEADROOM_SAVINGS_PROFILE=agent-90`, the effective config reports user/system-message compression and `target_ratio=0.10`, but the real chat path silently dropped all of it. OpenAI-compatible clients such as OpenCode kept protecting user messages and missed the configured profile. For contrast, `handlers/anthropic.py` passes `**proxy_pipeline_kwargs(self.config)` to every `apply()` call, and so does the dedicated OpenAI compress endpoint in this same module — only the two chat-completions `apply()` sites were missing it. Closes #1534 ## Fix Add `**proxy_pipeline_kwargs(self.config)` to both chat-path `apply()` calls (the token-mode branch and the non-token branch): ```python lambda: self.openai_pipeline.apply( messages=messages, model=model, model_limit=context_limit, context=extract_user_query(messages), frozen_message_count=openai_frozen_count, biases=_hook_biases, compression_policy=compression_policy, **proxy_pipeline_kwargs(self.config), # ← added ) ``` `proxy_pipeline_kwargs` is already imported in the module and is the exact helper the Anthropic handler and the OpenAI compress endpoint use, so the chat path now matches them. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/handlers/openai.py`: pass `**proxy_pipeline_kwargs(self.config)` on both `apply()` call sites in `handle_openai_chat` (token-mode and non-token branches). - `tests/test_proxy/test_openai_chat_savings_profile.py`: new regression test driving the chat handler with `savings_profile="agent-90"` and asserting the profile knobs reach `apply()`. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output The new test drives the real chat handler through the `create_app` + `TestClient` harness with a recording `apply()` stub. Before the fix it captures exactly the five kwargs the issue describes (no profile knobs); after the fix the profile knobs are present: ```text # before the fix (openai.py reverted, test kept) E AssertionError: assert None is True E + where None = {...}.get('compress_user_messages') # captured kwargs were: biases, compression_policy, messages, model, # model_limit, context, frozen_message_count — no profile knobs FAILED tests/test_proxy/test_openai_chat_savings_profile.py::test_chat_completions_threads_savings_profile_kwargs_into_apply # after the fix tests\test_proxy\test_openai_chat_savings_profile.py . ======================== 1 passed, 1 warning in 39.44s ======================== ``` No regression in the existing chat backend-path suite: ```text $ uv run pytest tests/test_proxy/test_openai_backend_path.py ======================== 5 passed, 1 warning in 15.78s ======================== $ uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_chat_savings_profile.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom built from this branch (`uv sync --extra dev`), proxy config `savings_profile="agent-90"`, `optimize=True`, `backend="anyllm"` with a mocked OpenAI upstream. - Exact command / steps: started the app with `create_app(config)`, replaced `proxy.openai_pipeline.apply` with a recording stub, and POSTed a real `/v1/chat/completions` request with a large user message so the compression decision fires. Inspected the kwargs the handler actually passed to `apply()`. - Observed result: before the fix the recorded `apply()` kwargs were `{biases, compression_policy, messages, model, model_limit, context, frozen_message_count}` — no profile knobs. After the fix the same call also carries `compress_user_messages=True`, `compress_system_messages=True`, `target_ratio=0.10`, `min_tokens_to_compress=120` (the agent-90 profile), matching the issue's "Expected". - Not tested: did not stand up a real OpenAI/OpenCode upstream end-to-end (no live key in this environment); the upstream is mocked and the assertion is on the kwargs the proxy threads into the compression pipeline, which is exactly what the bug was about. Did not run the full `mypy headroom` pass (two-line kwarg addition, no new types). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Two-line change plus comments; no new dependencies. Reuses the existing `proxy_pipeline_kwargs` helper, so behavior is consistent across Anthropic, the OpenAI compress endpoint, and now the OpenAI chat path. - @chopratejas flagging you for review — this aligns the OpenAI chat path with the savings-profile handling the other providers already had. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Tejas Chopra
|
248ae0f3e0
|
fix(proxy): freeze must forward cached (compressed) prefix byte-identical — stop token-mode cache busting (#1850)
The freeze path (both providers) emits the agent's ORIGINAL bytes for a frozen message, but the provider cached whatever we FORWARDED last turn (the compressed form). Forwarding original then mismatches the cached prefix and busts it from that point — re-creating the whole suffix. Measured on a real SWE-bench run: 100% of attributed misses were prefix_change, ~56% of ALL cache-writes were bust-induced (2.8M tokens), driving cache_create +150% and cost +41% vs baseline. Cache mode already avoided this via _extract_cache_stable_delta (replay the previously-forwarded prefix, compress only the delta). Token mode called apply(frozen_count) directly, which forwards original for the frozen region. Fix: add a shared, provider-agnostic overlay_cached_prefix() that replays the previously-forwarded (cached, compressed) prefix byte-identical, append-only guarded and idempotent, and apply it in BOTH the Anthropic and OpenAI handlers right before forwarding. This makes freezing byte-identical in every mode, so the only remaining difference between "token" and "cache" mode is how large a mutable (still-compressible) tail each leaves — not whether the frozen prefix busts the cache. Tests: - test_cache_prefix_overlay.py: the helper (replay, append-only guard, idempotence). - test_cross_turn_cache_safety.py: the invariant that was missing — drive the REAL tracker + freeze + overlay over multiple append-only turns against a simulated provider prefix cache and assert the forwarded prefix stays byte-identical turn-over-turn. Load-bearing: it fails (detects the bust) without the overlay. ## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> |
||
|
Kirill
|
6c48ac81f2
|
fix(proxy): honor x-headroom-base-url in dedicated OpenAI handlers (#1502)
## Description The dedicated OpenAI handlers (`/v1/chat/completions`, `/v1/responses`) ignore the `x-headroom-base-url` request header that the opencode/CLI transports already send on every routed request (`plugins/opencode/src/transport.ts`) and that the generic passthrough route already honors (`providers/proxy_routes.py:953`). As a result, OpenAI-compatible gateways (LiteLLM, CPA, self-hosted vLLM, Azure OpenAI) route correctly for passthrough traffic, but the dedicated chat/responses handlers fall back to the default `OPENAI_API_URL` and send the request — and the user's provider key — to the wrong upstream. This forces OpenCode users behind a custom gateway to run a hand-rolled plugin that re-spawns the proxy with `OPENAI_TARGET_API_URL` instead of the supported `HeadroomPlugin`. Refs #1503 (feature-request issue with full spec — API surface, failure modes, security considerations). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) Non-breaking: when the header is absent (the common case), behavior is identical to before — `_resolve_openai_upstream` falls back to `self.OPENAI_API_URL`. ## Changes Made - Added `OpenAIHandlerMixin._resolve_openai_upstream(request)` — returns `request.headers.get("x-headroom-base-url") or self.OPENAI_API_URL`. Prefers the header, falls back to the configured URL. - Used it at the two direct-path HTTP upstream sites: - `handle_openai_chat` → `build_copilot_upstream_url(self._resolve_openai_upstream(request), "/v1/chat/completions")` - `handle_openai_responses` → `build_copilot_upstream_url(self._resolve_openai_upstream(request), "/v1/responses")` - This makes the dedicated handlers behave identically to the catch-all passthrough and the Azure path (`_select_passthrough_base_url`, `providers/proxy_routes.py:66,:953`), which already read the same header. - The header is already stripped before forwarding by `helpers._strip_internal_headers`, so no upstream leakage / fingerprinting is introduced. - CHANGELOG entry under `### Bug Fixes`. ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) — not run locally (maturin native build not available in my env; covered by CI) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output New `tests/test_proxy/test_openai_upstream_header.py` pins the resolution contract (3 cases): ```text $ pytest tests/test_proxy/test_openai_upstream_header.py -q ... collected 3 items tests/test_proxy/test_openai_upstream_header.py ... [100%] ========================= 3 passed, 1 warning in 0.25s ========================= ``` Fail-before confirmed (unpatched handler raises `AttributeError: _resolve_openai_upstream`): ```text FAILED tests/test_proxy/test_openai_upstream_header.py::test_header_overrides_configured_url FAILED tests/test_proxy/test_openai_upstream_header.py::test_missing_header_falls_back_to_configured_url FAILED tests/test_proxy/test_openai_upstream_header.py::test_empty_header_falls_back_to_configured_url ========================= 3 failed, 1 warning in 0.29s ========================= ``` Lint/format: ```text $ ruff check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_upstream_header.py Ruff: No issues found $ ruff format --check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_upstream_header.py 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS (darwin), Python 3.12 (pipx install of `headroom-ai`), Headroom proxy `headroom proxy --port 8787` with `OPENAI_TARGET_API_URL=https://cpa.funxyz.fun` (an OpenAI-compatible gateway — "CLI Proxy API"). OpenCode with a custom `cpa` provider (`@ai-sdk/openai-compatible`, `baseURL: https://cpa.funxyz.fun/v1`) using the official `HeadroomPlugin`. - Exact command / steps: traced the bug in the installed package source — confirmed `handle_openai_chat` builds its upstream URL from `self.OPENAI_API_URL` only (`proxy/handlers/openai.py:2487`), never reading `x-headroom-base-url`, while `providers/proxy_routes.py:953` reads it for passthrough. Then applied this patch and re-imported the handler from the repo source via `PYTHONPATH`. - Observed result: before the patch, `/v1/chat/completions` requests ignored the `x-headroom-base-url: https://cpa.funxyz.fun` header (set by the opencode transport) and routed to the default upstream, failing against a non-OpenAI gateway — requiring a custom respawn-plugin workaround. After the patch, `_resolve_openai_upstream` returns the header value and the request forwards to the configured gateway; the official `HeadroomPlugin` works without the env-var workaround. Unit tests pass (3/3) and fail on the unpatched handler (3/3). - Not tested: full `uv sync` CI matrix (native `headroom._core` maturin build unavailable locally, so `headroom.proxy.server` import chain that pulls `transforms/content_router` can't be exercised here — the edited handler module imports fine and the focused unit tests exercise the new method directly). WebSocket/Codex paths (`handle_openai_responses_ws`, `_ws_http_fallback`) — intentionally out of scope (see Additional Notes). `mypy headroom` — deferred to CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — no public API/docs surface; the header is already documented as an internal control flag in `helpers.py:1489-1495` - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes **Scope boundary — WebSocket paths intentionally unchanged.** The two WS sites (`handle_openai_responses_ws`, `_ws_http_fallback`) are Codex-specific and left as-is: 1. They short-circuit to `chatgpt.com` under ChatGPT-session auth (not arbitrary gateways). 2. The WS path strips `x-headroom-base-url` from `upstream_headers` (`_strip_internal`, ~line 3756) before the upstream URL is built, and `_ws_http_fallback` receives already-stripped headers as a parameter. Honoring the header there would require threading it through the WS internals and changing a signature, for a path a custom OpenAI-compatible WebSocket gateway is unlikely to use. The HTTP paths cover the realistic gateway case. Happy to do it as a follow-up if maintainers want it. **Issue-first.** This is a behaviour change, so per CONTRIBUTING a feature-request issue (#1503) is open for triage with the full spec (API surface, user stories, failure modes, security). This PR implements it; holding for maintainer 👍 before treating as ready to merge. --------- Co-authored-by: ShutovKS <shutovks@example.local> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Vinay Gupta
|
d337e3b828
|
fix(proxy): handle streaming CCR retrieval (#1451)
## Description Fixes Anthropic-compatible streaming requests that can emit the internal `headroom_retrieve` CCR tool. When a `stream: true` request includes the CCR retrieve tool and response handling is enabled, Headroom now buffers the upstream call as `stream: false`, lets the existing CCR response handler retrieve and continue, and returns the final result as Anthropic SSE so streaming clients do not see the internal tool call. Closes #1450 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Detect direct Anthropic-compatible `stream: true` requests where `headroom_retrieve` is available and CCR response handling is enabled. - Route those requests through the existing buffered/non-stream CCR response handler, then convert the final response back to `text/event-stream`. - Fail closed with a 502 SSE error if a buffered response still contains `headroom_retrieve` after CCR handling, instead of leaking the internal tool to the client. - Preserve Anthropic `thinking`, `redacted_thinking`, signatures, and citations when converting response JSON back to SSE. - Add regression coverage for handled CCR retrieval, unused CCR tool availability, normal streaming passthrough, mixed client/CCR tool fail-closed behavior, and SSE conversion preservation. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ rtk gh pr checks 1451 --repo headroomlabs-ai/headroom CI Checks Summary: [ok] Passed: 20 [FAIL] Failed: 0 Relevant CI commands from .github/workflows/ci.yml: - ruff check . - ruff format --check . - mypy headroom --ignore-missing-imports - pytest tests scripts/tests $ rtk python3 -m py_compile headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/streaming.py tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_sse_thinking_blocks.py # passed, no output $ rtk pytest tests/test_sse_thinking_blocks.py -q Pytest: 6 passed $ MACOSX_DEPLOYMENT_TARGET=15.0 rtk uv run --python 3.13 pytest tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py -q Failed before test collection while building the local editable package: esaxx-rs build failed with fatal error: 'cstdint' file not found. ``` ## Real Behavior Proof - Environment: GitHub Actions CI on PR #1451 plus local macOS worktree `fix/1450-ccr-streaming-retrieve`. - Exact command / steps: CI ran lint, type checking, build, unit-test shards, dashboard tests, extras tests, and e2e jobs; locally ran syntax checks and the SSE conversion regression tests. - Observed result: CI passed 20 checks with 0 failures; local syntax checks passed; `tests/test_sse_thinking_blocks.py` passed with 6 tests. - Not tested: the new proxy-level regression test was not run locally because the local native extension build fails in `esaxx-rs` before proxy tests can collect; it is included in the CI-tested suite. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - Scope: this handles the direct Anthropic-compatible HTTP `/v1/messages` path. The configured Bedrock/backend streaming path does not share this CCR continuation machinery in this PR. - Documentation, CHANGELOG, code-comment, and local-full-test checklist items are N/A for this narrow bug fix or not true locally. |
||
|
Rod Boev
|
c19347c310
|
fix(opencode): preserve custom OpenAI gateway paths (#1596)
## Description Custom OpenAI-compatible gateways mounted under provider-specific prefixes could miss Headroom's dedicated OpenAI compression routes when used through the OpenCode transport. A request such as `https://open.bigmodel.cn/api/coding/paas/v4/chat/completions` was replayed to the proxy at `/api/coding/paas/v4/chat/completions`, so the proxy selected catch-all passthrough instead of `/v1/chat/completions`. This change keeps the proxy-facing entrypoints stable on `/v1/chat/completions` and `/v1/responses` for OpenAI-compatible suffixes, while preserving the original upstream path in an internal header so the dedicated OpenAI handlers can reconstruct the real provider URL. Closes #1582 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Normalize opencode-routed OpenAI-compatible `/chat/completions` and `/responses` requests onto the proxy's stable `/v1/*` routes. - Preserve the original upstream pathname in an internal `x-headroom-original-path` signal for dedicated OpenAI handler reconstruction. - Reconstruct dedicated OpenAI upstream URLs from `x-headroom-base-url` plus the preserved path prefix, while preserving request query strings and rejecting non-HTTP base hints. - Keep nearby non-OpenAI paths such as `/base/v1/messages` on existing passthrough behavior. - Add focused transport and proxy regression coverage for prefixed gateway paths, invalid fallback cases, and internal-header stripping. ## Testing - [x] Transport regression tests pass (`npm --prefix plugins/opencode test -- src/transport.test.ts`) - [x] Proxy regression tests pass (`uv run pytest tests/test_proxy/test_openai_transport_path_prefix.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_transport_path_prefix.py`) - [x] Type checking passes (`npm --prefix plugins/opencode run typecheck`) - [x] New tests added for the bugfix - [ ] Manual testing performed ### Test Output ```text npm --prefix plugins/opencode test -- src/transport.test.ts PASS, 11 tests passed. npm --prefix plugins/opencode run typecheck PASS uv run pytest tests/test_proxy/test_openai_transport_path_prefix.py -q PASS, 7 tests passed. uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_transport_path_prefix.py PASS, all checks passed. ``` ## Real Behavior Proof - Environment: Windows, Python 3.12 via `uv`, Node 18+, focused OpenCode transport and proxy handler tests. - Exact command / steps: on `origin/main`, copy the updated `plugins/opencode/src/transport.test.ts` into a base worktree and run `npm --prefix plugins/opencode test -- src/transport.test.ts`; on this branch, rerun that transport test plus `npm --prefix plugins/opencode run typecheck` and `uv run pytest tests/test_proxy/test_openai_transport_path_prefix.py -q`. - Observed result: the base worktree fails because prefixed `/chat/completions` and `/responses` requests still enter the proxy at their provider path, while this branch passes with `/v1/chat/completions` and `/v1/responses`, preserves `x-headroom-original-path`, reconstructs the provider-prefixed upstream URL and query string, falls back safely on invalid hints, and keeps nearby `/base/v1/messages` traffic on passthrough. - Not tested: full CI suite, live BigModel traffic, and generic catch-all passthrough compression. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This completes the transport contract introduced in https://github.com/headroomlabs-ai/headroom/pull/1573 by keeping prefixed OpenAI-compatible traffic on Headroom's stable `/v1/*` surface while preserving the real upstream path for dedicated-handler reconstruction. https://github.com/headroomlabs-ai/headroom/pull/1367 is adjacent global proxy configuration work for direct deployments; this PR is the per-request OpenCode transport fix for custom upstream path prefixes. `CHANGELOG.md` is intentionally unchanged because this repo's release pipeline generates changelog entries from conventional commits. This stays scoped to `/chat/completions` and `/responses` suffixes. Generic catch-all passthrough compression remains separate from this bugfix slice. |
||
|
Omar Garcia
|
8e0dadfe02
|
fix: restore token-mode compression on frozen prefixes (#1489)
## Description Fixes token-mode compression for continued Claude Code turns with a frozen prefix when the client has not already supplied `headroom_retrieve`. The previous guard returned before request-side compression could run in token mode. This keeps the non-token safety behavior, but lets token mode use the existing marker-triggered CCR tool injection override so emitted markers stay redeemable. Closes #1487. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Let Anthropic token mode run request-side compression even when the client did not pre-register `headroom_retrieve`. - Kept the deferred-injection skip for cache-mode coverage. - Added a regression for the frozen-prefix token-mode path. - Updated `CHANGELOG.md` for the user-facing behavior change. ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ rtk uv run pytest -q tests/test_proxy/test_anthropic_ccr_deferred_injection.py 15 passed, 1 warning in 2.73s $ rtk uv run ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy/test_anthropic_ccr_deferred_injection.py All checks passed! $ rtk uv run ruff format --check headroom/proxy/handlers/anthropic.py tests/test_proxy/test_anthropic_ccr_deferred_injection.py 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS, Python 3.12.9, local FastAPI `TestClient`, Anthropic proxy path, `mode=token`, `ccr_inject_tool=True`, frozen prefix count = 1, no client-supplied `headroom_retrieve`. - Exact command / steps: ran a local `rtk uv run python` repro that builds `create_app(ProxyConfig(...))`, forces compression on the Anthropic path, simulates a frozen prefix, and posts `/v1/messages`. - Observed result: local `TestClient` request returned `STATUS=200`; token-mode frozen-prefix compression ran once with `FROZEN_MESSAGE_COUNT=1`; the forwarded message was the CCR marker; forwarded tools included `headroom_retrieve`. ```text STATUS= 200 FROZEN_MESSAGE_COUNT= 1 COMPRESSION_CALLS= 1 FORWARDED_MESSAGES= [{'role': 'user', 'content': '[100 items compressed to 10. Retrieve more: hash=abc123def456abc123def456]'}] FORWARDED_TOOLS= ['headroom_retrieve'] ``` - Not tested: live Claude Code session against a real Anthropic upstream, full repo-wide `uv run pytest`, and `mypy headroom`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas (N/A: no new hard-to-follow block needed) - [x] I have made corresponding changes to the documentation (N/A: changelog update covers this user-facing bug fix) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A; proxy behavior only. ## Additional Notes The pytest run still emits the existing Starlette/httpx deprecation warning from `fastapi.testclient`; this PR does not touch that dependency path. |
||
|
Lucas Santos
|
43494ff526
|
fix(proxy): stop re-compressing headroom_retrieve output and emitting unredeemable markers (#1323)
## Description Two related CCR problems that both end in unreadable content. The first one (#1077) is an infinite loop. Any tool output over ~500 bytes gets replaced with a `<<ccr:hash>>` marker, and you call `headroom_retrieve` to get the original back. But the proxy then compresses the *retrieve response too*, so what comes back is a brand new marker. Retrieve that one and you get another marker. The second one (#1006), the proxy makes two independent decisions per request: SmartCrusher compresses, and the `headroom_retrieve` tool gets injected. The injection is deferred when there's a frozen message prefix (`frozen_message_count > 0`), but compression keeps running anyway. So the agent receives `[... compressed to N. Retrieve more: hash=...]` markers with no `headroom_retrieve` tool to redeem them. For #1077, SmartCrusher now skips `headroom_retrieve` results. Before crushing a tool message (OpenAI `role=tool`) or tool-result block (Anthropic `type=tool_result`), it checks whether that tool id maps to the CCR tool, and if so leaves it alone. Retrieved content stays readable. For #1006, compression and injection are no longer decided in isolation. The injection decision is extracted into `should_inject_ccr_tool`, which the Anthropic handler calls: when injection was deferred because of a frozen prefix but compression just emitted new markers, it injects the tool anyway, so a marker is never handed to an agent that can't act on it. The existing session-sticky dedup means sessions that already have the tool don't get it re-injected and don't lose their cache. Closes #1077 Closes #1006 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/transforms/smart_crusher.py`: exempt `headroom_retrieve` results from compression on both the OpenAI `role=tool` and Anthropic `type=tool_result` paths. - `headroom/proxy/helpers.py`: add `should_inject_ccr_tool`, the deferral-plus-override decision the handler used to inline, so the #1006 behaviour is testable at the decision point. - `headroom/proxy/handlers/anthropic.py`: call `should_inject_ccr_tool` to couple injection with compression; rename the misleading `frozen_prefix=` log key to `frozen_message_count=`. - `tests/test_transforms/test_smart_crusher_ccr_retrieve_exemption.py` and `tests/test_proxy/test_ccr_frozen_prefix_coupling.py`: new tests; the frozen-prefix test now drives `should_inject_ccr_tool` so it would fail if the override were removed. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --extra dev python -m pytest tests/test_proxy/test_ccr_frozen_prefix_coupling.py tests/test_transforms/test_smart_crusher_ccr_retrieve_exemption.py -q 5 passed, 1 skipped ruff: All checks passed! mypy: Success: no issues found ``` The SmartCrusher test skips locally because the Rust extension `.so` is built for a different OS, the same skip the existing SmartCrusher tests take locally. It runs in CI where the extension is built. ## Real Behavior Proof - Environment: macOS, Python 3.13, this branch. - Exact command / steps: `uv run --extra dev python -m pytest tests/test_proxy/test_ccr_frozen_prefix_coupling.py tests/test_transforms/test_smart_crusher_ccr_retrieve_exemption.py -q`. The frozen-prefix test calls `should_inject_ccr_tool` (the function the Anthropic handler now uses) with a frozen prefix and freshly emitted markers, then drives `apply_session_sticky_ccr_tool` end to end and asserts `headroom_retrieve` lands in the outbound tools. The exemption test runs a `headroom_retrieve` tool result through SmartCrusher on both the OpenAI and Anthropic shapes. - Observed result: 5 passed, 1 skipped. The retrieve tool is injected even under a frozen prefix once markers exist, and is not injected when no markers were emitted. Removing the handler override flips `should_inject_ccr_tool` and fails the test. - Not tested: a full live proxy session. The behaviours are covered at the decision, transform, and handler-call level by the new tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes This one touches compression gating, so it's worth a careful read on the injection coupling, that's the part where a wrong call would re-introduce data loss. 1. Tool results with no id mapping still compress, marked with `# ponytail:` comments. Only ids we can positively identify as the CCR tool are exempted. 2. The injection coupling keys off `injector.has_compressed_content`, so the tool only shows up when there's actually something to retrieve. --------- Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Rod Boev
|
3be2526b76
|
fix(proxy): add an Anthropic buffered read-timeout override (#1331)
## Description Buffered Anthropic `/v1/messages` requests still use Headroom's generic 300-second read timeout, which can produce proxy-generated `502 ReadTimeout` errors on long turns. This adds a dedicated buffered Anthropic timeout, keeps it applied across CCR and memory continuations plus batch paths, and makes the direct server entrypoint enforce the same positive-integer contract as the Click CLI. Closes #1261. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `anthropic_buffered_request_timeout_seconds` for buffered Anthropic reads. - Routed `/v1/messages`, CCR continuation, memory continuation, batch create, batch passthrough, and batch results through that timeout. - Enforced the same positive-integer validation for `HEADROOM_ANTHROPIC_BUFFERED_REQUEST_TIMEOUT_SECONDS` and `--anthropic-buffered-request-timeout-seconds` in both startup paths. - Added focused regressions and updated `CHANGELOG.md`. ## Testing - [x] `uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring` - [x] `uv run ruff check .` - [x] `uv run ruff format . --check` ### Test Output ```text $ uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring 17 passed in 3.42s $ uv run ruff check . All checks passed! $ uv run ruff format . --check 966 files already formatted ``` ## Real Behavior Proof - Environment: local FastAPI `TestClient` with stubbed retry and HTTP client seams - Exact command / steps: run `uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring`; the tests build `ProxyConfig(request_timeout_seconds=7, connect_timeout_seconds=3, anthropic_buffered_request_timeout_seconds=19)`, drive `/v1/messages`, `/v1/messages/batches`, `/v1/messages/batches/{batch_id}/results`, a CCR continuation, and a memory continuation through `TestClient`, then verify `HEADROOM_ANTHROPIC_BUFFERED_REQUEST_TIMEOUT_SECONDS=0` falls back to `600`, `--anthropic-buffered-request-timeout-seconds 0` is rejected, and default proxy timeouts stay `read=300` and `write=300` - Observed result: buffered Anthropic paths use `httpx.Timeout(connect=3, read=19, write=7, pool=3)`, continuation requests stay on that same budget, invalid zero-valued startup config is rejected or ignored back to the default, and unrelated proxy timeout defaults stay unchanged - Not tested: live upstream Anthropic latency beyond the focused stubbed-timeout regression ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have added tests that prove the fix - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable |
||
|
Rod Boev
|
2cae13dd79
|
fix(ccr): skip Anthropic marker emission when tool injection is deferred (#1273)
## Description Anthropic request-side CCR can still compress a turn into retrieval markers after the frozen-prefix cache guard suppresses `headroom_retrieve` registration. That leaves the model with marker-only context it cannot redeem, so the proxy silently drops recoverable data on exactly the turns where cache preservation deferred tool injection. This change couples the Anthropic request-side CCR path to tool availability so a turn never emits retrieve-only markers without the retrieval tool, even when token mode or cache-mode prefix replay could otherwise reuse already-compressed marker text. Closes #1006 After a collaborator merged current `main` into this branch, CI also picked up unrelated offline-memory failures from the merged base. Those follow-up changes are test-only: they keep the offline Hugging Face cache lanes skipping cleanly instead of failing in memory tests that are outside the CCR runtime path. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Couple Anthropic request-side CCR compression to the same frozen-prefix guard that already defers `headroom_retrieve` registration. - Keep the existing cache-preservation behavior: frozen-prefix turns stop emitting CCR retrieval markers instead of forcing tool injection into the cached prefix. - Make the skip decision use the effective frozen prefix after token-mode reclamping, so turns that genuinely reclamp to zero still keep normal reversible CCR behavior. - Bypass cached marker reuse in both token mode and cache-mode prefix replay when tool injection is deferred. - Add focused regressions for the Anthropic request-path seams under this bug: - frozen-prefix turns do not emit marker-only payloads - unfrozen turns still keep normal reversible CCR behavior - token-mode reclamp back to zero still compresses normally - existing `headroom_retrieve` tools keep reversible CCR on frozen turns - cache-mode delta reuse and exact-prefix replay both forward original content when retrieval is unavailable - Add a `CHANGELOG.md` entry because the proxy's user-visible Anthropic CCR behavior changes. - Add a shared test skip helper for offline Hugging Face cache misses and apply it to the merged `main` memory tests that were failing only in the offline CI shards after the branch picked up current `main`. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy/test_anthropic_ccr_deferred_injection.py`) - [x] Linting passes (`uv run ruff check . && uv run ruff format . --check`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy/test_anthropic_ccr_deferred_injection.py 14 passed, 1 warning in 34.19s uv run pytest tests/test_memory/test_skip_helpers.py tests/test_memory/test_embedder_mps_serialization.py::test_cpu_uses_shared_executor tests/test_memory/test_hierarchical.py::TestLocalEmbedder::test_embed_single tests/test_memory_bridge.py::TestMemoryBridgeImport::test_import_claude_code_memory tests/test_memory_handler_concurrent_init.py::test_real_localbackend_initializes_via_public_entrypoint tests/test_memory_system.py::TestLocalBackend::test_save_memory_basic -q 4 passed, 5 skipped, 11 warnings in 7.65s uv run ruff check . All checks passed! uv run ruff format . --check 968 files already formatted ``` ## Real Behavior Proof - Environment: local FastAPI `TestClient` for the Anthropic request path, plus Windows Python 3.12 offline-memory repros with `TRANSFORMERS_OFFLINE=1` - Exact command / steps: Run the focused CCR regression command and the offline-memory repro subset below on the merged branch state. - `uv run pytest tests/test_proxy/test_anthropic_ccr_deferred_injection.py` - `uv run pytest tests/test_memory/test_skip_helpers.py tests/test_memory/test_embedder_mps_serialization.py::test_cpu_uses_shared_executor tests/test_memory/test_hierarchical.py::TestLocalEmbedder::test_embed_single tests/test_memory_bridge.py::TestMemoryBridgeImport::test_import_claude_code_memory tests/test_memory_handler_concurrent_init.py::test_real_localbackend_initializes_via_public_entrypoint tests/test_memory_system.py::TestLocalBackend::test_save_memory_basic -q` - Scenario coverage from the CCR pytest command: - frozen prefix with deferred tool injection and cached marker text available in token mode - unfrozen turn with normal CCR marker emission - token mode where the tracked frozen prefix reclamps back to zero - frozen prefix where the client already supplied `headroom_retrieve` - cache-mode append-only delta reuse with a previously forwarded compressed prefix - cache-mode exact-prefix replay where the previous forwarded prefix already contained a marker - Scenario coverage from the offline-memory repro command: - direct local embedder startup on CPU with no cached HF model - hierarchical memory embedder startup with offline model cache missing - bridge import through `LocalBackend` - `MemoryHandler` public init warmup path - `LocalBackend` save path under the offline lane - Observed result: The CCR regression keeps marker-free forwarding on frozen turns without tool availability, and the merged-`main` offline-memory lanes now skip cleanly instead of failing unrelated CI shards. - frozen-prefix Anthropic turns without tool availability forward the original long transcript across both token-mode and cache-mode reuse paths, while unfrozen turns, reclamped token-mode turns, and frozen turns that already advertise `headroom_retrieve` keep the reversible CCR marker path - the merged-`main` offline-memory regressions now skip cleanly when the Hugging Face cache is unavailable instead of failing unrelated CI shards - Not tested: live Zed session cache-hit behavior, provider latency under real Anthropic upstreams, and online Hugging Face download lanes ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Runtime scope is still intentionally narrow to Anthropic request-side CCR. OpenAI, Gemini, streaming, and response-side CCR behavior are unchanged. - The only non-CCR diff is the test-only offline-memory follow-up required after current `main` was merged into the branch. - The focused proxy pytest run still emits the Windows-local `StarletteDeprecationWarning` from `fastapi.testclient`'s `httpx` bridge. The offline-memory repro command also emits existing datetime deprecation warnings and a pytest teardown warning around skipped offline lanes; none of those warnings were introduced by the CCR runtime change. |
||
|
Zhenjia ZHOU
|
6c68ff4e9f
|
perf(compression): take large cold-start contexts off the synchronous kompress path (#1171) (#1298)
## Description On a cold-start large context, kompress (ModernBERT ONNX) runs **synchronously on the request thread** — ~200–300s for ~1M tokens. It blows the 30s compression budget, leaks a non-preemptible worker, and cascades (executor saturation → queue timeouts on healthy requests); on timeout the request is forwarded **uncompressed** after eating 30s. This adds four layered, **default-off, fail-open** mitigations so the request path is never blocked on ML compression. Closes #1171 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Phase 0 — size gate** (`HEADROOM_KOMPRESS_MAX_TOKENS`, default 50000): route oversized text away from ModernBERT (→ LogCompressor / TextCrusher / passthrough) at the single `_try_ml_compressor` boundary. - **Phase 1 — cooperative deadline** (`HEADROOM_COMPRESSION_DEADLINE_MS`, default 20000): any kompress run self-terminates at the next chunk boundary past the budget, keeping the unprocessed tail verbatim. - **Phase 2 — TextCrusher** (`HEADROOM_TEXT_CRUSHER`): a new **native Rust** extractive prose compressor in `crates/headroom-core/src/transforms/text_crusher/`, exposed via PyO3 as `headroom._core.TextCrusher` with a thin Python wrapper. It **reuses the shared `crate::relevance::BM25Scorer`** rather than reimplementing BM25, and ships record/replay parity fixtures (mirroring the SmartCrusher Rust-core + Python-shim pattern). - **Phase 3 — off-path compression** (`HEADROOM_BACKGROUND_COMPRESSION`): forward uncompressed immediately and compress in a per-process background drain; a byte-identical cache hit on a later turn means the request never blocks on ML. - Benchmark (`benchmarks/text_crusher_quality_eval.py`), CHANGELOG entry, and docstrings documenting the fail-open limits. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`, new modules) - [x] New tests added for new functionality - [ ] Manual testing performed (Phase 0/1 gate-fire + deadline observed on real traffic in earlier iterations; Phase 3 off-path is unit- + byte-identity-tested, not yet live-validated) ### Test Output ```text $ pytest tests/test_transforms/ tests/test_cache/ \ tests/test_proxy/test_background_compression.py tests/test_proxy/test_phase3_byte_identity.py -q 501 passed, 37 skipped in 40.33s $ cargo test -p headroom-core --lib text_crusher test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 834 filtered out $ ruff check <changed files> All checks passed! $ mypy headroom/proxy/background_compression.py headroom/transforms/text_crusher.py Success: no issues found in 2 source files ``` New coverage: size-gate incl. the strategy-dispatch funnel (KOMPRESS + TEXT); partial-run deadline (chunk-0 compressed + chunk-1 verbatim tail); BackgroundCompressor (dedup / queue-full / fail-open); Phase 3 byte-identity round-trip; TextCrusher unit + parity. ## Real Behavior Proof - Environment: macOS, local dev — `uv` venv, Rust `_core` built via `uv pip install -e .`. - Exact command / steps: the `pytest` / `cargo test` / `ruff` / `mypy` commands shown under Test Output; quality eval `python benchmarks/text_crusher_quality_eval.py /tmp/squad_dev.json`. - Observed result: 501 Python + 3 Rust tests pass; ruff + mypy clean on changed/new modules. Quality eval: TextCrusher keeps ~94% of buried SQuAD answers at 30% size vs ~36% truncate/random; self-contained speed run ~333k words in ~76ms (one O(n) pass) — sub-second where ModernBERT takes minutes (fast-vs-slow contrast, not a same-input run). - Not tested: Phase 3 off-path on live traffic; multi-worker (per-process by design — see Additional Notes). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - **All four features are off by default and fail-open** — with the env flags unset the paths are no-ops for realistic inputs; on any error the request is forwarded (compressed if possible, else verbatim), never dropped. A full background queue / duplicate key surfaces as `deferred:dropped`. - **Known limits (documented in `background_compression.py`):** Phase 3 is per-process, in-memory, and token-mode-only — these are **lost-savings, never lost-correctness**, and consistent with the project's existing per-process compression cache + sticky-session multi-worker model. The startup multi-worker warning now names off-path background compression. - Phase 2 reuses the existing BM25 scorer; reuse did not improve answer-retention over a Python prototype (query-awareness dominates) — its value is the Rust speed + repo-conventional Rust-core/Python-shim shape. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Tejas Chopra
|
bd55a426bc
|
fix(proxy): scope CORS to loopback + gate operator/content endpoints (#1226)
## Description Locks down the proxy's browser- and network-facing attack surface, which matters most under a `--host 0.0.0.0` bind (the Docker default). The wildcard CORS policy (`allow_origins=["*"]` + `allow_credentials=True`) let any web page the user had open read the proxy's content endpoints — `/v1/retrieve` returns raw, uncompressed tool outputs (source, secrets) — via a cross-origin fetch to `127.0.0.1` (CWE-346). Several operator endpoints additionally leaked sensitive data or allowed unauthenticated state mutation to any network-reachable client. This PR scopes CORS to loopback origins and extends the project's existing `require_loopback` trust boundary (already used for `/admin/*` and `/debug/*`) to the remaining exposed endpoints. Closes #863. Supersedes #864 and #758 — see "Additional Notes". ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **CORS**: replaced `allow_origins=["*"]` + `allow_credentials=True` with a port-agnostic loopback origin regex (`https?://(localhost|127\.0\.0\.1|\[::1\])(:\d+)?`), `allow_credentials=False`, and methods/headers narrowed to `GET/POST` + `Content-Type/Authorization`. `HEADROOM_CORS_ORIGINS` (comma-separated) pins an explicit allowlist for Docker/remote dashboards; `*` opts back into the old wildcard. - **`/transformations/feed`** and **`/cache/clear`** gated behind `require_loopback` → 404 for non-loopback callers. The feed returns full prompt/completion bodies when `log_full_messages` is on; `/cache/clear` is unauthenticated state mutation (cache-eviction DoS / cost amplification). - **`/health`**: the `config` block (upstream API URLs, savings profile) is now served only to loopback callers; network callers get the `/readyz`-shape body (status/checks). `/livez` and `/readyz` remain unauthenticated probes for orchestration. - **`/stats`**: `recent_requests` / `request_logs` (per-request ids, providers, models, errors) and `config` are served only to loopback callers; aggregate counters stay public for remote monitoring. - Added `_request_is_loopback()` helper mirroring `require_loopback`'s two-gate check (loopback peer IP + loopback `Host` header, the DNS-rebinding defence) but degrading the payload instead of returning 404, so monitors keep the non-sensitive fields. - Tests: new `tests/test_proxy_cors.py` and `tests/test_proxy_loopback_gating.py`; updated 4 existing tests that assert the now-loopback-only data to use loopback clients. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/server.py tests/test_proxy_cors.py \ tests/test_proxy_loopback_gating.py tests/test_proxy_healthchecks.py \ tests/test_proxy_stats_recent_requests.py tests/test_proxy/test_transformations_feed.py All checks passed! $ mypy headroom --ignore-missing-imports Success: no issues found in 380 source files $ pytest tests/test_proxy_cors.py tests/test_proxy_loopback_gating.py \ tests/test_proxy/test_transformations_feed.py tests/test_proxy_healthchecks.py \ tests/test_proxy_stats_recent_requests.py tests/test_proxy_dashboard_stats_cache.py \ tests/test_proxy_compression_executor.py tests/test_header_isolation.py -q ======================== 82 passed, 1 skipped in 17.75s ======================== ``` ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), Python 3.12 venv; FastAPI `TestClient` driving the real `create_app()` ASGI app - Exact command / steps: issued requests as a non-loopback caller (`client.host=testclient`) vs a loopback caller (`base_url=http://127.0.0.1`, `client=("127.0.0.1", 9999)`), plus CORS preflights with varying `Origin` headers - Observed result: CORS — `http://evil.com` → no `access-control-allow-origin`; `http://localhost:8787` and `http://localhost:9000` → echoed (loopback allowed on any port); `access-control-allow-credentials` → absent. `/cache/clear` and `/transformations/feed` → 404 (network) / 200 (loopback). `/health` `config` block present for loopback only. `/stats` `recent_requests` present for loopback only, while the aggregate `tokens` block stays present for network callers. - Not tested: a live `headroom proxy` process bound on `0.0.0.0` reached from a second host (simulated via ASGI peer/Host instead); end-to-end browser DNS-rebinding (covered by the `Host`-header gate and its unit test) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — proxy/middleware change; behavior is captured under "Real Behavior Proof". ## Additional Notes **Supersedes two stale PRs that target the same issue but have drifted from `main`:** - **#864** (`fix(proxy): scope CORS to localhost`, @gabiudrescu) — correct instinct and the source of the tighter `GET/POST` + `Content-Type/Authorization` scoping kept here, but it derived the allowlist from the `HEADROOM_PORT` env var (wrong when `--port` is passed as a CLI flag), carried ~40 lines of unrelated punctuation churn, and is ~125 commits behind `main`. The port-agnostic regex used here resolves the reviewer's port concern. - **#758** (`security: adversarial review`, @neogenix) — bundled these same application-layer fixes with a large CI/CD + Docker supply-chain pass. It is a ~160-commit-behind draft whose `server.py` no longer merges cleanly (`main` independently adopted the same `require_loopback` pattern). The application-layer fixes are rebased onto current `main` here; the CI/Docker/supply-chain hardening from #758 is still valuable and would be welcome as a separate, rebased PR. Thanks to @gabiudrescu and @neogenix for the original analysis (#863). **Deliberate scope / follow-ups (not in this PR):** - `/stats` aggregate counters and the basic `/health` body remain readable on a `0.0.0.0` bind by design, so remote monitoring keeps working. Full lock-down is a one-line `Depends(require_loopback)` each if preferred. - The `/v1/retrieve*` family stays network-reachable; it can't be loopback-gated without breaking legitimate remote/containerized agents and needs auth instead — tracked separately. - `ruff check .` is scoped to changed paths above because the dashboard HTML template trips ruff's `invalid-syntax` (a known repo false-positive); `mypy` is run over the full `headroom` package. |
||
|
Zhenjia ZHOU
|
e8fc8a0d18
|
feat(proxy): cc-switch reconciler — keep Headroom in the request path alongside cc-switch (#1030)
## Description [cc-switch](https://github.com/farion1231/cc-switch) is a desktop provider manager for Claude Code and other coding agents; when a Claude Code provider is selected, it writes that provider's endpoint and token into `~/.claude/settings.json`. This PR adds an opt-in reconciler so Headroom can stay in Claude Code's request path when cc-switch rewrites that file during provider switches. The reconciler captures third-party Anthropic-compatible upstream URLs, points Claude back at the local Headroom proxy, and leaves official/empty OAuth settings direct unless explicitly opted in. This update also hardens the watcher so rapid settings rewrites that share the same float-second mtime are still detected. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added an opt-in `HEADROOM_CC_SWITCH_RECONCILE=1` watcher for cc-switch direct-injection mode. - Added loopback-only `GET/PUT /admin/upstream` runtime upstream inspection and override endpoints. - Preserved token/model settings while rewriting only `env.ANTHROPIC_BASE_URL` back to the local Headroom proxy. - Switched reconciler change detection from float-second `st_mtime` to nanosecond `st_mtime_ns` so rapid provider switches are not missed. - Added pytest coverage for capture/rewrite behavior, official-provider defaults, route-official opt-in, loop safety, enabled flags, and the same-float-mtime provider-switch case. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text python -m pytest tests/test_proxy/test_cc_switch_reconciler.py 12 passed in 0.17s python -m ruff check . All checks passed! python -m mypy headroom Success: no issues found in 359 source files python -m pytest 7 failed, 5996 passed, 492 skipped, 5814 warnings in 396.41s ``` ## Real Behavior Proof - Environment: macOS, branch `feat/cc-switch-reconciler`, Python 3.13.3. - Exact command / steps: Ran the focused reconciler pytest file, full repository ruff check, full `mypy headroom`, and full pytest from the local PR branch. - Observed result: All 12 reconciler tests passed, including the rapid provider-switch case where two writes share the same float mtime but differ by nanoseconds. Full `ruff check .` and `mypy headroom` passed. Full pytest completed with 7 failures outside the cc-switch reconciler test file. - Not tested: Live cc-switch plus Claude Code end-to-end switch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes Documentation and CHANGELOG updates are not included in this PR. The reconciler remains opt-in and off by default. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Eyal Mizrachi
|
addebdb29c
|
feat(proxy): make COMPRESSION_TIMEOUT_SECONDS configurable via env (#946) (#991)
## Description The compression-pipeline timeout was hard-coded at 30s, so slow CPUs and long Claude Code conversations had no recourse. #946 asks to wire `HEADROOM_COMPRESSION_TIMEOUT_SECONDS` through. Refs #946. ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - Read `HEADROOM_COMPRESSION_TIMEOUT_SECONDS` from the environment (float), falling back to 30 on an unparseable value. ## Testing - [x] Unit tests pass (`pytest`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_proxy/test_compression_timeout_config.py -q 4 passed in 0.09s ``` ## Real Behavior Proof - Environment: Linux, Python 3.13, editable build of this branch - Exact command / steps: `HEADROOM_COMPRESSION_TIMEOUT_SECONDS=88 python -c "import headroom.proxy.helpers as h; print(h.COMPRESSION_TIMEOUT_SECONDS)"` - Observed result: prints `88.0` (default `30.0`; an unparseable value falls back to `30.0`) - Not tested: a live compression actually exceeding the configured timeout under real load ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
rongabbay
|
7edb27ab24
|
feat(proxy): compress AWS Bedrock InvokeModel requests via configurable upstream (#720)
## Description
Clients that speak **Bedrock to a local gateway** can't get proxy-level
compression. Claude Code launched with `CLAUDE_CODE_USE_BEDROCK=1` (and
any AWS SDK pointed at a custom endpoint) POSTs
`/model/{id}/invoke[-with-response-stream]` to
`AWS_ENDPOINT_URL_BEDROCK_RUNTIME`, never `/v1/messages`. Those requests
fell through the catch-all and were forwarded **verbatim — no
compression**.
`--backend bedrock` is the opposite direction: it accepts Anthropic
input and re-signs to AWS. It can't accept Bedrock-format input or
forward to a custom upstream. So the "client speaks Bedrock → local
re-signing gateway → AWS" topology (internal gateways, LiteLLM,
LocalStack; see #510) got nothing.
This adds a Bedrock InvokeModel passthrough that compresses the request
body with the **same** `anthropic_pipeline` used for `/v1/messages` —
the Bedrock InvokeModel body for Anthropic models *is* the Anthropic
Messages shape (`{anthropic_version, system, messages, max_tokens, …}`,
model in the URL), so there's no translation and no new compression
logic. The routes register **only** when `--bedrock-api-url` is set, so
default behavior is completely unchanged.
**Limitation (important):** rewriting the body invalidates the caller's
**SigV4** signature (it covers a hash of the body). Point
`--bedrock-api-url` at a gateway that re-signs or doesn't verify the
inbound signature (an internal gateway, LiteLLM, LocalStack, a corporate
Bedrock proxy) — **never raw AWS**, which would 403. For direct-to-AWS
compression, use `--backend bedrock` (which re-signs). The two are
complementary. This is documented in the flag help, the handler
docstring, the proxy docs, and the CHANGELOG.
Closes #734. Refs #510 (the Bedrock slice of the provider-agnostic
umbrella).
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- New `--bedrock-api-url` flag (env: `BEDROCK_TARGET_API_URL`). When
set, registers `POST /model/{id}/invoke` and `POST
/model/{id}/invoke-with-response-stream`.
- `BedrockHandlerMixin` compresses the request body via the existing
`anthropic_pipeline`, then forwards to the configured upstream,
preserving path/query.
- Responses forwarded byte-faithfully (non-streaming JSON and the
streaming AWS event-stream alike — neither is parsed or mutated, since
all compression is request-side).
- `{model_id:path}` captures inference-profile ids with
dots/colons/slashes (e.g.
`us.anthropic.claude-sonnet-4-5-20250929-v1:0`).
- Fail-open: a malformed body or compression error forwards verbatim
rather than erroring.
- Routes register only when the flag is set — default behavior
unchanged.
- Files: `headroom/proxy/handlers/bedrock.py` (new —
`BedrockHandlerMixin`); `headroom/providers/proxy_routes.py` (gated
route registration); `headroom/cli/proxy.py`,
`headroom/proxy/server.py`, `headroom/proxy/models.py` (flag + config
wiring); `docs/content/docs/proxy.mdx`, `CHANGELOG.md` (docs).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_proxy/test_bedrock_passthrough.py -q
.............. [100%]
14 passed in 12.46s
```
`tests/test_proxy/test_bedrock_passthrough.py` (14 tests) covers: route
gating (absent unless configured), body compression, non-message fields
preserved, inference-profile id capture + re-encoding, byte-faithful
streaming, fail-open on malformed body and on pipeline exceptions,
bypass when `optimize=False` and via the `x-headroom-bypass` header,
upstream connect failure surfacing as a 502, the content-length
regression, outcome recorded with `provider="bedrock"`, and
`BEDROCK_TARGET_API_URL` env wiring. `ruff check`/`format` clean.
## Real Behavior Proof
- Environment: macOS, Python 3.12; forked proxy on `:8788` with
`--bedrock-api-url` pointed at a local re-signing Bedrock gateway;
provider Anthropic Claude on Bedrock.
- Exact command / steps: `headroom proxy --port 8788 --bedrock-api-url
http://127.0.0.1:<gateway>`, then `curl -X POST
http://127.0.0.1:8788/model/claude-haiku-4-5/invoke --data
@bedrock_invoke.json` (a ~52k-token conversation with a large assistant
turn).
- Observed result: valid Claude response returned and the gateway
received the compressed body — proxy `/stats` reports `52,095 → 3,979
tokens` (92.4%, 48,116 removed), and the gateway's reported
`input_tokens: 3709` confirms the compressed body reached the model.
- Not tested: raw direct-to-AWS (out of scope by design — SigV4; use
`--backend bedrock`); non-Anthropic Bedrock model bodies (e.g.
Titan/Llama) — only the Anthropic Messages-shaped invoke body is
handled.
<details><summary>Proxy <code>/stats</code> output + content-length bug
note</summary>
```json
"compression": {
"requests_compressed": 1,
"avg_compression_pct": 92.4,
"best_detail": "52,095 → 3,979 tokens",
"total_tokens_removed": 48116
}
```
The first iteration of this proof surfaced a real bug — a shrunk body
still carried the inbound `Content-Length`, so httpx raised `Too little
data for declared Content-Length`. Fixed by dropping
`content-length`/`content-encoding` on the rewritten path so httpx
recomputes them; covered by a regression test.
</details>
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
`mypy headroom` is left unchecked above — type checking runs in the CI
matrix rather than locally on my side; the new code carries type hints
on all public functions. Design spec / feature request: #734.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
gglucass
|
2269e40bde
|
feat(proxy): log compressed messages alongside original request (#261)
## Description Expose the post-compression message list that was actually sent upstream as a new `compressed_messages` field on `RequestLog`, paired with the existing (now consistently pre-compression) `request_messages`. Consumers of `/transformations/feed` — dashboards and any downstream observability — can now diff the two sides of a compression to see exactly what the pipeline stripped, replaced, or kept. Turns an abstract "saved N tokens" into a legible before/after. Gated by the same `log_full_messages` flag as `request_messages` so the two sides stay in sync; it's pointless to store one without the other. Also fixes a latent correctness bug: today's `request_messages` field is inconsistent across the four `RequestLog` construction sites — sometimes it's the pre-compression snapshot, sometimes it's the mutated `body["messages"]` (which is the compressed list, because the proxy mutates `body` in place before the log call). After this change, `request_messages` always means pre-compression and `compressed_messages` always means what went upstream. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) Note on "breaking": strictly speaking this is a semantic correction of an inconsistently-populated field, not a schema break. The field name `request_messages` is unchanged and the JSON shape is unchanged; what changes is that the field now consistently holds the pre-compression list. Consumers that treated it as "whatever messages we have" continue to work. Consumers that depended on the accidental post-compression value (if any existed) would shift to `compressed_messages`. ## Changes Made - **`headroom/proxy/models.py`**: `RequestLog` gains `compressed_messages: list[dict] | None = None`. Doc comment explains it's paired with `request_messages` and gated by the same `log_full_messages` flag. - **`headroom/proxy/handlers/anthropic.py`** (2 sites — Bedrock non-streaming and main non-streaming): `request_messages` now consistently sources from `original_messages` (the pre-compression snapshot at line 724), `compressed_messages` sources from `body["messages"]` (the compressed list after in-place mutation at line 1189). Both gated symmetrically. - **`headroom/proxy/handlers/streaming.py`** (2 sites — main streaming in `_finalize_stream_response`, Bedrock streaming in `_stream_response_bedrock`): same treatment. `_stream_response_bedrock` gains a new `original_messages: list[dict] | None = None` parameter so it has access to the pre-compression snapshot; the sole caller in `anthropic.py` now threads it through. - **`headroom/proxy/server.py`**: `/transformations/feed` adds `compressed_messages` to the JSON payload alongside the existing `request_messages` / `response_content`. *Split into a separate preceding commit is a one-time EOL normalization to LF — the file blob in history carries CRLF but `.gitattributes` declares `*.py text eol=lf`, so any contributor editing `server.py` triggers the same whole-file renormalization. Separating the two commits keeps this feature commit's diff at a single line.* - **`headroom/proxy/request_logger.py`**: `compressed_messages` is stripped from the JSONL file log and from `get_recent()` alongside the existing `request_messages` / `response_content` stripping. `get_memory_stats()` also counts it toward the deque's byte budget. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .` and `ruff format --check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (via the Headroom Desktop client that consumes `/transformations/feed` — confirmed both fields arrive and render) Test coverage added/extended: - `tests/test_proxy/test_request_logger.py` (new file): round-trip unit tests for `RequestLogger`. Confirms `get_recent` strips both sides (pre + post), `get_recent_with_messages` exposes both, and the JSONL file log drops both when `log_full_messages=False`. - `tests/test_proxy/test_transformations_feed.py`: extended to assert `compressed_messages` appears in the endpoint payload alongside `request_messages` / `response_content`. - `tests/test_proxy_streaming_request_logger.py`: existing include/omit tests updated to assert both sides populate when the flag is on and both are `None` when it's off. ## Test Output ``` $ uv run ruff check headroom tests All checks passed! $ uv run ruff format --check headroom tests 614 files already formatted $ uv run pytest tests/test_proxy/test_request_logger.py tests/test_proxy_streaming_request_logger.py tests/test_proxy/test_transformations_feed.py -v ... tests/test_proxy/test_request_logger.py::test_get_recent_strips_compressed_messages_alongside_request_and_response PASSED tests/test_proxy/test_request_logger.py::test_get_recent_with_messages_returns_compressed_messages PASSED tests/test_proxy/test_request_logger.py::test_jsonl_file_strips_both_sides_when_log_full_messages_disabled PASSED tests/test_proxy_streaming_request_logger.py::test_finalize_stream_response_logs_original_and_compressed_messages PASSED tests/test_proxy_streaming_request_logger.py::test_finalize_stream_response_omits_messages_when_log_full_messages_disabled PASSED tests/test_proxy/test_transformations_feed.py::test_transformations_feed_returns_messages PASSED ... ============================== 11 passed in 5.36s ============================== ``` ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas (the two-sided gating at each log site, the `_stream_response_bedrock` parameter addition, and the `get_memory_stats` accounting) - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes ### Non-Anthropic backends `handlers/openai.py` and `handlers/gemini.py` do not currently emit `RequestLog` entries at all — only Anthropic and the shared streaming paths do. This PR therefore only populates `compressed_messages` on Anthropic traffic (which is what `/transformations/feed` shows today). Wiring OpenAI and Gemini into `RequestLogger` end-to-end is a separate, larger gap worth its own PR. ### `server.py` EOL normalization The feature change in `server.py` is a single line. To keep the diff readable, the preceding commit is a whitespace-only `chore(proxy): normalize server.py to LF per .gitattributes` — the file blob was stored with CRLF terminators but `.gitattributes` declares `*.py text eol=lf`. Any contributor touching `server.py` triggers this renormalization; isolating it here keeps the feature commit reviewable. Happy to rebase / drop / reshape as preferred. ### Downstream desktop compatibility The Headroom Desktop client I work on now consumes `compressed_messages` and renders the pre/post pair side-by-side on the "Recent large compression" card. The desktop was updated to handle both shapes: proxies without the field render the legacy single "Request" block; proxies with the field render "Request (original, N tokens)" + "Request (compressed, M tokens)" where N/M come from `input_tokens_original` / `input_tokens_optimized`. No changes needed downstream if this PR lands. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
gglucass
|
8f374263d3
|
feat(transforms): attribute read_lifecycle + smart_crush tags (#249)
## What Enrich the `transforms_applied` tags emitted by `ReadLifecycleManager` and `SmartCrusher` so each tag carries the specific target it acted on, instead of being an opaque counter: - `read_lifecycle:<state>` -> `read_lifecycle:<state>:<file_path>` - `smart_crush:<n>` -> `smart_crush:<n>:<tool1,tool2,...>` (tool names resolved from the assistant's `tool_calls` / `tool_use` metadata; falls back to `smart_crush:<n>` when no name resolves) Downstream UIs can then show *what* a compression acted on (which file was a stale read, which tools had their output crushed), not just that it happened. ## Note on the rebase The original revision targeted `ToolCrusher` / `tool_crush:<n>`. That transform has since been retired and replaced by the Rust-backed `SmartCrusher` (which emits `smart_crush:<n>`), so the tool-name attribution moved to `smart_crusher.py`. The `read_lifecycle` half is unchanged. ## Response-header compatibility `x-headroom-transforms` is built as `",".join(transforms_applied)`. A tag containing a comma (tool-name lists; file paths) would make that header ambiguous to split back into tags. To keep the header backward compatible, `header_safe_transforms` (`headroom/proxy/cost.py`) collapses the enriched tags back to their legacy counter shape **for the header only** -- the full enriched detail still flows through the structured `transforms_applied` list (dashboards, request logs, activity feed). Applied at all three header sites (openai / anthropic / gemini handlers). Paths containing `:` survive in `transforms_applied` because consumers bound their split to 3 parts. ## Tests - `tests/test_transforms/test_read_lifecycle.py` -- OpenAI + Anthropic tag shape, colon-in-path preservation - `tests/test_transforms/test_smart_crusher_attribution.py` -- OpenAI + Anthropic tool-name resolution, dedup, no-name fallback, id/name-missing skips - `tests/test_proxy/test_header_safe_transforms.py` -- header normalization keeps the joined header unambiguous (incl. comma-in-path) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ashish
|
8db5efc6f9
|
fix(anthropic): CCR exception must re-raise, not silently swallow (#838)
## Summary
- When `ccr_response_handler.handle_response` throws on the Anthropic
path, the old code logged a `WARNING` and continued — silently returning
the raw `headroom_retrieve` tool-call block to the client (compressed
content never retrieved, client sees an unknown internal tool)
- The OpenAI handler already does `logger.error + raise` (commit
`
|
||
|
ashishpatel26
|
e8ecd08829 | fix(codex): fail open for proxy compression timeout | ||
|
chopratejas
|
20dc1f28f3 |
fix(proxy): Strands MCP bundle + backend path fixes + Codex fail-closed protection
Three logically-related sets of proxy changes ship in this branch:
1. Strands integration on the Bedrock path (HeadroomBundle + 4 OpenAI
handler fixes + LiteLLM cache stats + dep pin)
2. /stats MCP aggregation (cross-process events log → proxy summary)
3. Codex compression-failure fail-closed (WS + HTTP /v1/responses)
== 1. Strands integration on the Bedrock path ==
* HeadroomBundle (headroom/integrations/strands/bundle.py): single-helper
MCP wiring for a Strands Agent — Headroom MCP server (headroom_compress
/ headroom_retrieve / headroom_stats) plus optional Serena MCP and
optional in-process compression hook. Constructor builds unstarted
MCPClient instances per server; Strands' Agent owns the subprocess
lifecycle. Default config: MCP enabled, Serena enabled, hook OFF
(proxy is the single source of truth for compression). User-side
integration is two lines in any Strands app.
* headroom/proxy/handlers/openai.py — backend path now:
- calls PrefixCacheTracker.update_from_response (was direct-OpenAI only)
- intercepts CCR headroom_retrieve tool_calls server-side, mirroring
the Anthropic handler pattern; NO silent fallback, re-raises on
CCR errors (per feedback_no_silent_fallbacks)
- works for both non-streaming and streaming paths
* headroom/proxy/handlers/streaming.py: _stream_openai_via_backend now
accepts prefix_tracker + optimized_messages, parses cache stats from
the SSE final-usage frame (cache_creation_input_tokens added to the
state machine), records CCR retrieve feedback via a new
_record_ccr_feedback_from_openai_sse helper. Streaming CCR intercept
is intentionally out of scope (mirrors Anthropic streaming behaviour).
* headroom/backends/litellm.py: send_openai_message response usage block
now carries cache_read_input_tokens / cache_creation_input_tokens
(Anthropic/Bedrock dialect) and prompt_tokens_details.cached_tokens
(OpenAI dialect). Backwards-compatible — cold-start callers see the
same 3-key shape; cache keys appear only when the underlying provider
returns them. Pinned by test_no_cache_fields_means_no_cache_keys.
* headroom/proxy/auth_mode.py: ("strands-agents/", "strands") added to
CLIENT_UA_MAP. Production callers should also set X-Client: strands
since the default openai-python UA carries no Strands signal.
* pyproject.toml: huggingface-hub>=1.5.0,<2.0 pinned in [ml] so a sibling
install (e.g. strands-agents) can't drag the version below the floor
transformers 5.x requires (otherwise Kompress silently goes
"unavailable").
== 2. /stats MCP aggregation ==
* headroom/proxy/cost.py: _aggregate_mcp_events() reads the cross-process
shared events file the Headroom MCP server already writes to and
surfaces summary.mcp with three new keys:
- compressions (count of headroom_compress invocations)
- tokens_removed (sum of input - output across those)
- retrievals (count of headroom_retrieve — the load-bearing
over-compression alarm; if it grows linearly
with turn count, lossy compressors are
dropping info the model actually needs)
Defensive on every axis — missing MCP SDK, missing file, malformed
events, read errors — never blocks /stats.
* examples/strands_bundle_demo.py: stats panel prints the new fields so
the demo shows the full proxy-HTTP + MCP-tool story in one view.
== 3. Codex compression-failure fail-closed protection ==
Reported by Camille (2026-05-21): Codex threads were locking with
"ran out of room in the model's context window" after Headroom's
compression timed out on an oversized response.create frame and
forwarded the original ~1.7 MB frame to the upstream, which then
rejected it. Codex's auto-compact heuristic gates on the upstream-
reported total_usage_tokens (which Headroom had been shrinking on
earlier turns), so its compaction never fired and the thread locked.
Validated against open Codex issues (CLI + Desktop share codex-rs/core):
* #16068 — confirms compaction gates on total_usage_tokens,
estimated_token_count is computed but only logged
* #19806 — confirms image token estimator unbounded, contributes to
the same ContextManager.get_total_token_usage → auto-compaction chain
* headroom/proxy/helpers.py: decide_compression_failure_action() with a
unit-tested decision matrix:
- asyncio.TimeoutError → refuse, always
- non-timeout failure + frame > 256 KiB (configurable) → refuse
- non-timeout failure + small frame → forward (legacy)
Operator escape hatches:
- HEADROOM_WS_FAIL_OPEN_ON_COMPRESSION_FAILURE=1 restores legacy
- HEADROOM_WS_COMPRESSION_FAIL_THRESHOLD_BYTES tunes the threshold
* headroom/proxy/handlers/openai.py (WS /v1/responses): consults the
helper after compression failure. On refuse: close client websocket
code 1009 with "headroom: compression <reason> — please compact
context and retry" reason; set termination_cause for the outer
lifecycle finally; return.
* headroom/proxy/handlers/openai.py (HTTP /v1/responses): same helper.
On refuse: raise HTTPException(413) with a structured error body so
FastAPI's HTTPException handler emits a clean 413. The existing
`except HTTPException: raise` guard in this handler already ensures
the 413 propagates without being swallowed by the 502 catch-all.
Anthropic /v1/messages NOT changed in this branch: no equivalent bug
report on Anthropic-protocol clients, Claude Code (Anthropic-owned)
handles context overflow via its own cache_control/ephemeral
primitives, and Cursor/Aider don't maintain the local-Y estimate the
Codex bug requires. Deferred until a real report lands; the patch is
a one-liner reusing the same helper.
== Tests + verification ==
* tests/test_backends/test_litellm_cache_stats.py — 3 tests pinning
cache-stat surfacing across Anthropic/OpenAI dialects + backwards-
compat for no-cache responses.
* tests/test_proxy/test_openai_backend_path.py — 5 tests (Bedrock cache
fields, OpenAI fallback shape, CCR intercept with provider="openai",
CCR re-raise on exception, streaming signature contract).
* tests/test_proxy/test_mcp_stats_aggregation.py — 5 tests pinning the
aggregator across compress+retrieve mixes, empty events, unknown event
types, missing token fields, and read failures.
* tests/test_proxy/test_compression_failure_action.py — 12 tests pinning
the fail-closed decision matrix (timeout always refuses, small
transient passes through, oversize refuses, env override variants,
custom threshold, invalid threshold falls back, 0/negative ignored).
* examples/strands_bedrock_demo.py — model_id bumped from deprecated
Claude 3 Haiku to Sonnet 4.5 (the deprecated model now errors on
account access).
* examples/strands_via_proxy_demo.py — proxy + Bedrock cache + streaming
smoke test.
* examples/strands_mcp_dispatch_test.py — pure MCP round-trip probe.
* examples/strands_bundle_demo.py — full Strands + HeadroomBundle E2E
demo (this is the shape a real Strands user copies into their app).
Full pytest: 5327 passed, 178 skipped. The previously-failing
test_core_operations.py::TestAddBatch::test_add_batch_basic passes now
that the huggingface-hub pin in pyproject.toml unblocks transformers
imports.
E2E verified live against AWS Bedrock (Sonnet 4.5):
* cache_write=10,438 on turn A → cache_read=10,438 on turn B
* streaming SSE final usage frame carries cache_read_input_tokens
* 78.7% reduction on a 50 KB JSON tool_result via SmartCrusher (
dispatched per-content-type by ContentRouter)
* Strands Agent + HeadroomBundle: model autonomously called
headroom_compress + headroom_retrieve via MCP; CompressionStore
round-trip succeeded; final answer correct.
|
||
|
Garm
|
084678df7c |
fix(proxy): strip cache_control before hashing turn_id
compute_turn_id hashed the raw message dicts, which meant the same user-text message produced a different hash on each call of one agent loop because clients (notably Claude Code) move the cache_control breakpoint to the newest message per call. The user-text block carries cache_control on call 1 and not on call 2, so the serialized prefix differs and the turn_id rolls over. Effect downstream: every API call becomes its own "turn" and any prompt-level aggregation (e.g. the Headroom desktop app's prompt all-time record) collapses to the largest single call, not the sum across the prompt. Add a small recursive normalization pass that strips cache_control from the hashed prefix and from list-shaped system prompts before hashing. Two new tests cover cache_control moving between calls on both the messages array and the system prompt. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
58282bbc5e |
test(proxy): cover turn_id branches flagged by codecov
Patch coverage on helpers.py was 87% — 5 lines of compute_turn_id were untested. Add cases for: non-dict / non-user messages in the reverse scan, empty-string user content (should keep scanning), mixed text+tool_result content (agent-loop continuation, not a turn boundary), and system=None (hashes without the system segment). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garm
|
e8835affb8 |
docs(changelog): record turn_id feature; fix import order in new test
Follow-up to
|
||
|
Garm
|
c4464c066f |
feat(proxy): emit turn_id linking agent-loop API calls from one user prompt
Adds compute_turn_id() helper that hashes (model, system, messages prefix up to the last user text message). An agent loop sends the same user-text prefix across every iteration plus a growing tool chain, so this id is stable across the turn but rolls over when the user sends a new prompt. Stamps the id onto RequestLog at all three call sites (anthropic handler bedrock + direct branches, and the streaming handler) and surfaces it as turn_id in /transformations/feed so downstream consumers can aggregate savings per user prompt rather than per API call. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JerrettDavis
|
0aae886f77 |
feat: add live transformations feed to dashboard
- New /transformations/feed endpoint returning message diffs - Alpine.js drawer UI with virtual scrolling and auto-stream pause - Live Feed button hidden when log_full_messages=false - Added --log-messages CLI flag to enable full message logging - Backend stores request/response messages when enabled Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |