mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
21 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
JD Davis
|
8a1d38bc5d
|
fix(proxy): complete stateless Responses and buffered CCR lifecycle (#2997)
## Description
Consolidates the related OpenAI Responses ZDR/stateless continuation and
buffered CCR response-lifecycle corrections on current main. It
preserves client storage policy, makes Headroom-owned continuations
stateless across HTTP and WebSocket, and prevents buffered streaming
paths from committing a false HTTP 200 before the real upstream outcome
is known.
Closes #2675
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Preserves explicit and omitted Responses `store` policy instead of
forcing provider storage or disabling memory tools.
- Replays normalized input, replayable outputs, encrypted reasoning
content, and Headroom function outputs without `previous_response_id`.
- Applies the same stateless continuation policy to HTTP and WebSocket.
- Prevents transparent memory execution after client-visible WebSocket
output.
- Delays buffered CCR ASGI status/headers until the operation resolves
for Anthropic Messages and OpenAI Responses.
- Preserves real 429/5xx status and retry headers.
- Converts malformed non-JSON/non-SSE upstream 200 replies to a
sanitized 502 protocol error.
- Preserves valid JSON-to-SSE synthesis and existing SSE adaptation.
- Removes unreachable task cleanup left behind after replacing the old
keepalive polling loop with a direct awaited operation.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
118 passed across the changed HTTP/WS ZDR, lifecycle, and both-provider CCR suites
11526 tests collected with no collection errors
ruff check .: All checks passed
ruff format --check .: 1411 files already formatted
mypy headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py:
Success: no issues found in 2 source files
```
Exact-head CI is entirely green on
`
|
||
|
Pragadeesh
|
d76fce04a3
|
fix(proxy): adapt 200 SSE upstream replies on buffered /v1/responses instead of 502 (#2622)
## Description The buffered HTTP `/v1/responses` path (`_buffered_ccr_operation` in `headroom/proxy/handlers/openai.py`) assumed every upstream reply to a `stream: false` request is JSON. Some OpenAI-compatible upstreams answer with a valid `200 text/event-stream` body carrying Responses API events. `response.json()` raised `JSONDecodeError`, which is not in the narrow usage-extraction catch (`KeyError, TypeError, AttributeError`), so it escaped to the outer handler and the **successful** upstream reply was converted into a generic `502 proxy_error` — the client loses the response and typically retries, duplicating paid calls. The fix classifies the upstream reply at the ingestion boundary by its declared `Content-Type` (the SSE spec's own discriminator) instead of parsing by expectation: - **200 SSE with a terminal `response.completed` event** → the complete response object is reassembled from that event (`_openai_responses_from_sse`, the inverse of the existing `_openai_responses_to_sse`) and swapped in as a synthesized `application/json` response *before any parsing happens*. Everything downstream — usage extraction, CCR retrieval handling, memory-tool handling — runs unmodified. - **200 SSE without a recognizable terminal event** → the successful upstream body is forwarded to the client unchanged (sanitized headers) rather than fabricating a 502. Adapt only when the adaptation is provably faithful; otherwise pass through. - **Everything else** (normal JSON replies, non-200s) → byte-identical pre-existing behavior. Deliberately *not* done: widening the `except` clause (would leave `resp_json` unbound and break the downstream pipeline) and body sniffing (the declared media type is trusted; a mislabeled body keeps today's behavior). Closes #2613 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/openai.py`: new module-level helper `_openai_responses_from_sse()` (SSE-spec framing: blank-line event separation, multi-line `data:` joining, `\r` tolerance, at most one stripped space, `[DONE]` skipped; returns the terminal event's `response` object or `None`), placed next to its inverse `_openai_responses_to_sse()`. - `headroom/proxy/handlers/openai.py::_buffered_ccr_operation()`: content-type dispatch for 200 replies immediately after the upstream response (and after wire-debug capture, so debug logs keep the true upstream bytes) — adapt SSE→JSON when a terminal event exists, pass through unchanged when it doesn't. - `tests/test_openai_codex_routing.py`: two new handler-level tests (see below). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # Both new tests watched failing BEFORE the fix with the exact issue signature: # ERROR headroom.proxy:openai.py [req-1] OpenAI responses request failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0) # assert 502 == 200 $ pytest tests/test_openai_codex_routing.py -q 24 passed in 2.08s $ pytest tests/test_openai_codex_routing.py tests/test_ccr_response_handler_openai_responses.py tests/test_codex_responses_passthrough_bytes.py -q 38 passed, 1 warning in 13.80s $ pytest tests/test_output_shaper_responses.py tests/test_codex_responses_waste_signals.py tests/test_codex_openai_contract_parity.py tests/test_openai_beta_session_sticky.py tests/test_openai_max_completion_tokens.py tests/test_openai_response_cache_key.py tests/test_litellm_openai_passthrough.py -q 61 passed, 1 warning $ ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_routing.py All checks passed! $ mypy headroom/proxy/handlers/openai.py Success: no issues found in 1 source file ``` New tests: - `test_handle_openai_responses_non_stream_adapts_sse_upstream` — 200 SSE with `response.completed` → client gets 200 `application/json` with the reassembled response. - `test_handle_openai_responses_non_stream_passes_through_unparseable_sse` — 200 SSE with no terminal event → client gets 200 with the body unchanged, never a 502. ## Real Behavior Proof - Environment: macOS 26.5 (arm64), Python 3.12 via `uv`, headroom from source (editable install). Local fake OpenAI-compatible upstream (`http.server`) that answers every `POST` with `200 text/event-stream` containing a `response.completed` event and `data: [DONE]` — the upstream behavior reported in the issue. Proxy started with `OPENAI_TARGET_API_URL=http://127.0.0.1:9302 headroom proxy --port <port>`. - Exact command / steps: same-session A/B against real proxy processes — identical upstream and identical request, only the checked-out revision changed: ```bash curl -s -w "\nHTTP_STATUS=%{http_code} CONTENT_TYPE=%{content_type}\n" \ -X POST http://127.0.0.1:<port>/v1/responses \ -H "content-type: application/json" -H "authorization: Bearer sk-test" \ -d '{"model":"gpt-5.4","stream":false,"input":"hello"}' ``` - Observed result: unpatched `main` converts the successful upstream reply into the issue's 502; this branch returns the complete response as JSON. Full captures: **Before (unpatched `main`, port 8794):** ``` {"error":{"message":"An error occurred while processing your request. Please try again.","type":"server_error","code":"proxy_error"}} HTTP_STATUS=502 ``` **After (this branch, port 8795):** ``` {"id": "resp_sse_repro", "object": "response", "status": "completed", "model": "gpt-5.4", "output": [{"type": "message", "id": "msg_1", "role": "assistant", "content": [{"type": "output_text", "text": "hello from sse upstream"}]}], "usage": {"input_tokens": 2, "output_tokens": 1}} HTTP_STATUS=200 CONTENT_TYPE=application/json ``` - Not tested: a wild third-party SSE-answering upstream (the repro uses a local stub shaped per the issue report); the buffered-stream-CCR variant of this path against a live upstream (unit-tested only); Windows. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - Documentation checklist item is N/A — internal proxy behavior fix, no documented surface changes. - Known residual (pre-existing, out of this issue's scope): a **non-200** upstream reply with a non-JSON body (an SSE error stream, a gateway HTML error page) still follows the old `JSONDecodeError → 502` path, blurring a meaningful upstream error into a generic 502. This PR deliberately adapts only declared-SSE **200** replies, where reassembly from `response.completed` is provably faithful. Happy to file the non-200 case as a follow-up issue if maintainers want it tracked. |
||
|
inix
|
806d2e468a
|
fix(proxy): offload OpenAI and Gemini tokenizer counting off the event loop (#2498)
## Description The OpenAI and Gemini handlers resolved the tokenizer and counted the conversation inline on the event loop. When a model resolves to a HuggingFace tokenizer (the registry routes qwen, deepseek, llama, phi, falcon, and more there) a cold cache runs `AutoTokenizer.from_pretrained` behind a 10s `thread.join`, which freezes the whole server. That is the GH #1701 stall, now reachable from OpenAI and Gemini because `/v1/chat/completions` and `/v1/responses` are documented multi-provider passthroughs and receive those models. Anthropic already routed the same call through a fail-open `_count_tokens_offloaded` helper. This hoists that helper to the shared `HeadroomProxy` base and sends the OpenAI and Gemini sites through it too. No linked issue. This is the OpenAI and Gemini follow-on to #1738, which offloaded the Anthropic and batch paths. GH #1701 is the original freeze report. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Hoisted `_count_tokens_offloaded` from `AnthropicHandlerMixin` to the shared `HeadroomProxy` base, next to `_run_compression_in_executor`. It resolves and counts on the bounded compression executor and fails open to estimation on timeout, error, or executor quarantine. - Routed 6 inline sites through it: `handle_openai_chat`, `handle_openai_responses`, `handle_gemini_generate_content`, `handle_google_cloudcode_stream`, `handle_gemini_count_tokens`, and `handle_gemini_stream_generate_content` (resolve only, keeps its per-part `count_text` loop). - Removed 6 now-dead local `get_tokenizer` imports. - Left batch's per-line counts inline on purpose. They run on an already-warm tokenizer, so offloading them adds executor churn without touching the cold load. Batch's `pipeline.apply` was already offloaded in #1738. - Extended the wiring guard to all 7 provider handlers, added a quarantine fail-open test and a `count_text` fail-open test, and stubbed the method on 2 mixin-only handler doubles. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/server.py headroom/proxy/handlers/{anthropic,openai,gemini}.py tests/test_tokenizer_count_offload.py All checks passed! $ pytest tests/test_tokenizer_count_offload.py 6 passed in 4.39s # offload suite + all 26 handler-calling test files + handler-helpers/batch/tokenizers $ pytest tests/test_tokenizer_count_offload.py tests/test_openai_codex_routing.py tests/test_gemini_nonjson_status.py ... tests/test_tokenizers.py 377 passed, 15 skipped, 15 warnings in 86.60s (0:01:26) ``` ## Real Behavior Proof - Environment: macOS (Darwin), Python 3.13, proxy built from this branch. A synthetic tokenizer that sleeps 0.5s on resolve+count stands in for a cold HuggingFace `from_pretrained` load, with a 10ms asyncio loop-canary running alongside. - Exact command / steps: monkeypatch `headroom.tokenizers.get_tokenizer` to the 0.5s-sleeping tokenizer, then time a concurrent canary across two counts, the offloaded `await proxy._count_tokens_offloaded("qwen2.5-coder", messages)` and the old inline `get_tokenizer(model).count_messages(messages)`. - Observed result: the offloaded path kept the loop live at 41 canary ticks during the 509ms count, the inline path froze it to 0 ticks over 502ms, and both returned the same token count. Full run was 377 passed, 15 skipped, 0 failed. The new quarantine test confirms an unrelated compression timeout downgrades counting to estimation instead of raising a 500. - Not tested: live HuggingFace downloads and real qwen/deepseek traffic. No API keys in this environment, so the Gemini and OpenAI integration tests skip on `GEMINI_API_KEY`/`OPENAI_API_KEY`. `mypy headroom` did not finish locally (cold-times-out past 10 minutes on this box), so type-checking is left to CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - No linked issue. Follow-on to #1738. - Batch per-line counts stay inline: they run on an already-warm tokenizer, so offloading them adds executor churn without addressing the cold load. - Found a 6th site mid-implementation. `handle_gemini_stream_generate_content` also resolved the tokenizer inline but counts via a `count_text` loop, so it takes the resolve-only path. Verified `EstimatingTokenCounter.count_text` exists, so its fail-open branch does not crash. - `mypy headroom` cold-times-out locally (server.py pulls the full graph). Deferred to CI's Linux shards, same as prior PRs on this file. `ruff` and `pytest` run clean. - Documentation checkbox left unchecked: this change ships no user-facing doc update. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
John Xu
|
1c50eca8b3
|
fix(proxy): skip Responses memory tools for ChatGPT auth (#1579)
## Description Fix ChatGPT/Codex session-auth Responses proxy handling so the ChatGPT backend always receives an explicit `store=false`, while keeping Responses memory tools limited to the regular API-key path where stored responses are supported. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Detect ChatGPT auth before Responses memory-tool injection and force `store=false` for ChatGPT-auth Responses payloads. - Skip Responses memory tools and transparent memory-tool continuation handling for ChatGPT auth across HTTP, WebSocket first frames, WebSocket follow-up `response.create` frames, and WS-to-HTTP fallback. - Preserve API-key behavior after the current main merge: API-key requests that explicitly set `store=false` skip Responses memory tools, while API-key requests that receive injected memory tools are forced to `store=true` for continuation support. - Address Copilot formatter comments by making `_allow_responses_memory_tools` call sites formatter-stable. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --extra dev ruff format --check headroom/proxy/handlers/openai.py 1 file already formatted $ uv run --extra dev ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_routing.py tests/test_openai_codex_ws_timings.py tests/test_ws_http_fallback.py All checks passed! $ uv run --extra dev python -m pytest -q tests/test_openai_codex_routing.py tests/test_openai_codex_ws_timings.py tests/test_ws_http_fallback.py 37 passed in 0.34s ``` ## Real Behavior Proof - Environment: Local checkout of `fix/codex-store-false-memory-tools` using `uv run --extra dev`. - Exact command / steps: Ran the focused formatter, lint, and pytest commands listed in `Testing`. - Observed result: Formatting is stable, lint passes, and the focused OpenAI/Codex routing and fallback tests pass. - Not tested: Full test suite, `mypy headroom`, and a fresh live ChatGPT backend probe after the formatter-only follow-up. The original PR validation recorded that valid ChatGPT subscription backend requests return `200` with `store=false`, while identical `store=true` or omitted `store` requests return `400 Store must be set to false`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes - Post-deploy monitoring terms: `Responses: forced store=false for ChatGPT auth`, `WS Responses: forced store=false for ChatGPT auth`, `chatgpt_store_false`, `Memory: forced store=true for Responses memory tool continuation`, and upstream 400s containing `Store must be set to false`. - Expected healthy signals: ChatGPT-auth Responses requests keep `store=false` and no longer fail with `Store must be set to false`; API-key memory-tool flows still inject memory tools and can continue via `previous_response_id`. - Rollback trigger: any increase in ChatGPT-auth 400s, API-key memory-tool continuation failures, or missing memory tool injection on API-key Responses requests. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
nangsontay
|
e5b3a634df
|
fix(proxy): dedupe Codex WS request logging for accurate mixed-provider dashboards (#2189)
## Description Running Claude Code (Anthropic) and Codex (OpenAI) against the **same** Headroom proxy instance on one port produced incorrect, unstable dashboard data. The proxy core is provider-isolated and multi-provider-safe by design; the defect was in the observability layer. The Codex `/v1/responses` **WebSocket** handler was the only path in the proxy that wrote to the request logger by hand instead of through the unified `emit_request_outcome` funnel, and it did so twice per session close: the per-turn funnel record plus an unconditional cumulative session-summary `RequestLog`. This PR removes the duplicate summary log so Codex WS emits exactly one request log per turn, matching the HTTP provider paths. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Dropped the duplicate cumulative session-summary `RequestLog` in the Codex WS handler while preserving the per-turn `emit_request_outcome` path. - Preserved gated `request_messages` and `turn_id` on residual outcomes so dashboard telemetry keeps the useful attribution without double-counting tokens. - Ensured explicit `--anyllm-provider` wins over a leaked `HEADROOM_ANYLLM_PROVIDER` environment variable. - Registered retry delay settings that had drifted out of the settings registry. - Hardened tests against developer-shell `HEADROOM_*` / `ANTHROPIC_CUSTOM_HEADERS` leakage and stabilized several focused proxy/wrap test fixtures. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ .venv/bin/pytest tests/ -q -p no:cacheprovider 8529 passed, 537 skipped, 5831 warnings in 276.25s (0:04:36) $ .venv/bin/ruff check <touched files> All checks passed! ``` ## Real Behavior Proof - Environment: macOS (Darwin 25.4.0), Python 3.13.14, pytest 9.0.3, ruff via project venv, branch `fix/multi-provider-runtime`. - Exact command / steps: Ran the full test suite without pytest cache provider and Ruff on all touched files; used `git stash` to confirm the stale fake-config failures pre-existed this change. - Observed result: Full suite passed with no failures; Ruff passed; Codex WS now routes end-of-session logging through `emit_request_outcome`, emitting one request log per turn with the same accounting model as Anthropic HTTP turns. - Not tested: Live simultaneous Claude + Codex dashboard run. `mypy headroom` was not run to completion; a scoped run reported one pre-existing `settings_store.py:470` coercion error outside this diff. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A - server-side observability fix; no UI markup changed. ## Additional Notes - The proxy's multi-provider routing, header/auth isolation, and per-model cache keying are already correct and unchanged here; only the WS observability write path was double-counting. - Architectural assessment: `plans/reports/research-260714-0004-multi-provider-upstream-compression-report.md`; root-cause + resolution trail: `plans/reports/debug-assessment-260714-0011-dashboard-instability-mixed-claude-codex-report.md`. - No live simultaneous Claude + Codex dashboard run was performed; validation is from test coverage and code review of the WS logging path. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
gglucass
|
9fbd47ba6b
|
fix(proxy): strip Codex lite header on the HTTP /responses path (#1663)
## Description The WebSocket `/responses` handler already drops `X-OpenAI-Internal-Codex-Responses-Lite` before forwarding upstream (#1543) — OpenAI rejects newer Codex models (gpt-5.5 / gpt-5.4 / gpt-5.4-mini) when this client-only header leaks. The **HTTP POST `/responses`** handler (`handle_openai_responses`), however, forwards request headers verbatim after `_strip_internal_headers` (which removes only `x-headroom-*`), so on the HTTP path the lite header still reaches `chatgpt.com/backend-api/codex/responses`. This closes that remaining un-stripped path so both `/responses` transports behave identically. Closes # <!-- no tracking issue; found during a live support investigation --> ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/openai.py`: in `handle_openai_responses` (HTTP POST path), immediately after `headers = _strip_internal_headers(headers)`, drop any header whose lowercased name equals `_CODEX_RESPONSES_LITE_HEADER` — mirroring the existing WS-handler filter. No new imports (the constant is module-level); the WS path is unchanged. - `tests/test_openai_codex_routing.py`: add `test_handle_openai_responses_strips_codex_lite_header_upstream`, which pushes the lite header plus an adjacent header through the HTTP POST handler and asserts the lite header is dropped upstream while the adjacent header survives. ## Testing - [x] Unit tests pass (`pytest`) — directly-relevant files (see output) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed — no live upstream traffic (see Real Behavior Proof) ### Test Output ```text $ uv run --extra dev pytest tests/test_openai_codex_routing.py tests/test_openai_codex_ws_lifecycle.py -q 39 passed in 1.13s $ uv run ruff check . All checks passed! $ uv run --extra dev mypy headroom Success: no issues found in 404 source files ``` ## Real Behavior Proof - Environment: local `uv` venv (Python 3.10), no live provider required. - Exact command / steps: `uv run --extra dev pytest tests/test_openai_codex_routing.py::test_handle_openai_responses_strips_codex_lite_header_upstream tests/test_openai_codex_ws_lifecycle.py::test_ws_codex_responses_lite_header_is_not_forwarded_upstream` - Observed result: the new test drives a ChatGPT-auth HTTP POST `/responses` request carrying `X-OpenAI-Internal-Codex-Responses-Lite: true` and an adjacent `X-OpenAI-Debug: keep-me`; the captured upstream headers contain the adjacent header but not the lite header. The WS regression test still passes. - Not tested: live Codex traffic against OpenAI with real credentials. (Separately: for a WebSocket-only ChatGPT-auth client the lite signal is not carried as an HTTP header on the handshake — that case is out of scope here.) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (no doc-facing behavior change) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable — N/A (changelog is generated from conventional commits; commit is `fix(proxy): …`) ## Screenshots (if applicable) N/A — backend header-handling change. ## Additional Notes - Scope of checks: `pytest` was run on the two directly-relevant files (`test_openai_codex_routing.py`, `test_openai_codex_ws_lifecycle.py`), not the entire suite; `ruff check .` and `mypy headroom` were run repo-/package-wide. - Complements #1543 (WS path) by closing the HTTP POST path; it is the minimal mirror of that filter. - `Closes #` intentionally blank: found during a support investigation with no tracking issue. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Vinay Gupta
|
2a34a822f2
|
fix(proxy): preserve Responses passthrough bytes (#1598)
## Description
Fixes the Python `/v1/responses` forwarding path for encoded Codex
Desktop requests.
When Headroom receives a compressed Responses request, the request body
is decoded before JSON parsing. The handler then forwarded a rewritten
JSON body while preserving the inbound `Content-Encoding` header, so
upstream could receive plain JSON bytes that were still labeled as
`zstd`/`gzip`. This change keeps the decoded original bytes for true
passthrough requests, strips stale entity headers, and marks Responses
body mutations so memory/compression paths still use canonical
serialization.
Closes #1542
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Switched `/v1/responses` parsing to keep the decoded original request
bytes.
- Stripped stale `content-encoding` and `transfer-encoding` headers
before forwarding decoded JSON bodies.
- Wired Responses streaming and non-streaming forwarding through the
existing byte-faithful passthrough controls.
- Marked Responses memory and compression body mutations so mutated
requests continue to serialize canonically.
- Added regression tests for gzip and zstd encoded Responses passthrough
bodies.
## Testing
- [x] Unit tests pass (`pytest`) — GitHub CI test shards passed
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`) — GitHub CI ran `mypy
headroom --ignore-missing-imports`
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ /tmp/headroom-1542-testenv/bin/python -m ruff check .
All checks passed!
$ /tmp/headroom-1542-testenv/bin/python -m ruff format --check .
1014 files already formatted!
$ HEADROOM_REQUIRE_RUST_CORE=false PYTHONPATH=/Users/vinaygupta/Desktop/git/headroom-fix-1542-zstd-passthrough /tmp/headroom-1542-testenv/bin/python - <<'PY'
# Injected an in-memory headroom._core import stub for this local checkout,
# then ran pytest.main(["tests/test_openai_codex_routing.py", "-q"])
PY
19 passed in 0.55s
$ HEADROOM_REQUIRE_RUST_CORE=false PYTHONPATH=/Users/vinaygupta/Desktop/git/headroom-fix-1542-zstd-passthrough /tmp/headroom-1542-testenv/bin/python - <<'PY'
# Injected an in-memory headroom._core import stub for this local checkout,
# then ran pytest.main(["tests/test_proxy_byte_faithful_forwarding.py", "-q"])
PY
35 passed, 1 warning in 1.33s
$ HEADROOM_REQUIRE_RUST_CORE=false PYTHONPATH=/Users/vinaygupta/Desktop/git/headroom-fix-1542-zstd-passthrough /tmp/headroom-1542-testenv/bin/python - <<'PY'
# Injected the same in-memory headroom._core import stub,
# then ran pytest.main(["tests/test_proxy_compression_headers.py", "-q"])
PY
10 passed in 0.05s
GitHub CI on `
|
||
|
Devanshi Vyas
|
c632023cc1
|
fix(websocket): harden responses websocket origin handling (#1481)
## Description Validate browser WebSocket origins before accepting WS sessions. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - validate Responses WebSocket `Origin` before routing the session upstream - keep native clients that omit `Origin` working - allow loopback origins by default and support explicit origins via `HEADROOM_WS_ORIGINS` ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text .venv/bin/python -m pytest tests/test_openai_codex_routing.py Result: 19 passed .venv/bin/python -m ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_routing.py Result: All checks passed ``` ## Real Behavior Proof - Environment: macOS - Exact command / steps: `venv/bin/python -m pytest tests/test_openai_codex_routing.py``.venv/bin/python -m ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_routing.py` - Observed result:`19 passed` `All checks passed` - Not tested: NA ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable |
||
|
Michael Sam
|
6d3f39f213
|
feat: add dashboard agent usage stats (#814)
## Description Add a clear dashboard view for per-agent token usage so end users can see Cursor, Claude, Codex, and other detected clients with before/after token counts, tokens saved, and savings percentages. The stats API now exposes a stable `agent_usage` object that the dashboard renders near the top of the session view. Fixes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### New Files **Tests:** - `tests/test_dashboard_agent_usage.py` — Covers agent classification, exact per-request aggregation, and aggregate fallback behavior. ### Modified Files - `headroom/proxy/server.py` — Adds per-agent usage aggregation to `/stats` with before tokens, after tokens, output tokens, saved tokens, savings percentage, source, providers, and models. - `headroom/dashboard/templates/dashboard.html` — Adds a prominent Agent Usage panel with totals, coverage status, per-agent token-flow bars, request counts, before/after tokens, saved tokens, and share of savings. ## Testing - [x] Unit tests pass: `.venv312/bin/pytest tests/test_dashboard_agent_usage.py` - [x] Linting passes: `.venv312/bin/ruff check headroom/proxy/server.py tests/test_dashboard_agent_usage.py` - [x] Diff whitespace check passes: `git diff --check origin/main...HEAD` - [x] Dashboard smoke render: local proxy on `127.0.0.1:8790`, captured Chrome headless screenshot of `/dashboard` - [x] New tests added for new functionality ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing relevant unit tests pass locally with my changes - [ ] I have made corresponding changes to the documentation - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The agent usage panel uses exact request-log data when available. If detailed request logs are empty, it falls back to aggregate provider/model request counts and labels the coverage as aggregate fallback so users are not misled. |
||
|
ashishpatel26
|
e8ecd08829 | fix(codex): fail open for proxy compression timeout | ||
|
chopratejas
|
993c9076f9 |
refactor(proxy): migrate Codex WS + OpenAI HTTP + batch handlers; delete Databricks
Completes the migration of every ``metrics.record_request`` call site in ``headroom/proxy/handlers/`` onto the canonical funnel. After this commit, **zero ad-hoc record_request calls remain** across the entire handler subtree. Every request — regardless of provider, harness, or transport — flows through ``emit_request_outcome``. Migrated sites (this commit): * **handle_openai_responses_ws** (Codex WS) — 2 sites: - per-turn record (per ``response.completed``) - session-end residual (leftover tokens not captured per-turn) Pre-refactor these sites emitted only metrics + cost_tracker — no RequestLog, no PERF — so Codex traffic was invisible to ``headroom perf`` and the recent-requests feed. Funnel restores all four effects uniformly per turn. (Closes the visibility half of what #471's sibling PR addressed for the scheduler half.) The explicit session-summary RequestLog at session-end stays as a separate explicit log entry — it's a session-cumulative summary, distinct from per-turn observations. * **handle_openai_chat** — 3 sites: - response-cache hit (uses ``from_response_cache=True``) - backend-routed (LiteLLM/AnyLLM) non-streaming success - direct OpenAI non-streaming success * **handle_openai_responses** HTTP (Codex HTTP transport) — 1 site * **handle_passthrough** (OpenAI passthrough endpoints) — 1 site * **batch.py** handlers — 5 sites: - handle_google_batch_create - handle_google_batch_passthrough (Files API forward) - handle_google_batch_passthrough (list/get/cancel) - handle_google_batch_results (CCR-processed) - handle_batch_create (OpenAI batches) All converge on the funnel. Several gain request_id allocation they didn't have before (passthrough sites previously emitted ``request_id=None`` in logs). **Deleted: handle_databricks_invocations + its route + test cases.** Databricks was a 57-line thin wrapper at openai.py that parsed JSON, injected the model from URL into body, and delegated to ``handle_openai_chat``. It enabled ``databricks serving-endpoints query <model> --profile HEADROOM`` direct CLI use. No evidence of active users (no docs, no issues, no mentions). Databricks-hosted models still work via the standard ``/v1/chat/completions`` surface; LiteLLM has its own Databricks support too. If a user complains, this PR is a 30-minute revert. Architectural note: also updated 2 more test dummies (``_DummyOpenAIHandler`` in routing + WS lifecycle tests) to bind ``_record_request_outcome`` via the free function ``emit_request_outcome`` — same pattern as ``_run_compression_in_executor``. Final migration tally (from P0 audit + extensions): * **18 audit sites** + **5 batch.py sites discovered during migration** = 23 sites migrated * **1 site deleted** (Databricks) * **0 sites remaining** anywhere under ``handlers/`` Surface impact (this commit): * openai.py: −168 LOC (315 deletions − 147 insertions) * batch.py: +35 LOC (124 ins − 89 del; mostly comments) * proxy_routes.py: −4 LOC (Databricks route gone) * tests: +11 LOC (dummy `_record_request_outcome` bindings, 2 sites) * Net: ~−126 LOC in production handler code Tests * All 157 existing streaming/cache/Codex/anthropic/openai/backpressure/ routes tests pass with zero regressions. * ruff + ruff-format + mypy clean. This brings the cumulative refactor delta (across all 3 commits on this branch) to: contract introduced (outcome.py + funnel): ~+200 LOC fixed cost handler migrations (streaming + anthropic + gemini + openai + batch + WS): ~−700 LOC Databricks deletion: −57 LOC ────────────────────────────────────────────── ───────── Net production code delta: ~−557 LOC Plus +474 LOC of test coverage (RequestOutcome unit tests + funnel contract assertions). And every handler now emits identical observable outputs per request: same metrics shape, same cost_tracker shape, same RequestLog shape, same PERF format. The wire is uniform. |
||
|
chopratejas
|
7694f050fe |
fix: per-project memory storage so projects can no longer bleed memories (GH #462)
Memory retrieval was partitioned only by `x-headroom-user-id`. Claude
Code never sets that header, so every project a user worked on landed
in one global `default` bucket; the proxy then injected semantically
similar memories from that mixed bucket into every `/v1/messages`
request, regardless of which repo the session was actually about. The
injected `## Relevant Memories` block reads like a prompt-injection
payload and Claude has been seen to refuse to act on it, defeating the
feature.
This change makes leakage structurally impossible by giving each
resolved workspace its own SQLite database file. The wrong DB is
simply not open during a request.
- `headroom/memory/storage_router.py` (new) — `MemoryStorageMode`
(project/user/global), `ProjectResolver` (x-headroom-project-id →
x-headroom-cwd → --memory-project-root CLI override → env-block
parse: `Primary working directory:` / `Working directory:` / `cwd:`,
no regex), and `BackendRouter` with an LRU of open `LocalBackend`s
keyed by db_path.
- `proxy/memory_handler.py` — `MemoryConfig.storage_mode` defaults to
`PROJECT`. Provider handlers build a `RequestContext` once and pass
it through; `search_and_format_context`, `handle_memory_tool_calls`,
and the `_execute_*` methods route save/search/update/delete on the
per-project backend. Qdrant-neo4j gets a composite
`user::project_key` partition so external Mem0-style deployments
also isolate per project without a parallel collection.
- Fix C — injected block carries provenance:
`## Relevant Memories (workspace: <basename>, scope: project)`.
CCR proactive-expansion block gets a matching workspace tag.
- `memory/factory.py` — process-wide embedder cache so opening N
project DBs doesn't load the embedder N times. OpenAI key
validation runs ahead of the cache.
- CLI — `--memory-storage={project,user,global}` (default `project`),
`--memory-project-root` override, rewritten `--memory` help text,
banner reports storage mode.
- Migration UX — if the legacy single-file DB has content while
project mode is active, an INFO log points users at
`--memory-storage=global`. Bridge currently only syncs the legacy
DB; a WARN fires when bridge + project mode are combined.
Backward-compatible: legacy `~/.headroom/memory.db` untouched and
reachable via `--memory-storage=global`. `request_context` is
keyword-only on entry points so existing tests/mocks keep working.
Tests: 24 new (resolver tiers, LRU eviction, two-cwd isolation,
user-mode partition, legacy fallback, provenance headers); full
suite 5260 passing, ci-precheck green.
|
||
|
Tejas Chopra
|
eaf5980b4a | fix: stabilize codex compression, stats, and proxy lifecycle | ||
|
chopratejas
|
221109d95e |
fix: PR-C5 retire responses_converter.py — Rust owns /v1/responses
Phase C realignment final step. The Anthropic↔OpenAI Responses↔Chat Completions converter (`headroom/proxy/responses_converter.py`) was a fragile shim that mishandled Codex `phase`, multi-text-part rebuild, and unknown item types. It existed only because the Python compression pipeline operates on Chat Completions messages and Responses items needed to be coerced. After PR-C3 (Rust HTTP) and PR-C4 (Rust streaming + Conversations awareness) the Rust handler at `crates/headroom-proxy/src/handlers/responses.rs` processes Responses items natively without converting between shapes, so the converter has no remaining caller and is retired. Changes: - Delete `headroom/proxy/responses_converter.py` (336 lines). - Delete `tests/test_responses_converter.py` (408 lines) and `tests/test_proxy_responses_phase_preservation.py` (148 lines). Coverage moves to `crates/headroom-proxy/tests/integration_responses*`. - `handlers/openai.py::handle_openai_responses` (HTTP): drop the converter import, the list-input → messages conversion, the full compression dispatch (the `original_items is not None` guard was its only caller), the inflation-revert block, and the back-conversion. Memory injection is preserved via the existing `append_text_to_latest_user_input_item` helpers which operate on `body["input"]` directly. Telemetry vars stay zeroed. - `handle_openai_responses_ws` (WebSocket): retire the first-frame compression block plus the list-input branch in memory search. WS sessions now pass through unmodified; WS-side compression is a follow-up via Rust if ever needed. - Drop the now-unused `previous_response_id` local and the inner imports of `COMPRESSION_TIMEOUT_SECONDS`, `get_tokenizer`, `extract_user_query` from the WS handler. - Rename and update `test_handle_openai_responses_stream_keeps_compression` to assert `apply.call_count == 0` — the new contract is that Python compression on /v1/responses is retired. Acceptance criteria: - `git grep -n "responses_converter" -- headroom/ tests/` returns nothing. - `pytest -x` green (4543 passed, 410 skipped). - `cargo test --workspace` green (0 failures). - Live validation against OpenAI through the proxy: /v1/chat/completions, /v1/responses string + list input, /v1/responses SSE stream, and ws:// /v1/responses all forward correctly without invoking compression. Net diff in handlers/openai.py: +31 / -179. Refs REALIGNMENT/05-phase-C-rust-proxy.md PR-C5. |
||
|
chopratejas
|
aec5ba3253 |
fix: A6 — anthropic-beta and openai-beta deterministic merge + session-sticky
PR-A6 of the Phase A cache-safety lockdown. Eliminates P5-50 and preps
P0-6 (memory tool injection toggling).
Two cache-killer patterns the merge + tracker defeat:
1. Mid-session mutation: when memory was enabled the proxy did an
ad-hoc concat of `context-management-2025-06-27` onto the client
value (anthropic.py:1244-1248). The order varied with the client
value, breaking byte-stable headers across turns.
2. Token drop-out across turns: clients (Claude Code, Codex CLI) MAY
drop a beta token between turn N and turn N+1 even when the proxy
mutated turn N to add it. The cache hot zone is positional, so the
next turn's prefix bytes hash differently and the prefix-cache
read misses.
Changes
-------
`headroom/proxy/helpers.py`
* `merge_anthropic_beta` / `merge_openai_beta`: pure, deterministic,
order-preserving merge. Client tokens first (in their original
order), then Headroom-required tokens (in the order passed). Dedupe
is case-insensitive but preserves the original casing of the first
occurrence. No regex.
* `SessionBetaTracker`: bounded LRU keyed by (provider, session_id),
unioning client tokens with previously-seen tokens. OrderedDict
LRU; threading.RLock for thread safety (mirrors the
CompressionCache pattern from compression_cache.py).
* `get_session_beta_tracker` / `_reset_session_beta_tracker_for_test`
process-wide singleton with test reset.
* `log_beta_header_merge`: structured log per cache-affecting merge.
* Env-var knobs (NO HARDCODES):
- HEADROOM_BETA_HEADER_STICKY=enabled|disabled (default enabled).
- HEADROOM_BETA_TRACKER_MAX_SESSIONS (default 1000).
`headroom/proxy/handlers/anthropic.py`
* After `compute_session_id` (line ~744): record client
`anthropic-beta` against the session tracker, write the sticky
value back into `headers` if changed. Order matters: sticky-merge
FIRST so memory-injection has the canonical baseline.
* Memory-injection site (line ~1244): replace the ad-hoc concat with
`merge_anthropic_beta(headers["anthropic-beta"], required_tokens)`.
`headroom/proxy/handlers/openai.py`
* Chat-completions (line ~360): record/merge `openai-beta`.
* /v1/responses HTTP (line ~1213): compute `_responses_session_id`
and record/merge `openai-beta`.
* /v1/responses WS (line ~1711): replace the ad-hoc absent-only
inject with `merge_openai_beta(sticky, ["responses_websockets=
2026-02-06"])`. Replaces any case-variants of the existing key.
Tests
-----
`tests/test_anthropic_beta_session_sticky.py` (26 tests):
* Pure helper: empty inputs, only-client, only-headroom, ordering,
dedupe casing, deterministic memory-injection order, no-double-
inject when token already present.
* Tracker: sticky-on across turns even when client drops, casing
preservation, provider namespace independence, LRU eviction at
max_sessions, env-var validation (loud failures), thread safety
under 16-thread concurrent access, blank-input rejection.
`tests/test_openai_beta_session_sticky.py` (17 tests):
* Mirror of the anthropic suite for `OpenAI-Beta`.
* Plus WS-specific coverage: sticky-then-merge of
`responses_websockets=2026-02-06` against client baseline.
`tests/test_openai_codex_routing.py`
* Add `session_tracker_store` stub to `_DummyOpenAIHandler` so the
routing tests still exercise the responses HTTP handler now that
it computes a session_id for beta-merge.
Notes
-----
Build constraints honored:
* Configurable: HEADROOM_BETA_HEADER_STICKY,
HEADROOM_BETA_TRACKER_MAX_SESSIONS.
* No regex, no hardcodes (env-var bounds), no fallbacks (disabled
mode is operator opt-in for diagnostics, loud failures on invalid
values).
* Structured tracing log via `log_beta_header_merge`.
Acceptance:
* 43 new tests pass.
* `cargo test --workspace` green (no Rust changes).
* `make ci-precheck` green.
|
||
|
chopratejas
|
456a6b33af |
fix(test): stub _run_compression_in_executor on _DummyOpenAIHandler
The bounded compression executor introduced in this PR moved every handler's compression call from `asyncio.wait_for(asyncio.to_thread(...))` to `self._run_compression_in_executor(...)`, which lives on `HeadroomProxy` (server.py) and is inherited by handler mixins at runtime. The test's `_DummyOpenAIHandler` only inherits `OpenAIHandlerMixin`, not `HeadroomProxy`, so it lacks the method. The Responses API compression path caught the AttributeError and silently fell back — which made `test_handle_openai_responses_stream_keeps_compression` fail with `apply.call_count == 0`. Add a synchronous stub that just invokes the callable; tests don't need real thread-pool semantics. |
||
|
Adryan Eka Vandra
|
b0422cd27a
|
fix(proxy): retry responses transport without bypassing compression | ||
|
Tejas Chopra
|
c2451296df |
Add memory support for Codex via OpenAI Responses API handler
Inject memory context into instructions and memory tools into Responses API requests, and handle memory tool calls in responses. |
||
|
JerrettDavis
|
01fc1aaf94 |
test: fix codex routing response stub
Update the OpenAI Codex routing test double to match the response shape expected by the handler so the success path is exercised instead of falling into the generic failure branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
JerrettDavis
|
37f32a8922 |
test(openclaw): cover branch routing paths
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
JerrettDavis
|
92b7b09f70 | fix(openclaw): route codex through headroom proxy |