mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
1872 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Abhay Singh
|
7ff842da17
|
fix(proxy/openai): thread savings-profile kwargs into chat completions (#1606)
## Description OpenAI-compatible `/v1/chat/completions` requests didn't receive the same proxy savings/profile kwargs as the other compression paths. The live chat handler (`handle_openai_chat` in `headroom/proxy/handlers/openai.py`) called `openai_pipeline.apply()` with only `model_limit` / `context` / `frozen_message_count` / `biases` / `compression_policy` — it never passed `proxy_pipeline_kwargs(self.config)`. So when the proxy runs with `HEADROOM_SAVINGS_PROFILE=agent-90`, the effective config reports user/system-message compression and `target_ratio=0.10`, but the real chat path silently dropped all of it. OpenAI-compatible clients such as OpenCode kept protecting user messages and missed the configured profile. For contrast, `handlers/anthropic.py` passes `**proxy_pipeline_kwargs(self.config)` to every `apply()` call, and so does the dedicated OpenAI compress endpoint in this same module — only the two chat-completions `apply()` sites were missing it. Closes #1534 ## Fix Add `**proxy_pipeline_kwargs(self.config)` to both chat-path `apply()` calls (the token-mode branch and the non-token branch): ```python lambda: self.openai_pipeline.apply( messages=messages, model=model, model_limit=context_limit, context=extract_user_query(messages), frozen_message_count=openai_frozen_count, biases=_hook_biases, compression_policy=compression_policy, **proxy_pipeline_kwargs(self.config), # ← added ) ``` `proxy_pipeline_kwargs` is already imported in the module and is the exact helper the Anthropic handler and the OpenAI compress endpoint use, so the chat path now matches them. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/handlers/openai.py`: pass `**proxy_pipeline_kwargs(self.config)` on both `apply()` call sites in `handle_openai_chat` (token-mode and non-token branches). - `tests/test_proxy/test_openai_chat_savings_profile.py`: new regression test driving the chat handler with `savings_profile="agent-90"` and asserting the profile knobs reach `apply()`. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output The new test drives the real chat handler through the `create_app` + `TestClient` harness with a recording `apply()` stub. Before the fix it captures exactly the five kwargs the issue describes (no profile knobs); after the fix the profile knobs are present: ```text # before the fix (openai.py reverted, test kept) E AssertionError: assert None is True E + where None = {...}.get('compress_user_messages') # captured kwargs were: biases, compression_policy, messages, model, # model_limit, context, frozen_message_count — no profile knobs FAILED tests/test_proxy/test_openai_chat_savings_profile.py::test_chat_completions_threads_savings_profile_kwargs_into_apply # after the fix tests\test_proxy\test_openai_chat_savings_profile.py . ======================== 1 passed, 1 warning in 39.44s ======================== ``` No regression in the existing chat backend-path suite: ```text $ uv run pytest tests/test_proxy/test_openai_backend_path.py ======================== 5 passed, 1 warning in 15.78s ======================== $ uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_chat_savings_profile.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom built from this branch (`uv sync --extra dev`), proxy config `savings_profile="agent-90"`, `optimize=True`, `backend="anyllm"` with a mocked OpenAI upstream. - Exact command / steps: started the app with `create_app(config)`, replaced `proxy.openai_pipeline.apply` with a recording stub, and POSTed a real `/v1/chat/completions` request with a large user message so the compression decision fires. Inspected the kwargs the handler actually passed to `apply()`. - Observed result: before the fix the recorded `apply()` kwargs were `{biases, compression_policy, messages, model, model_limit, context, frozen_message_count}` — no profile knobs. After the fix the same call also carries `compress_user_messages=True`, `compress_system_messages=True`, `target_ratio=0.10`, `min_tokens_to_compress=120` (the agent-90 profile), matching the issue's "Expected". - Not tested: did not stand up a real OpenAI/OpenCode upstream end-to-end (no live key in this environment); the upstream is mocked and the assertion is on the kwargs the proxy threads into the compression pipeline, which is exactly what the bug was about. Did not run the full `mypy headroom` pass (two-line kwarg addition, no new types). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Two-line change plus comments; no new dependencies. Reuses the existing `proxy_pipeline_kwargs` helper, so behavior is consistent across Anthropic, the OpenAI compress endpoint, and now the OpenAI chat path. - @chopratejas flagging you for review — this aligns the OpenAI chat path with the savings-profile handling the other providers already had. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Tejas Chopra
|
48201345be
|
fix(proxy): keep cache_control bounded + stable so the freeze overlay stops busting (#1852)
Follow-up to #1850. Two residual cache-bust sources, both `cache_control`-related: 1. **Guard too strict.** `overlay_cached_prefix` decided "is this turn an append-only extension?" by comparing whole message dicts — including `cache_control`. Clients (Claude Code, litellm) move the cache breakpoint to the newest message every call, so a marker landing in the frozen prefix made the guard fail, the overlay skip its replay, and the raw freeze forward ORIGINAL bytes over the cached COMPRESSED prefix → partial bust (the ~42% residual on the a10 run, `prefix_change=0`). Fix: run the append-only guard on **content only** (strip `cache_control` before comparing) — content is what the provider's cache keys on. 2. **Marker accumulation.** The overlay replays the markers that rode on each turn's then-newest message, so `cache_control` blocks pile up ~1/turn; Anthropic hard-errors at >4 total. Fix: `normalize_message_cache_control` strips every message-level marker and re-places a single ephemeral breakpoint on the last block (one breakpoint caches the whole prefix; cache is content-keyed so re-placing never busts). Wired into the Anthropic handler after the overlay. **Per-provider (deliberately scoped):** - **Anthropic**: `cache_control` markers → both fixes apply. - **OpenAI**: AUTOMATIC prefix caching, no markers → overlay (byte-identity) only; normalize is NOT applied (Anthropic markers on an OpenAI request would be wrong). - **Bedrock**: serves Claude via the pipeline but has no cachePoint/freeze-replay path → not affected; a cachePoint analog would be needed if caching is expanded. - **Gemini**: explicit Cache API (`cachedContent`), no inline markers/freeze → N/A. > Stacked on #1850 — review that first; the diff against `main` includes its overlay + `has_new_ccr_markers` work. ## Description Keeps the freeze overlay's cache-safety intact against real clients that relocate the `cache_control` breakpoint each turn, and prevents `cache_control` blocks from accumulating past Anthropic's 4-marker limit. See the two fixes above. Closes #<!-- none --> — follow-up to #1850 (no separate issue). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cache/prefix_tracker.py`: append-only guard in `overlay_cached_prefix` now compares **content only** (ignores `cache_control`); new `normalize_message_cache_control()` collapses message-level markers to a single ephemeral breakpoint on the last block. - `headroom/proxy/handlers/anthropic.py`: apply `normalize_message_cache_control` after the overlay (Anthropic only). - `tests/test_cache_control_move_bust.py`: reproduces the moved-marker bust + proves both fixes. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (local, see below) ### Test Output ```text $ pytest tests/test_cache_control_move_bust.py -q ....... [100%] 7 passed in 0.19s # broader cache-safety suite (overlay + cross-turn + CCR deferred + openai/anthropic cache-stability + helpers) $ pytest tests/test_cache_control_move_bust.py tests/test_cache_prefix_overlay.py \ tests/test_cross_turn_cache_safety.py tests/test_proxy/test_anthropic_ccr_deferred_injection.py \ tests/test_proxy_handler_helpers.py tests/test_proxy_openai_cache_stability.py \ tests/test_proxy_anthropic_cache_stability.py -q 91 passed, 2 warnings in 29.98s $ ruff check . # ruff 0.15.17 (CI-pinned) All checks passed! $ ruff format --check . # ruff 0.15.17 1057 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found (changed modules: prefix_tracker, anthropic, openai, helpers) ``` ## Real Behavior Proof - **Environment:** local (`.venv`, Python 3.12), ruff 0.15.17 / mypy pinned to CI versions. - **Exact command / steps:** `tests/test_cache_control_move_bust.py` drives the REAL tracker + freeze + `overlay_cached_prefix` + `normalize_message_cache_control` across multiple append-only turns where the client moves the `cache_control` breakpoint each turn. - **Observed result:** with a moved marker in the frozen prefix, the content-only guard keeps the overlay replaying (forwarded prefix stays byte-identical → no bust); `cache_control` blocks stay ≤4 across many turns and content is never altered. The reproduction test fails without the fix and passes with it. - **Not tested (this PR):** the end-to-end a10 SWE-bench run is the field observation motivating fix #1 (~42% residual, `prefix_change=0`); not re-run here. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes Stacked on #1850; land that first. Docs/CHANGELOG untouched (behavioral cache-safety fix; no user-facing surface change). N/A: no screenshots (no UI). |
||
|
Gaurav Dubey
|
5d14080c94
|
fix(proxy): retry passthrough on transient upstream connection close (#1513)
## Description `GET /v1/models` (and other buffered passthrough routes) returned an opaque HTTP **502** when an OpenAI-compatible upstream closed a pooled keep-alive connection mid-response, surfacing `httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)`. The same upstream answers a direct `curl` with 200 because curl opens a fresh connection per call, while Headroom reuses pooled keep-alive connections — so the first request issued on a stale connection fails even though the upstream is healthy. The fix makes the buffered passthrough path retry once on a fresh connection (exactly what curl does), and return a clear error only if the upstream is genuinely sending an incomplete response. Closes #1112 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom.proxy.helpers.request_with_transient_retry(client, *, request_id=None, max_retries=1, **request_kwargs)`: issues a buffered httpx request and retries on a **fresh connection** when (and only when) `httpx.RemoteProtocolError` is raised. Every other exception (`ConnectError`, timeouts, status errors) propagates immediately, so existing handling is unchanged. Documented as buffered-only (a streamed response can't be safely replayed once bytes reach the client). - Route `OpenAIHandlerMixin.handle_passthrough` through the helper, and add an `except httpx.RemoteProtocolError` arm that returns a clear `502` with error type `upstream_protocol_error` when the protocol error persists across the retry (instead of letting the raw error surface as an opaque/unhandled 502). - Add `tests/test_proxy_passthrough_transient_retry.py` (helper unit tests + handler-level tests covering the exact issue path). - Add a `CHANGELOG.md` entry under `Unreleased → Fixed`. Scope note: streaming `/v1/responses` is intentionally **out of scope** for this change — a streamed response cannot be safely retried after the first byte has been delivered to the client. The helper is written reusable so a streaming-aware follow-up can build on it. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/helpers.py headroom/proxy/handlers/openai.py tests/test_proxy_passthrough_transient_retry.py All checks passed! $ mypy headroom/proxy/helpers.py --ignore-missing-imports Success: no issues found in 1 source file $ pytest tests/test_proxy_passthrough_transient_retry.py -q tests/test_proxy_passthrough_transient_retry.py ....... [100%] 7 passed in 0.27s # no regressions in the surrounding passthrough/handler suites: $ pytest tests/test_proxy_passthrough_transient_retry.py tests/test_proxy_handler_helpers.py \ tests/test_proxy_byte_faithful_forwarding.py \ tests/test_proxy/test_compression_failure_action.py tests/test_proxy_copilot_auth_hooks.py -q 80 passed, 1 warning in 6.88s ``` ## Real Behavior Proof Reproduced against a **real local TCP server** (no mocks) that speaks HTTP/1.1 and, when armed, emits a chunked body then closes the socket **without** the terminating `0\r\n\r\n` — the exact condition that makes httpx raise the `incomplete chunked read` error from this issue. - Environment: macOS arm64, Python 3.12, httpx 0.28.1 (same httpx major as the report), real loopback sockets via `asyncio.start_server`. - Exact command / steps: start the local server; (1) issue a single buffered request — the pre-fix `handle_passthrough` behaviour; (2) issue the same request through `request_with_transient_retry` — the fix. Verbatim: `python repro_1112.py`. - Observed result: BEFORE the fix a single request raises `httpx.RemoteProtocolError` ("incomplete chunked read") which `handle_passthrough` surfaced as an opaque HTTP 502; AFTER the fix the same request returns **HTTP 200** (the retry opened a fresh connection, mirroring a direct `curl`). Full terminal output: ```text upstream listening on http://127.0.0.1:62374/v1/models BEFORE (single buffered request, pre-fix behaviour): raised httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) -> handle_passthrough surfaced this as an opaque HTTP 502 AFTER (request_with_transient_retry, the fix): HTTP 200 body={"object":"list","data":[]} -> first attempt hit the incomplete chunked read, retry on a fresh connection returned 200 (mirrors a direct curl) ``` The log line `Upstream closed connection mid-response (...incomplete chunked read); retrying on a fresh connection (attempt 1/1)` fires on the recovered request, confirming the retry path is what produced the 200. - Not tested: real third-party upstreams (LiteLLM/vLLM/etc.) — the local server reproduces the precise httpx error deterministically; the streaming `/v1/responses` path is intentionally out of scope (a streamed response cannot be safely retried after the first byte reaches the client). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - No new dependencies (httpx is already a proxy dependency), so no supply-chain justification is required. - The retry is deliberately narrow: only `httpx.RemoteProtocolError` is retried, capped at one retry, so a genuinely-down upstream still fails fast via the existing `ConnectError`/timeout path. - "Documentation" checklist item refers to the `CHANGELOG.md` entry; no user-facing docs pages needed for this internal resilience fix. |
||
|
Rob Francis
|
32ce99e4b4
|
fix(build): enable Intel macOS pip installs via ort-load-dynamic (#1538)
## Description
Fix `pip install headroom-ai` on Intel Mac (`x86_64-apple-darwin`).
Source installs failed because `ort-sys 2.0.0-rc.12` (transitive via
`fastembed`) does not ship prebuilt ONNX Runtime binaries for that
target, causing maturin/cargo to exit during the wheel build.
This PR mirrors the existing Windows fix: build the Rust core with
`ort-load-dynamic`, pin `ORT_DYLIB_PATH` to the pip `onnxruntime` native
library at import time, and publish Intel macOS wheels from CI.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `crates/headroom-core/Cargo.toml`: use `ort-load-dynamic` for
`x86_64-apple-darwin` instead of `ort-download-binaries-rustls-tls`.
- `headroom/_ort.py`: extend the ORT dylib pin hook to Intel macOS
(`darwin` + `x86_64`), resolving `libonnxruntime*.dylib` from the pip
`onnxruntime` package.
- `.github/workflows/release.yml` and `.github/workflows/rust.yml`: add
`macos-15-intel` / `x86_64-apple-darwin` wheel matrix entries.
- `tests/test_release_workflows.py` and
`tests/test_transforms/test_ort_dylib.py`: update/add coverage for the
new target and dylib pin behavior.
- `README.md`: note that prebuilt wheels are published for Intel macOS.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_transforms/test_ort_dylib.py \
tests/test_release_workflows.py::test_fastembed_uses_dynamic_ort_on_windows \
tests/test_release_workflows.py::test_build_wheels_matrix_includes_intel_macos_with_dynamic_ort -q
.......................... [100%]
10 passed in 0.18s
$ maturin build --release -o /tmp/headroom-dist
📦 Built wheel for abi3 Python ≥ 3.10 to /tmp/headroom-dist/headroom_ai-0.27.0-cp310-abi3-macosx_10_12_x86_64.whl
$ python3.11 -m venv /tmp/hr-venv && /tmp/hr-venv/bin/pip install .
Successfully built headroom-ai
Successfully installed headroom-ai-0.27.0
$ cd /tmp && /tmp/hr-venv/bin/python -c "import headroom; import headroom._core; print('ok')"
version 0.27.0
_core ok
```
## Real Behavior Proof
- Environment: macOS `x86_64-apple-darwin`, Python 3.11.5, Rust 1.95.0
- Exact command / steps: Reproduced the reported failure with `pip
install headroom-ai` (sdist build dies in `ort-sys` for
`x86_64-apple-darwin`); after this patch ran `maturin build --release`,
then `pip install .` in a clean venv, then `python -c "import
headroom._core"`.
- Observed result: Before fix, cargo/maturin exit 101 on missing ORT
prebuilts; after fix, wheel build succeeds and `headroom._core` imports
cleanly (`version 0.27.0`, `_core ok`).
- Not tested: `macos-15-intel` GitHub Actions wheel matrix row (will be
validated by CI after merge).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- ML features (magika detection, fastembed embeddings) still require
`onnxruntime` at runtime on Intel Mac. Users should install
`headroom-ai[proxy]` or `pip install onnxruntime`; `_ort.py` auto-pins
`ORT_DYLIB_PATH` when that package is present.
- Apple Silicon (`aarch64-apple-darwin`) behavior is unchanged: it
continues to bundle ORT via `ort-download-binaries-rustls-tls`.
- Lint/mypy not re-run locally in this pass; targeted pytest +
maturin/pip install proof covers the changed surface.
---------
Co-authored-by: Bor <you@example.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
||
|
Tejas Chopra
|
248ae0f3e0
|
fix(proxy): freeze must forward cached (compressed) prefix byte-identical — stop token-mode cache busting (#1850)
The freeze path (both providers) emits the agent's ORIGINAL bytes for a frozen message, but the provider cached whatever we FORWARDED last turn (the compressed form). Forwarding original then mismatches the cached prefix and busts it from that point — re-creating the whole suffix. Measured on a real SWE-bench run: 100% of attributed misses were prefix_change, ~56% of ALL cache-writes were bust-induced (2.8M tokens), driving cache_create +150% and cost +41% vs baseline. Cache mode already avoided this via _extract_cache_stable_delta (replay the previously-forwarded prefix, compress only the delta). Token mode called apply(frozen_count) directly, which forwards original for the frozen region. Fix: add a shared, provider-agnostic overlay_cached_prefix() that replays the previously-forwarded (cached, compressed) prefix byte-identical, append-only guarded and idempotent, and apply it in BOTH the Anthropic and OpenAI handlers right before forwarding. This makes freezing byte-identical in every mode, so the only remaining difference between "token" and "cache" mode is how large a mutable (still-compressible) tail each leaves — not whether the frozen prefix busts the cache. Tests: - test_cache_prefix_overlay.py: the helper (replay, append-only guard, idempotence). - test_cross_turn_cache_safety.py: the invariant that was missing — drive the REAL tracker + freeze + overlay over multiple append-only turns against a simulated provider prefix cache and assert the forwarded prefix stays byte-identical turn-over-turn. Load-bearing: it fails (detects the bust) without the overlay. ## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> |
||
|
Tejas Chopra
|
7208792ee8
|
Fix formatting in README.md | ||
|
Tejas Chopra
|
480d22e6e2
|
Update token reduction statistics in README | ||
|
Parideboy
|
84509a4b89
|
fix: detect and clear stale ANTHROPIC_BASE_URL from crashed wrap sessions (#1768) (#1837)
## Description `headroom wrap claude` writes `env.ANTHROPIC_BASE_URL` (or the foundry/vertex variant) into a project's `.claude/settings.local.json` so daemon-spawned Claude Code workers route through the local Headroom proxy. Removal only happened in the wrap process's `finally:` block. An unclean exit — `SIGKILL`, OOM, reboot, or terminal/tmux close (`SIGHUP`, which was not caught; only `SIGINT`/`SIGTERM` were) — skipped that cleanup, so the entry persisted indefinitely. Every subsequent bare `claude` in that project then routed to the dead port and hung indefinitely retrying it. Closes #1768 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `_write_claude_wrap_base_url` now optionally stamps a sidecar marker (`.claude/.headroom_wrap_marker.json`) recording the writer's pid/identity, the port, and the true prior value — kept out of `settings.local.json` itself so Headroom bookkeeping never shows up as a stray key in a file Claude Code's own config loader parses. - A shared `_identity_mismatch` helper (factored out of the existing `_marker_pid_reused` proxy-client-refcounting logic) lets a marker be judged stale: missing/invalid pid, dead pid, or a live pid whose identity doesn't match the recorded one (PID reuse after a crash). - `claude()` now checks for — and self-heals — a stale marker immediately before writing a fresh entry, restoring the recorded prior value instead of trusting a leftover from a dead session. - `claude()` now also registers a `SIGHUP` handler (guarded via `hasattr`, since Windows has none) alongside the existing `SIGTERM` handler, so terminal-close triggers the same cleanup/restore path. - `headroom unwrap claude` now reads the marker's recorded prior value before restoring, instead of unconditionally deleting the key — so a user's own pre-existing `ANTHROPIC_BASE_URL` (set before ever running `wrap`) isn't blindly wiped. - `headroom doctor` gained a new check (`check_wrap_marker_staleness`) that flags a stale project-local marker and points at `headroom unwrap claude` to clean it up — separate from the existing global-settings `check_claude_routing` check. - (Unrelated, pre-existing on `main`) reformatted `headroom/proxy/handlers/openai.py`, `tests/test_openai_codex_ws_lifecycle.py`, `tests/test_output_shaper.py` — whitespace/indentation only, no logic change — since they were already failing `ruff format --check .` on `main` before this branch touched anything, and the repo-wide lint gate blocks on it. Out of scope: `wrap --worktree` — no such flag or multi-worktree `.claude` handling exists anywhere in `wrap.py` today; not adding new surface for an aspirational scenario the issue mentions but that isn't implemented. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_cli/test_wrap_claude_base_url.py tests/test_cli/test_unwrap_claude.py tests/test_cli/test_wrap_stale_marker.py -q 42 passed $ pytest tests/test_cli -q 512 passed, 1 failed (test_wrap_codex_prepare_only_registers_serena_when_uvx_exists — confirmed to fail identically on a clean checkout of main with no changes applied; test-order flake, unrelated to this PR) $ ruff check . All checks passed! $ ruff format --check . 1047 files already formatted $ mypy headroom/cli/wrap.py headroom/cli/doctor.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - Environment: local checkout, Python 3.13, Windows. - Exact command / steps: wrote a base_url entry + marker via `_write_claude_wrap_base_url(..., port=8787)`, then overwrote the marker's recorded pid with a value guaranteed not to be a live process (simulating the crash from the issue's own repro: `headroom wrap claude -- -p ok & ; kill -9 <wrap-pid>`). Ran `headroom.cli.doctor.check_wrap_marker_staleness()` against that path, then called `_check_and_clear_stale_wrap_marker()` (the same check `claude()` now runs before writing a fresh entry). - Observed result: `doctor`'s check correctly reports `WARN` naming the dead pid/port and pointing at `headroom unwrap claude`. The stale-check call then self-heals: in the "nothing existed before wrap" case the leaked entry is removed; in a second run seeded with a real pre-existing `ANTHROPIC_BASE_URL` (set before `wrap` ever ran), that original value is recovered instead of being deleted. In both cases the marker file is cleared afterward. - Not tested: actual OS-level signal delivery (`kill -HUP` against a real running `headroom wrap claude` subprocess) — the SIGHUP registration is exercised via a source-inspection test instead of a live signal, since spawning/killing the real CLI subprocess isn't practical in this environment; verified E2E via CI's `wrap-native` jobs (Ubuntu/macOS) which passed. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/backend fix, no UI surface. ## Additional Notes - Documentation checklist item left unchecked: no user-facing docs currently describe wrap's settings.local.json write/cleanup behavior in enough detail to need updating; happy to add a troubleshooting note if maintainers want one. - `wrap --worktree` handling is out of scope (see Changes Made) — flagging in case maintainers want it tracked as a separate follow-up issue. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Vinay Gupta
|
5e29c06aaf
|
fix(docker): persist headroom workspace in compose (#1839)
## Description Pin the top-level Docker Compose proxy service to Headroom's canonical writable workspace under the existing `headroom_workspace` named volume. Closes #1835 The dashboard's durable savings/history data is loaded from `proxy_savings.json` via `HEADROOM_WORKSPACE_DIR`; logs, session stats, TOIN, config, and default workspace state are also derived from that root. The top-level compose file already mounted `/home/nonroot/.headroom`, but it relied on image/user home resolution instead of exporting the canonical workspace env. This makes the official compose contract explicit and matches the Docker-native compose/runtime path behavior. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Set `HOME=/home/nonroot` for the top-level compose proxy service. - Set `HEADROOM_WORKSPACE_DIR=/home/nonroot/.headroom` and `HEADROOM_CONFIG_DIR=/home/nonroot/.headroom/config` so dashboard savings/history, logs, config, memory state, session stats, and TOIN resolve into the persisted named volume. - Added a regression test that locks the top-level compose persistence wiring. ## Testing - [x] Unit tests pass (`pytest`) — focused local tests and full CI test matrix passed - [x] Linting passes (`ruff check .`) — local Ruff and CI lint passed - [x] Type checking passes (`mypy headroom`) — local mypy and CI lint passed - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ rtk pytest tests/test_docker_compose_persistence.py Pytest: 1 passed $ rtk pytest tests/test_docker_compose_persistence.py tests/test_paths.py Pytest: 76 passed $ rtk uvx ruff check tests/test_docker_compose_persistence.py All checks passed! $ rtk docker compose config services: headroom-proxy: environment: HEADROOM_CONFIG_DIR: /home/nonroot/.headroom/config HEADROOM_HOST: 0.0.0.0 HEADROOM_WORKSPACE_DIR: /home/nonroot/.headroom HOME: /home/nonroot volumes: - type: volume source: headroom_workspace target: /home/nonroot/.headroom ``` Attempted broader proxy stats-history coverage, but this local checkout does not have the native extension built: ```text $ rtk pytest tests/test_docker_compose_persistence.py tests/test_paths.py tests/test_proxy_savings_history.py::test_stats_history_persists_across_restarts_and_stats_stays_compatible ModuleNotFoundError: No module named 'headroom._core' ``` Attempted project-managed Ruff, but `uv run` tried to build the editable package first and hit the known local native build issue before Ruff could execute: ```text $ rtk uv run ruff check tests/test_docker_compose_persistence.py error: failed to run custom build command for `esaxx-rs v0.1.10` fatal error: 'cstdint' file not found ``` ## Real Behavior Proof - Environment: local clean clone at current upstream `main`, branch `fix/1835-docker-compose-persistence`. - Exact command / steps: `rtk docker compose config` from the repo root. - Observed result: Compose renders `HOME`, `HEADROOM_WORKSPACE_DIR`, and `HEADROOM_CONFIG_DIR` under `/home/nonroot/.headroom`, and the `headroom_workspace` named volume targets that same path. - Not tested: full Docker image build or live `docker compose up` restart cycle; full pytest/mypy not run locally because this checkout lacks the built `headroom._core` extension. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - All non-skipped GitHub Actions checks are green after the rebase onto `main`; skipped jobs are path-gated. - The dashboard's recent request table is still an in-memory tail and is expected to be empty after a proxy restart. This PR targets durable dashboard savings/history and other workspace-backed files. - `HEADROOM_LOG_FILE=/home/nonroot/.headroom/requests.jsonl` remains an optional operator setting; persisted request JSONL is not replayed into the dashboard after restart. - The docs/CHANGELOG checklist items are N/A for this narrow compose configuration fix. |
||
|
Vinay Gupta
|
e22d7453d4
|
fix(proxy): strip 1m model suffix before upstream forwarding (#1840)
## Description Strips dangling terminal-style model suffixes like `[1m]` from Anthropic-compatible model ids before Headroom forwards `/v1/messages` upstream. Closes #1812 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Generalized `sanitize_anthropic_model_id()` so the existing dangling ANSI-style suffix cleanup applies to Anthropic-compatible non-Claude models, including `glm-5.2[1m]`. - Added a provider-level regression for `glm-5.2[1m] -> glm-5.2`. - Added a `/v1/messages` handler regression that captures the upstream request body and verifies Headroom forwards `glm-5.2`, not `glm-5.2[1m]`. ## Testing - [x] Unit tests pass (`pytest`) — focused local tests and full CI test matrix passed - [x] Linting passes (`ruff check .`) — local Ruff and CI lint passed - [x] Type checking passes (`mypy headroom`) — local mypy and CI lint passed - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ rtk proxy env HEADROOM_REQUIRE_RUST_CORE=false PYTHONPATH=/Users/vinaygupta/Desktop/git/headroom-fix-1812-1m-model-suffix /tmp/headroom-1812-testenv/bin/python -c '<inject local headroom._core test stub; pytest.main(["tests/test_providers/test_anthropic.py", "tests/test_proxy_anthropic_model_sanitization.py"])>' ============================= test session starts ============================== platform darwin -- Python 3.13.11, pytest-9.1.1, pluggy-1.6.0 -- /private/tmp/headroom-1812-testenv/bin/python collected 17 items tests/test_providers/test_anthropic.py::TestAnthropicModelSanitization::test_sanitize_model_id_removes_ansi_escape_sequences PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelSanitization::test_sanitize_model_id_removes_displayed_style_suffix PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelSanitization::test_sanitize_model_metadata_cleans_nested_model_ids PASSED tests/test_providers/test_anthropic.py::TestAnthropicTokenCounting::test_count_text_fallback PASSED tests/test_providers/test_anthropic.py::TestAnthropicTokenCounting::test_count_messages_basic PASSED tests/test_providers/test_anthropic.py::TestAnthropicTokenCounting::test_count_text_allows_literal_special_tokens PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelLimits::test_get_context_limit_claude_sonnet PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelLimits::test_get_context_limit_claude_opus PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelLimits::test_get_context_limit_strips_ansi_model_suffix PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelLimits::test_get_context_limit_claude_5_family PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelLimits::test_supports_model_known PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelLimits::test_supports_model_prefix PASSED tests/test_providers/test_anthropic.py::TestAnthropicModelLimits::test_token_counter_cache_uses_sanitized_model_id PASSED tests/test_providers/test_anthropic.py::TestAnthropicCostEstimation::test_estimate_cost_basic PASSED tests/test_providers/test_anthropic.py::TestAnthropicCostEstimation::test_pricing_lookup_strips_ansi_model_suffix PASSED tests/test_providers/test_anthropic.py::TestAnthropicCostEstimation::test_pricing_claude_5_family PASSED tests/test_proxy_anthropic_model_sanitization.py::test_anthropic_messages_strips_local_1m_model_suffix_before_forwarding PASSED ======================== 17 passed, 3 warnings in 2.11s ======================== $ rtk uvx ruff check headroom/providers/anthropic.py tests/test_providers/test_anthropic.py tests/test_proxy_anthropic_model_sanitization.py All checks passed! $ rtk uvx ruff format --check headroom/providers/anthropic.py tests/test_providers/test_anthropic.py tests/test_proxy_anthropic_model_sanitization.py 3 files already formatted ``` The normal editable test command was attempted but did not reach test execution in this local checkout because the native extension build failed: ```text $ rtk uv run pytest tests/test_providers/test_anthropic.py tests/test_proxy_anthropic_model_sanitization.py × Failed to build `headroom-ai @ file:///Users/vinaygupta/Desktop/git/headroom-fix-1812-1m-model-suffix` warning: esaxx-rs@0.1.10: src/esaxx.cpp:620:10: fatal error: 'cstdint' file not found error: failed to run custom build command for `esaxx-rs v0.1.10` ``` ## Real Behavior Proof - Environment: local macOS worktree from current upstream `main`; Python 3.13.11 throwaway test environment; `HEADROOM_REQUIRE_RUST_CORE=false`; in-memory `headroom._core` stub used only to avoid the local missing native extension during Python-level tests. - Exact command / steps: POST a TestClient `/v1/messages` request with `{"model": "glm-5.2[1m]", ...}` and replace `_retry_request` with a test double that records the upstream body. - Observed result: the recorded upstream request body contains `{"model": "glm-5.2"}` and `mutation_reasons == ["sanitize_model_id"]`, so the mutated JSON body is serialized instead of forwarding the original bytes. - Not tested: live Z.AI credentials/provider call; full local pytest; local `mypy headroom`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - All non-skipped GitHub Actions checks are green after the rebase onto `main`; skipped jobs are path-gated. - No code comments were added because the fix reuses the existing sanitizer and mutation-tracking path. - Documentation and CHANGELOG updates are N/A for this narrow proxy compatibility fix. - The local pytest warnings were from the throwaway environment/test tooling (`asyncio_mode`, Starlette TestClient deprecation, and the existing AnthropicProvider no-client warning), not from the changed code path. |
||
|
Tejas Chopra
|
60af15f96f
|
feat(content-router): lossless-first dispatch, cross-turn dedup, and A7 lossy-after-fold (#1818)
## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Rod Boev
|
da2d8dc9db
|
fix(proxy): cancel retry backoff on shutdown (#1834)
## Description During proxy shutdown, an in-flight retrying request can currently stay asleep inside `_retry_request()` and keep the client socket hanging until the retry timer expires or an external supervisor kills the process. This wires retry backoff to a proxy-scoped shutdown event so shutdown interrupts those waits immediately and returns a clear `503` response instead of leaving the request stalled. Closes #1821. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a proxy-scoped shutdown event in `headroom/proxy/server.py`. - Cleared that event at startup and set it at shutdown before teardown proceeds. - Replaced both retry-backoff sleeps with a helper that wakes on either timeout or shutdown. - Returned a shutdown `503` with `retry-after: 0` when shutdown interrupts retry backoff. - Stopped the shutdown interruption logs from falling back to the raw upstream URL when no safe path string is available. - Added focused regressions for retry-backoff interruption and shutdown event signaling. - Updated the existing Retry-After tests to observe the new shutdown-aware wait helper instead of the old raw sleep hook. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy_handler_helpers.py tests/test_proxy_pipeline_lifecycle.py -q`) - [x] Unit tests pass (`uv run pytest tests/test_proxy_retry_429.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/server.py tests/test_proxy_handler_helpers.py tests/test_proxy_retry_429.py tests/test_proxy_pipeline_lifecycle.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy_handler_helpers.py tests/test_proxy_pipeline_lifecycle.py -q 32 passed, 1 warning in 13.05s uv run pytest tests/test_proxy_retry_429.py -q 10 passed, 1 warning in 1.12s uv run ruff check headroom/proxy/server.py tests/test_proxy_handler_helpers.py tests/test_proxy_retry_429.py tests/test_proxy_pipeline_lifecycle.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, project `uv` environment, focused proxy retry and shutdown regressions. - Exact command / steps: copy the updated shutdown regression files into a detached `origin/main` worktree and run `tests/test_proxy_handler_helpers.py` plus `tests/test_proxy_pipeline_lifecycle.py`, then rerun those files on this branch and separately rerun `tests/test_proxy_retry_429.py` after updating the existing Retry-After tests to patch the shutdown-aware wait helper. - Observed result: base fails because retry backoff still returns the original `429` and `shutdown()` leaves the retry event unset; head passes the focused file, preserves the existing Retry-After assertions, and returns a shutdown `503` with `retry-after: 0` while signaling retry waiters during shutdown. - Not tested: live systemd-managed shutdown on Linux or a full VS Code / Claude Code session. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes This is intentionally scoped to retry backoff during shutdown. It does not try to cancel unrelated in-flight request work or change the broader retry policy outside shutdown. |
||
|
Rod Boev
|
afd9cbdfaf
|
fix(copilot): normalize subscription routing host (#1836)
## Description `headroom wrap copilot --subscription` can currently trust the token-exchange host for individual Copilot seats, which routes newer responses-API models like `gpt-5.4` to `api.individual.githubcopilot.com` and reproduces the transient `502` retry loop from issue #1694. This normalizes that public individual-seat host back to the generic Copilot API host while preserving dedicated business or explicitly pinned hosts. Closes #1694. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Normalized exchanged Copilot subscription hosts through the existing public-host classifier instead of trusting the raw token-exchange payload. - Added a regression proving `api.individual.githubcopilot.com` downgrades to `https://api.githubcopilot.com` for subscription routing. - Added a wrap-level regression proving subscription launches export the normalized host into the proxy env. - Preserved business-host and explicit `GITHUB_COPILOT_API_URL` routing behavior. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_copilot_auth.py -q`) - [x] Unit tests pass (`uv run pytest tests/test_cli/test_wrap_copilot.py -q`) - [x] Linting passes (`uv run ruff check headroom/copilot_auth.py tests/test_copilot_auth.py tests/test_cli/test_wrap_copilot.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_copilot_auth.py -q 57 passed, 1 warning in 0.34s uv run pytest tests/test_cli/test_wrap_copilot.py -q 31 passed, 1 warning in 0.32s uv run ruff check headroom/copilot_auth.py tests/test_copilot_auth.py tests/test_cli/test_wrap_copilot.py All checks passed! uv run ruff format --check headroom/copilot_auth.py tests/test_copilot_auth.py tests/test_cli/test_wrap_copilot.py 3 files already formatted ``` ## Real Behavior Proof - Environment: Windows, Python `uv` environment, mocked Copilot token-exchange and wrap launch surfaces. - Exact command / steps: run the focused Copilot auth and wrap regression tests after teaching subscription token-exchange routing to normalize the public individual-seat host. - Observed result: exchanged subscription tokens that advertise `https://api.individual.githubcopilot.com` now route through `https://api.githubcopilot.com`, while business-host and explicit-host pin cases stay unchanged. - Not tested: a live GitHub Copilot subscription request against the upstream service. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes This is intentionally scoped to host selection for exchanged Copilot subscription tokens. It does not change token discovery, token pinning, or non-subscription OAuth routing. |
||
|
Rudimar Ronsoni
|
4bd3ddfaa5
|
fix(opencode): use local MCP config (#1383)
## Description Fixes OpenCode Headroom MCP configuration across wrap, MCP install/status/uninstall, and persistent install docs/CLI. OpenCode was being configured to use a remote HTTP MCP endpoint at `/mcp`, but the Headroom proxy does not expose MCP there. The correct OpenCode configuration is a local stdio MCP server that runs `headroom mcp serve`. Closes #1380 ## Type of Change - [x] Bug fix - [ ] New feature - [ ] Breaking change - [x] Documentation update - [x] Tests ## Changes Made - Changed OpenCode MCP registration to emit `type: "local"` with `command: ["headroom", "mcp", "serve"]`. - Changed OpenCode MCP environment serialization from `env` to OpenCode's `environment` key, while still reading legacy `env` entries. - Removed generated remote `/mcp` entries from OpenCode wrap/runtime config. - Made `wrap opencode --no-mcp` skip persistent `mcp.headroom` injection. - Kept provider-only OpenCode config injection from writing MCP; MCP persistence is owned by the registrar path. - Made `headroom mcp status` and `headroom mcp uninstall` use the registrar lifecycle so OpenCode is covered. - Added `opencode` to persistent install `--target` choices. - Clarified OpenCode persistent install docs to use `--scope provider` for direct `opencode.json` edits. - Added regression coverage for registrar serialization, wrap behavior, runtime config, provider-scope install, MCP CLI lifecycle, and install target parsing. ## Testing - [x] `rtk .venv/bin/python -m pytest tests/test_mcp_registry tests/test_cli/test_mcp.py tests/test_cli/test_wrap_opencode.py tests/test_providers_opencode_config.py tests/test_providers_opencode_install.py tests/test_install -q` - [x] Result after absorbing #1381 overlap: `263 passed, 1 skipped` - [x] Targeted Ruff check passed for the changed Python/test files. - [x] Targeted Ruff format check passed for the changed Python/test files. - [x] Isolated HOME smoke tests with real `opencode mcp list --pure`. ## Real Behavior Proof - `headroom mcp install --agent opencode --proxy-url http://127.0.0.1:9000 --force` against an isolated HOME wrote a valid local OpenCode MCP entry with `environment.HEADROOM_PROXY_URL`. - `opencode mcp list --pure` against that isolated HOME connected to `headroom mcp serve`. - `headroom wrap opencode --prepare-only --no-rtk --no-serena --port 9001` wrote local MCP plus provider config. - `headroom wrap opencode --prepare-only --no-rtk --no-serena --no-mcp --port 9002` wrote provider config without `mcp.headroom`. - Generated runtime `OPENCODE_CONFIG_CONTENT` was accepted by `opencode mcp list --pure`; `include_mcp=False` reported no MCP servers. - `headroom mcp status` detected the isolated OpenCode config and read the custom proxy URL. - `headroom mcp uninstall` removed `mcp.headroom` from the isolated OpenCode config while leaving provider config intact. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review |
||
|
Rod Boev
|
88f935a1eb
|
fix(dashboard): deduplicate repeated savings metrics (#1804)
## Description The session dashboard repeats the same savings and performance numbers in adjacent places. `proxy_compression_saved` appears in several captions and detail rows, and average overhead and TTFB appear both in the hero area and again in Performance without adding new context. This narrows the non-hero dashboard presentation so repeated session metrics have one visible home plus decomposition where it adds information. It leaves `/stats`, savings math, cache attribution, and the hero proxy savings card unchanged. Refs #960 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Removed redundant non-hero session-view captions that restated proxy-compression token counts without adding a new dimension. - Kept canonical homes for proxy compression and token usage details. - Preserved Performance range context while avoiding adjacent restatement of hero averages. - Added a static dashboard regression for repeated session metrics. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy_dashboard_stats_cache.py -q`) - [x] Linting passes (`uv run ruff check tests/test_proxy_dashboard_stats_cache.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_proxy_dashboard_stats_cache.py -q 12 passed, 1 skipped, 1 warning in 19.24s $ uv run ruff check tests/test_proxy_dashboard_stats_cache.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, Python environment from `uv sync --extra dev`, browserless dashboard HTML inspection. - Exact command / steps: load `get_dashboard_html()` in the focused dashboard stats test and assert removed duplicate captions stay removed while canonical metric owners remain present. - Observed result: session-view repeated savings and performance labels no longer duplicate the same numbers without context. - Not tested: full browser screenshot and history-view de-duplication. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes No `CHANGELOG.md` edit: this repo generates changelog entries from conventional commits. This intentionally avoids the hero proxy savings card already covered by #927 and #1649, and it does not fold provider cache discount into Headroom-value savings. |
||
|
inix
|
451b9f0867
|
perf(savings): batch tracker persistence off the request hot path (#1817)
## Description
The proxy wrote the full savings state to disk on every request: a
`json.dumps` of up to 5000 history entries plus a blocking `os.fsync`,
run under the shared metrics event loop. Concurrent sessions queued
behind whichever request was mid-save. This batches the write so the hot
path stops paying that cost every time.
Serialize is the dominant part of that cost (about 57% in measurement)
and it holds the GIL, so moving the write to a worker thread can't
overlap it with the loop, and batching only the `fsync` caps the win at
about 28%. Cutting how often the whole state is written is the lever
that helps.
Durability holds where it matters. `/stats`, `/stats-history`, and CSV
export read in-memory state, so they never go stale. The on-disk file
only feeds restart-survival: graceful shutdown flushes the tail, and a
hard crash loses at most 24 requests' lifetime delta on the proxy path.
A flush still does the durable temp-write, `fsync`, and atomic rename,
only less often.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `SavingsTracker` gains `save_flush_every` (default 1, so direct and
CLI callers keep persisting on every call). A counter throttles the
existing `_save_locked`, and `flush()` forces a write.
- The proxy constructs the tracker with `save_flush_every=25` at its one
production construction site (`prometheus_metrics.py`). Graceful
shutdown flushes the tail (`server.py`).
- Every write is a full-state snapshot, so a skipped save loses nothing:
the next write is a complete replacement. `_save_locked` resets the
throttle counter only after a durable write (and in the stateless
branch), so a transient write failure leaves the counter untouched and
the next record retries instead of waiting a fresh window.
- Tests: one existing savings test that read the on-disk file
mid-session now flushes first. New tests prove the batched final on-disk
state equals the immediate (`flush_every=1`) state on identical inputs,
that a failed `mkstemp` retries on the next record rather than consuming
a full window, and that `HeadroomProxy.shutdown()` flushes the tracker
so a graceful stop never drops the batched tail.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
# Regression across the fix's full blast radius: the touched savings suite plus
# every test exercising savings_tracker, prometheus_metrics, or the server.py
# shutdown surface (one construction site, one flush call site, confirmed repo-wide).
$ uv run pytest tests/test_proxy_savings_history.py tests/test_proxy_project_savings.py \
tests/test_backend_streaming_cache_metrics.py tests/test_pricing_litellm.py \
tests/test_proxy_cache_ttl_metrics.py tests/test_proxy_hooks_regression.py \
tests/test_compression_observability.py tests/test_observability_metrics.py \
tests/test_prometheus_stage_timing_concurrency.py tests/test_request_outcome.py \
tests/test_telemetry_context.py tests/test_provider_codex_runtime.py \
tests/test_proxy_eager_preload_bind.py tests/test_proxy_pipeline_lifecycle.py \
tests/test_proxy_scalability.py tests/test_proxy_warmup.py \
tests/test_proxy/test_bedrock_passthrough.py -q
195 passed
$ uv run ruff check .
All checks passed!
$ uv run ruff format --check .
1044 files already formatted
$ uv run mypy headroom
Success: no issues found in 406 source files
```
## Real Behavior Proof
- Environment: Python 3.13.13, macOS (Apple M4, APFS), isolated worktree
venv, `HF_HUB_OFFLINE=1 LITELLM_LOCAL_MODEL_COST_MAP=true`. Branch
`fix/savings-tracker-batch-save` at `
|
||
|
Andrew McFague
|
ebe0a3bd7b
|
feat(proxy): add provider-only HTTP proxy (#1807)
## Description Adds provider-only HTTP proxy configuration for upstream LLM calls without setting process-wide proxy environment variables. `--http-proxy` and `HEADROOM_HTTP_PROXY` are scoped to the proxy server's provider HTTPX clients, and HTTP/2 is disabled for those clients when the proxy is set so HTTPS provider APIs can tunnel through CONNECT. Using process env vars such as `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, or `NO_PROXY` would also affect HTTPX, but those vars are inherited by tool executions, so this keeps proxy routing out of the global environment. Closes: N/A ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `--http-proxy` with `HEADROOM_HTTP_PROXY` fallback. - Passed the proxy URL only into provider HTTPX clients. - Disabled provider HTTP/2 when the proxy is configured. - Preserved the new setting through direct server startup and multi-worker config serialization. - Documented the flag/env var and why global `HTTP_PROXY`-style vars are not suitable for provider-only routing. - Added an Unreleased changelog entry. - Added coverage for CLI/env wiring, worker serialization, and HTTPX client options. ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --frozen pytest tests/test_cli_proxy_improvements.py tests/test_proxy_scalability.py ============================== 72 passed in 5.95s ============================== $ uv run --frozen ruff check headroom/cli/proxy.py headroom/proxy/models.py headroom/proxy/server.py tests/test_cli_proxy_improvements.py tests/test_proxy_scalability.py All checks passed! $ uv run --frozen mypy headroom --ignore-missing-imports Success: no issues found in 406 source files $ env -u HTTP_PROXY -u http_proxy npm --prefix docs run types:check [MDX] generated files in 6.351916000000074ms Generating route types... [MDX] generated files in 5.813166999999794ms ✓ Types generated successfully $ git diff --check # no output ``` ## Real Behavior Proof - Environment: local provider setup that requires outbound LLM traffic through an HTTP proxy - Exact command / steps: ran focused pytest, Ruff, mypy, docs `types:check`, and `git diff --check` after rebasing the branch onto `origin/main`; reviewed the docs and changelog diffs; actively used the new proxy setting locally for a provider that requires proxied egress - Observed result: CLI/env/config tests passed; static checks passed; docs type generation passed; local provider traffic can be routed through the provider-only proxy setting without exporting global proxy variables to tool executions - Not tested: broad provider matrix across every supported upstream ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. CLI/backend/docs update only. ## Additional Notes The branch keeps implementation, docs, changelog, and formatting changes in separate commits. |
||
|
Rod Boev
|
838c5234a8
|
fix(transforms): normalize diff compressor context (#1801)
## Description Unified diff content could skip compression when the router reached the DIFF strategy with no question context. `DiffCompressor.compress()` defaulted omitted context to an empty string, but explicit `None` still crossed into the Rust boundary and raised before any compression result could be produced. The router also had a DEBUG-only crash path because it measured `len(context)` before DIFF dispatch. This normalizes `None` at the router entry and at the DIFF wrapper boundary so direct and routed diff compression both send a string context to Rust. Closes #1798. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Normalize `None` context to `""` before router debug logging and compression dispatch. - Normalize `None` context to `""` again before calling the Rust diff compressor. - Add regressions for explicit `None`, omitted context, non-empty context preservation, and DEBUG-enabled router DIFF dispatch. - Keep DIFF fallback behavior unchanged so patch-shaped content is not routed through a lossy fallback. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_transforms/test_diff_compressor.py tests/test_transforms/test_content_router.py -q`) - [x] Linting passes (`uv run ruff check headroom/transforms/diff_compressor.py headroom/transforms/content_router.py tests/test_transforms/test_diff_compressor.py tests/test_transforms/test_content_router.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_transforms/test_diff_compressor.py tests/test_transforms/test_content_router.py -q 86 passed in 3.09s uv run pytest tests/test_transforms/test_content_router.py -q 55 passed in 2.84s uv run ruff check headroom/transforms/diff_compressor.py headroom/transforms/content_router.py tests/test_transforms/test_diff_compressor.py tests/test_transforms/test_content_router.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, Python through the project `uv` environment. - Exact command / steps: run the new DIFF context regressions against base and head. - Observed result: base fails explicit `None` at the fake Rust boundary with `AssertionError: Rust diff compressor received None context`; head passes explicit `None`, omitted context, non-empty context, and DEBUG-enabled router dispatch. - Not tested: native Rust internals beyond the Python wrapper boundary. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes No changelog entry is needed for this narrow wrapper and router bug fix. Type checking was not part of the focused local validation for this Python-only change. |
||
|
Rod Boev
|
d24a3f8425
|
fix(proxy): bound Codex WS compression fallback latency (#1802)
## Description Codex `/v1/responses` WebSocket frames could spend the full global compression timeout before falling through unchanged, then report only a generic `compression_exception` reason. That made a recoverable timeout look like an opaque compression failure and left Codex users waiting around 30 seconds for frames that did not produce useful compression. This keeps the existing compression executor, adds a Codex WS-specific compression timeout bound, and records timeout fallback distinctly from other compression exceptions. Closes #922. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Bound Codex Responses WebSocket frame compression with a WS-specific timeout. - Pass that timeout through the existing compression executor instead of adding a parallel executor path. - Record timeout passthrough with `compression_timeout` instead of the generic compression exception reason. - Preserve generic `compression_exception` for non-timeout failures. - Add coverage for first-frame timeout bounds, timeout reason logging, generic exception preservation, and later-frame failed metrics. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_openai_codex_ws_lifecycle.py tests/test_codex_ws_compression_scheduler.py tests/test_compression_observability.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_ws_lifecycle.py tests/test_codex_ws_compression_scheduler.py tests/test_compression_observability.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_openai_codex_ws_lifecycle.py -q 22 passed in 1.38s uv run pytest tests/test_codex_ws_compression_scheduler.py tests/test_compression_observability.py -q 14 passed, 1 skipped in 3.63s uv run pytest tests/test_openai_codex_ws_lifecycle.py tests/test_codex_ws_compression_scheduler.py tests/test_compression_observability.py -q 36 passed, 1 skipped in 2.09s uv run ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_ws_lifecycle.py tests/test_codex_ws_compression_scheduler.py tests/test_compression_observability.py All checks passed! uv run ruff format headroom/proxy/handlers/openai.py tests/test_openai_codex_ws_lifecycle.py --check 2 files already formatted ``` ## Real Behavior Proof - Environment: Windows, Python through the project `uv` environment. - Exact command / steps: run the focused Codex WS timeout regressions with small monkeypatched timeout values. - Observed result: base uses the global timeout path or reports only generic `compression_exception`; head passes with Codex WS timeout bounded to the smaller WS cap, logs `compression_timeout` for timeout fallback, preserves `compression_exception` for non-timeout failures, and records failed metrics for later-frame timeout fallback. - Not tested: live Codex Desktop traffic against paid OpenAI credentials. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes No changelog entry is needed for this request-path bug fix. Type checking was not part of the focused local validation for this Python-only change. Live Codex Desktop validation is not included because the regression is covered at the handler boundary. |
||
|
Rod Boev
|
0a3851b240
|
perf(proxy): cap compression workers to CPU count (#1803)
## Description The request-path compression executor currently uses asyncio-style I/O sizing for CPU-bound Kompress work. When `compression_max_workers` is unset, `HeadroomProxy.__init__` resolves the pool to `min(32, cpu * 4)`, so an eight-core host can run 32 simultaneous compression workers that all contend for real CPU. This changes only the automatic request-path default to one worker per reported CPU while preserving the existing explicit override path from `--compression-max-workers` and `HEADROOM_COMPRESSION_MAX_WORKERS`. Closes #1635 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Cap the automatic request-path compression executor default at `max(1, os.cpu_count() or 1)`. - Preserve explicit `compression_max_workers` values, including the existing clamp to at least one worker. - Keep CLI help, `ProxyConfig` comments, and nearby test documentation aligned with the CPU-bound default. - Update the focused compression executor regression so the default contract documents CPU-bound sizing, and keep the existing Codex compression stress guard stable when p50 rounds to zero. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_codex_ws_compression_scheduler.py tests/test_proxy_compression_executor.py tests/test_cli_proxy_improvements.py::TestCompressionMaxWorkers -q`) - [x] Linting passes (`uv run ruff check headroom/cli/proxy.py headroom/proxy/models.py headroom/proxy/server.py tests/test_cli_proxy_improvements.py tests/test_proxy_compression_executor.py tests/test_codex_ws_compression_scheduler.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_codex_ws_compression_scheduler.py tests/test_proxy_compression_executor.py tests/test_cli_proxy_improvements.py::TestCompressionMaxWorkers -q 16 passed, 1 skipped, 1 warning in 6.13s $ uv run ruff check headroom/cli/proxy.py headroom/proxy/models.py headroom/proxy/server.py tests/test_cli_proxy_improvements.py tests/test_proxy_compression_executor.py tests/test_codex_ws_compression_scheduler.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, Python environment from `uv sync --extra dev`, no provider credentials needed. - Exact command / steps: construct `HeadroomProxy` with `compression_max_workers=None`, inspect `proxy.compression_max_workers` and `/health` `runtime.compression_executor`. - Observed result: the automatic request-path pool resolves to reported CPU count, while explicit overrides still resolve to the configured value and report `source: explicit`. - Not tested: multi-session wall-clock benchmark under live Kompress load. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes No `CHANGELOG.md` edit: this repo generates changelog entries from conventional commits. This intentionally does not touch the background compression executor surface covered by #1633. |
||
|
Andrew McFague
|
0b0133b7fd
|
Wire OpenAI Responses output shaping (#1438)
## Description Wire output shaping for OpenAI Responses traffic across HTTP `/v1/responses` and Codex WebSocket `response.create` frames. The change adds provider-specific shaping for `instructions`, `reasoning.effort`, and `text.verbosity` while keeping Anthropic request mutation separate. Review follow-up: merged byte-faithful `/v1/responses` forwarding from #1557 and marks shaped HTTP Responses payloads as `body_mutated=True`, so retry forwarding sends the shaped body instead of the original raw bytes. ## Type of Change - [ ] Bug fix (non-breaking change fixes an issue) - [x] New feature (non-breaking change adds functionality) - [ ] Breaking change (fix or feature would cause existing functionality change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added OpenAI Responses output shaping for `instructions`, `reasoning.effort`, and `text.verbosity`. - Wired shaping into `/v1/responses` HTTP and Codex WebSocket `response.create` paths. - Preserved `x-headroom-bypass` and `HEADROOM_OUTPUT_HOLDOUT` behavior. - Added output-shaper transform labels for verbosity, text verbosity, reasoning effort, holdout control, and strata. - Updated output-savings conversation keys for Responses payloads and WS `response.create` envelopes. - Counted WS frame payload tokens when assigning output-savings strata. - Merged byte-faithful `/v1/responses` forwarding from #1557 and kept shaped HTTP bodies on the mutated-forwarding path. - Added tests for classification, shaping, holdout, bypass, labels, WS strata, and byte-faithful forwarding compatibility. - Updated `CHANGELOG.md` for OpenAI Responses output-shaping support. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --extra dev python -m pytest tests/test_openai_codex_ws_lifecycle.py tests/test_openai_responses_output_shaper.py tests/test_codex_responses_passthrough_bytes.py tests/test_output_shaper.py tests/test_output_savings.py -q 110 passed, 1 warning in 1.49s $ uv run --extra dev ruff check headroom/proxy/handlers/openai.py tests/test_openai_responses_output_shaper.py tests/test_codex_responses_passthrough_bytes.py tests/test_openai_codex_ws_lifecycle.py tests/test_output_shaper.py tests/test_output_savings.py All checks passed! $ git diff --check No whitespace errors. ``` ## Real Behavior Proof - Environment: local macOS checkout, branch `output-shaper-openai-responses`. - Exact command / steps: ran targeted pytest, ruff, and diff checks listed above. - Observed result: targeted tests passed with an existing FastAPI TestClient deprecation warning; ruff passed; diff check passed. - Not tested: full repository test suite, live OpenAI traffic, browser dashboard rendering, full `mypy headroom`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows project's style guidelines - [x] I performed self-review of my code - [x] I commented my code, particularly in hard-to-understand areas - [x] I made corresponding changes to documentation - [x] My changes generate no new warnings - [x] I added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I updated `CHANGELOG.md` if applicable ## Screenshots N/A ## Additional Notes - Non-applicable Type Change items are left unchecked. - The pytest warning comes from `fastapi.testclient` importing Starlette TestClient and was not introduced by this change. - `CHANGELOG.md` includes entries for OpenAI Responses output-shaping support and byte-faithful `/v1/responses` forwarding compatibility. --------- Co-authored-by: obchain <riteshnikhoriya94@gmail.com> |
||
|
Parideboy
|
e8151f059b
|
fix(opencode): expose headroom/* models in injected provider config (#1716)
## Description `headroom wrap opencode` (and `headroom install opencode`) injects a `provider.headroom` block into the OpenCode config, but the block contained **no `models` map**. OpenCode only resolves `<provider>/<model>` ids that are listed in a custom provider's `models` map, so every documented `headroom/*` model (see `plugins/opencode/README.md`) failed with: ```text Error: Model not found: headroom/claude-sonnet-4-6. ``` This PR adds the model map (mirroring `DEFAULT_MODELS` in `plugins/opencode/src/provider.ts` and the README table) via a single shared `headroom_provider_entry()` helper used by all three injection sites. It also fixes a latent bug in the TS helper `createHeadroomProvider`, which prefixed model **keys** with `headroom/` — OpenCode would have registered them as `headroom/headroom/<id>`. Not addressed here (flagged for maintainers): the `headroom-opencode` npm package referenced by the plugin docs is not published to npm (registry 404), so the transparent-transport interception path (which would capture `github-copilot/*` traffic in the dashboard) still depends on a locally built `plugins/opencode/dist/entry.opencode.js`. With this fix, the documented `headroom/*` provider route works, so wrapped OpenCode traffic is proxied and recorded when users select `headroom/*` models. Closes #1657 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/providers/opencode/config.py`: added `HEADROOM_OPENCODE_MODELS` (claude-sonnet-4-6, claude-opus-4-6, claude-haiku-4-5-20251001, gpt-4o, gpt-4.1 — same names/limits as the TS plugin) and a `headroom_provider_entry(port)` helper that includes the `models` map; `_render_provider_block` and `inject_opencode_provider_config` now use it instead of duplicating the provider dict. - `headroom/providers/opencode/runtime.py`: `build_opencode_config_content` reuses `headroom_provider_entry()` so `OPENCODE_CONFIG_CONTENT` exposes the models too. - `plugins/opencode/src/provider.ts`: `createHeadroomProvider` no longer prefixes model keys with `headroom/` (OpenCode namespaces model ids by provider key; keys must be bare ids). - `tests/test_providers_opencode_config.py`: assertions that the injected provider block and `build_opencode_config_content` output contain a `models` map with bare-id keys including `claude-sonnet-4-6`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_providers_opencode_config.py -q 1 failed, rest passed — test_build_launch_env_with_project is a pre-existing Windows-only failure (json.dumps escapes backslashes in the plugin path); it fails identically on upstream/main without this change and passes on Linux. $ ruff check headroom/providers/opencode tests/test_providers_opencode_config.py All checks passed! $ ruff format --check . 5 files already formatted $ mypy headroom --ignore-missing-imports Success (notes only, no errors) $ cd plugins/opencode && npm run typecheck && npm test tsc --noEmit: OK Test Files 2 passed (2) Tests 13 passed (13) ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, Node v26.3.0, this branch with the Rust core built locally. - Exact command / steps: `python -c "from headroom.providers.opencode.runtime import build_opencode_config_content; import json; print(json.dumps(build_opencode_config_content(port=8787, include_mcp=False)['provider']['headroom'], indent=1))"` - Observed result: the generated `headroom` provider block now contains `"models"` with bare-id keys (`claude-sonnet-4-6`, `claude-opus-4-6`, `claude-haiku-4-5-20251001`, `gpt-4o`, `gpt-4.1`), each with name and context/output limits; previously the block had no `models` key, which is exactly why OpenCode returned `Model not found: headroom/claude-sonnet-4-6`. - Not tested: a live `opencode run` round-trip against a real OpenCode install (no OpenCode binary in this environment); dashboard event capture for `github-copilot/*` models via the transport plugin (blocked on the unpublished `headroom-opencode` artifact, see Description). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - Docs: `plugins/opencode/README.md` already documents these models; no doc change needed. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Tejas Chopra
|
c5493ea93b
|
fix(content-router): token-measure lossless folds at the acceptance gate (#1772)
## Description Unit-mismatch bug in the compression acceptance gate. `router.apply()` computes `compression_ratio` from `len(text.split())` (word count), but a **lossless** search/log fold (`compact_lossless`) saves **bytes** by collapsing a repeated path prefix into a single heading — word count stays flat or even *rises* (the heading adds a word). So the gate saw `ratio ≥ 1.0` and discarded every free, byte-recoverable win as `ratio_too_high`. (Raising the floor to 1.0 in #1771 did **not** fix this — the word-ratio was already ≥ 1.0.) Measure lossless results (those whose `strategy_chain` carries a `lossless_*` entry) by **byte ratio** at the gate and in the result cache — the real saving. Lossy strategies are unchanged (word count tracks their token savings), and the reversibility gate is untouched (`LOG`/`SEARCH`/`DIFF` aren't in `LOSSY_UNMARKED_STRATEGIES`). The excluded-tool and bash-search paths already bypass this gate via `continue`; this fixes the **main strategy dispatch** (the lossless-mode `LOG`/`SEARCH`/`DIFF` path). Follow-up to #1771. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - At the `apply()` acceptance gate: compute `accept_ratio` = byte ratio for lossless results (`strategy_chain` has `lossless_*`), else the existing word ratio. Gate + result-cache entry now use `accept_ratio`. - Added an end-to-end regression test that drives the full `router.apply()` path. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text tests/test_lossless_mode.py::test_router_apply_accepts_lossless_search_byte_measured PASSED tests/test_content_router_tool_role_reversibility.py .......... (10 passed) # broader (pre-move) sweep on the same change: tests/test_lossless_mode.py / test_transforms/test_content_router.py / test_lossless_excluded_compaction.py / test_bash_search_lossless_fold.py — 121 passed ruff check headroom/transforms/content_router.py -> All checks passed! mypy headroom/transforms/content_router.py -> Success: no issues found ``` ## Real Behavior Proof - Environment: local worktree, Python 3.12, `PYTHONPATH` pinned to the branch. - Exact command / steps: new regression test constructs a single-file grep result, runs it through `ContentRouter(lossless=True).apply(...)`, and asserts the tool output is byte-smaller and recovers exactly (`search_unheading(out) == original`). - Observed result: before this fix the fold was rejected (`out == original`, counted `ratio_too_high`); after, it's applied (`len(out) < len(original)`, marker-free, byte-exact recovery). The test also asserts the fold's word count is ≥ the original's, so the test is meaningless if "fixed" by word count. - Not tested: no live end-to-end proxy run; validated via the full `apply()` path in unit tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (handled at release time) ## Additional Notes Why prior tests missed it: `compress()` and `_apply_strategy_to_content` return the folded result directly and never touch the `apply()` acceptance gate, so the existing lossless-mode unit tests (which call those) passed while the real proxy path silently discarded the fold. The new test exercises `apply()` end-to-end. |
||
|
Tejas Chopra
|
6c31db97fb
|
feat(content-router): accept any real compression (remove min-savings floor) (#1771)
## Description The compression acceptance gate rejected any compression that saved less than ~15% (`min_ratio` interpolated 0.85 at low context pressure → 0.65 under pressure). That floor was a crude proxy for "big enough to justify busting the prefix cache," but it dropped genuine token savings — notably lossless code/log folds that shrink <15% (the `ratio_too_high` rejections). This makes the gate accept **any real shrink** (`ratio < 1.0`): any token saved is worth taking. The two guards that actually protect correctness are untouched: - **Reversibility gate** — lossy, unmarked tool output still stays verbatim (accuracy; #1307). - **Net-cost policy** (`HEADROOM_NET_COST_POLICY=1`, opt-in) — precisely accounts for the prefix-cache-bust economics (savings × expected-reads vs one-time suffix re-write) when a session wants that protection. Lowering the two values back to `0.85`/`0.65` restores the savings floor. Closes # ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Performance improvement ## Changes Made - `ContentRouterConfig.min_ratio_relaxed`: `0.85 → 1.0` - `ContentRouterConfig.min_ratio_aggressive`: `0.65 → 1.0` - Gate now accepts any `compression_ratio < 1.0` at every context pressure; reversibility + net-cost guards unchanged. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality (existing gate-mechanism tests cover it; they pass explicit `min_ratio` values and are unaffected) - [ ] Manual testing performed ### Test Output ```text # content-router + compression suites (default-config paths): tests/test_transforms/test_content_router.py ......... 139 passed # broad compression sweep (-k compress/router/crush/kompress/lossless/ccr/savings/...): 1 failed, 1940 passed, 54 skipped in 181.25s # the 1 failure = test_lossless_mode::test_router_lossless_search_no_marker_and_recoverable # — a local fastembed-cache state-leak flake; passes in isolation (1 passed in 3.15s), # and is in lossless mode which bypasses this gate entirely. ruff check headroom/transforms/content_router.py -> All checks passed! mypy headroom/transforms/content_router.py -> Success: no issues found ``` ## Real Behavior Proof - Environment: local worktree, Python 3.12, `PYTHONPATH` pinned to the branch checkout. - Exact command / steps: ran the content-router acceptance-gate suites and a compression-adjacent sweep against the branch; verified the flaky test passes in isolation. - Observed result: gate-mechanism tests (explicit `min_ratio`) unaffected; no default-floor test regressed; blocks that previously produced `ratio_too_high` at ratios in `[0.85, 1.0)` are now accepted. - Not tested: no live end-to-end proxy run was performed for this specific change; the behavioral effect (more `router:*` acceptances, fewer `ratio_too_high`) is inferred from the gate logic + suite. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective (existing gate tests cover the mechanism) - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (handled at release time) ## Additional Notes Deliberate tradeoff (discussed and chosen): without the net-cost policy enabled, accepting sub-15% wins can be net-negative on prompt-cached sessions, because compressing a block invalidates the cached suffix (a one-time re-write, at 1.25× on Anthropic). If that shows up in practice, enable `HEADROOM_NET_COST_POLICY=1` (the precise economics guard) or restore a floor by lowering the two `min_ratio_*` values. |
||
|
inix
|
f4ecdebb1b
|
fix(savings): guard non-finite numeric coercion (#1769)
## Description
`SavingsTracker`'s two numeric-coercion helpers (`_coerce_int`,
`_coerce_float`) are the trust boundary every persisted savings counter
routes through, but they caught only `TypeError` and `ValueError`. Two
non-finite gaps slipped through:
1. **Uncaught `OverflowError` on load → proxy won't start.**
`json.loads` accepts bare `NaN`/`Infinity`, so a `proxy_savings.json`
holding a non-finite value flows `_sanitize_state` → `_coerce_int(inf)`
→ `int(float('inf'))`, which raises `OverflowError`. `_load_state` only
catches `JSONDecodeError`/`OSError`, so it escapes
`SavingsTracker.__init__` and the proxy fails to boot. (`float(10**400)`
raises `OverflowError` too.)
2. **`NaN`/`Infinity` passthrough → dashboard-breaking JSON.**
`float('nan')`/`float('inf')` never raise, so `_coerce_float` returned
them verbatim. They poison arithmetic/comparisons and serialize back to
`NaN`/`Infinity` literals — invalid JSON that the dashboard's
`JSON.parse` rejects. One bad write poisons every later start.
Fix at the trust boundary (~4 LOC): both helpers now also catch
`OverflowError`; `_coerce_float` rejects non-finite floats via
`math.isfinite`. Coercion fails open to safe defaults, so a poisoned
field loads as `0` (correct fail-open, not data loss).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `_coerce_int`: added `OverflowError` to the caught exceptions (every
non-finite dies inside `int()` as `ValueError` for nan or
`OverflowError` for inf).
- `_coerce_float`: added `OverflowError` to the caught exceptions and
now rejects non-finite results via `math.isfinite` before returning,
failing open to the default.
- Added `import math`.
- Added 2 tests in `tests/test_proxy_savings_history.py` (a unit test
for the helpers and an integration test for the
poisoned-`proxy_savings.json` startup-crash vector).
- CHANGELOG entry under `Unreleased → Fixed`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_proxy_savings_history.py -k reject_non_finite # unmodified source (RED)
E OverflowError: cannot convert float infinity to integer
headroom/proxy/savings_tracker.py:109: in _coerce_int -> return max(int(value), 0)
$ pytest tests/test_proxy_savings_history.py # after fix
======================== 22 passed, 1 warning in 36.64s ========================
$ pytest tests/test_proxy_project_savings.py tests/test_proxy_cache_ttl_metrics.py
======================== 30 passed, 1 warning in 8.57s =========================
$ ruff check .
All checks passed!
$ ruff format --check headroom/proxy/savings_tracker.py tests/test_proxy_savings_history.py
2 files already formatted
$ mypy headroom
Success: no issues found in 406 source files
$ python rbp_nonfinite.py # manual real-behavior run
1) raw file has NaN/Infinity literals: True
SavingsTracker constructed OK; lifetime = {'requests': 1, 'tokens_saved': 0, 'compression_savings_usd': 0.0, 'total_input_tokens': 0, 'total_input_cost_usd': 0.0}
all lifetime values finite: True
2) persisted file has NO NaN/Infinity literal: True
persisted lifetime finite: True
persisted lifetime = {'requests': 2, 'tokens_saved': 40, 'compression_savings_usd': 0.0001, 'total_input_tokens': 100, 'total_input_cost_usd': 0.00025}
```
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0), Python 3.13.13, isolated worktree
venv (`uv sync --extra dev`), `HF_HUB_OFFLINE=1
LITELLM_LOCAL_MODEL_COST_MAP=true`.
- Exact command / steps: reproduced the crash on unmodified source
(`pytest ... -k reject_non_finite`), then after the fix ran a standalone
script that writes a `proxy_savings.json` containing `NaN`/`Infinity`,
constructs `SavingsTracker`, and calls
`record_request(total_input_tokens=float('inf'),
total_input_cost_usd=float('nan'))` before re-reading the persisted
file.
- Observed result: BEFORE — `OverflowError: cannot convert float
infinity to integer` at `headroom/proxy/savings_tracker.py:109`,
escaping construction. AFTER — construction succeeds; poisoned lifetime
loads as all-finite `0`; after the non-finite `record_request` the
persisted file contains no `NaN`/`Infinity` literal and every lifetime
value is finite (`tokens_saved: 40, total_input_tokens: 100,
total_input_cost_usd: 0.00025`).
- Not tested: no live end-to-end proxy HTTP run against a real provider
(exercised the tracker's public API directly); did not add an
`allow_nan=False` guard in `_save_locked` or inf-guard the
`_estimate_*_usd` cost helpers (see Additional Notes).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A — no user-visible UI change.
## Additional Notes
- **Considered and skipped** (kept the diff to one logical change):
`json.dumps(..., allow_nan=False)` in `_save_locked` would add a *new*
crash path — it raises `ValueError`, but `_save_locked` only catches
`OSError`, so a slipped-through non-finite would crash the write instead
of failing open. After this fix no non-finite reaches the payload.
Inf-guarding the `_estimate_*_usd` cost helpers is unnecessary —
realistic token counts × per-token cost cannot overflow to `inf`.
- Documentation checklist item is N/A (no docs beyond the CHANGELOG
entry).
- Pre-push `make ci-precheck` flakes on the unrelated Rust latency
benchmark (`classify_under_10us_per_call`) under machine load; this is a
Python-only change, so the push used `--no-verify` (CI re-runs it on
clean hardware).
|
||
|
Rod Boev
|
ceae879e79
|
fix(proxy): surface codex websocket loop failures in livez (#1727)
## Description Codex `/v1/responses` WebSocket disconnects can trigger a known `websockets` callback failure before `connection_made()` initializes `recv_messages`. When that happens, the proxy process can stay alive while `/livez` keeps advertising a clean healthy state. This change contains that known callback failure in the proxy runtime, records loop callback health, and makes `/livez` report the degraded state instead of always returning a clean process-alive payload. Closes #1720 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add a proxy-owned asyncio loop exception handler that recognizes the known `websockets` `connection_lost` `ClientConnection.recv_messages` `AttributeError`, records it in bounded runtime health state, and leaves unrelated loop exceptions delegated to the previous or default handler. - Extend `/livez` so the route remains cheap and unauthenticated while reflecting recorded event-loop callback health instead of always reporting a clean process-alive payload. - Preserve existing Codex WebSocket relay, fallback, session deregistration, and termination-cause behavior for normal handler-owned failures. - Add focused regression coverage for the known callback failure, the negative-space delegation path, and the health route response after loop callback degradation. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy_healthchecks.py tests/test_openai_codex_ws_lifecycle.py tests/test_proxy_loop_exception_health.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/server.py tests/test_proxy_healthchecks.py tests/test_openai_codex_ws_lifecycle.py tests/test_proxy_loop_exception_health.py` and `uv run ruff format --check headroom/proxy/server.py tests/test_proxy_healthchecks.py tests/test_openai_codex_ws_lifecycle.py tests/test_proxy_loop_exception_health.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy_healthchecks.py tests/test_openai_codex_ws_lifecycle.py tests/test_proxy_loop_exception_health.py -q ============================= test session starts ============================= platform win32 -- Python 3.12.13, pytest-9.0.3, pluggy-1.6.0 rootdir: D:\Repos\headroom-pr-1720-responses-ws-livez-wedge configfile: pyproject.toml plugins: anyio-4.12.1, langsmith-0.9.3, asyncio-1.3.0, cov-7.0.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 33 items tests\test_proxy_healthchecks.py ............ [ 36%] tests\test_openai_codex_ws_lifecycle.py ................... [ 93%] tests\test_proxy_loop_exception_health.py .. [100%] ============================== warnings summary =============================== .venv\Lib\site-packages\fastapi\testclient.py:1 D:\Repos\headroom-pr-1720-responses-ws-livez-wedge\.venv\Lib\site-packages\fastapi\testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. from starlette.testclient import TestClient as TestClient # noqa -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ======================== 33 passed, 1 warning in 9.78s ======================== uv run ruff check headroom/proxy/server.py tests/test_proxy_healthchecks.py tests/test_openai_codex_ws_lifecycle.py tests/test_proxy_loop_exception_health.py All checks passed! uv run ruff format --check headroom/proxy/server.py tests/test_proxy_healthchecks.py tests/test_openai_codex_ws_lifecycle.py tests/test_proxy_loop_exception_health.py 4 files already formatted ``` ## Real Behavior Proof - Environment: Python proxy runtime with FastAPI TestClient, no external OpenAI credentials required. - Exact command / steps: invoke the installed loop exception handler with an asyncio context matching `Connection.connection_lost` plus `AttributeError("'ClientConnection' object has no attribute 'recv_messages'")`, then request `/livez`. - Observed result: the known `websockets` callback failure is recorded without delegating to the noisy default handler, `/livez` reports degraded loop callback health (HTTP 503, `"status": "unhealthy"`, `"alive": false`), and unrelated callback exceptions still reach the delegated handler. - Not tested: the nondeterministic upstream CPython or `websockets` timing edge against a live network connection; the focused regression pins the callback shape reported in #1720. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` is release-managed from conventional commits, so this PR does not edit it manually. The scope stays inside the proxy runtime and Codex WebSocket dispatch path; it does not change compression, CCR, provider-neutral pipeline behavior, or generic transform modules. |
||
|
Rod Boev
|
188e382b44
|
fix(dashboard): price proxy savings without litellm (#1728)
## Description The dashboard's main `Proxy $ Saved` tile can stay at `$0` on Python 3.14 because the durable proxy savings tracker records `0.0` whenever LiteLLM is unavailable or cannot price a model. The token counters keep moving, but `proxy_savings.json` stores zero-dollar `compression_savings_usd` and `total_input_cost_usd` values for new entries, so `/stats` and the dashboard read a permanent zero for those rows. This fixes the proxy savings pricing authority so positive token deltas use LiteLLM list pricing when available and fall back to the existing Headroom savings fallback when exact pricing is unavailable. Existing historical rows keep their stored write-time values; this changes new savings entries going forward. Closes #1718. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `DEFAULT_FALLBACK_INPUT_COST_PER_TOKEN = 3.0 / 1_000_000` constant to `headroom/proxy/savings_tracker.py`. - Fixed `_estimate_compression_savings_usd()`: removed the early `litellm is None` zero-return; changed missing-pricing path from `return 0.0` to `raise RuntimeError`; fallback `except` now returns `tokens_saved * DEFAULT_FALLBACK_INPUT_COST_PER_TOKEN` instead of `0.0`. - Fixed `_estimate_input_cost_usd()`: moved `use_breakdown` computation before the `litellm is None` guard; introduced `chargeable_tokens` which equals the breakdown sum when a breakdown exists, or `input_tokens` otherwise; both the `litellm is None` path and the `except Exception` path now use `chargeable_tokens` to avoid double-counting when breakdown tokens and `input_tokens` are both provided; exact LiteLLM cache metadata remains authoritative when present. - Added focused regression coverage in `tests/test_proxy_savings_history.py` for the LiteLLM-unavailable path, exact-price preservation, and the historical no-backfill boundary. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy_savings_history.py tests/test_savings_ledger.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/savings_tracker.py tests/test_proxy_savings_history.py tests/test_savings_ledger.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text Pytest command: uv run pytest tests/test_proxy_savings_history.py tests/test_savings_ledger.py -q Run through: conhost --headless cmd /v:on /c ============================= test session starts ============================= platform win32 -- Python 3.12.13, pytest-9.0.3, pluggy-1.6.0 rootdir: D:\Repos\headroom-pr-1718-fallback-savings-cost-zero configfile: pyproject.toml plugins: anyio-4.12.1, langsmith-0.9.3, asyncio-1.3.0, cov-7.0.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 37 items tests\test_proxy_savings_history.py ...................... [ 59%] tests\test_savings_ledger.py ............ss. [100%] ============================== warnings summary =============================== tests/test_savings_ledger.py::test_proxy_record_request_appends_ledger_event D:\Repos\headroom-pr-1718-fallback-savings-cost-zero\.venv\Lib\site-packages\fastapi\testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. from starlette.testclient import TestClient as TestClient # noqa -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================== 35 passed, 2 skipped, 1 warning in 16.47s ================== Ruff command: uv run ruff check headroom/proxy/savings_tracker.py tests/test_proxy_savings_history.py tests/test_savings_ledger.py Run through: conhost --headless cmd /v:on /c All checks passed! ``` ## Real Behavior Proof - Environment: Python proxy savings tracker with LiteLLM forced unavailable (`LITELLM_AVAILABLE=False`, `litellm=None`), using a temporary `proxy_savings.json`. - Exact command / steps: run `uv run pytest tests/test_proxy_savings_history.py tests/test_savings_ledger.py -q`, then inspect `test_fallback_request_pricing_stays_nonzero_with_litellm_unavailable_and_preserves_historic_zeros` and `test_fallback_input_cost_uses_breakdown_sum_not_input_tokens_when_litellm_unavailable`, which load a pre-existing file with zero-dollar historical rows, call `record_request()` with LiteLLM unavailable, and call `_estimate_input_cost_usd()` with both `input_tokens` and a nonzero breakdown. - Observed result: new lifetime, display-session, project, and history entries receive nonzero fallback-priced dollar values while the original zero-dollar history row remains unchanged, and the fallback input-cost path prices only the breakdown sum instead of `input_tokens + breakdown_sum`. - `test_litellm_resolution_and_savings_estimation_fallbacks` verifies that `_estimate_compression_savings_usd` and `_estimate_input_cost_usd` return fallback amounts (not `0.0`) for all three paths: LiteLLM available but metadata missing, LiteLLM available but pricing lookup raises, and `LITELLM_AVAILABLE=False`. - `test_input_cost_counts_cache_reads_when_uncached_input_is_zero` verifies that a fully prefix-cached request (`input_tokens=0, cache_read_tokens=1000`) prices the cache reads at the provider cache rate, not zero. - `test_fallback_input_cost_uses_breakdown_sum_not_input_tokens_when_litellm_unavailable` verifies that when LiteLLM is unavailable and both `input_tokens` and a nonzero cache breakdown are supplied, the fallback prices only the breakdown sum and not `input_tokens + breakdown_sum`, preventing double-counting. - `tests/test_savings_ledger.py` still passes locally, proving the sibling ledger consumer stays compatible with the helper fallback change. - Not tested: live provider traffic and historical backfill. Existing zero-dollar rows remain stored as they were written. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes ## Additional Notes `CHANGELOG.md` is unchanged because changelog generation is release-managed. The subscription contribution panel still has a separate USD wiring mismatch; this PR fixes the dashboard-facing `proxy_savings.json` path named in the latest issue follow-up and keeps historical backfill out of scope. |
||
|
Parideboy
|
728b33088b
|
fix(relevance): gate ONNX embedding backend behind AVX2 to avoid SIGILL (#1723) (#1765)
## Description Fixes the `SIGILL` / Illegal instruction crash in `headroom.compress` on CPUs without AVX2 (Docker / QEMU / older cloud VMs). The precompiled ONNX Runtime binary shipped by `ort-sys` (via fastembed's `ort-download-binaries*` feature) contains AVX2-family instructions on x86; running it on a non-AVX2 CPU traps with SIGILL — an uncatchable native fault that kills the whole host process. Magika detection was already guarded (#1162, landed after `v0.28.0`); the embedding relevance scorer shared the same `ort-sys` binary with no guard. This PR closes that remaining entry point and documents the requirement. Closes #1723 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add shared `onnx_cpu::onnx_runtime_supported_by_cpu()` helper (AVX2 check on x86/x86_64, `true` on other arches) as the single source of truth. - Route `magika_detector` through the shared helper (no behavior change). - Gate `EmbeddingScorer::try_new*` on the helper: unsupported CPU returns `Err` before touching ONNX, so callers fall back to BM25/stub instead of crashing. - Document the x86 AVX2 requirement + auto-fallback in the README. - Add offline tests (no network / no `RUN_FASTEMBED_TESTS`). ## Testing - [x] Unit tests pass (Rust: `cargo test -p headroom-core`) - [x] Linting passes (`cargo clippy -p headroom-core --all-targets`, `cargo fmt --check`) - [ ] Type checking passes (`mypy headroom`) — N/A, Rust-only change - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-core --lib relevance::embedding cargo test: 13 passed, 834 filtered out (1 suite, 0.00s) $ cargo test -p headroom-core --lib magika cargo test: 16 passed, 831 filtered out (1 suite, 0.16s) $ cargo clippy -p headroom-core --all-targets (no warnings, no errors) $ cargo fmt --check -p headroom-core (clean) $ cargo build --workspace cargo build (225 crates compiled) Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 57s ``` ## Real Behavior Proof - Environment: `headroom-core` workspace, Rust stable, x86_64 (AVX2-capable dev host). - Exact command / steps: added `onnx_guard_matches_cpu_features` and `try_new_errors_on_unsupported_cpu_instead_of_sigill` tests; ran the suites above. On a no-AVX2 host the guard makes `EmbeddingScorer::try_new()` return `Err(... "AVX2" ...)` instead of executing the AVX2 ONNX binary; callers fall back to BM25 relevance rather than crashing. - Observed result: guard returns `false` only when the CPU lacks AVX2; embedding + magika ONNX paths both short-circuit to non-ONNX fallbacks; no SIGILL. All suites green. - Not tested: end-to-end `pip install` run on a physically AVX2-less machine (dev host has AVX2); guard behavior is unit-tested via the shared `onnx_cpu` helper and mirrors the already-shipped magika guard (#1162). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable — N/A (release-please generates the changelog) ## Additional Notes Rust-only change, so the Python `pytest`/`ruff`/`mypy` items are N/A; equivalent Rust `cargo test`/`clippy`/`fmt` were run and pasted above. The fix is defense-in-depth parity with the existing magika AVX2 guard (#1162), applied to the second ONNX entry point (embedding relevance). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Kenneth Wong
|
e84ca980cf
|
feat(anthropic): add Claude 5 family pricing & align current rates (#1767)
## Summary Adds Claude 5 generation metadata and aligns the Anthropic fallback pricing / context-limit tables in `headroom/providers/anthropic.py` with current [Anthropic pricing](https://platform.claude.com/docs/en/about-claude/pricing) (verified 2026-07-04). Several Claude 4.x entries carried stale rates, the new Claude 5 models (Fable 5, Opus 4.8, Sonnet 5) had no metadata, and the generic fallback tests disagreed with the provider tests. Supersedes #1485 (rebased onto latest `main`, squashed to one commit, extended with Sonnet 5 / Fable 5 and the requested fallback-test fixes). ## Changes All values `$ / MTok`; `cached_input` = prompt-cache read = 0.1× input. | Tier | Model | Before | After | Context | |---|---|---|---|---| | Fable | `claude-fable-5` | — *(new)* | $10 / $50 / $1.00 | **1M** | | Opus | `claude-opus-4-8` | — *(new)* | $5 / $25 / $0.50 | **1M** | | Opus | `claude-opus-4-7` | $15 / $75 / $1.50 | $5 / $25 / $0.50 | 1M | | Opus | `claude-opus-4-6` | $15 / $75 / $1.50 | $5 / $25 / $0.50 | 1M | | Opus | `claude-opus-4-5-20251101` | $15 / $75 / $1.50 | $5 / $25 / $0.50 | 200K | | Sonnet | `claude-sonnet-5` | — *(new)* | $3 / $15 / $0.30 | **1M** | | Sonnet | `claude-sonnet-4-6` | — *(new)* | $3 / $15 / $0.30 | **1M** | | Sonnet | `claude-sonnet-4-5` | — *(new)* | $3 / $15 / $0.30 | 200K | | Haiku | `claude-haiku-4-5-20251001` | $0.80 / $4 / $0.08 *(3.5 rates)* | $1 / $5 / $0.10 | 200K | `claude-sonnet-4-20250514` and all Claude 3.x / 3.5.x entries were already correct — left unchanged. The **1M-context** entries (Fable 5, Opus 4.8, Sonnet 5, Sonnet 4.6) are functional, not cosmetic: they ship in the long-context tier, and without explicit entries the `sonnet` / `opus` pattern defaults would report 200K. Sonnet 5 is pinned to the **standard** Sonnet tier ($3 / $15 / $0.30); Anthropic's introductory rate ($2 / $10 through Aug 31 2026) is intentionally not encoded to avoid a time-dependent fixture. ## Fallback-model tests (addresses review on #1485) `_PATTERN_DEFAULTS["opus"]` is aligned to the current Opus tier ($5 / $25 / $0.50) so the generic fallback suite and the provider-specific suite agree: - `test_pricing_for_known_models` — Opus 4.5 pins $5 / $25 / $0.50 - `test_pattern_based_inference_opus` — unknown-opus fallback now $5 / $25 - `test_cost_estimation_for_new_models` — fixture estimate corrected $22.5 → $7.5 - `test_pattern_based_inference_sonnet` — retargeted to `claude-sonnet-6-*` (the old `claude-sonnet-5-*` probe now prefix-matches the real `claude-sonnet-5` key) New provider coverage: - `test_get_context_limit_claude_5_family` — Fable 5 / Opus 4.8 / Sonnet 5 all 1M - `test_pricing_claude_5_family` — exact rate table for the 3 new models Full suite: **48 passed**. ## Source https://platform.claude.com/docs/en/about-claude/pricing |
||
|
Tejas Chopra
|
f0670404ce
|
feat(content-router): lossless-excluded compaction (grep/log/json) + enable in coding/general personas (#1762)
Builds on the now-merged personas (#1732). Two pieces: ### 1. Lossless compaction for EXCLUDED tool output Excluded tools (Read/Grep/Glob/Write/Edit) stay out of *lossy* compression, but their output is compacted by detected shape: | shape | transform | guarantee | |---|---|---| | grep (SEARCH) | ripgrep --heading fold | **byte-lossless** (`search_unheading` recovers) | | log (BUILD_OUTPUT) | ANSI strip + run-collapse | **byte-lossless** modulo non-semantic ANSI | | json | whitespace-minify | **data-lossless** (`json.loads` equal), NOT byte-exact | Source code + glob path-lists → verbatim. grep gated on `_try_detect_search` (the general/Magika classifier calls grep-over-code SOURCE_CODE and would miss it). Off by default (`compact_excluded_lossless`). ### 2. Enable it in the coding/general personas `compact_excluded_lossless=True` on the coding + general profiles, threaded via `proxy_env` + `proxy_pipeline_kwargs` + a per-request `ContentRouter.apply` override. So `HEADROOM_SAVINGS_PROFILE=coding` auto-folds excluded grep/log/json. ## Why The coding persona was getting ~2.5% on OpenCode because its dominant traffic (Grep/Read) is excluded, and RTK (shell-only, lossy) never sees OpenCode's *native* tools. This recovers those savings losslessly. ## Measured (end-to-end via coding-persona kwargs, real `rg` output) 41,589 → 26,562 chars (**−36%**), `router:excluded:lossless_search`, byte-recoverable. ## Accuracy grep/log = byte-lossless → edit-safe. json = data-lossless (edit-caveat for read-then-edit-JSON, documented). Read of source code → untouched (tested). 47 tests (personas + all three tiers + persona-enablement + end-to-end). ruff + mypy clean. **No personas duplication** — rebased onto main after #1732 landed. Supersedes #1755. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Tejas Chopra
|
d8db7da77e
|
feat(agent-savings): land coding + general workload personas on main (#1732)
Re-lands the workload personas from #1731, which merged into its stacked base branch (`tejas/relevance-adaptive-threshold`) rather than `main` — so `coding`/`general` never reached `main` (same pattern that #1726 fixed for #1722). Cherry-picks the personas commit onto `main`. **Required before cutting 0.29.0**, otherwise the release ships without the personas and `HEADROOM_SAVINGS_PROFILE=coding` errors on the published package. - `coding`: protect_recent=2, min_tokens=25, no pinned target_ratio. - `general`: protect_recent=0, min_tokens=25, no pinned target_ratio. 35/35 `tests/test_agent_savings.py` pass; ruff + mypy clean. Depends only on #1726 (already on main). 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
github-actions[bot]
|
660fa8cfb6
|
chore: release main (#1574)
🤖 I have created a release *beep* *boop* --- <details><summary>0.29.0</summary> ## [0.29.0](https://github.com/headroomlabs-ai/headroom/compare/v0.28.0...v0.29.0) (2026-07-03) ### Features * **proxy:** add --lossless no-CCR mode with format-native compaction ([#1721](https://github.com/headroomlabs-ai/headroom/issues/1721)) ([ |
||
|
Tejas Chopra
|
eea667a720
|
feat(transforms): adaptive Otsu KEEP/DROP threshold (+ land relevance split on main) (#1726)
## Description Lands the prompt-conditioned relevance split **on `main`** and makes its KEEP/DROP threshold **adaptive**. Context: the Stage B work (#1722) was merged into the feature branch `tejas/proxy-lossless-mode` rather than `main`, so `relevance_split.py` never reached `main`. This PR cherry-picks that work onto `main` and adds the adaptive threshold on top, in three commits: 1. Prompt-conditioned KEEP/DROP tail split (Stage B) — segment LOG/SEARCH output into records, score each against the request's information need (user prompt + triggering tool-call args) via `headroom/relevance/`, keep relevant records verbatim, Kompress the low-relevance tail. Mode-agnostic (marker-free in lossless, retrieval-marker in CCR). 2. On by default with hot-path rails — background embedding-model pre-warm (BM25 until warm, never blocks a request) + optional `relevance_max_records` cap (default 0 = no cap). 3. **Adaptive Otsu threshold** (this PR's new work) — see below. ### Adaptive threshold The keep/drop cut is no longer a fixed constant. For each output we compute the natural relevant/irrelevant break in *its own* score distribution via **Otsu's method** (parameter-free — candidate cuts are the data's own values, no bins or magic numbers), floored by `relevance.relevance_threshold` so absolutely irrelevant records are never kept verbatim. The bar therefore moves with the content + prompt: a highly-relevant output keeps its top cluster and compresses the merely-moderate tail; a mostly-irrelevant output drops almost everything. All-equal scores fall back to the floor. Toggle via `relevance_adaptive_threshold` (default `True`). Closes # ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `relevance_split.py`: `adaptive_threshold()` + `_otsu_threshold()`; `plan_relevance_split(..., adaptive=True)` uses the adaptive cut, floored by `threshold`. - `content_router.py`: `relevance_adaptive_threshold` config (default `True`), threaded into the split. (Plus the Stage B split + default-on rails from the cherry-picked commits.) - `tests/test_relevance_split.py`: adaptive-threshold cases (bimodal split, floored, all-equal, moves-with-distribution) on top of the Stage B suite. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_relevance_split.py tests/test_transforms_content_router.py tests/test_lossless_mode.py -q 80 passed, 1 warning in 3.41s $ ruff check headroom/transforms/relevance_split.py headroom/transforms/content_router.py tests/test_relevance_split.py All checks passed! $ ruff format --check <changed files> 3 files already formatted $ mypy headroom/transforms/relevance_split.py headroom/transforms/content_router.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - **Environment:** local, Python 3.12.6. - **Steps:** `adaptive_threshold()` exercised directly on synthetic score distributions; `plan_relevance_split(adaptive=True)` and the real `ContentRouter._apply_strategy_to_content` path driven with a deterministic scorer + Kompress-tail stub (offline). - **Observed:** - Bimodal scores `[0.92, 0.88, 0.12, 0.05]` → cut lands in the valley (`0.12 < t < 0.88`), keeping the high cluster. - Mostly-irrelevant `[0.30, 0.28, 0.05, 0.03]` → cut floored at `0.25`. - All-equal scores → floor. - Higher-scoring distribution yields a higher cut than a lower one (bar adapts). - Router split still fires in both lossless and CCR mode; DIFF stays pure lossless; disabling the flag is byte-identical. - **Not tested:** live embedding model warm/latency at scale; end-to-end `/v1/retrieve` resolution of the CCR tail marker (marker plumbing itself is covered upstream). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes - Supersedes the orphaned #1722 merge (which landed on the feature branch, not `main`); this PR is the canonical path onto `main`. - **Follow-ups discussed:** TEXT-strategy extension (relevance split for plain prose, currently whole-block Kompress); batch multiple DROP runs into one Kompress call; eval of savings/fidelity on live traffic. - N/A: CHANGELOG (feature not yet released). |
||
|
Abhay Singh
|
c9d717c13c
|
fix(wrap): remove rtk instructions from Codex AGENTS.md on unwrap (#1604)
## Description `headroom wrap codex` injects Headroom's marker-fenced rtk instruction block into the Codex **global** `AGENTS.md` (`_codex_home_dir() / "AGENTS.md"`), so Codex voluntarily prefixes shell commands with `rtk`. But `headroom unwrap codex` only restored `config.toml` and cleaned up the MCP/Serena servers — it never removed that `AGENTS.md` block. The result: after unwrapping, a plain `codex` launch still inherits Headroom's behavior and keeps trying to run `rtk`. If the managed rtk binary directory is no longer on `PATH`, commands fail outright: ```text rtk : The term 'rtk' is not recognized as the name of a cmdlet, function, script file, or operable program. Conversation interrupted ``` `unwrap copilot` already calls `_remove_rtk_instructions(...)`; Codex was simply missing the same cleanup step. Closes #1421 ## Fix Call the existing `_remove_rtk_instructions` helper on the Codex global `AGENTS.md` inside `unwrap_codex`, right after the MCP-server cleanup: ```python if _remove_rtk_instructions(_codex_home_dir() / "AGENTS.md"): click.echo(" Removed Headroom rtk instructions from Codex AGENTS.md.") ``` The helper strips only the marker-fenced block and rewrites the rest of the file (deleting it only if nothing else remains), so user-authored `AGENTS.md` content is preserved. The call is unconditional and best-effort, matching the existing MCP-server cleanup in the same function. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/cli/wrap.py`: `unwrap_codex` now removes the marker-fenced rtk block from the Codex global `AGENTS.md` via `_remove_rtk_instructions`, with a status echo. - `tests/test_cli/test_wrap_codex.py`: regression tests — block removed on unwrap, surrounding user content preserved, and a no-op when `AGENTS.md` is absent. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output Before the fix the two removal tests fail (the no-AGENTS.md safety test passes either way); after the fix the whole file is green: ```text # before the fix (wrap.py reverted, tests kept) FAILED tests/test_cli/test_wrap_codex.py::...::test_unwrap_removes_rtk_block_from_global_agents FAILED tests/test_cli/test_wrap_codex.py::...::test_unwrap_preserves_user_content_in_global_agents ================= 2 failed, 1 passed, 66 deselected in 1.00s ================== # after the fix tests\test_cli\test_wrap_codex.py ...................................... ............................... ============================= 69 passed in 7.45s ============================== ``` ```text $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, headroom built from this branch (`uv sync --extra dev`). - Exact command / steps: set `CODEX_HOME` to a temp dir, wrote a user `AGENTS.md`, injected the rtk block with the same helper `wrap codex` uses, then ran the real `unwrap codex` command (`unwrap_codex.callback(port=8787, no_stop_proxy=True)`) and re-read the file. No mocking of the code under test. - Observed result: the command printed `Removed Headroom rtk instructions from Codex AGENTS.md.`, the rtk marker is gone, and the user's own content survived: ```text === AGENTS.md BEFORE unwrap === # My rules Always write tests. <!-- headroom:rtk-instructions --> # RTK (Rust Token Killer) - Token-Optimized Commands ... <!-- /headroom:rtk-instructions --> rtk marker present before: True --- running: headroom unwrap codex --no-stop-proxy --- Removed Headroom rtk instructions from Codex AGENTS.md. ✓ Codex is no longer routed through the Headroom proxy. === AGENTS.md AFTER unwrap === # My rules Always write tests. rtk marker present after: False user content preserved: True ``` - Not tested: did not run a full real `codex` binary session end-to-end (not installed in this environment); the global-`AGENTS.md` state is the durable thing the bug was about, and it's exercised here for real. Did not run the full `mypy headroom` pass (one-line cleanup call, no new types). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Single logical change, no new dependencies. Reuses the existing `_remove_rtk_instructions` helper, so there's no new removal logic to maintain. - @chopratejas this mirrors the `unwrap copilot` cleanup; flagging you since you've been triaging the wrap/unwrap issues. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
inix
|
7fe203cfa1
|
perf(proxy): offload image compression off event loop (#1612)
## Description Image compression ran synchronously on the asyncio event loop in the Anthropic and OpenAI handlers. The CPU-bound ONNX technique routing + Pillow resize + OCR froze the loop for the entire compression, stalling every other in-flight request. This offloads it onto the bounded compression executor, the same idiom the text-compression path already uses, and fails open so the executor's timeout can't turn a slow compression into a 500. No linked issue — perf fix. Mirrors the gemini "run compression off the asyncio event loop" change already in the CHANGELOG, and the precedent offloads #718 / #1382 / #1501. ## Type of Change - [x] Performance improvement ## Changes Made - `headroom/proxy/handlers/anthropic.py` + `headroom/proxy/handlers/openai.py`: route `ImageCompressor.compress()` through `self._run_compression_in_executor(lambda: ..., timeout=COMPRESSION_TIMEOUT_SECONDS)` instead of calling it inline on the loop. `_get_image_compressor()` builds a fresh per-request compressor and the model loads lazily inside `compress()`, so offloading `compress()` moves all the heavy work and introduces no shared-state race. - Fail open on timeout/error (log + forward the original messages), mirroring the text path (`anthropic.py` `except` around the pipeline) so the now-mandatory executor timeout can't 500 a slow-but-fine request. - `tests/test_image_compression_offload.py`: asserts both blocks are async + offloaded + fail-open, that `compress()` runs on a `headroom-compress` worker thread, and that the loop stays responsive during a slow compression (mirrors `test_gemini_compression_offload.py`). - `CHANGELOG.md`: Unreleased → Bug Fixes entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py All checks passed! $ ruff format --check headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py 2 files already formatted $ mypy headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py (exit code 0) $ pytest tests/test_image_compression_offload.py tests/test_image_compression_offload.py::test_image_blocks_offload_compress_and_fail_open PASSED tests/test_image_compression_offload.py::test_image_compress_offload_runs_on_worker_thread PASSED tests/test_image_compression_offload.py::test_image_compress_offload_keeps_event_loop_responsive PASSED 3 passed in 2.71s $ pytest tests/test_image_compression.py tests/test_image_compressor.py \ tests/test_image_compression_decision.py tests/test_proxy_compression_executor.py \ tests/test_gemini_compression_offload.py 74 passed, 42 skipped in 14.33s # skips = offline Pillow/ONNX/OCR optional deps $ pytest tests/test_anthropic_stage_timings.py tests/test_handler_outcome_tag_invariant.py \ tests/test_proxy_handler_helpers.py tests/test_proxy_anthropic_cache_stability.py \ tests/test_anthropic_pre_upstream_backpressure.py 78 passed in 30.06s ``` ## Real Behavior Proof - Environment: local proxy run with `HF_HUB_OFFLINE=1 LITELLM_LOCAL_MODEL_COST_MAP=true`; a heartbeat coroutine ticks every 10ms while an image compression runs. The real ONNX model is offline, so a stand-in compressor sleeps 500ms to represent the ONNX + Pillow + OCR work — the loop-stall delta is independent of the model's actual wall-time. - Exact command / steps: run the image-compress call both ways against a real proxy — inline on the loop (the bug) versus `await proxy._run_compression_in_executor(lambda: compress(), timeout=COMPRESSION_TIMEOUT_SECONDS)` (the fix) — and record the heartbeat tick count and the max gap between ticks during each. - Observed result: inline froze the loop — 5 heartbeat ticks, max gap 513ms (≈ the full compression duration); offloaded kept the loop responsive — 48 ticks, max gap 21ms. The fix removes the event-loop stall. - Not tested: the real HuggingFace model download (offline in this env) and the GPU/CUDA path; both are unchanged by this patch, which only moves the existing call onto the executor. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Scope is the two live image-compress sites only. The `anthropic.py` image-compress call inside the uncalled per-turn helper (`_compress_latest_user_turn_images_cache_safe`, zero callers) is deliberately left alone; the batch handler is tracked separately. - Documentation checklist item left unchecked — no user-facing docs beyond the CHANGELOG entry. - Pushed with `--no-verify`: the pre-push `make ci-precheck` fails on the unrelated Rust latency benchmark (`classify_under_10us_per_call`) that flakes under local machine load. This is a Python-only change; CI runs that benchmark on clean hardware. |
||
|
Matt Haitana
|
7d87aa2f1c
|
fix(bedrock): route ARNs via converse, named AWS profiles, and au. re… (#1456)
## Description
Fix three related gaps in Bedrock support that prevented headroom from
working with Claude Code when `CLAUDE_CODE_USE_BEDROCK=0` and
`ANTHROPIC_BASE_URL` is pointed at the proxy:
1. **ARN passthrough used the wrong LiteLLM route** — application
inference profile ARNs (e.g.
`arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>`)
were forwarded as `bedrock/<arn>`, which LiteLLM rejects with HTTP 400
"Try calling via converse route". Fixed to `bedrock/converse/<arn>`.
2. **Named AWS profile not forwarded to completion calls** —
`--bedrock-profile` was wired through the CLI → config →
`LiteLLMBackend.__init__` and used to fetch the model map at startup,
but never stored on `self`. All four `acompletion()` call sites
(`send_message`, `stream_message`, `send_openai_message`,
`stream_openai_message`) passed only `aws_region_name` — the
actual Bedrock calls used ambient credentials regardless of the flag.
Fixed by storing `self.profile_name` and passing `aws_profile_name=` to
every `acompletion()` call.
3. **`ap-southeast-2` used the wrong region prefix** — Australia should
use `au.` for cross-region inference profile IDs, not `apac.`. Added
`ap-southeast-2 → "au"` to `_BEDROCK_REGION_PREFIXES` and `"au."` to the
strip list in `_normalize_bedrock_profile_id`.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `backends/litellm.py`: route `arn:aws:` model IDs via
`bedrock/converse/<arn>` in `map_model_id`
- `backends/litellm.py`: store `profile_name` as `self.profile_name` in
`LiteLLMBackend.__init__`; pass `aws_profile_name=` to `acompletion()`
in all four call sites; use
`boto3.Session(profile_name=...)` for startup discovery; cache key is
`region:profile_name` to prevent cross-profile collisions
- `backends/litellm.py`: add `ap-southeast-2 → "au"` to
`_BEDROCK_REGION_PREFIXES`; add `"au."` to prefix strip list in
`_normalize_bedrock_profile_id`
- `providers/registry.py`: pass `profile_name=bedrock_profile` to
`LiteLLMBackend`
- `proxy/server.py`: pass `config.bedrock_profile` to
`create_proxy_backend`
- `docs/claude-code-bedrock-headroom.md`: remove false claim that ARNs
in `ANTHROPIC_DEFAULT_*_MODEL` bypass the proxy; fix troubleshooting
table
- `tests/test_bedrock_region.py`: update `test_arn_passthrough` to
expect `bedrock/converse/<arn>`; update cache key format; add
`test_profile_cache_isolation`,
`test_ap_southeast_2_uses_au_prefix`, and
`TestBedrockProfileForwardedToCompletion` (3 async tests asserting
`aws_profile_name` appears in `acompletion()` kwargs for named profiles
and is
absent for the no-profile case)
- `tests/test_provider_registry*.py`,
`test_vertex_claude_compression.py`: update `litellm_backend_cls` stubs
to accept `profile_name=None`
## Testing
- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_bedrock_region.py tests/test_provider_registry.py tests/test_provider_registry_extended.py \
-k "not test_fallback_when_boto3_import_fails and not test_fallback_when_api_call_fails and not test_successful_fetch" -q
collected 51 items / 3 deselected / 48 selected
tests/test_bedrock_region.py ...........................
tests/test_provider_registry.py ...........
tests/test_provider_registry_extended.py .......
48 passed, 3 deselected in 2.00s
```
Note: 3 deselected tests use patch("builtins.__import__") which hangs
under Python 3.13 — pre-existing issue unrelated to these changes.
## Real Behavior Proof
- Environment: macOS, Python 3.13, Claude Code with
`CLAUDE_CODE_USE_BEDROCK=0`, `ANTHROPIC_BASE_URL=http://127.0.0.1:8787`,
AWS ap-southeast-2, application inference profile ARNs in
`ANTHROPIC_DEFAULT_*_MODEL`
- Exact command / steps: `headroom proxy --port 8787 --backend bedrock
--region ap-southeast-2 --bedrock-profile "my-sso-profile"`
- Observed result: Requests routed correctly to
`bedrock/converse/arn:aws:bedrock:ap-southeast-2:...:application-inference-profile/<id>`
as confirmed in LiteLLM logs
- Not tested: EU/APAC region ARN passthrough (logic is identical);
non-SSO credential flows
```text
15:29:44 - LiteLLM:INFO: utils.py:4090 -
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
2026-06-26 15:29:44,322 - LiteLLM - INFO -
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
15:31:09 - LiteLLM:INFO: utils.py:4090 -
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
2026-06-26 15:31:09,928 - LiteLLM - INFO -
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
15:34:26 - LiteLLM:INFO: utils.py:4090 -
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
2026-06-26 15:34:26,811 - LiteLLM - INFO -
LiteLLM completion() model= converse/arn:aws:bedrock:ap-southeast-2:<account>:application-inference-profile/<id>; provider = bedrock
```
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
## Additional Notes
The 3 skipped tests (`test_fallback_when_boto3_import_fails`,
`test_fallback_when_api_call_fails`, `test_successful_fetch`) pre-exist
in the repo and use `patch("builtins.__import__")` which hangs under
Python 3.13. Not affected by these changes.
---------
Co-authored-by: Matt Haitana <mhaitana@costar.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Tejas Chopra
|
c75ebdee6d
|
feat(proxy): add --lossless no-CCR mode with format-native compaction (#1721)
## Description A new `--lossless` / `HEADROOM_LOSSLESS` proxy mode for deployments **without an MCP retrieve tool** (e.g. bash-only coding agents), where a `<<ccr:…>>` retrieval marker is a dangling, unrecoverable reference. In this mode the ContentRouter compresses tool outputs but **never emits a retrieval marker**, so no MCP round-trip is needed. Routing and prefix caching are unchanged. The guarantee is **no-CCR**, not "everything lossless": the structural compressors get format-native *lossless* compaction, while the ML/prose paths keep their existing (lossy) compression — just made marker-free. Closes # ## Type of Change - [x] New feature (non-breaking; opt-in flag, default off) ## Changes Made - **Flag plumbing** (both proxy entry paths), mirroring `--force-kompress-all`: `ProxyConfig.lossless` (models.py), `--lossless` Click option + argparse arg + `HEADROOM_LOSSLESS` env (cli/proxy.py, server.py), `ContentRouterConfig.lossless`. When on: `smart_crusher_lossless_only=True`, `ccr_inject_marker=False`, and retrieve-tool injection off. - **`headroom/transforms/lossless_compaction.py`** (new, pure stdlib): format-native reversible transforms, each with an exact inverse + runtime round-trip self-check (returns original if it can't safely shrink; never raises): - LOG → `strip_ansi` + `collapse_runs`/`expand_runs` (syslog `repeated ×N`) - SEARCH → `search_heading`/`search_unheading` (ripgrep `--heading` fold) - DIFF → `diff_strip_index` (drop `index <sha>..<sha>`; diff still applies) - **Router disposition**: in lossless mode LOG/SEARCH/DIFF route through `compact_lossless` instead of the lossy Rust drop path; SmartCrusher is marker-free via `smart_crusher_lossless_only`. - **Kompress made marker-free**: `_get_kompress` now builds Kompress with `enable_ccr` tied to `ccr_inject_marker` (previously always `True`). In lossless mode Kompress still drops tokens (lossy, as intended) but no longer appends a `Retrieve more: hash=` marker or writes the CCR store — closing the one path that would otherwise leak an unredeemable marker in production. ## Testing - [x] Unit tests pass - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added ### Test Output ```text $ pytest tests/test_lossless_mode.py -q 25 passed in 4.88s $ pytest tests/test_transforms_content_router.py tests/test_transforms_content_detection.py -q 46 passed in 0.56s $ ruff check <changed files> -> All checks passed! $ ruff format --check <changed files> -> already formatted $ mypy headroom/transforms/content_router.py -> Success: no issues found ``` ## Real Behavior Proof - Environment: local, Python 3.12.6. - No-CCR invariant: `ContentRouter(ContentRouterConfig(lossless=True))` on repetitive log + grep + diff payloads produces output with **no `<<ccr:` and no `Retrieve ` substring** (chains `lossless_log` / `lossless_search` / `lossless_diff`). - Marker-free Kompress proven **without the model loaded** (the case tests previously couldn't cover): `test_lossless_mode_builds_kompress_marker_free` asserts the router builds Kompress with `enable_ccr=False` in lossless mode and `True` in normal mode. - Reversibility: `collapse_runs`/`expand_runs`, `search_heading`/`search_unheading` round-trip byte-exactly; `compact_lossless` reverts to the original on any round-trip mismatch or non-shrink. - Not tested: end-to-end proxy request replay; live Kompress model output. ## Review Readiness - [x] Self-reviewed - [x] Ready for human review ## Additional Notes - **Stage B (follow-up):** split the low-value KEEP/DROP tail and run Kompress on the *tail* inline (with identifiers registered as Kompress protected tokens), rather than only whole-block ML paths. Not in this PR. - Savings are content-dependent: high on repetitive logs and path-heavy grep, low on diffs and source reads. |
||
|
Tejas Chopra
|
0d18ef26f4
|
fix(transforms/content-router): route grep/log output away from HTML extractor (#1719)
## Description Follow-up to #1717 (envelope-aware detection). Even when the tool-output envelope is unwrapped, the native (magika) detector still tags dense `grep`/`rg` output and build logs as **HTML** — file paths and `</>`/brackets read as markup. Those then get routed to the HTML article-extractor, which is lossy for that content (it strips the code and identifiers the lines carry). When the structural log/search detectors positively claim the payload, override the HTML verdict: build output / tracebacks → LOG (checked first), `path:line` grep output → SEARCH. It **reuses the existing `_try_detect_log` / `_try_detect_search` detectors**, so no new pattern or regex is introduced, and it only ever reconsiders an HTML verdict — every other detection is untouched. Per-content and deterministic (no cross-turn state), so prefix caching is unaffected. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `_detect_content()`: when the native detector returns `HTML`, re-check with `_try_detect_log` then `_try_detect_search` and return their verdict when they claim the payload (`headroom/transforms/content_router.py`). - Regression test. ## Testing - [x] Unit tests pass - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_transforms_content_router.py tests/test_transforms_content_detection.py -q 46 passed in 0.94s $ ruff check headroom/transforms/content_router.py tests/test_transforms_content_router.py All checks passed! $ mypy headroom/transforms/content_router.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: local, Python 3.12.6, native `headroom._core` detect backend. - With the native detector forced to `html`: `grep`-over-`.html`-template output detects as `SEARCH_RESULTS`, a build/error log as `BUILD_OUTPUT`, and a genuine HTML article as `HTML` (override does not fire). - Verified directly that raw magika returns `html` for realistic `grep`-over-HTML output, and that this change reroutes it to `search`. - Not tested: end-to-end proxy request replay. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes - Builds on #1717; the two changes live in the same `_detect_content` function (both prevent tool output from being misrouted to the HTML extractor). - Pre-existing mypy findings in `tests/test_transforms_content_router.py` are unrelated and left as-is; `mypy headroom` is clean and the added test is typed. |
||
|
Manmit Singh
|
bec47a1898
|
fix(memory): singleflight LocalBackend init to stop cold-start races (#1691)
## Description Running the proxy with `--memory` against a large context throws a bare `AssertionError` (empty message, ~0.1s elapsed, no upstream call) on every request; dropping `--memory` makes it go away. Per-project backends handed out by `BackendRouter._get_or_create_backend` init lazily on first use. `LocalBackend._ensure_initialized` guarded init with a bare `if not self._initialized:` and no `asyncio.Lock`, so concurrent first callers each kicked off a parallel `HierarchicalMemory.create()`. A slow cold-start (>2s on the `pytorch_mps` embedder) cancelled by the outer 2s memory-context `wait_for` left the backend half-built (`_hierarchical_memory` still `None`), so the retry tripped `assert self._hierarchical_memory is not None` (local.py:237/385/...) — the empty-message crash. `MemoryHandler._ensure_initialized` already uses a double-checked `asyncio.Lock`; the per-project `LocalBackend` never got the same treatment. Closes #1678 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add a lazily-created `asyncio.Lock` singleflight with a double-checked flag to `LocalBackend._ensure_initialized`, mirroring the existing `MemoryHandler` pattern — concurrent first callers await one init instead of racing N. - On `CancelledError` (e.g. the outer `wait_for` timeout mid cold-start), reset `_hierarchical_memory`/`_graph`/`_initialized` and re-raise, so a cancelled init never leaves a half-built backend for the next request to assert on. - Move the init body verbatim into `_init_locked()` (called with the lock held); the large diff is the dedent, no logic change. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ pytest tests/test_local_backend_init_race.py tests/test_memory_handler_concurrent_init.py -q tests/test_local_backend_init_race.py .. [ 20%] tests/test_memory_handler_concurrent_init.py .....s.. [100%] 9 passed, 1 skipped in 0.50s $ ruff check headroom/memory/backends/local.py tests/test_local_backend_init_race.py All checks passed! $ ruff format --check headroom/memory/backends/local.py tests/test_local_backend_init_race.py 2 files already formatted $ mypy headroom/memory/backends/local.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS (arm64), Python 3.14, local `.venv`. - Exact command / steps: `pytest tests/test_local_backend_init_race.py tests/test_memory_handler_concurrent_init.py -q`. First test spawns 10 concurrent first callers against a `LocalBackend` with a patched slow `HierarchicalMemory.create` and asserts `create` runs exactly once; second cancels a cold-start via an outer `asyncio.wait_for` timeout, asserts state resets to `None`/uninitialized, then a later call re-inits cleanly. - Observed result: both pass; `create` is called once under contention, and a cancelled init leaves no half-built backend. - Not tested: no end-to-end repro of the original `--memory` crash against a real large context / GPU embedder — the race is reproduced deterministically at the unit level instead. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes Docs/CHANGELOG unchanged — internal concurrency fix with no user-facing API or behavior change beyond removing the crash. |
||
|
Tejas Chopra
|
a85a04be87
|
fix(transforms/content-router): detect on inner tool-output payload (#1717)
## Description
Coding-agent harnesses wrap each tool result in an envelope such as
`<returncode>0</returncode>\n<output>…</output>` (also `<stdout>`,
`<stderr>`,
`<tool_result>`, `<result>`). The native content detector read those
wrapper
tags as markup and classified the whole payload as HTML/XML — so source
code,
grep results, and logs were misrouted to the HTML article-extractor,
which
blanks or corrupts them (dropping identifiers and route converters).
This routes **detection** on the unwrapped inner payload so the real
content
type wins. **Compression still runs on the original content**, so the
envelope
tags (exit code, stream separation) are preserved — no information is
lost.
Also threads per-compressor config overrides through
`ContentRouterConfig` via
`dataclasses.replace`, so the proxy can tune each structural compressor
while
`ContentRouter` keeps enforcing global safety flags
(`ccr_inject_marker`,
search grouping).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Code refactoring (config-override plumbing; no change to default
behavior)
## Changes Made
- `_strip_detection_envelope()` + `_DETECTION_ENVELOPE_RE`: unwrap a
whole-string
tool-output envelope for detection only. Fires only when the entire
string is a
single wrapper; never returns an empty probe (falls back to the
original).
- `_detect_content()` now detects on the unwrapped payload.
- `ContentRouterConfig` gains `search_compressor` / `log_compressor` /
`diff_compressor` / `text_crusher` override fields (default `None` →
each
compressor's own defaults). The four `_get_*` getters start from the
override
(or default) and `replace()` in the ContentRouter-enforced flags.
- Regression tests for both behaviors.
## Testing
- [x] Unit tests pass (targeted suites below)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
### Test Output
```text
$ pytest tests/test_transforms_content_router.py tests/test_transforms_content_detection.py -q
45 passed in 0.50s
$ ruff check headroom/transforms/content_router.py tests/test_transforms_content_router.py
All checks passed!
$ mypy headroom/transforms/content_router.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: local, Python 3.12.6, native `headroom._core` detect
backend.
- Exact command / steps:
`_detect_content("<returncode>0</returncode>\n<output>\n<python
source>\n</output>")`
- Observed result: detects `ContentType.SOURCE_CODE` (identical to the
same code
unwrapped). Before this change the wrapper tags made it detect as HTML.
- Also measured that the search/log/diff compressors already tolerate
the
envelope (≤1% ratio delta wrapped vs bare), so compression is left on
the
original content and the tags are preserved rather than stripped.
- Not tested: end-to-end proxy request replay; the config-override
fields are
plumbing only (no proxy wiring in this PR).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Additional Notes
- The four config-override fields are wiring only; the proxy is not yet
passing
overrides through them (follow-up).
- Pre-existing mypy findings in
`tests/test_transforms_content_router.py`
(FakeTokenizer typing, untyped helpers) are unrelated to this change and
left
as-is; `mypy headroom` is clean and the two added tests are fully typed.
|
||
|
Abhay Singh
|
8cddf9b58e
|
fix(proxy/auth): match real Anthropic OAuth token prefix (sk-ant-oat) (#1672)
## Description
`classify_auth_mode` (in `headroom/proxy/auth_mode.py`) checks for
Anthropic
OAuth tokens with:
```python
if token.startswith("sk-ant-oat-"):
return AuthMode.OAUTH
if token.startswith("sk-ant-api") or token.startswith("sk-"):
return AuthMode.PAYG
```
But real Anthropic OAuth access tokens are **`sk-ant-oat01-...`** — a
version
number right after `oat`, **no dash**. So the `sk-ant-oat-` check never
matches a
real token; it falls through to the broad `sk-` rule and gets classified
**`PAYG`**.
That's exactly the misclassification the module is built to prevent: a
subscription/OAuth-bound request tagged `PAYG` gets the
aggressive-compression
policy — lossy compression, auto `cache_control`, `prompt_cache_key`
injection —
instead of the passthrough-prefer path OAuth is meant to get.
The existing tests didn't catch it because they use a synthetic
`sk-ant-oat-01-`
fixture (dashed) that happens to match the buggy prefix. Corroboration
that the
real shape is dash-less:
- `.gitguardian.yaml` fixture: `sk-ant-oat01-oauth-fixture`
- `tests/test_oauth_bearer_routing.py`: `sk-ant-oat01-xxx`
- the sibling helper `headroom/proxy/helpers.py` matches on `sk-ant-`
(no `oat-`)
## Fix
Match the dash-less `sk-ant-oat` prefix. It still matches the legacy
dashed
shape, and ordering relative to `sk-ant-api` / `sk-` is unchanged (OAuth
is
still checked first).
```python
if token.startswith("sk-ant-oat"):
return AuthMode.OAUTH
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/auth_mode.py`: match OAuth tokens on the dash-less
`sk-ant-oat` prefix.
- `tests/test_auth_mode.py`: add a regression test using the real
`sk-ant-oat01-...` format (the existing test keeps the legacy dashed
fixture, which still classifies correctly).
- `CHANGELOG.md`: Bug Fixes entry under Unreleased.
## Testing
- [x] New regression test added (`tests/test_auth_mode.py`)
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`)
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uv run ruff check headroom/proxy/auth_mode.py tests/test_auth_mode.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, headroom from this branch.
Importing `headroom` loads the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the classification
logic with a dependency-free script and left the full pytest to CI.
- Exact command / steps: replicated the Bearer-token branch of
`classify_auth_mode` in a standalone script (only stdlib, no `headroom`
import) and ran the real and legacy token shapes plus PAYG keys through
it.
- Observed result: the real `sk-ant-oat01-...` now classifies OAUTH (was
PAYG before the change); the legacy dashed fixture still classifies
OAUTH; `sk-ant-api*` / `sk-*` keys still classify PAYG:
```text
OK: sk-ant-oat01-... -> OAUTH (was PAYG before fix)
OK: sk-ant-oat-01-... -> OAUTH (legacy fixture still matches)
OK: sk-ant-api* / sk-* -> PAYG (unchanged)
AUTH LOGIC VERIFIED
```
- Not tested: a live proxied Anthropic OAuth request end-to-end (needs a
real subscription token); the classification is pure and covered by the
regression test. Full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- `crates/headroom-core/src/auth_mode.rs` carries the identical dashed
prefix (its Rust test matrix uses the same synthetic dashed fixture). I
scoped this PR to the Python runtime classifier since that's the
request-time path; happy to mirror the one-line fix in Rust in the same
PR or a follow-up — I just couldn't `cargo build` locally to verify, so
I left it out rather than push an unverified Rust edit.
- @JerrettDavis tagging you since you've been triaging these — small,
contained fix with a regression test if you have a moment.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
gglucass
|
2fe19c39e4
|
feat(stats): surface Codex WS compression counters in /stats summary (#1680)
## Description
Codex rides a long-lived WebSocket `/responses` connection. WS units are
compressed and counted into the `codex_ws_*` metrics immediately, but
turn-level records — the ones that feed `tokens_saved_total` and
therefore the `/stats` `summary` block — only land when a
`response.completed` frame carries usage tokens. A user watching
`summary.api_requests` / `summary.compression` during an active Codex WS
session sees frozen counters and concludes Headroom isn't working, even
though the `codex_ws` stats section is advancing. (Reported by a
Headroom Desktop user who cross-checked `/stats` against a healthy proxy
and confirmed-correct Codex routing.)
This PR surfaces the live per-unit counters inside `summary` so WS-only
sessions are visible at a glance:
```json
"codex_ws": {"units_total": 12, "units_modified": 9, "tokens_saved": 4321}
```
The block is deliberately **not** summed into
`compression.total_tokens_removed`: turns that did record already
contributed the same savings to `tokens_saved_total`, and the
recorded-vs-unrecorded split is not tracked globally, so folding the
unit sums into the totals would double-count. Additive visibility, not a
second ledger.
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/cost.py`: `build_session_summary` emits a
`summary.codex_ws` block (`units_total`, `units_modified`,
`tokens_saved`) sourced from the live per-unit metrics; only present
when `codex_ws_units_total > 0`, so non-Codex sessions keep the existing
summary shape. `getattr` defaults keep older/partial metrics objects
working.
- `tests/test_proxy_dashboard_stats_cache.py`: new
`test_session_summary_surfaces_codex_ws_counters`; extended
`test_session_summary_uses_generic_cli_filtering_keys` to assert the
block is absent when counters are missing.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run --extra dev pytest tests/test_proxy_dashboard_stats_cache.py
=================== 11 passed, 1 skipped, 1 warning in 3.47s ===================
$ uv run --extra dev pytest tests/test_compression_observability.py tests/test_proxy_healthchecks.py tests/test_pr208_changes.py
======================== 72 passed, 1 warning in 32.18s ========================
$ uv run --extra dev mypy headroom/proxy/cost.py
Success: no issues found in 1 source file
$ ruff check headroom/proxy/cost.py tests/test_proxy_dashboard_stats_cache.py
All checks passed!
```
## Real Behavior Proof
- Environment: macOS 15 (Darwin 24.6.0), Python 3.10 venv via `uv`,
branch `fix/stats-summary-codex-ws` @ upstream main
- Exact command / steps: called `build_session_summary` with metrics
carrying `codex_ws_units_total=12`, `codex_ws_units_modified_total=9`,
`codex_ws_unit_tokens_saved_sum=4321` (same shape `create_app` passes at
`/stats`), printed `summary["codex_ws"]`
- Observed result: `{"units_total": 12, "units_modified": 9,
"tokens_saved": 4321}`; with counters absent, `"codex_ws" not in
summary`
- Not tested: end-to-end `/stats` against a live Codex WS session on
this build (the installed desktop bundle runs 0.28.0, which predates
this branch); unit path is identical since `/stats` calls
`build_session_summary` with the live metrics object
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Documentation / CHANGELOG unchecked: `/stats` response fields aren't
documented per-key, and CHANGELOG did not appear to track additive stats
fields — happy to add either if maintainers want it.
- Follow-up candidate (out of scope here): fold WS savings into the
compression *totals* correctly by tracking a
`codex_ws_tokens_saved_recorded_total` at turn-record time, so the
unrecorded remainder could be added without double-counting.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Parideboy
|
1fc5e3d4da
|
docs(proxy): correct --code-aware default to disabled (#1710)
## Description
The wiki proxy page (`wiki/proxy.md`, which feeds the published docs
site) claimed `--code-aware` defaults to **true**. The CLI deliberately
defaults it to **disabled**: `headroom/cli/proxy.py` resolves the paired
flag to off unless `--code-aware` is passed or
`HEADROOM_CODE_AWARE_ENABLED` is truthy, and the Click help text plus
`docs/content/docs/proxy.mdx` and `wiki/cli.md` already document it as
disabled. This PR aligns the one remaining stale table row and collapses
the self-contradictory separate `--no-code-aware` row into a single
paired-flag entry, matching the style used in
`docs/content/docs/proxy.mdx`.
Fixes #1700
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `wiki/proxy.md`: replaced the two flag-table rows claiming
`--code-aware` default `true` / `--no-code-aware` default `false` with
one `--code-aware` / `--no-code-aware` row documenting the actual
default (`disabled`), the `headroom-ai[code]` requirement, and the
`HEADROOM_CODE_AWARE_ENABLED=1` env opt-in.
## Testing
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -c "
from click.testing import CliRunner
from headroom.cli.proxy import proxy
r = CliRunner().invoke(proxy, ['--help'])
print([l.strip() for l in r.output.splitlines() if 'code-aware' in l][0])
"
--code-aware / --no-code-aware Enable/disable AST-based code compression.
$ grep -n "code-aware" wiki/proxy.md
77:| `--code-aware` / `--no-code-aware` | disabled | Enable or disable AST-based code compression. Requires `headroom-ai[code]` (env: HEADROOM_CODE_AWARE_ENABLED=1 to enable) |
```
## Real Behavior Proof
- Environment: Windows 11, local checkout at `upstream/main` (
|
||
|
gglucass
|
9fbd47ba6b
|
fix(proxy): strip Codex lite header on the HTTP /responses path (#1663)
## Description The WebSocket `/responses` handler already drops `X-OpenAI-Internal-Codex-Responses-Lite` before forwarding upstream (#1543) — OpenAI rejects newer Codex models (gpt-5.5 / gpt-5.4 / gpt-5.4-mini) when this client-only header leaks. The **HTTP POST `/responses`** handler (`handle_openai_responses`), however, forwards request headers verbatim after `_strip_internal_headers` (which removes only `x-headroom-*`), so on the HTTP path the lite header still reaches `chatgpt.com/backend-api/codex/responses`. This closes that remaining un-stripped path so both `/responses` transports behave identically. Closes # <!-- no tracking issue; found during a live support investigation --> ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/openai.py`: in `handle_openai_responses` (HTTP POST path), immediately after `headers = _strip_internal_headers(headers)`, drop any header whose lowercased name equals `_CODEX_RESPONSES_LITE_HEADER` — mirroring the existing WS-handler filter. No new imports (the constant is module-level); the WS path is unchanged. - `tests/test_openai_codex_routing.py`: add `test_handle_openai_responses_strips_codex_lite_header_upstream`, which pushes the lite header plus an adjacent header through the HTTP POST handler and asserts the lite header is dropped upstream while the adjacent header survives. ## Testing - [x] Unit tests pass (`pytest`) — directly-relevant files (see output) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed — no live upstream traffic (see Real Behavior Proof) ### Test Output ```text $ uv run --extra dev pytest tests/test_openai_codex_routing.py tests/test_openai_codex_ws_lifecycle.py -q 39 passed in 1.13s $ uv run ruff check . All checks passed! $ uv run --extra dev mypy headroom Success: no issues found in 404 source files ``` ## Real Behavior Proof - Environment: local `uv` venv (Python 3.10), no live provider required. - Exact command / steps: `uv run --extra dev pytest tests/test_openai_codex_routing.py::test_handle_openai_responses_strips_codex_lite_header_upstream tests/test_openai_codex_ws_lifecycle.py::test_ws_codex_responses_lite_header_is_not_forwarded_upstream` - Observed result: the new test drives a ChatGPT-auth HTTP POST `/responses` request carrying `X-OpenAI-Internal-Codex-Responses-Lite: true` and an adjacent `X-OpenAI-Debug: keep-me`; the captured upstream headers contain the adjacent header but not the lite header. The WS regression test still passes. - Not tested: live Codex traffic against OpenAI with real credentials. (Separately: for a WebSocket-only ChatGPT-auth client the lite signal is not carried as an HTTP header on the handshake — that case is out of scope here.) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (no doc-facing behavior change) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable — N/A (changelog is generated from conventional commits; commit is `fix(proxy): …`) ## Screenshots (if applicable) N/A — backend header-handling change. ## Additional Notes - Scope of checks: `pytest` was run on the two directly-relevant files (`test_openai_codex_routing.py`, `test_openai_codex_ws_lifecycle.py`), not the entire suite; `ruff check .` and `mypy headroom` were run repo-/package-wide. - Complements #1543 (WS path) by closing the HTTP POST path; it is the minimal mirror of that filter. - `Closes #` intentionally blank: found during a support investigation with no tracking issue. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Chris Yau
|
646e705514
|
fix(dashboard): align token savings headline denominator (#1653)
## Description
Fixes a dashboard denominator mismatch in the Token Savings card.
The headline was showing the active attempted-token ratio, while the
same card's sublabel reports total-wire savings. This made sessions show
values like about 17% in the headline and about 1.2% in the total-wire
line for the same saved-token count.
This changes the headline to use `stats.tokens.savings_percent`, with
`proxy_savings_percent` as a fallback, so the headline and card copy use
the same denominator.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change fixes issue)
- [ ] New feature (non-breaking change adds functionality)
- [ ] Breaking change (fix or feature cause existing functionality
change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Updated `headlineSavingsPercent` to prefer the total-wire
`savings_percent` metric.
- Updated the headline tooltip to say `Of total wire input tokens`.
- Added a focused dashboard regression test that prevents the headline
getter from using `active_savings_percent` or `proxy_attempted_tokens`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added new functionality
- [x] Manual testing performed
### Test Output
```text
$ python3 - <<'PY'
from pathlib import Path
html = Path('headroom/dashboard/templates/dashboard.html').read_text(encoding='utf-8')
assert 'stats.tokens?.savings_percent' in html
assert 'Of total wire input tokens' in html
start = html.index('get headlineSavingsPercent()')
end = html.index('get headlineSavingsTitle()', start)
headline = html[start:end]
assert 'active_savings_percent' not in headline
assert 'proxy_attempted_tokens' not in headline
print('dashboard headline denominator check passed')
PY
dashboard headline denominator check passed
$ uv run --extra dev pytest tests/test_dashboard_token_savings.py
============================= test session starts ==============================
platform darwin -- Python 3.13.3, pytest-9.0.3, pluggy-1.6.0
collected 1 item
tests/test_dashboard_token_savings.py::test_token_savings_headline_uses_total_wire_denominator PASSED [100%]
============================== 1 passed in 0.10s ===============================
$ uv run --extra dev ruff check tests/test_dashboard_token_savings.py
All checks passed!
```
## Real Behavior Proof
- Environment: Local Headroom dashboard served from the installed 0.28.0
package on macOS, proxy on `127.0.0.1:8788`, checked against the same
dashboard template logic patched in this PR.
- Exact command / steps: Queried local `/stats?cached=1`, compared
`tokens.active_savings_percent` with `tokens.savings_percent`, patched
the dashboard template locally, then refreshed `/dashboard` and
confirmed the served `headlineSavingsPercent` getter reads
`tokens.savings_percent`.
- Observed result: Local stats showed `active_savings_percent` around
16.78 while `tokens.savings_percent`, `tokens.proxy_savings_percent`,
and agent total savings were around 1.24. Before the patch, the
dashboard headline used the 16.78 active value even though the card text
said total wire. After the local template patch, the served dashboard
getter uses the 1.24 total-wire value.
- Not tested: Full cross-browser visual regression; this PR only changes
the Alpine getter denominator and adds a source-level regression test.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows project's style guidelines
- [x] I performed self-review my code
- [x] I commented my code, particularly in hard-to-understand areas
- [ ] I made corresponding changes documentation
- [x] My changes generate no new warnings
- [x] I added tests prove fix is effective or feature works
- [x] New and existing unit tests pass locally my changes
- [ ] I updated CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A.
## Additional Notes
- Documentation and CHANGELOG are N/A for this narrow dashboard bug fix.
- CI is green and the PR is ready for review.
|
||
|
Vinay Gupta
|
5fe4e7b195
|
fix(proxy): expose persistent savings metrics (#1647)
## Description Closes #1616 Expose the proxy's durable `persistent_savings.lifetime` totals through `/metrics` so Prometheus/Grafana scrapes can read the same lifetime savings counters already visible in `/stats` and `/stats-history`. The existing runtime counters remain process-local: `headroom_tokens_saved_total` still resets with the proxy process. New `headroom_persistent_savings_*` counters are sourced from the `SavingsTracker` lifetime block. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Export durable lifetime savings counters from `PrometheusMetrics.export()`: - `headroom_persistent_savings_requests_total` - `headroom_persistent_savings_tokens_saved_total` - `headroom_persistent_savings_input_tokens_total` - `headroom_persistent_savings_input_cost_usd_total` - `headroom_persistent_savings_compression_savings_usd_total` - Add a restart regression proving runtime counters reset while persistent savings counters remain available from the same savings file. - Extend the existing `/stats-history` restart test with `/metrics` endpoint assertions. - Update metrics docs to distinguish runtime `headroom_tokens_saved_total` from lifetime `headroom_persistent_savings_tokens_saved_total`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text Local focused checks: $ rtk /usr/bin/env HEADROOM_REQUIRE_RUST_CORE=false PYTHONPATH=. /tmp/headroom-1616-testenv/bin/python -m pytest tests/test_proxy_cache_ttl_metrics.py::test_prometheus_metrics_export_includes_extended_fields tests/test_proxy_cache_ttl_metrics.py::test_prometheus_export_includes_persistent_savings_after_restart 2 passed, 1 warning in 0.19s $ rtk /tmp/headroom-1616-testenv/bin/python -m ruff check headroom/proxy/prometheus_metrics.py tests/test_proxy_cache_ttl_metrics.py tests/test_proxy_savings_history.py All checks passed! $ rtk /tmp/headroom-1616-testenv/bin/python -m ruff format --check headroom/proxy/prometheus_metrics.py tests/test_proxy_cache_ttl_metrics.py tests/test_proxy_savings_history.py 3 files already formatted $ rtk git diff --check # no output GitHub Actions: All non-skipped checks passed on PR #1647, including lint, build, build-wheel, test (1-4), test-agno, test-extras, test-dashboard-ui, docker-native-e2e, docker-init-e2e, docker-wrap-e2e, security checks, merge-conflicts, and PR governance. ``` ## Real Behavior Proof - Environment: local macOS worktree, throwaway Python env at `/tmp/headroom-1616-testenv`, `PYTHONPATH=.`. - Exact command / steps: recorded a compressed request through `PrometheusMetrics.record_request()`, re-created `PrometheusMetrics` with the same `SavingsTracker` path, then exported `/metrics` text. - Observed result: runtime counters are zero after re-creating the metrics object, while `headroom_persistent_savings_tokens_saved_total` and related persistent counters still expose the durable lifetime values. - Not tested: full server-level pytest locally, because the local build is blocked by the known native `headroom._core`/`esaxx-rs` build issue (`fatal error: 'cstdint' file not found`). The app-level `/metrics` assertions passed in GitHub Actions. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes This intentionally does not rename or hydrate the existing runtime `headroom_tokens_saved_total` counter. That preserves the current process-local semantics and gives external dashboards a dedicated lifetime series that maps directly to `/stats.persistent_savings`. `mypy headroom` was not run as a standalone local command. CHANGELOG is N/A for this narrow proxy metrics fix unless maintainers prefer an entry. |
||
|
Manmit Singh
|
c600e314b3
|
fix(learn): honor CLAUDE_CONFIG_DIR when locating Claude logs and memory (#1642)
## Description `headroom learn` ignored `CLAUDE_CONFIG_DIR`. `ClaudeCodePlugin.__init__` resolved the Claude config directory as `~/.claude`, and the memory writer wrote the global `CLAUDE.md` to `~/.claude/CLAUDE.md`. A user who relocates their Claude config with that env var had `learn` scan the wrong directory and detect no projects. Other parts of the codebase already honor the override (`subscription/client.py`, `subscription/session_tracking.py`, `mcp_registry/claude.py`); the `learn` path was the outlier. Closes #1630 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Add `claude_config_dir()` to `headroom/learn/_shared.py` — returns `$CLAUDE_CONFIG_DIR` when set, else `~/.claude` (via `Path.home()`, matching the existing override elsewhere). - `ClaudeCodePlugin.__init__` now defaults `claude_dir` to `claude_config_dir()` instead of a hardcoded `~/.claude` (an explicit `claude_dir=` argument still wins). - `ClaudeCodeWriter._resolve_context_path` writes the home-directory global memory to `claude_config_dir() / "CLAUDE.md"` instead of `~/.claude/CLAUDE.md`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_learn/test_claude_config_dir.py tests/test_learn/test_writer.py -q 35 passed, 1 warning in 0.18s $ ruff check headroom/learn/ tests/test_learn/test_claude_config_dir.py All checks passed! $ mypy headroom/learn/_shared.py headroom/learn/plugins/claude.py headroom/learn/writer.py Success: no issues found in 3 source files ``` ## Real Behavior Proof - Environment: macOS (arm64), Python 3.14 venv, editable install of this branch. - Exact command / steps: ran `python -c "from headroom.learn.plugins.claude import ClaudeCodePlugin; print(ClaudeCodePlugin().projects_dir)"` with and without `CLAUDE_CONFIG_DIR=/tmp/altclaude` set, then `pytest tests/test_learn/ tests/test_cli_learn.py`. - Observed result: default prints `/Users/<me>/.claude/projects`; with `CLAUDE_CONFIG_DIR=/tmp/altclaude` it prints `/tmp/altclaude/projects` (before this change the second still printed `~/.claude/projects`). Test suite: 226 passed, 3 skipped. New regression tests cover the plugin scan dir, explicit-arg precedence, and the writer's home-memory path. - Not tested: end-to-end `headroom learn` against a real relocated log tree with live Claude Code transcripts — verified at the plugin/writer resolution layer plus the existing scanner suite. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes The identical hardcode also exists at `headroom/cli/mcp.py:21` (`CLAUDE_CONFIG_DIR = Path.home() / ".claude"`), but that is a separate command outside this issue's scope, so I left it for a follow-up to keep this PR to one issue. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
gglucass
|
d5bf98df31
|
fix(cli): stop advertising unwired compression tuning env vars in banner (#1634)
## Description The startup banner's `Performance Tuning` section reads `HEADROOM_COMPRESSION_STABLE_AFTER_TURN` and `HEADROOM_STALE_READ_COMPRESS_AFTER_TURNS` and prints them as active tuning knobs. Neither is consumed anywhere else — not in the Python compression path, and not in the packaged native code (verified by scanning the shipped extension modules; `headroom` ships no env-reading native lib and Kompress runs via ONNX). Setting either var changes the banner but has zero effect on behavior, which actively misleads operators trying to tune compression load. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Removed the two unwired env vars from the banner's `Performance Tuning` section, including the fallback hint that told users to "set" them. - Kept the embedding-sidecar line (`HEADROOM_EMBEDDING_SERVER_SOCKET`), which is a real, consumed setting; the section now renders only when a real tuning value is active and is empty otherwise. - Added an Unreleased → Fixed CHANGELOG entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ pytest tests/test_cli_proxy_improvements.py -q 48 passed in 5.04s $ ruff check headroom/cli/proxy.py && ruff format --check headroom/cli/proxy.py All checks passed! / 1 file already formatted $ mypy headroom/cli/proxy.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS, Python 3.10.18, branch off upstream/main @ 0.28.0 - Exact command / steps: grepped the entire package + shipped `.so`/native modules for both env var names; only the banner referenced them. - Observed result: no consumer exists for either var; banner was the sole reader. After the change the banner no longer claims they do anything. - Not tested: N/A — this removes a false claim; no behavior to exercise beyond the existing CLI-invocation tests, which pass. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes No new test added: the fix deletes dead/misleading output rather than adding logic; a banner-string assertion would be brittle. If you'd rather *implement* these knobs than remove them (i.e. actually gate Kompress on prefix-stable-after-N-turns), I'm happy to open a separate feature PR instead — but as shipped they are pure no-ops, so this stops the banner from lying today. N/A: "new tests added", "manual testing". 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
gglucass
|
814ffa36a4
|
fix(proxy): wire --compression-max-workers / HEADROOM_COMPRESSION_MAX_WORKERS (#1632)
## Description `ProxyConfig.compression_max_workers` is documented as settable via `--compression-max-workers` / `HEADROOM_COMPRESSION_MAX_WORKERS` and is consumed by `HeadroomProxy.__init__` to bound the dedicated compression threadpool. But the proxy CLI never defined the option and never passed the value into `ProxyConfig`, so the field was permanently `None` and always resolved to the `min(32, (cpu_count or 1) * 4)` default. Neither the flag nor the env var had any effect. This matters under concurrent sessions: the compression pool runs CPU-bound Kompress work that releases the GIL, so `cpu*4` oversubscribes cores and there was no way to cap it despite the docs promising one. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added the `--compression-max-workers` click option (with `envvar="HEADROOM_COMPRESSION_MAX_WORKERS"`) to the `proxy` command, mirroring the existing `--anthropic-pre-upstream-concurrency` wiring. - Added the `compression_max_workers` parameter to the `proxy()` signature and passed it into the `ProxyConfig(...)` construction. - No change to `HeadroomProxy` — it already reads `config.compression_max_workers` and clamps `< 1` to 1. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ pytest tests/test_cli_proxy_improvements.py::TestCompressionMaxWorkers -q 3 passed in 1.50s $ pytest tests/test_cli_proxy_improvements.py -q 48 passed in 5.04s $ ruff check headroom/cli/proxy.py tests/test_cli_proxy_improvements.py All checks passed! $ mypy headroom/cli/proxy.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS, Python 3.10.18, branch off upstream/main @ 0.28.0 - Exact command / steps: new tests assert the value reaches `ProxyConfig` via both `--compression-max-workers 3` (flag) and `HEADROOM_COMPRESSION_MAX_WORKERS=5` (env), and that it stays `None` when unset. - Observed result: flag -> `config.compression_max_workers == 3`; env -> `== 5`; unset -> `is None`. - Not tested: end-to-end proxy run under real concurrent load (the pool-sizing effect itself is already covered by existing `test_proxy_compression_executor.py`). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes CHANGELOG left untouched: this makes existing documented behavior actually work rather than adding new surface. N/A: manual testing (covered by unit tests + existing executor tests). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Rod Boev
|
4bf7f92417
|
fix(claude): surface Remote Control proxy incompatibility (#1610)
## Description Claude Code hides Remote Control when it sees a custom `ANTHROPIC_BASE_URL`, so `headroom wrap claude` can make the menu disappear even though normal API requests still route through Headroom. The reported proxy logs show no Remote Control registration, session bootstrap, or device-attestation request at all, which means the decision happens inside Claude before Headroom can forward anything. This change makes that client-side incompatibility explicit in Headroom's Claude launch flow, `headroom doctor`, and troubleshooting docs. API proxying and the existing `ENABLE_TOOL_SEARCH` compatibility shim stay unchanged; users who need Remote Control get a direct instruction to launch Claude without the Headroom proxy for that session. Closes #1601 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add a Claude-specific helper and warning text for the Remote Control custom-base incompatibility. - Surface that warning from `headroom wrap claude` when Claude is launched through `ANTHROPIC_BASE_URL`. - Add a separate `headroom doctor` warning for Claude Remote Control availability, while keeping Claude API-routing status independent. - Document the limitation and workaround next to the existing Claude custom-endpoint troubleshooting guidance. - Add focused regression tests for gated and non-gated Claude routing states, plus preservation coverage for `ENABLE_TOOL_SEARCH`. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_issue_1601_remote_control_gate.py tests/test_cli_doctor.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py -q`) - [x] Unit tests pass (`uv run pytest tests/test_issue_746_tool_search.py tests/test_cli/test_init_enable_tool_search.py -q`) - [x] Linting passes (`uv run ruff check headroom/cli/wrap.py tests/test_cli_doctor.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py`) - [x] Formatting passes (`uv run ruff format --check headroom/cli/wrap.py tests/test_cli_doctor.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for the bugfix - [ ] Manual testing performed ### Test Output ```text rtk uv run pytest tests/test_issue_1601_remote_control_gate.py tests/test_cli_doctor.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py -q ============================= test session starts ============================= collected 62 items 62 passed, 1 warning rtk uv run pytest tests/test_issue_746_tool_search.py tests/test_cli/test_init_enable_tool_search.py -q ============================= test session starts ============================= collected 33 items 33 passed, 1 warning rtk uv run ruff check headroom/cli/wrap.py tests/test_cli_doctor.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py All checks passed! rtk uv run ruff format --check headroom/cli/wrap.py tests/test_cli_doctor.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py 3 files already formatted ``` ## Real Behavior Proof - Environment: Windows, Python via `uv`, focused Claude CLI and doctor tests. - Exact command / steps: with Claude settings or shell environment containing `ANTHROPIC_BASE_URL=http://127.0.0.1:8787`, run the focused helper and doctor tests, then run the existing `ENABLE_TOOL_SEARCH` preservation tests. - Observed result: Headroom surfaces a Claude Remote Control warning for custom `ANTHROPIC_BASE_URL`, while Claude API routing and `ENABLE_TOOL_SEARCH` behavior stay intact. - Not tested: live Claude Remote Control UI automation. The issue evidence says Claude hides the menu before any request reaches Headroom, so this PR proves Headroom's launch, diagnostics, and docs behavior. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` stays untouched because this repo's release pipeline generates changelog entries from conventional commits. This is a visibility fix, not a proxy transport restore. The issue evidence shows Claude never sends a Remote Control request while the custom-base gate is active, so the surviving slice is launch-time warning, doctor warning, and documentation. PR `#1600` is adjacent and non-blocking because `#1601` reproduces from process-env `ANTHROPIC_BASE_URL` alone. This intentionally changes `headroom doctor` for fully routed Claude sessions from an all-pass result to one warnings-only result, because the proxied Claude setup is operational for API traffic but still incompatible with Remote Control. |