mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
1553 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
TenderDeve
|
85e8699451
|
fix(learn): keep traceback tail in tool-error digest preview (#2596)
## Description `_format_tool_call` in `headroom/learn/analyzer.py` built the error preview with a head-only slice — `tc.output[:200]`. For tracebacks the root cause (`ExceptionType: message`) is at the **tail**, so the digest showed only `Traceback (most recent call last):` plus the first frame and dropped the actual diagnosis. The issue reports 46% of 715 measured errors were truncated past the 200-char head. Closes #2590 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `_truncate_head_tail()` helper that collapses newlines and, when over budget, keeps both the head and the tail joined by `…`. - `_format_tool_call` now uses it for error output so the exception line survives truncation. Short errors are returned unchanged (no marker). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ uv run pytest tests/test_learn/test_analyzer.py::TestDigestBuilder -q 9 passed in 1.47s $ uv run ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! ``` ## Real Behavior Proof - Environment: headroom @ main, Python 3.14, uv - Exact command / steps: added a long synthetic traceback (`KeyError: 'the-actual-root-cause'` at the tail) as a failing tool call and built the digest. - Observed result: digest now contains both `Traceback` and `KeyError: 'the-actual-root-cause'`, separated by `…`; short errors have no `…`. - Not tested: mypy not run locally. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Truncation budget stays at 200 chars (now split head/tail). mypy not run locally; happy to adjust if CI flags anything. |
||
|
TenderDeve
|
18e1c3c9ba
|
fix(compression): report source-line span in CCR compression marker (#2597)
## Description The compression marker read `[N items compressed to M. Retrieve more: hash=...]`, where `items` counts whitespace-split **words**, not lines. So five lines of tool output could show as `[122 items compressed to 27...]`. A reader can't map "items" to lines and can't tell "this line was compressed away" from "this line was never in the output" — absence reads as evidence of absence, which per the report led to a materially wrong conclusion. Closes #2586 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Annotate the marker with the source line count — `[N items compressed to M (from L source lines). Retrieve more: hash=...]` — at both marker sites: `KompressCompressor.compress` / `compress_batch` (`kompress_compressor.py`) and the remote path (`kompress_remote.py`). - The machine-parsed `Retrieve more: hash=` token is left byte-for-byte unchanged, so CCR detection/retrieval is unaffected. Scope note: I intentionally kept the existing `items compressed to` phrasing rather than reword the unit, to avoid churning the marker format that's referenced across ~12 test fixtures and the `config.py` template. This is the minimal honesty fix; happy to go further (e.g. line-unit counts or unifying with the `config.py` template) if you'd prefer — see the issue thread where I asked about wording. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ uv run pytest tests/test_compression_units.py tests/test_compression_batches.py \ tests/test_ccr_marker_policy.py tests/test_ccr_tool_injection.py tests/test_session_probes.py -q 92 passed $ uv run pytest tests/test_ccr_marker_policy.py -q 8 passed # incl. new test_source_line_span_marker_is_still_detected $ uv run ruff check headroom/transforms/kompress_compressor.py headroom/transforms/kompress_remote.py tests/test_ccr_marker_policy.py All checks passed! ``` ## Real Behavior Proof - Environment: headroom @ main, Python 3.14, uv - Exact command / steps: added a marker in the new enriched format and ran it through the CCR marker detector. - Observed result: the retrieval hash is still detected from `[122 items compressed to 27 (from 5 source lines). Retrieve more: hash=...]`; existing compression/CCR suites unchanged. - Not tested: mypy not run locally; the full model-backed compress() marker path isn't unit-exercised (needs a real backend), so the new test targets the parser boundary instead. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Wording is adjustable per the issue discussion. The `config.py` marker template (a different code path with `Omitted`/`Expires` fields) is left untouched to keep this focused on the Kompress marker the report hit. |
||
|
gglucass
|
f54f04f5bf
|
feat(opencode): ship the transport plugin in pip installs (#2601)
## Description
The OpenCode transport plugin - the piece that gives `wrap opencode`
all-provider routing by tagging each request with `x-headroom-base-url`
- only exists in repo checkouts today. `headroom_opencode_plugin_path()`
resolves `plugins/opencode/dist/entry.opencode.js`, which pip wheels do
not ship, so every pip install silently degrades to the two-provider
(anthropic/openai) baseURL fallback. The function's own docstring
documents the gap ("a pip-only install that does not ship `plugins/`").
Shipping the existing build output is not enough: the regular tsup build
leaves `headroom-ai` and `@opencode-ai/plugin` as bare external imports,
which only resolve next to the checkout's `node_modules`. Copied into
site-packages, the file fails to load. This PR ships a self-contained
bundle inside the wheel instead.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `plugins/opencode/tsup.standalone.config.ts` + `npm run
build:standalone`: a second build of the loader entry with `noExternal:
[/.*/]` and `splitting: false` - a single self-contained file whose only
imports are node builtins.
- `headroom/providers/opencode/_dist/entry.opencode.js`: the committed
standalone bundle (452 KB). It sits inside the package directory, so
maturin's `python-source = "."` packaging picks it up into the wheel
with no build-system changes.
- `headroom_opencode_plugin_path()`: falls back to the packaged bundle.
Precedence otherwise unchanged: `HEADROOM_OPENCODE_PLUGIN_PATH` env
override, then a repo-checkout build (fresher during development), then
the packaged bundle.
- CI (`opencode-plugin.yml`): rebuilds the standalone bundle and fails
the run if the committed artifact drifted from source, with a one-line
fix instruction; workflow path triggers extended to
`headroom/providers/opencode/_dist/**`.
- `tests/test_providers_opencode_plugin_path.py`: packaged bundle exists
and is self-contained (no bare npm imports), env override wins, fallback
resolution order.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_providers_opencode_plugin_path.py \
tests/test_providers_opencode_config.py tests/test_providers_opencode_install.py
============================== 49 passed in 0.31s ==============================
$ uvx ruff check headroom/providers/opencode/runtime.py tests/test_providers_opencode_plugin_path.py
All checks passed!
$ uvx ruff format --check headroom/providers/opencode/runtime.py tests/test_providers_opencode_plugin_path.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/providers/opencode/runtime.py
Success: no issues found in 1 source file
$ cd plugins/opencode && npm run build:standalone
ESM dist-standalone/entry.opencode.js 452.28 KB
ESM Build success in 28ms
```
## Real Behavior Proof
- Environment: macOS 15 (arm64), opencode 1.18.5 (Homebrew), node 22 /
npm 10, isolated `XDG_*` dirs so no real user config was touched.
- Exact command / steps:
1. `npm run build:standalone` in `plugins/opencode`.
2. Started a local header-logging HTTP listener on `127.0.0.1:9977`
(stands in for the proxy; logs method, path, headers, returns 401).
3. Registered the standalone bundle by absolute path in a scratch
`opencode.json` (`"plugin":
["<abs>/dist-standalone/entry.opencode.js"]`) with a `google` provider
entry and a fake API key. Note: the bundle's directory has **no**
`node_modules` - this is exactly the site-packages situation.
4. `HEADROOM_PROXY_URL=http://127.0.0.1:9977 opencode run -m
google/gemini-2.5-flash "say hi"`.
- Observed result: the listener received `POST
/v1beta/models/gemini-2.5-flash:streamGenerateContent?alt=sse` with
`User-Agent: opencode/1.18.5 ...` - i.e. the plugin loaded standalone
and rerouted a provider that the baseURL fallback cannot cover (native
Gemini wire format) to the proxy URL from `HEADROOM_PROXY_URL`.
- Not tested: Windows path resolution (pure `pathlib`, no platform
branches); wheel-build byte-determinism of the tsup output across OSes
(the CI drift check will surface it on the first divergent build).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
Not applicable - CLI/packaging change.
## Additional Notes
- Documentation checklist item: unchecked because the only doc surface I
found is the `headroom_opencode_plugin_path()` docstring, which this PR
rewrites to describe the three-step resolution order. Happy to add a
line to `docs/content/docs/` if there is a preferred page.
- A committed build artifact is not free: the CI drift check keeps it
honest, and the byte-compare relies on tsup/esbuild determinism under
`npm ci` (pinned lockfile). If you'd rather avoid the committed artifact
entirely, the alternative is publishing `headroom-opencode` to npm (its
`package.json` is publish-ready) and registering the plugin by package
name - happy to rework in that direction; the wheel-bundled path has the
advantage of version-locking the plugin to the backend it ships with.
- Downstream motivation: Headroom Desktop manages a long-lived shared
proxy (no `wrap` launcher) and wants to register this plugin from the
installed wheel path so OpenCode users get all-provider routing there
too.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Rod Boev
|
f74d874777
|
fix(learn): detect the active OpenCode database (#2587)
## Description `headroom learn --agent opencode` can silently mine a frozen conversation corpus. `OpenCodePlugin` hardcodes `~/.local/share/opencode/opencode.db`, but source-built OpenCode writes `opencode-local.db` in the same directory. When both files exist, learn still succeeds against the stale packaged DB and ignores the live source-built corpus. This follows the report in https://github.com/headroomlabs-ai/headroom/issues/2581 and builds on the existing OpenCode learn path introduced in https://github.com/headroomlabs-ai/headroom/pull/559. This change keeps explicit constructor paths authoritative, honors `HEADROOM_OPENCODE_DB` when it is set, and otherwise selects the newest existing database between `opencode.db` and `opencode-local.db`, preferring canonical `opencode.db` on exact ties. It also updates the OpenCode learn docs line so the documented behavior matches the landed resolver. Closes #2581. The branch also carries one narrow CI repair requested during review: `headroom/cli/wrap.py` now binds the `unwrap claude` Click command back to `unwrap_claude` instead of the leak-warning helper, which restores the existing unwrap test surface and leaves the helper as an internal warning function. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - add a private OpenCode DB resolver in `headroom/learn/plugins/opencode.py` with precedence `db_path` then `HEADROOM_OPENCODE_DB` then newest existing default filename then canonical fallback - preserve canonical `opencode.db` for exact mtime ties and for canonical-only installs - add focused regression coverage for newer-local, explicit-path, canonical-only, equal-tie, missing-override, and end-to-end scanning cases - sync the OpenCode learn docs paragraph so it no longer claims `opencode.db` is the only supported default path - restore the `unwrap claude` Click command binding in `headroom/cli/wrap.py` and apply the repo formatter so the branch passes the existing unwrap test and lint gates ## Testing - [x] Unit tests pass (`uv run pytest tests/test_learn/test_opencode_scanner.py -q`) - [x] Linting passes (`uv run ruff check headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py`) - [x] Type checking passes (`uv run mypy headroom/learn/plugins/opencode.py`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_learn/test_opencode_scanner.py -q -k "newer_local_database" 1 passed, 9 deselected in 0.26s uv run pytest tests/test_learn/test_opencode_scanner.py -q 10 passed in 0.50s uv run ruff check headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py All checks passed! uv run ruff format headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py --check 2 files already formatted uv run mypy headroom/learn/plugins/opencode.py Success: no issues found in 1 source file rg -n "opencode-local\.db|HEADROOM_OPENCODE_DB|opencode\.db" docs/content/docs/opencode.mdx 78:`headroom learn` supports OpenCode as a scan target. It reads past sessions from the newer of `~/.local/share/opencode/opencode-local.db` and `~/.local/share/opencode/opencode.db`, or from `HEADROOM_OPENCODE_DB` when you set an explicit override, and writes corrections to your project's `AGENTS.md`. uv run pytest tests/test_cli/test_unwrap_claude.py -q -k "removes_mcp_rtk_and_stops_proxy or preserves_user_managed_serena or removes_headroom_installed_serena or keep_flags_skip_cleanup or restores_all_base_url_modes or stops_claude_owned_persistent_deployment or reports_ambiguous_same_port_persistent_deployment or warns_about_same_port_inherited_env or ignores_malformed_inherited_env_port" 9 passed, 5 deselected in 0.40s uv run ruff check . All checks passed! uv run ruff format --check . 1340 files already formatted ``` ## Real Behavior Proof - Environment: temporary SQLite databases exercised through the production `OpenCodePlugin()` constructor - Exact command / steps: run `uv run pytest tests/test_learn/test_opencode_scanner.py -q -k "newer_local_database"` against `origin/main` with the new regression test overlaid, then run the same command and the full `uv run pytest tests/test_learn/test_opencode_scanner.py -q` suite on the branch head - Observed result: the base reproduction fails with `AssertionError: assert 'Canonical' == 'Local'`, proving current main still selects the stale canonical DB; the branch head passes the reproduction row and the full 10-test scanner suite - Not tested: live user OpenCode corpus ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - `CHANGELOG.md` stays untouched because Headroom generates release notes from conventional commits. - The automatic chooser is intentionally limited to the two known default filenames, `opencode.db` and `opencode-local.db`. Other layouts can use `HEADROOM_OPENCODE_DB`. - The fix stays inside `headroom/learn/plugins/opencode.py`; no provider-neutral learn or pipeline code changes are planned. |
||
|
Eyal Mizrachi
|
0994ea04c8
|
fix(wrap): skip Serena project setup outside real project roots (#2574)
## Problem `headroom wrap` runs two per-project Serena steps against the cwd: `_scope_serena_languages()` (detect languages, pin them into `.serena/project.yml`) and `_index_serena_project()` (`serena project index`, to warm the symbol cache). Both assume the cwd *is* a project. Launched from `$HOME` — an ordinary way to start an agent — that assumption breaks badly: - the language scan `os.walk`s the entire home directory: `Downloads/`, VM images, backup trees, network mounts; - the pre-index then runs `serena project index` over the same tree and sits there until its full 300s timeout; - so the agent appears to **hang for minutes on every launch**, with no output after the Serena MCP registration line and nothing to suggest indexing is what's blocking; - and the scan writes `project.yml` into `~/.serena`, which is Serena's own config directory rather than a project's `.serena/`. A linked git worktree hits the same code from the other side: it's an ephemeral checkout, so it pays for a full cold index at a path that soon disappears — once per worktree, which adds up under any fan-out workflow. ## Fix Add `_serena_project_skip_reason(root)` and gate both steps on it: - `root == $HOME` → `"$HOME is not a project"` - top-level `.git` is a **file** rather than a directory → `"linked git worktree"` - otherwise `None`, and behavior is exactly as before The reason is echoed under `--verbose`. Nothing else changes: Serena MCP is still registered, instructions are still injected, and in the skipped cases Serena still indexes lazily on demand — so no capability is lost, only the wasted upfront scan. ## Testing Five unit tests in `tests/test_cli/test_wrap_serena_boost.py` covering an ordinary directory, a normal checkout (`.git` dir), `$HOME`, a linked worktree (`.git` file), and a non-existent root. Full file: 22 passed. `ruff format --check` and `ruff check` clean. Verified manually on the reported case: `claude` launched from `$HOME` now starts immediately instead of stalling on the index. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Abhay Singh
|
2a63ec70b6
|
fix(image): reuse image models instead of rebuilding them per request (#2513) (#2536)
## Description Fixes #2513. Image compression rebuilt its heavyweight models on every request: - `_compress_messages_worker` (`proxy/image_isolation.py`) created a new `ImageCompressor()` per call, and - `ImageCompressor.compress` (`image/compressor.py`) created a new `OnnxTechniqueRouter(use_siglip=...)` per image. Each `OnnxTechniqueRouter` loads native `ort.InferenceSession` models, and ONNX Runtime holds C++ memory that Python's GC does not eagerly reclaim. The image pool is a **persistent** single-worker `ProcessPoolExecutor`, so those sessions accumulated in the worker and RSS grew ~70 KB/request, reaching ~1.1 GB after ~15k image requests in a day (the log shows one `[RapidOCR] Using engine_name: onnxruntime` line per request, confirming reloads). ## Fix Load the models once and reuse them: - `ImageCompressor` caches the ONNX router on `self._onnx_router` (built lazily via `_get_onnx_router`) instead of building one per `compress()` call. - The isolation worker keeps a per-process `ImageCompressor` singleton (`_get_worker_compressor`) and reuses it across calls. - `_get_image_compressor()` (main process, used for the `has_images()` gate) returns a shared instance too. - Shared instances are marked `_is_singleton`, and `close()` is a no-op on them, so a caller's per-request `close()` no longer unloads the models the next request reuses. A non-singleton `close()` still releases the torch router and drops the cached ONNX router. RSS is now flat after the initial model load; behavior is otherwise unchanged. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/image/compressor.py`: add `_onnx_router` cache + `_get_onnx_router`, use it in `compress()`, add the `_is_singleton` flag, and make `close()` a no-op on a singleton (drop the cached ONNX router on a real close). - `headroom/proxy/image_isolation.py`: reuse a per-worker `ImageCompressor` singleton in `_compress_messages_worker` instead of building/closing one per call. - `headroom/proxy/helpers.py`: `_get_image_compressor()` returns a shared singleton instance. - `tests/test_image_compressor_singleton_reuse.py` (new): the ONNX router is built once and cached, singleton `close()` is a no-op while non-singleton `close()` releases, and both `_get_image_compressor` and the worker helper return a shared singleton. - `tests/test_proxy_handler_helpers.py`: updated the two existing `_get_image_compressor` tests that pinned the old fresh-per-call behavior to assert the singleton reuse instead (and reset the new module global so they stay isolated). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_image_compressor_singleton_reuse.py -q 5 passed # with the fix reverted, all five fail (router rebuilt per call, close() # unloads the shared models, helpers return fresh instances) $ uvx ruff@0.15.17 check headroom/image/compressor.py headroom/proxy/image_isolation.py headroom/proxy/helpers.py tests/test_image_compressor_singleton_reuse.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/image/compressor.py headroom/proxy/image_isolation.py headroom/proxy/helpers.py Success: no issues found in 3 source files ``` The pre-existing async tests in `tests/test_image_compression_isolation.py` (4 `@pytest.mark.asyncio` cases) fail identically on clean `main` in this environment because pytest-asyncio is not configured here; they are unrelated to this change and pass in CI. ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: with `OnnxTechniqueRouter` construction mocked, called `ImageCompressor._get_onnx_router()` twice and asserted a single construction; exercised `close()` on singleton vs non-singleton instances; and called `_get_image_compressor()` / `_get_worker_compressor()` twice each. Then reverted the three source files and re-ran. - Observed result: with the fix the ONNX router is constructed once and reused, singleton `close()` leaves `_router`/`_onnx_router` intact (no `release_models`), non-singleton `close()` releases and nulls them, and both helper accessors return the same `_is_singleton` instance; with the fix reverted every one of these fails (fresh construction / unconditional release / new instances). Ran against the actual modules. - Not tested: a live multi-hour image workload measuring RSS (the leak is inferred from the removed per-request model construction; the ONNX/torch model load itself is mocked here). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Tejas Chopra
|
b121223ec9
|
fix(install): default to cache mode, matching headroom proxy (#1893 follow-up) (#2563)
## Description `headroom install` and `headroom deploy` defaulted `--mode` to **token**, while `headroom proxy` and the server env default both resolve to **cache**. Because `install/planner.py:155` writes `"HEADROOM_MODE": proxy_mode` into the install base env, installing Headroom did not merely differ from running it directly — it **actively overrode** the good server default with the cache-busting one. | Entry point | Effective default | Where | |---|---|---| | `headroom proxy` | **cache** | `cli/proxy.py:1129` — `mode or HEADROOM_MODE or PROXY_MODE_CACHE` | | `proxy/server.py` env | **cache** | `server.py:4962`, commented *"delta-only compression at ~0 prefix-cache busts"* | | `headroom install` / `deploy` | **token** ❌ | `cli/install.py:455,615` | Cache mode freezes prior turns and compresses only the newest delta, so the cached prefix stays byte-identical. Token mode rewrites frozen history, which moves the bytes the provider hashed for its cache key and forces a full cold re-write of the entire prefix. Why that is expensive — measured on 35 local Claude Code sessions (23,018 turns, 8,985M prompt tokens): cache **writes** are ~46% of input spend from just 6.3% of tokens, and 714 warm turns that each re-wrote >100K tokens carried 83% of all warm-path write tokens (~26% of total input spend) at ~452K tokens per event. Full-prefix re-writes are the dominant cost in this workload, and token mode makes them more likely. **This is an oversight, not a deliberate divergence.** #1893 ("ship the coding profile as Headroom's out-of-box default posture") introduced the cache default but its diff touched only `agent_savings.py`, `cli/proxy.py`, and `proxy/server.py` — verified with `git show |
||
|
Parideboy
|
045f3dfe6f
|
fix(install): use CREATE_NO_WINDOW instead of DETACHED_PROCESS on Windows (#2527)
## Description On Windows, the detached agent process spawned by `install hook ensure` (and the `install restart` self-spawn) pops up a visible black console window repeatedly, because `DETACHED_PROCESS` makes `CREATE_NO_WINDOW` a no-op per the Win32 process-creation-flags docs. #2521 ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) ## Changes Made - `headroom/install/runtime.py`: `start_detached_agent()` now uses `CREATE_NO_WINDOW` instead of `DETACHED_PROCESS`, combined with `CREATE_NEW_PROCESS_GROUP` (unchanged detach/isolation semantics, window hidden). - `headroom/install/runtime.py`: `_spawn_detached_restart()` now also sets `CREATE_NO_WINDOW` on Windows (previously had no `creationflags` at all on that platform). - `tests/test_install/test_runtime.py`: updated the Windows branch of `test_start_detached_agent_and_run_foreground` to assert the actual `creationflags` value passed to `Popen`, instead of just monkeypatching an unused `DETACHED_PROCESS` attribute. ## Testing - [x] Added/updated tests - [x] Ran full local test suite ``` $ python -m pytest tests/test_install -q ======================= 137 passed, 1 skipped in 48.68s ======================= $ ruff check headroom/install/runtime.py tests/test_install/test_runtime.py All checks passed! $ ruff format --check headroom/install/runtime.py tests/test_install/test_runtime.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 506 source files ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.11, headroom repo local checkout - Exact command / steps: `python -m pytest tests/test_install/test_runtime.py -q`, plus manual read of `subprocess` Windows creation-flag semantics (`DETACHED_PROCESS` + child console allocation vs `CREATE_NO_WINDOW`) - Observed result: all 25 tests in `test_runtime.py` pass, including the updated assertion that `creationflags == CREATE_NO_WINDOW | CREATE_NEW_PROCESS_GROUP` on the Windows code path - Not tested: did not reproduce the original visible-console-popup repro end-to-end via live Claude Code hook invocation (no environment with the full hook-triggered respawn loop set up in this session); relying on the Win32 docs and the reporter's own local verification of the same flag swap ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
AxelRay
|
d50cfabedc
|
fix(proxy): report deferred Kompress status and promote health from cache (#2564)
## Description When Kompress preload is deferred until first request, startup still logs "not installed" even if ML deps are present. After the model later loads into the module cache, /readyz and /health can keep reporting kompress as unhealthy because reconcile only inspected attached compressor instances. This PR reports deferred startup accurately and promotes health from the live module cache once the model is ready, without starting loads from health checks. Closes #2560 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Treat eager-status `deferred` as installed-but-deferred at proxy startup and log that state instead of "not installed". - Promote `/readyz` and `/health` Kompress readiness from the module-level model cache when attached compressors are missing or not ready. - Keep health inspection free of lazy getters and download side effects. - Add regressions for deferred startup logging and cache-based health promotion. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ PYTHONPATH=/tmp/headroom-2561 python -m pytest tests/test_proxy_health.py tests/test_proxy_eager_preload_bind.py -q -o addopts= 21 passed, 1 warning in 2.73s $ ruff format --check headroom/proxy/server.py tests/test_proxy_health.py tests/test_proxy_eager_preload_bind.py 3 files already formatted $ ruff check headroom/proxy/server.py tests/test_proxy_health.py tests/test_proxy_eager_preload_bind.py All checks passed! ``` ## Real Behavior Proof - Environment: Linux VPS, Python 3.11 venv with headroom-ai 0.32.1 wheel for `_core`, checked out main + this branch overlayed for source under test - Exact command / steps: `PYTHONPATH=/tmp/headroom-2561 python -m pytest tests/test_proxy_health.py tests/test_proxy_eager_preload_bind.py -q -o addopts=`; `ruff format --check` and `ruff check` on the three changed files - Observed result: 21 focused tests passed, including deferred startup log regression and module-cache health promotion; ruff format/check clean - Not tested: live multi-request proxy with real ONNX model download on this host; install-status follow-up mentioned in the issue comment ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Scoped to Kompress status reporting only. The separate `headroom install status` ownership probe in the issue comment is left for a follow-up. Co-authored-by: axelray-dev <axelray-dev@users.noreply.github.com> |
||
|
AxelRay
|
4bd121493d
|
fix(proxy): allow request_scope import without fastapi (#2562)
## Description Base installs without the `proxy` extra crash during CLI command registration because `headroom.proxy.request_scope` imported FastAPI at module import time. That import is only needed for typing on `normalize_request_path`. This change keeps the FastAPI `Request` import under `TYPE_CHECKING` so the CLI path used by `headroom --help` no longer requires FastAPI. Closes #2561 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation update - [ ] Refactoring (no functional changes) - [ ] Performance improvement - [ ] Test update - [ ] Build/CI change - [ ] Other (please describe): ## Changes Made - Make the FastAPI `Request` import type-checking only in `headroom/proxy/request_scope.py` - Add a subprocess regression test that imports `request_scope` and `project_context` with FastAPI blocked and verifies `normalize_scope_path` ## Testing ### Test commands run ```bash PYTHONPATH=. python3 -m pytest tests/test_proxy_request_scope.py tests/test_request_scope_no_fastapi.py -q ruff format --check headroom/proxy/request_scope.py tests/test_request_scope_no_fastapi.py ruff check headroom/proxy/request_scope.py tests/test_request_scope_no_fastapi.py ``` ### Test Output ```text ========================= 5 passed, 1 warning in 0.93s ========================= 2 files already formatted All checks passed! ``` ## Real Behavior Proof ### Environment - Linux x86_64, Python 3.11.15 - Shallow sparse checkout of headroom main at commit parent of this PR - System/Hermes venv Python with pytest and ruff available ### Exact command ```bash PYTHONPATH=. python3 - <<'PY' import builtins, sys real = builtins.__import__ def imp(name, *a, **k): if name == "fastapi" or name.startswith("fastapi."): raise ModuleNotFoundError("No module named 'fastapi'") return real(name, *a, **k) builtins.__import__ = imp import headroom.proxy.request_scope as rs import headroom.proxy.project_context as pc rs.normalize_scope_path({"path": "/a"}, "/b") print("ok", "fastapi" not in sys.modules, hasattr(pc, "with_project_prefix")) PY ``` ### Observed result ```text ok True True ``` Importing the request-scope helpers no longer requires FastAPI, and scope path normalization still works. ### Not tested - Full base `pip install headroom-ai` (no extras) end-to-end on a clean venv without the monorepo source tree - Full monorepo `make ci-precheck` / cargo workspace - Live proxy traffic or FastAPI request path behavior beyond the existing unit test for `normalize_request_path` ## Review Readiness - [x] I have tested these changes locally - [x] I have added/updated tests where applicable - [x] I have updated documentation if needed (N/A) - [x] My code follows the project's style guidelines - [x] I have run linting/formatting checks - [x] I have considered security implications - [x] This PR is ready for review |
||
|
Tejas Chopra
|
a6d4921e82
|
feat(proxy/hooks): run fold-only (stream-safe) turn hooks on streaming OpenAI chat (#2549)
## Description Fixes the last harness gap in the turn-hook seam (the "B4" finding from the savings audit). The OpenAI chat handler gated hooks on `not stream`, so **streamed** `/v1/chat/completions` requests ran **no** turn hooks — the lossless-guard plugin's on_request fold and tool-schema shrink were skipped, unlike the Anthropic path (hooks run unconditionally). Affects opencode / Cursor / older OpenAI SDKs / some Copilot flows; **not** Claude Code (Anthropic path). The gate existed for a real reason: hooks that **re-drive** the model in `on_response` (defer a tool, reload it when asked) can't run mid-stream. But an **on_request fold** mutates the outbound request before the send — safe on a stream. ## Change - Add an opt-in `stream_safe` hook attribute (fold-only hooks set it). `run_request_hooks(ctx, stream_safe_only=…)` filters to stream-safe hooks when set. - OpenAI chat handler runs `on_request` on streaming with `stream_safe_only=stream`; buffered runs all hooks; the `on_response` re-drive (buffered response path) is untouched. - **Default off = conservative:** a hook is buffered-only unless it declares `stream_safe`, so **no behavior change** until a hook opts in. ## Type of Change - [x] Bug fix / feature (opt-in, backward-compatible) ## Testing ```text pytest tests/test_turn_hooks.py tests/test_openai_chat_turn_hooks.py -q → 25 passed ruff + mypy → clean ``` New test pins the filter: streaming runs only stream-safe hooks' on_request; buffered runs all. ## Notes The companion plugin PR (headroom-lossless-guard) sets `stream_safe = True` on its fold-only hook to actually claim the streaming savings. Anthropic path already ran hooks on streaming, so it's unaffected. ## Checklist - [x] Self-reviewed; tests pass; no CHANGELOG edit |
||
|
Tejas Chopra
|
c990cfb803
|
feat(wrap): reduce-at-source — SAFE quiet-CLI env defaults for the launched agent (#2548)
## Description Reduce-at-source, done **safely** in the wrap layer (not by rewriting commands in-flight): `headroom wrap` injects conservative quiet-CLI env defaults into the launched agent's environment so tools emit less noise at the source (which the proxy would otherwise strip post-hoc). Injected only when the user hasn't set them: `GIT_PAGER=cat`, `PIP_QUIET=1`, `PIP_DISABLE_PIP_VERSION_CHECK=1`, `npm_config_fund/audit/progress=false`; `PYTEST_ADDOPTS` **augmented** with `-q` (existing value preserved). Single chokepoint (`_launch_tool`), so it covers all wrapped tools. Opt out with `HEADROOM_WRAP_QUIET=0`. Closes # ## Type of Change - [x] Performance improvement / [x] New feature (opt-out) ## Safety Nothing that can suppress diffs, errors, summaries, or search results — no blanket `--silent`/`--quiet`. User-set values always win. ## Testing ```text pytest tests/test_wrap_quiet_cli.py → 5 passed (defaults injected; user value wins; PYTEST_ADDOPTS augmented; opt-out; on-by-default) ruff + mypy → clean ``` ## Scope note (honesty) A JSONL analysis of real Claude Code traffic shows this is a **modest** lever for that workload: non-TTY git already disables the pager (so `GIT_PAGER` is largely a no-op there), and pip/npm are low-traffic; `PYTEST_ADDOPTS=-q` is the clearest win. It's harmless and captures modest savings where those tools *are* used — the larger levers are post-output (the lossless-guard lossy tier) and the grep fold. ## Checklist - [x] Self-reviewed; tests pass; no CHANGELOG edit |
||
|
Tejas Chopra
|
7dc9a978ca
|
feat(lossless): factor shared directory prefix in the grep search fold (#2547)
## Description
The lossless search fold (`search_heading`) factors a repeated **file**
(many matches in one file → path once + `line:content` rows), but `grep
-rn` across many **distinct** files has one match each, so it saved ~0%
— the shared directory repeated on every row. This adds
`search_dir_heading`/`search_dir_unheading`, which factor the shared
**directory** across distinct files (dir once as a header,
`base:line:content` beneath). `compact_lossless('search')` now tries
both folds and keeps the smallest that round-trips exactly.
Matters because grep is ~23.5% of observed agent output tokens.
Closes #
## Type of Change
- [x] Performance improvement (lossless)
## Changes / Behavior
- File fold wins many-matches-one-file; dir fold wins the `grep -rn`
case (0% → ~16-40% depending on path depth / match length).
**Byte-lossless** — round-trip verified, fold discarded on any mismatch.
- Never touches source reads / diffs (unchanged class gating).
## Testing
```text
pytest tests/test_bash_search_lossless_fold.py -q → 30 passed
pytest test_lossless_excluded_compaction / _then_lossy / _mode → 72 passed
ruff + mypy → clean
```
Round-trip verified on: distinct-files (sorted), many-matches-one-file,
mixed+passthrough, colon-in-content.
## Note for reviewers
The dir-grouped output is byte-lossless but a slightly **non-standard**
format the model reads directly (`dir/` header + `base:line:content`) —
like the existing `rg --heading` fold but less standard. Low
comprehension risk; flagging it explicitly. If preferred, we can gate it
to only fire above a larger savings threshold.
## Checklist
- [x] Self-reviewed; tests pass; no CHANGELOG edit
|
||
|
Tejas Chopra
|
9f1ffefe83
|
feat(proxy/savings): aggregate tool-schema savings into Metrics + all reporting sinks (#2546)
## Description Companion to #2545 (the "sources" double-count fix) — this fixes the "sinks" half found in the same savings audit: **tool-schema / deferral savings were never aggregated into `Metrics`**. They lived only in per-request log tags, so every sink that reads `metrics.*` silently dropped them, and one CLI mode disagreed with another. Confirmed sinks that under-reported: - **Session-summary printout** — `Tokens saved:` is message-only; a 24K-tool-deferral turn printed `0`. - **`cost.py` session summary** (feeds `/stats.summary`) — `total_tokens_saved_with_rtk` etc. were message+CLI only. - **`/stats` `all_layers_tokens_saved`** — the advertised "total" excluded the `tool_search` layer it enumerates in `by_layer`. - **`headroom perf --format json/csv`** — omitted `tool_saved` while the **text** output of the same command showed it. Closes # ## Type of Change - [x] Bug fix (non-breaking) / observability correctness ## Changes Made - `PrometheusMetrics.tool_search_saved_total` — new counter, accumulated in `record_request` from a new `tool_search_saved` arg; `emit_request_outcome` fills it from the `tool_search_deferred_tokens` + `turn_hook_tools_saved_tokens` tags. **One source of truth.** - Fed into: session summary (`Tool schemas deferred:` line), `cost.py` summary (new `tool_schema_tokens_saved` + `total_tokens_saved_all_layers`; existing fields unchanged for back-compat), `/stats` `all_layers` total, and `build_perf_summary` (`tool_saved`). - Kept **distinct** from `tokens_saved_total` (message compression) — tool bytes never move `tok_before/after`, so it's a separate layer, not a merge (no double-count). ## Testing - [x] `ruff` + `ruff format --check` + `mypy` clean - [x] Regression tests + existing suites pass ### Test Output ```text pytest tests/test_savings_tool_search_aggregation.py tests/test_cli_perf_format.py -q → 18 passed pytest tests/test_cli_perf_format.py test_proxy_savings_history.py test_dashboard_token_savings.py test_bundled_tools_savings.py test_openai_chat_turn_hooks.py → 68 passed, 2 skipped mypy (metrics/outcome/cost/analyzer) → clean ``` ## Real Behavior Proof - Standalone: `record_request(tool_search_saved=1500)` then `(…=800)` → `metrics.tool_search_saved_total == 2300`, `tokens_saved_total == 200` (message stays separate); `build_perf_summary` over records with `tool_saved` 5000+3000 → `tool_saved == 8000`. ## Checklist - [x] Self-reviewed; no new warnings; tests pass; did **not** edit `CHANGELOG.md` ## Additional Notes Together, #2545 (record once) + this (surface every layer) make savings correct **and** complete end-to-end across `/stats`, the dashboard, `headroom perf`, the session summary, and cost/budget. The `/stats` `by_layer.tool_search` and dashboard card already showed the layer (windowed, from the log scan); this makes the lifetime/metrics-based sinks agree. |
||
|
Tejas Chopra
|
0845b26ee6
|
fix(proxy/cost): record each request's savings exactly once (drop 3 double-counts) (#2545)
## Description An audit of savings accounting found three **double-count** bugs: the P0 outcome-funnel refactor centralized cost + PERF recording in `emit_request_outcome`, but three pre-funnel emits were never removed, so they fire a second time on their paths. | Path | Stray emit | + Funnel | Effect | |---|---|---|---| | OpenAI chat direct, non-streaming | explicit `cost_tracker.record_tokens` (`handlers/openai.py` ~4140) | `outcome.py:418` | **2× spend / requests; budget period cost doubled** → `check_budget` can block at half the real spend | | OpenAI **Responses** buffered (Codex HTTP) | explicit `record_tokens` (~5223) | `outcome.py:418` | same | | Codex **WS** turns | explicit `PERF` log line (~7291) | `outcome.py:482` | `headroom perf` **double-counts** saved + requests every WS turn (analyzer sums per line, no dedup by request_id) | All three are pure duplicates: the funnel's `cost_tracker.record_tokens` is a **superset** of the explicit calls' args, and its PERF line uses the **same per-turn deltas** (verified: `7246-7249` == the explicit line's fields). The `/stats` headline was already correct (SavingsTracker fires once, inside the funnel) — only cost/budget and `headroom perf` were affected. Closes # ## Type of Change - [x] Bug fix (non-breaking) ## Changes Made - Remove the explicit `cost_tracker.record_tokens` on the OpenAI chat non-streaming path and the Responses buffered path — keep the `cache_write`/`uncached` computation the funnel needs. - Remove the duplicate WS PERF log line (+ its now-dead `_perf_*` locals and the now-unused `_summarize_transforms` import). - Add a regression test: cost is recorded exactly once on the non-streaming chat path (was 2×). ## Testing - [x] `ruff check` + `ruff format --check` clean; `mypy` clean - [x] Regression + existing tests pass ### Test Output ```text pytest tests/test_openai_chat_turn_hooks.py -q → 6 passed (incl. new double-count regression) pytest tests/test_openai_responses_context_compaction.py → 12 passed pytest tests/test_openai_codex_ws_lifecycle.py + timings + savings_deferral → 38 passed ruff/mypy → clean ``` ## Real Behavior Proof - **Verified by code trace**, not just tests: `grep cost_tracker.record_tokens` across the handler now returns only the funnel call (`outcome.py:418`); the explicit chat/Responses calls are gone. The WS funnel outcome (`openai.py:7246-7249`) feeds `outcome.py:482`'s PERF with the same deltas the deleted line used. - **Not covered:** a related finding (OpenAI-chat *streaming* skips turn hooks entirely, `openai.py:3484 "and not stream"`) is **intentionally deferred** — that gate protects re-drive-requiring hooks (tool-router deferral) which can't run mid-stream; a proper fix needs a per-hook "safe-on-stream" capability flag, out of scope here. ## Checklist - [x] Self-reviewed - [x] No new warnings; tests pass locally - [x] Did **not** edit `CHANGELOG.md` ## Additional Notes This is the "sources" half of the savings audit. A companion PR will fix the "sinks" half — tool-search/deferral savings are never aggregated into `Metrics`, so the session summary, `cost.py` summary, `headroom perf --json/csv`, and the `all_layers` total under-report them. |
||
|
Tejas Chopra
|
285176be54
|
fix(tokenizers): price Claude against a real BPE (tiktoken o200k) not a char estimate (#2543)
## Description
Claude's tokenizer is not public, so `get_tokenizer("claude-*")`
returned `EstimatingTokenCounter(chars_per_token=3.5)` — a
**character-ratio estimate**. The estimator's chars-per-token flips with
the *detected content type* (JSON 3.2 / code 3.5 / English 4.0 / CJK
1.5), so:
- the **same bytes** count differently before vs after compression → a
fold could appear to *increase* tokens;
- it **disagrees with the pipeline's** estimator on the same content.
Calibration vs a real BPE (tiktoken `o200k_base`) on representative
content:
| content | char-est(3.5) / o200k |
|---|---|
| English | **1.28×** (overcount 28%) |
| code | **0.83×** (undercount 17%) |
| JSON | 0.97× |
| diff | 1.02× |
i.e. the estimate is off **−17% … +28%**, and the error is
content-dependent (non-uniform) — which is exactly what produces
impossible `tok_after > tok_before` deltas.
This prices Claude against **tiktoken `o200k_base`** — a real,
deterministic, **monotone** BPE. It's not Claude's exact vocab, but it's
within ~10–20% and, crucially, **consistent before/after**, which is
what compression ratios and context-pressure gating actually need.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement
- [ ] Breaking change
- [ ] Documentation update
- [ ] Code refactoring (no functional changes)
## Changes Made
- `TiktokenCounter.__init__` accepts an explicit `encoding` override (so
a model can be priced against a chosen encoding regardless of name-based
resolution).
- `_create_anthropic` now returns `TiktokenCounter(model,
encoding="o200k_base")`, **failing open to the character estimator** if
the tiktoken vocab can't be loaded (reuses the existing `load_encoding`
timeout/guard).
- Updated the `MODEL_PATTERNS` comment and `test_get_anthropic_model` to
the new contract; added
`test_claude_priced_with_real_bpe_not_char_estimate`.
**Blast radius is contained:** `content_router` keeps its own
module-level estimator for routing decisions, so only the **handler's
reported counts** change. `tiktoken` is already a dependency.
**Composes with #2542** (tokenizer-consistent before/after): that PR
makes both endpoints use *one* tokenizer; this PR makes that one
tokenizer a *real BPE*. Together they fully eliminate the
inflated/phantom lines. This PR is based on `main` and is independently
valid.
## Calibration note (please review)
The one behavioral consumer of the handler's count is the
background-compression gate (`original_tokens >=
_background_compression_min_tokens`). Because o200k differs from the old
estimate by ~±20% depending on content, that threshold now trips on
slightly different requests. It's *more* accurate, but the threshold was
tuned against the estimator — worth a re-check. No other gate consumes
the handler count (routing uses `content_router`'s estimator).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check` + `ruff format --check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added
- [x] Manual testing performed
### Test Output
```text
$ ruff check <changed files> && ruff format --check <changed files>
All checks passed!
4 files already formatted
$ mypy headroom/tokenizers/registry.py headroom/tokenizers/tiktoken_counter.py
Success: no issues found in 2 source files
$ pytest tests/test_tokenizer.py tests/test_tokenizers.py -q
55 passed, 14 skipped
```
## Real Behavior Proof
- **Unit:** `get_tokenizer("claude-opus-4-8")` →
`TiktokenCounter(encoding_name="o200k_base")`; `count_text(sample)` ==
`tiktoken.get_encoding("o200k_base").encode(sample)` exactly; a
`tool_result` shrunk 300→3 words drops 508→13 tokens (folds register).
- **Live proxy** (Anthropic path, `--proxy-extension lossless_guard`):
foldable request logs `tok_before=694 tok_after=231 tok_saved=463
transforms=turn_hook` — real o200k-scale numbers (vs the estimator's 607
for the same payload), clean fold, no inflation.
- **Not tested:** exact agreement with Anthropic's *own* count_tokens
API (o200k is a proxy; a follow-up could calibrate against it offline).
GPT paths unchanged (already tiktoken).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] Documentation — N/A (internal; docstrings updated)
- [x] My changes generate no new warnings
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Follow-ups (not here): (1) extend the same real-BPE treatment to other
private-tokenizer providers (Gemini/Cohere/Moonshot), each with its own
calibration; (2) optional opt-in calibration against Anthropic's
`count_tokens` API to true-up the ~10–20% absolute offset.
|
||
|
Abhay Singh
|
fa4763761b
|
fix(proxy/cost): warn once per model when pricing lookup fails (#2504) (#2535)
## Description Fixes #2504. `CostTracker.estimate_cost` runs on the per-request cost path and logs a WARNING whenever LiteLLM can't price the model: ```python except Exception as e: logger.warning(f"Failed to get pricing for model {model}: {e}") return None ``` For a custom / OpenAI-compatible model LiteLLM can't resolve (e.g. `glm-5.2` via `--backend anyllm --anyllm-provider openai`), this fires on **every single request**, flooding `proxy.log` with hundreds of identical lines and burying genuinely useful warnings. The `LiteLLM not available` branch above it has the same per-request flooding shape. ## Fix Track already-warned models in a small module-level set and emit each pricing-failure warning (and the LiteLLM-unavailable warning) once per process. The set is bounded by the number of distinct model names seen. No new dependencies or config. The cost result itself is unchanged (`None` on failure); only the log volume changes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/cost.py`: add a module-level `_warned_pricing_models` set and `_warn_pricing_once` helper; route the pricing-failure and LiteLLM-unavailable warnings in `estimate_cost` through it. - `tests/test_cost_pricing_warning_dedup.py` (new): assert a repeated unresolvable model warns once, distinct models each warn once, and the LiteLLM-unavailable warning is deduped too. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cost_pricing_warning_dedup.py -q 3 passed # with the fix reverted, the module-level set does not exist, so the # dedup tests error/fail (the pre-fix code warned once per request) $ uvx ruff@0.15.17 check headroom/proxy/cost.py tests/test_cost_pricing_warning_dedup.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/cost.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: monkeypatched `_get_litellm_module` to a stub whose `cost_per_token` raises (and, separately, to `None`), called `CostTracker.estimate_cost("glm-5.2", ...)` five times and two distinct unresolvable models twice each, capturing `headroom.proxy` WARNING records with `caplog`. - Observed result: with the fix each model produces exactly one `Failed to get pricing for model ...` warning (and one `LiteLLM not available ...`) regardless of call count; the pre-fix code logged one per call. `estimate_cost` still returns `None` on failure. Ran against the actual module. - Not tested: a live multi-request session against a real unpriced model end to end. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Tejas Chopra
|
c371d5ad60
|
fix(proxy/perf): count turn-hook message folds in token accounting (#2520)
## Description
Turn hooks (the `headroom.proxy.turn_hooks` seam used by proxy
extensions, e.g. the lossless-guard plugin) fold tool_result / message
content in `on_request`, which runs **after** the pipeline has already
computed `optimized_tokens`. The saving was recorded to `/stats` via
`record_compression`, but was invisible to the `PERF` log line and
`headroom perf` (both read the pipeline's `original → optimized` delta).
Net effect: a plugin that folded 463 tokens still logged `tok_saved=0`.
This makes the per-turn token accounting count the hook's fold too,
across all three handler paths.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **Anthropic Messages handler** (`/v1/messages`): re-count messages
right after `run_request_hooks`, regardless of whether the hook replaced
the list or mutated it in place. Attribute the fold as a `turn_hook`
transform. Only ever lowers `optimized_tokens`.
- **OpenAI Chat handler** (`handle_openai_chat`,
`/v1/chat/completions`): same re-count. The existing code re-counted
hook-modified *tools* but not the *message* fold — this closes that gap
and adds the `turn_hook` transform tag.
- **OpenAI Responses handler** (`_compress_openai_responses_payload`,
`/v1/responses`): the seam previously only wrote hook-modified *tools*
back — a folded/replaced `input` list was silently dropped and
uncounted. Now snapshot the message-items token count **before** the
hook (an in-place fold would corrupt a post-hook baseline), write back a
replaced list, and add the fold delta to `tokens_saved` (the same
channel the tool-schema savings already ride to `/stats` and `headroom
perf`).
- Key detail: the identity check `ctx.messages is not <orig>` is
insufficient — the lossless-guard plugin mutates messages **in place**,
so an identity-gated re-count misses it. The re-count runs
unconditionally whenever a hook ran.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ ruff check headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py \
tests/test_openai_chat_turn_hooks.py tests/test_openai_responses_context_compaction.py
All checks passed!
$ mypy headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py
Success: no issues found in 2 source files
$ pytest tests/test_turn_hooks.py tests/test_openai_chat_turn_hooks.py \
tests/test_openai_responses_context_compaction.py -q
tests/test_turn_hooks.py ......... [ 34%]
tests/test_openai_chat_turn_hooks.py ..... [ 53%]
tests/test_openai_responses_context_compaction.py ............ [100%]
26 passed in 14.05s
```
New regression tests (each fails on the pre-fix code):
- `test_in_place_message_fold_is_counted` (chat path) — hook folds
message content in place; asserts `turn_hook` in `x-headroom-transforms`
and a recorded `tokens_saved > 0`.
- `test_responses_turn_hook_message_fold_is_applied_and_counted`
(Responses path) — hook folds a `function_call_output` in place; asserts
the outbound payload reflects the fold **and** `tokens_saved > 0`.
## Real Behavior Proof
- **Environment:** local proxy (`headroom proxy --port 8793
--proxy-extension lossless_guard`), `HEADROOM_LICENSE_DEV=1`,
`HEADROOM_PROTECT_TOOL_RESULTS=Bash` (so the fold is purely the plugin's
turn hook), model `claude-haiku-4-5`. Request carries a `gh --json`
object (folded to TOON) and a `docker pull` log.
- **Exact steps:** send the request → read the `PERF` line in
`~/.headroom/logs/proxy.log` and `GET /stats`.
- **Observed result:**
- Before this change: `PERF ... tok_before=607 tok_after=607 tok_saved=0
... transforms=none` while `/stats` reported `{"lossless_guard": 145}` —
i.e. the saving existed but perf showed nothing.
- After this change: `PERF ... tok_before=607 tok_after=484
tok_saved=123 ... transforms=turn_hook`, `/stats` still
`{"lossless_guard": 145}`. (`123` is the honest whole-request
`count_messages` delta; `145` is the per-content-string delta
`record_compression` measures — different scopes, both real and
positive.)
- **Not tested:** the OpenAI Chat and Responses paths were verified by
unit test, not a live client run — my live setup routes Claude Code
through the Anthropic handler only.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation — N/A
(internal accounting; no public API/doc surface)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Behavior is unchanged when no turn hook is registered
(`registered_turn_hooks() == []` → the re-count block is skipped), so
pure-OSS installs are byte-identical and unaffected. OSS's own pipeline
compression was already counted correctly (it runs before the hook);
this only surfaces the extension/turn-hook layer.
|
||
|
Rod Boev
|
4a8157fa0a
|
fix(copilot): derive GHE credential host from API URL (#800) (#2511)
## Description GHE Copilot credential discovery falls back straight to `github.com` when `GITHUB_COPILOT_HOST` is unset, even if the documented `GITHUB_COPILOT_API_URL` points at an enterprise host. This change keeps explicit-host precedence, then reuses the configured enterprise domain or a normalized custom API URL hostname for credential lookup, so Windows, macOS, Linux, GH CLI, and credential-file discovery search the same custom host instead of the public default. Closes #800. Attribution: https://github.com/headroomlabs-ai/headroom/issues/800#issuecomment-5044382263 narrowed the shared credential-host mismatch. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds new functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - Preserve explicit-host precedence, then fall back to the configured enterprise domain or a normalized custom API URL hostname only when the configured value is usable. - Normalize `api.` and `copilot-api.` prefixes before routing credential lookup, while keeping exact and segmented GitHub-hosted public domains plus public enterprise or malformed enterprise or API configuration fallback on `github.com`. - Add focused coverage for the base/head reproduction, explicit-host precedence, configured-enterprise precedence, public-enterprise, malformed-enterprise, and invalid-port fallback, prefixed-host normalization, adjacent-host exclusion, and GH CLI plus keychain forwarding. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_copilot_auth.py -q`) - [x] Linting passes - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_copilot_auth.py -q 105 passed in 0.58s uvx --from ruff==0.15.17 ruff check headroom/copilot_auth.py tests/test_copilot_auth.py All checks passed! uvx --from ruff==0.15.17 ruff format --check headroom/copilot_auth.py tests/test_copilot_auth.py 2 files already formatted git diff --check clean ``` ## Real Behavior Proof - Environment: Windows, isolated temporary credential file, local `origin/main` checkout plus this branch - Exact command / steps: With only `GITHUB_COPILOT_API_URL=https://api.ghe.example.com:8443/copilot` set and all other token sources disabled, run the same credential-file discovery reproduction against `origin/main` and this branch. - Observed result: `origin/main` selected `github.com` and resolved no token; the review branch selected `ghe.example.com` and resolved `gho-ghe`. - Not tested: live GitHub Enterprise Copilot tenant ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did not edit `CHANGELOG.md`; Headroom generates release notes from the PR title ## Additional Notes The change does not alter API routing, token exchange, discovery order, or credential matching breadth, and it keeps the live tenant claim out of the PR body until an enterprise user reruns it. |
||
|
Rod Boev
|
e4076bbe99
|
fix(grok): preserve business-seat auth while routing only inference (#2514)
## Description `headroom wrap grok` currently routes the whole session through `GROK_CLI_CHAT_PROXY_BASE_URL`. xAI's July 21, 2026 enterprise docs say that host carries both inference and settings, so the wrap displaces the native settings/auth path along with inference. A Grok account whose SuperGrok entitlement lives on a business account can then no longer resolve that seat and falls back to a login screen, even though native `grok` works for the same account. This change retargets the Grok provider slice to the narrower inference-only key, `GROK_MODELS_BASE_URL`, and leaves `GROK_CLI_CHAT_PROXY_BASE_URL` unset. Headroom still intercepts inference and model discovery through the existing `/v1/models` and chat-completions proxy paths, while the native `cli-chat-proxy.grok.com` settings host and `auth.x.ai` auth path stay intact. Closes #2489. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - switch the Grok provider env authority from `GROK_CLI_CHAT_PROXY_BASE_URL` to `GROK_MODELS_BASE_URL` - update the Grok wrap and unwrap docstrings to describe inference-only routing and the preserved native settings/auth path - update the compatibility matrix entry in `README.md` so the public docs match the new Grok routing key - add focused provider and wrap tests that assert the old chat-proxy key is absent and the project-prefixed inference URL is preserved - keep `grok_build` and the existing `/v1/models` proxy route unchanged, using them as preservation boundaries ## Testing - [x] Unit tests pass (`uv run pytest tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py tests/test_provider_grok_build.py -q`) - [x] Linting passes (`uv run ruff check headroom/providers/grok/runtime.py headroom/cli/wrap.py tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py tests/test_provider_grok_build.py -q uv run ruff check headroom/providers/grok/runtime.py headroom/cli/wrap.py tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py uv run ruff format headroom/providers/grok/runtime.py headroom/cli/wrap.py tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py --check ``` ## Real Behavior Proof - Environment: current Grok CLI plus a focused Headroom worktree - Exact command / steps: capture `grok --version`, re-check xAI's documented Grok env contract, run the focused Grok provider and wrap tests, and if a business-seat account is available locally launch `headroom wrap grok` to confirm the wrapped session no longer falls back to login - Observed result: Headroom emits only the inference-routing key, the old settings/auth key is absent, project prefixing still works, and the focused Grok tests pass - Not tested: local business-seat account on this host ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A - CLI and provider-routing change only. ## Additional Notes - `CHANGELOG.md` stays untouched because Headroom's release automation generates it from conventional commits. - The issue is reporter-only today, so the proof report records the validated `grok --version` and whether a real business-seat retest was reached locally or remains for the reporter. |
||
|
inix
|
806d2e468a
|
fix(proxy): offload OpenAI and Gemini tokenizer counting off the event loop (#2498)
## Description The OpenAI and Gemini handlers resolved the tokenizer and counted the conversation inline on the event loop. When a model resolves to a HuggingFace tokenizer (the registry routes qwen, deepseek, llama, phi, falcon, and more there) a cold cache runs `AutoTokenizer.from_pretrained` behind a 10s `thread.join`, which freezes the whole server. That is the GH #1701 stall, now reachable from OpenAI and Gemini because `/v1/chat/completions` and `/v1/responses` are documented multi-provider passthroughs and receive those models. Anthropic already routed the same call through a fail-open `_count_tokens_offloaded` helper. This hoists that helper to the shared `HeadroomProxy` base and sends the OpenAI and Gemini sites through it too. No linked issue. This is the OpenAI and Gemini follow-on to #1738, which offloaded the Anthropic and batch paths. GH #1701 is the original freeze report. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Hoisted `_count_tokens_offloaded` from `AnthropicHandlerMixin` to the shared `HeadroomProxy` base, next to `_run_compression_in_executor`. It resolves and counts on the bounded compression executor and fails open to estimation on timeout, error, or executor quarantine. - Routed 6 inline sites through it: `handle_openai_chat`, `handle_openai_responses`, `handle_gemini_generate_content`, `handle_google_cloudcode_stream`, `handle_gemini_count_tokens`, and `handle_gemini_stream_generate_content` (resolve only, keeps its per-part `count_text` loop). - Removed 6 now-dead local `get_tokenizer` imports. - Left batch's per-line counts inline on purpose. They run on an already-warm tokenizer, so offloading them adds executor churn without touching the cold load. Batch's `pipeline.apply` was already offloaded in #1738. - Extended the wiring guard to all 7 provider handlers, added a quarantine fail-open test and a `count_text` fail-open test, and stubbed the method on 2 mixin-only handler doubles. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/server.py headroom/proxy/handlers/{anthropic,openai,gemini}.py tests/test_tokenizer_count_offload.py All checks passed! $ pytest tests/test_tokenizer_count_offload.py 6 passed in 4.39s # offload suite + all 26 handler-calling test files + handler-helpers/batch/tokenizers $ pytest tests/test_tokenizer_count_offload.py tests/test_openai_codex_routing.py tests/test_gemini_nonjson_status.py ... tests/test_tokenizers.py 377 passed, 15 skipped, 15 warnings in 86.60s (0:01:26) ``` ## Real Behavior Proof - Environment: macOS (Darwin), Python 3.13, proxy built from this branch. A synthetic tokenizer that sleeps 0.5s on resolve+count stands in for a cold HuggingFace `from_pretrained` load, with a 10ms asyncio loop-canary running alongside. - Exact command / steps: monkeypatch `headroom.tokenizers.get_tokenizer` to the 0.5s-sleeping tokenizer, then time a concurrent canary across two counts, the offloaded `await proxy._count_tokens_offloaded("qwen2.5-coder", messages)` and the old inline `get_tokenizer(model).count_messages(messages)`. - Observed result: the offloaded path kept the loop live at 41 canary ticks during the 509ms count, the inline path froze it to 0 ticks over 502ms, and both returned the same token count. Full run was 377 passed, 15 skipped, 0 failed. The new quarantine test confirms an unrelated compression timeout downgrades counting to estimation instead of raising a 500. - Not tested: live HuggingFace downloads and real qwen/deepseek traffic. No API keys in this environment, so the Gemini and OpenAI integration tests skip on `GEMINI_API_KEY`/`OPENAI_API_KEY`. `mypy headroom` did not finish locally (cold-times-out past 10 minutes on this box), so type-checking is left to CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - No linked issue. Follow-on to #1738. - Batch per-line counts stay inline: they run on an already-warm tokenizer, so offloading them adds executor churn without addressing the cold load. - Found a 6th site mid-implementation. `handle_gemini_stream_generate_content` also resolved the tokenizer inline but counts via a `count_text` loop, so it takes the resolve-only path. Verified `EstimatingTokenCounter.count_text` exists, so its fail-open branch does not crash. - `mypy headroom` cold-times-out locally (server.py pulls the full graph). Deferred to CI's Linux shards, same as prior PRs on this file. `ruff` and `pytest` run clean. - Documentation checkbox left unchecked: this change ships no user-facing doc update. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Tejas Chopra
|
5d23a0aec2
|
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499)
## Description
`tokensave` was a **downloaded third-party Rust binary**
(`aovestdipaperino/tokensave`) that `headroom wrap` registered as a
code-graph MCP server. This removes it entirely and standardises on
**Serena** as the code-memory MCP — which was already the default in
`wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads
or executes a binary of its own for code memory.
The change is a *removal + safe transition*, not a behaviour flip:
Serena was already the default, so existing users move over
automatically. This PR also folds in a small README repositioning
(Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx
are third-party tools we don't control), since it's the same
tooling-stack story.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Code refactoring (no functional changes)
## Changes Made
- **Removed the external tool:** `headroom/graph/tokensave_installer.py`
(the download-and-execute path), `build_tokensave_spec`, and the
`_ensure_tokensave_binary` / `_index_tokensave_project` /
`_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`.
- **`--code-memory`** now offers `serena` (default) or `none` — the
`tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as
its (default-on) code-memory MCP.
- **Tombstone / transition (ledger-verified):** `headroom wrap` **and**
`headroom unwrap` remove a previously Headroom-installed `tokensave` MCP
entry so upgrading users stop launching it, and print that the leftover
`~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to
delete. A user-managed `tokensave` entry is left untouched. Mirrors the
existing `codebase-memory-mcp` retirement.
- **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and
`--no-tokensave` resolve to Serena instead of erroring; `--no-serena`
now means "no code memory". **No state migration needed** — both tools'
indexes are regenerable caches of the source, so Serena simply
re-indexes.
- **Docs/README:** replaced the "tokensave binary trust model" section
with a Serena note + an "Upgrading from tokensave?" callout; retired the
RTK "first-class part of our stack" framing.
- **Tests:** deleted the tokensave-only test files
(`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`,
`test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py`
for the new resolver/dispatch, fixed a codex test that patched a removed
symbol.
Net: **+102 / −1187 lines.**
## Testing
- [x] Unit tests pass (`pytest`) — targeted to the affected areas
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py
All checks passed!
$ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py
Success: no issues found in 12 source files
$ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \
tests/test_cli/test_wrap_claude_vertex_proxy_env.py \
tests/test_cli/test_wrap_claude_finally_unbound.py -q
======================== 120 passed in 97.86s (0:01:37) ========================
$ pytest tests/test_cli --collect-only -q
========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal
```
## Real Behavior Proof
- **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on
branch `tejas/remove-tokensave`.
- **Exact command / steps:**
- `python -c "from headroom.integrations.strands.bundle import
HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False,
enable_serena_mcp=False); print(len(b.tools))"` → confirms the module
imports after `build_tokensave_spec` removal (the import that my change
would otherwise break).
- CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`)
writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context",
"codex"`) to the codex config and **no** tokensave entry.
- `_resolve_code_memory` unit tests confirm: default → `serena`;
`HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`;
`--code-memory bogus` → `ClickException`.
- **Observed result:** import OK (`tools: 0`); Serena registered,
tokensave absent; resolver behaves as above; 120/120 tests pass.
- **Not tested:** a live `headroom wrap` against a real agent on a
machine with a *previously-installed* tokensave MCP entry — the
tombstone-removal path is covered by unit tests with a fake
registrar/ledger, not an end-to-end run.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title
## Additional Notes
- `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden,
deprecated flags (no-ops or mapped) so existing scripts don't break.
- `--code-graph` is unchanged — it's the proxy's live file-watcher flag
and was never the tokensave MCP; only the dead tokensave hook behind it
was removed.
- `CHANGELOG.md` intentionally left untouched.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Rod Boev
|
5bd2266f16
|
fix(kompress): raise the default execution-slot wait (#2456)
## Description Concurrent Kompress requests currently fail open after a 25 ms execution-slot wait even though ordinary ONNX inference can hold the single slot for hundreds of milliseconds. This raises the existing default wait to 3000 ms while retaining concurrency one, the `HEADROOM_KOMPRESS_EXECUTION_TIMEOUT_MS` override, the tighter acquire and request budgets, and passthrough after a genuine timeout. The reproduction and validated 3000 ms setting come from https://github.com/headroomlabs-ai/headroom/issues/2451 Closes #2451 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Raise the default Kompress execution-slot wait from 25 ms to 3000 ms. - Start the Kompress request deadline at call entry and carry it through single-item acquire, single-to-batch delegation, and sequential-fallback lineage. - Cap the raised execution-slot wait by that live request deadline on both single-item and batch acquire paths. - Keep the per-backend default concurrency at one and preserve all tighter time budgets. - Add queued single-item, batch, request-deadline, carried-deadline lineage, and router-watchdog lifecycle regressions at the same owner layer that currently fails. - Preserve the explicit short-timeout fail-open path. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_kompress_failsafe.py tests/test_kompress_request_nonblocking.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py -v`) - [x] Linting passes (`uv run ruff check headroom/transforms/kompress_compressor.py tests/test_kompress_failsafe.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py`) - [x] Formatting passes (`uv run ruff format headroom/transforms/kompress_compressor.py tests/test_kompress_failsafe.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py --check`) - [x] New regression tests prove the saturation fix - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_kompress_failsafe.py tests/test_kompress_request_nonblocking.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py -v 37 passed in 4.22s uv run ruff check headroom/transforms/kompress_compressor.py tests/test_kompress_failsafe.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py All checks passed! uv run ruff format headroom/transforms/kompress_compressor.py tests/test_kompress_failsafe.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py --check 4 files already formatted ``` ## Real Behavior Proof - Environment: worktree Python environment from `uv sync --extra dev`, focused pytest with real Python threads and `threading.BoundedSemaphore` - Exact command / steps: hold the sole execution slot with the environment override unset, start queued single-item and batch compression workers, wait until each worker proves it reached a blocked acquire on the shared execution semaphore, release the slot, rerun the explicit 1 ms timeout preservation case, then set `HEADROOM_COMPRESSION_DEADLINE_MS=10` and repeat the held-slot single-item and batch acquires plus a router single-cache-miss run whose Kompress load sleeps past the request deadline. - Observed result: The queued single-item and batch workers each proved a real blocked acquire before release, then acquired after release and compressed, while `HEADROOM_KOMPRESS_EXECUTION_TIMEOUT_MS=1` still passed through promptly, the 10 ms request deadline capped the raised default wait so both held-slot paths failed open before 200 ms without reaching model inference, the single-to-batch and sequential-fallback lineage regressions proved later branches inherit the original request start instead of resetting it, and the router lifecycle proof showed the carried deadline now allows slow Kompress load to start but still expires before model inference after the outer request has already failed open. - Not tested: live ONNX proxy savings under sustained concurrent load ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` stays unchanged because the release pipeline generates changelog entries from conventional commits. The fail-open path from #1430 stays intact; this change stops it from firing spuriously under ordinary queueing. |
||
|
Parideboy
|
a09ba6c087
|
fix(learn): treat unreadable candidate paths as absent in project decode (#2446)
## Description `headroom learn` crashes with an uncaught `PermissionError` when the current user's username contains a dash. `_decode_project_path` (in `headroom/learn/plugins/claude.py`) probes speculative candidate paths when reconstructing an original filesystem path from a Claude Code encoded project directory name. When the username is e.g. `marco-rocha`, one candidate becomes `/home/marco/rocha`, which can collide with another user's home directory whose parent isn't stat-able. `Path.exists()` calls `os.stat` internally, raising `PermissionError` instead of returning `False`, so the whole `learn` command crashes before returning any recommendations. Fixes #2443 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `_path_exists()` to `headroom/learn/plugins/claude.py` — a thin wrapper around `Path.exists()` that returns `False` on any `OSError` (including `PermissionError`), mirroring the existing `OSError` handling already used in `_greedy_path_decode`. - Route every speculative candidate-path existence check in the decode path through `_path_exists()`: the Windows drive/path probes in `_decode_windows_path`, the `simple` POSIX candidate and greedy-branch bases in `_decode_project_path`/`_greedy_path_decode`, and the decoded `project_path`/`CLAUDE.md` checks in `discover_projects`. - Add regression tests covering the exact issue shape (`PermissionError` on `/home/marco/rocha`) and the `_path_exists` helper directly. - Leave `CHANGELOG.md` untouched — release-please generates it from conventional commits. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_learn/test_scanner.py::TestDecodePermissionError -q`) - [x] Linting passes (`ruff check`, `ruff format --check` on the two changed files) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_learn/test_scanner.py::TestDecodePermissionError -q collected 2 items tests\test_learn\test_scanner.py .. [100%] 2 passed in 1.86s $ ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, local dev checkout of headroom on branch off upstream/main - Exact command / steps: Simulated the issue by monkeypatching `Path.exists` to raise `PermissionError` for the colliding candidate `/home/marco/rocha`, then calling `_decode_project_path("-home-marco-rocha-butterfly-sylphina")` - Observed result: Before the fix the call propagates `PermissionError` (crash, matching the reported traceback); after the fix it returns without raising and the unreadable candidate is treated as non-existent. Both regression tests pass. - Not tested: End-to-end `headroom learn --apply` on a real Linux multi-user box with an actually unreadable `/home/<prefix>` — reproduced via the documented minimal logic instead. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Abhay Singh
|
3e976712e7
|
fix(proxy/output-shaping): tolerate a non-string system block text in steering (#2435)
## Description
`apply_verbosity_steering` (the Anthropic output-shaping path) scans the
`system` block list to find and update an existing steering block:
```python
if isinstance(system, list):
for block in system:
if isinstance(block, dict) and block.get("text", "").startswith(_STEERING_SENTINEL):
```
`.get("text", "")` only substitutes the default when the key is
**absent**. A malformed client block with a null text (`{"type": "text",
"text": null}`) returns `None`, so `None.startswith(...)` raises
`AttributeError`. In the output-shaping treatment arm that call runs
inside `shape_request`, which is not individually guarded, so the
exception propagates and 502s the request.
The OpenAI chat sibling in the same module already defends against this
exact case (`isinstance(part.get("text"), str)`), so the Anthropic path
is the inconsistent one.
## Fix
Guard that the block text is a string before `startswith`, mirroring the
OpenAI sibling. Well-formed bodies are unchanged: the steering block is
still replaced idempotently when a level changes, or appended when
absent. The malformed block is left untouched.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/output_steering.py`: string-guard the system block
text before `startswith` in `apply_verbosity_steering`.
- `tests/test_output_steering.py`: regression asserting a `system` list
containing a `{"text": null}` block does not crash and still appends the
steering block.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_output_steering.py -q
9 passed
# with the fix reverted, the new test fails (AttributeError on None.startswith):
$ git stash push -- headroom/proxy/output_steering.py
$ python -m pytest "tests/test_output_steering.py::test_anthropic_steering_tolerates_non_string_system_block_text" -q
1 failed
$ uvx ruff@0.15.17 check headroom/proxy/output_steering.py tests/test_output_steering.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/output_steering.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called the real `apply_verbosity_steering` with
`system=[{"type":"text","text":None},{"type":"text","text":"Real system
prompt."}]`; also confirmed the OpenAI sibling
`apply_openai_chat_verbosity_steering` handles the same shape.
- Observed result: pre-fix the Anthropic call raised `AttributeError:
'NoneType' object has no attribute 'startswith'` while the OpenAI
sibling returned True; post-fix the Anthropic call returns True, leaves
the malformed block as-is, appends the steering block, and stays
idempotent on a repeat. Ran against the actual module.
- Not tested: a live client that sends a null system block text end to
end.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Abhay Singh
|
77b26c093c
|
fix(proxy/streaming): tolerate malformed content in _response_to_sse (#2481)
## Description
`StreamingMixin._response_to_sse` rebuilds an Anthropic SSE stream from
a buffered response dict. It iterated the content and read usage with no
type guards:
```python
for idx, block in enumerate(response.get("content", [])):
if block.get("type") == "text":
...
...
"usage": {"output_tokens": response.get("usage", {}).get("output_tokens", 0)},
```
`response` here is provider- and reconstruction-controlled.
`.get("content", [])` only falls back when the key is absent, so a
present-but-null `content` returns `None` and `enumerate(None)` raises
`TypeError`. A non-list `content` (e.g. a bare string) makes
`block.get(...)` raise `AttributeError`, and a null element inside the
list hits the same `AttributeError`. `response.get("usage",
{}).get(...)` breaks the same way on `usage: null`.
This matters because the Anthropic buffered CCR path calls it inside an
`except ValueError` guard only:
```python
try:
sse_events = self._response_to_sse(resp_json, "anthropic")
except ValueError as sse_err:
...
```
A `TypeError`/`AttributeError` from any of the shapes above escapes that
guard and 500s the streamed request. The sibling
`_record_ccr_feedback_from_response` in the same class already guards
`content` for list-ness and skips non-dict blocks, so this closes the
asymmetry.
## Fix
Coerce `content` to a list before iterating (non-list becomes empty),
skip any non-dict block, and coerce a non-dict `usage` to `{}` before
reading `output_tokens`. Well-formed responses render byte-for-byte as
before.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/streaming.py`: list-guard `content`, skip
non-dict blocks, and dict-guard `usage` in `_response_to_sse`.
- `tests/test_sse_thinking_blocks.py`: regression rendering responses
with null/non-list content, a null block element, and null usage, plus a
check that a valid block alongside a null element still renders.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_sse_thinking_blocks.py -q
14 passed
# with the fix reverted, the new test fails with
# TypeError: 'NoneType' object is not iterable
$ uvx ruff@0.15.17 check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/streaming.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called
`StreamingMixin()._response_to_sse(response, "anthropic")` with four
malformed bodies (`content: null`, `content: "not-a-list"`, `content:
[null, {text}]`, `usage: null`); then reverted `streaming.py` and re-ran
the same inputs.
- Observed result: with the fix each body produces a well-formed SSE
envelope (message_start ... message_stop) and the valid block alongside
a null element still emits its text_delta; with the fix reverted the
`content: null` body raises `TypeError: 'NoneType' object is not
iterable` and the others raise `AttributeError`. Ran against the actual
module via `tests/test_sse_thinking_blocks.py`.
- Not tested: a live upstream returning a malformed buffered response
end to end through the CCR path.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Abhay Singh
|
7524854da7
|
fix(doctor): don't crash on a valid-but-non-object settings.json (#2482)
## Description
`headroom doctor` parses `~/.claude/settings.json` in two checks:
```python
try:
payload = json.loads(settings_path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
return CheckResult(... WARN "could not parse" ...)
...
env_block = payload.get("env")
```
`json.loads` returns a non-dict for any valid JSON that is not an
object: `[]`, `null`, `42`, `"a string"`. None of those raise
`JSONDecodeError`, so they slip past the `except (OSError, ValueError)`
guard, and the following `payload.get("env")` raises `AttributeError`.
`AttributeError` is not in the caught tuple, so it escapes and crashes
`doctor` with a traceback. That is the worst moment for it: `doctor` is
the command a user runs precisely because their config is suspect, and a
hand-edited or reset `settings.json` holding `[]` or `null` is exactly
the kind of file it should report on, not fall over on.
Two functions have this shape: `check_claude_routing` (the `.get` is
after the `try` returns) and `check_claude_remote_control_gate` (the
`.get` is inside a `try` whose `except` is also `(OSError,
ValueError)`).
## Fix
Guard `payload` for dict-ness in both checks. `check_claude_routing` now
returns the same WARN it already returns for unparseable files, with a
"not a JSON object" summary; `check_claude_remote_control_gate` treats a
non-object as having no `env` block, so the shell environment still
drives the gate. Well-formed object settings behave exactly as before.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/doctor.py`: guard `payload` for dict-ness in
`check_claude_routing` and `check_claude_remote_control_gate` before
calling `.get`.
- `tests/test_cli_doctor.py`: parametrized regressions feeding `[]`,
`null`, `42`, and a bare string to both checks.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_cli_doctor.py -q
68 passed
# with the fix reverted, the new tests fail with
# AttributeError: 'list' object has no attribute 'get'
$ uvx ruff@0.15.17 check headroom/cli/doctor.py tests/test_cli_doctor.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/cli/doctor.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: wrote a `settings.json` containing `[]` (and
`null`, `42`, `"a string"`) into a tmp path and called
`check_claude_routing(path, 8787)` and
`check_claude_remote_control_gate(path, {"ANTHROPIC_BASE_URL":
"http://127.0.0.1:8787"})`; then reverted `doctor.py` and re-ran.
- Observed result: with the fix both checks return a WARN result instead
of raising; with the fix reverted both raise `AttributeError: 'list'
object has no attribute 'get'` (and the analogous message for
`null`/`42`/string). Ran against the actual module via
`tests/test_cli_doctor.py`.
- Not tested: the full `headroom doctor` CLI end to end against a real
`~/.claude/settings.json`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Rod Boev
|
46293f4daf
|
fix(codex): detect keyring-backed ChatGPT auth (#2478)
## Description Headroom currently treats missing `auth.json` as “not ChatGPT auth” for Codex, which breaks keyring-backed ChatGPT sessions on Codex CLI 0.144.6 because those sessions intentionally may not store credentials in the file. This updates the Codex auth detector to keep the existing file-backed fast path and fall back to Codex-owned auth metadata when the session is keyring-backed or auto-backed, so `requires_openai_auth = true` is emitted only for real ChatGPT logins. Closes #2474 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - extend Codex auth detection so keyring-backed and auto-backed sessions can be classified from Codex-owned auth metadata when `auth.json` is absent - preserve the current file-backed ChatGPT, API-key, malformed-file, and fail-closed behaviors - add focused install-layer regression coverage for the new keyring path and adjacent negative space ## Testing - [x] Unit tests pass (`uv run pytest tests/test_install/test_codex_install.py -q`) - [x] Linting passes (`uv run ruff check headroom/providers/codex/install.py tests/test_install/test_codex_install.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_install/test_codex_install.py -q ============================= test session starts ============================= platform win32 -- Python 3.12.13, pytest-9.0.3, pluggy-1.6.0 collected 9 items tests\test_install\test_codex_install.py ......... [100%] ============================== 9 passed in 0.24s ============================== uv run ruff check headroom/providers/codex/install.py tests/test_install/test_codex_install.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, Codex CLI 0.144.6 available locally, Python 3.12.13 via `uv` - Exact command / steps: `codex login status`; `Measure-Command { codex login status > $null }`; focused pytest and Ruff commands above - Observed result: `codex login status` returns `stdout=''` and `stderr='Logged in using ChatGPT\n'`; the status probe measured `71.40` ms; pytest reports `9 passed in 0.24s`; keyring ChatGPT emits `requires_openai_auth = true`, non-ChatGPT and failed probes omit it, and file-backed ChatGPT/API-key cases remain true/false - Not tested: live local keyring-backed Codex login ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` stays untouched because Headroom generates release notes from conventional commits. The PR should only claim the Codex-owned detection path and focused local regression coverage; the live keyring session proof remains a follow-up owner check. |
||
|
Rod Boev
|
a2e42fb877
|
fix(proxy): keep buffered CCR streams alive (#2479)
## Description Buffered CCR streaming currently waits for the full upstream response before sending any bytes back to the client. On the Anthropic path this shows up as `API Error: Stream idle timeout - no chunks received`, and the same buffer-then-synthesize mechanism still exists on the `/v1/responses` CCR path. This adds a narrow buffered-stream heartbeat layer so the client sees early stream activity while Headroom preserves the existing server-side retrieval round trip and final synthesized provider events. Closes #2465 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - open buffered CCR streams early and emit client-visible `event: ping` heartbeats while the buffered upstream call is still in flight - preserve the existing terminal Anthropic and Responses synthesis helpers instead of replacing their event-building logic - preserve early non-streaming failure semantics before the first heartbeat, including normal 429 passthrough and normal JSON 502 failures - log late buffered-task exceptions server-side and record one failed provider metric on that post-keepalive branch, while keeping the client-facing SSE error sanitized - add focused delayed-upstream regression coverage for both buffered provider paths, their early-failure branches, and their late-failure branches ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_proxy/test_openai_responses_ccr.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_proxy/test_openai_responses_ccr.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_proxy/test_openai_responses_ccr.py -q ======================= 17 passed, 1 warning in 42.47s ======================== uv run ruff check headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_proxy/test_openai_responses_ccr.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, Python via `uv`, proxy handler tests with gated buffered upstream fixtures - Exact command / steps: run the focused Anthropic and Responses CCR suites above; delayed-upstream tests consume the first client-visible SSE event before releasing the upstream, then consume the synthesized final events - Observed result: both buffered paths emitted `event: ping` before upstream release; pre-keepalive 429 responses preserved their real status and headers, pre-keepalive exceptions returned the normal JSON 502 shape, late transport failures recorded one failed provider metric and one server error log before emitting one sanitized SSE error, Anthropic preserved `done`, and Responses preserved `Resolved!` - Not tested: live slow upstream run with Claude Code or a real Responses client ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` stays untouched because Headroom generates release notes from conventional commits. The PR should only claim the local buffered-stream contract and focused regression coverage; live client proof remains an owner check on a slow real upstream. |
||
|
Abhay Singh
|
43a7b578a1
|
fix(backends): don't crash the OpenAI->Anthropic converter on empty choices (#2484)
## Description `_to_anthropic_response` in both backends converts a non-streaming OpenAI-shape response to Anthropic shape and indexes the first choice directly: ```python # headroom/backends/litellm.py choice = litellm_response.choices[0] # headroom/backends/anyllm.py choice = response.choices[0] ``` A non-streaming upstream response can be HTTP 200 with an **empty** `choices` list: Azure OpenAI content filtering does exactly this, and any OpenAI-compatible gateway can return a usage-only / filtered turn the same way. With `choices: []`, `choices[0]` raises `IndexError`, which surfaces as a 500 for the request instead of a normal (if empty) turn. This is an intra-file asymmetry: the streaming siblings in the same two files already guard it (`if not chunk.choices: continue` / `if hasattr(chunk, "choices") and chunk.choices:`), and `headroom/proxy/handlers/openai.py` documents the exact hazard in `_apply_stream_usage_option`: "the common `chunk.choices[0].delta` pattern then raises IndexError" on a usage-only `choices: []` chunk. The non-streaming converters just never got the same guard. ## Fix Return a valid empty assistant turn (`content: []`, `stop_reason: "end_turn"`, usage still mapped) when `choices` is empty, before indexing. The client gets a clean empty response instead of a 500, matching how the streaming path already tolerates the same shape. Non-empty responses are unchanged. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/backends/litellm.py`: empty-`choices` guard at the top of `_to_anthropic_response`, returning an empty assistant turn with mapped usage. - `headroom/backends/anyllm.py`: same guard in its `_to_anthropic_response`. - `tests/test_litellm_nonstream_cache_usage.py`, `tests/test_backend_anyllm.py`: regressions passing an empty-`choices` response through each converter and asserting an empty turn instead of IndexError. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_litellm_nonstream_cache_usage.py::test_to_anthropic_response_empty_choices_returns_empty_turn tests/test_backend_anyllm.py::test_to_anthropic_response_empty_choices_returns_empty_turn -q 2 passed # with the fix reverted, both fail with # IndexError: list index out of range $ uvx ruff@0.15.17 check headroom/backends/litellm.py headroom/backends/anyllm.py tests/test_backend_anyllm.py tests/test_litellm_nonstream_cache_usage.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/backends/litellm.py headroom/backends/anyllm.py Success: no issues found in 2 source files ``` Note: `tests/test_backend_anyllm.py` has 7 `@pytest.mark.asyncio` tests that fail locally because pytest-asyncio is not configured in this environment (`Unknown config option: asyncio_mode`); they are unrelated to this change and pass in CI. The two new tests here are synchronous and pass locally. ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: built a response stand-in with `choices=[]` and a usage object, called `LiteLLMBackend._to_anthropic_response` (on a bare `object.__new__` instance) and `AnyLLMBackend._to_anthropic_response` (via the file's fake-backend fixture); then reverted both backend files and re-ran. - Observed result: with the fix each converter returns `{type: message, role: assistant, content: [], stop_reason: end_turn, usage: {...}}` with the input/output token counts mapped; with the fix reverted both raise `IndexError: list index out of range`. Ran against the actual modules via the two test files. - Not tested: a live Azure OpenAI content-filtered response routed through the backend end to end. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Abhay Singh
|
07cf547607
|
fix(proxy/gemini): tolerate malformed parts on the compression path (#2486)
## Description
Three helpers on the Gemini compression path read a content entry's
`parts` and iterate it without type guards:
```python
# _has_non_text_parts
parts = content.get("parts", [])
for part in parts: ...
# _rebuild_gemini_contents
had_text = any("text" in p for p in content.get("parts", []))
# _gemini_contents_to_messages
parts = content.get("parts", [])
text_parts = [p.get("text", "") for p in parts if "text" in p]
```
`parts` is request-controlled and `.get("parts", [])` only falls back
when the key is absent, so:
- a present-but-null `parts` returns `None`, and `for part in None` /
`any(... for p in None)` raises `TypeError`;
- a list carrying a bare string (a client that treats `parts` as a
string array) makes `p.get("text", "")` raise `AttributeError`, while
`"text" in p` silently does substring matching first;
- a null element in the list crashes the same way.
Any of these 500s the request on the compression path, on data that
parsed as valid JSON.
## Fix
Route all three helpers through a shared `_dict_parts(content)` that
returns the dict entries of `parts`, coercing a non-dict content or a
non-list `parts` to an empty list and dropping non-dict elements.
`_gemini_contents_to_messages` also reads `role` defensively for a
non-dict content entry. Conversion now degrades gracefully (the
malformed part contributes nothing) instead of raising. Well-formed
requests are unchanged.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/gemini.py`: add `_dict_parts`; use it in
`_has_non_text_parts`, `_rebuild_gemini_contents`, and
`_gemini_contents_to_messages`; read `role` defensively for a non-dict
content entry.
- `tests/test_gemini_function_response_waste.py`: regressions for null
`parts`, bare-string part elements, a null part element,
`_has_non_text_parts` on malformed parts, and a non-dict content entry.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_gemini_function_response_waste.py -q
16 passed
# with the fix reverted, the new malformed-parts tests fail with
# TypeError: 'NoneType' object is not iterable (and AttributeError on string parts)
$ uvx ruff@0.15.17 check headroom/proxy/handlers/gemini.py tests/test_gemini_function_response_waste.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/gemini.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: built a real `HeadroomProxy` and called
`_gemini_contents_to_messages` / `_has_non_text_parts` with contents
carrying `parts: null`, `parts: ["bare string", {text}]`, `parts: [null,
{text}]`, and a non-dict content entry; then reverted `gemini.py` and
re-ran.
- Observed result: with the fix each malformed shape converts without
raising and the valid text part is still emitted (`[{"role": "user",
"content": "kept"}]`); with the fix reverted the null-`parts` and
null-element cases raise `TypeError: 'NoneType' object is not iterable`
and the string-element case raises `AttributeError: 'str' object has no
attribute 'get'`. Ran against the actual module via
`tests/test_gemini_function_response_waste.py`.
- Not tested: a live Gemini request with malformed `parts` routed
through the full proxy compression pipeline end to end.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
gglucass
|
2195ba7d91
|
fix(proxy/openai): don't record Codex WS savings without input accounting (#2493)
## Description
On the Codex WS Responses path (`handle_openai_responses_ws`),
`tokens_saved` accumulates at compression time (our own token count),
while input tokens only arrive with a usage frame on
`response.completed`. A turn that is compressed but never completes —
cancelled mid-response (Esc in Codex), or an upstream error before the
usage frame — records `tokens_saved > 0` with `input_tokens == 0`
through the outcome funnel.
That writes a savings-with-zero-spend checkpoint into the savings
tracker: `compression_savings_usd` advances while `total_input_tokens` /
`total_input_cost_usd` stay flat. `/stats-history` then serves daily
buckets with `compression_savings_usd_delta > 0` and
`total_input_tokens_delta == 0`, which savings dashboards flag as a
data-integrity anomaly ("graph shows compression savings but zero tokens
spent on recent day(s)").
Both WS record sites have the hazard:
- the per-turn metrics closure (`_record_ws_response_metrics`) records
per-field-clamped deltas, so a usage-less turn contributes a
savings-only outcome;
- the session-end residual flush records `residual_tokens_saved` with
`residual_input_tokens` possibly 0.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/handlers/openai.py`:
- New module-level pure helper `_deferrable_savings_delta(input_delta,
saved_delta)` — returns 0 when `saved_delta > 0` with `input_delta <=
0`, passes everything else through unchanged.
- Per-turn metrics closure: gate `saved_delta` through the helper, and
advance `ws_recorded_tokens_saved_total += saved_delta` (previously `=
tokens_saved`) so deferred savings stay pending and ride along with the
next usage-carrying turn instead of being silently dropped.
- Session-end residual flush: gate `residual_tokens_saved` through the
same helper — savings that never saw a usage frame by session close are
dropped rather than recorded against zero spend (the spend for those
turns is genuinely unknown).
- `tests/test_codex_ws_savings_deferral.py`: truth-table test for the
helper; a bookkeeping walk asserting deferred savings land with the next
usage-carrying turn; and a source-level regression guard for the
closure-internal wiring (same idiom as
`test_codex_ws_compression_scheduler.py`, since the WS closures have no
unit harness yet).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed (real-behavior script below)
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_codex_ws_savings_deferral.py tests/test_openai_codex_ws_lifecycle.py tests/test_codex_ws_compression_scheduler.py tests/test_openai_codex_ws_timings.py tests/test_proxy_savings_history.py
83 passed, 1 skipped (pre-existing pending-harness skip)
$ ruff check headroom/proxy/handlers/openai.py tests/test_codex_ws_savings_deferral.py
All checks passed!
$ uv run --frozen --extra dev mypy headroom/proxy/handlers/openai.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS arm64 (Darwin 24.6.0), Python 3.12, this branch
checked out in the repo, run via `uv run --frozen python`.
- Exact command / steps: `uv run --frozen python rbp_demo.py` — a
real-behavior script exercising the REAL `SavingsTracker` (persistence +
`/stats-history` rollup via `history_response()`) and the REAL
`_deferrable_savings_delta` from this branch, no mocks. Scenario: turn 1
compressed (1200 saved) then cancelled before its usage frame (recorded
on day 1), turn 2 compressed (600 more) and completed with
`input_tokens=40000` (day 2); "BEFORE" records what the unfixed handler
emitted, "AFTER" walks the fixed bookkeeping. Additionally, a real
production `~/.headroom/proxy_savings.json` (5000 checkpoints, live
proxy in daily Claude Code + Codex use) was scanned for consecutive
checkpoint pairs where `compression_savings_usd` grew while
`total_input_tokens` stayed flat — one such pair was present
(`provider=openai, model=gpt-5.4-mini`, a Codex WS turn), exactly the
shape this PR removes at the source.
- Observed result: the unfixed recording produces a day-1
`/stats-history` bucket with `compression_savings_usd_delta > 0` and
`total_input_tokens_delta == 0` (the flagged anomaly); the fixed
bookkeeping produces no such bucket and preserves the full 1800 tokens
of savings, paired with the usage-carrying turn. Full output:
```text
BEFORE (unfixed recording): [{'tokens_saved': 1200, 'compression_savings_usd_delta': 0.0009, 'total_input_tokens_delta': 0},
{'tokens_saved': 600, 'compression_savings_usd_delta': 0.00045, 'total_input_tokens_delta': 40000}]
AFTER (fixed recording): [{'tokens_saved': 1800, 'compression_savings_usd_delta': 0.00135, 'total_input_tokens_delta': 40000}]
desync bucket present before fix: True
desync bucket present after fix: False
total savings preserved after fix: True
```
- Not tested: a live end-to-end WS session against the real OpenAI
upstream with a mid-response cancel (needs a real Codex client +
billable upstream). The per-turn/residual closure wiring is covered by
the source-level regression guard instead, per the pending-harness note
in `test_codex_ws_compression_scheduler.py`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A —
internal accounting fix, no user-facing docs affected)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from the Conventional Commit PR title (a CI guard
enforces this)
## Additional Notes
- Sessions that end on a cancelled turn under-report savings slightly
(the deferred savings are dropped at close because their spend is
genuinely unknown). This is the honest trade-off: the alternative —
recording savings against zero spend — is the desync this PR removes.
- `attempted_input_tokens` is intentionally not gated: a cancelled turn
still records its attempted delta, keeping funnel-drop visibility.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Tejas Chopra
|
1329ed7f1a
|
feat(proxy): make /v1/compress usable as a gateway/Kong sidecar (#2458)
## Description Makes the compression-only `POST /v1/compress` endpoint usable as a **network compression sidecar** behind an API gateway (Kong, LiteLLM, ...), and fixes a latent content-detector hang that silently zeroed compression on non-Windows hosts. Motivated by a LiteLLM-sidecar deployment whose team documented five build-time patches; this ports the ones that belong upstream, generalized so they cover any aliasing gateway (not just LiteLLM). Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **`lossy_inline` compress mode** (`config.mode="lossy_inline"`, alias `"lossless_then_lossy"`): lossless byte/data fold first, then Kompress the folded remainder, with `ccr_inject_marker=False` so every compressor emits **inline, marker-free** output — no `<<ccr:…>>` markers and no CCR store write, so the result is safe to forward straight to a provider with no retrieval round-trip. The mode inherits the deployment's `enable_kompress`. - **`HEADROOM_COMPRESS_ALLOW_REMOTE`** opt-in: drops the loopback dependency on the `/v1/compress` route **only** so an authorized in-network gateway can reach it. Default is unchanged (loopback-only); inbound `HEADROOM_PROXY_TOKEN` auth still applies. - **`HEADROOM_MODEL_ALIAS_MAP`** (gateway-agnostic, fail-soft): one shared resolver in `pricing/litellm_pricing.py` reduces a gateway-aliased model name (e.g. `claude-opus`) to a priced `litellm.model_cost` key, trying the mapped target as-is and with a `bedrock/` / `vertex_ai/` prefix stripped. `proxy/savings_tracker.py` now delegates to it, so the live (`/stats`) and persisted (`/stats-history`) dollar figures price identically. - **`get_context_limit`**: an operator-configured limit (`HEADROOM_MODEL_LIMITS` / `~/.headroom/models.json`) now wins **before** the dynamic LiteLLM lookup, so an aliased name no longer falls through to the 128K default and skews compression. - **fix(content_router): first-call detector watchdog on all platforms.** The native content detector can deadlock on first use (#575, previously flagged Windows-only). The watchdog was `win32`-only, so on macOS/Linux a first-use hang was unbounded → `_detect_content` never returned → the `/v1/compress` executor timeout fired → fail-open → **`tokens_before=0`, silent zero compression**. Now the native detector runs under the watchdog on the first call on every platform; once it returns it is marked verified and the direct fast path is used (zero steady-state overhead). A hang degrades to pure-Python detection with a clear warning. `win32` behavior is unchanged. - Thread `waste_signals` / `pipeline_timing` into the already-present `/v1/compress` outcome record so the guardrail path populates the dashboard panels like the forward-proxy paths. Deliberately **not** ported: the sidecar's LiteLLM-specific `GET /model/info` HTTP fetch (urllib/ssl/threading/TTL). Kong has no such endpoint; the static `HEADROOM_MODEL_ALIAS_MAP` covers any gateway with no network dependency on the pricing path. ## Testing - [x] Unit tests pass (targeted — see output) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check <changed files> All checks passed! $ mypy <changed source files> Success: no issues found in 6 source files $ pytest tests/test_gateway_sidecar_ports.py tests/test_proxy_compress_endpoint.py -q tests/test_gateway_sidecar_ports.py ........ [ 34%] tests/test_proxy_compress_endpoint.py ............... [100%] ============================= 23 passed in 20.62s ============================== ``` ## Real Behavior Proof - **Environment:** macOS (darwin/arm64), Python 3.12, `.venv`; Kompress offloaded to a Modal endpoint via `HEADROOM_KOMPRESS_ENDPOINT`. - **Exact command / steps:** posted typical tool-output payloads to `POST /v1/compress` (via the FastAPI `TestClient`, loopback) in both `default` and `lossy_inline` modes; separately reproduced the detector hang with `faulthandler.dump_traceback_later`. - **Observed result:** - Real savings through the endpoint (structural/lossless, Kompress off): **JSON 150 records 13,982→9,514 (32.0%)**, **logs 314 lines 12,240→9,549 (22.0%)**, **search 200 hits 5,231→3,471 (33.6%)**. `lossy_inline` emits **zero** CCR markers. - `faulthandler` pinned the pre-fix hang to `content_router.py:_detect_content` → native `_rust_detect`. With the fix, the first call degrades at the 5s watchdog with `"Native content detector hung … using pure-Python detection"` and compression proceeds (previously it hung and the endpoint returned `tokens_before=0`). - Modal Kompress warm latency measured ~0.8s/call; the learned pass compresses prose further (62→56 words on a sample). - **Not tested:** full `pytest` suite (ran the two affected test files only); the native-detector hang was reproduced on a local macOS/arm64 build — the fix's degrade path is verified, but a healthy-native CI Linux run should confirm the fast (verified) path there. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - The five-item context comes from a downstream LiteLLM sidecar's `PATCHES.md`; item #3 (record an outcome from the guardrail path) was already upstreamed — this PR only adds the missing `waste_signals`/`pipeline_timing` threading. Item #2 (observability read-only exemption when `HEADROOM_PROXY_TOKEN` is set) is not addressed here. - All new config is opt-in and fail-soft; with nothing set, behavior is byte-identical to today. |
||
|
Ehsan
|
f4070c44cb
|
fix(transforms/cross-turn-dedup): don't renumber-fold zero-padded line prefixes (#2369)
## Description
On an HTTP tool-output re-read, `cross_turn_dedup` folds a contiguous
span that
already appeared in an earlier block into a compact pointer, and when
the line
numbers shifted by a constant it carries the offset as a `delta` so the
original
bytes recover as `int(number) + delta`. The module states this renumber
path is
"strictly lossless" for UNPADDED numbers only.
`_LINENO_RE = ^(\d+)(:|\t)(.*)$` does not enforce the "unpadded"
restriction: `\d+`
also matches a LEADING-ZERO prefix. A timestamped log row such as
`08:00:01 ...`
is read as line number `8`, not as data, so a re-read shifted by a
constant (a
later window of the same hourly log) folds under a uniform delta.
Recovery then
renders `str(int("08") + 1)` = `"9"`, not `"09"`: the round-trip is not
byte-exact. This is a lossy (false-positive) fold in a module whose
stated
posture is to prefer false negatives (`CONTRIBUTING.md:129`,
`cross_turn_dedup.py:45-50`).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/transforms/cross_turn_dedup.py`: restrict `_LINENO_RE` to
`[1-9]\d*`
so a leading-zero run stays non-numbered and can fold only on an EXACT
match
(delta 0), never under a lossy renumber. Real `grep -n` / `sed -n` / `rg
-n`
numbers never carry a leading zero, so the intended renumber-fold
feature is
unchanged. Added a comment stating why the character class is
load-bearing.
- `tests/test_cross_turn_dedup.py`: added a delta-aware reconstruction
helper and
three regression tests (the existing `_reconstruct` asserts delta is
absent, so
it never exercised the numbered path this bug lives on).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
Three named scenarios, one test each:
1. `test_zero_padded_prefix_not_folded_lossily`: a padded shifted
re-read is left
verbatim (`spans_folded == 0`). This fails on `main` (it folds under a
delta).
2. `test_unpadded_renumber_still_folds_and_recovers_exactly`: an
unpadded `grep -n`
read renumbered by `+5` still folds and reconstructs byte-exact (feature
guard).
3. `test_padded_content_exact_redisplay_still_folds`: the same padded
rows
re-displayed verbatim still fold with delta 0 (surgical-scope guard).
### Test Output
```text
--- ruff check ---
All checks passed!
--- ruff format --check ---
2 files already formatted
--- mypy ---
Success: no issues found in 1 source file
--- pytest: 3 new tests on the branch (fixed) ---
3 passed, 14 deselected
--- pytest: revert regex to \d+ (simulate main): the regression test must FAIL ---
1 failed
```
## Real Behavior Proof
- Environment: clean `python:3.12-slim` Docker, `PYTHONPATH` at the
source tree,
core deps installed by name (tiktoken, pydantic, litellm, click, rich,
opentelemetry-api, pyyaml, tomlkit), `ruff==0.15.17`, `mypy==1.20.2`.
- Exact command / steps: import the module and print provenance, then
run ruff,
ruff format, mypy on the two changed files, then `pytest` the three new
tests
on the branch, then revert only the regex to `\d+` and re-run the
regression
test.
- Observed result: module `cross_turn_dedup.py` (sha256
7ff204b65f40617d505895841aa1cfc1ee54bf63ffe6520198f0aa05a850aa8d), regex
now `^([1-9]\d*)(:|\t)(.*)$`; ruff, ruff format, mypy all green; branch
`3 passed`, reverted-regex main `1 failed`. Breakdown:
- `module: /src/headroom/transforms/cross_turn_dedup.py`
- `sha256:
7ff204b65f40617d505895841aa1cfc1ee54bf63ffe6520198f0aa05a850aa8d`
- `regex : ^([1-9]\d*)(:|\t)(.*)$`
- ruff, ruff format, mypy: all green (output above).
- Branch: `3 passed`. Reverted-regex main: the regression test `1
failed`.
- Not tested: the router-level and Rust-backed integration tests in this
file
(`test_apply_*`, `test_dedup_*`) need the compiled `headroom._core`
extension,
which is not built in this lightweight container; they are
`ModuleNotFoundError`
on both `main` and this branch here, so they were not exercised. The
change is a
pure-stdlib regex in a pure-stdlib function; the unit-level
`dedup_blocks` tests
above cover it directly. I also did not measure how often real-world
tool output
hits the leading-zero shifted shape; the argument is the module's own
strictly-lossless contract, not observed field frequency.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A: no
doc surface)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Per `CONTRIBUTING.md` ("Bug or small fix -> Open a PR with repro +
test"), this
goes straight to a PR rather than an issue. One concern only; no
dependency or
generated-file changes.
|
||
|
Parideboy
|
c811007f81
|
fix(kompress): match all ONNX backends with startswith, not exact "onnx" (#2448)
## Description With `HEADROOM_KOMPRESS_BACKEND=onnx_coreml`, every Kompress compression call and the startup canary crash with `'_OnnxModel' object has no attribute 'parameters'`, so Kompress silently degrades to passthrough and `/health` reports `kompress: unhealthy, backend: null`. Root cause: `headroom/transforms/kompress_compressor.py` gated the ONNX-vs-PyTorch branch with an exact string match `backend == "onnx"`. But `_load_kompress_onnx` returns `onnx_coreml` (CoreML) or `onnx_cpu` — never the bare string `onnx`. So under `onnx_coreml` the code built PyTorch tensors and dispatched to a device via `next(model.parameters())`, which the `_OnnxModel` wrapper doesn't implement. This is the accelerated backend Apple Silicon users reach for, so the fast path is exactly the broken one. Fixes #2442 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Change the four exact-match `backend == "onnx"` sites in `headroom/transforms/kompress_compressor.py` to `backend.startswith("onnx")`, matching the convention already used by `_model_device_type`: `_timed_canary`, `compress`, `compress_batch`, and the batch-parallelism guard in `_should_use_sequential_fallback`. - Update the guard comment ("ONNX CPU provider" → "ONNX EPs") since it now covers all ONNX execution providers. - Add regression tests exercising `_timed_canary` on `onnx_coreml` (must take the numpy path and never touch `.parameters()`) with a negative control proving the PyTorch branch still dispatches to a device. - Leave `CHANGELOG.md` untouched — release-please generates it from conventional commits. - Out of scope: the secondary `/health` under-reporting the issue flags as informational (deferred-preload warmup object never flips to `loaded`). ## Testing - [x] Unit tests pass (`python -m pytest tests/test_transforms/test_kompress_compressor.py::TestOnnxBackendPrefixGating -q`) - [x] Linting passes (`ruff check`, `ruff format --check` on the two changed files) - [x] Type checking passes (`mypy headroom/transforms/kompress_compressor.py --ignore-missing-imports`) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_transforms/test_kompress_compressor.py::TestOnnxBackendPrefixGating -q collected 2 items tests\test_transforms\test_kompress_compressor.py .. [100%] 2 passed in 2.20s $ ruff check headroom/transforms/kompress_compressor.py tests/test_transforms/test_kompress_compressor.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, local dev checkout on a branch off upstream/main (no Apple Silicon / CoreML hardware available) - Exact command / steps: Ran the new `TestOnnxBackendPrefixGating` regression; then temporarily reverted one site back to `backend == "onnx"` and re-ran to confirm the test discriminates. - Observed result: With the fix, `_timed_canary(model, tokenizer, "onnx_coreml")` returns a float and never touches `.parameters()`. Reverting one site makes the onnx_coreml test fail (it takes the `pt` tensor path and hits the paramless model), proving the test catches the exact bug. The issue reporter separately verified the fix on real Apple Silicon hardware (onnxruntime 1.27.0, CoreMLExecutionProvider): zero occurrences of the error afterward and compression completing on the CoreML session. - Not tested: End-to-end run on real CoreML hardware from this environment — reproduced via the unit-level device-dispatch seam instead; hardware confirmation is in the issue. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Rod Boev
|
2eca5ee114
|
fix(copilot): normalize subscription API routing (#2441) (#2455)
## Description PR https://github.com/headroomlabs-ai/headroom/pull/2445 added the missing OpenCode subscription path, but the shared Copilot subscription resolver still lets Business and Enterprise payload hosts route through segmented `*.githubcopilot.com` domains and still drops an explicit `GITHUB_COPILOT_API_URL` pin on two resolution paths. This follow-up moves the final hosted-route decision back into the shared resolver, normalizes `api.business.githubcopilot.com` and `api.enterprise.githubcopilot.com` to the generic host by default, and makes the explicit pin win on token exchange, explicit API token, and Copilot-token candidate resolution. Both `headroom wrap copilot --subscription` and `headroom wrap opencode --copilot-subscription` inherit the same fix because they already consume the same `CopilotSubscriptionTokenResolution.api_url`. Refs #2441. Attribution: https://github.com/headroomlabs-ai/headroom/pull/2445#issuecomment-5026212395 reported and narrowed the Business or Enterprise regression, and https://github.com/headroomlabs-ai/headroom/pull/2445#issuecomment-5026863498 scoped the shared-resolver follow-up that this change implements. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Centralize subscription hosted-route selection so explicit `GITHUB_COPILOT_API_URL` pins win on token exchange, explicit API token, and Copilot-token candidate resolution. - Normalize `api.business.githubcopilot.com` and `api.enterprise.githubcopilot.com` to `https://api.githubcopilot.com` by default, extending the existing individual-seat normalization. - Extend focused auth and wrapper tests so both subscription wrappers prove the corrected shared resolver output and the private-proxy isolation contract stays intact. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_copilot_auth.py -q`) - [x] Unit tests pass (`uv run pytest tests/test_cli/test_wrap_copilot.py -q`) - [x] Unit tests pass (`uv run pytest tests/test_cli/test_wrap_opencode.py -q`) - [x] Unit tests pass (`uv run pytest tests/test_cli/test_wrap_persistent.py -q`) - [x] Linting passes (`uv run ruff check .`) - [x] Formatting check passes (`uv run ruff format . --check`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_copilot_auth.py -q -> 84 passed uv run pytest tests/test_cli/test_wrap_copilot.py -q -> 31 passed uv run pytest tests/test_cli/test_wrap_opencode.py -q -> 44 passed in 143.46s uv run pytest tests/test_cli/test_wrap_persistent.py -q -> 31 passed uv run ruff check . -> All checks passed! uv run ruff format . --check -> 1331 files already formatted ``` ## Real Behavior Proof - Environment: Windows - Exact command / steps: Run the focused auth, Copilot wrapper, OpenCode wrapper, and persistent-proxy pytest files after implementing the shared resolver change, then ask lucasp1337 to rerun the Business or Enterprise `--copilot-subscription` scenario from PR https://github.com/headroomlabs-ai/headroom/pull/2445#issuecomment-5026212395 on a real seat. - Observed result: Focused auth and wrapper pytest runs passed locally, including the enterprise-host exchange reproduction row, explicit-pin precedence on all three producer paths, both subscription wrapper routes, and the private-proxy isolation regression. Live Business or Enterprise success stays behind reporter retest. - Not tested: live Business or Enterprise tenant run ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes No `CHANGELOG.md` edit is needed because Headroom generates release notes from conventional commits. Risk for maintainers: PR https://github.com/headroomlabs-ai/headroom/pull/641 manually validated a Business seat against the GitHub-returned hosted domain in June on `gpt-5.4`, so generic-by-default could affect tenants that genuinely require a dedicated host. This follow-up keeps the documented escape hatch intact by making `GITHUB_COPILOT_API_URL` win on every path. Live-seat proof boundary: lucasp1337 offered to retest on a Business or Enterprise seat in PR https://github.com/headroomlabs-ai/headroom/pull/2445#issuecomment-5026212395. Keep any live success claim behind that rerun. |
||
|
Parideboy
|
8c8fae0d0b
|
fix(proxy): reassemble server_tool_use.input from streamed partial_json (#2449)
## Description
Under `--target-ratio 0.4` a session died mid-run with a fatal Anthropic
400:
```
messages.13.content.0.server_tool_use.input: Input should be an object
```
Root cause is not compression of the request: the request path passes
structured blocks through byte-for-byte. It is **SSE stream
reconstruction**. When the proxy rebuilds a full Anthropic message from
the streamed response (non-stream retry, buffered, and CCR round-trip
paths), the `content_block_stop` handler parsed the accumulated
`_partial_json` into `input` only for blocks whose type was exactly
`tool_use`. A `server_tool_use` block streams its input identically via
`input_json_delta`, so its input was never reassembled: the block kept
the empty start-event `input: {}` and leaked the internal
`_partial_json` scratch key. That reconstructed block becomes assistant
history, and on the next turn the client replays it, so Anthropic
rejects `server_tool_use.input`. `--target-ratio` only makes the
buffered/reconstructed path more likely; it does not itself rewrite the
block.
Refs #2438 (Finding 2). Findings 1 (prompt-cache regression) and 3
(compression not engaging) are architectural and tracked separately.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/streaming.py` (`_parse_sse_to_response`):
gate the `content_block_stop` `_partial_json` → `input` parse on the
presence of `_partial_json`, not `type == "tool_use"`, so
`server_tool_use` (and any future tool-ish block) is reassembled. Always
strip the scratch key; `input` is always a parsed object (`{}` on
malformed/empty JSON).
- `headroom/ccr/response_handler.py`
(`StreamingCCRHandler._reconstruct_anthropic_response`): same
stop-handler fix, and relax the `input_json_delta` accumulator that was
likewise gated on `type == "tool_use"` so server_tool_use partial JSON
is accumulated at all.
- Regression tests in `tests/test_sse_thinking_blocks.py` and
`tests/test_ccr_response_handler_extra.py`: a `server_tool_use` whose
input arrives via `input_json_delta` must reconstruct to the parsed
object with no `_partial_json` leak.
- Leave `CHANGELOG.md` untouched, release-please generates it.
## Testing
- [x] Unit tests pass (`python -m pytest
tests/test_sse_thinking_blocks.py
tests/test_ccr_response_handler_extra.py -q`)
- [x] Linting passes (`ruff check`, `ruff format --check` on the four
changed files)
- [x] Type checking passes (`mypy headroom/proxy/handlers/streaming.py
headroom/ccr/response_handler.py --ignore-missing-imports`)
- [x] New tests added for new functionality
### Test Output
```text
$ python -m pytest tests/test_sse_thinking_blocks.py tests/test_ccr_response_handler_extra.py -q
26 passed in 3.36s
$ ruff check <changed files>
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.13, local dev checkout on a branch
off upstream/main
- Exact command / steps: Fed a synthetic Anthropic SSE stream with a
`server_tool_use` block whose `input` arrives as `input_json_delta`
partial JSON through both reconstructors (`_parse_sse_to_response`,
`_reconstruct_anthropic_response`); then temporarily restored the `type
== "tool_use"` guard and re-ran.
- Observed result: With the fix, the reconstructed block has `input ==
{"query": ...}` and no `_partial_json` key. With the old guard the test
fails, `input` stays `{}` and the scratch key leaks, reproducing the
malformed block that Anthropic rejects on replay.
- Not tested: End-to-end multi-turn `--target-ratio` session against the
live Anthropic API from this environment, reproduced at the
reconstruction seam instead; the reporter observed the 400 on real
traffic.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Parideboy
|
bec4cce8a9
|
feat(telemetry): record provider cache read/write/uncached tokens per request (#2450)
## Description The per-request JSONL feed (`--log-file`) collapsed all cache signal into a single `cache_hit: bool`, defined as `cache_read_tokens > 0 or from_response_cache`. A call that was billed cache-*creation* (write) with zero reads is therefore indistinguishable from a real cache-*read* hit. On Claude Code traffic where the proxy pays repeated cache writes, this hides the real economics from users (the issue's "cache_hit inverts the user's real economics" telemetry complaint). The provider-truth counters already ride on `RequestOutcome`, `cache_read_tokens`, `cache_write_tokens`, `uncached_input_tokens`, parsed from the upstream response usage on every path (`handlers/anthropic.py`, `handlers/streaming.py`, `backends/litellm.py`) but were dropped when the `RequestLog` entry was constructed in `emit_request_outcome`. This surfaces them per call. Refs #2438 (Finding 1, telemetry sub-item). The core prompt-cache preservation regression (Finding 1) and Finding 3 are architectural and tracked separately. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Add `cache_read_tokens`, `cache_write_tokens`, `uncached_input_tokens` (optional, default `0`) to `RequestLog` (`headroom/proxy/models.py`). Optional so existing consumers and serialized logs stay backward compatible. - Populate the three fields at the single log-emit site in `emit_request_outcome` (`headroom/proxy/outcome.py`) from the values already on `RequestOutcome`. `cache_hit` is unchanged. - Add `tests/test_proxy_cache_telemetry.py`: drive `emit_request_outcome` through the real proxy funnel with logging enabled and assert the JSONL entry carries the write/uncached deltas even when `cache_hit` is False; plus a default-value backward-compat check. - Leave `CHANGELOG.md` untouched release-please generates it. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_proxy_cache_telemetry.py -q`) - [x] Linting passes (`ruff check`, `ruff format --check` on the three changed files) - [x] Type checking passes (`mypy headroom/proxy/models.py headroom/proxy/outcome.py --ignore-missing-imports`) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_proxy_cache_telemetry.py -q 2 passed, 1 warning in 8.51s $ ruff check headroom/proxy/models.py headroom/proxy/outcome.py tests/test_proxy_cache_telemetry.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, local dev checkout on a branch off upstream/main - Exact command / steps: Built a `RequestOutcome` with `cache_read_tokens=0, cache_write_tokens=800, uncached_input_tokens=200` and ran it through `emit_request_outcome` against a real proxy app (`create_app`) with `log_requests=True` and a temp `log_file`, then read the JSONL back. - Observed result: The written entry carries `cache_read_tokens=0`, `cache_write_tokens=800`, `uncached_input_tokens=200` a cache-write-only call is now distinguishable from a cache-read hit in the log, where previously only `cache_hit` (False here) was recorded. - Not tested: A live Anthropic call end to end from this environment, the funnel is exercised with a synthetic outcome carrying real provider-usage values instead. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Rod Boev
|
9089e7f7d3
|
feat(opencode): support Copilot subscription backend for headroom models (#2441) (#2445)
## Description `headroom wrap opencode` currently routes `headroom/*` models only to ordinary Anthropic or OpenAI backends. A user with a GitHub Copilot subscription cannot point those `headroom/*` requests at the Copilot seat while keeping Headroom compression and stats, even though Headroom already has the validated subscription resolver, the proxy seed path, and the OpenCode provider route needed to do it. This PR adds `--copilot-subscription` to the OpenCode wrap command. It reuses the existing Copilot subscription token resolver, passes the validated endpoint and token seed into the existing proxy startup path, rejects unsupported runtime modes, and treats any non-empty Copilot API token as a private session seed so token-only sessions do not reuse a shared proxy. The generated OpenCode provider still targets the local proxy, and subscription secrets stay out of OpenCode config, environment, and terminal output. Closes #2441 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom wrap opencode --copilot-subscription` in `headroom/cli/wrap.py`. - Reuse the existing validated Copilot subscription resolver through one small required-resolution helper shared with the dedicated Copilot wrapper. - Pass the resolved endpoint and token seed into `_ensure_proxy()` as `openai_api_url`, `copilot_api_token`, `copilot_refresh_oauth_token`, and `copilot_api_token_expires_at`. - Reject `--copilot-subscription` with `--no-proxy`, `--prepare-only`, and translated backends before proxy or OpenCode launch. - Validate subscription mode before snapshotting OpenCode config, so rejected invocations don't create stale backups. - Scrub inherited Copilot proxy seed variables from the OpenCode child environment. - Treat any non-empty Copilot API token as a private session seed so token-only sessions do not reuse shared or persistent proxies. - Add focused OpenCode and persistent-proxy coverage for seed handoff, guard failures, direct-token isolation, secret non-disclosure, and unchanged non-subscription behavior. - Leave `CHANGELOG.md` untouched because Headroom generates changelog entries from conventional commits. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_cli/test_wrap_opencode.py tests/test_cli/test_wrap_persistent.py tests/test_cli/test_wrap_copilot.py -q`, `106 passed in 136.65s`) - [x] Linting passes (`uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_opencode.py tests/test_cli/test_wrap_persistent.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text Targeted subscription tests pass: OpenCode `7 passed, 37 deselected in 0.36s`, persistent proxy `2 passed, 29 deselected in 0.34s`, and dedicated Copilot `11 passed, 20 deselected in 0.39s`. Coverage includes inherited resolver-input env scrubbing, HEADROOM_BACKEND rejection, no-backup-on-rejection, OpenCode-only scrub scoping, and private-proxy teardown on config-injection failure. The full focused command `uv run pytest tests/test_cli/test_wrap_opencode.py tests/test_cli/test_wrap_persistent.py tests/test_cli/test_wrap_copilot.py -q` passed with `106 passed in 136.65s`. Ruff check and format check pass. ``` ## Real Behavior Proof - Environment: Windows, `uv` development environment, local CLI tests, no live Copilot seat on this host - Exact command / steps: run the focused OpenCode and persistent-proxy tests with a mocked `CopilotSubscriptionTokenResolution`, then capture the proof rows for seed handoff, direct-token isolation, guards, and secret non-disclosure - Observed result: Targeted subscription and proxy-seed tests pass, including OpenCode-only resolver-input env scrubbing and private-proxy teardown on config-injection failure; the full focused command passed with `106 passed in 136.65s`; Ruff check and format check pass. - Not tested: live Copilot subscription seat run on this host ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - Feature approval comes from the open `enhancement` label on https://github.com/headroomlabs-ai/headroom/issues/2441. - Keep the final live-backend claim behind manual owner proof. Local CLI tests can prove config, guard, secret, and proxy-seed behavior, but they cannot prove a real Copilot seat on this host. - `CHANGELOG.md` remains untouched because Headroom's release pipeline generates changelog entries from conventional commits. |
||
|
inix
|
4aac068814
|
fix(proxy/metrics): move the savings-ledger append off the event loop (#2439)
## Description `PrometheusMetrics.record_request` appends one durable JSONL event per compressed request. That append is synchronous: `open` + `fcntl.flock` + `write`, plus a full-file rewrite once the ledger passes 1 MB. It runs on the event loop, inside `self._lock`. `export()` takes that same lock and holds it for the entire Prometheus serialization, so a slow ledger write stops `/metrics` cold. In a repro run of 200 compressed requests, `/metrics` completed zero scrapes and the event loop never yielded once across 6.4 seconds. The append now runs in a thread, outside the lock. `savings_ledger` already takes its own `flock` across processes, so the metrics lock was never what made the write safe. Both halves are one change. Awaiting inside the lock would hold it for the whole write rather than just the syscall, which is worse than what is on main today. The file already documents this hazard against itself. `record_stage_timings` (`prometheus_metrics.py:867-874`) picks a plain `threading.Lock` over `self._lock` specifically because "the async lock is also held by `export()` during Prometheus scrapes." The ledger append was the pattern that docstring warns about. No filed issue for this one. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Move the `savings_ledger.record_savings_event` call in `record_request` out of `async with self._lock` and run it through `asyncio.to_thread`. The call site keeps its keyword arguments verbatim; `to_thread` forwards `**kwargs`, so no `functools.partial` wrapper is needed. - Keep the `await`. Callers still see the event on disk when `record_request` returns, which `tests/test_savings_ledger_before_forwarded.py` asserts synchronously. - Add `tests/test_savings_ledger_offload.py`: lock scope, event-loop responsiveness, durability on return, and both arms of the `tokens_saved > 0 and not stateless` gate. `savings_ledger.py` is untouched. It stays synchronous so the MCP `headroom_compress` caller in `ccr/mcp_server.py:789` does not have to change. Sizing the executor is left alone on purpose. `asyncio.to_thread` uses the default pool, which is the documented tool for blocking I/O and already the idiom here (`helpers.py:1297`, `server.py:1694`, `:3557`, `:3613`, `:4244`). The compression pools are sized `max(1, os.cpu_count())` for CPU-bound work, and `PrometheusMetrics` holds no reference to `HeadroomProxy` anyway, so reaching them would mean a new constructor parameter. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_savings_ledger_offload.py tests/test_savings_ledger.py tests/test_savings_ledger_before_forwarded.py -q ======================== 26 passed, 1 warning in 4.08s ========================= $ ruff check . && ruff format --check headroom/proxy/prometheus_metrics.py tests/test_savings_ledger_offload.py All checks passed! 2 files already formatted $ mypy headroom/proxy/prometheus_metrics.py Success: no issues found in 1 source file ``` Broader sweep across the blast radius, 145 test files matching savings / metrics / outcome / stats / proxy / handler / server / ledger / cost / prometheus, each run under a per-file wall-clock watchdog: ```text 138 files pass, 1470 tests passed 7 non-green: HANG tests/test_agent_savings.py HANG tests/test_ccr_mcp_server.py HANG tests/test_netcost_gate.py HANG tests/test_proxy_compress_endpoint.py HANG tests/test_proxy_mode_benchmark.py HANG tests/test_read_maturation_handler_nobust.py FAIL tests/test_proxy_copilot_auth_hooks.py::test_openai_passthrough_applies_copilot_auth Same 7 files re-run with headroom/proxy/prometheus_metrics.py reverted to |
||
|
Tejas Chopra
|
a7dcb9e91c
|
feat(transforms): pluggable lossless-compaction provider seam (#2433)
## Description
Adds an optional external provider for the information-preserving
compaction of protected (excluded) tool output, mirroring the existing
`proxy_extension` / `compressor` extension seams. Lets an out-of-tree
extension supply its own reversible compaction for excluded tools
without forking the router. Default behavior is unchanged.
Closes #
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
## Changes Made
- New `headroom/transforms/lossless_provider.py`:
`set_lossless_provider` / `get_lossless_provider`. Contract: `content ->
(compacted, kind) | None`, where `compacted` must be byte-recoverable
(or data-lossless for structured data), and the provider must be
deterministic and per-block so the prefix cache stays byte-stable across
turns.
- `ContentRouter._lossless_compact_excluded` consults a registered
provider first and is **authoritative** when one is set; it falls back
to the built-in folds only if the provider raises. With no provider
registered (the default) behavior is byte-for-byte identical to before.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
### Test Output
```text
$ python -m pytest tests/test_lossless_excluded_compaction.py -q
tests/test_lossless_excluded_compaction.py ........... [100%]
11 passed in 0.60s
$ ruff check headroom/transforms/lossless_provider.py headroom/transforms/content_router.py
All checks passed!
$ mypy headroom/transforms/lossless_provider.py headroom/transforms/content_router.py
Success: no issues found in 2 source files
```
## Real Behavior Proof
- Environment: local, Python 3.12 venv,
`ContentRouter(ContentRouterConfig())`.
- Exact command / steps: (1) default — call
`_lossless_compact_excluded(GREP)` with no provider; (2) register
`set_lossless_provider(lambda c: ("<<folded>>","custom"))` and call
again; (3) register a provider that raises.
- Observed result: (1) built-in search-heading fold `("…","search")`;
(2) returns `("<<folded>>","custom")` — provider is authoritative,
built-in not run; provider returning `None` yields `None` (no built-in
fallback); (3) provider exception → falls back to the built-in fold.
Covered by the 3 new tests.
- Not tested: the broad `tests/test_transforms/test_content_router.py`
suite stalls locally on model downloads (HF/ONNX); CI runs it.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
|
||
|
Abhay Singh
|
c400f90810
|
fix(copilot): preserve /v1 for the Anthropic /v1/messages endpoint (#2409) (#2414)
## Description Fixes #2409. GitHub Copilot Claude requests routed through Headroom return `404 page not found`. Copilot serves Claude models at `/v1/messages`, but Headroom forwards them to `/messages`, so the upstream 404s (observed on OpenCode's GitHub Copilot provider for `claude-haiku-4.5` / `claude-sonnet-4.6`, Headroom 0.32.0). ## Root cause `build_copilot_upstream_url` strips the `/v1` prefix from every Copilot path: ```python if normalized_path.startswith("/v1/"): normalized_path = normalized_path[3:] ``` That is correct for Copilot's **OpenAI-compatible** surface, which has no `/v1` (`/chat/completions`, `/responses`, `/embeddings`). But Copilot's **Anthropic** surface for Claude models is `/v1/messages` — with the `/v1`. Stripping it produces `https://api.githubcopilot.com/messages`, which 404s. Confirmed against the current code: ```text build_copilot_upstream_url("https://api.githubcopilot.com", "/v1/messages") -> "https://api.githubcopilot.com/messages" # 404 ``` ## Fix Keep `/v1` for the messages endpoint; still strip it for the OpenAI paths: ```python if normalized_path.startswith("/v1/") and not normalized_path.startswith("/v1/messages"): normalized_path = normalized_path[3:] ``` Now `/v1/messages` (and `/v1/messages/batches`) route to `.../v1/messages`, while `/v1/chat/completions` -> `/chat/completions` and `/v1/responses` -> `/responses` are unchanged, on both the public and GHE Copilot hosts. Non-Copilot upstreams are untouched. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/copilot_auth.py`: exclude `/v1/messages` from the `/v1`-strip in `build_copilot_upstream_url`. - `tests/test_copilot_auth.py`: assert `/v1/messages` (+ batches, + GHE host) keep `/v1` while the OpenAI paths still strip it. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/copilot_auth.py tests/test_copilot_auth.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/copilot_auth.py Success: no issues found in 1 source file # copilot_auth is import-light, so I ran the real function in the project venv # (uv sync): /v1/messages -> .../v1/messages, /v1/chat/completions -> /chat/completions. ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`. - Exact command / steps: called the real `build_copilot_upstream_url` before and after the change for `/v1/messages`, `/v1/messages/batches`, `/v1/chat/completions`, `/v1/responses`, `/v1/embeddings`, and a non-Copilot host. - Observed result: before, `/v1/messages` -> `.../messages` (the 404); after, `.../v1/messages`. Batches keep `/v1` too; the OpenAI paths still strip `/v1` (`/chat/completions`, `/responses`, `/embeddings`); `https://api.anthropic.com/v1/messages` is unchanged. Ran against the actual module. - Not tested: a live OpenCode -> Copilot Claude round trip; the added unit tests assert the URL construction directly. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes `copilot_auth` is a light module, so I verified the fix against the real function in the venv (output above) in addition to the unit tests. Scope is limited to the messages endpoint (the reported 404); every other Copilot path is byte-identical to before. |
||
|
Abhay Singh
|
0cbc0e8e54
|
fix(proxy/openai): replay incremental events in buffered Responses SSE (#2410) (#2415)
## Description Fixes #2410. When a streaming `/v1/responses` request has `headroom_retrieve` available, Headroom forces a non-streaming (`stream:false`) upstream call so CCR retrieval can be resolved server-side, then reconstructs the complete response as SSE for the client. GitHub Copilot returns 200 with real output tokens, but OpenCode shows no assistant response. Root cause: `_openai_responses_to_sse` emitted only two events — `response.created` and `response.completed`: ```python created_response = {**response, "status": "in_progress", "output": []} events = [("response.created", created_response), ("response.completed", response)] ``` Clients that read the whole answer off the terminal `response.completed` event work, but OpenCode and the Vercel AI SDK render output from the **incremental** item/text events (`response.output_item.added`, `response.output_text.delta`, ...). With those absent, the SDK displays nothing. ## Fix Reconstruct the real Responses event sequence: ``` response.created (status in_progress, empty output) response.in_progress for each output item: response.output_item.added (message items start with empty content) for each message content part: response.content_part.added (text blanked) response.output_text.delta (the text) response.output_text.done response.content_part.done response.output_item.done (full item) response.completed (full response) data: [DONE] ``` Non-message items (reasoning, function_call, ...) get `output_item.added` + `output_item.done` with the full item. Every event carries a contiguous `sequence_number`. The terminal `response.completed` still carries the full response, so clients that key off it are unaffected; clients that stream now receive the deltas they need. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/openai.py`: rewrite `_openai_responses_to_sse` to replay the incremental output-item/content-part/output-text events between `response.created`/`response.in_progress` and `response.completed`. - `tests/test_openai_responses_buffered_sse.py`: new test asserting the incremental `output_text.delta` (visible text), the per-item sequence for message vs non-message items, the empty-output case, and contiguous sequence numbers. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/proxy/handlers/openai.py tests/test_openai_responses_buffered_sse.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/openai.py # no errors in the changed file # _openai_responses_to_sse is a pure module-level function, so I ran the new # tests against the real code in the project venv (uv sync): 3 passed. ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`. - Exact command / steps: fed the real `_openai_responses_to_sse` a completed response with a reasoning item and a message item whose content is `output_text: "Hello world"`, plus an empty-output response and a function_call-only response. - Observed result: the stream now contains `response.output_text.delta` with `"Hello world"` at `output_index=1, content_index=0`, wrapped by `content_part.added/done` and `output_item.added/done`, with the reasoning and function_call items emitted as `output_item.added/done` and preserved whole; `response.created`/`in_progress` carry empty output while `response.completed` carries the full output; sequence numbers are `0..n`. Ran against the actual module. - Not tested: a live OpenCode -> Copilot Responses round trip; the added tests assert the event stream directly. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes `_openai_responses_to_sse` is a pure function, so I verified the fix against the real code in the venv (output above) in addition to the unit tests. This mirrors the incremental replay the Anthropic buffered path already does in `StreamingMixin._response_to_sse` (content_block_start/delta/stop), bringing the Responses buffered-CCR path to the same fidelity. |
||
|
Abhay Singh
|
170b04a74d
|
fix(install): carry upstream-routing env overrides into supervised deployments (#2429)
## Description Fixes #2240. `headroom install apply` builds the persistent deployment's environment from the `HEADROOM_*` family plus any explicit `--env KEY=VALUE`. It never captured the provider upstream-routing overrides that the interactive `headroom proxy` reads from the environment through `resolve_api_overrides` (`ANTHROPIC_TARGET_API_URL` and its `*_TARGET_API_URL` siblings). A supervised runner (launchd, systemd, cron, Windows service/task) starts from a bare environment, so those exports never reach the persistent proxy. The result: a user who exports `ANTHROPIC_TARGET_API_URL` pointing at their gateway and runs `install apply` gets a proxy that silently forwards to the default Anthropic endpoint instead. That is both a correctness bug and a routing surprise (traffic and keys can go to the wrong host). ## Fix Capture the documented `*_TARGET_API_URL` overrides from the current environment and merge them into the manifest env underneath the explicit `--env` map, so an explicit `--env` still wins. Scope notes: - Only URL overrides are auto-captured. The `*_TARGET_API_HEADERS` variables can carry bearer tokens, so those are deliberately left to an explicit `--env` rather than being persisted into the on-disk manifest implicitly. - The proxy already resolves these vars correctly at runtime; this only makes `install apply` hand them to the supervised process the same way the interactive proxy would inherit them. - `headroom deploy` (the Docker path) is left unchanged here; this targets the exact reported `install apply` flow. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/install.py`: add `_PASSTHROUGH_URL_ENV_VARS` and `_capture_passthrough_env`, and merge the captured overrides under the parsed `--env` map in `install_apply` before building the manifest. - `tests/test_cli/test_install_cli.py`: unit test for the capture helper (skips empty/unrelated vars), plus CliRunner tests that a set `ANTHROPIC_TARGET_API_URL` reaches `build_manifest`'s env and that an explicit `--env` overrides the captured value. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cli/test_install_cli.py -k "capture or captures or overrides" -q 3 passed $ uvx ruff@0.15.17 check headroom/cli/install.py tests/test_cli/test_install_cli.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/cli/install.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: ran the new CliRunner tests, which export `ANTHROPIC_TARGET_API_URL` via monkeypatch, invoke `install apply` with the supervisor side effects stubbed, and capture the kwargs handed to `build_manifest`. Also called the real `_capture_passthrough_env` and real `build_manifest` directly to confirm the value lands in `manifest.base_env`. - Observed result: with the var exported, `build_manifest` received it in `extra_env` and `manifest.base_env["ANTHROPIC_TARGET_API_URL"]` held the gateway URL; with an explicit `--env ANTHROPIC_TARGET_API_URL=...` the explicit value won; empty and unrelated vars were skipped. Ran against the actual modules. - Not tested: a live launchd/systemd run forwarding to a real gateway. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Abhay Singh
|
313c290df9
|
fix(proxy/openai): None-guard usage token counts on the chat path (#2431)
## Description
`handle_openai_chat` reads token counts from the response usage to
record metrics and update the prefix tracker:
```python
usage = backend_response.body.get("usage", {})
output_tokens = usage.get("completion_tokens", 0)
total_input_tokens = usage.get("prompt_tokens", optimized_tokens)
```
`.get(key, default)` only falls back when the key is **absent**. When an
OpenAI-compatible backend emits a key with a **null** value (providers
do this on a stopped or empty turn, the same shape that caused the
Gemini crash in #2347), `.get` returns `None`. That `None` then flows
into:
- `_infer_openai_cache_write_tokens(total_input_tokens,
cache_read_tokens)` → `max(input_tokens - cache_read_tokens, 0)` (a
`None - int` → `TypeError`),
- `uncached_input_tokens = max(0, total_input_tokens - cache_read_tokens
- cache_write_tokens)`, and
- `RequestOutcome(output_tokens=..., optimized_tokens=...)`, whose
fields are `int` and which the metrics recorder increments.
Both chat usage-extraction sites are affected. On the direct-provider
branch the arithmetic runs **outside** the surrounding `try`, so a
single such response raises an uncaught `TypeError` and 500s the
request; on the backend branch it corrupts outcome recording.
## Fix
Coerce the three counts with the existing module-level `_usage_int`
guard (`max(int(value), 0)`, 0 on failure) at both sites, matching the
streaming path, the already-guarded cache keys in the same block
(`usage.get("cache_read_input_tokens", 0) or 0`), and the Gemini fix in
#2347. A normal integer usage is unchanged; only a null (or absent)
value now becomes the fallback/0. `prompt_tokens` keeps its
`optimized_tokens` fallback so our own input estimate is used when the
count is missing.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/openai.py`: `_usage_int`-guard
`completion_tokens` / `prompt_tokens` / `cached_tokens` at both
non-streaming usage-extraction sites in `handle_openai_chat`.
- `tests/test_proxy/test_openai_chat_savings_profile.py`: regression
driving a `/v1/chat/completions` request whose backend usage reports
null `prompt_tokens` / `completion_tokens`, asserting a 200 instead of a
crash.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_proxy/test_openai_chat_savings_profile.py -q
2 passed
# with the fix reverted, the new test fails (the null-usage response 500s):
$ git stash push -- headroom/proxy/handlers/openai.py
$ python -m pytest tests/test_proxy/test_openai_chat_savings_profile.py::test_chat_completions_survives_null_usage_token_counts -q
1 failed
$ uvx ruff@0.15.17 check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_chat_savings_profile.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/openai.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: ran the new FastAPI `TestClient` regression,
which drives the real `handle_openai_chat` through a mock backend
returning `usage: {prompt_tokens: null, completion_tokens: null,
total_tokens: null}`; then reverted only `openai.py` and re-ran the same
test.
- Observed result: with the fix the request returns 200; with the fix
reverted the same request fails (the null count reaches the `max(...)`
arithmetic and outcome recording). Ran against the actual handler via
the app.
- Not tested: a live third-party OpenAI-compatible gateway emitting null
usage.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Abhay Singh
|
17ff13ccbe
|
fix(install): migrate deployments off the retired chopratejas image repo (#2427)
## Description Fixes #2426. Persistent Docker deployments store their image in the deployment manifest. The image org moved from the personal `ghcr.io/chopratejas/headroom` repo to the project org `ghcr.io/headroomlabs-ai/headroom`, and the personal repo is frozen at 0.27.0. Because the manifest image is only ever read back verbatim (`build_runtime_command`, `docker run`, status output), a deployment created before the move keeps pulling 0.27.0 forever, several minor versions behind the CLI, with no drift signal to the user. Two related gaps: - `headroom/install/state.py` reads the recorded image straight back with no migration, so an old manifest is stuck on the dead repo. - `headroom/cli/install.py` `deploy --image` still defaulted to `ghcr.io/chopratejas/headroom:latest`, so brand new deploys through that command also pinned the retired repo (the `install-apply` default was already correct). ## Fix - Rewrite the retired repo to the org repo when a manifest is loaded, in both `load_manifest` and `list_manifests`, preserving whatever tag was recorded. The rewrite is surgical: it only matches the exact retired `ghcr.io/chopratejas/headroom` repo and leaves already-current images and any third-party image untouched. The migrated value persists on the next apply/save. - Change the `deploy --image` default to `ghcr.io/headroomlabs-ai/headroom:latest` so it matches `install-apply`. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/install/state.py`: add `_migrate_deprecated_image` and apply it in `load_manifest` and `list_manifests` before constructing the manifest. - `headroom/cli/install.py`: `deploy --image` default now points at the org repo. - `tests/test_install/test_state.py`: new tests covering load and list migrating the retired repo (tag preserved) and leaving current/third-party images untouched. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/install/state.py headroom/cli/install.py tests/test_install/test_state.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/install/state.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`. - Exact command / steps: wrote a manifest.json pinning `ghcr.io/chopratejas/headroom:latest` (and `:0.27.0`) under a temp home, then called the real `load_manifest` and `list_manifests`. - Observed result: both returned a manifest with `image == ghcr.io/headroomlabs-ai/headroom:latest` (tag preserved on the `0.27.0` case too); an already-current image and a third-party image passed through unchanged. Ran against the actual module. - Not tested: a live `docker run` against the migrated image. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Abhay Singh
|
b976378c3e
|
test(pricing): stop asserting DeepSeek pricing freshness on wall-clock time (#2428)
## Description The shared `test` job is currently failing on every open PR because of a wall-clock time-bomb in the DeepSeek pricing tests, not because of any code change. `tests/test_providers/test_deepseek.py::TestDeepSeekPricingModule::test_registry_staleness_and_source_url` asserted: ```python assert not registry.is_stale() ``` `PricingRegistry.is_stale()` returns `(date.today() - last_updated) > timedelta(days=30)`. The DeepSeek registry ships `LAST_UPDATED = date(2026, 6, 19)`, so this assertion holds only while the current date stays within 30 days of that constant. Once it lapses, the test fails on time alone, turning the `test` shard red for every unrelated PR in the repo. It is failing right now (31 days past `LAST_UPDATED`). This is not testing code behavior: it only checks that the machine's clock is within 30 days of a hardcoded date. The sibling Anthropic and OpenAI registries are 560 days old and make no such assertion, so DeepSeek is the odd one out here rather than a deliberate freshness gate. ## Fix Drop the freshness assertion and keep the meaningful `source_url` check, renaming the test to `test_registry_source_url` to match what it now verifies. The staleness mechanism stays fully and time-independently covered by `tests/test_pricing.py::test_registry_staleness_and_warning`, which builds registries with `date.today() - timedelta(days=30)` (asserts not stale) and `date.today() - timedelta(days=31)` (asserts stale) plus the warning text. So this removes a fragile environmental assertion without reducing real coverage, and aligns DeepSeek with the Anthropic/OpenAI registries. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `tests/test_providers/test_deepseek.py`: remove the wall-clock-dependent `assert not registry.is_stale()`, keep the `source_url` assertion, rename the test to `test_registry_source_url`, and add a comment explaining why freshness is not asserted here. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check tests/test_providers/test_deepseek.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17`. - Exact command / steps: with the current date at 31 days past `LAST_UPDATED`, ran the registry's `is_stale()` and the fixed test body against the real modules, plus the mechanism test from `tests/test_pricing.py`. - Observed result: `get_deepseek_registry().is_stale()` is `True` on the current date (which is exactly what broke the old assertion); the fixed `test_registry_source_url` body passes regardless of the date; and `test_registry_staleness_and_warning` still passes, so the staleness mechanism remains covered. - Not tested: a live DeepSeek pricing fetch (out of scope; pricing values are unchanged). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Tejas Chopra
|
fd0e1a8afe
|
feat(wrap): boost Serena — symbol-first guidance, wrap-time pre-index, repo-language scoping (#2425)
When Serena is the active code-memory engine, `headroom wrap` now does three things (all best-effort, timeout-guarded, non-fatal, and fully inert when Serena/uvx are absent — mirroring the existing RTK/tokensave patterns): 1. **Symbol-first guidance** — injects a marker-guarded, idempotent block into the agent's hint file (`CLAUDE.md` for Claude; `AGENTS.md` for Codex/Grok/OpenCode) steering it to prefer Serena's `get_symbols_overview` / `find_symbol` / `find_referencing_symbols` / `find_declaration` over whole-file reads. This is the highest-leverage change — Serena only saves tokens if the agent actually uses it. 2. **Repo-language scoping** — detects the languages present in the repo (extension scan, pruning `.git`/`node_modules`/`.venv`/etc.) and pins them into `.serena/project.yml`'s `languages` list, so Serena doesn't spin up superfluous language servers. Conservative: only rewrites a single-line flow list or creates a minimal `project.yml`; a custom/block-style entry is left untouched to avoid corrupting hand-authored config. 3. **Wrap-time pre-index** — runs `serena project index` so the first symbol query isn't cold. Order is inject → scope → index (scope before index so the pre-index respects the scope). No new env vars, no settings_store drift, no behavior change outside the Serena path. The `languages` key and extension→language mapping were verified from Serena's local source (`project.template.yml`, `ProjectConfig`, `solidlsp/ls_config.py`), not the web. ## Testing New `tests/test_cli/test_wrap_serena_boost.py` (16 tests: injection idempotency + content, language detection incl. ignore-dirs, mocked pre-index/project.yml write incl. failure/timeout no-op). Updated `test_serena_migrate.py`'s fixture to neutralize the new side-effecting calls. Offline: 46 passed; ruff 0.15.17 + mypy clean. |
||
|
inix
|
7052d52dcb
|
fix(proxy/openai): cache under looked-up messages (#2420)
## Description
The OpenAI chat path caches responses under a different key than it
looked them up by. `handle_openai_chat` calls `cache.get(messages, ...)`
at request start, then the `pre_compress` hook reassigns `messages`
before `cache.set(messages, ...)`. When a deployment configures a
message-rewriting hook, the handler stores every response under a key no
future lookup can produce. The response cache never hits and fills with
unreachable entries until eviction, with no error signal.
This is the OpenAI twin of the anthropic fix in #2124 (which closed
#327). Same snapshot pattern: capture the lookup messages once before
the hook runs, reuse them verbatim at `cache.set`.
Related to #327, follow-on to #2124 (which fixed the anthropic side
only).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- Snapshot `cache_lookup_messages = messages` before the `pre_compress`
hook in `handle_openai_chat`, and cache the response under that snapshot
at `cache.set`. Mirrors the shipped anthropic pattern in
`handlers/anthropic.py`.
- Add `tests/test_openai_response_cache_key.py`: drives two identical
`/v1/chat/completions` requests through a message-rewriting
`pre_compress` hook against the real `SemanticCache`, and asserts the
repeat is served from cache (upstream called once) rather than re-sent.
This exercises the real cache-key function, which a get/set-argument
check does not.
- Document the ordering invariant at the snapshot: image compression
also rebinds `messages` but runs upstream of the snapshot, so a future
reorder that moved it below would reintroduce the drift.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_openai_response_cache_key.py tests/test_proxy_openai_cache_key_integration.py tests/test_backend_nonstreaming_cache_metrics.py tests/test_openai_codex_routing.py -q
31 passed, 1 warning in 17.43s
$ ruff check headroom/proxy/handlers/openai.py tests/test_openai_response_cache_key.py
All checks passed!
$ mypy headroom
Success: no issues found in 505 source files
```
## Real Behavior Proof
- Environment: headroom at `upstream/main`
|
||
|
inix
|
9b016f2b64
|
perf(content_router): dedupe content detection (#2419)
## Description
ContentRouter ran the native content detector two to three times on
identical content, on the hottest path in the proxy (every compressed
message, every request). This cuts it to once.
`_detect_content` isn't cheap and isn't memoized. It strips a detection
envelope, runs the Rust/Magika ONNX classifier, then several regex
passes. `compress()` ran it once for debug logging that's off by
default, then `_determine_strategy()` recomputed it (plus
`is_mixed_content`) on the same content. That's twice per `compress()`,
and three times on the `apply()` cache-miss path.
Closes: N/A (no filed issue, surfaced by an internal
contribution-backlog audit).
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `compress()` computes `is_mixed_content` and `_detect_content` once,
then threads both into `_determine_strategy` through new optional params
(`mixed`, `detection`).
- `_determine_strategy` uses the passed values when present, and
computes them itself when they're `None`. Its one private caller
changes. Any other caller keeps working.
- Added `tests/test_content_router_detection_dedup.py`. One test asserts
`compress()` detects exactly once (it fails before the fix at `assert 2
== 1`). The other asserts the threaded result routes the same as the
recomputed one across content types.
- Updated two existing `_determine_strategy` test doubles to take the
new kwargs.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_content_router_detection_dedup.py tests/test_transforms_content_router.py \
tests/test_transforms/test_content_router.py tests/test_transforms_content_detection.py -q
135 passed in 8.98s
$ pytest tests/test_transforms/ tests/test_content_router_*.py tests/test_router_*.py \
tests/test_lossless_excluded_compaction.py -q
423 passed, 62 skipped in 54.93s
$ ruff check .
All checks passed!
$ mypy headroom
Success: no issues found in 505 source files
```
## Real Behavior Proof
- Environment: macOS (Darwin), Python 3.13, headroom worktree on this
branch off `upstream/main`, `HF_HUB_OFFLINE=1
LITELLM_LOCAL_MODEL_COST_MAP=true`. A counter wraps the real
`_detect_content` and delegates to it, so real routing and compression
run.
- Exact command / steps: run the real router over one representative
message and count `_detect_content` calls on the fixed tree, then `git
stash` the source and count again on the unfixed tree. Covered
`router.compress(blob)` and `router.apply([tool_msg])`.
- Observed result: `compress()` dropped from 2 detection calls to 1, and
`apply()` dropped from 3 to 2, on the same input with the same routing
strategy (`text`) and the same output. The once-only test flips from
`assert 2 == 1` before to passing after.
- Not tested: production Magika ONNX timing. This dev env has no
onnxruntime, so the detector ran its regex fallback tier, which makes
the saved cost a floor, not a ceiling. I also scoped out the Tier B
extension (threading the `apply()` Pass-1 detection into `compress()`)
on purpose.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A
## Additional Notes
Scope is the default routing path. `force_kompress` already uses the
cheaper regex detector, so it never paid the redundant native cost.
`_compress_mixed` re-detects per split section, but that's different
content (sub-sections), so it's out of scope.
The `apply()` Pass-1 detection stays. It gates the `is_code` protection
check for every message, including cache hits that never reach
`compress()`. Threading it into `compress()` would widen a shared task
tuple and change the public `compress()` signature, all for a
cache-miss-only save, so I left it as a possible follow-up.
Doc checklist item is N/A (internal perf dedup, no user-facing docs
change). This is a Python-only change, so the first push will use
`--no-verify` for the known `ci-precheck` Rust-latency bench flake
(`classify_under_10us_per_call`), which runs clean in CI.
|