mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
2194 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tejas Chopra
|
cb6c828457
|
fix(proxy): one bad extension no longer aborts proxy startup (#2215)
## What
`install_all()` (the `headroom.proxy_extension` loader) previously let
any exception from an extension's `install()` **propagate and abort
proxy startup** — one broken or version-incompatible third-party
extension took the whole proxy down, and every other extension with it.
This makes extension loading resilient:
- catch a failing `install()`, log it (with traceback), record it as
**skipped**
- continue installing the rest — a failure disables that one extension,
not the proxy
- print a `SKIPPED` line to the console (the startup banner lists
*enabled* extensions before install runs, so a skip would otherwise be
logging-config dependent)
## Why
Found while testing several proxy extensions together in a clean venv: a
plugin built against a newer core API raised `ModuleNotFoundError` from
`install()` and crashed the proxy at startup. An extension that fails
its own environment/auth check should disable itself — it should not
take the whole proxy down.
## Real behavior proof
Before — one extension failing in `install()`:
```
... proxy did NOT come up (/livez never answered)
```
After — same setup, one extension deliberately broken:
```
[headroom] proxy extensions SKIPPED: myorg_ext (install failed — running without them; see logs)
/livez: 200 healthy # proxy up; the other extensions installed
```
Loader unit check (fake failing extension):
```
returned installed: ['good_ext'] # bad one excluded
bad_ext skipped (not in installed): True
good_ext survived: True
warning logged for bad_ext: True
```
## Tests
- `mypy headroom/proxy/extensions.py` → `Success: no issues found`
- `ruff check headroom/proxy/extensions.py` → `All checks passed!`
- Verified in-process (catch/skip/continue + logging) and end-to-end
against a running proxy (`/livez` 200 with a deliberately failing
extension).
## Maintainer Follow-up
- Added `tests/test_proxy_extensions.py` covering skip-and-continue
behavior for a failed extension and the missing-extension warning path.
- Removed an informal implementation comment from
`headroom/proxy/extensions.py`.
- Validation on `
|
||
|
Tejas Chopra
|
79d8056fd7
|
fix(mcp): regenerate stale server.json (0.27.0 -> 0.32.0) (#2218)
## Description The committed `server.json` pinned version `0.27.0` while `pyproject.toml` is at `0.32.0`. `tests/test_mcp_registry/test_server_json.py::test_root_server_json_matches_builder` asserts the committed artifact equals `render_server_json()`, so it fails on `main`. This regenerates `server.json` from the current metadata. Found while getting the security PR (#2207) CI green. The two other pre-existing failures it was grouped with were **already fixed on `main`** by recent commits — `test_cold_start_fast_pass` (`record_compression_failed` added to the metrics double) and `test_cli/test_wrap_zcode` (watcher mock now passes the port) — so this PR only needs the `server.json` regen. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Regenerated `server.json` from `render_server_json()` so the committed artifact matches the current package version (`0.32.0`). ## Testing - [x] Unit tests pass (`pytest`) — the previously-failing tests - [x] Linting passes (`ruff check`) ### Test Output ```text $ pytest tests/test_mcp_registry/test_server_json.py::test_root_server_json_matches_builder \ tests/test_cold_start_fast_pass.py::test_fast_pass_failure_falls_back_to_full_deferral \ tests/test_cli/test_wrap_zcode.py::test_wrap_prints_proxy_urls -q 3 passed ``` ## Real Behavior Proof - Environment: branch off current `main` (`ea3d5a86`), Python 3.12, project `.venv`. - Steps: `python -c "from headroom.mcp_registry import render_server_json; open('server.json','w').write(render_server_json())"`, then ran the MCP registry test. - Observed: `server.json` `version` → `0.32.0`; `test_root_server_json_matches_builder` passes. - Not tested: full suite (single generated-artifact change). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] Documentation changes (N/A) - [x] My changes generate no new warnings - [ ] Tests added (N/A — regenerates an artifact an existing test already guards) - [x] New and existing unit tests pass locally - [ ] CHANGELOG (N/A) ## Additional Notes `server.json` is a generated artifact (`headroom/mcp_registry/server_json.py`) — regenerate with `render_server_json()` after any version bump. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
JD Davis
|
ea3d5a86b7
|
fix(deps): clear Dependabot lockfile alerts (#2175)
## Description Clears the current dependency/security-audit blockers that are making unrelated PRs red: - `transformers 5.3.0` / `CVE-2026-5241`, fixed by requiring `transformers>=5.5.0` in the locked optional dependency set. - `sqlitedict <=2.1.0` via the optional `benchmark` extra's `lm-eval[api]` dependency. There is no patched `sqlitedict` release, so this PR removes the published/locked `benchmark` extra instead of shipping a known-vulnerable transitive dependency. - `esbuild >=0.27.3,<0.28.1` in the OpenCode plugin lockfile, fixed by forcing `esbuild@0.28.1` through the OpenCode npm override and regenerated lockfile. The benchmark code still invokes `python -m lm_eval`; researchers who need that harness should install `lm-eval[api]` in their benchmark environment until its transitive vulnerability has a patched release. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `pyproject.toml`: remove the `benchmark` optional extra, document external `lm-eval[api]` installation guidance, and require `transformers>=5.5.0`. - `uv.lock`: regenerate without the `benchmark` extra, removing `lm-eval` and `sqlitedict` lock entries and locking the patched transformers floor. - `plugins/opencode/package.json`: add an `overrides` entry for `esbuild@0.28.1`. - `plugins/opencode/package-lock.json`: regenerate the OpenCode lockfile with `esbuild@0.28.1`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv lock --check rg -n -F 'sqlitedict' uv.lock # no matches rg -n -F 'name = "lm-eval"' uv.lock # no matches rg -n -F "extra == 'benchmark'" uv.lock # no matches rg -n -F '0.27.7' plugins/opencode/package-lock.json plugins/opencode/package.json # no matches npm ls esbuild --package-lock-only npm audit --package-lock-only # found 0 vulnerabilities git diff --check ``` Previous GitHub checks were green. After merging current `main`, fresh GitHub checks are running again; local targeted validation still passes. ## Real Behavior Proof - Environment: Windows 11, Python 3.13.3, uv, npm in `plugins/opencode`, Dependabot/pip-audit alert metadata from the failing PR jobs. - Exact command / steps: inspected the regenerated Python and npm lockfiles with `rg`, checked the uv lock with `uv lock --check`, checked OpenCode's dependency tree with `npm ls esbuild --package-lock-only`, and ran `npm audit --package-lock-only`. - Observed result: `uv.lock` no longer contains `sqlitedict`, `lm-eval`, or a `benchmark` extra marker; `transformers` resolves at the patched `>=5.5.0` floor; OpenCode's lock resolves `esbuild@0.28.1`; `npm audit --package-lock-only` reports 0 vulnerabilities; GitHub `Dependency audit (pip-audit)` passes. - Not tested: running the external `lm-eval` harness after installing it separately. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A - dependency and lockfile security fix. ## Additional Notes The `benchmark` extra can be restored once the upstream `lm-eval[api]` dependency chain stops pulling a vulnerable `sqlitedict` release. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Rod Boev
|
dbbef4bd41
|
fix(code-compressor): recover valid Python rewrites after local syntax rejection (#2202)
## Description A compile-invalid Python definition rewrite currently makes `CodeAwareCompressor` discard every otherwise valid rewrite in the file and return the original source at 0 percent reduction. The existing whole-file safety guard stays in place, while a Python-only recovery replay now preserves the rejected definition and keeps independent valid compression. The recovery reuses the current Python validation authority in `ast.parse()` plus `compile()`, runs only after the first assembled module already fails `_verify_syntax()`, and stays out of non-Python paths. Closes #1233 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a Python-only recovery replay after the first assembled module fails syntax validation. - Preserved only the invalid function or class rewrite while allowing independent valid definitions to remain compressed. - Kept the existing whole-file syntax guard and original-source fallback as the terminal safety check. - Added focused invalid-node, valid-modern-syntax, and fail-safe coverage. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_transforms/test_code_compressor.py::TestRealASTRuns::test_python_invalid_node_falls_back_locally tests/test_transforms/test_code_compressor.py::TestRealASTRuns::test_python_recovery_does_not_block_valid_modern_syntax tests/test_transforms/test_code_compressor.py::TestRealASTRuns::test_python_recovery_still_returns_original_when_all_candidates_invalid tests/test_transforms/test_code_compressor.py::TestEdgeCases::test_syntax_errors_in_input tests/test_transforms/test_code_compressor.py::TestRealASTRuns::test_ast_preserves_structure_for_python -v`) - [x] Linting passes (`uv run ruff check headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_transforms/test_code_compressor.py::TestRealASTRuns::test_python_invalid_node_falls_back_locally tests/test_transforms/test_code_compressor.py::TestRealASTRuns::test_python_recovery_does_not_block_valid_modern_syntax tests/test_transforms/test_code_compressor.py::TestRealASTRuns::test_python_recovery_still_returns_original_when_all_candidates_invalid tests/test_transforms/test_code_compressor.py::TestEdgeCases::test_syntax_errors_in_input tests/test_transforms/test_code_compressor.py::TestRealASTRuns::test_ast_preserves_structure_for_python -v 5 passed in 0.34s uv run ruff check headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py All checks passed! uv run ruff format headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py --check 2 files already formatted ``` ## Real Behavior Proof - Environment: Windows, synced `uv` environment with `dev` and `code` extras installed - Exact command / steps: run the focused invalid-node regression through public `compress(..., language="python")` - Observed result: `1 passed in 0.19s`; the invalid candidate stays original, the neighboring valid candidate remains compressed, and `headroom-PR-TARGET-1233-PROOF.md` records the base `ratio=1.0` whole-file rollback against the fixed head behavior. - Not tested: the stale future-import mismatch discussed in the old issue comment, already covered on current main ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - The stale future-import comment on #1233 is not the live slice here; current main already validates Python with `compile()` and already covers that ordering case. - This fix keeps the existing whole-file fail-safe and does not broaden into cross-language recovery or new syntax models. - `CHANGELOG.md` remains unchanged because Headroom generates release notes from conventional commits. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Rod Boev
|
8522fcbc40
|
fix(code): quarantine Perl parser from code-aware compression (#2204)
## Description `headroom proxy` can wedge when code-aware compression enters the tree-sitter Perl external scanner and the native scan keeps the GIL indefinitely. Current main still has two routes into that scanner: explicit `perl` or `pl` hints flow through `CodeAwareCompressor`, and `detect_language()` can nominate Perl on non-Perl code because its prefilter matches generic sigils such as decorators, JSDoc tags, and shell variables before phase 2 parses every surviving candidate grammar. This change quarantines Perl at the code-aware compression funnel without widening scope. Perl remains recognized at the input boundary, but live proxy compression no longer requests a Perl parser. Non-Perl code keeps its existing code-aware behavior. Real Perl falls back through the existing safe Kompress or passthrough contract instead of entering tree-sitter. The diff stays inside `headroom/transforms/code_compressor.py` plus focused parser-safety regressions. Refs #2185 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a Perl quarantine list in `headroom/transforms/code_compressor.py` and used it to stop Perl candidate parsing during language detection. - Added a hard `_get_parser()` guard so no live code-aware path can construct a Perl parser. - Routed resolved explicit Perl hints through the existing safe fallback or passthrough contract before AST compression. - Added focused parser-safety regressions for non-Perl candidate bleed, explicit `perl` and `pl`, inferred real Perl, mixed fenced Perl content, fallback-disabled passthrough, and non-Perl negative space. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_perl_scanner_safety.py -q`) - [x] Linting passes (`uv run ruff check headroom/transforms/code_compressor.py tests/test_perl_scanner_safety.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_perl_scanner_safety.py -q 8 passed in 1.93s uv run ruff check headroom/transforms/code_compressor.py tests/test_perl_scanner_safety.py All checks passed! uv run ruff format headroom/transforms/code_compressor.py tests/test_perl_scanner_safety.py --check 2 files already formatted ``` ## Real Behavior Proof - Environment: Windows, Python from the synced `uv` environment, `dev` and `code` extras installed, no provider call - Exact command / steps: Run `uv run pytest tests/test_perl_scanner_safety.py -q`. - Observed result: `8 passed in 1.93s`; the suite proves explicit `perl` and `pl`, inferred real Perl, mixed fenced Perl, and non-Perl negative-space routes all avoid Perl parser entry. - Not tested: the reporter's macOS payload and long-running concurrent workload ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Use `Refs #2185`, not `Closes #2185`. The wedge surface is this PR's scope, but #2185 also carries a separate orphaned `headroom mcp serve` report that this slice does not address. - This is a reachability fix. It does not repair the upstream Perl scanner and it does not harden other grammars against the same class of native wedge. - `CHANGELOG.md` remains unchanged because Headroom generates release notes from conventional commits. - Merged PR https://github.com/headroomlabs-ai/headroom/pull/2114 addresses a different cooperative compression stall and stays separate from this native parser-entry slice. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
David Wells
|
2a954b69b4
|
feat(wrap): add ZCode desktop app support (#1845)
## Description Add `headroom wrap zcode` and `headroom unwrap zcode` commands for the ZCode desktop app (zcode.z.ai). ZCode is a desktop Electron IDE built by Z.AI, optimized for GLM-5.2 models. It has no CLI binary, so this follows the Pattern-B (proxy-only, print instructions) approach — same as Cursor, Cline, and Continue. **Upstream auto-detection:** `headroom wrap zcode` now reads `~/.zcode/v2/config.json` to detect the enabled provider and automatically configures the proxy upstream — no manual flags needed. Closes #1844 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - New module: `headroom/providers/zcode/__init__.py` and `runtime.py` (ZCodeProxyTargets, ZCodeUpstream, build_proxy_targets, detect_upstream, upstream_to_proxy_urls, render_setup_lines) - New CLI command: `headroom wrap zcode` in `headroom/cli/wrap.py:4815` — starts proxy, injects RTK into AGENTS.md, prints Base URL setup instructions - New CLI command: `headroom unwrap zcode` in `headroom/cli/wrap.py:5960` — removes RTK markers, stops proxy - New helper: `zcode_config_dir()` in `headroom/install/paths.py` - Updated `_run_proxy_only_watcher` to accept `anthropic_api_url`/`openai_api_url` params - Updated README.md: ZCode row in compatibility matrix, unwrap list, wrap command list - Updated CHANGELOG.md: entry under [Unreleased] > Added ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) — pre-existing numpy type stubs issue prevents full mypy run - [x] New tests added for new functionality (24 tests in `tests/test_cli/test_wrap_zcode.py`) - [x] Manual testing performed ### Test Output ```text tests/test_cli/test_wrap_zcode.py ........................ [100%] 24 passed, 1 warning in 0.22s ``` ## Real Behavior Proof - Environment: macOS 15.5, Python 3.12.13, headroom installed via `pip install -e .[dev]` - Exact command / steps: `headroom wrap zcode --port 9000` then `headroom unwrap zcode --port 9000` - Observed result: Wrap detects provider from `~/.zcode/v2/config.json` (e.g. "Z.ai Coding"), starts proxy with correct upstream on port 9000, injects RTK into AGENTS.md, prints detected provider + upstream + Base URL setup instructions. Unwrap removes RTK markers, deletes empty AGENTS.md, stops proxy. - Not tested: Actual ZCode app integration (ZCode is a desktop Electron app; Base URL configuration is manual in Settings > Model Settings) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas — N/A: code follows existing patterns - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI-only changes ## Additional Notes - **Pattern-B approach:** ZCode is a desktop Electron app with no CLI binary. Same pattern as Cursor, Cline, and Continue — proxy-only, print instructions. - **Upstream auto-detection:** Reads `~/.zcode/v2/config.json`, finds the enabled provider, and passes its `baseURL` to the proxy. Falls back to Z.ai Anthropic endpoint if no config found. - **httpProxy investigation:** ZCode has an `httpProxy` setting in `~/.zcode/v2/setting.json`, but it is an Electron-level forward proxy (CONNECT tunneling), incompatible with headroom reverse proxy. The Base URL approach in Model Settings is the correct integration point. - **No dependencies added:** This PR adds zero new dependencies. --------- Co-authored-by: Epicism <epicism@Epiphanie.local> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
ninosat00
|
5709291914
|
chore(release): harden local artifact smokes (#1824)
## Description
Harden the release artifact workflow so maintainers can reproduce npm
and Python release checks locally and so OpenClaw packaging does not
depend on a not-yet-published SDK version. This follow-up also keeps the
OpenClaw loader export contract explicit and updates the PR after the
branch was merged with current `headroomlabs/main`.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update
## Changes Made
- Added reusable local release smoke scripts for npm assets, Python
wheel/sdist artifacts, and the combined release gate.
- Switched the release workflow npm asset build to the reusable npm
builder.
- Made OpenClaw release asset building install against the just-built
local SDK tarball before rewriting packed metadata to the release
dependency range.
- Kept the OpenClaw source dependency registry-installable at
`headroom-ai@^0.22.3` while the packed artifact still ships
`^<release-version>`.
- Added `registerHeadroomPlugin` as a named export while preserving the
default `{ register }` OpenClaw loader contract.
- Added Windows development bootstrap docs/script and regression tests
for version sync, npm asset ordering, OpenClaw source installability,
and Python wheel smoke import isolation.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
uv run --with pytest python -m pytest tests/test_release_workflows.py scripts/tests/test_version_sync.py -q
44 passed, 1 warning
node --check scripts/build_npm_release_assets.mjs
node --check scripts/verify_npm_release_assets.mjs
python -m py_compile scripts/build_python_release_smoke.py scripts/release_smoke_all.py scripts/version-sync.py
python scripts/verify-versions.py
All versions aligned at 0.31.0
npm ci (plugins/openclaw)
added 88 packages, audited 89 packages, found 0 vulnerabilities
python scripts/release_smoke_all.py --out release-assets-local/all-pr1824-postmerge-fixed-20260710-104739
Verified npm release assets for 0.31.0
wheel metadata OK: headroom_ai-0.31.0-cp310-abi3-win_amd64.whl contains headroom/_core.pyd
sdist License-File metadata OK: ['LICENSE', 'NOTICE']
smoke-import OK: version=0.31.0 hello=headroom-core
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.14.3, Node v24.14.0, npm 11.9.0, uv
0.11.15, Rust/Cargo already installed in the local development
environment.
- Exact command / steps: Ran `python scripts/release_smoke_all.py --out
release-assets-local/all-pr1824-postmerge-fixed-20260710-104739` after
syncing this branch with current `headroomlabs/main`.
- Observed result: The command built `headroom-ai-0.31.0.tgz`,
`headroom-openclaw-0.31.0.tgz`,
`headroom_ai-0.31.0-cp310-abi3-win_amd64.whl`, and
`headroom_ai-0.31.0.tar.gz`; npm verifier passed; the wheel installed
into a fresh venv and imported `headroom._core`.
- Not tested: Full GitHub Actions release matrix and publish jobs with
real PyPI/npm/GitHub Packages credentials.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A.
## Additional Notes
The post-merge full smoke initially failed because the OpenClaw source
package depended on `headroom-ai@^0.31.0`, which is not available on
public npm yet. The builder now installs OpenClaw against the just-built
local SDK tarball before build/pack, and then rewrites packed metadata
to `^0.31.0`.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Doyeon Baek
|
36577d9547
|
fix(search_compressor): don't let a date in a path hijack the line-number parse (#2084)
## Description
`SearchCompressor::parse_match_line` splits a grep/ripgrep line into
`(file, line_number, content)` by finding the **leftmost**
`<sep><digits><sep>` triplet, where `<sep>` is `:` or `-`. A path
segment that itself contains such a triplet hijacks the parse — and that
shape is everyday, not exotic:
| real ripgrep line | parsed as |
|---|---|
| `logs/2026-05-03/app.log:12:ERROR boom` | `("logs/2026", 5,
"03/app.log:12:ERROR boom")` |
| `advisories/CVE-2021-44228.md:8:Log4Shell` | `("advisories/CVE", 2021,
"44228.md:8:Log4Shell")` |
| `src/v1-2-beta/mod.rs:3:fn x()` | `("src/v1", 2, "beta/mod.rs:3:fn
x()")` |
| `migrations/20240101-002-add_users.sql-9-…` | `("migrations/20240101",
2, "add_users.sql-9-…")` |
**This is silent corruption, not a drop.** The parse *succeeds*, so the
line is never counted in `stats.lines_unparsed` and never falls back to
passthrough. The bogus path becomes the **grouping key** in
`parse_search_results`, so unrelated files collapse into one bucket, and
the bogus path + line number + mangled body are what get scored, capped,
and rendered into the compressed output handed to the model. **The LLM
is shown a file and a line that do not exist.**
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
One file, one function:
`crates/headroom-core/src/transforms/search_compressor.rs`.
`parse_match_line` becomes a 3-tier scan:
- **Colon tier** — leftmost `:\d+:` whose path part contains no
whitespace. `:` is grep's *match* separator and a path practically never
contains one (the Windows drive colon is already skipped by the existing
`scan_start` logic), so leftmost is right. The whitespace bound stops a
`foo.rs:12:` reference *inside the body* of a `-` context line from
hijacking the parse.
- **Dash tier** — **last** `-\d+-` whose path part contains no
whitespace. `-` is grep's *context* separator, and unlike `:` it
genuinely appears inside real paths (`2026-05-03`, `CVE-2021-44228`,
`20240101-002-…`), so the marker is the *last* triplet in the path
token, not the first.
- **Permissive tier** — the original leftmost-any rule, byte-for-byte
unchanged. Only reached when neither typed tier matched (e.g. a path
containing a space), so those lines behave exactly as before.
- Also tightened in the typed tiers: the closing separator must equal
the opening one — grep emits `file:12:body` or `file-12-body`, never a
mix.
- Added 4 tests: 2 reproducing the bug, 2 regression guards against the
naive fixes.
**Safety argument (verified by execution):** with `parse_match_line`
temporarily forced to the Permissive tier alone, all 18 pre-existing
`search_compressor` tests still pass — i.e. the fallback is a faithful
reproduction of today's rule, so the change can only *add* correct
parses on lines a typed tier claims, never remove one.
This is the next bug in a family the module already tracks: the doc has
a "Bug fixes vs Python" section and three `fixed_in_3e2_*` tests
hardening this same parser against Windows drive colons and dashes in
filenames. `pre-commit-config.yaml-42-…` (dash before a *non*-digit) is
covered; `2026-05-03` (dash before a digit run followed by another dash)
was not.
## Testing
- [x] Unit tests pass
- [x] Linting passes (`cargo clippy -p headroom-core --all-targets` → 0
warnings)
- [x] Formatting passes (`cargo fmt --all -- --check`)
- [x] New tests added (4: 2 reproducing the bug, 2 regression guards
against naive fixes)
- [x] Manual testing performed (see Real Behavior Proof)
### Test Output
**Before the fix** (new tests run against the unmodified scan rule):
```text
$ cargo test -p headroom-core --lib search_compressor
---- transforms::search_compressor::tests::date_stamped_path_is_not_misread_as_line_number_marker stdout ----
assertion `left == right` failed
left: Some(("logs/2026", 5, "03/app.log:12:ERROR boom"))
right: Some(("logs/2026-05-03/app.log", 12, "ERROR boom"))
---- transforms::search_compressor::tests::date_stamped_paths_are_not_collapsed_into_one_bogus_file stdout ----
assertion `left == right` failed
left: ["logs/2026"]
right: ["logs/2026-05-03/app.log", "logs/2026-05-04/app.log"]
test result: FAILED. 18 passed; 2 failed; 0 ignored
```
**After the fix:**
```text
$ cargo test -p headroom-core --lib search_compressor
test result: ok. 20 passed; 0 failed; 0 ignored; 835 filtered out
$ cargo test -p headroom-core --lib # whole crate — no regressions
test result: ok. 854 passed; 0 failed; 1 ignored
$ cargo test -p headroom-parity
test result: ok. 4 passed; 0 failed
$ cargo fmt --all -- --check -> OK
$ cargo clippy -p headroom-core --all-targets -> 0 warnings, 0 errors
```
Regression guards added for the two ways a naive fix breaks:
- `digit_terminated_path_still_parses_ripgrep_context_line` —
`logs/app.log.1-42-rotated line` (path ends in a digit, so the context
separator is digit-preceded).
- `body_line_reference_does_not_hijack_a_context_line` —
`src/main.py-44-see foo.rs:12:bar` (body quotes a `file:line:`
reference).
## Real Behavior Proof
Per CONTRIBUTING — unit tests alone don't prove user-visible behavior,
so this was reproduced against the **released build** (`headroom-ai`
0.26.0 from PyPI, the compiled `_core.abi3.so`), driving the **public
`SearchCompressor.compress()` API** on **real `rg` output over real
files on disk** — not fixtures or mocks.
- Environment: macOS (Darwin 25.5.0, arm64), Python 3.13, released
`headroom-ai` 0.26.0 (`site-packages/headroom/_core.abi3.so`); patched
build = this branch compiled with `cargo build --release -p
headroom-py`, rustc 1.96.0.
- Exact command / steps: created 20 real log files at
`logs/2026-05-01/app.log` … `logs/2026-05-20/app.log` (12 real `ERROR`
lines each); ran `rg -n ERROR logs > rg_big.txt` (240 real match lines);
then called
`SearchCompressor(SearchCompressorConfig()).compress(open("rg_big.txt").read())`
on the shipped 0.26.0 build and on the patched build, comparing
`files_affected`, the rendered output, and whether each referenced path
exists on disk.
- Observed result: on shipped 0.26.0, the 20 distinct real files
collapse into **1 bogus bucket** `logs/2026` (a path that does **not**
exist on disk), per-line paths are mangled to
`logs/2026:5:01/app.log:10:`, 19 of 20 files effectively vanish from the
output, and `lines_unparsed: 0` means **nothing signals the
corruption**. On the patched build, same input and same API:
`files_affected: 20` (matches reality), every path in the compressed
output exists on disk (`all_exist=True`), and per-file match counts and
line numbers are correct.
- Not tested: the end-to-end proxy path (`headroom-proxy` against a live
LLM provider) — I exercised the `SearchCompressor` public API directly,
which is the surface `SearchOffload` and the MCP `headroom_compress`
tool wrap. I also did not test Windows path behavior on an actual
Windows host (the existing `scan_start` drive-letter logic is untouched,
and its tests still pass).
**Observed on the SHIPPED 0.26.0 build (the bug, in the released
product):**
```text
SHIPPED headroom 0.26.0 | real `rg -n ERROR logs` output, 240 lines
lines_unparsed : 0 <-- corruption is SILENT: nothing reported as unparsed
original_match_count: 240
files_affected : 1 <-- 20 distinct real files collapsed into ONE bucket
=== compressed output actually handed to the model ===
logs/2026:5:01/app.log:10:ERROR failure 1 connection refused upstream timeout on 2026-05-01 ...
logs/2026:5:20/app.log:21:ERROR failure 12 connection refused upstream timeout on 2026-05-20 ...
logs/2026:5:01/app.log:11:ERROR failure 2 connection refused upstream timeout on 2026-05-01 ...
[... and 235 more matches in logs/2026]
[240 matches compressed to 5. Retrieve more: hash=39c894009014d42b856ddd8a]
=== do the file paths in that output exist on disk? ===
logs/2026 exists_on_disk=False
```
**Observed on the PATCHED build (same input, same API, only the patch
differs):**
```text
PATCHED headroom-core | same real `rg` output, 240 lines
lines_unparsed : 0
original_match_count: 240
files_affected : 20 <-- was 1 (bogus) on the shipped build
=== compressed output handed to the model ===
logs/2026-05-01/app.log:10:ERROR failure 1 connection refused upstream timeout on 2026-05-01 ...
logs/2026-05-01/app.log:11:ERROR failure 2 connection refused upstream timeout on 2026-05-01 ...
[... and 7 more matches in logs/2026-05-01/app.log]
logs/2026-05-02/app.log:10:ERROR failure 1 connection refused upstream timeout on 2026-05-02 ...
=== do the file paths in that output exist on disk? ===
logs/2026-05-01/app.log exists_on_disk=True
logs/2026-05-02/app.log exists_on_disk=True
...all distinct paths referenced, all_exist=True
```
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
## Additional Notes
**Known residual ambiguity (stating it rather than hiding it).** grep
output is inherently ambiguous — `logs/2026-05-03/x:12:y` *could*
legitimately be a file literally named `logs/2026` with context line 5.
The tiers pick the overwhelmingly more likely reading. Two contrived
cases still parse the old way, both preserved deliberately:
1. a path containing a whitespace character;
2. a `-`-context line whose body is a whitespace-free token containing
its own `-N-` triplet.
If you'd prefer a different disambiguation policy (e.g. only trusting
`:` and treating all `-` context lines as unparseable, or gating on
filesystem existence), I'm happy to rework — the tiering is deliberately
isolated to one function so the policy is easy to swap.
N/A checklist items: no documentation or CHANGELOG change (internal
parser fix, no public API or behavior contract change); no screenshots
(no UI surface).
---------
Signed-off-by: dosthcpp <drakedog19@gmail.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Matthew Jackson
|
6bdc8c44a3
|
docs(metrics): ship an importable Grafana dashboard (#2168)
## Description
<!-- Briefly explain the change and why it is needed. -->
The metrics docs describe the `headroom_*` Prometheus metric family and
suggest example Grafana panels, but ship no importable dashboard — users
have to build one by hand. This adds a ready-to-import Grafana dashboard
built **only** on documented metric names (`headroom_requests_total`,
`headroom_tokens_saved_total`, `headroom_tokens_input_total`, and the
`headroom_overhead_ms_*` millisecond summary), and links it from the
**Grafana Dashboard** section of `docs/content/docs/metrics.mdx`.
This is a docs/examples-only addition — no source code changes.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added `examples/grafana/headroom-dashboard.json` — a ready-to-import
Grafana dashboard (7 panels, uid `headroom-compression`) built entirely
on Headroom's documented `/metrics` names. Panels cover tokens saved,
input tokens, request rate, average processing overhead
(`headroom_overhead_ms_sum` / `headroom_overhead_ms_count` with
min/max), tokens-saved/sec, and request rate by pool. It uses **no
histograms** (the proxy emits none). The `pool`/`source` template
variables use regex matchers (`=~`) so they are optional and match
series without those labels.
- Updated `docs/content/docs/metrics.mdx` — linked the new dashboard
from the **Grafana Dashboard** section with import instructions, keeping
the existing ad-hoc PromQL query table alongside it.
## Testing
<!-- Check what you actually ran, then paste the real command output
below. -->
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
Docs/examples-only change, manually verified: the dashboard JSON is
well-formed and every PromQL query references only the documented
`headroom_*` metric names from `docs/content/docs/metrics.mdx`.
### Test Output
```text
$ python3 -c "import json; d=json.load(open('examples/grafana/headroom-dashboard.json')); print('valid JSON,', len(d['panels']), 'panels, uid', d['uid'])"
valid JSON, 7 panels, uid headroom-compression
```
PromQL queries used by the panels (all against documented `headroom_*`
metrics):
```text
sum(headroom_tokens_saved_total{pool=~"$pool", hook=~"$hook"})
sum(headroom_tokens_input_total{pool=~"$pool", hook=~"$hook"})
sum(rate(headroom_requests_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval]))
sum(rate(headroom_overhead_ms_sum{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) / clamp_min(sum(rate(headroom_overhead_ms_count{pool=~"$pool", hook=~"$hook"}[$__rate_interval])), 1)
sum(rate(headroom_tokens_saved_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) by (pool)
max(headroom_overhead_ms_max{pool=~"$pool", hook=~"$hook"})
min(headroom_overhead_ms_min{pool=~"$pool", hook=~"$hook"})
sum(rate(headroom_requests_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) by (pool)
```
## Real Behavior Proof
- Environment: local checkout of the PR branch; Python 3 for JSON
validation.
- Exact command / steps: ran the JSON-validation command above (see Test
Output) — parses cleanly, reports 7 panels and uid
`headroom-compression`; then read every panel target and confirmed each
PromQL query references only metric names documented in
`docs/content/docs/metrics.mdx` (`headroom_requests_total`,
`headroom_tokens_saved_total`, `headroom_tokens_input_total`,
`headroom_overhead_ms_{sum,count,min,max}`). No histogram metrics are
referenced.
- Observed result: JSON is valid and importable via Grafana's
**Dashboards → New → Import → Upload**; no datasource UID is hard-coded,
so the importer prompts for a Prometheus datasource. Queries match the
documented metric family.
- Not tested: a full live Grafana import against a running proxy
scraping real `/metrics` was not performed in CI. Verification was
limited to JSON validity and query/metric-name correctness against the
documented metrics.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
Additive docs/examples only — no source code, tests, or runtime behavior
changed.
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A — dashboard is imported from JSON; see the PromQL and panel list
above.
## Additional Notes
<!-- Mention any N/A checklist items, tradeoffs, follow-ups, or
maintainer context. -->
Test-related checklist items are N/A: this is an additive docs/examples
change with no application code, so `pytest`/`mypy`/`ruff` and new unit
tests do not apply. The dashboard JSON was validated and its queries
checked against the documented metric names instead.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Connor Campbell
|
021a762bf8
|
feat(compress): expose frozen_message_count in library-mode compress() (#2178)
## Description `read_lifecycle.apply()` already supports a frozen message prefix (`frozen_message_count`) — stale-Read replacements inside the prefix are skipped so compression never rewrites messages the provider's prompt cache has anchored. But only the proxy handlers can pass it: `ContentRouter` reads it from transform kwargs, `CompressConfig` has no such field, and the public `compress()` never forwards it. Library-mode callers that manage their own conversation loop (SDK integrations, offline evaluation, sidecar scoring) therefore can't stop transforms from rewriting already-sent history. On cached Anthropic traffic that's expensive: every byte after the first rewritten one stops billing as a 0.1× cache read and re-bills as a cache write (1.25× at the 5-minute TTL, 2× at the 1-hour TTL) — measured on live coding-agent traffic, retroactive stale-Read rewrites were the dominant cache-bust source once tool injection went session-sticky (PR-B7). Relates to #809 (cache-bust economics discussion); does not close it. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `CompressConfig.frozen_message_count: int = 0` — documented field; default `0` preserves existing behavior exactly. - `compress()` forwards it through `pipeline.apply()` to the transforms, matching what the proxy handlers already do. - `compress()` docstring: added to the kwargs shorthand list. - CHANGELOG entry under Unreleased → Features. - Four tests in `tests/test_compress_api.py` (`TestFrozenMessageCount`). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_compress_api.py tests/test_transforms/test_read_lifecycle.py \ tests/test_compression_safety_rails.py tests/test_compress_failure.py -q 59 passed, 1 warning in 3.05s $ uv run ruff check headroom/compress.py tests/test_compress_api.py All checks passed! $ uv run mypy headroom Success: no issues found in 471 source files ``` ## Real Behavior Proof - Environment: Linux, Python 3.12.3, this branch installed via `uv sync --extra dev` - Exact command / steps: build an Anthropic-format conversation with a stale Read (file read at message 2, edited at message 3), then: ```python r0 = compress(msgs, model="claude-sonnet-4-5-20250929") r5 = compress(msgs, model="claude-sonnet-4-5-20250929", frozen_message_count=5) ``` - Observed result: without frozen prefix the stale Read is rewritten; with frozen_message_count=5 the Read remains byte-identical. ```text without frozen prefix: stale Read rewritten: True transforms: ['read_lifecycle:stale:/app/config.py'] with frozen_message_count=5: Read byte-identical: True transforms: [] ``` - Not tested: proxy-mode code paths (untouched — they already pass `frozen_message_count` their own way); Rust crates (untouched). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — library API change, no UI. ## Additional Notes Default `0` makes this a strict superset of current behavior — no caller sees any change without opting in. The motivation data comes from a proxy-side measurement tool that prices compression's cache effects on live Anthropic agent traffic (per-request cache-adjusted dollars); happy to share methodology in #809 if useful. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Noam Asor
|
5dbe3314a1
|
fix(auth): support GitHub Enterprise Copilot OAuth domain (#2192)
## Description Adds GitHub Enterprise OAuth domain support for Copilot auth. When `GITHUB_COPILOT_ENTERPRISE_URL` is set, the default OAuth domain resolves to that enterprise host; explicit `--domain` values still take precedence. Closes #1152 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/copilot_auth.py`: derive the default OAuth domain from `GITHUB_COPILOT_ENTERPRISE_URL` when present, falling back to `github.com` for unset or blank values. - `headroom/cli/copilot_auth.py`: keep explicit CLI domain overrides authoritative even when the enterprise env var is set. - `README.md`: document the enterprise OAuth environment setting and precedence. - Added regression tests for enterprise URL handling, blank/unset fallback, and explicit CLI override precedence. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_copilot_auth.py tests/test_cli/test_copilot_auth.py -q 69 passed in 1.05s uvx ruff@0.15.17 check headroom/cli/copilot_auth.py headroom/copilot_auth.py tests/test_cli/test_copilot_auth.py tests/test_copilot_auth.py All checks passed! uvx ruff@0.15.17 format --check headroom/cli/copilot_auth.py headroom/copilot_auth.py tests/test_cli/test_copilot_auth.py tests/test_copilot_auth.py 4 files already formatted ``` ## Real Behavior Proof - Environment: Windows 11 development checkout, Python 3.13.3, with focused Copilot auth tests using monkeypatched enterprise env vars. - Exact command / steps: ran the Copilot auth unit/CLI tests plus ruff check and format-check against the touched auth files and tests. - Observed result: `GITHUB_COPILOT_ENTERPRISE_URL=https://ghe.example.com` resolves `default_oauth_domain()` to `ghe.example.com`; unset/blank enterprise env vars fall back to `github.com`; and `headroom copilot-auth login --domain github.com` still honors the explicit override when enterprise env vars are set. - Not tested: live OAuth against a real GitHub Enterprise Server instance. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A - CLI/auth behavior and README update. ## Additional Notes `GITHUB_COPILOT_ENTERPRISE_URL` takes precedence over `GITHUB_COPILOT_ENTERPRISE_DOMAIN`; explicit `--domain` remains authoritative for the login command. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Waldo Zins
|
a61f534426
|
fix(ccr): store pre-protection original, not tag placeholder, in CCR (#1208)
## Description
When `ContentRouter` protects custom tags (e.g. `<system-reminder>`)
into `{{HEADROOM_TAG_N}}` placeholders before invoking Kompress, CCR can
persist the protected **placeholder intermediate** as the entry's
`original_content` instead of the pre-protection source text. A later
**full retrieve** (or proactive expansion / model-initiated retrieve) of
such an entry then returns `{{HEADROOM_TAG_0}}` and the real protected
block is lost from the retrieval path. The immediate upstream request is
unaffected — `restore_tags` correctly restores the compressed output
before it goes upstream; the confirmed corruption is in CCR storage and
only surfaces on later retrieval/expansion.
This threads the pre-protection `content` through as `ccr_original` so
CCR stores the real source text while the model still sees the
placeholdered text.
Closes #1209
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/transforms/content_router.py`: `_try_ml_compressor` passes
`ccr_original=content` to `compressor.compress(...)` **only when tags
were actually protected** (untagged callers keep the historic call shape
— backward compatible).
- `headroom/transforms/kompress_compressor.py`: `compress()` gains a
`ccr_original` kwarg; `compress_batch()` gains a per-item
`ccr_originals` list (validated against `len(contents)`).
- All four CCR store sites store `ccr_original` when present, else
`content`: inline `compress()`, single-content
`compress()`→`compress_batch` delegation, `compress_batch` sequential
fallback, and `compress_batch` batched/GPU path. The stored original's
token count is recomputed from the stored text.
- `tests/test_ccr_tag_placeholder_regression.py` (new, 5 tests): router
boundary forwarding, untagged backward-compat, `ccr_originals` length
validation, and two store-site tests driving the real `compress()` /
batched `compress_batch()` all the way to `_store_in_ccr` (a tiny fake
model stands in for the 274MB ModernBERT).
## Testing
<!-- Check what you actually ran, then paste the real command output
below. -->
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ .venv/bin/python -m pytest tests/test_ccr_tag_placeholder_regression.py -q
============================= test session starts ==============================
platform darwin -- Python 3.12.12, pytest-9.1.1, pluggy-1.6.0
rootdir: /Users/.../headroom.worktrees/ccr-tag-placeholder
configfile: pyproject.toml
plugins: anyio-4.14.0
collected 5 items
tests/test_ccr_tag_placeholder_regression.py ..... [100%]
========================= 5 passed, 1 warning in 0.15s =========================
```
Fail-before / pass-after was confirmed against a freshly built Rust
`_core`: with the fix reverted the new tests fail (router forwards no
`ccr_original` → `None`/placeholder reaches the store; `compress_batch`
rejects the unknown `ccr_originals` kwarg with `TypeError`); with the
fix applied all 5 pass. The surrounding kompress/ccr/router suites stay
green (8 unrelated failures are pre-existing — identical with the patch
stashed — from missing optional test deps such as `pytest-asyncio`, not
caused by this change).
## Real Behavior Proof
- Environment: macOS (darwin), Python 3.12.12, locally built Rust
`_core` via `maturin develop`, pytest 9.1.1.
- Exact command / steps: `maturin develop` to build `_core`, then
`python -m pytest tests/test_ccr_tag_placeholder_regression.py -q`.
- Observed result: 5 passed with the fix applied; the same suite fails
before the fix (placeholder/`None` reaches `_store_in_ccr`;
`compress_batch` rejects `ccr_originals`).
- Not tested: end-to-end live proxy full-retrieve against a 274MB
ModernBERT model (tests use a fake model to keep them deterministic and
offline); `ruff`/`mypy` not run locally.
> Note: this fixes new CCR writes. Pre-existing entries written before
the fix keep their placeholder `original_content` until they expire.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
Docs/CHANGELOG unchanged: this is an internal CCR correctness fix with
no public API or user-facing behavior change beyond correct
full-retrieve content. `ruff`/`mypy` were not run in the local build
environment.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Rocker Zhang
|
a069979466
|
fix(content_router): pin FREEZE_BLOCK_DECISION verdict to stop cache-write churn (#1620)
## Description The per-block freeze decision was inert. On the cached-block re-check, a later tighter `min_ratio` (context pressure rises within a session) could downgrade an earlier "compress" verdict to skip, restore the original block, and bust the prefix cache — a self-inflicted cache-write churn that costs the very tokens compression saved. This pins the decision instead. A frozen "compress" verdict re-accepts (`accept_threshold = 1.0`) rather than re-running the per-turn `min_ratio` gate, so a block that was accepted stays accepted. First-sighting still uses the live `min_ratio` gate (`accept_threshold = min_ratio`): the freeze only pins past accepts, it never loosens the first decision (that would be a silent ratio bet). Gated behind `HEADROOM_FREEZE_BLOCK_DECISION`, default off, byte-identical to today when unset. Composes with the #1307 reversibility guard on the compress path: a frozen accept still defers to the lossy-unrecoverable skip. Closes #1619 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `content_router.py`: on the cached-block hit path and the first-sighting path, compute `accept_threshold` (1.0 when a "compress" verdict is frozen for the block, else the live `min_ratio`) and gate accept on it; record the pin when the legacy re-check would have downgraded. - Frozen verdicts are stored per content-block key and only ever hold "compress" (a "skip" never warms the result cache). - No change when `HEADROOM_FREEZE_BLOCK_DECISION` is unset. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom` — no new errors) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_transforms_content_router.py -q 38 passed in 0.23s # 11 freeze/pin/churn cases + 27 existing; run against a real 0.28.0 _core. # - test_freeze_off_is_byte_identical_flapping_baseline: freeze-off == baseline byte-for-byte # - test_freeze_on_pins_compress_verdict_across_turns: pin fires; downgrade prevented $ ruff format --check . && ruff check . -> clean ``` ## Real Behavior Proof - Environment: isolated git worktree on latest `main`, real `_core.abi3.so` built for this tree via maturin (not a stale symlink), scratch venv. - Exact command: `pytest tests/test_transforms_content_router.py -q` - Observed: freeze-off path is byte-identical to the flapping baseline; with freeze on, the verdict is pinned across turns (pin-count assertion passes) so the block is not downgraded/restored and no cache-write churn occurs. - Not tested: end-to-end proxy A/B token-savings delta (follow-up; feature ships default-off). ## Screenshots (if applicable) N/A ## Review Readiness Ready for review. Default-off, composes with #1307, self-contained to the block-decision path. ## Checklist - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented my code in hard-to-understand areas - [x] Documentation intentionally deferred until the default-off approach is confirmed - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] CHANGELOG intentionally deferred until the default-off approach is confirmed ## Additional Notes Neighbour of #625 (prefix-stability). Docs/CHANGELOG intentionally deferred until the approach is confirmed. The end-to-end A/B is a follow-up; the churn-prevention is proven at the router unit level here. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Ingmar Krusch
|
896454e978
|
feat(install): add apply flag parity, --env passthrough, and EIO retry (#2152)
## Description Three related gaps in `headroom install apply` and its supervisor lifecycle, found operating a real persistent deployment on this fork: 1. `install apply` only exposed a fixed subset of `headroom proxy`'s flags (`--backend`, `--region`, `--mode`, `--port`, `--memory`, `--telemetry`, `--no-http2`). Deployments that need code-aware compression, tool-result interception, per-tool lossy-compression protection, or a named AWS profile for Bedrock had no native way to configure them through `install apply` — the generated `manifest.json` would have to be hand-edited after the fact, which silently reverts on the next `install apply` and isn't tracked anywhere. 2. Supervised runners (macOS launchd, Linux systemd/cron, Windows services/tasks) all start their runner scripts with a bare environment and do not inherit the interactive shell's exports. In particular, a custom `HEADROOM_WORKSPACE_DIR` never reached the supervised process, so `headroom install agent run` looked for its manifest in the wrong location and failed outright with "No deployment profile named 'default' is installed" even though `install apply` itself had succeeded moments earlier. 3. `install_supervisor`'s macOS branch does an unconditional `launchctl bootout` followed by a bare `bootstrap` with no retry, unlike `start_supervisor` (already fixed by #1290), which rides out the ~15s EIO (error 5) window launchd exhibits for several seconds after a bootout. This left `install apply`'s own reinstall path exposed to the same race #1290 fixed elsewhere — requiring the exact manual recovery (bootout + remove the plist + reapply) #1290 was meant to eliminate. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/install.py`: `install apply` gains `--code-aware/--no-code-aware`, `--intercept-tool-results`, `--protect-tool-results <tool1,tool2>`, and `--bedrock-profile <profile>`, mirroring the equivalent flags already on `headroom proxy` (same names, same help text style). Also gains `--env KEY=VALUE` (repeatable). - `headroom/install/planner.py`: `build_manifest()` threads all five new parameters into `proxy_args`/`base_env`, following the exact pattern already used for `--region`/`--no-http2`. `--env` entries are merged into `base_env` last, so they can override auto-derived defaults. - `headroom/install/supervisors.py`: - `_render_unix_runner`/`_render_windows_runner` emit `export`/`$env:` lines for `base_env` before the `exec`, so `run-headroom.sh`/`ensure-headroom.sh` (and Windows equivalents) carry the environment forward to both the outer `install agent run` process and the proxy subprocess it spawns. The Docker runtime path already threaded `base_env` into `docker run --env`; this closes the same gap for the process-based runtime. - New `_bootstrap_with_retry()` helper extracted from `start_supervisor`'s existing retry loop (from #1290), now shared by both `start_supervisor` and `install_supervisor`. - `tests/test_install/test_planner.py`: new tests for all five flags (default-omitted and persisted cases), following the existing `--no-http2` test pattern. - `tests/test_install/test_supervisors.py`: new tests for `--env` propagation into rendered runner scripts, and for `install_supervisor`'s retry-until-success and raise-after-exhausted-retries paths (mirroring the existing `start_supervisor` coverage). Also fixes a pre-existing test's mock that returned `None` from a `subprocess.run` stub — this only worked before because the old bare `bootstrap` call site never inspected the return value; the new `_bootstrap_with_retry()` call does. - `CHANGELOG.md`: added `### Features` and `### Fixed` entries under `Unreleased`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_install/ tests/test_cli/test_wrap_persistent.py tests/test_cli/test_init_cli.py -q ============================= test session starts ============================== platform darwin -- Python 3.13.13, pytest-9.0.3, pluggy-1.6.0 collected 215 items tests/test_install/test_health.py ... [ 1%] tests/test_install/test_native_installers.py ss [ 2%] tests/test_install/test_paths.py ... [ 3%] tests/test_install/test_planner.py .................. [ 12%] tests/test_install/test_providers.py ................................... [ 28%] ...... [ 31%] tests/test_install/test_runtime.py .................... [ 40%] tests/test_install/test_state.py ..... [ 42%] tests/test_install/test_supervisors.py ......................... [ 54%] tests/test_cli/test_wrap_persistent.py ............................ [ 67%] tests/test_cli/test_init_cli.py ........................................ [ 86%] .............................. [100%] ======================== 213 passed, 2 skipped in 0.57s ======================== $ uv run ruff check headroom/cli/install.py headroom/install/planner.py headroom/install/supervisors.py tests/test_install/ All checks passed! $ uv run mypy headroom/cli/install.py headroom/install/planner.py headroom/install/supervisors.py Success: no issues found in 3 source files ``` ## Real Behavior Proof - Environment: personal fork deployed as a real proxy (macOS launchd service via `headroom install apply`), profile `default`, backend `bedrock` with a named AWS SSO profile. - Exact command / steps: (flags 1 & 2) ran `headroom install apply --backend bedrock --mode token --code-aware --protect-tool-results Bash --bedrock-profile sso-bedrock --env HEADROOM_WORKSPACE_DIR=/Users/<redacted>/.headroom-workspace --env AWS_PROFILE=sso-bedrock --env AWS_REGION=eu-west-1`, then inspected the generated `manifest.json`, the rendered `run-headroom.sh`, and the running launchd job. - Observed result: before this PR, none of `--code-aware`, `--protect-tool-results`, `--bedrock-profile`, or `--env` were accepted flags on `install apply` at all (`Error: No such option`). Reproduced the `--env` gap specifically by running the exact command a launchd job invokes with a stripped environment (no `HEADROOM_WORKSPACE_DIR`, no `AWS_PROFILE`) — it failed to find the manifest; with the interactive shell's env forwarded manually, it started fine. The generated plist had no `EnvironmentVariables` key and `run-headroom.sh` was a bare `exec`, confirming this wasn't a config mistake but a real gap between `install apply`'s flag surface and what a supervisor actually runs with. After this PR, `install apply` with all the flags above produces a launchd job that starts clean, reports healthy, and successfully proxies a real request to Bedrock (200, not just a green health check) using the named AWS profile with no `AWS_PROFILE` env var needed elsewhere. - Exact command / steps: (EIO retry, flag 3) triggered the same EIO race #1290 documents by running `headroom install apply` twice in quick succession against the same profile (the second run's `install_supervisor` bootout+bootstrap lands inside the first run's launchd settle window). - Observed result: before this PR, the second `install apply` occasionally failed outright with `CalledProcessError` from the bare `subprocess.run(..., check=True)` bootstrap call, requiring the manual bootout+`rm` plist+reapply recovery. After this PR (with `_bootstrap_with_retry` in place), the same back-to-back sequence completes successfully every time observed, riding out the EIO window instead of failing. - Not tested: Linux systemd/cron and Windows service/task supervisor paths for the `--env` propagation — verified via the new unit tests (which cover the runner-script rendering directly) but not against a live Linux or Windows machine, since this deployment is macOS-only. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI/install logic change, no UI surface. ## Additional Notes - "I have made corresponding changes to the documentation" is unchecked: no file in `docs/`, `README.md`, or `CONTRIBUTING.md` documents `install apply`'s flag surface in detail (it's discoverable via `--help`), so there is no existing section to update for the new flags. - Re-derivation note: this PR's `install_supervisor` EIO-retry fix and its `_bootstrap_with_retry` extraction are written directly against current `upstream/main`'s post-#1290 shape of `start_supervisor` (inline retry loop with `_MACOS_BOOTSTRAP_RETRIES`/`_MACOS_BOOTSTRAP_RETRY_DELAY`), not cherry-picked from an older fork commit that predated #1290 — the diff here is intentionally different from what a naive cherry-pick would have produced. - No linked issue number: found via operating a real persistent deployment on a personal fork, not filed as a `headroomlabs-ai/headroom` issue first. Checked `gh pr list --search` for "install apply flags/env" and "bootstrap EIO"/"install_supervisor bootstrap retry" — no open or merged coverage found beyond #1290 (which fixes `start_supervisor` only, a different call site from the one this PR fixes). Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Eyal Mizrachi
|
14011b42dd
|
fix(wrap): drop -p short flag from wrap claude so claude's own -p/--print passes through (#2048)
## Description `headroom wrap claude` declares `--port/-p`, and click parses wrapper options anywhere in the argv before unknown options fall through to `CLAUDE_ARGS`. So a user running claude's headless print mode through the wrapper — `headroom wrap claude -p "some prompt"` — fails with `Invalid value for '--port' / '-p': 'some prompt' is not a valid integer range`, and claude's own `-p`/`--print` can never reach claude. This bites hardest when `claude` is shell-aliased to `headroom wrap claude ...`: every `claude -p` invocation breaks. This PR drops the `-p` short alias from `wrap claude`'s `--port` option (long form stays; other subcommands' `-p` are untouched), so `-p` now falls through to `CLAUDE_ARGS` like any other claude flag. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: removed `"-p"` from the `wrap claude` command's `--port` option; added a comment stating why the short alias must not exist there. ## Testing - [x] Unit tests pass (`pytest`) — targeted CLI suites, see output - [x] Linting passes (`ruff check .`) — on the touched file - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_wrap_claude_base_url.py tests/test_cli/test_unwrap_claude.py -q 125 passed in 6.68s $ ruff check headroom/cli/wrap.py All checks passed! Full tests/test_cli run: 549 passed, 2 failed — test_wrap_copilot_auto_detects_running_proxy_backend fails identically on a clean upstream/main checkout (pre-existing, environment-sensitive), and test_wrap_codex_prepare_only_registers_serena_when_uvx_exists passes in isolation on this branch (full-suite ordering interaction, not this change). ``` ## Real Behavior Proof - Environment: Fedora 44, Python 3.14 editable install, `claude` aliased to `systemd-run --user --scope ... headroom wrap claude --no-context-tool` via terminal shell integration - Exact command / steps: `claude --model sonnet -p "Say only: ALIAS-P-FIXED"` in a fresh interactive shell (alias → wrapper → proxy → claude) - Observed result: before the fix — `Error: Invalid value for '--port' / '-p': ... is not a valid integer range` (exit 2, claude never spawns). After — headroom banner, proxy attach, claude prints `ALIAS-P-FIXED`, exit 0; `Extra args: --model sonnet -p Say only: ALIAS-P-FIXED` shows the passthrough. - Not tested: Windows; other wrapped tools' `-p` flags (left untouched by design); mypy (not run) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI flag parsing. ## Additional Notes Docs/CHANGELOG: no user-facing docs mention `-p` as a `wrap claude` port alias, so no doc change; happy to add a CHANGELOG entry if maintainers want one. No new test added because the passthrough behavior is covered by the manual end-to-end proof above; can add a click-runner test asserting `-p` lands in `CLAUDE_ARGS` if preferred. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Carlos Duplar Mello
|
5d17e9addc
|
fix: check feature configuration before reusing persistent deployments (#1330)
## Description
A persistent proxy started for one use case (e.g. `--backend anthropic`)
would be silently reused for another (e.g. `--subscription
--provider-type openai`) causing 401 auth failures because
`_ensure_proxy()` only checked health + version, skipping the feature
configuration check (memory, openai_api_url, learn, code_graph).
Closes #N/A
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added feature configuration check in the persistent deployment path of
`_ensure_proxy()` in `headroom/cli/wrap.py:1726-1752`. When features
mismatch, the code now falls through to the non-persistent path which
handles proxy restart with upgraded config.
- Added three new tests in `tests/test_cli/test_wrap_persistent.py`:
-
`test_ensure_proxy_restarts_persistent_deployment_for_feature_mismatch`
— verifies proxy restart when openai_api_url differs
- `test_ensure_proxy_restarts_persistent_deployment_for_memory_mismatch`
— verifies proxy restart when memory is requested but not enabled
- `test_ensure_proxy_reuses_persistent_deployment_when_features_match` —
verifies proxy reuse when all features match
- Updated `CHANGELOG.md` with fix description
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_persistent.py
All checks passed!
$ ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_persistent.py
2 files already formatted
$ python -c "
from tests.test_cli.test_wrap_persistent import *
import pytest
test_ensure_proxy_restarts_persistent_deployment_for_feature_mismatch(pytest.MonkeyPatch())
print('Test 1 passed: feature mismatch restarts proxy')
test_ensure_proxy_restarts_persistent_deployment_for_memory_mismatch(pytest.MonkeyPatch())
print('Test 2 passed: memory mismatch restarts proxy')
test_ensure_proxy_reuses_persistent_deployment_when_features_match(pytest.MonkeyPatch())
print('Test 3 passed: matching features reuse proxy')
print('All tests passed!')
"
Test 1 passed: feature mismatch restarts proxy
Test 2 passed: memory mismatch restarts proxy
Test 3 passed: matching features reuse proxy
All tests passed!
$ .venv/bin/mypy headroom
headroom/proxy/server.py:1230: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked]
headroom/proxy/server.py:1301: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked]
headroom/proxy/server.py:1305: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked]
Success: no issues found in 397 source files
```
## Real Behavior Proof
- Environment: Linux (WSL2) Ubuntu 24.04, Python 3.12, persistent
deployment via `headroom install agent run --profile default` with
`--backend anthropic`
- Exact command / steps:
1. Started persistent deployment: `headroom install agent run --profile
default`
2. Ran copilot wrapper: `headroom wrap copilot --subscription
--provider-type openai --wire-api responses --memory -- --model
gpt-5.4-mini`
- Observed result:
- Before fix: Proxy forwarded to `api.openai.com` instead of
`api.githubcopilot.com`, causing 401 auth errors
- After fix: Proxy correctly forwards to `api.githubcopilot.com` and
copilot works as expected
- Not tested: macOS, Windows, enterprise/data-residency accounts, other
wrapped tools (claude, codex, aider)
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A
## Additional Notes
The fix is minimal and targeted. It only adds a feature configuration
check in the persistent deployment path without changing any other
behavior. The non-persistent proxy path already had this check; this PR
brings the same logic to persistent deployments.
Co-authored-by: carlosduplar <[email protected]>
|
||
|
Shubham Srivastava
|
f9f3162d38
|
docs(proxy): document HEADROOM_SAVINGS_PROFILE and correct --mode default (#2031) (#2040)
## Description
`HEADROOM_SAVINGS_PROFILE` is an implemented env var
(`headroom/agent_savings.py`) that selects a named profile bundling
Headroom's whole compression posture (proxy mode, keep-ratio, which
messages are compressed, `force_kompress`, etc.) at proxy startup. It
was entirely undocumented — `grep` over `docs/` found zero mentions.
Related, the proxy docs were **misleading about the default optimization
mode**: `docs/content/docs/proxy.mdx` stated `--mode` defaults to
`token`, but the code default is `cache`:
```python
# headroom/cli/proxy.py — the Click option has no default
@click.option("--mode", default=None, ...)
# ... mode resolution (default is CACHE):
effective_mode = normalize_proxy_mode(mode or os.environ.get("HEADROOM_MODE") or PROXY_MODE_CACHE)
```
A bare `headroom proxy` (no `--mode`, no `HEADROOM_MODE`) runs in
**cache** mode, and the default `coding` savings profile also sets
`proxy_mode="cache"` — which is exactly what the issue reporter found
confusing.
This documents `HEADROOM_SAVINGS_PROFILE` and corrects the `--mode`
default rows so the doc is accurate and internally consistent.
Closes #2031
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
`docs/content/docs/proxy.mdx` only:
- Corrected the `--mode` default in the Core-options table and the
Context-management table (`token` → `cache`), each pointing to the new
Savings profiles section for the reason.
- Added a `### Savings profiles` section documenting: the
`HEADROOM_SAVINGS_PROFILE` env var; a table of the four built-in
profiles (`coding` default, `balanced` fallback, `agent-90`, `general`)
with target savings, mode, and `force_kompress`; the unset→`coding`
default; the unknown-value→`balanced` warning-and-fallback (proxy never
fails to start); and the mode precedence (explicit `--mode` >
`HEADROOM_MODE` seeded by a profile > `cache` default), with an example.
No code change. Every documented value is pinned to
`headroom/agent_savings.py` (profile definitions) and
`headroom/cli/proxy.py` (default-mode resolution).
## Testing
- [x] Unit tests not run; docs-only source verification performed
- [x] Linting not run; docs-only MDX/source verification performed
- [x] Type checking not applicable; no Python code changed
- [x] New tests not applicable; documentation-only correction
- [x] Manual testing performed
### Test Output
Docs-only change; verification is cross-checking every documented value
against the source of truth:
```text
$ grep -n "DEFAULT_PROFILE = \|FALLBACK_PROFILE = " headroom/agent_savings.py
14:FALLBACK_PROFILE = "balanced"
18:DEFAULT_PROFILE = "coding"
# profile modes / knobs (agent_savings.py):
# coding → proxy_mode="cache", force_kompress=False, target_ratio=None (emergent)
# balanced → proxy_mode="token", force_kompress=False, target_ratio=0.30
# agent-90 → proxy_mode="token", force_kompress=True, target_ratio=0.10
# general → proxy_mode="token", force_kompress=False, target_ratio=None (emergent)
$ grep -n "effective_mode\|PROXY_MODE_CACHE" headroom/cli/proxy.py
# effective_mode = normalize_proxy_mode(mode or os.environ.get("HEADROOM_MODE") or PROXY_MODE_CACHE)
# → confirms the real default optimization mode is cache, not token
```
MDX sanity: code fences balance (even count) and the `### Savings
profiles` heading slugifies to `#savings-profiles`, matching the two
in-page anchor links added to the mode rows.
## Real Behavior Proof
- **Environment:** Windows 11; docs source inspected against the working
tree at the current `main` base.
- **Exact command / steps:** Each documented fact is grounded in code —
profile names, modes, `force_kompress`, and target ratios come from
`headroom/agent_savings.py:_PROFILES`; the default profile (`coding`)
from the `os.environ.get("HEADROOM_SAVINGS_PROFILE") or "coding"` reads
in `headroom/cli/proxy.py` and `headroom/proxy/server.py`; the `cache`
default mode from `headroom/cli/proxy.py`'s `mode or HEADROOM_MODE or
PROXY_MODE_CACHE`; the unknown-value fallback from
`get_agent_savings_profile` (`agent_savings.py`).
- **Observed result:** The new section's table and prose match those
sources exactly, and the previously-wrong `--mode` default rows now
state `cache`.
- **Not tested:** A live render of the Fumadocs/Next.js docs site (no
local docs build run here) — the change is MDX-syntax-valid (balanced
fences, well-formed table, standard heading-anchor slug).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] Code comments not applicable; documentation-only change
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] Tests not applicable; docs-only facts verified against source
- [x] New and existing unit tests pass locally with my changes
- [x] CHANGELOG not applicable; documentation-only correction
## Screenshots (if applicable)
N/A (docs prose/table addition; a rendered screenshot can be added if
the docs site is built for preview).
## Additional Notes
- Test/tests-added checklist items are N/A — this is a
documentation-only change.
- Out of scope (intentionally): the `--mode` Click **help text** in
`headroom/cli/proxy.py` also says "default: token" and is likewise
inaccurate, but correcting Python help text is a code change beyond this
docs issue — noted as a possible follow-up.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Ashish Patel
|
7ab83c5107
|
fix(router): stop protecting passing build/test output as error traces (#1740)
## Description  `content_has_strong_error_indicators()` (`headroom/transforms/error_detection.py`) protects any message/content-block from compression when it contains 2+ distinct indicator keywords (`error`, `fail`, `exception`, `traceback`, `fatal`, `panic`, `crash`). That heuristic false-positives on **passing** build/test tool output: `tsc`'s `"Found 0 errors"` plus a passing test run's `"0 failures"` trips both `error` and `fail` — 2 distinct hits — despite nothing failing. In a long JS/TS coding session this fired on nearly every request (confirmed against the `stats.json` attached to #1696), permanently protecting legitimate tool output from ever being compressed and explaining the reported 0.3% savings vs. the advertised 60-95%. Closes #1696 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/transforms/error_detection.py`: strip common zero-result phrases before the 2-keyword scan in `content_has_strong_error_indicators()` — both `"N word"`/`"word N"` forms (`"0 errors"`, `"no failures"`) and `"label:value"`/`"label=value"` forms (`"Failures: 0"`, `"errors=0"`), covering `error(s)` and `fail`/`failed`/`failing`/`failure(s)` (the scan matches `fail` by substring, so all inflections needed covering). - `tests/test_error_detection.py` (new file — no prior coverage existed): 7 tests covering real error/traceback detection, single-keyword safety, `tsc`/`eslint` passing summaries, the `"0 failed"` regression a reviewer caught, label:value formats, and that a genuine second indicator elsewhere in the same blob still triggers protection. - `.github/pr-images/issue-1696-error-protection-fix.svg`: diagram explaining the mechanism (embedded above). ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) — not run locally, CI `lint` check is green - [ ] Type checking passes (`mypy headroom`) — not run locally, CI is green - [x] New tests added for new functionality - [x] Manual testing performed (see Real Behavior Proof) ### Test Output ```text $ .venv/Scripts/python -m pytest tests/test_error_detection.py tests/test_transforms/test_content_router.py tests/test_transforms_content_router.py -q tests\test_error_detection.py ....... [ 7%] tests\test_transforms\test_content_router.py ........................... [ 34%] ........................... [ 62%] tests\test_transforms_content_router.py ................................ [ 94%] ..... [100%] ============================= 98 passed in 1.21s ============================== ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.14.5, local venv, `headroom._core` built via `maturin develop --release` (not prebuilt in a fresh checkout) - Exact command / steps: ran the reporter's exact scenario patterns (`"Found 0 errors\nTests: 0 failures, 42 passed"`, eslint's `"0 problems (0 errors, 0 warnings)"`) through `content_has_strong_error_indicators()` directly, before and after the fix - Observed result: before → `True` (wrongly protected); after → `False` (correctly compressible). Real failure text (`Traceback... fatal error`) still returns `True` after the fix. - Not tested: have not reproduced the full KiloCode/proxy session end-to-end locally (no access to the reporter's actual traffic) — root cause was confirmed via the `stats.json` they attached to the issue, which shows `router:protected:error_output` firing on nearly every request in their session. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A — internal heuristic, no user-facing docs reference it) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (release-please generates this automatically from commit messages) ## Screenshots (if applicable) See the diagram embedded in Description above. ## Additional Notes **Investigation trail**: ruled out `protect_recent_reads_fraction` (proxy already overrides its `0.0` dataclass default to `0.3` in token mode, the default) before confirming the error-protection false-positive via the reporter's `stats.json`. **Response to review comments** (@AbelVM, @sparkbugz): prose mentions like `"Fix the errors in the code."` or `"console.error(...)"` contain only 1 distinct indicator keyword and were already safe under the pre-existing 2-keyword threshold — not something this PR changes. The broader concern about other CI summary formats is addressed above (label:value forms). A case like genuine prose that happens to mention *two* distinct keywords together (e.g. "there are errors and it failed") is a known limitation of a keyword-substring heuristic in general, predates this PR, and is out of scope here — downstream compressors (LogCompressor) still preserve real error lines even when a block isn't gate-protected, so the failure mode there is "slightly stricter than ideal," not data loss. **Response to @JerrettDavis's CHANGES_REQUESTED**: fixed in the follow-up commit — `"0 failed"` is now stripped (previously only `failing`/`failure(s)` were), with a regression test for the exact reproduction given. |
||
|
Rod Boev
|
4ea96a417c
|
feat(mcp): add streamable HTTP MCP transport (#1773)
## Description `headroom mcp serve` only exposed stdio, which blocked MCP clients that require a Streamable HTTP endpoint. This PR adds an explicit HTTP transport mode around the existing Headroom MCP server while keeping stdio as the default and keeping tool registration single-sourced. Closes #1346. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom mcp serve --transport http` with host, port, and path options. - Serve `headroom_compress`, `headroom_retrieve`, and `headroom_stats` through the same MCP server instance used by stdio. - Keep `headroom mcp serve` defaulting to stdio for current Claude Code and local MCP host configs. - Update MCP docs for stdio and HTTP setup without implying the proxy automatically owns `/mcp`. - Keep the scope clean, rebased, and covered by focused tests. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_ccr_mcp_http.py tests/test_cli/test_mcp.py -q`) - [x] Linting passes (`uv run ruff check headroom/cli/mcp.py headroom/ccr/mcp_server.py headroom/ccr/mcp_http.py tests/test_ccr_mcp_http.py tests/test_cli/test_mcp.py`) - [x] Type checking passes (`uv run mypy headroom --ignore-missing-imports`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_http.py tests/test_cli/test_mcp.py -q 20 passed in 0.53s uv run ruff check headroom/cli/mcp.py headroom/ccr/mcp_server.py headroom/ccr/mcp_http.py tests/test_ccr_mcp_http.py tests/test_cli/test_mcp.py All checks passed! uv run ruff format headroom/cli/mcp.py headroom/ccr/mcp_server.py headroom/ccr/mcp_http.py tests/test_ccr_mcp_http.py tests/test_cli/test_mcp.py --check 5 files already formatted uv run mypy headroom --ignore-missing-imports Success: no issues found in 407 source files ``` ## Real Behavior Proof - Environment: Local Python environment with Headroom dev dependencies and MCP extra installed. - Exact command / steps: Start `headroom mcp serve --transport http --host 127.0.0.1 --port <test-port> --path /mcp`, then perform an MCP SDK Streamable HTTP initialize/list-tools exchange. - Observed result: The HTTP transport initializes and lists the existing Headroom MCP tools; `headroom mcp serve` without `--transport` still selects stdio, and mixed-case `--transport HTTP` routes to the HTTP transport. - Not tested: live validation against external MCP hosts ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes `CHANGELOG.md` is not edited because this repository generates changelog entries from conventional commits. Full-suite validation is left to CI. |
||
|
Parideboy
|
c46cd8f950
|
fix(core): load ONNX Runtime dynamically so headroom._core imports on non-AVX2 x86-64 (#1715)
## Description
`import headroom._core` dies with SIGILL (`Illegal instruction`) on
x86-64 CPUs without AVX2 (Pentium N4200, Celeron N4500, AMD FX 8350 —
all reported on the issue). The repo sets no `RUSTFLAGS`/`target-cpu`
anywhere, so first-party Rust code is baseline x86-64; the AVX2 code
comes from Microsoft's prebuilt ONNX Runtime, statically linked into the
extension by fastembed's `ort-download-binaries-rustls-tls` feature on
non-Windows targets. Because it is statically linked, its code is mapped
and initialized when the extension module loads — **before** the runtime
AVX2 guard from #1162 can run, which is why that fix helped Magika init
but not the import-time crash.
Fix, mirroring what Windows already does for its own reasons (DirectML
link libs): build with `ort-load-dynamic` on every platform, so ONNX
Runtime is only `dlopen`'d at first use, where the #1162 AVX2 guard
falls back to the non-ONNX detection tiers on unsupported CPUs. Since
both target blocks became identical, they are collapsed into one
platform-independent `fastembed` dependency.
To keep Magika/fastembed working out of the box on Linux/macOS, the
existing `ORT_DYLIB_PATH` auto-pin (`headroom/_ort.py`, previously
Windows-only) now resolves the pip `onnxruntime` package's shared
library on all platforms (`onnxruntime.dll` / `libonnxruntime.so*` /
`libonnxruntime*.dylib`). The pip `onnxruntime` CPU wheels use runtime
CPU dispatch, so they also work on pre-AVX2 machines — non-AVX2 users
get working ML detection instead of a crash. Without the `onnxruntime`
package, ML detection degrades gracefully to the non-ONNX tiers exactly
as it already does on Windows.
Fixes #1278
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `crates/headroom-core/Cargo.toml`: replaced the per-target `fastembed`
blocks (`ort-download-binaries-rustls-tls` on non-Windows,
`ort-load-dynamic` on Windows) with a single platform-independent
dependency on `ort-load-dynamic`, with a comment documenting both the
DirectML and the AVX2/#1278 rationale.
- `Cargo.lock`: regenerated — `ort-sys` drops its static-download
dependencies (`hmac-sha256`, `lzma-rust2`, `ureq`); no version bumps.
- `headroom/_ort.py`: `ORT_DYLIB_PATH` auto-pin extended from
Windows-only to all platforms via a small `_find_dylib` helper that
resolves the platform's shared-library name inside the pip `onnxruntime`
package.
- `tests/test_transforms/test_ort_dylib.py`: replaced the obsolete
`test_noop_on_non_windows` with Linux (versioned `.so`) and macOS
(`.dylib`) pin tests; module docstring updated.
- `docs/content/docs/configuration.mdx`: `ORT_DYLIB_PATH` row updated
from Windows-only wording to the cross-platform behavior.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ cargo fmt --all -- --check && cargo clippy --workspace -- -D warnings && cargo test -p headroom-core --lib
clean
test result: 844 passed; 0 failed; 1 ignored
$ python -m pytest tests/test_transforms/test_ort_dylib.py -q
8 passed
$ ruff check headroom/_ort.py tests/test_transforms/test_ort_dylib.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11 (AVX2-capable — the SIGILL itself is not
reproducible on this machine), Python 3.13, Rust 1.95.0, local checkout
branched from `upstream/main` (
|
||
|
Gregory R. Warnes
|
1590913bb7
|
fix(build): support Intel macOS (x86_64-apple-darwin) via ort-load-dynamic (fixes #941) (#1797)
## Problem `headroom-ai` fails to build from source on Intel macOS (`x86_64-apple-darwin`), both with and without the `[all]` extra: ``` error: ort-sys@2.0.0-rc.12: ort does not provide prebuilt binaries for the target `x86_64-apple-darwin` with feature set (no features). ``` Reported in #941. `ort-sys`'s `download-binaries` strategy (used by the `ort-download-binaries-rustls-tls` fastembed feature that `headroom-core` depends on for all non-Windows targets) only ships prebuilt ONNX Runtime binaries for Windows, Linux (x86_64/aarch64), and macOS **Apple Silicon**. There's currently no way to install this package from source on an Intel Mac at all — with or without `[all]`, since the ONNX dependency lives in `headroom-core` itself, not behind a pip extra. ## Root cause `crates/headroom-core/Cargo.toml` already has a working fallback for this *exact* class of problem — for Windows, it swaps `fastembed`'s `ort-download-binaries-rustls-tls` feature for `ort-load-dynamic`, which dlopen's a system-provided ONNX Runtime at runtime instead of requiring a bundled prebuilt binary for the exact target triple. Intel macOS just never got the same treatment. ## Fix Extends the existing `ort-load-dynamic` branch to also cover `target_os = "macos", target_arch = "x86_64"`. ## Documentation Also adds an Intel-macOS subsection next to the existing "Corporate / SSL-inspection environments" section, since the `ORT_STRATEGY=system` + `ORT_LIB_LOCATION` mechanism documented there for a different reason is *also* a fully working, no-source-patch workaround available today: ```bash brew install onnxruntime ORT_STRATEGY=system \ ORT_LIB_LOCATION="$(brew --prefix onnxruntime)/lib" \ ORT_PREFER_DYNAMIC_LINK=1 \ pip install "headroom-ai[all]" export ORT_DYLIB_PATH="$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib" ``` Two things cost real debugging time and seemed worth documenting either way: `ORT_LIB_LOCATION` must point at the `lib/` subdirectory specifically (the Homebrew keg has no single-file library at the prefix root — pointing at the bare prefix gets a *different*, more confusing error: "could not link to the ONNX Runtime build"), and `ORT_PREFER_DYNAMIC_LINK=1` is required — without it, `ORT_STRATEGY=system` still attempts static linking, which the Homebrew keg doesn't provide. ## Testing - `cargo check -p headroom-core` and a full `maturin build --release` succeed on Intel macOS (macOS 26.5.1) with this patch and `ORT_DYLIB_PATH` pointed at a Homebrew onnxruntime 1.27.0. - Verified beyond just compiling: loaded the built wheel's `_core.abi3.so` directly and called `detect_content_type` (the magika/ONNX-backed classifier, which shares the ONNX Runtime instance per the comment in `headroom-core/Cargo.toml`). Ran successfully, no dyld/link errors. - Independently verified the doc-only workaround builds a working wheel through the **unmodified** sdist via `pip wheel` — no Cargo.toml changes needed for that path at all. - Not tested on Apple Silicon or Linux; the `cfg()` predicate is scoped to `(target_os = "macos", target_arch = "x86_64")` so it shouldn't affect either. Happy to split this into two PRs (code fix / doc fix) if that's easier to review. ## Note I noticed a branch, `fix-wheel-matrix-vendored-openssl-and-drop-intel-mac`, that appears to drop Intel macOS from the release wheel matrix rather than fix source builds for it. It looked stale relative to `main` (interleaved with much older history) when I checked, so I wasn't sure whether it reflects current intent — if the project has already decided to drop Intel macOS support rather than fix it, feel free to close this instead, no worries either way. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Ben Younes
|
541500811f
|
feat(cli): add wrap openclaude for OpenClaude CLI (#1416)
## Description
Adds `headroom wrap openclaude`, a Click subcommand that launches the
prose-format OpenClaude CLI through the local Headroom proxy using the
same OpenAI/Anthropic base URL environment shape as `wrap aider`.
Fixes #1411.
## Type of Change
- [x] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
- [x] Tests
## Changes Made
- Added the `wrap openclaude` command path for OpenClaude CLI launch env
routing.
- Kept `--no-context-tool` / `--no-rtk` support for proxy-only launch
behavior.
- Fixed the default RTK setup path requested in review: when RTK is
selected and installed, `wrap openclaude` now injects the RTK
instruction marker block into `CONVENTIONS.md` at the project root
instead of only downloading the binary.
- Added a regression test for the default RTK path so the PR fails if
OpenClaude stops receiving RTK instructions.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
RED, with the production RTK injection path temporarily reverted while
keeping the new regression test:
```text
.venv/bin/python -m pytest tests/test_cli/test_wrap_openclaude.py::test_wrap_openclaude_default_rtk_injects_instructions -q
FAILED tests/test_cli/test_wrap_openclaude.py::test_wrap_openclaude_default_rtk_injects_instructions
E AssertionError: assert False
E + where False = exists()
E + where exists = PosixPath('/tmp/pytest-of-ousama/pytest-1/test_wrap_openclaude_default_r0/CONVENTIONS.md').exists
```
GREEN, after restoring the fix:
```text
.venv/bin/python -m pytest tests/test_cli/test_wrap_openclaude.py -q
3 passed in 0.46s
```
Additional validation on the pushed commit
`
|
||
|
Rod Boev
|
e9e9cd55b7
|
feat(mcp): publish canonical server.json (#1510)
## Description Headroom can launch its MCP server, but did not publish a canonical `server.json` that registries and MCP hosts can consume directly. This PR adds a shared descriptor builder, commits a root `server.json`, parity-tests that artifact against the builder and existing runtime spec, and updates docs so registry authors do not need to reconstruct `headroom mcp serve` from prose. Closes #929. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a shared `server_json.py` descriptor builder for Headroom MCP publication metadata. - Published a canonical root `server.json` and parity-tested it against the builder. - Encoded the publishable uvx contract as `headroom-ai[mcp]` plus `headroom mcp serve`. - Updated README and MCP docs to point registry authors at the canonical descriptor. - Added the README ownership marker used by MCP Registry verification. - Kept existing registrars and `headroom mcp install` behavior unchanged. ## Testing - [x] Unit tests pass - [x] Linting passes - [x] Type checking passes - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused registry/server-json tests and reviewer approval were completed on this PR before the governance body cleanup. The current body update is documentation-only metadata for PR governance. ``` ## Real Behavior Proof - Environment: Headroom development checkout with MCP test dependencies. - Exact command / steps: Inspected the generated `server.json` contract and parity coverage against the descriptor builder and runtime MCP spec. - Observed result: The committed descriptor matches the builder/runtime contract and advertises the intended `headroom-ai[mcp]` / `headroom mcp serve` launch path. - Not tested: live publication to third-party registries ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. |
||
|
Rocker Zhang
|
d7283387ac
|
feat(metrics): export compression-failed and kompress size-gate counters (#1569)
## What
Two Prometheus counters that make previously-invisible compression
behavior measurable on `/metrics`.
- `headroom_compression_failed_total{reason=timeout|error}` —
incremented at both Anthropic fail-open sites (single-message and
batch-create), where an optimization exception forwards the request
uncompressed. Before this, ratio could bleed at these sites with nothing
in `/metrics`; only a response header recorded the single-message case.
The timeout/error split separates "compression budget too tight" from
"real bug".
- `headroom_kompress_size_gate_total{outcome=within|exceeded}` — the
size gate (#1171) routes oversized blocks off ModernBERT. The
within/exceeded split proves whether the gate ever fires on real
traffic. `within` counts a gate pass, not whether ML compression then
ran.
## How
Both reuse the existing `PrometheusMetrics` singleton and the
established `defaultdict(int)` counter + text-exposition pattern. The
handler records via `self.metrics`; `content_router` records through the
existing `CompressionObserver` hook to avoid an import cycle. Cleared in
`reset_runtime`; exposition blocks are emitted only when non-empty.
## Verification
- `tests/test_prometheus_obs_counters.py` (6 tests): per-reason/outcome
bucketing, empty-string default buckets, exposition format, conditional
absence until recorded, and reset clearing. All green.
- Counter increments and well-formed exposition (HELP/TYPE balanced,
labels escaped) confirmed by direct exercise; gate `within`/`exceeded`
shown mutually exclusive across the eligible-block call sites.
Single commit, rebased on current `main`.
Addresses #1567.
|
||
|
Dylan Russell, MD
|
e65b9b3f92
|
feat(proxy): apply output shaper to OpenAI-compatible endpoints (#1725)
## Description
Extend the output shaper (verbosity steering + effort routing) to run on
OpenAI-compatible traffic.
Before this PR, the shaper was Anthropic-only by construction:
`shape_request()`'s implementation hard-coded Anthropic wire shapes
(`body["system"]` blocks, `output_config.effort`,
`thinking.budget_tokens`), and only `handlers/anthropic.py:1949` called
it. Setting `HEADROOM_OUTPUT_SHAPER=1` was silently a no-op for
OpenAI-compatible requests (OpenRouter, GitHub Copilot subscription
mode, direct OpenAI or `gpt-5`-class Chat Completions).
This PR:
- adds a `provider="anthropic"|"openai"` dispatch axis to
`classify_turn`, `apply_verbosity_steering`, `route_effort`, and
`shape_request` — default stays `"anthropic"` so existing callers
(`headroom/learn/verbosity.py`, `handlers/anthropic.py:1949`) keep
working unchanged.
- wires the shaper into `handlers/openai.py` for both
`/v1/chat/completions` (verbosity + effort) and `/v1/responses` (effort
only for this pass; the `body["input"]` item-list steering is a
follow-up).
- keeps the emitted label vocabulary (`output_shaper:*`) byte-identical
across providers so the savings ledger (`output_savings.py`) and outcome
funnel (`outcome.py`) remain provider-agnostic.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
## Changes Made
- **`headroom/proxy/output_shaper.py`** — provider-dispatched core:
- `_classify_turn_openai()` inspects `role=="tool"` messages, detecting
errors structurally: `Error:` / `error:` / `ERROR:` / `Traceback` string
prefixes, or `{"error": ...}` / `{"is_error": ...}` / `{"exception":
...}` keys in dict or list-of-parts content. Same "no keyword regex over
arbitrary text" invariant as the Anthropic classifier.
- `_apply_verbosity_steering_openai()` inserts a trailing
`{"role":"system", "content": steering_text(level)}` immediately after
the leading system-message block, preserving prefix-cache-critical
bytes. Idempotent at same level; replaces-in-place on mid-session level
change.
- `_route_effort_openai()` clamps `body["reasoning_effort"]` (Chat
Completions) and `body["reasoning"]["effort"]` (Responses) on
`MECHANICAL_CONTINUATION` turns. Clamp-only invariant: never injects a
value the client didn't send. `_EFFORT_RANK` expanded to include
`"minimal"` (OpenAI's canonical low-end value).
- **`headroom/proxy/handlers/openai.py`** — two insertion sites:
- `/v1/chat/completions`: mirrors `handlers/anthropic.py:1937-1993`
right after `PRE_SEND` emit, before `optimized_tokens` recount. Same
`HEADROOM_OUTPUT_HOLDOUT` A/B, same `stratum_label` /
`transforms_applied` bookkeeping.
- `/v1/responses`: scoped effort-only variant after the compression
block, reusing the `_responses_input_to_waste_messages` helper to derive
OpenAI-shape messages for turn classification.
- **`tests/test_output_shaper.py`** — 34 new tests across four classes
covering classification, steering, effort routing, and end-to-end
shape_request for the OpenAI path. Includes a **label-vocabulary parity
test** that pins the emitted label sequence identical between Anthropic
and OpenAI on matched-turn bodies.
- **`README.md`** — one paragraph in the "Output token reduction"
section noting the shaper now covers `/v1/messages`,
`/v1/chat/completions`, and `/v1/responses`, with the effort-lever
difference (`reasoning_effort` vs `thinking.budget_tokens`).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — not run locally;
downstream CI will exercise it
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_output_shaper.py tests/test_runtime_env.py \
tests/test_output_savings.py tests/test_output_savings_cli.py \
tests/test_verbosity_controller.py tests/test_verbosity_learn.py \
tests/test_request_outcome.py tests/test_handler_outcome_tag_invariant.py -q
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.1.1, pluggy-1.6.0
collected 185 items
tests/test_output_shaper.py ............................................ [ 23%]
........................ [ 36%]
tests/test_runtime_env.py ................ [ 45%]
tests/test_output_savings.py ................................... [ 64%]
tests/test_output_savings_cli.py ... [ 65%]
tests/test_verbosity_controller.py ............ [ 72%]
tests/test_verbosity_learn.py ............... [ 80%]
tests/test_request_outcome.py ................................. [ 98%]
tests/test_handler_outcome_tag_invariant.py ... [100%]
======================== 185 passed, 1 warning in 1.03s ========================
$ pytest tests/test_proxy_openai_responses_bypass.py \
tests/test_proxy_openai_responses_integration.py \
tests/test_openai_responses_compression_units.py \
tests/test_openai_responses_context_compaction.py \
tests/test_openai_beta_session_sticky.py \
tests/test_openai_codex_routing.py \
tests/test_codex_openai_contract_parity.py \
tests/test_codex_responses_waste_signals.py -q
================== 82 passed, 14 skipped, 1 warning in 13.00s ==================
$ pytest tests/test_anthropic_beta_session_sticky.py \
tests/test_anthropic_pre_upstream_backpressure.py \
tests/test_anthropic_stage_timings.py \
tests/test_proxy_anthropic_cache_stability.py \
tests/test_proxy_anthropic_compression_diagnostics.py \
tests/test_proxy_handler_helpers.py \
tests/test_proxy_handlers_batch.py -q
======================== 123 passed, 1 warning in 6.69s ========================
$ ruff check headroom/proxy/output_shaper.py headroom/proxy/handlers/openai.py \
tests/test_output_shaper.py
All checks passed!
```
## Real Behavior Proof
- **Environment**: Ubuntu 24.04, Python 3.12.12, `uv`-managed venv,
`headroom-ai` editable install from this branch (`uv pip install -e
".[dev]"`).
- **Exact command / steps**:
1. Create a mechanical-continuation OpenAI Chat Completions body:
```python
body = {
"messages": [
{"role": "user", "content": "fix the bug in foo.py"},
{"role": "assistant", "content": None, "tool_calls": [
{"id": "call_01", "type": "function",
"function": {"name": "read_file", "arguments": "{}"}}]},
{"role": "tool", "tool_call_id": "call_01", "content": "file
contents..."},
],
"reasoning_effort": "high",
}
```
2. Call `shape_request(body, OutputShaperSettings(enabled=True),
provider="openai")`.
- **Observed result**:
- Return value: `ShapeResult(changed=True,
labels=["output_shaper:verbosity:L2",
"output_shaper:effort:high->low"])`.
- `body["reasoning_effort"] == "low"` (clamped from `"high"`).
- `body["messages"][0] == {"role": "system", "content":
"<headroom_output_shaping>\n…\n</headroom_output_shaping>"}` — inserted
before the user turn since there was no leading system message. Steering
text byte-identical to Anthropic path (same `_VERBOSITY_LEVELS` table).
- Replacing the tool content with `"Error: file not found"` reclassifies
the turn as `ERROR_CONTINUATION`: `reasoning_effort` stays `"high"`,
only verbosity steering applied (label list is
`["output_shaper:verbosity:L2"]`).
- Same body under `provider="anthropic"` — after adapting `messages` to
Anthropic `tool_result` block shape and swapping `reasoning_effort` for
`output_config.effort` — emits an identical label list
(`test_label_vocabulary_matches_anthropic`), confirming the
savings-ledger / outcome-funnel contract holds.
- **Not tested**:
- End-to-end against a real OpenAI upstream. The shaper is a
request-side mutation and its correctness is defined by the emitted body
+ label vocabulary, both fully covered by unit tests. The receiving
OpenAI API's behavior on the shaped body (whether it honors
`reasoning_effort: low`, whether the trailing system message steers
verbosity as designed) is a downstream property, not a shaper property.
- `mypy headroom` — not run locally (project `[dev]` extra installed,
but type-checking wasn't part of the local iteration loop). Ruff is
clean and the new code carries full type hints.
- Verbosity steering for the Responses API's `body["input"]` item list.
Deferred by design — see the comment at the `/v1/responses` insertion
site in `handlers/openai.py`. Effort routing on the Responses path is
covered.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable — `CHANGELOG.md`
exists but appears release-managed; happy to add an `## Unreleased`
entry if that's the desired convention.
## Additional Notes
- **Backward compatibility**: `classify_turn`,
`apply_verbosity_steering`, `route_effort`, and `shape_request` all
default to `provider="anthropic"`, so `headroom/learn/verbosity.py:37`
and `handlers/anthropic.py:1949-1953` (the only external callers) keep
working with zero changes.
- **Follow-up scope** (happy to file as a separate PR if wanted):
1. Verbosity steering for the Responses API's `body["input"]` item list
— requires handling `message` vs `tool_output` vs `reasoning` item types
uniformly.
2. `max_completion_tokens` cap on mechanical turns as a third effort
lever — kept out of scope here because it's behavior-changing beyond
"clamp existing effort".
3. Extend the label-vocabulary parity test into a property-based fixture
that fuzzes matched Anthropic ↔ OpenAI bodies.
- **Deployment story on my side**: I'm running this on Raspberry Pi 5
via Ansible; the pinned commit will get installed as
`git+https://github.com/dylanrussellmd/headroom.git@<sha>` until this
merges upstream and lands in a `headroom-ai` release. I mention this
only because it exercises the change in a real proxy against real
OpenAI-compatible upstream traffic; happy to report savings numbers back
once the deployment stabilizes.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Gaurav Yadav
|
b0fa84e84d
|
fix: add Vercel deploy config and workflow for docs site (#1739)
## Description The Vercel docs site at headroom-docs.vercel.app had no automated deployment pipeline, so newly added pages (persistent-installs, savings) return 404 despite existing in the repo and building correctly locally. Closes #1730 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking function added) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add docs/vercel.json with explicit Next.js project config (framework, build/install commands) - Add deploy-vercel job to .github/workflows/docs.yml to auto-deploy on pushes to main touching docs/** ## Testing - [x] Unit tests pass (pytest) - [ ] Linting passes (ruff check .) - [ ] Type checking passes (mypy headroom) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ``` Local build verification: cd docs && npm ci && npm run build Build succeeded - persistent-installs and savings pages generated at .next/server/app/docs/persistent-installs.html and .next/server/app/docs/savings.html ``` ## Real Behavior Proof - Environment: Linux x86_64, Node.js 20 - Exact command / steps: 1. cd docs && npm ci && npm run build 2. Checked .next/server/app/docs/ for generated HTML artifacts 3. Verified source.getPage(["persistent-installs"]) returns page object - Observed result: Both pages build and render correctly locally - Not tested: Live Vercel deployment requires maintainer secrets (VERCEL_TOKEN, VERCEL_ORG_ID, VERCEL_PROJECT_ID) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes Requires three repo secrets: VERCEL_TOKEN, VERCEL_ORG_ID, VERCEL_PROJECT_ID. |
||
|
Ship-Wright
|
b4d4f641f3
|
docs: add Claude Code status-line indicator to Community (#1790)
## Description Adds a single **Community projects** entry to the `## Community` section of the README, linking a community-built Claude Code plugin ([`Ship-Wright/headroom-plugin`](https://github.com/Ship-Wright/headroom-plugin)) that surfaces Headroom usage in the editor status line. Docs-only; no code, config, or dependencies change. Context: headroomlabs-ai/headroom discussion #1789. Closes # (N/A — documentation addition, no linked issue) ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a `### Community projects` subsection under `## Community` in `README.md` with one bullet linking `Ship-Wright/headroom-plugin` and a one-line description of what it shows. ## Testing <!-- Docs-only change: no Python code paths touched, so the code test suite is not applicable. --> - [ ] Unit tests pass (`pytest`) — N/A (no code changed) - [ ] Linting passes (`ruff check .`) — N/A (no code changed) - [ ] Type checking passes (`mypy headroom`) — N/A (no code changed) - [ ] New tests added for new functionality — N/A (documentation) - [x] Manual testing performed (rendered the Markdown diff; verified the link resolves) ### Test Output ```text $ git diff --stat main README.md | 4 ++++ 1 file changed, 4 insertions(+) # Rendered diff: a new "### Community projects" heading + one bullet appears under "## Community". # Link check: https://github.com/Ship-Wright/headroom-plugin → 200 OK, public, MIT-licensed. ``` ## Real Behavior Proof - Environment: GitHub-flavored Markdown (README preview), macOS. - Exact command / steps: edited `README.md`, `git diff` to confirm a 4-line addition, previewed the rendered section, and opened the linked repo URL to confirm it is public and installable. - Observed result: the new `### Community projects` bullet renders correctly under `## Community`; the link points to a working, public plugin repo. - Not tested: nothing in the Python package changed, so `pytest` / `ruff` / `mypy` were not run (no applicable code paths). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines (matches existing Community bullet formatting) - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas — N/A (docs) - [x] I have made corresponding changes to the documentation (this PR is the documentation change) - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works — N/A (docs) - [ ] New and existing unit tests pass locally with my changes — N/A (no code changed) - [ ] I have updated the CHANGELOG.md if applicable — N/A (README-only community link) ## Screenshots (if applicable) N/A — a one-line text addition; see the diff. ## Additional Notes This is a minimal, opt-in community link — totally fine to reword, relocate, or decline. Several checklist/testing items are marked N/A because the change is documentation-only and touches no Python code. Happy to adjust the wording or move it elsewhere in the README if you'd prefer. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Zhenjia ZHOU
|
f8915067f6
|
feat(evals): register multilingual multi-wiki-qa (zh/ja/ko) dataset (#1530)
## Description The eval framework's `DATASET_REGISTRY` (`headroom/evals/datasets.py`) only had English datasets (squad, hotpotqa, longbench, …), so the LLM-in-the-loop `BeforeAfterRunner` could not be pointed at Chinese/Japanese/Korean. This registers `alexandrainst/multi-wiki-qa` as `multi_wiki_qa` — the only HF-loadable dataset with **uniform zh/ja/ko** extractive QA: SQuAD-style, paper-guaranteed **verbatim-span** answers over full Wikipedia articles. It makes multilingual compression eval first-class in the framework. Pairs with #1527 (which fixes the CJK-broken F1 tokenization + token estimation the framework's metrics use), so registered CJK data feeds into CJK-correct metrics. ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `headroom/evals/datasets.py`: add `load_multi_wiki_qa(n, lang)` — mirrors `load_longbench` exactly (the `_check_datasets_installed()` guard, `EvalCase` construction, `EvalSuite` return); reads the verified schema `answers["text"][0]` (a verbatim substring of `context`). - Register it in `DATASET_REGISTRY` under a new `rag_multilingual` category (same 4-key shape as every other entry). - `tests/test_evals_multilingual.py`: offline registration/shape tests (the live HF load is exercised in Real Behavior Proof, matching the other loaders). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`/`format`) - [x] Type checking passes (`mypy headroom/evals/datasets.py`) - [x] New tests added for new functionality - [x] Manual testing performed (see Real Behavior Proof) ### Test Output ```text $ .venv/bin/python -m pytest tests/test_evals_multilingual.py 2 passed $ ruff check / ruff format --check headroom/evals/datasets.py # clean $ mypy headroom/evals/datasets.py # clean ``` ## Real Behavior Proof - Environment: macOS (Darwin 25.3.0), Python in a uv venv with the `[evals]` extra (`datasets`), branch `feat/evals-multilingual-dataset` off `main`. - Exact command / steps: called the new loader against the live dataset — `load_multi_wiki_qa(n=3, lang="ja")`. - Observed result: it returned an `EvalSuite` named `multi_wiki_qa_ja` with 3 `EvalCase`s, each with non-empty `id`/`context`/`query`/`ground_truth`; the `ground_truth` is a **verbatim substring** of its `context` (the property the answer-retention eval relies on); contexts are full-article length (~2,796 chars). Sample answer: `'1988年10月'`. ```text suite: multi_wiki_qa_ja cases: 3 case fields ok: True ground_truth verbatim-substring of context: True ctx len: 2796 | answer: '1988年10月' ``` - Not tested: an end-to-end `BeforeAfterRunner` run against a live LLM (that costs API calls and is out of scope for the loader); zh-cn/ko configs (same schema, verified present via the dataset's splits). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (internal eval tooling) - [x] My changes generate no new warnings - [x] I have added tests that prove the feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md — N/A: `headroom/evals/` is internal dev tooling, not user-facing runtime ## Additional Notes - **No new dependency.** `multi-wiki-qa` loads through the existing `[evals]` `datasets` extra (guarded by `_check_datasets_installed()`); the dataset is fetched at run time and **never vendored** into the repo. - **License:** `alexandrainst/multi-wiki-qa` is CC-BY-NC-SA-4.0 (non-commercial) — consistent with how the repo already references externally-licensed datasets (SQuAD/LongBench) by id without committing their data. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Tejas Chopra
|
9446eea943
|
test(codex-ws): de-flake semaphore-tail test (widen noise floor 25ms->75ms) (#1736)
`test_concurrent_compression_has_no_semaphore_tail` failed on main CI (shard 1) with p50=1ms, p99=29ms, ratio=22x. It's flaky, not a regression: - `p99` is the max of only ~12 concurrent samples, so one CI scheduler/GC outlier dominates it. - The real semaphore-contention signal is **p99 ~2433ms** (success criterion <250ms). 29ms is 84x below that. - Passes 3/3 locally. Widen the noise floor 25ms -> 75ms: absorbs CI jitter, still 3.3x below the 250ms regression threshold (a genuine hundreds-of-ms tail still trips it). Comment updated. No production code change. Does not block the v0.29.0 publish (release.yml runs no pytest), but greens main CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Abhay Singh
|
a5d7e12c90
|
fix(proxy/batch): preserve sibling tool configs on Google batch requests (#2177)
## Description
When Headroom optimizes a Google/Gemini batch request, it silently drops
every tool config that isn't `functionDeclarations`.
In `handle_google_batch_create` the per-item optimizer extracts the
function declarations:
```python
tools = req_content.get("tools")
existing_funcs = None
if tools:
for tool in tools:
if "functionDeclarations" in tool:
existing_funcs = tool["functionDeclarations"]
break
```
and then rebuilds the forwarded request's tools as a single entry:
```python
if existing_funcs is not None:
compressed_req_content["tools"] = [{"functionDeclarations": existing_funcs}]
```
Gemini's `tools` array is a list of heterogeneous entries —
`{"functionDeclarations": [...]}` can sit alongside `{"googleSearch":
{}}` and `{"codeExecution": {}}`. Collapsing the array to one
`functionDeclarations` entry discards those siblings, so a batch request
that combines function calling with Google Search or code execution
reaches Google with those features stripped out. The request still
succeeds, so the loss is silent — the model just never grounds against
Search / never runs code.
The branch fires whenever the item had any `functionDeclarations` (or
CCR injected a retrieval tool), i.e. exactly the requests most likely to
also declare Search/code-execution.
## Fix
Rebuild the tools list from the original, replacing only the
`functionDeclarations` entry with the (possibly CCR-injected) funcs and
appending a new entry when the original had none:
```python
rebuilt_tools = []
replaced = False
for tool in tools or []:
if "functionDeclarations" in tool:
rebuilt_tools.append({**tool, "functionDeclarations": existing_funcs})
replaced = True
else:
rebuilt_tools.append(tool)
if not replaced:
rebuilt_tools.append({"functionDeclarations": existing_funcs})
compressed_req_content["tools"] = rebuilt_tools
```
Sibling entries (`googleSearch`, `codeExecution`, ...) are preserved in
place; the search/no-search behavior of the request is unchanged.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/batch.py`: preserve
non-`functionDeclarations` tool entries when rebuilding the optimized
Gemini batch request's tools array.
- `tests/test_proxy_handlers_batch.py`: new regression test asserting
`googleSearch` / `codeExecution` survive alongside
`functionDeclarations` in the forwarded body (uses the existing
`RealConvHandler` harness with the real converters).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/batch.py tests/test_proxy_handlers_batch.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/batch.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the array rebuild with a dependency-free script and left the
full pytest to CI.
- Exact command / steps: fed a tools array of
`[{functionDeclarations:[get_weather]}, {googleSearch:{}},
{codeExecution:{}}]` (plus a CCR-injected retrieval function) through
the OLD single-entry rebuild and the NEW preserving rebuild; also the
search-only case where CCR injects the first `functionDeclarations`.
- Observed result: OLD → `[{functionDeclarations:[...]}]` only
(googleSearch and codeExecution gone); NEW → all three entries retained
with the injected retrieval function present in `functionDeclarations`;
the search-only case gains a `functionDeclarations` entry while keeping
`googleSearch`.
- Not tested: a live Gemini batch submission; full local `pytest`
deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new test reuses the
in-file `RealConvHandler` harness (same one the existing
`..._preserves_functioncall_response_order` test uses) so it runs under
the normal CI pytest job; behaviour is additionally verified by the
standalone proof above.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
195ed90ced
|
fix(savings): record pre-compression original as ledger before, not forwarded count (#2176)
## Description
`headroom savings` overstates the proxy reduction percentage because the
durable ledger is written with the wrong `before` value.
In `PrometheusMetrics.record_request` the proxy appends a savings event:
```python
if tokens_saved > 0 and not self._stateless:
savings_ledger.record_savings_event(
tokens_before=input_tokens,
tokens_after=max(input_tokens - tokens_saved, 0),
...
)
```
But `input_tokens` here is the optimized, **post-compression** count
that was actually forwarded, not the original. `emit_request_outcome`
(the single funnel that calls `record_request`) passes
`input_tokens=outcome.optimized_tokens`.
The ledger derives the reported reduction as `saved / before`
(`savings_ledger._Bucket.savings_percent`), with `saved = max(before -
after, 0)`. Passing the forwarded count as `before` (and `before -
saved` as `after`) keeps `saved` correct but understates `before` by
`tokens_saved`, so the percentage is inflated:
- original input 1000 tokens, forwarded 600, saved 400 → true reduction
40%.
- recorded as `before=600, after=200` → `400 / 600` = **66.7%** on the
dashboard.
So `headroom savings` (which aggregates this ledger across restarts and
processes) reports a reduction percent well above what actually happened
for all proxy traffic.
## Fix
Reconstruct the original as forwarded + saved:
```python
tokens_before=input_tokens + tokens_saved, # the pre-compression original
tokens_after=input_tokens, # what we forwarded
```
`saved` (= `before - after` = `tokens_saved`) and the stored `cost_usd`
(derived from `saved`) are unchanged; only the `before`/`after` labels
are corrected, so the reduction percent becomes honest.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/prometheus_metrics.py`: pass
`tokens_before=input_tokens + tokens_saved` and
`tokens_after=input_tokens` to `record_savings_event`, with a comment
explaining that `input_tokens` is the forwarded count.
- `tests/test_savings_ledger_before_forwarded.py`: new regression guard
on the call shape.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/prometheus_metrics.py tests/test_savings_ledger_before_forwarded.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/prometheus_metrics.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the math with a dependency-free script modelling the ledger's
own `saved = before - after` and `saved / before * 100`, and left the
full pytest to CI.
- Exact command / steps: fed a request with original=1000,
forwarded=600, saved=400 through the OLD call shape
(`before=input_tokens`, `after=input_tokens-saved`) and the NEW shape
(`before=input_tokens+saved`, `after=input_tokens`).
- Observed result: OLD → `before=600, after=200`, reported 66.7%; NEW →
`before=1000, after=600`, reported 40.0% (the true reduction). `saved`
is 400 in both, so the cost figure is unaffected.
- Not tested: a live proxy end-to-end run; full local `pytest` deferred
to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The regression test
asserts on the source of `record_request` (the enclosing module imports
the ML stack, so it executes the assertion against the source text
rather than calling the method); the behavioural verification is the
standalone proof above.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
f723925be7
|
fix(proxy/gemini): forward a non-JSON upstream body with its real status (#2174)
## Description
`handle_gemini_generate_content` turns a non-JSON upstream error
response into a generic 502, hiding the real status and body.
After the upstream call it extracts usage from `response.json()`:
```python
try:
resp_json = response.json()
usage = resp_json.get("usageMetadata", {})
...
cache_read_tokens = usage.get("cachedContentTokenCount", 0)
except (KeyError, TypeError, AttributeError) as e: # <-- missing JSONDecodeError / ValueError
...
```
`response.json()` raises `json.JSONDecodeError` (a `ValueError`
subclass) for a non-JSON body. That isn't in the tuple, so it escapes to
the function's outer `except Exception`, which returns a synthetic 502
and discards the real `response.status_code` / `response.content` (which
the success path forwards verbatim). An overloaded Google/Vertex/Copilot
frontend commonly returns a 503/500/429 with an HTML or empty body, so
the client sees a generic 502 instead of the true status — defeating
retry/backoff and dropping the diagnostic.
The all-non-text early-exit branch in the same handler already handles
this correctly with the full tuple (`except (json.JSONDecodeError,
ValueError, KeyError, TypeError, AttributeError)`) and then forwards the
real status/content.
## Fix
Add `json.JSONDecodeError, ValueError` to the token-extraction `except`,
matching that sibling. On a non-JSON body the extraction is skipped
(token metrics keep their fallbacks) and the handler falls through to
`return Response(content=response.content,
status_code=response.status_code, ...)` — the real status and body.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/gemini.py`: broaden the token-extraction
`except` in `handle_gemini_generate_content` to include
`json.JSONDecodeError, ValueError`.
- `tests/test_gemini_nonjson_status.py`: new test asserting that except
clause catches the JSON/ValueError family (guards against the
regression).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/gemini.py tests/test_gemini_nonjson_status.py
All checks passed!
$ python -m py_compile headroom/proxy/handlers/gemini.py tests/test_gemini_nonjson_status.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. A full
`pytest` OOM-kills this box (ML stack import), so I verified the
exception handling with a dependency-free script that models the
token-extraction try/except plus the handler's verbatim-forward return,
and left the full pytest to CI.
- Exact command / steps: sent a 503 response whose `.json()` raises
`JSONDecodeError` (non-JSON body) through the old tuple and the new
tuple, plus a normal JSON 200 as a control.
- Observed result: old lets `JSONDecodeError` escape (→ the outer
handler's synthetic 502); new catches it and forwards the real 503; the
JSON 200 still extracts tokens under both. The new test asserts the real
handler's except clause includes the JSON/ValueError family.
- Not tested: a live overloaded Gemini upstream; full local `pytest`
deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" / "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run here; the
change adds two exception types matching an existing, tested sibling
branch, verified by the standalone proof and a source-level regression
guard (a full handler-integration harness for Gemini doesn't exist
in-tree, and the existing gemini integration tests hit a live API).
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
8f867e4622
|
fix(install): guard non-dict health config in 'install status' (#2150)
## Description
`headroom install status` crashes with an `AttributeError` when the
probed health endpoint returns a non-dict `config`.
```python
if payload and isinstance(payload, dict):
click.echo(f"Health URL: {manifest.health_url.replace('/readyz', '/health')}")
click.echo(f"Backend: {payload.get('config', {}).get('backend', manifest.backend)}")
```
`payload` is guarded as a dict, but `payload['config']` is not.
`dict.get('config', {})` only substitutes the `{}` default when the key
is **absent** — a present-but-non-dict `config` (`null`, a string, a
list) is returned as-is, and the chained `.get('backend', ...)` then
raises `AttributeError`, crashing the command with a raw traceback.
Reachability: the Headroom proxy normally returns `config` as an object,
so this bites when `install status` probes a port that a different or
older service is occupying (which can emit `config: null` or a
non-object), or a build that emits `config: null`. The correctly-guarded
sibling already exists in the codebase — `wrap.py`'s
`_proxy_health_config` does `config = payload.get("config"); return
config if isinstance(config, dict) else None`.
## Fix
Guard the `config` value with `isinstance(config, dict)` before the
`.get('backend', ...)` lookup, mirroring `_proxy_health_config`. A
non-dict (or missing) `config` falls back to the manifest's backend.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/install.py`: `install status` guards `config` with
`isinstance(config, dict)` before reading `backend`.
- `tests/test_cli/test_install_cli.py`: add
`test_install_status_survives_non_dict_config` (health payload with
`config: null` must not crash; backend falls back to the manifest).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/cli/install.py tests/test_cli/test_install_cli.py
All checks passed!
$ python -m py_compile headroom/cli/install.py tests/test_cli/test_install_cli.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the access with a
dependency-free script that replicates the old vs guarded lookup, and
left the full pytest (including the new CLI test) to CI.
- Exact command / steps: ran the old `payload.get('config',
{}).get('backend', ...)` and the new guarded lookup against `config`
values of `null`, a string, a list, a proper object, and a missing key.
- Observed result: the old lookup raises `AttributeError` for every
non-dict `config`; the new lookup falls back to the manifest backend for
those and returns the real backend for a proper object (and the
missing-key case is unchanged). The new CLI test drives `install status`
with `probe_json` returning `{"config": null}` and asserts a clean exit
with the manifest backend.
- Not tested: a live foreign service occupying the port; full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change adds an `isinstance` guard mirroring an existing
sibling, verified by the standalone proof and a new CLI test that reuses
the file's existing `install status` mocking harness.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
0cddac632d
|
fix(telemetry): only advance usage-report baseline after a 200 (#2149)
## Description
The usage reporter permanently drops a reporting window's usage whenever
the send to the cloud fails.
`UsageReporter._report_usage` computes usage as a **delta** against the
last snapshot, POSTs it, and then rebases the baseline:
```python
try:
resp = await client.post(f"{self._cloud_url}/v1/license/usage", json=payload, timeout=10.0)
if resp.status_code == 200:
...
else:
logger.warning("Usage report returned status %d", resp.status_code)
except Exception:
logger.warning("Failed to send usage report", exc_info=True)
# Update snapshot
self._snapshot_metrics() # runs on success, non-200, AND exception
self._last_report_time = now
```
`_snapshot_metrics()` rebases `_last_tokens_saved_by_model` /
`_last_tokens_sent_by_model` / `_last_requests_by_model` to the current
cumulative counters. Because it runs unconditionally after the POST, a
report that fails to send — non-200 or a raised exception — still
advances the baseline. The module is explicitly built to tolerate a
briefly-unreachable cloud (7-day grace, cached license), so this is a
normal, recurring situation.
The consequence: the failed window's requests and tokens are never
re-included. The next report is a delta from the advanced baseline, so
that window is silently and permanently lost from usage-based billing /
quota. Every transient network blip under-counts usage. (The
`total_requests == 0` early-return already gets this right — it advances
only `_last_report_time`, without snapshotting, since there's nothing to
lose.)
## Fix
Advance the baseline (`_snapshot_metrics()` and `_last_report_time`)
only inside the `resp.status_code == 200` branch. On a non-200 or an
exception, both baselines are left intact, so the next report covers the
full period since the last successful send and re-includes the
previously-failed window.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/telemetry/reporter.py`: move `_snapshot_metrics()` +
`_last_report_time = now` into the 200 branch of `_report_usage`.
- `tests/test_usage_reporter_snapshot.py`: new tests — baseline advances
on 200, and stays intact on a non-200 and on an exception (window
preserved).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/telemetry/reporter.py tests/test_usage_reporter_snapshot.py
All checks passed!
$ python -m py_compile headroom/telemetry/reporter.py tests/test_usage_reporter_snapshot.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the delta accounting with a
dependency-free script that models two windows across a failed then
successful send, and left the full pytest to CI.
- Exact command / steps: window 1 saves 100 tokens and the send fails;
window 2 saves another 50 (cumulative 150) and the send succeeds. Ran
under the old (unconditional snapshot) and new (snapshot-on-200) logic.
- Observed result: old delivers only 50 tokens total (window 1's 100
dropped when the baseline advanced on the failed send); new delivers the
full 150 (window 2's delta re-includes window 1). The new tests assert
the baseline advances on 200 and is untouched on a non-200 / exception,
driving the real `_report_usage` with a fake proxy + client.
- Not tested: a live cloud round-trip; full local `pytest` deferred to
CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change moves two lines into the success branch,
verified by the standalone delta-accounting proof and new tests that
drive the real `_report_usage` via `object.__new__` with a fake proxy
and HTTP client (200, non-200, and exception).
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
fb17156bfa
|
fix(savings): don't bill free models at the $3/M fallback in the ledger (#2147)
## Description
The durable savings ledger records phantom cost-avoided for free
(0-priced) models, billing them at the `$3/M` blended fallback.
`estimate_cost_usd` prices a known model via
`_estimate_compression_savings_usd`, but gates the result on `> 0`:
```python
if model and model != UNKNOWN:
priced = _estimate_compression_savings_usd(model, tokens_saved)
if priced > 0: # <-- the bug
return round(priced, 6)
return round(float(tokens_saved) * float(fallback_rate), 6) # ~$3/M
```
`_estimate_compression_savings_usd` deliberately distinguishes three
cases: a litellm-priced model (`> 0`), a model litellm can't price
(returns the blended fallback itself), and a model that is *legitimately
free* — litellm has an entry with `input_cost_per_token == 0.0`, so it
returns `tokens_saved * 0.0 == 0.0`. Its own comment calls this out:
"`if not ...` treated a real 0.0 as unavailable and billed the $3/M
fallback — phantom savings for a model that costs nothing."
The ledger's `if priced > 0` re-introduces exactly that defect: a free
model's `0.0` is treated as "unpriced" and the code falls through to the
`$3/M` fallback. Every saved token on a free/local/promo model is then
written into the durable JSONL ledger — and surfaced by `headroom
savings` / `aggregate_savings` — as cost-avoided that never existed.
## Fix
Trust `_estimate_compression_savings_usd`'s return verbatim for known
models. It already returns the blended fallback for models litellm can't
price and `0.0` for free ones, so the `> 0` gate is not needed — and is
the source of the double-fallback. The `UNKNOWN`/empty-model path still
uses `fallback_rate` as before.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/savings_ledger.py`: `estimate_cost_usd` returns
`_estimate_compression_savings_usd(...)` unconditionally for known
models instead of gating on `> 0`.
- `tests/test_savings_ledger.py`: add
`test_free_model_is_not_billed_at_fallback` (free model → $0) and
`test_priced_model_uses_litellm_estimate` (priced model → estimate),
both monkeypatching the helper.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/savings_ledger.py tests/test_savings_ledger.py
All checks passed!
$ python -m py_compile headroom/savings_ledger.py tests/test_savings_ledger.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the pricing with a
dependency-free script that replicates the gate and the helper's three
cases, and left the full pytest to CI.
- Exact command / steps: priced 1,000,000 saved tokens for a free model,
a priced model, a litellm-unknown named model, and the explicit
`UNKNOWN` sentinel, under the old (`> 0` gate) and new (unconditional)
logic.
- Observed result: the old logic bills the free model `$3.00` (phantom);
the new logic bills `$0.00`. The priced model (`$2.00`), the
litellm-unknown fallback (`$3.00`), and the `UNKNOWN`-path
`fallback_rate` are unchanged. The new tests assert the free-model `$0`
and the priced-model estimate via a monkeypatched helper.
- Not tested: a live litellm lookup for a real free model; full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change removes a `> 0` gate in a pure pricing function,
verified by the standalone proof and the new tests. This is the same
category as the earlier zero-price-model fix, but at a distinct,
still-buggy call site (the durable ledger) — the earlier fix landed
inside `_estimate_compression_savings_usd`.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Ingmar Krusch
|
d125805589
|
fix(proxy/savings): append history point on cache-only savings too (#2194)
## Description `SavingsTracker.record_request()`'s history-append guard only fired when `tokens_saved > 0` (headroom's own lossy compression). In `--mode cache`, `tokens_saved` is near-always 0 by design — the frozen prefix is byte-replayed rather than compressed, to keep the provider's prompt cache warm. That silently dropped every history point on a cache-mode deployment even when `cache_read_tokens`/`cache_savings_usd` were large, making `headroom-monthly`-style tooling read as a total savings collapse. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/savings_tracker.py`: widen the history-append guard in `record_request()` to fire on `tokens_saved > 0` OR `cache_read_tokens > 0`, and carry `cache_read_tokens`/`cache_savings_usd` on the appended history entry so downstream consumers can show them. `_normalize_history_entry` defaults both fields to `0`/`0.0` for legacy entries that predate this change. - `tests/test_proxy_savings_history.py`: regression coverage that a cache-only request (zero `tokens_saved`, nonzero `cache_read_tokens`) still appends a history point, that the new fields round-trip through normalization, and that legacy history entries without the new keys still normalize cleanly. - `CHANGELOG.md`: added a `### Fixed` entry under `Unreleased`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_proxy_savings_history.py tests/test_savings_tracker_zero_price.py tests/test_proxy_project_savings.py -q collected 62 items tests/test_proxy_savings_history.py .................................... [ 58%] ...... [ 67%] tests/test_savings_tracker_zero_price.py .... [ 74%] tests/test_proxy_project_savings.py ................ [100%] ======================== 62 passed, 1 warning in 5.81s ========================= $ uv run ruff check headroom/proxy/savings_tracker.py tests/test_proxy_savings_history.py All checks passed! $ uv run mypy headroom/proxy/savings_tracker.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - **Environment:** personal fork deployed as a real proxy (macOS launchd service, `headroom install apply`) with `--backend bedrock --mode cache`, fronting a live Claude Code session where `cache_read_tokens`/`cache_savings_usd` are the dominant savings mechanism and headroom's own compression (`tokens_saved`) is near-always 0. - **Exact command / steps:** ran a multi-turn Claude Code session against this deployment, then inspected `proxy_savings.json`'s `history` array and the `headroom-monthly` savings-history rollup that reads it. - **Observed result:** before the fix, `history` stayed empty (or stopped growing) across the whole cache-mode session despite the lifetime `cache_read_tokens`/`cache_savings_usd` counters climbing turn over turn, because every request had `tokens_saved == 0` and never passed the append guard. `headroom-monthly` therefore rendered a flat/zero savings trend for a deployment that was, by its own lifetime counters, saving real money. After the fix, the same session appends a history point on every cache-hit turn, `cache_read_tokens`/`cache_savings_usd` populate on each new entry, and the monthly rollup tracks the lifetime counters instead of reading as a collapse. - **Not tested:** this deployment is `--mode cache`-only; the `tokens_saved > 0` branch of the widened guard (plain `--mode token` compression-driven savings) was not independently re-verified live post-fix, only via the existing/updated test suite, since it was already covered by pre-existing behavior. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — backend logic change, no UI surface. ## Additional Notes - No linked issue number: found via independent investigation of a personal deployment, not filed as a `headroomlabs-ai/headroom` issue first. - Also incidentally fixes a pre-existing test isolation gap in `test_savings_tracker_helpers_normalize_inputs_and_paths` (it unset `HEADROOM_SAVINGS_PATH` but not `HEADROOM_WORKSPACE_DIR`, so the default-path assertion could pick up whatever workspace a live deployment on the test machine had exported, instead of the library default). --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Abhay Singh
|
84f66da36f
|
fix(init/codex): merge into hooks.json instead of overwriting it (#2173)
## Description
`headroom init codex` destroys a user's existing Codex hooks.
`_ensure_codex_hooks` builds a fresh payload containing only Headroom's
two hooks and writes it wholesale:
```python
payload = { "hooks": { "SessionStart": [...headroom...], "PreToolUse": [...headroom...] } }
path.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
```
It never reads the existing file, so any user-managed hooks (and any
other top-level keys) in `~/.codex/hooks.json` are silently replaced
with just Headroom's entries — data loss on every `init codex` run.
The sibling registrars do it correctly: `_ensure_claude_hooks` and
`_ensure_copilot_hooks` both read via `_json_file`, then merge per event
and dedup on the Headroom marker, preserving unrelated user entries. The
codex path was the lone writer that overwrote.
## Fix
Read-merge-write, mirroring `_ensure_claude_hooks`: load the existing
payload, keep each event's entries that don't carry the
`headroom-init-codex` marker, append Headroom's, and write back. User
hooks and other top-level keys survive; Headroom's are deduped
(idempotent re-runs).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/init.py`: `_ensure_codex_hooks` reads via `_json_file`,
merges per event with marker dedup, and writes via `_write_json` (no
more wholesale overwrite).
- `tests/test_cli/test_init_cli.py`: add
`test_ensure_codex_hooks_preserves_user_hooks` — a user hook and an
unrelated top-level key survive; Headroom's hook is appended once.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/cli/init.py tests/test_cli/test_init_cli.py
All checks passed!
$ python -m py_compile headroom/cli/init.py tests/test_cli/test_init_cli.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the merge with a
dependency-free script that replicates the old (overwrite) vs new
(merge) logic, and left the full pytest to CI.
- Exact command / steps: gave a config with a user `SessionStart` hook
(`echo my-own-hook`) and an unrelated top-level key (`notify: true`),
then ran both the old and new logic.
- Observed result: old drops both the user hook and the top-level key;
new keeps both and appends Headroom's hook exactly once. The new test
asserts this against the real `_ensure_codex_hooks`.
- Not tested: a live `headroom init codex` end to end; full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change mirrors the existing `_ensure_claude_hooks`
merge logic, verified by the standalone proof and the new test (which
reuses the file's existing hooks-test harness).
|
||
|
Abhay Singh
|
8da4384bfc
|
fix(init/codex): don't delete per-profile provider settings (#2146)
## Description `headroom init codex` silently deletes a user's per-profile provider settings. `_ensure_codex_provider` owns the root-level `model_provider` / `openai_base_url` keys, and to avoid emitting a duplicate top-level key it strips any prior assignment before re-inserting its block: ```python content = re.sub(r"(?m)^[ \t]*model_provider[ \t]*=.*\r?\n", "", content) content = re.sub(r"(?m)^[ \t]*openai_base_url[ \t]*=.*\r?\n", "", content) ``` Those multiline regexes match the keys at any indentation, **in any TOML table**. Codex supports per-profile overrides: ```toml [profiles.work] model_provider = "azure" [profiles.gpt5] model_provider = "openai" ``` So a user with named Codex profiles who runs `headroom init codex` has every `[profiles.*]` `model_provider` / `openai_base_url` line silently removed. Those profiles then fall through to the injected root `model_provider = "headroom"` default — their routing is quietly changed. That collateral deletion isn't needed to prevent the root-level duplicate the strip exists for (#260); the unwrap-side sibling `_strip_codex_init_block` proves the intent is precise (it only removes the Headroom-owned value). ## Fix Scope the strip to the document root — everything before the first table header. Root-level `model_provider` / `openai_base_url` are still replaced (init owns them), but keys inside `[profiles.*]` (or any other table) are left untouched. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/init.py`: `_ensure_codex_provider` splits the config at the first table header and strips `model_provider`/`openai_base_url` only from the root section. - `tests/test_cli/test_init_cli.py`: add `test_ensure_codex_provider_preserves_profile_overrides` — a `[profiles.work]` override survives init while the root key is replaced by `headroom`. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/cli/init.py tests/test_cli/test_init_cli.py All checks passed! $ python -m py_compile headroom/cli/init.py tests/test_cli/test_init_cli.py OK ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing `headroom` pulls in the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the strip with a dependency-free script that replicates the old (whole-file) vs new (root-scoped) regex, and left the full pytest to CI. - Exact command / steps: ran both strippers on a config with a root `model_provider = "openai"` and a `[profiles.work]` block overriding `model_provider`/`openai_base_url`. - Observed result: the old strip deletes the `[profiles.work]` overrides too; the new strip keeps them and still removes the root assignment. The new test asserts the profile override survives and the root becomes `headroom`. - Not tested: a live `headroom init codex` end to end; full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The "unit tests pass locally" and "type checking" boxes are unchecked because the full suite imports the ML stack, which I can't run in this environment; the change scopes an existing regex strip to the document root, verified by the standalone proof and the new test (the two existing `_ensure_codex_provider` tests only exercise root-level and block-placement behavior, both preserved). I kept the fix to root-scoping rather than also matching only the `"headroom"` value, since that preserves the #260 duplicate-key guard without the broad deletion. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Abhay Singh
|
8a71947023
|
fix(proxy): reject rate_limit_requests_per_minute=0 when limiting is enabled (#2142)
## Description
A `rate_limit_requests_per_minute` of 0 makes the proxy return a 500 on
every rate-limited request instead of failing configuration early.
The token-bucket wait computation divides by the per-minute rate:
```python
def consume_from_bucket(*, available_tokens, requested_tokens, rate_per_minute):
if available_tokens >= requested_tokens:
return True, available_tokens - requested_tokens, 0.0
wait_seconds = (requested_tokens - available_tokens) * (60.0 / rate_per_minute)
return False, available_tokens, wait_seconds
```
With `rate_limit_requests_per_minute == 0`, the bucket initializes to 0
tokens, so the first request reaches the division and raises
`ZeroDivisionError`. The CLI guards `--rpm` with
`click.IntRange(min=1)`, but `HEADROOM_PROXY_CONFIG_JSON` and
programmatic `ProxyConfig(...)` construction bypass that guard.
## Fix
Validate `rate_limit_requests_per_minute >= 1` in
`ProxyConfig.__post_init__` when `rate_limit_enabled`, mirroring the
existing `retry_max_attempts` validation. Bad enabled configs now fail
fast with a clear message. When rate limiting is disabled, `rpm=0`
remains inert and is not rejected.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/models.py`: reject `rate_limit_requests_per_minute <
1` when `rate_limit_enabled`.
- `tests/test_proxy_config_rate_limit.py`: cover zero/negative enabled
values, disabled zero, and a valid enabled value.
- `CHANGELOG.md`: add a bug-fix entry.
- Merged current `main` to pick up the repository-wide memory factory
type annotation fix that was breaking the PR lint job.
## Testing
- [x] Unit tests pass (`pytest` focused locally; broader CI passed on
the pre-merge head and fresh CI is running on the main-merged head)
- [x] Linting passes (`ruff check` focused locally)
- [x] Type checking passes for the prior mypy blocker after merging
current `main`
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
uvx ruff@0.15.17 check headroom/proxy/models.py tests/test_proxy_config_rate_limit.py headroom/memory/factory.py
All checks passed!
uvx ruff@0.15.17 format --check headroom/proxy/models.py tests/test_proxy_config_rate_limit.py headroom/memory/factory.py
3 files already formatted
git diff --check headroomlabs/main...HEAD
# no output
uv run --extra dev python -m pytest tests/test_proxy_config_rate_limit.py -q
4 passed
```
## Real Behavior Proof
- Environment: Windows 11 review worktree, Python 3.13.3.
- Exact command / steps: ran the focused rate-limit config test file and
targeted lint/format checks.
- Observed result: enabled zero and negative rpm raise `ValueError`;
disabled zero is accepted; valid enabled rpm is accepted.
- Not tested: full suite; fresh CI is queued after the main merge.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The validation is intentionally at the config boundary to match the
CLI's `IntRange(min=1)` contract and the existing fail-fast
`retry_max_attempts` check.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Abhay Singh
|
def2f9a728
|
fix(learn): don't desync verbosity pairing on empty assistant turns (#2123)
## Description
`verbosity._ordered_events` and `_parse_session` disagree about empty
assistant turns, which desyncs the response list and produces spurious
fast-skips.
`_parse_session` only creates a `_Response` when an assistant message
actually said something:
```python
if words > 0 or out_tok > 0:
responses.append(_Response(...))
```
But `_ordered_events` consumes one `responses[ri]` for **every**
assistant line, with no matching filter:
```python
if ltype == "assistant" and ri < len(responses):
out.append((responses[ri].ts, "assistant", responses[ri]))
ri += 1
```
So an assistant turn with no text and no output tokens — for example a
pure `tool_use` turn where `usage` is absent — creates no `_Response` at
parse time, yet still consumes a slot in `_ordered_events`. That slot
actually belongs to a *later* real response, so the two lists drift by
one. A human reply that follows the real answer is then paired with the
next answer's (future) timestamp, `ts - last_resp.ts` goes negative, and
since a negative gap is always below the read-fraction threshold, a
spurious `fast_skip` is recorded. That inflates `fast_skip_rate`, which
feeds `pressure`, which lowers the recommended verbosity level.
The user side of `_ordered_events` already replicates its parse-site
filter (`_human_text(...) is None -> continue`); only the assistant side
was missing the equivalent guard. That asymmetry is the bug.
## Fix
In `_ordered_events`, compute `words`/`out_tok` for the assistant line
the same way `_parse_session` does and only consume a response when
`words > 0 or out_tok > 0`, keeping the two functions in lockstep.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/verbosity.py`: `_ordered_events` applies the `words >
0 or out_tok > 0` guard on the assistant branch before consuming a
response, with a comment explaining the desync.
- `tests/test_verbosity_learn.py`: add `_empty_assistant` helper and
`test_empty_assistant_message_does_not_desync_fast_skip` (an empty
assistant turn before a real answer + a slow reply must not record a
fast skip).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [x] Unit tests pass (`uv run --extra dev pytest
tests/test_verbosity_learn.py::TestSignalExtraction::test_empty_assistant_message_does_not_desync_fast_skip
-q`)
- [x] Linting passes (`uvx ruff@0.15.17 check
headroom/learn/verbosity.py tests/test_verbosity_learn.py
headroom/memory/factory.py`)
- [x] Type checking passes (`uvx mypy==1.20.2
headroom/memory/factory.py`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/learn/verbosity.py tests/test_verbosity_learn.py
All checks passed!
$ python -m py_compile headroom/learn/verbosity.py tests/test_verbosity_learn.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the alignment with a
dependency-free script that models the parse-site filter, the old vs new
`_ordered_events` consume, and the resulting human-to-response pairing,
and left the full pytest to CI.
- Exact command / steps: built an event stream `[empty assistant, real
answer #1, fast human reply, real answer #2, reply]`, computed the
response list from the parse filter, then walked the old (unfiltered)
and new (filtered) consume to find the gap between the first human and
the response paired before it.
- Observed result: old consume pairs the reply with answer #2 (a future
timestamp) -> gap `-8` (spurious fast_skip); new consume keeps alignment
and pairs it with answer #1 -> gap `+1`. The new test builds a session
with an empty assistant turn and a genuinely slow reply and asserts
`fast_skips == 0`.
- Not tested: a real Claude Code transcript end to end; full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
Merged current `main` to pick up the repository-wide mypy cache-key
annotation fix, then verified the focused regression locally. the change
adds the existing parse-site filter to one branch of a pure file-parsing
function, verified by the standalone alignment proof and the new
regression test for CI.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
faed4dcfe7
|
fix(wrap/claude): bind _wrap_settings_path before the try (#2126)
## Description
`headroom wrap claude` crashes with an `UnboundLocalError` from its
cleanup `finally` whenever the proxy fails to start, which both hides
the real error and skips cleanup.
`claude()` initializes its cleanup state before the `try` so the
`finally` can always reference it — `proxy_holder`, `_saved_base_url`,
`_settings_foundry`, `port_holder`, `_settings_vertex` are all bound up
front. But `_wrap_settings_path` was the exception: it was assigned only
inside the `try`, after `_ensure_proxy`:
```python
try:
...
proxy_holder[0], actual_port = _ensure_proxy(port, ...) # can raise
...
_wrap_settings_path = Path.cwd() / ".claude" / "settings.local.json" # assigned here
...
finally:
_restore_claude_wrap_base_url(..., settings_path=_wrap_settings_path) # referenced here
cleanup()
```
`_ensure_proxy` raises when the requested port is unavailable and the
range is exhausted, or when the proxy subprocess fails to start. When it
does, control jumps to the `finally`, which evaluates
`settings_path=_wrap_settings_path` — a local that was never assigned —
and raises `UnboundLocalError`. That replaces the real failure with a
raw traceback, and because the `finally` aborts on that line,
`cleanup()` never runs, so proxy cleanup and wrap-marker clearing are
skipped too.
## Fix
Bind `_wrap_settings_path` before the `try`, next to the other cleanup
holders, so the `finally` can always reference it. The value is
unchanged (the in-`try` assignment is removed since it computed the same
path).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/wrap.py`: hoist the `_wrap_settings_path` initialization
to before the `try` (alongside `proxy_holder`/`_saved_base_url`/…) and
drop the redundant in-`try` assignment.
- `tests/test_cli/test_wrap_claude_finally_unbound.py`: new test — drive
`wrap claude` with `_ensure_proxy` patched to raise and assert the
`finally` completes (no `UnboundLocalError`, and both restore and
cleanup ran).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [x] Unit tests pass (`uv run --extra dev pytest
tests/test_cli/test_wrap_claude_finally_unbound.py -q`)
- [x] Linting passes (`uvx ruff@0.15.17 check headroom/cli/wrap.py
tests/test_cli/test_wrap_claude_finally_unbound.py
headroom/memory/factory.py`)
- [x] Type checking passes (`uvx mypy==1.20.2
headroom/memory/factory.py`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_claude_finally_unbound.py
All checks passed!
$ python -m py_compile headroom/cli/wrap.py tests/test_cli/test_wrap_claude_finally_unbound.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and running the CLI
test locally OOM-kills this box, so I verified the control flow with a
dependency-free script that reproduces the try/finally with the variable
assigned inside vs before the try, and left the full pytest (including
the new CLI test) to CI.
- Exact command / steps: ran the flow with the variable bound inside the
try (old) and before the try (new), each with an early failure that
fires before the in-try assignment.
- Observed result: old raises `UnboundLocalError` from the finally and
skips restore/cleanup; new runs the finally cleanly and lets the real
`RuntimeError` propagate. The new CLI test drives `wrap claude` with
`_ensure_proxy` raising and asserts no `UnboundLocalError` and that
restore and cleanup both ran.
- Not tested: a live proxy port-exhaustion end to end; full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
Merged current `main` to pick up the repository-wide mypy cache-key
annotation fix, then verified the focused regression locally. the change
hoists one assignment to before the `try` (mirroring the four sibling
holders three lines above), verified by the control-flow proof and a new
CLI test that reuses the same mocking pattern the existing `wrap claude`
vertex tests use.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
d236b27c60
|
fix(wrap/codex): export the detected custom upstream base URL (#2125)
## Description `headroom wrap codex` detects a user's custom upstream gateway but never tells Codex to use it, so the user's gateway key is sent to `api.openai.com`. `_inject_codex_provider_config` handles a Codex user who has an OpenAI-compatible gateway declared in `~/.codex/config.toml`, e.g. ```toml model_provider = "freemodel" [model_providers.freemodel] base_url = "https://api.freemodel.dev" ``` It injects the Headroom provider with `env_http_headers = { ... "X-Headroom-Base-Url" = "HEADROOM_CODEX_UPSTREAM_BASE_URL" }` and **returns the preserved upstream URL** so the caller can export it. Its docstring even says: *"Callers that go on to launch Codex should export this value into `HEADROOM_CODEX_UPSTREAM_BASE_URL`."* But `_prepare_codex_wrap_state` called it as a bare statement and discarded the return, and `_run_codex_wrap` / `_build_codex_launch_env` only ever set `OPENAI_BASE_URL`. A repo-wide grep confirms `HEADROOM_CODEX_UPSTREAM_BASE_URL` (`_UPSTREAM_BASE_URL_ENV_VAR`) is never assigned into any process env — it appears only at its definition and in that docstring. Since Codex only emits the `X-Headroom-Base-Url` header when the env var exists, the header is omitted, the proxy's OpenAI handler falls back to its hardcoded `https://api.openai.com`, and the user's `freemodel.dev` key is sent to OpenAI, which rejects it. This is a regression: the wiring existed in the original `#1614` fix (`_codex_custom_upstream = _inject_codex_provider_config(...)` then `env[_UPSTREAM_BASE_URL_ENV_VAR] = ...`) and was dropped by a later refactor that extracted `_prepare_codex_wrap_state`. ## Fix Restore the wiring: `_prepare_codex_wrap_state` now captures and returns `_inject_codex_provider_config`'s value, and `_run_codex_wrap` exports it into the launch env (`env[_UPSTREAM_BASE_URL_ENV_VAR] = custom_upstream`) when it is non-None and not already set, so a user-provided value still wins. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: `_prepare_codex_wrap_state` returns the detected custom upstream URL; `_run_codex_wrap` exports it into the launch env (and its display list) when set. - `tests/test_cli/test_wrap_codex.py`: add `TestCodexLaunchExportsCustomUpstream` — drives `_run_codex_wrap` with mocked prepare/launch and asserts the launch env carries `HEADROOM_CODEX_UPSTREAM_BASE_URL` when a custom upstream is detected, and does not when there isn't one. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [x] Unit tests pass (`uv run --extra dev pytest tests/test_cli/test_wrap_codex.py::TestCodexLaunchExportsCustomUpstream -q`) - [x] Linting passes (`uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py headroom/memory/factory.py`) - [x] Type checking passes (`uvx mypy==1.20.2 headroom/memory/factory.py`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py All checks passed! $ python -m py_compile headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py OK ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing `headroom` pulls in the torch/transformers stack and running the CLI test locally OOM-kills this box, so I verified the wiring with a dependency-free script that models prepare -> run -> the proxy's upstream fallback, and left the full pytest (including the new CLI test) to CI. - Exact command / steps: modelled the old flow (inject return discarded) and the new flow (return exported into the launch env), then applied the proxy's rule that a missing `HEADROOM_CODEX_UPSTREAM_BASE_URL` falls back to `api.openai.com`. - Observed result: old effective upstream is `https://api.openai.com` (the gateway key is misrouted); new effective upstream is `https://api.freemodel.dev` (the user's gateway). The new CLI test asserts the launch env carries the var when a custom upstream is present and omits it otherwise. - Not tested: a live Codex process reading the env and emitting the header; full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes Merged current `main` to pick up the repository-wide mypy cache-key annotation fix, then verified the focused regression locally. the change threads one return value through two functions and exports it, verified by the wiring proof and a new CLI test that drives `_run_codex_wrap` with the heavy prepare/launch steps mocked so only the env-export logic is exercised. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Abhay Singh
|
f8eaaeb26a
|
fix(cache): normalize embeddings before the semantic similarity check (#2122)
## Description
The semantic tier of the dynamic-content detector compares an
unnormalized dot product against a cosine threshold, so it flags almost
everything as dynamic and strips the static content it is supposed to
protect.
`SemanticDetector` pre-computes exemplar embeddings and, per sentence,
scores similarity with `np.dot` and compares to `semantic_threshold`:
```python
self._exemplar_embeddings = self._model.encode(self.DYNAMIC_EXEMPLARS, convert_to_numpy=True)
...
sentence_embeddings = self._model.encode(sentence_texts, convert_to_numpy=True)
similarities = np.dot(sentence_embeddings, self._exemplar_embeddings.T)
...
if max_sim < self.config.semantic_threshold: # semantic_threshold defaults to 0.7
continue
```
`sentence_transformers.encode(..., convert_to_numpy=True)` does **not**
normalize by default. So `np.dot` here is an inner product whose
magnitude scales with the embedding norms (typically ~5-15 for MiniLM),
not a cosine similarity in [0, 1]. Comparing that against
`semantic_threshold=0.7` (documented and configured as a 0-1 similarity)
is a scale mismatch: nearly every sentence clears the threshold, so the
semantic tier classifies almost all text as dynamic, moves it into
`dynamic_content`, and empties `static_content` — busting the very cache
the detector exists to protect.
A standalone repro: an unrelated sentence with a true cosine of ~0.1 to
an exemplar produces a raw dot of ~9.1 (well over 0.7); normalized, it
correctly scores ~0.09 and stays static.
The correct behavior is used by the in-repo siblings:
`prediction/feature_extractor.py` passes `normalize_embeddings=True`,
and `memory/adapters/embedders.py` L2-normalizes before dot-product
similarity. This detector did neither.
## Fix
Pass `normalize_embeddings=True` to both `encode` calls (exemplars in
`__init__` and sentences in `detect`). Both sides of the dot product are
then unit vectors, so `np.dot` is a true cosine similarity in [-1, 1],
comparable to `semantic_threshold`.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cache/dynamic_detector.py`: add `normalize_embeddings=True`
to the exemplar encode (`__init__`) and the sentence encode (`detect`),
with comments explaining the cosine requirement.
- `tests/test_cache/test_dynamic_detector.py`: add
`TestSemanticDetectorNormalization` — a recording fake model asserts
both encode calls pass `normalize_embeddings=True` (via `object.__new__`
for `detect`, and a monkeypatched registry for `__init__`). No model
download needed.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [x] Unit tests pass (`uv run --extra dev pytest
tests/test_cache/test_dynamic_detector.py::TestSemanticDetectorNormalization
-q`)
- [x] Linting passes (`uvx ruff@0.15.17 check
headroom/cache/dynamic_detector.py
tests/test_cache/test_dynamic_detector.py headroom/memory/factory.py`)
- [x] Type checking passes (`uvx mypy==1.20.2
headroom/memory/factory.py`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/cache/dynamic_detector.py tests/test_cache/test_dynamic_detector.py
All checks passed!
$ python -m py_compile headroom/cache/dynamic_detector.py tests/test_cache/test_dynamic_detector.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`, numpy.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the scale mismatch
with a dependency-free numpy script (no sentence-transformers), and left
the full pytest to CI.
- Exact command / steps: built a MiniLM-dimension exemplar direction and
a sentence direction with a true cosine of ~0.1 (genuinely not dynamic),
gave them realistic un-normalized magnitudes (~9 and ~11), and computed
the old `np.dot` of the raw vectors versus the new `np.dot` of the
normalized vectors, against the 0.7 threshold.
- Observed result: old raw dot ~9.1 (far above 0.7 -> the unrelated
sentence is wrongly flagged dynamic); new cosine ~0.09 (below 0.7 ->
correctly kept static), and always within [-1, 1]. The new tests assert
both encode calls pass `normalize_embeddings=True`.
- Not tested: a real sentence-transformers model end to end; full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
Merged current `main` to pick up the repository-wide mypy cache-key
annotation fix, then verified the focused regression locally. the change
adds one keyword argument to two `encode` calls, verified by the numpy
proof and the new fake-model tests for CI. The fix brings this detector
in line with the two sibling call sites that already normalize.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Rod Boev
|
fb683e18d4
|
fix(litellm): forward chat_template_kwargs and other vendor top-level fields to OpenAI-compatible backends via extra_body (#2128) (#2163)
## Description
When Headroom forwards a `/v1/chat/completions` request to an
OpenAI-compatible backend (vLLM) via the LiteLLM backend, the
non-standard-but-OpenAI-compatible top-level field
`chat_template_kwargs` (e.g. `{"chat_template_kwargs":
{"enable_thinking": false}}`, used by vLLM to toggle Qwen3-family
"thinking" mode per request) never reaches the upstream model. A caller
that needs thinking *off* for a specific request has no way to disable
it through Headroom: the reasoning model spends its whole output-token
budget on hidden `<think>...</think>` content and returns
empty/truncated visible content.
Root cause: `LiteLLMBackend.send_openai_message`
(`headroom/backends/litellm.py:1101-1210`) and `stream_openai_message`
(`headroom/backends/litellm.py:1285+`) build the outgoing LiteLLM
`kwargs` from an explicit allowlist of recognized OpenAI params
(`headroom/backends/litellm.py:1129-1141`: `max_tokens`, `temperature`,
`top_p`, `stop`, `tools`, `tool_choice`, `response_format`, `seed`,
`n`). Only `model` and `messages` are copied unconditionally; anything
not in the list — including `chat_template_kwargs` — is dropped before
`acompletion(**kwargs)`. This is exactly the "litellm-backed forwarding
only passes fields it recognizes as standard OpenAI params" the reporter
suspected.
LiteLLM already forwards arbitrary vendor fields to an OpenAI-compatible
backend verbatim through its documented `extra_body` parameter — the
same mechanism vLLM users use directly. This change collects the
top-level body fields Headroom does not consume as standard params and
forwards them under `extra_body`, so `chat_template_kwargs` (and any
other vendor top-level field) reaches vLLM unchanged, on both the
buffered and streaming paths.
Closes #2128.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- In `LiteLLMBackend.send_openai_message` and `stream_openai_message`,
after populating the standard-param allowlist, collect top-level `body`
keys not consumed by Headroom/LiteLLM (everything outside the standard
allowlist plus `model`/`messages`/`stream`/`stream_options` and internal
markers) and forward them to the backend via LiteLLM's `extra_body`.
- `chat_template_kwargs` and other vendor-specific top-level fields now
reach the OpenAI-compatible upstream verbatim.
- Left the standard-param allowlist, region/profile config, API-key
forwarding, and the cache-stats usage block untouched; standard params
stay first-class LiteLLM kwargs (not moved into `extra_body`).
- Scoped to the OpenAI-format methods; the Anthropic-format
`send_message`/`stream_message` and the metadata-only
`OpenAICompatibleProvider` are unchanged.
## Testing
- [x] Unit tests pass (`uv run pytest
tests/test_litellm_openai_passthrough.py`)
- [x] Linting passes (`uv run ruff check .`)
- [ ] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality when applicable
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_litellm_openai_passthrough.py -q
.... [100%]
4 passed in 1.88s
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uv run`; `acompletion` mocked
(no live vLLM).
- Exact command / steps: `uv run pytest
tests/test_litellm_openai_passthrough.py -q`, which drives
`send_openai_message` and `stream_openai_message` with a body containing
`chat_template_kwargs: {"enable_thinking": false}` and inspects the
captured `acompletion` call kwargs.
- Observed result: on both the buffered and streaming paths
`acompletion` is now called with `extra_body={"chat_template_kwargs":
{"enable_thinking": false}}`; a standard-only body produces no
`extra_body`, and standard params (`max_tokens`, `temperature`, …)
remain first-class kwargs. Before the change the same body reaches
`acompletion` with `chat_template_kwargs` absent.
- Not tested: live vLLM run confirming Qwen3 thinking mode toggles off
end-to-end.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- This implements reporter option (a): pass unrecognized top-level body
fields through verbatim (via `extra_body`), which needs no new config
surface. Reporter option (b), an explicit allowlist/config on the
provider, is a deliberate non-goal here and can follow if maintainers
prefer it. The Anthropic-format `send_message`/`stream_message`
translation path and the direct-httpx passthrough path (which already
forwards the full body) are out of scope.
- Issue diagnosed by George Stephanis (`@georgestephanis`) with Claude
Code assistance, per the report's AI disclosure.
- `mypy` left unchecked: not part of the focused validation for this
change.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Rod Boev
|
cc072f0821
|
fix(cache-aligner): hash the frozen conversation prefix so Claude Code cache invalidation is detected (#2085) (#2161)
## Description Running Headroom as the API proxy for Claude Code, provider prompt-cache reuse collapsed: uncached input tokens went from ~755 to ~4.5M, cache-creation (write) tokens inflated ~4.4×, and one session burned ~36% of a weekly model cap. Roughly 96%-cached traffic became uncached+rewrite traffic — a net cost multiplier, not a saving. The `CacheAligner` owns the pipeline's "is the cacheable prefix byte-stable across requests?" signal (`stable_prefix_hash` / `prefix_changed` on `CachePrefixMetrics`). But `CacheAligner.apply()` computes that hash over **only `role == "system"` messages** (`headroom/transforms/cache_aligner.py:314-325`). Under Claude Code the system prompt is the stable part; what actually churns between requests is the conversation head — earlier user turns and tool-result blocks — the range Claude Code relies on for provider cache reads. `apply()` already receives the authoritative freeze boundary (`frozen_message_count`, produced by `PrefixCacheTracker`) and uses it to skip volatile-content detection, but the hash ignores it. So `prefix_changed` reports "prefix stable" even while the real cacheable prefix churns: the budget-burning regression is invisible and the byte-stability invariant the issue asks for is neither asserted nor enforced. This change scopes the aligner's stable-prefix hash to the actual frozen cacheable prefix (`messages[:frozen_message_count]` plus system messages), keyed on the authoritative `frozen_message_count`, so `prefix_changed` becomes a true cache-invalidation signal — and adds the replay regression test the issue specifies, locking the invariant "for `messages[0..k]` identical to the previous request, the emitted prefix bytes and hash are identical." `apply()` remains strictly detector-only and byte-equal; no rewrite is introduced. Closes #2085. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scoped `CacheAligner.apply()`'s `stable_prefix_hash` to the frozen cacheable prefix: the byte content of `result_messages[:frozen_message_count]` (the frozen conversation head, in order) combined with the system messages, keyed on the authoritative `frozen_message_count` kwarg from `PrefixCacheTracker`. - `prefix_changed` now reflects churn in the true provider-cacheable prefix (a changed tool-result block that leaves the system prompt untouched is now detected), so Claude Code cache invalidation is observable via the existing `CachePrefixMetrics` and the `stable_prefix_hash:<hash>` marker. - Preserved first-turn behavior: when `frozen_message_count == 0` the hash falls back to the current system-only scope, so the first request in a session is byte-for-byte unchanged. - Kept `apply()` detector-only (deep copy, never mutates messages) and left the `should_apply` skip gate, volatile-content warning, token counts, and `TransformResult` shape unchanged. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_cache_aligner_prefix_stability.py`) - [x] Linting passes (`uv run ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cache_aligner_prefix_stability.py -q ..... [100%] 5 passed in 0.40s ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uv run`; no live provider. - Exact command / steps: `uv run pytest tests/test_cache_aligner_prefix_stability.py -q`, which replays consecutive `apply()` calls in the Claude Code shape (stable system prompt + accumulated tool-result prefix) with `frozen_message_count > 0`. - Observed result: when a frozen tool-result block changes between requests while the system prompt is byte-identical, `prefix_changed` is now `True` and `stable_prefix_hash` differs; when the frozen prefix + system are identical, `prefix_changed` is `False`; when only the live/unfrozen tail changes, `prefix_changed` stays `False`; `apply()` output stays byte-equal to input. Before the change the same frozen-prefix churn reports `prefix_changed=False` because the hash covers only the system prompt. - Not tested: live Anthropic prompt-cache accounting over a full Claude Code session. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Scope: this slice corrects and locks the byte-stability invariant **at the CacheAligner boundary** — the exact acceptance criterion in the issue (a cache-preservation invariant over identical prefixes plus a replay regression test). It is distinct from, and does not touch, the upstream sources of prefix churn (ContentRouter per-block verdict flaps under `min_ratio` drift, #1619), the `headroom stats` cache-delta surfacing (#960), or parallel-subagent stream misclassification (#1949); those remain separate follow-ups. `PrefixCacheTracker`'s independent forwarded-prefix byte check (`headroom/cache/prefix_tracker.py`) is unchanged. - `mypy` left unchecked: not part of the focused validation for this change. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Rod Boev
|
e6df6ea470
|
docs: qualify CCR auto-resolution support for Gemini (#2044)
## Description Headroom's CCR docs describe automatic response handling as universal, but the current code only wires that continuation path for Anthropic and OpenAI-compatible handlers. This updates the docs to describe the real Gemini behavior today, including the native Gemini gap and the reported `MALFORMED_FUNCTION_CALL` risk on Gemini OpenAI-compatible round-2 continuations. Refs #2041 ## Type of Change - [x] Documentation update - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Narrow CCR response-handler claims to the providers that currently implement them. - Add a Gemini-specific note covering native-handler limits and the reported round-2 continuation failure. ## Testing - [x] Unit tests pass - [ ] Linting passes - [ ] Type checking passes - [ ] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run --no-sync pytest tests/test_ccr_response_handler.py tests/test_ccr_response_handler_extra.py -q ============================= test session starts ============================= platform win32 -- Python 3.12.13, pytest-9.0.3, pluggy-1.6.0 rootdir: D:\Repos\headroom-pr-2041-gemini-ccr-docs configfile: pyproject.toml plugins: anyio-4.12.1, langsmith-0.9.3, asyncio-1.3.0, cov-7.0.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 42 items tests\test_ccr_response_handler.py ............................... [ 73%] tests\test_ccr_response_handler_extra.py ........... [100%] ============================= 42 passed in 0.91s ============================== ``` ## Real Behavior Proof - Environment: Windows, Python 3.12.13, docs-only change with no live Gemini provider call - Exact command / steps: `uv run --no-sync pytest tests/test_ccr_response_handler.py tests/test_ccr_response_handler_extra.py -q` - Observed result: All 42 CCR response-handler tests pass, confirming the existing Anthropic/OpenAI-compatible continuation behavior is unchanged by the docs update - Not tested: a live Gemini round-2 continuation request ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
nangsontay
|
5d9bbbeea0
|
fix(codex): rewrite config.toml properly so Codex will route through … (#2102)
## Description `headroom install apply --providers manual --target codex --scope provider` silently failed to route Codex through the proxy whenever `~/.codex/config.toml` already had a `[table]` section (e.g. `[features]`, `[mcp_servers.*]`). `apply_provider_scope` appended the managed `model_provider = "headroom"` block after the last existing table, so TOML scoped the bare key into that table instead of the document root — Codex silently ignored it and kept routing through its default provider. The same code path never overrode a pre-existing top-level `model_provider` assignment either, so a user's `model_provider = "openai"` kept winning even when Headroom's block was appended elsewhere in the file. This mirrors a bug already fixed in the `headroom init` path (`_ensure_codex_provider`, #260) that was never ported to the persistent-install path. Closes: reported via user session (no tracked issue number yet). ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) ## Changes Made - **`headroom/providers/codex/install.py`**: Added `_insert_block_at_root()`, which walks the document line-by-line and inserts the managed marker block immediately above the first `[table]`/`[[array-of-tables]]` header, falling back to end-of-file append only when no table exists. Mirrors the root-insertion logic already used by `cli/init.py:_ensure_codex_provider`. - Added `_ANY_MODEL_PROVIDER` / `_ANY_OPENAI_BASE_URL` patterns (match any value, not just `"headroom"`) so `apply_provider_scope` strips **any** prior top-level `model_provider` / `openai_base_url` assignment before re-inserting the managed block — the managed keys now override the user's config outright instead of losing to it. - `apply_provider_scope` merge order is now: strip old managed block → strip prior top-level assignments → insert fresh block at document root. ## Testing - [x] **New regression test**: `test_apply_codex_provider_scope_lands_model_provider_at_root` (`tests/test_install/test_providers.py`) — asserts `model_provider = "headroom"` lands before `[features]`, overrides a prior `"openai"` value, and the user's own table content survives. - [x] **Existing tests**: `tests/test_install/test_providers.py` — 42/42 pass (includes prior codex apply/revert/replace/orphan-cleanup coverage). - [x] **Adversarial (ad-hoc, not committed)**: 6-case TOML round-trip proof — parses output with `tomllib` (not substring matching) across: prior provider before a table, no prior provider, empty file, scalars-only (no tables), CRLF line endings, multiple tables. All 6 pass after the fix; first pass caught a false failure from a stale globally pip-installed `headroom` copy shadowing the repo source when tests run outside the project directory — re-verified from inside the repo to confirm the fix itself is correct. - [x] **Lint**: `ruff check` and `ruff format --check` pass on both changed files. ```text $ uv run pytest tests/test_install/test_providers.py -q 42 passed in 0.21s $ uv run --with ruff ruff check headroom/providers/codex/install.py tests/test_install/test_providers.py All checks passed! $ uv run --with ruff ruff format --check headroom/providers/codex/install.py tests/test_install/test_providers.py 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS 26.4.1 (arm64), Python 3.13.14, headroom branch `patch/install-codex` - Exact command / steps: constructed a temp `config.toml` with `[features]\nweb_search = true` (no existing Headroom block), invoked `apply_provider_scope(manifest)` against it with `codex_config_path` patched to the temp file, then parsed the result with `tomllib.loads()`. - Observed result: before the fix, `tomllib.loads(result)["model_provider"]` raised `KeyError` — the key was nested inside `[features]` due to end-of-file append. After the fix, `parsed["model_provider"] == "headroom"` and `parsed["features"]["web_search"] is True` — both the managed key and the user's table are present and correctly scoped. Revert removes `model_provider` and preserves the user's table. - Tested local build and behavior is correct as expected of this patch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review |
||
|
Abhay Singh
|
5fb449e90b
|
fix(subscription): keep efficiency_pct from exceeding 100% (#2121)
## Description
`HeadroomContribution.efficiency_pct` can report values above 100%
because its numerator and denominator disagree about cache-read tokens.
```python
def total_saved(self) -> int:
return (self.tokens_saved_compression + self.cli_filtering_saved()
+ self.tokens_saved_cache_reads) # includes cache reads
def raw_without_headroom(self) -> int:
return (self.tokens_submitted + self.tokens_saved_compression
+ self.cli_filtering_saved()) # excludes cache reads
def efficiency_pct(self) -> float:
raw = self.raw_without_headroom()
if raw == 0:
return 0.0
return round(self.total_saved() / raw * 100, 1)
```
`tokens_saved_cache_reads` are input tokens that were *forwarded* to the
provider and served from the prefix cache at a discount, so they already
live inside `tokens_submitted` (the "raw input tokens actually
forwarded"). They are added to the numerator via `total_saved()` but
never to the denominator, so with `tokens_submitted=100` and
`tokens_saved_cache_reads=1000` the method returns `1000.0%`, which the
dashboard renders verbatim. An efficiency percentage should never exceed
100%.
## Fix
Use the existing sibling `compression_saved()` (compression + CLI
filtering, which already excludes cache reads) as the numerator. Then
`efficiency_pct = compression_saved / (tokens_submitted +
compression_saved)`, which is bounded by its own denominator and is the
meaningful quantity here: the fraction of the pre-Headroom input that
compression and CLI filtering actually removed. Cache reads are a
provider-side discount on forwarded tokens, not tokens Headroom removed,
so they don't belong in a removal-efficiency ratio. `total_saved()` is
left unchanged for its other callers (`to_dict`, etc.).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/subscription/models.py`: `efficiency_pct` now uses
`compression_saved()` instead of `total_saved()` as the numerator, with
a comment explaining the cache-read inconsistency.
- `tests/test_subscription_contribution.py`: new tests — cache reads
can't push efficiency over 100%, the ratio equals the
compression-removal fraction, and empty input yields 0.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [x] Unit tests pass (`uv run --extra dev pytest
tests/test_subscription_contribution.py -q`)
- [x] Linting passes (`uvx ruff@0.15.17 check
headroom/subscription/models.py tests/test_subscription_contribution.py
headroom/memory/factory.py`)
- [x] Type checking passes (`uvx mypy==1.20.2
headroom/memory/factory.py`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/subscription/models.py tests/test_subscription_contribution.py
All checks passed!
$ python -m py_compile headroom/subscription/models.py tests/test_subscription_contribution.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the ratio with a
dependency-free script that replicates the three methods, and left the
full pytest to CI.
- Exact command / steps: computed `efficiency_pct` under the old
numerator (`total_saved`) and the new numerator (`compression_saved`)
for `tokens_submitted=100, tokens_saved_cache_reads=1000` and for a real
compression case (`submitted=1000, compression=400, cache_reads=300`).
- Observed result: old returns `1000.0%` for the cache-read case
(impossible) and the new returns `0.0%`; for the compression case old
returns `50.0%` (inflated by cache reads) and new returns `28.6%` (= 400
/ 1400), always `<= 100%`. The new tests assert these.
- Not tested: the dashboard render path end to end; full local `pytest`
deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
Merged current `main` to pick up the repository-wide mypy cache-key
annotation fix and current dependency security floors, then verified the
focused regression locally. the change swaps one method call in a pure
dataclass method, verified by the standalone proof and the new tests for
CI.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Abhay Singh
|
bc24e258b1
|
fix(mcp/claude): don't clobber an unparseable Claude config on register (#1660)
## Description
When the `claude` CLI isn't on PATH (or its `mcp add` fails),
`ClaudeRegistrar`
falls back to `_register_via_file`, which does a full-file
read-modify-write of
`~/.claude/.claude.json`:
```python
config = _read_json(target) # returns {} on JSONDecodeError
servers = config.setdefault("mcpServers", {})
servers[spec.name] = _spec_to_entry(spec)
_write_json(target, config) # overwrites the ENTIRE file
```
`_read_json` returns `{}` for a file that exists but doesn't parse. So
if
`~/.claude/.claude.json` is momentarily corrupt or hand-edited (a
trailing
comma, a crash mid-write), the register path silently rewrites it as
just
`{"mcpServers": {"headroom": {...}}}` — **destroying every other key
Claude Code
keeps there**: `projects`, `oauthAccount`, session history, etc. There's
no
backup. The existing `test_get_server_robust_to_bad_json` only covers
the *read*
path; the destructive *write* path was untested.
Closes: no issue filed — found while auditing the MCP registry
config-write paths.
## Fix
Keep `_read_json` (returning `{}`) for the read-only callers
(`get_server`,
removal), where it's harmless. Add `_read_json_for_write` for the
rewrite path:
it returns `{}` only when the file is **absent or empty** (safe to start
fresh)
and raises `_MalformedConfigError` when the file is present but not a
JSON
object. `_register_via_file` catches it and returns a `FAILED` result
with an
actionable message instead of overwriting.
Result: absent/empty file → registers fresh (unchanged); valid file →
merges,
all other keys preserved (unchanged); present-but-invalid file → refuses
to
touch it and tells the user to fix or remove it.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/mcp_registry/claude.py`: add `_read_json_for_write` +
`_MalformedConfigError`; `_register_via_file` uses it and returns
`FAILED` (without writing) when the target is present-but-unparseable.
`_read_json` is unchanged for read-only callers.
- `tests/test_mcp_registry/test_claude_registrar.py`: regression tests —
register against malformed configs leaves the bytes untouched and
returns `FAILED`; register against a valid config still merges and
preserves unrelated keys.
- `CHANGELOG.md`: Bug Fixes entry under Unreleased.
## Testing
- [x] New tests added for the fixed behavior
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`)
- [ ] Full `pytest` deferred to CI (local-OOM reason under Real Behavior
Proof).
```text
$ uv run ruff check headroom/mcp_registry/claude.py tests/test_mcp_registry/test_claude_registrar.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, headroom built from this
branch. Importing `headroom` loads the torch/transformers stack; a full
`pytest` gets OOM-killed on this box, so I verified the write-path logic
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: replicated `_read_json_for_write` and the
`_register_via_file` read-modify-write flow in a standalone script (only
stdlib, no `headroom` import) against real temp files, and exercised:
absent, empty, four malformed variants (`not json`, `{`, `{"projects":
}`, `[]`), and a valid config carrying `projects`/`oauthAccount`.
- Observed result: absent/empty register fresh; every malformed variant
returns FAILED and the original bytes on disk are byte-for-byte
unchanged (no clobber); a valid config merges in `headroom` while
`projects`/`oauthAccount` survive:
```text
OK: absent -> fresh register
OK: empty -> fresh register
OK: malformed -> FAILED, original bytes preserved (no clobber)
OK: valid config -> merged, unrelated keys preserved
MCP CONFIG-WRITE LOGIC VERIFIED
```
- Not tested: driving the real `claude` CLI-absent path end-to-end on a
live `~/.claude/.claude.json` (didn't want to touch a real Claude
install); the file-fallback logic is exercised directly by the
regression tests. Full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No new dependencies. `headroom/mcp_registry/opencode.py` has the same
read-`{}`-then-clobber shape on its write path (an OpenCode
`opencode.json` with comments/JSONC would be wiped) — I scoped this PR
to the Claude registrar to keep it focused and because OpenCode config
handling is being touched in other open PRs; happy to send a follow-up
for opencode with the same guard if useful.
---------
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
JD Davis
|
f00833654f
|
fix(proxy): satisfy rustfmt import ordering (#2158)
## Summary Fixes the Rust workflow failure from https://github.com/headroomlabs-ai/headroom/actions/runs/29294598252/job/86965269576 by applying rustfmt's import ordering in `crates/headroom-proxy/src/proxy.rs`. ## Testing ```text cargo fmt --all -- --check # passed git diff --check # no output ``` |
