mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
2386 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tejas Chopra
|
5383c6bf2f
|
fix(release): sync generated version metadata on the release branch (#2659)
## Description
The 0.33.0 release PR (#2339) has sat in `changes-requested` since
2026-07-17. Root cause: **release-please only rewrites `pyproject.toml`
and its configured `extra-files`**, but other tracked files also carry
the version — and `server.json` is asserted byte-for-byte against
`render_server_json()`, which derives its version from `pyproject.toml`.
So the bump alone fails
`tests/test_mcp_registry/test_server_json.py::test_root_server_json_matches_builder`
(the `test (2)` shard) on every regenerated release PR.
Nothing in the repo regenerated `server.json` at all, so it fell behind
every release.
Unblocks #2339.
### Why the release *build* passes but the release PR does not
`release.yml` already runs `scripts/version-sync.py` immediately before
its own `verify-versions.py` gate (lines 145 and 278). That is why
`build` and `build-wheels` are green on #2339 despite the drift — it
syncs in the workspace, uncommitted. The regular CI test job does
**not** sync, so the fix has to be committed to the branch.
This also explains why reviewers kept seeing `verify-versions.py` fail
locally while CI's build jobs passed: the verifier is never run
un-synced inside `release.yml`.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **`scripts/version-sync.py`**: also write `server.json`. It was the
one version-carrying file with no writer anywhere. Values are rewritten
in place so key order and formatting keep matching the builder's
byte-for-byte output (verified: the file is pure ASCII and round-trips
exactly through `json.dumps(..., indent=2) + "\n"`).
- **`.github/workflows/release-metadata-sync.yml`** (new): on a push to
`release-please--branches--**`, run version-sync → gate on
verify-versions → commit if changed.
- **Keyed off the branch push** because release-please force-regenerates
that branch on every merge to main. That is precisely what wiped the
hand-pushed metadata fixes on #2339 (`2a86c8ff`, `d5ea4dc5`) — a push
trigger re-heals after every regeneration instead of being lost.
- **Uses the same PAT as `release-please.yml`**: a `GITHUB_TOKEN` push
does not trigger workflows, so the release PR's checks would never
re-run against the synced commit and would stay red.
- **Idempotent**: the self-triggered rerun finds no diff and exits
before pushing, so the loop terminates after one no-op run.
- **Corrected pre-existing drift on `main`**: the agent-hooks plugin
manifests, both marketplace manifests, and `.releasemetadata` were
stranded at **0.31.0** — never bumped for 0.32.0 either.
`verify-versions.py` now passes on `main`.
### Why not more `extra-files` entries
That would need ~13 jsonpath entries restating what `version-sync.py`
already knows, and a jsonpath that fails to match **fails silently** —
the same class of failure this PR removes, discoverable only after a
real release PR regenerates. There is also no precedent for nested
jsonpath (`$.packages[0].version`, `$.metadata.version`) in the config
today; both existing entries are plain `$.version`. Running the script
keeps one source of truth, and files added to it later are covered with
no change here.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — no `headroom/` sources
touched
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m pytest scripts/tests/ tests/test_release_workflows.py tests/test_mcp_registry/ -q
207 passed in 2.69s
$ ruff check scripts/version-sync.py scripts/tests/test_version_sync.py tests/test_release_workflows.py
All checks passed!
$ ruff format --check <same>
3 files already formatted
$ actionlint .github/workflows/release-metadata-sync.yml
(clean)
```
New tests:
- `test_server_json_version_is_synchronized` — version-sync moves both
`server.json` version fields and preserves the other keys.
- `test_release_metadata_sync_runs_on_release_please_branch` — asserts
the trigger, the sync→verify→commit ordering, the no-op guard, and the
PAT.
- `test_version_sync_covers_every_file_the_verifier_gates` — guards
`version-sync.py` and `verify-versions.py` against drifting apart again,
which is the root cause here.
## Real Behavior Proof
- **Environment:** macOS (Darwin arm64), Python 3.12, repo venv.
- **Exact command / steps:** reproduced the CI failure locally by
simulating release-please's partial bump, then applying the fix.
**Reproducing the exact `test (2)` failure** — set `pyproject` to 0.33.0
while `server.json` stays at 0.32.0, as release-please leaves it:
```text
$ python -m pytest tests/test_mcp_registry/test_server_json.py -q
FAILED tests/test_mcp_registry/test_server_json.py::test_root_server_json_matches_builder
1 failed, 3 passed
```
**After `version-sync.py`:**
```text
$ python scripts/version-sync.py && python -m pytest tests/test_mcp_registry/test_server_json.py -q
4 passed
```
**Both gates green on a simulated 0.33.0 bump:**
```text
$ python scripts/version-sync.py --version 0.33.0
Version synchronized to 0.33.0
$ python scripts/verify-versions.py
All versions aligned at 0.33.0
$ python -m pytest tests/test_mcp_registry/test_server_json.py -q
4 passed
```
**Idempotency** (the property the workflow's loop-termination relies
on): re-running against an already-synced tree leaves `pyproject.toml`,
`server.json`, `openclaw`, and `sdk/typescript` untouched.
- **Not tested:** the workflow has not executed on a real release-please
branch regeneration — that can only be exercised once this is on `main`
and release-please next updates #2339. The PAT push path and the
self-trigger no-op are reasoned from `release-please.yml`'s existing
token comment and from local idempotency, not observed in CI.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
**Context on the v0.32.0 release failure, since it is easy to misread as
"images never build".** Every artifact built for v0.32.0 — all 5 wheel
platforms including Windows, all 16 Docker builds + 8 manifests +
`promote-latest`, npm, and GitHub Packages. Only `publish-pypi` failed
(PyPI attestations, already fixed by `
|
||
|
Robert Schorr
|
b3f016b866
|
fix(mcp): pin mcp dependency to <2.0.0 to prevent server startup crash (#2642)
## Description
The MCP Python SDK recently released version `2.0.0`, which introduced
breaking changes to the high-level server interface (removing
`.list_tools()` and `.call_tool()` decorators on `mcp.server.Server`).
Because `headroom-ai` specified `mcp>=1.28.1` without an upper bound,
installing or upgrading `headroom-ai` pulled in `mcp 2.0.0`. When
`headroom mcp serve` was started by an MCP client (such as OpenCode or
Claude Code), the server crashed immediately on startup with
`AttributeError: 'Server' object has no attribute 'list_tools'`,
resulting in the connection closing error (`headroom MCP error -32000:
Connection closed`).
This PR pins the `mcp` dependency to `<2.0.0` (`mcp>=1.28.1,<2.0.0`) in
`pyproject.toml` so compatible 1.x SDK releases (e.g. `1.29.0`) are
used.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Pinned `mcp` dependency to `"mcp>=1.28.1,<2.0.0"` under both `proxy`
dependencies and the `mcp` extra in `pyproject.toml`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ ruff check pyproject.toml headroom/ccr/mcp_server.py
All checks passed!
$ pytest tests/test_ccr_mcp_server.py tests/test_cli/test_mcp.py tests/test_cli/test_mcp_status.py
============================== 45 passed in 1.03s ==============================
```
## Real Behavior Proof
- Environment: macOS (Darwin arm64), Python 3.14.5, uv
- Exact command / steps: Executed uv sync --all-extras and sent stdio
JSON-RPC initialize and tools/list requests to .venv/bin/headroom mcp
serve.
- Observed result: Server initializes cleanly and returns JSON-RPC
response
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2024-11-05","capabilities":{"experimental":{},"tools":{"listChanged":false}},"serverInfo":{"name":"headroom","version":"1.29.0"}}}
with no startup AttributeError or closed pipe errors.
- Not tested: N/A
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did not edit CHANGELOG.md — it is generated by release-please
from my Conventional Commit PR title (a CI guard enforces this)
## Screenshots (if applicable)
N/A
## Additional Notes
Capping mcp<2.0.0 ensures stability with current headroom releases while
a future update can adopt MCP SDK 2.x interface changes if desired.
|
||
|
JD Davis
|
2dc7e4ab27
|
test: add fluent Headroom harness (#2650)
## Description Adds `headroom.testing`, a fluent, contractual test harness for building Headroom scenarios and suites that can be simulated locally, orchestrated, deployed through the proxy, and handed off to `headroom-bench` / `agent-evals` with bench-native manifests. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom.testing.Headroom` fluent scenario builder with provider/platform/configuration facets such as `WithBedrock`, `OnAppleSilicon`, `Configure`, `WithCompression`, `WithCCR`, `WithCache`, `WithPrefixFreeze`, `WithReadMaturation`, and `WithMemory`. - Add contractual coverage over the current `HeadroomConfig` and `ProxyConfig` dataclass surfaces, including full JSON-ready proxy deployment payloads. - Add no-key local simulations, scenario/suite orchestration, guarantee evaluation, deployment plans, and a local proxy lifecycle context manager. - Add `headroom-bench` handoff artifacts, including `agent_evals.models.RunManifest`-compatible JSON without taking a runtime dependency on `agent-evals`. - Add demonstration tests for providers, feature facets, manifests, suites, guarantees, deployment payloads, and the no-key simulation path. - Fix unversioned OTEL meter lookup typing so `mypy headroom` remains green on current `main`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text python -m ruff check . All checks passed! python -m mypy headroom headroom\proxy\server.py:1680: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] headroom\proxy\server.py:1691: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] Success: no issues found in 512 source files python -m pytest tests/test_cli/test_subprocess_utf8_encoding.py tests/test_testing_harness.py -q 24 passed, 1 warning in 4.68s ``` ## Real Behavior Proof - Environment: Windows, Python 3.13.13, branch `feat/headroom-test-harness` rebased on `headroomlabs-ai/main`. - Exact command / steps: built a `Headroom.WithOpenAI().WithCompression(mode="cache", kompress=False).Build()` scenario and entered `scenario.deploy_local(port=19192, timeout_s=20)`. - Observed result: proxy launched, `/readyz` succeeded, handle returned `http://127.0.0.1:19192`, `OPENAI_BASE_URL=http://127.0.0.1:19192/v1`, and context-manager teardown completed. - Exact command / steps: emitted `scenario.agent_evals_manifest(...).to_dict()` and validated it with the current cloned `headroom-bench` `agent_evals.models.RunManifest` pydantic model. - Observed result: validation succeeded with arms `a0_direct`, `a1_passthrough`, and `b_headroom` for provider `openai`. - Not tested: upstream-provider API calls requiring real OpenAI/Anthropic/Bedrock keys; phase-1 validation intentionally stays no-key/local. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A. ## Additional Notes The pytest warning shown above is the existing OpenAI pricing-data staleness warning from cost estimation. The harness does not call upstream providers during local simulation. |
||
|
Tejas Chopra
|
e0d2cd0c5a
|
fix(cache): preserve cache_control ttl when re-anchoring a breakpoint (#2651)
## Description
`normalize_message_cache_control` deliberately reuses the client's
marker verbatim so an explicit `cache_control.ttl` (e.g. `"1h"`)
survives breakpoint consolidation instead of silently downgrading to the
5-minute default (#2375).
Two other sites also strip a breakpoint and re-place it, and both
hardcoded a bare `{"type": "ephemeral"}` — undoing that guarantee.
A downgrade is invisible: the request still succeeds, and the cost shows
up later as a full prefix re-write on every idle gap past 5 minutes.
Measured over 10,409 local Claude Code API requests, cache writes are
**6.1% of raw input tokens but 44.8% of the price-weighted input bill**
(5m write 1.25x vs read 0.1x), and **89% of those write tokens are
re-writes of content cached one request earlier**. Honoring a 1h TTL
when the client asks for it is the cheapest thing we can do about that.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/transforms/read_maturation.py` — `relocate_cache_breakpoint`
now carries the stripped marker forward when re-anchoring before the
held-Read region. This is the one that mattered most: it runs **after**
`normalize_message_cache_control` in the Anthropic handler
(`anthropic.py:1747` vs `:1642`), so it had the final say — a 1h client
with read maturation enabled was being downgraded to 5m.
- `headroom/proxy/helpers.py` — `inject_tool_search_deferral` keeps the
dropped marker when moving the tools-array breakpoint off a now-deferred
tool onto the last resident real tool.
- Both fall back to a bare ephemeral only when the client sent no ttl,
and neither invents a breakpoint where none existed.
- `headroom/transforms/compression_policy.py` — comment only. Notes that
`CACHE_WRITE_MULTIPLIER` is hardcoded to the 5m tier (1.25x), so a
client already on 1h caching (2.0x) has its mutations gated with a ~40%
under-stated write penalty. Harmless while the net-cost gate stays
default-off (`HEADROOM_NET_COST_POLICY`); names the plumbing needed if
it is ever enabled.
Both changed code paths sit behind off-by-default flags
(`HEADROOM_READ_MATURATION`, `HEADROOM_TOOL_SEARCH`), so this is a
latent-bug fix with **no default behavior change**.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_cache_ttl_preserved.py tests/test_read_maturation.py \
tests/test_read_maturation_handler_nobust.py tests/test_cache_control_move_bust.py -q
tests/test_cache_ttl_preserved.py ..... [ 12%]
tests/test_read_maturation.py ...................... [ 67%]
tests/test_read_maturation_handler_nobust.py ... [ 75%]
tests/test_cache_control_move_bust.py .......... [100%]
============================= 40 passed in 15.52s ==============================
$ ruff check headroom/ tests/ --exclude headroom/dashboard/templates
All checks passed!
$ mypy headroom
Success: no issues found in 509 source files
```
Broader regression sweep over every cache/breakpoint-adjacent suite:
```text
$ python -m pytest tests/ -q -k "read_maturation or tool_search or cache_control or prefix_tracker or ttl_preserved"
204 passed, 10145 deselected in 59.95s
```
## Real Behavior Proof
- **Environment:** macOS 25.4.0 (arm64), Python 3.12.6, pytest 9.0.2,
branched from `main` at
|
||
|
Zhenjia ZHOU
|
e825588bfb
|
fix(ccr): sliding idle-window TTL with max-lifetime ceiling in the Rust core backends (#2604) (#2631)
## Description
Rust-core counterpart of the CCR mid-session expiry fix. #2604 (and its
duplicate #2616) report that the 30-minute wall-clock TTL kills entries
in the middle of a normal multi-agent burst: the clock starts at
compression time and never refreshes, so an entry the session keeps
touching still dies.
#2607 fixes this on the Python side by turning the TTL into an idle
window that restarts on every successful retrieval, bounded by an
absolute max lifetime (8x the idle TTL) — but it explicitly notes the
caveat that the Rust core still measures TTL from insertion. This PR
closes that gap: all three Rust CCR backends (`InMemoryCcrStore`,
`SqliteCcrStore`, `RedisCcrStore`) now use the same sliding idle-window
+ max-lifetime-ceiling semantics as the Python `CompressionStore`.
Scoped to the Rust core only; it deliberately does not touch
`DEFAULT_TTL`'s value (1800), which #2607 bumps to 3600 — happy to
rebase in lockstep whichever lands first.
Refs #2604, #2616. Complements #2607.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `crates/headroom-core/src/ccr/mod.rs`:
`DEFAULT_MAX_LIFETIME_MULTIPLIER = 8` + `max_lifetime_for()` helper;
documents the idle-window semantics.
- `in_memory.rs`: entries track `last_accessed`; a hit refreshes it
under the shard write lock (`get_mut`), expiry checks idle window OR max
lifetime, and the existing `remove_if` TOCTOU protection now uses the
same predicate. New `with_capacity_and_ttls` constructor for independent
control of window and ceiling.
- `sqlite.rs`: new `last_accessed` column (legacy DBs migrated in place
via `ALTER TABLE`, backfilled from `created_at` so old rows keep their
original expiry baseline); lazy purge and the lookup honour both bounds;
a hit touches the row under the same connection mutex as the read. New
`open_with_ttls` constructor.
- `redis.rs`: a hit re-arms the key's expiry, capped by a companion
`{prefix}:{hash}:born` key whose remaining TTL marks the absolute
ceiling; entries written by pre-sliding builds (no born key) are
backfilled rather than dropped.
- `tests/ccr_backends.rs`: 6 new tests — sliding-window survival and
max-lifetime cap for in-memory and SQLite, legacy-schema migration, and
a gated Redis sliding test.
No public API is broken: existing constructors keep their signatures and
derive the ceiling as 8x the idle TTL.
## Testing
- [x] Unit tests pass (`cargo test -p headroom-core`)
- [x] Linting passes (`cargo clippy -p headroom-core --all-features`,
`cargo fmt --check`)
- [ ] Type checking passes (`mypy headroom`) — N/A, no Python files
touched
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ cargo test -p headroom-core --test ccr_backends
test result: ok. 12 passed; 0 failed; 0 ignored (8.12s)
$ cargo test -p headroom-core ccr # all ccr-named tests across suites
38 passed, 949 filtered out (13 suites)
$ cargo test -p headroom-core --test ccr_roundtrip --test live_zone_ccr
18 passed (2 suites)
$ cargo check -p headroom-core --features redis # cfg-gated backend compiles
Finished `dev` profile in 25.09s
$ cargo clippy -p headroom-core --all-features
No issues found
```
## Real Behavior Proof
- Environment: macOS (Darwin 25.5), local checkout at upstream `main`
(
|
||
|
Zhenjia ZHOU
|
e86c6390ce
|
fix(rust): port CJK-aware relevance-query matching to CodeCompressor (#2634)
## Description The Rust port of `CodeCompressor` (#1154, parity-only) did not carry over the CJK-aware relevance-query matching from `headroom/transforms/code_compressor.py` (`_CONTEXT_DELIMS` / `_CJK_CHARS` / `_query_context_tokens()` / `_symbol_in_context()`, lines 2353-2387, called from lines 987/1009): - Rust tokenized the context with an ASCII-only delimiter class `[\s,;:.()\[\]{}"']+`, so a CJK query (no spaces, CJK punctuation) collapses into a single blob and never isolates an ASCII symbol name. - The substring-fallback guard `chars().count() > 3` had no CJK relaxation, so a short ASCII name glued to CJK text (e.g. `run` in `修复run函数的报错`, `db` in `请保留db相关的逻辑`) could never receive the +3.0 context boost — while Python does boost it. Same `(code, context)` input, different `symbol_scores`. This PR ports the two Python helpers with identical semantics: - `query_context_tokens()` — delimiter class extended with the CJK/full-width punctuation and ideographic space from Python's `_CONTEXT_DELIMS`; returns `(words, lowered, has_cjk)` with CJK detection over U+3000-U+9FFF, U+AC00-U+D7AF, U+FF00-U+FFEF (Python's `_CJK_CHARS`). - `symbol_in_context()` — exact token match, plus the substring fallback gated by `> 3` **characters** (Python `len()`, not bytes), relaxed when the query contains CJK. The call site in `analyze_symbols` now uses these helpers; no other behavior changed. Pure-ASCII query behavior is identical to before (exact token match, `>3`-gated substring fallback), which the tests pin down. Closes #2630 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `crates/headroom-core/src/transforms/code_compressor.rs`: extract `query_context_tokens()` / `symbol_in_context()` free functions mirroring the Python helpers (CJK/full-width delimiter class, CJK detection, CJK-relaxed `>3`-character guard); replace the inline ASCII-only tokenization + guard in `analyze_symbols` with calls to them. - Unit tests mirroring `tests/test_transforms/test_code_compressor_cjk.py` case-for-case, plus a character-vs-byte guard test and an end-to-end `compress_with` test asserting `symbol_scores`. ## Testing - [x] Unit tests pass (`cargo test -p headroom-core` — Rust-only change; Python untouched) - [x] Linting passes (`cargo fmt --check`, `cargo clippy -p headroom-core --all-targets` — no new warnings) - [ ] Type checking passes (`mypy headroom`) — N/A, no Python changes - [x] New tests added for new functionality - [x] Manual testing performed New tests (all in `code_compressor.rs` `mod tests`): - `cjk_query_isolates_wrapped_ascii_symbol` — full-width parens isolate `parse_config` - `cjk_query_matches_short_ascii_name_glued_to_cjk` — `db` (len 2) glued to CJK matches via the relaxed guard - `english_short_name_substring_still_gated` — `db` vs "keep the database helper" must NOT match (ASCII guard unchanged) - `english_exact_token_match_unchanged`, `english_long_name_substring_fallback_unchanged`, `empty_context_matches_nothing` - `guard_counts_chars_not_bytes` — the guard is a character count, matching Python `len()` - `cjk_context_boosts_named_symbol_end_to_end` — full `compress_with` run asserting `symbol_scores` (red on main, green here — see proof) ### Test Output ```text $ cargo test -p headroom-core --lib -- code_compressor::tests test transforms::code_compressor::tests::empty_and_short_passthrough ... ok test transforms::code_compressor::tests::empty_context_matches_nothing ... ok test transforms::code_compressor::tests::estimate_tokens_uses_chars_div_4_min_1 ... ok test transforms::code_compressor::tests::py_round3_matches_cpython ... ok test transforms::code_compressor::tests::py_round_int_is_half_to_even ... ok test transforms::code_compressor::tests::cjk_query_isolates_wrapped_ascii_symbol ... ok test transforms::code_compressor::tests::english_short_name_substring_still_gated ... ok test transforms::code_compressor::tests::cjk_query_matches_short_ascii_name_glued_to_cjk ... ok test transforms::code_compressor::tests::english_exact_token_match_unchanged ... ok test transforms::code_compressor::tests::english_long_name_substring_fallback_unchanged ... ok test transforms::code_compressor::tests::guard_counts_chars_not_bytes ... ok test transforms::code_compressor::tests::cjk_context_boosts_named_symbol_end_to_end ... ok test transforms::code_compressor::tests::detect_language_basic ... ok test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 899 filtered out; finished in 0.05s $ cargo test -p headroom-core # per-binary summaries lib .......................... ok. 911 passed; 0 failed; 1 ignored auth_mode .................... ok. 16 passed; 0 failed cache_control ................ ok. 14 passed; 0 failed ccr_backends ................. ok. 7 passed; 0 failed ccr_roundtrip ................ ok. 15 passed; 0 failed code_compressor_parity ....... ok. 1 passed; 0 failed (recorded byte-parity fixtures) live_zone_ccr ................ ok. 3 passed; 0 failed live_zone_dispatch ........... ok. 6 passed; 0 failed live_zone_thresholds ......... ok. 2 passed; 0 failed live_zone_token_validation ... ok. 3 passed; 0 failed recommendations_loader ....... ok. 4 passed; 0 failed tokenizer_proptest ........... ok. 5 passed; 0 failed doc-tests .................... ok. 1 passed; 0 failed; 2 ignored $ cargo fmt --check # clean $ cargo clippy -p headroom-core --all-targets # no new warnings ``` ## Real Behavior Proof - Environment: macOS (Darwin 25.5.0), repo-pinned Rust toolchain (`rust-toolchain.toml`), branch based on current `main`. - Exact command / steps: the end-to-end test was written first and run against unmodified `main` (red), then after the fix (green). Input: Python source with two signal-symmetric functions `run` and `keep`; context `修复run函数的报错`. The Python reference gives `run` the boost (`_symbol_in_context('run', ...) == True`, `_symbol_in_context('keep', ...) == False`, verified against the live Python implementation), so expected normalized scores are `run = 1.0`, `keep = 0.0`. - Observed result: on unmodified main the end-to-end test fails (`left: 0.5, right: 1.0` — the CJK query `修复run函数的报错` gives `run` no boost, both symbols collapse to 0.5, while Python scores `run=1.0, keep=0.0`); on this branch all 8 new tests pass and the same query boosts `run` to 1.0, matching Python. Full output: Before (unmodified `main` + new test only — Rust gives no boost, both symbols collapse to 0.5): ```text ---- transforms::code_compressor::tests::cjk_context_boosts_named_symbol_end_to_end stdout ---- thread '...cjk_context_boosts_named_symbol_end_to_end' panicked at crates/headroom-core/src/transforms/code_compressor.rs:1890:9: assertion `left == right` failed: run must get the context boost left: 0.5 right: 1.0 test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 904 filtered out ``` After (this branch): the same test passes, including its ASCII control case (`fix the runner` must NOT boost `run` — scores stay 0.5/0.5), proving pure-ASCII behavior is unchanged. The recorded byte-parity fixture suite (`code_compressor_parity`) also still passes. - Not tested: real proxy traffic end-to-end (change is confined to the symbol-scoring context boost inside the Rust compressor; the Python implementation is the behavioral reference and is untouched). `kompress_parity` was not run locally — it is model-gated and my sandbox blocks the model fetch; it is unrelated to this change and CI covers its skip path. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (internal behavior fix) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes For background, the Python-side CJK handling comes from the merged CJK sweep work (#2220 and follow-ups); #1154 predates part of it, which is likely how the port missed it. Longer names wrapped in full-width punctuation happened to still match in Rust via the substring fallback, but the token set itself was wrong; this PR restores exact-token semantics for those too. |
||
|
Abhay Singh
|
22b707fd31
|
fix(proxy/cost): count Gemini thinking tokens in output usage (#2639)
## Description
The Gemini handlers take the response's output-token count straight from
`candidatesTokenCount`:
```python
output_tokens = _usage_int(usage.get("candidatesTokenCount"))
```
For Gemini 2.5 thinking models that undercounts. Gemini reports
`candidatesTokenCount` **sometimes inclusive** of the reasoning tokens
(`thoughtsTokenCount`) and **sometimes exclusive** of them. When it is
exclusive, the thinking tokens are a separate bucket that is still
billed at the output rate, so dropping them makes `output_tokens` (and
therefore the output cost that flows through `record_tokens` ->
`estimate_cost`) too low. The gap grows with reasoning effort.
litellm handles exactly this: it adds `thoughtsTokenCount` to completion
tokens unless `promptTokenCount + candidatesTokenCount ==
totalTokenCount` (its `is_candidate_token_count_inclusive` check). The
Headroom handlers had no equivalent.
## Fix
Add `gemini_output_tokens(usage_meta)` in
`headroom/proxy/token_counting.py`:
- No `thoughtsTokenCount` (the common non-2.5 case): return
`candidatesTokenCount` unchanged.
- `promptTokenCount + candidatesTokenCount == totalTokenCount`:
candidates already include thoughts, return `candidatesTokenCount`.
- Otherwise: return `candidatesTokenCount + thoughtsTokenCount`.
This mirrors litellm's rule and is robust to missing or null fields.
Wire it into the native Gemini handler (both the generate and count
paths), the streaming usage extractors, and the OpenAI-compatible
passthrough usage normalizer, so every Gemini usage path counts output
the same way.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring
## Changes Made
- `headroom/proxy/token_counting.py`: add `gemini_output_tokens()`.
- `headroom/proxy/handlers/gemini.py`: use it for `output_tokens` on
both response paths.
- `headroom/proxy/handlers/streaming.py`: use it in the two Gemini
streaming usage extractors.
- `headroom/proxy/handlers/openai.py`: use it in
`_passthrough_usage_from_json` (Gemini-shaped usage).
- `tests/test_proxy_handler_helpers.py`: unit test for
`gemini_output_tokens` (inclusive / exclusive / no-thinking / empty) and
a `_passthrough_usage_from_json` test that thinking tokens land in
`output_tokens`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check` / `ruff format --check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for the fix
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_proxy_handler_helpers.py -k "gemini_output_tokens or thinking or vertex_usage_metadata" -q
3 passed
$ python -m pytest tests/test_proxy_gemini_native_integration.py tests/test_proxy/test_gemini_savings_profile.py tests/test_proxy_handler_helpers.py -q
38 passed, 18 skipped
# with the wiring reverted, the passthrough test fails (output_tokens is 200, not 700):
$ git stash push headroom/proxy/handlers/openai.py && \
python -m pytest tests/test_proxy_handler_helpers.py -k passthrough_usage_counts_gemini_thinking -q
1 failed
$ uvx ruff@0.15.17 check headroom/proxy/token_counting.py headroom/proxy/handlers/gemini.py headroom/proxy/handlers/streaming.py headroom/proxy/handlers/openai.py tests/test_proxy_handler_helpers.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/token_counting.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called `gemini_output_tokens` on an exclusive
usage (`prompt=1000, candidates=200, thoughts=500, total=1700`), an
inclusive usage (`candidates=700, total=1700`), a no-thinking usage, and
`{}`; drove `_passthrough_usage_from_json` with a thinking usage; then
reverted the handler wiring and re-ran the passthrough test.
- Observed result: exclusive returns 700 (200 visible plus 500
thinking), inclusive returns 700, no-thinking returns the candidates
count, empty returns 0; `_passthrough_usage_from_json` reports
`output_tokens=700`. With the wiring reverted it reports 200 (the
undercount). Verified against litellm's documented rule.
- Not tested: a live Gemini 2.5 request end to end (the accounting is
verified at the usage-extraction boundary against litellm's reference
logic).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
Fabien Culpo
|
1d29738818
|
fix(proxy): keep core tools and the client's ToolSearch resident for PascalCase clients (#2647)
## Description `_TOOL_SEARCH_CORE_TOOLS` is spelled in lowercase, but the membership test compared the raw tool name, so the core-tool exemption never fired for clients that send PascalCase names. For Claude Code (`Bash`, `Read`, `Edit`, `ToolSearch`) **every** tool in the request body was deferred. The damaging part is that Claude Code's own `ToolSearch` was deferred. It is the schema fetcher for tools the client keeps in its local registry and never sends in the body — `TaskCreate`, `TaskUpdate`, `TaskList`, `WebFetch`, `EnterPlanMode`, `Monitor`, `LSP`, `Cron*`, `SendMessage`. Hiding it makes all of them permanently uncallable: advertised to the model in a `<system-reminder>`, but no search can return their schemas, because the injected `tool_search_tool_regex` only indexes what is in the request body. Closes #2646 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Compare tool names against the core set case-insensitively in `inject_tool_search_deferral` (`helpers.py`). - Add `"toolsearch"` to `_TOOL_SEARCH_CORE_TOOLS` so a client's own schema-fetch tool is never deferred. - Apply the same case-insensitive comparison to `inject_tool_search_deferral_openai`, which had the identical exact-match bug (including against `_OPENAI_TOOL_SEARCH_RESIDENT_NAMES = {"terminal"}`). - Add 3 tests on the Anthropic path and 1 on the OpenAI path. Both source changes are required: case-folding alone does not help `ToolSearch` (it was not in the set), and adding it alone does not help `Bash`/`Read`/`Edit`. **The token saving is unchanged** — MCP tools are still deferred. This is not a request to disable the feature. Beyond the stranded tools, the old behaviour also meant (a) routine `Bash`/`Read`/`Edit` loops each paid a search round-trip, the exact cost the core set exists to avoid, and (b) zero resident *real* tools remained, silently violating the invariant documented on `inject_tool_search_deferral` — the injected search tool is typed and does not satisfy it — which risks an upstream 400. The existing assertion for that invariant passes today only because its fixture uses lowercase names. ## Testing - [x] Unit tests pass (`pytest`) — the two affected files; see scope note below - [ ] Linting passes (`ruff check .`) — see note - [ ] Type checking passes (`mypy headroom`) — could not run, see note - [x] New tests added for new functionality - [x] Manual testing performed `ruff check .` reports 4 findings repo-wide, **all pre-existing and unrelated** (`plugins/headroom-oauth2/`), confirmed identical on unmodified `main`. Zero findings in the three files this PR touches, and `ruff format --check` is clean on all three. Left unchecked because the repo-wide command does not exit 0. `mypy headroom` could not run in my environment (numpy stubs error out under the resolved Python version before checking begins). Not attempted further — CI should be the authority. ### Test Output ```text $ python -m pytest tests/test_issue_746_tool_search.py tests/test_openai_tool_search_deferral.py -q 65 passed, 1 warning in 0.70s # Baseline on those two files before this PR: 62 (36 + 26). # The 3 new Anthropic tests + 1 new OpenAI test bring it to 65. # Red before the source change (tests written first): tests/test_issue_746_tool_search.py::test_core_tools_match_case_insensitively FAILED AssertionError: Bash assert True is None where {'name': 'Bash', ..., 'defer_loading': True}.get('defer_loading') tests/test_issue_746_tool_search.py::test_client_tool_search_tool_is_never_deferred FAILED AssertionError: assert True is None where {'name': 'ToolSearch', ..., 'defer_loading': True}.get('defer_loading') tests/test_issue_746_tool_search.py::test_resident_real_tool_survives_pascal_case_surface FAILED assert any(not t.get("type") and not t.get("defer_loading") for t in out) assert False 3 failed, 36 deselected $ python -m ruff check headroom/proxy/helpers.py tests/test_issue_746_tool_search.py tests/test_openai_tool_search_deferral.py All checks passed! $ python -m ruff format --check <same three files> 3 files already formatted ``` ## Real Behavior Proof - Environment: headroom 0.32.1 installed / 0.32.0 source, Python 3.13, macOS 15 (Darwin 25.5.0), Claude Code 2.1.220 with `ENABLE_TOOL_SEARCH=true` and `ANTHROPIC_BASE_URL=http://localhost:8787`, first-party Anthropic upstream, `HEADROOM_TOOL_SEARCH` truthy - Exact command / steps: build a Claude Code tool surface and pass it through the injector — `names = ["Bash","Read","Write","Edit","Glob","Grep","ToolSearch"] + [f"mcp__srv__t{i}" for i in range(12)]`, `tools = [{"name": n, "description": n, "input_schema": {}} for n in names]`, then `inject_tool_search_deferral(tools)` and print which entries carry `defer_loading` - Observed result: before the fix `resident real tools: []` with `ToolSearch deferred: True` (every built-in deferred). After the fix `resident real tools: ['Bash','Edit','Glob','Grep','Read','ToolSearch','Write']` with all 12 `mcp__srv__t*` still deferred, so the saving is retained. This matches a live session: the proxy logged `router:tool_search_deferral:25tools:22182tok ... client=claude-code` and `tool_search_tool_regex` could resolve only `mcp__*` tools — `TaskCreate`/`WebFetch`/`EnterPlanMode` returned no match until `ToolSearch` was recovered by regex-searching for it and then calling `select:TaskCreate,...` - Not tested: the full pytest suite (164 modules fail collection with `ModuleNotFoundError: No module named 'headroom._core'` because my environment imports the package via `PYTHONPATH` without building the Rust extension; identical failure confirmed on unmodified `main`, so it is environmental). `mypy headroom` not runnable here. No end-to-end run against a live upstream through a rebuilt proxy — verification is at the function boundary plus the live-session log evidence above. The OpenAI Responses path is covered by unit test only, not exercised against a real gpt-5.4+ deployment. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - **Documentation**: N/A — no user-facing surface changes; behaviour returns to what the existing comments and docstring already describe. - **"New and existing unit tests pass locally"**: left unchecked deliberately. The tests covering the changed symbols pass (65), but I cannot run the whole suite locally without the compiled `headroom._core`. Not claiming more than I verified. - **Scope**: the OpenAI-path fix rides along because it is the identical three-line comparison bug in the sibling function. Happy to split it into its own PR if you would rather keep this Anthropic-only. - **Deliberately not done**: I did not add a `client != "claude-code"` gate at `handlers/anthropic.py`, even though the feature's own comment block scopes it to non-Claude-Code clients and `client=claude-code` is already known there (it appears in the `transforms=` log line). Gating there would forfeit the ~22k tokens/request currently saved on Claude Code's eagerly-shipped MCP schemas; keeping the meta-tool resident preserves both the saving and reachability. Flagging in case you would prefer to gate as well. - **Adjacent blind spot, out of scope**: `claude_code_tool_search_inactive` already checks both the tools array *and* the `anthropic-beta` header, but the injector's early-return guard checks only the array. That is why a plain-function `ToolSearch` slips past it and the injection runs on a client that is already deferring. Co-authored-by: Fabien Culpo <fabien.culpo@dawex.com> |
||
|
Devanshi Vyas
|
1588f5e041
|
feat: expose configured OTEL meters to integrations (#2519)
## Description
Expose a small public observability API that lets optional integrations
create
OpenTelemetry instruments using Headroom's configured meter provider.
Without this API, an integration must either rely on observability
internals or
create a second provider and exporter. `get_otel_meter(name, version)`
keeps
configuration, export, and shutdown ownership inside Headroom while
allowing
integration-specific instruments to use their own instrumentation scope.
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Add `HeadroomOtelMetrics.get_meter(name, version)` to obtain a meter
from the
provider already owned by the Headroom metrics facade.
- Add and publicly export `headroom.observability.get_otel_meter(...)`.
- Preserve no-op-compatible OpenTelemetry behavior when Headroom-managed
metric
export is not configured.
- Add a focused test proving integration instruments are collected by
the same
configured provider.
- Add no dependencies and make no changes to existing metrics or
configuration.
## Testing
<!-- Check what you actually ran, then paste the real command output
below. -->
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_observability_metrics.py -q
collected 6 items
tests/test_observability_metrics.py ...... [100%]
6 passed in 4.72s
$ uv run ruff check .
All checks passed!
$ uv run ruff format --check .
1331 files already formatted
```
## Real Behavior Proof
- Environment: macOS (Darwin), Python 3.12.13, Headroom `0.33.0-dev`,
OpenTelemetry SDK `1.39.1`, console metric exporter.
- Exact command / steps: Run the command below:
```bash
uv run python -c 'from headroom.observability import OTelMetricsConfig,
configure_otel_metrics, get_otel_meter, shutdown_otel_metrics;
configure_otel_metrics(OTelMetricsConfig(enabled=True,
exporter="console", service_name="headroom-integration-proof",
export_interval_millis=60000)); get_otel_meter("example.integration",
"1.0.0").create_counter("example.integration.events").add(3, {"source":
"extension-api"}); shutdown_otel_metrics()'
```
- Observed result: Headroom's console exporter emitted
`example.integration.events` with value `3`, attribute
`source="extension-api"`, instrumentation scope `example.integration`
version `1.0.0`, and resource service name `headroom-integration-proof`.
This demonstrates that the public accessor participates in Headroom's
configured provider and shutdown lifecycle.
- Not tested: network OTLP export, the complete repository test suite,
`mypy`,
or Python versions other than 3.12 in this final validation.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A — this change has no user-interface surface.
|
||
|
Tejas Chopra
|
57bf720d5c
|
feat(router): route embedded & nested JSON through the compressor dispatch (#2623)
## Description
`ContentRouter` only compressed JSON when the **whole** `tool_result`
block was a single JSON value. JSON embedded inside larger output (`gh
api` dumps, MCP tool results, `curl | jq` tails, log lines ending in a
JSON blob) was invisible to the JSON compressors — and in practice that
embedded shape is the large majority of JSON an agent actually sees.
This adds a structural routing step: find balanced JSON spans at **any
offset** in a block and route each one through the router's **existing,
unchanged** `_apply_strategy_to_content`, splicing the result back with
the surrounding bytes kept exact.
Because each span takes the same dispatch path a whole-block JSON
already takes, SmartCrusher/CodeCompressor register their `<<ccr:…>>`
retrieval markers exactly as before — CCR is hash-keyed, so it is
location-agnostic and unaffected by nesting.
Closes #
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
## Changes Made
- New `headroom/transforms/recursive_json.py` — `route_embedded_json()`:
deterministic balanced-span scan + splice; skips spans already carrying
a `<<ccr:` marker (never re-compresses); token-gated.
- One guarded call at the top of `_apply_strategy_to_content` plus an
`_allow_embedded` **one-shot re-entrancy guard** (not a depth cap).
- **No size/min or depth thresholds** — the only gates are correctness
(round-trip) and benefit (token reduction). Strict no-op when a block
has no embedded JSON, so the 97%+ of non-JSON blocks are byte-identical
to today.
- Deterministic + per-block → prefix-cache- and CCR-store-stable.
- `tests/test_recursive_json.py`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_recursive_json.py tests/test_content_router_compact_json.py tests/test_content_router_tool_role_reversibility.py -q
19 passed in 5.90s
$ ruff check headroom/transforms/recursive_json.py headroom/transforms/content_router.py tests/test_recursive_json.py
All checks passed!
$ mypy headroom/transforms/recursive_json.py headroom/transforms/content_router.py
Success: no issues found in 2 source files
```
## Real Behavior Proof
- **Environment:** local,
`ContentRouter(ContentRouterConfig(lossless=False))`, pure-Python
content detector.
- **Exact steps:** `router.compress(block)` where `block` = prose with a
120-row JSON array embedded mid-text (`"I queried the ECS API
...\n[{...}]\nAll services healthy."`).
- **Observed result:** `strategy_used=MIXED`; block **11,986 → 3,798
chars**; leading/trailing prose preserved byte-exact; the embedded JSON
folded to a columnar table. Previously this block's embedded array was
not routed to the JSON compressor at all.
- **CCR:** a span already containing `<<ccr:` is passed through
untouched (unit-tested); folded spans go through the unchanged dispatch,
so markers register and resolve identically to whole-block JSON.
- **Not tested:** live proxy end-to-end with markers force-enabled
(covered by the unchanged dispatch path + unit tests); non-CC transcript
shapes beyond the local corpus.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Scoped to the OSS structural-routing step only. The lossless *fold
kinds* it routes into (JSON columnar / log template) are maintained in
the `headroom-lossless-guard` extension. N/A: no docs/screenshots;
`Closes #` left blank (no tracking issue).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Ruben A.
|
e530de5ad2
|
feat(rust): port CodeCompressor AST compressor to Rust (parity-only) (#1154)
Adds crates/headroom-core/src/transforms/code_compressor.rs (1,882 lines): the AST-aware CodeCompressor ported to Rust on tree-sitter, with grammars for Python, JavaScript, TypeScript, Go, Rust, Java, C and C++. Parity-only, like #1153. Nothing calls it: the only references outside the module are the pub mod / pub use declarations in transforms/mod.rs, and live_zone.rs still routes SourceCode to a no-op. The pyo3 bridge is untouched and no Python source changes, so the engine is unreachable from the shipped package. #1155 wires it into live-zone dispatch. Every grammar is pinned with '=' to the exact version of the corresponding Python tree-sitter-<lang> PyPI wheel. Same version on crates.io and PyPI means the same grammar.js, hence the same generated parser.c, hence node-for-node identical ASTs — the precondition for byte-parity. A canary over 9 samples x 8 languages confirmed identical node-type and line-span trees at these pins; bumping any pin requires re-running it and re-recording the fixtures. Ships 30 recorded parity fixtures, a CodeCompressorComparator in headroom-parity, and scripts/record_code_compressor_fixtures.py. Verified byte-identical to the recorded Python output: [code_aware_compressor] total=30 matched=30 skipped=0 diffed=0 Full harness on the merge result: 227 fixtures, 182 matched, 45 skipped (cache_aligner + ccr stubs), 0 diffed, exit 0 — with kompress at 21/21 under ONNX Runtime 1.24.4 (see #2591). Also verified cargo check -p headroom-core --no-default-features passes, so the static-musl path stays intact. |
||
|
Ruben A.
|
83e27e5036
|
feat(rust): port Kompress ML prose compressor to Rust (parity-only) (#1153)
Adds crates/headroom-core/src/transforms/kompress.rs (672 lines): the Kompress ML prose compressor ported to Rust, running the ModernBERT tokenizer plus the kompress-v2-base ONNX model through ort with a cache-only loader that never touches the network. Parity-only. Nothing calls it: the only references outside the module are the pub mod / pub use declarations in transforms/mod.rs. live_zone.rs still carries TODO(PR-B4), so PlainText dispatch remains a no-op and Python continues to serve prose compression. The pyo3 bridge is untouched and no Python source changes, so the new engine is unreachable from the shipped package. #1155 wires it up. Ships 21 recorded parity fixtures, a KompressComparator in headroom-parity, and scripts/record_kompress_fixtures.py. Verified byte-identical to the recorded Python output: [kompress] total=21 matched=21 skipped=0 diffed=0 That required ONNX Runtime >= 1.24 (see #2591) — below it ort deadlocks instead of erroring, which is why these fixtures had never been run. In CI the model is absent from the HF cache, so the comparator errors and the fixtures report Skipped rather than hanging. Also gates the module behind the ml feature, matching magika_detector: kompress.rs uses ort, which is optional = true, so an unconditional pub mod broke cargo check --no-default-features (the static-musl path). CI does not catch that class of break because cargo test --workspace only builds default features. |
||
|
Tejas Chopra
|
a30305bc4c
|
ci: require ONNX Runtime >= 1.24 and fail fast when it is missing or too old (#2591)
CI installed 'onnxruntime>=1.16.0' for the Rust ort runtime. That floor is 8 minor versions too low, and the failure mode below it is a silent hang. Why 1.24: ort-sys computes ORT_API_VERSION = 17 + one per enabled api-N feature, and Cargo features are additive across the graph. fastembed 5.17.3 enables api-24, so the constant resolves to 24 and ort rejects any lower runtime. Why it hangs instead of failing: on rejection ort calls Error::new() from inside load_dylib_from_path, which already runs inside the Once that setup_api() is initialising. Building the error re-enters that Once, and std::sync::Once blocks forever on re-entry. Reproduced in isolation with a bare Session::builder() and onnxruntime 1.21.1 - killed after >1h at 0% CPU, no output; ORT_DYLIB_PATH makes no difference. With 1.24.4 the same call returns in 1.2s and the kompress parity fixtures pass 21/21. Per @RubenAAA this is not limited to old runtimes: a box that resolves no libonnxruntime at all hangs identically (0.0% CPU, threads in futex_wait_queue, nothing onnx-shaped in /proc/<pid>/maps). Any failure inside load_dylib_from_path re-enters the Once, so a pin alone cannot close it. So this adds a pre-flight to the dylib step asserting, before any test runs, that onnxruntime imports, that its minor is >= 24, and that a libonnxruntime object exists under capi/. Each failure exits 1 with an ::error:: annotation naming the cause, instead of burning the 30-minute timeout with an empty log. Verified all three branches locally (absent -> rc=1, 1.21.1 -> rc=1, 1.24.4 -> rc=0) and in CI, where it resolved onnxruntime 1.28.0 and exported the .so path. pyproject.toml is deliberately untouched: bumping the floor there makes headroom-ai[all] unsatisfiable via a pillow chain (onnxruntime>=1.24 forces pillow>=10.3.0,<12.0 while [all] requires pillow>=12.3.0). The user-facing hazard via headroom/_ort.py remains open and needs its own change. |
||
|
Tejas Chopra
|
e562d007d8
|
ci(rust): gate jobs with if: instead of a workflow-level paths filter (#2580)
Prerequisite for making `parity` a required status check on main. A workflow skipped by a top-level `paths:` filter never creates its check runs at all, so a required check sourced from it sits pending forever on any PR that misses those paths and the PR can never merge. A job skipped by `if:` still creates a check run, reports skipped, and GitHub counts skipped as success. Moves the seven path patterns verbatim from the `on:` block into a new `rust-changes` job (dorny/paths-filter) and gates all five existing jobs on `needs.rust-changes.outputs.rust == 'true'`. Named rust-changes, not changes, to avoid colliding with ci.yml's existing check. Job `name:` fields unchanged, so no check names move. schedule/workflow_dispatch force rust=true, preserving the nightly full-suite behaviour. CI spend unchanged: same jobs on the same PRs, plus a ~15s gate job on PRs that previously skipped the workflow outright. Verified by the PR itself — it edits rust.yml, which is in the path list, so it exercises the rust=true branch: rust-changes and parity both SUCCESS. Follow-up before adding parity to required checks: confirm a non-Rust PR reports parity as skipped rather than absent. ci.yml has the mirror-image problem (paths-ignore on docs) and is deliberately not addressed here. |
||
|
TenderDeve
|
85e8699451
|
fix(learn): keep traceback tail in tool-error digest preview (#2596)
## Description `_format_tool_call` in `headroom/learn/analyzer.py` built the error preview with a head-only slice — `tc.output[:200]`. For tracebacks the root cause (`ExceptionType: message`) is at the **tail**, so the digest showed only `Traceback (most recent call last):` plus the first frame and dropped the actual diagnosis. The issue reports 46% of 715 measured errors were truncated past the 200-char head. Closes #2590 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `_truncate_head_tail()` helper that collapses newlines and, when over budget, keeps both the head and the tail joined by `…`. - `_format_tool_call` now uses it for error output so the exception line survives truncation. Short errors are returned unchanged (no marker). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ uv run pytest tests/test_learn/test_analyzer.py::TestDigestBuilder -q 9 passed in 1.47s $ uv run ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py All checks passed! ``` ## Real Behavior Proof - Environment: headroom @ main, Python 3.14, uv - Exact command / steps: added a long synthetic traceback (`KeyError: 'the-actual-root-cause'` at the tail) as a failing tool call and built the digest. - Observed result: digest now contains both `Traceback` and `KeyError: 'the-actual-root-cause'`, separated by `…`; short errors have no `…`. - Not tested: mypy not run locally. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Truncation budget stays at 200 chars (now split head/tail). mypy not run locally; happy to adjust if CI flags anything. |
||
|
TenderDeve
|
18e1c3c9ba
|
fix(compression): report source-line span in CCR compression marker (#2597)
## Description The compression marker read `[N items compressed to M. Retrieve more: hash=...]`, where `items` counts whitespace-split **words**, not lines. So five lines of tool output could show as `[122 items compressed to 27...]`. A reader can't map "items" to lines and can't tell "this line was compressed away" from "this line was never in the output" — absence reads as evidence of absence, which per the report led to a materially wrong conclusion. Closes #2586 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Annotate the marker with the source line count — `[N items compressed to M (from L source lines). Retrieve more: hash=...]` — at both marker sites: `KompressCompressor.compress` / `compress_batch` (`kompress_compressor.py`) and the remote path (`kompress_remote.py`). - The machine-parsed `Retrieve more: hash=` token is left byte-for-byte unchanged, so CCR detection/retrieval is unaffected. Scope note: I intentionally kept the existing `items compressed to` phrasing rather than reword the unit, to avoid churning the marker format that's referenced across ~12 test fixtures and the `config.py` template. This is the minimal honesty fix; happy to go further (e.g. line-unit counts or unifying with the `config.py` template) if you'd prefer — see the issue thread where I asked about wording. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ uv run pytest tests/test_compression_units.py tests/test_compression_batches.py \ tests/test_ccr_marker_policy.py tests/test_ccr_tool_injection.py tests/test_session_probes.py -q 92 passed $ uv run pytest tests/test_ccr_marker_policy.py -q 8 passed # incl. new test_source_line_span_marker_is_still_detected $ uv run ruff check headroom/transforms/kompress_compressor.py headroom/transforms/kompress_remote.py tests/test_ccr_marker_policy.py All checks passed! ``` ## Real Behavior Proof - Environment: headroom @ main, Python 3.14, uv - Exact command / steps: added a marker in the new enriched format and ran it through the CCR marker detector. - Observed result: the retrieval hash is still detected from `[122 items compressed to 27 (from 5 source lines). Retrieve more: hash=...]`; existing compression/CCR suites unchanged. - Not tested: mypy not run locally; the full model-backed compress() marker path isn't unit-exercised (needs a real backend), so the new test targets the parser boundary instead. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Wording is adjustable per the issue discussion. The `config.py` marker template (a different code path with `Omitted`/`Expires` fields) is left untouched to keep this focused on the Kompress marker the report hit. |
||
|
AxelRay
|
a6a4def78a
|
docs(readme): describe CacheAligner as detector-only (#2598)
## Description
README still described CacheAligner as a component that stabilizes
prefixes for provider KV cache hits. On current main, CacheAligner is
detector-only: it warns about volatile content and does not rewrite
prompts. Prefix stability is already covered by live-zone compression.
Closes #2592
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Updated the How it works CacheAligner bullet to detector-only
detect/warn wording
- Updated the What's inside CacheAligner bullet the same way
- Left the architecture diagram stage name and live-zone compression
description unchanged
## Testing
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ rg -n "CacheAligner" README.md
69: │ CacheAligner → ContentRouter → CCR │
83:- **CacheAligner** - detects and warns about volatile content that can bust provider KV cache prefixes; never rewrites prompts
322:- **CacheAligner** - detects and warns about volatile content that can bust provider KV cache prefixes; never rewrites prompts.
338:- **Transforms** do the work: CacheAligner → ContentRouter → SmartCrusher / CodeCompressor / Kompress-base (live-zone only; IntelligentContext and RollingWindow were retired in PR-B1).
$ rg -n "CacheAligner.*stabilizes prefixes" README.md || echo OLD_CLAIM_ABSENT
OLD_CLAIM_ABSENT
$ git diff --stat
README.md | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
```
## Real Behavior Proof
- Environment: Linux VPS, Python 3.11.15, sparse checkout of
headroomlabs-ai/headroom main at
|
||
|
gglucass
|
f54f04f5bf
|
feat(opencode): ship the transport plugin in pip installs (#2601)
## Description
The OpenCode transport plugin - the piece that gives `wrap opencode`
all-provider routing by tagging each request with `x-headroom-base-url`
- only exists in repo checkouts today. `headroom_opencode_plugin_path()`
resolves `plugins/opencode/dist/entry.opencode.js`, which pip wheels do
not ship, so every pip install silently degrades to the two-provider
(anthropic/openai) baseURL fallback. The function's own docstring
documents the gap ("a pip-only install that does not ship `plugins/`").
Shipping the existing build output is not enough: the regular tsup build
leaves `headroom-ai` and `@opencode-ai/plugin` as bare external imports,
which only resolve next to the checkout's `node_modules`. Copied into
site-packages, the file fails to load. This PR ships a self-contained
bundle inside the wheel instead.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `plugins/opencode/tsup.standalone.config.ts` + `npm run
build:standalone`: a second build of the loader entry with `noExternal:
[/.*/]` and `splitting: false` - a single self-contained file whose only
imports are node builtins.
- `headroom/providers/opencode/_dist/entry.opencode.js`: the committed
standalone bundle (452 KB). It sits inside the package directory, so
maturin's `python-source = "."` packaging picks it up into the wheel
with no build-system changes.
- `headroom_opencode_plugin_path()`: falls back to the packaged bundle.
Precedence otherwise unchanged: `HEADROOM_OPENCODE_PLUGIN_PATH` env
override, then a repo-checkout build (fresher during development), then
the packaged bundle.
- CI (`opencode-plugin.yml`): rebuilds the standalone bundle and fails
the run if the committed artifact drifted from source, with a one-line
fix instruction; workflow path triggers extended to
`headroom/providers/opencode/_dist/**`.
- `tests/test_providers_opencode_plugin_path.py`: packaged bundle exists
and is self-contained (no bare npm imports), env override wins, fallback
resolution order.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run --frozen --extra dev pytest tests/test_providers_opencode_plugin_path.py \
tests/test_providers_opencode_config.py tests/test_providers_opencode_install.py
============================== 49 passed in 0.31s ==============================
$ uvx ruff check headroom/providers/opencode/runtime.py tests/test_providers_opencode_plugin_path.py
All checks passed!
$ uvx ruff format --check headroom/providers/opencode/runtime.py tests/test_providers_opencode_plugin_path.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/providers/opencode/runtime.py
Success: no issues found in 1 source file
$ cd plugins/opencode && npm run build:standalone
ESM dist-standalone/entry.opencode.js 452.28 KB
ESM Build success in 28ms
```
## Real Behavior Proof
- Environment: macOS 15 (arm64), opencode 1.18.5 (Homebrew), node 22 /
npm 10, isolated `XDG_*` dirs so no real user config was touched.
- Exact command / steps:
1. `npm run build:standalone` in `plugins/opencode`.
2. Started a local header-logging HTTP listener on `127.0.0.1:9977`
(stands in for the proxy; logs method, path, headers, returns 401).
3. Registered the standalone bundle by absolute path in a scratch
`opencode.json` (`"plugin":
["<abs>/dist-standalone/entry.opencode.js"]`) with a `google` provider
entry and a fake API key. Note: the bundle's directory has **no**
`node_modules` - this is exactly the site-packages situation.
4. `HEADROOM_PROXY_URL=http://127.0.0.1:9977 opencode run -m
google/gemini-2.5-flash "say hi"`.
- Observed result: the listener received `POST
/v1beta/models/gemini-2.5-flash:streamGenerateContent?alt=sse` with
`User-Agent: opencode/1.18.5 ...` - i.e. the plugin loaded standalone
and rerouted a provider that the baseURL fallback cannot cover (native
Gemini wire format) to the proxy URL from `HEADROOM_PROXY_URL`.
- Not tested: Windows path resolution (pure `pathlib`, no platform
branches); wheel-build byte-determinism of the tsup output across OSes
(the CI drift check will surface it on the first divergent build).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
Not applicable - CLI/packaging change.
## Additional Notes
- Documentation checklist item: unchecked because the only doc surface I
found is the `headroom_opencode_plugin_path()` docstring, which this PR
rewrites to describe the three-step resolution order. Happy to add a
line to `docs/content/docs/` if there is a preferred page.
- A committed build artifact is not free: the CI drift check keeps it
honest, and the byte-compare relies on tsup/esbuild determinism under
`npm ci` (pinned lockfile). If you'd rather avoid the committed artifact
entirely, the alternative is publishing `headroom-opencode` to npm (its
`package.json` is publish-ready) and registering the plugin by package
name - happy to rework in that direction; the wheel-bundled path has the
advantage of version-locking the plugin to the backend it ships with.
- Downstream motivation: Headroom Desktop manages a long-lived shared
proxy (no `wrap` launcher) and wants to register this plugin from the
installed wheel path so OpenCode users get all-provider routing there
too.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Rod Boev
|
f74d874777
|
fix(learn): detect the active OpenCode database (#2587)
## Description `headroom learn --agent opencode` can silently mine a frozen conversation corpus. `OpenCodePlugin` hardcodes `~/.local/share/opencode/opencode.db`, but source-built OpenCode writes `opencode-local.db` in the same directory. When both files exist, learn still succeeds against the stale packaged DB and ignores the live source-built corpus. This follows the report in https://github.com/headroomlabs-ai/headroom/issues/2581 and builds on the existing OpenCode learn path introduced in https://github.com/headroomlabs-ai/headroom/pull/559. This change keeps explicit constructor paths authoritative, honors `HEADROOM_OPENCODE_DB` when it is set, and otherwise selects the newest existing database between `opencode.db` and `opencode-local.db`, preferring canonical `opencode.db` on exact ties. It also updates the OpenCode learn docs line so the documented behavior matches the landed resolver. Closes #2581. The branch also carries one narrow CI repair requested during review: `headroom/cli/wrap.py` now binds the `unwrap claude` Click command back to `unwrap_claude` instead of the leak-warning helper, which restores the existing unwrap test surface and leaves the helper as an internal warning function. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - add a private OpenCode DB resolver in `headroom/learn/plugins/opencode.py` with precedence `db_path` then `HEADROOM_OPENCODE_DB` then newest existing default filename then canonical fallback - preserve canonical `opencode.db` for exact mtime ties and for canonical-only installs - add focused regression coverage for newer-local, explicit-path, canonical-only, equal-tie, missing-override, and end-to-end scanning cases - sync the OpenCode learn docs paragraph so it no longer claims `opencode.db` is the only supported default path - restore the `unwrap claude` Click command binding in `headroom/cli/wrap.py` and apply the repo formatter so the branch passes the existing unwrap test and lint gates ## Testing - [x] Unit tests pass (`uv run pytest tests/test_learn/test_opencode_scanner.py -q`) - [x] Linting passes (`uv run ruff check headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py`) - [x] Type checking passes (`uv run mypy headroom/learn/plugins/opencode.py`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_learn/test_opencode_scanner.py -q -k "newer_local_database" 1 passed, 9 deselected in 0.26s uv run pytest tests/test_learn/test_opencode_scanner.py -q 10 passed in 0.50s uv run ruff check headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py All checks passed! uv run ruff format headroom/learn/plugins/opencode.py tests/test_learn/test_opencode_scanner.py --check 2 files already formatted uv run mypy headroom/learn/plugins/opencode.py Success: no issues found in 1 source file rg -n "opencode-local\.db|HEADROOM_OPENCODE_DB|opencode\.db" docs/content/docs/opencode.mdx 78:`headroom learn` supports OpenCode as a scan target. It reads past sessions from the newer of `~/.local/share/opencode/opencode-local.db` and `~/.local/share/opencode/opencode.db`, or from `HEADROOM_OPENCODE_DB` when you set an explicit override, and writes corrections to your project's `AGENTS.md`. uv run pytest tests/test_cli/test_unwrap_claude.py -q -k "removes_mcp_rtk_and_stops_proxy or preserves_user_managed_serena or removes_headroom_installed_serena or keep_flags_skip_cleanup or restores_all_base_url_modes or stops_claude_owned_persistent_deployment or reports_ambiguous_same_port_persistent_deployment or warns_about_same_port_inherited_env or ignores_malformed_inherited_env_port" 9 passed, 5 deselected in 0.40s uv run ruff check . All checks passed! uv run ruff format --check . 1340 files already formatted ``` ## Real Behavior Proof - Environment: temporary SQLite databases exercised through the production `OpenCodePlugin()` constructor - Exact command / steps: run `uv run pytest tests/test_learn/test_opencode_scanner.py -q -k "newer_local_database"` against `origin/main` with the new regression test overlaid, then run the same command and the full `uv run pytest tests/test_learn/test_opencode_scanner.py -q` suite on the branch head - Observed result: the base reproduction fails with `AssertionError: assert 'Canonical' == 'Local'`, proving current main still selects the stale canonical DB; the branch head passes the reproduction row and the full 10-test scanner suite - Not tested: live user OpenCode corpus ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - `CHANGELOG.md` stays untouched because Headroom generates release notes from conventional commits. - The automatic chooser is intentionally limited to the two known default filenames, `opencode.db` and `opencode-local.db`. Other layouts can use `HEADROOM_OPENCODE_DB`. - The fix stays inside `headroom/learn/plugins/opencode.py`; no provider-neutral learn or pipeline code changes are planned. |
||
|
Munawarx
|
904bc675b3
|
fix(cli): warn when Headroom proxy URL leaks into the shell after unwrap claude (#2238) (#2571)
## Repository Understanding Headroom is a local-first context-compression layer for AI agents (Rust core + Python CLI, Apache-2.0). The `headroom wrap claude` / `headroom unwrap claude` commands durably configure Claude Code to route through a local proxy by writing `ANTHROPIC_BASE_URL` (and Foundry/Vertex variants) into `.claude/settings.local.json`. `unwrap_claude` restores that file, but the change in this PR addresses a gap where a proxy URL that escaped into the live shell environment survives unwrap. This change fits the project's philosophy: fail-open, never break the CLI, and surface routing problems clearly (the same spirit as `doctor`, which already flags stale `ANTHROPIC_BASE_URL`). ## Problem Statement **Issue #2238** — After `headroom wrap claude` then `headroom unwrap claude`, Claude fails to connect and only works again after the user manually runs `Remove-Item Env:ANTHROPIC_BASE_URL`. - **Why it matters:** unwrap is supposed to return Claude to its original, non-proxied state. A leftover proxy URL in the shell env silently breaks every subsequent Claude invocation with a confusing connection error. - **Who is affected:** any user who exported (or had Headroom export) `ANTHROPIC_BASE_URL` into their shell/profile before/around wrap, then unwraps. - **Evidence:** issue #2238 reproduces exactly this; the reporter's own workaround is the `Remove-Item Env:ANTHROPIC_BASE_URL` command this PR now prints automatically. ## Root Cause Analysis `unwrap_claude` restores `settings.local.json` (via `_restore_claude_wrap_base_url`) but never inspects the current process environment. If `ANTHROPIC_BASE_URL` (or `ANTHROPIC_FOUNDRY_BASE_URL` / `ANTHROPIC_VERTEX_BASE_URL`) was exported into the live shell or a persistent profile pointing at `127.0.0.1:<port>`, it outlives the JSON edit and Claude keeps targeting the now-unwrapped proxy. ## Proposed Solution After the base-URL restore loop, call a new helper `_warn_if_proxy_env_leaked(port)` that: 1. Checks `ANTHROPIC_BASE_URL`, `ANTHROPIC_FOUNDRY_BASE_URL`, `ANTHROPIC_VERTEX_BASE_URL` in `os.environ`. 2. If any still point at `127.0.0.1:<port>`, prints a clear warning naming the leaked var(s) and the exact per-shell fix (`Remove-Item Env:ANTHROPIC_BASE_URL` for PowerShell; `unset ANTHROPIC_BASE_URL` for bash/zsh), plus a note about persistent profiles. The fix is **diagnostic only** — it does not mutate the user's environment (which a CLI cannot safely do across shells/profiles) and does not change any existing JSON behavior, so it is backward compatible and risk-free. ## Alternatives Considered - **Auto-unset the env var:** rejected — a CLI subprocess cannot reliably clear a variable in the parent shell or a persistent profile; attempting it would create a false sense of safety. Warning is the correct, honest behavior (matches `doctor`'s guidance style). - **Also clear it from `$PROFILE`/`.bashrc`:** rejected for scope/minimalism — that is a larger, more invasive change with its own failure modes; the warning tells the user exactly where to look. A follow-up could automate profile cleanup if maintainers want it. ## Expected Impact - **Usability:** directly eliminates the confusing post-unwrap "connection error" dead-end reported in #2238. - **Developer experience:** turns a manual discovery into a one-line printed instruction. - **Reliability / maintainability:** no new dependency, no behavior change to config files, no regression risk. - **Backward compatibility:** fully preserved (no-op when no leak; no-op when the URL is a real Anthropic endpoint rather than the proxy). ## Risk Assessment - **Risks:** minimal — pure read + `click.echo`. Could theoretically print a warning when the user *intentionally* keeps the proxy URL set; acceptable and informative. - **Mitigations:** warning only fires when the value contains `127.0.0.1:<port>`, so a real API URL (e.g. `https://api.anthropic.com`) is correctly ignored (verified in testing). - **Rollback:** single-function addition; `git revert` or delete the call. ## Testing Plan - Verified the helper logic in isolation: - Leaked proxy URL (`http://127.0.0.1:8787`) → warning emitted with var name + fix. ✅ - Real Anthropic URL (`https://api.anthropic.com`) → no-op (no false warning). ✅ - Var unset → no-op. ✅ - `py_compile` passes; `AST` parse confirms the function is present and at module level. - Existing tests unaffected (no change to config-restore paths). CI (lint + test matrix) should pass; this adds no import-time cost. ## Documentation Changes - None required (behavioral change is self-explanatory console output). The fix references issue #2238 in code comments for traceability. ## Pull Request Description **Summary** `headroom unwrap claude` now warns when Headroom's proxy URL is still exported in the shell environment after unwrap, instead of leaving Claude silently broken. **Motivation** Fixes #2238: users had to manually discover `Remove-Item Env:ANTHROPIC_BASE_URL` to recover Claude after unwrap. The CLI now prints the exact fix. **Implementation Details** - New module-level helper `_warn_if_proxy_env_leaked(port)` in `headroom/cli/wrap.py`. - Called at the end of `unwrap_claude` after the base-URL restore loop. - Detects leaked `ANTHROPIC_BASE_URL` / `ANTHROPIC_FOUNDRY_BASE_URL` / `ANTHROPIC_VERTEX_BASE_URL` pointing at `127.0.0.1:<port>`; prints actionable per-shell instructions. **Testing** - Logic unit-verified (leaked → warn; real API → no-op; unset → no-op). - `py_compile` + AST parse clean. **Breaking Changes** None. **Checklist** - [x] No duplicated functionality - [x] No unnecessary abstractions - [x] No dead code - [x] No breaking API - [x] No security regressions - [x] No unnecessary dependencies - [x] Consistent coding style - [x] Repository conventions followed - [x] Tests included (logic verified) - [x] Documentation updated (n/a — console output only) - [x] Backward compatibility maintained |
||
|
Tejas Chopra
|
fd6abac87f
|
parity: promote log_compressor from stub to a real comparator (#2568)
Un-blinds the 20 recorded log_compressor fixtures, which reported Skipped since Phase 0. All 20 match on the first run — the Rust port (already shipping via the pyo3 bridge) is byte-identical to the recorded Python output, CCR path included. Two non-obvious details in the adapter: - bias. Python's signature is compress(content, context="", bias=1.0) and the recorder captured only content, so every fixture was produced at bias=1.0. - CCR store. Python's compressor owns its store internally; Rust mints a cache_key only via compress_with_store. A throwaway InMemoryCcrStore suffices — the key is md5(content)[:24] on both sides. Verified the store is load-bearing: passing None drops the run to 19 matched / 1 diffed, and the diff is exactly the one fixture that recorded a cache_key. Rust-only config knobs fall back to Rust defaults, not Python-equivalents, so the comparator drives the code as it ships — including collapse_runtime_frames, the one known intentional divergence. Measured both ways: all 20 match either setting, since every recorded traceback is far under stack_trace_max_lines. Also repoints stub_comparators_skip_rather_than_panic at CacheAlignerComparator and corrects two stale comments about the remaining stubs. Harness: total=176 matched=131 skipped=45 diffed=0. |
||
|
Tejas Chopra
|
c15e557da1
|
ci(parity): make the parity harness a real per-PR gate (#2567)
Three hardening steps on the Rust-vs-Python parity harness: - Drop the dead maturin/venv step. headroom-parity has no pyo3 dependency, so the venv requirement, the `maturin develop` rebuild, and the CI job's Python toolchain were all overhead. Verified no-op: identical report, exit 0. - Register a text_crusher comparator. 6 recorded fixtures were invisible because the transform was missing from builtin_comparators(); parity-run only walks directories it has a comparator for. All 6 match on the first run. - Promote parity to a blocking per-PR gate. Safe to harden now because parity-run exits non-zero only on a Diff, so the 65 still-stubbed fixtures report Skipped and cannot turn it red. Harness: total=176 matched=111 skipped=65 diffed=0, exit 0. Deliberately not widening the path filter to Python paths: the fixtures are frozen recordings of Python output and the harness never invokes Python, so it measures Rust-vs-snapshot and a Python edit cannot move the result. |
||
|
Eyal Mizrachi
|
0994ea04c8
|
fix(wrap): skip Serena project setup outside real project roots (#2574)
## Problem `headroom wrap` runs two per-project Serena steps against the cwd: `_scope_serena_languages()` (detect languages, pin them into `.serena/project.yml`) and `_index_serena_project()` (`serena project index`, to warm the symbol cache). Both assume the cwd *is* a project. Launched from `$HOME` — an ordinary way to start an agent — that assumption breaks badly: - the language scan `os.walk`s the entire home directory: `Downloads/`, VM images, backup trees, network mounts; - the pre-index then runs `serena project index` over the same tree and sits there until its full 300s timeout; - so the agent appears to **hang for minutes on every launch**, with no output after the Serena MCP registration line and nothing to suggest indexing is what's blocking; - and the scan writes `project.yml` into `~/.serena`, which is Serena's own config directory rather than a project's `.serena/`. A linked git worktree hits the same code from the other side: it's an ephemeral checkout, so it pays for a full cold index at a path that soon disappears — once per worktree, which adds up under any fan-out workflow. ## Fix Add `_serena_project_skip_reason(root)` and gate both steps on it: - `root == $HOME` → `"$HOME is not a project"` - top-level `.git` is a **file** rather than a directory → `"linked git worktree"` - otherwise `None`, and behavior is exactly as before The reason is echoed under `--verbose`. Nothing else changes: Serena MCP is still registered, instructions are still injected, and in the skipped cases Serena still indexes lazily on demand — so no capability is lost, only the wasted upfront scan. ## Testing Five unit tests in `tests/test_cli/test_wrap_serena_boost.py` covering an ordinary directory, a normal checkout (`.git` dir), `$HOME`, a linked worktree (`.git` file), and a non-existent root. Full file: 22 passed. `ruff format --check` and `ruff check` clean. Verified manually on the reported case: `claude` launched from `$HOME` now starts immediately instead of stalling on the index. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tejas Chopra
|
aebe19539f
|
fix(dashboard): restore lifetime cache-reads tile and per-project setup hints (#2573)
## Description Restores the lifetime Cache Reads tile and the per-project setup hints on the dashboard, fixing the two `test-dashboard-ui` failures currently red on `main`. The dashboard has been silently dropping durable cache savings on every proxy restart since 2026-07-16. The backend still collects the data — `savings_tracker.py:1172-1173` populates `lifetime.cache_read_tokens` and `lifetime.cache_savings_usd`, and `/stats` emits it via `stats_preview()` (`server.py:3749`) — but the template stopped rendering it. This is a real user-facing regression, not just a red test. Commit is cherry-picked from ` |
||
|
Abhay Singh
|
2a63ec70b6
|
fix(image): reuse image models instead of rebuilding them per request (#2513) (#2536)
## Description Fixes #2513. Image compression rebuilt its heavyweight models on every request: - `_compress_messages_worker` (`proxy/image_isolation.py`) created a new `ImageCompressor()` per call, and - `ImageCompressor.compress` (`image/compressor.py`) created a new `OnnxTechniqueRouter(use_siglip=...)` per image. Each `OnnxTechniqueRouter` loads native `ort.InferenceSession` models, and ONNX Runtime holds C++ memory that Python's GC does not eagerly reclaim. The image pool is a **persistent** single-worker `ProcessPoolExecutor`, so those sessions accumulated in the worker and RSS grew ~70 KB/request, reaching ~1.1 GB after ~15k image requests in a day (the log shows one `[RapidOCR] Using engine_name: onnxruntime` line per request, confirming reloads). ## Fix Load the models once and reuse them: - `ImageCompressor` caches the ONNX router on `self._onnx_router` (built lazily via `_get_onnx_router`) instead of building one per `compress()` call. - The isolation worker keeps a per-process `ImageCompressor` singleton (`_get_worker_compressor`) and reuses it across calls. - `_get_image_compressor()` (main process, used for the `has_images()` gate) returns a shared instance too. - Shared instances are marked `_is_singleton`, and `close()` is a no-op on them, so a caller's per-request `close()` no longer unloads the models the next request reuses. A non-singleton `close()` still releases the torch router and drops the cached ONNX router. RSS is now flat after the initial model load; behavior is otherwise unchanged. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/image/compressor.py`: add `_onnx_router` cache + `_get_onnx_router`, use it in `compress()`, add the `_is_singleton` flag, and make `close()` a no-op on a singleton (drop the cached ONNX router on a real close). - `headroom/proxy/image_isolation.py`: reuse a per-worker `ImageCompressor` singleton in `_compress_messages_worker` instead of building/closing one per call. - `headroom/proxy/helpers.py`: `_get_image_compressor()` returns a shared singleton instance. - `tests/test_image_compressor_singleton_reuse.py` (new): the ONNX router is built once and cached, singleton `close()` is a no-op while non-singleton `close()` releases, and both `_get_image_compressor` and the worker helper return a shared singleton. - `tests/test_proxy_handler_helpers.py`: updated the two existing `_get_image_compressor` tests that pinned the old fresh-per-call behavior to assert the singleton reuse instead (and reset the new module global so they stay isolated). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_image_compressor_singleton_reuse.py -q 5 passed # with the fix reverted, all five fail (router rebuilt per call, close() # unloads the shared models, helpers return fresh instances) $ uvx ruff@0.15.17 check headroom/image/compressor.py headroom/proxy/image_isolation.py headroom/proxy/helpers.py tests/test_image_compressor_singleton_reuse.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/image/compressor.py headroom/proxy/image_isolation.py headroom/proxy/helpers.py Success: no issues found in 3 source files ``` The pre-existing async tests in `tests/test_image_compression_isolation.py` (4 `@pytest.mark.asyncio` cases) fail identically on clean `main` in this environment because pytest-asyncio is not configured here; they are unrelated to this change and pass in CI. ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: with `OnnxTechniqueRouter` construction mocked, called `ImageCompressor._get_onnx_router()` twice and asserted a single construction; exercised `close()` on singleton vs non-singleton instances; and called `_get_image_compressor()` / `_get_worker_compressor()` twice each. Then reverted the three source files and re-ran. - Observed result: with the fix the ONNX router is constructed once and reused, singleton `close()` leaves `_router`/`_onnx_router` intact (no `release_models`), non-singleton `close()` releases and nulls them, and both helper accessors return the same `_is_singleton` instance; with the fix reverted every one of these fails (fresh construction / unconditional release / new instances). Ran against the actual modules. - Not tested: a live multi-hour image workload measuring RSS (the leak is inferred from the removed per-request model construction; the ONNX/torch model load itself is mocked here). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Tejas Chopra
|
b121223ec9
|
fix(install): default to cache mode, matching headroom proxy (#1893 follow-up) (#2563)
## Description `headroom install` and `headroom deploy` defaulted `--mode` to **token**, while `headroom proxy` and the server env default both resolve to **cache**. Because `install/planner.py:155` writes `"HEADROOM_MODE": proxy_mode` into the install base env, installing Headroom did not merely differ from running it directly — it **actively overrode** the good server default with the cache-busting one. | Entry point | Effective default | Where | |---|---|---| | `headroom proxy` | **cache** | `cli/proxy.py:1129` — `mode or HEADROOM_MODE or PROXY_MODE_CACHE` | | `proxy/server.py` env | **cache** | `server.py:4962`, commented *"delta-only compression at ~0 prefix-cache busts"* | | `headroom install` / `deploy` | **token** ❌ | `cli/install.py:455,615` | Cache mode freezes prior turns and compresses only the newest delta, so the cached prefix stays byte-identical. Token mode rewrites frozen history, which moves the bytes the provider hashed for its cache key and forces a full cold re-write of the entire prefix. Why that is expensive — measured on 35 local Claude Code sessions (23,018 turns, 8,985M prompt tokens): cache **writes** are ~46% of input spend from just 6.3% of tokens, and 714 warm turns that each re-wrote >100K tokens carried 83% of all warm-path write tokens (~26% of total input spend) at ~452K tokens per event. Full-prefix re-writes are the dominant cost in this workload, and token mode makes them more likely. **This is an oversight, not a deliberate divergence.** #1893 ("ship the coding profile as Headroom's out-of-box default posture") introduced the cache default but its diff touched only `agent_savings.py`, `cli/proxy.py`, and `proxy/server.py` — verified with `git show |
||
|
Parideboy
|
045f3dfe6f
|
fix(install): use CREATE_NO_WINDOW instead of DETACHED_PROCESS on Windows (#2527)
## Description On Windows, the detached agent process spawned by `install hook ensure` (and the `install restart` self-spawn) pops up a visible black console window repeatedly, because `DETACHED_PROCESS` makes `CREATE_NO_WINDOW` a no-op per the Win32 process-creation-flags docs. #2521 ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) ## Changes Made - `headroom/install/runtime.py`: `start_detached_agent()` now uses `CREATE_NO_WINDOW` instead of `DETACHED_PROCESS`, combined with `CREATE_NEW_PROCESS_GROUP` (unchanged detach/isolation semantics, window hidden). - `headroom/install/runtime.py`: `_spawn_detached_restart()` now also sets `CREATE_NO_WINDOW` on Windows (previously had no `creationflags` at all on that platform). - `tests/test_install/test_runtime.py`: updated the Windows branch of `test_start_detached_agent_and_run_foreground` to assert the actual `creationflags` value passed to `Popen`, instead of just monkeypatching an unused `DETACHED_PROCESS` attribute. ## Testing - [x] Added/updated tests - [x] Ran full local test suite ``` $ python -m pytest tests/test_install -q ======================= 137 passed, 1 skipped in 48.68s ======================= $ ruff check headroom/install/runtime.py tests/test_install/test_runtime.py All checks passed! $ ruff format --check headroom/install/runtime.py tests/test_install/test_runtime.py 2 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 506 source files ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.11, headroom repo local checkout - Exact command / steps: `python -m pytest tests/test_install/test_runtime.py -q`, plus manual read of `subprocess` Windows creation-flag semantics (`DETACHED_PROCESS` + child console allocation vs `CREATE_NO_WINDOW`) - Observed result: all 25 tests in `test_runtime.py` pass, including the updated assertion that `creationflags == CREATE_NO_WINDOW | CREATE_NEW_PROCESS_GROUP` on the Windows code path - Not tested: did not reproduce the original visible-console-popup repro end-to-end via live Claude Code hook invocation (no environment with the full hook-triggered respawn loop set up in this session); relying on the Win32 docs and the reporter's own local verification of the same flag swap ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
AxelRay
|
d50cfabedc
|
fix(proxy): report deferred Kompress status and promote health from cache (#2564)
## Description When Kompress preload is deferred until first request, startup still logs "not installed" even if ML deps are present. After the model later loads into the module cache, /readyz and /health can keep reporting kompress as unhealthy because reconcile only inspected attached compressor instances. This PR reports deferred startup accurately and promotes health from the live module cache once the model is ready, without starting loads from health checks. Closes #2560 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Treat eager-status `deferred` as installed-but-deferred at proxy startup and log that state instead of "not installed". - Promote `/readyz` and `/health` Kompress readiness from the module-level model cache when attached compressors are missing or not ready. - Keep health inspection free of lazy getters and download side effects. - Add regressions for deferred startup logging and cache-based health promotion. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ PYTHONPATH=/tmp/headroom-2561 python -m pytest tests/test_proxy_health.py tests/test_proxy_eager_preload_bind.py -q -o addopts= 21 passed, 1 warning in 2.73s $ ruff format --check headroom/proxy/server.py tests/test_proxy_health.py tests/test_proxy_eager_preload_bind.py 3 files already formatted $ ruff check headroom/proxy/server.py tests/test_proxy_health.py tests/test_proxy_eager_preload_bind.py All checks passed! ``` ## Real Behavior Proof - Environment: Linux VPS, Python 3.11 venv with headroom-ai 0.32.1 wheel for `_core`, checked out main + this branch overlayed for source under test - Exact command / steps: `PYTHONPATH=/tmp/headroom-2561 python -m pytest tests/test_proxy_health.py tests/test_proxy_eager_preload_bind.py -q -o addopts=`; `ruff format --check` and `ruff check` on the three changed files - Observed result: 21 focused tests passed, including deferred startup log regression and module-cache health promotion; ruff format/check clean - Not tested: live multi-request proxy with real ONNX model download on this host; install-status follow-up mentioned in the issue comment ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Scoped to Kompress status reporting only. The separate `headroom install status` ownership probe in the issue comment is left for a follow-up. Co-authored-by: axelray-dev <axelray-dev@users.noreply.github.com> |
||
|
AxelRay
|
4bd121493d
|
fix(proxy): allow request_scope import without fastapi (#2562)
## Description Base installs without the `proxy` extra crash during CLI command registration because `headroom.proxy.request_scope` imported FastAPI at module import time. That import is only needed for typing on `normalize_request_path`. This change keeps the FastAPI `Request` import under `TYPE_CHECKING` so the CLI path used by `headroom --help` no longer requires FastAPI. Closes #2561 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation update - [ ] Refactoring (no functional changes) - [ ] Performance improvement - [ ] Test update - [ ] Build/CI change - [ ] Other (please describe): ## Changes Made - Make the FastAPI `Request` import type-checking only in `headroom/proxy/request_scope.py` - Add a subprocess regression test that imports `request_scope` and `project_context` with FastAPI blocked and verifies `normalize_scope_path` ## Testing ### Test commands run ```bash PYTHONPATH=. python3 -m pytest tests/test_proxy_request_scope.py tests/test_request_scope_no_fastapi.py -q ruff format --check headroom/proxy/request_scope.py tests/test_request_scope_no_fastapi.py ruff check headroom/proxy/request_scope.py tests/test_request_scope_no_fastapi.py ``` ### Test Output ```text ========================= 5 passed, 1 warning in 0.93s ========================= 2 files already formatted All checks passed! ``` ## Real Behavior Proof ### Environment - Linux x86_64, Python 3.11.15 - Shallow sparse checkout of headroom main at commit parent of this PR - System/Hermes venv Python with pytest and ruff available ### Exact command ```bash PYTHONPATH=. python3 - <<'PY' import builtins, sys real = builtins.__import__ def imp(name, *a, **k): if name == "fastapi" or name.startswith("fastapi."): raise ModuleNotFoundError("No module named 'fastapi'") return real(name, *a, **k) builtins.__import__ = imp import headroom.proxy.request_scope as rs import headroom.proxy.project_context as pc rs.normalize_scope_path({"path": "/a"}, "/b") print("ok", "fastapi" not in sys.modules, hasattr(pc, "with_project_prefix")) PY ``` ### Observed result ```text ok True True ``` Importing the request-scope helpers no longer requires FastAPI, and scope path normalization still works. ### Not tested - Full base `pip install headroom-ai` (no extras) end-to-end on a clean venv without the monorepo source tree - Full monorepo `make ci-precheck` / cargo workspace - Live proxy traffic or FastAPI request path behavior beyond the existing unit test for `normalize_request_path` ## Review Readiness - [x] I have tested these changes locally - [x] I have added/updated tests where applicable - [x] I have updated documentation if needed (N/A) - [x] My code follows the project's style guidelines - [x] I have run linting/formatting checks - [x] I have considered security implications - [x] This PR is ready for review |
||
|
Tejas Chopra
|
58555c5be0
|
docs(configuration): document cold-prefix hook flags + bound the TTL observation log (#2557)
## Description Follow-up to #2555. Documents the cold-prefix hook / reasoning-compaction / cache-TTL-learner flags (what to set for what, and whether each can be on by default), and makes two small safety fixes so the learning seam is production-ready and free when off. ## Type of Change - [x] Documentation update - [x] Performance improvement (learning seam is now free when disabled) ## Changes Made - **docs/content/docs/configuration.mdx** — env-var table rows for `HEADROOM_THINKING_COMPACT` (+`_KEEP_LAST`), `HEADROOM_COLD_RECOMPACT`, `HEADROOM_DEDUPE`, `HEADROOM_CACHE_TTL_LEARN`, `HEADROOM_KOMPRESS_ENDPOINT`, plus a **Cold-prefix hook & reasoning compaction** section: what to set for what, how cold detection reads the real TTL (CC config vs learned), and a per-flag "can this be on by default?" analysis. - **docs/content/docs/cache-optimization.mdx** — a cold-prefix recompaction section linking to the flags. - **headroom/cache/ttl_observations.py** — the observation log is now size-bounded (single-backup rotation) and respects `HEADROOM_STATELESS`. - **headroom/proxy/handlers/openai.py** — the extra `classify_cache_miss` attribution is gated behind `observations_enabled()` so it costs nothing when learning is off. Everything remains **off by default**. ## Testing - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] Manual testing performed (module self-check) ### Test Output ```text $ ruff check headroom/cache/ttl_observations.py headroom/proxy/handlers/openai.py All checks passed! $ mypy headroom/cache/ttl_observations.py headroom/proxy/handlers/openai.py Success: no issues found in 2 source files $ python headroom/cache/ttl_observations.py ttl_observations self-check OK ``` ## Real Behavior Proof - Environment: local repo, Python 3.12 venv. - Exact command / steps: ran the module self-check (covers gated-off no-write, gated-on write, learned-table read with model→provider fallback) and ruff+mypy. - Observed result: self-check passes; when `HEADROOM_CACHE_TTL_LEARN` is unset no file is written; when `HEADROOM_STATELESS` is truthy no file is written; the observation log rotates to `.1` past the size cap. - Not tested: live multi-turn provider run (unchanged from #2555, which carried the live Kimi/CC proofs); docs render is Markdown/MDX only. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] New and existing checks pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - Default-on stance (in the docs): `THINKING_COMPACT` stays opt-in (rewrites model inputs); `COLD_RECOMPACT` is a candidate to default for Claude Code once TTL detection is field-validated; `CACHE_TTL_LEARN` is the safest to default on (observation-only, bounded, stateless-aware) — kept opt-in for now. |
||
|
Tejas Chopra
|
cb8f4b6436
|
feat(proxy): model-aware cold-prefix hook — reasoning compaction (Kimi/GLM) + cold recompaction (CC) (#2555)
## Description
Adds a **model-aware cold-prefix cache-miss hook** plus **plain-text
reasoning compaction**, both off by default behind flags. Motivation:
prior-turn reasoning and stale prefix content are re-sent and (for some
models) re-billed every turn; when the prompt cache has lapsed,
rewriting the prefix is free. What we do depends on the model's
reasoning shape.
| | plain-text reasoning (Kimi/GLM/DeepSeek) | encrypted reasoning
(Claude/Codex) |
|---|---|---|
| **warm turn** | Kompress reasoning (deterministic → cache-stable) |
leave it (encrypted; can't shrink) |
| **cold turn** | drop the full reasoning block | dedupe + drop
superseded reads (recompact whole prefix) |
Closes #
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
- [x] Performance improvement
## Changes Made
- `headroom/transforms/thinking_compactor.py` (new): shape-driven
reasoning compaction for the OpenAI-chat path — Kimi `reasoning_content`
field + GLM/DeepSeek inline `<think>` spans; deterministic memoized
Kompress (warm) or drop (cold); `keep_last_turns` protects the active
reasoning; no-ops on encrypted-reasoning models.
- `headroom/transforms/cold_prefix.py` (new): the cold-decision surface
— `is_cold_prefix` (idle > TTL + margin), `has_plaintext_reasoning`,
`cold_recompact_messages` (lossless whole-prefix dedupe/superseded), and
`anthropic_cache_ttl_seconds` (reads CC's **real** cache TTL from
request `cache_control.ttl` + `DISABLE_/ENABLE_/FORCE_PROMPT_CACHING_*`
env controls instead of a hardcoded 300s guess).
- `headroom/proxy/handlers/openai.py`: PRE_SEND reasoning compaction
(warm Kompress / cold drop).
- `headroom/proxy/handlers/anthropic.py`: cold-prefix recompaction —
token mode via `frozen_message_count=0`, and **cache mode** via a
whole-prefix lossless recompaction that skips the byte-identical
splice/overlay on a confirmed-cold turn; cold decision uses CC's real
TTL.
- Flags (all off by default): `HEADROOM_THINKING_COMPACT` (+
`HEADROOM_THINKING_COMPACT_KEEP_LAST`), `HEADROOM_COLD_RECOMPACT`.
## Testing
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality (module self-checks)
- [x] Manual testing performed (live provider calls)
### Test Output
```text
$ ruff check headroom/transforms/cold_prefix.py headroom/transforms/thinking_compactor.py \
headroom/proxy/handlers/openai.py headroom/proxy/handlers/anthropic.py
All checks passed!
$ mypy <same 4 files>
Success: no issues found in 4 source files
$ python headroom/transforms/cold_prefix.py
cold_prefix self-check OK
$ python headroom/transforms/thinking_compactor.py
thinking_compactor self-check OK
```
## Real Behavior Proof
- Environment: live Kimi K2.7 via Fireworks
(`accounts/fireworks/models/kimi-k2p7-code`) + real Modal Kompress
endpoint; Claude models via Anthropic API.
- Exact command / steps: 2-turn replay — turn 1 produces reasoning; turn
2 re-sends it through the transform; compare `usage.prompt_tokens`.
- Observed result:
- Kimi reasoning resend is real, billable plain text: WITH reasoning =
2,643 vs WITHOUT = 1,085 input tokens (+1,558/block).
- Warm Kompress (real Modal endpoint): 2,427 → 2,190 prompt_tokens.
- **Cold drop: 2,330 → 714** (= none-baseline; full block removed).
- opencode confirmed to resend `reasoning_content` across turns (real
`opencode run` trace).
- Cold recompaction (dedupe/superseded) on a real 3.4M-token Claude Code
prefix: ~3.7%.
- CC TTL detection self-check pins the bug: `is_cold_prefix` at
idle=400s is `False` under the real 1h TTL (safe) but `True` under the
old 300s guess (would bust a warm cache).
- Not tested: Codex/Responses API path (deferred — no prefix tracker
there, encrypted reasoning); cross-provider live cache-mode cold turn on
a real >TTL idle gap (measured on captured prefixes instead).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my feature works (module
self-checks)
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
- All behavior is flag-gated and off by default — zero change for
existing users.
- Follow-ups: (1) Codex/Responses API wiring (gap #2, deferred); (2) a
cross-provider cache-TTL learner (estimate real TTL per provider from
JSONL cache-bust observations) so Kimi/OpenAI cold detection is
empirical rather than the 300s fallback — candidate for an enterprise
plugin.
|
||
|
Tejas Chopra
|
a6d4921e82
|
feat(proxy/hooks): run fold-only (stream-safe) turn hooks on streaming OpenAI chat (#2549)
## Description Fixes the last harness gap in the turn-hook seam (the "B4" finding from the savings audit). The OpenAI chat handler gated hooks on `not stream`, so **streamed** `/v1/chat/completions` requests ran **no** turn hooks — the lossless-guard plugin's on_request fold and tool-schema shrink were skipped, unlike the Anthropic path (hooks run unconditionally). Affects opencode / Cursor / older OpenAI SDKs / some Copilot flows; **not** Claude Code (Anthropic path). The gate existed for a real reason: hooks that **re-drive** the model in `on_response` (defer a tool, reload it when asked) can't run mid-stream. But an **on_request fold** mutates the outbound request before the send — safe on a stream. ## Change - Add an opt-in `stream_safe` hook attribute (fold-only hooks set it). `run_request_hooks(ctx, stream_safe_only=…)` filters to stream-safe hooks when set. - OpenAI chat handler runs `on_request` on streaming with `stream_safe_only=stream`; buffered runs all hooks; the `on_response` re-drive (buffered response path) is untouched. - **Default off = conservative:** a hook is buffered-only unless it declares `stream_safe`, so **no behavior change** until a hook opts in. ## Type of Change - [x] Bug fix / feature (opt-in, backward-compatible) ## Testing ```text pytest tests/test_turn_hooks.py tests/test_openai_chat_turn_hooks.py -q → 25 passed ruff + mypy → clean ``` New test pins the filter: streaming runs only stream-safe hooks' on_request; buffered runs all. ## Notes The companion plugin PR (headroom-lossless-guard) sets `stream_safe = True` on its fold-only hook to actually claim the streaming savings. Anthropic path already ran hooks on streaming, so it's unaffected. ## Checklist - [x] Self-reviewed; tests pass; no CHANGELOG edit |
||
|
Tejas Chopra
|
c990cfb803
|
feat(wrap): reduce-at-source — SAFE quiet-CLI env defaults for the launched agent (#2548)
## Description Reduce-at-source, done **safely** in the wrap layer (not by rewriting commands in-flight): `headroom wrap` injects conservative quiet-CLI env defaults into the launched agent's environment so tools emit less noise at the source (which the proxy would otherwise strip post-hoc). Injected only when the user hasn't set them: `GIT_PAGER=cat`, `PIP_QUIET=1`, `PIP_DISABLE_PIP_VERSION_CHECK=1`, `npm_config_fund/audit/progress=false`; `PYTEST_ADDOPTS` **augmented** with `-q` (existing value preserved). Single chokepoint (`_launch_tool`), so it covers all wrapped tools. Opt out with `HEADROOM_WRAP_QUIET=0`. Closes # ## Type of Change - [x] Performance improvement / [x] New feature (opt-out) ## Safety Nothing that can suppress diffs, errors, summaries, or search results — no blanket `--silent`/`--quiet`. User-set values always win. ## Testing ```text pytest tests/test_wrap_quiet_cli.py → 5 passed (defaults injected; user value wins; PYTEST_ADDOPTS augmented; opt-out; on-by-default) ruff + mypy → clean ``` ## Scope note (honesty) A JSONL analysis of real Claude Code traffic shows this is a **modest** lever for that workload: non-TTY git already disables the pager (so `GIT_PAGER` is largely a no-op there), and pip/npm are low-traffic; `PYTEST_ADDOPTS=-q` is the clearest win. It's harmless and captures modest savings where those tools *are* used — the larger levers are post-output (the lossless-guard lossy tier) and the grep fold. ## Checklist - [x] Self-reviewed; tests pass; no CHANGELOG edit |
||
|
Tejas Chopra
|
7dc9a978ca
|
feat(lossless): factor shared directory prefix in the grep search fold (#2547)
## Description
The lossless search fold (`search_heading`) factors a repeated **file**
(many matches in one file → path once + `line:content` rows), but `grep
-rn` across many **distinct** files has one match each, so it saved ~0%
— the shared directory repeated on every row. This adds
`search_dir_heading`/`search_dir_unheading`, which factor the shared
**directory** across distinct files (dir once as a header,
`base:line:content` beneath). `compact_lossless('search')` now tries
both folds and keeps the smallest that round-trips exactly.
Matters because grep is ~23.5% of observed agent output tokens.
Closes #
## Type of Change
- [x] Performance improvement (lossless)
## Changes / Behavior
- File fold wins many-matches-one-file; dir fold wins the `grep -rn`
case (0% → ~16-40% depending on path depth / match length).
**Byte-lossless** — round-trip verified, fold discarded on any mismatch.
- Never touches source reads / diffs (unchanged class gating).
## Testing
```text
pytest tests/test_bash_search_lossless_fold.py -q → 30 passed
pytest test_lossless_excluded_compaction / _then_lossy / _mode → 72 passed
ruff + mypy → clean
```
Round-trip verified on: distinct-files (sorted), many-matches-one-file,
mixed+passthrough, colon-in-content.
## Note for reviewers
The dir-grouped output is byte-lossless but a slightly **non-standard**
format the model reads directly (`dir/` header + `base:line:content`) —
like the existing `rg --heading` fold but less standard. Low
comprehension risk; flagging it explicitly. If preferred, we can gate it
to only fire above a larger savings threshold.
## Checklist
- [x] Self-reviewed; tests pass; no CHANGELOG edit
|
||
|
Tejas Chopra
|
9f1ffefe83
|
feat(proxy/savings): aggregate tool-schema savings into Metrics + all reporting sinks (#2546)
## Description Companion to #2545 (the "sources" double-count fix) — this fixes the "sinks" half found in the same savings audit: **tool-schema / deferral savings were never aggregated into `Metrics`**. They lived only in per-request log tags, so every sink that reads `metrics.*` silently dropped them, and one CLI mode disagreed with another. Confirmed sinks that under-reported: - **Session-summary printout** — `Tokens saved:` is message-only; a 24K-tool-deferral turn printed `0`. - **`cost.py` session summary** (feeds `/stats.summary`) — `total_tokens_saved_with_rtk` etc. were message+CLI only. - **`/stats` `all_layers_tokens_saved`** — the advertised "total" excluded the `tool_search` layer it enumerates in `by_layer`. - **`headroom perf --format json/csv`** — omitted `tool_saved` while the **text** output of the same command showed it. Closes # ## Type of Change - [x] Bug fix (non-breaking) / observability correctness ## Changes Made - `PrometheusMetrics.tool_search_saved_total` — new counter, accumulated in `record_request` from a new `tool_search_saved` arg; `emit_request_outcome` fills it from the `tool_search_deferred_tokens` + `turn_hook_tools_saved_tokens` tags. **One source of truth.** - Fed into: session summary (`Tool schemas deferred:` line), `cost.py` summary (new `tool_schema_tokens_saved` + `total_tokens_saved_all_layers`; existing fields unchanged for back-compat), `/stats` `all_layers` total, and `build_perf_summary` (`tool_saved`). - Kept **distinct** from `tokens_saved_total` (message compression) — tool bytes never move `tok_before/after`, so it's a separate layer, not a merge (no double-count). ## Testing - [x] `ruff` + `ruff format --check` + `mypy` clean - [x] Regression tests + existing suites pass ### Test Output ```text pytest tests/test_savings_tool_search_aggregation.py tests/test_cli_perf_format.py -q → 18 passed pytest tests/test_cli_perf_format.py test_proxy_savings_history.py test_dashboard_token_savings.py test_bundled_tools_savings.py test_openai_chat_turn_hooks.py → 68 passed, 2 skipped mypy (metrics/outcome/cost/analyzer) → clean ``` ## Real Behavior Proof - Standalone: `record_request(tool_search_saved=1500)` then `(…=800)` → `metrics.tool_search_saved_total == 2300`, `tokens_saved_total == 200` (message stays separate); `build_perf_summary` over records with `tool_saved` 5000+3000 → `tool_saved == 8000`. ## Checklist - [x] Self-reviewed; no new warnings; tests pass; did **not** edit `CHANGELOG.md` ## Additional Notes Together, #2545 (record once) + this (surface every layer) make savings correct **and** complete end-to-end across `/stats`, the dashboard, `headroom perf`, the session summary, and cost/budget. The `/stats` `by_layer.tool_search` and dashboard card already showed the layer (windowed, from the log scan); this makes the lifetime/metrics-based sinks agree. |
||
|
Tejas Chopra
|
0845b26ee6
|
fix(proxy/cost): record each request's savings exactly once (drop 3 double-counts) (#2545)
## Description An audit of savings accounting found three **double-count** bugs: the P0 outcome-funnel refactor centralized cost + PERF recording in `emit_request_outcome`, but three pre-funnel emits were never removed, so they fire a second time on their paths. | Path | Stray emit | + Funnel | Effect | |---|---|---|---| | OpenAI chat direct, non-streaming | explicit `cost_tracker.record_tokens` (`handlers/openai.py` ~4140) | `outcome.py:418` | **2× spend / requests; budget period cost doubled** → `check_budget` can block at half the real spend | | OpenAI **Responses** buffered (Codex HTTP) | explicit `record_tokens` (~5223) | `outcome.py:418` | same | | Codex **WS** turns | explicit `PERF` log line (~7291) | `outcome.py:482` | `headroom perf` **double-counts** saved + requests every WS turn (analyzer sums per line, no dedup by request_id) | All three are pure duplicates: the funnel's `cost_tracker.record_tokens` is a **superset** of the explicit calls' args, and its PERF line uses the **same per-turn deltas** (verified: `7246-7249` == the explicit line's fields). The `/stats` headline was already correct (SavingsTracker fires once, inside the funnel) — only cost/budget and `headroom perf` were affected. Closes # ## Type of Change - [x] Bug fix (non-breaking) ## Changes Made - Remove the explicit `cost_tracker.record_tokens` on the OpenAI chat non-streaming path and the Responses buffered path — keep the `cache_write`/`uncached` computation the funnel needs. - Remove the duplicate WS PERF log line (+ its now-dead `_perf_*` locals and the now-unused `_summarize_transforms` import). - Add a regression test: cost is recorded exactly once on the non-streaming chat path (was 2×). ## Testing - [x] `ruff check` + `ruff format --check` clean; `mypy` clean - [x] Regression + existing tests pass ### Test Output ```text pytest tests/test_openai_chat_turn_hooks.py -q → 6 passed (incl. new double-count regression) pytest tests/test_openai_responses_context_compaction.py → 12 passed pytest tests/test_openai_codex_ws_lifecycle.py + timings + savings_deferral → 38 passed ruff/mypy → clean ``` ## Real Behavior Proof - **Verified by code trace**, not just tests: `grep cost_tracker.record_tokens` across the handler now returns only the funnel call (`outcome.py:418`); the explicit chat/Responses calls are gone. The WS funnel outcome (`openai.py:7246-7249`) feeds `outcome.py:482`'s PERF with the same deltas the deleted line used. - **Not covered:** a related finding (OpenAI-chat *streaming* skips turn hooks entirely, `openai.py:3484 "and not stream"`) is **intentionally deferred** — that gate protects re-drive-requiring hooks (tool-router deferral) which can't run mid-stream; a proper fix needs a per-hook "safe-on-stream" capability flag, out of scope here. ## Checklist - [x] Self-reviewed - [x] No new warnings; tests pass locally - [x] Did **not** edit `CHANGELOG.md` ## Additional Notes This is the "sources" half of the savings audit. A companion PR will fix the "sinks" half — tool-search/deferral savings are never aggregated into `Metrics`, so the session summary, `cost.py` summary, `headroom perf --json/csv`, and the `all_layers` total under-report them. |
||
|
Tejas Chopra
|
285176be54
|
fix(tokenizers): price Claude against a real BPE (tiktoken o200k) not a char estimate (#2543)
## Description
Claude's tokenizer is not public, so `get_tokenizer("claude-*")`
returned `EstimatingTokenCounter(chars_per_token=3.5)` — a
**character-ratio estimate**. The estimator's chars-per-token flips with
the *detected content type* (JSON 3.2 / code 3.5 / English 4.0 / CJK
1.5), so:
- the **same bytes** count differently before vs after compression → a
fold could appear to *increase* tokens;
- it **disagrees with the pipeline's** estimator on the same content.
Calibration vs a real BPE (tiktoken `o200k_base`) on representative
content:
| content | char-est(3.5) / o200k |
|---|---|
| English | **1.28×** (overcount 28%) |
| code | **0.83×** (undercount 17%) |
| JSON | 0.97× |
| diff | 1.02× |
i.e. the estimate is off **−17% … +28%**, and the error is
content-dependent (non-uniform) — which is exactly what produces
impossible `tok_after > tok_before` deltas.
This prices Claude against **tiktoken `o200k_base`** — a real,
deterministic, **monotone** BPE. It's not Claude's exact vocab, but it's
within ~10–20% and, crucially, **consistent before/after**, which is
what compression ratios and context-pressure gating actually need.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement
- [ ] Breaking change
- [ ] Documentation update
- [ ] Code refactoring (no functional changes)
## Changes Made
- `TiktokenCounter.__init__` accepts an explicit `encoding` override (so
a model can be priced against a chosen encoding regardless of name-based
resolution).
- `_create_anthropic` now returns `TiktokenCounter(model,
encoding="o200k_base")`, **failing open to the character estimator** if
the tiktoken vocab can't be loaded (reuses the existing `load_encoding`
timeout/guard).
- Updated the `MODEL_PATTERNS` comment and `test_get_anthropic_model` to
the new contract; added
`test_claude_priced_with_real_bpe_not_char_estimate`.
**Blast radius is contained:** `content_router` keeps its own
module-level estimator for routing decisions, so only the **handler's
reported counts** change. `tiktoken` is already a dependency.
**Composes with #2542** (tokenizer-consistent before/after): that PR
makes both endpoints use *one* tokenizer; this PR makes that one
tokenizer a *real BPE*. Together they fully eliminate the
inflated/phantom lines. This PR is based on `main` and is independently
valid.
## Calibration note (please review)
The one behavioral consumer of the handler's count is the
background-compression gate (`original_tokens >=
_background_compression_min_tokens`). Because o200k differs from the old
estimate by ~±20% depending on content, that threshold now trips on
slightly different requests. It's *more* accurate, but the threshold was
tuned against the estimator — worth a re-check. No other gate consumes
the handler count (routing uses `content_router`'s estimator).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check` + `ruff format --check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added
- [x] Manual testing performed
### Test Output
```text
$ ruff check <changed files> && ruff format --check <changed files>
All checks passed!
4 files already formatted
$ mypy headroom/tokenizers/registry.py headroom/tokenizers/tiktoken_counter.py
Success: no issues found in 2 source files
$ pytest tests/test_tokenizer.py tests/test_tokenizers.py -q
55 passed, 14 skipped
```
## Real Behavior Proof
- **Unit:** `get_tokenizer("claude-opus-4-8")` →
`TiktokenCounter(encoding_name="o200k_base")`; `count_text(sample)` ==
`tiktoken.get_encoding("o200k_base").encode(sample)` exactly; a
`tool_result` shrunk 300→3 words drops 508→13 tokens (folds register).
- **Live proxy** (Anthropic path, `--proxy-extension lossless_guard`):
foldable request logs `tok_before=694 tok_after=231 tok_saved=463
transforms=turn_hook` — real o200k-scale numbers (vs the estimator's 607
for the same payload), clean fold, no inflation.
- **Not tested:** exact agreement with Anthropic's *own* count_tokens
API (o200k is a proxy; a follow-up could calibrate against it offline).
GPT paths unchanged (already tiktoken).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] Documentation — N/A (internal; docstrings updated)
- [x] My changes generate no new warnings
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Follow-ups (not here): (1) extend the same real-BPE treatment to other
private-tokenizer providers (Gemini/Cohere/Moonshot), each with its own
calibration; (2) optional opt-in calibration against Anthropic's
`count_tokens` API to true-up the ~10–20% absolute offset.
|
||
|
Tejas Chopra
|
1cc53c9c92
|
fix(proxy/perf): tokenizer-consistent token accounting + surface tool-schema savings (#2542)
## Description Follow-up to #2520 (turn-hook message-fold accounting). While validating that PR on live Claude Code traffic, two accounting defects surfaced: 1. **Impossible/misleading token deltas.** The handler and the compression pipeline use *different* token estimators — the handler's `EstimatingTokenCounter(3.5)` (or real tiktoken on OpenAI) vs `content_router`'s adaptive `EstimatingTokenCounter()`. Cross-assigning `original_tokens` (handler) against `optimized_tokens = result.tokens_after` (pipeline) put the two endpoints on different scales, producing **`tok_after > tok_before` on 101/783 PERF lines** and phantom savings on `transforms=none` lines. It also made the turn-hook recount fire on the *scale difference* rather than a real fold, emitting a **spurious `turn_hook` tag with `tok_saved=0`**. 2. **Tool-schema savings were invisible.** Tool deferral (`defer_loading`) and turn-hook tool shrink save thousands of tool-schema tokens, but `tok_before`/`tok_after` count **messages only** — so a tool-heavy turn logged `tok_saved=0` while genuinely saving ~29k tool-schema tokens, reading as "no compression." Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Performance improvement - [ ] Breaking change - [ ] Documentation update - [ ] Code refactoring (no functional changes) ## Changes Made - **Tier B.1 — tokenizer-consistent accounting.** On the Anthropic and OpenAI-chat paths, recount **both** endpoints with the **same** tokenizer (pre-compression snapshot vs final outbound messages) right before the outcome is recorded. This puts `tok_before`/`tok_after` on one scale (fixes the inflated + phantom lines), and it subsumes any turn-hook fold. `turn_hook` is now attributed **only** when the hook itself reduced tokens (same-tokenizer pre vs post), not when a recount merely normalized a scale difference. OpenAI preserves its existing tool-schema delta folding. - **Surface tool-schema savings.** New `tool_saved=` field on the PERF line (summed from `tool_search_deferred_tokens` + `turn_hook_tools_saved_tokens` tags) and a separate `Tool saved` line in `headroom perf`. Additive + backward-compatible (key=value parse; old lines default to 0). `tok_saved` still means message savings, so ratios and calibrated thresholds are unaffected. - The OpenAI **Responses** path already accumulates per-transform deltas (each consistent within its own transform), so it isn't exposed to the cross-scale subtraction bug and needs no change. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check` + `ruff format --check`) - [x] Type checking passes (`mypy`) - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py \ headroom/proxy/outcome.py headroom/perf/analyzer.py All checks passed! $ ruff format --check <same files> 4 files already formatted $ mypy <same files> Success: no issues found in 4 source files $ pytest tests/test_turn_hooks.py tests/test_openai_chat_turn_hooks.py \ tests/test_openai_responses_context_compaction.py -q 26 passed in 14.34s ``` ## Real Behavior Proof - **Environment:** local proxy `headroom proxy --port 8793 --proxy-extension lossless_guard`, with `HEADROOM_LOSSLESS_GUARD_LOSSY=1`, `HEADROOM_TOOL_SEARCH=1`, model `claude-haiku-4-5` (Anthropic path = the one Claude Code uses). - **Observed, before vs after this PR:** - foldable tool_result: `tok_before=607 tok_after=178 tok_saved=429 transforms=turn_hook` (real fold, correctly attributed) - plain multi-turn (nothing foldable): `tok_before=3700 tok_after=3700 tok_saved=0 transforms=none` — **no inflation, no spurious `turn_hook`** (before this PR: same request showed a spurious `turn_hook`) - tool-heavy (12 tools): `tok_saved=0 tool_saved=1794` → `headroom perf` shows `Total saved: … (messages)` **and** `Tool saved: 1,794 tokens (tool schemas, deferral)` (before: the 1,794 was invisible) - **Not tested:** OpenAI chat/Responses paths verified by unit test, not a live client run (local setup routes Claude Code through the Anthropic handler only). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] Documentation changes — N/A (internal accounting; `tool_saved` is self-describing in `headroom perf`) - [x] My changes generate no new warnings - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - Behavior is unchanged when no turn hook is registered for the *attribution* tag; the consistency recount runs unconditionally so pure-OSS installs also get correct before/after (it only ever makes the two endpoints comparable — it never fabricates savings). - Follow-up (separate PR, intentionally not here): swap the char-estimator for a real BPE (tiktoken `o200k_base`) for private-tokenizer models like Claude. That's the "Tier B.2" accuracy upgrade; it shifts absolute numbers ~10–20% and touches calibrated thresholds, so it needs its own recalibration pass. |
||
|
Abhay Singh
|
fa4763761b
|
fix(proxy/cost): warn once per model when pricing lookup fails (#2504) (#2535)
## Description Fixes #2504. `CostTracker.estimate_cost` runs on the per-request cost path and logs a WARNING whenever LiteLLM can't price the model: ```python except Exception as e: logger.warning(f"Failed to get pricing for model {model}: {e}") return None ``` For a custom / OpenAI-compatible model LiteLLM can't resolve (e.g. `glm-5.2` via `--backend anyllm --anyllm-provider openai`), this fires on **every single request**, flooding `proxy.log` with hundreds of identical lines and burying genuinely useful warnings. The `LiteLLM not available` branch above it has the same per-request flooding shape. ## Fix Track already-warned models in a small module-level set and emit each pricing-failure warning (and the LiteLLM-unavailable warning) once per process. The set is bounded by the number of distinct model names seen. No new dependencies or config. The cost result itself is unchanged (`None` on failure); only the log volume changes. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/cost.py`: add a module-level `_warned_pricing_models` set and `_warn_pricing_once` helper; route the pricing-failure and LiteLLM-unavailable warnings in `estimate_cost` through it. - `tests/test_cost_pricing_warning_dedup.py` (new): assert a repeated unresolvable model warns once, distinct models each warn once, and the LiteLLM-unavailable warning is deduped too. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cost_pricing_warning_dedup.py -q 3 passed # with the fix reverted, the module-level set does not exist, so the # dedup tests error/fail (the pre-fix code warned once per request) $ uvx ruff@0.15.17 check headroom/proxy/cost.py tests/test_cost_pricing_warning_dedup.py All checks passed! $ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/cost.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, project venv (`uv sync --extra proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv. - Exact command / steps: monkeypatched `_get_litellm_module` to a stub whose `cost_per_token` raises (and, separately, to `None`), called `CostTracker.estimate_cost("glm-5.2", ...)` five times and two distinct unresolvable models twice each, capturing `headroom.proxy` WARNING records with `caplog`. - Observed result: with the fix each model produces exactly one `Failed to get pricing for model ...` warning (and one `LiteLLM not available ...`) regardless of call count; the pre-fix code logged one per call. `estimate_cost` still returns `None` on failure. Ran against the actual module. - Not tested: a live multi-request session against a real unpriced model end to end. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Tejas Chopra
|
c371d5ad60
|
fix(proxy/perf): count turn-hook message folds in token accounting (#2520)
## Description
Turn hooks (the `headroom.proxy.turn_hooks` seam used by proxy
extensions, e.g. the lossless-guard plugin) fold tool_result / message
content in `on_request`, which runs **after** the pipeline has already
computed `optimized_tokens`. The saving was recorded to `/stats` via
`record_compression`, but was invisible to the `PERF` log line and
`headroom perf` (both read the pipeline's `original → optimized` delta).
Net effect: a plugin that folded 463 tokens still logged `tok_saved=0`.
This makes the per-turn token accounting count the hook's fold too,
across all three handler paths.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **Anthropic Messages handler** (`/v1/messages`): re-count messages
right after `run_request_hooks`, regardless of whether the hook replaced
the list or mutated it in place. Attribute the fold as a `turn_hook`
transform. Only ever lowers `optimized_tokens`.
- **OpenAI Chat handler** (`handle_openai_chat`,
`/v1/chat/completions`): same re-count. The existing code re-counted
hook-modified *tools* but not the *message* fold — this closes that gap
and adds the `turn_hook` transform tag.
- **OpenAI Responses handler** (`_compress_openai_responses_payload`,
`/v1/responses`): the seam previously only wrote hook-modified *tools*
back — a folded/replaced `input` list was silently dropped and
uncounted. Now snapshot the message-items token count **before** the
hook (an in-place fold would corrupt a post-hook baseline), write back a
replaced list, and add the fold delta to `tokens_saved` (the same
channel the tool-schema savings already ride to `/stats` and `headroom
perf`).
- Key detail: the identity check `ctx.messages is not <orig>` is
insufficient — the lossless-guard plugin mutates messages **in place**,
so an identity-gated re-count misses it. The re-count runs
unconditionally whenever a hook ran.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ ruff check headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py \
tests/test_openai_chat_turn_hooks.py tests/test_openai_responses_context_compaction.py
All checks passed!
$ mypy headroom/proxy/handlers/anthropic.py headroom/proxy/handlers/openai.py
Success: no issues found in 2 source files
$ pytest tests/test_turn_hooks.py tests/test_openai_chat_turn_hooks.py \
tests/test_openai_responses_context_compaction.py -q
tests/test_turn_hooks.py ......... [ 34%]
tests/test_openai_chat_turn_hooks.py ..... [ 53%]
tests/test_openai_responses_context_compaction.py ............ [100%]
26 passed in 14.05s
```
New regression tests (each fails on the pre-fix code):
- `test_in_place_message_fold_is_counted` (chat path) — hook folds
message content in place; asserts `turn_hook` in `x-headroom-transforms`
and a recorded `tokens_saved > 0`.
- `test_responses_turn_hook_message_fold_is_applied_and_counted`
(Responses path) — hook folds a `function_call_output` in place; asserts
the outbound payload reflects the fold **and** `tokens_saved > 0`.
## Real Behavior Proof
- **Environment:** local proxy (`headroom proxy --port 8793
--proxy-extension lossless_guard`), `HEADROOM_LICENSE_DEV=1`,
`HEADROOM_PROTECT_TOOL_RESULTS=Bash` (so the fold is purely the plugin's
turn hook), model `claude-haiku-4-5`. Request carries a `gh --json`
object (folded to TOON) and a `docker pull` log.
- **Exact steps:** send the request → read the `PERF` line in
`~/.headroom/logs/proxy.log` and `GET /stats`.
- **Observed result:**
- Before this change: `PERF ... tok_before=607 tok_after=607 tok_saved=0
... transforms=none` while `/stats` reported `{"lossless_guard": 145}` —
i.e. the saving existed but perf showed nothing.
- After this change: `PERF ... tok_before=607 tok_after=484
tok_saved=123 ... transforms=turn_hook`, `/stats` still
`{"lossless_guard": 145}`. (`123` is the honest whole-request
`count_messages` delta; `145` is the per-content-string delta
`record_compression` measures — different scopes, both real and
positive.)
- **Not tested:** the OpenAI Chat and Responses paths were verified by
unit test, not a live client run — my live setup routes Claude Code
through the Anthropic handler only.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation — N/A
(internal accounting; no public API/doc surface)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Behavior is unchanged when no turn hook is registered
(`registered_turn_hooks() == []` → the re-count block is skipped), so
pure-OSS installs are byte-identical and unaffected. OSS's own pipeline
compression was already counted correctly (it runs before the hook);
this only surfaces the extension/turn-hook layer.
|
||
|
Rod Boev
|
4a8157fa0a
|
fix(copilot): derive GHE credential host from API URL (#800) (#2511)
## Description GHE Copilot credential discovery falls back straight to `github.com` when `GITHUB_COPILOT_HOST` is unset, even if the documented `GITHUB_COPILOT_API_URL` points at an enterprise host. This change keeps explicit-host precedence, then reuses the configured enterprise domain or a normalized custom API URL hostname for credential lookup, so Windows, macOS, Linux, GH CLI, and credential-file discovery search the same custom host instead of the public default. Closes #800. Attribution: https://github.com/headroomlabs-ai/headroom/issues/800#issuecomment-5044382263 narrowed the shared credential-host mismatch. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds new functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - Preserve explicit-host precedence, then fall back to the configured enterprise domain or a normalized custom API URL hostname only when the configured value is usable. - Normalize `api.` and `copilot-api.` prefixes before routing credential lookup, while keeping exact and segmented GitHub-hosted public domains plus public enterprise or malformed enterprise or API configuration fallback on `github.com`. - Add focused coverage for the base/head reproduction, explicit-host precedence, configured-enterprise precedence, public-enterprise, malformed-enterprise, and invalid-port fallback, prefixed-host normalization, adjacent-host exclusion, and GH CLI plus keychain forwarding. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_copilot_auth.py -q`) - [x] Linting passes - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_copilot_auth.py -q 105 passed in 0.58s uvx --from ruff==0.15.17 ruff check headroom/copilot_auth.py tests/test_copilot_auth.py All checks passed! uvx --from ruff==0.15.17 ruff format --check headroom/copilot_auth.py tests/test_copilot_auth.py 2 files already formatted git diff --check clean ``` ## Real Behavior Proof - Environment: Windows, isolated temporary credential file, local `origin/main` checkout plus this branch - Exact command / steps: With only `GITHUB_COPILOT_API_URL=https://api.ghe.example.com:8443/copilot` set and all other token sources disabled, run the same credential-file discovery reproduction against `origin/main` and this branch. - Observed result: `origin/main` selected `github.com` and resolved no token; the review branch selected `ghe.example.com` and resolved `gho-ghe`. - Not tested: live GitHub Enterprise Copilot tenant ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did not edit `CHANGELOG.md`; Headroom generates release notes from the PR title ## Additional Notes The change does not alter API routing, token exchange, discovery order, or credential matching breadth, and it keeps the live tenant claim out of the PR body until an enterprise user reruns it. |
||
|
Rod Boev
|
e4076bbe99
|
fix(grok): preserve business-seat auth while routing only inference (#2514)
## Description `headroom wrap grok` currently routes the whole session through `GROK_CLI_CHAT_PROXY_BASE_URL`. xAI's July 21, 2026 enterprise docs say that host carries both inference and settings, so the wrap displaces the native settings/auth path along with inference. A Grok account whose SuperGrok entitlement lives on a business account can then no longer resolve that seat and falls back to a login screen, even though native `grok` works for the same account. This change retargets the Grok provider slice to the narrower inference-only key, `GROK_MODELS_BASE_URL`, and leaves `GROK_CLI_CHAT_PROXY_BASE_URL` unset. Headroom still intercepts inference and model discovery through the existing `/v1/models` and chat-completions proxy paths, while the native `cli-chat-proxy.grok.com` settings host and `auth.x.ai` auth path stay intact. Closes #2489. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - switch the Grok provider env authority from `GROK_CLI_CHAT_PROXY_BASE_URL` to `GROK_MODELS_BASE_URL` - update the Grok wrap and unwrap docstrings to describe inference-only routing and the preserved native settings/auth path - update the compatibility matrix entry in `README.md` so the public docs match the new Grok routing key - add focused provider and wrap tests that assert the old chat-proxy key is absent and the project-prefixed inference URL is preserved - keep `grok_build` and the existing `/v1/models` proxy route unchanged, using them as preservation boundaries ## Testing - [x] Unit tests pass (`uv run pytest tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py tests/test_provider_grok_build.py -q`) - [x] Linting passes (`uv run ruff check headroom/providers/grok/runtime.py headroom/cli/wrap.py tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py tests/test_provider_grok_build.py -q uv run ruff check headroom/providers/grok/runtime.py headroom/cli/wrap.py tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py uv run ruff format headroom/providers/grok/runtime.py headroom/cli/wrap.py tests/test_provider_grok.py tests/test_cli/test_wrap_grok.py --check ``` ## Real Behavior Proof - Environment: current Grok CLI plus a focused Headroom worktree - Exact command / steps: capture `grok --version`, re-check xAI's documented Grok env contract, run the focused Grok provider and wrap tests, and if a business-seat account is available locally launch `headroom wrap grok` to confirm the wrapped session no longer falls back to login - Observed result: Headroom emits only the inference-routing key, the old settings/auth key is absent, project prefixing still works, and the focused Grok tests pass - Not tested: local business-seat account on this host ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A - CLI and provider-routing change only. ## Additional Notes - `CHANGELOG.md` stays untouched because Headroom's release automation generates it from conventional commits. - The issue is reporter-only today, so the proof report records the validated `grok --version` and whether a real business-seat retest was reached locally or remains for the reporter. |
||
|
inix
|
806d2e468a
|
fix(proxy): offload OpenAI and Gemini tokenizer counting off the event loop (#2498)
## Description The OpenAI and Gemini handlers resolved the tokenizer and counted the conversation inline on the event loop. When a model resolves to a HuggingFace tokenizer (the registry routes qwen, deepseek, llama, phi, falcon, and more there) a cold cache runs `AutoTokenizer.from_pretrained` behind a 10s `thread.join`, which freezes the whole server. That is the GH #1701 stall, now reachable from OpenAI and Gemini because `/v1/chat/completions` and `/v1/responses` are documented multi-provider passthroughs and receive those models. Anthropic already routed the same call through a fail-open `_count_tokens_offloaded` helper. This hoists that helper to the shared `HeadroomProxy` base and sends the OpenAI and Gemini sites through it too. No linked issue. This is the OpenAI and Gemini follow-on to #1738, which offloaded the Anthropic and batch paths. GH #1701 is the original freeze report. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Hoisted `_count_tokens_offloaded` from `AnthropicHandlerMixin` to the shared `HeadroomProxy` base, next to `_run_compression_in_executor`. It resolves and counts on the bounded compression executor and fails open to estimation on timeout, error, or executor quarantine. - Routed 6 inline sites through it: `handle_openai_chat`, `handle_openai_responses`, `handle_gemini_generate_content`, `handle_google_cloudcode_stream`, `handle_gemini_count_tokens`, and `handle_gemini_stream_generate_content` (resolve only, keeps its per-part `count_text` loop). - Removed 6 now-dead local `get_tokenizer` imports. - Left batch's per-line counts inline on purpose. They run on an already-warm tokenizer, so offloading them adds executor churn without touching the cold load. Batch's `pipeline.apply` was already offloaded in #1738. - Extended the wiring guard to all 7 provider handlers, added a quarantine fail-open test and a `count_text` fail-open test, and stubbed the method on 2 mixin-only handler doubles. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/proxy/server.py headroom/proxy/handlers/{anthropic,openai,gemini}.py tests/test_tokenizer_count_offload.py All checks passed! $ pytest tests/test_tokenizer_count_offload.py 6 passed in 4.39s # offload suite + all 26 handler-calling test files + handler-helpers/batch/tokenizers $ pytest tests/test_tokenizer_count_offload.py tests/test_openai_codex_routing.py tests/test_gemini_nonjson_status.py ... tests/test_tokenizers.py 377 passed, 15 skipped, 15 warnings in 86.60s (0:01:26) ``` ## Real Behavior Proof - Environment: macOS (Darwin), Python 3.13, proxy built from this branch. A synthetic tokenizer that sleeps 0.5s on resolve+count stands in for a cold HuggingFace `from_pretrained` load, with a 10ms asyncio loop-canary running alongside. - Exact command / steps: monkeypatch `headroom.tokenizers.get_tokenizer` to the 0.5s-sleeping tokenizer, then time a concurrent canary across two counts, the offloaded `await proxy._count_tokens_offloaded("qwen2.5-coder", messages)` and the old inline `get_tokenizer(model).count_messages(messages)`. - Observed result: the offloaded path kept the loop live at 41 canary ticks during the 509ms count, the inline path froze it to 0 ticks over 502ms, and both returned the same token count. Full run was 377 passed, 15 skipped, 0 failed. The new quarantine test confirms an unrelated compression timeout downgrades counting to estimation instead of raising a 500. - Not tested: live HuggingFace downloads and real qwen/deepseek traffic. No API keys in this environment, so the Gemini and OpenAI integration tests skip on `GEMINI_API_KEY`/`OPENAI_API_KEY`. `mypy headroom` did not finish locally (cold-times-out past 10 minutes on this box), so type-checking is left to CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - No linked issue. Follow-on to #1738. - Batch per-line counts stay inline: they run on an already-warm tokenizer, so offloading them adds executor churn without addressing the cold load. - Found a 6th site mid-implementation. `handle_gemini_stream_generate_content` also resolved the tokenizer inline but counts via a `count_text` loop, so it takes the resolve-only path. Verified `EstimatingTokenCounter.count_text` exists, so its fail-open branch does not crash. - `mypy headroom` cold-times-out locally (server.py pulls the full graph). Deferred to CI's Linux shards, same as prior PRs on this file. `ruff` and `pytest` run clean. - Documentation checkbox left unchecked: this change ships no user-facing doc update. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Tejas Chopra
|
5d23a0aec2
|
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499)
## Description
`tokensave` was a **downloaded third-party Rust binary**
(`aovestdipaperino/tokensave`) that `headroom wrap` registered as a
code-graph MCP server. This removes it entirely and standardises on
**Serena** as the code-memory MCP — which was already the default in
`wrap`. Serena runs on demand via `uvx`, so Headroom no longer downloads
or executes a binary of its own for code memory.
The change is a *removal + safe transition*, not a behaviour flip:
Serena was already the default, so existing users move over
automatically. This PR also folds in a small README repositioning
(Headroom = the proxy; Serena is the recommended companion; RTK/lean-ctx
are third-party tools we don't control), since it's the same
tooling-stack story.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Code refactoring (no functional changes)
## Changes Made
- **Removed the external tool:** `headroom/graph/tokensave_installer.py`
(the download-and-execute path), `build_tokensave_spec`, and the
`_ensure_tokensave_binary` / `_index_tokensave_project` /
`_setup_tokensave_mcp` helpers; dropped the dead `_setup_code_graph`.
- **`--code-memory`** now offers `serena` (default) or `none` — the
`tokensave` choice is gone. The strands `HeadroomBundle` uses Serena as
its (default-on) code-memory MCP.
- **Tombstone / transition (ledger-verified):** `headroom wrap` **and**
`headroom unwrap` remove a previously Headroom-installed `tokensave` MCP
entry so upgrading users stop launching it, and print that the leftover
`~/.local/bin/tokensave` binary and `.tokensave/` folders are safe to
delete. A user-managed `tokensave` entry is left untouched. Mirrors the
existing `codebase-memory-mcp` retirement.
- **Graceful for existing users:** `HEADROOM_CODE_MEMORY=tokensave` and
`--no-tokensave` resolve to Serena instead of erroring; `--no-serena`
now means "no code memory". **No state migration needed** — both tools'
indexes are regenerable caches of the source, so Serena simply
re-indexes.
- **Docs/README:** replaced the "tokensave binary trust model" section
with a Serena note + an "Upgrading from tokensave?" callout; retired the
RTK "first-class part of our stack" framing.
- **Tests:** deleted the tokensave-only test files
(`test_graph_tokensave.py`, `test_cli/test_tokensave_helpers.py`,
`test_cli/test_tokensave_setup.py`), rewrote `test_wrap_code_memory.py`
for the new resolver/dispatch, fixed a codex test that patched a removed
symbol.
Net: **+102 / −1187 lines.**
## Testing
- [x] Unit tests pass (`pytest`) — targeted to the affected areas
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ ruff check headroom/cli/wrap.py headroom/mcp_registry/ headroom/integrations/strands/ tests/test_wrap_code_memory.py tests/test_cli/conftest.py tests/test_cli/test_wrap_codex.py
All checks passed!
$ mypy headroom/cli/wrap.py headroom/mcp_registry headroom/integrations/strands/bundle.py
Success: no issues found in 12 source files
$ pytest tests/test_wrap_code_memory.py tests/test_cli/test_wrap_codex.py \
tests/test_cli/test_wrap_claude_vertex_proxy_env.py \
tests/test_cli/test_wrap_claude_finally_unbound.py -q
======================== 120 passed in 97.86s (0:01:37) ========================
$ pytest tests/test_cli --collect-only -q
========================= 713 tests collected in 1.82s ========================= # no import errors after symbol removal
```
## Real Behavior Proof
- **Environment:** macOS (darwin), Python 3.12.6, local `.venv`, on
branch `tejas/remove-tokensave`.
- **Exact command / steps:**
- `python -c "from headroom.integrations.strands.bundle import
HeadroomBundle; b=HeadroomBundle(enable_headroom_mcp=False,
enable_serena_mcp=False); print(len(b.tools))"` → confirms the module
imports after `build_tokensave_spec` removal (the import that my change
would otherwise break).
- CliRunner-driven `wrap codex --prepare-only` (in `test_wrap_codex.py`)
writes `[mcp_servers.serena]` (with `command = "uvx"`, `"--context",
"codex"`) to the codex config and **no** tokensave entry.
- `_resolve_code_memory` unit tests confirm: default → `serena`;
`HEADROOM_CODE_MEMORY=tokensave` → `serena`; `--no-serena` → `none`;
`--code-memory bogus` → `ClickException`.
- **Observed result:** import OK (`tools: 0`); Serena registered,
tokensave absent; resolver behaves as above; 120/120 tests pass.
- **Not tested:** a live `headroom wrap` against a real agent on a
machine with a *previously-installed* tokensave MCP entry — the
tombstone-removal path is covered by unit tests with a fake
registrar/ledger, not an end-to-end run.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title
## Additional Notes
- `--no-tokensave` / `--serena` / `--no-serena` are retained as hidden,
deprecated flags (no-ops or mapped) so existing scripts don't break.
- `--code-graph` is unchanged — it's the proxy's live file-watcher flag
and was never the tokensave MCP; only the dead tokensave hook behind it
was removed.
- `CHANGELOG.md` intentionally left untouched.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
Rod Boev
|
5bd2266f16
|
fix(kompress): raise the default execution-slot wait (#2456)
## Description Concurrent Kompress requests currently fail open after a 25 ms execution-slot wait even though ordinary ONNX inference can hold the single slot for hundreds of milliseconds. This raises the existing default wait to 3000 ms while retaining concurrency one, the `HEADROOM_KOMPRESS_EXECUTION_TIMEOUT_MS` override, the tighter acquire and request budgets, and passthrough after a genuine timeout. The reproduction and validated 3000 ms setting come from https://github.com/headroomlabs-ai/headroom/issues/2451 Closes #2451 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Raise the default Kompress execution-slot wait from 25 ms to 3000 ms. - Start the Kompress request deadline at call entry and carry it through single-item acquire, single-to-batch delegation, and sequential-fallback lineage. - Cap the raised execution-slot wait by that live request deadline on both single-item and batch acquire paths. - Keep the per-backend default concurrency at one and preserve all tighter time budgets. - Add queued single-item, batch, request-deadline, carried-deadline lineage, and router-watchdog lifecycle regressions at the same owner layer that currently fails. - Preserve the explicit short-timeout fail-open path. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_kompress_failsafe.py tests/test_kompress_request_nonblocking.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py -v`) - [x] Linting passes (`uv run ruff check headroom/transforms/kompress_compressor.py tests/test_kompress_failsafe.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py`) - [x] Formatting passes (`uv run ruff format headroom/transforms/kompress_compressor.py tests/test_kompress_failsafe.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py --check`) - [x] New regression tests prove the saturation fix - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_kompress_failsafe.py tests/test_kompress_request_nonblocking.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py -v 37 passed in 4.22s uv run ruff check headroom/transforms/kompress_compressor.py tests/test_kompress_failsafe.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py All checks passed! uv run ruff format headroom/transforms/kompress_compressor.py tests/test_kompress_failsafe.py tests/test_content_router_single_item_deadline.py tests/test_transforms/test_kompress_deadline.py --check 4 files already formatted ``` ## Real Behavior Proof - Environment: worktree Python environment from `uv sync --extra dev`, focused pytest with real Python threads and `threading.BoundedSemaphore` - Exact command / steps: hold the sole execution slot with the environment override unset, start queued single-item and batch compression workers, wait until each worker proves it reached a blocked acquire on the shared execution semaphore, release the slot, rerun the explicit 1 ms timeout preservation case, then set `HEADROOM_COMPRESSION_DEADLINE_MS=10` and repeat the held-slot single-item and batch acquires plus a router single-cache-miss run whose Kompress load sleeps past the request deadline. - Observed result: The queued single-item and batch workers each proved a real blocked acquire before release, then acquired after release and compressed, while `HEADROOM_KOMPRESS_EXECUTION_TIMEOUT_MS=1` still passed through promptly, the 10 ms request deadline capped the raised default wait so both held-slot paths failed open before 200 ms without reaching model inference, the single-to-batch and sequential-fallback lineage regressions proved later branches inherit the original request start instead of resetting it, and the router lifecycle proof showed the carried deadline now allows slow Kompress load to start but still expires before model inference after the outer request has already failed open. - Not tested: live ONNX proxy savings under sustained concurrent load ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` stays unchanged because the release pipeline generates changelog entries from conventional commits. The fail-open path from #1430 stays intact; this change stops it from firing spuriously under ordinary queueing. |
||
|
Parideboy
|
a09ba6c087
|
fix(learn): treat unreadable candidate paths as absent in project decode (#2446)
## Description `headroom learn` crashes with an uncaught `PermissionError` when the current user's username contains a dash. `_decode_project_path` (in `headroom/learn/plugins/claude.py`) probes speculative candidate paths when reconstructing an original filesystem path from a Claude Code encoded project directory name. When the username is e.g. `marco-rocha`, one candidate becomes `/home/marco/rocha`, which can collide with another user's home directory whose parent isn't stat-able. `Path.exists()` calls `os.stat` internally, raising `PermissionError` instead of returning `False`, so the whole `learn` command crashes before returning any recommendations. Fixes #2443 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `_path_exists()` to `headroom/learn/plugins/claude.py` — a thin wrapper around `Path.exists()` that returns `False` on any `OSError` (including `PermissionError`), mirroring the existing `OSError` handling already used in `_greedy_path_decode`. - Route every speculative candidate-path existence check in the decode path through `_path_exists()`: the Windows drive/path probes in `_decode_windows_path`, the `simple` POSIX candidate and greedy-branch bases in `_decode_project_path`/`_greedy_path_decode`, and the decoded `project_path`/`CLAUDE.md` checks in `discover_projects`. - Add regression tests covering the exact issue shape (`PermissionError` on `/home/marco/rocha`) and the `_path_exists` helper directly. - Leave `CHANGELOG.md` untouched — release-please generates it from conventional commits. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_learn/test_scanner.py::TestDecodePermissionError -q`) - [x] Linting passes (`ruff check`, `ruff format --check` on the two changed files) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_learn/test_scanner.py::TestDecodePermissionError -q collected 2 items tests\test_learn\test_scanner.py .. [100%] 2 passed in 1.86s $ ruff check headroom/learn/plugins/claude.py tests/test_learn/test_scanner.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, local dev checkout of headroom on branch off upstream/main - Exact command / steps: Simulated the issue by monkeypatching `Path.exists` to raise `PermissionError` for the colliding candidate `/home/marco/rocha`, then calling `_decode_project_path("-home-marco-rocha-butterfly-sylphina")` - Observed result: Before the fix the call propagates `PermissionError` (crash, matching the reported traceback); after the fix it returns without raising and the unreadable candidate is treated as non-existent. Both regression tests pass. - Not tested: End-to-end `headroom learn --apply` on a real Linux multi-user box with an actually unreadable `/home/<prefix>` — reproduced via the documented minimal logic instead. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Abhay Singh
|
3e976712e7
|
fix(proxy/output-shaping): tolerate a non-string system block text in steering (#2435)
## Description
`apply_verbosity_steering` (the Anthropic output-shaping path) scans the
`system` block list to find and update an existing steering block:
```python
if isinstance(system, list):
for block in system:
if isinstance(block, dict) and block.get("text", "").startswith(_STEERING_SENTINEL):
```
`.get("text", "")` only substitutes the default when the key is
**absent**. A malformed client block with a null text (`{"type": "text",
"text": null}`) returns `None`, so `None.startswith(...)` raises
`AttributeError`. In the output-shaping treatment arm that call runs
inside `shape_request`, which is not individually guarded, so the
exception propagates and 502s the request.
The OpenAI chat sibling in the same module already defends against this
exact case (`isinstance(part.get("text"), str)`), so the Anthropic path
is the inconsistent one.
## Fix
Guard that the block text is a string before `startswith`, mirroring the
OpenAI sibling. Well-formed bodies are unchanged: the steering block is
still replaced idempotently when a level changes, or appended when
absent. The malformed block is left untouched.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/output_steering.py`: string-guard the system block
text before `startswith` in `apply_verbosity_steering`.
- `tests/test_output_steering.py`: regression asserting a `system` list
containing a `{"text": null}` block does not crash and still appends the
steering block.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_output_steering.py -q
9 passed
# with the fix reverted, the new test fails (AttributeError on None.startswith):
$ git stash push -- headroom/proxy/output_steering.py
$ python -m pytest "tests/test_output_steering.py::test_anthropic_steering_tolerates_non_string_system_block_text" -q
1 failed
$ uvx ruff@0.15.17 check headroom/proxy/output_steering.py tests/test_output_steering.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/output_steering.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called the real `apply_verbosity_steering` with
`system=[{"type":"text","text":None},{"type":"text","text":"Real system
prompt."}]`; also confirmed the OpenAI sibling
`apply_openai_chat_verbosity_steering` handles the same shape.
- Observed result: pre-fix the Anthropic call raised `AttributeError:
'NoneType' object has no attribute 'startswith'` while the OpenAI
sibling returned True; post-fix the Anthropic call returns True, leaves
the malformed block as-is, appends the steering block, and stays
idempotent on a repeat. Ran against the actual module.
- Not tested: a live client that sends a null system block text end to
end.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Abhay Singh
|
77b26c093c
|
fix(proxy/streaming): tolerate malformed content in _response_to_sse (#2481)
## Description
`StreamingMixin._response_to_sse` rebuilds an Anthropic SSE stream from
a buffered response dict. It iterated the content and read usage with no
type guards:
```python
for idx, block in enumerate(response.get("content", [])):
if block.get("type") == "text":
...
...
"usage": {"output_tokens": response.get("usage", {}).get("output_tokens", 0)},
```
`response` here is provider- and reconstruction-controlled.
`.get("content", [])` only falls back when the key is absent, so a
present-but-null `content` returns `None` and `enumerate(None)` raises
`TypeError`. A non-list `content` (e.g. a bare string) makes
`block.get(...)` raise `AttributeError`, and a null element inside the
list hits the same `AttributeError`. `response.get("usage",
{}).get(...)` breaks the same way on `usage: null`.
This matters because the Anthropic buffered CCR path calls it inside an
`except ValueError` guard only:
```python
try:
sse_events = self._response_to_sse(resp_json, "anthropic")
except ValueError as sse_err:
...
```
A `TypeError`/`AttributeError` from any of the shapes above escapes that
guard and 500s the streamed request. The sibling
`_record_ccr_feedback_from_response` in the same class already guards
`content` for list-ness and skips non-dict blocks, so this closes the
asymmetry.
## Fix
Coerce `content` to a list before iterating (non-list becomes empty),
skip any non-dict block, and coerce a non-dict `usage` to `{}` before
reading `output_tokens`. Well-formed responses render byte-for-byte as
before.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/streaming.py`: list-guard `content`, skip
non-dict blocks, and dict-guard `usage` in `_response_to_sse`.
- `tests/test_sse_thinking_blocks.py`: regression rendering responses
with null/non-list content, a null block element, and null usage, plus a
check that a valid block alongside a null element still renders.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_sse_thinking_blocks.py -q
14 passed
# with the fix reverted, the new test fails with
# TypeError: 'NoneType' object is not iterable
$ uvx ruff@0.15.17 check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/streaming.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: called
`StreamingMixin()._response_to_sse(response, "anthropic")` with four
malformed bodies (`content: null`, `content: "not-a-list"`, `content:
[null, {text}]`, `usage: null`); then reverted `streaming.py` and re-ran
the same inputs.
- Observed result: with the fix each body produces a well-formed SSE
envelope (message_start ... message_stop) and the valid block alongside
a null element still emits its text_delta; with the fix reverted the
`content: null` body raises `TypeError: 'NoneType' object is not
iterable` and the others raise `AttributeError`. Ran against the actual
module via `tests/test_sse_thinking_blocks.py`.
- Not tested: a live upstream returning a malformed buffered response
end to end through the CCR path.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Abhay Singh
|
7524854da7
|
fix(doctor): don't crash on a valid-but-non-object settings.json (#2482)
## Description
`headroom doctor` parses `~/.claude/settings.json` in two checks:
```python
try:
payload = json.loads(settings_path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
return CheckResult(... WARN "could not parse" ...)
...
env_block = payload.get("env")
```
`json.loads` returns a non-dict for any valid JSON that is not an
object: `[]`, `null`, `42`, `"a string"`. None of those raise
`JSONDecodeError`, so they slip past the `except (OSError, ValueError)`
guard, and the following `payload.get("env")` raises `AttributeError`.
`AttributeError` is not in the caught tuple, so it escapes and crashes
`doctor` with a traceback. That is the worst moment for it: `doctor` is
the command a user runs precisely because their config is suspect, and a
hand-edited or reset `settings.json` holding `[]` or `null` is exactly
the kind of file it should report on, not fall over on.
Two functions have this shape: `check_claude_routing` (the `.get` is
after the `try` returns) and `check_claude_remote_control_gate` (the
`.get` is inside a `try` whose `except` is also `(OSError,
ValueError)`).
## Fix
Guard `payload` for dict-ness in both checks. `check_claude_routing` now
returns the same WARN it already returns for unparseable files, with a
"not a JSON object" summary; `check_claude_remote_control_gate` treats a
non-object as having no `env` block, so the shell environment still
drives the gate. Well-formed object settings behave exactly as before.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/doctor.py`: guard `payload` for dict-ness in
`check_claude_routing` and `check_claude_remote_control_gate` before
calling `.get`.
- `tests/test_cli_doctor.py`: parametrized regressions feeding `[]`,
`null`, `42`, and a bare string to both checks.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_cli_doctor.py -q
68 passed
# with the fix reverted, the new tests fail with
# AttributeError: 'list' object has no attribute 'get'
$ uvx ruff@0.15.17 check headroom/cli/doctor.py tests/test_cli_doctor.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/cli/doctor.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: wrote a `settings.json` containing `[]` (and
`null`, `42`, `"a string"`) into a tmp path and called
`check_claude_routing(path, 8787)` and
`check_claude_remote_control_gate(path, {"ANTHROPIC_BASE_URL":
"http://127.0.0.1:8787"})`; then reverted `doctor.py` and re-ran.
- Observed result: with the fix both checks return a WARN result instead
of raising; with the fix reverted both raise `AttributeError: 'list'
object has no attribute 'get'` (and the analogous message for
`null`/`42`/string). Ran against the actual module via
`tests/test_cli_doctor.py`.
- Not tested: the full `headroom doctor` CLI end to end against a real
`~/.claude/settings.json`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
Rod Boev
|
46293f4daf
|
fix(codex): detect keyring-backed ChatGPT auth (#2478)
## Description Headroom currently treats missing `auth.json` as “not ChatGPT auth” for Codex, which breaks keyring-backed ChatGPT sessions on Codex CLI 0.144.6 because those sessions intentionally may not store credentials in the file. This updates the Codex auth detector to keep the existing file-backed fast path and fall back to Codex-owned auth metadata when the session is keyring-backed or auto-backed, so `requires_openai_auth = true` is emitted only for real ChatGPT logins. Closes #2474 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - extend Codex auth detection so keyring-backed and auto-backed sessions can be classified from Codex-owned auth metadata when `auth.json` is absent - preserve the current file-backed ChatGPT, API-key, malformed-file, and fail-closed behaviors - add focused install-layer regression coverage for the new keyring path and adjacent negative space ## Testing - [x] Unit tests pass (`uv run pytest tests/test_install/test_codex_install.py -q`) - [x] Linting passes (`uv run ruff check headroom/providers/codex/install.py tests/test_install/test_codex_install.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_install/test_codex_install.py -q ============================= test session starts ============================= platform win32 -- Python 3.12.13, pytest-9.0.3, pluggy-1.6.0 collected 9 items tests\test_install\test_codex_install.py ......... [100%] ============================== 9 passed in 0.24s ============================== uv run ruff check headroom/providers/codex/install.py tests/test_install/test_codex_install.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows, Codex CLI 0.144.6 available locally, Python 3.12.13 via `uv` - Exact command / steps: `codex login status`; `Measure-Command { codex login status > $null }`; focused pytest and Ruff commands above - Observed result: `codex login status` returns `stdout=''` and `stderr='Logged in using ChatGPT\n'`; the status probe measured `71.40` ms; pytest reports `9 passed in 0.24s`; keyring ChatGPT emits `requires_openai_auth = true`, non-ChatGPT and failed probes omit it, and file-backed ChatGPT/API-key cases remain true/false - Not tested: live local keyring-backed Codex login ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` stays untouched because Headroom generates release notes from conventional commits. The PR should only claim the Codex-owned detection path and focused local regression coverage; the live keyring session proof remains a follow-up owner check. |