mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
1137 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Ayush Kumar Jha
|
daeff69a75
|
fix(wrap): surface Claude Remote Control base-URL gate accurately (#1… (#1883)
…779) Claude Code 2.1.196 deterministically disables first-party Remote Control (/remote-control, /rc) behind a custom ANTHROPIC_BASE_URL, which Headroom always sets. Make the wrap/doctor warning accurate (state the disable as fact, name the /rc command, detect the installed version), suppress it for auth modes that never had RC (API key, Bedrock/Vertex/Foundry) and for builds older than 2.1.196, co-report the sibling #746/#1158 gates session-accurately, and fix is_custom_anthropic_base_url host handling (scheme-less hosts, malformed URLs). UX/notice-only; no request bytes touched. ## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Shubham Srivastava
|
c3db8e47f8
|
fix(install): default docker image to headroomlabs-ai GHCR registry (#1867) (#2039)
## Description
The GitHub repository was transferred from `chopratejas/headroom` to
`headroomlabs-ai/headroom`. GitHub 301-redirects transferred repos for
web and git operations, but **GitHub Container Registry (GHCR) does
not** — the old package `ghcr.io/chopratejas/headroom` is now orphaned
and frozen (its `latest` tag stopped advancing at `0.27.0`), while CI
publishes new images to `ghcr.io/headroomlabs-ai/headroom` (the workflow
derives the path from `${{ github.repository }}`).
Headroom's install tooling still defaulted to the dead path, so
`headroom install apply --preset persistent-docker`, `headroom init`,
and the standalone install scripts all pulled a stale `0.27.0` image
instead of the current release.
This changes every docker-image **default** to
`ghcr.io/headroomlabs-ai/headroom:latest`.
Closes #1867
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/install/models.py` — `InstallManifest.image` default.
- `headroom/cli/install.py` — `--image` Click option default.
- `headroom/cli/init.py` — two `InstallManifest(...)` image args.
- `scripts/install.sh` / `scripts/install.ps1` — `IMAGE_DEFAULT` /
`$ImageDefault` plus the `--image` help-text default.
- `docker/docker-compose.native.yml` — image default in both services.
- `tests/test_install/test_planner.py` (4) and
`tests/test_install/test_runtime.py` (5) — updated the assertions that
pinned the old image (including `assert "<image>" in command`), so they
now verify the corrected registry threads through the planner and docker
runtime command.
Scope note: `github.com/chopratejas/...` links and the plugin
marketplace slug are intentionally **not** changed — GitHub redirects
those, so they still work. Only the genuinely-dead GHCR image references
are touched. No new dependency, no new abstraction.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_install/ -q
99 passed, 1 skipped in 48.34s
$ uv run ruff check headroom/install/models.py headroom/cli/install.py headroom/cli/init.py
All checks passed!
$ grep -rn "ghcr.io/chopratejas/headroom" headroom/ scripts/ docker/ tests/
# (no source matches — every default now points at headroomlabs-ai)
```
The install-test assertions pinned the old image, so they fail against
the old defaults and pass after the fix — they are the regression guard.
## Real Behavior Proof
- **Environment:** Windows 11, Python 3.13.5, headroom installed from
this branch (editable, via `uv`).
- **Exact command / steps:** The dead-registry claim is verifiable at
the registry level, independent of a release:
```text
docker pull ghcr.io/chopratejas/headroom:latest # old default → 0.27.0
(frozen / orphaned)
docker pull ghcr.io/headroomlabs-ai/headroom:latest # new default →
current release
```
And the install pipeline now emits the correct image (covered by
`tests/test_install/test_runtime.py`, which asserts the resolved docker
command contains `ghcr.io/headroomlabs-ai/headroom:latest`).
- **Observed result:** `grep` confirms no `ghcr.io/chopratejas/headroom`
default remains in code, scripts, compose, or tests;
`tests/test_install/` is green (99 passed) with the corrected image
asserted end-to-end through planner → runtime command.
- **Not tested:** A live `docker pull` of both tags on this specific
machine (no local Docker daemon guaranteed) — the registry difference is
reproducible by anyone running the two `docker pull` commands above; and
an end-to-end `headroom install apply --preset persistent-docker`
against a real Docker host.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A
## Additional Notes
- Documentation checklist item is N/A — no user-facing docs surface
beyond the CHANGELOG entry.
- `mypy` left unchecked — not run as part of this verification; the
change is a string-default swap with no type-level surface.
- The `--image` help text and `docker-compose.native.yml` were included
so every user-facing default is consistent; only the dead GHCR image
string was changed.
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Ingmar Krusch
|
3d0e59e518
|
fix(content-router): protect_tool_results must not be weakened by profile-derived read_protection_window (#2105)
## Description `ContentRouter.apply()` computes `read_protection_window` from `protect_recent_reads_fraction`, where `0.0` (the sentinel `--protect-tool-results` sets, per #1374's documented contract) means "protect all excluded-tool output regardless of conversation depth." The method then unconditionally overwrote that window with a per-request `read_protection_window` kwarg whenever one was present. `proxy_pipeline_kwargs()` supplies that kwarg on every request from the active `AgentSavingsProfile.protect_recent` (the default `coding` profile sets `protect_recent=2`), so in practice only the last 2 messages ever kept read-protection regardless of `--protect-tool-results` — older excluded-tool output (`Read`, `Glob`, `Grep`, `Write`, `Edit` results) silently fell through to lossy Kompress compression. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/transforms/content_router.py`: the runtime `read_protection_window` kwarg may now only *narrow* the window when `self.config.protect_recent_reads_fraction > 0`. It can no longer override the `0.0` ("protect everything") sentinel that `--protect-tool-results` sets. - `tests/test_content_router_exclude_tools.py`: regression coverage that `--protect-tool-results`-equivalent config (`protect_recent_reads_fraction=0.0`) stays fully protected even when a savings-profile kwarg would otherwise shrink the window. - `tests/test_transforms/test_content_router.py`: unit coverage of the precedence logic itself (kwarg narrows when fraction > 0, kwarg is ignored when fraction == 0.0). - `CHANGELOG.md`: added an `### Bug Fixes` entry under `Unreleased`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_content_router_exclude_tools.py tests/test_transforms/test_content_router.py -q ============================= test session starts ============================== platform darwin -- Python 3.13.13, pytest-9.0.3, pluggy-1.6.0 collected 64 items tests/test_content_router_exclude_tools.py ...... [ 9%] tests/test_transforms/test_content_router.py ........................... [ 51%] ............................... [100%] ============================== 64 passed in 2.77s ============================== $ uv run ruff check headroom/transforms/content_router.py tests/test_content_router_exclude_tools.py tests/test_transforms/test_content_router.py All checks passed! $ uv run mypy headroom/transforms/content_router.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - **Environment:** personal fork deployed as a real proxy (macOS launchd service, `headroom install apply`) with `--backend bedrock --mode token --code-aware --protect-tool-results Bash`, `HEADROOM_SAVINGS_PROFILE=coding` (library default `protect_recent=2`), fronting a live Claude Code session. - **Exact command / steps:** in a long-running Claude Code session against this deployment, `Read` a source file, continue the conversation past 2 more assistant turns (so the file's `Read` result ages past the profile's `protect_recent=2` window), then have the agent re-read or reference the same file. - **Observed result:** before the fix, the aged `Read` output for a plain (non-code) file came back as `[N items compressed to M. Retrieve more: hash=...]` despite `--protect-tool-results` being set and `Read` sitting in `DEFAULT_EXCLUDE_TOOLS` — confirmed by direct proxy log inspection (`content_router.py`'s override silently winning over the `0.0` sentinel) and by byte-diffing the installed pipx package against this same fork's git source to rule out a stale build. After applying the fix, the same sequence leaves the aged `Read` output intact (no compression marker) — verified via `pytest` regression tests plus a fresh live-session check post-deploy. - **Not tested:** this deployment has since switched to `--mode cache` (upstream's tested/benchmarked default for the `coding` profile as of `68676daa`), where the whole `read_protection_window` mechanism this bug lives in is structurally unreachable for anything inside the frozen prefix — so the precedence fix in this PR is primarily relevant to `token`-mode deployments (or any deployment where cache mode's frozen-prefix boundary hasn't yet advanced past the affected message). It has not been independently re-verified live under `--mode token` after the most recent rebase onto `main` (only the automated test suite was rerun post-rebase). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — backend logic change, no UI surface. ## Additional Notes - "I have made corresponding changes to the documentation" is unchecked: no file in `docs/`, `README.md`, or `CONTRIBUTING.md` documents `read_protection_window`, `protect_recent_reads_fraction`, or `--protect-tool-results` precedence at all, so there was no existing section to update, and no new section was added either. This is arguably a pre-existing documentation gap this PR doesn't close. - No linked issue number: this was found via independent investigation of a personal deployment, not filed as a `headroomlabs-ai/headroom` issue first. Co-authored-by: Ingmar Krusch <ingmar.krusch@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Abhay Singh
|
1725cd1f83
|
fix(memory): key the embedder cache on ollama_base_url (#2109)
## Description
The process-wide embedder cache can hand a caller an embedder bound to
the wrong Ollama server.
`_create_embedder` caches by `(backend, model)`:
```python
key = (
config.embedder_backend.value if hasattr(...) else str(...),
config.embedder_model or "",
)
```
But the Ollama branch constructs the embedder with the server URL:
```python
embedder = OllamaEmbedder(base_url=config.ollama_base_url, model_name=config.embedder_model)
```
So two configs in the same process that share a backend and model but
point at different Ollama servers (for example a per-project storage
router, or a fail-over host) collide on the same cache key. The first
call builds and caches an `OllamaEmbedder` bound to server A; the second
call, asking for server B, gets server A's embedder back and silently
embeds against the wrong host.
The code already reasoned about the analogous `openai_api_key` omission
and worked around it with an up-front validation guard (see the comment
above the key), but `ollama_base_url` has no such guard, so it just
resolves to the wrong server.
## Fix
Add `config.ollama_base_url` to the cache key. Same server still hits
the cache (one model load); a different server gets its own embedder.
Non-Ollama backends are unaffected (the URL just becomes an extra,
constant key component).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/memory/factory.py`: include `config.ollama_base_url` in the
embedder cache key, with a comment explaining why.
- `tests/test_memory/test_factory_embedder_cache.py`: new file with
`test_ollama_embedder_cache_keys_on_base_url` (different servers get
different embedders) and
`test_ollama_embedder_cache_reuses_same_base_url` (same server still
caches). Kept out of `test_factory.py` because that module skips
wholesale without `hnswlib`, which these cases don't need.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/memory/factory.py tests/test_memory/test_factory_embedder_cache.py
All checks passed!
$ python -m py_compile headroom/memory/factory.py tests/test_memory/test_factory_embedder_cache.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the cache-key behavior with a
dependency-free script that models the `(backend, model)` vs `(backend,
model, base_url)` keys against a simulated cache, and left the full
pytest to CI.
- Exact command / steps: created two configs with the same backend and
model but `ollama_base_url` of `http://gpu1:11434` and
`http://gpu2:11434`, and resolved each through the old key and the new
key against a shared cache.
- Observed result: the old key serves the same embedder object for both,
and the config asking for `gpu2` is handed the `gpu1`-bound embedder;
the new key gives each config its own embedder bound to its own server.
The new tests assert distinct embedders with the right `_base_url` for
different servers, and cache reuse for the same server.
- Not tested: a live Ollama round-trip (`OllamaEmbedder` construction is
offline — it stores the URL and lazily creates its client); full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change adds one component to a cache-key tuple in a
pure function, verified by the standalone proof and the two new tests
for CI. The tests construct only the lightweight (offline) Ollama
embedder, so they don't need a running server or the vector-index deps.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
6979b5245e
|
fix(tokenizers): use o200k_base for gpt-4.1/gpt-4.5/o4 families (#2108)
## Description
`get_encoding_for_model` returns the wrong tiktoken encoding for the
current OpenAI flagship families, so their token counts are computed
with the wrong vocabulary.
The prefix table is ordered most-specific-first, but it has no entry for
the `gpt-4.1` / `gpt-4.5` / `o4` families:
```python
for prefix, encoding in (
("gpt-4o", "o200k_base"),
("gpt-4-turbo", "cl100k_base"),
("gpt-4", "cl100k_base"),
("gpt-3.5", "cl100k_base"),
("o1", "o200k_base"),
("o3", "o200k_base"),
):
if model.startswith(prefix):
return encoding
return DEFAULT_ENCODING # cl100k_base
```
- `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.5-*` all start with `gpt-4`, so they
match the `gpt-4` prefix and get `cl100k_base`.
- `o4-mini` matches no prefix and falls through to the `cl100k_base`
default.
All three families use `o200k_base`. Since `count_text`/`count_messages`
tokenize with the resolved encoding, every token count for those models
is computed against the wrong BPE vocabulary, which skews budget gating
and the compress/skip decision for a large slice of current OpenAI
traffic.
## Fix
Add explicit `gpt-4.1` and `gpt-4.5` prefixes (ordered ahead of `gpt-4`,
which they would otherwise match) and an `o4` prefix, all mapping to
`o200k_base`. Plain `gpt-4` and `gpt-3.5` snapshots still resolve to
`cl100k_base`, and `gpt-4o` still wins for the 4o family.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/tokenizers/tiktoken_counter.py`: add `gpt-4.1`/`gpt-4.5`
prefixes ahead of `gpt-4`, and an `o4` prefix, all mapping to
`o200k_base`.
- `tests/test_tokenizers.py`: add
`test_gpt41_and_o4_families_use_o200k`.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/tokenizers/tiktoken_counter.py tests/test_tokenizers.py
All checks passed!
$ python -m py_compile headroom/tokenizers/tiktoken_counter.py tests/test_tokenizers.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the resolution with a
dependency-free script that runs the old and new prefix tables, and left
the full pytest to CI.
- Exact command / steps: resolved `gpt-4.1`, `gpt-4.1-mini`,
`gpt-4.5-preview`, and `o4-mini` under the old table and the new table,
plus `gpt-4o-*`, `gpt-4-2025-*`, `gpt-4-turbo-*`, `gpt-3.5-turbo`, and
`o1-mini` as regression guards.
- Observed result: old table returns `cl100k_base` for all four (wrong);
new table returns `o200k_base`; the guard models are unchanged
(`gpt-4o-*` and `o1-*` stay `o200k_base`,
`gpt-4*`/`gpt-4-turbo*`/`gpt-3.5*` stay `cl100k_base`). The new test
asserts the four families resolve to `o200k_base` and a plain `gpt-4`
snapshot stays `cl100k_base`.
- Not tested: loading the actual tiktoken vocabularies to count tokens
end to end; full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change adds three ordered prefix entries to a pure
function, verified by the standalone proof and the new regression test
for CI. I intentionally left `gpt-5` out since I didn't want to assert
an encoding I couldn't confirm here; happy to add it in a follow-up if
you can confirm the intended mapping.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
eecb81e847
|
fix(cache/ccr): don't count a successful eviction as a retrieval (#2106)
## Description
The compression feedback learner treats a *successful* compression as
evidence that it should compress less, which inverts the learning
signal.
When `CompressionStore` evicts an entry that was never retrieved, it
emits a synthetic event to tell the learner the compression was fine
(the model never needed the original):
```python
success_event = RetrievalEvent(..., retrieval_type="eviction_success")
self._pending_feedback_events.append(success_event)
```
`process_pending_feedback` forwards every pending event to
`CompressionFeedback.record_retrieval` unconditionally. But
`record_retrieval` has no branch for `"eviction_success"` — and since
that string isn't `"full"`, it lands in the `else`:
```python
self._total_retrievals += 1
pattern.total_retrievals += 1
if event.retrieval_type == "full":
pattern.full_retrievals += 1
else:
pattern.search_retrievals += 1 # <-- eviction_success counted here
```
So a compression that worked is booked as a *search retrieval*, which
raises the tool's `retrieval_rate` and `search_rate`.
`get_compression_hints` reads a high retrieval rate as "we're
compressing too aggressively" and recommends larger `max_items` / lower
aggressiveness (or `skip_compression`). Net effect: the more often
compression succeeds, the more the learner backs off from compressing. A
standalone repro books a single successful eviction as a 100% retrieval
rate.
Every sibling consumer of the event distinguishes the type — telemetry
and TOIN both receive `retrieval_type="eviction_success"` and handle it
as its own thing. Only the local feedback counter ignores the
distinction.
## Fix
Recognize `"eviction_success"` in `record_retrieval` and leave it out of
the retrieval counters. The compression itself is already counted by
`record_compression` at store time, so an entry that is compressed and
never retrieved already yields a low retrieval rate — which is the
correct "compression worked" signal. Genuine `full`/`search` retrievals
are unchanged.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cache/compression_feedback.py`: early-return in
`record_retrieval` for `retrieval_type == "eviction_success"` so it is
not counted as a retrieval, with a comment explaining the signal.
- `tests/test_ccr_feedback.py`: add
`test_eviction_success_is_not_counted_as_retrieval` (asserts the
counters stay at zero after a successful eviction, and that a genuine
retrieval afterward still counts).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/cache/compression_feedback.py tests/test_ccr_feedback.py
All checks passed!
$ python -m py_compile headroom/cache/compression_feedback.py tests/test_ccr_feedback.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the counting logic with a
dependency-free script that replicates
`record_compression`/`record_retrieval` and the
`retrieval_rate`/`search_rate` properties, and left the full pytest to
CI.
- Exact command / steps: recorded one compression, then a
`retrieval_type="eviction_success"` event, under the old counting (no
branch) and the new counting (early return), plus a genuine `search`
retrieval as a control.
- Observed result: old counting books the successful eviction as a
retrieval — `retrieval_rate=1.0`, `search_rate=1.0` — so the learner
would back off from compressing; new counting leaves
`retrieval_rate=0.0` and `total_retrievals=0`; a real retrieval
afterward still increments to 1. The new test asserts exactly this.
- Not tested: an end-to-end store-evict-then-hint cycle through
`CompressionStore.process_pending_feedback`; full local `pytest`
deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the fix is a single early-return in a pure counting method,
verified by the standalone proof and the new regression test (which
reuses the existing `test_ccr_feedback.py` pattern) for CI. Scope is
deliberately limited to the local feedback learner — telemetry and TOIN
already receive the `eviction_success` type and handle it separately.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
aa788164fd
|
fix(proxy/anthropic): preserve non-2xx upstream status through security scan (#2100)
## Description
When enterprise security scanning is enabled, the non-streaming
`/v1/messages` handler can turn a failed upstream response into an HTTP
200, so the client never sees the error.
After the upstream call, the handler parses the body unconditionally:
```python
resp_json = None
try:
resp_json = response.json()
except (json.JSONDecodeError, ValueError):
...
```
Every response-mutating block that follows gates on a successful
upstream — CCR handling (`... and response.status_code == 200 ...`), the
response cache (`if self.cache and response.status_code == 200`), and
the buffered-stream CCR block (`if buffered_stream_ccr and
response.status_code == 200 ...`). The enterprise-security block was the
exception:
```python
if self.security and _security_ctx and resp_json:
resp_json = self.security.scan_response(resp_json, _security_ctx)
response = httpx.Response(status_code=200, content=json.dumps(resp_json).encode(), headers=response_headers)
if not buffered_stream_ccr:
return Response(content=response.content, status_code=response.status_code, headers=response_headers)
```
No status check, and the rebuilt response hardcodes `status_code=200`.
`_retry_request` returns 429 (rate limit), 529 (overloaded), and other
4xx responses verbatim to this caller, so when security is configured
any of those — whose JSON error body parses fine — is rebuilt as an HTTP
200 and returned. The client sees success, so its retry/backoff logic
never fires on a rate limit or overload, exactly when it matters most.
## Fix
Gate the security block on a 200 upstream, the same condition the
sibling CCR/cache/buffered-stream blocks already use. A non-2xx response
falls through to the final `return Response(...,
status_code=response.status_code, ...)` and keeps its real status; a 200
is still scanned and returned as before.
```python
if self.security and _security_ctx and resp_json and response.status_code == 200:
...
```
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/anthropic.py`: gate the enterprise-security
response-scan branch on `response.status_code == 200`.
- `tests/test_anthropic_pre_upstream_backpressure.py`: reuse the
existing `_DummyAnthropicHandler` harness (adds optional `security` and
`upstream_status` params, both defaulting to today's behavior) and add
`test_security_scan_preserves_non_200_upstream_status` (429/529/400)
plus a 200 positive-control test.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/anthropic.py tests/test_anthropic_pre_upstream_backpressure.py
All checks passed!
$ python -m py_compile headroom/proxy/handlers/anthropic.py tests/test_anthropic_pre_upstream_backpressure.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and running the handler
test locally OOM-kills this box, so I verified the branch logic with a
dependency-free script that models the security block plus the
fallthrough return, and left the full pytest (including the new handler
test) to CI.
- Exact command / steps: modelled the non-streaming return path with
enterprise security configured, running a `429` upstream through the old
branch (no status gate, rebuilds 200) and the new branch (gated on 200,
falls through), plus a `200` upstream as a control. Also traced
`_retry_request` in `server.py` to confirm it returns 429/529/4xx
verbatim to this caller (only 5xx raises), so the branch is reachable
for those statuses.
- Observed result: old branch returns HTTP 200 for a 429 upstream
(laundered); new branch returns 429; a 200 upstream returns 200 under
both. The new parametrized handler test asserts the returned status
equals the upstream status for 429/529/400, and the control test asserts
a 200 upstream still returns 200.
- Not tested: a live enterprise-security plugin (`scan_response` is an
out-of-repo component; the test uses a passthrough stub); full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the fix is a one-condition gate mirroring three sibling
blocks in the same method, and the regression test reuses the file's
existing, proven `handle_anthropic_messages` harness (the added handler
params default to current behavior, so existing tests are unaffected).
The branch-logic proof and the new tests cover the fix for CI. The
security block only runs when an enterprise-security component is
configured; without it, this path is inert.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
c7b5a24b4f
|
fix(learn): don't shadow TIMEOUT/CONNECTION with the generic RUNTIME_ERROR (#2099)
## Description `classify_error` (used by `headroom learn` to categorize failed tool calls) miscategorizes timeouts and connection failures as generic runtime errors. The pattern list is checked in order, first match wins, and it puts the generic catch-all *before* the specific categories: ```python (re.compile(r"Traceback \(most recent|Exception:|Error:", re.I), ErrorCategory.RUNTIME_ERROR), (re.compile(r"timed? ?out|TimeoutError|deadline exceeded", re.I), ErrorCategory.TIMEOUT), ... (re.compile(r"ConnectionError|ConnectionRefused|ECONNREFUSED|network", re.I), ErrorCategory.CONNECTION_ERROR), ``` Every Python exception repr is `XxxError: ...` (or `Exception: ...`), so the generic `Error:`/`Exception:` pattern matches first. A tool result of `"TimeoutError: timed out after 30s"` is classified `RUNTIME_ERROR` instead of `TIMEOUT`; `"ConnectionError: [Errno 111] Connection refused"` is classified `RUNTIME_ERROR` instead of `CONNECTION_ERROR`. The dedicated `TIMEOUT` and `CONNECTION_ERROR` categories — which explicitly list `TimeoutError` and `ConnectionError` — are therefore unreachable for the most common (colon-repr) message shape; they only fire for tokenless phrasings like `deadline exceeded`. That mislabels the learn digest's per-category error stats. ## Fix Check the two specific categories (`TIMEOUT`, `CONNECTION_ERROR`) before the generic `RUNTIME_ERROR` catch-all. A generic exception repr with no timeout/connection token still classifies as `RUNTIME_ERROR`, so existing behavior for those is unchanged (including the opencode scanner's `"Error: command failed with exit code 1"` → `RUNTIME_ERROR`). Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/learn/_shared.py`: move the `TIMEOUT` and `CONNECTION_ERROR` patterns above the generic `RUNTIME_ERROR` pattern, with a comment explaining the ordering. - `tests/test_learn/test_error_classification.py`: new tests asserting `TimeoutError:`/`ConnectionError:` reprs classify specifically, a generic `Error:` stays `RUNTIME_ERROR`, and non-error text is `UNKNOWN`. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/learn/_shared.py tests/test_learn/test_error_classification.py All checks passed! $ python -m py_compile headroom/learn/_shared.py tests/test_learn/test_error_classification.py OK ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing `headroom` pulls in the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the ordering with a dependency-free script that replicates the pattern list under both the old and new orderings, and left the full pytest to CI. - Exact command / steps: classified `"ConnectionError: [Errno 111] Connection refused"` and `"TimeoutError: timed out after 30s"` under the old order (RUNTIME before TIMEOUT/CONNECTION) and the new order (TIMEOUT/CONNECTION before RUNTIME), plus the opencode scanner's `"Error: command failed with exit code 1"` as a regression guard. - Observed result: old order classifies both as `RUNTIME_ERROR`; new order classifies them as `CONNECTION_ERROR` and `TIMEOUT` respectively; the guard string stays `RUNTIME_ERROR` under both orderings, so the existing opencode scanner test is unaffected. - Not tested: a full `headroom learn` digest run; full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The "unit tests pass locally" and "type checking" boxes are unchecked because the full suite imports the ML stack, which I can't run in this environment; the change is a reordering of two entries in a pure pattern list, verified by the standalone proof (which also confirms the one existing test that touches this path stays green) and the new regression tests for CI. Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Abhay Singh
|
e0232df9b4
|
fix(tokenizers): resolve HF tokenizer names by most-specific prefix (#2096)
## Description
`get_tokenizer_name` can pick the wrong tokenizer for a versioned model,
which silently produces wrong token counts.
For a model that isn't a literal key in `MODEL_TO_TOKENIZER`, it falls
back to prefix matching:
```python
for key, value in MODEL_TO_TOKENIZER.items():
if model_lower.startswith(key):
return value
```
That returns the first key the model merely *starts with*, in
dict-insertion order. The table lists short family keys before their
more-specific siblings — `"qwen"` (→ `Qwen/Qwen-7B`) appears before
`"qwen2"`/`"qwen2-7b"`/`"qwen2.5"`. So
`get_tokenizer_name("qwen2-7b-instruct")` matches `"qwen"` first and
returns the **Qwen1** tokenizer, not Qwen2. Qwen1 and Qwen2 have
different vocabularies, so every `count_text`/`count_messages` for that
model is off. `qwen2.5-*` and `deepseek-v2.x` are mis-resolved the same
way.
The sibling tiktoken resolver already documents and guards this exact
pitfall — `get_encoding_for_model` uses an explicit most-specific-first
prefix list with a comment that scanning "for the first key that merely
starts with the prefix is order-dependent and wrong." The HuggingFace
resolver is the one that still scans insertion order.
## Fix
Match the **longest** (most-specific) prefix instead of the first in
insertion order:
```python
for key in sorted(MODEL_TO_TOKENIZER, key=len, reverse=True):
if model_lower.startswith(key):
return MODEL_TO_TOKENIZER[key]
```
Direct-key lookups and the shorter-family fallback (e.g. `deepseek-chat`
→ `deepseek-ai/deepseek-llm-7b-base`) are unchanged.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/tokenizers/huggingface.py`: `get_tokenizer_name` prefix
matching now iterates keys longest-first and returns the most-specific
match.
- `tests/test_huggingface_tokenizer_timeout.py`: add
`test_get_tokenizer_name_prefers_most_specific_prefix`.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/tokenizers/huggingface.py tests/test_huggingface_tokenizer_timeout.py
All checks passed!
$ python -m py_compile headroom/tokenizers/huggingface.py tests/test_huggingface_tokenizer_timeout.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified against the real key table
with a dependency-free script that parses `MODEL_TO_TOKENIZER` out of
the source and runs both the old (insertion-order) and new
(longest-first) scans, then left the full pytest to CI.
- Exact command / steps: resolved `qwen2-7b-instruct`, `qwen2.5-turbo`,
and `deepseek-v2.5` under both strategies, plus `deepseek-chat` as a
regression guard.
- Observed result: old scan returns `Qwen/Qwen-7B` (Qwen1) for both
qwen2 models and `deepseek-ai/deepseek-llm-7b-base` (v1) for
`deepseek-v2.5`; new scan returns `Qwen/Qwen2-7B`, `Qwen/Qwen2.5-7B`,
and `deepseek-ai/DeepSeek-V2` respectively. `deepseek-chat` resolves
identically under both (`deepseek-ai/deepseek-llm-7b-base`), so the
existing timeout test's model is unaffected.
- Not tested: loading the actual HuggingFace tokenizers
(network/`transformers`); full local `pytest` deferred to CI (OOM, per
above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change is a localized swap of the prefix-scan order in
a pure function, verified by the standalone proof (run against the real
key table) and the new regression test for CI. This mirrors the
same-class fix already present in the sibling tiktoken resolver.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
6137967083
|
fix(pricing): alias retired claude-3-sonnet to Sonnet-tier price, not Haiku (#2095)
## Description The `MODEL_ALIASES` fallback prices the retired Claude 3 Sonnet as Claude 3 Haiku — a different, ~12x cheaper tier. `MODEL_ALIASES` maps models that LiteLLM's cost DB no longer knows about to a current key "that has equivalent pricing" (per the module comment). The two Claude 3.5 Sonnet entries follow that rule — both map to `claude-sonnet-4-20250514`, which is the same `$3 / $15` per-1M tier. But the Claude 3 Sonnet entry was: ```python "claude-3-sonnet-20240229": "claude-3-haiku-20240307", ``` `claude-3-sonnet-20240229` was a Sonnet-tier model at `$3.00 / $15.00` per 1M (input/output). `claude-3-haiku-20240307` is `$0.25 / $1.25`. So whenever LiteLLM lacks the retired Sonnet key and resolution falls through to this alias (via `resolution_candidates` / `pricing_lookup_candidates`), every cost and savings figure for that model is understated **~12x on both input and output**. That's the opposite of the "equivalent pricing" the alias table promises, and it silently biases dashboards/ledger numbers for anyone still routing that model. ## Fix Alias the retired Claude 3 Sonnet to `claude-sonnet-4-20250514` — the same-price ($3/$15) target the sibling retired-Sonnet aliases already use — so the fallback preserves the tier instead of downgrading it. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/pricing/litellm_model_resolution.py`: change the `claude-3-sonnet-20240229` alias target from `claude-3-haiku-20240307` to `claude-sonnet-4-20250514`, with a comment explaining the tier. - `tests/test_pricing_litellm_model_resolution.py`: add `test_retired_claude_3_sonnet_aliases_to_sonnet_tier_not_haiku`. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/pricing/litellm_model_resolution.py tests/test_pricing_litellm_model_resolution.py All checks passed! $ python -m py_compile headroom/pricing/litellm_model_resolution.py tests/test_pricing_litellm_model_resolution.py OK ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing `headroom` pulls in the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I checked the tier delta with a dependency-free script against the public list prices and left the full pytest to CI. - Exact command / steps: compared the old alias target (`claude-3-haiku-20240307`, $0.25/$1.25) against the new one (`claude-sonnet-4-20250514`, $3.00/$15.00), which matches the retired Claude 3 Sonnet's own $3/$15 tier. - Observed result: the Haiku target underpriced input 12x ($3.00 / $0.25) and output 12x ($15.00 / $1.25). The new regression test asserts the alias contains no `haiku` and equals the same-tier target used by the other retired-Sonnet aliases. - Not tested: an end-to-end resolution through a live LiteLLM cost DB (the alias only fires when LiteLLM lacks the retired key); full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The "unit tests pass locally" and "type checking" boxes are unchecked because the full suite imports the ML stack, which I can't run in this environment; this is a one-line data fix in a pure module, verified by the tier-delta proof and the new regression test for CI. Reachability is bounded — the alias only matters when LiteLLM's cost DB doesn't already know the retired model — but when it does fire the price is off by a full tier. Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Abhay Singh
|
cf6367add4
|
fix(cache/semantic): don't evict an unrelated entry on an update at capacity (#2094)
## Description
`SemanticCache.put` can evict a perfectly good, unrelated entry when it
merely updates a key that is already cached.
The method runs its at-capacity eviction loop *before* it computes the
entry's key:
```python
self._cleanup_expired()
# Evict if at capacity
while len(self._cache) >= self.config.max_entries:
self._evict_oldest()
...
key = messages_hash or self._generate_key(query)
...
self._cache[key] = entry
```
So when the same key is stored again while the cache is full (a
duplicate store, or a retried request that produces the same
`messages_hash`), the loop fires because `len == max_entries`, evicts
the LRU-oldest *distinct* entry, and only then overwrites the existing
key in place. Writing to an already-present key does not grow the map,
so nothing needed to be evicted — but an unrelated live entry is now
gone, and the next `get` for it is a false miss.
Concretely, with `max_entries=2` and keys `[h1, h2]`, re-storing `h2`
evicts `h1`, leaving `[h2]` even though only two distinct keys were ever
stored.
The sibling `CompressionCache.store_compressed` gets this right: it
deletes the existing key first, inserts, and only then trims — so
re-storing a present key never drops an unrelated entry.
## Fix
Compute the key first, then run the eviction loop only while the key is
genuinely new:
```python
key = messages_hash or self._generate_key(query)
while key not in self._cache and len(self._cache) >= self.config.max_entries:
self._evict_oldest()
```
An in-place update of an existing key no longer evicts anything; adding
a new key still trims to make room exactly as before.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cache/semantic.py`: move the cache-key computation above the
eviction loop and gate the loop on `key not in self._cache` so an
in-place update never evicts.
- `tests/test_cache/test_semantic.py`: add
`test_update_at_capacity_does_not_evict_unrelated_entry`.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/cache/semantic.py tests/test_cache/test_semantic.py
All checks passed!
$ python -m py_compile headroom/cache/semantic.py tests/test_cache/test_semantic.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the eviction logic with a
dependency-free script that replicates the `OrderedDict` +
`_evict_oldest` (popitem last=False) behavior for the old vs new loop,
and left the full pytest to CI.
- Exact command / steps: with `max_entries=2`, store `h1` then `h2`,
then re-store the already-present `h2`, under both the old loop (evict
before key dedup) and the new loop (evict only when key is new).
- Observed result: old loop leaves `['h2']` and `get(h1)` returns `None`
(h1 wrongly evicted); new loop leaves `['h1', 'h2']` with `get(h1)`
intact and `h2` updated. The regression test asserts h1 survives and h2
reflects the update.
- Not tested: a live embedding-backed cache round-trip; full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change is a localized reordering of two existing
statements plus a loop guard, verified by the standalone proof and the
new regression test for CI. This is a different defect from the earlier
messages-hash keying fix — that one was about which slot a request maps
to; this one is about eviction dropping a live entry on an in-place
update.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
ae10d6c99d
|
fix(tokenizers): don't tokenize image blocks as text in TiktokenCounter (#2093)
## Description
`TiktokenCounter.count_messages` explodes the token count for any
content block that isn't plain text or an OpenAI `image_url`.
The multi-part content loop handles exactly two shapes:
```python
if part.get("type") == "text":
total += self.count_text(part.get("text", ""))
elif part.get("type") == "image_url":
... # 85 / 170 tokens by detail
else:
total += self.count_text(str(part)) # <-- everything else
```
Every other block shape reaching the `else` gets `str(part)`-ified and
tokenized as text. That includes Anthropic's `{"type": "image",
"source": {"type": "base64", "data": "<...>"}}`, `tool_result`,
`tool_use`, and the Strands SDK blocks. Over the wire the image `data`
is a base64 string, so a 1MB image turns into ~1.4M characters of "text"
and is counted as **~330K tokens** for a single image (a ~218x overcount
in a standalone repro). Anything that relies on the count — budget
gating, the compress/skip decision, savings math — is thrown off for
multimodal requests that route through the tiktoken counter.
The base class already solved this: `BaseTokenizer._count_content_parts`
prices `image`/`image_url`/`input_image` at a flat bounded estimate and
has a comment stating it exists specifically to stop "a 1MB image =
~330K fake tokens". The tiktoken override just never delegated to it for
the non-text shapes.
## Fix
Delegate unknown block shapes in the `else` branch to
`self._count_content_parts([part])` instead of stringifying them. `text`
and `image_url` keep the existing tiktoken-specific handling (including
the 85/170 detail split); everything else now gets the base handler's
bounded pricing.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/tokenizers/tiktoken_counter.py`: the `count_messages`
multi-part `else` branch delegates to the base
`_count_content_parts([part])` rather than `count_text(str(part))`.
- `tests/test_tokenizers.py`: add
`test_count_messages_image_block_is_not_stringified` — a base64 image
block inside list content must stay bounded (well under the tens of
thousands of tokens the blob would produce as text).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/tokenizers/tiktoken_counter.py tests/test_tokenizers.py
All checks passed!
$ python -m py_compile headroom/tokenizers/tiktoken_counter.py tests/test_tokenizers.py
OK
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17`. Importing
`headroom` pulls in the torch/transformers stack and a full `pytest`
gets OOM-killed on this box, so I verified the magnitude with a
dependency-free script that models the old `count_text(str(part))` path
against the base handler's bounded image estimate, and left the full
pytest to CI.
- Exact command / steps: built a ~1MB PNG as an Anthropic `image` block
with the payload base64-encoded (as it arrives over the wire), computed
the old path (`len(str(part)) / ~4` chars-per-token) versus the new path
(base handler prices an image block at a flat 1600).
- Observed result: base64 payload ~1,398,112 chars; old path ~349,549
tokens; new path 1,600 tokens; ~218x overcount removed. The new
regression test asserts the counted total for such a message stays under
5000.
- Not tested: a live tiktoken end-to-end count through the proxy; full
local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" and "type checking" boxes are unchecked
because the full suite imports the ML stack, which I can't run in this
environment; the change is a localized delegation to an existing base
method, verified by the standalone magnitude proof and the new
regression test for CI. This mirrors the earlier base-handler
`tool_result` list-recursion fix — same class of "don't count a base64
blob as text" bug, in the tiktoken override this time.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
b097ef3e25
|
fix(install): don't let host env override the manifest in persistent-docker (#2090)
## Description In persistent-docker deployments a stale host env var can silently override the value the deployment manifest pinned for the container. `build_runtime_command` builds the `docker run` argv in two passes: 1. It emits the manifest's pinned env as `--env NAME=VALUE` (from `base_env` plus the deployment env). 2. It then walks `os.environ` and, for every name matching a `PASSTHROUGH_ENV_PREFIXES` prefix, appends a bare `--env NAME` so the host value is forwarded into the container. A manifest-pinned name and a host-exported name can collide when they share a passthrough prefix. `HEADROOM_BACKEND` is the clearest case: the manifest pins `--env HEADROOM_BACKEND=anthropic` in pass 1, and pass 2 also matches the `HEADROOM_` prefix and appends a bare `--env HEADROOM_BACKEND`. Docker resolves duplicate `--env` flags last-wins, and the bare passthrough comes last, so a stale host export `HEADROOM_BACKEND=anyllm` wins and the container runs a different backend than its deployment config says. `start_persistent_docker` runs the resulting command through `subprocess.run` with the parent process environment, so whatever the operator happened to have exported leaks in and overrides the manifest. The fix skips the bare passthrough for any name the manifest already pins, so the pinned value stands while unrelated host secrets (API keys and so on) are still passed through as before. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/install/runtime.py`: skip the bare `--env NAME` passthrough when `NAME` is already pinned by the manifest (`and name not in runtime_env`). - `tests/test_install/test_runtime.py`: add `test_build_runtime_command_docker_manifest_env_beats_host_passthrough`, which exports a conflicting `HEADROOM_BACKEND` and asserts the command keeps the manifest value and emits no bare passthrough for it. - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 format headroom/install/runtime.py tests/test_install/test_runtime.py 2 files left unchanged $ uvx ruff@0.15.17 check headroom/install/runtime.py tests/test_install/test_runtime.py All checks passed! $ python -m py_compile headroom/install/runtime.py tests/test_install/test_runtime.py OK ``` ## Real Behavior Proof - Environment: local checkout, Python 3.11, `uvx ruff@0.15.17`. - Exact command / steps: ran a standalone script that reproduces the two-pass argv build and models Docker's duplicate `--env` last-wins resolution, with the manifest pinning `HEADROOM_BACKEND=anthropic` and the host exporting `HEADROOM_BACKEND=anyllm`. - Observed result: the old build resolves the effective `HEADROOM_BACKEND` to the host value `anyllm` (bare passthrough wins); the new build keeps the manifest value `anthropic` and emits no bare `HEADROOM_BACKEND` token, while a non-pinned passthrough (`ANTHROPIC_API_KEY`) is still forwarded. - Not tested: I did not run the full `pytest` suite locally because it pulls in the ML stack; the new regression test is left for CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The "unit tests pass locally" and "type checking" boxes are unchecked because the full suite imports the ML dependencies, which I can't run in this environment; the change is a pure function over `build_runtime_command`, verified by the standalone proof above and covered by the new regression test for CI. Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Vinay Gupta
|
bd8de9f382
|
fix(proxy): keep recent stats request rows (#1922)
## Description `/stats.recent_requests` was built from the request-log tail, but then filtered out any row whose compact token fields were incomplete. That made the dashboard/API summary diverge from the raw request counters: `requests.by_model` could show many recent requests for a model while `recent_requests` only showed the few rows with complete compression-token accounting. This fixes the stats payload so the compact `recent_requests` table mirrors the latest request-log rows without masking unknown token accounting as measured zero. Missing/non-finite compact numeric fields now remain `null`, and each row exposes `token_accounting_status` plus `has_exact_tokens` so API clients and the dashboard can distinguish complete, partial, and missing token accounting. Closes #1914 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - Removed the token-completeness filter from `_build_recent_request_payload()`. - Preserved unknown compact recent-request numeric fields as `null`. - Added `token_accounting_status` and `has_exact_tokens` to compact recent request rows. - Updated the dashboard recent-requests table to render unknown token/latency fields as `unknown`. - Hardened `build_session_summary()` against token-incomplete request-log rows and surfaced an `unknown_token_accounting` bucket. - Added TypeScript SDK fields for the compact recent-request stats contract. - Added a regression test covering missing, partial, and complete token-accounting rows. ## Testing - [x] Focused stats regression passes - [x] Adjacent summary/MCP tests pass - [x] TypeScript SDK typecheck/tests pass - [x] Ruff check passes - [x] Ruff format check passes - [x] Diff whitespace check passes - [ ] Full suite not run ### Test Output ```text $ uv run --no-project ... pytest tests/test_proxy_stats_recent_requests.py -q 4 passed, 1 warning $ uv run --no-project ... pytest tests/test_proxy_dashboard_stats_cache.py::test_session_summary_uses_generic_cli_filtering_keys tests/test_proxy_dashboard_stats_cache.py::test_session_summary_surfaces_codex_ws_counters tests/test_ccr_mcp_server.py -q 19 passed, 1 skipped $ npm run typecheck tsc --noEmit $ npm test 296 passed, 33 skipped $ uv run --no-project --with ruff ruff check headroom/proxy/server.py headroom/proxy/cost.py headroom/ccr/mcp_server.py tests/test_proxy_stats_recent_requests.py All checks passed! $ uv run --no-project --with ruff ruff format --check headroom/proxy/server.py headroom/proxy/cost.py headroom/ccr/mcp_server.py tests/test_proxy_stats_recent_requests.py 4 files already formatted $ git diff --check clean ``` ## Verification - Principal engineer review: no blockers after the final finite-number/accounting-status alignment. - Senior developer review: no blockers; implementation and focused coverage look solid for #1914. - Architect/design review: original API/UI contract blocker resolved; unknowns remain visible as `null`/`unknown` instead of measured zero. ## Notes The normal editable `uv run pytest ...` path is blocked locally by the known native build issue in `esaxx-rs` (`fatal error: 'cstdint' file not found`) while building the Rust extension on this machine. I validated the Python-only stats path with a temporary `headroom._core` import stub and `HEADROOM_REQUIRE_RUST_CORE=false`; no repository files were changed for that stub. ## Review Readiness - [x] I have performed a self-review - [ ] This PR is ready for human review |
||
|
Rod Boev
|
10ed14e7f6
|
fix(proxy): keep Kompress warmup off the startup path (#2001)
## Description Proxy startup can enter cached Kompress native model initialization before binding its port. On the RHEL/CentOS 7-family environment reported in #1908, that path terminates in a deterministic `libarrow.so.2400` jemalloc-thread segfault with no Python traceback. Cache-only preload still initializes native libraries when the model is already cached. Defer Kompress model and tokenizer loading out of startup while preserving the existing lazy request path and the eager warmups for non-Kompress components. Closes #1908. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Stop cached and uncached Kompress models from entering native preload during proxy startup. - Record enabled Kompress as deferred until its existing lazy request path needs it. - Preserve disabled-Kompress routing and non-Kompress eager warmups. - Add lifecycle, cache-state, disabled-mode, and warmup-preservation regression coverage. - Document the startup behavior change in the changelog. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_kompress_preload_deferral.py tests/test_kompress_request_nonblocking.py -q`) - [x] Linting passes (`uv run ruff check headroom/transforms/content_router.py tests/test_kompress_preload_deferral.py tests/test_kompress_request_nonblocking.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_kompress_preload_deferral.py tests/test_kompress_request_nonblocking.py -q 17 passed in 1.28s uv run ruff check headroom/transforms/content_router.py tests/test_kompress_preload_deferral.py tests/test_kompress_request_nonblocking.py All checks passed ``` ## Real Behavior Proof - Environment: Red OS 7.3 or equivalent RHEL/CentOS 7-family system, glibc 2.17, Python 3.11, pyarrow 24.0.0, onnxruntime 1.27.0, cached Kompress model - Exact command / steps: start `headroom proxy` with Kompress enabled, wait 30 seconds, query the loopback health endpoint, verify deferred Kompress warmup in logs, then inspect `journalctl -k` for new `libarrow.so.2400` or `jemalloc_bg_thd` faults - Observed result: automated coverage now proves startup avoids the cached Kompress preload boundary, keeps non-Kompress warmups live, and preserves `unavailable` status when dependencies are absent - Not tested: the native reporter-host run ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots Not applicable. ## Additional Notes Focused local validation included `tests/test_kompress_request_nonblocking.py` alongside the startup regression suite. The change is scoped to startup warmup; it does not claim to repair the external `libarrow.so` or jemalloc incompatibility when Kompress later executes. `HEADROOM_DISABLE_KOMPRESS=1` remains the supported narrow workaround for hosts that cannot run the native path. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Rudimar Ronsoni
|
c4ddcb93a7
|
fix(codex): skip sockets in session home overlay (#2104)
## Description Prevent `headroom wrap codex` from failing when the active `CODEX_HOME` contains a Unix socket. The session overlay copied every entry with `shutil.copytree()`, which raises `shutil.Error` when it reaches Git's `fsmonitor--daemon.ipc` socket. The overlay now skips socket entries while continuing to copy regular Codex state and surface unrelated copy errors. Closes #2103 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Ignore filesystem sockets while seeding the temporary Codex session home. - Add a regression test with a real nested `fsmonitor--daemon.ipc` socket and a regular sibling file. ## Testing - [x] Focused unit tests pass (`pytest tests/test_cli/test_wrap_codex.py -q`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New regression test added - [ ] Manual interactive testing performed ### Test Output ```text Docker, Linux arm64, Python 3.12.12 pytest tests/test_cli/test_wrap_codex.py -q 88 passed in 5.92s ruff check . All checks passed! ruff format --check . 1191 files already formatted mypy headroom --ignore-missing-imports Success: no issues found in 469 source files ``` ## Real Behavior Proof - Environment: isolated Docker container on Linux arm64 with Python 3.12.12 and Rust 1.95.0 - Exact command / steps: bind a real Unix socket at `vendor_imports/skills/.git/fsmonitor--daemon.ipc`, then enter `_codex_session_home_overlay()` through the focused pytest regression - Observed result: the regular sibling file is copied, the socket is omitted, the source socket remains active, and the overlay exits cleanly - Not tested: an interactive Codex launch against the live host `~/.codex` ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing focused Codex wrapper tests pass with my changes ## Additional Notes The filter is intentionally limited to socket entries. Permission errors and failures involving regular files still propagate from `shutil.copytree()`. |
||
|
Abhay Singh
|
20968a4fa4
|
fix(wrap/opencode): unwrap removes the rtk block from AGENTS.md (#2025)
## Description `headroom wrap opencode` injects the marker-fenced rtk guidance block — "prefix shell commands with `rtk`" — into **both** instruction files (`headroom/cli/wrap.py`): ```python # wrap opencode project_agents = Path.cwd() / "AGENTS.md" _inject_rtk_instructions(project_agents, verbose=verbose) global_agents = _opencode_home_dir() / "AGENTS.md" _inject_rtk_instructions(global_agents, verbose=verbose) ``` But `unwrap_opencode` only restores the OpenCode config and cleans up MCP servers — it never removes that rtk block. So after `unwrap opencode`, both `AGENTS.md` files still contain the marker-fenced instruction, and a plain `opencode` launch keeps following "prefix shell commands with `rtk`" and fails once the managed rtk binary is off PATH. `unwrap_codex` (#1421) and `unwrap_copilot` both already do this cleanup via `_remove_rtk_instructions`; opencode was simply never given the equivalent — a wrap/unwrap asymmetry. Closes: no issue filed — found while auditing wrap/unwrap symmetry across agents. ## Fix In `unwrap_opencode`, after the MCP cleanup, strip the rtk block from both files it was injected into, mirroring `unwrap_codex`: ```python for _agents_md in (Path.cwd() / "AGENTS.md", _opencode_home_dir() / "AGENTS.md"): if _remove_rtk_instructions(_agents_md): click.echo(f" Removed Headroom rtk instructions from {_agents_md}.") ``` Best-effort and unconditional, matching the existing MCP cleanup and the codex/copilot unwrap paths. `_remove_rtk_instructions` already no-ops when the file or marker is absent. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/cli/wrap.py`: `unwrap_opencode` removes the rtk block from the project and global `AGENTS.md`. - `tests/test_cli/test_wrap_opencode.py`: add `test_unwrap_opencode_removes_rtk_from_agents_md` (wrap injects into both, unwrap removes from both). ## Testing - [x] New regression test added (`tests/test_cli/test_wrap_opencode.py`) - [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17` - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uvx ruff@0.15.17 check headroom/cli/wrap.py tests/test_cli/test_wrap_opencode.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.10, headroom from this branch. Importing `headroom` pulls in the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the wrap→unwrap round-trip through the new Click-runner test (which drives the real command) and reasoned through the marker logic; the full pytest runs on CI. - Exact command / steps: the added test runs `wrap opencode --no-mcp` (asserts `_RTK_MARKER` present in both the project and global `AGENTS.md`), then `unwrap opencode`, and asserts the marker is gone from both. - Observed result (the assertions the test enforces): before the fix, `unwrap opencode` left `_RTK_MARKER` in both files; after the fix both are clean: ```text after wrap: _RTK_MARKER in project AGENTS.md ✓ _RTK_MARKER in global AGENTS.md ✓ after unwrap: _RTK_MARKER absent (project) ✓ _RTK_MARKER absent (global) ✓ ``` - Not tested: launching a real `opencode` binary (mocked in the test, as the existing wrap tests do). Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + the new Click-runner test path; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Directly parallels the merged `unwrap codex` rtk cleanup (#1421); no new dependencies. - @JerrettDavis tagging you — same class as the codex rtk fix, just the opencode side that was missed. Thanks! --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Abhay Singh
|
6ecbdd6b52
|
fix(proxy/savings): don't bill fallback rate for free (0-priced) models (#2024)
## Description
Two savings/cost estimators in `headroom/proxy/savings_tracker.py` read
`input_cost_per_token`
from litellm and use a falsy check to decide whether the price is known:
```python
# _estimate_compression_savings_usd
input_cost_per_token = info.get("input_cost_per_token")
if not input_cost_per_token:
raise RuntimeError("input cost unavailable")
return float(tokens_saved) * float(input_cost_per_token)
except Exception:
return float(tokens_saved) * float(DEFAULT_FALLBACK_INPUT_COST_PER_TOKEN) # $3/M
```
`if not input_cost_per_token` is true for **both** a missing key
(`None`) *and* a legitimate
`0.0`. So a genuinely **free** model — free-tier / local / vendored-at-0
entries, which litellm
does carry — is treated as "price unavailable" and billed the
`DEFAULT_FALLBACK_INPUT_COST_PER_TOKEN`
($3/M) fallback. The result is fabricated dollar savings (and, in
`_estimate_input_cost_usd`,
fabricated cost) for a model that costs nothing. The same pattern is at
`_estimate_input_cost_usd`.
(`_estimate_cache_savings_usd` also uses `if not ...`, but there both
branches correctly resolve
to `$0` for a free model, so it is left unchanged.)
Closes: no issue filed — found while auditing the cost/savings
estimators.
## Fix
Distinguish "missing" from "legitimately zero" with an explicit `is
None` check, so a present
`0.0` flows through as `$0` while an absent key still falls back:
```python
input_cost_per_token = info.get("input_cost_per_token")
if input_cost_per_token is None:
raise RuntimeError("input cost unavailable")
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/savings_tracker.py`: `is None` check (instead of `if
not ...`) in `_estimate_compression_savings_usd` and
`_estimate_input_cost_usd`.
- `tests/test_savings_tracker_zero_price.py`: free model → `$0`, unknown
model → fallback, paid model → real price.
## Testing
- [x] New regression tests added
(`tests/test_savings_tracker_zero_price.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/proxy/savings_tracker.py tests/test_savings_tracker_zero_price.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the estimator logic
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: ran a free model (`input_cost_per_token: 0.0`),
an unknown model (key absent), and a paid model through both the old `if
not ...` and new `is None` logic.
- Observed result: the old logic bills $3/M for the free model; the new
logic charges $0 while still falling back for the unknown model and
leaving the paid model unchanged:
```text
FREE old=3.0000 new=0.0000
UNKNOWN old=3.0000 new=3.0000
PAID old=3.0000 new=3.0000
PHANTOM-COST FIX VERIFIED (free model: $3.00 phantom -> $0.00; unknown still falls back)
```
- Not tested: a full `record_request` round-trip persisted to the
savings file (needs the heavy stack). The fix is confined to the two
estimators and the new tests drive them directly with a stubbed litellm.
Full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Two one-line `is None` fixes plus tests; no new dependencies. Same
falsy-zero class as the `HEADROOM_MIN_TOKENS=0` (#1886) and Copilot
`remaining: 0` (#1997) fixes.
- @JerrettDavis tagging you — small one, surfaces phantom savings for $0
models. Thanks!
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Rod Boev
|
31abb696dd
|
fix(memory): honor explicit store=false on Responses requests (#2017)
## Description This PR addresses the source-backed `store=false` mutation documented inside #1944. Headroom currently injects Responses memory tools by silently flipping explicit `store=false` to `true`, which Codex-backed Responses requests reject. The fix respects explicit `store=false` by skipping only the tool-continuation memory path for those requests. Memory context injection stays unchanged. Refs #1944 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Honor explicit `store=false` on `/v1/responses`. - Skip only Responses memory tools that depend on stored-response continuation. - Preserve current behavior when the client does not opt out of storage. - Add focused regression coverage and a changelog note. ## Testing - [x] Unit tests pass - [x] Linting passes - [ ] Type checking passes - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_openai_responses_context_compaction.py -q 11 passed uv run ruff check headroom/proxy/handlers/openai.py tests/test_openai_responses_context_compaction.py All checks passed uv run ruff format --check headroom/proxy/handlers/openai.py tests/test_openai_responses_context_compaction.py 2 files already formatted ``` ## Real Behavior Proof - Environment: Responses client with explicit `store=false` - Exact command / steps: send a memory-enabled `/v1/responses` request with `store=false` - Observed result: the handler now preserves explicit `store=false` and skips only the Responses memory-tool injection path that depends on stored-response continuation; focused regression coverage proves stored/default requests still allow the path - Not tested: the Desktop disconnect tracked separately in #1944 ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The large Codex Desktop mid-stream disconnect remains a separate external-proof problem. This PR is intentionally limited to the explicit `store=false` mutation proven in the same issue thread. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Abhay Singh
|
f542b70413
|
fix(proxy/memory): capture user text blocks for the retrieval query (#2064)
## Description
`extract_memory_query_sources` (`headroom/proxy/memory_query_policy.py`)
builds the text used to
retrieve relevant memories. It captures `latest_user` **only** when a
user message's `content` is
a plain `str`:
```python
if role == "user":
if isinstance(content, list):
_append_anthropic_tool_results(content, tool_outputs=..., lookback_tools=...)
elif isinstance(content, str) and not latest_user:
latest_user = content
```
But the standard Anthropic `/v1/messages` shape (used by Claude Code)
sends the user turn as a
**list of content blocks** — `content=[{"type":"text","text":"help me
refactor auth"}]`. That
routes into `_append_anthropic_tool_results`, which extracts only
`type=="tool_result"` blocks
and **never reads the `type=="text"` blocks** — so the actual user
prompt is discarded.
Downstream (`handlers/anthropic.py` → `MemoryQuery.from_messages` →
`to_embedding_input`):
- On a **first turn** (no prior assistant/tool context) the embedding
input is `""`, and the
memory handler then returns `None` — **memory injection is silently
skipped entirely**.
- With history present, the query is assembled from stale assistant/tool
context **minus the
current question**, so retrieval targets the wrong text.
Closes: no issue filed — found while auditing the memory retrieval query
policy.
## Fix
In the list-content user branch, also collect the `text` blocks into
`latest_user` (guarded by
`if not latest_user` so the latest turn wins), alongside the existing
tool-result extraction.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/memory_query_policy.py`: capture Anthropic user `text`
blocks into `latest_user`.
- `tests/test_memory_query_policy.py`: add
`test_extract_sources_captures_anthropic_user_text_blocks` and
`test_extract_sources_captures_user_text_alongside_tool_result`.
## Testing
- [x] New regression tests added (`tests/test_memory_query_policy.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/proxy/memory_query_policy.py tests/test_memory_query_policy.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the extraction logic
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: ran a text-block user turn (and a mixed
text+tool_result turn, a plain-string turn, and multiple user turns)
through the old and new logic.
- Observed result: the old logic drops the user text (empty query →
injection skipped); the new logic captures it, still gathers tool
output, and keeps the plain-string / latest-turn behavior:
```text
text-block user: OLD user_text='' NEW user_text='help me refactor auth'
MEMORY QUERY TEXT-BLOCK FIX VERIFIED (old drops user text; new captures it)
```
- Not tested: a full memory retrieval round-trip through the embedder
(needs the heavy stack). The fix is confined to
`extract_memory_query_sources` and the new tests drive it directly. Full
local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Small, contained fix in the query-source extractor; no new
dependencies. The existing
`test_extract_sources_handles_anthropic_tool_result_without_user_text`
still passes (its list turn has no text block).
- @JerrettDavis tagging you — this silently disables memory injection
for the standard Claude Code request shape on a first turn, so it seemed
worth surfacing. Thanks!
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
a5bdc5491f
|
fix(memory/sqlite): don't emit OFFSET without LIMIT in query (#2063)
## Description
`SQLiteMemoryStore.query` (`headroom/memory/adapters/sqlite.py`) builds
pagination like this:
```python
if filter.limit is not None:
query += " LIMIT ?"
params.append(filter.limit)
if filter.offset > 0:
query += " OFFSET ?"
params.append(filter.offset)
```
SQLite's grammar allows `OFFSET` **only** as part of a `LIMIT` clause.
So a `MemoryFilter` with
an offset but no limit produces `... ORDER BY created_at DESC OFFSET ?`,
which SQLite rejects:
```
sqlite3.OperationalError: near "OFFSET": syntax error
```
Both `offset` and `limit` are public `MemoryFilter` fields (`ports.py`:
`limit` defaults to
`None`, `offset` to `0`), so any caller paginating with an offset but no
explicit limit crashes.
Closes: no issue filed — found while auditing the memory store query
builder.
## Fix
When an offset is present without a limit, emit SQLite's unbounded
`LIMIT -1` so `OFFSET` is
grammatically valid:
```python
if filter.offset > 0:
if filter.limit is None:
query += " LIMIT -1"
query += " OFFSET ?"
params.append(filter.offset)
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/memory/adapters/sqlite.py`: emit `LIMIT -1` when paginating
with an offset but no limit.
- `tests/test_memory/test_hierarchical.py`: add
`test_query_offset_without_limit` (offset skips rows; offset past the
end returns `[]`; no crash).
## Testing
- [x] New regression test added
(`tests/test_memory/test_hierarchical.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/memory/adapters/sqlite.py tests/test_memory/test_hierarchical.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I reproduced the exact SQL
against a real stdlib `sqlite3` in-memory DB (the store's query is pure
SQL) and left the full pytest to CI.
- Exact command / steps: built the same `ORDER BY ... [LIMIT] [OFFSET]`
query for `offset=2, limit=None` with the old and new logic and ran it
against a 5-row table.
- Observed result: the old builder raises the exact `OperationalError`;
the new builder skips `offset` rows and returns the rest, and
`LIMIT`-only / `LIMIT`+`OFFSET` still work:
```text
OLD offset-no-limit: OperationalError -> near "OFFSET": syntax error
NEW offset-no-limit: rows=[2, 1, 0]
SQLITE OFFSET-WITHOUT-LIMIT FIX VERIFIED (old crashes; new paginates)
```
- Not tested: the full `HierarchicalMemory` stack (needs the heavy
embedder). The new test drives `SQLiteMemoryStore.query` directly with
`save_batch` + `MemoryFilter`. Full local `pytest` deferred to CI (OOM,
per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone SQLite check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- One-line grammar fix plus a test; no new dependencies.
- @JerrettDavis tagging you — a paginating caller (offset, no limit)
currently crashes the memory store query; quick one. Thanks!
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
4056117d90
|
fix(proxy/gemini): preserve non-text content across the compression round-trip (#2079)
## Description Two related content-loss bugs in the Gemini `contents[]` <-> `messages[]` compression round-trip. Both drop or misplace real user content that entries with **non-text** parts should carry through untouched. They share the same theme (non-text preservation), so they're bundled here as two commits. ### 1. Google batch handler restores preserved entries by the wrong index (`handlers/batch.py`) The `batchGenerateContent` handler restored preserved (non-text) entries with the raw-index loop that commit #836 (`_rebuild_gemini_contents`) replaced in the three non-batch Gemini handlers: ```python for orig_idx, original_content in preserved_contents.items(): if orig_idx < len(optimized_contents): optimized_contents[orig_idx] = original_content ``` `preserved_indices` are indices into the **original** `contents[]`, but `optimized_contents` is a **shorter** list (text-less entries produce no message). Indexing `optimized_contents` by `orig_idx` overwrites the wrong entry and drops any preserved entry whose original index is past the optimized length. For: ```python [user text, model functionCall, user functionResponse, model text] ``` the batch was forwarded to Google as **two** entries: the model's answer overwritten by the functionCall, and the functionResponse dropped. Unlike `gemini.py` there is no `if optimized_messages != messages` gate, so it runs on every mixed batch item. **Fix:** use the shared `_rebuild_gemini_contents` interleaving helper. ### 2. Code-execution parts not detected as non-text (`handlers/gemini.py`) `_has_non_text_parts` only recognized `inlineData`/`fileData`/`functionCall`/`functionResponse`. Gemini's code-execution feature emits `executableCode` and `codeExecutionResult` parts, echoed back in `contents[]` on later turns. Because they weren't detected: - a mixed `text`+`executableCode` entry lost its code payload (only the text survived the round-trip); - a text-less `executableCode`+`codeExecutionResult` entry was treated as a phantom in `_rebuild_gemini_contents` — it consumed the next optimized message, dropping the whole code turn and shifting a following user turn into the model's role slot (corrupting role alternation). **Fix:** add both keys to the non-text detection so those entries are preserved verbatim. Closes: no issue filed — both found while auditing the Gemini contents<->messages round-trip. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/handlers/batch.py`: use `_rebuild_gemini_contents` instead of the raw-index restore loop. - `headroom/proxy/handlers/gemini.py`: recognize `executableCode` / `codeExecutionResult` in `_has_non_text_parts`. - `tests/test_proxy_handlers_batch.py`: add `test_handle_google_batch_create_preserves_functioncall_response_order`, driving the handler with the **real** Gemini converters (the existing batch tests stub them, which hid the bug); mix `GeminiHandlerMixin` into the shared `DummyBatchHandler` so `_rebuild_gemini_contents` is available. - `tests/test_google_multimodal.py`: extend the parametrized `test_each_non_text_key_detected` to the two new keys, and add `test_code_execution_entry_survives`. ## Testing - [x] New regression tests added (`tests/test_proxy_handlers_batch.py`, `tests/test_google_multimodal.py`) - [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17` - [ ] Full `pytest` deferred to CI (local-OOM reason below). ```text $ uvx ruff@0.15.17 check headroom/proxy/handlers/batch.py headroom/proxy/handlers/gemini.py \ tests/test_proxy_handlers_batch.py tests/test_google_multimodal.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.10, headroom from this branch. Importing `headroom` pulls in the torch/transformers stack and a full `pytest` gets OOM-killed on this box, so I verified the interleaving/detection with dependency-free scripts (replicating the Gemini converters, the old loop, and `_rebuild_gemini_contents`) and left the full pytest to CI. - Exact command / steps: ran two standalone scripts. Script 1 rebuilds a Gemini batch request with `preserved_indices` holding a `functionCall`/`functionResponse` pair and compares the old raw-index loop against `_rebuild_gemini_contents`. Script 2 feeds a `codeExecutionResult` entry through `_has_non_text_parts` and the preserve path with and without the two new allowlist keys. Also ran `uvx ruff@0.15.17 check` on the changed files and tests. - Observed result: the old batch loop drops the `functionResponse` and overwrites the answer (4 parts collapse to 2); `_rebuild_gemini_contents` keeps all 4. Without the new keys the code-execution entry is dropped/shifted (2 parts, code absent); with them it survives intact (3 parts, code present). Lint clean. See the two blocks below. Batch fix (bug #1): ```text preserved_indices: [1, 2] OLD result parts: ['text', 'functionCall'] len 2 NEW result parts: ['text', 'functionCall', 'functionResponse', 'text'] len 4 GEMINI BATCH REBUILD FIX VERIFIED (old drops response + overwrites answer; new keeps all 4) ``` Code-execution fix (bug #2): ```text (b) OLD len=2 NEW len=3 (a) OLD has code=False NEW has code=True GEMINI CODE-EXECUTION PRESERVE FIX VERIFIED (old drops/shifts; new keeps intact) ``` - Not tested: a live Google/Gemini round-trip (handlers stubbed, as the existing tests do). Full local `pytest` deferred to CI (OOM, per above). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes — ran lint + standalone logic checks; full pytest deferred to CI (local OOM, disclosed above) - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Two small behavioral changes (one loop -> shared helper, two keys added to an allowlist) plus regression tests; no new dependencies. Both complete/extend the non-text preservation the non-batch handlers already do (the #836 line). - @JerrettDavis tagging you since you reviewed the recent Gemini fixes. Both of these drop content (functionResponse/images on batch; code-execution on the normal round-trip), so they seemed worth surfacing together. Thanks. --------- Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Abhay Singh
|
113894600c
|
fix(cache/ccr): don't evict a live entry on a duplicate store at capacity (#2082)
## Description
`CompressionStore.store` (`headroom/cache/compression_store.py`) runs
eviction **before** it knows
whether the incoming `hash_key` is new or a re-store of an
already-present key:
```python
with self._lock:
self._evict_if_needed() # <-- runs first
existing = self._backend.get(hash_key)
if existing is not None:
... # duplicate / collision: overwrite in place
self._stale_heap_entries += 1
self._backend.set(hash_key, entry)
```
When the store is full and the incoming key **already exists** (a
duplicate re-store),
`_evict_if_needed()` removes the oldest *distinct* entry to "make room"
— but then `set()` merely
overwrites the existing key in place, so no room was ever needed. Net
effect: `count` drops to
`max_entries - 1` and a **live, never-retrieved entry is destroyed**.
That entry's `<<ccr:...>>`
marker, still sitting in the conversation history, then resolves to a
404 on `/v1/retrieve`.
This is not a corner case: the CCR mirror bridge
(`_mirror_single_hash_to_python_store` in
`smart_crusher.py`) re-`store()`s the same `explicit_hash` every turn a
`<<ccr:…>>` marker is
re-encountered, and markers persist across turns — so a full store
silently deletes a live sibling
entry on each duplicate.
Concrete (with the repo's `max_entries=3` fixture): store c0, c1, c2
(full), then re-store c1
(same content ⇒ same hash). Eviction pops the oldest (c0), deletes it,
then c1 is overwritten in
place. Final state: {c1, c2}, count 2, and **c0 is gone** — its marker
is now unredeemable.
Closes: no issue filed — found while auditing the compression store.
## Fix
Decide novelty before evicting: only `_evict_if_needed()` for a
genuinely new key; a
duplicate/replace overwrites in place (no eviction). The
collision/duplicate logging and
stale-heap accounting are unchanged.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/cache/compression_store.py`: `store()` reads `existing`
first and only evicts when the key is new.
- `tests/test_compression_store.py`: add
`test_duplicate_store_at_capacity_does_not_evict` (re-store an existing
hash at capacity keeps all entries and count at `max_entries`).
## Testing
- [x] New regression test added (`tests/test_compression_store.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/cache/compression_store.py tests/test_compression_store.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the store/evict
logic with a dependency-free script (replicating the backend + eviction
heap) and left the full pytest to CI.
- Exact command / steps: filled a `max_entries=3` store with c0/c1/c2,
then re-stored c1 (duplicate), through the old (evict-first) and new
(check-first) logic; also confirmed a genuinely new key still evicts the
oldest.
- Observed result: the old logic drops c0 (count 2); the new keeps all
three; and a new key at capacity still evicts the oldest:
```text
OLD: after duplicate re-store of h1 -> keys=['h1', 'h2'] count=2
NEW: after duplicate re-store of h1 -> keys=['h0', 'h1', 'h2'] count=3
NEW still evicts oldest for a genuinely new key at capacity
DUPLICATE-STORE EVICTION FIX VERIFIED (old drops a live entry; new keeps it)
```
- Not tested: a full proxy CCR round-trip (needs the heavy stack). The
fix is confined to `store()` and the new test drives it directly with
the `max_entries=3` fixture. Existing eviction tests use distinct keys
and stay green. Full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Internal reordering only, no signature change; no call sites or
backend mocks break.
- @JerrettDavis tagging you — this silently drops a live CCR entry
(making its marker 404) whenever a duplicate hash is re-stored at
capacity, which the mirror bridge does routinely. Thanks.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
dfb1d37ed6
|
fix(tokenizers): recurse into list-content tool_result blocks (#2081)
## Description
`_count_content_parts` (`headroom/tokenizers/base.py`) counts a native
Anthropic `tool_result`
block like this:
```python
elif part_type == "tool_result":
content = part.get("content", "")
if isinstance(content, str):
total += self.count_text(content)
else:
total += self._count_serialized(content) # list content -> json.dumps + sample
```
When `content` is a **list of blocks** (the standard shape when a tool
returns an image), it falls
into `_count_serialized`, which `json.dumps`'s the block and counts the
resulting string as text.
A base64 image is a multi-hundred-KB string, so it's priced as ordinary
text:
- a ~200KB screenshot → ~70,000 tokens; a 1MB image → ~350,000 tokens,
- versus the ~1,600 the image branch (`total += 1600`) would assign — a
**50-200x overcount**.
This is the shape computer-use / MCP screenshot tools produce, and it's
reached in production via
`get_tokenizer(model).count_messages` in the Anthropic proxy handler
(the count runs on the raw
inbound messages before any image compression). The effect: a single
screenshot can make the
context read as far larger than reality (appearing to blow past Claude's
200K window), triggering
unnecessary / over-aggressive compression and corrupting the
tokens-before metric.
The sibling **Strands** `toolResult` branch a few lines below already
handles this correctly — it
recurses into list content. Only the native `tool_result` branch was
missed.
Closes: no issue filed — found while auditing the token counters.
## Fix
Recurse into the nested blocks when `tool_result` content is a list,
mirroring the Strands branch,
so an image block is priced structurally (~1600).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/tokenizers/base.py`: `_count_content_parts` recurses into
list-content `tool_result` blocks instead of serializing them.
- `tests/test_tokenizers.py`: add
`test_tool_result_list_recurses_into_image_block` (a base64 image in a
`tool_result` list is priced ~1600, not tens of thousands).
## Testing
- [x] New regression test added (`tests/test_tokenizers.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/tokenizers/base.py tests/test_tokenizers.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the count logic with
a dependency-free script (replicating `_count_content_parts`) and left
the full pytest to CI.
- Exact command / steps: ran a `tool_result` carrying a ~280KB base64
image (nested in a list) through the old serialize path and the new
recurse path.
- Observed result: the old path prices the base64 as text (44x overcount
here); the new path recurses to the image branch (~1600); text-only and
dict content are unchanged:
```text
screenshot-in-tool_result: OLD=70022 NEW=1600 ratio=44x overcount
TOOL_RESULT LIST RECURSE FIX VERIFIED (old prices base64 as text; new -> image 1600)
```
- Not tested: a full proxy count over a real screenshot request (needs
the heavy stack). The fix is confined to `_count_content_parts` and the
new test drives `count_messages` directly. The existing `tool_result`
tests use dict content and stay green. Full local `pytest` deferred to
CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No signature change; internal-only edit to `_count_content_parts`,
consistent with the Strands branch already in the same function.
- @JerrettDavis tagging you — this makes a single tool-returned image
read as tens of thousands of tokens, over-triggering compression, so it
seemed worth surfacing. Thanks.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Tejas Chopra
|
1cc99792ac
|
fix(version): mark source-checkout builds as -dev (#2072)
## Description
`headroom --version` and the dashboard show `0.32.0` from a source
checkout, but the latest published release is `0.31.0`. That `0.32.0` is
not a real release: on a git checkout `get_version()` predicts the
*next* release from conventional commits since the last tag (`v0.31.0` +
`feat:` commits → `0.32.0`) and renders it identically to a shipped
version — so a dev build looks published.
This appends `-dev` on the source-checkout path so a dev build is never
mistaken for the published release.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/_version.py`: the source-checkout branch of `get_version()`
now returns `f"{source_version}-dev"`.
- `tests/test_package_init_lazy.py`: updated the source-tree version
test to assert the `-dev` suffix.
Released installs are unaffected: pip wheels and Docker images with a
baked `BUILD_VERSION` never take the source-checkout path, so they still
report clean release versions (`0.31.0` / `v0.31.0`). The suffix makes
`is_release_version()` return `False` and `normalize_release_version()`
return `None`, which every comparison site already handles — e.g.
`wrap.py`'s `_proxy_needs_version_restart` requires both sides to
normalize, so a dev build short-circuits to "no restart" (no behavior
change).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality (updated the existing
source-tree test)
- [x] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_package_init_lazy.py tests/test_cli_doctor.py -q
============================== 12 passed in 1.79s ==============================
============================== 51 passed in 0.56s ==============================
$ ruff check headroom/_version.py tests/test_package_init_lazy.py
All checks passed!
$ mypy headroom/_version.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: local source checkout (macOS), `.venv`, latest release
tag `v0.31.0`
- Exact command / steps: `headroom --version`
- Observed result:
- Before: `headroom, version 0.32.0` — indistinguishable from a release
- After: `headroom, version 0.32.0-dev`
- Not tested: behavior inside a built Docker image / installed pip wheel
— unchanged by design, since those paths never compute a source-tree
version.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
Scope kept to a bare `-dev` marker, which answers "is this a release?".
Appending the short git SHA (`-dev+g<sha>`) to distinguish individual
dev builds in bug reports is an easy follow-up if wanted. Docs/CHANGELOG
unchecked as N/A for a dev-only version-string fix.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Abhay Singh
|
b4f807f21a
|
fix(proxy/cost): price cache savings by most-used model, not first-seen (#2023)
## Description
`build_prefix_cache_stats` (`headroom/proxy/cost.py`) values each
provider's cache-read savings
using a single "base input price per token". It derives that price by
scanning
`cost_tracker._tokens_sent_by_model` and **breaking on the first**
provider-matching model that
has a price — even though the comment says "most-used model":
```python
# Get the base input price per token for the most-used model on this provider
input_price_per_token = None
if cost_tracker:
for model_name in cost_tracker._tokens_sent_by_model: # insertion order, NOT usage order
...
if is_match:
price_per_1m = cost_tracker._get_list_price(model_name)
if price_per_1m:
input_price_per_token = price_per_1m / 1_000_000
break # first match wins
```
`_tokens_sent_by_model` is insertion-ordered, so the price used depends
on which model was
*recorded first*, not on usage volume. A Claude Code session sends both
Sonnet (main loop) and
Haiku (titles/subagents). If Haiku ($0.80/M) was seen before Sonnet
($3/M), **all** of the
provider's cache-read savings are priced at Haiku's rate — understating
the dashboard's cache
savings by ~3.75×. Reverse the order and it overstates.
Closes: no issue filed — found while auditing the cache-savings pricing.
## Fix
Pick the provider-matching, priced model with the **highest token
volume** instead of breaking
on the first match:
```python
best_tokens = -1
for model_name, tokens_sent in cost_tracker._tokens_sent_by_model.items():
if is_match and tokens_sent > best_tokens:
price_per_1m = cost_tracker._get_list_price(model_name)
if price_per_1m:
input_price_per_token = price_per_1m / 1_000_000
best_tokens = tokens_sent
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/cost.py`: select the highest-volume provider-matching
model (with a known price) rather than the first-recorded one.
- `tests/test_proxy_cache_ttl_metrics.py`: add
`test_prefix_cache_stats_prices_by_most_used_model` using real distinct
per-model prices. (The existing cache-stats tests monkeypatch
`_get_list_price` to a constant `100.0`, which masked the
model-selection logic — hence the bug slipped through.)
## Testing
- [x] New regression test added
(`tests/test_proxy_cache_ttl_metrics.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/proxy/cost.py tests/test_proxy_cache_ttl_metrics.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the selection logic
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: ran a `{haiku: 500, sonnet: 50000}` token map
(Haiku recorded first, Sonnet the higher volume) through both the old
first-match and new highest-volume selection with real prices.
- Observed result: the old logic picks Haiku's $0.80/M (first-inserted);
the new logic picks Sonnet's $3/M (highest volume) and is
insertion-order independent:
```text
OLD picks Haiku price: 0.80/M (first-inserted)
NEW picks Sonnet price: 3.00/M (highest volume)
-> old understates the input price by 3.75x (3.75x)
NEW is insertion-order independent
COST MOST-USED-MODEL FIX VERIFIED
```
- Not tested: rendering the live dashboard (needs the running app). The
fix is confined to the price-selection loop and the new test drives
`build_prefix_cache_stats` directly. Full local `pytest` deferred to CI
(OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No new dependencies; a single-loop change plus a test with realistic
prices.
- @JerrettDavis tagging you — this skews the dashboard's per-provider
cache-savings dollar figure by the ratio between a provider's models
(≈3.75× for Sonnet/Haiku), so it seemed worth surfacing. Thanks!
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
415e03c168
|
fix(mcp/codex): don't clobber an unparseable/non-table config.toml (#2062)
## Description
`CodexRegistrar.register_server` (`headroom/mcp_registry/codex.py`)
guards against clobbering a
user-managed `[mcp_servers.<name>]` entry — but **only inside the `if
existing is not None`
branches**. `existing` comes from `get_server`, which returns `None` in
two cases that are *not*
"nothing there":
1. the `config.toml` is **unparseable** (`_load_toml` catches
`TOMLDecodeError` and returns `{}`), and
2. `mcp_servers` (or `mcp_servers.<name>`) is present but **not a
table** (`get_server` returns `None` via its `isinstance` guards).
With `existing is None`, all three protection branches are skipped and
control falls straight to
`_write_block`, which blindly appends a fresh `[mcp_servers.<name>]`
table.
So for a **valid** TOML file like:
```toml
[mcp_servers]
headroom = "not-a-table"
```
`register_server(headroom_spec)` appends `[mcp_servers.headroom]`,
producing a file that defines
`mcp_servers.headroom` **both** as a string and as a table — a duplicate
key that `tomllib`/codex
then reject, **corrupting a previously-valid user config**. The
unparseable-file case similarly
appends our block into a file that can't be parsed.
This is the exact Codex sibling of the claude (#1660) and opencode
(#1661) clobber-guard fixes;
codex never received it.
Closes: no issue filed — found while auditing the registrars for the
#1660/#1661 class.
## Fix
Add `_unmergeable_reason(name)` — returns why the existing file can't be
safely merged (present
but unparseable, or a non-table `mcp_servers` / `mcp_servers.<name>`),
else `None`. In
`register_server`, when `existing is None`, refuse with
`RegisterStatus.FAILED` (leaving the file
untouched) instead of appending. Absent/empty/valid configs are
unaffected.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/mcp_registry/codex.py`: add `_unmergeable_reason`; refuse in
`register_server` when the existing config is unparseable or defines a
non-table `mcp_servers`/`mcp_servers.<name>`.
- `tests/test_mcp_registry/test_codex_registrar.py`: add tests for
unparseable TOML, non-table `mcp_servers.headroom`, and non-table
`mcp_servers` (all refuse + file untouched).
## Testing
- [x] New regression tests added
(`tests/test_mcp_registry/test_codex_registrar.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/mcp_registry/codex.py tests/test_mcp_registry/test_codex_registrar.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the
`_unmergeable_reason` logic with a dependency-free script (stdlib
`tomllib`) and left the full pytest to CI.
- Exact command / steps: ran the two clobber cases (non-table entry,
unparseable TOML) and the safe cases (absent/empty/valid/other-server)
through the guard.
- Observed result: the guard refuses exactly the two corrupting cases
and allows every valid config:
```text
REFUSE [non-table entry (valid TOML)]: non-table mcp_servers.headroom
REFUSE [unparseable TOML]: not valid TOML (Invalid value (at line 1, column 8))
ALLOW [absent]: reason=None
ALLOW [empty]: reason=None
ALLOW [valid, no mcp_servers]: reason=None
ALLOW [valid, mcp_servers table w/ other server]: reason=None
CODEX CLOBBER-GUARD VERIFIED (refuses non-table/unparseable; allows valid configs)
```
- Not tested: a live `codex` launch reading the config (mocked in the
registrar tests). Full local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Completes the registrar clobber-guard trio (claude #1660, opencode
#1661, codex here); no new dependencies.
- @JerrettDavis tagging you — same class you already reviewed for
claude/opencode, just the codex side. Thanks!
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
石岳峰
|
4f3d5ab341
|
fix(install): add orjson to [proxy] extra for LiteLLM provider backends (#2074)
## Description Add `orjson` to the `[proxy]` extra so `uv tool install "headroom-ai[all]"` installs a runtime dependency required by LiteLLM provider backends (e.g. OpenRouter). ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature - [ ] Breaking change - [ ] Documentation update ## Changes Made - Add `orjson>=3.9.14; platform_python_implementation != 'PyPy'` to `[proxy]`. - Regression test in `tests/test_optional_dependencies.py`. - Minimal `uv.lock` update (proxy/all optional-deps + metadata only). ## Motivation Fixes #2056. `headroom-ai[all]` installs `litellm` (core dep) but not `orjson`. LiteLLM provider backends import `orjson` at runtime; LiteLLM only declares it under `litellm[proxy]`, not base deps. Headroom does not depend on `litellm[proxy]` (would pull the full LiteLLM proxy server stack). ## Testing - `uv run --extra dev python -m pytest tests/test_optional_dependencies.py -q` — 2 passed - `uvx --from ruff==0.15.17 ruff check tests/test_optional_dependencies.py` - `uvx --from ruff==0.15.17 ruff format --check tests/test_optional_dependencies.py` ## Real Behavior Proof - **Setup:** Ubuntu, Python 3.12.3 - **Verified:** dependency graph test confirms `orjson` is selected for `[proxy]` and `[all]` extras after this patch. - **Not tested locally:** full `uv tool install` end-to-end (local sdist build requires Rust/C++ toolchain unavailable in this environment). Reporter workaround `uv tool install ... --with orjson` confirms the missing transitive dep diagnosis. ## Review Readiness - [x] I have performed a self-review of my code - [x] This PR is ready for human review ## Notes - Reporter used Python 3.14.4; `litellm` is intentionally skipped on 3.14 (GH #956). This PR fixes the missing `orjson` install path for supported Python versions using `[all]`. Co-authored-by: syf2211 <syf2211@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Andrew Barnes
|
099c66432b
|
feat(proxy): expose retry delay configuration (#2077)
## Description Expose Headroom's existing retry-delay configuration through the proxy CLI and environment so operators can tune upstream backoff without changing code. Existing 1000 ms / 30000 ms defaults remain unchanged. Closes #2030 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - add CLI flags for initial and maximum upstream retry delays - support `HEADROOM_RETRY_BASE_DELAY_MS` and `HEADROOM_RETRY_MAX_DELAY_MS` - validate non-negative values and forward them into `ProxyConfig` - cover explicit CLI values and environment-variable wiring ## Testing - [x] Focused unit tests pass (`pytest`) - [x] Touched-file linting passes (`ruff check`) - [ ] Type checking passes (`mypy headroom`) — not run - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_cli_proxy_improvements.py::TestRetryDelayValidation tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring::test_headroom_retry_delays_from_env -q 4 passed uv run ruff check headroom/cli/proxy.py tests/test_cli_proxy_improvements.py All checks passed! uv run ruff format --check headroom/cli/proxy.py tests/test_cli_proxy_improvements.py 2 files already formatted ``` ## Real Behavior Proof - Environment: local Python test environment - Exact command / steps: invoke the focused Click CLI tests with explicit delay flags and `HEADROOM_RETRY_*` environment variables - Observed result: all four cases passed; parsed values reached `ProxyConfig`, while invalid negative values were rejected - Not tested: live upstream retry timing or the full repository test suite ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings in the scoped checks - [x] I have added tests that prove the feature works - [x] Relevant existing and new unit tests pass locally - [ ] Documentation and changelog updates — not applicable for these self-documenting CLI options ## Additional Notes The existing runtime backoff helper still caps the base delay against the maximum. This change only exposes values already supported by `ProxyConfig`. Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Andrew Barnes
|
f53f720eb5
|
feat(wrap): allow project RTK instruction opt-out (#2078)
## Description Add an opt-out for project-level RTK guidance when wrapping OpenCode, so teams can preserve an existing repository `AGENTS.md` while still installing RTK and its global OpenCode instructions. Closes #1980 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - add `--no-project-rtk` to `headroom wrap opencode` - leave the repository `AGENTS.md` untouched when requested - continue installing RTK and its global OpenCode guidance - cover preservation of existing team instructions ## Testing - [x] Focused unit test passes (`pytest`) - [x] Touched-file linting passes (`ruff check`) - [ ] Type checking passes (`mypy headroom`) — not run - [x] New test added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_cli/test_wrap_opencode.py::test_wrap_opencode_no_project_rtk_only_skips_project_agents_md -q 1 passed uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_opencode.py All checks passed! uv run ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_opencode.py 2 files already formatted ``` ## Real Behavior Proof - Environment: local Python test environment - Exact command / steps: invoke the focused OpenCode wrapper test with an existing project `AGENTS.md` and project RTK guidance disabled - Observed result: the existing project instructions remain byte-for-byte untouched while the global RTK guidance path still runs - Not tested: manual OpenCode launch or the full repository test suite ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] My changes generate no new warnings in the scoped checks - [x] I have added a test that proves the feature works - [x] Relevant existing and new unit coverage passes locally - [ ] Documentation and changelog updates — not applicable for this self-documenting CLI option ## Additional Notes The opt-out is deliberately narrow: it suppresses only the project `AGENTS.md` mutation, not RTK installation or global OpenCode configuration. Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
GUOHAO LIU
|
12a38d3180
|
feat(proxy): persist per-model savings breakdown in proxy_savings.json (#2055)
## Description
Persist per-model savings breakdown in `proxy_savings.json` so per-model
stats survive proxy restarts (Closes #1913). Previously only the
in-memory Prometheus metrics kept per-model data, which reset on
restart.
Add `by_model` dict keyed by normalized model name, each entry following
the lifetime aggregate shape with a derived `savings_percent`.
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/savings_tracker.py`: add `_empty_by_model_entry()` and
`_normalize_by_model()` helpers; add `_record_by_model_locked()` and
`_by_model_snapshot_locked()` methods to SavingsTracker; include
`by_model` in `_default_state()`, `_sanitize_state()`, `snapshot()`,
`stats_preview()`, and `history_response()`; update `record_request()`
and `record_compression_savings()` to accumulate per-model counters
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_proxy_savings_history.py -x -q
39 passed in 11.26s
$ uv run ruff check headroom/proxy/savings_tracker.py
All checks passed!
```
## Real Behavior Proof
- Environment: Linux, headroom main @
|
||
|
GUOHAO LIU
|
f1663ea557
|
fix(health): exclude kompress from aggregate readiness + adversarial PBT (#2066)
## Description Kompress's model-not-ready state (e.g. after fresh install before first compression cycle) was being incorrectly reported as a proxy-wide failure in the aggregate readiness endpoint, because the health check treated it the same as a hard failure. This PR: 1. Excludes kompress from the aggregate readiness check (Closes #1842) 2. Adds adversarial + PBT tests to verify the exclusion behavior 3. Surfaces model-not-ready state to operators via dedicated log ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/health.py`: exclude kompress from aggregate readiness check - `headroom/proxy/checks.py`: surface kompress model-not-ready state in health log - `tests/test_proxy_health.py`: add adversarial + PBT tests for kompress exclusion - `tests/test_proxy_health.py`: add model-not-ready edge case coverage ## Testing - [x] Unit tests pass - [x] Linting passes - [x] Adversarial edge cases covered ### Test Output ```text $ uv run pytest tests/test_proxy_health.py -x -q -v (adversarial + PBT tests pass) ``` ## Real Behavior Proof - Environment: Linux, headroom main - Exact command / steps: `uv run pytest tests/test_proxy_health.py -x -q` - Observed result: All tests pass including new adversarial/PBT coverage - Not tested: End-to-end with live kompress instance in model-not-ready state ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: lennney <lennney@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
GUOHAO LIU
|
c9a7755a28
|
fix(proxy): handle content-part outputs in Codex Responses compression (#2052)
## Description
Fixes 0% savings when wrapping Codex (Closes #2050). `_slot_text` and
the
lossless-excluded fold in the OpenAI Responses compression path only
handled
`function_call_output` items whose `output` field is a plain string,
silently
skipping items whose `output` is an array of content parts (valid per
OpenAI
spec). Use `_responses_part_text()` — which already handles both — so
these
items reach the ContentRouter and accrue compression savings.
Also extend `_responses_input_item_text_bytes` to count text bytes
inside
content-part arrays in the `output` field, matching its existing
treatment of
the `content` field.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `_slot_text()` (openai.py:1260): use `_responses_part_text()` instead
of
`isinstance(output, str)` to extract text from both string and
content-part
outputs
- Lossless excluded fold (openai.py:1362): same change — use
`_responses_part_text()` so excluded-tool outputs with content parts can
still be losslessly compacted
- `_responses_input_item_text_bytes()` (openai.py:547): extend byte
counting
to handle content-part arrays in the `output` field, matching existing
`content` field handling
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_openai_responses_compression_units.py -x -q
16 passed in 1.19s
$ uv run pytest tests/ -k "openai and responses and compress" -x -q
19 passed, 14 skipped in 18.17s
$ uv run ruff check headroom/proxy/handlers/openai.py
All checks passed!
```
## Real Behavior Proof
- Environment: Linux (6.8.0-124-generic), Python 3.12.3, headroom main @
|
||
|
Abhay Singh
|
cd3d5aa10c
|
fix(tokenizers): price CJK in the fixed-ratio estimator path (#2080)
## Description
`EstimatingTokenCounter.count_text` (`headroom/tokenizers/estimator.py`)
prices dense scripts
(CJK / Kana / Hangul) at ~1 token per 1.5 chars, because at the Latin
ratio they undercount 4-6x.
But that correction is applied **only on the auto-detect path**; the
fixed-ratio early return
divides by the Latin ratio with no adjustment:
```python
if self._fixed_ratio is not None:
return max(1, int(len(text) / self._fixed_ratio + 0.5)) # no CJK split
# auto path (below) does the split:
cjk_chars = self._count_cjk_chars(text)
other_chars = len(text) - cjk_chars
base_count = int(other_chars / ratio + cjk_chars / self.CHARS_PER_TOKEN_CJK + 0.5)
```
The registry builds **every** provider-calibrated counter with a fixed
ratio — Anthropic 3.5,
Google 4.0, Cohere 4.0, Moonshot 3.1 (`registry.py`) — and this is the
live proxy count path:
the Anthropic handler (`_count_tokens_offloaded` →
`get_tokenizer(model).count_messages`) and the
Gemini handlers resolve to these counters. So a CJK-heavy conversation
reads as ~40-55% of its
true token size:
- a large CJK context can fall under the size / backpressure /
background-compression gates and
**skip compression** entirely;
- every `x-headroom-tokens-before` metric for CJK traffic is materially
wrong.
(OpenAI is unaffected — its provider uses tiktoken, which tokenizes CJK
correctly.)
Git blame confirms this is an oversight: commit `
|
||
|
wzy-del
|
c365c7ff81
|
fix(proxy): only queue mid-turn messages for opt-in clients with explicit session header (#1951)
## Description
Mid-turn steering wrongly queues **concurrent independent streams**.
When two streaming `/v1/messages` requests share the same model + system
prompt and arrive concurrently (no `x-headroom-session-id` header), the
proxy misclassifies the second as a "mid-turn message", returns `202
{"event":"headroom_queued"}`, and never forwards it upstream. A standard
Anthropic SDK client that made a *streaming* call receives a non-SSE 202
→ empty event stream → `AssertionError` (`assert
self.__final_message_snapshot is not None` in
`anthropic/lib/streaming/_messages.py`), and fails after retries.
**Root cause.** Without an `x-headroom-session-id` header,
`_get_session_key()` falls back to `md5(model + system[:500])` (mirrors
`prefix_tracker.compute_session_id`). That key is intentionally coarse
and cannot distinguish genuinely concurrent, independent streams that
share a model + system prompt (e.g. a main conversation plus its
background / parallel requests), so the second stream hits `session_key
in self._active_streams` and gets queued.
A queued message is only ever drained back to the client via the custom
`headroom_pending_messages` SSE event, which a standard Anthropic SDK
does not understand — so mid-turn steering is effectively a private
protocol for clients that **opt in** via `x-headroom-session-id`. A
client that never sends the header can never participate in the queue;
for it, the 202 is simply a broken streaming response.
Note: "send a unique header per request" is **not** a workaround — the
same header also drives `prefix_tracker.compute_session_id()`, so
unique-per-stream ids break prompt caching while a shared id keeps
colliding.
Closes #1949
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Add `StreamingMixin._should_queue_mid_turn()` helper that gates
mid-turn queuing behind an explicit `x-headroom-session-id` header.
- Header-less concurrent streams are now forwarded upstream normally;
only opt-in (header-bearing) callers can be queued.
- Prefix-tracker / cache-alignment behavior is untouched — the header
still drives `compute_session_id()` exactly as before.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest tests/test_mid_turn_steering.py
6 passed
```
New test `test_should_queue_only_with_explicit_session_header`: a
header-less concurrent stream must not queue; an explicit-header opt-in
must. All existing `test_mid_turn_steering.py` cases pass an explicit
header and are unaffected.
## Real Behavior Proof
- Environment: macOS, `headroom-ai` 0.30.0 (installed via `uv tool`),
proxy running `headroom proxy --port 8799 --no-http2 --mode cache`,
upstream = an Anthropic-compatible gateway. Client = Hermes Agent
(Anthropic SDK, streaming) driving a main conversation plus concurrent
background/parallel requests that share the same model + system prompt.
- Exact command / steps:
1. Reproduce on stock 0.30.0: concurrent streaming requests without
`x-headroom-session-id` → second stream returns `202
{"event":"headroom_queued"}` → client raises `AssertionError` in
`anthropic/lib/streaming/_messages.py`.
2. Correlate logs: count of `AssertionError` in the client error log vs
count of `202` in the proxy access log for the window — **48 == 48**,
timestamps line up 1:1.
3. Apply this patch to the running package, restart the proxy, re-run
the same concurrent workload.
- Observed result: after the fix, **0 × 202 / all requests 200**, no new
`AssertionError`, and `cache_hit_pct` stayed ~99% (prefix caching
intact). Header-bearing opt-in clients still queue mid-turn as before.
- Not tested: behavior under a client that deliberately sends a
*changing* `x-headroom-session-id` per request (out of scope —
documented as a caching anti-pattern, not a supported mode).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Docs / CHANGELOG unchanged: this is a proxy-internal correctness fix
with no user-facing config surface.
- The fix is deliberately minimal and conservative — it only narrows
*when* queuing engages (explicit opt-in header), leaving the
prefix-tracker, cache-alignment, and body-rewrite paths byte-for-byte
identical.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
alex33d
|
71cbb6aaad
|
feat(proxy): extend output shaping to the OpenAI Responses path (Codex HTTP + WS) (#1943)
## Description
Output shaping (`HEADROOM_OUTPUT_SHAPER`) so far only runs on the
Anthropic `/v1/messages` path (`shape_request` is called only from
`handlers/anthropic.py`). Codex traffic over `/v1/responses` — HTTP and
WebSocket — is never shaped, so subscription Codex users get no
output-token reduction. On a fleet where Codex is the majority of
traffic, that's the largest unshaped output-token pool.
This ports both output-shaping levers to the OpenAI Responses format
with the same contracts as the Anthropic path, wired at the single
funnel all three call paths already share.
Closes #
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
## Changes Made
- `output_shaper.py` — Responses-format counterparts of the existing
levers:
- `classify_responses_turn()`: structural turn classifier over the
`input` item list. The trailing run of tool-output items
(`function_call_output`, `custom_tool_call_output`,
`local_shell_call_output`, `computer_call_output`) is a mechanical
continuation; a trailing user message is a new ask. Error detection is
structural JSON fields only (`exit_code`/`success`/`error`, incl. the
common `{"output":…, "metadata":{…}}` nesting) — never prose — mirroring
the Anthropic `is_error` handling so error turns keep full effort.
- `apply_responses_verbosity_steering()`: appends the byte-stable
steering block to the tail of the `instructions` string. Idempotent per
level, replaced in place on level change — within a conversation every
shaped turn sends identical `instructions` bytes, so the provider prefix
cache stays hot after the first shaped turn (same contract as the
Anthropic system-tail append).
- `route_responses_effort()`: lowers an explicitly-present
`reasoning.effort` on mechanical continuations only. Never injects
`reasoning`, never raises an effort, leaves new asks and error
continuations untouched. Responses gets its own rank table (`minimal`
floor).
- `shape_responses_request()`: the `shape_request` counterpart (same
settings, labels, level-resolution contract).
- `output_savings.py` — `conversation_key_from_responses_body()`:
conversation-stable holdout key (model + first user input text) so whole
conversations land in one A/B arm.
- `handlers/openai.py` — `_shape_openai_responses_payload()` (module
helper, never raises) called inside
`_compress_openai_responses_payload_in_executor`'s closure — the single
funnel for HTTP `/v1/responses`, the WS first frame, and WS subsequent
frames. Runs before compression so the classifier sees the client's
input as sent; serialization stays off the event loop. Shaper labels
ride the existing transforms channel so `outcome.py record_from_labels`
feeds the output-savings ledger unchanged. The `modified` flag is forced
only when shaping actually mutated the payload — an unshaped control-arm
request never breaks byte-faithful forwarding.
- `tests/test_output_shaper_responses.py` — 40 tests covering the
classifier (incl. error sniff + prose-never-inspected), steering
(idempotency, level change, byte stability, non-string instructions),
effort routing (never-inject/never-raise, error/new-ask untouched),
conversation key stability, and the handler helper
(disabled/treatment/full-holdout arms).
Off by default; same env gates as the Anthropic path, all hot-reloadable
via `/admin/runtime-env`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_output_shaper.py tests/test_output_shaper_responses.py \
tests/test_output_savings.py tests/test_verbosity_controller.py \
tests/test_verbosity_learn.py tests/test_codex_openai_contract_parity.py \
tests/test_codex_responses_waste_signals.py -q
154 passed
$ ruff check headroom/proxy/output_shaper.py headroom/proxy/output_savings.py \
headroom/proxy/handlers/openai.py tests/test_output_shaper_responses.py
All checks passed!
$ mypy headroom/proxy/output_shaper.py headroom/proxy/output_savings.py --ignore-missing-imports
exit 0
```
## Real Behavior Proof
- Environment: macOS, Python 3.13, proxy run from this branch
(`PYTHONPATH=. headroom proxy --port 8790`, `HEADROOM_OUTPUT_SHAPER=1
HEADROOM_VERBOSITY_LEVEL=2`), real OpenAI upstream (fake API key —
shaping happens pre-upstream; upstream 401s prove the request went
through the full pipeline).
- Exact command / steps: POST three `/v1/responses` bodies — (a)
trailing `function_call_output` with `exit_code:0` (mechanical), (b)
plain user ask, (c) trailing `function_call_output` with `exit_code:1`
(error).
- Observed result: mechanical turn got `effort:high->low` + L2 steering;
new ask and error turns kept full effort with L2 steering only — proxy
request log `transforms_applied` below.
```text
(a) ["output_shaper:stratum:gpt|mechanical_continuation|xs|tools", "output_shaper:verbosity:L2", "output_shaper:effort:high->low"]
(b) ["output_shaper:stratum:gpt|new_user_ask|xs|notools", "output_shaper:verbosity:L2"]
(c) ["output_shaper:stratum:gpt|error_continuation|xs|notools", "output_shaper:verbosity:L2"]
```
Mechanical turn gets `reasoning.effort` high→low; new ask and error
continuation keep full effort; all three get the byte-stable L2 steering
on the `instructions` tail.
- Not tested: a live Codex WebSocket session end-to-end against the
ChatGPT backend (the WS paths share the exact executor funnel exercised
above);
`test_codex_ws_compression_scheduler.py::test_concurrent_compression_has_no_semaphore_tail`
fails in my env on a clean tree too (no compiled `headroom._core` in a
source checkout) — unrelated.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Abhay Singh
|
1843346283
|
fix(proxy/vertex): route google-publisher requests to the request region (#2069)
## Description
The Vertex `publisher=google` routes forward to a **fixed** upstream
host, ignoring the
request's region. In `headroom/providers/proxy_routes.py`,
`vertex_generate_content`,
`vertex_stream_generate_content`, and `vertex_count_tokens` all do:
```python
del api_version, project, location # <-- location discarded
if publisher == "google":
return await proxy.handle_gemini_generate_content(
request, model,
_api_target(proxy, "vertex"), # <-- single fixed host (default us-central1)
"vertex:google",
)
```
The sibling Anthropic `rawPredict` route already does this correctly —
it keeps `location` and
passes `_vertex_target_for_location(proxy, location)`, which derives the
regional host from the
path.
So a request to
`.../locations/europe-west1/publishers/google/models/gemini-2.0-flash:generateContent`
(with the proxy left at the default Vertex URL) is forwarded to
`https://us-central1-aiplatform.googleapis.com/...europe-west1...` — a
`us-central1` host serving a
`europe-west1` path. Vertex requires the host region to match the path
location, so it rejects the
request. `_vertex_target_for_location` and the region-aware Anthropic
routing landed together in
`
|
||
|
Abhay Singh
|
19201e842f
|
fix(proxy/openai): respect explicit stream_options.include_usage (#2026)
## Description
On the direct OpenAI `/v1/chat/completions` streaming path, the handler
injects
`stream_options.include_usage = True` so it can count tokens from the
trailing usage chunk —
but it does so **unconditionally**, including flipping an explicit
client `include_usage: false`
to `true` (`headroom/proxy/handlers/openai.py`):
```python
if "stream_options" not in body:
body["stream_options"] = {"include_usage": True}
elif isinstance(body.get("stream_options"), dict):
body["stream_options"]["include_usage"] = True # overrides an explicit `false`
```
When the client passed `stream_options: {"include_usage": false}` (or a
dict that set some
other key), the upstream is nevertheless asked for usage and appends a
terminal usage-only
frame:
```
data: {"id":...,"choices":[],"usage":{...}}
data: [DONE]
```
The extremely common client pattern `for chunk in stream:
chunk.choices[0].delta.content`
then raises `IndexError` on that empty-`choices` frame — for a usage
chunk the client
explicitly opted out of.
Closes: no issue filed — found while auditing the streaming
request-shaping.
## Fix
Only fill in `include_usage` when the client left the choice open — no
`stream_options` at all,
or a `stream_options` dict that doesn't mention `include_usage`. An
explicit `true`/`false` is
respected. Extracted into a small `_apply_stream_usage_option(body)`
helper (mirroring the
existing `_normalize_openai_max_tokens`) for a clean unit-test seam:
```python
stream_options = body.get("stream_options")
if stream_options is None:
body["stream_options"] = {"include_usage": True}
elif isinstance(stream_options, dict) and "include_usage" not in stream_options:
stream_options["include_usage"] = True
```
Scope note: this respects an explicit client choice, which is the
unambiguous defect. The
separate question of whether to strip the synthetic usage chunk when
Headroom injected the
option itself (the no-`stream_options` default, kept for token-counting)
touches the raw SSE
byte stream and is intentionally left out of this change.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/handlers/openai.py`: add
`_apply_stream_usage_option(body)` and call it from the streaming chat
path; it no longer overrides an explicit client `include_usage`.
- `tests/test_proxy/test_openai_stream_usage_option.py`: cover explicit
`false` (respected), explicit `true` (preserved), absent (injected), and
dict-without-key (filled in).
## Testing
- [x] New regression tests added
(`tests/test_proxy/test_openai_stream_usage_option.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/openai.py tests/test_proxy/test_openai_stream_usage_option.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the decision logic
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: ran a client body with `stream_options:
{include_usage: false}` (plus the explicit-true, absent, and
dict-without-key cases) through the old unconditional injection and the
new helper.
- Observed result: the old logic flips the client's `false` to `true`;
the new logic respects it:
```text
explicit false: OLD -> {'include_usage': True} NEW -> {'include_usage': False}
INCLUDE_USAGE RESPECT-CLIENT FIX VERIFIED (old flips false->true; new respects false)
```
- Not tested: a full streaming round-trip through a live OpenAI upstream
(needs the heavy stack + a key). The fix is confined to the
request-shaping helper and the new tests drive it directly. Full local
`pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Small, contained change plus a helper and tests; no new dependencies.
The backend-path injection (`test_backend_anyllm` /
`test_backend_streaming_cache_metrics`) is untouched — those pass an
explicit `include_usage: true`, which is preserved.
- @JerrettDavis tagging you — this one makes a client that sent
`include_usage: false` hit an `IndexError` on the usage chunk, so it
seemed worth surfacing. Thanks!
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
JD Davis
|
0415dc8765
|
refactor(proxy): isolate output verbosity policy (#1963)
## Description Extracts output verbosity steering text and sentinel replacement into a pure `output_verbosity_policy` module. `output_shaper` continues to mutate Anthropic/OpenAI request bodies, while the byte-stable steering block and replacement rules now live behind deterministic, directly tested policy functions. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Added `headroom.proxy.output_verbosity_policy` for steering sentinels, level text, `steering_text`, and `replace_or_append_steering_block`. - Updated `output_shaper` to delegate pure steering text/replacement rules while preserving existing public imports and request mutation behavior. - Added direct policy tests for byte-stable steering text, append, replacement, malformed sentinel handling, and idempotency. - Included the current LiteLLM callback signature compatibility shim required for repo-wide mypy on main-based slices. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text python -m pytest tests/test_output_verbosity_policy.py tests/test_output_shaper.py tests/test_litellm_callback.py -q 58 passed in 6.23s python -m ruff check . All checks passed! python -m ruff format --check . 1095 files already formatted python -m mypy headroom --ignore-missing-imports Success: no issues found in 409 source files gitleaks protect --staged --no-banner --redact no leaks found ``` ## Real Behavior Proof - Environment: Windows, Python 3.13.13, clean worktree based on `headroomlabs/main`. - Exact command / steps: targeted pytest, ruff, format check, repo-wide mypy, staged gitleaks scan. - Observed result: output verbosity policy/shaper/callback tests pass; static checks pass; no staged secrets detected. - Not tested: live provider calls; this slice preserves existing request mutation behavior and only moves pure steering rules. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes Documentation and changelog updates are not applicable for this internal architecture slice. PR-specific GHAS checks will be monitored after opening. |
||
|
JD Davis
|
f359f21424
|
refactor(proxy): extract beta header merge policy (#1993)
## Description Extracts deterministic beta-header token parsing and merge rules from `headroom.proxy.helpers` into a focused module. Existing helper names remain available for Anthropic/OpenAI handlers and the session beta tracker. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Added `headroom.proxy.beta_header_merge` for beta token splitting and deterministic merge behavior. - Re-exported the existing `merge_anthropic_beta` and `merge_openai_beta` helper names from `helpers.py` for compatibility. - Added direct unit tests for token splitting, ordering, case-insensitive dedupe, empty required tokens, and provider wrappers. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text python -m pytest tests/test_beta_header_merge.py tests/test_anthropic_beta_session_sticky.py tests/test_openai_beta_session_sticky.py 48 passed in 0.38s python -m ruff check . All checks passed! python -m ruff format --check . 1069 files already formatted python -m mypy headroom --ignore-missing-imports Success: no issues found in 410 source files gitleaks protect --staged --no-banner --redact no leaks found ``` ## Real Behavior Proof - Environment: Windows, Python 3.13.13 - Exact command / steps: Ran focused beta merge tests, existing Anthropic/OpenAI beta sticky suites, full ruff, format check, mypy, and staged gitleaks scan. - Observed result: Existing beta merge and tracker behavior remains green while extracted merge rules are covered directly. - Not tested: Full repository pytest suite locally; CI covers the broader matrix. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes Documentation and changelog updates are not applicable for this internal refactor. The default-branch Dependabot alerts reported during push are pre-existing and unrelated to this PR. |
||
|
Chester
|
d0ecc9a556
|
fix(memory): track MCP retrieval access (#2065)
## Description Track successful native MCP `memory_search` retrievals in persistent memory metadata. Returned memories now increment `access_count` and update `last_accessed`, so MCP usage contributes to memory budget and retention signals. Closes #2061 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add an atomic, deduplicated `MemoryStore.record_access` operation. - Expose access recording through `HierarchicalMemory` and `LocalBackend`, invalidating stale cache entries. - Record only the final active memories actually returned by MCP search. - Fail open if usage metadata cannot be written. - Add SQLite and MCP regression coverage. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text pytest tests/test_memory --ignore=tests/test_memory/test_learn_flag.py -q 368 passed, 142 skipped, 158 warnings in 3.28s pytest tests/test_memory/test_hierarchical.py tests/test_memory/test_mcp_server.py tests/test_memory/test_factory.py -q 40 passed, 53 skipped, 158 warnings in 0.75s ``` ## Real Behavior Proof - Environment: macOS, Python 3.13, SQLite memory store. - Exact command / steps: save two memories; call `record_access` with duplicate IDs plus a missing ID; read both rows; call it again for one row. - Observed result: each existing memory increments once per call, duplicates do not double-count, missing IDs are ignored, and `last_accessed` advances to the supplied timestamp. - Not tested: the full repository suite and `tests/test_memory/test_learn_flag.py`; the source checkout does not include the compiled `headroom._core` Rust extension. Ruff and mypy were not available in the local development environment; CI remains authoritative for those checks. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes Documentation and changelog changes are not included because this is an internal retrieval-metadata correction with no user-facing configuration change. Access tracking is intentionally fail-open so a metadata write failure cannot suppress a valid memory search result. --------- Co-authored-by: xuyidiao <xuyidiao@bytedance.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Abhay Singh
|
ec6e60ea3e
|
fix(proxy/anthropic): scope session id by top-level system prompt (#2070)
## Description
`SessionTrackerStore.compute_session_id`
(`headroom/cache/prefix_tracker.py`) computes a fallback
session id (when no `x-headroom-session-id` header is present) from
`model` + system-prompt text.
But it harvests system text **only** from `messages` entries with `role
== "system"`:
```python
for msg in messages:
if msg.get("role") == "system":
... # collect system text
system_content = json.dumps(system_parts, ...)
key = f"{model}:{system_content}"
```
Anthropic's `/v1/messages` carries the system prompt as a **top-level**
`body["system"]` field —
it never sends `role:"system"` entries inside `messages`. And
`x-headroom-session-id` is a
Headroom-internal header no client sends. So for every genuine Anthropic
request `system_parts`
is empty and the id collapses to `md5(f"{model}:[]")` — **every
conversation on the same model
shares one session id**, and therefore one `PrefixCacheTracker` and all
session-sticky state.
The colliding state cross-contaminates across conversations
(`anthropic.py:1052`):
- sticky `headroom_retrieve` / memory tools keyed purely on `session_id`
(no content guard) get
injected into another conversation's tool list — busting its tools cache
and adding tools its
client never requested;
- sticky `anthropic-beta` header tokens leak across conversations;
- `frozen_message_count` and the per-session compression cache
cross-contaminate.
(The sibling `StreamingMixin._get_session_key` already reads
`body.get("system")` and its docstring
claims to mirror `compute_session_id` — which it did not.)
Closes: no issue filed — found while auditing the session/prefix
tracker.
## Fix
Add an optional `system` parameter to `compute_session_id` and fold its
text (a plain string or a
list of `{"type":"text"}` blocks) into the hash. The Anthropic handler
passes `body.get("system")`.
OpenAI callers don't pass it (defaults to `None`), so their behavior is
unchanged.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/cache/prefix_tracker.py`: `compute_session_id` accepts an
optional `system` and folds it into the id.
- `headroom/proxy/handlers/anthropic.py`: pass
`system=body.get("system")` when computing the session id.
- `tests/test_cache/test_prefix_tracker.py`: add
`test_compute_session_id_distinguishes_top_level_system` (distinct
systems → distinct ids; list-form == string-form; `system=None`
unchanged).
## Testing
- [x] New regression test added
(`tests/test_cache/test_prefix_tracker.py`)
- [x] Linting/formatting clean — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/cache/prefix_tracker.py headroom/proxy/handlers/anthropic.py tests/test_cache/test_prefix_tracker.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the hash logic with
a dependency-free script and left the full pytest to CI.
- Exact command / steps: computed ids for two conversations with the
same model and messages but different top-level `system` prompts,
through the old (never-folds-system) and new logic.
- Observed result: the old logic collapses both to one id (the leak);
the new logic separates them, folds list-form system the same as
string-form, and leaves the `system=None` (OpenAI) path unchanged:
```text
OLD: A=97d8857ba27010bb B=97d8857ba27010bb same=True
NEW: A=1e838c0f6e3980a6 B=18ec49bfa8240852 same=False
SESSION-ID SYSTEM FIX VERIFIED (old collapses Anthropic convos; new separates them)
```
- Not tested: a full two-conversation proxy run asserting no sticky-tool
leakage (needs the heavy stack). The fix is confined to
`compute_session_id` + the one handler call site, and the new test
drives the method directly. Full local `pytest` deferred to CI (OOM, per
above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- Backward-compatible: the new `system` parameter defaults to `None`, so
the OpenAI call sites (`openai.py`) need no change and their session ids
are identical.
- @JerrettDavis tagging you — this one lets one Anthropic conversation's
sticky tools/headers leak into another on the same model, so it seemed
worth surfacing. Thanks!
|
||
|
Abhay Singh
|
cbb775015e
|
fix(subscription/copilot): preserve remaining=0 for exhausted quota (#1997)
## Description
`parse_copilot_quota` reads each category's remaining count like this
(`headroom/subscription/copilot_quota.py`):
```python
remaining = raw.get("remaining") or raw.get("quota_remaining")
```
When a Copilot category is fully consumed, the `/copilot_internal/user`
API sends
`remaining: 0`. The `or` chain treats that legitimate `0` as falsy and —
since the real
per-category payload emits `remaining`, not the `quota_remaining` alias
— collapses it to
`None`:
```python
{"entitlement": 300, "remaining": 0} # fully spent
# raw.get("remaining") -> 0 (falsy) -> raw.get("quota_remaining") -> None -> remaining = None
```
With `remaining = None`, the derived properties break:
- `CopilotQuotaCategory.used` (needs `remaining is not None`) → `None`
instead of `entitlement`
- `used_percent`, when the API also omits `percent_remaining` for that
category → `None`
`to_dict` then emits `remaining: None, used: None, used_percent: None`,
so the dashboard
renders a **100%-exhausted** quota as `used: -` and a **0% green** gauge
— telling the user
they have full quota left when they have none.
Only the `remaining` field has this falsy-zero bug;
`entitlement`/`percent_remaining` are
already parsed with a plain `.get()`, and `overage_count`'s `or 0` is
benign because `0` is
its intended default.
Closes: no issue filed — found while auditing the subscription/quota
parsing.
## Fix
Use an explicit `is None` check, matching how the sibling fields are
parsed:
```python
remaining = raw.get("remaining")
if remaining is None:
remaining = raw.get("quota_remaining")
```
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/subscription/copilot_quota.py`: parse `remaining` with an
explicit `is None` check so a legitimate `0` survives (alias fallback
only when the key is truly absent).
- `tests/test_copilot_quota.py`: add
`test_fully_exhausted_remaining_zero_is_preserved` (remaining `0` →
`used == entitlement`, `used_percent == 100`).
## Testing
- [x] New regression test added (`tests/test_copilot_quota.py`)
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`) — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/subscription/copilot_quota.py tests/test_copilot_quota.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the parse +
`used`/`used_percent` logic with a dependency-free script and left the
full pytest to CI.
- Exact command / steps: ran a fully-exhausted category (`entitlement:
300, remaining: 0`, no alias/percent) through both the old `or`
expression and the new `is None` check, then through the
`used`/`used_percent` property logic.
- Observed result: the old path yields `remaining=None → used=None,
used_percent=None` (the misleading 0%/green); the new path preserves `0`
and reports 100%:
```text
OLD remaining: None used=None used_percent=None
NEW remaining: 0 used=300 used_percent=100.0
-> OLD renders exhausted quota as unknown (0%/green); NEW shows 300/300 = 100%
OK alias fallback + normal values preserved
COPILOT QUOTA ZERO-REMAINING FIX VERIFIED
```
- Not tested: rendering the actual dashboard HTML (needs the running
app). The fix is confined to the parse function and the new test asserts
the parsed `used`/`used_percent`. Full local `pytest` deferred to CI
(OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- One-line falsy-zero fix plus a test; no new dependencies.
- @JerrettDavis tagging you — small one, but it makes the Copilot
dashboard show a spent quota as 100% instead of a green 0%, so worth a
quick look when you have a moment.
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
gglucass
|
112d95b618
|
feat(proxy): report new-content-relative input savings rate in /stats (#2058)
## Description The whole-request savings ratios in `/stats` (`proxy_savings_percent`, `savings_percent`) divide by a per-request recount of the full transcript: a session at turn 200 has had its history counted 200 times into the denominator. Long-running cached sessions — 1M-context models especially, since they never compact — therefore read as ~0% savings no matter how well compression performs on content that actually newly enters context. Field example that motivated this: one day of 1M-context Claude Code traffic saved 641K tokens against ~13.4M tokens of genuinely new content (~4.8%), but displayed as 0.14% because the summed full-transcript denominator was 475M. This PR adds a new-content-relative rate alongside the existing fields: - `tokens.new_input_tokens` — provider-billed non-cache-read input (uncached + cache-write tokens, summed from response usage across providers; the cache accumulators already track both). - `tokens.new_input_savings_percent` — `saved / (new_input + saved)`. Tokens Headroom removed never reached the provider, so they're added back to form the baseline: "of the input that would have newly entered context, what fraction did Headroom remove?" Purely additive — no existing field changes, no new accumulators. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/server.py`: compute `new_input_tokens` from `prefix_cache_stats["totals"]` (already built for `/stats`) and emit the two new fields in the `tokens` block. Rate is guarded on `new_input_tokens > 0`: the cache accumulators only see requests with cache activity, so a deployment with no cache metrics (e.g. Bedrock) would otherwise divide savings by themselves and report ~100% — it reports 0 instead. - `tests/test_stats_new_input_savings_rate.py`: endpoint-level tests via `TestClient(create_app(...))` — a long-cached-session request shows 9.09% new-content rate while `proxy_savings_percent` stays diluted at 0.5%; and the no-cache-usage-data case reports 0. - `CHANGELOG.md`: Features entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --frozen --extra dev pytest tests/test_stats_new_input_savings_rate.py -v tests/test_stats_new_input_savings_rate.py::test_stats_reports_new_input_savings_rate PASSED tests/test_stats_new_input_savings_rate.py::test_stats_new_input_rate_is_zero_without_cache_usage_data PASSED ========================= 2 passed, 1 warning in 6.78s ========================= $ uv run --frozen --extra dev pytest tests/test_proxy_savings_history.py tests/test_dashboard_token_savings.py tests/test_proxy_cache_ttl_metrics.py ======================== 57 passed, 1 warning in 10.70s ======================== $ uv run --frozen --extra dev mypy headroom/proxy/server.py Success: no issues found in 1 source file $ ruff check headroom/proxy/server.py tests/test_stats_new_input_savings_rate.py All checks passed! $ ruff format --check headroom/proxy/server.py tests/test_stats_new_input_savings_rate.py 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS 15 (darwin 24.6.0), Python 3.10 via `uv run --frozen --extra dev`. - Exact command / steps: `TestClient(create_app(config))`, record a request shaped like a late turn of a long cached session (`input_tokens=1_000_000, tokens_saved=5_000, cache_read=900_000, cache_write=45_000, uncached=5_000`), then `GET /stats`. - Observed result: `tokens.new_input_tokens == 50_000`, `tokens.new_input_savings_percent == 9.09`, while `proxy_savings_percent` stays `0.5` — the dilution the new field exists to correct, reproduced side by side. - Not tested: not run against a live proxy with real provider traffic; `ruff`/`mypy` run scoped to the changed files rather than the whole repo. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — JSON API addition; dashboard adoption can follow separately. ## Additional Notes - No linked issue; companion to the nested tool_result image token-counting fix (same investigation — that PR fixes the inflated numerator/denominator counts, this one fixes the metric that divides by transcript recounts). - Caveat worth a reviewer's eye: the numerator (`tokens_saved_total`, local tokenizer) and denominator (provider-reported usage) come from different counters. They're on the same scale, but the rate is honest-approximate rather than exact — comment in code says so. - Deliberately did not change the dashboard headline or any existing field semantics; consumers can opt into the new rate. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Abhay Singh
|
18e5680be3
|
fix(install): only validate requested targets on the manual path (#1659)
## Description
`resolve_targets()` runs the provider-scope "unsupported targets"
validation
**before** it dispatches on `provider_mode`:
```python
if scope == ConfigScope.PROVIDER.value:
unsupported = [t for t in requested if t and t not in valid]
if unsupported:
raise click.ClickException("Provider scope supports only ...; unsupported targets: ...")
if provider_mode == ALL: return [t.value for t in valid_targets] # ignores `requested`
if provider_mode == AUTO: ... # ignores `requested`
# manual: filters `requested`
```
But `all` and `auto` never consult the requested target list — only the
manual
path does. So an unsupported entry that those modes would simply ignore
instead
makes the call raise. Concretely:
```
headroom install apply --scope provider --providers all --target cursor
→ ClickException: Provider scope supports only claude, codex, openclaw, and
opencode; unsupported targets: cursor
```
...when it should just return the full provider set. (`--target` is a
click
`Choice` that accepts all 7 targets regardless of scope/mode, so this is
reachable from the CLI.) The user-scope equivalent,
`resolve_targets("all",
["cursor"])`, happily ignores `cursor` and returns all user targets — so
provider
scope is inconsistent with user scope for identical, ignored input.
Closes: no issue filed — found while auditing `install` target
resolution.
## Fix
Move the provider-scope validation so it runs only on the manual path
(the only
mode that reads `requested`). `all` returns the full provider set and
`auto`
returns detected/default targets, neither raising on an ignored
requested list.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/install/planner.py`: move the provider-scope "unsupported
targets" check below the `all`/`auto` dispatch, into the manual path.
- `tests/test_install/test_planner.py`: regression tests — `all` and
`auto` ignore an unsupported requested target under provider scope; the
manual path still rejects it.
- `CHANGELOG.md`: Bug Fixes entry under Unreleased.
## Testing
- [x] New tests added for the fixed behavior
(`tests/test_install/test_planner.py`)
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`)
- [ ] Full `pytest` deferred to CI (see Real Behavior Proof for the
local-OOM reason).
```text
$ uv run ruff check headroom/install/planner.py tests/test_install/test_planner.py
All checks passed!
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, headroom built from this
branch. Importing `headroom` loads the torch/transformers stack and a
full `pytest` gets OOM-killed on this box, so I verified the control
flow with a dependency-free script (only stdlib) and left the full
pytest to CI.
- Exact command / steps: replicated `resolve_targets`'s control flow
(with the fix, `click.ClickException` stubbed, and stand-in target
lists) in a standalone script — no `headroom` import — and exercised
`all`/`auto`/`manual` under provider scope with an unsupported `cursor`
entry, plus the user-scope and manual-dedup regressions.
- Observed result: `all`/`auto` return the provider targets without
raising, `manual` still raises on the unsupported target, and the
pre-existing manual-dedup and user-scope behavior is unchanged:
```text
OK: all + [cursor] + provider -> provider set (no raise)
OK: auto + [cursor] + provider -> [claude, codex] (no raise)
OK: manual + [cursor] + provider -> raises (preserved)
OK: manual dedupe/filter + user-scope all unchanged
PLANNER LOGIC VERIFIED
```
- Not tested: a full `headroom install apply` end-to-end (would require
the heavy stack and a real deployment); the change is confined to pure
target-list resolution. Full local `pytest` deferred to CI (OOM, per
above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No new dependencies; a control-flow move plus tests.
---------
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
Kenneth Wong
|
560319cef4
|
fix(dashboard): serve per-request metadata to trusted-gateway peers (#1766)
## Description The dashboard's per-request metadata — the `recent_requests` / `request_logs` tail and the `config` block (which echoes upstream API URLs + backend settings) — is gated to loopback callers via `_request_is_loopback`. It requires **both** a loopback peer IP (`request.client.host == 127.0.0.1`) and a loopback `Host` header. When Headroom runs in a **bridge-network container** (Docker/podman, or Apple Containerization / `mocker`), a browser on the host reaches the proxy through the container gateway, so `request.client.host` is the **gateway IP** (e.g. `172.18.0.1`, or `192.168.64.1` on macOS vmnet), not `127.0.0.1`. `include_sensitive` is therefore `False`, and the "Recent Requests" table renders empty even though the operator is browsing locally at `http://127.0.0.1:8787/dashboard`. `curl` from **inside** the container (real `127.0.0.1` peer) confirmed the data is present and populated — only the host-browser path was being stripped. The fix treats a peer inside an operator-configured trusted-gateway CIDR (`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` — the same allow-list already used by `forwarded_headers.py` to sanitize `X-Forwarded-*`) as loopback-equivalent, while **retaining the loopback `Host`-header gate as the DNS-rebinding defence**. It is opt-in and empty by default, so there is **no behavior change** unless the operator explicitly allow-lists their container gateway. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/server.py` — `_request_is_loopback` now: (1) always enforces the loopback `Host`-header gate first; (2) returns `True` for a genuine loopback peer; (3) additionally returns `True` for a peer inside `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` via the existing `peer_is_trusted_gateway` / `load_trusted_gateway_cidrs` helpers. - `tests/test_proxy_loopback_gating.py` — added `test_stats_metadata_served_to_trusted_gateway_peer`: gateway peer stripped without the allow-list, served with it, and DNS-rebinding (non-loopback `Host`) still rejected even for a trusted gateway peer. - `CHANGELOG.md` — Unreleased → Fixed entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_proxy_loopback_gating.py -q 14 passed, 1 warning in 3.56s $ ruff check headroom/proxy/server.py tests/test_proxy_loopback_gating.py All checks passed! ``` ## Real Behavior Proof - Environment: Headroom 0.29.0 in a `mocker compose` (Apple Containerization) bridge container on macOS; host browser at `http://127.0.0.1:8787/dashboard`. - Exact command / steps: before the fix, `mocker compose exec headroom-proxy sh -c 'curl -s http://127.0.0.1:8787/stats'` (peer = real `127.0.0.1`) returned a populated `recent_requests` array, while the host browser saw an empty table. After adding `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` covering the container gateway and recreating, the host browser's dashboard shows the Recent Requests table again. - Observed result: dashboard per-request table restored for the host browser; aggregate-only view unchanged for untrusted network callers. - Not tested: IPv6 gateway CIDRs (the underlying `peer_is_trusted_gateway` supports them; not exercised in this environment). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes Pure opt-in: `HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` is empty by default, so `_request_is_loopback` behavior is byte-identical to today unless an operator allow-lists a gateway CIDR. Reuses the existing trusted-gateway machinery rather than introducing a new config surface. Docs/compose examples intentionally omitted — deployment-specific. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
JD Davis
|
e6243f65c9
|
refactor(providers): split proxy route adapters (#1934)
## Description Refactors provider-specific proxy routing into provider-owned helper modules so `headroom/providers/proxy_routes.py` primarily registers routes and delegates behavior. This keeps Codex, OpenAI Responses/images, model metadata, Vertex, Cloud Code, passthrough target selection, and request path normalization logic testable outside the route table. Closes # ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - Extracted Codex routing helpers for headers, endpoint URLs, image forwarding, response subpaths, and model metadata. - Moved provider target selection, route specs, OpenAI Responses/images helpers, Vertex runtime helpers, Cloud Code path normalization, passthrough telemetry, and request scope normalization into focused modules. - Kept `proxy_routes.py` as route registration/delegation and preserved current-main `/v1/messages` custom-base behavior. - Added focused provider/proxy tests for the extracted modules and route delegation behavior. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text python -m pytest tests/test_package_init_lazy.py::test_codex_package_import_stays_runtime_only tests/test_provider_cloudcode_runtime.py tests/test_provider_codex_endpoints.py tests/test_provider_codex_headers.py tests/test_provider_codex_images.py tests/test_provider_codex_model_metadata.py tests/test_provider_codex_responses.py tests/test_provider_model_metadata.py tests/test_provider_openai_images.py tests/test_provider_openai_responses.py tests/test_provider_proxy_targets.py tests/test_provider_route_specs.py tests/test_provider_vertex_runtime.py tests/test_proxy_request_scope.py tests/test_provider_proxy_routes.py::test_provider_passthrough_routes_forward_expected_targets tests/test_provider_proxy_routes.py::test_proxy_route_helpers_prefer_legacy_targets_and_gemini_passthrough tests/test_provider_proxy_routes.py::test_provider_specific_routes_delegate_to_expected_proxy_handlers tests/test_provider_proxy_routes.py::test_openai_response_websocket_aliases_delegate_to_openai_ws_handler tests/test_provider_proxy_routes.py::test_openai_response_subpath_passthrough_returns_502_on_http_failure tests/test_provider_proxy_routes.py::test_openai_response_subpath_passthrough_uses_openai_target tests/test_provider_proxy_routes.py::test_openai_response_subpath_aliases_and_chatgpt_auth_use_expected_targets tests/test_provider_proxy_routes.py::test_openai_image_routes_use_codex_backend_under_chatgpt_auth tests/test_provider_proxy_routes.py::test_openai_image_codex_response_strips_stale_compression_headers tests/test_provider_proxy_routes.py::test_openai_image_edits_api_key_auth_falls_through_to_openai_passthrough tests/test_provider_proxy_routes.py::test_openai_image_edits_preserves_multipart_body_under_chatgpt_auth tests/test_provider_proxy_routes.py::test_gemini_batch_embed_contents_passthrough_uses_gemini_target tests/test_provider_proxy_routes.py::test_v1_models_fetches_codex_registry_under_chatgpt_auth tests/test_provider_proxy_routes.py::test_v1_models_falls_back_to_synthetic_list_under_chatgpt_auth tests/test_provider_proxy_routes.py::test_v1_models_get_single_dynamic_under_chatgpt_auth tests/test_provider_proxy_routes.py::test_v1_models_still_forwards_under_non_chatgpt_auth tests/test_provider_proxy_routes.py::test_v1_models_routes_claude_code_gateway_discovery_to_anthropic tests/test_provider_proxy_routes.py::test_anthropic_model_metadata_strips_ansi_model_ids tests/test_custom_base_passthrough_telemetry.py tests/test_proxy_passthrough.py tests/test_proxy_google_cloudcode_route_aliases.py tests/test_proxy_project_savings.py::test_with_project_prefix_round_trips_through_split tests/test_vertex_claude_compression.py ============================ 102 passed in 34.83s ============================= python -m ruff check headroom/providers/cloudcode headroom/providers/codex headroom/providers/vertex headroom/providers/model_metadata.py headroom/providers/openai_images.py headroom/providers/openai_responses.py headroom/providers/proxy_targets.py headroom/providers/route_specs.py headroom/providers/proxy_routes.py headroom/proxy/handlers/openai.py headroom/proxy/passthrough.py headroom/proxy/request_scope.py headroom/proxy/project_context.py tests/test_package_init_lazy.py tests/test_provider_cloudcode_runtime.py tests/test_provider_codex_endpoints.py tests/test_provider_codex_headers.py tests/test_provider_codex_images.py tests/test_provider_codex_model_metadata.py tests/test_provider_codex_responses.py tests/test_provider_model_metadata.py tests/test_provider_openai_images.py tests/test_provider_openai_responses.py tests/test_provider_proxy_targets.py tests/test_provider_route_specs.py tests/test_provider_vertex_runtime.py tests/test_proxy_request_scope.py tests/test_provider_proxy_routes.py tests/test_custom_base_passthrough_telemetry.py tests/test_proxy_passthrough.py tests/test_proxy_google_cloudcode_route_aliases.py tests/test_proxy_project_savings.py tests/test_vertex_claude_compression.py All checks passed! python -m compileall -q headroom\providers\cloudcode headroom\providers\codex headroom\providers\vertex headroom\providers\model_metadata.py headroom\providers\openai_images.py headroom\providers\openai_responses.py headroom\providers\proxy_targets.py headroom\providers\route_specs.py headroom\providers\proxy_routes.py headroom\proxy\handlers\openai.py headroom\proxy\passthrough.py headroom\proxy\request_scope.py headroom\proxy\project_context.py # no output; exited 0 git commit -m "refactor(providers): split proxy route adapters" Sync plugin versions.....................................................Passed check for merge conflicts................................................Passed ruff.....................................................................Passed ruff-format..............................................................Passed mypy.....................................................................Passed ``` ## Real Behavior Proof - Environment: Windows PowerShell, Python 3.13.13, branch `jd/provider-route-slices` based on `headroomlabs/main`. - Exact command / steps: Ran the focused provider/proxy pytest suite, focused ruff command, compileall over changed Python modules, and commit hooks. - Observed result: Provider/proxy route behavior tests passed; lint, formatting, and mypy passed. - Not tested: Full pytest suite, live upstream provider calls, and manual end-to-end proxy traffic. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes Documentation and CHANGELOG updates are N/A for this internal refactor. The full pytest suite was not run; coverage here is focused on provider/proxy routing behavior touched by this slice. --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |
||
|
GUOHAO LIU
|
c41cf444c7
|
fix(proxy): allow HEAD method on catch-all passthrough route (#2035)
## Description Claude Code sends `HEAD /` against `ANTHROPIC_BASE_URL` as a connectivity preflight (UA `Bun/1.4.0`). The proxy catch-all route only accepted `GET/POST/PUT/DELETE`, so `HEAD /` returned 405. This made the preflight read as "endpoint down", obscuring the real Remote Control gate message. Closes #2032 ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) ## Changes Made - Add `"HEAD"` to the catch-all passthrough route methods list (`proxy_routes.py:1017`) - `handle_passthrough` already uses `method=request.method` generically — HEAD is forwarded upstream correctly - Add regression test: `test_head_root_returns_200_not_405` ## Testing - [x] Unit test with TestClient - [x] Adversarial: 10 HEAD variants (root, query, nested paths, URL-encoded, XSS query, custom headers, POST-only routes) - [x] Design scan: verified no other `methods=` definitions need HEAD (specific `@app.get` routes auto-handle HEAD) ``` $ uv run pytest tests/test_proxy_passthrough_integration.py tests/test_proxy_cors.py -q 16 passed, 19 skipped # Adversarial: 10 HEAD variants ALL PASSED: 10/10 ✅ HEAD / → 421 (upstream, not 405) ✅ HEAD /?query → 421 ✅ HEAD /v1/models → 401 ✅ HEAD /health → 404 ✅ HEAD /deep/nested → 404 ✅ HEAD /%E4%B8%AD%E6%96%87 → 404 ✅ HEAD / XSS+null query → 421 ✅ HEAD / x-headroom-base-url → 502 ✅ HEAD / Authorization → 421 ✅ HEAD /v1/messages → 404 ``` ## Real Behavior Proof - Environment: Python 3.12, headroom dev install, Ubuntu 24.04 - Exact command / steps: (1) `python3 -c "import urllib.request; req = urllib.request.Request(http://127.0.0.1:8787/, method=HEAD); print(urllib.request.urlopen(req, timeout=5).status)"` → no longer 405; (2) `uv run pytest tests/test_proxy_passthrough_integration.py::test_head_root_returns_200_not_405` → PASSED; (3) `uv run ruff check . && uv run ruff format --check . && uv run mypy headroom --ignore-missing-imports` → 0 errors - Observed result: HEAD / no longer returns 405. Proxy forwards HEAD upstream for all paths. Claude Code preflight reads the correct 421/redirect instead of falsely reporting proxy down. - Not tested: Windows/macOS (route definition is platform-independent) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review Co-authored-by: lennney <lennney@users.noreply.github.com> |
||
|
Abhay Singh
|
38306a331c
|
fix(proxy/gemini): thread savings-profile kwargs into apply() (#1994)
## Description
The native Gemini / Vertex-google compression handlers build the
pipeline call like this
(`headroom/proxy/handlers/gemini.py`, three sites —
`handle_gemini_generate_content` ~L487,
`handle_google_cloudcode_stream` ~L843, `handle_gemini_count_tokens`
~L1104):
```python
result = await self._run_compression_in_executor(
lambda: self.openai_pipeline.apply(
messages=messages,
model=model,
model_limit=context_limit,
context=extract_user_query(messages),
waste_messages=waste_messages,
), # <-- no **proxy_pipeline_kwargs(self.config)
...
)
```
Every other live compression path threads
`proxy_pipeline_kwargs(self.config)` into `apply()`
— `handlers/openai.py` (chat + responses), `handlers/anthropic.py`, and
the `/v1/compress`
endpoint. The Gemini handler never imports or calls it, so the savings
profile and the
ProxyConfig compression knobs never reach the pipeline for Gemini/Vertex
requests.
The proxy pipeline is built with only `transforms` + `provider`
(`server.py`), and
`ContentRouter` reads the accuracy-sensitive knobs per-call from
`**kwargs`. With the kwargs
missing, Gemini falls back to router defaults instead of the
profile/config values:
- `min_tokens_to_compress` → hardcoded **50** instead of the
coding-profile **25** / `config.min_tokens_to_crush`
- `protect_recent` → router default instead of the profile / configured
value
- `target_ratio` → **None** instead of the CLI default **0.4** used
everywhere else
- `max_items_after_crush`, `smart_crusher_with_compaction`,
`force_kompress` → router defaults
So Gemini/Vertex requests compress with a materially different (and
inconsistent) posture than
Claude/Codex/Cursor, and any user-tuned `HEADROOM_SAVINGS_PROFILE` /
`HEADROOM_TARGET_RATIO` / `HEADROOM_MIN_TOKENS` /
`HEADROOM_PROTECT_RECENT` is ignored on this
path.
This is the exact bug **#1534** fixed for the OpenAI chat path.
Closes: no issue filed — found while auditing profile/config threading
across the provider handlers.
## Fix
Import `proxy_pipeline_kwargs` (as `openai.py`/`anthropic.py` do) and
add
`**proxy_pipeline_kwargs(self.config)` to all three Gemini
`openai_pipeline.apply(...)` calls.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/handlers/gemini.py`: import `proxy_pipeline_kwargs`;
thread `**proxy_pipeline_kwargs(self.config)` into the three
`openai_pipeline.apply(...)` call sites (generateContent, Cloud Code
stream, countTokens).
- `tests/test_proxy/test_gemini_savings_profile.py`: drive the native
`/v1beta/models/{model}:generateContent` route with
`savings_profile="agent-90"` and assert the profile knobs
(`compress_user_messages`, `target_ratio`, `min_tokens_to_compress`,
`compress_system_messages`) reach `apply()`.
## Testing
- [x] New regression test added
(`tests/test_proxy/test_gemini_savings_profile.py`), mirroring the #1534
chat-path test
- [x] Linting passes (`ruff check`) and formatting is clean (`ruff
format --check`) — run with the CI-pinned `ruff==0.15.17`
- [ ] Full `pytest` deferred to CI (local-OOM reason below).
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/gemini.py tests/test_proxy/test_gemini_savings_profile.py
All checks passed!
$ uvx ruff@0.15.17 format --check headroom/proxy/handlers/gemini.py tests/test_proxy/test_gemini_savings_profile.py
2 files already formatted
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, headroom from this branch.
Importing `headroom` pulls in the torch/transformers stack and a full
`pytest` gets OOM-killed on this box, so I verified the kwargs threading
with a dependency-free script and left the full pytest to CI.
- Exact command / steps: mechanically reproduced the call site —
`apply(**base)` (old) vs `apply(**base,
**proxy_pipeline_kwargs(config))` (new) — and captured the kwargs each
produced.
- Observed result: the old call omits the profile knobs entirely; the
new call threads them:
```text
OLD gemini apply() kwargs: ['context', 'messages', 'model', 'model_limit', 'waste_messages']
-> profile knobs DROPPED (savings profile / config ignored)
NEW gemini apply() kwargs: ['compress_system_messages', 'compress_user_messages', 'context',
'max_items_after_crush', 'messages', 'min_tokens_to_compress', 'model', 'model_limit',
'protect_recent', 'target_ratio', 'waste_messages']
-> profile knobs THREADED: target_ratio=0.10, min_tokens_to_compress=120, compress_user/system=True
GEMINI KWARGS THREADING VERIFIED
```
- Not tested: a full request through a live Gemini/Vertex upstream
(needs the heavy stack + a key). The new test drives the native route
with a mocked upstream and asserts the kwargs reach `apply()`. Full
local `pytest` deferred to CI (OOM, per above).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes — ran
lint + a standalone logic check; full pytest deferred to CI (local OOM,
disclosed above)
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No new dependencies; one import plus three call-site kwargs and a
test.
- @JerrettDavis tagging you — this is the Gemini sibling of the #1534
chat-path fix; the profile/config knobs are currently ignored for the
whole Gemini/Vertex surface, so it may be worth a look when you have a
moment.
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
Yevhen Koval
|
ec55ddcfb3
|
feat(transforms): first-class C# support in CodeAwareCompressor (#1926)
Refs #1664 ## Description First-class C# support in `CodeAwareCompressor` via the tree-sitter `csharp` grammar, at parity with Java/C++/Rust: `using` directives, namespace headers, and type/member signatures preserved verbatim; method/constructor/destructor/operator/local-function bodies compressed; malformed input passes through unchanged. **No new dependencies** — the grammar ships inside the already-pinned `tree-sitter-language-pack==0.13.0` (resolves only as `"csharp"`; `c_sharp`/`cs` raise `LookupError`). Spec and maintainer go-ahead in the issue. Closes #1664 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `CodeLanguage.CSHARP` + full `_LANG_CONFIGS` entry; `_LANGUAGE_PREFILTER` and `content_detector` patterns chosen to be C#-distinctive (so Java doesn't mis-tag). - New data-driven `LangConfig` fields (pattern of #1334's `class_body_node_types`): `container_node_types` — block-scoped `namespace { }` routed through class compression so members compress without the wrapper being re-emitted verbatim; `opaque_node_types` — `#if`…`#endif` wrappers preserved verbatim without recursion (recursing + wrapper re-emit duplicated whole files, up to ~1.9x input on real repos); `#if` blocks wrapping only usings are emitted with the imports so they stay ahead of type declarations. - Shared-path fixes surfaced by real C# repos, each guarded and covered by a fail-before test: keep an Allman `{` on its own line in class reconstruction (K&R path byte-for-byte unchanged; Allman Java now compresses instead of falling back); line-based child extraction no longer swallows the following line for nodes ending at column 0 (C# `#region`/`#endregion` span their trailing newline — the over-slice duplicated the next member's signature or the closing brace); uncaptured top-level nodes preceding the first captured node (license banners, `#region License`) are emitted first instead of relocated below the code (tree-sitter-c-sharp rejects top-level `#region` after a type declaration, so relocation forfeited compression for the whole file). - `TestCSharpSupport` (8 tests) + a C# case in the parametrized member-container test; CHANGELOG entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_transforms/test_code_compressor.py -q 2 failed, 78 passed, 1 warning, 4 errors # the 2 failures / 4 errors reproduce # identically on main in the same env # (network-dependent tokenizer setup) Fail-before: with both changed sources reverted to main, the new C#-scoped selection reports "10 failed, 5 passed" (the 5 other languages keep passing); on the branch: "15 passed". $ ruff check headroom/transforms/code_compressor.py headroom/transforms/content_detector.py tests/test_transforms/test_code_compressor.py All checks passed! $ ruff format --check <same files> 3 files already formatted ``` ## Real Behavior Proof - Environment: Linux 6.8.0 aarch64, Python 3.10.12; `uv run --no-project --with "tree-sitter-language-pack==0.13.0" --with "tree-sitter>=0.25.2,<0.26" --with "pydantic>=2.0.0"`; real `CodeAwareCompressor` (`CodeCompressorConfig(enable_ccr=False)`, otherwise defaults), no mocks. - Exact command / steps: cloned two real .NET repos at depth 1 (`github.com/JamesNK/Newtonsoft.Json @ 4f73e74`, `github.com/App-vNext/Polly @ 7a1d10f`), ran `python proof_csharp.py <repo>` over every `.cs` file (chars/4 token estimate; tiktoken BPE download unavailable in my sandbox). Script in the collapsed section below. - Observed result: 16.1% tokens saved on Newtonsoft.Json (945/945 syntax-valid), 37.8% on Polly (797/797 syntax-valid), zero content duplication; full output: ```text repo: Newtonsoft.Json (945 .cs files) tokens before: 1,777,691 after: 1,490,629 saved: 287,062 (16.1%) files compressed: 479 pass-through: 466 inflated(>before): 19 syntax_valid: 945/945 latency ms P50: 0.7 P95: 18.7 P99: 44.1 max: 255.0 mean: 3.5 repo: Polly (797 .cs files) tokens before: 1,100,523 after: 684,303 saved: 416,220 (37.8%) files compressed: 693 pass-through: 104 inflated(>before): 15 syntax_valid: 797/797 latency ms P50: 0.8 P95: 11.6 P99: 28.9 max: 74.1 mean: 2.4 ``` After rebasing onto current `main` (which touched the same transform files via #1906/#1747/#1668) I re-ran the Polly proof on the rebased tree: 37.8% saved, 797/797 syntax-valid, P99 28.5ms — unchanged. Signatures/properties verbatim, bodies elided with call summaries, `using` order and preproc balance intact; residual "inflated" files are +2…+209 chars of assembly blank lines, not duplicated content. Newtonsoft is the adversarial case (multi-targeting: heavy `#if`, `#region`, Allman) — its conditional regions stay verbatim by design. Latency at parity with Java (<50ms P99; max is the pre-existing symbol-analysis cost on ~1800+-line files, shared with other languages). - Not tested: proxy end-to-end path with C# through `ContentRouter` (tested the `CodeAwareCompressor` API directly); CCR retrieval round-trips (`enable_ccr=False` in proof runs); exact tiktoken counts (chars/4 estimate — relative ratios are tokenizer-independent); Windows/macOS; full native `uv run pytest` with the Rust extension (ran the complete `test_code_compressor.py` in a lightweight venv; its 2 failures/4 errors reproduce identically on `main`); `mypy`. <details> <summary>proof_csharp.py (reproducible)</summary> ```python """Real behavior proof: run the real CodeAwareCompressor over a .NET repo.""" import pathlib import statistics import sys import time from headroom.transforms.code_compressor import ( CodeAwareCompressor, CodeCompressorConfig, ) try: import tiktoken ENC = tiktoken.get_encoding("cl100k_base") def toks(s: str) -> int: return len(ENC.encode(s, disallowed_special=())) except Exception: def toks(s: str) -> int: return len(s) // 4 target = pathlib.Path(sys.argv[1]) comp = CodeAwareCompressor(CodeCompressorConfig(enable_ccr=False)) tot_before = tot_after = 0 n_files = n_compressed = n_valid = n_passthrough = n_inflated = 0 times_ms: list[float] = [] for f in sorted(target.rglob("*.cs")): try: code = f.read_text(encoding="utf-8-sig", errors="replace") except OSError: continue t0 = time.perf_counter() r = comp.compress(code, language="csharp") times_ms.append((time.perf_counter() - t0) * 1000) n_files += 1 b, a = toks(code), toks(r.compressed) tot_before += b tot_after += a if r.compressed == code: n_passthrough += 1 else: n_compressed += 1 if r.syntax_valid: n_valid += 1 if a > b: n_inflated += 1 times_ms.sort() p = lambda q: times_ms[min(int(len(times_ms) * q), len(times_ms) - 1)] print(f"repo: {target.name} ({n_files} .cs files)") print(f" tokens before: {tot_before:,} after: {tot_after:,} saved: {tot_before - tot_after:,} ({(1 - tot_after / tot_before) * 100:.1f}%)") print(f" files compressed: {n_compressed} pass-through: {n_passthrough} inflated(>before): {n_inflated}") print(f" syntax_valid: {n_valid}/{n_files}") print(f" latency ms P50: {p(0.50):.1f} P95: {p(0.95):.1f} P99: {p(0.99):.1f} max: {times_ms[-1]:.1f} mean: {statistics.mean(times_ms):.1f}") ``` </details> ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — terminal evidence above. ## Additional Notes - Dependency justification: none added, none bumped; the `csharp` grammar is inside the already-pinned `tree-sitter-language-pack==0.13.0` wheel; `uv.lock` untouched. - Architecture: malformed input passes through byte-identical; every risky construct prefers the false negative (verbatim) over corruption; invalid reassembly falls back to the original via the existing validation gate (observed live); no new imports at module load; P99 <50ms on both proof repos. - Known v1 limitations (deliberate false negatives, possible follow-ups): expression-bodied members and property accessor bodies stay verbatim; declarations inside `#if` regions stay verbatim. - Related pre-existing finding, out of scope: C/C++ exhibit the same `#if`-wrapper duplication on `main` (an `#if`-wrapped C++ class is emitted twice, ratio 1.62). Happy to file separately. - `mypy` unchecked above because I did not run it in my environment. |