## Problem
pyo3 marks `_native::Parser` as `#[pyclass(unsendable)]`, which causes a
hard thread-assertion panic when a parser created on one thread is
accessed from another:
```
thread '<unnamed>' panicked at pyo3-0.28.3/src/impl_/pyclass.rs:1055:9:
assertion `left == right` failed: _native::Parser is unsendable, but sent to another thread
left: ThreadId(2)
right: ThreadId(1)
```
The prior implementation stored parsers in a module-level `dict[str,
Any]` (`_tree_sitter_languages`). When `_run_compression_in_executor`
dispatched compression work to a `ThreadPoolExecutor`, pool workers
grabbed parsers from that shared dict that were originally created on
the main asyncio thread and panicked.
This produces a 500 on every request where code compression is attempted
via a pool thread.
## Fix
Replace the global dict with `threading.local()` so each thread creates
and owns its own parser instances. No cross-thread parser access is
possible.
```python
# before
_tree_sitter_languages: dict[str, Any] = {} # shared — crosses threads
# after
_tree_sitter_local = threading.local() # per-thread — isolated
```
`is_tree_sitter_loaded()` and `unload_tree_sitter()` updated to operate
on the current thread's local cache (semantics unchanged for
single-threaded callers).
## Tests
9 regression tests added in
`tests/test_transforms/test_tree_sitter_thread_safety.py`:
- Thread isolation: two threads get distinct parser instances
- Within-thread reuse: same thread gets the same cached instance
- Thread pool: parsers usable from `ThreadPoolExecutor` workers without
panic
- Concurrent workers: each distinct pool thread owns a unique parser
- `is_tree_sitter_loaded` / `unload_tree_sitter` lifecycle
Also adds a `filterwarnings` entry for
`PytestUnraisableExceptionWarning`: pyo3 emits this when short-lived
test threads drop parsers at teardown; it does not occur in production
where pool threads are long-lived.
## Relation to #564
PR #564 proposes the same `threading.local()` approach but was blocked
on missing tests (`CHANGES_REQUESTED`). This PR includes the full test
suite.
> ⚠️ This PR was opened using Codex (gpt-5.5 on `xhigh`)
Fixes#730.
## Summary
- centralize Codex config path resolution in `headroom wrap codex` so it
honors `CODEX_HOME` when set
- make the Codex MCP registrar use `$CODEX_HOME/config.toml` instead of
always writing to `~/.codex/config.toml`
- route optional memory MCP and global Codex `AGENTS.md` injection
through the same Codex home helper
- make `headroom unwrap codex` print a warning, but still succeed, when
`CODEX_HOME` is unset and the default Codex config has no Headroom
markers
- add regression coverage for provider injection, prepare-only wrapping,
MCP registration under a custom Codex home, and the ambiguous unwrap
warning
- update `CHANGELOG.md` under `Unreleased > Bug Fixes`
## Real behavior proof
Setup tested on:
- Linux `7.0.10-2-cachyos`
- Python 3.14.3 via `uv`
- local fork branch `fix/codex-home`
- custom Codex home created outside `~/.codex`
Exact command run after the patch:
```bash
tmp_home=$(mktemp -d)
mkdir -p "$tmp_home/codex_custom"
HOME="$tmp_home" USERPROFILE="$tmp_home" CODEX_HOME="$tmp_home/codex_custom" \
UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \
uv run --with fastapi --with uvicorn --with httpx --with websockets \
headroom wrap codex --no-context-tool --no-serena --prepare-only --port 8787
find "$tmp_home" -maxdepth 3 -type f -print | sort
sed -n '1,140p' "$tmp_home/codex_custom/config.toml"
test -e "$tmp_home/.codex/config.toml" && echo yes || echo no
HOME="$tmp_home" USERPROFILE="$tmp_home" \
UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 \
uv run --with fastapi --with uvicorn --with httpx --with websockets \
headroom unwrap codex --no-stop-proxy
```
After-fix evidence + observed result:
Interactive check:
- A full `CODEX_HOME="$HOME/.codex_p" headroom wrap codex --no-serena`
launch was tested locally after the patch and worked as expected.
- The prepare-only proof below shows the same config path behavior
without requiring an interactive Codex session in CI/reviewer
environments.
```text
MCP retrieve tool: registered (restart OpenAI Codex CLI if it was already running)
Codex config: injected Headroom provider (WS + HTTP) into /tmp/tmp.UPUvloGNYE/codex_custom/config.toml
--- files ---
/tmp/tmp.UPUvloGNYE/codex_custom/config.toml
/tmp/tmp.UPUvloGNYE/codex_custom/config.toml.headroom-backup
--- custom config ---
# --- Headroom proxy (auto-injected by headroom wrap codex) ---
model_provider = "headroom"
openai_base_url = "http://127.0.0.1:8787/v1"
# --- end Headroom ---
# --- Headroom MCP server ---
[mcp_servers.headroom]
command = "headroom"
args = ["mcp", "serve"]
# --- end Headroom MCP server ---
# --- Headroom proxy (auto-injected by headroom wrap codex) ---
[model_providers.headroom]
name = "OpenAI via Headroom proxy"
base_url = "http://127.0.0.1:8787/v1"
supports_websockets = true
# --- end Headroom ---
--- default config exists? ---
no
Warning: found no Headroom wrap markers in the default Codex config. If you wrapped Codex with CODEX_HOME, rerun unwrap with the same environment variable, e.g. CODEX_HOME=/path/to/codex-home headroom unwrap codex.
Nothing to undo: .../.codex/config.toml has no Headroom wrap markers.
```
What I did not test:
- Windows/macOS path behavior
- full repository test suite, because this local Python 3.14 environment
hits optional dependency/build constraints outside this patch
## Testing
```bash
UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with pytest --with fastapi --with uvicorn --with httpx --with websockets pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_cli/test_wrap_codex.py -q
UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py
UV_SKIP_WHEEL_FILENAME_CHECK=1 PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 uv run --with ruff ruff format --check headroom/cli/wrap.py headroom/mcp_registry/codex.py tests/test_cli/test_wrap_codex.py tests/test_mcp_registry/test_codex_registrar.py
```
Results:
```text
63 passed, 1 warning in 1.48s
All checks passed!
4 files already formatted
```
Notes:
- Python 3.14 required `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` for the
editable build.
- `uv` required `UV_SKIP_WHEEL_FILENAME_CHECK=1` because the existing
`uv.lock` has a GitPython wheel filename/version mismatch.
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/chopratejas/headroom/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Description
Brief description of changes and motivation.
Fixes#781
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
add light mode to dashboard
## Summary
- Adds `evaluate_tool_schema_compaction()` and
`generate_tool_schema_cases()` to `CompressionOnlyRunner`
- Four built-in cases cover the property-name vs annotation-key
distinction: `title`, `deprecated`, `readOnly`, and all four at once
- Each case asserts: byte count shrinks (annotations stripped), all
`must_preserve` property names survive in `properties`, no `required`
entry points to a stripped key, root-level schema annotations
(`$schema`, `title`) are dropped
- Wires the new eval into `.github/workflows/eval.yml` alongside the
existing CCR round-trip smoke step — runs on every PR touching
`headroom/transforms/**`, `headroom/evals/**`, or
`headroom/compress.py`, at zero API cost
## Motivation
PR #785 fixed a bug where the compaction pass stripped property *names*
that happened to match DROP_KEYS (e.g. a field literally called
`title`). This eval encodes the invariant that fix established so future
changes to the compaction logic can't silently regress it.
## Test plan
- [ ] `pytest
tests/test_evals_metrics.py::test_tool_schema_compaction_integrity` —
all 4 cases pass, `total_tokens_saved > 0`
- [ ] CI smoke step "Run tool schema compaction integrity eval (zero
cost)" passes with no API key required
## Real behavior proof
```
$ pytest tests/test_evals_metrics.py::test_tool_schema_compaction_integrity -v
PASSED [100%]
1 passed in 0.53s
```
Zero API calls, zero cost. Runs in under 1 second.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- Lazy-import `create_app`/`run_server` in `headroom/proxy/__init__.py`
via PEP 562 `__getattr__` to prevent CLI crash when `fastapi` is not
installed (i.e., installed without `[proxy]` extras)
- Fix `.pre-commit-config.yaml` to use `python3` instead of `python`
(unavailable on macOS Homebrew)
- Add graceful `ImportError` skip in `scripts/sync-plugin-versions.py`
for environments without dev dependencies
Fixes#441
## Test plan
- [x] `headroom --help` works without `[proxy]` extras installed
- [x] `headroom proxy --help` works with `[proxy]` extras installed
- [x] `headroom proxy --port 18787` starts and serves traffic
- [x] Lazy imports resolve correctly: `from headroom.proxy import
create_app, run_server`
- [x] `AttributeError` raised for invalid attributes on `headroom.proxy`
- [x] Pre-commit hooks pass (ruff, ruff-format, mypy,
sync-plugin-versions)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Code Bot <claude-code@smartwatermelon.github>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
Replaces `chopratejas/kompress-base` with
**`chopratejas/kompress-v2-base`** as the default Kompress
text-compression model (the fallback for content not handled by
structured compressors), using a new **weight-only int8 ONNX** artifact
that is fp32-equivalent at 2.2x less memory.
## Why
v2 is the same dual-head ModernBERT (token classifier + span CNN),
LoRA-trained on the v2 dataset. The HF repo originally shipped PyTorch
weights only — pointing Headroom at it naively would have forced the
heavy `[ml]` (torch) path on every `[proxy]` install. We exported ONNX
artifacts reproducing the v1 loader contract (single `final_scores`
output) and published them to the HF repo.
## Eval (labeled dataset_v2 test split, n=500, threshold 0.5)
| artifact | size | f1 | must_keep_recall | keep_rate | fp32 agreement |
|---|---|---|---|---|---|
| fp32 (reference) | 601 MB | 0.9128 | 0.9770 | 0.8100 | 100% |
| **int8-wo (default)** | **261 MB** | **0.9130** | **0.9765** |
**0.8097** | **99.6%** |
| fp16 | 301 MB | 0.9129 | 0.9771 | 0.8099 | 99.9% |
| int4-wo | 230 MB | 0.9102 | 0.9825 | 0.8354 ⚠️ | 94.5% |
| int8-dynamic | 271 MB | 0.9041 | 0.9868 | 0.8857 ⚠️ | 90.5% |
Weight-only int8 (MatMulNBits) keeps activations fp32, avoiding the
upward score bias that makes dynamic int8 keep ~7% more tokens (≈40%
less compression savings). Quantized candidates were generated and
eval-gated by a Modal job in the kompress repo
(`modal_jobs/export_onnx_v2.py`) against the labeled test split.
## Changes
- Default model id → `chopratejas/kompress-v2-base`
- ONNX artifact resolution tries candidates in order (**int8-wo → fp32 →
v1 int8**), falling through on download miss **or session-load failure**
— onnxruntime builds without the MatMulNBits 8-bit kernel fall back to
fp32 instead of losing Kompress
- `HEADROOM_KOMPRESS_ONNX_FILENAME` env pins an exact artifact
- `scripts/export_kompress_v2_onnx.py`: reproducible fp32 export (loads
the merged v2 checkpoint, traces the `final_scores` contract, verifies
vs PyTorch)
- `.gitignore`: local `onnx/` artifacts dir; allowlist the export script
## Testing
- End-to-end: fresh `KompressCompressor().preload()` resolves int8-wo
from HF, loads on onnxruntime 1.23 (CPU, no torch), compresses with
error/traceback content preserved
- fp32 export verified lossless vs PyTorch (max |Δscore| = 0.0, 100%
keep agreement)
- ruff check + format clean, mypy clean, 63 targeted tests pass
## Description
Adds first-class GCP Vertex AI proxy routing for publisher REST
endpoints so Vertex requests are forwarded to a configurable regional
Vertex host instead of falling through to the generic
OpenAI/Anthropic/Gemini passthrough selection.
Fixes#792
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update
## Changes Made
- Added a `vertex` provider target with `VERTEX_TARGET_API_URL` and
`--vertex-api-url` support.
- Registered explicit Vertex publisher routes for Google
`generateContent`, `streamGenerateContent`, `countTokens` and Anthropic
publisher `rawPredict`, `streamRawPredict` passthrough.
- Added startup banner/routing output for Vertex AI.
- Added focused tests for provider target resolution, CLI/env config,
banner output, and route delegation.
- Added `wiki/vertex.md` with usage examples and Google Cloud source
links.
## Sources
- Vertex AI Gemini inference reference:
https://docs.cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference
- Google Cloud REST authentication:
https://docs.cloud.google.com/docs/authentication/rest
- Google Application Default Credentials:
https://docs.cloud.google.com/docs/authentication/application-default-credentials
## Testing
- [x] Linting passes (`python -m ruff check .`)
- [x] New tests added for new functionality
- [x] Focused unit tests pass
- [ ] Full unit suite completed locally
- [ ] Rust tests completed locally
- [ ] Type checking passes locally
## Test Output
```text
$ python -m ruff check .
All checks passed!
$ python -m pytest tests/test_provider_registry.py tests/test_provider_proxy_routes.py tests/test_cli_proxy_env.py tests/test_banner_upstream_targets.py -q
57 passed, 1 warning in 13.75s
```
Local limitations:
- `python -m pytest tests scripts/tests -q` timed out after 1 hour on
this Windows machine before completing.
- `cargo test -p headroom-proxy --test integration_vertex_raw_predict`
could not run because `cargo` is not installed on PATH in this
environment.
- The commit hook's `mypy` step fails locally on an existing Windows
`fcntl` typing issue in `headroom/subscription/tracker.py`; `ruff`,
`ruff-format`, and plugin-version hooks passed, and the commit was made
with only `mypy` skipped.
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] I have added tests that prove the feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Problem
`.pre-commit-config.yaml` already has `ruff` + `ruff-format` configured,
and `pre-commit>=3.0.0` is already in `[dev]` deps — but `make
install-git-hooks` never called `pre-commit install`. Every
contributor's repo had the hook **config** but no running hook.
PR #772 merged with inline-comment spacing and import-order violations
that ruff would have caught automatically. The maintainer had to add a
separate fixup commit (`fix: format issue 728 regression test`) to clean
it up.
## Changes
**`scripts/install-git-hooks.sh`** — after installing the pre-push hook,
also run `pre-commit install`. Falls back to `.venv/bin/pre-commit` when
`pre-commit` is not on `PATH`, with a clear warning if neither is found:
```
✅ installed: .git/hooks/pre-push
Runs 'make ci-precheck' before every git push.
✅ installed: .git/hooks/pre-commit (ruff lint + format via pre-commit)
```
**`CONTRIBUTING.md`** — update PR workflow step 2 to mention `make
install-git-hooks` so contributors know to run it after `pip install`:
```
2. pip install -e ".[dev]" then make install-git-hooks — installs ruff on
every commit and ci-precheck on every push.
```
## No behaviour change for existing code
Only the local dev setup script is touched. Nothing in the proxy, tests,
or CI pipeline changes.
## Real behavior proof
- **OS**: macOS darwin arm64
- **Steps**: ran `bash scripts/install-git-hooks.sh` with venv
available, then attempted a commit with a badly-formatted file
- **Result**: ruff caught and auto-fixed it before the commit landed
```
✅ installed: .git/hooks/pre-push
Runs 'make ci-precheck' before every git push.
Bypass (use sparingly): git push --no-verify
pre-commit installed at .git/hooks/pre-commit
✅ installed: .git/hooks/pre-commit (ruff lint + format via pre-commit)
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- add stale triage for inactive issues and PRs with conservative close
windows
- add PR health labeling for branches that are behind, conflicted, or
failing checks
- create the maintenance labels idempotently before applying them
## Validation
- `go run github.com/rhysd/actionlint/cmd/actionlint@latest
.github/workflows/pr-health.yml .github/workflows/stale.yml`
- `act workflow_dispatch -W .github/workflows/pr-health.yml --dryrun`
- `act workflow_dispatch -W .github/workflows/stale.yml --dryrun`
- `git diff --cached --check`
Note: local `pre-commit` was not installed, so the commit was created
with `--no-verify` after the workflow-specific validation above passed.
Fixes#759
## Summary
`_compact_openai_tool_schema_value()` strips every key matching
`_OPENAI_TOOL_SCHEMA_DROP_KEYS` (which includes `title`, `readOnly`,
`deprecated`, `writeOnly`, etc.) regardless of where in the schema tree
it appears. This is wrong when those same strings are used as **property
names** inside a `properties` object — they're valid business fields,
not annotation metadata.
The result is an invalid strict schema sent upstream:
```
"required key 'title' not in properties"
```
**Root cause (single function, two lines):**
```python
# before — drops "title" everywhere, even as a property name
for key, child in value.items():
if key in _OPENAI_TOOL_SCHEMA_DROP_KEYS:
continue
compacted[key] = _compact_openai_tool_schema_value(child)
```
**Fix — add `_parent_key` context, skip drop only when not inside
`properties`:**
```python
def _compact_openai_tool_schema_value(value, _parent_key=None):
...
for key, child in value.items():
if _parent_key != "properties" and key in _OPENAI_TOOL_SCHEMA_DROP_KEYS:
continue
compacted[key] = _compact_openai_tool_schema_value(child, key)
```
Schema-level annotations (e.g. `title: "ReadFileParameters"` at schema
root) are **still stripped**. Only property names whose string value
happens to match a drop-key are preserved.
## Test plan
- [x] Added
`test_openai_tool_schema_compaction_preserves_property_named_title` in
`tests/test_openai_responses_context_compaction.py` — reproduces the
exact OMP `eval` tool schema from the issue report
- [x] All 9 existing compaction tests still pass (including
`test_openai_tool_schema_compaction_preserves_invocation_shape` which
verifies schema-level `title` is still stripped)
```
tests/test_openai_responses_context_compaction.py::test_openai_tool_schema_compaction_preserves_invocation_shape PASSED
tests/test_openai_responses_context_compaction.py::test_openai_tool_schema_compaction_preserves_property_named_title PASSED
tests/test_openai_responses_context_compaction.py::test_openai_tool_schema_compaction_is_deterministic PASSED
9 passed
```
## Real behavior proof
- **OS**: macOS darwin arm64, Python 3.11.0
- **Tested**: ran the new and existing compaction tests locally against
the patched handler
- **Not tested**: live OMP / Venice.ai / Codex endpoint (no API key for
those); the fix is a pure schema-transform function with no network side
effects
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Description
Codex's subscription/rate-limit window (the `x-codex-*` headers) was
being
**stripped on every transport Codex actually uses**, so session/weekly
usage
never reached the Codex CLI's own `/status` display, Headroom
`/stats`/dashboard,
or any consumer that sniffs the client-facing handshake. This PR
restores it on
**both** the WebSocket and streaming-SSE paths — the two halves of #577
— in one
place.
Fixes#577
**Supersedes #582 and #590.** This PR incorporates #582's SSE fix
(carried verbatim
with a `Co-authored-by` trailer) and additionally forwards the window
onto the client
`101` on the WS path, which #582/#590's capture-only WS code cannot do.
Both can be
closed as superseded once this merges — GitHub closing keywords only
auto-close
issues (hence `Fixes #577` above), not PRs, so #582/#590 need a manual
close.
### WebSocket (`gpt-5.4+`)
OpenAI delivers `x-codex-*` **only** on the upstream WS handshake
response, never
in data frames. `handle_openai_responses_ws` accepted the client WS
*before* it
connected upstream and never read `upstream.response.headers`, so the
window was
dropped. This reorders the handler to **connect upstream first**,
extract the
`x-codex-*` subset, then **accept the client WS with those headers
attached** to
the `101`, and refresh the Python state for `/stats` parity.
### Streaming SSE (incorporated from #582, @m16khb)
Codex CLI almost always streams. `streaming.py` neither captured
`x-codex-*` into
`CodexRateLimitState` nor forwarded it — the forwarded-header filter
matched only
the substring `"ratelimit"`, which `x-codex-*` does not contain. This
calls
`update_from_headers()` **before** the `>=400` early-return (so a
streaming 429/5xx
still refreshes the window, matching the non-streaming handlers) and
widens the
forward filter to pass `x-codex-*`.
> Credit: the SSE fix is @m16khb's work from #582, carried here verbatim
with a
> `Co-authored-by` trailer so the maintainer gets a single PR covering
both
> transports. This supersedes #582/#590's **WS** capture (which only
writes
> `/stats`); the connect-before-accept reorder additionally forwards the
window to
> the client `101`, which capture-only cannot do. #590's optional
snapshot
> persistence is intentionally left out (separable; hot-path sync write;
doesn't
> help the `101`-sniff consumers).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `openai.py`: add `_extract_codex_handshake_headers()` (strictly
`x-codex-*`, via
`raw_items()` to avoid `MultipleValuesError`; never
`set-cookie`/`authorization`).
- `openai.py`: reorder `handle_openai_responses_ws` — connect-only retry
loop runs
before `accept()`; `accept(headers=...)` carries the forwarded window;
first
client frame read afterward. HTTP fallback preserved; it now also
refreshes
`/stats` from the HTTP response headers.
- `streaming.py`: capture `x-codex-*` on all statuses + widen the
forwarded-header
filter (from #582).
### Diff-size note
The bulk of the `openai.py` line count is **whitespace-only
relocation**: the relay
block dedents one level out of the old per-attempt `async with`. Logical
change is
~290 lines. **Review with `?w=1`.** In API-key mode the handshake
carries no
`x-codex-*`, so the accept-header list is empty and the path behaves
exactly as
before — the fix only activates for ChatGPT-subscription auth.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
- WS: `test_ws_connect_happens_before_accept`,
`test_ws_forwards_codex_headers_to_client_accept`
(only `x-codex-*` forwarded; `set-cookie`/`authorization` excluded;
`/stats` refreshed),
`test_ws_connect_failure_falls_back_to_http`,
`test_ws_first_frame_timeout_after_connect_closes_upstream`.
- Fallback: `test_fallback_refreshes_codex_rate_limit_state`.
- SSE:
`test_codex_rate_limit_headers_captured_and_forwarded_in_streaming`,
`test_codex_rate_limit_captured_on_streaming_429` (from #582).
- Wire-level e2e: `tests/e2e_ws_codex_usage_headers.py` boots the real
proxy + fake
upstream + real `websockets` client and reads the client `101` — closes
the gap
the unit tests stub (that uvicorn/starlette actually write
`accept(headers=...)`).
## Test Output
```
$ uv run pytest tests/test_proxy_streaming_ratelimit_headers.py \
tests/test_ws_http_fallback.py \
tests/test_openai_codex_ws_lifecycle.py \
tests/test_openai_codex_ws_timings.py \
tests/test_codex_rate_limits.py -q
63 passed in 0.83s
$ .venv/bin/python tests/e2e_ws_codex_usage_headers.py
[codex-hdr-e2e] client 101 headers:
x-codex-primary-used-percent: 42
x-codex-primary-window-minutes: 300
x-codex-secondary-used-percent: 7
x-codex-secondary-window-minutes: 10080
[codex-hdr-e2e] /stats reflects codex window (primary-used=42)
=== CODEX-HDR E2E ALL GREEN ===
$ uv run ruff check . && uv run ruff format --check <touched files>
All checks passed!
```
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- **Why connect-before-accept (not capture-only).** Once `accept()`
sends the `101`,
headers can no longer be added; the `x-codex-*` window only exists after
we connect
upstream. Capturing into Python state (as #582/#590's WS code does)
fixes `/stats`
but not the Codex CLI's native display or any `101`-sniffing consumer —
those need
the headers *on the client handshake*, which requires the reorder.
- **Security.** Forwarding is filtered strictly to `x-codex-*`;
`set-cookie`,
`authorization`, and all other upstream headers are never forwarded to
the client
(asserted by both the unit test and the e2e).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Contract Schemas
Per maintainer request: a JSON Schema (draft 2020-12) artifact
enshrining the OpenAI
interaction expectations this changeset relies on, so drift is
detectable later.
Committed following the repo's parity convention:
- schema:
`tests/parity/fixtures/codex_openai_contracts/codex-openai-interaction.schema.json`
- test: `tests/test_codex_openai_contract_parity.py` binds the schema to
the **live code**
in both directions, so drift fails CI rather than living only in this
description -
every declared `x-codex-*` header must be consumed by
`parse_codex_rate_limits`, and
`_extract_codex_handshake_headers` must forward exactly the declared
subset and never
`set-cookie`/`authorization`. No new dependency (does not pull in
`jsonschema`).
It covers, as `$defs`:
- `WSUpstreamHandshakeResponse` / `StreamingUpstreamResponseHeaders` -
the upstream
`x-codex-*` header family (full superset, with per-header wire pattern +
the parsed
semantic type) the WS and SSE captures read. Source of truth:
`parse_codex_rate_limits`.
- `ClientForwardedHandshakeHeaders` - the WS-101 **allow/deny**
contract: only
`x-codex-*` may be forwarded; `set-cookie`/`authorization` are
explicitly forbidden
(`propertyNames` + `not`).
- `ClientForwardedStreamingHeaders` - the wider SSE forward set
(`*ratelimit*` OR `x-codex*`).
- `WSClientRequestFrame` / `WSRelayEvent` / `HTTPFallbackRequestBody` -
the WS frame
envelopes and the unwrapped HTTP-fallback POST body.
- `CodexRateLimitStatsOutput` - the headroom `/stats` shape the parity
tests assert.
Validated with `jsonschema` (Draft202012 `check_schema` passes; positive
instances from
the e2e validate; negative instances - a leaked `set-cookie`, a fallback
body still
carrying a top-level `type` - are correctly rejected).
<details>
<summary><code>codex-openai-interaction.schema.json</code> (draft
2020-12)</summary>
```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/chopratejas/headroom/contracts/codex-openai-interaction.schema.json",
"title": "Codex <-> OpenAI interaction contracts (PR #794)",
"description": "Enshrines the OpenAI interaction expectations this changeset depends on, so drift is detectable. Header values are transported as strings on the wire; the `x-headroom-parsed-type` annotation on each records the semantic type the parser (headroom/subscription/codex_rate_limits.py) coerces them to. Sources: codex_rate_limits.parse_codex_rate_limits (header family + gating), openai._extract_codex_handshake_headers (WS-101 forward filter), streaming.py (SSE forward filter).",
"$defs": {
"OpenAICodexWindowHeaders": {
"title": "x-codex-*-{primary,secondary} window headers",
"description": "A rolling rate-limit/subscription window. A window is materialized iff its `*-used-percent` header is present and numeric; `*-window-minutes` and `*-reset-at` are optional. `primary` and `secondary` are independent and either may be absent.",
"type": "object",
"properties": {
"x-codex-primary-used-percent": {
"type": "string",
"pattern": "^\\d+(?:\\.\\d+)?$",
"x-headroom-parsed-type": "float (0-100, NaN-guarded)",
"description": "Percent of the primary window consumed. Gates creation of the primary window."
},
"x-codex-primary-window-minutes": {
"type": "string",
"pattern": "^\\d+$",
"x-headroom-parsed-type": "int",
"description": "Primary window size in minutes."
},
"x-codex-primary-reset-at": {
"type": "string",
"pattern": "^\\d+$",
"x-headroom-parsed-type": "int (Unix epoch seconds)",
"description": "Absolute reset time of the primary window."
},
"x-codex-secondary-used-percent": {
"type": "string",
"pattern": "^\\d+(?:\\.\\d+)?$",
"x-headroom-parsed-type": "float (0-100, NaN-guarded)",
"description": "Percent of the secondary window consumed. Gates creation of the secondary window."
},
"x-codex-secondary-window-minutes": {
"type": "string",
"pattern": "^\\d+$",
"x-headroom-parsed-type": "int"
},
"x-codex-secondary-reset-at": {
"type": "string",
"pattern": "^\\d+$",
"x-headroom-parsed-type": "int (Unix epoch seconds)"
}
},
"additionalProperties": true
},
"OpenAICodexCreditsHeaders": {
"title": "x-codex-credits-* headers",
"description": "OpenAI credits balance. A credits snapshot is materialized iff `x-codex-credits-has-credits` is present; `unlimited` defaults to false; `balance` is optional.",
"type": "object",
"properties": {
"x-codex-credits-has-credits": {
"type": "string",
"pattern": "^(?:[Tt][Rr][Uu][Ee]|[Ff][Aa][Ll][Ss][Ee]|[01])$",
"x-headroom-parsed-type": "bool (true|false|1|0, case-insensitive)",
"description": "Gates creation of the credits snapshot."
},
"x-codex-credits-unlimited": {
"type": "string",
"pattern": "^(?:[Tt][Rr][Uu][Ee]|[Ff][Aa][Ll][Ss][Ee]|[01])$",
"x-headroom-parsed-type": "bool (defaults false when absent/unparseable)"
},
"x-codex-credits-balance": {
"type": "string",
"x-headroom-parsed-type": "str (empty -> null)",
"description": "Free-form server string, e.g. \"$5.00\"."
}
},
"additionalProperties": true
},
"OpenAICodexMetaHeaders": {
"title": "x-codex meta headers",
"type": "object",
"properties": {
"x-codex-limit-name": {
"type": "string",
"x-headroom-parsed-type": "str (empty -> null)",
"description": "Active limit/model label, e.g. \"gpt-5.2-codex-sonic\"."
},
"x-codex-promo-message": {
"type": "string",
"x-headroom-parsed-type": "str (empty -> null)",
"description": "Server announcement. Also gates snapshot creation when present."
}
},
"additionalProperties": true
},
"OpenAICodexRateLimitHeaders": {
"title": "Full x-codex-* header family OpenAI may emit",
"description": "Superset of every x-codex-* header headroom reads. parse_codex_rate_limits returns a snapshot iff at least one of: a primary window, a secondary window, a credits snapshot, or a non-empty promo message is present; otherwise null (treated as a non-Codex response). All members are individually optional.",
"type": "object",
"allOf": [
{ "$ref": "#/$defs/OpenAICodexWindowHeaders" },
{ "$ref": "#/$defs/OpenAICodexCreditsHeaders" },
{ "$ref": "#/$defs/OpenAICodexMetaHeaders" }
],
"additionalProperties": true
},
"WSUpstreamHandshakeResponse": {
"title": "OpenAI WS handshake (101) response headers consumed by the WS fix",
"description": "On the Codex WebSocket transport the x-codex-* window is delivered ONLY on the upstream handshake response (never in data frames). handle_openai_responses_ws reads upstream.response.headers here. This is the contract the connect-before-accept reorder depends on: if OpenAI ever moves these headers off the handshake (e.g. into a frame), the WS half of the fix goes stale.",
"$ref": "#/$defs/OpenAICodexRateLimitHeaders"
},
"StreamingUpstreamResponseHeaders": {
"title": "OpenAI streaming/HTTP response headers consumed by the SSE fix",
"description": "On the streaming SSE/HTTP transport the same x-codex-* headers ride the HTTP response. streaming.py captures them on ALL statuses (including >=400) via update_from_headers, and forwards a wider set to the client (see ClientForwardedStreamingHeaders).",
"$ref": "#/$defs/OpenAICodexRateLimitHeaders"
},
"ClientForwardedHandshakeHeaders": {
"title": "Headers forwarded onto the CLIENT-facing WS 101 (allow/deny contract)",
"description": "_extract_codex_handshake_headers forwards ONLY headers whose (lowercased) name starts with `x-codex-`. Every other upstream handshake header - notably set-cookie and authorization - MUST NOT appear on the client 101. Enforced by propertyNames below and asserted by the unit tests + tests/e2e_ws_codex_usage_headers.py.",
"type": "object",
"propertyNames": {
"pattern": "^[Xx]-[Cc][Oo][Dd][Ee][Xx]-"
},
"not": {
"anyOf": [
{ "required": ["set-cookie"] },
{ "required": ["Set-Cookie"] },
{ "required": ["authorization"] },
{ "required": ["Authorization"] }
]
},
"additionalProperties": { "type": "string" }
},
"ClientForwardedStreamingHeaders": {
"title": "Headers forwarded to the client on the streaming SSE path",
"description": "streaming.py forwards a header iff `\"ratelimit\" in name.lower()` OR `name.lower().startswith(\"x-codex\")`. This is a SUPERSET of the WS allow-list: it additionally passes generic *ratelimit* headers (e.g. the Anthropic streaming path) which do not contain the x-codex prefix.",
"type": "object",
"propertyNames": {
"pattern": "(?:[Rr][Aa][Tt][Ee][Ll][Ii][Mm][Ii][Tt])|^[Xx]-[Cc][Oo][Dd][Ee][Xx]"
},
"additionalProperties": { "type": "string" }
},
"WSClientRequestFrame": {
"title": "Client -> proxy WS data frame (Responses API over WS)",
"description": "Codex sends the request as a response.create envelope. The HTTP fallback unwraps `.response` for the POST body, forces stream=true, and strips any top-level `type`. A flattened variant (no envelope, fields at top level) is also tolerated by the fallback.",
"type": "object",
"properties": {
"type": { "const": "response.create" },
"response": {
"type": "object",
"properties": {
"model": { "type": "string", "description": "e.g. gpt-5.4" },
"input": {
"description": "String prompt or Responses-API structured input array.",
"type": ["string", "array"]
},
"stream": { "type": "boolean" }
},
"required": ["model"],
"additionalProperties": true
}
},
"required": ["type", "response"],
"additionalProperties": true
},
"WSRelayEvent": {
"title": "proxy -> client WS data frame (relayed Responses API event)",
"description": "SSE `data:` payloads relayed verbatim as WS text frames. `[DONE]` sentinels are dropped (not relayed). Every relayed event is a JSON object carrying a `type`. response.completed additionally carries usage under `response.usage`. anyOf (not oneOf): an error event also satisfies the looser lifecycle shape, which is fine.",
"anyOf": [
{
"title": "lifecycle event",
"type": "object",
"properties": {
"type": {
"type": "string",
"examples": [
"response.created",
"response.output_item.added",
"response.completed"
]
},
"response": { "type": "object", "additionalProperties": true }
},
"required": ["type"],
"additionalProperties": true
},
{
"title": "error event",
"type": "object",
"properties": {
"type": { "const": "error" },
"error": {
"type": "object",
"properties": { "message": { "type": "string" } },
"required": ["message"],
"additionalProperties": true
}
},
"required": ["type", "error"],
"additionalProperties": true
}
]
},
"HTTPFallbackRequestBody": {
"title": "proxy -> OpenAI HTTP POST body on WS->HTTP fallback",
"description": "Derived from WSClientRequestFrame: the inner `.response` object, with `stream` forced to true and any top-level `type` removed.",
"type": "object",
"properties": {
"model": { "type": "string" },
"stream": { "const": true },
"input": { "type": ["string", "array"] }
},
"required": ["model", "stream"],
"not": { "required": ["type"] },
"additionalProperties": true
},
"CodexRateLimitStatsOutput": {
"title": "headroom /stats output for the codex tracker (CodexRateLimitSnapshot.to_dict)",
"description": "Internal (headroom-emitted) shape produced from the headers above; the WS and SSE update_from_headers parity tests assert this is refreshed. Included so drift in our own surface is also caught.",
"type": "object",
"properties": {
"limit_id": { "const": "codex" },
"limit_name": { "type": ["string", "null"] },
"primary": { "$ref": "#/$defs/CodexWindowDict" },
"secondary": { "$ref": "#/$defs/CodexWindowDict" },
"credits": {
"oneOf": [
{ "type": "null" },
{
"type": "object",
"properties": {
"has_credits": { "type": "boolean" },
"unlimited": { "type": "boolean" },
"balance": { "type": ["string", "null"] }
},
"required": ["has_credits", "unlimited", "balance"],
"additionalProperties": false
}
]
},
"promo_message": { "type": ["string", "null"] },
"captured_at": { "type": "number", "description": "Unix epoch seconds (float)." }
},
"required": ["limit_id", "limit_name", "primary", "secondary", "credits", "promo_message", "captured_at"],
"additionalProperties": false
},
"CodexWindowDict": {
"oneOf": [
{ "type": "null" },
{
"type": "object",
"properties": {
"used_percent": { "type": "number" },
"window_minutes": { "type": ["integer", "null"] },
"window_label": { "type": "string", "description": "e.g. \"5h\", \"7d\"-style label; \"unknown\" when window_minutes is null." },
"resets_at": { "type": ["integer", "null"], "description": "Unix epoch seconds." },
"seconds_until_reset": { "type": ["integer", "null"] }
},
"required": ["used_percent", "window_minutes", "window_label", "resets_at", "seconds_until_reset"],
"additionalProperties": false
}
]
}
}
}
```
</details>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: m16khb <m16khb@gmail.com>
## Description
Adds per-provider attribution to the durable savings history rollups so
consumers can show how savings and spend are distributed across
providers within a given time period.
Today the per-provider numbers on `/dashboard` come from
`cache_by_provider`/`requests_by_provider` in `prometheus_metrics.py`,
which are cumulative-since-start counters with no timestamp. The
time-series that powers the savings history (`/stats-history` ->
`SavingsTracker.history_response`) had no provider dimension at all, so
a per-time-period provider breakdown was impossible. This change threads
the provider into the history buckets.
Fixes #(none)
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `prometheus_metrics.py`: forward the `provider` already known at the
`record_request` call site into `SavingsTracker.record_request`.
- `savings_tracker.py`: add an optional `provider` arg to
`record_request` and `record_compression_savings`, persist it on each
history checkpoint, and preserve it through `_normalize_history_entry`.
- `savings_tracker.py`: in `_build_rollup`, attribute each checkpoint's
delta to its provider, emitting a `by_provider` map per bucket
(`tokens_saved`, `compression_savings_usd_delta`,
`total_input_tokens_delta`, `total_input_cost_usd_delta`). Each
checkpoint is produced by a single request, so its delta is wholly owned
by one provider. Providers only appear in a bucket where they moved a
counter; legacy checkpoints with no provider collapse into `"unknown"`.
- Additive and schema-compatible: `schema_version` is unchanged, old
persisted state loads unchanged (provider defaults to `"unknown"`), and
existing rollup fields are untouched. CSV export is unaffected (it
filters to its fixed columns).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
Added `test_savings_tracker_rollup_attributes_savings_per_provider`,
covering: two providers sharing one hour bucket, a bucket with a single
provider, per-provider deltas summing back to the bucket total, and a
no-provider checkpoint collapsing to `"unknown"`. Updated three existing
exact-equality history-shape assertions to include the new `provider`
field.
## Test Output
```
$ uv run pytest tests/test_proxy_savings_history.py -q
14 passed, 2 warnings in 7.75s
$ uv run ruff check headroom/proxy/savings_tracker.py headroom/proxy/prometheus_metrics.py tests/test_proxy_savings_history.py
All checks passed!
$ uv run ruff format --check ...
3 files already formatted
$ uv run mypy headroom/proxy/savings_tracker.py headroom/proxy/prometheus_metrics.py
Success: no issues found in 2 source files
```
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The new `by_provider` field is purely additive; existing consumers that
read the flat bucket totals are unaffected. The keys are providers
(`anthropic`, `openai`, ...), not per-client/connector identities -- the
proxy buckets requests by provider at record time, so finer per-client
attribution would be a separate change.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Description
Sets pyo3 params to support python above 3.13
Fixes #(408
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Updated Cargo.toml
## Testing
Describe the tests you ran to verify your changes:
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
## Test Output
```
Compiles
```
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
## What
Loosen over-pinned Python dependency constraints and add missing upper
bounds in `pyproject.toml`. Also bump the neo4j Docker image and uv
builder version.
## Why
Several dependencies had constraints that either blocked security
patches or allowed silent major-version jumps:
- `litellm==1.82.3` was an exact pin — every security patch release
requires a manual lockfile bump
- `transformers`, `sentence-transformers` had no upper bound and have
already crossed major version boundaries without a constraint gate
- `neo4j>=5.20.0` had no upper cap; the driver has already reached 6.x
in the wild
- `mem0ai>=0.1.100` had a pre-1.0 floor while the locked version is
already 1.0.11
- `langchain-core`, `langchain-openai`, `qdrant-client`, `uvicorn` had
no upper bound on a range with active major-version churn
- `docker-compose.yml` pinned neo4j at `5.15.0`, which is 11 patch
releases behind the current 5.x LTS
- `Dockerfile` pinned uv at `0.11.16`; latest stable is `0.11.18`
## How
Constraint changes only — no code changes, no `uv lock --upgrade`. The
existing locked versions all satisfy the new bounds (we added caps, not
floors). `uv` re-resolved the lockfile to format revision 3 (adds
`upload-time` metadata fields) and cleaned up the defunct `llmlingua`
extra entries.
| Dependency | Before | After |
|---|---|---|
| `litellm` | `==1.82.3` | `>=1.82.3,<2.0` |
| `transformers` | `>=4.30.0` | `>=4.30.0,<6.0` |
| `sentence-transformers` | `>=2.2.0` | `>=2.2.0,<6.0` |
| `neo4j` | `>=5.20.0` | `>=5.20.0,<7.0` |
| `mem0ai` | `>=0.1.100` | `>=1.0.0,<2.0` |
| `langchain-core` | `>=0.2.0` | `>=0.2.0,<4.0` |
| `langchain-openai` | `>=0.1.0` | `>=0.1.0,<2.0` |
| `qdrant-client` | `>=1.9.0` | `>=1.9.0,<2.0` |
| `uvicorn` | `>=0.23.0` | `>=0.23.0,<1.0` |
| neo4j Docker image | `5.15.0` | `5.26` |
| uv (Dockerfile ARG) | `0.11.16` | `0.11.18` |
## Breaking changes
None. All currently installed versions fall within the new ranges.
Installers that previously resolved `litellm` to an older exact pin may
now resolve newer patch releases — which is the desired behavior.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
Fixes#728
## Summary
- `apply_session_sticky_ccr_tool` and
`apply_session_sticky_memory_tools` always return a list — returning
`[]` when `existing_tools=None` and nothing was injected
- The old handler guard `if tools is not None:` evaluated `True` for
`[]`, causing `body["tools"] = []` to be sent upstream on every request
- vLLM-based providers (Venice.ai, etc.) strictly reject empty `tools`
arrays with a 400 error
**Fix:** Change the guard in both the OpenAI and Anthropic handlers
from:
```python
if tools is not None:
body["tools"] = tools
```
to:
```python
if tools or _original_tools is not None:
body["tools"] = tools
```
The `_original_tools` variable is already defined in both handlers
(`_original_tools = body.get("tools")`). This condition correctly
handles all four cases:
| Scenario | `tools` | `_original_tools` | Result |
|---|---|---|---|
| No client tools, no injection | `[]` | `None` | `False` → don't inject
✅ |
| No client tools, CCR injected | `[ccr_tool]` | `None` | `True` →
inject ✅ |
| Client sent `tools: []` | `[]` | `[]` | `True` → preserve ✅ |
| Client sent tools | `[A, ...]` | `[A, ...]` | `True` → preserve ✅ |
## Test plan
- [x] New test file `tests/test_issue_728_empty_tools_injection.py` with
7 tests covering the guard condition and helper behavior
- [x] All 51 existing CCR/golden-bytes tests still pass
- [x] Zero changes to helper function return types or signatures
## Real behavior proof
Tested against the helpers directly:
```
tests/test_issue_728_empty_tools_injection.py::TestHandlerGuardCondition::test_no_tools_no_injection_does_not_inject PASSED
tests/test_issue_728_empty_tools_injection.py::TestHandlerGuardCondition::test_client_sent_empty_tools_is_preserved PASSED
tests/test_issue_728_empty_tools_injection.py::TestHandlerGuardCondition::test_ccr_injection_sets_body_tools PASSED
tests/test_issue_728_empty_tools_injection.py::TestHandlerGuardCondition::test_client_tools_always_set PASSED
tests/test_issue_728_empty_tools_injection.py::TestCCRHelperNoToolsNoCompression::test_returns_empty_list_and_false_when_no_session_ccr PASSED
tests/test_issue_728_empty_tools_injection.py::TestCCRHelperNoToolsNoCompression::test_returns_tool_list_when_compression_occurred PASSED
tests/test_issue_728_empty_tools_injection.py::TestCCRHelperNoToolsNoCompression::test_no_double_injection_when_client_pre_registered_ccr_tool PASSED
7 passed in 1.81s
```
**What I did not test:** end-to-end against a live Venice.ai endpoint
(no API key available), or passthrough mode with a real vLLM backend.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
## Problem
The release pipeline's `docker-manifest` jobs fail at action resolution:
```
Unable to resolve action `sigstore/cosign-installer@v4`, unable to find version `v4`
```
`sigstore/cosign-installer` has no `v4`; its current major is `v3`. This
broke the multi-arch manifest assembly and `promote-latest` on the
v0.24.0 release run (and would break every release). Per-arch image
builds and **PyPI/npm/GitHub-Packages publishing were unaffected**.
## Fix
`.github/workflows/docker.yml`: `sigstore/cosign-installer@v4` → `@v3`.
## Verification
Resolves the only failing jobs in release run
[27184823371](https://github.com/chopratejas/headroom/actions/runs/27184823371).
After merge, the docker-manifest + promote-latest steps will resolve the
action and run.
## Problem
When the compression pipeline returns empty/whitespace content for a
**non-empty** user message, the proxied request reaches Anthropic with
empty message content and is rejected with:
```
HTTP 400 — messages.N: user messages must have non-empty content
```
This kills the whole request (not just compression), so a single bad
compression output takes down the turn.
## Root cause
`ContentRouter.compress()` returns whatever the selected transform
produced. Nothing asserts the invariant that **compression must never
blank out non-empty input**. Any transform path that yields
`""`/whitespace from non-empty input therefore surfaces as a 400 at the
API boundary.
In production this was triggered by a `pyo3` `unsendable` panic in the
tree-sitter parser path (cross-thread parser reuse) that produced empty
content. That specific panic is already addressed on `main` by the
thread-local parser fix (38aefc1d). **This PR is the complementary,
transform-agnostic safety net** — it catches *any* future path that
could blank out content, independent of the tree-sitter panic.
## Fix
A final guard in `compress()`: if input is non-empty but the compressed
result is empty/whitespace, fall back to the original content
(passthrough) and log a warning.
```python
if (
content
and content.strip()
and (result.compressed is None or not str(result.compressed).strip())
):
logger.warning(
"content_router: compression produced EMPTY output from non-empty "
"input (%d chars, strategy=%s); falling back to original to avoid 400.",
len(content),
getattr(result.strategy_used, "value", result.strategy_used),
)
result.compressed = content
```
- 18 lines, single file (`headroom/transforms/content_router.py`).
- No behavior change on the normal path (only activates when output
would otherwise be empty).
- `py_compile` clean.
## Testing
Verified against a token-mode proxy under cross-thread
`ThreadPoolExecutor` load: previously-failing requests (empty-content
400) now pass through with original content preserved; no 400s observed.
Normal compression output is unaffected.
Co-authored-by: yoonhwan <yoonhwan.ko@byourz.com>
## Summary
- add a containerized differential network capture harness for Claude
Code direct vs Claude Code routed through Headroom
- capture both Headroom client-side traffic and Headroom upstream
traffic with sanitized mitmproxy JSONL output
- add `headroom capture network-diff` to compare captures and produce
Markdown/JSON reports, including Anthropic tool-count/tool-byte deltas
for deferred-tool investigations
- add an on-demand GitHub Actions workflow for the harness; it only runs
via `workflow_dispatch`, with live Claude Code/Anthropic capture gated
on `ANTHROPIC_API_KEY`
- document the workflow and ignore generated capture artifacts
## Validation
- `C:\git\headroom\.venv\Scripts\python.exe -m pytest
tests/test_network_diff_capture.py`
- `ruff check headroom/capture headroom/cli/capture.py
tests/test_network_diff_capture.py`
- `ruff format --check headroom/capture headroom/cli/capture.py
tests/test_network_diff_capture.py`
- `C:\git\headroom\.venv\Scripts\python.exe -m mypy
headroom/capture/network_diff.py headroom/cli/capture.py`
- `docker compose -f
docker/differential-network-capture/docker-compose.yml --profile run
config`
- `docker compose -f
docker/differential-network-capture/docker-compose.yml --profile run
build claude-direct`
- `docker run --rm -e CLAUDE_COMMAND="claude --version"
headroom-network-diff-claude-direct:latest`
- parsed `.github/workflows/network-diff-capture.yml` with PyYAML and
confirmed manual-only trigger
Live Claude API capture was not run locally because `ANTHROPIC_API_KEY`
is not set in this environment. The workflow can run it manually in
GitHub Actions when that secret is present; otherwise it emits a visible
skip warning and uploads a skipped artifact.
## Notes
- Full pre-commit mypy still fails on unrelated Windows `fcntl`
attributes in `headroom/subscription/tracker.py`; the feature commit
skipped only that hook after narrow mypy passed for the new modules.
- `tests/test_release_workflows.py` has two Windows-local failures
because it shells out to a missing Unix/Rust command; unrelated workflow
checks in that file passed before those failures.
- Motivated by
https://github.com/chopratejas/headroom/issues/746#issuecomment-4651276818
/ Issue #746.
Closes#741
## What
Headroom is commonly deployed alongside Claude Code, which sets
`NODE_EXTRA_CA_CERTS` to a custom CA bundle for corporate or internal
CAs. Node.js inherits this automatically; Python's `httpx` does not.
Every upstream request silently failed with `SSL:
CERTIFICATE_VERIFY_FAILED`, causing 502s and retry loops in the client.
## Changes
- New `headroom/proxy/ssl_context.py` with `build_ssl_context()` helper
that checks `SSL_CERT_FILE` → `REQUESTS_CA_BUNDLE` →
`NODE_EXTRA_CA_CERTS` (first match wins) and builds an `ssl.SSLContext`
with the custom CA bundle loaded
- `HeadroomProxy.start()` calls `build_ssl_context()` and passes the
result as `verify=` to `httpx.AsyncClient`; falls back to `verify=True`
(default httpx behaviour) when no env var is set
- Logs which env var and path was used at `INFO` level; warns on
set-but-missing paths
- 10 unit tests covering: no env var → `None`, each var returns
`SSLContext`, priority order, nonexistent paths skipped
## Priority order
1. `SSL_CERT_FILE` — standard POSIX/Python ssl override
2. `REQUESTS_CA_BUNDLE` — standard requests/httpx convention
3. `NODE_EXTRA_CA_CERTS` — Node.js / Claude Code convention
Routing Claude Code through the proxy disabled its on-demand tool loading:
with a custom ANTHROPIC_BASE_URL and ENABLE_TOOL_SEARCH unset, Claude Code
stops deferring MCP/system tool schemas behind the server-side Tool Search
Tool and materializes them all into local context (~25K tokens) — the
opposite of what a context-optimization proxy should do.
Root cause is a client-side gate in Claude Code (isToolSearchEnabledOptimistic):
deferral is disabled when ENABLE_TOOL_SEARCH is unset AND provider is
first-party AND the base-URL host is not api.anthropic.com. It is a one-way
URL check, not a capability handshake, so no proxy/response header can flip
it. The only lever is the ENABLE_TOOL_SEARCH env var Claude Code reads at
startup.
Changes:
- wrap claude: inject ENABLE_TOOL_SEARCH into the launched Claude Code env
(default "true"; --tool-search true|auto|auto:N|false; a pre-set env value
is respected; blank is treated as unset). Keeps deferral on through the proxy.
- proxy: emit a one-time, actionable hint when a Claude Code request is
detected loading tools eagerly (for users who run `claude` manually). Gated
on a cheap one-shot flag and wrapped so it can never fail a request.
- docs: troubleshooting section with before/after verification.
- tests: 30 unit tests (value validation, injection precedence, detection,
hint content, one-shot guard).
Display the resolved upstream API targets (Anthropic, OpenAI, Gemini,
Cloud Code) in the proxy startup banner so users can verify their
custom endpoint configuration at a glance.
The new UPSTREAM TARGETS section appears between the Backend line and
the FEATURES section. URLs are resolved through the existing
resolve_api_targets pipeline, which normalizes trailing /v1 suffixes
and applies defaults for unconfigured providers.
Closes#583
Co-authored-by: vipin-si <vipin-si@users.noreply.github.com>
The existing suppression in headroom/memory/adapters/embedders.py fires at
module-import time for that file, but sentence_transformers is imported lazily
(only when SentenceTransformer() is instantiated in a worker process). This
means the log levels are set too late: httpx and huggingface_hub have already
emitted their INFO/WARNING records by the time embedders.py is first imported.
Fix: move the suppression to headroom/cli/proxy.py module level. This file is
imported during CLI registration (_register_commands in main.py), well before
any worker forks or ML initialisation. Setting log levels here guarantees they
are in place for every code path that eventually loads sentence_transformers.
Changes:
- logging.getLogger("httpx").setLevel(WARNING) -- suppress manifest HEAD/GET INFO
- logging.getLogger("huggingface_hub").setLevel(ERROR)
- logging.getLogger("huggingface_hub.utils._http").setLevel(ERROR)
- logging.getLogger("sentence_transformers").setLevel(WARNING)
- os.environ.setdefault("HF_HUB_DISABLE_IMPLICIT_TOKEN", "1") -- pre-empt env check
- os.environ.setdefault("HF_HUB_DISABLE_PROGRESS_BARS", "1")
- warnings.filterwarnings to suppress unauthenticated/HF_TOKEN UserWarnings
Eliminates ~50 noisy startup log lines with 8 workers (6 HEAD requests each).
Complementary to PR #619 which adds the same suppression in embedders.py;
this PR adds the MISSING earlier suppression point.
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
Co-authored-by: Devanshi Vyas <dnv2103@columbia.edu>
* fix(deps): add missing runtime deps to [code] and [proxy] extras
- Add gunicorn>=21.0.0 to the [proxy] extra
The proxy docs (docs/content/docs/proxy.mdx and wiki/proxy.md) show
gunicorn as the recommended production deployment server:
pip install gunicorn
gunicorn headroom.proxy.server:app --worker-class uvicorn.workers.UvicornWorker
Users installing headroom-ai[proxy] for production get uvicorn (already
declared) but had to discover and install gunicorn manually. Adding it
to [proxy] removes that friction.
Investigation notes:
- [code] only needs tree-sitter-language-pack (already declared).
code_compressor.py has zero numpy imports. The kompress fallback
inside code_compressor.py is guarded by ImportError and requires [ml].
- numpy is correctly declared in [relevance] (numpy>=1.24.0) and pulled
transitively by sentence-transformers in [memory]. It is NOT needed
under [code].
- tree-sitter is a transitive dep of tree-sitter-language-pack (requires
tree-sitter>=0.25.2) so it does not need an explicit entry.
* docs(changelog): add entry for gunicorn proxy dep fix
style(tests): ruff format test_provider_proxy_routes.py (blank lines after docstrings)
* fix(deps): move gunicorn to [proxy-prod] extra, add Windows guard
- Remove gunicorn from [proxy] so dev, CI, and Windows users are not
forced to install a Unix-only package that does nothing on Windows
- Add new [proxy-prod] extra that includes [proxy] + gunicorn with a
sys_platform != 'win32' environment marker
- Production users: pip install 'headroom-ai[proxy,proxy-prod]'
- Update CHANGELOG to reflect the new extra name
* fix(devcontainer): bump uv floor to >=0.11.0 for lockfile compatibility
uv 0.6.17 (previously pinned) cannot parse lockfiles generated by
uv >= 0.11.x. The validate CI job (triggered by pyproject.toml
changes) was failing with 'Failed to parse uv.lock'. Loosening the
pin to >=0.11.0 picks up the matching format parser while keeping the
Docker layer cacheable with a range rather than an exact pin.
* fix(devcontainer): skip gitpython wheel filename check in uv sync
gitpython 3.1.47 on PyPI has wheel gitpython-3.1.46-py3-none-any.whl
(wrong filename). uv >=0.11.19 strict filename validation rejects this
lockfile entry. UV_SKIP_WHEEL_FILENAME_CHECK=1 bypasses the check until
the upstream lockfile is regenerated with a corrected entry.
* fix(deps): correct gitpython version in uv.lock to match actual wheel
gitpython 3.1.47 on PyPI was uploaded with sdist/wheel files named
gitpython-3.1.46.*. The version field in uv.lock said 3.1.47 but all
download URLs reference 3.1.46 files, causing uv >=0.11.19 to refuse
to parse the lockfile with a version-mismatch error.
Change the version field to 3.1.46 so the entry is internally
consistent. Also revert the now-unnecessary UV_SKIP_WHEEL_FILENAME_CHECK
workaround from post-create.sh.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
* feat(perf): add structured summary/record builders to analyzer
parse_log_files() already returns a fully-structured PerfReport, but
the only way to read it was the colored text report. Add reusable
machine-readable views so CI guards, dashboards, and agent harnesses
can consume perf data without scraping ANSI text:
- build_perf_summary(report) -> dict with the aggregated KPIs
(savings_pct, cache_hit_pct, by_model, by_transform, ...), mirroring
format_report() numbers exactly.
- perf_records_as_dicts(report) -> per-record list for --raw output.
- PERF_RECORD_FIELDS: shared column order for CSV/raw consumers.
Pure additions; no behaviour change to existing callers. Part of #595.
* feat(perf): add --format {text,json,csv} to headroom perf
Adds a machine-readable output path to the perf command (issue #595):
- --format json: aggregated summary (default) or, with --raw, a JSON
array of per-record dicts.
- --format csv: per-model breakdown (default) or, with --raw, one row
per PERF record using the shared PERF_RECORD_FIELDS column order.
- --format text (default): unchanged human-readable report.
Enables CI guards (jq '.savings_pct < 70'), dashboards, and agent
wrappers to consume perf data without scraping ANSI text.
Closes#595.
* test(perf): cover --format json/csv and structured builders
Unit tests for build_perf_summary (totals, savings/cache pct,
by_model/by_transform, empty-report zero-division guard) and
perf_records_as_dicts, plus CliRunner integration tests for
--format json, json --raw, csv, csv --raw, the unchanged text
default, and rejection of an unknown format. Part of #595.
* fix(perf): rename transform loop var to satisfy mypy
The structured-summary builder reused `recs` for both the per-model
(list[PerfRecord]) and per-transform (list[TransformRecord]) groupings, so
mypy flagged the second assignment as an incompatible-type reuse
(analyzer.py:704). Rename the transform loop variable to `t_recs` so each
loop keeps a single element type. No behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Kumario1 <ramsakal.ipec@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs: add enterprise.md
* docs: add link to enterprisemd in README
* fix(copilot): restore generic endpoint for non-subscription OAuth (#610) (#612)
* fix(copilot): restore generic endpoint for non-subscription OAuth (#610)
0.23.0 re-pointed the shared Copilot OAuth branch from the generic
api.githubcopilot.com host to the account-specific endpoints.api host
returned by /copilot_internal/user, and made resolve_copilot_api_url
ignore the GITHUB_COPILOT_API_URL override whenever a token resolved.
That change was meant to add --subscription, but it also altered the
pre-existing non-subscription OAuth flow that worked on 0.22.4. The
account host does not serve newer models (e.g. gpt-5.4) on the responses
API, so wrapped requests began failing with unsupported-model errors
while plain Copilot and 0.22.4 kept working.
Restore 0.22.4 routing for non-subscription OAuth (generic host, still
overridable) and keep account resolution only for --subscription.
resolve_copilot_api_url now honors GITHUB_COPILOT_API_URL first, so the
override escape hatch works for every path. BYOK is unaffected.
Add a regression suite that mocks a successful user-info response, the
real-world path the prior test never exercised (it relied on the network
call failing in CI and falling back to the generic host).
* fix(copilot): route subscription + OAuth through the generic host (#610)
The 0.23.0 endpoint resolution derived the Copilot API host from
/copilot_internal/user (endpoints.api), which returns a segmented host
(e.g. api.individual.githubcopilot.com) that does not serve newer models
on the responses API and is not the host the official Copilot client
routes with (that comes from the token-exchange endpoint). --subscription
used the identical resolution, so it carried the same latent regression
as the non-subscription OAuth path.
Make Copilot host resolution override -> generic for BOTH --subscription
and the implicit OAuth path, and stop using user-info to route. Accounts
that require a dedicated host (enterprise / data residency) pin it via
GITHUB_COPILOT_API_URL. resolve_copilot_api_url no longer makes a network
call; _fetch_copilot_user_info is retained for token validation.
Update the subscription smoke tests that encoded the old account-host
assumption, and add wrap-level + unit coverage that --subscription routes
to the generic host even when user-info advertises an account host, and
that the GITHUB_COPILOT_API_URL override flows through both paths.
* docs(copilot): document generic-host routing + enterprise override (#610)
Spell out the routing contract introduced by the #610 fix so enterprise
users have a supported path. Headroom routes wrapped Copilot hosted
traffic (--subscription and OAuth) to the generic api.githubcopilot.com,
and accounts on a dedicated host (Enterprise Cloud data residency, egress
proxy) pin it via GITHUB_COPILOT_API_URL.
- copilot --help: note the generic host + GITHUB_COPILOT_API_URL override.
- TESTING-copilot-subscription.md: add "API host & Enterprise / data
residency" section; correct the stale api.*.githubcopilot.com claim; and
invite enterprise tenants who want token-exchange-based auto-detection to
open an issue.
- integration-guide.md: short hosted-host + override note in the Copilot
section.
* fix(wrap): report unbindable proxy ports (#602)
* fix(proxy): fail-open on corrupt golden bytes instead of RuntimeError (#603)
* fix(proxy): fail-open on corrupt golden bytes instead of RuntimeError
Permanent session corruption: once golden bytes become unreadable
(UnicodeDecodeError / JSONDecodeError), every subsequent request for
the session raised RuntimeError, returning 500 until proxy restart.
Fix: log at ERROR level and recover — skip the corrupt memory tool,
or regenerate a fresh CCR definition — rather than propagating
RuntimeError and permanently breaking the session.
Also change proxy_inbound_request_aborted from logger.info to
logger.error with exc_info=True so tracebacks appear in logs.
Closes: proxy silent-500 sessions in the wild (observed 2026-06-04)
* fix(tests): re-enable headroom log propagation in corrupt-bytes tests
configure_proxy_logging() sets headroom_logger.propagate = False to prevent
duplicate writes when the proxy redirects stderr to a log file. In CI the
proxy initialises its logging stack before the test suite, leaving propagation
disabled. pytest's caplog handler attaches to the root logger, so records that
stop at the headroom logger are never captured.
Added _enable_headroom_log_propagation autouse fixture that temporarily
re-enables propagation for the duration of each test, making caplog capture
work regardless of the surrounding logging configuration.
* fix(tests): remove unused imports from corrupt-bytes regression tests
Remove json, SessionCcrTracker, and SessionToolTracker imports that were
imported but never referenced in the test body. Fixes ruff F401 and I001
lint errors reported by CI.
---------
Co-authored-by: Patrick Ancillotti <patrick.ancillotti@people.inc>
* fix(startup): suppress log noise from litellm, trafilatura, HF hub, and tiktoken warning
* fix(startup): suppress httpx INFO logs from sentence_transformers HEAD checks
* docs(changelog): add entry for startup log noise suppression fixes
* refactor(startup): extract hf_hub_download_local_first into onnx_runtime
The three _hub_download/_hub_dl helpers in embedders.py, onnx_router.py, and
kompress_compressor.py are identical -- try local cache first, fall back to
network download. Extract into a single hf_hub_download_local_first() function
in onnx_runtime.py (the natural home for shared ORT/HF utilities) and update
all three callers to use it.
* fix(lint): sort imports and remove unused _FALLBACK_WARNING_SHOWN import
* fix(lint): cast hf_hub_download return to str for mypy no-any-return
---------
Co-authored-by: Patrick Ancillotti <patrick.ancillotti@people.inc>
* ci: speed up GitHub Actions - path filters, caching, timeouts, version upgrades
Performance improvements:
- init-e2e.yml, wrap-e2e.yml: add path filters so e2e Docker builds only run when
e2e-related files change (saves ~10 min per irrelevant PR push)
- init-e2e.yml, wrap-e2e.yml: add concurrency groups to cancel superseded PR runs
- ci.yml: add pip caching to lint and build jobs
- ci.yml: cache actionlint + act binaries in workflow-validation (skip curl on hits)
- eval.yml: add pip caching to smoke-test and weekly-suite jobs
- docs.yml: add pip caching for mkdocs-material install
- rust.yml: replace cargo install --locked cargo-audit/deny with taiki-e/install-action
(prebuilt binaries; saves 2-5 min per audit run)
Bug fixes:
- docker.yml: fix actions/checkout@v6 -> @v4 (v6 does not exist; would break all
Docker builds on every release/PR touching docker paths)
Version upgrades:
- wagoid/commitlint-github-action: @v5 -> @v6
- devcontainers.yml: docker/setup-buildx-action@v3 -> @v4 (align with docker.yml)
Safety improvements:
- ci.yml: add timeout-minutes to all 13 jobs (changes, lint, build-wheel,
prefetch-model, test x4, test-extras, test-agno, commitlint, build,
workflow-validation, docker-native-e2e, windows-native-wrapper, macos-native-wrapper)
- docker.yml: add timeout-minutes to docker-build (75m), docker-manifest (20m),
promote-latest (10m)
- eval.yml: add timeout-minutes to smoke-test (30m); bump weekly-suite 60->90m
- rust.yml: add timeout-minutes to test (30m), wheels (45m), audit (20m)
Observed wall-clock impact on recent PRs:
- Init E2E and Wrap E2E were running on every single PR push regardless of content
- CI workflow was taking 12-17 min; path filters reduce unnecessary e2e runs to 0
* fix(ci): bust actionlint+act cache when workflow file changes
Static cache key 'ci-tools-actionlint-act-v1' never invalidated on
tool version updates. Switched to hashFiles('.github/workflows/ci.yml')
so the cache busts automatically whenever the download scripts are
updated to point at a newer release.
Flagged by adversarial review (Architecture + Testing/Reliability personas).
* fix(ci): add missing Dockerfile COPY paths to e2e path filters
e2e/init/Dockerfile and e2e/wrap/Dockerfile COPY files not covered
by the initial path filter set:
init-e2e: Cargo.toml, Cargo.lock, rust-toolchain.toml, uv.lock,
.claude-plugin, .github/plugin/**, plugins/headroom-agent-hooks/**
wrap-e2e: Cargo.toml, Cargo.lock, rust-toolchain.toml, uv.lock,
sdk/typescript/**, plugins/openclaw/**
Without these, a Rust toolchain bump or SDK change on a PR would
skip the e2e gate entirely, only catching it on the merge to main.
Flagged by adversarial review (Domain/Correctness persona).
* fix(devcontainer): upgrade uv to >=0.7.0 to parse uv.lock revision=3
* fix(devcontainer): set UV_SKIP_WHEEL_FILENAME_CHECK=1 in post-create.sh for gitpython wheel
* ci: bump actions/checkout and actions/setup-node to v5 (Node.js 20 EOL Jun 16)
* fix(devcontainer): export UV_SKIP_WHEEL_FILENAME_CHECK so uv run also skips wheel check
* ci: bump all GitHub Actions to latest versions (Node.js 24)
* fix(test): accept release-please-action v4 or v5 in workflow assertion
* fix(format): ruff format test_release_workflows.py
BM25Scorer.score_batch() ranks a real corpus of documents but weighted
every matched term with a constant idf=log(2.0), so a ubiquitous noise
word counted the same as a discriminative UUID. The _compute_idf() helper
needed to do this properly already existed (and was unit-tested) but was
never wired into scoring.
- Implement _compute_idf() with the standard floored BM25 IDF its docstring
documents: log((N - n + 0.5) / (n + 0.5) + 1).
- Thread an optional per-term idf_map through _bm25_score(); single-document
score() keeps the neutral log(2.0) weight (no corpus to estimate from).
- score_batch() now computes document frequency across the batch and builds
the IDF map, so rare/discriminative terms outrank corpus-wide terms in the
ranking that CompressionStore.search() and HybridScorer consume.
Adds tests covering the IDF formula and the batch ranking behaviour.
* fix(proxy): fail-open on corrupt golden bytes instead of RuntimeError
Permanent session corruption: once golden bytes become unreadable
(UnicodeDecodeError / JSONDecodeError), every subsequent request for
the session raised RuntimeError, returning 500 until proxy restart.
Fix: log at ERROR level and recover — skip the corrupt memory tool,
or regenerate a fresh CCR definition — rather than propagating
RuntimeError and permanently breaking the session.
Also change proxy_inbound_request_aborted from logger.info to
logger.error with exc_info=True so tracebacks appear in logs.
Closes: proxy silent-500 sessions in the wild (observed 2026-06-04)
* fix(tests): re-enable headroom log propagation in corrupt-bytes tests
configure_proxy_logging() sets headroom_logger.propagate = False to prevent
duplicate writes when the proxy redirects stderr to a log file. In CI the
proxy initialises its logging stack before the test suite, leaving propagation
disabled. pytest's caplog handler attaches to the root logger, so records that
stop at the headroom logger are never captured.
Added _enable_headroom_log_propagation autouse fixture that temporarily
re-enables propagation for the duration of each test, making caplog capture
work regardless of the surrounding logging configuration.
* fix(tests): remove unused imports from corrupt-bytes regression tests
Remove json, SessionCcrTracker, and SessionToolTracker imports that were
imported but never referenced in the test body. Fixes ruff F401 and I001
lint errors reported by CI.
---------
Co-authored-by: Patrick Ancillotti <patrick.ancillotti@people.inc>
* fix(copilot): restore generic endpoint for non-subscription OAuth (#610)
0.23.0 re-pointed the shared Copilot OAuth branch from the generic
api.githubcopilot.com host to the account-specific endpoints.api host
returned by /copilot_internal/user, and made resolve_copilot_api_url
ignore the GITHUB_COPILOT_API_URL override whenever a token resolved.
That change was meant to add --subscription, but it also altered the
pre-existing non-subscription OAuth flow that worked on 0.22.4. The
account host does not serve newer models (e.g. gpt-5.4) on the responses
API, so wrapped requests began failing with unsupported-model errors
while plain Copilot and 0.22.4 kept working.
Restore 0.22.4 routing for non-subscription OAuth (generic host, still
overridable) and keep account resolution only for --subscription.
resolve_copilot_api_url now honors GITHUB_COPILOT_API_URL first, so the
override escape hatch works for every path. BYOK is unaffected.
Add a regression suite that mocks a successful user-info response, the
real-world path the prior test never exercised (it relied on the network
call failing in CI and falling back to the generic host).
* fix(copilot): route subscription + OAuth through the generic host (#610)
The 0.23.0 endpoint resolution derived the Copilot API host from
/copilot_internal/user (endpoints.api), which returns a segmented host
(e.g. api.individual.githubcopilot.com) that does not serve newer models
on the responses API and is not the host the official Copilot client
routes with (that comes from the token-exchange endpoint). --subscription
used the identical resolution, so it carried the same latent regression
as the non-subscription OAuth path.
Make Copilot host resolution override -> generic for BOTH --subscription
and the implicit OAuth path, and stop using user-info to route. Accounts
that require a dedicated host (enterprise / data residency) pin it via
GITHUB_COPILOT_API_URL. resolve_copilot_api_url no longer makes a network
call; _fetch_copilot_user_info is retained for token validation.
Update the subscription smoke tests that encoded the old account-host
assumption, and add wrap-level + unit coverage that --subscription routes
to the generic host even when user-info advertises an account host, and
that the GITHUB_COPILOT_API_URL override flows through both paths.
* docs(copilot): document generic-host routing + enterprise override (#610)
Spell out the routing contract introduced by the #610 fix so enterprise
users have a supported path. Headroom routes wrapped Copilot hosted
traffic (--subscription and OAuth) to the generic api.githubcopilot.com,
and accounts on a dedicated host (Enterprise Cloud data residency, egress
proxy) pin it via GITHUB_COPILOT_API_URL.
- copilot --help: note the generic host + GITHUB_COPILOT_API_URL override.
- TESTING-copilot-subscription.md: add "API host & Enterprise / data
residency" section; correct the stale api.*.githubcopilot.com claim; and
invite enterprise tenants who want token-exchange-based auto-detection to
open an issue.
- integration-guide.md: short hosted-host + override note in the Copilot
section.
