## Problem
pyo3 marks `_native::Parser` as `#[pyclass(unsendable)]`, which causes a
hard thread-assertion panic when a parser created on one thread is
accessed from another:
```
thread '<unnamed>' panicked at pyo3-0.28.3/src/impl_/pyclass.rs:1055:9:
assertion `left == right` failed: _native::Parser is unsendable, but sent to another thread
left: ThreadId(2)
right: ThreadId(1)
```
The prior implementation stored parsers in a module-level `dict[str,
Any]` (`_tree_sitter_languages`). When `_run_compression_in_executor`
dispatched compression work to a `ThreadPoolExecutor`, pool workers
grabbed parsers from that shared dict that were originally created on
the main asyncio thread and panicked.
This produces a 500 on every request where code compression is attempted
via a pool thread.
## Fix
Replace the global dict with `threading.local()` so each thread creates
and owns its own parser instances. No cross-thread parser access is
possible.
```python
# before
_tree_sitter_languages: dict[str, Any] = {} # shared — crosses threads
# after
_tree_sitter_local = threading.local() # per-thread — isolated
```
`is_tree_sitter_loaded()` and `unload_tree_sitter()` updated to operate
on the current thread's local cache (semantics unchanged for
single-threaded callers).
## Tests
9 regression tests added in
`tests/test_transforms/test_tree_sitter_thread_safety.py`:
- Thread isolation: two threads get distinct parser instances
- Within-thread reuse: same thread gets the same cached instance
- Thread pool: parsers usable from `ThreadPoolExecutor` workers without
panic
- Concurrent workers: each distinct pool thread owns a unique parser
- `is_tree_sitter_loaded` / `unload_tree_sitter` lifecycle
Also adds a `filterwarnings` entry for
`PytestUnraisableExceptionWarning`: pyo3 emits this when short-lived
test threads drop parsers at teardown; it does not occur in production
where pool threads are long-lived.
## Relation to #564
PR #564 proposes the same `threading.local()` approach but was blocked
on missing tests (`CHANGES_REQUESTED`). This PR includes the full test
suite.
## Description
Sets pyo3 params to support python above 3.13
Fixes #(408
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Updated Cargo.toml
## Testing
Describe the tests you ran to verify your changes:
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
## Test Output
```
Compiles
```
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
## What
Loosen over-pinned Python dependency constraints and add missing upper
bounds in `pyproject.toml`. Also bump the neo4j Docker image and uv
builder version.
## Why
Several dependencies had constraints that either blocked security
patches or allowed silent major-version jumps:
- `litellm==1.82.3` was an exact pin — every security patch release
requires a manual lockfile bump
- `transformers`, `sentence-transformers` had no upper bound and have
already crossed major version boundaries without a constraint gate
- `neo4j>=5.20.0` had no upper cap; the driver has already reached 6.x
in the wild
- `mem0ai>=0.1.100` had a pre-1.0 floor while the locked version is
already 1.0.11
- `langchain-core`, `langchain-openai`, `qdrant-client`, `uvicorn` had
no upper bound on a range with active major-version churn
- `docker-compose.yml` pinned neo4j at `5.15.0`, which is 11 patch
releases behind the current 5.x LTS
- `Dockerfile` pinned uv at `0.11.16`; latest stable is `0.11.18`
## How
Constraint changes only — no code changes, no `uv lock --upgrade`. The
existing locked versions all satisfy the new bounds (we added caps, not
floors). `uv` re-resolved the lockfile to format revision 3 (adds
`upload-time` metadata fields) and cleaned up the defunct `llmlingua`
extra entries.
| Dependency | Before | After |
|---|---|---|
| `litellm` | `==1.82.3` | `>=1.82.3,<2.0` |
| `transformers` | `>=4.30.0` | `>=4.30.0,<6.0` |
| `sentence-transformers` | `>=2.2.0` | `>=2.2.0,<6.0` |
| `neo4j` | `>=5.20.0` | `>=5.20.0,<7.0` |
| `mem0ai` | `>=0.1.100` | `>=1.0.0,<2.0` |
| `langchain-core` | `>=0.2.0` | `>=0.2.0,<4.0` |
| `langchain-openai` | `>=0.1.0` | `>=0.1.0,<2.0` |
| `qdrant-client` | `>=1.9.0` | `>=1.9.0,<2.0` |
| `uvicorn` | `>=0.23.0` | `>=0.23.0,<1.0` |
| neo4j Docker image | `5.15.0` | `5.26` |
| uv (Dockerfile ARG) | `0.11.16` | `0.11.18` |
## Breaking changes
None. All currently installed versions fall within the new ranges.
Installers that previously resolved `litellm` to an older exact pin may
now resolve newer patch releases — which is the desired behavior.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
* fix(deps): add missing runtime deps to [code] and [proxy] extras
- Add gunicorn>=21.0.0 to the [proxy] extra
The proxy docs (docs/content/docs/proxy.mdx and wiki/proxy.md) show
gunicorn as the recommended production deployment server:
pip install gunicorn
gunicorn headroom.proxy.server:app --worker-class uvicorn.workers.UvicornWorker
Users installing headroom-ai[proxy] for production get uvicorn (already
declared) but had to discover and install gunicorn manually. Adding it
to [proxy] removes that friction.
Investigation notes:
- [code] only needs tree-sitter-language-pack (already declared).
code_compressor.py has zero numpy imports. The kompress fallback
inside code_compressor.py is guarded by ImportError and requires [ml].
- numpy is correctly declared in [relevance] (numpy>=1.24.0) and pulled
transitively by sentence-transformers in [memory]. It is NOT needed
under [code].
- tree-sitter is a transitive dep of tree-sitter-language-pack (requires
tree-sitter>=0.25.2) so it does not need an explicit entry.
* docs(changelog): add entry for gunicorn proxy dep fix
style(tests): ruff format test_provider_proxy_routes.py (blank lines after docstrings)
* fix(deps): move gunicorn to [proxy-prod] extra, add Windows guard
- Remove gunicorn from [proxy] so dev, CI, and Windows users are not
forced to install a Unix-only package that does nothing on Windows
- Add new [proxy-prod] extra that includes [proxy] + gunicorn with a
sys_platform != 'win32' environment marker
- Production users: pip install 'headroom-ai[proxy,proxy-prod]'
- Update CHANGELOG to reflect the new extra name
* fix(devcontainer): bump uv floor to >=0.11.0 for lockfile compatibility
uv 0.6.17 (previously pinned) cannot parse lockfiles generated by
uv >= 0.11.x. The validate CI job (triggered by pyproject.toml
changes) was failing with 'Failed to parse uv.lock'. Loosening the
pin to >=0.11.0 picks up the matching format parser while keeping the
Docker layer cacheable with a range rather than an exact pin.
* fix(devcontainer): skip gitpython wheel filename check in uv sync
gitpython 3.1.47 on PyPI has wheel gitpython-3.1.46-py3-none-any.whl
(wrong filename). uv >=0.11.19 strict filename validation rejects this
lockfile entry. UV_SKIP_WHEEL_FILENAME_CHECK=1 bypasses the check until
the upstream lockfile is regenerated with a corrected entry.
* fix(deps): correct gitpython version in uv.lock to match actual wheel
gitpython 3.1.47 on PyPI was uploaded with sdist/wheel files named
gitpython-3.1.46.*. The version field in uv.lock said 3.1.47 but all
download URLs reference 3.1.46 files, causing uv >=0.11.19 to refuse
to parse the lockfile with a version-mismatch error.
Change the version field to 3.1.46 so the entry is internally
consistent. Also revert the now-unnecessary UV_SKIP_WHEEL_FILENAME_CHECK
workaround from post-create.sh.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
The repo had drifted: pyproject.toml said 0.9.1 but PyPI's latest
published headroom-ai was 0.22.3. release_version.py papered over
this by taking max(canonical, latest_tag) at release time;
release-please does NOT do that — it trusts the manifest verbatim.
Left as-is, release-please would propose 0.9.2 on the next merge
and PyPI would reject it ("400 Cannot publish version lower than
latest"), looping the bot forever.
Fix: align every version-bearing file to 0.22.3 (the truth on
PyPI). Done via `scripts/version-sync.py --version 0.22.3`:
- .release-please-manifest.json
- pyproject.toml
- sdk/typescript/package.json
- plugins/openclaw/package.json (+ headroom-ai dep range -> ^0.22.3)
- .claude-plugin/marketplace.json
- .github/plugin/marketplace.json
- plugins/headroom-agent-hooks/.claude-plugin/plugin.json
- plugins/headroom-agent-hooks/.github/plugin/plugin.json
After this lands, the bot's next release PR will propose 0.22.4
(patch) or 0.23.0 (minor) depending on conventional-commit traffic
since v0.22.3.
Three logically-related sets of proxy changes ship in this branch:
1. Strands integration on the Bedrock path (HeadroomBundle + 4 OpenAI
handler fixes + LiteLLM cache stats + dep pin)
2. /stats MCP aggregation (cross-process events log → proxy summary)
3. Codex compression-failure fail-closed (WS + HTTP /v1/responses)
== 1. Strands integration on the Bedrock path ==
* HeadroomBundle (headroom/integrations/strands/bundle.py): single-helper
MCP wiring for a Strands Agent — Headroom MCP server (headroom_compress
/ headroom_retrieve / headroom_stats) plus optional Serena MCP and
optional in-process compression hook. Constructor builds unstarted
MCPClient instances per server; Strands' Agent owns the subprocess
lifecycle. Default config: MCP enabled, Serena enabled, hook OFF
(proxy is the single source of truth for compression). User-side
integration is two lines in any Strands app.
* headroom/proxy/handlers/openai.py — backend path now:
- calls PrefixCacheTracker.update_from_response (was direct-OpenAI only)
- intercepts CCR headroom_retrieve tool_calls server-side, mirroring
the Anthropic handler pattern; NO silent fallback, re-raises on
CCR errors (per feedback_no_silent_fallbacks)
- works for both non-streaming and streaming paths
* headroom/proxy/handlers/streaming.py: _stream_openai_via_backend now
accepts prefix_tracker + optimized_messages, parses cache stats from
the SSE final-usage frame (cache_creation_input_tokens added to the
state machine), records CCR retrieve feedback via a new
_record_ccr_feedback_from_openai_sse helper. Streaming CCR intercept
is intentionally out of scope (mirrors Anthropic streaming behaviour).
* headroom/backends/litellm.py: send_openai_message response usage block
now carries cache_read_input_tokens / cache_creation_input_tokens
(Anthropic/Bedrock dialect) and prompt_tokens_details.cached_tokens
(OpenAI dialect). Backwards-compatible — cold-start callers see the
same 3-key shape; cache keys appear only when the underlying provider
returns them. Pinned by test_no_cache_fields_means_no_cache_keys.
* headroom/proxy/auth_mode.py: ("strands-agents/", "strands") added to
CLIENT_UA_MAP. Production callers should also set X-Client: strands
since the default openai-python UA carries no Strands signal.
* pyproject.toml: huggingface-hub>=1.5.0,<2.0 pinned in [ml] so a sibling
install (e.g. strands-agents) can't drag the version below the floor
transformers 5.x requires (otherwise Kompress silently goes
"unavailable").
== 2. /stats MCP aggregation ==
* headroom/proxy/cost.py: _aggregate_mcp_events() reads the cross-process
shared events file the Headroom MCP server already writes to and
surfaces summary.mcp with three new keys:
- compressions (count of headroom_compress invocations)
- tokens_removed (sum of input - output across those)
- retrievals (count of headroom_retrieve — the load-bearing
over-compression alarm; if it grows linearly
with turn count, lossy compressors are
dropping info the model actually needs)
Defensive on every axis — missing MCP SDK, missing file, malformed
events, read errors — never blocks /stats.
* examples/strands_bundle_demo.py: stats panel prints the new fields so
the demo shows the full proxy-HTTP + MCP-tool story in one view.
== 3. Codex compression-failure fail-closed protection ==
Reported by Camille (2026-05-21): Codex threads were locking with
"ran out of room in the model's context window" after Headroom's
compression timed out on an oversized response.create frame and
forwarded the original ~1.7 MB frame to the upstream, which then
rejected it. Codex's auto-compact heuristic gates on the upstream-
reported total_usage_tokens (which Headroom had been shrinking on
earlier turns), so its compaction never fired and the thread locked.
Validated against open Codex issues (CLI + Desktop share codex-rs/core):
* #16068 — confirms compaction gates on total_usage_tokens,
estimated_token_count is computed but only logged
* #19806 — confirms image token estimator unbounded, contributes to
the same ContextManager.get_total_token_usage → auto-compaction chain
* headroom/proxy/helpers.py: decide_compression_failure_action() with a
unit-tested decision matrix:
- asyncio.TimeoutError → refuse, always
- non-timeout failure + frame > 256 KiB (configurable) → refuse
- non-timeout failure + small frame → forward (legacy)
Operator escape hatches:
- HEADROOM_WS_FAIL_OPEN_ON_COMPRESSION_FAILURE=1 restores legacy
- HEADROOM_WS_COMPRESSION_FAIL_THRESHOLD_BYTES tunes the threshold
* headroom/proxy/handlers/openai.py (WS /v1/responses): consults the
helper after compression failure. On refuse: close client websocket
code 1009 with "headroom: compression <reason> — please compact
context and retry" reason; set termination_cause for the outer
lifecycle finally; return.
* headroom/proxy/handlers/openai.py (HTTP /v1/responses): same helper.
On refuse: raise HTTPException(413) with a structured error body so
FastAPI's HTTPException handler emits a clean 413. The existing
`except HTTPException: raise` guard in this handler already ensures
the 413 propagates without being swallowed by the 502 catch-all.
Anthropic /v1/messages NOT changed in this branch: no equivalent bug
report on Anthropic-protocol clients, Claude Code (Anthropic-owned)
handles context overflow via its own cache_control/ephemeral
primitives, and Cursor/Aider don't maintain the local-Y estimate the
Codex bug requires. Deferred until a real report lands; the patch is
a one-liner reusing the same helper.
== Tests + verification ==
* tests/test_backends/test_litellm_cache_stats.py — 3 tests pinning
cache-stat surfacing across Anthropic/OpenAI dialects + backwards-
compat for no-cache responses.
* tests/test_proxy/test_openai_backend_path.py — 5 tests (Bedrock cache
fields, OpenAI fallback shape, CCR intercept with provider="openai",
CCR re-raise on exception, streaming signature contract).
* tests/test_proxy/test_mcp_stats_aggregation.py — 5 tests pinning the
aggregator across compress+retrieve mixes, empty events, unknown event
types, missing token fields, and read failures.
* tests/test_proxy/test_compression_failure_action.py — 12 tests pinning
the fail-closed decision matrix (timeout always refuses, small
transient passes through, oversize refuses, env override variants,
custom threshold, invalid threshold falls back, 0/negative ignored).
* examples/strands_bedrock_demo.py — model_id bumped from deprecated
Claude 3 Haiku to Sonnet 4.5 (the deprecated model now errors on
account access).
* examples/strands_via_proxy_demo.py — proxy + Bedrock cache + streaming
smoke test.
* examples/strands_mcp_dispatch_test.py — pure MCP round-trip probe.
* examples/strands_bundle_demo.py — full Strands + HeadroomBundle E2E
demo (this is the shape a real Strands user copies into their app).
Full pytest: 5327 passed, 178 skipped. The previously-failing
test_core_operations.py::TestAddBatch::test_add_batch_basic passes now
that the huggingface-hub pin in pyproject.toml unblocks transformers
imports.
E2E verified live against AWS Bedrock (Sonnet 4.5):
* cache_write=10,438 on turn A → cache_read=10,438 on turn B
* streaming SSE final usage frame carries cache_read_input_tokens
* 78.7% reduction on a 50 KB JSON tool_result via SmartCrusher (
dispatched per-content-type by ContentRouter)
* Strands Agent + HeadroomBundle: model autonomously called
headroom_compress + headroom_retrieve via MCP; CompressionStore
round-trip succeeded; final answer correct.
Several signals AI agents and search engines use to discover and
install a project were misaligned or missing:
* ``docs/app/layout.tsx`` set ``metadataBase`` to
``https://chopratejas.github.io/headroom/`` while the live docs run
on Vercel — every page's ``og:url`` and ``twitter:url`` resolved to
a URL that returns 404 for ``/llms.txt``. Now points at the live
Vercel host (overridable via ``NEXT_PUBLIC_SITE_URL`` for a future
custom domain). Adds explicit ``openGraph`` and ``twitter`` metadata
so social shares render a card with the project's pitch.
* No ``llms.txt`` at the GitHub repo root. AI agents crawling
``github.com/chopratejas/headroom/`` saw only the README. The new
``llms.txt`` follows the llmstxt.org convention: 1-line pitch,
canonical docs links, copy-paste install commands (pip / npm /
Docker / proxy / ``headroom wrap``), and entry points for the
library, proxy, MCP server, and SDK integrations. Points at the
Fumadocs-generated ``/llms.txt`` and ``/llms-full.txt`` for the
full picture.
* ``pyproject.toml`` ``Documentation`` URL pointed at the GitHub
README anchor. Updated to point at the docs site so PyPI visitors
land on searchable docs, and adds an ``AI / LLM Index`` URL
pointing at the Fumadocs ``/llms.txt``.
* No explicit AI-bot allow list. Added ``docs/app/robots.ts`` (Next
13+ App Router convention) with explicit allows for GPTBot,
ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot,
ChatGPT-User, Cohere-AI, CCBot, and Applebot-Extended. Wildcard
allow as the catch-all. Advertises the sitemap.
* No ``sitemap.xml`` route. Added ``docs/app/sitemap.ts`` that pulls
every Fumadocs page out of ``source`` (same source backing
``/llms.txt``, search, and OG images) so search and AI crawlers
can enumerate doc pages without scraping HTML.
* README didn't tell AI agents where to look. Added a 2-line
pointer near the top nav row: read ``/llms.txt`` here, or fetch
the live index / full docs blob.
Also tightened the GitHub repo description and added five topics
(``claude-code``, ``cursor``, ``tokens``, ``prompt-engineering``,
``typescript``) via ``gh repo edit`` — that's already live on the
repo, not part of this commit.
No Python or Rust code changes; ``make ci-precheck`` was run to
confirm the test slice still passes.
Address user-reported UX gaps across the CLI surface:
- code-aware: add --code-aware/--no-code-aware (+ HEADROOM_CODE_AWARE_ENABLED env)
to the Click CLI. PR #411 had added these only to the orphaned argparse main;
the user-facing CLI couldn't reach the flag. Banner status text "remove
--no-code-aware to enable" referenced a flag that didn't exist — fix to point
at the actual flag/env. Surface code-aware in the click banner and add
print_banner=False plumbing to run_server so the click path doesn't print
two banners back-to-back.
- --mode: hide alias clutter via metavar=[token|cache] and rewrite help to
lead with the two real modes. Legacy aliases (token_mode/token_savings/...)
still validate.
- perf --hours: was documented but ignored. Records are now actually filtered,
the report shows the actual time-range covered, and the count of records
filtered out (so users can tell when raising --hours helps).
- perf TOIN: replace the hash-keyed pattern dump with a strategy-distribution
view + recommendation-eligibility from the live store — actionable signal
rather than opaque rows.
- code-graph: clarify in --help that it indexes cwd / project root.
- wrap: spell out supported tools, wrap-vs-proxy distinction, and that
`headroom wrap opencode` isn't a thing (use `proxy` directly for opencode;
openclaw is not opencode).
- mcp: note that mcp__headroom__headroom_retrieve is correct MCP namespacing,
not a doubled-prefix bug. Renaming would break the proxy's tool injection.
- LLMLingua cleanup: remove [llmlingua] extra from pyproject (no live code
uses it). Delete wiki/llmlingua.md and clean retired flag/class references
in 6 other wiki pages. Point at [ml] (Kompress) where ML compression is
documented.
- init -g openclaw: strip mcpServers from existing plugin entries before
re-writing — newer openclaw schemas reject it, leaving stale entries from
older installs unhealable. Pinned with regression test.
Tests: mock_run_server signatures in two existing tests accept **kwargs
(needed for the new print_banner plumbing). New test for the openclaw
mcpServers strip. Full suite: 4847 passed, 262 skipped, 0 failed.
Every release since v0.20.16 has uploaded 12 wheels but no sdist. The
underlying failure is a 400 from PyPI:
400 License-File NOTICE does not exist in distribution file
headroom_ai-X.Y.Z.tar.gz at headroom_ai-X.Y.Z/NOTICE
Two-part regression:
1. The hatch -> maturin migration in 2a91cbb (single-wheel maturin build
backend, May 4) replaced `[tool.hatch.build.targets.sdist].include`,
which listed both `LICENSE` and `NOTICE`, with maturin's own include
directive that only carried `LICENSE` over. Maturin's PEP 639 license
auto-discovery still emits `License-File: NOTICE` into the sdist's
PKG-INFO (because NOTICE exists at the project root and matches the
default glob), so the sdist tarball declares a license file it
doesn't physically contain. PyPI's PEP 639 validator rejects with
400. Wheels were unaffected because maturin auto-injects both files
into `*.dist-info/licenses/`.
2. CI showed "publish-pypi" green for ~22 releases despite this break
because twine was bailing earlier with `400 File already exists` on
the wheels (the version detector kept computing the same v0.21.5).
PR #412 added `skip-existing: true` (May 6) to make wheel re-uploads
idempotent. With wheels now silently skipping, twine proceeded to
upload the sdist for the first time in three weeks - and the
dormant License-File error surfaced as a hard 400.
Fix:
- Add `NOTICE` alongside `LICENSE` in `[tool.maturin].include` for the
`sdist` format. Both files now ship in the tarball, matching what
PEP 639 already declares in PKG-INFO.
- Replace the existing "verify sdist contains LICENSE" check with a
generic "every License-File entry in PKG-INFO resolves to a real
tarball member" check. This catches the same bug class for any
future addition (COPYING, AUTHORS, etc.) without another bespoke
literal.
Verified locally:
$ maturin sdist --out dist
Including license file `LICENSE`
Including license file `NOTICE`
Including files matching "LICENSE"
Including files matching "NOTICE"
Built source distribution to dist/headroom_ai-0.9.1.tar.gz
$ tar -tzf dist/headroom_ai-0.9.1.tar.gz | grep -E '(LICENSE|NOTICE)$'
headroom_ai-0.9.1/LICENSE
headroom_ai-0.9.1/NOTICE
$ twine check dist/headroom_ai-0.9.1.tar.gz
Checking dist/headroom_ai-0.9.1.tar.gz: PASSED
Root cause: `headroom-ai[all]==0.20.16` fails to install on Python 3.13
because `rapidocr-onnxruntime` 1.4.0–1.4.4 wheels declare
`requires-python: <3.13,>=3.6`. After 1.4.x the rapidocr ecosystem
split: `rapidocr-onnxruntime` (bundled-ORT, capped at <3.13) vs
`rapidocr` 3.x (engine-agnostic core, supports 3.13+, returns
RapidOCROutput dataclass instead of v1's tuple).
Fix:
1. pyproject.toml — environment-marker hybrid in [image]:
- rapidocr-onnxruntime>=1.4.0,<2; python_version<'3.13'
- rapidocr>=3.0,<4; python_version>='3.13'
- onnxruntime>=1.7,<2; python_version>='3.13'
ORT remains the engine on every Python version; bundle and speed
unchanged, just split into two packages on 3.13+.
2. headroom/image/compressor.py — runtime adapter:
_resolve_rapidocr() tries v1 first, falls back to v3 when v1 is
missing, returns (None, None) when neither installed. Cached at
module scope. Detection at runtime (not Python-version-based) so
users can install either package on any Python version.
_ocr_extract branches on resolved api_version:
- v1: (list[(box, text, score)], elapsed) tuple — unchanged
- v3: RapidOCROutput dataclass with .txts / .scores / .boxes
attrs (each may be None when nothing detected)
Defensive None-handling, length-mismatch detection, structured
log events for both branches.
Smoke test (real install verified before commit):
pip install rapidocr onnxruntime pillow
→ result type: RapidOCROutput
→ fields: txts (None when empty), scores (None when empty), boxes
Confirms the v3 None-coercion is necessary.
Tests: 11 new unit tests in tests/test_image_ocr_api_compat.py covering:
- Resolver: v1 preferred, v3 fallback, both missing
- v1 path: tuple parses, low-confidence None, empty result None
- v3 path: dataclass parses, low-confidence None, None attrs handled,
mismatched lengths logged + None
- Backend missing: returns None gracefully
All 11 pass; `make ci-precheck` PASSED.
Closes#372.
Devcontainer validate jobs were failing on PR #360 with:
× Failed to fetch:
https://pypi.netflix.net/packages/.../nvidia_nvshmem_cu12-3.4.5-...whl
├─▶ Request failed after 3 retries
╰─▶ operation timed out
`pypi.netflix.net` is Netflix's internal PyPI mirror. It got into the
lockfile because my local `~/.config/uv/uv.toml` had:
index-url = "https://pypi.netflix.net/simple"
Running `uv lock` from that machine baked Netflix-internal URLs into
uv.lock for every package. Public CI runners (and any external
contributor) can't resolve them.
Two fixes:
1. Add `[[tool.uv.index]]` block to pyproject.toml pinning public PyPI
as the project's default index. uv now ignores user-level config when
resolving for this project, regardless of who runs `uv lock`. This
prevents the same contamination from any developer in the future.
2. Regenerate uv.lock against public PyPI. All package URLs now point
at `https://files.pythonhosted.org/...` and `https://pypi.org/simple/`.
Zero references to `pypi.netflix.net` remain in the lockfile.
Verified: `grep -c "pypi.netflix" uv.lock` returns 0.
Eliminates the dual-package architecture that was the root cause of #355.
`pip install headroom-ai` now produces ONE wheel containing both the Python
source (headroom/*.py) and the compiled Rust extension (headroom/_core.so).
No more separate `headroom-core-py` package, no more chicken-and-egg with
PyPI publication, no more wheelhouse / PIP_FIND_LINKS / composite-action
plumbing in CI.
This is the canonical pattern used by cryptography, polars, ruff,
pydantic-core, and other Rust-as-core Python packages. Honors the
"Rust as core engine" direction.
## What changed
- pyproject.toml: `[build-system]` swapped from hatchling to maturin.
`[tool.hatch.*]` deleted; `[tool.maturin]` added pointing at
`crates/headroom-py/Cargo.toml` for the cdylib. `python-source = "."`
picks up the root `headroom/` package directly (dashboard HTML
templates and other non-Python files included automatically).
- crates/headroom-py/pyproject.toml: deleted. The crate is no longer a
separate published package; its Cargo.toml stays as the cdylib build
target invoked via `[tool.maturin] manifest-path`.
- crates/headroom-py/python/: deleted (placeholder layout for the old
separate package).
## CI updates
- ci.yml: `test` / `test-extras` / `test-agno` jobs simplified — Rust
toolchain set up before `pip install -e .` (which now invokes maturin
via build-system). Removed the "build wheel + symlink .so" dance.
`build` job swapped from `python -m build` (hatch) to
`maturin build` + `maturin sdist`.
- release.yml: collapsed dual-package matrix into one. New `build-wheels`
matrix produces cross-platform wheels for cp310/11/12/13 ×
{linux x86_64, linux aarch64, macos x86_64, macos aarch64}. New
`collect-dist` aggregator merges artifacts. publish-pypi consumes the
merged dist.
- init-native-e2e.yml: dropped windows-latest from the matrix —
upstream `esaxx-rs` (/MT) and `ort-sys` (/MD) link with conflicting
MSVC C runtime libraries, so the Rust extension cannot build for
win_amd64 today. Tracked as a follow-up; not a blocker for Linux+macOS.
- headroom-e2e-setup: composite action now sets up Rust toolchain +
Swatinem/rust-cache before `pip install -e .[proxy]`.
- eval.yml, publish.yml, rust.yml: same pattern — rust toolchain before
install. rust.yml's wheels job builds from root pyproject.toml (no
more `-m crates/headroom-py/Cargo.toml`).
- e2e/init/Dockerfile, e2e/wrap/Dockerfile: install rust + maturin in
the build stage; copy `crates/` + workspace `Cargo.toml/lock` so the
install can build the extension. Dropped `HEADROOM_REQUIRE_RUST_CORE=false`
from wrap-e2e — the image now ships the full Rust core.
- Dockerfile (main): simplified — no more Layer 2/3 dance with
`headroom-core-py` install + symlink. Single `uv pip install` builds
+ installs everything.
- .devcontainer/Dockerfile: rust toolchain + libssl-dev + maturin
added so `uv sync` builds the extension inside the devcontainer.
## Lockfile + script
- uv.lock: regenerated. No `headroom-core-py` entries remain.
- scripts/build_rust_extension.sh: simplified from a symlink-into-tree
workaround to a thin wrapper around `pip install -e .`. The maturin
build-backend handles placement automatically.
## Local validation (all green on macOS aarch64)
1. Clean venv `pip install -e .` → `from headroom._core import …` works.
2. `maturin build --release` → 13.8 MB wheel, 336 files including
`headroom/_core.cpython-311-darwin.so` (32 MB cdylib) and
`headroom/dashboard/templates/dashboard.html`.
3. `pip install <wheel>` in fresh venv → import works.
4. Wheel contents verified via `unzip -l`.
5. `pytest tests/test_transforms/test_diff_compressor.py` — 29 passed.
6. `pytest tests/test_relevance.py` — 30 passed.
7. `cargo build --workspace` + `cargo test --workspace` — all green.
8. `make ci-precheck` — 176 Python tests + Rust + commitlint green.
## Migration notes
Users on `pip install headroom-ai` get the Rust core automatically
(linux + macos wheels). sdist installs require rust toolchain available
locally — pip will build via maturin.
Closes#355
Supersedes #357 (workarounds-based fix abandoned in favor of
architectural fix)
Adds tests/test_realignment_live_multi_turn.py with 9 OPT-IN live tests
that validate the load-bearing claims of the Phase A+B megamerge against
real upstream APIs (Anthropic, OpenAI, Gemini). Each test maps to one or
more realignment PRs:
1. test_anthropic_cache_hit_across_two_turns — A2/A6/E
Identical cache_control'd system+messages on two turns must
eventually produce cache_read_input_tokens > 0. Guards the cache
hot zone invariant (I2): proxy must not mutate frozen prefix bytes.
Uses a bounded retry loop (max 4 attempts) to absorb Anthropic's
eventually-consistent prompt-cache write latency without masking
a real "proxy broke cache stability" regression.
2. test_anthropic_cache_stable_when_live_zone_compresses — B2/B3
Turn 2 mutates only the LATEST user content (8KB+ JSON tail);
cache_read on turn 2 must still be > 0 AND the proxy must emit
compression headers — proving the live-zone block dispatcher
ran on the new tail without disturbing the cached prefix.
3. test_anthropic_cache_control_passthrough_byte_faithful — A3/A4
Wraps proxy._retry_request to snapshot the upstream-bound body
and assert cache_control on system blocks survives verbatim,
and user content is not flattened from list to string form.
4. test_openai_chat_completions_multi_turn_through_proxy — A8/B
Three-turn conversation through /v1/chat/completions; each
turn returns valid content, prior assistant turns survive in
the messages list (proxy doesn't drop them).
5. test_openai_streaming_sse_chunks_arrive_in_order — A8 (SSE wire)
Streams /v1/chat/completions; asserts each event is
'data: ...\\n\\n', terminator is 'data: [DONE]\\n\\n',
reassembled content non-empty, no malformed events.
6. test_gemini_multi_turn_through_proxy — Gemini reach
Two-turn conversation through native
/v1beta/models/{model}:generateContent. Proves Gemini handler
wiring stayed intact through the megamerge.
7. test_ccr_marker_round_trip_live — B7 (CCR)
Pre-populates compression_store with a fixture entry, embeds
a CCR marker on a tool_result, verifies (a) headroom_retrieve
tool is injected into the upstream tools array (PR-B7
always-on), and (b) /v1/retrieve returns the original bytes
by hash with all rows intact. Pre-populating the Python store
(vs. driving SmartCrusher's internal Rust store) matches the
established pattern in tests/test_proxy_ccr.py and exercises
the surface served by /v1/retrieve.
8. test_memory_tail_injection_does_not_modify_system_prompt_live — B6/A2
Spins up a memory-enabled proxy with MemoryMode.AUTO_TAIL,
seeds LocalBackend, captures upstream-bound body. Asserts:
(a) system prompt byte-identical to input; (b) memory text
lands on latest user message tail; (c) earlier messages
untouched. Guards the live-zone-only injection contract.
9. test_classify_auth_mode_routes_payg_vs_oauth — Phase F-prep / B5
NOT a live API call. Sends three header shapes through the
proxy (x-api-key=..., Bearer sk-ant-oat01-..., Bearer
sk-ant-api03-...), captures dispatcher headers via a wrap on
_retry_request, and asserts the canonical auth-mode classifier
maps each correctly. Codifies the Phase F contract.
Conventions:
* file-level pytestmark = pytest.mark.live → excluded by default
via 'pytest -m "not live"'. Adds a 'live' marker registration in
pyproject.toml's [tool.pytest.ini_options].markers.
* each test skipif's on the relevant API key — no silent fallbacks,
no real-API runs against fake keys.
* uses tests/_dotenv.py helpers (load_env_overrides + autouse_apply_env)
rather than re-implementing env loading.
* model IDs and thresholds live in a top-of-file LIVE_CONFIG dict
(no hardcodes); Anthropic primary/fallback resolves at runtime per
key entitlement.
* assertions are direction-only (cache_read > 0, tokens_after <=
tokens_before) — never tied to upstream pricing/tokenizer drift.
* shared module-scoped TestClient fixture for performance; CCR and
memory tests build dedicated proxies for their config-specific paths.
Verification:
* pytest tests/test_realignment_live_multi_turn.py -v
→ 9 passed, 0 skipped, 0 failed in ~25s (with all keys set)
* pytest -m "not live" --tb=short -q
→ 4694 passed, 265 skipped, 9 deselected — same baseline as today
* make ci-precheck → green (rust + python + commitlint)
Per-realignment-plan: REALIGNMENT/04-phase-B-live-zone.md.
Replace `sentence-transformers` (PyTorch-backed) with `fastembed` (ONNX-
backed) so Python and Rust call into the same library + same model for
relevance scoring. Both sides run BAAI/bge-small-en-v1.5 (33M params,
384 dims, ~30 MB int8-quantized ONNX) auto-downloaded from HF Hub.
Cross-language verification on ('authentication failed for user',
'login error'): Python=0.7505, Rust=0.7507, delta ~0.0002 — well below
the relevance_threshold (0.3) buffer SmartCrusher uses for keep/drop
decisions, so the two implementations agree on every observable
SmartCrusher output. (True byte-equal would require both to load the
identical ONNX weights file — Rust's `fastembed` crate and Python's
`fastembed` package can pick different upstream artifacts; deferred.)
Why fastembed:
- removes torch from the relevance/ path (Phase 6: drop torch from
Python).
- ~2-3x faster than sentence-transformers' all-MiniLM-L6-v2 for the
same input shape.
- bge-small-en-v1.5 outranks all-MiniLM-L6-v2 on MTEB by ~6 points.
- self-contained: no longer reads ML_MODEL_DEFAULTS.sentence_transformer
from utils config.
What this does, in plain terms:
Headroom's proxy now ships with three CLI tools (ast-grep, difftastic,
scc) that it can use to shrink tool_result payloads before they reach
the model. The goal is simple: when Claude Code (or Codex, Aider, etc.)
asks the model to reason about a big file or diff, we swap the verbose
output for a compact, same-meaning version. Fewer tokens per turn, same
answers, lower bill.
Today a single interceptor is wired: ast-grep on Read. When an agent
reads a large code file, the proxy replaces the file body with an
outline of its top-level functions/classes plus docstrings. In live
tests that cut prompt tokens 74–76% on both OpenAI and Anthropic,
same answer either way.
How it works:
- `pip install headroom-ai` now installs ast-grep via a PyPI wheel
(core dep). difftastic and scc are fetched once at proxy startup
from pinned upstream GitHub releases and cached per-user.
- A generic registry (`headroom/proxy/interceptors/`) lets us add more
tool-aware rewrites in one file each: declare `matches()` and
`transform()`, call `register()`, done. No proxy or metrics plumbing
per tool.
- Safety rails built in: pass-through when a Read specifies a line
range; second Read of the same file in a conversation returns full
content (progressive disclosure); any failing interceptor logs and
skips, never crashes a request.
Opt-in for now:
- Off by default while this ships. Turn on with
`headroom proxy --intercept-tool-results` or
`HEADROOM_INTERCEPT_ENABLED=1`, so we can measure before flipping
defaults.
What users see after turning it on:
- First `headroom wrap claude` boot is ~5s longer (binaries fetched).
Every subsequent run is cache-only.
- Existing `transforms_applied` field in metrics gets entries like
`interceptor:ast-grep`, so savings show up in current dashboards
and HTML reports with no UI change.
Other housekeeping in this PR:
- uv.lock moved to .gitignore — regenerated locally per environment.
- 35 unit + integration tests, ruff + mypy clean.
- Dead-code audit done: removed `binaries.run()`, `needs_filesystem`
plumbing, unused `_kind` tuple elements, unused `tool_output`
parameter, and the never-set HEADROOM_SKIP_TOOLS_BOOTSTRAP env.
Add websockets to the dev extra so the repro harness smoke test can import
its websocket client dependency in the CI test matrix. Also apply ruff
formatting to the files the formatter check was rejecting so the 3.12 lint
job passes.
Adds `headroom/proxy/extensions.py` — a generic entry-point hook under
the `headroom.proxy_extension` group. External packages register an
`install(app, config)` callable that runs once at proxy startup and is
free to add ASGI middleware, routes, mutate config, or raise to
fail-closed (e.g., license check failure aborts startup).
Why: Headroom OSS keeps a deliberately minimal surface, but several
high-value capabilities — PII redaction + tool-call vaulting, data-
residency routing, multi-tenant RBAC, compliance audit, vision/voice
privacy — only make sense as separately-distributed packages. A small,
stable plugin contract lets those live outside this repo without
requiring the OSS to know about them.
Contract:
[project.entry-points."headroom.proxy_extension"]
my_extension = "my_pkg.extension:install"
def install(app: FastAPI, config: ProxyConfig) -> None: ...
An extension that raises from install() is a deliberate fail-closed
signal and aborts startup. Entry-point load failures are logged and
skipped so one broken third-party package cannot take the proxy down.
Changes:
* New: headroom/proxy/extensions.py (~55 LOC)
* headroom/proxy/server.py: invoke install_all(app, config) in
create_app() immediately after CORS middleware registration
* pyproject.toml: register slow and real_llm pytest markers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three fixes for Docker-native install with --memory:
1. Add sqlite-vec to proxy extras — memory's vector index now installs
with pip install headroom-ai[proxy]. No separate pip install needed.
2. Fix AUTO vector backend fallback — was: SQLITE_VEC → HNSW → crash.
Now: SQLITE_VEC → HNSW → clear error message listing install options.
3. Fix ONNX pthread_setaffinity_np error in Docker containers — set
intra/inter thread count to 1 in SessionOptions. Prevents the
"Invalid argument" error on containers with limited CPU affinity.
No manual install needed. headroom wrap claude --code-graph:
1. Checks if binary exists (PATH or ~/.local/bin/)
2. Downloads from GitHub releases if not found (~15MB, one-time)
3. Indexes project + starts file watcher
Same pattern as rtk auto-download. Platform-aware:
darwin-arm64, darwin-amd64, linux-arm64, linux-amd64, windows-amd64.
Also: add watchdog>=4.0.0 to proxy extras.
Add a default devcontainer and a compose-backed memory-stack profile, validate them in CI, and document the contributor workflow.
Also lock the memory-stack dependencies, pin related container tooling, and sync the latest healthcheck shutdown fix for stubbed memory handlers.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a shared observability layer for OTEL metrics and Langfuse tracing.
Instrument the shared compression pipeline once so SDK and proxy paths can
reuse the same metrics and spans without scattering telemetry concerns.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Transcode technique now runs RapidOCR to extract text from images.
Falls back to full_low if OCR confidence < 70% or no text detected.
Token counting is now done AFTER compression by measuring the actual
output — no hardcoded estimates. OCR text counted by char length,
resized images re-estimated from new dimensions.
RapidOCR added to headroom-ai[image] extra (~15MB ONNX models, ~180ms CPU).
4 new OCR tests (extraction, blank image, confidence threshold, full pipeline).
29 total image compression tests passing.
SiglipTokenizer from transformers hard-requires sentencepiece at runtime.
Without it, the image router silently falls back to PRESERVE (no compression).
Users on pip install headroom-ai[image] or [all] were missing this dependency.
- Fix beacon spam: file lock ensures only one beacon per proxy regardless
of worker count. Workers > 1 caused N beacons firing N rows per cycle.
- Beacon upsert: on_conflict=session_id prevents duplicate rows.
- Beacon stop() guard: skip final report if uptime < 2 minutes.
- Fix dashboard cost: savings_usd now uses model list price (monotonic),
not moving average. Separate breakdown for compression/cache/rtk.
Fixes#83
KompressCompressor now tries ONNX Runtime first (156MB INT8 model),
falls back to PyTorch only if ONNX unavailable. No torch needed for
text compression — just onnxruntime (~50MB) + transformers (tokenizer).
Changes:
- Add onnxruntime + transformers to [proxy] extra in pyproject.toml
- Add _OnnxModel wrapper with get_scores/get_keep_mask interface
- _load_kompress() tries ONNX first, falls back to PyTorch
- is_kompress_available() returns True if EITHER backend available
- compress() handles both numpy (ONNX) and tensor (PyTorch) outputs
Dependency impact:
Before: pip install headroom-ai[proxy] → no text compression
After: pip install headroom-ai[proxy] → Kompress ONNX INT8 (156MB)
[ml] extra still available for full PyTorch (600MB, GPU support)
Codex v0.117.0+ with newer models uses WebSocket instead of HTTP POST
for the Responses API. Added @app.websocket("/v1/responses") handler that:
- Accepts ws:// connections and forwards to wss://api.openai.com
- Compresses input on first message using existing pipeline
- Relays all response events bidirectionally
- Handles SSL (certifi), graceful disconnect, missing websockets lib
Tested with real OpenAI API: basic text, large tool outputs (200 records),
parallel function calls, instructions preservation.
Addresses #79
