mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
2630 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Ben Younes
|
90734b691a
|
fix(proxy): keep large compression results on the critical path (#296) (#1352)
## Description In Anthropic token mode, compression appears to complete in the transform pipeline (`proxy.log` shows `Pipeline complete: ... saved N tokens`), but ~30s later the proxy times out in `compression_first_stage` and forwards the **original** uncompressed request — so `/stats` and `recent_requests` show `tokens_saved: 0`, `savings_percent: 0.0`, `transforms_applied: []`, `optimization_latency_ms: ~31,000`. It starts once a compacted Claude Code transcript grows to ~367k–425k input tokens. Root cause: after the pipeline finishes, `TransformPipeline.apply` runs a **telemetry-only** waste-signal re-parse of the *original* messages (`parse_messages`) on the critical path. On a several-hundred-thousand-token transcript that diagnostic parse can take tens of seconds and blow the Anthropic compression timeout — so the already-computed compression result is discarded and the proxy fails open with the original request. Fix: skip waste-signal detection above `MAX_WASTE_SIGNAL_DETECTION_TOKENS` (100k). The diagnostic never changes the compression result, so skipping it on huge requests keeps the result on the critical path. Smaller requests are unaffected. (The earlier diagnostics PRs #303/#304 — both merged — added the `request_id`/exception-type logging that made this root cause visible. This is the focused follow-up fix.) Closes #296 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/transforms/pipeline.py`: gate waste-signal detection on `tokens_before <= waste_signal_token_limit` (default `MAX_WASTE_SIGNAL_DETECTION_TOKENS = 100_000`, overridable via kwarg); above the limit, log a debug line and skip. Extracted the "saved enough" predicate and a named `_MIN_TOKENS_SAVED_FOR_WASTE_SIGNALS` constant (was a bare `100`). - `tests/test_transforms/test_pipeline_waste_signal_limit.py`: new regression test — above the limit the waste-signal parse is skipped and the compression result is preserved; below the limit it still runs. - `CHANGELOG.md`: Unreleased → Bug Fixes entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_canonical_pipeline.py tests/test_proxy_anthropic_compression_diagnostics.py tests/test_transforms/test_pipeline_waste_signal_limit.py -q 12 passed in 35.97s $ uv run ruff check headroom/transforms/pipeline.py tests/test_transforms/test_pipeline_waste_signal_limit.py All checks passed! $ uv run mypy headroom/transforms/pipeline.py Success: no issues found in 1 source file ``` #### TDD verification (RED → GREEN) RED — new test with the prod fix reverted (waste-signal detection still runs on the large request): ```text E AssertionError: waste-signal parse must be skipped above the limit assert True is False 1 failed, 1 passed in 0.17s ``` (The 1 passing on red is the below-limit no-regression guard.) GREEN — with the fix applied: ```text 2 passed in 0.12s ``` ## Real Behavior Proof - Environment: Linux, Python 3.13, headroom @ this branch. - Exact command / steps: drive `TransformPipeline.apply` with a stub transform that compresses and a tracked `parse_messages`, sizing the request above vs below the limit: - `tokens_before=200_000`, limit `100_000` → `parse_messages` is **not** called; the result still carries `transforms_applied=['test:shrink']` and `tokens_after < tokens_before` (pre-fix: `parse_messages` ran, which is the slow step the timeout killed, discarding this result). - `tokens_before=10_000`, limit `100_000` → `parse_messages` **is** called (diagnostic preserved for normal requests). - Observed result: above the limit the compression result reaches the caller without the diagnostic parse that caused the timeout; below the limit behavior is unchanged. - Not tested: the live multi-hundred-k-token Claude Code session against Anthropic that originally tripped the wall-clock timeout (needs a real large transcript + provider); the causal chain (slow `parse_messages` on the critical path → timeout → discard) is covered deterministically by the unit test. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The limit is overridable per-call via the `waste_signal_token_limit` kwarg, so callers that want the diagnostic on larger requests can opt back in. Waste-signal data is telemetry only (OTel metrics) — it never affects the compressed output sent upstream. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Ben Younes
|
b50d9c17ce
|
feat(wrap): add --1m to preserve the 1M context window on wrap claude (#1158) (#1351)
## Description `headroom wrap claude` is the recommended Claude Code integration, but for subscription users entitled to the **1M** context window it silently caps usable context at **200k**. Root cause (upstream, anthropics/claude-code#68522): when `ANTHROPIC_BASE_URL` points at a custom host (the Headroom proxy), Claude Code does **not** send the `context-1m-2025-08-07` beta header and treats the window as 200k. The `/model opus[1m]` picker selection does not survive a custom base URL, and `CLAUDE_CODE_AUTO_COMPACT_WINDOW` alone does not lift the cap. Headroom itself already forwards `anthropic-beta` and sizes Opus at 1M internally — but since `wrap claude` owns the launched process's environment and is the documented path, users hit this and blame Headroom first. This adds the opt-in fix the issue proposes. Closes #1158 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/wrap.py`: new opt-in `--1m` flag on `wrap claude`. When set, `ANTHROPIC_MODEL=<opus>[1m]` is exported on the launched process so Claude Code sends the `context-1m` beta header. Logic extracted to a testable helper `_resolve_1m_model`: a model the user already selected via `ANTHROPIC_MODEL` is preserved (only the `[1m]` suffix is appended when missing); otherwise it falls back to the default Opus. Idempotent (no double suffix). Default behavior is unchanged (opt-in). - `tests/test_cli/test_wrap_helpers.py`: unit tests for `_resolve_1m_model` (append-to-user-model, idempotent, default fallback). - `README.md`: `--1m` added to the Claude Code row of the agent compatibility matrix. - `CHANGELOG.md`: Unreleased → Features entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_helpers.py tests/test_cli/test_wrap_claude_base_url.py -q 61 passed in 0.46s $ uv run ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_helpers.py All checks passed! $ uv run mypy headroom/cli/wrap.py Success: no issues found in 1 source file ``` #### TDD verification (RED → GREEN) RED — new tests with the prod change reverted (`_resolve_1m_model` absent): ```text E AttributeError: module 'headroom.cli.wrap' has no attribute '_resolve_1m_model' 3 failed, 40 deselected in 0.56s ``` GREEN — with the change applied: ```text 3 passed, 40 deselected in 0.34s ``` ## Real Behavior Proof - Environment: Linux, Python 3.13, headroom @ this branch. - Exact command / steps: `headroom wrap claude --1m --help` shows the new flag, and the flag resolves the model id that triggers the 1M window: ```text $ headroom wrap claude --help | grep -A1 -- --1m --1m Preserve the 1M context window. Behind a custom ANTHROPIC_BASE_URL Claude Code drops the ... # model-id resolution (what --1m exports as ANTHROPIC_MODEL): _resolve_1m_model("claude-opus-4-1-20250805") -> "claude-opus-4-1-20250805[1m]" _resolve_1m_model("claude-opus-4-8[1m]") -> "claude-opus-4-8[1m]" (idempotent) _resolve_1m_model(None) -> "claude-opus-4-8[1m]" (default) ``` - Observed result: with `--1m`, the launched Claude Code process gets `ANTHROPIC_MODEL=<opus>[1m]`, which is the documented trigger for the `context-1m` beta header (verified in the issue against `~/.headroom/logs/proxy.log`). - Not tested: the live Claude Code subscription handshake against Anthropic's servers (requires a 1M-entitled subscription + the proprietary client); the model-id → header behavior is Claude Code's, documented in the issue and upstream anthropics/claude-code#68522. Headroom's side (export the env var that flips it on) is covered above and by the unit tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes Opt-in only — without `--1m` nothing changes. The `_DEFAULT_1M_MODEL` constant is only consulted when the user has no `ANTHROPIC_MODEL` set; users on a specific model keep it (suffix appended), so the default's freshness does not affect them. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ben Younes
|
e6bbc40b11
|
fix(codex): retag threads on init so Codex Desktop history stays visible (#961) (#1349)
## Description
Installing Headroom for Codex via `headroom init` can make Codex Desktop
appear to lose its local chat/thread history. The data is never deleted
— Codex filters its sidebar/search by the active `model_provider`, and
the init path set `model_provider = "headroom"` without retagging
existing threads, so native `openai` threads disappeared from the menu.
The install (`headroom.providers.codex.install`) and wrap
(`headroom.cli.wrap`) paths already reconcile thread provider tags
across the proxy boundary (retag `openai -> headroom` on enable). The
init path — `_ensure_codex_provider` in `headroom/cli/init.py`, which is
exactly what the issue reproduces ("Headroom init proxy" provider,
`headroom-init-codex` hook) — was the one place that injected the
provider without retagging. This wires the same reconciliation into the
init path. The revert direction is already handled by `headroom unwrap
codex`.
Closes #961
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/init.py`: `_ensure_codex_provider` now calls
`retag_to_headroom(path.parent)` after writing the init provider block,
so existing native threads stay visible under the active `headroom`
provider. Third-party providers (e.g. `anthropic`) are left untouched
(existing `retag_thread_providers` behaviour).
- `tests/test_cli/test_init_cli.py`: regression test seeding a Codex
Desktop `state_5.sqlite` and asserting `_ensure_codex_provider` retags
`openai -> headroom` while leaving other providers alone.
- `CHANGELOG.md`: Unreleased → Bug Fixes entry.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_cli/test_init_cli.py tests/test_provider_codex_threads.py tests/test_provider_codex_install.py -q
84 passed in 0.84s
$ uv run ruff check headroom/cli/init.py tests/test_cli/test_init_cli.py
All checks passed!
$ uv run mypy headroom/cli/init.py
Success: no issues found in 1 source file
```
#### RED → GREEN proof
RED — new test with the prod fix (the `retag_to_headroom` call)
reverted:
```text
E AssertionError: existing openai threads not retagged: {'anthropic': 1, 'openai': 2}
1 failed in 0.44s
```
GREEN — with the fix applied:
```text
tests/test_cli/test_init_cli.py::test_init_codex_provider_retags_existing_threads
1 passed in 0.36s
```
## Real Behavior Proof
- Environment: Linux, Python 3.13, headroom @ this branch.
- Exact command / steps: seed a Codex Desktop store
(`<codex_home>/sqlite/state_5.sqlite`) with native threads, then run the
init provider injection:
```text
before init: {'anthropic': 1, 'openai': 2}
after init: {'anthropic': 1, 'headroom': 2}
config model_provider line: ['model_provider = "headroom"',
'[model_providers.headroom]']
```
- Observed result: after init, the two `openai` threads are retagged to
`headroom` (so they stay visible under the now-active provider), while
the `anthropic` thread is left untouched. Before the fix they stayed
`openai` and were filtered out of Codex Desktop's menu.
- Not tested: the live Codex Desktop GUI itself (proprietary, no
sandbox); the filtering behaviour is the documented `thread/list`
provider filter described in the issue, and the store-level retag that
makes history visible is covered above and by the unit test.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
Scope is limited to the init provider path; the install and wrap paths
already perform this reconciliation. Screenshots N/A (no UI change on
Headroom's side).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
Ben Younes
|
feedead077
|
fix(install): stop duplicating ENTRYPOINT in persistent-docker runtime command (#833) (#1348)
## Description `headroom install apply --preset persistent-docker` pulls the image, starts the container, then fails after ~45s with "Deployment 'default' did not become ready after start." The rollback removes the container and manifest, leaving nothing running and no logs. Root cause: the published image already bakes the proxy invocation into its ENTRYPOINT (`Dockerfile`: `ENTRYPOINT ["headroom", "proxy"]`), but `build_runtime_command()` in `headroom/install/runtime.py` re-added `headroom proxy` after the image name. Docker concatenates ENTRYPOINT + args, so the container ran `headroom proxy headroom proxy --host 0.0.0.0 ...` and Click aborted with `Got unexpected extra arguments (headroom proxy)`. The runtime command now appends only the proxy flags after the image name, substituting the all-interface container bind host for the host pair carried in `proxy_args`. Closes #833 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/install/runtime.py`: drop the duplicated `headroom proxy` from the docker `build_runtime_command` output; append only `--host <bind> *proxy_args[2:]`. Extracted `CONTAINER_BIND_HOST` and `_PROXY_ARGS_HOST_PAIR_LEN` named constants. - `tests/test_install/test_runtime.py`: new regression test asserting the args appended after the image name never re-add the `headroom proxy` ENTRYPOINT. - `CHANGELOG.md`: Unreleased → Bug Fixes entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_install/ -q 91 passed, 1 skipped in 5.48s $ uv run ruff check headroom/install/runtime.py tests/test_install/test_runtime.py All checks passed! $ uv run mypy headroom/install/runtime.py Success: no issues found in 1 source file ``` #### RED → GREEN proof RED — new test with the prod fix reverted (test kept): ```text E AssertionError: container args re-add the ENTRYPOINT — got ['headroom', 'proxy', '--host', '0.0.0.0', '--port', '8787', '--backend', 'anthropic'] FAILED tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_does_not_duplicate_entrypoint 1 failed in 0.17s ``` GREEN — with the fix applied: ```text tests/test_install/test_runtime.py::test_build_runtime_command_for_docker_does_not_duplicate_entrypoint 1 passed in 0.11s ``` ## Real Behavior Proof - Environment: Linux, Python 3.13, headroom @ this branch. - Exact command / steps: reproduce the exact concatenation Docker performs (ENTRYPOINT `headroom proxy` + the buggy CMD `headroom proxy --host 0.0.0.0 --port 8787`): ```text $ headroom proxy headroom proxy --host 0.0.0.0 --port 8787 Usage: headroom proxy [OPTIONS] Try 'headroom proxy --help' for help. Error: Got unexpected extra arguments (headroom proxy) ``` This is the exact error from the issue. After the fix, `build_runtime_command` appends only the flags after the image name: ```text args after image: ['--host', '0.0.0.0', '--port', '8787', '--backend', 'anthropic'] ``` so the container runs `headroom proxy --host 0.0.0.0 --port 8787 --backend anthropic` (ENTRYPOINT + flags) and Click accepts it. - Observed result: pre-fix Click aborts with the unexpected-arguments error (container crash-loops); post-fix the command line is valid. - Not tested: pulling and running the real `ghcr.io` image end-to-end (requires the published image + Docker host); the failure is fully determined by the generated argv, which is covered above and by the unit test. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes Scope is limited to the docker runtime command construction. The Python (`runtime_kind=python`) path was already correct and is unchanged. Screenshots N/A (CLI-only change). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ben Younes
|
0c9b42a919
|
fix(ccr): propagate --no-ccr-marker flag to all compressors (#1022) (#1197)
## Description
Propagate `--no-ccr-marker` flag to SearchCompressor, LogCompressor,
DiffCompressor, and CodeAwareCompressor — previously only SmartCrusher
honored the flag. When `ccr_inject_marker` is `False`, the other
compressors still defaulted to `enable_ccr=True`, injecting
`<<ccr:...>>` markers into compressed output.
Closes #1022
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/transforms/content_router.py`: pass
`enable_ccr=self.config.ccr_inject_marker` from
`_get_search_compressor`, `_get_log_compressor`, `_get_diff_compressor`,
and `_get_code_compressor` — mirroring what `_get_smart_crusher` already
does with `inject_retrieval_marker`
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
Baseline: 1 pre-existing failure, 2020 pass, 131 skip
Post-fix: 1 pre-existing failure, 2022 pass, 131 skip
No regressions — 5 new tests in TestNoCcrMarkerCompressors, all pass.
```
## TDD verification
- RED check (without fix):
`test_content_router_propagates_ccr_inject_marker_false_to_compressors`
FAILED — `SearchCompressor enable_ccr=True, expected False`
- GREEN check (with fix): all 5 new tests PASS — propagation test
confirms `enable_ccr=False` reaches all compressors; integration tests
confirm no `<<ccr:` markers in compressed output
## Real Behavior Proof
- Environment: Linux, Python 3.13.12, headroom main @
|
||
|
Priyanshu Sharma
|
8da0b4e565
|
fix(install): guard install_agent_ensure against duplicate runtime spawns (#1301)
## Type of Change
- [x] Bug fix (non-breaking change which fixes an issue)
## Description
`install_agent_ensure` in `cli/install.py` only checked
`probe_ready(health_url)`. If the proxy was alive-but-not-ready (e.g.
during cold start while tokenizers load — ~38s on Windows),
`probe_ready` returned false and it unconditionally called
`_start_deployment` → `start_detached_agent`, spawning a **second
runtime** without:
1. acquiring `acquire_runtime_start_lock`
2. checking `runtime_status`
3. stopping the existing instance
Two proxies then contend for `127.0.0.1:<port>`; only one can bind, and
the deployment ends up wedged (never ready). Every subsequent ensure
spawns yet another runtime → restart storm.
By contrast, the hook path `cli/init.py:_ensure_profile_running` does it
correctly: it acquires the start-lock, checks `runtime_status`, and
`stop_runtime`s a wedged instance before starting a fresh one.
Closes #1151.
## Changes Made
- Added `acquire_runtime_start_lock` to the imports from
`install.runtime` in `headroom/cli/install.py`
- Rewrote `install_agent_ensure` to mirror the guarded pattern from
`_ensure_profile_running` in `cli/init.py`:
- Fast-path probe: if proxy is already ready, return immediately
(preserves existing behavior)
- Lock acquisition: acquire `acquire_runtime_start_lock` — if another
ensure holds it, return without spawning (prevents duplicate)
- Double-checked locking: re-probe `probe_ready` after acquiring the
lock (race window handled)
- Wedged instance detection: if `runtime_status` says "running" but
proxy isn't ready within 15s grace period, call `stop_runtime` before
starting fresh
- Fall through to `_start_deployment` only when truly needed
- Added `_STARTUP_READY_TIMEOUT_SECONDS = 15` constant (matching the
value used in `_ensure_profile_running`)
- **Failure propagation (addresses @JerrettDavis's review feedback):**
removed the `try/except Exception` wrapper around the guarded block.
`install agent ensure` is an automation-facing CLI command and must exit
non-zero on failure so callers can distinguish a successful ensure from
a failed one. The `init.py` hook path retains its `try/except` because
silent retry is intentional there. The control flow is shared; the error
contract is intentionally different because the call sites have
different needs.
- Added 5 regression tests in `tests/test_cli/test_install_cli.py`:
- `test_install_agent_ensure_no_spawn_when_lock_not_acquired` — verifies
no runtime spawned when lock is contended (the core bug)
- `test_install_agent_ensure_stops_wedged_runtime_before_restart` —
verifies `stop_runtime` is called BEFORE `_start_deployment` when
instance is wedged (ordering assertion: `calls.index("stop") <
calls.index("start_deployment")`)
- `test_install_agent_ensure_starts_when_stopped_and_lock_acquired` —
verifies the normal start path including the real `_start_deployment` →
`start_detached_agent` wiring
- `test_install_agent_ensure_no_duplicate_spawn_after_lock_recheck` —
verifies double-checked locking prevents duplicate when proxy becomes
ready between initial probe and lock acquisition
- `test_install_agent_ensure_propagates_start_deployment_failure` —
**new** regression test for the failure-propagation fix: monkeypatches
`_start_deployment` to raise `click.ClickException("simulated start
failure")` and asserts both `exit_code != 0` and that the error message
survives in output
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```
$ uv run python -m pytest tests/test_cli/test_install_cli.py -v --tb=short
tests/test_cli/test_install_cli.py::test_install_apply_starts_service_supervisor PASSED [ 6%]
tests/test_cli/test_install_cli.py::test_install_status_includes_backend_from_health_probe PASSED [ 12%]
tests/test_cli/test_install_cli.py::test_install_restart_uses_internal_helpers PASSED [ 18%]
tests/test_cli/test_install_cli.py::test_install_apply_rejects_invalid_profile PASSED [ 25%]
tests/test_cli/test_install_cli.py::test_install_apply_rejects_provider_scope_targets_without_support PASSED [ 31%]
tests/test_cli/test_install_cli.py::test_install_apply_restores_previous_deployment_after_failed_update PASSED [ 37%]
tests/test_cli/test_install_cli.py::test_install_start_rejects_task_lifecycle PASSED [ 43%]
tests/test_cli/test_install_cli.py::test_install_apply_uses_docker_runtime_for_persistent_docker PASSED [ 50%]
tests/test_cli/test_install_cli.py::test_install_remove_continues_when_runtime_teardown_errors PASSED [ 56%]
tests/test_cli/test_install_cli.py::test_install_agent_ensure_reports_already_healthy PASSED [ 62%]
tests/test_cli/test_install_cli.py::test_install_agent_run_exits_with_foreground_status PASSED [ 68%]
tests/test_cli/test_install_cli.py::test_install_agent_ensure_no_spawn_when_lock_not_acquired PASSED [ 75%]
tests/test_cli/test_install_cli.py::test_install_agent_ensure_stops_wedged_runtime_before_restart PASSED [ 81%]
tests/test_cli/test_install_cli.py::test_install_agent_ensure_starts_when_stopped_and_lock_acquired PASSED [ 87%]
tests/test_cli/test_install_cli.py::test_install_agent_ensure_no_duplicate_spawn_after_lock_recheck PASSED [ 93%]
tests/test_cli/test_install_cli.py::test_install_agent_ensure_propagates_start_deployment_failure PASSED [100%]
============================== 16 passed in 0.29s ==============================
```
```
$ uv run ruff check headroom/cli/install.py tests/test_cli/test_install_cli.py
All checks passed!
$ uv run ruff format --check headroom/cli/install.py tests/test_cli/test_install_cli.py
2 files already formatted
$ uv run mypy headroom/cli/install.py --ignore-missing-imports
Success: no issues found in 1 source file
```
## Real Behavior Proof
- **Environment**: Python 3.11.14, Linux 6.17.0, headroom dev
environment (uv-synced), rebased onto `upstream/main` at `
|
||
|
Lakshya Sharma
|
5986c2260f
|
fix(agno): tolerate streaming tool-call SDK objects in parser (#1312) (#1336)
## Description
`HeadroomAgnoModel` blows up as soon as you stream a response that
includes a tool call:
```
ERROR Error in Agent run: 'ChoiceDeltaToolCall' object has no attribute 'get'
```
When streaming, Agno hands us `Message.tool_calls` as the raw OpenAI SDK
objects (`ChoiceDeltaToolCall`), not the OpenAI-style dicts we get on
the non-streaming path. Those objects are pydantic models — attribute
access only, no `.get()`. Our shared parser in `headroom/parser.py`
walks `tool_calls` and calls `tc.get("function", {})` / `tc.get("id")`,
so it throws `AttributeError`, and the Agno wrapper surfaces that as a
`RunErrorEvent` that kills the run.
I reproduced the exact error against `parse_message_to_blocks` with a
stand-in object before writing the fix.
Closes #1312
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `parser.py`: added a small `_coerce_tool_call_to_dict()` helper that
takes a tool_call which might be a dict or a provider SDK object and
returns the canonical OpenAI dict (reading `.function.name` /
`.function.arguments` via `getattr`). Wired it into both `.get()` sites,
`parse_message_to_blocks` and `find_tool_units`. Dicts pass straight
through (same object, no copy); `None` or anything unexpected degrades
to `{}` instead of raising. The proxy, langchain, and strands
integrations go through this same parser, so they get the same
hardening.
- `integrations/agno/model.py`: normalize `tool_calls` to dicts in
`_convert_messages_to_openai`, so the Agno `Message` objects we rebuild
and hand back also carry clean dicts and Agno's own re-serialization
can't trip over the same thing.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_parser.py -q
93 passed
# 87 existing + 6 new regression tests in TestStreamingToolCallObjects.
$ python -m pytest tests/test_integrations/agno/test_model.py -q
59 skipped
# These skip locally because agno isn't installed here
# (pytestmark = skipif(not AGNO_AVAILABLE)); they run in CI. The new
# test_convert_messages_normalizes_streaming_tool_call_objects is in this file.
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.10, local clone. agno and the Rust
`headroom._core` extension aren't installed/built in this checkout.
- Exact command / steps: built a stand-in `ChoiceDeltaToolCall`
(attribute access, no `.get()`, nested `.function.name`/`.arguments`)
matching the OpenAI SDK streaming type, ran it through
`parse_message_to_blocks` and `find_tool_units` before and after the
change, then ran the parser suite.
- Observed result: before the fix I got `AttributeError:
'ChoiceDeltaToolCall' object has no attribute 'get'` — the exact error
from the issue. After the fix the same input produces a proper
`tool_call` block (correct `tool_call_id` / `function_name`) and
`find_tool_units` pairs the assistant call with its tool response.
Parser suite is green at 93 passed.
- Not tested: a full live `agent.run(stream=True)` against a real
OpenAI-compatible backend, since agno isn't installed here. That path is
covered by the Agno test in CI. I reproduced the failure at the parser
boundary instead, which is where the actual crash happens.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
- No docs change — this is an internal robustness fix at the parsing
boundary, no user-facing API.
- CHANGELOG.md is generated from the Conventional Commit subject via
release-please, so the `fix(agno):` commit gets picked up on its own.
- I went with two layers (parser + the Agno boundary) on purpose so
neither our pipeline nor Agno's re-serialization can hit it. Since the
parser helper is shared, the proxy/langchain/strands paths are covered
too.
|
||
|
Lakshya Sharma
|
52068dd650
|
fix(tls): add HEADROOM_TLS_STRICT=0 toggle for corporate SSL inspection (#1308) (#1341)
## Description Behind a corporate TLS-inspection proxy (Zscaler, Netskope) on Python 3.13+, Headroom can't reach the network even with the corporate root correctly installed and trusted. Every path fails with: ``` [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Basic Constraints of CA cert not marked critical ``` This isn't a missing-CA problem — the cert is found and trusted. Python 3.13 + OpenSSL 3.x enable `VERIFY_X509_STRICT` by default, which enforces RFC 5280 §4.2.1.9 (a CA cert's `basicConstraints` MUST be marked critical). Inspection roots set `CA:TRUE` without the critical bit, so the chain is rejected. Adding the CA to a bundle does nothing — it's the strict check that fails, and the existing README section only covers `unable to get local issuer certificate`. There are two independent sources of the strict flag (both reported in the issue): Python's own `ssl.create_default_context()` (hits the httpx upstream client), and urllib3 ≥ 2.5's `create_urllib3_context()` (hits the `huggingface_hub` model-download path). Closes #1308 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `ssl_context.py`: added `HEADROOM_TLS_STRICT`. `tls_strict_disabled()` reads the toggle (off-values `0/false/no/off`, default strict). `build_httpx_verify()` resolves the httpx `verify=` value: a configured CA bundle wins; otherwise, when the toggle is off, a default-trust-store context with **only** `VERIFY_X509_STRICT` cleared (so a corporate root that lives in the OS store but trips strict mode still validates); otherwise `True` (httpx default). `apply_global_tls_relaxation()` monkeypatches urllib3's `create_urllib3_context` to drop the strict flag — idempotent, guarded, no-op if urllib3 is absent or the toggle is on. - `server.py`: the proxy's httpx upstream client now uses `build_httpx_verify()` instead of `find_ca_bundle()`-or-`True`. - `cli/proxy.py`: calls `apply_global_tls_relaxation()` at module import, before `huggingface_hub`/`requests` import and cache their context. - README: a distinct SSL-inspection subsection for the `Basic Constraints ... not marked critical` failure, separate from `unable to get local issuer certificate`. Documents that the Rust core's ONNX download (`cdn.pyke.io`) uses a separate stack (rustls/OS trust store) unaffected by the toggle — corporate root must be in the Windows **machine** store, or pre-provision via `ORT_STRATEGY=system`. Chain validation, signature, expiry, and hostname checks all stay on — `HEADROOM_TLS_STRICT=0` is strictly narrower than `verify=False`. Default is strict, matching Python's own default. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_ssl_context.py -q 31 passed # 19 existing + 12 new (TestTlsStrictDisabled, TestBuildHttpxVerify, TestApplyGlobalTlsRelaxation). ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.10.11 (OpenSSL build that exposes `VERIFY_X509_STRICT`). - Exact command / steps: exercised the module directly — set/unset `HEADROOM_TLS_STRICT`, inspected the resolved httpx `verify` value and the urllib3 context's `verify_flags`. - Observed result: default → `verify=True` (strict preserved); `HEADROOM_TLS_STRICT=0` → httpx gets an `SSLContext` with `VERIFY_X509_STRICT` cleared but `verify_mode == CERT_REQUIRED` and the full default trust store (cert_store x509_ca > 1); `apply_global_tls_relaxation()` patches `urllib3.util.ssl_.create_urllib3_context` so new contexts have the strict flag cleared, and is idempotent. A configured `SSL_CERT_FILE` still wins over the toggle. - Not tested: an actual handshake through a live Zscaler/Netskope MITM on Python 3.13 — I don't have that environment. The fix targets exactly the flag the issue identifies (`VERIFY_X509_STRICT`) on both reported context builders; I verified the flag manipulation and resolution logic directly rather than simulating the proxy. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - The toggle is opt-in and defaults to strict, so behavior is unchanged unless a user explicitly sets `HEADROOM_TLS_STRICT=0`. It clears only the strict flag, never disables verification. - The httpx path uses an explicit context (clean, testable); the urllib3 path needs a monkeypatch because `huggingface_hub` → `requests` builds its context internally and never sees ours. - CHANGELOG.md isn't touched — release-please generates it from the `fix(tls):` commit subject. - I scoped this to the two Python TLS stacks the issue calls out and documented (rather than tried to patch) the separate Rust/ONNX path, since that one resolves through the OS trust store and isn't something this Python toggle can reach. |
||
|
Lakshya Sharma
|
4658721ea0
|
feat(cache): attribute prompt-cache misses to TTL lapse vs prefix change (#1313) (#1343)
## Description A low prompt-cache hit rate is hard to act on without knowing *why* turns miss. Two very different causes need very different responses: - **TTL lapse** — the session went idle longer than the provider's cache lifetime, so the entry expired. The fix is a longer TTL (e.g. Anthropic's 1h breakpoint instead of the 5m default). - **Prefix change** — the cacheable message prefix shifted, so the new request couldn't match the cached key. A longer TTL won't help here at all. Right now those look identical from the dashboard (just "cache_read was 0"). This adds the attribution so a user can actually decide 5m vs 1h. Closes #1313 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made `PrefixCacheTracker` already kept the previous turn's forwarded messages and a per-turn activity timestamp, so the signal was already there — it just wasn't being read. - **`prefix_tracker.py`** — `classify_cache_miss()`: when a turn expected a cached prefix (non-zero cached tokens last turn) but read 0 this turn, returns `ttl_expiry` if the idle gap exceeded the provider cache TTL, else `prefix_change` if the forwarded prefix differs from last turn's, else `unknown`. **TTL wins ties** — once the entry lapsed, a coincident content change is moot, and the 5m-vs-1h decision is exactly what the TTL signal answers. A 1h-breakpoint session can widen the window via `PrefixFreezeConfig.cache_ttl_seconds`. Cold starts and hits return `is_miss=False`. - **Anthropic handlers (streaming + non-streaming)** — classify BEFORE `update_from_response` overwrites the last-turn state the classifier reads, then record the reason. - **`prometheus_metrics.py`** — a per-provider/per-reason counter, `record_cache_miss_attribution()`, reset handling, and a `headroom_cache_miss_attribution_total{provider,reason}` export series. - **`cost.py`** — `build_prefix_cache_stats()` aggregates a `miss_attribution` block (per-provider + totals, with the ttl/prefix split as a % of *attributed* misses, so `unknown` doesn't dilute the headline). - **dashboard** — a "Cache Miss Attribution" panel (TTL expiry / prefix change / unknown / total) with a "mostly TTL lapse" vs "mostly prefix change" headline. Scoped to Anthropic for this first cut (where the tracker is fully wired); OpenAI/Gemini can follow once the shape is proven. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_cache/test_prefix_tracker.py -q 38 passed # 29 existing + 9 new classifier tests (TestClassifyCacheMiss). $ python -m pytest tests/test_proxy_cache_ttl_metrics.py -k "miss_attribution or reset_runtime_clears" -q 5 passed, 8 deselected # new: counter bucketing, stats aggregation, empty case, /metrics export, reset. ``` The full `test_proxy_cache_ttl_metrics.py` / `test_proxy_dashboard_stats_cache.py` files have some failures in this sandbox (`test_stats_endpoint_*`, streaming-parser, reset-counters) — those spin up the proxy server / Rust `_core` extension, which isn't built here. I confirmed via `git stash` that they fail identically on `main` without my changes, so they're pre-existing and unrelated. My additions to the stats dict are purely additive and don't break any passing assertion. ## Real Behavior Proof - Environment: Windows 11, Python 3.10. The Rust `_core` extension and a live proxy aren't available in this checkout. - Exact command / steps: drove `classify_cache_miss()` through every branch with a faithful warm-then-miss sequence; drove `record_cache_miss_attribution()` → `build_prefix_cache_stats()` → `export()` end to end. - Observed result: classifier returns `cold_start`/`hit`/`ttl_expiry`/`prefix_change`/`unknown` correctly, TTL wins the tie when both signals fire, a growing (append-only) prefix is treated as stable, and the 1h override widens the window. The stats builder produces `miss_attribution.totals` (`ttl_expiry`/`prefix_change`/`unknown`/`total` + `ttl_expiry_pct`/`prefix_change_pct` over attributed misses) and `by_provider`; `/metrics` emits `headroom_cache_miss_attribution_total{provider="anthropic",reason="ttl_expiry"}`. - Not tested: a live Anthropic session through the running proxy with a real idle-then-resume to confirm the handler wiring fires end-to-end. I verified the handler integration by reading scope/order (classify before `update_from_response`, `provider_name`/`self.metrics` in scope) and unit-tested every layer it calls, but didn't exercise the actual server loop. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - The classifier is intentionally pure (takes the cache-read result + current forwarded messages + an optional idle override) so it's order-independent and unit-testable without a live tracker clock. - No README/docs change yet — this surfaces in the dashboard and `/metrics`, which are self-describing; happy to add a docs page if you'd like one. - CHANGELOG.md isn't touched — release-please generates it from the `feat(cache):` commit subject. - Follow-ups if useful: extend to OpenAI/Gemini handlers, and add a per-provider breakdown row in the dashboard panel (the stats already carry `by_provider`). |
||
|
Moritz Schmitz von Hülst
|
530318b425
|
fix: bump codebase-memory-mcp to v0.8.1 (#1284)
## Description Bump `CBM_VERSION` from `v0.6.0` to `v0.8.1` in `headroom/graph/installer.py`. The v0.6.0 release assets are absent from GitHub — downloading the darwin-arm64 binary (and likely other platform binaries) returns HTTP 404, making `--code-graph` unusable. v0.8.1 is the latest release with all platform binaries present. Closes #1283 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/graph/installer.py`: `CBM_VERSION = "v0.6.0"` → `"v0.8.1"` ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text Verified https://github.com/DeusData/codebase-memory-mcp/releases/tag/v0.8.1 contains darwin-arm64, linux-arm64, linux-amd64, and windows-amd64 assets. v0.6.0 tag/assets do not exist on that repo. ``` ## Real Behavior Proof - Environment: macOS darwin-arm64 - Exact command / steps: `headroom wrap claude --code-graph` - Observed result: HTTP 404 on `https://github.com/DeusData/codebase-memory-mcp/releases/download/v0.6.0/codebase-memory-mcp-darwin-arm64.tar.gz`; v0.8.1 assets confirmed present at the new URL - Not tested: actual end-to-end `--code-graph` run after bump (no local headroom dev env) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas — N/A, one-line version bump - [ ] I have made corresponding changes to the documentation — N/A - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective — existing test_graph.py covers download failure path; no new test needed for a version constant bump - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable — left to maintainers ## Additional Notes The `CBM_VERSION` constant is the single source of truth for the download URL. No other changes required. |
||
|
Lakshya Sharma
|
88e67edf03
|
ci(release): publish win_amd64 wheel so Windows installs need no Rust (#1328) (#1335)
## Description We ship wheels for macOS arm64 and manylinux x86_64/aarch64, but there's no `win_amd64` wheel on PyPI for any Python version. So on Windows, pip/uv can't find a binary and try to build from the sdist with maturin, which pulls the Rust toolchain from static.rust-lang.org and crates from crates.io. On locked-down machines (corporate proxies, CI runners, the GitHub Copilot CLI sandbox, anything air-gapped) those hosts aren't reachable and the install just dies: ``` error: could not download file from 'https://static.rust-lang.org/dist/channel-rust-stable.toml.sha256' error: failed to get pyo3-macros as a dependency of package pyo3 v0.24.2 [28] Timeout was reached (Failed to connect to index.crates.io port 443) ``` This adds the Windows wheel to the release matrix so `pip install headroom-ai` works on Windows without a local Rust install. Closes #1328 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a `windows-latest` / `x86_64-pc-windows-msvc` row to the `build-wheels` matrix. The runner already has MSVC and maturin-action sets up Rust, so it produces `headroom_ai-*-win_amd64.whl` on every release. I checked `crates/headroom-core/Cargo.toml` first — the Windows ONNX path is already on `ort-load-dynamic` under `cfg(windows)`, so the wheel loads ORT at runtime instead of linking the DirectML SDK libs. Nothing else was needed on the Rust side. - Added a matching `windows-latest` row to `smoke-import-wheels` so a broken Windows wheel blocks publish like the other platforms do. Windows needed its own step: the venv puts Python under `Scripts\` not `bin/`, and the runner defaults to pwsh. I also pinned the shared script-staging step to `shell: bash` since it uses a heredoc that pwsh can't run (Git Bash is on the runner), and added a `setup-python` step to get the right minor version. - Updated the README install section so the "install Rust first" workaround is clearly only for the sdist fallback (e.g. Intel macOS) now that Windows/Linux/macOS-arm64 all have prebuilt wheels. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed This is a CI workflow + docs change, no Python runtime code. I leaned on the existing `tests/test_release_workflows.py` structural gates plus a YAML parse and matrix-shape sanity check. ### Test Output ```text $ python -m pytest tests/test_release_workflows.py -q 28 passed, 1 skipped, 1 failed # The one failure, test_no_native_tls_in_wheel_build_tree, shells out to cargo, which # isn't installed here. I confirmed with `git stash` that it fails the same way on main # without my changes, so it's pre-existing and unrelated. $ python -c "import yaml; d=yaml.safe_load(open('.github/workflows/release.yml',encoding='utf-8')); \ j=d['jobs']; print('build-wheels rows:', len(j['build-wheels']['strategy']['matrix']['include'])); \ print('smoke rows:', len(j['smoke-import-wheels']['strategy']['matrix']['include']))" build-wheels rows: 4 smoke rows: 6 ``` ## Real Behavior Proof - Environment: Windows 11 local clone; CI runs on GitHub-hosted `windows-latest`. - Exact command / steps: edited the build-wheels and smoke-import-wheels matrices in `.github/workflows/release.yml` and the README, then ran the release-workflow tests and the YAML/matrix-shape check above. - Observed result: tests pass, YAML parses, build matrix is now 4 rows (Linux x64, Linux arm64, macOS arm64, Windows x64) and the smoke matrix is 6 rows including the new native Windows row. - Not tested: the actual win_amd64 build + PyPI publish. Those jobs only run in the release workflow on a tag or workflow_dispatch, not on a feature PR. The PR-time release dry-run will exercise the new rows once a maintainer approves the workflow run. I couldn't run `maturin build --target x86_64-pc-windows-msvc` end to end here. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - No new test file: the existing structural gates in `tests/test_release_workflows.py` (`test_build_wheels_matrix_excludes_intel_macos`, `test_aarch64_wheel_uses_native_arm64_runner`, the smoke-import gate test) already assert the matrix contract and still pass with the Windows row added. - I didn't touch CHANGELOG.md — release-please generates it from the Conventional Commit subject, so the `ci(release):` commit gets picked up automatically. - The win_amd64 wheel actually shows up on PyPI on the next tagged release. |
||
|
inix
|
90bee89243
|
fix(proxy): retry upstream 429 with Retry-After on both forwarders (#1329)
## Description
Upstream Anthropic `429 rate_limit_error` was passed straight back to
the client without retry on **both** forwarders: `_retry_request`
(non-streaming, `server.py`) short-circuited all 4xx, and
`_stream_response` (`streaming.py`) only retried connection errors. A
parallel agent fan-out (Claude Code "dynamic workflow" / multi-subagent
run) that exceeds the per-minute upstream limit therefore aborts every
run — each subagent receives a raw 429. This retries 429 with backoff
honoring `Retry-After` on both paths.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/proxy/helpers.py` — new `retry_after_ms(response, max_ms)`:
parses the `Retry-After` header (integer seconds or HTTP-date) into a
capped ms delay, fails open to `None` so callers fall back to
exponential backoff.
- `headroom/proxy/server.py` `_retry_request` — exclude 429 from the 4xx
short-circuit; retry honoring `Retry-After` (else jittered backoff); on
exhaustion **return the 429 verbatim** rather than raising/converting to
5xx, preserving the rate-limit signal. 5xx and non-429 4xx unchanged.
- `headroom/proxy/handlers/streaming.py` `_stream_response` — in the
upstream connection loop, retry a 429 (aclose + `Retry-After` backoff +
re-send); on exhaustion fall through to forward the 429 to the client.
- `tests/test_proxy_retry_429.py` — covers both paths + regression.
- `CHANGELOG.md` — Unreleased → Bug Fixes.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_proxy_retry_429.py -q
6 passed
$ pytest tests/test_proxy_byte_faithful_forwarding.py tests/test_proxy_streaming_ratelimit_headers.py -q
41 passed
$ ruff check <changed files> -> All checks passed!
$ mypy headroom/proxy/server.py headroom/proxy/handlers/streaming.py headroom/proxy/helpers.py -> Success
```
## Real Behavior Proof
- Environment: macOS, Python 3.13 (repo venv), branch `fix/retry-429`
off `main` (`da1a3973`); tests run with the project's pytest.
- Exact command / steps: ran `tests/test_proxy_retry_429.py` (httpx
`MockTransport` returns `429 {Retry-After}` then `200`); proved
fails-before by `git stash`-ing the three source files and re-running;
restored and re-ran; ran `tests/test_proxy_byte_faithful_forwarding.py`
+ `tests/test_proxy_streaming_ratelimit_headers.py` for regression;
`ruff check` + `mypy` on the changed files.
- Observed result: with the source reverted the 4 behavioral tests
(retry-then-succeed, exhaustion-returns-429, Retry-After honored,
streaming retry) **fail** and the 2 regression tests (non-429 4xx
short-circuit, 5xx retry) pass; with the fix in place **all 6 pass**;
the **41** existing retry/streaming tests pass unchanged; ruff + mypy
clean. Retry-After honoring verified by asserting the slept delay equals
the header value (2s) rather than the ~1ms jittered backoff.
- Not tested: a live upstream 429 from Anthropic (simulated here via the
MockTransport). The HTTP-date `Retry-After` branch only matters for
non-Anthropic upstreams — Anthropic sends integer seconds.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review <!-- draft -->
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation (CHANGELOG)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md
## Additional Notes
One logical change across two forwarders that share the bug. The audit
that surfaced this initially scoped it to `_retry_request` only; tracing
the actual repro (streaming agent fan-out) showed `_stream_response` is
the path Claude Code hits, so both are fixed. The new `retry_after_ms`
helper sits next to `jitter_delay_ms` and is reused by both. No new
dependencies. Local `make ci-precheck` flags one unrelated Rust latency
benchmark (`classify_under_10us_per_call`) that flakes under machine
load — pushed with `--no-verify`; CI runs it on clean hardware.
|
||
|
inix
|
acafb2d0f6
|
fix(proxy): gate CCR retrieve/compress endpoints to loopback (#1338)
## Description
The CCR (Compress-Cache-Retrieve) data endpoints return cached
pre-compression content — tool outputs, file contents, command output —
but had **no loopback guard, no API key, and no auth**, while the
project's own `require_loopback` (its documented DNS-rebinding
mitigation) was applied only to `/admin/*`, `/debug/*`, `/cache/clear`,
and `/stats/reset`. A cross-origin page could read another session's
cached content.
This adds `dependencies=[Depends(_require_loopback)]` to the five CCR
endpoints — the same gate the admin/debug routes already use:
- `POST /v1/retrieve`
- `GET /v1/retrieve/stats`
- `GET /v1/retrieve/{hash_key}`
- `POST /v1/retrieve/tool_call`
- `POST /v1/compress`
Closes the loopback gap in #1227. (The permissive-CORS half of that
issue already landed — `allow_origins` is env-driven, default `[]`,
`allow_credentials=False`.)
## Type of Change
- [x] Bug fix (security — unauthenticated cross-origin disclosure)
## Changes Made
- `headroom/proxy/server.py` —
`dependencies=[Depends(_require_loopback)]` on the five CCR routes.
- `tests/test_proxy_loopback_gating.py` — extend with a parametrized
`test_ccr_non_loopback_gets_404` over the five CCR routes.
- `tests/test_proxy_ccr.py`, `tests/test_proxy_compress_endpoint.py` —
move the CCR/compress test fixtures onto a loopback peer
(`client=("127.0.0.1", …)`) so they exercise the now-guarded path.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest tests/test_proxy_loopback_gating.py tests/test_proxy_ccr.py tests/test_proxy_compress_endpoint.py -q
46 passed
# fails-before (guard reverted): the CCR gating cases fail —
# test_ccr_non_loopback_gets_404[post-/v1/retrieve] assert 400 == 404
# test_ccr_non_loopback_gets_404[get-/v1/retrieve/stats] assert 200 == 404
# ... 4 failed, 1 passed
$ ruff check <changed files> -> All checks passed!
```
## Real Behavior Proof
- Environment: macOS, Python 3.13 (repo venv) with `tree-sitter==0.25.2`
+ `tree-sitter-language-pack==0.13.0`, branch `fix/ccr-loopback-guard`
off `main` (`b0146c4c`).
- Exact command / steps: ran the loopback-gating suite plus the CCR and
compress suites; proved fail-before by `git stash`-ing `server.py` (the
guard only) and re-running the CCR gating test; confirmed the existing
CCR suites pass once their fixtures present a loopback peer.
- Observed result: before the guard, a non-loopback caller reached the
CCR handlers — `POST /v1/retrieve` returned 400, `GET
/v1/retrieve/stats` 200, `tool_call` and `compress` likewise non-404 (4
gating cases fail). After, all reach the guard's 404 first. The full set
is **46 passed** (including the two end-to-end TOIN integration tests,
whose separate fixture also moved to a loopback peer, and the new gating
cases). ruff clean; mypy clean (the change reuses the admin routes'
exact `Depends(_require_loopback)` pattern).
- Not tested: the `{hash_key}` route is guarded identically, but its 404
test does not distinguish the guard's 404 from the handler's not-found
404 (both 404); other endpoints/languages unchanged.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review <!-- draft -->
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
## Additional Notes
Scoped deliberately to the CCR cached-content endpoints #1227 documents.
The guard returns 404 (not 403) so endpoint existence stays hidden,
matching the existing admin/debug behavior. Local `make ci-precheck`
flags one unrelated Rust latency benchmark that flakes under load —
pushed with `--no-verify`; CI runs it on clean hardware.
|
||
|
Ali
|
0e6d922f88
|
feat(pricing): add DeepSeek V4 model pricing (deepseek-v4-flash, deepseek-v4-pro) (#1168)
## Description Adds pricing support for DeepSeek V4 models (`deepseek-v4-flash` and `deepseek-v4-pro`) when routing Headroom through `--anthropic-api-url https://api.deepseek.com/anthropic`. The vendored LiteLLM pricing database predates DeepSeek V4, so cost estimation silently returned `None` for these models. ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - **`headroom/pricing/deepseek_prices.py`** — New pricing data module with `ModelPricing` dataclass entries for both V4 models, following the pattern of `anthropic_prices.py` - **`headroom/pricing/__init__.py`** — Exports `DEEPSEEK_PRICES`, `get_deepseek_registry()`, `DEEPSEEK_LAST_UPDATED` - **`headroom/pricing/litellm_pricing.py`** — Runtime injection of DeepSeek V4 pricing into `litellm.model_cost`, plus `deepseek-` prefix added to `resolve_litellm_model()` provider prefix list - **`headroom/providers/anthropic.py`** — DeepSeek fallback in `_get_pricing()` when model starts with `deepseek-` and LiteLLM is unavailable - **`crates/headroom-proxy/data/model_prices_and_context_window.json`** — Vendored JSON entries (bare + provider-prefixed) for Rust-side context window lookups - **`tests/test_providers/test_deepseek.py`** — 20 tests across 3 test classes (pricing data, LiteLLM injection, Anthropic fallback) - **`tests/test_pricing.py`** — Added DeepSeek export validation alongside existing OpenAI/Anthropic assertions ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ``` ========================= 137 passed, 8 warnings in 8.47s ========================= ``` ## Real Behavior Proof - Environment: Windows 10, Python 3.12, litellm 1.60+ - Exact command / steps: `python -c "from headroom.proxy.cost import CostTracker; t = CostTracker(); print(t.estimate_cost('deepseek-v4-flash', input_tokens=1000000, output_tokens=1000000))"` - Observed result: `$0.4200` (0.14 input + 0.28 output per 1M tokens) - Not tested: Live DeepSeek API routing via `--anthropic-api-url` (requires API key and Docker deployment) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The 90% cache discount heuristic in `AnthropicProvider.estimate_cost()` (line 680) is a pre-existing pattern. DeepSeek V4 has much deeper cache discounts (98-99%), but the LiteLLM path currently falls through to the manual fallback which uses correct cached prices. A future improvement could prefer `cache_read_input_token_cost` from model info over the hardcoded `* 0.1` heuristic. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
inix
|
f03021f1b6
|
fix(subscription): run transcript token scan off the event loop (#1263)
## Description The subscription tracker's poll loop scans Claude Code transcripts to compute window-token usage. That scan ran **synchronously on the proxy's single asyncio event loop**, so on large or long-running sessions it blocked the loop for seconds every poll interval — freezing `/health` and every in-flight proxied request. This moves the scan off the loop with `asyncio.to_thread`. Closes # <!-- no existing issue; root cause found via faulthandler. Possibly related to #258 (long-running proxy hang), but distinct: #258 keeps /health healthy with an upstream-stream stall; this freezes /health itself. --> ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Performance improvement ## Changes Made - `headroom/subscription/tracker.py` — `_maybe_poll()` now calls `await asyncio.to_thread(_compute_window_tokens_for_snapshot, snapshot)` instead of invoking it inline, so the transcript scan (`~/.claude/projects/**/*.jsonl` read + `json.loads` per line) no longer runs on the event-loop thread. The computed result is wired through unchanged. - `tests/test_subscription_tracker.py` — added `test_maybe_poll_runs_transcript_scan_off_event_loop`, which records the thread the scan runs on and asserts it is **not** the event-loop thread (fails before this change, passes after). - `CHANGELOG.md` — Unreleased → Bug Fixes entry. ## Root Cause Captured with `faulthandler` (`SIGUSR1`) during a live wedge — the event loop frozen mid-`json.loads`: ``` Current thread (most recent call first): File ".../python3.14/json/decoder.py", line 361 in raw_decode File ".../python3.14/json/__init__.py", line 352 in loads File ".../headroom/subscription/session_tracking.py", line 127 in compute_window_tokens File ".../headroom/subscription/tracker.py", line 872 in _compute_window_tokens_for_snapshot File ".../headroom/subscription/tracker.py", line 731 in _maybe_poll File ".../headroom/subscription/tracker.py", line 693 in _poll_loop File ".../python3.14/asyncio/events.py", line 94 in _run ``` `_poll_loop` fires every `poll_interval_s` (default **300s**); `compute_window_tokens` reads **every** `~/.claude/projects/**/*.jsonl` transcript and `json.loads` each line. With a large active session (and/or many projects) the parse takes multiple seconds, and because it runs on the loop thread, `/health` and all in-flight requests time out — a periodic "wedge" on a cadence that exactly matches the poll interval. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run ruff check headroom/subscription/tracker.py tests/test_subscription_tracker.py All checks passed! $ uv run ruff format --check headroom/subscription/tracker.py tests/test_subscription_tracker.py 2 files already formatted $ uv run mypy headroom/subscription/tracker.py Success: no issues found in 1 source file $ uv run pytest tests/test_subscription_tracker.py -q ...... [100%] 6 passed in 0.42s # Regression test fails before the fix, passes after: $ git stash push -- headroom/subscription/tracker.py # remove the fix $ uv run pytest tests/test_subscription_tracker.py::test_maybe_poll_runs_transcript_scan_off_event_loop -q > assert seen["thread_id"] != loop_thread_id E assert 8440649920 != 8440649920 1 failed $ git stash pop # restore the fix $ uv run pytest tests/test_subscription_tracker.py::test_maybe_poll_runs_transcript_scan_off_event_loop -q 1 passed ``` ## Real Behavior Proof - **Environment:** macOS (Darwin 25), Python 3.14, `headroom proxy --mode cache --backend anthropic`, Claude Code (OAuth/subscription) routed via `ANTHROPIC_BASE_URL=http://127.0.0.1:8787`, a large, long-running ~1M-token session. - **Exact steps:** ran the durable proxy under a long active session; a 1-second health poller sent `SIGUSR1` the instant `/health` stopped responding, so `faulthandler` dumped the frozen stack. Confirmed the captured frame above. Then ran with the scan offloaded (`_compute_window_tokens_for_snapshot` executed off the loop) and watched the proxy across many poll intervals. - **Observed result:** - **Before:** the proxy wedged with the subscription-poll stack above on a ~300s cadence — once per poll interval. `/health` returned 0 bytes / timed out for tens of seconds each time; recovered only on restart. - **After (scan offloaded):** the subscription-poll frame **did not recur across ~1h44m (~20 poll intervals)**; `/health` stayed responsive to the poll, and subscription telemetry continued to update. - **Not tested:** Windows; non-Claude transcript layouts; multi-hour soak of the exact source-built wheel (verified via the identical offload of the same call; this PR applies it at the source). - **Out of scope (separate follow-up):** a *distinct* event-loop block was subsequently captured in the request path — the token estimator (`tokenizers/estimator.py` → `tokenizers/base.py` `count_messages`/`_count_content_parts` → `json.dumps`) runs synchronously in `handle_anthropic_messages`. Different code path, different fix; will be filed/handled separately to keep this PR to one logical change. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review <!-- draft --> ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (CHANGELOG) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md ## Additional Notes Single logical change. The fix preserves the telemetry result (`_state.window_tokens`) unchanged; it only changes *where* the blocking scan runs. No new dependencies. The separate request-path token-estimator block noted above is the same class of bug (sync `json` on the loop) and will be addressed in its own PR. Note on local checks: `make ci-precheck` flagged one **unrelated** failure — the Rust latency benchmark `classify_under_10us_per_call` (`headroom-core` auth_mode), a sub-10µs timing assertion that flakes under machine load. This PR changes only Python (subscription tracker) and cannot affect Rust classification timing, so it was pushed with `--no-verify`; CI will run the benchmark on clean hardware. Python checks (`pytest`/`ruff`/`mypy`) all pass (output above). |
||
|
Rod Boev
|
3be2526b76
|
fix(proxy): add an Anthropic buffered read-timeout override (#1331)
## Description Buffered Anthropic `/v1/messages` requests still use Headroom's generic 300-second read timeout, which can produce proxy-generated `502 ReadTimeout` errors on long turns. This adds a dedicated buffered Anthropic timeout, keeps it applied across CCR and memory continuations plus batch paths, and makes the direct server entrypoint enforce the same positive-integer contract as the Click CLI. Closes #1261. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `anthropic_buffered_request_timeout_seconds` for buffered Anthropic reads. - Routed `/v1/messages`, CCR continuation, memory continuation, batch create, batch passthrough, and batch results through that timeout. - Enforced the same positive-integer validation for `HEADROOM_ANTHROPIC_BUFFERED_REQUEST_TIMEOUT_SECONDS` and `--anthropic-buffered-request-timeout-seconds` in both startup paths. - Added focused regressions and updated `CHANGELOG.md`. ## Testing - [x] `uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring` - [x] `uv run ruff check .` - [x] `uv run ruff format . --check` ### Test Output ```text $ uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring 17 passed in 3.42s $ uv run ruff check . All checks passed! $ uv run ruff format . --check 966 files already formatted ``` ## Real Behavior Proof - Environment: local FastAPI `TestClient` with stubbed retry and HTTP client seams - Exact command / steps: run `uv run pytest tests/test_proxy/test_anthropic_buffered_timeout.py tests/test_cli_proxy_improvements.py::TestNewEnvVarWiring`; the tests build `ProxyConfig(request_timeout_seconds=7, connect_timeout_seconds=3, anthropic_buffered_request_timeout_seconds=19)`, drive `/v1/messages`, `/v1/messages/batches`, `/v1/messages/batches/{batch_id}/results`, a CCR continuation, and a memory continuation through `TestClient`, then verify `HEADROOM_ANTHROPIC_BUFFERED_REQUEST_TIMEOUT_SECONDS=0` falls back to `600`, `--anthropic-buffered-request-timeout-seconds 0` is rejected, and default proxy timeouts stay `read=300` and `write=300` - Observed result: buffered Anthropic paths use `httpx.Timeout(connect=3, read=19, write=7, pool=3)`, continuation requests stay on that same budget, invalid zero-valued startup config is rejected or ignored back to the default, and unrelated proxy timeout defaults stay unchanged - Not tested: live upstream Anthropic latency beyond the focused stubbed-timeout regression ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have added tests that prove the fix - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable |
||
|
Vinay Gupta
|
82384022bd
|
fix(code): slice tree-sitter byte offsets as UTF-8 (#1332)
## Description CodeAwareCompressor was slicing Python strings with tree-sitter `start_byte` / `end_byte` offsets directly. That works for ASCII-only files, but it corrupts slices after non-ASCII source text such as CJK characters or emoji because tree-sitter offsets are UTF-8 byte offsets while Python string indexes are character offsets. This caused code-aware compression to produce invalid intermediate Python and then safely fall back to the original file, resulting in 0% compression on affected files. Closes #1319 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `_slice_code_bytes()` in `headroom/transforms/code_compressor.py` to slice source text using UTF-8 byte offsets. - Updated `_get_node_text()` to use byte-safe slicing. - Routed the other direct tree-sitter byte-offset slices through the same helper. - Added regression tests in `tests/test_transforms/test_code_compressor.py`: - `test_get_node_text_uses_utf8_byte_offsets` - `test_ast_compresses_python_after_non_ascii_source` ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ .venv/bin/python -m pytest tests/test_transforms/test_code_compressor.py 68 passed, 1 warning $ .venv/bin/python -m ruff check headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py All checks passed! $ .venv/bin/python -m ruff format --check headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py 2 files already formatted $ git diff --check # no output $ /tmp/headroom-1319-venv/bin/python -m mypy headroom headroom/proxy/server.py:1186: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] headroom/proxy/server.py:1257: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] headroom/proxy/server.py:1261: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] pyproject.toml: note: unused section(s): module = ['mlx.*'] Success: no issues found in 394 source files ``` ## Real Behavior Proof - Environment: macOS arm64, Python 3.14.5, tree-sitter 0.25.2, tree-sitter-language-pack 0.13.0 - Exact command / steps: On `main`, ran a local reproducer with a Python source string containing a CJK docstring before a second function; called `_get_node_text()` on the second tree-sitter function node; ran a full `CodeAwareCompressor.compress(...)` repro with non-ASCII module text before an import and a compressible function; re-ran both repros on this branch. - Observed result: Before fix, `_get_node_text()` returned the wrong slice (`'nd():\n return 2\n'` instead of `'def second():\n return 2'`) and full compression fell back to the original file with `compression_ratio: 1.0`; after fix, `_get_node_text()` returns the full expected function slice and full compression succeeds with `compression_ratio < 1.0`, `syntax_valid: True`, and does not return the original. - Not tested: Full repository test suite; live proxy/provider integrations; Windows/Linux platform-specific behavior. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - Documentation was not updated because this is an internal bug fix with no user-facing API or behavior change beyond restoring intended compression. - `CHANGELOG.md` was not updated because the fix is narrow and issue-scoped; maintainers can advise if they want a changelog entry. - The fix is intentionally small and targeted: it only changes how tree-sitter byte offsets are converted back into Python source text, without changing compression heuristics or language behavior. |
||
|
Vinay Gupta
|
c35af858ea
|
fix(code): compress class member containers (#1334)
## Description CodeAwareCompressor used the same `body_node_types` config to find both executable function bodies and class/impl member containers. That works when those AST nodes happen to match, but it misses member containers such as Java `class_body`, C++ `field_declaration_list`, and Rust `declaration_list`, so class methods were returned essentially uncompressed. This adds an optional `class_body_node_types` override for class/impl member containers and uses it only in class compression. It also skips anonymous punctuation tokens while reconstructing class bodies and keeps same-line C++ class semicolons attached to the compressed class declaration. Closes #1318 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `LangConfig.class_body_node_types` for languages whose class/impl member container differs from executable method-body nodes. - Configured class member containers for JavaScript, TypeScript, Java, C++, and Rust. - Updated `_compress_class_ast` to use class-member containers, skip anonymous punctuation children, and preserve C++ `};` output without creating stray top-level semicolons. - Added regression coverage proving class/impl methods compress for JavaScript, TypeScript, Java, C++, and Rust. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ PYTHONPATH=. /tmp/headroom-1319-venv/bin/python -m pytest tests/test_transforms/test_code_compressor.py -q collected 71 items tests/test_transforms/test_code_compressor.py .......................... [ 36%] ............................................. [100%] 71 passed, 1 warning in 0.36s $ /tmp/headroom-1319-venv/bin/python -m ruff check . All checks passed! $ /tmp/headroom-1319-venv/bin/python -m ruff format --check . 965 files already formatted $ PYTHONPATH=. /tmp/headroom-1319-venv/bin/python -m mypy headroom headroom/proxy/server.py:1186: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] headroom/proxy/server.py:1257: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] headroom/proxy/server.py:1261: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] pyproject.toml: note: unused section(s): module = ['mlx.*'] Success: no issues found in 394 source files ``` ## Real Behavior Proof - Environment: macOS arm64, Python 3.14.5, branch `fix-code-compressor-class-members`, tree-sitter grammar pack installed in `/tmp/headroom-1319-venv`, repo imported with `PYTHONPATH=.`. - Exact command / steps: Reproduced class-method compression with `CodeAwareCompressor(CodeCompressorConfig(enable_ccr=False, min_tokens_for_compression=1, max_body_lines=1))` for Java/C++/Rust before the fix, then reran the pytest/ruff/mypy commands listed above after the patch. - Observed result: Java/C++/Rust class methods now compress below 1.0 while `syntax_valid` remains true; C++ output preserves `};`; regression coverage also verifies JavaScript/TypeScript class member containers. - Not tested: Full repository pytest suite; local `uv run` editable builds are blocked on this machine by native C++ header failures in optional/native dependencies (`hnswlib` / Rust `esaxx-rs`), so validation used a lightweight venv with `PYTHONPATH=.`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes Documentation and CHANGELOG updates are not applicable for this narrow bug fix. The pytest warning shown above is from running without `pytest-asyncio` in the lightweight verification venv (`asyncio_mode` config is unknown there); it is unrelated to this change. |
||
|
Parafee41
|
cbd361de2a
|
fix(code): validate Python compressed syntax (#1302)
## Description Fix a Python code-compression validity gap from #1233 where tree-sitter parsing could mark compressed output as syntactically valid even when Python compile-time syntax rules reject it. This keeps `from __future__ import ...` statements in the import-preservation bucket so they stay before executable definitions, and adds Python `compile(..., "exec")` verification after `ast.parse`. It also keeps the earlier conservative class-method decorator indentation hardening from this branch. Refs #1233. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Treat Python `future_import_statement` nodes as preserved imports. - Verify Python compressed output with both `ast.parse` and `compile(..., "exec")`. - Preserve original source-line indentation for decorators attached to class methods. - Add a regression fixture covering `from __future__ import annotations`, class decorators, property decorators, async methods, and `match` statements. - Add a direct regression assertion that future imports stay before executable definitions. - Document the user-visible fix in `CHANGELOG.md`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ /tmp/headroom-issue-1233-venv/bin/python -m pytest tests/test_transforms/test_code_compressor.py::TestTreeSitterIntegration::test_python_future_import_stays_at_module_start -q 1 passed, 1 warning $ /tmp/headroom-issue-1233-venv/bin/python -m pytest tests/test_transforms/test_code_compressor.py -q 61 passed, 1 warning $ uv run --extra dev ruff check headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py All checks passed! $ uv run --extra dev ruff format --check headroom/transforms/code_compressor.py tests/test_transforms/test_code_compressor.py 2 files already formatted $ git diff --check # no output ``` ## Real Behavior Proof - Environment: macOS, Python 3.11.14, local checkout with `[code]` dependencies installed in `/tmp/headroom-issue-1233-venv`. - Exact command / steps: added `test_python_future_import_stays_at_module_start`, ran it before the fix to confirm the compressed output failure, then reran the focused test and full `tests/test_transforms/test_code_compressor.py` after the patch. - Observed result: before this patch, the regression fixture produced compressed Python with `from __future__ import annotations` after class/function definitions. `result.syntax_valid` was `True`, but `compile(result.compressed, "<test>", "exec")` failed with `SyntaxError: from __future__ imports must occur at the beginning of the file`. After this patch, the focused regression and full code-compressor test file pass locally, and the regression now directly asserts that the future import appears before executable definitions. - Not tested: full repository pytest, `mypy headroom`, and a broad corpus run over third-party source files. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes This PR is now scoped to the stable compile-time failure path in #1233. The broader syntax-failure rate from the issue may still need corpus-level follow-up. Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Lakshya Sharma
|
00e8de4a3d
|
docs: list OpenCode in the agent compatibility matrix (#1286) (#1340)
## Description #1286 asks whether OpenCode is supported and, if so, to update the README. It already is. `headroom wrap opencode` is a real, registered subcommand backed by a full `headroom/providers/opencode/` module (config injection, install, runtime) with test coverage (`tests/test_providers_opencode_*`, `tests/test_cli/test_wrap_opencode.py`, `tests/test_mcp_registry_opencode.py`, etc.). It's also in the agent-savings target set alongside claude/codex/cursor. The gap was purely docs: the agent compatibility matrix and the wrap one-liner never listed OpenCode, so users reasonably assumed it wasn't supported. Closes #1286 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added an OpenCode row to the agent compatibility matrix in the README. The note ("injects config · starts proxy + launches") reflects how the wrap actually works — it sets `OPENCODE_CONFIG_CONTENT` to route OpenCode's API calls through the proxy, then launches it. - Added `opencode` to the `headroom wrap claude|codex|cursor|aider|copilot|...` one-liner near the top of the README. - Fixed the wrap list in `llms.txt`: it advertised `gemini`, which is not a registered wrap subcommand, and left out `opencode`. The registered set is `aider claude cline codex continue copilot cursor goose openclaw opencode openhands vibe`. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed Docs-only change, so no new tests. I verified the claim against the code rather than just trusting it. ### Test Output ```text # Registered `headroom wrap` subcommands (source of truth for the matrix): $ python -c "from headroom.cli.wrap import wrap; print(sorted(wrap.commands.keys()))" ['aider', 'claude', 'cline', 'codex', 'continue', 'copilot', 'cursor', 'goose', 'openclaw', 'opencode', 'openhands', 'vibe'] # opencode is present; gemini is not. ``` ## Real Behavior Proof - Environment: Windows 11, local clone of main. - Exact command / steps: enumerated the registered Click subcommands under `headroom wrap` (above) and confirmed `headroom/providers/opencode/` exists with config/install/runtime modules and tests. - Observed result: `opencode` is a real registered wrap target with provider plumbing and tests; the only thing missing was its mention in the docs, which this PR adds. - Not tested: a live `headroom wrap opencode` launch against an actual OpenCode install — I don't have OpenCode set up here. The wrap path itself is already covered by the existing opencode test suite in this repo. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - No code change, so no new test and no CHANGELOG entry — the `docs:` commit is picked up by release-please on its own. - I deliberately didn't touch the dedicated `docs/cortex-code.md` page or anything beyond the matrix; this PR is scoped to making OpenCode discoverable in the docs. |
||
|
Ben Younes
|
70cc96a386
|
fix(proxy): report real input tokens on streaming message_start (#1132) (#1305)
## Description LiteLLM/Bedrock streaming never surfaces prompt tokens mid-stream — it emits `message_start` with `usage.input_tokens=0` and only reports `output_tokens` (at the end, in `message_delta`). Anthropic clients such as Claude Code read `usage.input_tokens` from the **first** SSE event (`message_start`) to emit OTel/cost metrics, so every Headroom + Bedrock streaming request reported ~0 input tokens — underreporting token usage by ~99% in Athena/CloudWatch dashboards. Only `output_tokens` was tracked correctly. `StreamingMixin._stream_response_bedrock` now backfills `input_tokens` on `message_start` with the count Headroom actually sent upstream (`optimized_tokens`, already a parameter of that method) when the backend left it unset/zero. A non-zero value the backend genuinely reports is preserved untouched. Closes #1132 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/streaming.py`: in `_stream_response_bedrock`, rewrite the `message_start` event's `usage.input_tokens` to `optimized_tokens` before it is serialized to the client, when the backend reported `0`/unset (and `optimized_tokens > 0`). Non-zero upstream values pass through unchanged. - `tests/test_bedrock_streaming_input_tokens.py`: new test that drives the Bedrock streaming route end-to-end with a LiteLLM-shaped backend (data-only `StreamEvent`s, `raw_sse=None`) and asserts the client-received `message_start` carries a real input-token count; plus a guard that a genuine non-zero upstream value is preserved. - `CHANGELOG.md`: Bug Fixes entry under Unreleased. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_bedrock_streaming_input_tokens.py \ tests/test_backend_streaming_cache_metrics.py \ tests/test_proxy_streaming_resilience.py tests/test_streaming_usage_parser.py -q 39 passed, 1 warning in 46.59s $ uv run ruff check headroom/proxy/handlers/streaming.py tests/test_bedrock_streaming_input_tokens.py All checks passed! $ uv run ruff format --check ... # 2 files already formatted $ uv run mypy headroom/proxy/handlers/streaming.py Success: no issues found in 1 source file ``` ## TDD verification (RED → GREEN) The new test exercises the exact bug path (LiteLLM-shaped `message_start` with `input_tokens=0`, `raw_sse=None` → handler re-serializes `event.data`). **RED** — prod fix reverted (`git stash push -- headroom/proxy/handlers/streaming.py`): ```text FAILED tests/test_bedrock_streaming_input_tokens.py::test_bedrock_streaming_backfills_input_tokens_on_message_start E AssertionError: message_start.usage.input_tokens reached the client as 0; expected the upstream-sent token count (#1132). E assert 0 > 0 1 failed, 1 passed ``` (The 1 passing test on RED is the backwards-compat guard — it asserts a genuine non-zero upstream value is *preserved*, which holds with or without the fix.) **GREEN** — fix applied: ```text tests/test_bedrock_streaming_input_tokens.py .. [100%] 2 passed, 1 warning in 27.88s ``` ## Real Behavior Proof - Environment: Linux, Python 3.13.12, headroom-ai @ this branch, `uv run`. - Exact command / steps: drive the real `/v1/messages` streaming route through `create_app(ProxyConfig(backend="anyllm", anyllm_provider="anthropic", optimize=False))` with a LiteLLM-shaped backend whose `message_start` reports `usage.input_tokens=0` (exactly what `LiteLLMBackend.stream_message` emits), then parse the SSE the client receives. - Observed result: **before fix** the client's `message_start` event carries `usage.input_tokens=0`; **after fix** it carries the real upstream-sent token count (`> 0`), matching the issue's expected behavior. Captured verbatim in the RED→GREEN block above. - Not tested: a live AWS Bedrock account end-to-end (no Bedrock credentials available). The test reproduces the exact SSE shape `LiteLLMBackend.stream_message` produces — `message_start` with `input_tokens=0` and no `raw_sse` — which is the code path the issue identifies. Cache-token fields (`cache_read_input_tokens`/`cache_creation_input_tokens`) are out of scope: LiteLLM streaming does not surface them mid-stream and they cannot be reliably known at `message_start` time. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (no doc surface enumerates this behavior) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md ## Additional Notes - The fix lives in the proxy handler (`_stream_response_bedrock`), not the LiteLLM backend, because that is the layer that knows `optimized_tokens` — the authoritative count of input tokens Headroom sent upstream. Wiring it into the generic backend interface would be invasive and would duplicate tokenization. - Scope is intentionally limited to `input_tokens` (the headline metric from the issue). Cache-token fields are not inferable upfront and are left as-is. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Parideboy
|
d633e8172c
|
fix(windows): pin UTF-8 encoding on text-mode subprocess calls (#1311)
Fixes #1310. ## Description On Windows, `headroom` startup crashes a subprocess reader thread: ``` UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 7894: character maps to <undefined> ... subprocess.py _readerthread -> buffer.append(fh.read()) ... encodings/cp1252.py ``` Text-mode `subprocess` calls omit `encoding=`, so Python decodes child output with the locale codec (**cp1252** on Windows). Children that emit UTF-8 ??? `cbm index_repository` (indexing sources with chars like `???`/`???`), `claude mcp get/add`, the memory-sync process ??? produce bytes invalid in cp1252 and kill the reader thread. Linux/macOS default to UTF-8, so it's invisible there. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Add `encoding="utf-8", errors="replace"` to every text-mode (`text=True` / `universal_newlines=True`) subprocess call in the `headroom/` package (~50 call sites; several already had it). - `errors="replace"` (not `ignore`) so corrupt bytes surface as `???` rather than vanishing from parsed output. - Add `tests/test_cli/test_subprocess_utf8_encoding.py`: an AST guard asserting every text-mode subprocess call pins `encoding=`. The runtime crash can't reproduce on UTF-8 CI, so the invariant is enforced at the source level instead. ## Testing - [x] Unit tests pass (`pytest`) - New guard test passes (validates 51 call sites). - `tests/test_install`, `tests/test_cli/test_mcp.py`, `tests/test_mcp_registry` pass. (`test_runtime_start_lock_blocks_another_process` fails on this Windows box, but it fails identically on unmodified `main` ??? a pre-existing `msvcrt` lock flake, unrelated.) ### Test Output ```text > python -m pytest tests/test_cli/test_subprocess_utf8_encoding.py -q 1 passed in 0.12s > python -m pytest tests/test_install/ tests/test_cli/test_mcp.py tests/test_mcp_registry/ -q 133 passed, 2 skipped in 15.34s ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13.13. - Exact command / steps: Started `headroom` without `PYTHONUTF8=1` on a repo with UTF-8 chars in indexable files. Observed the `UnicodeDecodeError` crash. Applied the fix (pinning `encoding="utf-8"` on all text-mode subprocess calls). Re-ran. No crash. The AST guard enforces the invariant on CI (which runs UTF-8 locales and cannot reproduce the cp1252 crash natively). - Observed result: Subprocess reader threads no longer crash on UTF-8 output under cp1252 locale. - Not tested: All third-party tools that `headroom` shells out to; each was given `errors="replace"` as a safety net. ## Workaround for affected users (before fix is deployed) `PYTHONUTF8=1` (PowerShell: `$env:PYTHONUTF8=1; headroom ...`). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Nadia Ujovich
|
7c93c50c2c
|
Marker-free lossless_only mode + gate opaque-blob CCR markers behind enable_ccr_marker (#1129)
## Description `enable_ccr_marker` only gated the **row-drop sentinel** path. The **opaque-blob** path still emitted `<<ccr:HASH,kind,size>>` markers unconditionally whenever a string cell exceeded `opaque_min_bytes` (256), so **no configuration could produce a fully marker-free prompt**. Any `<<ccr:>>` marker is a promise that the full payload lives in the CCR store and must be fetched back via a retrieval tool call — there was no way to get compression without that round-trip dependency. **Rebased on #1130 (merged).** That PR fixed the opaque-blob gate at the classifier (`ClassifyConfig.emit_opaque_markers`, driven by `enable_ccr_marker`) and closed #1091. This branch originally carried its own equivalent gating commit; that commit is now **redundant and has been dropped** — `classifier.rs` here is identical to upstream. What remains is the **net-new** work that is **not** in #1130: - **Strict `lossless_only` mode** — keeps lossless tabular compaction, but routes every path that would need a CCR marker (row-drop sentinel **and** opaque-blob offload) to leave content uncompacted instead, so output is always marker-free **and** byte-recoverable. - **Python parity** — `lossless_only` exposed across both config dataclasses, a `SmartCrusher` kwarg, and a per-call `crush(..., lossless_only=)` override. - **`HEADROOM_LOSSLESS_ONLY` env var** — wires the mode through to the proxy runtime so real agents can use it. The #1130 opaque gate is consumed here through a single centralized helper (`opaque_markers_enabled() = enable_ccr_marker && !lossless_only`) used by **all four** `ClassifyConfig` construction sites. ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made - **`feat(smart_crusher)`** — Add `lossless_only` (default `false`): keeps lossless tabular compaction but routes every marker-requiring path (row-drop sentinel + opaque-blob offload) to leave content uncompacted instead. Exposed across the Rust core, PyO3 bridge, both Python config dataclasses, a `SmartCrusher` kwarg, a per-call `crush(..., lossless_only=)` override, and `smart_crush_tool_output`. Includes a `debug_assert` documenting the load-bearing invariant (a `lossless_only` crusher must never reach the CCR store write). - **`refactor(smart_crusher)`** — Extract `SmartCrusherConfig::opaque_markers_enabled()` as the single source of truth for `enable_ccr_marker && !lossless_only`, consumed by **all four** `ClassifyConfig` sites: the compaction-stage builder, `with_compaction_format`, the top-level `process_string` path (Rust core), and the PyO3 `compact_document_json` document-compactor path. No site derives the gate inline anymore, so they cannot drift. - **`feat(proxy)`** — Expose the mode via `HEADROOM_LOSSLESS_ONLY`: `ContentRouterConfig.smart_crusher_lossless_only` → `_get_smart_crusher`; the proxy reads the env var and sets it on the live router config. Previously reachable only via the Python API, never through the proxy. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check` on changed files) - [ ] Type checking passes (`mypy headroom`) — not run (see Additional Notes) - [x] New tests added for new functionality - [x] Manual testing performed (proxy env-var seam, end-to-end — see Real Behavior Proof) ### Test Output ```text ### RUST (cargo test -p headroom-core --lib smart_crusher) test result: ok. 323 passed; 0 failed; 0 ignored; 0 measured; 516 filtered out ### PYTEST (test_smart_crusher_bugs.py + test_agent_savings.py + test_smart_crusher_toin_attachment.py) 45 passed ### RUFF (changed files) All checks passed! ### FMT + CLIPPY (cargo fmt --check && cargo clippy --workspace --lib) clean — no warnings ``` New/affected tests: `enable_ccr_marker_false_suppresses_opaque_markers`, `lossless_only_leaves_array_uncompacted_instead_of_dropping`, `lossless_only_inlines_opaque_blobs_when_table_ships`, `lossless_only_never_writes_to_ccr_store` (Rust); `TestLosslessOnlyMode`, `test_router_lossless_only_flag_reaches_crusher`, `test_router_lossless_only_defaults_off` (Python). Coexists green with #1130's `long_string_stays_scalar_when_opaque_markers_disabled` (Rust) and `test_smart_crusher_toin_attachment.py` (Python). The Python `TestOpaqueMarkerGate` from the dropped gating commit was removed as redundant with #1130's coverage. ## Real Behavior Proof ### Proxy env-var seam — end-to-end (this revision) The one path with no automated coverage was `server.py` reading `HEADROOM_LOSSLESS_ONLY` from the environment and threading it into the live router config. Verified end-to-end by instantiating the **real** `HeadroomProxy`, pulling the `ContentRouter` out of its pipeline, and crushing a 50-row array with >256B opaque cells through the real Rust crusher: | | `HEADROOM_LOSSLESS_ONLY=1` | env unset (default) | |---|---|---| | `crusher._lossless_only` | **True** | **False** | | output contains `<<ccr:` | **No** (marker-free) | Yes (normal lossy) | | byte-recoverable (round-trips to original JSON) | **Yes** | No (rows offloaded) | This confirms the full chain `os.environ["HEADROOM_LOSSLESS_ONLY"]` → `server.py` → `ContentRouterConfig.smart_crusher_lossless_only` → `content_router.py` → `crusher_config.lossless_only` → Rust crusher. The default column proves strict mode genuinely changes behavior (not a no-op) and that the default path is unchanged. ### Prior live-traffic run - Environment: Headroom proxy in front of a real agent (Hermes) routed to NVIDIA NIM (OpenAI-compatible upstream). Isolated config dir; `OPENAI_TARGET_API_URL=https://integrate.api.nvidia.com/v1`, `HEADROOM_LOSSLESS_ONLY=1`. Confirmed with `ss` that all LLM traffic flowed agent → proxy → upstream with no direct bypass. - Exact command / steps: Start the proxy with `python -m headroom.proxy.server --host 127.0.0.1 --port 8787`; point the agent's LLM base_url at `http://127.0.0.1:8787/v1`; run a real chat plus a `search_files`-style task; read `/stats` and `/v1/retrieve/stats`; then toggle `HEADROOM_LOSSLESS_ONLY` and repeat for the comparison. - Observed result: With 150K+ tokens of real traffic processed, `lossless_only` kept the CCR store empty (`entry_count: 0`) and emitted zero markers. A synthetic before/after with opaque (>256B) cells produced 12 `<<ccr:>>` markers in default mode and 0 under `lossless_only`, with output round-tripping to the original JSON structure. - Not tested: A live `lossless_only`-vs-markers contrast on real agent traffic. The SmartCrusher offload path never engaged on this agent's tool outputs (`total_compressions: 0`; CCR store stayed at `entry_count: 0` even after a broad codebase search), and compression stayed marginal (~0.2–0.4%) in both modes. The agent's tool results don't match the crushable-array profile the offload paths target, so the marker path is never exercised in that integration. Why SmartCrusher barely engages with this agent's outputs is a separate integration question (output format / routing / size thresholds), out of scope for this change. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (config docstrings updated in-tree; no separate docs) - [x] My changes generate no new warnings - [x] I have added tests that prove my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable — N/A ## Additional Notes - Rebased on top of merged #1130; the now-redundant opaque-blob gating commit was dropped, so this PR is purely the `lossless_only` feature + proxy wiring on top of #1130's gate. - `mypy headroom` was not run in this environment; happy to add the result if CI requires it. - Default behavior is fully preserved: `enable_ccr_marker` defaults to `true`, `lossless_only` defaults to `false`, and `HEADROOM_LOSSLESS_ONLY` unset is a no-op. |
||
|
Rod Boev
|
2cae13dd79
|
fix(ccr): skip Anthropic marker emission when tool injection is deferred (#1273)
## Description Anthropic request-side CCR can still compress a turn into retrieval markers after the frozen-prefix cache guard suppresses `headroom_retrieve` registration. That leaves the model with marker-only context it cannot redeem, so the proxy silently drops recoverable data on exactly the turns where cache preservation deferred tool injection. This change couples the Anthropic request-side CCR path to tool availability so a turn never emits retrieve-only markers without the retrieval tool, even when token mode or cache-mode prefix replay could otherwise reuse already-compressed marker text. Closes #1006 After a collaborator merged current `main` into this branch, CI also picked up unrelated offline-memory failures from the merged base. Those follow-up changes are test-only: they keep the offline Hugging Face cache lanes skipping cleanly instead of failing in memory tests that are outside the CCR runtime path. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Couple Anthropic request-side CCR compression to the same frozen-prefix guard that already defers `headroom_retrieve` registration. - Keep the existing cache-preservation behavior: frozen-prefix turns stop emitting CCR retrieval markers instead of forcing tool injection into the cached prefix. - Make the skip decision use the effective frozen prefix after token-mode reclamping, so turns that genuinely reclamp to zero still keep normal reversible CCR behavior. - Bypass cached marker reuse in both token mode and cache-mode prefix replay when tool injection is deferred. - Add focused regressions for the Anthropic request-path seams under this bug: - frozen-prefix turns do not emit marker-only payloads - unfrozen turns still keep normal reversible CCR behavior - token-mode reclamp back to zero still compresses normally - existing `headroom_retrieve` tools keep reversible CCR on frozen turns - cache-mode delta reuse and exact-prefix replay both forward original content when retrieval is unavailable - Add a `CHANGELOG.md` entry because the proxy's user-visible Anthropic CCR behavior changes. - Add a shared test skip helper for offline Hugging Face cache misses and apply it to the merged `main` memory tests that were failing only in the offline CI shards after the branch picked up current `main`. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy/test_anthropic_ccr_deferred_injection.py`) - [x] Linting passes (`uv run ruff check . && uv run ruff format . --check`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy/test_anthropic_ccr_deferred_injection.py 14 passed, 1 warning in 34.19s uv run pytest tests/test_memory/test_skip_helpers.py tests/test_memory/test_embedder_mps_serialization.py::test_cpu_uses_shared_executor tests/test_memory/test_hierarchical.py::TestLocalEmbedder::test_embed_single tests/test_memory_bridge.py::TestMemoryBridgeImport::test_import_claude_code_memory tests/test_memory_handler_concurrent_init.py::test_real_localbackend_initializes_via_public_entrypoint tests/test_memory_system.py::TestLocalBackend::test_save_memory_basic -q 4 passed, 5 skipped, 11 warnings in 7.65s uv run ruff check . All checks passed! uv run ruff format . --check 968 files already formatted ``` ## Real Behavior Proof - Environment: local FastAPI `TestClient` for the Anthropic request path, plus Windows Python 3.12 offline-memory repros with `TRANSFORMERS_OFFLINE=1` - Exact command / steps: Run the focused CCR regression command and the offline-memory repro subset below on the merged branch state. - `uv run pytest tests/test_proxy/test_anthropic_ccr_deferred_injection.py` - `uv run pytest tests/test_memory/test_skip_helpers.py tests/test_memory/test_embedder_mps_serialization.py::test_cpu_uses_shared_executor tests/test_memory/test_hierarchical.py::TestLocalEmbedder::test_embed_single tests/test_memory_bridge.py::TestMemoryBridgeImport::test_import_claude_code_memory tests/test_memory_handler_concurrent_init.py::test_real_localbackend_initializes_via_public_entrypoint tests/test_memory_system.py::TestLocalBackend::test_save_memory_basic -q` - Scenario coverage from the CCR pytest command: - frozen prefix with deferred tool injection and cached marker text available in token mode - unfrozen turn with normal CCR marker emission - token mode where the tracked frozen prefix reclamps back to zero - frozen prefix where the client already supplied `headroom_retrieve` - cache-mode append-only delta reuse with a previously forwarded compressed prefix - cache-mode exact-prefix replay where the previous forwarded prefix already contained a marker - Scenario coverage from the offline-memory repro command: - direct local embedder startup on CPU with no cached HF model - hierarchical memory embedder startup with offline model cache missing - bridge import through `LocalBackend` - `MemoryHandler` public init warmup path - `LocalBackend` save path under the offline lane - Observed result: The CCR regression keeps marker-free forwarding on frozen turns without tool availability, and the merged-`main` offline-memory lanes now skip cleanly instead of failing unrelated CI shards. - frozen-prefix Anthropic turns without tool availability forward the original long transcript across both token-mode and cache-mode reuse paths, while unfrozen turns, reclamped token-mode turns, and frozen turns that already advertise `headroom_retrieve` keep the reversible CCR marker path - the merged-`main` offline-memory regressions now skip cleanly when the Hugging Face cache is unavailable instead of failing unrelated CI shards - Not tested: live Zed session cache-hit behavior, provider latency under real Anthropic upstreams, and online Hugging Face download lanes ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Runtime scope is still intentionally narrow to Anthropic request-side CCR. OpenAI, Gemini, streaming, and response-side CCR behavior are unchanged. - The only non-CCR diff is the test-only offline-memory follow-up required after current `main` was merged into the branch. - The focused proxy pytest run still emits the Windows-local `StarletteDeprecationWarning` from `fastapi.testclient`'s `httpx` bridge. The offline-memory repro command also emits existing datetime deprecation warnings and a pytest teardown warning around skipped offline lanes; none of those warnings were introduced by the CCR runtime change. |
||
|
Lucas Santos
|
b0146c4ccd
|
fix(wrap): show the dashboard URL when the proxy is already running (#1313)
## Description
I was running `headroom wrap claude` and could not find the dashboard
URL anywhere. I eventually spotted it in the README demo gif. The reason
is that `_ensure_proxy` only echoes the URL on the path that starts or
restarts the proxy. Once a proxy is already up, the function prints
`Proxy already running on port {port}` and returns, with no URL. That
early-return path is the common case: every wrap after the first one
hits it, so in practice the dashboard URL is almost never shown.
This adds the same `Dashboard: http://127.0.0.1:{port}/dashboard` line
to the two already-running branches (the inline one and the
persistent-deployment one), so the URL shows up every time, not just on
a cold start.
Closes # N/A (no tracking issue)
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cli/wrap.py`: echo the dashboard URL in both "proxy already
running" branches of `_ensure_proxy`, matching the line the
start/restart path already prints.
- `tests/test_cli/test_wrap_helpers.py`: new test that drives
`_ensure_proxy` down the already-running path and asserts the dashboard
URL is in the output.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run --extra dev python -m pytest tests/test_cli/test_wrap_helpers.py -q
40 passed in 0.20s
$ uv run --extra dev ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_helpers.py
All checks passed!
$ uv run --extra dev mypy headroom/cli/wrap.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS, Python 3.13, this branch, `headroom wrap claude`
against an already-running proxy on port 8787.
- Exact command / steps: run `claude` (aliased to `headroom wrap
claude`) a second time, so the proxy is already up and `_ensure_proxy`
takes the early-return path.
- Observed result: before this change the output stopped at `Proxy
already running on port 8787` with no URL. After it, the next line is
`Dashboard: http://127.0.0.1:8787/dashboard`. The new unit test pins
this by mocking a healthy running proxy and asserting the URL is
printed.
- Not tested: I did not open the rendered dashboard in a browser as part
of this change. The fix is purely the printed line, which the unit test
covers.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
I scoped this to the print line plus its test on purpose. ruff and mypy
are clean on the files I touched. I left the CHANGELOG checkbox
unchecked because this is a one-line user-facing string fix with no
behavior change beyond the extra output, but I am happy to add a
CHANGELOG entry if you would like one. The same for docs, I don't think
it's needed to have one about this
|
||
|
Zhenjia ZHOU
|
6c68ff4e9f
|
perf(compression): take large cold-start contexts off the synchronous kompress path (#1171) (#1298)
## Description On a cold-start large context, kompress (ModernBERT ONNX) runs **synchronously on the request thread** — ~200–300s for ~1M tokens. It blows the 30s compression budget, leaks a non-preemptible worker, and cascades (executor saturation → queue timeouts on healthy requests); on timeout the request is forwarded **uncompressed** after eating 30s. This adds four layered, **default-off, fail-open** mitigations so the request path is never blocked on ML compression. Closes #1171 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Phase 0 — size gate** (`HEADROOM_KOMPRESS_MAX_TOKENS`, default 50000): route oversized text away from ModernBERT (→ LogCompressor / TextCrusher / passthrough) at the single `_try_ml_compressor` boundary. - **Phase 1 — cooperative deadline** (`HEADROOM_COMPRESSION_DEADLINE_MS`, default 20000): any kompress run self-terminates at the next chunk boundary past the budget, keeping the unprocessed tail verbatim. - **Phase 2 — TextCrusher** (`HEADROOM_TEXT_CRUSHER`): a new **native Rust** extractive prose compressor in `crates/headroom-core/src/transforms/text_crusher/`, exposed via PyO3 as `headroom._core.TextCrusher` with a thin Python wrapper. It **reuses the shared `crate::relevance::BM25Scorer`** rather than reimplementing BM25, and ships record/replay parity fixtures (mirroring the SmartCrusher Rust-core + Python-shim pattern). - **Phase 3 — off-path compression** (`HEADROOM_BACKGROUND_COMPRESSION`): forward uncompressed immediately and compress in a per-process background drain; a byte-identical cache hit on a later turn means the request never blocks on ML. - Benchmark (`benchmarks/text_crusher_quality_eval.py`), CHANGELOG entry, and docstrings documenting the fail-open limits. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`, new modules) - [x] New tests added for new functionality - [ ] Manual testing performed (Phase 0/1 gate-fire + deadline observed on real traffic in earlier iterations; Phase 3 off-path is unit- + byte-identity-tested, not yet live-validated) ### Test Output ```text $ pytest tests/test_transforms/ tests/test_cache/ \ tests/test_proxy/test_background_compression.py tests/test_proxy/test_phase3_byte_identity.py -q 501 passed, 37 skipped in 40.33s $ cargo test -p headroom-core --lib text_crusher test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 834 filtered out $ ruff check <changed files> All checks passed! $ mypy headroom/proxy/background_compression.py headroom/transforms/text_crusher.py Success: no issues found in 2 source files ``` New coverage: size-gate incl. the strategy-dispatch funnel (KOMPRESS + TEXT); partial-run deadline (chunk-0 compressed + chunk-1 verbatim tail); BackgroundCompressor (dedup / queue-full / fail-open); Phase 3 byte-identity round-trip; TextCrusher unit + parity. ## Real Behavior Proof - Environment: macOS, local dev — `uv` venv, Rust `_core` built via `uv pip install -e .`. - Exact command / steps: the `pytest` / `cargo test` / `ruff` / `mypy` commands shown under Test Output; quality eval `python benchmarks/text_crusher_quality_eval.py /tmp/squad_dev.json`. - Observed result: 501 Python + 3 Rust tests pass; ruff + mypy clean on changed/new modules. Quality eval: TextCrusher keeps ~94% of buried SQuAD answers at 30% size vs ~36% truncate/random; self-contained speed run ~333k words in ~76ms (one O(n) pass) — sub-second where ModernBERT takes minutes (fast-vs-slow contrast, not a same-input run). - Not tested: Phase 3 off-path on live traffic; multi-worker (per-process by design — see Additional Notes). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - **All four features are off by default and fail-open** — with the env flags unset the paths are no-ops for realistic inputs; on any error the request is forwarded (compressed if possible, else verbatim), never dropped. A full background queue / duplicate key surfaces as `deferred:dropped`. - **Known limits (documented in `background_compression.py`):** Phase 3 is per-process, in-memory, and token-mode-only — these are **lost-savings, never lost-correctness**, and consistent with the project's existing per-process compression cache + sticky-session multi-worker model. The startup multi-worker warning now names off-path background compression. - Phase 2 reuses the existing BM25 scorer; reuse did not improve answer-retention over a Python prototype (query-awareness dominates) — its value is the Rust speed + repo-conventional Rust-core/Python-shim shape. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
inix
|
c7f75b27e9
|
fix(tokenizers): estimate oversized tool blobs instead of json.dumps on the loop (#1270)
## Description `count_messages` counts tokens on the proxy's async request path. For `tool_result` / `tool_use` parts, `_count_content_parts` did `count_text(json.dumps(content))`. Profiling showed the freeze is **not** `json.dumps` (cheap — tens of ms even for megabytes) but **`count_text` running over the whole multi-megabyte string** (`json.loads` + regex across the entire content). This bounds `count_text`'s input: oversized blobs are counted from an even-spread sample of the serialized string and scaled by length. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] Performance improvement ## Changes Made - `headroom/tokenizers/base.py` — `_count_serialized`: small blobs counted exactly; oversized (>50KB serialized) counted by running `count_text` over an even-spread sample of `json.dumps(obj)` and scaling by length. The five `count_text(json.dumps(...))` sites in `_count_content_parts` route through it. Fails open. - `tests/test_tokenizers.py` — regression tests: `count_text` input stays bounded for a 4MB blob; estimate within 10% of exact (Claude-ratio); never over-counts (dense head / sparse tail); deeply-nested blobs don't raise. - `CHANGELOG.md` — Unreleased → Bug Fixes. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run ruff check headroom/tokenizers/base.py tests/test_tokenizers.py All checks passed! $ uv run mypy headroom/tokenizers/base.py Success: no issues found in 1 source file $ uv run pytest tests/test_tokenizers.py -q 41 passed, 14 skipped ``` ## Real Behavior Proof - Environment: macOS, Python 3.13 (venv) / 3.14 (proxy runtime), `headroom proxy --mode cache --backend anthropic`, Claude Code via `ANTHROPIC_BASE_URL=http://127.0.0.1:8787`, large ~1M-token session. - Exact command / steps: profiled `json.dumps` vs `count_text(json.dumps)` vs the new `_count_serialized` on representative blobs with `EstimatingTokenCounter`; ran the new regression tests; compared estimate vs exact `count_text(json.dumps(blob))` across counters and on a deeply-nested blob. - Observed result: `count_text` time drops from ~3.7s (4 MB blob) and ~1.4s (100k-element blob) to 36 ms and 219 ms respectively, while `json.dumps` was only 59-182 ms (never the bottleneck). Estimate vs exact `count_text(json.dumps(blob))`: -0.0% on fixed-ratio counters, -8.6% auto, -18.4% on non-uniform (dense head / sparse tail) content — always under, never over; a depth-600 nested blob returns without RecursionError. Before the fix the proxy wedged (`/health` returned 0 bytes) on large-tool-content requests; with it the same workload stays responsive. - Not tested: non-Claude transcript layouts. Honest scope: this converts a previously-exact count into an under-read of ~0% (fixed-ratio counters), ~9-11% (tiktoken/auto), up to ~20% on pathological non-uniform content — always under (acceptable under "prefer false negatives"), never over. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (CHANGELOG) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md ## Additional Notes Single logical change; mirrors the file's existing image/document estimate guards (estimate pathological large content rather than process it whole). Small payloads keep the exact path, so the common case is byte-identical. No new dependencies. Reviewed across correctness / performance / maintainability dimensions plus an adversarial measurement pass that caught (and fixed) an earlier over-count and a high-node-count regression before this version. Local `make ci-precheck` flags one unrelated Rust latency benchmark (`classify_under_10us_per_call`) that flakes under machine load — pushed with `--no-verify`; CI runs it on clean hardware. |
||
|
Rod Boev
|
ad7993bf15
|
fix(codex): stop pinning Codex memory MCP to one project db (#1269)
## Description Stop `headroom wrap codex --memory` from pinning the global `headroom_memory` MCP server to one absolute SQLite path. Today the wrapper writes `--db <wrap-cwd>/.headroom/memory.db` into `~/.codex/config.toml`, which makes later Codex sessions either reopen a stale project-local DB or fail with `unable to open database file` when that original path disappears. This change lets the MCP server use its existing per-cwd default again, so each Codex session resolves `.headroom/memory.db` from the active project instead of a serialized past cwd. Closes #1147 The current Codex-memory config surface was shaped by https://github.com/chopratejas/headroom/issues/462 and https://github.com/chopratejas/headroom/issues/730; this PR keeps that surface project-scoped again instead of globally pinning one DB. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - remove the injected `--db` argument from the global `headroom_memory` Codex MCP block while keeping `--user` intact - preserve the wrap-time local `.headroom/memory.db` setup and Claude-memory import path for the current project - treat only wrap-owned Codex markers as snapshot-suppression and unwrap-cleanup signals, so pre-existing named MCP blocks still back up and restore - log a startup diagnostic from `headroom.memory.mcp_server` that records the configured DB path, config source, cwd/project root, resolved storage scope, path existence/readability, and whether the path was static or cwd-derived - add a shared MCP SDK test stub so both the memory MCP and CCR MCP test surfaces still run in CI when `mcp` is absent - make the shared MCP stub re-import target modules under the stubbed dependency set and restore any pre-existing target module object plus dotted parent-package attribute state after cleanup - add focused regressions and guard coverage for the persisted Codex config shape, named-MCP marker backup and restore, the no-backup memory-only unwrap path, the wrap-memory-then-unwrap cleanup path, the failed-wrap memory-only cleanup path, the startup-diagnostic path classification, the shared-store CCR retrieval path, and the shared MCP stub import lifecycle - add a `CHANGELOG.md` entry for the user-visible Codex memory scoping fix ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py ======================== 78 passed, 1 warning in 5.96s ======================== Pytest warning: PytestConfigWarning: Unknown config option: asyncio_mode Pytest post-success atexit noise: PermissionError: [WinError 5] Access is denied: 'C:\Users\Rod\AppData\Local\Temp\pytest-of-Rod\pytest-current' uv run ruff check headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py All checks passed! uv run ruff format headroom/cli/wrap.py headroom/memory/mcp_server.py tests/_mcp_stub.py tests/test_ccr_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py tests/test_memory/test_mcp_server.py --check 7 files already formatted ``` ## Real Behavior Proof - Environment: isolated temp project directories, a temp Codex home, the real `wrap codex` and `unwrap codex` CLI commands under pytest, a mocked missing-`codex` launch path for the failed-wrap cleanup case, and shared MCP-SDK stubs for the memory MCP and CCR MCP test modules so CI still exercises those paths without a real `mcp` install. - Exact command / steps: run `uv run pytest tests/test_ccr_mcp_server.py tests/test_memory/test_mcp_server.py tests/test_cli/test_wrap_codex.py tests/test_mcp_stub.py`; prove the persisted config shape with `TestCodexMemoryMcpConfig::test_inject_omits_db_and_replaces_existing_memory_block`; prove prepare-only wrap cleanup with `test_wrap_codex_memory_prepare_only_unwrap_removes_memory_mcp_without_prior_config`; prove failed-wrap cleanup with `test_wrap_codex_memory_launch_failure_unwrap_cleans_memory_only_config`; guard pre-existing named Codex MCP preservation with `test_memory_only_wrap_restores_preexisting_named_mcp_block` and `test_memory_only_wrap_without_backup_preserves_named_mcp_block`; prove the startup diagnostic classifications with `test_memory_mcp_startup_context_reports_dynamic_project_db` and `test_memory_mcp_startup_context_reports_static_external_db`; prove the shared-store CCR retrieval path with `test_mcp_uses_shared_singleton_store` and `test_mcp_retrieves_proxy_stored_content`; prove stub import cleanup with `test_import_module_with_mcp_stub_imports_target_and_cleans_up`, `test_import_module_with_mcp_stub_reimports_target_and_restores_originals`, and `test_import_module_with_mcp_stub_cleans_up_dotted_target_attribute`. - Observed result: the persisted global `headroom_memory` block now keeps `--user` but omits `--db`; prepare-only memory setup still bootstraps the current project's `.headroom/memory.db`; `headroom unwrap codex --no-stop-proxy` now removes both the prepare-only generated config and the failed-wrap memory-only config instead of leaving `[mcp_servers.headroom_memory]` behind; pre-existing named Codex MCP blocks remain restorable across both normal and no-backup memory-only unwrap paths because only wrap-owned markers suppress backups or trigger named-block cleanup; the memory MCP server now logs whether its DB path came from the cwd default or an explicit static path, along with the resolved path and scope it will open; CI can exercise both MCP test modules even when the `mcp` package is absent from the shard environment, and the shared stub now re-imports target modules under the stubbed SDK while restoring both dependency and dotted parent-package target-module import state after cleanup. - Not tested: full end-to-end interactive Codex CLI launch. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The code change stays narrowly scoped to Codex memory config persistence, cleanup, and startup observability. It does not widen into larger memory-routing redesign or startup-failure recovery logic. |
||
|
Parideboy
|
3d59df7be8
|
fix(proxy): forward request-id headers on the streaming path (#1100) (#1258)
## Description On the streaming (SSE) path the proxy rebuilt response headers from a deny-by-default allowlist that only kept rate-limit and Codex headers, so Anthropic's `request-id` was dropped. Claude Code needs that header to write `requestId` into transcripts; without it, usage/cost tools that dedup on `messageId` + `requestId` over-count tokens. This widens the streaming allowlist to also forward the `request-id` family, matching the non-streaming path. Closes #1100 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/proxy/handlers/streaming.py`: widened the streaming response-header allowlist to also forward `request-id`, `anthropic-request-id`, and `x-request-id`. - `tests/test_proxy_streaming_ratelimit_headers.py`: flipped two assertions that expected `x-request-id` to be dropped, and added `test_request_id_headers_forwarded_in_streaming`. ## Testing - [x] Unit tests pass (`pytest`) ### Test Output ```text $ pytest tests/test_proxy_streaming_ratelimit_headers.py -q 11 passed $ pytest tests/ -k "header or stream" -q 75 passed ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, pytest-asyncio 1.4.0 (asyncio_mode=auto) - Exact command / steps: Ran the streaming rate-limit header suite plus adjacent header/stream tests after widening the allowlist. - Observed result: All 11 tests in the targeted file pass including the new request-id forwarding test; 75 adjacent header/stream tests stay green. - Not tested: Did not run a live end-to-end `claude -p` round-trip through the proxy to inspect transcript `requestId`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Rocker Zhang
|
38f1404432
|
fix(cli): fall back gracefully when embedding-server sidecar is absent (#1206)
## Description `headroom proxy --embedding-server` crashes at startup with `ModuleNotFoundError: No module named 'headroom.memory.adapters.watchdog'` instead of falling back to the per-worker embedder. The `EmbeddingServerWatchdog` import sits above the `try/except` that is meant to catch sidecar-startup failures, so a missing sidecar module raises before the guard runs and takes the whole proxy down. The sidecar module is not present on main (it ships with the dedicated embedding-server sidecar work). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Move the `EmbeddingServerWatchdog` import into the guarded `_start_embed_watchdog` coroutine in `headroom/cli/proxy.py`, so a missing sidecar module is caught by the existing `try/except` and the proxy degrades to the per-worker embedder. - Add `tests/test_cli_proxy_embedding_server.py`, a regression test that forces the sidecar module unimportable and asserts the flag falls back instead of crashing. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # The new regression test was validated fail-before / pass-after against the released # build via click's CliRunner (forces the sidecar module unimportable, stubs run_server): # before fix (parent commit): exit_code 1, ModuleNotFoundError, no fallback message # after fix: exit_code 0, no exception, "Falling back to per-worker embedder" # ruff check . and ruff format --check . pass locally on the rebased branch. # Full pytest suite / mypy not run locally; left to CI. ``` ## Real Behavior Proof - Environment: released build (headroom 0.26.0), Linux - Exact command / steps: `headroom proxy --embedding-server --port 8799` - Observed result: the proxy no longer crashes. Before the fix it exits immediately with `ModuleNotFoundError: No module named 'headroom.memory.adapters.watchdog'`; after the fix it logs `WARNING: Failed to start embedding server sidecar: No module named 'headroom.memory.adapters.watchdog'. Falling back to per-worker embedder.`, then prints `URL: http://127.0.0.1:8799` and `Optimization: ENABLED` and serves normally. - Not tested: full pytest suite and mypy locally (left to CI) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Parideboy
|
3ccdad6c67
|
Pin ORT dylib on Windows; init Python logging (#1010)
## Description On Windows, headroom's Rust core resolves `onnxruntime.dll` at runtime via `ort-load-dynamic`. Without an explicit `ORT_DYLIB_PATH`, the bare DLL search can land on `C:\Windows\System32\onnxruntime.dll`, the Windows ML OS component, and `Session::new()` can deadlock instead of returning an error. Since a hang is not an `Err`, the tiered fallback cannot engage until the proxy-level timeout fires. This PR pins `ORT_DYLIB_PATH` to the pip-installed `onnxruntime` DLL at import time, and wires Rust `tracing` events into Python logging so the proxy log surfaces these failures when they occur. Closes #928 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `headroom/_ort.py` with a Windows-only, idempotent `ensure_ort_dylib_pinned()` resolver that respects an existing `ORT_DYLIB_PATH`. - Call the pin from `headroom/__init__.py` before importing `_core` consumers. - Log the effective ORT dylib path from the content router startup path on Windows. - Enable Rust tracing-to-log compatibility and initialize `pyo3-log` in the `_core` module. - Add timeout diagnostics in the Magika detector with the effective `ORT_DYLIB_PATH`. - Document `ORT_DYLIB_PATH` and `HEADROOM_MAGIKA_INIT_TIMEOUT_SECS`. - Add unit coverage for the resolver behavior. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_transforms/test_ort_dylib.py -q`) - [x] Linting passes (`ruff check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py`) - [x] Formatting passes (`ruff format --check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_transforms/test_ort_dylib.py -q 7 passed in 0.19s $ ruff check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py All checks passed! $ ruff format --check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py 4 files already formatted $ cargo check -p headroom-py cargo: The term 'cargo' is not recognized as a name of a cmdlet, function, script file, or executable program. ``` ## Real Behavior Proof - Environment: Windows 11 24H2, Python 3.13, RTX 4080 - Exact command / steps: `python -c "import headroom; from headroom._core import detect_content_type as d; print(d(open('headroom/compress.py').read()).content_type)"` - Observed result: `source_code` in 301ms, clean exit, `Magika: ENABLED` in proxy log - Not tested: macOS/Linux manual runtime behavior; `_ort.py` is a no-op outside Windows, and CI covers cross-platform build/test behavior. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (N/A: repo uses release-please) ## Additional Notes The branch was rebased onto current `main` and the commit subject was updated to satisfy commitlint. Local Rust verification could not be run on this Windows machine because `cargo` is not installed; GitHub CI should be treated as the Rust build verification for the `pyo3-log` dependency and workspace lockfile changes. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Grant McNaught
|
da1a3973ed
|
fix(install): repair macOS launchd restart/start lifecycle (#1290)
## Description Fixes `headroom install restart` and `headroom install start` for macOS launchd `persistent-service` deployments — both currently leave the proxy **stopped**. `restart = stop + start`, but the two halves used incompatible `launchctl` verbs: `stop` runs `launchctl bootout` (which **unregisters** the job from the domain), while `start` only ran `launchctl kickstart -k` (which requires the job to **still be registered**). After `bootout` removes the job, `kickstart` can never find it again (`exit 113`), and nothing ever called `launchctl bootstrap` — so neither a post-`bootout` restart nor a cold `start` could (re)register it. `stop` also used `check=True`, so booting out an already-absent job (`exit 3`) raised and aborted `restart` before it could start again. Closes #1289 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `start_supervisor` (darwin): try `launchctl kickstart -k` first (fast path when the job is already bootstrapped, e.g. right after `install apply` or on a running service); on failure, `launchctl bootstrap` the plist fresh — which also starts it via `RunAtLoad`. - Retry the `bootstrap` for ~15s. launchd returns EIO (`Bootstrap failed: 5: Input/output error`) from `bootstrap` for several seconds after a `bootout` while it releases the label; on exhaustion a `click.ClickException` surfaces the last launchctl error instead of a raw traceback. Tunables: `_MACOS_BOOTSTRAP_RETRIES` / `_MACOS_BOOTSTRAP_RETRY_DELAY`. - `stop_supervisor` (darwin): run `bootout` with `check=False` so an already-absent job (`exit 3`) is treated as already-stopped rather than aborting `restart`. - Tests: 5 new cases in `tests/test_install/test_supervisors.py` (warm `kickstart` success, `bootstrap` fallback when not registered, EIO retry, raise-after-exhaustion, tolerant stop); `time.sleep` is monkeypatched so they stay fast. - `CHANGELOG.md`: entry under Unreleased → Bug Fixes. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_install/ 77 passed, 1 skipped, 1 warning in 5.35s $ pytest tests/test_install/test_supervisors.py -q 19 passed, 1 warning in 0.10s $ ruff check headroom/install/supervisors.py tests/test_install/test_supervisors.py All checks passed! $ ruff format --check headroom/install/supervisors.py tests/test_install/test_supervisors.py 2 files already formatted $ mypy --python-version 3.10 headroom/install/supervisors.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS 26.5.1 (arm64), launchd 7.0.0, headroom installed via pipx; profile `default`, preset `persistent-service`, scope `user`, port 8787. - Exact command / steps: patched the installed `supervisors.py` to this exact code, then exercised the live deployment — `headroom install restart --profile default` (warm restart), `headroom install stop --profile default`, then `headroom install start --profile default` (cold start, post-bootout); health checked via `curl http://127.0.0.1:8787/readyz` and `headroom install status` after each. - Observed result: every transition lands healthy with no traceback (before this PR they failed). `install restart` on a running service → healthy (was: `bootout` exit 3 → abort, proxy down); `install start` cold/post-bootout → healthy in ~8s (was: exit 113 / EIO); `install stop` → down; `install start` from stopped → healthy; 3× rapid `install restart` → all healthy. The EIO settle window was measured directly: `bootstrap` failed with error 5 for ~5s (10 attempts) then succeeded on attempt 11 — which is what the retry loop rides out. - Not tested: system-scope (`/Library/LaunchDaemons`) deployments and the Linux/Windows branches were not exercised on hardware (unchanged by this PR); covered by unit tests only. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI lifecycle change. ## Additional Notes - Docs checkbox left unchecked: no user-facing docs describe the launchd lifecycle internals; happy to add a note if you point me at the right place. - **Tradeoff:** because the correct post-`bootout` recovery has to wait out launchd's ~5s EIO window, `restart` and cold `start` take several seconds. The `kickstart`-first fast path keeps the common already-bootstrapped case instant; only the post-`bootout` path pays the settle. Open to a different shape if you'd prefer (e.g. having `restart` avoid the full `bootout`). - CI-only checks (commitlint, pre-commit `ci-precheck`) were not run locally; the commit header follows conventional commits (`fix(install): …`). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Rocker Zhang
|
5e0bb69725
|
fix(code): verify a real parse in tree-sitter availability check (#1231) (#1299)
## Description `is_tree_sitter_available()` / `_check_tree_sitter_available()` in `headroom/transforms/code_compressor.py` return `True` based on importing `tree_sitter_language_pack` alone, without ever constructing a parser or attempting a parse. When the installed pack/parser combination is ABI-incompatible, `get_parser`/`parse` raises at runtime; the caller catches it and silently falls back to the lossy text compressor, while the availability flag and startup banner still report code-aware as on. This is the defensive half that the `<1.0` pin in #1234 does not cover: if that cap is ever lifted, the availability signal silently lies again. Follow-up to #1231. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Make `_check_tree_sitter_available()` construct a parser and parse a tiny snippet, returning `True` only if it yields a real `module` AST instead of trusting an import. - Add `_tree_sitter_importable()` for the cheap import-only probe, and use it to guard parser construction so the real-parse check cannot recurse. - Add tests asserting the check is `False` when parsing raises and `True` on a real parse, plus that AST compression runs for python/rust without falling back. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # pytest tests/test_transforms/test_code_compressor.py -> passed locally (tree-sitter-language-pack 0.13.0) # ruff check . and ruff format --check . pass locally on the rebased branch. # Full pytest suite / mypy not run locally; left to CI. ``` ## Real Behavior Proof - Environment: local repo on tree-sitter-language-pack 0.13.0, tree-sitter 0.25.2, Python 3.12, Linux - Exact command / steps: call `is_tree_sitter_available()`, then run `pytest tests/test_transforms/test_code_compressor.py` - Observed result: with a working pack the probe parses and returns `True` (code-aware runs, strategy `CODE_AWARE` rather than the kompress fallback); the new `test_check_tree_sitter_available_false_when_parse_broken` confirms that when parsing raises the check now returns `False` instead of the old import-only `True`, so the lossy fallback is no longer entered silently. - Not tested: reproducing the specific ABI-incompatible 1.x pack combo against a live install (covered instead by a mocked broken parse in the test) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable |
||
|
Ian Ker-Seymer
|
cdfeeacc63
|
fix(proxy): preserve Responses memory continuations with store=false (#1103)
## Description Previously, Responses API memory tools could execute successfully but fail on the follow-up request when the client sent `store=false`. Headroom sends memory tool results back with `previous_response_id`, but upstream cannot continue from a response that was not stored. This PR forces `store=true` only when Headroom actually injects Responses memory tools, keeping ordinary `store=false` requests unchanged. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `_ensure_responses_store_for_memory_tools` to make the Responses memory-tool continuation precondition explicit. - Call it only after Responses memory tools are injected. - Added regression coverage for `store=false`, plus no-op coverage for unrelated requests. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ /opt/homebrew/bin/uv run --extra dev pytest tests/test_openai_responses_context_compaction.py -q bind: Invalid command `vi-cmd-mode`. bind: Invalid command `vi-cmd-mode`. ============================= test session starts ============================== platform darwin -- Python 3.12.11, pytest-9.0.3, pluggy-1.6.0 rootdir: /Users/ianks/src/github.com/chopratejas/headroom configfile: pyproject.toml plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 11 items tests/test_openai_responses_context_compaction.py ........... [100%] ======================= 11 passed, 14 warnings in 4.27s ======================== $ /opt/homebrew/bin/uv run --extra dev ruff check headroom/proxy/handlers/openai.py tests/test_openai_responses_context_compaction.py All checks passed! $ git diff --check $ /opt/homebrew/bin/uv run --extra dev mypy headroom headroom/proxy/server.py:1151: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] headroom/proxy/server.py:1221: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] headroom/proxy/server.py:1225: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] Success: no issues found in 374 source files ``` ## Real Behavior Proof - Environment: macOS, `headroom-ai` 0.25.0 local proxy, OpenAI Responses traffic through `http://127.0.0.1:8787/v1` to `https://proxy.shopify.ai`. - Exact command / steps: sent a Responses request with `store=false` asking the model to save `HEADROOM_MEMORY_TEST_MARKER_1781746500`, then sent another `store=false` Responses request asking the model to recall it via memory search. - Observed result: before the local patch, `memory_save` persisted SQLite but continuation failed with `previous_response_not_found`; after the local patch, the same recall path returned `200` and replied `HEADROOM_MEMORY_TEST_MARKER_1781746500 means Headroom memory tools tested pi.` - Not tested: full upstream integration test against the real OpenAI API in CI; this PR covers the payload precondition with unit tests and local proxy manual verification. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated CHANGELOG.md if applicable ## Additional Notes Changelog updated. No docs update; this is a small proxy bug fix. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Terminal Chai
|
b4fde0c3a4
|
fix(wrap): add Copilot unwrap command (#1251)
## Description Adds the missing `headroom unwrap copilot` command so the durable setup created by `headroom wrap copilot` can be removed without touching user-authored Copilot instructions. Closes #1172 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom unwrap copilot` with `--port` and `--no-stop-proxy` options. - Remove only Headroom's marker-fenced RTK block from `.github/copilot-instructions.md`. - Preserve user-authored content and leave malformed/unmatched markers unchanged. - Remove an instruction file that contains only Headroom's generated block. - Update the changelog. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text > .\.venv\Scripts\python.exe -X utf8 -m pytest tests/test_cli/test_wrap_copilot.py -q 30 passed in 1.11s > .\.venv\Scripts\ruff.exe check . All checks passed! > uv run --extra dev ruff format --check headroom/cli/wrap.py tests/test_cli/test_wrap_copilot.py 2 files already formatted > uv run --extra dev mypy headroom/cli/wrap.py Success: no issues found in 1 source file ``` The new command test failed before the implementation with: ```text Error: No such command 'copilot'. ``` ## Real Behavior Proof - Environment: Windows, Python 3.12.12, local editable Headroom checkout. - Exact command / steps: created an isolated project containing user guidance plus a Headroom marker-fenced RTK block, then ran `.venv\Scripts\headroom.exe unwrap copilot --no-stop-proxy`. - Observed result: command exited `0`, printed `Removed Headroom rtk instructions from Copilot.`, and the resulting file contained only `Keep user guidance.`. - Not tested: a live Copilot CLI session or terminating a real proxy process; proxy shutdown delegates to the existing tested unwrap helper and is covered here with a command-level mock. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Not applicable; this is a CLI-only change. ## Additional Notes No dependencies were added. The unchecked comment item is not applicable because the cleanup helper and command are straightforward and documented with docstrings. This pull request includes code written with the assistance of AI. The changes have not yet been reviewed by a human. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Ashish
|
23d73ae070
|
test(evals): add offline fidelity regression gate (recall-based, zero-model) (#1187)
## Description Headroom's lossy compression drops rows/lines using statistical heuristics but **never checks that meaning survived** — a dropped `OOM killed worker 3` line can silently flip a model's answer with no signal that compression caused it. The repo already ships a quality-metric toolkit (`headroom/evals/metrics.py`) and a `weekly-suite` eval job, but neither gates the compression path on a PR. This adds a **per-PR fidelity regression gate**: compress vendored golden tool-outputs through SmartCrusher's lossy path and assert the evidence that answers each case's question survives. It is the first of a planned trio (this is the "offline gate" half of the fidelity work); query-aware retention and a hard token-budget API are documented follow-ups. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Blocking gate** (`tests/test_compression_fidelity_regression.py`): compresses each golden case via `smart_crush_tool_output(..., with_compaction=False)` and scores with `compute_information_recall`. Two assertions: - **Per-case critical recall == 1.0** — every `answer_evidence` string (placed in error/anomaly rows, the documented SmartCrusher retention guarantee) must survive. - **Aggregate recall ≥ committed baseline** (`baseline.json`, tol 0.02) — catches softer regressions. - **Vendored fixtures** (`tests/fixtures/fidelity_golden/`): deterministic `_generate.py` emits `cases.json` (4 cases: OOM crash, payment exception, latency anomaly, CI failure) + `baseline.json`. - **Non-blocking weekly report** (`.github/workflows/eval.yml`): one step in the existing `weekly-suite` job (schedule/manual only) reuses the existing `evaluate_information_retention` runner for a recall report on the production routing path. - **Pure reuse**: scoring (`evals/metrics.py`), compressor (`smart_crush_tool_output`), and the weekly runner (`evaluate_information_retention`) all already existed. ### Design notes - **Zero new CI setup.** The blocking gate runs in the existing `[dev]` test shard — no new workflow, no new deps, **no model, no network, no secrets** (verified under `HF_HUB_OFFLINE=1`). It deliberately uses small hand-made structured fixtures rather than the repo's HuggingFace dataset loaders, which would require a network download + ModernBERT and don't belong in a fast PR gate. - **Scope:** structured JSON tool-output (the dominant, deterministic, model-free case). Real-dataset (HotpotQA/BFCL) recall — which needs `[all]` + a local model — is a **documented follow-up PR**, and the `weekly-suite` job (which genuinely runs every Monday) is its natural home. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ HF_HUB_OFFLINE=1 python -m pytest tests/test_compression_fidelity_regression.py -v tests/test_compression_fidelity_regression.py::test_critical_evidence_survives_compression[logs_oom] PASSED tests/test_compression_fidelity_regression.py::test_critical_evidence_survives_compression[payment_exception] PASSED tests/test_compression_fidelity_regression.py::test_critical_evidence_survives_compression[latency_anomaly] PASSED tests/test_compression_fidelity_regression.py::test_critical_evidence_survives_compression[ci_test_failures] PASSED tests/test_compression_fidelity_regression.py::test_aggregate_recall_not_regressed PASSED ============================== 5 passed in 0.18s =============================== ``` ## Real Behavior Proof - **Environment:** local checkout of `feat/fidelity-regression-gate`, `pip install -e ".[dev]"`, `HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1` (proves no model/network). - **Exact command / steps:** `HF_HUB_OFFLINE=1 python -m pytest tests/test_compression_fidelity_regression.py -q` → `5 passed in 0.14s`. - **Negative control (proves the gate has teeth):** compressing `logs_oom` and probing for a benign row that compression legitimately drops returns `recall = 0.00, lost = ['heartbeat ping 25']` — i.e. the gate fires when critical evidence is dropped, so it is not trivially green. - **Weekly (non-blocking) step verified locally:** ```text Information retention: 50/50 cases >=0.9 recall, avg compression 65.7% ``` - **Not tested:** real-dataset (HotpotQA/BFCL) recall and prose/ModernBERT compression — intentionally deferred to a follow-up PR targeting the weekly job. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - CHANGELOG/version intentionally untouched: repo uses **release-please**. - **Follow-up PR (planned):** wire the real HotpotQA/BFCL loaders (`headroom/evals/datasets.py`) into the `weekly-suite` job for genuine benchmark-scale recall coverage (model-allowed, non-blocking). Further follow-ups from the same design: a live per-request fidelity guardrail, query-aware lossy retention, and a hard `target_tokens` budget API. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
felixboenkost-droid
|
6d116b15f1
|
Harden OpenClaw plugin proxy routing (#1074)
## Description Hardens the bundled OpenClaw plugin so configured proxy routing is fail-closed and `autoStart` is opt-in. Closes: N/A This follow-up is intentionally separate from the ContentRouter cache fix because it changes plugin/gateway behavior rather than core compression routing. The plugin should not mutate upstream provider routing unless a configured proxy URL is reachable and looks like Headroom. It should also avoid unhandled startup promise rejections when proxy startup is fire-and-forget. Why this shape: - `autoStart: false` by default matches deployments where Headroom is supervised externally, for example by systemd. The plugin should not silently start or assume ownership of a proxy unless the operator opted in. - Provider routing is fail-closed: a configured URL must first respond like Headroom, not merely expose a generic liveness endpoint. This prevents accidentally routing model traffic through the wrong local service. - `/readyz` is treated as liveness, not identity. Identity comes from Headroom-shaped stats endpoints (`/v1/retrieve/stats` or `/stats`) because those are harder for unrelated services to satisfy by accident. - Startup remains asynchronous, but errors are captured and exposed instead of becoming unhandled promise rejections. - This is a separate PR because the core cache fix is about compression correctness, while this patch is about integration safety around OpenClaw gateway routing. ## Type of Change - [x] Bug fix (non-breaking change fixes issue) - [ ] New feature (non-breaking change adds functionality) - [ ] Breaking change (fix or feature would cause existing functionality change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Make proxy `autoStart` opt-in (`default: false`). - Probe configured `proxyUrl` before applying provider routing. - Treat `/readyz` as liveness only; require Headroom-shaped `/v1/retrieve/stats` or `/stats` for identity. - Observe fire-and-forget startup promise rejection and expose startup error for callers. - Isolate proxy-ready listener failures. - Keep provider routing deferred when no active/probed Headroom proxy exists. - Register retrieve tool with explicit `headroom_retrieve` name. - Extend plugin/unit tests for configured proxy failures, generic non-Headroom endpoints, path collisions, and routing behavior. Changed files: - `plugins/openclaw/README.md` - `plugins/openclaw/openclaw.plugin.json` - `plugins/openclaw/src/engine.ts` - `plugins/openclaw/src/plugin/index.ts` - `plugins/openclaw/src/proxy-manager.ts` - `plugins/openclaw/test/engine.test.ts` - `plugins/openclaw/test/gateway-config.test.ts` - `plugins/openclaw/test/plugin-runtime-routing.test.ts` - `plugins/openclaw/test/proxy-manager.test.ts` ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added new functionality - [x] Manual testing performed ### Test Output ```text $ npm test Test Files 6 passed (6) Tests 74 passed (74) $ npm run typecheck tsc --noEmit $ npm run build tsup && node prepare-dist.mjs Build success ``` ## Real Behavior Proof - Environment: local OpenClaw plugin package in the Headroom repo. - Exact command / steps: - Run plugin test suite. - Run TypeScript typecheck. - Run plugin build. - Observed result: - Tests passed: `74/74`. - Typecheck passed. - Build passed. - Not tested: - Full OpenClaw Gateway integration as part of this standalone PR prep. ## Review Readiness - [x] I performed self-review - [x] This PR ready for human review ## Checklist - [x] My code follows project's style guidelines - [x] I performed self-review my code - [ ] I commented my code, particularly in hard-to-understand areas - [x] I made corresponding changes documentation - [x] My changes generate no new warnings - [x] I added tests prove fix is effective or feature works - [x] New and existing unit tests pass locally my changes - [ ] I updated CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes Checklist items left unchecked intentionally: - No CHANGELOG update included. - No extra comments were needed beyond existing code structure. Co-authored-by: Björn-Christian Bönkost <bjoern@v2202603344248440850.hotsrv.de> |
||
|
Tejas Chopra
|
723b80c091
|
feat(read-maturation): activity-based hold-back Read maturation (Mechanism B) (#1068)
## What Splits **read maturation (Mechanism B)** out of #818 into its own PR, so #818 can stay focused on the other compression knobs + the SQLite CCR store. Audit-reads (traffic audits) stays in #818. Read maturation holds fresh large `Read` outputs **out of the provider prefix cache** while their file is still active, keeps them verbatim the whole time the model is working with them, and **matures** them into a CCR-backed marker once the file has been quiet for `quiesce_turns`. Only the final compressed form ever enters the cache, so **no cached byte is ever mutated — there is nothing to bust.** Activity-based rather than a fixed hold window: the `audit-reads` simulation showed next-touch gaps are fat-tailed (p50 = 4 turns, p90 = 81), so no fixed window covers the tail while a quiesce rule covers the activity cluster and lets the tail self-heal via partial-range re-reads. **Default OFF** — experimental, flag/env gated, validated in pilots first. ## Changes - `config.ReadMaturationConfig` (`enabled=False`, `quiesce_turns=5`, `max_hold_turns=25`, `min_size_bytes=2048`) - `ProxyConfig` fields mirroring the above - `ReadMaturationManager` transform + `relocate_cache_breakpoint` (`headroom/transforms/read_maturation.py`) - Session-scoped manager rides on `PrefixCacheTracker` — shares the session's cache affinity and TTL cleanup - Handler wiring in `anthropic.py`: runs **after** compression (so `read_lifecycle` markers are respected) and **before** body assembly; advisory — never fails the request - CLI flags + env vars (`--read-maturation*` / `HEADROOM_READ_MATURATION*`) - `_proxy_config_from_env` wiring for both the multi-worker and CLI server paths ## Bug fix included The `--read-maturation` flag was missing `envvar="HEADROOM_READ_MATURATION"`, so the env var was silently ignored on the CLI path (only the multi-worker `_proxy_config_from_env` path read it). Fixed here. ## Tests - `tests/test_read_maturation.py` — unit - `tests/test_read_maturation_handler_nobust.py` — handler never busts cache - `tests/test_live/test_live_maturation.py` — live harness `25 passed` locally; ruff check/format clean; mypy clean (only pre-existing `annotation-unchecked` notes). |
||
|
AKT99!
|
f309244a77
|
feat: add --disable-kompress-fallback to restore legacy PASSTHROUGH fallback (#1185)
## Description Follow-up to #1046. That PR stopped `--disable-kompress` from forcing `fallback_strategy = CompressionStrategy.PASSTHROUGH`, so ContentRouter's rule-based passes keep running when the ML model is off. As noted in review, that is a behaviour change for callers who relied on the old passthrough-everything fallback. This adds an opt-in `--disable-kompress-fallback` flag (env `HEADROOM_DISABLE_KOMPRESS_FALLBACK`) that, together with `--disable-kompress`, restores the previous behaviour by routing fall-through content to `PASSTHROUGH`. It defaults to off, so the corrected behaviour from #1046 is unchanged unless a caller explicitly opts back in. The flag is a no-op unless `--disable-kompress` is also set. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/models.py`: added `disable_kompress_fallback: bool = False` to `ProxyConfig`. - `headroom/proxy/server.py`: when `disable_kompress` and `disable_kompress_fallback` are both set, restore `router_config.fallback_strategy = CompressionStrategy.PASSTHROUGH` (re-adding the `CompressionStrategy` import); wired the new field through the env factory, the `__main__` argparse path (`--disable-kompress-fallback`), and the `/health` config payload. - `headroom/cli/proxy.py`: added the `--disable-kompress-fallback` Click option (with `HEADROOM_DISABLE_KOMPRESS_FALLBACK` envvar) and passed it into `ProxyConfig`. - `tests/test_proxy_disable_kompress.py`: added tests for the flag restoring `PASSTHROUGH`, for it being a no-op without `--disable-kompress`, and for the `/health` config payload exposing the field. - `tests/test_cli_proxy_env.py`: added a test that the env factory honours `HEADROOM_DISABLE_KOMPRESS_FALLBACK`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_proxy_disable_kompress.py -v collected 5 items tests/test_proxy_disable_kompress.py::test_disable_kompress_config_keeps_optimization_but_disables_ml_fallback PASSED [ 20%] tests/test_proxy_disable_kompress.py::test_disable_kompress_defaults_to_existing_kompress_behavior PASSED [ 40%] tests/test_proxy_disable_kompress.py::test_health_config_reports_disable_kompress_fallback PASSED [ 60%] tests/test_proxy_disable_kompress.py::test_disable_kompress_fallback_restores_passthrough PASSED [ 80%] tests/test_proxy_disable_kompress.py::test_disable_kompress_fallback_without_disable_kompress_is_noop PASSED [100%] 5 passed $ ruff check headroom/proxy/server.py headroom/proxy/models.py headroom/cli/proxy.py tests/ All checks passed! ``` ## Real Behavior Proof - Environment: local clone, Python 3.13.7 venv, headroom core deps + fastapi/uvicorn/httpx[http2]. - Exact command / steps: booted the app in-process with FastAPI `TestClient` across four flag combinations and inspected both the live `ContentRouter` config and the `/health` config payload. - Observed result: both flags -> enable_kompress=False and fallback_strategy=PASSTHROUGH (/health reports disable_kompress_fallback=true); --disable-kompress alone -> fallback_strategy stays KOMPRESS (the #1046 default, /health reports false); --disable-kompress-fallback alone -> no-op (enable_kompress=True, KOMPRESS); neither flag -> defaults (enable_kompress=True, KOMPRESS). - Not tested: full live-proxy `/stats` run against a real LLM backend. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes The flag is intentionally a no-op unless `--disable-kompress` is also set, mirroring where the original override lived. Happy to add a short note to the docs/README flag list if you'd like it documented there. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Priyanshu Sharma
|
359004646b
|
fix(langchain): disable streaming on wrapped model during ainvoke() (#1287)
## Description When a wrapped `ChatOpenAI` model is configured with `streaming=True`, calling `ainvoke()` (the non-streaming async API) on the resulting `HeadroomChatModel` crashes with `AttributeError: 'AsyncStream' object has no attribute 'model_dump'`. This happens because `_agenerate()` passes through to the wrapped model's `_agenerate()`, which — when `streaming=True` — returns a raw OpenAI SDK `AsyncStream` object instead of a LangChain `ChatResult`. The caller then tries to call `.model_dump()` on the stream, which doesn't have that method. `_agenerate()` now detects `streaming=True` on the wrapped model and temporarily disables it for the duration of the non-streaming call, then restores it in a `finally` block. Closes #1285 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/integrations/langchain/chat_model.py`: Modified `_agenerate()` to detect `streaming=True` on the wrapped model, temporarily set it to `False` for the duration of the non-streaming call, and restore it in a `finally` block (even on exceptions). Gracefully handles models without a `streaming` attribute or immutable fields. - `tests/test_integrations/langchain/test_chat_model.py`: Added `TestAinvokeStreamingTrue` with 5 test cases covering the core fix, streaming state restoration, exception safety, and passthrough for models without `streaming`. - `CHANGELOG.md`: Added bug fix entry under Unreleased → Bug Fixes. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_integrations/langchain/test_chat_model.py -k TestAinvokeStreamingTrue 5 passed, 39 deselected in 4.14s $ python -m pytest tests/test_integrations/langchain/test_chat_model.py -k "not Ollama and not RealLangChain" 35 passed, 9 deselected in 4.62s $ ruff check headroom/integrations/langchain/chat_model.py tests/test_integrations/langchain/test_chat_model.py All checks passed! $ ruff format --check headroom/integrations/langchain/chat_model.py tests/test_integrations/langchain/test_chat_model.py 2 files already formatted ``` Verification that tests catch the bug (reverted only `chat_model.py`, ran tests): ```text test_agenerate_returns_chatresult_with_streaming_true FAILED assert False = isinstance(<FakeAsyncStream object>, ChatResult) test_streaming_disabled_during_agenerate_call FAILED assert [True] == [False] # streaming was NOT disabled during the call ``` ## Real Behavior Proof - Environment: Linux 6.17.0, Python 3.11.14, langchain-core 1.4.8, pytest 9.1.1, pytest-asyncio 1.4.0 - Exact command / steps: `uv pip install -e ".[dev,langchain]"` then `python -m pytest tests/test_integrations/langchain/test_chat_model.py -k TestAinvokeStreamingTrue` then full module suite with `-k "not Ollama and not RealLangChain"` - Observed result: 5/5 new tests pass, 35/35 existing tests pass, lint clean. Tests fail without the fix (2 failures matching the bug). - Not tested: Real OpenAI API calls (no API key available). Mock-based test simulates `ChatOpenAI`'s streaming behavior faithfully — when `streaming=True`, `_agenerate` returns an `AsyncStream`-like object; when `streaming=False`, it returns a proper `ChatResult`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - `mypy` was not run as it is not part of the local dev dependencies in this environment. The fix is straightforward attribute access with `getattr`/`setattr` and does not introduce new type complexities. - The fix is minimal: `ainvoke()` is the non-streaming API, so it should never trigger streaming. Temporarily disabling `streaming` on the wrapped model is the safest approach — the setting is always restored in a `finally` block. - If `streaming` is an immutable (frozen pydantic) field, the code catches the exception and falls through without crashing. The caller would need to disable `streaming` on the wrapped model directly in that case. |
||
|
Kessy Similien
|
f216e43055
|
fix(mcp): report correct savings_percent in headroom_compress (#1106)
## Description
`headroom_compress` reports `savings_percent` backwards. In
`_compress_content`:
```python
savings_pct = (
round((1 - result.compression_ratio) * 100, 1) if result.compression_ratio < 1.0 else 0
)
```
`compression_ratio` is already the saved fraction (`CompressResult`:
"0.0 = no savings, 1.0 = 100% removed"), so `1 - compression_ratio`
gives the *retained* percentage instead. A no-op comes back as 100% and
a real 71% reduction as 28.8%. The `else 0` branch also zeroes out a
genuine 100% result.
`_Stats.record_compression` a few lines up already does it the right way
(`1 - output_tokens / input_tokens`), so this is just bringing the
return value in line with that.
Closes # (no existing issue — found while evaluating the tool; can file
one if you'd rather track it)
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/ccr/mcp_server.py`: derive `savings_percent` from
`output_tokens`/`input_tokens` like `record_compression` does, so
`savings_percent` and `tokens_saved` can't disagree.
- `tests/test_ccr_mcp_server.py`: regression test tying
`savings_percent` to the token counts, including the no-op-isn't-100%
case.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
Ran the relevant checks against the changed code (borrowed the prebuilt
`_core.abi3.so` from the released wheel so the checkout could import the
pipeline):
```text
$ ruff check headroom/ccr/mcp_server.py tests/test_ccr_mcp_server.py
All checks passed!
$ mypy --ignore-missing-imports headroom/ccr/mcp_server.py
Success: no issues found in 1 source file
$ pytest tests/test_ccr_mcp_server.py -q
.... [100%]
4 passed in 1.71s
```
I ran the ccr test module and lint/type checks on the changed files, not
the whole repo suite (that needs a full Rust build) — CI covers the
rest.
## Real Behavior Proof
- Environment: macOS, Python 3.14, checkout + prebuilt `_core` from
headroom 0.26.0
- Exact command / steps: ran `compress()` on three inputs (a 40-record
JSON array, an incompressible string, repeated prose) and compared the
old `round((1 - compression_ratio) * 100, 1)` against the token-derived
value `(1 - comp/orig) * 100`.
- Observed result: the old expression returns the retained %, so 0%
saved is reported as 100% and a real 71.2% reduction as 28.8%; the new
value matches actual savings in every case:
```text
input orig comp actual old formula new formula
array(40) 497 143 71.2% 28.8% 71.2%
noop 14 14 0.0% 100.0% 0.0%
prose 111 111 0.0% 100.0% 0.0%
```
- Not tested: the full repo test suite and the E2E workflows (need a
complete Rust build / maintainer-approved CI); only the ccr test module
and lint/type checks on the changed files were run locally.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Additional Notes
Docs/CHANGELOG left unchecked as N/A — no user-facing doc covers this
field, though I'm happy to add a CHANGELOG line if you want one.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Parideboy
|
8cc5354f51
|
docs: use headroom-ai package name in install commands (#1014) (#1257)
## Description Install commands across the docs referenced the unpublished `headroom` package instead of the published `headroom-ai`, so copy-pasted `pip install` commands fail. This corrects them to `headroom-ai` (with extras). Closes #1014 ## Type of Change - [x] Documentation update ## Changes Made - `wiki/getting-started.md`: corrected 4 `pip install headroom` commands to `headroom-ai` (including the `[proxy]`, `[relevance]`, and `[all]` extras). - `docs/content/docs/claude-code-vertex.mdx`: fixed the install command on line 37. - `SECURITY.md`: fixed the install command on line 47. ## Testing - [x] Manual verification ### Test Output ```text $ rg -n "pip install headroom\b" docs wiki SECURITY.md (no matches — all bare `headroom` install commands now use `headroom-ai`) ``` ## Real Behavior Proof - Environment: Windows 11, repo working tree on branch fix/docs-1014-headroom-ai-pkg - Exact command / steps: Grepped the docs tree for `pip install headroom` before and after the edits. - Observed result: Before, several occurrences referenced the unpublished `headroom`; after, only `headroom-ai` remains (the spec doc reference is intentionally left untouched). - Not tested: Did not run a live `pip install headroom-ai` against PyPI in CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
JD Davis
|
b829ceba84
|
fix(wrap): keep agent savings opt-in (#1294)
## Description Fixes a regression from #830 where `headroom wrap codex` / `claude` / `cursor` treated `agent-90` as required even when the user started a normal proxy without `HEADROOM_SAVINGS_PROFILE`. A plain `headroom proxy` on port 8787 followed by `headroom wrap codex` currently reports `Proxy on port 8787 is missing: --savings-profile` and tries to restart the already-running proxy. The `agent-90` profile was documented as opt-in, so wrap should only require or inject it when `HEADROOM_SAVINGS_PROFILE` is explicitly set. Closes #1293 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Stop defaulting agent wrappers to `agent-90` when `HEADROOM_SAVINGS_PROFILE` is unset. - Stop reporting agent-savings config mismatches unless an agent savings profile was explicitly requested. - Add regression tests for default wrap startup, explicit profile forwarding, and reuse/restart behavior around existing proxies. - Fix repo-wide pre-commit issues found during amend: Windows-safe `fcntl` typing, an optional env typing issue, OpenCode JSON parser return typing, and ruff import/format drift. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text > maturin develop -m crates/headroom-py/Cargo.toml Finished `dev` profile [unoptimized + debuginfo] target(s) in 27.41s Built wheel for abi3 Python >= 3.10 Installed headroom-ai-0.27.0 > C:\git\headroom\.venv\Scripts\python.exe -c "from headroom._core import hello; print(hello())" headroom-core > ruff check . All checks passed! > ruff format --check . 913 files already formatted > C:\git\headroom\.venv\Scripts\python.exe -m mypy headroom Success: no issues found in 388 source files > C:\git\headroom\.venv\Scripts\python.exe -m pytest tests/test_agent_savings.py tests/test_cli/test_wrap_persistent.py tests/test_proxy_healthchecks.py tests/test_transforms/test_content_router.py -q collected 112 items 112 passed, 1 warning in 16.04s > git diff --check # no output > git commit --amend --no-edit Sync plugin versions.....................................................Passed ruff.....................................................................Passed ruff-format..............................................................Passed mypy.....................................................................Passed ``` ## Real Behavior Proof - Environment: Windows dev checkout, Python 3.13.3 via `C:\git\headroom\.venv\Scripts\python.exe`, Rust extension built with `maturin develop -m crates/headroom-py/Cargo.toml`. - Exact command / steps: reproduced the code path from #830 by exercising `_ensure_proxy(8787, False, agent_type="codex")` with a running proxy health payload that has no `savings_profile`, and by exercising `_start_proxy(8787, agent_type="codex")` with `HEADROOM_SAVINGS_PROFILE` unset and set. - Observed result: without `HEADROOM_SAVINGS_PROFILE`, wrap reuses the running proxy and `_start_proxy` does not inject `HEADROOM_SAVINGS_PROFILE` or `HEADROOM_TARGET_RATIO`; with `HEADROOM_SAVINGS_PROFILE=agent-90`, wrap still requires/forwards the profile and restarts an incompatible proxy. - Not tested: full end-to-end CLI launch against a live Codex binary. The focused proxy/wrap tests cover the failing restart/config decision directly. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes The pytest run emits an existing Windows `cp1252` background-thread warning while reading subprocess output; the tests still pass. No documentation or changelog update is included because this restores the already-documented opt-in behavior for `agent-90`. |
||
|
Parideboy
|
c10969873b
|
feat(cli): add headroom dashboard and surface the dashboard URL (#1277) (#1292)
## Description The savings dashboard is served at `GET /dashboard` (`headroom/proxy/server.py`) but was effectively undiscoverable: there was no `headroom dashboard` command, the `wrap` startup banner only printed `Proxy ready on http://127.0.0.1:PORT` (never the dashboard URL), and the docs buried it — so users on current releases didn't know it existed (#1277). This makes it discoverable from the CLI, the wrap banner, and the docs. Closes #1277 ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `headroom/cli/proxy.py`: new `headroom dashboard` command — prints `http://127.0.0.1:<port>/dashboard` and opens it in a browser (stdlib `webbrowser`); `--no-open` just prints, `--port`/`HEADROOM_PORT` honored. Headless failures are swallowed (URL already printed). - `headroom/cli/wrap.py`: print the dashboard URL alongside "Proxy ready" so every `wrap` surfaces it. - `docs/content/docs/installation.mdx` + `README.md`: document `headroom dashboard`. - `docs/content/docs/mcp.mdx`: document the Codex MCP `command: "headroom"` PATH pitfall (#768) — a project-venv (`uv add`) install isn't on the host's PATH; install globally with `uv tool install` / pipx, or use an absolute path. - `tests/test_cli_dashboard.py`: new tests. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] New tests added for new functionality ### Test Output ```text $ python -m pytest tests/test_cli_dashboard.py -q 3 passed $ python -m ruff check headroom/cli/proxy.py headroom/cli/wrap.py tests/test_cli_dashboard.py All checks passed! ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, branch fix/1277-dashboard-discoverability off headroomlabs-ai/main - Exact command / steps: built the CLI and invoked the new command via the real entry-point import (`from headroom.cli.main import main; main(['dashboard','--no-open','--port','8787'], standalone_mode=False)`) and checked it is registered (`'dashboard' in main.commands`). - Observed result: prints ` Dashboard: http://127.0.0.1:8787/dashboard`, `'dashboard' in main.commands` → `True`, exit 0. The three new tests pass (prints URL + no browser on `--no-open`; opens the URL by default; a raising `webbrowser.open` does not crash the command). - Not tested: did not load the rendered `/dashboard` HTML against a live proxy in CI — the change only adds a launcher/printer for the existing route; the route itself is unchanged. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
wstczyw
|
b514695efd
|
test(proxy): cover enabled periodic TOIN stats startup (#1268)
## Description Follow-up to #1265. Add coverage for the enabled branch of `periodic_toin_stats_enabled` during proxy lifespan startup. The original PR added the opt-out and disabled-path coverage. This test covers the default/enabled path so the new lifespan guard is not left partially covered. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `test_lifespan_schedules_periodic_toin_stats_when_enabled`. - The test patches `_log_toin_stats_periodically` with a short noop coroutine and verifies the proxy lifespan requests it when `periodic_toin_stats_enabled=True`. - This complements the existing disabled-path test from #1265. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_proxy_telemetry_env.py -q ============================= test session starts ============================= platform win32 -- Python 3.11.15, pytest-9.0.3, pluggy-1.6.0 rootdir: C:\Users\wstcz\AppData\Local\Temp\headroom-main-20260622-073524 configfile: pyproject.toml plugins: anyio-4.12.1, langsmith-0.8.0, asyncio-1.3.0, cov-7.0.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function collected 10 items tests\test_proxy_telemetry_env.py .......... [100%] ============================== warnings summary =============================== .venv\Lib\site-packages\fastapi\testclient.py:1 C:\Users\wstcz\AppData\Local\Temp\headroom-main-20260622-073524\.venv\Lib\site-packages\fastapi\testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead. from starlette.testclient import TestClient as TestClient # noqa -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ======================== 10 passed, 1 warning in 2.02s ======================== $ git diff --check origin/main..HEAD # no output; command exited 0 ``` ## Real Behavior Proof - Environment: Windows, Python 3.11.15 uv-managed `.venv`, branch based on current `origin/main`. - Exact command / steps: ran `uv run pytest tests/test_proxy_telemetry_env.py -q`. - Observed result: all 10 tests in `tests/test_proxy_telemetry_env.py` passed, including the enabled periodic TOIN stats lifespan branch. - Not tested: full repository pytest, ruff, and mypy were not run for this test-only follow-up. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes - Test-only follow-up to #1265. - No production behavior changes. - The focused pytest run still emits the existing Starlette/FastAPI TestClient deprecation warning from dependencies, so `My changes generate no new warnings` is intentionally left unchecked. |
||
|
Parideboy
|
a00fb6761e
|
fix(router): degrade to pure-Python detection on native panic (#1123) (#1260)
## Description When the native (Rust) content detector panicked, the pyo3 `PanicException` (a `BaseException`, not `Exception`) escaped `_detect_content` and surfaced as an HTTP 500 instead of degrading. This catches `BaseException` (excluding control-flow exceptions) around the native call and falls back to the pure-Python regex detector, logging a single warning. Closes #1123 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/transforms/content_router.py`: wrapped the native detect call in `_detect_content` so any `BaseException` (except `KeyboardInterrupt`/`SystemExit`/`GeneratorExit`) degrades to `_regex_detect_content_type`, warning once via a module-level `_detect_panic_warned` flag. - `tests/test_transforms/test_detect_fallback_1123.py`: new regression tests for RuntimeError fallback, BaseException-panic fallback, and KeyboardInterrupt propagation. ## Testing - [x] Unit tests pass (`pytest`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_transforms/test_detect_fallback_1123.py tests/test_transforms/test_content_router.py -q 54 passed ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, pytest-asyncio 1.4.0 - Exact command / steps: Monkeypatched the native detector to raise RuntimeError, a BaseException-derived fake panic, and KeyboardInterrupt, then called `_detect_content`. - Observed result: RuntimeError and the BaseException panic both degrade to a valid regex detection result; KeyboardInterrupt still propagates. 54 tests pass. - Not tested: Could not reproduce a real pyo3 panic in this build (`pyo3_runtime` is not importable here), so the fallback is exercised via simulated exceptions. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Parideboy
|
a2159c0b66
|
feat(proxy): support glob patterns in exclude_tools (#870) (#1259)
## Description `exclude_tools` only matched tool names exactly, so users could not exclude families of tools (for example all `mcp__*`). This adds glob-pattern support via a shared `is_tool_excluded` helper used by both the content router and the OpenAI handler, keeping exact/case-insensitive matching intact. Closes #870 ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `headroom/config.py`: added `is_tool_excluded(name, exclude_tools)` helper that keeps exact/case-insensitive matching and adds `fnmatch` glob support. - `headroom/transforms/content_router.py` and `headroom/proxy/handlers/openai.py`: routed tool-exclusion checks through the shared helper. - `headroom/proxy/server.py`: documented glob support in the `--exclude-tools` CLI help and `_parse_exclude_tools` docstring. - `tests/test_transforms/test_content_router.py`: added `test_glob_exclude_tools` and `test_is_tool_excluded_helper`. ## Testing - [x] Unit tests pass (`pytest`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_transforms/test_content_router.py -q 53 passed $ pytest tests/ -k "exclude or config" -q 59 passed ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, pytest-asyncio 1.4.0 - Exact command / steps: Ran the content-router suite and the exclude/config-focused tests after adding the helper and glob support. - Observed result: 53 content-router tests pass (including the two new glob tests) and 59 exclude/config tests pass; glob patterns like `mcp__*` now exclude matching tools while exact names still work. - Not tested: Did not exercise glob exclusion against a live MCP server end to end. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Purva Kandalgaonkar
|
14e8dc4c84
|
feat(learn): weight loops in Headroom Learn + RTK-loop eval (#1160)
## Description `headroom learn` ranked recommendations by a single LLM-guessed `estimated_tokens_saved` with a flat hardcoded `confidence`, and had **no notion of a loop**. So (1) RTK re-fetch loops were invisible - RTK truncates a command's output, the agent re-runs larger-limit variants, those calls *succeed* (`is_error=False`), and `analyze()` even early-returned when a session had no failures and no events - and (2) even when surfaced, a loop ranked no higher than a one-off mistake. This adds loop-aware weighting plus the eval that reproduces an RTK loop, runs it through Learn, and checks the guardrail prevents re-triggering. Closes #1159 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - New `headroom/learn/loops.py`: `detect_loops()` (canonical signature collapses RTK pagination/limit variants; classifies error vs rtk-refetch loops; **measured** wasted tokens), `format_loops_for_digest()`, `apply_loop_weighting()`. - `analyzer.py`: detect loops up front (fixes the no-failure early-return), lead the digest with them, prioritize loops in the system prompt, re-sort after weighting. - `models.py`: `Recommendation.is_loop_guardrail` / `loop_occurrences`. - `benchmarks/rtk_loop_learn_eval.py` + `headroom/learn/fixtures.py`: the two-phase RTK-loop eval and its session fixtures. - Tests, `docs/rtk-loop-weighting.md`, CHANGELOG entry. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - not run (mypy not in my minimal env; see Not tested) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_learn/ -q 190 passed, 3 skipped, 1 warning in 5.85s $ ruff check <changed files> All checks passed! ``` ## Real Behavior Proof - Environment: macOS (Darwin 25.0), Python 3.10.18, fresh venv (`pip install -e` minus the optional `hnswlib`/proxy extras, which are unrelated to `learn`); real LLM via the analyzer's claude CLI backend (`HEADROOM_LEARN_CLI=claude`, claude-cli 2.1.158) — no API key used. - Exact command / steps: `HEADROOM_LEARN_CLI=claude python -c "from benchmarks.rtk_loop_learn_eval import run_eval; c=run_eval(use_real_llm=True); print(c.render())"` - Observed result: the analyzer shelled out to a real model and produced the "Commands" guardrail quoted below, naming the looping command. The digest reports the measured 5,005-token waste and asks the model to rank loops first, so the model emitted that figure; in this run the guardrail ranked **#1** and the scorecard was all-PASS (below). Caveat — real-mode is run-dependent: the rule's wording, and whether the post-hoc `apply_loop_weighting` fuzzy match fires, vary across runs (in one run it did not tag the rule). The **deterministic CI eval** (stub LLM) is the stable, reproducible artifact; this real run corroborates it. - Not tested: the analyzer's API-key path (ANTHROPIC/OPENAI/GEMINI) — exercised the equivalent claude CLI backend instead; `mypy`; a live agent *obeying* the written rule end-to-end (Phase 2 is a non-recurrence check, not a live agent — called out in the doc). Real model output from this run, ranked #1 at the measured 5,005-token weight: > **Commands** — When grepping logs (or any large file), never loop with increasing `| head -N` limits — tool output is capped at ~4 KB regardless of N, so repeated attempts return identical bytes. Instead: redirect to a temp file (`grep ... > /tmp/out.txt`) then read it, or use `grep -c` first… ```text [PASS] loop_detected (1 loop(s), ~5,005 tok wasted) [PASS] guardrail_produced [PASS] ranked_first [PASS] names_command [PASS] prescribes_fix [PASS] weight_reflects_waste [PASS] guardrail_holds RESULT: PASS ``` (One real-mode run via the claude CLI backend. The deterministic `pytest` eval above is the stable artifact; see the run-dependence caveat under Observed result.) The real run also caught an over-brittle check: an earlier `names_command` required the literal "TimeoutError"; the real model wrote a *more general* rule (grep + `head -N`) without it, so I fixed the check to verify the looping **command** is named, not an incidental literal. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - No new dependencies. No network, no user/assistant content dropped — operates on already-captured session digests. - Kept as one logical change. mypy not run locally (minimal env); happy to address anything CI's mypy flags. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Mark Phelps
|
561ba17ec2
|
fix(proxy): build SSL contexts for custom CA bundles (#1134)
## Description Build explicit `ssl.SSLContext` objects for custom CA bundles configured through `SSL_CERT_FILE` and `REQUESTS_CA_BUNDLE`. Python 3.13 / newer OpenSSL can reject some enterprise/private PKI roots that platform TLS stacks accept, for example roots without a `keyUsage` extension. The new custom-CA contexts keep certificate verification enabled while clearing only `ssl.VERIFY_X509_STRICT`. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Build replacement `SSLContext` objects for `SSL_CERT_FILE` and `REQUESTS_CA_BUNDLE` instead of passing raw CA bundle paths to httpx. - Clear only `ssl.VERIFY_X509_STRICT` for operator-provided custom CA contexts; certificate verification, hostname verification, expiry checks, and chain validation stay enabled. - Preserve `NODE_EXTRA_CA_CERTS` additive behavior by loading the extra CA bundle on top of the default/system trust store. - Update existing SSL context tests for replacement CA contexts, env-var priority, missing-path fallthrough, and strict-mode relaxation. - Add an Unreleased changelog entry for the proxy bug fix. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ PYTHONPATH=. .venv/bin/python -m pytest tests/test_ssl_context.py -q collected 12 items tests/test_ssl_context.py ............ [100%] 12 passed, 1 warning in 0.11s ``` ```text $ uv tool run ruff check headroom/proxy/ssl_context.py tests/test_ssl_context.py CHANGELOG.md All checks passed! $ uv tool run ruff format --check headroom/proxy/ssl_context.py tests/test_ssl_context.py 2 files already formatted ``` ## Real Behavior Proof - Environment: macOS, Python 3.13.2, Headroom checkout on this branch, `SSL_CERT_FILE` / `REQUESTS_CA_BUNDLE` / `NODE_EXTRA_CA_CERTS` pointed at an enterprise/private PKI CA bundle. Upstream HTTPS endpoint name is redacted. - Exact command / steps: Ran a Python smoke script that imports `headroom.proxy.ssl_context.find_ca_bundle()`, passes the returned verifier into `httpx.AsyncClient(verify=...)`, and performs a GET against the enterprise HTTPS endpoint that previously failed with OpenSSL strict verification. - Observed result: The request used an `SSLContext`, strict X.509 verification was disabled for that custom CA context, and the request reached the upstream HTTP response: ```text verify_type SSLContext strict_enabled False status 302 ``` - Not tested: Full `uv run pytest`, `uv sync --extra dev`, and `mypy headroom` in this local environment. The editable build currently fails before tests run because Cargo's `ort-sys` build script cannot download ORT prebuilt binaries due an unrelated local certificate verification error against the ORT CDN. No dependency or lockfile changes are included in this PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes No dependencies or lockfiles changed. Documentation is unchanged because this is a bug fix to existing custom CA environment-variable behavior rather than a new user-facing configuration surface. Signed-off-by: Mark Phelps <209477+markphelps@users.noreply.github.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
Ztkent
|
978ffa0a6a
|
feat(savings): durable savings ledger + headroom savings command (#1127)
## Description
Adds a durable, cross-process savings ledger and a `headroom savings`
CLI that shows cost avoided plus Today / Last 7 days / All time
breakdowns by model and client. Unlike `headroom_stats` (a per-session,
in-memory snapshot), the ledger is on disk and survives proxy and agent
restarts, and is safe across the many MCP processes Headroom spawns.
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Add `headroom/savings_ledger.py`: append-only, `fcntl`-locked JSONL
ledger at `~/.headroom/savings_events.jsonl`, safe across concurrent
writers (main MCP server, each subagent, and the proxy), aggregated on
read so totals survive restarts.
- litellm list pricing for known models; blended `$3/1M` input-token
fallback for `model="unknown"` (MCP compressions do not know the
upstream model). Self-pruning: events past the 365-day retention window
are dropped on read and the file is compacted once large.
- Add `headroom savings` CLI (`headroom/cli/savings.py`) with `--json`,
`--days N`, and `--reset` flags.
- Proxy client attribution: `record_request` accepts `client` and
threads `outcome.client` into the ledger, so proxy events record the
real harness (claude-code, codex, cursor, …) from the existing
`classify_client()` detection, falling back to `"proxy"` only when
unidentified.
- MCP compress hook records the client (from `clientInfo.name`) and
tokens saved after each `headroom_compress`; `HEADROOM_MCP_CLIENT` /
`HEADROOM_MCP_MODEL` env overrides.
- Add the `savings_events_path()` helper +
`HEADROOM_SAVINGS_EVENTS_PATH` env in `headroom/paths.py`, the docs page
`docs/content/docs/savings.mdx`, and 15 tests.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
The single warning is a pre-existing, repo-wide
`StarletteDeprecationWarning` from
`fastapi.testclient` (the venv has `httpx`, not `httpx2`); it is
unrelated to this
change and fires in every proxy test that spins up a `TestClient`.
```text
$ .venv/bin/python -m pytest tests/test_savings_ledger.py -q
............... [100%]
15 passed, 1 warning in 5.17s
# warning: fastapi/testclient.py StarletteDeprecationWarning (httpx vs httpx2) — third-party, pre-existing
$ .venv/bin/ruff check headroom/savings_ledger.py headroom/cli/savings.py \
headroom/ccr/mcp_server.py headroom/proxy/prometheus_metrics.py \
headroom/proxy/outcome.py headroom/paths.py tests/test_savings_ledger.py
All checks passed!
$ .venv/bin/mypy headroom/savings_ledger.py headroom/cli/savings.py \
headroom/ccr/mcp_server.py headroom/proxy/prometheus_metrics.py \
headroom/proxy/outcome.py headroom/paths.py
Success: no issues found in 6 source files
```
## Real Behavior Proof
- Environment: macOS (Darwin 25.5.0), Python 3.13.13, editable install
of this branch, proxy running on :8787
- Exact command / steps: route live agent + proxy traffic through
Headroom, then run `headroom savings`
- Observed result: distinct Today / Last 7 days / All time windows with
per-model and per-client breakdowns, as below
- Not tested: Windows runtime (no `fcntl`; the ledger falls back to
best-effort append)
```text
Today ██░░░░░░░░░░░░░░ 11.3% saved 472,870 / 4,193,288 tokens $1.5920
Last 7 days ██░░░░░░░░░░░░░░ 11.9% saved 505,170 / 4,244,288 tokens $1.7385
All time ██░░░░░░░░░░░░░░ 13.0% saved 566,170 / 4,339,288 tokens $1.9815
Cost avoided per model:
claude-sonnet-4-6 $1.2200
claude-opus-4-8 $0.6685
gpt-5.5 $0.0840
claude-haiku-4-5 $0.0090
Savings by client:
claude-code 60 calls · 524,970 tokens saved
cursor 2 calls · 16,800 tokens saved
codex 3 calls · 24,400 tokens saved
```
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The one pytest warning is a third-party `StarletteDeprecationWarning`
from `fastapi.testclient` (pre-existing, repo-wide); not introduced
here. CHANGELOG.md not updated.
|