mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
1553 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tejas Chopra
|
6e4425a6bd
|
feat(wrap): default code-memory to Serena (dashboard browser off) behind unified --code-memory (#2413)
## What
Two commits:
1. **Unify code-memory MCP selection behind `--code-memory
{tokensave|serena|none}`** (+ `HEADROOM_CODE_MEMORY`), collapsing the
`--serena`/`--no-serena`/`--no-tokensave` flag tangle into one selector.
Old flags remain as hidden deprecated aliases that map into it. Shared
across the code-memory-capable subcommands (claude/codex/grok).
2. **Default the engine to Serena**, with its **dashboard browser
suppressed**.
## Why Serena as default
Serena is a mature, offline, symbol-level code-navigation MCP with broad
language coverage (LSP-backed) — the strongest zero-account default for
reducing tokens by letting the agent query
symbols/definitions/references instead of reading whole files. It
attacks the *protected-reads* volume the proxy deliberately doesn't
compress, so it's complementary to the pipeline compressors.
## Browser suppression (in Serena's own settings)
`_ensure_serena_dashboard_disabled()` sets
`web_dashboard_open_on_launch: false` in `~/.serena/serena_config.yml`
when Serena is set up, so wrapped sessions don't spawn a browser tab.
The dashboard backend stays reachable manually at `localhost:24282`.
This lives in Serena's config (authoritative), not just a startup flag.
## Schema-overhead note
Serena injects tool schemas per request; that cost is deferred by the
tool-search deferral the coding profile already enables
(`HEADROOM_TOOL_SEARCH=1`), so tools load on demand — the navigation
benefit without a standing schema tax on turns that don't navigate.
## Selection / escape hatches
`--code-memory serena` (default) · `tokensave` (lighter/faster) · `none`
(disable). Deprecated `--serena`/`--no-serena`/`--no-tokensave` still
work.
## Testing
Updated the primary/backup policy test to the serena-primary default;
code-memory selector + serena disable/migrate tests pass. Local: 21
passed (policy + code-memory); ruff + mypy clean. Full suite in CI.
|
||
|
Tejas Chopra
|
446ec26003
|
feat(transforms): dispatch kompress/text via the compressor registry + forward question (#2411)
## What
Completes the if/elif → registry migration in the content router:
**KOMPRESS and TEXT** now dispatch through the `kompress` built-in
adapter (`_registry_compress`), like every other strategy. Also **fixes
a latent bug** in `_invoke_kompress` that dropped the QA-aware
`question` argument (hardcoded `None`) — `question` now rides
`CompressInput.config['question']` and is forwarded into
`_try_ml_compressor`, so QA-aware compression content is preserved.
## Intentionally NOT byte-identical (one approved change)
The sole behavior change is the KOMPRESS/TEXT **token metric**: reported
`compressed_tokens` is now `_estimate_tokens(output.content)` — the
router's calibrated estimate, consistent with `original_tokens` and
every other registry-dispatched strategy — instead of the Kompress
model's own tuple count. **Compressed content is preserved byte-for-byte
in all paths.**
## Decision-impact analysis (traced every reader of `compressed_tokens`)
No content, routing, keep/drop, fallback, or lossless-then-lossy
decision reads this metric for KOMPRESS/TEXT: they're not in
`fallback_eligible_strategy` nor `{SEARCH,LOG,HTML}`, and the
STAGE-0/general layering calls `_try_ml_compressor` directly (unchanged,
already forwards `question`). The only downstream value-reader is
`_record_to_toin`'s skip gate (`original_tokens <= compressed_tokens`) —
**telemetry/learning only**, never affects returned content or routing,
and arguably more correct now (both sides on the same `_estimate_tokens`
scale). Consciously accepted.
## Tests
Rewrote the PR-C2 deferral-pinning tests →
registry-dispatch-matches-direct (content matches the direct
`_try_ml_compressor(..., question)` call; token assertion switched
`==<model count>` → `==_estimate_tokens(output)`, the only assertion
change, solely due to the approved metric switch). Added a
QA-differential test (question changes content) + an adapter-level
`question`-forwarding test. Offline suite: 96 passed; ruff 0.15.17 +
mypy clean.
**Note:** the full content-router CI suite may require further test
updates for any test that exercises the real KOMPRESS/TEXT branch and
asserts the returned count equals the model's tuple `compressed_tokens`
— those should switch to `_estimate_tokens(output)`. (The broad
content_router/compression selection wasn't run locally — it needs
ONNX/HF.)
After this, the router's per-strategy dispatch is fully
registry-resolved.
|
||
|
Tejas Chopra
|
a986d878b1
|
test: make copilot-flag fixture tolerate headroom not installed (#2407)
## Description
The autouse `_reset_copilot_routing_flag` fixture in `tests/conftest.py`
did an unconditional `from headroom.copilot_auth import
reset_request_routed_to_copilot` for **every** test. That import pulls
in the whole package (`headroom/__init__` → `compress.py` →
`observability` → `opentelemetry`).
The `macos-native-wrapper` and `windows-native-wrapper` CI jobs run
`tests/test_install/test_native_installers.py` with **only `pytest`
installed** (see `.github/workflows/ci.yml` — those jobs `pip install
pytest` and nothing else). Those tests drive the installer shell scripts
via `subprocess` and never import headroom, so the autouse fixture
errored at setup:
```
tests/conftest.py:40: in _reset_copilot_routing_flag
from headroom.copilot_auth import reset_request_routed_to_copilot
headroom/__init__.py:86: from .compress import ...
headroom/compress.py:65: from .observability import get_otel_metrics
headroom/observability/metrics.py:11: from opentelemetry import metrics
E ModuleNotFoundError: No module named 'opentelemetry'
```
Guard the import: when headroom isn't importable there is no routing
flag to reset, so the fixture is a no-op. No production code changes;
behavior is unchanged whenever headroom is installed (all other jobs).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `tests/conftest.py`: wrap the `_reset_copilot_routing_flag` fixture's
`headroom.copilot_auth` import in `try/except ModuleNotFoundError` →
yield-and-return when headroom is absent.
## Testing
- [x] Ran the exact CI command locally
- [x] Linting passes (`ruff check`)
### Test Output
```text
$ ruff check tests/conftest.py
All checks passed!
$ pytest tests/test_install/test_native_installers.py -q
collected 2 items
tests/test_install/test_native_installers.py ss [100%]
============================== 2 skipped in 0.11s ==============================
```
(2 skipped = Docker not available on the local box; the point is **no
more "ERROR at setup"**. Before this change the same run reported `1
error in 0.11s` with the `opentelemetry` traceback above.)
## Real Behavior Proof
- Environment: macOS, Python 3.12, headroom installed (normal path
exercised).
- Exact command / steps: `pytest
tests/test_install/test_native_installers.py -q`
- Observed result: no setup error; fixture takes the normal
(headroom-present) path — 2 tests skipped for lack of Docker.
- Not tested: the headroom-absent branch can't be reproduced locally
(headroom is installed here); it is exactly the CI job's environment,
which this PR's CI run will exercise.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Additional Notes
Scope is intentionally the native-wrapper failures only. The separate
`test-dashboard-ui` red X is unrelated (a stale UI-text assertion:
`element(s) not found — "Completed 128 Failed 0 Rate Limited 0 Cached
96"`) and is not addressed here. Checklist items about
docs/CHANGELOG/new-tests are N/A — this is a test-harness resilience
fix, not a behavior change.
|
||
|
Tejas Chopra
|
7c7bf43057
|
feat(transforms): dispatch smart_crusher via the compressor registry (defer kompress/text ML boundary) (#2404)
## What Final increment of the adapter phase (builds on #2391/#2399/#2400). Flips the **SMART_CRUSHER** primary `.crush()` invocation in `_apply_strategy_to_content` to registry-resolved dispatch, following the CODE_AWARE/HTML pattern. The shared SmartCrusher→Kompress→Log fallback block is unchanged. ## Byte-identical (SMART_CRUSHER) The `smart_crusher` adapter delegates to the same `_get_smart_crusher().crush(content, query=context, bias=bias)` (same cached getter, same method), so `output.content == result.compressed`; the branch recomputes the same `_estimate_tokens` metric; the `if crusher:` guard and the entire fallback chain / `strategy_chain` / `decision_reason` mutations are preserved verbatim. ## Deferred — KOMPRESS and TEXT (honest contract limitation) The `kompress` adapter can't reproduce the direct `_try_ml_compressor(content, context, question)` byte-for-byte, for two independent reasons: 1. **`question` is dropped** — the adapter hardcodes `None`, so QA-aware compression content would diverge. 2. **Token count differs** — the historical branch returns Kompress's own `compressed_tokens` (a word count taken *before* the CCR marker is appended), while the registry path recomputes `_estimate_tokens` over the marker-augmented output. Structurally different numbers whenever Kompress actually compresses. Flipping them would require evolving the adapter/`CompressOutput` contract (forward `question`; carry the compressor's own token count), which is a separate change and would touch the ML boundary — so they're left byte-for-byte here. ## Testing New `tests/test_router_registry_smartcrusher.py`: SMART_CRUSHER success (differential vs a real crush), query/bias forwarding, Kompress-fallback (`[smart_crusher, kompress]`) and Log-fallback (`[smart_crusher, kompress, log]`) chains; plus KOMPRESS/TEXT tests that *pin the deferral facts* (token mismatch + `question` forwarding). Offline suite: 94 passed; ruff (0.15.17) + mypy clean. Full content-router CI suite is the authoritative byte-identical gate. No new config/env. Reversibility gate, external dispatch (#2388), default behavior unchanged. |
||
|
Tejas Chopra
|
7ebda67ef6
|
feat(transforms): add compressed signal + dispatch code_aware/html/diff via registry (#2400)
## What Third increment of the adapter phase (builds on #2391/#2399). Adds a `compressed: bool` field to `CompressOutput` and uses it to flip the **fallback/passthrough** strategies — CODE_AWARE and HTML (and DIFF where clean) — to registry-resolved dispatch, byte-identically. ## The contract addition (the enabling piece) `CompressOutput.compressed: bool = True` — lets a compressor signal **passthrough** (did-not-compress, `content` is the original unchanged) vs a real result. This is what the router's `None`-driven fallback/passthrough branches needed to move to the registry without changing behavior. Default `True`, so existing and external compressors are unaffected. ## How (byte-identical) A new `_registry_compress` helper returns the `CompressOutput` (or `None` when the built-in is unavailable, preserving the `_get_*` guard's passthrough). The flipped branches map that back to their historical `compressed is None` semantics: - **CODE_AWARE:** a passthrough (`not output.compressed` / `None`) sets local `compressed = None`, so the existing `_try_ml_compressor` Kompress fallback + `lossless_then_lossy` no-shrink retry + `strategy`/`strategy_chain` mutations run **verbatim**. - **HTML:** a `None`/passthrough falls through to the bottom passthrough exactly as before (`strategy_chain == [html, passthrough]`). ## Deferred SMART_CRUSHER, KOMPRESS, TEXT, PASSTHROUGH — the SmartCrusher→Kompress→Log fallback chain + the ML boundary — are the next (final) increment, left byte-for-byte here. Reversibility gate, external dispatch (#2388), default behavior unchanged. No new config/env. ## Testing `tests/test_router_registry_dispatch.py` + `tests/test_builtin_compressor_adapters.py` extended: differential tests for CODE_AWARE (success AND None→Kompress-fallback with matching `strategy_chain`, ML mocked), HTML (success AND None→`[html, passthrough]`), and the adapter `compressed=False`-on-None mapping. Offline suite: 88 passed; ruff + mypy clean. The full content-router suite in CI is the authoritative byte-identical gate. |
||
|
Abhay Singh
|
89319fbcad
|
fix(ccr): guard empty/malformed OpenAI choices in _extract_assistant_message (#2389)
## Description
`CCRResponseHandler._extract_assistant_message` extracts the assistant
message from an upstream response while building the CCR
retrieval-continuation history. The OpenAI branch is not defensive about
an empty or malformed `choices` array:
```python
elif provider == "openai":
message = response.get("choices", [{}])[0].get("message", {})
```
`response.get("choices", [{}])` only falls back to `[{}]` when the key
is **absent**. When `choices` is present but empty (`[]`) or carries a
null first element (`[null]`), this raises on the success path:
- `choices: []` → `[][0]` → `IndexError`
- `choices: [null]` → `None.get(...)` → `AttributeError`
OpenAI-compatible gateways can return those shapes on content-filtered
or usage-only responses. The sibling **Google** branch a few lines below
already guards this (`candidates = response.get("candidates", []); if
candidates: ... else: parts = []`), and so does `ccr/tool_calls.py` (it
checks `isinstance(choices, list)`, non-empty, and
`isinstance(first_choice, dict)`). Only this OpenAI branch was missed.
## Fix
Guard the list and the first element the same way the siblings do:
```python
elif provider == "openai":
choices = response.get("choices")
first = choices[0] if isinstance(choices, list) and choices else {}
message = first.get("message", {}) if isinstance(first, dict) else {}
return {
"role": "assistant",
"content": message.get("content"),
"tool_calls": message.get("tool_calls"),
}
```
A well-formed response is unaffected; an empty/null/absent `choices` now
yields `{"role": "assistant", "content": None, "tool_calls": None}`
instead of raising.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/ccr/response_handler.py`: guard empty/non-list `choices` and
a non-dict first element in the OpenAI branch of
`_extract_assistant_message`.
- `tests/test_ccr_response_handler.py`: add
`TestExtractAssistantMessageEdgeCases` (empty `choices`, `[null]`,
absent, and the normal case).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/ccr/response_handler.py tests/test_ccr_response_handler.py
All checks passed!
$ uvx ruff@0.15.17 format --check headroom/ccr/response_handler.py tests/test_ccr_response_handler.py
2 files already formatted
# Verified against the REAL imported module (headroom.ccr.response_handler is
# light — no ML imports), so this ran locally in the project venv:
$ python -c "from headroom.ccr.response_handler import CCRResponseHandler as H; h=H(); \
assert h._extract_assistant_message({'choices': []}, 'openai') == {'role':'assistant','content':None,'tool_calls':None}"
# (no IndexError; normal case still extracts content/tool_calls)
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17`.
- Exact command / steps: imported the real `CCRResponseHandler` and
called `_extract_assistant_message` with `{"choices": []}`, `{"choices":
[null]}`, `{}` (absent), and a normal `{"choices": [{"message":
{...}}]}`.
- Observed result: the OLD code raised `IndexError` on `[]` and
`AttributeError` on `[null]`; the NEW code returns `{"role":
"assistant", "content": None, "tool_calls": None}` for all three
malformed shapes and still extracts `content`/`tool_calls` from a
well-formed response. Because `response_handler` has no ML imports, this
ran against the actual module, not a replica.
- Not tested: a live CCR retrieval round trip through a gateway that
emits empty choices; the added unit tests drive
`_extract_assistant_message` directly.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
`headroom/ccr/response_handler.py` is a light module (no ML imports), so
unlike most of my recent PRs I verified the fix by importing the real
class in the project venv (output above), in addition to the added unit
tests. This aligns the OpenAI branch with the already-defensive Google
branch and `ccr/tool_calls.py`.
|
||
|
Rod Boev
|
d6a1af40d5
|
fix(proxy): skip max_tokens rename for backend-routed openai chat (#2401)
## Description OpenAI-format `POST /v1/chat/completions` requests routed through `--backend litellm-vertex` fail when the client includes `max_tokens`. The proxy currently runs its direct-OpenAI compatibility shim before backend dispatch, renames `max_tokens` to `max_completion_tokens`, then the LiteLLM path no longer recognizes that field as standard and sweeps it into `extra_body`. Vertex rejects the resulting request with `extra_body: Extra inputs are not permitted`. This change scopes the rename shim to the direct OpenAI path only. Backend-routed chat requests now keep `max_tokens`, which LiteLLM already forwards correctly for the Vertex Anthropic path. Direct GPT-5 and o-series compatibility stays unchanged. Closes #2392. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Thread a backend-owned translation flag into `_normalize_openai_max_tokens`. - Skip the legacy-to-completion-token rename on backend-routed OpenAI chat requests. - Keep the direct OpenAI compatibility path covered with a backend-owned translation no-op test. - Add buffered and streaming handler-level regressions for the exact `litellm-vertex` request shape, proving the request survives the `/v1/chat/completions` normalization boundary with vendor fields intact. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/handlers/openai.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py -q ......sss............ [100%] 20 passed, 3 skipped, 1 warning in 42.13s $ uv run ruff check headroom/proxy/handlers/openai.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py All checks passed! $ uv run ruff format headroom/proxy/handlers/openai.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py --check 5 files already formatted ``` ## Real Behavior Proof - Environment: Windows, synced Headroom development environment, mocked LiteLLM provider boundary, no paid GCP credentials required - Exact command / steps: run `uv run pytest tests/test_proxy/test_openai_backend_path.py tests/test_openai_streaming_backend.py tests/test_openai_max_completion_tokens.py tests/test_litellm_openai_passthrough.py -q`, using the issue payload shape `{"model":"claude-sonnet-4-6","max_tokens":32,"messages":[{"role":"user","content":"hi"}],"chat_template_kwargs":{"enable_thinking":false}}` through `POST /v1/chat/completions` - Observed result: buffered and streaming `litellm-vertex` requests keep `max_tokens` as a named backend kwarg, preserve `chat_template_kwargs` in `extra_body`, omit `max_completion_tokens` from `extra_body`, and return success through the handler boundary. Direct-path normalization still renames legacy `max_tokens`. - Not tested: live Vertex AI request ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - `CHANGELOG.md`: N/A, the release pipeline generates it from the conventional-commit subject. - Scope is intentionally narrow: this fixes the exact backend-routed `max_tokens` failure and does not broaden `extra_body` hardening for unrelated OpenAI fields. |
||
|
Rod Boev
|
54526bc858
|
fix(proxy): promote Kompress health after runtime load (#2402)
## Description
`/readyz` can keep reporting Kompress as `{"ready": false, "status":
"unhealthy", "backend": null}` after the live compressor has already
become ready. Startup intentionally records Kompress as `deferred`
without loading the model, `WarmupRegistry.merge_transform_status()`
stores that only as metadata, and the health check later serializes the
stale warmup slot instead of the live runtime compressor state. The
request path can already see the real readiness signal through
`KompressCompressor.is_ready()`, but nothing promotes the health surface
after startup.
This change keeps startup behavior untouched and reconciles Kompress
health from the live compressor right before `/readyz` serializes
component state. It adds side-effect-free runtime backend accessors for
local and remote Kompress implementations, promotes the warmup slot only
when the runtime compressor is ready, preserves loaded state on
transient inspection failures, and keeps Kompress excluded from
aggregate readiness.
Closes #2386
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/transforms/kompress_compressor.py`: add a side-effect-free
`ready_backend()` accessor that returns the cached backend for the
current model or `None`.
- `headroom/transforms/kompress_remote.py`: add `ready_backend()`
returning `"remote"` for the always-ready remote adapter.
- `headroom/proxy/server.py`: derive Kompress health from the live
enabled `ContentRouter` instances, promote the warmup slot only when
runtime readiness is real, respect per-provider re-enable overrides, and
preserve loaded state on transient inspection failures.
- `tests/test_proxy_health.py`: add focused regression, override,
pending, remote, no-instantiation, disabled, fail-open, and
aggregate-readiness coverage.
- `tests/test_kompress_preload_deferral.py`: keep startup-deferral proof
current if a helper needs the new accessor surface.
## Testing
- [x] Unit tests pass (`uv run pytest tests/test_proxy_health.py
tests/test_kompress_preload_deferral.py
tests/test_kompress_request_nonblocking.py -q`)
- [x] Linting passes (`uv run ruff check headroom/proxy/server.py
headroom/transforms/kompress_compressor.py
headroom/transforms/kompress_remote.py tests/test_proxy_health.py
tests/test_kompress_preload_deferral.py`)
- [x] Formatting passes (`uv run ruff format headroom/proxy/server.py
headroom/transforms/kompress_compressor.py
headroom/transforms/kompress_remote.py tests/test_proxy_health.py
tests/test_kompress_preload_deferral.py --check`)
- [x] New tests added for new functionality when applicable
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_proxy_health.py tests/test_kompress_preload_deferral.py tests/test_kompress_request_nonblocking.py -q
................................ [100%]
32 passed, 1 warning in 2.06s
$ uv run ruff check headroom/proxy/server.py headroom/transforms/kompress_compressor.py headroom/transforms/kompress_remote.py tests/test_proxy_health.py tests/test_kompress_preload_deferral.py
All checks passed!
$ uv run ruff format headroom/proxy/server.py headroom/transforms/kompress_compressor.py headroom/transforms/kompress_remote.py tests/test_proxy_health.py tests/test_kompress_preload_deferral.py --check
4 files already formatted
```
## Real Behavior Proof
- Environment: Windows host, local FastAPI test app with the same
`HeadroomProxy`, `WarmupRegistry`, and `/readyz` route used in
production
- Exact command / steps: run `uv run pytest tests/test_proxy_health.py
tests/test_kompress_preload_deferral.py
tests/test_kompress_request_nonblocking.py -q`, covering a deferred
startup slot, a pending resident compressor, a global-disable plus
`disable_kompress_anthropic=False` override, and a router whose lazy
getters would raise if health instantiated them
- Observed result: deferred runtime readiness promotes to `{"enabled":
true, "ready": true, "status": "healthy", "backend": "onnx"}`, a pending
resident compressor stays `{"ready": false, "backend": null}`, a
per-provider override re-enables health even when the global flag is
off, and the health path never instantiates Kompress
- Not tested: live remote Kompress endpoint behavior beyond the local
remote-adapter contract
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have made corresponding changes to the documentation if needed
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
## Additional Notes
- `CHANGELOG.md` stays untouched because Headroom generates changelog
entries from conventional commits.
- Kompress remains a soft component already excluded from aggregate
readiness. This PR fixes only the per-component health report.
- The health path must remain read-only; it must not call `preload()`,
`ensure_background_load()`, `compress()`, or any network or model I/O.
|
||
|
Tejas Chopra
|
fc9c63f18c
|
refactor(transforms): dispatch simple built-in strategies via the compressor registry (#2399)
## What Second increment of the adapter phase (builds on #2391). Flips the content router's per-strategy dispatch in `_apply_strategy_to_content` from the hardcoded if/elif to **registry-resolved** — but only for the *clean, single-compressor* strategies: **SEARCH, LOG, TABULAR, CONFIG**. Each resolves its compressor by name from `compressor_registry` and runs it over the pure-data `CompressInput`/`CompressOutput` contract via a shared `_registry_compress_content` helper, then maps back to the branch's exact historical return shape. ## Byte-identical by construction - The built-in adapter delegates to the SAME `_get_<name>()` getter + method with the same args (`context`→query, `bias`→budget), so returned content is identical to the old direct call. - Each flipped branch **keeps its `enable_*` gate and `_get_*` availability guard** — so the built-in-unavailable → passthrough behavior is preserved and the adapter's `None`→content collapse is never reached. - Each branch **recomputes its token count with its own historical metric** (`_estimate_tokens` for search/log/tabular; `len(split())` for config). - `content_type` in `CompressInput` is inert (built-ins don't consume it), so it can't shift output. ## Deferred (left byte-for-byte as-is) — and why - **CODE_AWARE** — has a Kompress/ML fallback chain (`compressed is None` → `_try_ml_compressor`, plus a `lossless_then_lossy` no-shrink retry) that mutates `strategy`/`strategy_chain`. Not a clean single call. - **HTML** — uses `.extract().extracted` (different shape) and relies on `None` extraction falling through to bottom passthrough (`[html, passthrough]`); the adapter's `None`→content collapse would change the chain. Not byte-identical through the entry. - **SMART_CRUSHER** (fallback chain), **KOMPRESS/TEXT** (ML boundary), **PASSTHROUGH**, **DIFF** — untouched per plan. The reversibility gate, external-compressor dispatch (#2388), and default (nothing-selected) behavior are unchanged. No new config/env. ## Testing New `tests/test_router_registry_dispatch.py` (6 tests): differential test per flipped strategy asserting registry-dispatch output == old direct-dispatch output (content + branch token metric + `[strategy]` chain), plus assertions that deferred SMART_CRUSHER and KOMPRESS are unchanged. Offline suite: 78 passed; ruff + mypy clean. The broad content-router suite (HF-Hub/ONNX) is deferred to CI — **that full suite is the authoritative byte-identical gate for the flipped strategies.** |
||
|
Tejas Chopra
|
981616c60e
|
feat(transforms): make built-in compressors real Compressor implementations (adapters) (#2391)
## What
Turns each built-in registry entry into a working `Compressor` (the
`compressor_registry` contract): `compress(CompressInput) ->
CompressOutput` delegates to the same underlying built-in method the
content router already invokes in `_apply_strategy_to_content`, reached
through the router's own `_get_*` getter so config flows through
identically. Token counts use the router's `_estimate_tokens`;
`lossless` mirrors the descriptor; `recoverable` is `{}` (built-ins
persist CCR recovery to the store as a side effect, not via their return
value).
Adapted: `smart_crusher, code_aware, search, log, tabular, config, html,
kompress`.
## Behavior change
**None — additive by construction.** Dispatch, the `_get_*` getters,
fallback chains, the reversibility gate, and config are all unchanged.
The router still dispatches built-ins via its existing if/elif and never
routes a request through the registry;
`_resolve_active_external_compressors` filters built-in entries out of
the opt-in external-dispatch path *by type* (the class name
`_BuiltinCompressorEntry` is load-bearing). A default request is
byte-identical: `_active_external_compressors == []`, external dispatch
is an inert guard, and adapters are reachable only via
`compressor_registry.get()/active()`.
## `image` — documented passthrough (not a guess)
`ImageCompressor.compress(messages)` operates on image blocks inside
message dicts, not `str` content, and isn't on the
`_apply_strategy_to_content` path, so there's no faithful `str→str`
delegation. Its adapter is a documented non-raising passthrough rather
than a fabricated one.
## Testing
`tests/test_builtin_compressor_adapters.py` — differential tests
asserting each adapter's output matches the built-in's direct output
(JSON→smart_crusher, CSV→tabular, log lines→log, grep→search,
config→config, Python→code_aware, HTML→html); kompress is mocked (no ML
inference); every registry entry has a working non-raising `compress`.
Updated the obsolete guard test in `test_compressor_selection.py`.
Offline suite: 72 passed; ruff + mypy clean. (Broad
content-router/compression suite deferred to CI — it needs HF-Hub/ONNX
model loads.)
This is PR-A of the adapter phase (built-ins become Compressor
implementations); flipping the router's dispatch to registry-resolved is
the follow-up. Builds on #2370/#2371/#2373/#2388.
|
||
|
Abhay Singh
|
6cdfd3f64d
|
fix(proxy/openai): feed chat/completions traffic into the traffic learner (#2333)
## Description Addresses the chat/completions portion of #2060. The live traffic learner is wired into the Anthropic `/v1/messages` handler and, since then, the OpenAI Responses HTTP handler (`_observe_openai_responses_traffic`, called from `handle_openai_responses`). But `handle_openai_chat` has **no** ingestion call site: ```text headroom/proxy/handlers/openai.py handle_openai_responses -> _observe_openai_responses_traffic (wired) handle_openai_chat -> (no traffic_learner call) (gap) ``` So OpenAI-compatible clients that route through `/v1/chat/completions` — GitHub Copilot CLI, opencode, OpenAI SDKs — run through an apparently healthy proxy with Learn enabled while producing no learned patterns: the learner starts, but it never receives their tool results or user messages. ## Fix Observe the original client payload (before memory/compression mutates it) at the top of `handle_openai_chat`, mirroring the Responses and Anthropic ingestion paths: ```python await self._observe_openai_chat_traffic(original_client_messages, request_id=request_id) ``` `_observe_openai_chat_traffic` is the chat counterpart of `_observe_openai_responses_traffic`: same lazy backend wiring, same `on_tool_result` / `on_messages` lifecycle, same fail-soft `try/except`. The one format-specific piece is tool-result extraction. chat/completions encodes tool calls differently from Anthropic — the call is on an assistant message's `tool_calls` array (`id` -> function `name` + `arguments`) and each result is a separate `role: "tool"` message keyed by `tool_call_id`, so the existing `extract_tool_results_from_messages` (which scans for Anthropic `tool_use`/`tool_result` blocks) finds nothing. A new `TrafficLearner.extract_tool_results_from_openai_messages`: - builds the `tool_call_id -> function` map from assistant `tool_calls`; - for each `role: "tool"` message, resolves the tool name and joins string-or-list content; - parses the OpenAI `arguments` JSON string into a dict, so the downstream environment/recovery extractors (which call `input.get("command")`, `input.get("file_path")`, ...) see the same shape as an Anthropic `tool_use.input` instead of a raw string; - sniffs `is_error` from the output (chat tool messages carry no error flag). It returns the same `{tool_name, input, output, is_error}` shape as the Anthropic extractor, so `on_tool_result` stays format-agnostic. User-message preference extraction (`on_messages`) already reads plain `role`/`content`, so it consumes chat messages unchanged. Scope: this wires the **chat/completions** path. Codex WebSocket ingestion (`handle_openai_responses_ws`) additionally needs per-`response.create` evaluation plus transcript-replay baselining on reconnect, so it is intentionally left as a follow-up rather than half-implemented here. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/memory/traffic_learner.py`: add `extract_tool_results_from_openai_messages` (OpenAI chat tool-result extraction with `arguments` JSON parsed to a dict). - `headroom/proxy/handlers/openai.py`: add `_observe_openai_chat_traffic` and call it from `handle_openai_chat` on the original client payload. - `tests/test_memory/test_traffic_learner.py`: cover the OpenAI extractor (name resolution, arguments parsing, list content, error sniff, malformed/orphan handling, empty case). - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/memory/traffic_learner.py headroom/proxy/handlers/openai.py tests/test_memory/test_traffic_learner.py All checks passed! $ uvx ruff@0.15.17 format --check <same files> 3 files already formatted $ uvx mypy@1.20.2 --ignore-missing-imports headroom/memory/traffic_learner.py # clean for this file (the one reported error is a pre-existing # headroom/_subprocess.py:18 no-any-return, unrelated to this change and # present on main with these edits stashed) ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx mypy@1.20.2`. A full `pytest` OOMs this box (ML-stack import), so I reproduced the extractor with a dependency-free script and left the full pytest to CI. - Exact command / steps: replicated `extract_tool_results_from_openai_messages` and ran it over a typical chat round-trip (assistant `tool_calls` for `bash` + `read_file`, then two `role: "tool"` results, one erroring and one with list content), plus malformed-`arguments`, orphan-`tool_call_id`, and no-tool cases. - Observed result: tool names resolved from the call-id map; `arguments` parsed to a dict so `input.get("command")` works; list content joined; `is_error` sniffed from output; malformed arguments degrade to `{}` and an orphan id yields `unknown` without raising. The added unit tests assert the same through a real `TrafficLearner`. - Not tested: a live Copilot CLI session end to end; the added tests drive `TrafficLearner.extract_tool_results_from_openai_messages` directly, matching the existing `test_extract_tool_results_from_messages` pattern. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The "unit tests pass locally" box is unchecked because a local pytest run imports the ML stack and OOMs this box; the added tests reuse the existing `TrafficLearner(backend=None, ...)` harness in `test_traffic_learner.py` (no real backend) and run under the normal CI pytest job, and the extractor behavior is corroborated by the standalone proof above. This PR is deliberately scoped to `/v1/chat/completions`; I'm happy to follow up with the Codex WebSocket ingestion path (which needs the transcript-replay baselining discussed in the issue) as a separate change if useful. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Tejas Chopra
|
e3c7964038
|
feat(proxy): route selected external compressors through the content router (#2388)
## What
Scope 3 of the pluggable-compressor system: a **selected external
`headroom.compressor` plugin now compresses real traffic**. Opt-in via
`--compressor` / `HEADROOM_COMPRESSORS` — external (non-built-in) names
flow to `ContentRouterConfig.active_external_compressors`, resolved once
against the registry in `__init__` (built-in inventory entries filtered
out).
## How
A single guarded branch at the top of `_apply_strategy_to_content`,
immediately before the built-in if/elif. When a selected external
compressor declares the block's detected content type (exact MIME,
`text/*`, or `*` wildcard), the block runs through the pure-data
`Compressor` contract; otherwise it falls through to the built-in path
unchanged.
## Cache safety (by construction)
The branch lives **inside the per-block strategy dispatch**, which only
runs on non-frozen, already-compressible blocks — the frozen/cached
prefix is split off upstream in `apply()`. So a selected external
compressor **can never rewrite cached-prefix content and bust the prompt
cache**; it inherits the exact same cache-preservation the built-ins
have.
## Fail-open + fidelity
Raise, malformed/non-`CompressOutput`, empty-from-non-empty, or
expansion all fall back to the built-in path. Tokens are counted with
the router's own estimator (never the compressor's self-report). Any
`recoverable` (hash→original) map is mirrored to the CCR store like
SmartCrusher, so `/v1/retrieve/{hash}` resolves. Reached only in
lossy/CCR mode (lossless-only sessions return earlier), so it can't
inject unrecoverable loss.
## Behavior change
**None by default.** With no external compressor selected, the branch is
a single cheap guard and everything below is byte-identical to today.
## Testing
`tests/test_router_external_dispatch.py` — end-to-end dispatch of a
selected external compressor, recoverable-map retrievability,
non-hex-hash skip, fail-open on raise/malformed/empty/expansion,
not-selected & non-matching-content-type leave the built-in path
unchanged, wildcard selection. Offline suite: 84 passed (this file +
selection + registry + settings_store). ruff + mypy clean.
Note: the broad content-router/compression suite exercises real HF-Hub
model downloads + local ONNX inference and is slow/flaky in some local
envs — deferred to CI.
Stacks on #2370/#2371/#2373 (all merged).
|
||
|
Serge ARADJ
|
d7a8cdbee1
|
feat(proxy): label GitHub Copilot traffic as "copilot" in the outcome… (#2377)
## Description Requests routed to the GitHub Copilot API travel on the OpenAI or Anthropic wire, so the proxy handlers stamp the *wire* provider (`openai` / `anthropic`) on the outcome. As a result, Copilot traffic is attributed to OpenAI/Claude in the dashboard's per-request provider stats, hiding the real upstream. (This is distinct from the existing **Copilot Quota** panel, which is separate from per-request provider attribution.) This labels Copilot traffic as `copilot` in the single outcome funnel. `build_copilot_upstream_url()` is already the one routing chokepoint every Copilot surface goes through (OpenAI `/chat/completions` + `/responses` and the Anthropic `/v1/messages` route all build their upstream URL there), so it flags the request via a task-local `ContextVar`; `emit_request_outcome()` reads the flag and relabels the provider. The relabel runs before the `>= 500` failed guard, so a failed Copilot request is attributed to `copilot` too. Non-Copilot traffic never sets the flag and is untouched. ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `headroom/copilot_auth.py`: add a task-local `_request_routed_to_copilot` `ContextVar` with `mark_request_routed_to_copilot()` / `request_routed_to_copilot()` helpers; set the flag in `build_copilot_upstream_url()` whenever the base is a Copilot API URL (the existing `is_copilot_api_url` check). `/v1` path normalization is unchanged. - `headroom/proxy/outcome.py`: in `emit_request_outcome()`, when the request was routed to Copilot and the wire provider is `openai`/`anthropic`, relabel the outcome provider to `copilot` (before the 5xx guard). - `tests/test_copilot_provider_label.py`: new tests for the chokepoint marking and the outcome relabel. ## Testing - [x] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) — ran on the changed files only (clean) - [ ] Type checking passes (`mypy headroom`) — ran on the changed files only (clean) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_copilot_provider_label.py tests/test_outcome_records_5xx_as_failed.py -q tests/test_copilot_provider_label.py ..... [ 71%] tests/test_outcome_records_5xx_as_failed.py .. [100%] 7 passed $ python -m pytest tests/test_copilot_auth.py -k "build_copilot_upstream_url or copilot_api_url" -q 8 passed, 58 deselected # existing /v1-stripping behavior preserved $ python -m ruff check headroom/copilot_auth.py headroom/proxy/outcome.py tests/test_copilot_provider_label.py All checks passed! $ python -m mypy headroom/copilot_auth.py headroom/proxy/outcome.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - Environment: Python 3.11, headroom installed with the `proxy` extra. - Exact command / steps: the unit tests above drive `build_copilot_upstream_url()` followed by `emit_request_outcome()` in an isolated context and assert the recorded provider. - Observed result: an `anthropic`/`openai` outcome for a request routed to `https://api.githubcopilot.com` is recorded as provider `copilot`; a request not routed to Copilot is recorded under its wire provider unchanged. - Not tested: end-to-end against a live Copilot subscription (no live seat in the test environment). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes - The flag is a `ContextVar` (task-local), so it cannot bleed across concurrent requests; each request that is not routed to Copilot simply reads the `False` default. - No `CHANGELOG.md` edits (release-please generates it from the Conventional Commit PR title). --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Zhenjia ZHOU
|
8906d3a676
|
fix(cache): preserve client cache_control ttl when consolidating breakpoints (#2382)
## Description
`normalize_message_cache_control()` consolidates message-level
`cache_control` breakpoints (strip all, re-place exactly one) to stay
under Anthropic's 4-block limit. The re-placed marker was hardcoded to
`{"type": "ephemeral"}`, so a client using 1-hour caching
(`cache_control: {"type": "ephemeral", "ttl": "1h"}`) was silently
downgraded to the 5-minute default on every consolidated turn — no
error, no signal, just quietly worse cache economics.
Fix: track the newest client marker while stripping, and re-place **that
marker verbatim** (a copy). Headroom keeps owning *where* the breakpoint
goes; the client keeps owning *what it says*. Older replayed markers
don't win — if the client's newest marker has no `ttl`, we don't
resurrect a stale `1h` (covered by a dedicated regression test).
Fixes #2375.
## Type of Change
- [x] Bug fix (silent 1h→5m cache downgrade)
## Changes Made
- `headroom/cache/prefix_tracker.py`:
`normalize_message_cache_control()` records the last marker dict seen in
message order and re-places a copy of it instead of a hardcoded
`{"type": "ephemeral"}`; docstring documents the ownership split.
- `tests/test_cache_control_move_bust.py`: 3 new tests — ttl preserved,
newest-marker-wins over stale ttls, ttl survives an 8-turn conversation
loop.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff` + `mypy`, CI-pinned settings)
- [x] Reproduced the bug first (2 new tests failed on the old code),
then verified the fix
### Test Output
```text
$ .venv/bin/python -m pytest tests/test_cache_control_move_bust.py -q
10 passed
# Before the fix, the two new ttl tests fail exactly as #2375 describes:
# FAILED ...::test_normalize_preserves_ttl_of_newest_marker
# FAILED ...::test_normalize_ttl_survives_many_turns
$ ruff check headroom/cache/prefix_tracker.py tests/test_cache_control_move_bust.py # All checks passed!
$ ruff format --check <both files> # already formatted
$ mypy headroom/cache/prefix_tracker.py --ignore-missing-imports # Success: no issues
```
## Real Behavior Proof
- Environment: macOS (Darwin), Python in a uv venv, branch
`fix/cache-control-ttl-preserve` off `main` (`56c7d4a5`).
- Exact command / steps: drove `normalize_message_cache_control`
directly with a 2-message conversation whose marker carries `ttl: "1h"`,
printed the re-placed marker before/after the fix, and ran the new
regression tests against the unfixed code first.
- Observed result: before — output marker `{'type': 'ephemeral'}` (ttl
silently dropped); after — output marker `{'type': 'ephemeral', 'ttl':
'1h'}` with marker count still exactly 1 (the ≤4-block guarantee is
untouched).
- Not tested: a live Anthropic round-trip asserting
`cache_creation.ephemeral_1h_input_tokens` (needs a billed API call);
the marker dict forwarded on the wire is what the assertion pins.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation — N/A
(docstring updated)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md — N/A
## Additional Notes
- The `test_normalize_newest_marker_wins_over_stale_ttl` test also
guards against over-fixing (e.g. "any 1h seen anywhere wins"), which
would pin users to 1h pricing after they switch back to the default.
|
||
|
Zhenjia ZHOU
|
f57e959a50
|
fix(wrap): emit bare dotted keys for Codex --config overrides (#2383)
## Description `headroom wrap codex` with a custom provider emits `--config` overrides whose dotted key quotes **every** segment (`"model_providers"."litellm_prod"."base_url"=…`). Codex's override parser matches dotted segments literally and silently ignores quoted ones, so the overrides are dropped, the session keeps the provider's real `base_url`, and traffic **bypasses Headroom entirely** — the exact silent-bypass reported in #2358 on Codex 0.144.5. I reproduced the parser behavior differentially on a local Codex **0.144.1** (see Real Behavior Proof): a bare key is parsed (Codex errors on the injected value), the same key quoted produces no error at all — the override is silently discarded. Fix: `_codex_dotted_key()` now emits segments **bare** whenever they are valid bare keys (`[A-Za-z0-9_-]+`, which covers `model_providers`, every normal provider id, and hyphenated header names like `X-Headroom-Base-Url`), and quotes only segments where bare emission would corrupt the dotted path (e.g. a provider id containing a dot). Fixes #2358. ## Type of Change - [x] Bug fix (silent proxy bypass for custom-provider Codex wraps) ## Changes Made - `headroom/cli/wrap.py`: `_codex_dotted_key()` quotes only non-bare-key segments; docstring explains the observed Codex parser behavior. The default `openai` path (`openai_base_url=…`, already bare) is unchanged. - `tests/test_cli/test_wrap_codex.py`: the custom-provider launch test now pins the bare form for all three overrides (`base_url`, `supports_websockets`, `env_http_headers.X-Headroom-Base-Url`), plus 2 direct unit tests (bare-when-safe incl. hyphens, quote-only-unsafe). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff` + `mypy`, CI-pinned settings) - [x] Reproduced the parser behavior on a real Codex CLI first, then fixed ### Test Output ```text $ .venv/bin/python -m pytest tests/test_cli/test_wrap_codex.py -q 93 passed # Before the fix the updated assertions fail (generated args still fully quoted): # FAILED ...::test_codex_session_launch_settings_preserve_custom_provider_identity # FAILED ...::test_codex_dotted_key_emits_bare_segments_when_safe # FAILED ...::test_codex_dotted_key_quotes_only_unsafe_segments $ ruff check headroom/cli/wrap.py tests/test_cli/test_wrap_codex.py # All checks passed! $ ruff format --check <both> # formatted $ mypy headroom/cli/wrap.py --ignore-missing-imports # Success: no issues ``` ## Real Behavior Proof - Environment: macOS (Darwin), Codex CLI **0.144.1** (`/opt/homebrew/bin/codex`), empty temp `CODEX_HOME` (no auth, no network side effects), branch `fix/codex-config-bare-keys` off `main` (`56c7d4a5`). - Exact command / steps: differential probe of the override parser — `codex exec --skip-git-repo-check -c 'profile="__nope__"' 'hi'` (bare key) vs `codex exec --skip-git-repo-check -c '"profile"="__nope__"' 'hi'` (quoted key), each run once against a fresh empty `CODEX_HOME`. - Observed result: bare key → Codex **parsed the override** and failed fast on it (`Error: legacy profile = "__nope__" config is no longer supported…`); quoted key → **no error referencing the override at all**, Codex proceeded to start a session (banner printed) — the quoted override was silently discarded. That is precisely the #2358 bypass mechanism: every generated custom-provider override was quoted, hence dropped, hence traffic went straight to the real upstream. - Not tested: an end-to-end wrapped session against a live LiteLLM upstream on Codex 0.144.5 (the reporter's exact version); the parser behavior above is version-adjacent (0.144.1) and the argv shape is pinned by unit tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (docstring added) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md — N/A ## Additional Notes - Segments that genuinely need quoting (a provider id with a dot) keep their quotes: on parsers that ignore quoted segments those overrides still won't apply, but bare emission would corrupt a *different* key path, which is strictly worse. Such ids are rare; the common LiteLLM/custom-provider case is fully bare after this fix. |
||
|
Zhenjia ZHOU
|
a84b28af0e
|
fix(proxy): warn when --compressor selection matches no built-in (#2385)
## Description A `--compressor` selection that matches no built-in name (e.g. `smart_krusher`, a typo of `smart_crusher`) silently disables **all** built-in compression: the proxy starts healthy, the dashboard shows ~0 savings, and nothing explains why. The all-off *semantics* is deliberate and stays untouched — `test_only_external_name_disables_all_builtins` pins the opt-in "exactly these" contract, and external/registry names are a legitimate input class. What's missing is any **signal**: a typo and an external compressor name are indistinguishable at this seam, and the registry's own unregistered-name warning (`CompressorRegistry.select`) never runs on this path. Fix: `_apply_compressor_selection` now logs one warning when the selection contains unmatched names — - **nothing matched** (the typo case): says plainly that every built-in compressor is now disabled and lists the valid names + `*`; - **mixed**: names the unmatched entries as assumed registry names. Selection results are byte-identical before/after. Fixes #2384. ## Type of Change - [x] Bug fix (observability for a silent misconfiguration; no behavior change) ## Changes Made - `headroom/proxy/server.py`: `_apply_compressor_selection` computes the unmatched set and emits one `headroom.proxy` warning (two phrasings: nothing-matched vs mixed); docstring updated. - `tests/test_compressor_selection.py`: 3 new tests — typo-only selection warns (and flags stay all-off, pinning the unchanged contract), mixed selection warns only about the unmatched name, matched/wildcard selections stay warning-free. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff` + `mypy`, CI-pinned settings) - [x] Wrote the failing tests first, then the warning ### Test Output ```text $ .venv/bin/python -m pytest tests/test_compressor_selection.py -q 26 passed # Before the fix the two new warning tests fail (no log records emitted). $ ruff check headroom/proxy/server.py tests/test_compressor_selection.py # All checks passed! $ ruff format --check <both> # formatted $ mypy headroom/proxy/server.py --ignore-missing-imports # Success: no issues ``` ## Real Behavior Proof - Environment: macOS (Darwin), Python in a uv venv, branch `fix/compressor-selection-warn` off `main` (`56c7d4a5`). - Exact command / steps: configured stdlib logging at WARNING and called `_apply_compressor_selection(ContentRouterConfig(), {"smart_krusher"})` — the exact typo scenario from #2384. - Observed result: `WARNING headroom.proxy: compressor selection smart_krusher matches no built-in compressor — every built-in compressor is now disabled. If this is a typo, valid names are: code_aware, config, html, image, kompress, log, search, smart_crusher, tabular (or '*' for all).` with `enable_smart_crusher = False` (contract unchanged). Before the fix the same call produced zero log output. - Not tested: a full `headroom proxy --compressor smart_krusher` process launch; the seam is exercised directly and the proxy wires it unconditionally. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A (docstring updated) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md — N/A ## Additional Notes - Deliberately warn-only: erroring here would break legitimate external/registry selections and could brick startup on a stale `HEADROOM_COMPRESSORS` settings value. If you'd rather hard-fail just the CLI-typed path, happy to follow up. |
||
|
Tejas Chopra
|
56c7d4a59e
|
feat(proxy): select built-in compressors via --compressor + registry inventory (#2373)
## What - Adds an opt-in `--compressor` / `HEADROOM_COMPRESSORS` selection that narrows the active built-in compressors, mapped onto the existing `ContentRouterConfig` `enable_*` flags at the proxy config seam. Recognized names: `smart_crusher, kompress, code_aware, search, log, tabular, config, html, image`; `"*"` selects all. - Builds a name-addressable compressor registry in `ContentRouter`: a metadata-only descriptor per built-in plus opt-in discovery of `headroom.compressor` entry points (the seam added in #2370). ## Why Built-in compressors were only reachable through a hardcoded if/elif; there was no supported way to select a subset (7 of the `enable_*` flags had no external surface) or to see the built-ins as a name-addressable set alongside third-party ones. ## Behavior change **None by default.** `--compressor` unset (the default) leaves every `enable_*` flag at its dataclass default, so the request path is byte-identical to today. The registry is inventory-only — built-ins are still constructed and dispatched by the existing if/elif; `_BuiltinCompressorEntry.compress` deliberately raises (never called), and registry construction is fail-open. Routing an external compressor *through* the pipeline is a deliberate follow-up. ## How - `server.py`: `BUILTIN_COMPRESSOR_FLAGS` map + `_apply_compressor_selection(router_config, compressors)` (no-op when `None`/empty; runs before the `disable_kompress` override so that stays authoritative). - `models.py`: `ProxyConfig.compressors: set[str] | None = None`. - `cli/proxy.py`: `--compressor` (repeatable, comma-split, `HEADROOM_COMPRESSORS`), mirroring `--proxy-extension`. - `content_router.py`: built-in descriptors + `_build_compressor_registry()` (register built-ins, then fail-open `discover()`), exposed as `self.compressor_registry`. Dispatch unchanged. ## Testing `tests/test_compressor_selection.py` — 23 tests: selection mapping (None/empty/whitespace = byte-identical defaults, single/multi/wildcard, external-only disables built-ins, unrecognized ignored), `ProxyConfig` field, registry inventory (descriptors cover the 9 names, valid cost tiers, router exposes registry, inventory doesn't auto-activate, built-in `compress` guard, discovery merges external, fail-open on discovery error). Local: 23 passed; ruff + mypy clean on changed files. Full suite runs in CI. Stacks conceptually on #2370 (registry seam); rebased onto `main` after that merged. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Tejas Chopra
|
02eb90f243
|
feat(metrics): record per-extension token savings (#2371)
## What Adds `PrometheusMetrics.record_extension_savings(key, saved)` so proxy extensions can report the tokens they save, and surfaces the per-extension totals in the `/stats` payload. ## Why Proxy extensions that perform their own token reduction currently have no supported way to report their savings to the metrics object — there is no method for it, so that telemetry is silently dropped. This adds the recording method and exposes the aggregate alongside the existing per-strategy compression breakdown. ## How - New `extension_savings: dict[str, int]` counter on `PrometheusMetrics`, populated lazily per extension-supplied `key` (no hardcoded list of extensions). - `record_extension_savings(key, saved)` accumulates positive savings per key, mirroring how `record_compression` aggregates `tokens_saved_by_strategy` (lock-free `defaultdict(int)`, atomic under the GIL for these key types); non-positive values are ignored. - Cleared in `reset_runtime()` with the other in-memory counters. - Surfaced in `/stats` as `extension_savings`, next to `compressions_by_strategy` / `tokens_saved_by_strategy`. No new Prometheus series. ## Behavior change None to existing metrics. ## Testing - Two focused tests in `tests/test_compression_observability.py` (per-key accumulation incl. zero/negative ignored; surfaced in `/stats` via `create_app`) → 2 passed (13 in file). - `ruff check` / `ruff format` → clean; `mypy` → clean on changed source. |
||
|
Tejas Chopra
|
a02073e332
|
feat(transforms): add pluggable compressor registry + headroom.compressor entry point (#2370)
## What Adds a pluggable compressor registry and a `headroom.compressor` entry-point group so compressors can be registered, discovered, and selected by name. - Pure-data contract (`CompressorDescriptor`, `CompressInput`, `CompressOutput`, `Compressor` Protocol). Only plain types (`str`/`int`/`bool`/`list`/`dict`) cross the boundary — no tokenizer, store, or config objects — so the same contract can be implemented outside Python. - `CompressorRegistry`: starts empty and accepts explicit registrations by name; discovers external compressors from the `headroom.compressor` group fail-open (mirrors the existing pipeline-extension discovery); resolves an opt-in selection (`select`/`active`) — nothing active by default, `"*"` for all, otherwise a name allowlist with unknown names logged and skipped. Discovery loads compressors but never invokes `compress`. ## Why Compressors are currently constructed and dispatched via a hardcoded chain in the content router; there is no way to add or select one without editing the router. This lands a name-addressable seam so that becomes possible. ## Behavior change None. Purely additive — not wired into `content_router`, the proxy server, or config, and constructing the registry has no global side effects. Router integration is a deliberate follow-up. ## Testing - `pytest tests/test_compressor_registry.py -q` → 11 passed (contract round-trip, registration, opt-in selection semantics, wildcard, unknown-name skip, monkeypatched entry-point discovery, discovery-never-runs-compress). - `ruff check` / `ruff format` → clean; `mypy` → no issues. |
||
|
Tejas Chopra
|
44136ed042
|
fix(wrap): make RTK opt-in (off by default) across wrap subcommands (#2344)
## Description
RTK CLI-command filtering was set up **by default** across ~16 `wrap`
subcommands (copilot, codex, aider, cursor, cline, continue, goose,
openhands, opencode, grok, omp, openclaude, vibe, …) via `if not
no_rtk:` — so users got rtk hooks / instruction injection without opting
in. `wrap claude` was the lone exception (already gated on
`--context-tool`).
This makes RTK **opt-in (off by default)** everywhere, so Headroom's own
savings are what's measured unless a user explicitly wants rtk.
Closes #
## Type of Change
- [x] Bug fix (behavior change: default flip)
## Changes Made
- **Central gate** `_rtk_opt_in()` — RTK runs only when explicitly
enabled via `--rtk` or `HEADROOM_RTK=1`. Guards the 3 RTK entry points
(`_setup_rtk`, `_ensure_rtk_binary`, `_inject_rtk_instructions`) so they
no-op by default — one small change instead of editing ~30 call sites.
- **`--rtk` opt-in flag** on all 18 tool subcommands via a shared
eager-callback option (`expose_value=False`, sets `HEADROOM_RTK=1`; no
subcommand signature changes).
- `wrap claude`'s legacy `--context-tool` still opts in (mirrored into
the gate).
- **`--no-rtk` kept** as an accepted, now-redundant no-op (back-compat).
- lean-ctx and all non-RTK behavior untouched.
## Testing
```text
pytest tests/test_wrap_rtk_opt_in.py -> 4 passed
ruff check / format -> clean
mypy headroom -> Success: no issues found in 504 source files
```
Verified `--rtk` appears in `wrap {claude,codex,copilot} --help`;
`_rtk_opt_in()` is False by default, True with `HEADROOM_RTK=1`; entry
points no-op + write nothing when off.
## Real Behavior Proof
- Env: local `.venv`, click CliRunner.
- Steps: import wrap; assert gate default-off / env-on; assert
`_setup_rtk`/`_ensure_rtk_binary` return None and
`_inject_rtk_instructions` returns False + writes no file when not opted
in; assert `--rtk` in subcommand help.
- Observed: all pass. Not tested: a live end-to-end wrap launch (proxy
spawn).
## Notes
Part 2 of 3 (RTK opt-in). Separate PRs cover the code-graph MCP engine
and the proxy-option cleanup. No `CHANGELOG.md` edit — release-please
generates it from the PR title (per the changelog guard).
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Abhay Singh
|
1b8c11ebfb
|
fix(proxy/openai): apply output shaping on /v1/chat/completions (#2328)
## Description Fixes #2302. Output shaping (`HEADROOM_OUTPUT_SHAPER=1`) verbosity steering is wired into the Anthropic `/v1/messages` handler and the OpenAI `/v1/responses` handler, but never into `handle_openai_chat`. OpenAI-compatible clients that route through `/v1/chat/completions` — GitHub Copilot CLI, opencode, older SDKs — therefore got zero output savings, and `headroom output-savings` reported: ``` No shaped requests recorded yet. ``` `handle_openai_chat` referenced verbosity only for cache-key construction, never for actual shaping. The shared helpers (`OutputShaperSettings`, `resolve_verbosity_level`, `assign_arm`, `classify_turn`) existed but were not called from the chat path. ## Fix Run the same shaping block the Anthropic handler already uses, at the end of `handle_openai_chat` (after every other body mutation, before the upstream forward, skipped under `x-headroom-bypass`): - conversation-stable holdout via `assign_arm(conversation_key_from_body(body), holdout)` — `conversation_key_from_body` already reads `messages`, so it works unchanged for a chat body; - stratum labelling on the transforms channel so the outcome funnel feeds the output-savings ledger from the chat path; - for the treatment arm, verbosity steering via a new `shape_openai_chat_request`. The one genuinely new piece is a chat-specific steering injector. Anthropic carries the system prompt in a top-level `system` field and Responses in `instructions`; **chat/completions carries it as a `role: "system"` message inside `messages`**, which neither existing injector touches. `apply_openai_chat_verbosity_steering`: - appends the byte-stable steering block to the tail of the last `system`/`developer` message (idempotent via the `<headroom_output_shaping>` sentinel, and it swaps cleanly when the level changes); - handles both string content and the content-part list form (`[{"type": "text", ...}]`); - inserts a `role: "system"` message at the front only when the request has no system message. Because a whole conversation is stably treatment or control and the block text is fixed per level, a treatment conversation's steering is byte-stable across turns, so the provider prefix cache is not thrashed. Effort routing is intentionally not applied on this path — `route_effort` writes Anthropic-shaped `output_config`/thinking config with no portable chat/completions equivalent — so only the token-reducing verbosity lever runs. Mutating `body` in place is enough on this path; the outbound request serializes `body` fresh, so no body-mutation tracker is needed. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/output_steering.py`: add `apply_openai_chat_verbosity_steering` (inject the steering block into the chat `messages` system prompt). - `headroom/proxy/output_shaper.py`: add `shape_openai_chat_request` (verbosity-only chat shaper) and export both new names. - `headroom/proxy/handlers/openai.py`: run the holdout/stratum + shaping block at the end of `handle_openai_chat`, mirroring the Anthropic handler and respecting bypass. - `tests/test_output_steering.py`: cover the injector (append, idempotency, level swap, insert-when-absent, list content, level-0 no-op). - `tests/test_output_shaper.py`: cover `shape_openai_chat_request` (disabled no-op, applies steering, level override, stable second pass). - `CHANGELOG.md`: Bug Fixes entry. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uvx ruff@0.15.17 check headroom/proxy/output_steering.py headroom/proxy/output_shaper.py headroom/proxy/handlers/openai.py tests/test_output_steering.py tests/test_output_shaper.py All checks passed! $ uvx ruff@0.15.17 format --check <same files> 5 files already formatted $ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/output_steering.py headroom/proxy/output_shaper.py Success: no issues found in 2 source files ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx mypy@1.20.2`. A full `pytest` OOMs this box (ML-stack import), so I reproduced the injector with a dependency-free script and left the full pytest to CI. - Exact command / steps: replicated `apply_openai_chat_verbosity_steering` (and the `steering_text`/`replace_or_append_steering_block` primitives it uses) and exercised: an existing string system message, an existing content-part list, no system message, re-apply at the same level, and a level swap. - Observed result: the steering block is appended to the system message while user turns and message order are untouched; re-applying at the same level is a no-op; a level change replaces the block (exactly one remains); a request with no system message gets one inserted at the front; level 0 is a no-op. The added unit tests assert the same through `shape_openai_chat_request`. - Not tested: a live Copilot CLI `/v1/chat/completions` round trip; the added tests drive the pure shaper/injector directly, matching the existing `test_output_shaper.py` / `test_output_steering.py` patterns. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes The "unit tests pass locally" box is unchecked because a local pytest run imports the ML stack and OOMs this box; the added tests are pure (no ML imports) and run under the normal CI pytest job, and the injector behavior is corroborated by the standalone proof above. Effort routing on chat/completions is deliberately out of scope here (no portable equivalent to the Anthropic effort levers); this PR restores the verbosity-steering savings the issue reports as missing, and effort routing for chat can follow separately if wanted. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Rod Boev
|
cf5fa644b6
|
fix(wrap): stop same-port persistent routing during claude unwrap (#2340) (#2350)
## Description `headroom unwrap claude` currently removes Claude-local wrap state but can still leave Claude effectively routed through Headroom when the same port belongs to a managed persistent deployment. The command already knows how to discover same-port persistent manifests, but its stop path only kills the current pid and never uses that deployment metadata. This patch keeps the existing local settings cleanup, then applies an ownership-aware same-port audit: Claude-owned deployments are stopped through the install lifecycle path, while ambiguous same-port residue is surfaced with exact remediation instead of a false clean-success claim. Refs #2340. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - extend the Claude unwrap stop path in `headroom/cli/wrap.py` to distinguish local pid stops, Claude-owned same-port persistent deployments, and ambiguous same-port residue - reuse the install lifecycle teardown path for Claude-owned persistent deployments instead of re-implementing supervisor cleanup - keep the existing Claude-local settings, hook, and base-url cleanup unchanged - add focused CLI regressions that prove a matching Claude-owned deployment is stopped during unwrap, ambiguous same-port residue is reported truthfully, and different-port manifests stay untouched ## Testing - [x] Unit tests pass (`uv run pytest tests/test_cli/test_unwrap_claude.py tests/test_cli/test_wrap_persistent.py -q`) - [x] Linting passes (`uv run ruff check headroom/cli/wrap.py tests/test_cli/test_unwrap_claude.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text uv run pytest tests/test_cli/test_unwrap_claude.py tests/test_cli/test_wrap_persistent.py -q ============================= 43 passed in 0.51s ============================== uv run pytest tests/test_cli/test_unwrap_claude.py -q ============================= 13 passed in 0.41s ============================== uv run pytest tests/test_cli/test_wrap_persistent.py -q ============================= 30 passed in 0.39s ============================== uv run ruff check headroom/cli/wrap.py tests/test_cli/test_unwrap_claude.py All checks passed! uv run ruff format headroom/cli/wrap.py tests/test_cli/test_unwrap_claude.py --check 2 files already formatted ``` ## Real Behavior Proof - Environment: Windows worktree `D:\Repos\headroom-pr-2340-claude-unwrap-effective-routing` with `uv sync --extra dev` - Exact command / steps: run the focused pytest and Ruff commands above, then run a constructed `CliRunner` replay against both `D:\Repos\headroom` and this branch with the same same-port Claude-owned manifest harness - Observed result: base prints `base: exit=0; local=[8787]; deactivated=[]; stopped=[]` and still routes through the pid-only helper; head prints `head: exit=0; local=[]; deactivated=['unwrap-2340']; stopped=['unwrap-2340']` and reports `Stopped Claude-owned persistent deployment 'unwrap-2340' on port 8787.` - Not tested: a live macOS launchd deployment on this host ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` is not applicable because Headroom derives release notes from conventional commits. Scope stays below the broader uninstall workflow in open PR `#749`: this patch makes Claude unwrap truthful and ownership-aware, but it does not remove install artifacts or introduce a new uninstall command. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Abhay Singh
|
b75999017f
|
fix(transforms/kompress-remote): keep compress fail-open on malformed 200 (#2320)
## Description
`RemoteKompressCompressor` (the opt-in `HEADROOM_KOMPRESS_ENDPOINT`
remote compression client) documents a fail-open contract in its own
docstring:
> Fails OPEN: any network/HTTP error returns the content verbatim so a
flaky endpoint degrades compression rather than breaking the proxy.
But only the network call and the `compressed` field check actually run
inside the fail-open guard. The metadata coercions run **after** the
`except`, outside it:
```python
try:
resp = self._client.post(...)
resp.raise_for_status()
data = resp.json()
compressed = data["compressed"]
if not isinstance(compressed, str):
raise TypeError("...")
except Exception as e: # fail OPEN
logger.warning("Remote Kompress failed (%s); passing through", e)
return self._passthrough(content, n_words)
result = KompressResult(
compressed=compressed,
original=content,
original_tokens=int(data.get("original_tokens", n_words)),
compressed_tokens=int(data.get("compressed_tokens", len(compressed.split()))),
compression_ratio=float(data.get("compression_ratio", 1.0)), # <-- outside the guard
model_used=str(data.get("model_used", self.config.model_id)),
)
```
So a hosted `/compress` endpoint that returns a 200 with a valid
`compressed` string but a malformed metadata field escapes the guard and
raises out of `compress`, breaking the proxy request instead of passing
through. The most realistic trigger is an explicit JSON `null`:
`data.get("compression_ratio", 1.0)` returns `None` for a **present**
key (the default only applies to a missing key), and `float(None)`
raises `TypeError`. A non-numeric string like `"original_tokens":
"lots"` raises `ValueError` the same way. Since the whole point of the
flag is to support arbitrary self-hosted endpoints, a slightly-off but
well-meaning endpoint (sending `null` for a field it could not compute)
takes down the request path this class exists to protect.
## Fix
Move the response parsing (the `KompressResult` construction with its
`int`/`float`/`str` coercions) inside the fail-open `try`, so any
malformed field degrades to verbatim passthrough like every other
bad-response case:
```python
try:
...
compressed = data["compressed"]
if not isinstance(compressed, str):
raise TypeError("...")
result = KompressResult(
compressed=compressed,
original=content,
original_tokens=int(data.get("original_tokens", n_words)),
compressed_tokens=int(data.get("compressed_tokens", len(compressed.split()))),
compression_ratio=float(data.get("compression_ratio", 1.0)),
model_used=str(data.get("model_used", self.config.model_id)),
)
except Exception as e: # fail OPEN
logger.warning("Remote Kompress failed (%s); passing through", e)
return self._passthrough(content, n_words)
```
No behavior change on a well-formed response; only the malformed-200
path changes (raise to passthrough).
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/transforms/kompress_remote.py`: move the `KompressResult`
construction and its field coercions inside the fail-open `try`.
- `tests/test_transforms/test_kompress_remote.py`: add
`test_remote_kompress_null_numeric_field_fails_open` (explicit JSON
`null`) and `test_remote_kompress_non_numeric_field_fails_open`
(non-numeric string), both asserting verbatim passthrough.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/transforms/kompress_remote.py tests/test_transforms/test_kompress_remote.py
All checks passed!
$ uvx ruff@0.15.17 format --check headroom/transforms/kompress_remote.py tests/test_transforms/test_kompress_remote.py
2 files already formatted
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/transforms/kompress_remote.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOMs this box (ML-stack import), so I
reproduced the control flow with a dependency-free script and left the
full pytest to CI.
- Exact command / steps: modeled the OLD (coercions outside the `try`)
and NEW (inside the `try`) parsing against a 200 body `{"compressed":
"short result", "compression_ratio": null}` and against a well-formed
body.
- Observed result: OLD raised `TypeError` on the null field (proxy
request breaks); NEW returned passthrough; a well-formed body still
compressed under NEW. The added tests assert both malformed cases
(`null` and non-numeric string) return the original content with
`compression_ratio == 1.0`.
- Not tested: a live remote Kompress endpoint; the added tests drive
`RemoteKompressCompressor` through an `httpx.MockTransport`, matching
the existing test harness in this file.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because a local pytest
run imports the ML stack and OOMs this box; the added tests reuse the
existing `httpx.MockTransport` harness in `test_kompress_remote.py` and
run under the normal CI pytest job, and the behavior is corroborated by
the standalone proof above.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Abhay Singh
|
44a174fef4
|
fix(backends/litellm): guard None completion_tokens in usage mapping (#2322)
## Description
`_anthropic_usage_from_litellm` maps a LiteLLM `Usage` object to the
Anthropic response shape on the buffered (non-streaming) backend path.
Every numeric field is `None`-guarded with `int(... or 0)` except
`output_tokens`:
```python
cache_read = int(getattr(litellm_usage, "cache_read_input_tokens", 0) or 0)
cache_write = int(getattr(litellm_usage, "cache_creation_input_tokens", 0) or 0)
...
prompt_tokens = int(getattr(litellm_usage, "prompt_tokens", 0) or 0)
usage: dict[str, Any] = {
"input_tokens": max(prompt_tokens - cache_read - cache_write, 0),
"output_tokens": getattr(litellm_usage, "completion_tokens", 0), # <-- no guard
}
```
The `getattr(..., 0)` default only fires when the attribute is
**absent**. LiteLLM's `Usage` is a pydantic model that always carries
`completion_tokens`, so the default never applies; when a provider
leaves the value `None`, `output_tokens` becomes `None`.
That `None` then propagates:
- `LiteLLMBackend.complete_message` builds the Anthropic-shaped body
with `"usage": usage`.
- The buffered anthropic-backend handler reads `output_tokens =
usage.get("output_tokens", 0)` (again, a present key returns its `None`
value, not the default) and passes it to
`RequestOutcome(output_tokens=...)`, whose field is declared `int`.
- The outcome-recording path does arithmetic on it, e.g. Prometheus
`self.tokens_output_total += output_tokens`, which raises `TypeError:
unsupported operand type(s) for +=: 'int' and 'NoneType'`.
So a provider that returns usage with a `None` completion count breaks
metrics recording for that request on any `--backend litellm` /
Bedrock/Vertex deployment.
## Fix
Guard the field the same way as its three siblings, so the mapping
always emits an `int`:
```python
"output_tokens": int(getattr(litellm_usage, "completion_tokens", 0) or 0),
```
No change for the normal case (an integer count passes through
unchanged); only a `None` (or absent) value now becomes `0` instead of
`None`.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/backends/litellm.py`: `None`-guard `output_tokens` in
`_anthropic_usage_from_litellm`.
- `tests/test_litellm_nonstream_cache_usage.py`: add
`test_output_tokens_none_coerced_to_zero` asserting a `None` completion
count maps to `int` `0`.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/backends/litellm.py tests/test_litellm_nonstream_cache_usage.py
All checks passed!
$ uvx ruff@0.15.17 format --check headroom/backends/litellm.py tests/test_litellm_nonstream_cache_usage.py
2 files already formatted
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/backends/litellm.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOMs this box (ML-stack import), so I
reproduced the field logic with a dependency-free script and left the
full pytest to CI.
- Exact command / steps: modeled the OLD (`getattr(..., 0)`) and NEW
(`int(getattr(..., 0) or 0)`) field derivations for a usage object with
`completion_tokens=None`, an integer, and the attribute absent, then
simulated the downstream `total += output_tokens`.
- Observed result: OLD produced `None` for the `None` case and the
downstream `+=` raised `TypeError`; NEW produced `0`/`7`/`0`
respectively and the `+=` succeeded. The added unit test asserts
`usage["output_tokens"] == 0` and `isinstance(..., int)`.
- Not tested: a live LiteLLM/Bedrock request that returns a `None`
completion count; the added test drives `_anthropic_usage_from_litellm`
directly with a `SimpleNamespace`, matching the existing tests in this
file.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because a local pytest
run imports the ML stack and OOMs this box; the added test uses the same
`SimpleNamespace`-driven, dependency-light pattern as the neighbouring
tests in `test_litellm_nonstream_cache_usage.py` and runs under the
normal CI pytest job, and the behavior is corroborated by the standalone
proof above.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Abhay Singh
|
f64aac9733
|
fix(proxy/gemini): None-guard token counts from usageMetadata (#2347)
## Description
The non-streaming Gemini/Vertex handler reads token counts straight from
the response's `usageMetadata`:
```python
try:
usage = resp_json.get("usageMetadata", {})
total_input_tokens = usage.get("promptTokenCount", optimized_tokens)
output_tokens = usage.get("candidatesTokenCount", 0)
cache_read_tokens = usage.get("cachedContentTokenCount", 0)
except (...):
...
uncached_input_tokens = max(0, total_input_tokens - cache_read_tokens) # OUTSIDE the try
```
`.get(key, default)` only falls back when the key is **absent**. When
`usageMetadata` carries a key with a **null** value — which Gemini can
do on a safety-blocked turn that produced no candidates — `.get` returns
`None`. That `None` then reaches:
- `max(0, total_input_tokens - cache_read_tokens)` (a `None - int` →
`TypeError`), and
- `RequestOutcome(output_tokens=...)`, whose field is `int` and which
the metrics recorder increments (`tokens_output_total += output_tokens`
→ `TypeError`).
Both run on the success (non-`except`) path, so a single such response
crashes the request and its outcome recording. The Gemini streaming path
already guards these with a `_usage_int` helper; the non-streaming path
(two sites) did not.
## Fix
Coerce the three counts with `int(... or fallback)`, matching the
streaming `_usage_int` guard and the LiteLLM usage mappings:
```python
total_input_tokens = int(usage.get("promptTokenCount", optimized_tokens) or optimized_tokens)
output_tokens = int(usage.get("candidatesTokenCount", 0) or 0)
cache_read_tokens = int(usage.get("cachedContentTokenCount", 0) or 0)
```
No change for a normal integer usage; only a `None` (or absent) value
now becomes the fallback/0. Applied to both non-streaming
usage-extraction sites in `handlers/gemini.py`.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/gemini.py`: `int(... or fallback)`-guard
`promptTokenCount` / `candidatesTokenCount` / `cachedContentTokenCount`
at both non-streaming usage sites.
- `tests/test_proxy/test_gemini_savings_profile.py`: add a regression
driving a `generateContent` request whose
`usageMetadata.candidatesTokenCount` is `null`, asserting a 200, an
`int` `output_tokens == 0`, and `uncached_input_tokens == 20` (the
`max(0, …)` no longer raises).
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/gemini.py tests/test_proxy/test_gemini_savings_profile.py
All checks passed!
$ uvx ruff@0.15.17 format --check headroom/proxy/handlers/gemini.py tests/test_proxy/test_gemini_savings_profile.py
2 files already formatted
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/gemini.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOMs this box (ML-stack import), so I
reproduced the extraction with a dependency-free script and left the
full pytest to CI.
- Exact command / steps: modeled the OLD (bare `.get`) and NEW (`int(...
or fallback)`) derivations for a blocked response
(`candidatesTokenCount: null`, valid prompt count), a null
`promptTokenCount`, a normal response, and an absent-usage response.
- Observed result: OLD raised `TypeError` at `max(0, None - …)` for a
null prompt count and left `output_tokens = None` (which crashes the
int-typed outcome/metrics recorder) for a null candidate count; NEW
produced `(20, 0)` for the blocked case, `(15, 0)` for the null-prompt
case (the `optimized_tokens` fallback), `(60, 30)` for a normal
response, and the fallbacks for absent usage. The added
`create_app`/`TestClient` test drives the handler end to end and asserts
a 200 with `int` outcome counts.
- Not tested: a live Gemini safety-blocked response; the added test uses
a mocked `_retry_request` returning a `usageMetadata` with a null count,
matching the existing Gemini test harness in this file.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because a local pytest
run imports the ML stack and OOMs this box; the added test reuses the
existing `create_app`/`TestClient` + mocked-`_retry_request` harness in
`test_gemini_savings_profile.py` and runs under the normal CI pytest
job, and the behavior is corroborated by the standalone proof above.
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Gautam Sharma
|
793d20fb2a
|
fix(subscription): read newest transcript tail (#2310)
## Description
Large Claude Code transcript files were capped by reading the first 10
MB of each append-only JSONL file. Because recent entries are appended
at the end, current-window and weighted token usage could silently omit
the newest activity.
Oversized transcripts are now read from EOF. If the capped tail begins
within a JSONL record, only that partial record is discarded. A
complete record beginning exactly at the boundary remains included.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation-only change
- [ ] Refactoring
## Changes Made
- Read the newest capped transcript bytes instead of the oldest prefix.
- Determine the tail offset using the opened file handle.
- Inspect the preceding byte to distinguish a partial record from an
exact line boundary.
- Remove partial bytes before UTF-8 decoding.
- Preserve existing behavior for transcripts below the 10 MB cap.
- Add direct session-tracking tests for aggregation and boundary
behavior.
- Add an Unreleased changelog entry.
## Testing
- [x] Added regression tests
- [x] Focused tests pass
- [x] Subscription test suite passes
- [x] Ruff checks pass
- [x] Mypy passes
- [x] Changed files pass formatting checks
- [ ] Entire repository test suite passes without baseline failures
Commands and results:
- `uv run --extra dev --frozen pytest
tests/test_subscription_session_tracking.py -q`
- `4 passed`
- Subscription-focused suite
- `53 passed`
- `uv run --extra dev --frozen ruff check .`
- Passed
- `uv run --extra dev --frozen mypy headroom --ignore-missing-imports`
- Success across 504 source files
- Changed-file Ruff formatting
- Passed
- `uv run --extra dev --frozen pytest -q`
- `9364 passed, 565 skipped, 4 failed`
- The four existing, unrelated failures are:
- `test_l2_appends_transform_label`
- `test_recovery_records_sockets_and_secures_both_backups`
- `test_dashboard_uses_cached_stats_and_lazy_history_feed_polling`
- `test_smart_crusher_log_fallback_runs_for_valid_json`
Repository-wide `ruff format --check .` identifies pre-existing
formatting drift only in the untouched
`headroom/proxy/handlers/anthropic.py`.
## Real Behavior Proof
A focused reproduction created a 10,485,787-byte transcript with a
marker entry appended after the 10 MB boundary.
Before the fix:
```text
{'file_bytes': 10485787, 'line_count': 1, 'newest_entry_present': False}
After the fix:
{'file_bytes': 10485787, 'line_count': 1, 'newest_entry_present': True}
The regression tests additionally verify that:
1. Recent token usage beyond the cap contributes to raw and weighted
totals.
2. A partial initial JSONL record is discarded.
3. A complete record exactly at the tail boundary is preserved.
4. Small transcripts retain their existing behavior.
Environment: macOS arm64, CPython 3.12.13.
Not tested: mutation of the transcript during the individual file read
by a live Claude Code process. Reads remain bounded to a single recent
snapshot.
## Review Readiness
- [x] I have performed a self-review before requesting human review.
- [x] This PR is ready for human review.
## Checklist
- [x] The change follows existing project style and error-handling
conventions.
- [x] Tests cover the reported failure and relevant boundary cases.
- [x] The 10 MB memory/read cap remains enforced.
- [x] No unrelated files or formatting changes are included.
- [x] No temporary logging or debug code remains.
- [x] The changelog has been updated.
## Additional Notes
The four full-suite failures listed above occur outside the modified
subscription code and are unrelated to this PR. All tests covering
transcript reading and subscription tracking pass.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Gautam Sharma
|
0924755591
|
fix(memory): serialize MCP backend initialization (#2309)
## Description
The Memory MCP server previously assigned its backend before
asynchronous embedder and vector-index warm-up completed. A tool call
arriving
during the handshake could therefore receive a partially initialized
backend.
Backend initialization is now atomic and shared between concurrent
callers. The backend is published only after warm-up succeeds. Failed
candidates are closed and discarded so later calls can retry with a
fresh backend.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation-only change
- [ ] Refactoring
## Changes Made
- Keep the initializing backend local until warm-up completes
successfully.
- Share one initialization task between handshake and concurrent tool
calls.
- Await the shared task before exposing the backend to tool handlers.
- Shield shared initialization from cancellation by an individual tool
caller.
- Close failed or cancelled backend candidates.
- Clear failed initialization state so subsequent calls can retry.
- Retrieve and log background initialization failures.
- Add regression tests for handshake races, failure recovery, and
concurrent initialization.
- Add an Unreleased changelog entry.
## Testing
- [x] Added regression tests
- [x] Focused test suite passes
- [x] Ruff checks pass
- [x] Mypy passes
- [x] Changed files pass formatting checks
- [ ] Entire repository test suite passes without baseline failures
Commands and results:
- `uv run --extra dev --frozen pytest
tests/test_memory/test_mcp_server.py -q`
- `12 passed`
- `uv run --extra dev --frozen ruff check .`
- Passed
- `uv run --extra dev --frozen ruff format --check
headroom/memory/mcp_server.py tests/test_memory/test_mcp_server.py`
- Passed
- `uv run --extra dev --frozen mypy headroom --ignore-missing-imports`
- Success across 504 source files
- `uv run --extra dev --frozen pytest -q`
- `9363 passed, 565 skipped, 4 failed`
- The four failures are existing, unrelated failures outside the changed
code:
- `test_l2_appends_transform_label`
- `test_recovery_records_sockets_and_secures_both_backups`
- `test_dashboard_uses_cached_stats_and_lazy_history_feed_polling`
- `test_smart_crusher_log_fallback_runs_for_valid_json`
Repository-wide `ruff format --check .` also identifies pre-existing
formatting drift in the untouched
`headroom/proxy/handlers/anthropic.py`.
## Real Behavior Proof
The regression tests exercise the affected lifecycle directly:
1. Start backend initialization through the MCP handshake.
2. Suspend warm-up before it completes.
3. Issue a memory tool call and verify its handler is not invoked.
4. Release warm-up and verify the tool receives the initialized backend.
5. Force background initialization to fail and verify the candidate is
closed.
6. Issue another tool call and verify initialization retries with a
fresh backend.
7. Start two tool calls concurrently and verify only one backend is
constructed.
Observed behavior:
- Tool calls remain pending while handshake warm-up is incomplete.
- A partially initialized backend never reaches a tool handler.
- Failed candidates are closed and discarded.
- A later tool call successfully retries initialization.
- Concurrent calls share one initialization task and backend.
Environment: macOS arm64, CPython 3.12.13.
Not tested: a live stdio MCP client using the real ONNX model and
database. The affected initialization lifecycle is covered with
deterministic asynchronous regression tests.
## Review Readiness
- [x] I have performed a self-review before requesting human review.
- [x] This PR is ready for human review.
## Checklist
- [x] The implementation follows the repository’s existing style and
error-handling conventions.
- [x] Tests cover the reported race, concurrent initialization, and
failure recovery.
- [x] Failed initialization does not leave a partially published
backend.
- [x] Failed backend candidates are closed before retry.
- [x] No unrelated files or formatting changes are included.
- [x] No temporary logging, debug code, or commented-out code remains.
- [x] Public behavior changes are documented in the changelog.
- [x] The branch has been rebased from the intended base and is ready
for review.
## Additional Notes
The four full-suite failures listed above occur outside the changed
Memory MCP code and are unrelated to this PR. All tests covering the
modified initialization lifecycle pass.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
6decbd1e6e
|
fix(proxy/streaming): preserve non-standard content-block fields on SSE reconstruction (#2271)
## Description
When the proxy reconstructs a full response from an Anthropic SSE
stream, it silently drops the payload of any content block that isn't
`text` / `tool_use` / `thinking` / `redacted_thinking`.
`_parse_sse_to_response` builds each block on `content_block_start`:
```python
current_block = {"type": btype, "index": block_index}
if btype == "text":
current_block["text"] = block.get("text", "")
elif btype == "tool_use":
current_block["id"] = block.get("id")
current_block["name"] = block.get("name")
current_block["input"] = {}
elif btype == "thinking":
...
elif btype == "redacted_thinking":
...
blocks_by_index[block_index] = current_block
```
There's no branch for other block types. A `server_tool_use` or
`web_search_tool_result` block (Anthropic server-side tools) therefore
reconstructs as a bare `{"type": ..., "index": ...}`, losing its `id`,
`name`, `input`, and content.
This reconstructed response is what `has_memory_tool_calls` and the CCR
feedback recorder inspect, so a stream that used a server-side tool
feeds detection a gutted block. The sibling reconstructor
`_reconstruct_anthropic_response` (in
`headroom/ccr/response_handler.py`) already handles this correctly with
`elif btype: current_block = dict(block)` — this path just wasn't
updated.
## Fix
Add an `elif btype:` branch that copies through all of the block's
fields (except `type`, already set), mirroring the sibling:
```python
elif btype:
for _k, _v in block.items():
if _k != "type":
current_block[_k] = _v
```
Standard blocks are untouched; non-standard blocks keep their fields.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/streaming.py`: add the non-standard-block
field copy in `_parse_sse_to_response`'s `content_block_start` handler.
- `tests/test_sse_thinking_blocks.py`: new test asserting a
`server_tool_use` block keeps `id` / `name` / `input`.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/handlers/streaming.py tests/test_sse_thinking_blocks.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/streaming.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the block-construction logic with a dependency-free script
and left the full pytest to CI.
- Exact command / steps: ran a `server_tool_use` content_block_start
through the OLD (special-cases only) and NEW (`elif btype:` copy) logic,
plus a `text` block as a control.
- Observed result: OLD produces `{"type": "server_tool_use", "index":
0}` (id/name/input gone); NEW keeps `id`/`name`/`input`; the `text`
block is identical under both.
- Not tested: a live server-tool stream end-to-end; full local `pytest`
deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new test reuses the
existing `_Parser(StreamingMixin)` harness in
`tests/test_sse_thinking_blocks.py`, so it runs under the normal CI
pytest job; behaviour is additionally verified by the standalone proof
above.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
1612f06a4c
|
fix(ccr): don't crash tool-call detection on a null function/functionCall (#2269)
## Description
CCR tool-call detection crashes when an upstream response carries a tool
call whose `function` (or `functionCall`) field is explicitly `null`.
`is_ccr_tool_call` and `parse_tool_call` both read the nested name like
this:
```python
tool_call.get("function", {}).get("name")
tool_call.get("functionCall", {}).get("name")
```
`dict.get("function", {})` only substitutes `{}` when the key is
**missing**. When the key is present but `null` — `{"id": "call_1",
"type": "function", "function": null}`, which upstreams (and gateways
like LiteLLM/OpenRouter) emit for a partial or streamed tool call — the
result is `None`, and `None.get("name")` raises `AttributeError`.
These functions run over the untrusted upstream response
(`has_ccr_tool_calls` → `is_ccr_tool_call` for every tool call, and
`parse_tool_call` on the retrieve path), so a single malformed tool call
takes down CCR detection for the whole response. The sibling
`tool_call_id_for_provider` in the same module already guards this shape
(`if isinstance(function_call, dict)`); these two paths just weren't
updated to match.
## Fix
Coalesce with `or {}` so a `null` (or any falsy) value collapses to
`{}`:
```python
(tool_call.get("function") or {}).get("name")
(tool_call.get("functionCall") or {}).get("name")
```
and in `parse_tool_call`:
```python
function = tool_call.get("function") or {}
function_call = tool_call.get("functionCall") or {}
```
A null tool call now reports "not a CCR call" and is passed through as a
normal tool, and real CCR calls are still detected.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/ccr/tool_calls.py`: `is_ccr_tool_call` coalesces `function`
/ `functionCall` with `or {}`.
- `headroom/ccr/tool_injection.py`: `parse_tool_call` coalesces
`function` (openai) and `functionCall` (google) with `or {}`.
- `tests/test_ccr_tool_calls.py`, `tests/test_ccr_tool_injection.py`:
new tests covering a null-function tool call in detection and parsing.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/ccr/tool_calls.py headroom/ccr/tool_injection.py tests/test_ccr_tool_calls.py tests/test_ccr_tool_injection.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/ccr/tool_calls.py headroom/ccr/tool_injection.py
Success: no issues found in 2 source files
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the detection logic with a dependency-free script and left
the full pytest to CI.
- Exact command / steps: ran an OpenAI tool call `{"function": null}`
(plus a real CCR call) through the OLD `get("function", {})` form and
the NEW `get("function") or {}` form.
- Observed result: OLD raises `AttributeError` on the null function; NEW
returns `False`/`None` for it and still detects the real CCR call and
both `functionCall`/`name` shapes.
- Not tested: a live upstream emitting a null-function tool call; full
local `pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new tests live
alongside the existing CCR tool-call tests so they run under the normal
CI pytest job; behaviour is additionally verified by the standalone
proof above.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
8b7e797ed4
|
fix(proxy/memory): don't crash memory tool-call detection on a null function (#2272)
## Description
`MemoryHandler` crashes when an upstream response carries a tool call
whose `function` field is explicitly `null`.
Three sites read the nested name like `tool_call.get("function",
{}).get("name")`:
- `has_memory_tool_calls` (line ~1043) — over the response's tool calls.
- `handle_tool_calls` (line ~1110/1119) — resolving the tool name and
arguments.
- the memory tool-injection dedup (line ~562) — over the request's
tools.
`dict.get("function", {})` only substitutes `{}` for a *missing* key. A
present-but-null `{"id": "c1", "type": "function", "function": null}` —
a shape upstreams and gateways emit for a partial or streamed tool call
— makes the result `None`, and `None.get("name")` raises
`AttributeError`.
`has_memory_tool_calls` and `handle_tool_calls` both iterate the
untrusted upstream response, so a single malformed tool call takes down
memory tool-call detection and handling for the whole response.
## Fix
Coalesce `function` with `or {}` at all three sites, so a null (or any
falsy) value collapses to `{}`:
```python
name = tc.get("name") or (tc.get("function") or {}).get("name")
args_str = tc.get("arguments") or (tc.get("function") or {}).get("arguments") or "{}"
```
Real tool calls resolve exactly as before.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/memory_handler.py`: coalesce `function` with `or {}`
in `has_memory_tool_calls`, `handle_tool_calls`, and the tool-injection
dedup.
- `tests/test_memory_handler_null_function.py`: new tests that a
null-function tool call doesn't crash detection and the real memory call
is still seen.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/memory_handler.py tests/test_memory_handler_null_function.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/memory_handler.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the name-resolution logic with a dependency-free script and
left the full pytest to CI.
- Exact command / steps: ran a `{"function": null}` tool call (plus a
real `memory_save` call) through the OLD `get("function", {})` and NEW
`get("function") or {}` name resolution.
- Observed result: OLD raises `AttributeError` on the null function; NEW
returns `None` for it and still resolves the real `memory_save` name and
a plain `{"name": "memory"}`.
- Not tested: a live upstream emitting a null-function tool call; full
local `pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. `has_memory_tool_calls`
and `_extract_tool_calls` use no instance state, so the new test
exercises them on a bare instance via `object.__new__` — it runs under
the normal CI pytest job, and the standalone proof above corroborates
it. This is the memory-handler sibling of the same null-`function`
hazard I'm fixing in the CCR tool-call detection and the memory tool
adapter.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
ef1e7e403b
|
fix(proxy/memory): don't crash the memory tool adapter on a null function/arguments (#2270)
## Description
The memory tool adapter crashes when an upstream response carries a tool
call whose `function` or `arguments` field is explicitly `null`.
`_get_tool_name`, `_get_tool_id`, and `_get_tool_input` all read the
nested function like this:
```python
str(tool_call.get("function", {}).get("name", ""))
tool_call.get("function", {}).get("arguments", "{}")
```
Two distinct crashes:
1. **Null `function`** → `AttributeError`. `dict.get("function", {})`
only substitutes `{}` for a *missing* key. A present-but-null `{"id":
"c1", "type": "function", "function": null}` (which upstreams and
gateways emit for partial/streamed tool calls) makes the result `None`,
and `None.get("name")` raises.
2. **Null `arguments`** → `TypeError`. `tool_call.get("function",
{}).get("arguments", "{}")` returns `None` when `arguments` is null, and
`json.loads(None)` raises `TypeError` — which the surrounding `except
json.JSONDecodeError` does **not** catch.
Both parse the untrusted upstream response inside `handle_tool_calls`,
so a single malformed tool call takes down memory tool handling. Notably
`parse_tool_call` in `headroom/ccr/tool_injection.py` already catches
the `json.loads(None)` `TypeError` with an explicit comment, so the
null-arguments hazard is known in the codebase; this path just wasn't
hardened.
## Fix
- Coalesce `function` / `functionCall` with `or {}` so a null value
collapses to `{}`.
- Coalesce the arguments string with `or "{}"` and add `TypeError` to
the `except`, so a null `arguments` yields `{}` instead of crashing.
Real tool calls parse exactly as before.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/memory_tool_adapter.py`: coalesce
`function`/`functionCall` (`or {}`) in
`_get_tool_name`/`_get_tool_id`/`_get_tool_input`; coalesce the
arguments string (`or "{}"`) and catch `TypeError`.
- `tests/test_memory_tool_adapter_null_fields.py`: new tests for null
function, null arguments, and that real calls still parse.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/memory_tool_adapter.py tests/test_memory_tool_adapter_null_fields.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/memory_tool_adapter.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the parse helpers with a dependency-free script and left the
full pytest to CI.
- Exact command / steps: ran `{"function": null}` and `{"function":
{"arguments": null}}` (plus a real `memory_save` call) through the OLD
and NEW `_get_tool_name`/`_get_tool_input` logic.
- Observed result: OLD raises `AttributeError` on the null function and
`TypeError` on the null arguments; NEW returns `""`/`{}` for both and
still parses the real call to `{"content": "hi"}`.
- Not tested: a live upstream emitting a null field; full local `pytest`
deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The parse helpers read
only their `tool_call` argument (no instance state), so the new test
exercises them on a bare instance via `object.__new__` — it runs under
the normal CI pytest job, and the standalone proof above corroborates
it.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
eed80dd4ba
|
fix(learn/claude): don't abort the whole scan on a null message line (#2299)
## Description
A single Claude session-log line with an explicit `{"message": null}`
crashes the entire `headroom learn` run.
`ClaudeCodePlugin._scan_session` reads the message object in four
places:
```python
usage = d.get("message", {}).get("usage", {}) # assistant line
...
msg = d.get("message", {}) # _extract_tool_uses
msg = d.get("message", {}) # _extract_tool_results
msg = d.get("message", {}) # _extract_user_events
```
`dict.get("message", {})` only substitutes `{}` for a **missing** key. A
present-but-null `{"type": "assistant", "message": null}` yields `None`,
and `None.get(...)` raises `AttributeError`.
The per-file guard only catches I/O errors:
```python
try:
with open(jsonl_path, ...) as f:
for line in f:
...
except (OSError, UnicodeDecodeError) as e:
...
return None
```
so the `AttributeError` propagates out of `_scan_session`, past
`scan_project` (which has no try/except around the scan), and aborts the
whole `learn` invocation — every project, not just the one bad line. One
malformed line takes down the entire run.
## Fix
Coalesce the message with `or {}` at all four sites, so a null (or any
falsy) value collapses to `{}`:
```python
usage = (d.get("message") or {}).get("usage", {})
msg = d.get("message") or {}
```
The malformed line is now skipped and scanning continues; valid lines
are parsed exactly as before.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/learn/plugins/claude.py`: coalesce `d.get("message")` with
`or {}` in `_scan_session` and the three `_extract_*` helpers.
- `tests/test_learn/test_subagent_scanning.py`: new test that a session
containing `{"message": null}` lines scans without crashing and still
parses the valid tool call.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/learn/plugins/claude.py tests/test_learn/test_subagent_scanning.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/learn/plugins/claude.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the per-line handling with a dependency-free script and left
the full pytest to CI.
- Exact command / steps: ran an assistant line `{"message": null}` (plus
a real assistant line and a missing-message line) through the OLD
`get("message", {})` and NEW `get("message") or {}` logic.
- Observed result: OLD raises `AttributeError` on the null message; NEW
returns `0` for it and still counts `42` input tokens for the real line
and `0` for a missing-message line.
- Not tested: a full `learn` run over a real history containing such a
line; full local `pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new test reuses the
existing `ClaudeCodePlugin` scanner harness in
`tests/test_learn/test_subagent_scanning.py`, so it runs under the
normal CI pytest job; behaviour is additionally verified by the
standalone proof above.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Abhay Singh
|
ec12e18186
|
fix(savings): don't fabricate output savings for a free (zero-priced) model (#2298)
## Description
`_estimate_output_savings_usd` reports phantom output-shaping savings
for a model whose output price is legitimately `0.0`.
It reads the per-token output price from litellm and treats a falsy
value as "unavailable":
```python
output_cost_per_token = info.get("output_cost_per_token")
if not output_cost_per_token:
raise RuntimeError("output cost unavailable")
return float(tokens_saved) * float(output_cost_per_token)
```
`if not output_cost_per_token` is `True` for both a **missing** price
(`None`) *and* a real **`0.0`** (a free / local / vendored-at-zero
model). So for a free model it raises, hits the `except`, and bills the
saved output tokens at `DEFAULT_FALLBACK_OUTPUT_COST_PER_TOKEN` ($15/M)
— fabricating output savings for a model that costs nothing.
This is the exact bug that `_estimate_compression_savings_usd` was
already fixed for (it now uses `if input_cost_per_token is None:`, with
a comment explaining that `if not ...` "treated a real 0.0 as
unavailable and billed the $3/M fallback — phantom savings").
`_estimate_input_cost_usd` carries the same fix.
`_estimate_output_savings_usd` is the one that was missed.
## Fix
Fall back only when the price is truly missing:
```python
if output_cost_per_token is None:
raise RuntimeError("output cost unavailable")
```
A `0.0` price now correctly yields `$0` output savings; a missing price
still falls back to the estimate; a real price is unchanged.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/savings_tracker.py`: `_estimate_output_savings_usd`
falls back on `output_cost_per_token is None` instead of `not
output_cost_per_token`.
- `tests/test_savings_tracker_zero_price.py`: new tests (free → $0,
unknown → fallback, paid → real price), alongside the existing
compression/input-cost zero-price tests.
- `CHANGELOG.md`: Bug Fixes entry.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ uvx ruff@0.15.17 check headroom/proxy/savings_tracker.py tests/test_savings_tracker_zero_price.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/savings_tracker.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, `uvx ruff@0.15.17` / `uvx
mypy@1.20.2`. A full `pytest` OOM-kills this box (ML stack import), so I
reproduced the estimator with a dependency-free script and left the full
pytest to CI.
- Exact command / steps: ran 1,000,000 saved output tokens through the
OLD `if not ...` and NEW `is None` logic for a free model
(`output_cost_per_token = 0.0`), a paid model, and an unknown model
(`None`).
- Observed result: OLD bills the free model at the $15/M fallback
(`$0.015` phantom savings); NEW returns `$0.00`. The paid model is
unchanged; the unknown model still falls back under both.
- Not tested: a live proxy run pricing a free model end-to-end; full
local `pytest` deferred to CI (OOM).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The "unit tests pass locally" box is unchecked because the full suite
imports the ML stack, which I can't run here. The new tests reuse the
`_fake_litellm` harness in `tests/test_savings_tracker_zero_price.py`
(the same file that pins the compression/input-cost zero-price
behavior), so they run under the normal CI pytest job; behaviour is
additionally verified by the standalone proof above.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
Sebastian Schkudlara
|
517bf992cf
|
fix(proxy): quarantine compression while timed-out workers run (#2292)
## Description
A request-side `asyncio.wait_for()` timeout stops waiting, but it cannot
preempt an executor thread that already started. The proxy counted those
late workers and still admitted more compression, so repeated slow calls
could consume the whole compression pool and charge every request
another full timeout.
This change tracks running post-timeout workers as timeout debt and
quarantines request-path compression while that debt is non-zero. New
attempts raise `CompressionQuarantinedError` before executor admission,
using an `asyncio.TimeoutError` subclass so Python 3.10 handlers apply
the existing compression-failure policy. Quarantine clears automatically
after all known timed-out workers genuinely exit.
Mitigates #946 and #810. It does not attempt to kill the first running
thread; Python cannot safely preempt it.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Track started, finished, timed-out, and debt-recorded state under the
existing compression metrics lock.
- Reject new compression before enqueue while timed-out workers remain;
clear quarantine on the final worker exit.
- Preserve queued-timeout behavior: work cancelled before worker start
does not activate quarantine; a cancellation/start race is
conservatively tracked as running debt.
- Add `/health` and `/stats` runtime fields for quarantine state, worker
debt, activations, and skips.
- Add `headroom_compression_quarantine_total{event="activated|skipped"}`
Prometheus counters.
- Add regression, recovery, queue-race, runtime-payload, export, reset,
and Python 3.10 exception-class coverage.
- Update `CHANGELOG.md`; no dependency or lockfile changes.
## Reproduction
On base commit `
|
||
|
GUOHAO LIU
|
8951a264a2
|
fix(proxy): preserve content-part array structure in excluded-tool lossless fold write-back (#2261)
## Description
When a tool result is excluded from lossy compression (e.g. grep via
`HEADROOM_EXCLUDE_TOOLS`), the OpenAI Responses adapter performs a
byte-lossless fold on the output text. However, the excluded-tool fold
path joined all content-part text with `_responses_part_text()` and
recorded a `("output", None)` slot, which caused `_set_slot_text` to
replace the entire `output` with a plain string.
For content-part arrays (valid per OpenAI spec: `[{type: output_text,
text: "..."}, {type: input_image, ...}]`), this destroyed the array
structure — non-text parts like images and refusals were silently
dropped.
Closes #2235
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/openai.py` — In the excluded-tool lossless
fold path, detect list (content-part) outputs and fold each
`input_text`/`output_text` part individually using `("output_part",
index)` slots, matching the eligibility rule already used by
`_slot_texts()` in the normal compression path
- `tests/test_openai_responses_compression_units.py` — Strengthen
existing content-part test to assert output remains a list; add new test
with mixed parts (output_text + input_image + refusal) to verify
structure preservation and byte-identical non-text parts
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Formatting passes (`ruff format --check .`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ uv run pytest tests/test_openai_responses_compression_units.py -q --no-header
26 passed in 1.04s
$ uv run pytest tests/test_openai_responses_compression_units.py tests/test_openai_responses_context_compaction.py tests/test_openai_responses_traffic_learner.py -q --no-header
39 passed in 6.43s
```
## Real Behavior Proof
- Environment: Linux 6.8.0-124-generic, Python 3.12.3, headroom main @
|
||
|
Tanmay Garg
|
4e2bbfee3f
|
fix(opencode): Use opencode.jsonc when present (#1590)
## Description Fix OpenCode proxy injection so it respects user configurations that use the `.jsonc` extension, preventing Headroom from creating a duplicate `.json` file that overrides it. Closes #1588 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Updated `opencode_config_path` in `paths.py` to check for `.jsonc` - Updated backup creation in `config.py` to preserve the original extension ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text N/A ``` ## Real Behavior Proof - Environment: local headroom dev - Exact command / steps: creating a dummy `.config/opencode/opencode.jsonc` and running `headroom wrap opencode`. - Observed result: Headroom successfully injects into `.jsonc` and creates a backup named `opencode.jsonc.headroom-backup`. - Not tested: N/A ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) ## Additional Notes |
||
|
guyoron1
|
f42ce4a239
|
fix: harden fd lifecycle and SystemError handling in runtime and proxy kill (#1556)
## Description Make `pid_alive()` safe on Windows even when `psutil` is not installed, and harden `_kill_proxy_by_pid` exception handling for stale PIDs. ### Problem `headroom._subprocess.pid_alive()` falls back to `os.kill(pid, 0)` when `psutil` cannot be imported. On Windows, CPython routes `os.kill(pid, 0)` through `TerminateProcess` — a destructive call that **kills the target process**. Since `psutil` is not a declared runtime dependency in `pyproject.toml`, a normal lightweight install can hit that fallback, meaning `runtime_status()` can silently terminate a live proxy. ### Fix - **`headroom/_subprocess.py`**: On `win32`, bypass `os.kill` entirely and probe via `kernel32.OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION)`. If `ctypes` also fails, return `True` conservatively (assume alive) to prevent false-negative liveness from causing callers to kill a running process. - **`headroom/cli/wrap.py`**: Widen `_kill_proxy_by_pid` exception handlers on both SIGTERM and SIGKILL paths to catch `OSError` and `SystemError` (Windows `WinError 87`), preventing crashes from stale/invalid PIDs. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom/_subprocess.py`) ### New Tests - `test_pid_alive_win32_no_psutil_never_calls_os_kill` — simulates `win32` + broken `psutil`, asserts `os.kill` is never called and the `kernel32.OpenProcess` path is used instead - `test_pid_alive_win32_no_psutil_no_ctypes_returns_conservative` — simulates `win32` + broken `psutil` + broken `ctypes`, asserts `os.kill` is never called and `True` is returned conservatively |
||
|
Gautam Sharma
|
5279c33b19
|
fix(memory): preserve semantically similar memories (#2303)
## Description
Prevent memory_save from automatically deleting semantically similar but
distinct memories. The previous fire-and-forget deduplication path
deleted existing memories at cosine similarity scores of 0.92 or higher
after the save had already returned success. Similarity remains
available as a consolidation hint, while supersession now requires an
explicit memory_update or memory_delete operation.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Removed the automatic background deletion scheduled by memory_save.
- Removed the automatic-dedup threshold and background coroutine that
were no longer needed.
- Preserved the existing similarity search and consolidation hint.
- Kept explicit memory_update and memory_delete behavior unchanged.
- Added a regression test proving that distinct memories survive even at
0.99 simulated similarity.
- Added an Unreleased changelog entry.
## Testing
- [x] Unit tests pass (pytest)
- [x] Linting passes (ruff check .)
- [x] Type checking passes (mypy headroom)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
$ uv run --extra dev --frozen pytest
tests/test_memory_handler_native_ops.py
33 passed
$ uv run --extra dev --frozen ruff check .
All checks passed!
$ uv run --extra dev --frozen ruff format --check
headroom/proxy/memory_handler.py tests/test_memory_handler_native_ops.py
2 files already formatted
$ uv run --extra dev --frozen mypy headroom --ignore-missing-imports
Success: no issues found in 504 source files
$ uv run --extra dev --frozen pytest
9361 passed, 565 skipped, 4 failed
The four full-suite failures are unrelated to this diff: the Anthropic
compaction test passed in isolation; the Codex recovery test exceeded
the macOS AF_UNIX path limit; the dashboard test expects text absent
from the existing implementation; and the content-router test expects a
fallback absent from the existing strategy chain.
The repository-wide format check also flags pre-existing formatting in
the untouched headroom/proxy/handlers/anthropic.py.
## Real Behavior Proof
- Environment: macOS on Apple Silicon, CPython 3.12.13, real
LocalBackend, temporary SQLite database, and the local
sentence-transformers
embedding backend; no external provider or model API.
- Exact command / steps: Ran uv run --extra dev --frozen python with a
temporary database, saved User's primary backend framework at work is
FastAPI., queried its similarity to User's primary backend framework at
home is FastAPI., saved the second fact through
MemoryHandler._execute_save, and listed the user's memories.
- Observed result: The real embedding similarity was 0.9387, above the
former 0.92 deletion threshold. The second save returned saved,
included the consolidation hint, retained the original memory, and left
both distinct facts in the database (memory_count: 2).
- Not tested: Live OpenAI or Anthropic provider calls, a deployed proxy
or MCP client session, and Qdrant or Neo4j memory backends. These
paths share the handler policy changed here; backend-specific explicit
update and delete behavior is unchanged.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable
## Additional Notes
The documentation and code-comment checklist items are not applicable
because this change removes unsafe behavior without introducing a new
public interface or complex implementation. The full-suite checkbox
remains unchecked because four unrelated tests failed locally, as
documented above.
|
||
|
Rod Boev
|
26b43f64d6
|
fix(proxy): keep anthropic ccr compression active across deferred injection (#2291) (#2297)
## Description Large native Claude Code requests on the Anthropic path can still forward with zero request compression after CCR tool injection is deferred on a frozen prefix. The stale skip branch treats deferred injection as a reason to bypass request compression entirely, even though the later sticky CCR path already knows when new markers actually require the tool. This removes that stale bypass so compression still runs while the reversible CCR path stays intact. Closes #2291. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Remove the stale `should_skip_ccr_request_compression` branch from `headroom/proxy/handlers/anthropic.py`, so deferred CCR tool injection no longer bypasses request compression in token, non-cache, or cache mode. - Keep the existing sticky CCR injection path as the only place that decides whether historical markers need the retrieval tool reintroduced. - Update `tests/test_proxy/test_anthropic_ccr_deferred_injection.py` to cover the two broken zero-compression cases and preserve the already-reversible frozen-prefix path. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy/test_anthropic_ccr_deferred_injection.py::test_cache_mode_compresses_delta_but_replays_cached_prefix_when_markers_are_historical tests/test_proxy/test_anthropic_ccr_deferred_injection.py::test_non_token_non_cache_mode_keeps_compression_and_injects_tool_for_new_markers tests/test_proxy/test_anthropic_ccr_deferred_injection.py::test_existing_retrieve_tool_keeps_reversible_ccr_path_when_prefix_is_frozen tests/test_openai_tool_search_deferral.py tests/test_openai_responses_compression_units.py -q`) - [x] Linting passes (`uv run ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text ======================= 53 passed, 1 warning in 12.71s ======================== All checks passed! 1310 files already formatted ``` ## Real Behavior Proof - Environment: synced branch and base worktrees on a local Windows proxy test host - Exact command / steps: run the updated Anthropic deferred-injection regressions directly against the base package tree and the branch package tree, then run the focused branch pytest suite above - Observed result: the base package tree fails the two updated zero-compression regressions (`FAIL test_cache_mode_compresses_delta_but_replays_cached_prefix_when_markers_are_historical`, `FAIL test_non_token_non_cache_mode_keeps_compression_and_injects_tool_for_new_markers`) while preserving the already-reversible path; the branch package tree prints three `PASS` lines for the same trio and keeps the neighboring OpenAI suites green in the 53-test focused run - Not tested: a live upstream Claude Code request with the reporter's exact provider/model credentials ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes `CHANGELOG.md` is not applicable here because Headroom's release pipeline derives it from conventional commits. Scope is limited to the Anthropic CCR request-compression seam that current #2291 evidence exercises. OpenAI tool-search deferral is untouched because the current live issue is a native Claude Code path and the concrete stale skip branch on `origin/main` is in `anthropic.py`. Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
Tejas Chopra
|
1d79e70f95
|
fix(tests): repair three main-branch test failures (#2306)
## Description `main` CI is red on three independent test failures. All three are **test-side** bugs (stale cache, semantic merge conflict, stale mock) — no product code regressed. Each test passed in isolation but failed on `main`, and each also blocks the `chore: release main` PR (#1923). Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - **`test_l2_appends_transform_label`** — `tool_desc_max_chars()` memoises into a module global. An earlier test in shard 1 reads it with the env unset, pinning the cache to `0`, so this test's `setenv("HEADROOM_TOOL_DESC_MAX_CHARS=20")` was swallowed (`assert 0 == 20`). Reset the cache before reading and after, mirroring the sibling `test_l2_skips_label_when_disabled`. - **`test_dashboard_uses_cached_stats_and_lazy_history_feed_polling`** — semantic merge conflict: #2198 (persist lifetime metrics) intentionally retired the session-card `Filtered (lifetime)` row and moved CLI-filtering lifetime into the history tab as `Lifetime Saved`, while the assertion from #1433 still checked the old string. Assert the current `Lifetime Saved` label. - **`test_smart_crusher_log_fallback_runs_for_valid_json`** — stale mock: #1857 made token counting whitespace-aware, so the router now rates the JSON above the naive `len(content.split())==8` the no-op kompress mock reported, making it look like a saving and short-circuiting before the Log fallback. Mock now reports `_estimate_tokens(content)` to match the router. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) ### Test Output ```text $ pytest tests/test_anthropic_compaction_transforms.py \ tests/test_proxy_dashboard_stats_cache.py \ tests/test_transforms_content_router.py -q 78 passed, 1 skipped in 12.14s $ ruff check <the three files> All checks passed! $ ruff format --check <the three files> 3 files already formatted ``` ## Real Behavior Proof - Environment: local `.venv`, Python 3.12.6, pytest 9.0.2 (same three tests that fail on the `main` CI shards 1/3/4). - Exact command / steps: ran the three previously-failing tests by node id — all pass. Reproduced the shard-isolation failure for #1 by calling `tool_desc_max_chars()` with the env unset (cache → 0) before the test, confirmed the reset makes it pass. - Observed result: 3/3 target tests pass; 78 passed / 1 skipped across the three full files. - Not tested: full suite (unchanged product code); CI shards will re-run on this PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes `mypy headroom` (the CI-enforced scope) is unaffected — these edits touch only `tests/`, which CI does not type-check. Once this lands on `main`, the `chore: release main` PR (#1923) drops to just the `test_root_server_json_matches_builder` failure, which is the release version-bump `server.json` regen (not a code bug). |
||
|
Tejas Chopra
|
718c8dc559
|
fix(proxy): repair main lint (ruff-format drift + mypy host_header) (#2268)
## Description
`main`'s `lint` CI job is currently **red** (latest main `
|
||
|
kaz
|
eac49656a1
|
feat(wrap): add headroom wrap kimi for Kimi CLI (#1426)
## Description Adds `headroom wrap kimi`, routing Kimi CLI through the Headroom proxy. Kimi CLI speaks an OpenAI-compatible `/chat/completions` API (its `kosong` backend wraps `AsyncOpenAI`) and lets the base URL be overridden via `KIMI_BASE_URL`. This wrapper points it at the local proxy. Kimi's own OAuth bearer is forwarded upstream unchanged, so — unlike the Copilot subscription path — no extra login or token exchange is needed. ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `headroom/providers/kimi/`: new slice; `build_launch_env` sets `KIMI_BASE_URL` with the per-project base-URL prefix, mirroring the aider/vibe slices. - `headroom/cli/wrap.py`: `kimi` subcommand; falls back to the `kimi-cli` binary when `kimi` is not on `PATH`; `--kimi-api-url` overrides the upstream coding endpoint (default `https://api.kimi.com/coding/v1`). - `tests/test_cli/test_wrap_kimi.py`: 8 tests for the wrap command. - `README.md`: Kimi CLI row in the agent-compatibility matrix. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_cli/test_wrap_kimi.py -q ........ [100%] 8 passed in 0.36s $ ruff check headroom/providers/kimi headroom/cli/wrap.py tests/test_cli/test_wrap_kimi.py All checks passed! $ ruff format --check headroom/providers/kimi headroom/cli/wrap.py tests/test_cli/test_wrap_kimi.py 4 files already formatted ``` ## Real Behavior Proof - Environment: macOS; Kimi CLI (`kimi` / `kimi-cli`); `headroom proxy` started with `--openai-api-url https://api.kimi.com/coding/v1`. - Exact command / steps: start `headroom proxy --port 8787 --openai-api-url https://api.kimi.com/coding/v1`, then `curl -s http://localhost:8787/v1/chat/completions` with the Kimi OAuth bearer and a one-line `kimi-for-coding` chat request (`"Reply with exactly: PONG"`). - Observed result: `HTTP 200`; `choices[0].message.content == "PONG"` from `kimi-for-coding`; the OAuth bearer was forwarded and accepted upstream; the per-project path `/p/<name>/v1/chat/completions` also returned `HTTP 200`. - Not tested: Windows/Linux PATH discovery; the `--learn` / `--memory` live paths beyond flag wiring. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes ## Additional Notes - `ruff check` and `ruff format --check` pass locally; `mypy` was run on the new `headroom/providers/kimi` slice only (clean), so the full-tree `mypy headroom` box is left unchecked and is left to CI. - The slice deliberately reuses `codex.proxy_base_url` and `with_project_prefix`, identical to the aider/vibe wrappers, so per-project savings attribution works without Kimi sending custom headers. - Kimi's separate search/fetch services are out of scope for `KIMI_BASE_URL` and continue to hit Kimi directly; only the LLM `/chat/completions` traffic is compressed. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Ashish
|
46d4378cf7
|
feat(evals): weekly HotpotQA answer-recall report on the prose path (#1188)
## Description
Follow-up to **#1187** (the offline fidelity gate). That gate is
hermetic and **structured-only** (JSON tool outputs via Rust
compressors) so it can block every PR with zero setup. This PR adds the
genuinely-uncovered piece: **prose answer-recall on a real dataset
(HotpotQA)** in the **model-allowed weekly job**, where compression
routes through Kompress (ModernBERT).
> **Stacked on #1187.** Until that merges, this PR's diff shows its
commit too; it reduces to just `
|
||
|
Semianchuk Vitalii
|
63f74aa3e6
|
fix: replace computer_call_output with apply_patch_call_output in output_shaper (#2250)
The _RESPONSES_TOOL_OUTPUT_TYPES frozenset in output_shaper.py had computer_call_output instead of apply_patch_call_output, making it inconsistent with the canonical definitions in handlers/openai.py and output_turn_policy.py. This caused apply_patch_call_output items to be misclassified, preventing effort routing optimization for those turns. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
AxelRay
|
81d40a6437
|
fix(proxy): record Prometheus metrics for POST /v1/compress (#2247)
## Description `POST /v1/compress` compressed messages correctly but never recorded Prometheus business metrics. Standalone compress microservice deployments (including LiteLLM `guardrail: headroom`) left `headroom_requests_total`, token counters, latency, and by_model/by_provider families at zero. This wires the existing request-outcome funnel into `handle_compress` so success and timeout paths update the same counters as reverse-proxy handlers, and hard failures call `record_failed`. Closes #2244 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - On successful compression, record a `RequestOutcome` with `provider="compress"`, model, token before/after/saved, latency, transforms, tags, and client. - On compression timeout (fail-open), record zero-savings outcome plus `record_compression_failed("timeout")`. - On hard compression errors (503), call `metrics.record_failed(provider="compress")`. - Leave bypass header and empty-message early returns unrecorded (no real compression work). - Response schemas and status codes unchanged. - Add regression tests for success, timeout, and hard-failure metric recording. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text uv run pytest tests/test_proxy_compress_endpoint.py -q # 16 passed uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy_compress_endpoint.py # passed ``` ## Real Behavior Proof - Environment: local checkout of this branch; FastAPI TestClient fixtures for `/v1/compress` (loopback client) - Exact command / steps: - `uv run pytest tests/test_proxy_compress_endpoint.py -q` - `uv run ruff check headroom/proxy/handlers/openai.py tests/test_proxy_compress_endpoint.py` - Observed result: - Success path awaits `_record_request_outcome` with `tokens_saved = max(0, before-after)` and `provider="compress"` - Timeout path records zero-savings outcome and `record_compression_failed("timeout")` - Hard failure path awaits `record_failed(provider="compress")` and still returns 503 - Not tested: live multi-process scrape of `GET /metrics` while a real headroom process handles LiteLLM guardrail POSTs ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes - Focused compress endpoint suite only; full-repo mypy was not run. - No public API or response-schema changes. |
||
|
akothari-godaddy
|
7ddcbcb616
|
perf: surface optimization overhead diagnostics (#1212)
## Summary - add overhead diagnostics to perf JSON output - report optimization p50/p95/p99, slow request percentage, per-stage totals/percentiles, and top slow requests - update text report and recommendations to point at the slowest stage and HEADROOM_COMPRESSION_TIMEOUT_SECONDS when optimization is consistently slow ## Verification - python -m py_compile headroom/perf/analyzer.py tests/test_cli_perf_format.py - pytest tests/test_cli_perf_format.py could not run locally because pytest is not installed in this Python environment |
||
|
TUTU244
|
412db40a0b
|
fix(code): pin tree-sitter-language-pack <1.0.0 in [code] extra (#1219)
## Description <!-- Briefly explain the change and why it is needed. --> Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Paste relevant command output or artifact links here ``` ## Real Behavior Proof - Environment: - Exact command / steps: - Observed result: - Not tested: ## Review Readiness - [ ] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) Add screenshots to help explain your changes. ## Additional Notes <!-- Mention any N/A checklist items, tradeoffs, follow-ups, or maintainer context. --> --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
John Xu
|
1c50eca8b3
|
fix(proxy): skip Responses memory tools for ChatGPT auth (#1579)
## Description Fix ChatGPT/Codex session-auth Responses proxy handling so the ChatGPT backend always receives an explicit `store=false`, while keeping Responses memory tools limited to the regular API-key path where stored responses are supported. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Detect ChatGPT auth before Responses memory-tool injection and force `store=false` for ChatGPT-auth Responses payloads. - Skip Responses memory tools and transparent memory-tool continuation handling for ChatGPT auth across HTTP, WebSocket first frames, WebSocket follow-up `response.create` frames, and WS-to-HTTP fallback. - Preserve API-key behavior after the current main merge: API-key requests that explicitly set `store=false` skip Responses memory tools, while API-key requests that receive injected memory tools are forced to `store=true` for continuation support. - Address Copilot formatter comments by making `_allow_responses_memory_tools` call sites formatter-stable. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --extra dev ruff format --check headroom/proxy/handlers/openai.py 1 file already formatted $ uv run --extra dev ruff check headroom/proxy/handlers/openai.py tests/test_openai_codex_routing.py tests/test_openai_codex_ws_timings.py tests/test_ws_http_fallback.py All checks passed! $ uv run --extra dev python -m pytest -q tests/test_openai_codex_routing.py tests/test_openai_codex_ws_timings.py tests/test_ws_http_fallback.py 37 passed in 0.34s ``` ## Real Behavior Proof - Environment: Local checkout of `fix/codex-store-false-memory-tools` using `uv run --extra dev`. - Exact command / steps: Ran the focused formatter, lint, and pytest commands listed in `Testing`. - Observed result: Formatting is stable, lint passes, and the focused OpenAI/Codex routing and fallback tests pass. - Not tested: Full test suite, `mypy headroom`, and a fresh live ChatGPT backend probe after the formatter-only follow-up. The original PR validation recorded that valid ChatGPT subscription backend requests return `200` with `store=false`, while identical `store=true` or omitted `store` requests return `400 Store must be set to false`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A. ## Additional Notes - Post-deploy monitoring terms: `Responses: forced store=false for ChatGPT auth`, `WS Responses: forced store=false for ChatGPT auth`, `chatgpt_store_false`, `Memory: forced store=true for Responses memory tool continuation`, and upstream 400s containing `Store must be set to false`. - Expected healthy signals: ChatGPT-auth Responses requests keep `store=false` and no longer fail with `Store must be set to false`; API-key memory-tool flows still inject memory tools and can continue via `previous_response_id`. - Rollback trigger: any increase in ChatGPT-auth 400s, API-key memory-tool continuation failures, or missing memory tool injection on API-key Responses requests. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
Aashish Tamsya
|
420dc9077b
|
feat(grok-build): add Grok Build wrap command and MCP integration (#1629)
## Description
Adds first-class Grok Build support to Headroom so Grok CLI sessions can
route through the local proxy for context compression and savings
tracking.
This PR introduces `headroom wrap grok-build` / `headroom unwrap
grok-build`, a `grok_build` provider slice, Grok MCP registrar support,
and install/telemetry wiring so Grok traffic is attributed correctly in
the proxy and dashboard.
Review follow-up (`9368c413`): when users already own
`[model.grok-build]` in `~/.grok/config.toml`, wrap rewrites `base_url`
in that table in place instead of appending a duplicate header (invalid
TOML).
## Type of Change
- [x] New feature (non-breaking change that adds functionality)
## Changes Made
- Added `headroom/providers/grok_build/` with runtime helpers,
reversible `~/.grok/config.toml` injection, and install env builders.
- Added `headroom wrap grok-build` and `headroom unwrap grok-build` CLI
commands.
- Added `GrokRegistrar` for Headroom MCP registration in Grok config.
- Wired `grok_build` into install planner/registry, agent savings,
telemetry, and proxy client detection (`grok/` user agent).
- **Review fix:** rewrite `base_url` inside an existing user-owned
`[model.grok-build]` table in place (`# was: …` metadata).
- Added regression tests + docs (`grok-build.mdx`, `proxy.mdx`) and
CHANGELOG entry.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ pytest -q tests/test_provider_grok_build.py tests/test_mcp_registry/test_grok_registrar.py
============================== 12 passed in 1.13s ==============================
```
See **Screenshots** below for terminal captures (pytest, review-fix
in-place rewrite, proxy `/readyz`, unwrap).
## Real Behavior Proof
- Environment: macOS, Python 3.11.12 venv, feat/grok-build @ `
|
||
|
panamarob30-jpg
|
3f5067022f
|
[codex] Simulate Codex read maturation risk (#1395)
## Summary - add Codex support to `audit-reads --codex --simulate-maturation` - classify Codex edit targets from `apply_patch`, `sed -i`, `tee`, and shell redirects so maturation risk can count edits - include focused tests for Codex maturation metrics, CLI JSON/text output, and edit-risk buckets - refresh `uv.lock` to match the current `pyproject.toml` version/extras ## Validation - `uv run ruff check headroom/audit/codex.py headroom/audit/maturation.py headroom/audit/__init__.py headroom/cli/audit.py tests/test_audit_codex.py` - `uv run pytest tests/test_audit_codex.py tests/test_audit_reads.py tests/test_read_maturation.py tests/test_read_maturation_handler_nobust.py -q` - live local run: `uv run headroom audit-reads --codex --path /home/robert-briscoe/.codex/sessions --simulate-maturation` Co-authored-by: Robert Briscoe <robert@briscoe.dev> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
