mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
5 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
JD Davis
|
3077ac81e8
|
feat: add deterministic runtime rollout controls (#1490)
## Description Establish one centrally resolved, observable, deterministic, versioned runtime rollout-control mechanism for Headroom. Runtime rollout controls which behaviors an already-built artifact may expose; it does not select or qualify a Headroom release/version. ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Bug fix (non-breaking change that fixes rollout enforcement regressions) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made - Added `RolloutChannel`, `HEADROOM_ROLLOUT_CHANNEL`, `--rollout-channel`, and a versioned immutable `RolloutSnapshot` shared by Python configuration boundaries. - Added schema/policy versions, canonical registry and snapshot SHA-256 identities, per-feature decision reasons, disable precedence, unsafe qualification poisoning, strict CLI validation, and fail-closed environment handling. - Added `headroom rollout status --json`, Python `/stats.rollout`, and Rust `/rollout/status` runtime provenance. - Added equivalent Rust snapshot semantics and shared Python/Rust policy vectors while retaining language-specific feature registries. - Enforced rollout policy at alternate Python server composition roots so `HEADROOM_READ_MATURATION=1` cannot bypass its beta gate. - Preserved typed rollout snapshots across multi-worker serialization with schema, policy, registry, snapshot-digest, type, and feature-name validation. - Made loopback runtime output-shaper updates replace the immutable snapshot atomically for request readers, retain explicit request/disable provenance, preserve channel and kill-switch precedence, invalidate cached stats, and return the effective rollout decision. - Made `headroom learn --verbosity --apply` report a channel-blocked update instead of claiming the shaper is live. - Made explicit CLI feature flags fail loudly when their current channel blocks them. - Made persistent interceptor installation select canary automatically, or reject an explicitly insufficient channel unless the break-glass override is set. - Updated architecture, proxy, rollout, learn, and output-shaper documentation with required channels and hot-reload semantics. ## Testing - [x] Unit tests pass - [x] Linting passes (`ruff check .` and `ruff format --check .`) - [x] Type checking passes (`mypy headroom --ignore-missing-imports`) - [x] New regression tests added for every corrected behavior - [x] Rust tests and production-target Clippy pass - [x] Documentation build passes ### Test Output ```text Focused rollout coverage suite 57 passed; headroom.rollout + rollout CLI: 98% coverage Affected proxy/rollout/transform/governance suites 222 passed; 0 failed Final changed regression suites 100 passed; 0 failed Cross-module hot-reload isolation regression 6 passed; 0 failed cargo test -p headroom-core -p headroom-proxy --quiet headroom-core: 924 passed; 1 ignored headroom-proxy and integration suites: all passed cargo clippy -p headroom-core -p headroom-proxy --lib --bins -- -D warnings cargo fmt --all -- --check ruff check . ruff format --check . mypy headroom --ignore-missing-imports git diff --check All passed cd docs && npm run build Compiled successfully; 164 static pages generated ``` The unsharded Windows-only CI selection exposed unrelated baseline failures, principally the existing `sqlite:///C:\\...` URL parser producing an invalid `\\C:\\...` path. At commit ` |
||
|
Rod Boev
|
55c700c686
|
fix(proxy): register interceptor in explicit transforms list when HEADROOM_INTERCEPT_ENABLED (#1376)
## Description `headroom proxy --intercept-tool-results` sets `HEADROOM_INTERCEPT_ENABLED=1` but the interceptor is never registered. The proxy server constructs its transform pipeline with an explicit list (`server.py:645-648`), bypassing `_build_default_transforms` (`pipeline.py:113-118`) where the env-var check lives. The flag is silently ignored. This PR mirrors the env-var check in `server.py` immediately after the explicit transforms list, inserting `ToolResultInterceptorTransform()` at index 0 when `HEADROOM_INTERCEPT_ENABLED` is set (any truthy value). This matches the truthiness-based activation in `_build_default_transforms` at `pipeline.py:113-114`. Closes #829 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/server.py`: after the explicit transforms list (~line 692), check `os.environ.get("HEADROOM_INTERCEPT_ENABLED")` (truthy, matching `pipeline.py`) and prepend `ToolResultInterceptorTransform()` to both Anthropic and OpenAI pipelines - `tests/test_tool_result_interceptors.py`: two tests covering interceptor presence when env var is set and absence when unset - `CHANGELOG.md`: bug fix entry ## Testing - [x] Unit tests pass (`uv run pytest tests/test_tool_result_interceptors.py -v -k "proxy_pipeline"`) - [x] Linting passes (`uv run ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) — N/A: repo does not enforce mypy in CI - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # paste actual pytest -v output here after running ``` ## Real Behavior Proof - Environment: headroom proxy with `HEADROOM_INTERCEPT_ENABLED=1` - Exact command / steps: construct `HeadroomProxy(ProxyConfig())` with env var set, inspect `anthropic_pipeline.transforms` - Observed result: `ToolResultInterceptorTransform` present at index 0 in the pipeline transforms list - Not tested: end-to-end interception of a live streaming response; interaction with Bedrock pipeline path ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes No `ProxyConfig` plumbing is needed because the CLI already sets the env var at `proxy.py:741,759`. The fix is ~8 LOC in server.py. The activation uses bare truthiness (`os.environ.get(...)`) to match `pipeline.py:113-114`, so any non-empty value enables the interceptor. PR #831 (luv-jeri) is stale and labeled "status: needs author action" since 2026-06-19; this is an independent clean fix. |
||
|
chopratejas
|
504f20a1f7 |
fix(review-3): address third-pass PR #210 feedback
7 new "important" findings from the third review pass:
1. base.py:158 — JSONDecodeError on OpenAI tool_call arguments is now
logged at DEBUG (was silently replaced with {}). Callers can now
diagnose why range-key checks didn't fire on malformed calls.
2. base.py:234,248 — logger.warning for interceptor transform() and
key() failures now passes `exc_info=True` so stack traces are
preserved in logs.
3. base.py:306 — progressive disclosure now pre-seeds `fired` from the
frozen prefix. A file first Read in the cached prefix no longer gets
re-outlined when the model Reads it again in the mutable tail.
apply_to_messages() now takes a `frozen_count` kwarg and handles the
split internally; the Transform adapter simplifies.
4. astgrep.py:114 — broadened `except` around binaries.resolve() to
catch the full BinaryError hierarchy plus KeyError + OSError. In
offline mode this is the difference between a debug log and a
warning on every single request.
5. astgrep.py:181 — chmod 0700 failure on the temp dir now logs at
DEBUG. Silent swallow meant a multi-tenant host could leave
untrusted content world-readable with no indication the hardening
skipped.
6. cli/proxy.py:297 — explicit `--intercept-tool-results` now fails
fast (`sys.exit(1)`) when the critical tool can't be installed.
Previously it warned and started with non-functional interceptors.
7. pipeline.py:85 — interceptor gate now checks
`HeadroomConfig.intercept_tool_results` first, env var second.
Non-CLI callers (SDK, tests, embedded) can enable via config
instead of having to touch os.environ. Added the config field with
default False.
Plus a new test: `test_progressive_disclosure_respects_frozen_prefix_history`
proves a file Read in the frozen prefix blocks re-outline in the tail.
46 tests total, ruff + mypy clean.
|
||
|
chopratejas
|
e165a579f8 |
fix(review): address PR #210 feedback — race, query-params, SHA logging, frozen prefix, etc
Addresses all 24 inline comments across the two review passes. **CRITICAL fixes:** - binaries.py: PID-scoped partial-file name prevents concurrent `headroom proxy` starts from clobbering each other's downloads. - binaries.py: strip URL query params before computing the download filename (was breaking archive-type detection for mirror URLs with `?token=...`). - cli/tools.py: `--force` cleanup now logs failures and bumps exit_code instead of silently swallowing exceptions. **HIGH fixes:** - binaries.py: log at INFO when SHA256 is unpinned; expose `sha_pinned` in doctor's status output. - proxy/interceptors/base.py: `_FAILURES` counter + `interceptor_failure_counts()` getter; incremented on every `matches()`/`transform()`/`key()` exception so dashboards can distinguish "nothing eligible" from "everything crashing". - cli/proxy.py: validate critical tools resolved when `--intercept-tool-results` is set; warn (don't fail) if a dependency is missing. - proxy/interceptors/base.py: compute `tokens_before` from the original messages via `count_messages()` instead of back-calculating from `tokens_after + sum(saved)` (which double-counted message-level overhead). - proxy/interceptors/astgrep.py: write untrusted tool_output into a private mode-0700 `tempfile.mkdtemp()` directory, not directly into shared `/tmp`. - proxy/interceptors/base.py: `ToolResultInterceptorTransform.apply()` now honors `frozen_message_count` — leading cached-prefix messages are passed through untouched to preserve provider prefix caches. **MEDIUM fixes:** - proxy/interceptors/base.py: pre-built O(1) tool_use index replaces the O(n²) per-tool-result linear scan. - proxy/interceptors/base.py: broken `progressive_disclosure_key()` now skips the interceptor entirely rather than firing without key protection. - proxy/interceptors/astgrep.py: distinguish ast-grep rc=1 (no matches) from rc>=2 (real errors — bad syntax, missing grammar, corrupt binary). - proxy/interceptors/astgrep.py: count JSON parse failures; warn when all lines fail to parse (indicates version mismatch). - binaries.py: musl detection falls back to checking `/lib/ld-musl-*.so.1` when `ldd` is absent (Alpine). - proxy/interceptors/astgrep.py: use `tempfile.mkdtemp()` + `shutil.rmtree` instead of `NamedTemporaryFile(delete=False)`; cleans up on Windows. - binaries.py: chmod failures on POSIX now log a warning (only swallow on Windows where .exe is implicitly executable). - tests/test_binaries.py: `test_mirror_substitution` now uses `monkeypatch.setenv()` instead of raw `os.environ` manipulation. - tools.json: add `linux-x86_64-musl` and `linux-aarch64-musl` entries for `difft`; document the shared-asset strategy for both tools. - proxy/interceptors/astgrep.py: log a debug line when `progressive_disclosure_key()` returns None for a tool whose tool_input shape we don't recognize. - proxy/interceptors/base.py: moved `import json` to module top (was inside `_find_tool_use` hot loop). - binaries.py: fix bare `.gz` detection — now explicitly excludes `.tar.gz`/`.tgz` instead of relying on a brittle "no dots" heuristic. - proxy/interceptors/astgrep.py: provenance comment on each `_RANGE_KEYS` entry so future maintainers know which tool defined which key. - cli/tools.py: comment explaining os.execv's lack of Python finalizer cleanup. - proxy/interceptors/base.py: `InterceptionResult` now `frozen=True`. **Test gaps closed:** - Interceptor failure isolation (transform() raises → request survives, counter increments). - Broken key() skips interceptor entirely. - Refuse-to-enlarge guard (rewrite larger than original → pass through). - Orphaned tool_result (no matching tool_use) doesn't crash. - ToolResultInterceptorTransform.apply() happy path + frozen_message_count. - ensure_tools() partial failure (one tool fetch fails, others succeed, proxy still starts). - Mirror URL with query params doesn't leak into download filename. 44 tests total; ruff + mypy clean. |
||
|
chopratejas
|
7cd67ef06d |
fix: bundle ast-grep/difftastic/scc + generic tool_result interceptor framework
What this does, in plain terms: Headroom's proxy now ships with three CLI tools (ast-grep, difftastic, scc) that it can use to shrink tool_result payloads before they reach the model. The goal is simple: when Claude Code (or Codex, Aider, etc.) asks the model to reason about a big file or diff, we swap the verbose output for a compact, same-meaning version. Fewer tokens per turn, same answers, lower bill. Today a single interceptor is wired: ast-grep on Read. When an agent reads a large code file, the proxy replaces the file body with an outline of its top-level functions/classes plus docstrings. In live tests that cut prompt tokens 74–76% on both OpenAI and Anthropic, same answer either way. How it works: - `pip install headroom-ai` now installs ast-grep via a PyPI wheel (core dep). difftastic and scc are fetched once at proxy startup from pinned upstream GitHub releases and cached per-user. - A generic registry (`headroom/proxy/interceptors/`) lets us add more tool-aware rewrites in one file each: declare `matches()` and `transform()`, call `register()`, done. No proxy or metrics plumbing per tool. - Safety rails built in: pass-through when a Read specifies a line range; second Read of the same file in a conversation returns full content (progressive disclosure); any failing interceptor logs and skips, never crashes a request. Opt-in for now: - Off by default while this ships. Turn on with `headroom proxy --intercept-tool-results` or `HEADROOM_INTERCEPT_ENABLED=1`, so we can measure before flipping defaults. What users see after turning it on: - First `headroom wrap claude` boot is ~5s longer (binaries fetched). Every subsequent run is cache-only. - Existing `transforms_applied` field in metrics gets entries like `interceptor:ast-grep`, so savings show up in current dashboards and HTML reports with no UI change. Other housekeeping in this PR: - uv.lock moved to .gitignore — regenerated locally per environment. - 35 unit + integration tests, ruff + mypy clean. - Dead-code audit done: removed `binaries.run()`, `needs_filesystem` plumbing, unused `_kind` tuple elements, unused `tool_output` parameter, and the never-set HEADROOM_SKIP_TOOLS_BOOTSTRAP env. |