Two bugs collided to break the Docker-native install CI:
1. `ensure_tools()` ran unconditionally at every proxy startup, even when
`--intercept-tool-results` was not passed. The feature is opt-in, so
there's no reason to pay the binary-fetch cost (or risk a failure) when
nothing will use them.
2. The fetch loop caught PlatformNotSupported / OfflineError /
BinaryFetchError / Sha256Mismatch but not `PermissionError`. In
containerized environments where the home dir / cache dir isn't
writable, `binary_path.parent.mkdir()` raises PermissionError
(subclass of OSError), which propagated out of ensure_tools() and
crashed proxy startup.
Fixes:
- Move ensure_tools() inside the `if intercept_tool_results:` branch in
cli/proxy.py so the base case never triggers a fetch.
- Catch OSError (covers PermissionError, ENOSPC, etc.) in ensure_tools()
so sandboxed / readonly filesystems degrade to no-op instead of
crashing. Interceptors fall back to pass-through when their tool isn't
resolvable.
Adds regression test `test_ensure_tools_survives_readonly_cache_dir` that
points the cache at a chmod-0500 parent and asserts ensure_tools()
returns without raising.
Addresses all 24 inline comments across the two review passes.
**CRITICAL fixes:**
- binaries.py: PID-scoped partial-file name prevents concurrent `headroom proxy`
starts from clobbering each other's downloads.
- binaries.py: strip URL query params before computing the download filename
(was breaking archive-type detection for mirror URLs with `?token=...`).
- cli/tools.py: `--force` cleanup now logs failures and bumps exit_code
instead of silently swallowing exceptions.
**HIGH fixes:**
- binaries.py: log at INFO when SHA256 is unpinned; expose `sha_pinned` in
doctor's status output.
- proxy/interceptors/base.py: `_FAILURES` counter + `interceptor_failure_counts()`
getter; incremented on every `matches()`/`transform()`/`key()` exception so
dashboards can distinguish "nothing eligible" from "everything crashing".
- cli/proxy.py: validate critical tools resolved when
`--intercept-tool-results` is set; warn (don't fail) if a dependency is
missing.
- proxy/interceptors/base.py: compute `tokens_before` from the original
messages via `count_messages()` instead of back-calculating from
`tokens_after + sum(saved)` (which double-counted message-level overhead).
- proxy/interceptors/astgrep.py: write untrusted tool_output into a private
mode-0700 `tempfile.mkdtemp()` directory, not directly into shared `/tmp`.
- proxy/interceptors/base.py: `ToolResultInterceptorTransform.apply()` now
honors `frozen_message_count` — leading cached-prefix messages are passed
through untouched to preserve provider prefix caches.
**MEDIUM fixes:**
- proxy/interceptors/base.py: pre-built O(1) tool_use index replaces the
O(n²) per-tool-result linear scan.
- proxy/interceptors/base.py: broken `progressive_disclosure_key()` now
skips the interceptor entirely rather than firing without key protection.
- proxy/interceptors/astgrep.py: distinguish ast-grep rc=1 (no matches) from
rc>=2 (real errors — bad syntax, missing grammar, corrupt binary).
- proxy/interceptors/astgrep.py: count JSON parse failures; warn when all
lines fail to parse (indicates version mismatch).
- binaries.py: musl detection falls back to checking `/lib/ld-musl-*.so.1`
when `ldd` is absent (Alpine).
- proxy/interceptors/astgrep.py: use `tempfile.mkdtemp()` + `shutil.rmtree`
instead of `NamedTemporaryFile(delete=False)`; cleans up on Windows.
- binaries.py: chmod failures on POSIX now log a warning (only swallow on
Windows where .exe is implicitly executable).
- tests/test_binaries.py: `test_mirror_substitution` now uses
`monkeypatch.setenv()` instead of raw `os.environ` manipulation.
- tools.json: add `linux-x86_64-musl` and `linux-aarch64-musl` entries for
`difft`; document the shared-asset strategy for both tools.
- proxy/interceptors/astgrep.py: log a debug line when
`progressive_disclosure_key()` returns None for a tool whose tool_input
shape we don't recognize.
- proxy/interceptors/base.py: moved `import json` to module top (was inside
`_find_tool_use` hot loop).
- binaries.py: fix bare `.gz` detection — now explicitly excludes
`.tar.gz`/`.tgz` instead of relying on a brittle "no dots" heuristic.
- proxy/interceptors/astgrep.py: provenance comment on each `_RANGE_KEYS`
entry so future maintainers know which tool defined which key.
- cli/tools.py: comment explaining os.execv's lack of Python finalizer
cleanup.
- proxy/interceptors/base.py: `InterceptionResult` now `frozen=True`.
**Test gaps closed:**
- Interceptor failure isolation (transform() raises → request survives,
counter increments).
- Broken key() skips interceptor entirely.
- Refuse-to-enlarge guard (rewrite larger than original → pass through).
- Orphaned tool_result (no matching tool_use) doesn't crash.
- ToolResultInterceptorTransform.apply() happy path + frozen_message_count.
- ensure_tools() partial failure (one tool fetch fails, others succeed,
proxy still starts).
- Mirror URL with query params doesn't leak into download filename.
44 tests total; ruff + mypy clean.
What this does, in plain terms:
Headroom's proxy now ships with three CLI tools (ast-grep, difftastic,
scc) that it can use to shrink tool_result payloads before they reach
the model. The goal is simple: when Claude Code (or Codex, Aider, etc.)
asks the model to reason about a big file or diff, we swap the verbose
output for a compact, same-meaning version. Fewer tokens per turn, same
answers, lower bill.
Today a single interceptor is wired: ast-grep on Read. When an agent
reads a large code file, the proxy replaces the file body with an
outline of its top-level functions/classes plus docstrings. In live
tests that cut prompt tokens 74–76% on both OpenAI and Anthropic,
same answer either way.
How it works:
- `pip install headroom-ai` now installs ast-grep via a PyPI wheel
(core dep). difftastic and scc are fetched once at proxy startup
from pinned upstream GitHub releases and cached per-user.
- A generic registry (`headroom/proxy/interceptors/`) lets us add more
tool-aware rewrites in one file each: declare `matches()` and
`transform()`, call `register()`, done. No proxy or metrics plumbing
per tool.
- Safety rails built in: pass-through when a Read specifies a line
range; second Read of the same file in a conversation returns full
content (progressive disclosure); any failing interceptor logs and
skips, never crashes a request.
Opt-in for now:
- Off by default while this ships. Turn on with
`headroom proxy --intercept-tool-results` or
`HEADROOM_INTERCEPT_ENABLED=1`, so we can measure before flipping
defaults.
What users see after turning it on:
- First `headroom wrap claude` boot is ~5s longer (binaries fetched).
Every subsequent run is cache-only.
- Existing `transforms_applied` field in metrics gets entries like
`interceptor:ast-grep`, so savings show up in current dashboards
and HTML reports with no UI change.
Other housekeeping in this PR:
- uv.lock moved to .gitignore — regenerated locally per environment.
- 35 unit + integration tests, ruff + mypy clean.
- Dead-code audit done: removed `binaries.run()`, `needs_filesystem`
plumbing, unused `_kind` tuple elements, unused `tool_output`
parameter, and the never-set HEADROOM_SKIP_TOOLS_BOOTSTRAP env.
