Two fixes for the init-native-e2e matrix surfaced on PR #256:
1. Composite action installed `headroom` without extras, but
`headroom/cli/__init__.py` eagerly imports `proxy.server` (via
`cli/proxy.py`), which requires `fastapi`. All 6 POSIX jobs hit
`ModuleNotFoundError: No module named 'fastapi'` before `init` ran.
Fix: install `-e .[proxy]` to match the Docker e2e image.
2. On Windows, shims are `.cmd` files and Git Bash's `which` cannot
resolve them (exact-match only). Python's `shutil.which` (used by
`headroom init`) honors PATHEXT and finds the shim fine, but the
pre-flight `which` step failed first. Fix: use `Get-Command` via
`pwsh` for the Windows verification step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test added in bb91cfe used ``CliRunner(mix_stderr=False)`` to keep
stderr separate from stdout for assertion purposes. That parameter was
removed in Click 8.2. The repo's pyproject.toml pins ``click>=8.1.0``,
so either Click 8.1 (needs mix_stderr) or Click 8.2+ (must omit it)
could appear in CI.
Switch to reading ``result.stderr`` when the attribute is populated,
falling back to ``result.output`` (combined stream) otherwise. This
covers every Click 8.x variant without branching on the installed
version.
Verified in the Docker e2e image (Click 8.3.3): all 45 tests in
tests/test_cli/test_init_cli.py pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Existing Docker init-e2e runs on ubuntu only. Platform-specific bugs
(Windows path separators in written hook commands, PowerShell-vs-bash
matcher strings, macOS keychain prompts, shutil.which PATHEXT quirks)
slip past it. Add a matrix workflow that drops a noop shim for each
target agent and runs ``headroom init -g <target>`` on each of the
three supported OSes, then asserts the settings file was written to
the platform-correct location.
Matrix: [ubuntu-latest, macos-latest, windows-latest] x [claude,
codex, copilot]. ``openclaw`` is excluded because it delegates to
``headroom wrap openclaw`` which needs a real OpenClaw CLI and can't
be stubbed with a noop shim; the Docker suite already covers its
negative path.
Common setup (Python install, editable headroom install, shim drop,
PATH wiring) is factored into a composite action at
.github/actions/headroom-e2e-setup so follow-up per-command workflows
(install-native-e2e, wrap-native-e2e) can be near-copies that only
supply their matrix and assertion blocks. The composite action uses
the cross-platform shim scripts from e2e/_lib/make_shim.{sh,ps1} that
landed with the harness refactor.
Scoped trigger: pull_request touching init code OR the harness, plus
pushes to main and manual dispatch. This avoids burning CI minutes on
every push to unrelated feature branches while still gating every PR
that could regress init behavior.
Not verified locally: Windows runner behavior. Reviewer should watch
the first matrix run on PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Port e2e/init/run.py onto the shared harness and extend coverage so
issue #245 (bare ``headroom init -g`` with no agents) is locked in:
* ``seq_claude_local`` / ``seq_copilot_global`` / ``seq_codex_local`` —
the original scenario, now expressed as a sequence of Cases sharing
one scratch so the manifest-merge behavior (claude + codex targets)
is still exercised end-to-end
* ``bare_init_g_no_shims`` — regression guard for issue #245: asserts
the new guided error mentions every probed target and the concrete
``headroom init -g <agent>`` example
* ``bare_init_g_with_all_shims`` — complementary happy path with all
four shims present; asserts all three configurable agents report
``Configured ... (user scope)`` on stdout
* ``init_g_{claude,codex,copilot}_explicit`` — one case per
subcommand, each with only its own shim on PATH, asserting exit 0
and the correct per-agent settings file is written
* ``init_g_openclaw_missing`` — negative path for openclaw when its
binary isn't installed (delegates to ``headroom wrap openclaw`` which
can't be shimmed cheaply)
* ``init_verbose_no_shims`` — smoke test for ``headroom init -v``
ensuring ``detect_init_targets``, ``global_scope=True``, and every
agent name appear on stderr
Dockerfile is updated to COPY e2e/__init__.py and e2e/_lib/ so the
harness is importable inside the container. A new e2e/__init__.py
marks the tree as a package.
One small harness fix rides along: ``_resolve_headroom_bin`` captures
the absolute path to headroom before ``with_clean_path`` narrows PATH.
This is required for any case run inside a venv-scoped image - the
real ``headroom`` lives outside the shim dir and would otherwise be
hidden by the scrubbed PATH. Same bug would have bitten every future
command suite, so the fix belongs in the harness rather than run.py.
Verified locally inside the Docker image: all 10 cases pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The sync-plugin-versions pre-commit hook recomputes plugin semver from
git history + conventional-commits bump rules. Adding the feat(init)
-v/--verbose commit triggers a minor bump (0.11.4 -> 0.12.0). Land
that bump as its own chore so subsequent test/ci commits on this
branch aren't flagged as drift by the hook.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When users hit an init regression it's opaque why: no visible state
about which agents were probed, which paths were written, which
subprocesses ran. Add a top-level flag to ``headroom init`` that routes
debug-level logging from the ``headroom.cli.init`` logger to stderr.
Instrumented decision points:
* detect_init_targets / _probe_init_targets — scope + per-target
shutil.which result
* _write_json, _ensure_claude_hooks, _ensure_copilot_hooks,
_ensure_codex_hooks, _ensure_codex_provider — file paths being written
* _apply_user_env — chosen scope (windows vs unix) and env-var keys
* _run_checked — each subprocess command + exit code + truncated
stdout/stderr (useful when ``claude plugin install`` fails)
* _run_init_targets — target dispatch order and resolved profile
* top-level init callback — all flag values and invoked_subcommand
Log output goes to stderr so stdout stays clean for pipes. The handler
attached by ``_enable_verbose_logging`` is idempotent - nested
subcommand invocations don't duplicate output. The logger does not
propagate to the root logger, so enabling ``headroom init -v`` does not
affect the rest of the process.
The flag is declared on the parent Click group. Subcommands (claude,
codex, copilot, openclaw) inherit the enabled logger automatically
because the group callback runs before dispatch.
Added tests cover:
* ``init -v`` emits the expected markers to stderr, including
``detect_init_targets``, ``global_scope=True``, and each agent name
* ``_enable_verbose_logging`` is safe to call repeatedly (handler
remains singular)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes#245.
Running ``headroom init -g`` with no supported agents on PATH previously
produced a single-line ClickException that read like the -g flag had
been removed:
Error: No supported user init targets were auto-detected. Specify one explicitly.
This left reporter #245 concluding the feature was gone. Replace that
message with a structured diagnostic that:
* states which scope (user / local) was tried
* lists every target probed (claude, codex, copilot, openclaw) and the
shutil.which() result for each
* explicitly confirms that -g / --global is still a supported flag
* shows the concrete per-target invocation for each agent
(``headroom init -g claude``, ...) so the user knows the escape hatch
The implementation factors ``detect_init_targets`` into a ``_probe_init_targets``
helper that returns ``[(name, which_result)]``. ``detect_init_targets``
keeps its existing signature so the test suite and external imports
aren't broken; the new helper backs both the auto-detection path and
the diagnostic error formatter.
Unit tests in tests/test_cli/test_init_cli.py cover:
* the end-to-end message shape (structural markers + every target name +
the example invocation)
* the local-scope variant omitting global-only agents (copilot / openclaw)
* that found binaries are surfaced with their absolute path so users can
debug cases where shutil.which returns an unexpected result
No behavior change when at least one target is detected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Centralize Docker / CI e2e test helpers so per-command suites can be
declarative and future commands (install, wrap, ...) can reuse the same
shim/PATH/assertion primitives without duplicating infrastructure.
The harness provides:
* Case dataclass describing one test as argv + shims + expected exit /
stdout / stderr / files / custom callbacks
* make_shim() factory producing cross-platform executable shims (.sh on
POSIX, .cmd on Windows) with noop / fail / record-args behaviors
* with_clean_path() context manager that isolates PATH to a minimal
known-good value plus any extras supplied by the case
* agent_settings_path() locator mirroring headroom.cli.init so tests can
assert the right file was written without touching private init state
* run_cases() for independent cases and run_case_sequence() for cases
that must share scratch state (e.g. manifest-merge scenarios)
Shell / PowerShell shim-creation scripts are also shipped for CI steps
that need to drop a shim without spinning up Python first.
No behavior change in this commit - pure infrastructure. The init suite
and new subcommand suites consume the harness in follow-up commits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Running the repo's sync-plugin-versions pre-commit hook updates
.claude-plugin/marketplace.json, .github/plugin/marketplace.json, and
the two headroom-agent-hooks plugin.json manifests to the release
semver computed from git tags (0.11.4 at time of branch). Landing this
first keeps subsequent commits on this branch from tripping the
hook's auto-fix path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`headroom wrap codex` injects a `model_provider = "headroom"` block
plus a `[model_providers.headroom]` table into `~/.codex/config.toml`
so Codex routes both HTTP and WebSocket traffic through the proxy. The
matching `unwrap codex` subcommand did not exist, so the injected
block stayed in `config.toml` forever — the moment the proxy stopped,
Codex (CLI and macOS app) started erroring with
`Missing environment variable: OPENAI_API_KEY`, and users had to hand-
edit the file to recover.
Fix:
* `_inject_codex_provider_config` now snapshots the pre-wrap file to
`~/.codex/config.toml.headroom-backup` before the first modification
and leaves that snapshot untouched on subsequent wrap runs. The
injection is also rewritten to use two self-contained marker-
delimited blocks (top-level key and provider table) so stripping
them never consumes user content that sits between them.
* `_inject_memory_mcp_config` takes the same snapshot, so
`wrap codex --memory` without a full provider injection is still
fully reversible.
* New `_restore_codex_provider_config` helper and `unwrap codex`
click command:
* backup present → restore byte-for-byte and delete the backup;
* backup absent but Headroom block present → strip the block and
keep surrounding user content;
* config contained only Headroom content → remove the file so
Codex falls back to defaults;
* nothing to undo → safe no-op.
Codex is the only wrap target that modifies a persistent user config
file: claude/aider/cursor/copilot all go through env vars or project-
scoped files only, so this bug was unique to Codex.
Tests:
* `tests/test_cli/test_wrap_codex.py` adds 20 new cases covering the
strip/snapshot helpers directly, round-trip idempotency of
wrap → wrap → unwrap, handling of malformed prior configs, and
end-to-end CliRunner invocations of `headroom wrap codex
--prepare-only` / `headroom unwrap codex` against a temp `$HOME`.
* All 153 existing `tests/test_cli/` tests continue to pass.
Plugin manifest versions were re-synced from `pyproject.toml` (0.11.2)
by the `sync-plugin-versions` pre-commit hook; the previous values
(0.10.3) had drifted.
Reported by @raenaryl in Discord on 0.6.3; confirmed still broken on
current `main` (0.11.x).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI runs `ruff format --check`, which flagged the two multi-line
expressions added in the previous commit. Pure whitespace reflow —
no behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compute_turn_id hashed the raw message dicts, which meant the same
user-text message produced a different hash on each call of one agent
loop because clients (notably Claude Code) move the cache_control
breakpoint to the newest message per call. The user-text block carries
cache_control on call 1 and not on call 2, so the serialized prefix
differs and the turn_id rolls over. Effect downstream: every API call
becomes its own "turn" and any prompt-level aggregation (e.g. the
Headroom desktop app's prompt all-time record) collapses to the
largest single call, not the sum across the prompt.
Add a small recursive normalization pass that strips cache_control from
the hashed prefix and from list-shaped system prompts before hashing.
Two new tests cover cache_control moving between calls on both the
messages array and the system prompt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When `headroom learn` re-surfaced a section heading that already existed
in CLAUDE.md / MEMORY.md, the writer replaced that section wholesale —
but the LLM never saw the prior block, so it emitted condensed bullets
like "X is *also* large — same rule as Y, Z" assuming Y and Z would
remain siblings. After replacement, Y and Z were gone and the "also"
dangled.
This threads the project's current `<!-- headroom:learn -->` block (from
both CLAUDE.md and MEMORY.md) into the digest as a "Prior Learned
Patterns" section, and extends the system prompt to make the re-emission
contract explicit: re-stating a section replaces it wholesale, so the
LLM must copy forward prior bullets it still agrees with. Prior sections
the LLM omits entirely are still carried forward by the writer (#231
behavior preserved as a safety net).
Changes:
- New `extract_marker_block(file_content)` helper in `learn.writer` that
returns the raw marker block (delimiters included) or None.
- New `_build_prior_patterns_section(project)` in `learn.analyzer` reads
`project.context_file` and `project.memory_file` via the new helper
and formats a labeled section ahead of the per-session event stream.
- `_build_digest` emits the prior-patterns section when present; char
budget accounting unchanged (prior blocks are small).
- `_SYSTEM_PROMPT` gains a "Prior Learned Patterns" rule block telling
the LLM how to integrate prior bullets (preserve / revise / drop-only-
if-contradicted) and warning against unresolved cross-references.
- Tests: 6 new `TestPriorPatternsInjection` cases (present/absent files,
no-marker-block, both-files, end-to-end via mocked `_call_llm`); 4 new
`TestExtractMarkerBlock` cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /livez-unaffected test flaked deterministically on the Python 3.10
matrix job of this PR while passing on 3.11/3.12/3.13 and on main. Root
cause: the 3-request warmup did not cover every lazy-init path the
restructured proxy triggers on first request, so one measured sample
(consistently index 2 of 20) came in at 336-356ms instead of <1ms.
Compounding this, the assertion called `statistics.quantiles(n=100)[98]`
"p99" on only 20 samples — which collapses to `max(latencies)` and fails
on any single stall.
Fix the test for real, not just for this PR:
- Bump warmup from 3 to 10 to clear all lazy-init paths exposed by the
upstream canonical-pipeline restructure. CI traces placed the rogue
sample at measured-index 2 (request #6 overall), so 10 is comfortably
past every observed lazy boundary.
- Stop mislabelling `max(latencies)` as p99. With 20 samples, drop the
single worst outlier and assert on the next-worst. A genuine regression
(semaphore actually blocking /livez) still fails hard because every
sample would cluster near the drained timeout; a single GC/scheduler
jitter no longer trips the assertion.
- Drop now-unused `statistics` import.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
test_livez_unaffected_under_anthropic_backpressure timed out at 336s
against a 100s threshold on the Python 3.10 runner. Timing-sensitive
test, unrelated to this PR's changes (helpers.py, models.py,
handlers/*.py, server.py). Pushing an empty commit because contributor
PRs cannot re-run individual failed jobs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Anthropic subscription window tracker polled GET /api/oauth/usage
every 10s by default and kept polling even when the proxy was idle
(via the cached OAuth token in ~/.claude/.credentials.json). On Pro
accounts this blew through Anthropic's metadata rate limit within
minutes and produced a steady stream of 429s.
Raise the default interval to 300s, raise the upper cap to 3600s, and
expose --subscription-poll-interval / --no-subscription-tracking CLI
flags (with matching HEADROOM_SUBSCRIPTION_POLL_INTERVAL and
HEADROOM_NO_SUBSCRIPTION_TRACKING env vars) so users can tune further
or turn the poller off entirely.
Fixes#242
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflicts:
- CHANGELOG.md: upstream landed Live flush + traffic-learner fix entries.
Placed turn_id entry at the top of the first [Unreleased] ### Added
section so both features coexist; preserved upstream ordering.
- headroom/proxy/server.py: upstream restructured the file substantially,
producing a whole-file conflict. Took upstream's version and
re-applied the single-line `"turn_id": log.get("turn_id")` addition
to the /transformations/feed response dict at its new location.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
