mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
40 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tejas Chopra
|
b77d612913
|
fix(copilot): send VS Code inline completions to the host that serves them (#3112)
## Description
#3077 stopped Copilot's inline completions being forwarded to
`api.openai.com` (the corporate-blocked host in the original report) —
but sent them to the **CAPI host**, which does not serve that endpoint.
Copilot has two surfaces on two different hosts, and GitHub's own client
library keeps them apart:
```js
_getCAPIUrl(t) -> t?.endpoints.api || "https://api.githubcopilot.com"
_getProxyUrl(t) -> t?.endpoints.proxy || DEFAULT_PROXY_BASE_URL
DEFAULT_PROXY_BASE_URL = "https://copilot-proxy.githubusercontent.com"
```
building completions as
`${proxyBaseURL}/v1/engines/<engine>/completions` (`@vscode/copilot-api`
0.5.2). Probed unauthenticated against the live hosts:
| host | `POST /v1/engines/<e>/completions` |
|---|---|
| `copilot-proxy.githubusercontent.com` | **401** — exists, needs auth |
| `proxy.individual.githubcopilot.com` | **401** — CNAME to the above |
| `api.githubcopilot.com` | **404** — does not serve this path |
So the destination #3077 chose could not have worked. Three separate
defects were in the way, each sufficient on its own to keep completions
broken.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `copilot_auth.py`: added `DEFAULT_COMPLETIONS_PROXY_URL` and made it
the default in `copilot_completions_base_url()`, replacing the CAPI
host.
- `copilot_auth.py`: the "custom deployment keeps its own host" rule now
excludes public Copilot hosts. Without this, `headroom wrap vscode` —
the common setup, and the one that exports
`GITHUB_COPILOT_API_URL=<resolved subscription URL>` — resolved straight
back to the 404 host. **This was a bug in my own first cut of the fix,
found by testing the real `wrap vscode` environment rather than just the
routing table.**
- `copilot_auth.py`: added `is_copilot_completions_host()` and
`is_copilot_upstream_url()` (chat ∪ completions). The completions host
was recognised as Copilot **nowhere**, so `apply_copilot_api_auth`
attached no credentials (401 — routing correctly to a host we then
failed to authenticate against) and `build_copilot_upstream_url` skipped
`mark_request_routed_to_copilot()`, mislabelling the provider in
telemetry.
- The union is applied at exactly those two call sites.
`is_copilot_api_url` is left alone, so validation of a token payload's
`endpoints.api` and the Responses-API preference check keep their strict
chat-only meaning. All six call sites were read before choosing this.
- `proxy_targets.py`: the "already a Copilot host" guard now keys on the
*completions* host. A CAPI host is not a completions host, so it must
still be redirected; a genuine per-SKU completions host or operator
override is still left untouched.
- `providers/copilot/vscode.py`, `cli/wrap.py`,
`docs/…/vscode-copilot.mdx`: stop writing/printing
`github.copilot.advanced.debug.overrideAuthType`. No such setting exists
in the modern Copilot Chat extension — the only one left after
`GitHub.copilot` was deprecated in early 2026. Its full `advanced.*`
surface is `authPermissions`, `authProvider`, `debug.overrideCapiUrl`,
`debug.overrideProxyUrl`, `debug.use*Fetcher`. It is still *recognised*
so a stale hand-written copy is detected, just never emitted.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
tests/test_copilot_vscode_completions_routing.py 59 passed
Copilot-related suites 293 passed, 8 skipped
Full suite:
3 failed, 11250 passed, 581 skipped in 342.50s
```
The 3 failures are pre-existing and environmental, identical to a
plain-`main` baseline on this machine: no `cargo`
(`test_no_native_tls_in_wheel_build_tree`), no `codex` CLI
(`test_learn/test_integration.py`), and
`test_run_server_installs_cancelled_error_filter`, which fails under
full-suite ordering on `main` too.
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0), Python 3.12.13, worktree off
`main` @ `
|
||
|
JD Davis
|
1aa701adaa
|
fix(vscode): persist compatible Claude modes and route Copilot CAPI (#2986)
## Description Fixes #2492, #2028, and #2827. Claude daemon workers consume project settings rather than reliably inheriting wrapper environment state, while the Claude VS Code webview cannot render deferred-tool response blocks. Separately, recent Copilot Chat versions use the whole CAPI override for generation; the legacy proxy override alone only sends model discovery through Headroom. This PR carries both integrations through to the actual consumers instead of only changing their launch-time surface configuration. ## Type of Change - [x] Bug fix - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] Build / CI ## Changes Made - Persist the resolved Claude ENABLE_TOOL_SEARCH value into project settings for daemon workers and restore it transactionally after wrap exits. - Use compatibility-safe Foundry and Claude VS Code defaults while preserving explicit user choices. - Configure both Copilot overrideProxyUrl and overrideCapiUrl in the reversible managed VS Code settings block. - Route Copilot unprefixed POST /chat/completions and HTTP /responses requests through the real compression handlers. - Keep /responses out of the Codex WebSocket aliases because Copilot and Codex use different WebSocket wire protocols. - Extend wrap E2E assertions for both the Claude webview mode and Copilot CAPI routing. ## Testing - [x] 127 combined Claude, Copilot, route-integration, and MCP dependency-contract tests pass. - [x] Ruff check passes on all changed Python files. - [x] Ruff format check passes. - [x] Python compilation and git diff --check pass. ## Runtime Safety Standalone Claude CLI defaults remain unchanged. Explicit Claude tool-search values retain precedence, and project settings are restored through the existing cleanup path. Copilot model/session helper endpoints continue through generic passthrough, while only validated HTTP generation paths receive explicit compression routes. Existing Codex WebSocket behavior is unchanged. ## Review Readiness - [x] Current main and MCP v1 compatibility retained - [x] Worker-facing Claude persistence covered - [x] Reversible Copilot and Claude settings behavior covered - [x] Copilot generation routes covered at registration and proxy integration layers - [x] Ready for review |
||
|
JD Davis
|
13a310a00d
|
feat(claude): support Claude Code in VS Code (#2752)
## Description Add first-class Headroom support for the official Claude Code extension in VS Code. The new wrapper starts the local proxy, configures the Claude Code user settings consumed by the embedded extension process, preserves authentication and model selection, and provides a conflict-safe reversible unwrap lifecycle. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom wrap vscode-claude` and `headroom unwrap vscode-claude`. - Configure project-scoped `ANTHROPIC_BASE_URL` plus `ENABLE_TOOL_SEARCH=true` in Claude Code user settings while preserving existing values. - Respect `CLAUDE_CONFIG_DIR`, macOS/Linux home paths, Windows `USERPROFILE`, custom `--settings-file`, and `--no-configure`. - Add durable Headroom-owned restore state and refuse malformed settings or conflicting user edits. - Add unit, CLI, and Docker-harness e2e coverage for configuration, real proxy forwarding, and restoration. - Document setup, remote development, undo, and troubleshooting. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ UV_NO_SYNC=1 uv run pytest -q tests/test_provider_claude_vscode_config.py tests/test_cli/test_wrap_vscode_claude.py tests/test_cli/test_wrap_vscode.py tests/test_cli/test_wrap_claude_base_url.py tests/test_provider_copilot_vscode_config.py tests/test_copilot_auth.py 160 passed in 0.45s $ UV_NO_SYNC=1 uv run ruff check . All checks passed! $ UV_NO_SYNC=1 uv run mypy headroom Success: no issues found in 512 source files $ npm run build # from docs/ Compiled successfully; generated 155 static pages ``` ## Real Behavior Proof - Environment: macOS, Python 3.13 editable install, isolated temporary HOME and Claude settings, local mock Anthropic Messages upstream. - Exact command / steps: invoked the new `verify_vscode_claude_wrap` e2e function, which launched real `headroom wrap vscode-claude`, waited for proxy readiness, POSTed an Anthropic `/v1/messages` request through the generated project-scoped URL, stopped the wrapper, then ran `headroom unwrap vscode-claude`. - Observed result: HTTP 200 with the mock Claude response through Headroom; generated settings retained unrelated values and enabled tool deferral; unwrap restored the original Claude settings. - Not tested: real Anthropic account traffic or the full Docker image locally because Docker Desktop was unavailable. The same e2e function is wired into the existing Docker wrap CI job. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable; this adds CLI configuration and proxy routing without changing VS Code UI. ## Additional Notes The wrapper deliberately leaves the endpoint configured when stopped so requests fail closed instead of silently bypassing Headroom. `headroom unwrap vscode-claude` restores the exact prior managed values and preserves unrelated settings. --------- Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net> |
||
|
JD Davis
|
007446c73a
|
feat(copilot): proxy VS Code models transparently (#2687)
## Description Make Headroom a transparent proxy for VS Code GitHub Copilot. Users keep using Copilot's normal model picker—GPT-4.1, Claude Sonnet, Claude Opus, and other models in their entitlement—while Headroom silently forwards the selected model instead of registering or requiring a separate "Headroom" model. This also fixes GitHub's device OAuth exchange by sending form-encoded request bodies, matching the endpoint contract. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `headroom wrap vscode` to start a Copilot-seeded subscription proxy and safely configure VS Code's shipped Copilot proxy override. - Add `headroom unwrap vscode` for reversible cleanup. - Preserve VS Code's selected model by changing only the proxy URL/auth override; no custom model is registered and no model preference is written. - Support stable VS Code settings locations on macOS, Windows, and Linux, plus `--settings-file` for Insiders, portable, and other installations. - Edit JSONC settings with a marker-owned block while preserving unrelated bytes, comments, ordering, and trailing commas. - Refuse malformed markers, invalid JSONC, or unmanaged existing Copilot overrides instead of overwriting user configuration. - Fix SIGINT cleanup so the managed settings block is removed and normal shutdown exits successfully. - Fix Copilot device OAuth start/poll requests to use `application/x-www-form-urlencoded`. - Add a compatibility matrix, setup/removal flow, credential behavior, remote-development guidance, enterprise notes, troubleshooting, and verification documentation. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ .venv/bin/pytest -q tests/test_provider_copilot_vscode_config.py tests/test_cli/test_wrap_vscode.py tests/test_cli/test_wrap_helpers.py tests/test_copilot_auth.py tests/test_cli/test_copilot_auth.py tests/test_provider_copilot_wrap.py tests/test_cli/test_wrap_copilot.py tests/test_copilot_provider_label.py tests/test_copilot_subscription_smoke.py 244 passed in 0.59s $ .venv/bin/ruff check <changed Python files and tests> All checks passed! $ .venv/bin/mypy headroom/providers/copilot/vscode.py Success: no issues found in 1 source file $ cd docs && npm run types:check fumadocs-mdx && next typegen && tsc --noEmit # exited 0 $ git diff --check # exited 0 ``` The full 10,179-test suite was also sampled through approximately 83%, but was stopped because of its runtime. It exposed existing failures in `test_recover_codex.py`, `test_wrap_stale_marker.py`, and `test_proxy_health.py`; therefore the broad `pytest`, repository-wide Ruff, and repository-wide mypy boxes are intentionally not checked. ## Real Behavior Proof - Environment: macOS arm64, VS Code 1.131.0, built-in GitHub Copilot 0.59.0, Headroom 0.33.1-dev. - Exact command / steps: 1. Completed `headroom copilot login` with GitHub's device flow. 2. Ran `.venv/bin/headroom wrap vscode --port 8788`. 3. Confirmed VS Code retained its ordinary Copilot model catalog and made `GET /models` through Headroom with `GitHubCopilotChat/0.59.0` and `editor-version: vscode/1.131.0`. 4. Sent native Copilot `/p/headroom/chat/completions` requests through the same endpoint using `gpt-4.1`, `claude-sonnet-4.6`, and `claude-opus-4.7`. - Observed result: - All three completion requests returned HTTP 200. - GPT-4.1 resolved upstream to `gpt-4.1-2025-04-14`; Sonnet and Opus retained their exact selected IDs. - All returned the requested exact marker content. - VS Code's settings contained only the Headroom proxy URL and token auth override—no Headroom model or model-selection setting. - The proxy health endpoint remained ready with `openai_api_url` set to `https://api.githubcopilot.com`. - Not tested: - Physical Windows or Linux hosts (their path/config behavior is covered by unit tests). - WSL, dev containers, SSH remotes, VS Code Insiders, or enterprise Copilot deployments end-to-end. - Every model in the live Copilot catalog. - A fully submitted chat from VS Code's UI automation; the real extension's catalog request and native completion paths were verified separately. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing targeted unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) Not applicable; this integration intentionally has no separate UI or model entry. ## Additional Notes The integration uses VS Code Copilot's shipped advanced/debug proxy endpoint seam. The managed settings block is deliberately narrow and reversible. Remote extension hosts may need their own reachable proxy/configuration as documented. --------- Co-authored-by: JerrettDavis <2610199+JerrettDavis@users.noreply.github.com> |
||
|
Tejas Chopra
|
e0ce4b1d48
|
fix: remove rtk and lean-ctx CLI context tools (#2677)
## Description Removes both third-party CLI context tools — **rtk** and **lean-ctx** — and with them the context-tool selector itself. Headroom no longer downloads, installs or configures either one, and there is no replacement. The previous pass (#2344) gated only three entry points inside `headroom/cli/wrap.py`. That left the feature reachable in practice: | Gap | Effect | |---|---| | `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global --auto-patch` from bash/PowerShell, **bypassing the Python gate entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook regardless of `HEADROOM_RTK` | | `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was broken by default**: `rtk_required=True` met a gate returning `None` → `SystemExit(1)`. Invisible because all 8 openhands tests patched `_ensure_rtk_binary` to a fake path | | `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to `rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) | | No cleanup path | Nothing removed artifacts an earlier default had installed, so a machine that once ran the old default kept rtk in the loop forever (#1669, #1955) | Also worth noting: the rtk binary download had **no SHA or signature verification** — only `rtk --version` as a smoke test. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [x] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [x] Code refactoring (no functional changes) ## Changes Made **Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages, `headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` / `_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` / `--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers, `benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path filters. **Fails loudly, not silently** — `--context-tool` / `--no-context-tool` / `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in shell profiles, aliases and CI jobs, and accepting them as a no-op would read as Headroom having quietly stopped working. The installers reject them too, which matters more than it looks: their arg parsers forward the first unknown flag **and everything after it** to the wrapped tool, so a leftover `--no-rtk` would have silently swallowed a following `--port` and then been ignored downstream. **New `headroom/context_tool_cleanup.py`** — deleting the code cannot help a machine that already ran the old default, since the hooks, binaries and injected guidance are durable on disk. `purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and removes the registered hook entries, the generated hook scripts, the Headroom-managed `~/.local/bin` symlinks, the vendored `~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server entry and the marker-fenced instruction blocks. Deliberately conservative: idempotent, **skips** a malformed config rather than overwriting it, and only unlinks a symlink resolving inside Headroom's own bin dir so a user's own build is untouched. It reports on **stderr**, because `wrap/unwrap openclaw --prepare-only` emit machine-readable JSON on stdout as their entire contract. Skipped for `wrap selfheal` (runs from a SessionStart hook; must not race Claude Code's writer for `~/.claude.json`) and for `--help`, which must stay read-only. **Client-config hardening** (discovered while investigating a "corrupted Serena settings file" report) — `wrap.py` reset a settings file to `{}` when an existing file would not parse, then wrote that back. One hand-edited typo or a transient `EACCES`/`EINTR` on a valid file destroyed the user's `permissions`, `env` and `hooks`, on **every `headroom wrap claude`**. It now refuses to write. Separately, `fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`), fixing all 14 non-atomic client-config writes at once; it follows symlinks rather than replacing them (dotfile managers) and preserves an existing file's mode. **Deliberately kept** — `rtk` stays in the wrapper-peel list in `transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as shell-command grammar, so `rtk cat f` is still classified as a file read for anyone running their own rtk install, which the purge intentionally leaves alone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates All checks passed! $ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates 1255 files already formatted $ mypy headroom/ Success: no issues found in 508 source files $ pytest tests/test_context_tool_cleanup.py -q 11 passed $ pytest tests/test_fsutil.py -q 12 passed $ pytest tests/test_cli/test_wrap_codex.py -q # 89 tests 89 passed in 431.68s $ pytest tests/test_cli/test_wrap_opencode.py -q 39 passed in 257.46s $ pytest tests/test_cli/test_wrap_helpers.py -q 45 passed $ pytest tests/test_paths.py -q 75 passed $ pytest tests/test_cli/test_unwrap_claude.py -q 14 passed $ pytest tests/test_proxy_savings_history.py -q 39 passed $ pytest tests/test_cli/test_wrap_copilot.py -q 27 passed $ pytest tests/test_cli/test_wrap_zcode.py -q 20 passed $ pytest tests/test_subscription_tracker.py -q 9 passed $ pytest tests/test_proxy_dashboard_stats_cache.py -q 5 passed, 1 skipped ``` Repo-wide grep for 14 removed symbols (`headroom.rtk`, `headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`, `_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`, `wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`, `tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`, `*.html`: **zero hits**. Notable test changes: `test_wrap_openhands.py` no longer patches `_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0 unpatched — the regression that was previously masked. `test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed (every test drove RTK instruction injection). A new `test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed` proves a pre-removal `subscription_state.json` still loads. ## Real Behavior Proof - **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @ this branch, real `~/.headroom` and `~/.claude` on the dev machine. - **Exact command / steps and observed result:** ```text # 1. Retired flag fails loudly instead of silently no-op'ing $ headroom wrap codex --prepare-only --context-tool rtk Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they rewrote shell commands through a third-party binary Headroom no longer manages. Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL; `headroom wrap` uninstalls what they left behind on first run. $ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ... # 2. install.sh rejects the retired flags (extracted parse_wrap_args harness) ['--no-rtk', '--port', '9999'] rc=1 ERROR: CLI context tools ... Drop --no-rtk ['--context-tool=rtk'] rc=1 ERROR: CLI context tools ... Drop --context-tool $ bash -n scripts/install.sh # syntax OK # 3. Purge ran against the real machine, which had all the orphaned artifacts $ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..." removed ~/.headroom/bin/lean-ctx (51 MB) removed ~/.headroom/bin/rtk (7.7 MB) removed ~/.local/bin/rtk (symlink into ~/.headroom/bin) removed ~/.claude/hooks/rtk-rewrite.sh removed 8 lean-ctx-* hook scripts # ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged # → ~59 MB reclaimed, no unrelated key touched # 4. stdout stays machine-readable while the purge reports (planted a fake artifact) $ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err $ cat out {"enabled":true,"config":{"proxyPort":8787,...}} # parses as JSON $ cat err Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk # 5. --help is inert (planted artifact survives), a real run purges $ headroom wrap codex --help → artifact survived: CORRECT $ headroom wrap openclaw --prepare-only → purged: CORRECT # 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json top-level keys 90 -> 90; projects 19 -> 19; LOST keys: none all content outside mcpServers byte-identical: True ``` Dashboard rendered via the Playwright test after the panel removal: "Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`, and "Token Usage" reads Before Compression → Proxy Removed → After Compression with no "Filtered (this session)" row. Nothing below the removed panel broke. - **Not tested:** Windows and Linux (macOS only) — `install.ps1` is verified by brace-balance and inspection, not executed, since no `pwsh` is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated but not run; it needs the Docker e2e image. `serena project index` interaction is exercised in the stacked base PR. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes **Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge that first; this PR's base should then be retargeted to `main`, or it will read as containing that fix too. **Breaking-change migration for users:** - Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`, `--context-tool`, `--no-context-tool` from any alias, script or CI job, and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error rather than being ignored, so the failure is immediate and self-explaining. - Previously-installed artifacts are purged automatically on the next `wrap`/`unwrap`; no manual cleanup needed. - `headroom perf --json` no longer carries a `cli_filtering` key, and `/stats` no longer returns a `context_tool` section. **Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from `README.md`, `docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`, `docs/observability.md` and the matching `wiki/` pages. `REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED rather than deleted, to keep the planning record. **Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as dead code — its only caller was the `except KeyboardInterrupt` guarding the binary download, so with no download there is nothing slow left to interrupt. |
||
|
Tejas Chopra
|
44136ed042
|
fix(wrap): make RTK opt-in (off by default) across wrap subcommands (#2344)
## Description
RTK CLI-command filtering was set up **by default** across ~16 `wrap`
subcommands (copilot, codex, aider, cursor, cline, continue, goose,
openhands, opencode, grok, omp, openclaude, vibe, …) via `if not
no_rtk:` — so users got rtk hooks / instruction injection without opting
in. `wrap claude` was the lone exception (already gated on
`--context-tool`).
This makes RTK **opt-in (off by default)** everywhere, so Headroom's own
savings are what's measured unless a user explicitly wants rtk.
Closes #
## Type of Change
- [x] Bug fix (behavior change: default flip)
## Changes Made
- **Central gate** `_rtk_opt_in()` — RTK runs only when explicitly
enabled via `--rtk` or `HEADROOM_RTK=1`. Guards the 3 RTK entry points
(`_setup_rtk`, `_ensure_rtk_binary`, `_inject_rtk_instructions`) so they
no-op by default — one small change instead of editing ~30 call sites.
- **`--rtk` opt-in flag** on all 18 tool subcommands via a shared
eager-callback option (`expose_value=False`, sets `HEADROOM_RTK=1`; no
subcommand signature changes).
- `wrap claude`'s legacy `--context-tool` still opts in (mirrored into
the gate).
- **`--no-rtk` kept** as an accepted, now-redundant no-op (back-compat).
- lean-ctx and all non-RTK behavior untouched.
## Testing
```text
pytest tests/test_wrap_rtk_opt_in.py -> 4 passed
ruff check / format -> clean
mypy headroom -> Success: no issues found in 504 source files
```
Verified `--rtk` appears in `wrap {claude,codex,copilot} --help`;
`_rtk_opt_in()` is False by default, True with `HEADROOM_RTK=1`; entry
points no-op + write nothing when off.
## Real Behavior Proof
- Env: local `.venv`, click CliRunner.
- Steps: import wrap; assert gate default-off / env-on; assert
`_setup_rtk`/`_ensure_rtk_binary` return None and
`_inject_rtk_instructions` returns False + writes no file when not opted
in; assert `--rtk` in subcommand help.
- Observed: all pass. Not tested: a live end-to-end wrap launch (proxy
spawn).
## Notes
Part 2 of 3 (RTK opt-in). Separate PRs cover the code-graph MCP engine
and the proxy-option cleanup. No `CHANGELOG.md` edit — release-please
generates it from the PR title (per the changelog guard).
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
Rudimar Ronsoni
|
dec60de976
|
fix(codex): preserve wrapped sessions and recover state (#2160)
## Description Closes #2159. Codex wrappers currently launch against a disposable `CODEX_HOME`, so session state created during a wrapped run can disappear when that temporary directory is removed. This change launches Codex against its durable home, keeps proxy routing process-local, and adds recovery for retained temporary homes and pinned recovery sources. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Launch Codex against its durable `CODEX_HOME` and apply routing through process-local config overrides after the actual proxy port is resolved. - Preserve custom provider identity and reject providers that cannot be redirected safely. - Detect dangling temporary Codex homes before interactive wraps and offer recovery. - Add `headroom recover codex` with automatic discovery, repeatable `--source`, preview, confirmation, retained backups, and rollback on failure. - Search Python's temp root, `$TMPDIR`, `/tmp`, `/private/tmp`, and macOS `/private/var/folders/*/*/T` for retained `headroom-codex-home-*` directories. - Reuse `source-pinned/` copies left by interrupted or failed recovery attempts after the original temporary home has disappeared. - Report deleted temporary homes still referenced by SQLite rollout paths without treating paths pasted into prompts or errors as filesystem evidence. - Audit the durable thread index, rollout files, and history when no source remains, including indexed chat counts and history-only orphan records. - Normalize legacy localhost `headroom` providers in both SQLite thread rows and rollout `session_meta`, including retries after an earlier broken recovery, while preserving user-defined remote providers named `headroom`. - Merge compatible config, JSONL, rollout, SQLite, credential, and regular-file state without propagating deletions or runtime artifacts. - Rewrite recovered thread rollout paths to the durable home and restore legacy Headroom thread providers to the active provider. - Validate SQLite schemas, SQLx migration checksums, integrity, and foreign keys, and quarantine malformed JSONL. - Preserve failed targets with an atomic rename before rollback, avoiding recursive-deletion races with live SQLite runtime files. - Document discovery, migration, retained backups, rollback behavior, and the limits of deleted-source recovery. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed in isolated Docker containers ### Test Output ```text $ uv run pytest tests/test_cli/test_wrap_codex.py tests/test_cli/test_recover_codex.py -q 122 passed $ uv run ruff check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py All checks passed! $ uv run ruff format --check headroom/cli/recover.py headroom/providers/codex/recovery.py tests/test_cli/test_recover_codex.py 3 files already formatted $ uv run mypy headroom/cli/recover.py headroom/providers/codex/recovery.py Success: no issues found in 2 source files ``` All validation ran in `ghcr.io/astral-sh/uv:python3.12-bookworm` against a writable disposable copy of a read-only source mount. Codex was not installed or launched, and no real user Codex state was read or modified. The tests cover multi-root discovery, deleted-reference reporting, retained pinned-source recovery, durable SQLite path relocation, SQLite and rollout provider normalization, idempotent repair after an earlier broken recovery, remote provider preservation, unrelated dangling target rows, backup retention, atomic rollback, malformed-state quarantine, SQLite validation, and Windows-safe handle closure. The repository shim E2E was not launched locally because this recovery work intentionally avoids launching Codex. Upstream CI exercises wrapper E2E in isolated environments. ## Real Behavior Proof - Environment: `ghcr.io/astral-sh/uv:python3.12-bookworm`, Python 3.12, a writable disposable checkout copied from a read-only source mount, at head ` |
||
|
Rod Boev
|
ad9d086f43
|
feat(codex): keep wrap routing session-scoped (#1507)
## Description Keeps Codex wrap routing session-scoped so routing state from one wrapped session does not leak into another. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Scope Codex wrap routing state to the active session. - Avoid cross-session routing contamination for wrapped Codex traffic. - Keep changes focused on wrap/proxy routing behavior. ## Testing - [x] Unit tests pass - [x] New tests added for new functionality when applicable - [x] Manual testing performed ### Test Output ```text Focused tests/review were completed before this governance body cleanup. The current branch is conflicted and still needs merge resolution before final merge readiness. ``` ## Real Behavior Proof - Environment: Headroom development/review context. - Exact command / steps: Reviewed session-scoped Codex wrap routing behavior and existing focused coverage. - Observed result: Routing state is scoped to the active wrap session rather than shared globally across sessions. - Not tested: Current conflicted branch after merge resolution; conflicts still need to be resolved before merge. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes This body was normalized by a maintainer after approval so the governance parser reflects the already-reviewed PR state. The PR remains blocked by merge conflicts. |
||
|
Parideboy
|
1573f1fd07
|
fix: use rtk native Cursor hook instead of injecting .cursorrules (#756) (#1846)
## Description `headroom wrap cursor` unconditionally injected an `rtk`-usage instructions block into `.cursorrules`. rtk itself supports a native hook for Cursor (`rtk init --agent cursor`) — the same registration mechanism headroom already uses for Claude Code — which rewrites shell commands transparently with zero custom-instructions text needed. Headroom never tried that path for Cursor, so users got a redundant `.cursorrules` file duplicating guidance the native hook already provides silently. A follow-up commit hardens the switch: `register_agent_hooks` returns `True` on rtk exit 0, but some rtk builds exit 0 without writing `~/.cursor/hooks.json`. headroom now trusts the on-disk hook file, not the exit code, before skipping the `.cursorrules` fallback. Closes #756 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/rtk/installer.py`: generalized `register_claude_hooks` into `register_agent_hooks(rtk_path, *, agent="claude")`, which passes `--agent <agent>` to `rtk init` for non-Claude agents. `register_claude_hooks` kept as a thin wrapper for backward compatibility. Added `RTK_NATIVE_HOOK_AGENTS` documenting which agents rtk supports a native hook for. - `headroom/cli/wrap.py`: `wrap cursor` now calls `register_agent_hooks(rtk_path, agent="cursor")` first, and only skips the `.cursorrules` fallback when `~/.cursor/hooks.json` is actually on disk; otherwise it falls back to `_inject_rtk_instructions(...)`. - Tests: `tests/test_rtk_installer.py` and `tests/test_cli/test_wrap_bridge.py` cover the native-hook path, the on-disk verification, and the `.cursorrules` fallback. - `CHANGELOG.md`: added an entry under `## Unreleased` / `### Fixed`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m ruff format --check headroom/ tests/ e2e/ 953 files already formatted $ python -m ruff check headroom/cli/wrap.py headroom/rtk/installer.py tests/test_cli/test_wrap_bridge.py tests/test_rtk_installer.py All checks passed! $ python -m pytest tests/test_cli/test_wrap_bridge.py -k cursor -q 3 passed ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.13, local checkout; `python -m pytest` / `ruff` run directly. - Exact command / steps: `python -m pytest tests/test_cli/test_wrap_bridge.py -k cursor -q` — the first test mocks `register_agent_hooks` to write `~/.cursor/hooks.json` and asserts `.cursorrules` is NOT created; the second mocks it to write nothing and asserts `.cursorrules` IS created with the `headroom:rtk-instructions` marker; the third exercises the explicit registration-failure fallback. - Observed result: `3 passed`. Native-hook path skips `.cursorrules` only when the hook file exists on disk; every other outcome falls back to `.cursorrules`, so Cursor always gets RTK guidance. - Not tested: real `rtk` binary writing `~/.cursor/hooks.json` end-to-end — that path is covered by the `docker-wrap-e2e` CI job, not locally. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI-only change. ## Additional Notes Scope: rtk's native-hook-capable agents include `claude`, `cursor`, `windsurf`, `cline`, `kilocode`, `antigravity`, `pi`, `hermes`, but only `cursor` and `claude` have a corresponding `headroom wrap` subcommand today, so this fix only changes `wrap cursor` behavior. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Rod Boev
|
22def93177
|
fix(mcp): register managed installs with a resolvable headroom command (#1386)
## Description Managed Headroom installs can register the MCP server with a bare `headroom mcp serve` command even when the active runtime lives in a venv outside `PATH`. That leaves Claude and Codex with a registration they cannot re-launch reliably, and Claude eventually fails with `Failed to reconnect to headroom: ENOENT`. This PR reuses the existing runtime command resolver when building the shared Headroom MCP spec, so the generated registration follows the active install instead of assuming `headroom` is globally discoverable. It also updates the shared-builder and registrar tests so the proof rows now flow through `build_headroom_spec()` and prove the same resolved command contract on both the Claude CLI path and the Codex TOML path. A follow-up CI fix keeps the Docker init E2E expectation aligned with that same resolver-backed contract. Closes #487 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/mcp_registry/install.py`: build the Headroom MCP server spec from the canonical runtime command resolver instead of hardcoding `headroom mcp serve` - `tests/test_mcp_registry/test_install.py`: cover the shared builder's direct-binary and module-fallback command shapes - `tests/test_mcp_registry/test_claude_registrar.py`: prove the Claude CLI registration forwards the resolved command vector end to end - `tests/test_mcp_registry/test_codex_registrar.py`: prove the Codex registrar writes the same resolved command vector into TOML - `e2e/init/run.py`: derive the Docker init E2E's expected Claude MCP registration argv from `resolve_headroom_command()` so the CI harness follows the same runtime contract - `CHANGELOG.md`: note the managed-install MCP registration fix ## Testing - [x] Unit tests pass (`uv run pytest tests/test_mcp_registry/test_install.py -v`, `uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v`, `uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v`) - [x] Linting passes (`uv run ruff check .`) - [ ] Type checking passes (`uv run mypy headroom`) or explain N/A truthfully - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_mcp_registry/test_install.py -v ============================= 12 passed in 0.13s ============================== $ uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v ============================= 24 passed in 0.18s ============================== $ uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v ============================= 25 passed in 0.20s ============================== $ uv run python -c "from e2e.init.run import _expected_headroom_mcp_call; print(_expected_headroom_mcp_call('http://127.0.0.1:9011'))" ['mcp', 'add', 'headroom', '-s', 'user', '-e', 'HEADROOM_PROXY_URL=http://127.0.0.1:9011', '--', '.../headroom', 'mcp', 'serve'] $ uv run ruff check e2e/init/run.py All checks passed! $ uv run ruff format e2e/init/run.py --check 1 file already formatted $ uv run ruff check . All checks passed! $ uv run ruff format . --check 987 files already formatted ``` `uv run mypy headroom` was not run locally; this repo's focused local gate for the touched Python registry path is the targeted pytest set plus Ruff. ## Real Behavior Proof - Environment: managed-install-safe MCP registration path, Python 3.11+, no provider required - Exact command / steps: run the focused MCP registry pytest files, inspect the captured Claude CLI argv and rendered Codex TOML block, and verify the Docker init E2E expectation derives its Claude MCP argv from the same runtime helper - Observed result: the persisted MCP registration uses a resolvable command tied to the active Headroom runtime instead of bare `headroom`, while `HEADROOM_PROXY_URL` handling stays unchanged - Not tested: full live Claude reconnect against a real managed venv, unless that is run during implementation ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - Scoped to the MCP registration slice in `#487`. The RTK hook rewriting thread from the same issue is intentionally out of scope here. - `@erikpr1994` isolated the managed-install `ENOENT` failure mode in the issue thread and narrowed it to the bare-command MCP registration path. - If existing owned registrations with the old bare-command contract need an in-place upgrade path, that should be handled explicitly in the final diff rather than left implicit. |
||
|
Parideboy
|
487aa71a3c
|
ci: restore green lint (reformat for ruff 0.15.17, fix mypy no-any-return, pin linters) (#1295)
## Description
The CI lint job (`ruff check .` → `ruff format --check .` → `mypy
headroom`) was red on `main`
and therefore on every open PR, for two unrelated reasons that the early
ruff failure was masking:
1. **ruff**: the lint job installs `ruff` unpinned, and ruff 0.15.17
began enforcing import-block
sorting (`I001`) and formatting that older ruff accepted → `ruff check
.` / `ruff format --check .`
fail on files nobody touched.
2. **mypy**: `headroom/providers/opencode/config.py` had two `return
json.loads(...)` statements in
a function declared `-> dict[str, Any]`; `json.loads` is typed `Any`, so
`mypy headroom` fails
with `no-any-return` (reproduced on mypy 1.20.2 — not a version-specific
quirk).
This restores a green lint baseline and pins both linters so a future
release can't silently break
CI again.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `.github/workflows/ci.yml`: pin `ruff==0.15.17` and `mypy==1.20.2` in
the lint job.
- Applied `ruff check --fix .` (6 `I001` import-sort fixes) and `ruff
format .` (10 files) across
the repo — import ordering and whitespace only, no behavior change.
- `headroom/providers/opencode/config.py`: narrow both
`_parse_json_loose` return sites with an
`isinstance(parsed, dict)` guard, so the `dict[str, Any]` annotation is
true at runtime
(non-dict JSON falls back to `{}`) and mypy's `no-any-return` is
resolved.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`, `ruff format --check .`, `mypy
headroom --ignore-missing-imports`)
### Test Output
```text
$ python -m ruff check .
All checks passed!
$ python -m ruff format --check .
913 files already formatted
$ python -m mypy headroom/providers/opencode/config.py --ignore-missing-imports
Success: no issues found in 1 source file
$ python -m pytest tests/test_providers_opencode_config.py -q
37 passed
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.13, ruff 0.15.17, mypy 1.20.2,
branch ci/fix-ruff-lint off
headroomlabs-ai/main
- Exact command / steps: reproduced the red lint (latest ruff: 6 `I001`
+ 10 unformatted files;
the mypy failure was read from the #1295 CI lint log —
`config.py:125,133 no-any-return`, and
re-confirmed locally on mypy 1.20.2). Applied the ruff auto-fix/format,
added the dict guard,
pinned both linters, and re-ran each lint step.
- Observed result: `ruff check .` → "All checks passed!"; `ruff format
--check .` → "913 files
already formatted"; `mypy` on the fixed file → "Success: no issues
found"; full `mypy headroom`
reports only Unix `fcntl` attributes that don't exist on this Windows
box (present on the Linux
CI runner, where the prior run showed exactly the two now-fixed errors).
37 opencode-config
tests pass.
- Not tested: did not run the full OS/Python test matrix — the change is
formatting + two CI
dependency pins + a two-line type-narrowing guard, with no runtime
behavior change for dict JSON.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Rudimar Ronsoni
|
b4571cc346
|
feat: headroom wrap opencode / unwrap opencode CLI (#1105)
## Summary This PR implements transparent `headroom wrap opencode` support without asking users to edit OpenCode provider URLs, choose an extra CLI flag, or maintain a static provider list. The wrapper now lives at the runtime transport boundary: OpenCode keeps its user/provider config, while Headroom intercepts outbound provider traffic in-process and routes it through the local Headroom proxy. ## What changed ### Transparent OpenCode wrapping - `headroom wrap opencode` injects the `headroom-opencode` plugin through `OPENCODE_CONFIG_CONTENT`. - Existing OpenCode provider URLs are preserved. We do not rewrite user config URLs to point at Headroom. - Existing `OPENAI_BASE_URL` and `ANTHROPIC_BASE_URL` env vars are preserved. - Local OpenCode traffic, localhost traffic, and Headroom proxy traffic bypass the shim to avoid loops. ### Runtime transport interception - Added an OpenCode plugin transport shim that wraps: - `globalThis.fetch` - `http.request` / `http.get` - `https.request` / `https.get` - External provider calls are routed to the local Headroom proxy. - The original upstream origin is passed through `x-headroom-base-url`, so the proxy can forward to the real provider without changing OpenCode config. - External `http2.connect` is blocked loudly instead of allowing direct provider traffic to leak outside Headroom. ### Live provider additions Provider coverage is no longer based on a static config scan. Because routing happens at outbound request time, providers added mid-session are routed through Headroom automatically as long as they use the covered Node transport paths. ### Subagent and child-process coverage - The parent OpenCode plugin sets a packaged Node preload shim through `NODE_OPTIONS=--import=.../hook-shim/handler.js`. - The transport shim patches `child_process.spawn`, `exec`, `execFile`, and `fork` so child Node processes receive the Headroom preload even when OpenCode passes a custom `env`. - The child-process shim fails closed if it loads without `HEADROOM_OPENCODE_TRANSPORT_PROXY_URL`. - This closes the subagent leak path where a child Node process could otherwise start without Headroom transport interception. ## Why this goes beyond PR #1089 PR #1089 improves OpenCode provider registration, but it still focuses on provider config shape. This PR moves the enforcement boundary to runtime transport interception. This PR goes further because: - No provider URL rewriting is required. - New providers added mid-session are covered automatically. - Subagents and child Node processes inherit the Headroom transport shim. - Direct external HTTP/2 paths fail loudly instead of leaking. - The wrap remains transparent to the user's OpenCode provider config. - The wrapper is fail-closed for unsupported child-process preload state. ## Additional robustness fixes While validating the change in Docker, the full Python suite exposed unrelated Linux/container robustness issues. These are fixed in this PR so the suite is green: - Binary cache handling now treats cache paths under a non-writable existing parent as unavailable, including when tests run as root in Docker. - `release_version.py` honors `MANUAL_VER` before git calls so direct script execution works outside a `.git` checkout. - Test logger isolation now resets relevant Headroom child loggers so proxy logging setup cannot poison later `caplog` tests. - The scanner missing-path test now uses a guaranteed missing `tmp_path` child instead of relying on `/nonexistent/path`. ## Validation All implementation validation was run inside Docker. - Full Python suite from a fresh Docker copy: `6605 passed, 523 skipped`. - Ruff on changed Python/OpenCode paths: passed. - OpenCode plugin typecheck: passed. - OpenCode plugin tests: `9 passed`. - OpenCode plugin build: passed. - Hook shim preload smoke test: passed. ## Notes This PR intentionally does not add a CLI option. `headroom wrap opencode` means full wrap. Either Headroom wraps OpenCode transparently, or the path fails loudly instead of silently leaking provider traffic. --------- Co-authored-by: Rudimar Ronsoni <6081613+rudironsoni@users.noreply.github.com> |
||
|
Tejas Chopra
|
bc12acef59
|
fix(e2e): align Codex wrap e2e with global-only RTK guidance (#1240) (#1254)
## Description `main` is red on the **Wrap E2E** workflow and on CI's **`docker-native-e2e`** job. Both run `e2e/wrap/run.py` and fail on the same assertion: ``` e2e/wrap/run.py:553 assert_true(project_agents.exists(), "Codex wrap should create project AGENTS.md") AssertionError: Codex wrap should create project AGENTS.md ``` PR #1240 (`fix(wrap): keep Codex RTK guidance global`) intentionally moved Codex RTK guidance to the global `~/.codex/AGENTS.md` and stopped writing a project-level `AGENTS.md` (a project `AGENTS.md` is now created only when `wrap codex --memory` is used, for memory guidance). #1240 updated its unit test (`tests/test_cli/test_wrap_codex.py`) but not the wrap **e2e** harness, so `verify_codex_wrap` still asserted the old project-level behavior. This corrects the e2e harness to match the shipped behavior — it is a stale-test fix, not a behavior change. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `e2e/wrap/run.py` `verify_codex_wrap`: removed the two now-false project-level assertions (`project_agents.exists()` and the project RTK-marker check) and the unused `project_agents` variable. - Kept the global assertions (`~/.codex/AGENTS.md` exists + contains the RTK marker) — these already match the shipped behavior. - Added a comment documenting that Codex RTK guidance is global-only (#1240) and a project `AGENTS.md` appears only with `--memory`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check e2e/wrap/run.py All checks passed! $ python -m pytest tests/test_cli/test_wrap_codex.py -q ============================== 57 passed in 6.13s ============================== # includes test_wrap_codex_injects_rtk_globally_without_changing_project_agents, # which asserts the RTK marker lands in ~/.codex/AGENTS.md and the project # AGENTS.md is left byte-for-byte unchanged — the contract this e2e now matches. ``` ## Real Behavior Proof - Environment: macOS (darwin 25.4.0), Python 3.12 venv; root-caused from the failing CI logs and verified the behavior contract via the unit suite (the Docker wrap-e2e itself runs in CI) - Exact command / steps: read the failing step logs for CI run `27912260743` and Wrap E2E run `27912260746` (both fail at `e2e/wrap/run.py:553`); confirmed via `headroom/cli/wrap.py:3679` that RTK injects only into `~/.codex/AGENTS.md`; ran `pytest tests/test_cli/test_wrap_codex.py` and `ruff check e2e/wrap/run.py` - Observed result: 57/57 codex-wrap unit tests pass; `test_wrap_codex_injects_rtk_globally_without_changing_project_agents` confirms the RTK marker is written to `~/.codex/AGENTS.md` while the project `AGENTS.md` is left unchanged — exactly what the corrected e2e asserts. ruff clean. - Not tested: the full Docker `Wrap E2E` / `docker-native-e2e` jobs locally (require Docker + a wheel build); they run on this PR's CI to confirm the fix turns both jobs green. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — e2e harness fix; evidence is under "Real Behavior Proof". ## Additional Notes - `mypy` / "new tests added" are unchecked: this is a test-only correction to an existing e2e assertion, no production code or new test surface. - Root-cause detail: a project-level `AGENTS.md` is created by `wrap codex` only inside the `if memory:` branch (`headroom/cli/wrap.py:3704`/`3715`); the e2e runs `wrap codex -- --help` without `--memory`, so no project file is created — the assertion could never pass after #1240. - `ruff check .` scoped to the changed file here (the dashboard HTML template trips ruff's `invalid-syntax`, a known repo false-positive). |
||
|
Logan Kang
|
dff6a19946
|
fix(codex): write canonical hooks feature flag and migrate deprecated codex_hooks (#743)
## Description `headroom init codex` writes the hooks feature flag into `.codex/config.toml` under the key `codex_hooks`. Codex renamed the canonical key to `hooks` and kept `codex_hooks` as a legacy alias (openai/codex#20522). Current Codex builds warn about `[features].codex_hooks` and tell users to use `[features].hooks` instead, so configs written by headroom should stop emitting the deprecated key. This PR switches headroom to write the canonical `hooks` key and **migrates existing configs in place**. The migration is the tricky part: a config can already contain `codex_hooks`, `hooks`, or both, in any order, inside or outside headroom's marker block — and a naive replace can emit a *duplicate* `hooks` key, which is invalid TOML that Codex rejects outright. The fix strips every `codex_hooks` line up front (any value, anywhere) — mirroring the existing top-level key cleanup in `_ensure_codex_provider` (#260) — then guarantees `hooks` is present without ever duplicating it, and respects a user-managed `hooks` value that lives outside our marker block. Fixes: N/A (no tracking issue — surfaced while aligning with Codex >= 0.129; related upstream context: openai/codex#20522 and the warning behavior discussed in openai/codex#22148) ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Write the canonical `hooks` key instead of the deprecated `codex_hooks` in `_ensure_codex_feature_flag` (`headroom/cli/init.py`). - Strip any `codex_hooks` line (any value, inside or outside the marker block) before ensuring the flag, so re-running `init` migrates a legacy config instead of leaving a stale key or producing a duplicate `hooks` key (invalid TOML). - Respect a user-managed `hooks` value found outside headroom's marker block (e.g. `hooks = false`); only the deprecated alias is removed. - Make the insert/create paths match `_replace_marker_block`'s normalisation so re-running `init` is byte-idempotent. - Extract a `_codex_feature_block()` helper to remove the 4x duplicated marker block assembly. - Add regression tests for the previously-broken edge cases. ## Testing - [x] Unit tests pass (`pytest`) — affected module fully green (see output) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom/cli/init.py`) - [x] New tests added for new functionality - [x] Manual testing performed (reproduced each edge case against the patched function via `tomllib.loads`) ## Test Output ``` $ pytest -v tests/test_cli/test_init_cli.py -k "feature_flag or hooks_feature" tests/test_cli/test_init_cli.py::test_init_codex_merges_feature_flag_into_existing_table PASSED tests/test_cli/test_init_cli.py::test_init_codex_creates_hooks_feature_flag_on_first_init PASSED tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_replaces_existing_marker PASSED tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_legacy_codex_hooks_key PASSED tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_when_both_keys_present PASSED tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_when_keys_reversed PASSED tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_drops_legacy_key_outside_marker PASSED tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_is_idempotent PASSED tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_creates_features_section_when_missing PASSED ======================= 9 passed, 45 deselected in 0.24s ======================= $ pytest -q tests/test_cli/test_init_cli.py 54 passed $ ruff check . All checks passed! $ mypy headroom/cli/init.py Success: no issues found in 1 source file ``` Note: the broader `tests/test_cli/` run has one unrelated failure (`test_wrap_copilot_auto_detects_running_proxy_backend`) caused by a real proxy already bound to port 8787 in the local environment — it fails identically on a clean checkout without this change. ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (none required) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (managed by release-please; generated from the conventional commit, not edited by hand) ## Screenshots (if applicable) N/A ## Additional Notes - **Why the duplicate-key path matters:** TOML forbids duplicate keys, so a `[features]` table containing both `codex_hooks` and `hooks` (which the old in-place migration could produce) makes Codex reject `config.toml` entirely. The new "strip then ensure" approach can never emit two `hooks` lines. - **Version provenance:** the `codex_hooks` -> `hooks` rename landed in openai/codex#20522, first shipped in Codex `rust-v0.129.0`. `codex_hooks` remains a working legacy alias, but current Codex builds can warn users to move to `[features].hooks`. - **Idempotency:** running `headroom init codex` repeatedly now produces a byte-stable `config.toml`, so there is no churn on re-init. |
||
|
Focused Instability
|
914a60a2b0
|
feat(proxy): per-project savings breakdown on the dashboard (claude, codex, aider, copilot, cursor) (#803)
## Summary Adds a **per-project savings breakdown** to the proxy dashboard, covering **all wrap-supported agents: Claude Code, Codex, aider, Copilot, and Cursor**. Spec and rationale in #802 (feature-request issue — happy to adjust scope per maintainer feedback). How it works — two attribution channels, by client capability: **Header channel** (clients that can send custom headers): - `headroom wrap claude` appends `X-Headroom-Project: <basename(cwd)>` to `ANTHROPIC_CUSTOM_HEADERS` (a user-supplied x-headroom-project header always wins; other user headers preserved). - `headroom wrap codex` extends the injected `[model_providers.headroom]` block with `env_http_headers = { "X-Headroom-Project" = "HEADROOM_PROJECT" }` and sets `HEADROOM_PROJECT` per launch — Codex only sends the header when the env var is set, so the static config stays inert outside wrap. **Base-URL prefix channel** (clients that cannot send custom headers): - The proxy accepts `/p/<url-encoded-name>/...`; middleware strips the prefix before routing (before Starlette caches the URL) and binds the project. The explicit header wins over the prefix. - `headroom wrap aider` points `OPENAI_API_BASE` / `ANTHROPIC_BASE_URL` at the prefixed URL. - `headroom wrap copilot` points `COPILOT_PROVIDER_BASE_URL` at the prefixed URL (BYOK anthropic/openai provider types and the GitHub-subscription path). - `headroom wrap cursor` prints the prefixed Override Base URL in its setup instructions, with a note explaining the attribution. **Shared plumbing:** - New `headroom/proxy/project_context.py`: header classification, `/p/` prefix split/strip, URL-prefix builder, and a contextvar bound per request (HTTP middleware + WS accept for the Codex responses bridge); the outcome funnel resolves it (explicit `RequestOutcome.project` wins), stamps `RequestLog.tags["project"]`, and forwards it through `PrometheusMetrics.record_request` into the `SavingsTracker`. - `SavingsTracker`: persisted state gains a `projects` map (requests, tokens saved, savings USD, input tokens/cost, last activity). Schema v2 → v3 with transparent forward migration (v2 files load cleanly, `projects` starts empty). Names sanitized (printable-only, 128-char cap); map capped at 50 projects, evicting the smallest bucket. - `/stats` exposes `savings.per_project` (and `projects`/`projects_limit` inside `persistent_savings`); `/stats-history` exposes `projects`. - Dashboard gains a "Per-Project Savings" table mirroring the per-model table (Alpine `x-text` only — names are user-supplied, no HTML injection). **Behavior changes:** none for unattributed traffic — no header and no prefix means no bucket, aggregate totals exactly as before (regression-tested: legacy `/stats` and `/stats-history` shapes are pinned by tests). ## Real behavior proof **Setup tested on:** macOS (Darwin 25.5.0), Python 3.11.9, `pip install -e ".[dev]"`, proxy on spare ports with isolated `HEADROOM_SAVINGS_PATH`; real Claude Code CLI (subscription auth) and real Codex CLI (ChatGPT auth) as clients. **Header channel — exact steps:** ``` HEADROOM_SAVINGS_PATH=/tmp/headroom-proof-savings.json .venv/bin/python -m headroom.cli proxy --port 9123 # background cd /tmp/proof-alpha && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap claude --port 9123 --no-rtk --no-mcp --no-serena -- -p "Reply with exactly: OK" cd /tmp/proof-beta && headroom wrap codex --port 9123 --no-rtk --no-mcp --no-serena -- exec --skip-git-repo-check "Reply with exactly: OK" ``` **Observed** (copied live `/stats` output; all runs replied `OK` through the proxy): ```json { "proof-beta": { "requests": 3, "tokens_saved": 140, "compression_savings_usd": 0.0007, "total_input_tokens": 38581, "total_input_cost_usd": 0.360433, "last_activity_at": "2026-06-10T09:19:17Z", "savings_percent": 0.36 }, "proof-alpha": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 9050, "total_input_cost_usd": 0.32245, "last_activity_at": "2026-06-10T09:18:09Z", "savings_percent": 0.0 } } ``` `proof-beta` aggregates one Claude Code turn + one Codex `exec` run (Codex attribution flows through `env_http_headers`). **Prefix channel — exact steps** (the same mechanism the aider/copilot/cursor wraps emit, driven by a real client): ``` .venv/bin/python -m headroom.cli proxy --port 9124 # background, isolated savings path ANTHROPIC_BASE_URL="http://127.0.0.1:9124/p/aider-style-project" claude -p "Reply with exactly: OK" ``` **Observed:** ```json { "aider-style-project": { "requests": 1, "tokens_saved": 0, "compression_savings_usd": 0.0, "total_input_tokens": 8571, "total_input_cost_usd": 1.018575, "last_activity_at": "2026-06-10T10:10:13Z", "savings_percent": 0.0 } } ``` `/stats-history` returned `schema_version: 3` with the same `projects` keys; `GET /dashboard` HTML contains the new "Per-Project Savings" table; persisted state survived a tracker reload; a hand-written v2 savings file loaded cleanly with an empty `projects` map. **What I did NOT test live:** the actual aider/Copilot/Cursor binaries end-to-end (their wraps emit exactly the prefixed URLs exercised above — unit tests pin the emitted env/URLs); Codex subscription-mode routing via the built-in `openai` provider (no provider headers there — such traffic simply stays unattributed); multi-worker uvicorn; Windows. ## Tests - `tests/test_proxy_project_savings.py` (17 tests): sanitization, header classification, `/p/` prefix split + URL-builder round-trip, tracker aggregation/persistence/migration/cardinality-cap/state-sanitization, funnel→`/stats` end-to-end, middleware binding for header + prefix + precedence, plus regression tests pinning legacy `/stats`/`/stats-history` shape and unattributed-traffic totals. - Wrap/provider tests extended: `test_cli/test_wrap_helpers.py`, `test_cli/test_wrap_codex.py`, `test_provider_aider.py`, `test_provider_cursor.py`, `test_provider_copilot_wrap.py` (prefixed env/URLs, user-override wins, no duplicate header, TOML block contents, block strip, setup-line note). - Full `pytest` suite run locally; `ruff check` + `ruff format` clean on all touched files. - `CHANGELOG.md` updated. ## Dependencies None added or bumped. Closes #802 (pending maintainer 👍 per CONTRIBUTING — raised there first with the full spec). --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com> |
||
|
chopratejas
|
c375fa156d |
fix(cli): wrap subcommands for cline, continue, goose, openhands
Adds four new `headroom wrap <agent>` subcommands so the proxy can front-end Cline (VS Code), Continue (VS Code/JetBrains), Goose (Block CLI), and OpenHands (CLI), extending the existing claude/codex/aider/copilot/ cursor pattern. Phase G PR-G1 of the realignment work. Architectural decision: extended the existing `headroom/cli/wrap.py` module in-place rather than splitting it into a `headroom/cli/wrap/` package. The spec at REALIGNMENT/09-phase-G-rtk-observability.md mentions per-agent files under a package, but the existing five wrap subcommands all live in the single module and the extension is small relative to the file. Keeping the file together preserves the simple import surface used by tests (`from headroom.cli import wrap as wrap_mod`). Per-agent wiring: - cline → injects RTK block into `.clinerules` at project root (Cline is a VS Code extension; API base URL is configured in the UI, so the command prints config instructions and blocks on the proxy). - continue → injects RTK guidance into the `systemMessage` field of `.continue/config.json` (idempotent; refuses malformed JSON or non-object roots; supports `--config` for custom paths). - goose → injects RTK block into `.goosehints` at project root and launches the goose CLI with OPENAI_BASE_URL / OPENAI_API_BASE / ANTHROPIC_BASE_URL env vars pointed at the proxy. - openhands → injects RTK guidance via the `OPENHANDS_INSTRUCTIONS` env var at launch (no on-disk artifact) and sets OPENAI_BASE_URL / ANTHROPIC_BASE_URL / LLM_BASE_URL. Preserves any pre-existing OPENHANDS_INSTRUCTIONS content. Tests: - tests/test_cli/test_wrap_cline.py: 4 tests covering prepare-only injection, idempotence, --no-context-tool, and existing content preservation. - tests/test_cli/test_wrap_continue.py: 8 tests covering the new `_inject_continue_rtk_systemmessage` helper (new-file, existing keys, idempotence, malformed JSON, non-object roots) and the click command surface (default path, custom --config). - tests/test_cli/test_wrap_goose.py: 5 tests covering env-var wiring, `.goosehints` injection, idempotence, missing-binary error, and --no-context-tool. - tests/test_cli/test_wrap_openhands.py: 6 tests covering env-var wiring, OPENHANDS_INSTRUCTIONS injection, preservation of existing instructions, idempotence, missing-binary error, and --no-context-tool. E2E: extended `e2e/wrap/run.py` with `--prepare-only` smoke tests for all four new wrappers (full launches require agent CLIs not present in the e2e image; unit tests cover the env-var wiring). |
||
|
chopratejas
|
2ae88a6874 |
fix(tests): widen wrap-e2e openclaw startup timeout from 5s to 30s
The wrap-e2e harness passed `--startup-timeout-ms 5000` to `headroom wrap openclaw`, leaving zero slack for the openclaw plugin's auto-start launcher to bring up the headroom proxy before the 5s health-check deadline. On a busy CI runner, cold Python import of `headroom.cli` plus pyo3 dlopen plus FastAPI app boot routinely lands in the 4–8s range, so this was always a coin-flip. Evidence: run 25897154424 failed on main with the exact code that passed pre-merge on PR #474's docker-wrap-e2e check (run 25897085244). Both logs show identical openclaw "Config warnings" output — that's normal noise, not the cause. The differentiating line is `[plugins] Headroom proxy started and reachable` (pass) vs `[plugins] Headroom proxy unavailable: health check failed` (fail). 30s matches what other wrap-e2e callers already use as a working margin for the headroom proxy boot path; the runtime default for `headroom wrap openclaw --startup-timeout-ms` is 20s. |
||
|
Tejas Chopra
|
8714c64a20 |
test(init-e2e): expect mcp add in seq_claude_local install sequence
|
||
|
JerrettDavis
|
4f654212d5 |
style(ci): apply ruff format to bug-3 fix files
Three files modified in the previous commit (
|
||
|
JerrettDavis
|
4071d57134 |
fix(ci): update tests to assert absence of requires_openai_auth (bug 3, #406)
- Restore build_provider_section() to headroom/providers/codex/install.py without requires_openai_auth (was removed entirely; pre-existing test test_provider_codex_install.py imports it and would fail to collect) - Flip test_codex_provider_section_preserves_openai_oauth to assert requires_openai_auth is ABSENT, not present (old behavior was wrong) - Fix test_provider_codex_runtime.py:337 same way — init config must NOT contain requires_openai_auth - Fix Ruff B023 lint error in test_providers.py:492 — capture loop variable config_path in lambda default arg (_p=config_path) - Fix e2e/init/run.py _verify_codex_local and _verify_codex_global to assert requires_openai_auth is absent, not present - Fix e2e/wrap/run.py verify_codex_wrap same way All unit tests pass locally (82 affected tests green). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JerrettDavis
|
06428d20fd |
fix: preserve Codex OAuth proxy delivery
Preserve Codex OAuth-safe provider config across init, wrap, and persistent install paths, and strengthen coverage so Codex requests are proven to reach Headroom and the mock upstream. The wrap e2e now sends a real chat-completions probe and checks Headroom /stats. Runtime tests cover temporary launch env, install env, init config, provider-scope config delivery, and the Python 3.11 ws bootstrap path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
chopratejas
|
6f2c0a8400 |
fix(ci): rustls-everywhere — eliminate openssl-sys from build tree
# Root cause of the wheel-build cascade We have shipped 5 release-pipeline hot-fixes in 12 hours, each addressing a different symptom of the same architectural problem: 1. PR #363 — npm artifact downloads + tried `yum openssl-devel` 2. PR #367 — vendored OpenSSL in `headroom-proxy` + dropped Intel mac 3. PR #369 — Debian-cross perl install (`perl` not `libipc-cmd-perl`) 4. PR #370 — moved `openssl/vendored` from headroom-proxy to headroom-py 5. (this PR) — ELIMINATE OpenSSL entirely Each fix exposed a different missing system package or feature flag in a different build surface (manylinux x86_64 vs aarch64-cross-Debian vs macOS Intel vs e2e/wrap Dockerfile vs e2e/init Dockerfile vs main Dockerfile vs devcontainer). We were playing whack-a-mole because every Cargo dep change to the OpenSSL surface required matching system-package updates in 6+ different Dockerfiles and workflows, and the PR-level CI didn't exercise all of them. # Why this PR is the structural fix `fastembed` exposes clean rustls feature flags: - `hf-hub-rustls-tls` (replaces default `hf-hub-native-tls`) - `ort-download-binaries-rustls-tls` (replaces default `…native-tls`) By disabling fastembed's default features and enabling the rustls variants explicitly, we remove `native-tls` (and therefore `openssl-sys`, `openssl`, `openssl-src`, perl modules, OpenSSL build-time deps, vendored OpenSSL ~30s build cost) from the entire workspace dep tree. Verified locally: $ cargo tree -p headroom-py -i openssl-sys error: package ID specification `openssl-sys` did not match any packages $ cargo tree -p headroom-py -i native-tls error: package ID specification `native-tls` did not match any packages $ cargo build --release -p headroom-py Finished `release` profile [optimized] target(s) in 25.57s (Down from 1m+ with vendored OpenSSL.) # Cleanups enabled by this change - crates/headroom-py/Cargo.toml — dropped the `openssl/vendored` workaround from PR #370. - crates/headroom-proxy/Cargo.toml — same dep removed. - e2e/wrap/Dockerfile — dropped `yum install openssl-devel pkgconfig perl-IPC-Cmd`. Comment retained explaining why. - e2e/init/Dockerfile — same. - Dockerfile (main) — dropped `pkg-config libssl-dev` from apt-get. - .devcontainer/Dockerfile — dropped `pkg-config libssl-dev`. - .github/workflows/release.yml — removed the entire before-script-linux block (perl install probe + multi-package-manager dispatch + fail-loud assertion). No longer needed. # Regression gate Three new structural tests in tests/test_release_workflows.py: - test_no_openssl_sys_in_wheel_build_tree — runs `cargo tree -p <crate> -i openssl-sys` for headroom-py / headroom-proxy / headroom-core. If openssl-sys reappears (a future native-tls enabler creeping in via a new dep), this fails AT PR TIME with an actionable message. - test_no_native_tls_in_wheel_build_tree — same shape, native-tls is the proximate cause. - test_fastembed_uses_rustls_features — checks the Cargo.toml so a future "let me bump fastembed and forget the features" doesn't silently re-introduce OpenSSL. Plus two cleanup gates: - test_dockerfiles_no_longer_install_openssl_devel - test_release_yml_does_not_install_openssl_or_perl_for_wheels All 13 release-workflow tests pass. `make ci-precheck` PASSED. # What this teaches us about rollouts (per user's ultrathink ask) The 5-fix cascade exposed three meta-problems: 1. PR checks don't block merges. PR #370 had docker-init-e2e, docker-wrap-e2e, docker-native-e2e all FAILED yet got merged. Branch protection should require these checks. Operator action needed (cannot fix in code). 2. Local validation is misleading. `cargo build -p headroom-py` from the workspace root used the workspace lockfile and looked green; CI did fresh resolution against headroom-py's manifest alone where the feature wasn't enabled. Lesson: verify structural invariants with `cargo tree -e features` before trusting that a build "works." 3. 6+ build surfaces with independent system-dep state. Every Cargo change required matching updates in 6 places. The structural answer (this PR) is to NOT depend on system OpenSSL at all. Where structural fixes are not possible, the answer is a single shared scripts/install-rust-build-deps.sh — but with this PR there's nothing left to install. |
||
|
chopratejas
|
9ae696eb75 |
fix(e2e): pin marketplace source via env var in init Dockerfile
`seq_claude_local` e2e assertion in e2e/init/run.py expects `claude plugin marketplace add /workspace` but the actual command was `claude plugin marketplace add chopratejas/headroom`. Root cause: `_marketplace_source()` in headroom/cli/init.py walks `Path(__file__).resolve().parents[2]` to find `.claude-plugin/ marketplace.json`. Before the single-wheel refactor, that path was `/workspace/headroom/cli/init.py` -> parents[2] = `/workspace`, where `.claude-plugin/marketplace.json` exists (COPY'd into the e2e image). After the refactor, `headroom` is installed from a wheel into site-packages, so `__file__` is now under `/opt/headroom-venv/.../site-packages/headroom/cli/init.py` -> parents[2] is the site-packages dir, which has no plugin manifest. The function then falls back to the remote `chopratejas/headroom`. Fix: set `HEADROOM_MARKETPLACE_SOURCE=/workspace` in the e2e/init runtime ENV. The function honors this override before doing the filesystem walk. The local `.claude-plugin/marketplace.json` is already COPY'd into `/workspace/.claude-plugin/`, so the override points at a valid source. |
||
|
chopratejas
|
73a4782917 |
fix(ci): switch e2e runtime to python:3.11-slim (trixie, glibc 2.41)
PR #360's previous attempt (multi-stage manylinux_2_28 build) still failed with the same `__isoc23_strtoll` undefined-symbol ImportError. Local repro showed the wheel built inside manylinux_2_28 has THREE glibc 2.38+ C23 symbol references (`__isoc23_strtol`, `__isoc23_strtoll`, `__isoc23_strtoull`) embedded by one of our transitive C/C++ deps during cc-rs compilation — most likely libstdc++'s `<cstdlib>` resolving `std::strtoll` to the C23 variant when the manylinux toolchain has newer-glibc-aware headers. We can't easily fix the source of that emission downstream. Path of least resistance: switch the e2e runtime stage from a glibc-2.36 base to one with glibc 2.38+. Verified on Mac (linux/arm64 native): the same wheel that fails on `node:22-bookworm` (glibc 2.36) imports cleanly on `python:3.11-slim` (now trixie, glibc 2.41). ## Changes - e2e/init/Dockerfile: stage 2 base `node:22-trixie` → `python:3.11-slim`. The init harness only needs Python; no Node 22. Drops apt-get install of python3/python3-pip/python3-venv (already in the base image) and the `ln -sf` python alias. - e2e/wrap/Dockerfile: stage 2 base `node:22-bookworm` → `python:3.11-slim`. The wrap harness needs both Python 3.11 (aider-chat==0.86.2 requires Python <3.12) AND Node 22 (codex, openclaw). Trixie's default python3 is 3.13 — too new for aider — so we build on top of `python:3.11-slim` (trixie + py 3.11) and install Node 22 from NodeSource. - Both: stage 1 `--interpreter` reverted from python3.13 to python3.11 to match the runtime. ## Verification (local, linux/arm64) docker buildx build -f e2e/wrap/Dockerfile.aarch64-test \ --platform linux/arm64 -t headroom-wrap-test . → stage 1 manylinux build green → stage 2 `from headroom._core import DiffCompressor` → OK → stage 2 aider-chat install in progress (separate venv) ## Production-side note (out of scope for this PR) `pip install headroom-ai` from PyPI on a glibc-2.36 host (e.g. Debian 12, Ubuntu 22.04) will hit the same ImportError once the wheel matrix publishes. python:3.X-slim is now trixie (glibc 2.41) for ALL of 3.10/3.11/3.12/3.13, so users on those base images are unaffected. Tracking the underlying cc-rs symbol-emission bug as a separate issue. |
||
|
chopratejas
|
b31a34b4ac |
fix(ci): multi-stage manylinux build for e2e dockerfiles + release workflow test
## Two distinct failures on PR #360 ### docker-init-e2e + docker-wrap-e2e + docker-native-e2e Building headroom-ai from source inside `node:22-bookworm` produced a `_core.so` that referenced `__isoc23_strtoll` (a glibc 2.38+ symbol). The same image's runtime libc.so.6 (whatever it actually ships) can't resolve it at import time: ImportError: /workspace/headroom/_core.cpython-311-x86_64-linux-gnu.so: undefined symbol: __isoc23_strtoll Most likely cause: cc-rs invoking the bookworm gcc against headers that have C23 wrappers exposed (libc6-dev backport, gcc 13 default mode, or something similar), generating object code that references a symbol the runtime libc.so doesn't actually have. Fix: multi-stage docker build. Stage 1 builds the wheel inside `quay.io/pypa/manylinux_2_28_x86_64` (AlmaLinux 8, glibc 2.28 baseline). Stage 2 (node:22-bookworm) just installs the prebuilt wheel — no rust toolchain needed at runtime, no build inside the runtime image. Same pattern release.yml already uses for cross-platform wheel matrix. Removed `COPY headroom/` and `COPY pyproject.toml` from the runtime stage to prevent the source-only `headroom/` from shadowing the installed wheel via cwd (Python would import the .py-only package and miss `_core.so`). ### test (3.10/3.11/3.12/3.13) The release-workflows test asserts the literal `needs:` list of the create-release job. The single-wheel maturin refactor added `build-wheels` and `collect-dist` jobs between `build` and the publish jobs; create-release now waits for those too. Updated the assertion + added explicit checks for the new `needs.<job>.result == 'success'` guards. |
||
|
chopratejas
|
2ae57725e7 |
fix(ci): pre-install rustfmt+clippy components in all Dockerfiles
rust-toolchain.toml at the repo root requests
`components = ["rustfmt", "clippy"]`. When `pip install -e .` invokes
maturin → cargo from inside `/workspace`, rustup auto-detects the
toolchain file and tries to add the missing components on top of the
`--profile minimal` install we did earlier. The install fails with:
info: downloading component clippy
info: rolling back changes
error: failed to install component: 'rustfmt-preview-x86_64-unknown-linux-gnu',
detected conflict: 'bin/cargo-fmt'
— rustup's auto-component install hits a `bin/cargo-fmt` conflict
inside the toolchain it just installed. The fix is to install the
required components up-front via `-c rustfmt -c clippy`, so the
toolchain matches what rust-toolchain.toml expects on first cargo run
and rustup never needs to mutate it.
Applied to: Dockerfile (main), e2e/init/Dockerfile, e2e/wrap/Dockerfile,
.devcontainer/Dockerfile. Also pinned the main Dockerfile's toolchain
from `stable` to `1.95.0` so all four images now match the lockfile
(prevents drift if rust-toolchain.toml is bumped later).
|
||
|
chopratejas
|
2a91cbb4b4 |
refactor: single-wheel maturin build backend (fixes #355)
Eliminates the dual-package architecture that was the root cause of #355. `pip install headroom-ai` now produces ONE wheel containing both the Python source (headroom/*.py) and the compiled Rust extension (headroom/_core.so). No more separate `headroom-core-py` package, no more chicken-and-egg with PyPI publication, no more wheelhouse / PIP_FIND_LINKS / composite-action plumbing in CI. This is the canonical pattern used by cryptography, polars, ruff, pydantic-core, and other Rust-as-core Python packages. Honors the "Rust as core engine" direction. ## What changed - pyproject.toml: `[build-system]` swapped from hatchling to maturin. `[tool.hatch.*]` deleted; `[tool.maturin]` added pointing at `crates/headroom-py/Cargo.toml` for the cdylib. `python-source = "."` picks up the root `headroom/` package directly (dashboard HTML templates and other non-Python files included automatically). - crates/headroom-py/pyproject.toml: deleted. The crate is no longer a separate published package; its Cargo.toml stays as the cdylib build target invoked via `[tool.maturin] manifest-path`. - crates/headroom-py/python/: deleted (placeholder layout for the old separate package). ## CI updates - ci.yml: `test` / `test-extras` / `test-agno` jobs simplified — Rust toolchain set up before `pip install -e .` (which now invokes maturin via build-system). Removed the "build wheel + symlink .so" dance. `build` job swapped from `python -m build` (hatch) to `maturin build` + `maturin sdist`. - release.yml: collapsed dual-package matrix into one. New `build-wheels` matrix produces cross-platform wheels for cp310/11/12/13 × {linux x86_64, linux aarch64, macos x86_64, macos aarch64}. New `collect-dist` aggregator merges artifacts. publish-pypi consumes the merged dist. - init-native-e2e.yml: dropped windows-latest from the matrix — upstream `esaxx-rs` (/MT) and `ort-sys` (/MD) link with conflicting MSVC C runtime libraries, so the Rust extension cannot build for win_amd64 today. Tracked as a follow-up; not a blocker for Linux+macOS. - headroom-e2e-setup: composite action now sets up Rust toolchain + Swatinem/rust-cache before `pip install -e .[proxy]`. - eval.yml, publish.yml, rust.yml: same pattern — rust toolchain before install. rust.yml's wheels job builds from root pyproject.toml (no more `-m crates/headroom-py/Cargo.toml`). - e2e/init/Dockerfile, e2e/wrap/Dockerfile: install rust + maturin in the build stage; copy `crates/` + workspace `Cargo.toml/lock` so the install can build the extension. Dropped `HEADROOM_REQUIRE_RUST_CORE=false` from wrap-e2e — the image now ships the full Rust core. - Dockerfile (main): simplified — no more Layer 2/3 dance with `headroom-core-py` install + symlink. Single `uv pip install` builds + installs everything. - .devcontainer/Dockerfile: rust toolchain + libssl-dev + maturin added so `uv sync` builds the extension inside the devcontainer. ## Lockfile + script - uv.lock: regenerated. No `headroom-core-py` entries remain. - scripts/build_rust_extension.sh: simplified from a symlink-into-tree workaround to a thin wrapper around `pip install -e .`. The maturin build-backend handles placement automatically. ## Local validation (all green on macOS aarch64) 1. Clean venv `pip install -e .` → `from headroom._core import …` works. 2. `maturin build --release` → 13.8 MB wheel, 336 files including `headroom/_core.cpython-311-darwin.so` (32 MB cdylib) and `headroom/dashboard/templates/dashboard.html`. 3. `pip install <wheel>` in fresh venv → import works. 4. Wheel contents verified via `unzip -l`. 5. `pytest tests/test_transforms/test_diff_compressor.py` — 29 passed. 6. `pytest tests/test_relevance.py` — 30 passed. 7. `cargo build --workspace` + `cargo test --workspace` — all green. 8. `make ci-precheck` — 176 Python tests + Rust + commitlint green. ## Migration notes Users on `pip install headroom-ai` get the Rust core automatically (linux + macos wheels). sdist installs require rust toolchain available locally — pip will build via maturin. Closes #355 Supersedes #357 (workarounds-based fix abandoned in favor of architectural fix) |
||
|
chopratejas
|
55dfc19e1d |
fix(ci): unblock A0 Docker e2e — install pkg-config + opt out wrap-e2e
Two CI failures introduced by Hotfix-A0's deployment-stage smoke test: 1. docker-native-e2e: the new maturin step in the builder stage failed with "Could not find openssl via pkg-config". The workspace transitively depends on `openssl-sys` (via reqwest's native-tls path in some dep chain). The previous Dockerfile only installed `build-essential`/`g++`/`curl`/`ca-certificates` — enough for the proxy binary build because cached target/ artefacts already had openssl-sys compiled, but the fresh maturin invocation hits a cold build and needs the dev headers. Add `pkg-config` + `libssl-dev`. 2. docker-wrap-e2e: this image is a `node:22-bookworm` base that installs headroom in editable mode for CLI-routing-only tests (aider, codex, openclaw via the wrap subcommand). It deliberately does NOT build the Rust extension. After A0, the proxy `lifespan` startup refuses to start when `headroom._core` can't import — so the wrap-e2e proxy port never opens, the harness's /health check times out, and the test fails. The wrap-e2e scope doesn't cover compression behaviour, so set `HEADROOM_REQUIRE_RUST_CORE=false` to start in degraded Python-only mode. Compression is exercised end-to-end by the smoke-test and docker-native-e2e jobs which build via the main Dockerfile. The remaining 3 PR check failures (validate * 3) were transient PyPI download failures (`nvidia-cuda-cupti-cu12==12.8.90`, `safetensors==0.7.0`) — unrelated to the realignment branch; they need a re-run, not a code change. |
||
|
JerrettDavis
|
48e2431510 |
test(init): extend Docker e2e with bare/shim/per-subcommand cases
Port e2e/init/run.py onto the shared harness and extend coverage so issue #245 (bare ``headroom init -g`` with no agents) is locked in: * ``seq_claude_local`` / ``seq_copilot_global`` / ``seq_codex_local`` — the original scenario, now expressed as a sequence of Cases sharing one scratch so the manifest-merge behavior (claude + codex targets) is still exercised end-to-end * ``bare_init_g_no_shims`` — regression guard for issue #245: asserts the new guided error mentions every probed target and the concrete ``headroom init -g <agent>`` example * ``bare_init_g_with_all_shims`` — complementary happy path with all four shims present; asserts all three configurable agents report ``Configured ... (user scope)`` on stdout * ``init_g_{claude,codex,copilot}_explicit`` — one case per subcommand, each with only its own shim on PATH, asserting exit 0 and the correct per-agent settings file is written * ``init_g_openclaw_missing`` — negative path for openclaw when its binary isn't installed (delegates to ``headroom wrap openclaw`` which can't be shimmed cheaply) * ``init_verbose_no_shims`` — smoke test for ``headroom init -v`` ensuring ``detect_init_targets``, ``global_scope=True``, and every agent name appear on stderr Dockerfile is updated to COPY e2e/__init__.py and e2e/_lib/ so the harness is importable inside the container. A new e2e/__init__.py marks the tree as a package. One small harness fix rides along: ``_resolve_headroom_bin`` captures the absolute path to headroom before ``with_clean_path`` narrows PATH. This is required for any case run inside a venv-scoped image - the real ``headroom`` lives outside the shim dir and would otherwise be hidden by the scrubbed PATH. Same bug would have bitten every future command suite, so the fix belongs in the harness rather than run.py. Verified locally inside the Docker image: all 10 cases pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JerrettDavis
|
3ca2ce08ae |
refactor(e2e): extract reusable harness into e2e/_lib
Centralize Docker / CI e2e test helpers so per-command suites can be declarative and future commands (install, wrap, ...) can reuse the same shim/PATH/assertion primitives without duplicating infrastructure. The harness provides: * Case dataclass describing one test as argv + shims + expected exit / stdout / stderr / files / custom callbacks * make_shim() factory producing cross-platform executable shims (.sh on POSIX, .cmd on Windows) with noop / fail / record-args behaviors * with_clean_path() context manager that isolates PATH to a minimal known-good value plus any extras supplied by the case * agent_settings_path() locator mirroring headroom.cli.init so tests can assert the right file was written without touching private init state * run_cases() for independent cases and run_case_sequence() for cases that must share scratch state (e.g. manifest-merge scenarios) Shell / PowerShell shim-creation scripts are also shipped for CI steps that need to drop a shim without spinning up Python first. No behavior change in this commit - pure infrastructure. The init suite and new subcommand suites consume the harness in follow-up commits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JerrettDavis
|
a278a7b0ba |
test: cover init install flows end to end
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
JerrettDavis
|
01ce6c710b |
test(e2e): assert HEADROOM_WORKSPACE_DIR / HEADROOM_CONFIG_DIR reach container at runtime
Fills the deferred TODO in docker-native-install.sh. After 'install
apply' and before 'install stop', we now:
1. 'docker inspect' the running headroom-${PROFILE} container and assert
both canonical env vars are present in Config.Env with the expected
/tmp/headroom-home/.headroom and .../config values.
2. 'docker exec env' inside the container and assert the same vars are
visible to processes running under the proxy entrypoint (proves not
just Config.Env but actual runtime visibility).
Unit tests in tests/test_install/{test_runtime,test_native_installers}
already lock install-time env forwarding; this completes the runtime
half of the guarantee.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
JerrettDavis
|
4a87753713 |
feat(docker): forward HEADROOM_WORKSPACE_DIR and HEADROOM_CONFIG_DIR into containers
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
JerrettDavis
|
865bef2216 |
fix: harden persistent install wrappers and review gaps
Align Docker-native wrapper help and runtime behavior with the Python install contract, including persistent deployment metadata, baked install-image defaults, and explicit unsupported wrap targets. Harden the Python persistent-install path with profile validation, safer provider-scope handling, Windows environment restoration, runtime parity improvements, and rollback-safe apply/update behavior. Update README, Docker install docs, CI, and focused regressions to cover the Windows BOM failure, wrapper parity, compose coverage, and Docker-native wrap behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
JerrettDavis
|
b325a06aae |
feat: harden persistent install wrappers
Tighten Docker-native bash and PowerShell wrapper validation for wrap and proxy flows, pin the bash wrapper to the install-time interpreter, clean up failed persistent container starts, and extend docs, CI, e2e, and native installer coverage for persistent Docker installs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Tejas Chopra
|
e4f72569c7
|
Merge pull request #109 from JerrettDavis/feat/openclaw-upstream-gateway
feat(openclaw): route configurable gateway providers through headroom |
||
|
chopratejas
|
c9ac5f6270 |
Fix e2e cleanup: ignore errors from OpenClaw leftover files
TemporaryDirectory cleanup fails with OSError when OpenClaw gateway leaves behind lock/session files. ignore_cleanup_errors=True lets the test pass while the OS cleans up /tmp on reboot. |
||
|
JerrettDavis
|
37f32a8922 |
test(openclaw): cover branch routing paths
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
JerrettDavis
|
ca728a4d35 |
fix(ci): harden wrap e2e validation
Make the Docker wrap e2e harness validate live proxy env wiring for Codex and Aider, start a real OpenClaw gateway in-container, and clear the repo-wide Ruff issues that were keeping the Python 3.12 CI job red. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
JerrettDavis
|
1967859ef6 |
feat(ci): add docker wrap e2e workflow
Add a Docker-based end-to-end harness that validates Headroom's Codex, Aider, Cursor, and OpenClaw wrap flows without calling real model providers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |