akadmin/headroom
1
0
Fork 0
mirror of https://github.com/headroomlabs-ai/headroom.git synced 2026-08-27 14:17:10 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
2606 commits 410 branches 169 tags 142 MiB
Commit graph

13 commits

Author SHA1 Message Date
Rod Boev
22def93177
fix(mcp): register managed installs with a resolvable headroom command (#1386) 
## Description

Managed Headroom installs can register the MCP server with a bare
`headroom mcp serve` command even when the active runtime lives in a
venv outside `PATH`. That leaves Claude and Codex with a registration
they cannot re-launch reliably, and Claude eventually fails with `Failed
to reconnect to headroom: ENOENT`.

This PR reuses the existing runtime command resolver when building the
shared Headroom MCP spec, so the generated registration follows the
active install instead of assuming `headroom` is globally discoverable.
It also updates the shared-builder and registrar tests so the proof rows
now flow through `build_headroom_spec()` and prove the same resolved
command contract on both the Claude CLI path and the Codex TOML path. A
follow-up CI fix keeps the Docker init E2E expectation aligned with that
same resolver-backed contract.

Closes #487

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/mcp_registry/install.py`: build the Headroom MCP server spec
from the canonical runtime command resolver instead of hardcoding
`headroom mcp serve`
- `tests/test_mcp_registry/test_install.py`: cover the shared builder's
direct-binary and module-fallback command shapes
- `tests/test_mcp_registry/test_claude_registrar.py`: prove the Claude
CLI registration forwards the resolved command vector end to end
- `tests/test_mcp_registry/test_codex_registrar.py`: prove the Codex
registrar writes the same resolved command vector into TOML
- `e2e/init/run.py`: derive the Docker init E2E's expected Claude MCP
registration argv from `resolve_headroom_command()` so the CI harness
follows the same runtime contract
- `CHANGELOG.md`: note the managed-install MCP registration fix

## Testing

- [x] Unit tests pass (`uv run pytest
tests/test_mcp_registry/test_install.py -v`, `uv run pytest
tests/test_mcp_registry/test_claude_registrar.py -v`, `uv run pytest
tests/test_mcp_registry/test_codex_registrar.py -v`)
- [x] Linting passes (`uv run ruff check .`)
- [ ] Type checking passes (`uv run mypy headroom`) or explain N/A
truthfully
- [x] New tests added for new functionality when applicable
- [ ] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_mcp_registry/test_install.py -v
============================= 12 passed in 0.13s ==============================

$ uv run pytest tests/test_mcp_registry/test_claude_registrar.py -v
============================= 24 passed in 0.18s ==============================

$ uv run pytest tests/test_mcp_registry/test_codex_registrar.py -v
============================= 25 passed in 0.20s ==============================

$ uv run python -c "from e2e.init.run import _expected_headroom_mcp_call; print(_expected_headroom_mcp_call('http://127.0.0.1:9011'))"
['mcp', 'add', 'headroom', '-s', 'user', '-e', 'HEADROOM_PROXY_URL=http://127.0.0.1:9011', '--', '.../headroom', 'mcp', 'serve']

$ uv run ruff check e2e/init/run.py
All checks passed!

$ uv run ruff format e2e/init/run.py --check
1 file already formatted

$ uv run ruff check .
All checks passed!

$ uv run ruff format . --check
987 files already formatted
```

`uv run mypy headroom` was not run locally; this repo's focused local
gate for the touched Python registry path is the targeted pytest set
plus Ruff.

## Real Behavior Proof

- Environment: managed-install-safe MCP registration path, Python 3.11+,
no provider required
- Exact command / steps: run the focused MCP registry pytest files,
inspect the captured Claude CLI argv and rendered Codex TOML block, and
verify the Docker init E2E expectation derives its Claude MCP argv from
the same runtime helper
- Observed result: the persisted MCP registration uses a resolvable
command tied to the active Headroom runtime instead of bare `headroom`,
while `HEADROOM_PROXY_URL` handling stays unchanged
- Not tested: full live Claude reconnect against a real managed venv,
unless that is run during implementation

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- Scoped to the MCP registration slice in `#487`. The RTK hook rewriting
thread from the same issue is intentionally out of scope here.
- `@erikpr1994` isolated the managed-install `ENOENT` failure mode in
the issue thread and narrowed it to the bare-command MCP registration
path.
- If existing owned registrations with the old bare-command contract
need an in-place upgrade path, that should be handled explicitly in the
final diff rather than left implicit.
 2026-06-26 23:39:00 -05:00
Logan Kang
dff6a19946
fix(codex): write canonical hooks feature flag and migrate deprecated codex_hooks (#743) 
## Description

`headroom init codex` writes the hooks feature flag into
`.codex/config.toml`
under the key `codex_hooks`. Codex renamed the canonical key to `hooks`
and kept
`codex_hooks` as a legacy alias (openai/codex#20522). Current Codex
builds warn
about `[features].codex_hooks` and tell users to use `[features].hooks`
instead,
so configs written by headroom should stop emitting the deprecated key.

This PR switches headroom to write the canonical `hooks` key and
**migrates
existing configs in place**. The migration is the tricky part: a config
can
already contain `codex_hooks`, `hooks`, or both, in any order, inside or
outside
headroom's marker block — and a naive replace can emit a *duplicate*
`hooks`
key, which is invalid TOML that Codex rejects outright.

The fix strips every `codex_hooks` line up front (any value, anywhere) —
mirroring the existing top-level key cleanup in `_ensure_codex_provider`
(#260)
— then guarantees `hooks` is present without ever duplicating it, and
respects a
user-managed `hooks` value that lives outside our marker block.

Fixes: N/A (no tracking issue — surfaced while aligning with Codex >=
0.129;
related upstream context: openai/codex#20522 and the warning behavior
discussed
in openai/codex#22148)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Write the canonical `hooks` key instead of the deprecated
`codex_hooks` in
  `_ensure_codex_feature_flag` (`headroom/cli/init.py`).
- Strip any `codex_hooks` line (any value, inside or outside the marker
block)
before ensuring the flag, so re-running `init` migrates a legacy config
instead
of leaving a stale key or producing a duplicate `hooks` key (invalid
TOML).
- Respect a user-managed `hooks` value found outside headroom's marker
block
  (e.g. `hooks = false`); only the deprecated alias is removed.
- Make the insert/create paths match `_replace_marker_block`'s
normalisation so
  re-running `init` is byte-idempotent.
- Extract a `_codex_feature_block()` helper to remove the 4x duplicated
marker
  block assembly.
- Add regression tests for the previously-broken edge cases.

## Testing

- [x] Unit tests pass (`pytest`) — affected module fully green (see
output)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom/cli/init.py`)
- [x] New tests added for new functionality
- [x] Manual testing performed (reproduced each edge case against the
patched
      function via `tomllib.loads`)

## Test Output

```
$ pytest -v tests/test_cli/test_init_cli.py -k "feature_flag or hooks_feature"
tests/test_cli/test_init_cli.py::test_init_codex_merges_feature_flag_into_existing_table PASSED
tests/test_cli/test_init_cli.py::test_init_codex_creates_hooks_feature_flag_on_first_init PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_replaces_existing_marker PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_legacy_codex_hooks_key PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_when_both_keys_present PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_migrates_when_keys_reversed PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_drops_legacy_key_outside_marker PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_is_idempotent PASSED
tests/test_cli/test_init_cli.py::test_ensure_codex_feature_flag_creates_features_section_when_missing PASSED
======================= 9 passed, 45 deselected in 0.24s =======================

$ pytest -q tests/test_cli/test_init_cli.py
54 passed

$ ruff check .
All checks passed!

$ mypy headroom/cli/init.py
Success: no issues found in 1 source file
```

Note: the broader `tests/test_cli/` run has one unrelated failure
(`test_wrap_copilot_auto_detects_running_proxy_backend`) caused by a
real proxy
already bound to port 8787 in the local environment — it fails
identically on a
clean checkout without this change.

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (none
required)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable (managed by
release-please;
      generated from the conventional commit, not edited by hand)

## Screenshots (if applicable)

N/A

## Additional Notes

- **Why the duplicate-key path matters:** TOML forbids duplicate keys,
so a
`[features]` table containing both `codex_hooks` and `hooks` (which the
old
in-place migration could produce) makes Codex reject `config.toml`
entirely.
  The new "strip then ensure" approach can never emit two `hooks` lines.
- **Version provenance:** the `codex_hooks` -> `hooks` rename landed in
openai/codex#20522, first shipped in Codex `rust-v0.129.0`.
`codex_hooks`
remains a working legacy alias, but current Codex builds can warn users
to
  move to `[features].hooks`.
- **Idempotency:** running `headroom init codex` repeatedly now produces
a
  byte-stable `config.toml`, so there is no churn on re-init.
 2026-06-12 12:49:34 -05:00
Tejas Chopra
8714c64a20 test(init-e2e): expect mcp add in seq_claude_local install sequence 
d9d8972 wired auto-MCP registration into ``init`` so ``[Retrieve
more: hash=…]`` markers stay live for users who never ran
``headroom mcp install`` separately, but the ``seq_claude_local``
e2e assertion was still pinned to the pre-MCP two-command sequence
and failed in docker-native-e2e on main.

The ``-e HEADROOM_PROXY_URL=…`` arg is only emitted when the proxy
port differs from the 8787 default; this case sets ``--port 9011``,
so the env arg is included in the expected argv.
 2026-05-08 17:38:34 -07:00
JerrettDavis
4071d57134 fix(ci): update tests to assert absence of requires_openai_auth (bug 3, #406) 
- Restore build_provider_section() to headroom/providers/codex/install.py
  without requires_openai_auth (was removed entirely; pre-existing test
  test_provider_codex_install.py imports it and would fail to collect)
- Flip test_codex_provider_section_preserves_openai_oauth to assert
  requires_openai_auth is ABSENT, not present (old behavior was wrong)
- Fix test_provider_codex_runtime.py:337 same way — init config must
  NOT contain requires_openai_auth
- Fix Ruff B023 lint error in test_providers.py:492 — capture loop
  variable config_path in lambda default arg (_p=config_path)
- Fix e2e/init/run.py _verify_codex_local and _verify_codex_global to
  assert requires_openai_auth is absent, not present
- Fix e2e/wrap/run.py verify_codex_wrap same way

All unit tests pass locally (82 affected tests green).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 12:18:47 -05:00
JerrettDavis
06428d20fd fix: preserve Codex OAuth proxy delivery 
Preserve Codex OAuth-safe provider config across init, wrap, and

persistent install paths, and strengthen coverage so Codex requests

are proven to reach Headroom and the mock upstream.

The wrap e2e now sends a real chat-completions probe and checks

Headroom /stats. Runtime tests cover temporary launch env, install

env, init config, provider-scope config delivery, and the Python

3.11 ws bootstrap path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-05 21:03:42 -05:00
chopratejas
6f2c0a8400 fix(ci): rustls-everywhere — eliminate openssl-sys from build tree 
# Root cause of the wheel-build cascade

We have shipped 5 release-pipeline hot-fixes in 12 hours, each
addressing a different symptom of the same architectural problem:

1. PR #363 — npm artifact downloads + tried `yum openssl-devel`
2. PR #367 — vendored OpenSSL in `headroom-proxy` + dropped Intel mac
3. PR #369 — Debian-cross perl install (`perl` not `libipc-cmd-perl`)
4. PR #370 — moved `openssl/vendored` from headroom-proxy to headroom-py
5. (this PR) — ELIMINATE OpenSSL entirely

Each fix exposed a different missing system package or feature flag in
a different build surface (manylinux x86_64 vs aarch64-cross-Debian vs
macOS Intel vs e2e/wrap Dockerfile vs e2e/init Dockerfile vs main
Dockerfile vs devcontainer). We were playing whack-a-mole because every
Cargo dep change to the OpenSSL surface required matching system-package
updates in 6+ different Dockerfiles and workflows, and the PR-level CI
didn't exercise all of them.

# Why this PR is the structural fix

`fastembed` exposes clean rustls feature flags:
- `hf-hub-rustls-tls`               (replaces default `hf-hub-native-tls`)
- `ort-download-binaries-rustls-tls` (replaces default `…native-tls`)

By disabling fastembed's default features and enabling the rustls
variants explicitly, we remove `native-tls` (and therefore `openssl-sys`,
`openssl`, `openssl-src`, perl modules, OpenSSL build-time deps,
vendored OpenSSL ~30s build cost) from the entire workspace dep tree.

Verified locally:

    $ cargo tree -p headroom-py -i openssl-sys
    error: package ID specification `openssl-sys` did not match any packages

    $ cargo tree -p headroom-py -i native-tls
    error: package ID specification `native-tls` did not match any packages

    $ cargo build --release -p headroom-py
    Finished `release` profile [optimized] target(s) in 25.57s

(Down from 1m+ with vendored OpenSSL.)

# Cleanups enabled by this change

- crates/headroom-py/Cargo.toml — dropped the `openssl/vendored`
  workaround from PR #370.
- crates/headroom-proxy/Cargo.toml — same dep removed.
- e2e/wrap/Dockerfile — dropped `yum install openssl-devel pkgconfig
  perl-IPC-Cmd`. Comment retained explaining why.
- e2e/init/Dockerfile — same.
- Dockerfile (main) — dropped `pkg-config libssl-dev` from apt-get.
- .devcontainer/Dockerfile — dropped `pkg-config libssl-dev`.
- .github/workflows/release.yml — removed the entire before-script-linux
  block (perl install probe + multi-package-manager dispatch + fail-loud
  assertion). No longer needed.

# Regression gate

Three new structural tests in tests/test_release_workflows.py:

- test_no_openssl_sys_in_wheel_build_tree — runs `cargo tree -p <crate>
  -i openssl-sys` for headroom-py / headroom-proxy / headroom-core. If
  openssl-sys reappears (a future native-tls enabler creeping in via a
  new dep), this fails AT PR TIME with an actionable message.
- test_no_native_tls_in_wheel_build_tree — same shape, native-tls is
  the proximate cause.
- test_fastembed_uses_rustls_features — checks the Cargo.toml so a
  future "let me bump fastembed and forget the features" doesn't
  silently re-introduce OpenSSL.

Plus two cleanup gates:
- test_dockerfiles_no_longer_install_openssl_devel
- test_release_yml_does_not_install_openssl_or_perl_for_wheels

All 13 release-workflow tests pass. `make ci-precheck` PASSED.

# What this teaches us about rollouts (per user's ultrathink ask)

The 5-fix cascade exposed three meta-problems:

1. PR checks don't block merges. PR #370 had docker-init-e2e,
   docker-wrap-e2e, docker-native-e2e all FAILED yet got merged.
   Branch protection should require these checks. Operator action
   needed (cannot fix in code).

2. Local validation is misleading. `cargo build -p headroom-py` from
   the workspace root used the workspace lockfile and looked green;
   CI did fresh resolution against headroom-py's manifest alone where
   the feature wasn't enabled. Lesson: verify structural invariants
   with `cargo tree -e features` before trusting that a build "works."

3. 6+ build surfaces with independent system-dep state. Every Cargo
   change required matching updates in 6 places. The structural answer
   (this PR) is to NOT depend on system OpenSSL at all. Where structural
   fixes are not possible, the answer is a single shared
   scripts/install-rust-build-deps.sh — but with this PR there's
   nothing left to install.
 2026-05-03 23:26:04 -07:00
chopratejas
9ae696eb75 fix(e2e): pin marketplace source via env var in init Dockerfile 
`seq_claude_local` e2e assertion in e2e/init/run.py expects
`claude plugin marketplace add /workspace` but the actual command
was `claude plugin marketplace add chopratejas/headroom`.

Root cause: `_marketplace_source()` in headroom/cli/init.py walks
`Path(__file__).resolve().parents[2]` to find `.claude-plugin/
marketplace.json`. Before the single-wheel refactor, that path was
`/workspace/headroom/cli/init.py` -> parents[2] = `/workspace`,
where `.claude-plugin/marketplace.json` exists (COPY'd into the
e2e image). After the refactor, `headroom` is installed from a
wheel into site-packages, so `__file__` is now under
`/opt/headroom-venv/.../site-packages/headroom/cli/init.py` ->
parents[2] is the site-packages dir, which has no plugin manifest.
The function then falls back to the remote `chopratejas/headroom`.

Fix: set `HEADROOM_MARKETPLACE_SOURCE=/workspace` in the e2e/init
runtime ENV. The function honors this override before doing the
filesystem walk. The local `.claude-plugin/marketplace.json` is
already COPY'd into `/workspace/.claude-plugin/`, so the override
points at a valid source.
 2026-05-03 14:57:30 -07:00
chopratejas
73a4782917 fix(ci): switch e2e runtime to python:3.11-slim (trixie, glibc 2.41) 
PR #360's previous attempt (multi-stage manylinux_2_28 build) still
failed with the same `__isoc23_strtoll` undefined-symbol ImportError.
Local repro showed the wheel built inside manylinux_2_28 has THREE
glibc 2.38+ C23 symbol references (`__isoc23_strtol`, `__isoc23_strtoll`,
`__isoc23_strtoull`) embedded by one of our transitive C/C++ deps
during cc-rs compilation — most likely libstdc++'s `<cstdlib>` resolving
`std::strtoll` to the C23 variant when the manylinux toolchain has
newer-glibc-aware headers. We can't easily fix the source of that
emission downstream.

Path of least resistance: switch the e2e runtime stage from a
glibc-2.36 base to one with glibc 2.38+. Verified on Mac (linux/arm64
native): the same wheel that fails on `node:22-bookworm` (glibc 2.36)
imports cleanly on `python:3.11-slim` (now trixie, glibc 2.41).

## Changes

- e2e/init/Dockerfile: stage 2 base `node:22-trixie` →
  `python:3.11-slim`. The init harness only needs Python; no Node 22.
  Drops apt-get install of python3/python3-pip/python3-venv (already in
  the base image) and the `ln -sf` python alias.
- e2e/wrap/Dockerfile: stage 2 base `node:22-bookworm` →
  `python:3.11-slim`. The wrap harness needs both Python 3.11
  (aider-chat==0.86.2 requires Python <3.12) AND Node 22 (codex,
  openclaw). Trixie's default python3 is 3.13 — too new for aider —
  so we build on top of `python:3.11-slim` (trixie + py 3.11) and
  install Node 22 from NodeSource.
- Both: stage 1 `--interpreter` reverted from python3.13 to python3.11
  to match the runtime.

## Verification (local, linux/arm64)

  docker buildx build -f e2e/wrap/Dockerfile.aarch64-test \
    --platform linux/arm64 -t headroom-wrap-test .

→ stage 1 manylinux build green
→ stage 2 `from headroom._core import DiffCompressor` → OK
→ stage 2 aider-chat install in progress (separate venv)

## Production-side note (out of scope for this PR)

`pip install headroom-ai` from PyPI on a glibc-2.36 host (e.g. Debian
12, Ubuntu 22.04) will hit the same ImportError once the wheel matrix
publishes. python:3.X-slim is now trixie (glibc 2.41) for ALL of
3.10/3.11/3.12/3.13, so users on those base images are unaffected.
Tracking the underlying cc-rs symbol-emission bug as a separate issue.
 2026-05-03 14:30:46 -07:00
chopratejas
b31a34b4ac fix(ci): multi-stage manylinux build for e2e dockerfiles + release workflow test 
## Two distinct failures on PR #360

### docker-init-e2e + docker-wrap-e2e + docker-native-e2e

Building headroom-ai from source inside `node:22-bookworm` produced a
`_core.so` that referenced `__isoc23_strtoll` (a glibc 2.38+ symbol).
The same image's runtime libc.so.6 (whatever it actually ships) can't
resolve it at import time:

    ImportError: /workspace/headroom/_core.cpython-311-x86_64-linux-gnu.so:
      undefined symbol: __isoc23_strtoll

Most likely cause: cc-rs invoking the bookworm gcc against headers that
have C23 wrappers exposed (libc6-dev backport, gcc 13 default mode, or
something similar), generating object code that references a symbol the
runtime libc.so doesn't actually have.

Fix: multi-stage docker build. Stage 1 builds the wheel inside
`quay.io/pypa/manylinux_2_28_x86_64` (AlmaLinux 8, glibc 2.28 baseline).
Stage 2 (node:22-bookworm) just installs the prebuilt wheel — no rust
toolchain needed at runtime, no build inside the runtime image. Same
pattern release.yml already uses for cross-platform wheel matrix.

Removed `COPY headroom/` and `COPY pyproject.toml` from the runtime
stage to prevent the source-only `headroom/` from shadowing the
installed wheel via cwd (Python would import the .py-only package and
miss `_core.so`).

### test (3.10/3.11/3.12/3.13)

The release-workflows test asserts the literal `needs:` list of the
create-release job. The single-wheel maturin refactor added
`build-wheels` and `collect-dist` jobs between `build` and the publish
jobs; create-release now waits for those too. Updated the assertion +
added explicit checks for the new `needs.<job>.result == 'success'`
guards.
 2026-05-03 14:08:25 -07:00
chopratejas
2ae57725e7 fix(ci): pre-install rustfmt+clippy components in all Dockerfiles 
rust-toolchain.toml at the repo root requests
`components = ["rustfmt", "clippy"]`. When `pip install -e .` invokes
maturin → cargo from inside `/workspace`, rustup auto-detects the
toolchain file and tries to add the missing components on top of the
`--profile minimal` install we did earlier. The install fails with:

    info: downloading component clippy
    info: rolling back changes
    error: failed to install component: 'rustfmt-preview-x86_64-unknown-linux-gnu',
           detected conflict: 'bin/cargo-fmt'

— rustup's auto-component install hits a `bin/cargo-fmt` conflict
inside the toolchain it just installed. The fix is to install the
required components up-front via `-c rustfmt -c clippy`, so the
toolchain matches what rust-toolchain.toml expects on first cargo run
and rustup never needs to mutate it.

Applied to: Dockerfile (main), e2e/init/Dockerfile, e2e/wrap/Dockerfile,
.devcontainer/Dockerfile. Also pinned the main Dockerfile's toolchain
from `stable` to `1.95.0` so all four images now match the lockfile
(prevents drift if rust-toolchain.toml is bumped later).
 2026-05-03 13:35:01 -07:00
chopratejas
2a91cbb4b4 refactor: single-wheel maturin build backend (fixes #355) 
Eliminates the dual-package architecture that was the root cause of #355.
`pip install headroom-ai` now produces ONE wheel containing both the Python
source (headroom/*.py) and the compiled Rust extension (headroom/_core.so).
No more separate `headroom-core-py` package, no more chicken-and-egg with
PyPI publication, no more wheelhouse / PIP_FIND_LINKS / composite-action
plumbing in CI.

This is the canonical pattern used by cryptography, polars, ruff,
pydantic-core, and other Rust-as-core Python packages. Honors the
"Rust as core engine" direction.

## What changed

- pyproject.toml: `[build-system]` swapped from hatchling to maturin.
  `[tool.hatch.*]` deleted; `[tool.maturin]` added pointing at
  `crates/headroom-py/Cargo.toml` for the cdylib. `python-source = "."`
  picks up the root `headroom/` package directly (dashboard HTML
  templates and other non-Python files included automatically).
- crates/headroom-py/pyproject.toml: deleted. The crate is no longer a
  separate published package; its Cargo.toml stays as the cdylib build
  target invoked via `[tool.maturin] manifest-path`.
- crates/headroom-py/python/: deleted (placeholder layout for the old
  separate package).

## CI updates

- ci.yml: `test` / `test-extras` / `test-agno` jobs simplified — Rust
  toolchain set up before `pip install -e .` (which now invokes maturin
  via build-system). Removed the "build wheel + symlink .so" dance.
  `build` job swapped from `python -m build` (hatch) to
  `maturin build` + `maturin sdist`.
- release.yml: collapsed dual-package matrix into one. New `build-wheels`
  matrix produces cross-platform wheels for cp310/11/12/13 ×
  {linux x86_64, linux aarch64, macos x86_64, macos aarch64}. New
  `collect-dist` aggregator merges artifacts. publish-pypi consumes the
  merged dist.
- init-native-e2e.yml: dropped windows-latest from the matrix —
  upstream `esaxx-rs` (/MT) and `ort-sys` (/MD) link with conflicting
  MSVC C runtime libraries, so the Rust extension cannot build for
  win_amd64 today. Tracked as a follow-up; not a blocker for Linux+macOS.
- headroom-e2e-setup: composite action now sets up Rust toolchain +
  Swatinem/rust-cache before `pip install -e .[proxy]`.
- eval.yml, publish.yml, rust.yml: same pattern — rust toolchain before
  install. rust.yml's wheels job builds from root pyproject.toml (no
  more `-m crates/headroom-py/Cargo.toml`).
- e2e/init/Dockerfile, e2e/wrap/Dockerfile: install rust + maturin in
  the build stage; copy `crates/` + workspace `Cargo.toml/lock` so the
  install can build the extension. Dropped `HEADROOM_REQUIRE_RUST_CORE=false`
  from wrap-e2e — the image now ships the full Rust core.
- Dockerfile (main): simplified — no more Layer 2/3 dance with
  `headroom-core-py` install + symlink. Single `uv pip install` builds
  + installs everything.
- .devcontainer/Dockerfile: rust toolchain + libssl-dev + maturin
  added so `uv sync` builds the extension inside the devcontainer.

## Lockfile + script

- uv.lock: regenerated. No `headroom-core-py` entries remain.
- scripts/build_rust_extension.sh: simplified from a symlink-into-tree
  workaround to a thin wrapper around `pip install -e .`. The maturin
  build-backend handles placement automatically.

## Local validation (all green on macOS aarch64)

1. Clean venv `pip install -e .` → `from headroom._core import …` works.
2. `maturin build --release` → 13.8 MB wheel, 336 files including
   `headroom/_core.cpython-311-darwin.so` (32 MB cdylib) and
   `headroom/dashboard/templates/dashboard.html`.
3. `pip install <wheel>` in fresh venv → import works.
4. Wheel contents verified via `unzip -l`.
5. `pytest tests/test_transforms/test_diff_compressor.py` — 29 passed.
6. `pytest tests/test_relevance.py` — 30 passed.
7. `cargo build --workspace` + `cargo test --workspace` — all green.
8. `make ci-precheck` — 176 Python tests + Rust + commitlint green.

## Migration notes

Users on `pip install headroom-ai` get the Rust core automatically
(linux + macos wheels). sdist installs require rust toolchain available
locally — pip will build via maturin.

Closes #355
Supersedes #357 (workarounds-based fix abandoned in favor of
architectural fix)
 2026-05-03 13:16:41 -07:00
JerrettDavis
48e2431510 test(init): extend Docker e2e with bare/shim/per-subcommand cases 
Port e2e/init/run.py onto the shared harness and extend coverage so
issue #245 (bare ``headroom init -g`` with no agents) is locked in:

* ``seq_claude_local`` / ``seq_copilot_global`` / ``seq_codex_local`` —
  the original scenario, now expressed as a sequence of Cases sharing
  one scratch so the manifest-merge behavior (claude + codex targets)
  is still exercised end-to-end
* ``bare_init_g_no_shims`` — regression guard for issue #245: asserts
  the new guided error mentions every probed target and the concrete
  ``headroom init -g <agent>`` example
* ``bare_init_g_with_all_shims`` — complementary happy path with all
  four shims present; asserts all three configurable agents report
  ``Configured ... (user scope)`` on stdout
* ``init_g_{claude,codex,copilot}_explicit`` — one case per
  subcommand, each with only its own shim on PATH, asserting exit 0
  and the correct per-agent settings file is written
* ``init_g_openclaw_missing`` — negative path for openclaw when its
  binary isn't installed (delegates to ``headroom wrap openclaw`` which
  can't be shimmed cheaply)
* ``init_verbose_no_shims`` — smoke test for ``headroom init -v``
  ensuring ``detect_init_targets``, ``global_scope=True``, and every
  agent name appear on stderr

Dockerfile is updated to COPY e2e/__init__.py and e2e/_lib/ so the
harness is importable inside the container. A new e2e/__init__.py
marks the tree as a package.

One small harness fix rides along: ``_resolve_headroom_bin`` captures
the absolute path to headroom before ``with_clean_path`` narrows PATH.
This is required for any case run inside a venv-scoped image - the
real ``headroom`` lives outside the shim dir and would otherwise be
hidden by the scrubbed PATH. Same bug would have bitten every future
command suite, so the fix belongs in the harness rather than run.py.

Verified locally inside the Docker image: all 10 cases pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 16:12:27 -05:00
JerrettDavis
a278a7b0ba test: cover init install flows end to end 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-21 20:15:11 -05:00