refactor: single-wheel maturin build backend (fixes #355)

Eliminates the dual-package architecture that was the root cause of #355.
`pip install headroom-ai` now produces ONE wheel containing both the Python
source (headroom/*.py) and the compiled Rust extension (headroom/_core.so).
No more separate `headroom-core-py` package, no more chicken-and-egg with
PyPI publication, no more wheelhouse / PIP_FIND_LINKS / composite-action
plumbing in CI.

This is the canonical pattern used by cryptography, polars, ruff,
pydantic-core, and other Rust-as-core Python packages. Honors the
"Rust as core engine" direction.

## What changed

- pyproject.toml: `[build-system]` swapped from hatchling to maturin.
  `[tool.hatch.*]` deleted; `[tool.maturin]` added pointing at
  `crates/headroom-py/Cargo.toml` for the cdylib. `python-source = "."`
  picks up the root `headroom/` package directly (dashboard HTML
  templates and other non-Python files included automatically).
- crates/headroom-py/pyproject.toml: deleted. The crate is no longer a
  separate published package; its Cargo.toml stays as the cdylib build
  target invoked via `[tool.maturin] manifest-path`.
- crates/headroom-py/python/: deleted (placeholder layout for the old
  separate package).

## CI updates

- ci.yml: `test` / `test-extras` / `test-agno` jobs simplified — Rust
  toolchain set up before `pip install -e .` (which now invokes maturin
  via build-system). Removed the "build wheel + symlink .so" dance.
  `build` job swapped from `python -m build` (hatch) to
  `maturin build` + `maturin sdist`.
- release.yml: collapsed dual-package matrix into one. New `build-wheels`
  matrix produces cross-platform wheels for cp310/11/12/13 ×
  {linux x86_64, linux aarch64, macos x86_64, macos aarch64}. New
  `collect-dist` aggregator merges artifacts. publish-pypi consumes the
  merged dist.
- init-native-e2e.yml: dropped windows-latest from the matrix —
  upstream `esaxx-rs` (/MT) and `ort-sys` (/MD) link with conflicting
  MSVC C runtime libraries, so the Rust extension cannot build for
  win_amd64 today. Tracked as a follow-up; not a blocker for Linux+macOS.
- headroom-e2e-setup: composite action now sets up Rust toolchain +
  Swatinem/rust-cache before `pip install -e .[proxy]`.
- eval.yml, publish.yml, rust.yml: same pattern — rust toolchain before
  install. rust.yml's wheels job builds from root pyproject.toml (no
  more `-m crates/headroom-py/Cargo.toml`).
- e2e/init/Dockerfile, e2e/wrap/Dockerfile: install rust + maturin in
  the build stage; copy `crates/` + workspace `Cargo.toml/lock` so the
  install can build the extension. Dropped `HEADROOM_REQUIRE_RUST_CORE=false`
  from wrap-e2e — the image now ships the full Rust core.
- Dockerfile (main): simplified — no more Layer 2/3 dance with
  `headroom-core-py` install + symlink. Single `uv pip install` builds
  + installs everything.
- .devcontainer/Dockerfile: rust toolchain + libssl-dev + maturin
  added so `uv sync` builds the extension inside the devcontainer.

## Lockfile + script

- uv.lock: regenerated. No `headroom-core-py` entries remain.
- scripts/build_rust_extension.sh: simplified from a symlink-into-tree
  workaround to a thin wrapper around `pip install -e .`. The maturin
  build-backend handles placement automatically.

## Local validation (all green on macOS aarch64)

1. Clean venv `pip install -e .` → `from headroom._core import …` works.
2. `maturin build --release` → 13.8 MB wheel, 336 files including
   `headroom/_core.cpython-311-darwin.so` (32 MB cdylib) and
   `headroom/dashboard/templates/dashboard.html`.
3. `pip install <wheel>` in fresh venv → import works.
4. Wheel contents verified via `unzip -l`.
5. `pytest tests/test_transforms/test_diff_compressor.py` — 29 passed.
6. `pytest tests/test_relevance.py` — 30 passed.
7. `cargo build --workspace` + `cargo test --workspace` — all green.
8. `make ci-precheck` — 176 Python tests + Rust + commitlint green.

## Migration notes

Users on `pip install headroom-ai` get the Rust core automatically
(linux + macos wheels). sdist installs require rust toolchain available
locally — pip will build via maturin.

Closes #355
Supersedes #357 (workarounds-based fix abandoned in favor of
architectural fix)

scripts/build_rust_extension.sh

@@ -1,26 +1,19 @@
 #!/usr/bin/env bash
-# Build the Rust → Python extension (headroom._core) and link it into the
-# in-tree `headroom/` package so `import headroom._core` resolves.
+# Build + install the Rust extension (headroom._core) into the active venv.
 #
-# Why a wrapper script: `maturin develop` builds the `.so` and installs it
-# into the venv's site-packages, but the in-tree `headroom/` source
-# directory (loaded via `pip install -e .`) shadows that on sys.path.
-# Python finds `headroom/__init__.py` at the project root before reaching
-# the maturin overlay, so `import headroom._core` fails. Symlinking the
-# built `.so` into `headroom/` fixes the lookup with zero copies.
+# With single-wheel architecture (post-#355), `pip install -e .` invokes
+# maturin (declared in pyproject.toml's `[build-system]`) which builds the
+# Rust extension and installs it into site-packages alongside the Python
+# source. Earlier versions of this script symlinked the .so into the
+# in-tree `headroom/` directory because the dual-package layout left the
+# .so in `crates/headroom-py/python/headroom/`. That dance is no longer
+# needed — maturin places the .so directly in the editable install's
+# overlay and Python's import system finds it.
 #
-# Hotfix-A0 (2026-05-02): the script now also runs an end-to-end import
-# verification with the `hello()` marker so a partial / stale build is
-# caught before the proxy is started. This mirrors the lifespan smoke
-# test in `headroom.proxy.server._check_rust_core` so dev-time and
-# deploy-time both catch the same class of failure.
-#
-# Idempotent. Safe to run repeatedly. Requires `maturin` in PATH (i.e.
-# inside the project venv).
+# Idempotent. Safe to run repeatedly.
 
 set -euo pipefail
 
-# Move to repo root so all relative paths below are stable.
 cd "$(dirname "$0")/.."
 
 log() {
@@ -32,59 +25,34 @@ fail() {
     exit 1
 }
 
-# Step 1: pre-flight. Maturin must be on PATH and a venv must be active —
-# `maturin develop` writes into site-packages, and we want that write to
-# land in the same env the proxy will run in.
-if ! command -v maturin >/dev/null 2>&1; then
-    fail "maturin not found on PATH. Activate the venv first: source .venv/bin/activate"
-fi
-
+# Pre-flight: a venv should be active. The install would otherwise write
+# into the system Python.
 if [[ -z "${VIRTUAL_ENV:-}" ]]; then
-    log "warning: VIRTUAL_ENV is unset; maturin will install into the system Python."
+    log "warning: VIRTUAL_ENV is unset; pip will install into the system Python."
     log "         If that is not what you want, abort and 'source .venv/bin/activate' first."
 fi
 
-# Step 2: build + install via `maturin develop` (in-place editable install
-# with C extensions). This produces a `.so` under
-# crates/headroom-py/python/headroom/.
-log "step 1/3: maturin develop"
-maturin develop -m crates/headroom-py/Cargo.toml \
-    || fail "maturin develop failed (see output above)"
-
-# Step 3: locate the built artifact. `maturin develop` writes
-# `_core.cpython-<ver>-<platform>.{so,dylib,pyd}` into the package dir.
-SO_FILE=$(find crates/headroom-py/python/headroom -maxdepth 1 \
-    \( -name "_core.cpython-*.so" -o -name "_core.cpython-*.dylib" -o -name "_core.pyd" \) \
-    2>/dev/null | head -1 || true)
-
-if [[ -z "${SO_FILE}" ]]; then
-    fail "maturin develop succeeded but produced no _core.* binary in crates/headroom-py/python/headroom/"
+if ! command -v cargo >/dev/null 2>&1; then
+    fail "cargo not found on PATH. Install Rust toolchain (rustup) first."
 fi
 
-# Step 4: symlink into the in-tree package dir so the in-tree
-# `headroom/__init__.py` resolves the `_core` submodule. Only the symlink
-# style is supported; copy semantics drift on every rebuild.
-LINK_NAME="headroom/$(basename "${SO_FILE}")"
-ln -sf "$(pwd)/${SO_FILE}" "${LINK_NAME}" \
-    || fail "failed to symlink ${SO_FILE} into ${LINK_NAME}"
-log "step 2/3: linked ${LINK_NAME} -> ${SO_FILE}"
+# Build + install in one shot. The `[build-system] build-backend = "maturin"`
+# in pyproject.toml means pip drives maturin under the hood. The resulting
+# wheel contains both the Python source and the compiled `headroom/_core.so`,
+# and pip installs them into the editable overlay together.
+log "pip install -e . (drives maturin via build-backend)"
+python -m pip install -e . || fail "pip install -e . failed (see output above)"
 
-# Step 5: end-to-end import verification. This is the same check the
-# proxy lifespan runs at startup. Failing here means the build produced
-# something that can't be loaded — fix the build, don't fix the proxy.
-log "step 3/3: verifying \`from headroom._core import hello\`"
+# End-to-end verification — same shape as Phase A0's startup smoke check.
+log "verifying \`from headroom._core import DiffCompressor, SmartCrusher\`"
 python -c '
 import sys
 try:
-    from headroom._core import hello, DiffCompressor
+    from headroom._core import DiffCompressor, SmartCrusher
 except Exception as exc:
     print(f"verify FAILED: {type(exc).__name__}: {exc}", file=sys.stderr)
     sys.exit(1)
-marker = hello()
-if marker != "headroom-core":
-    print(f"verify FAILED: hello() returned {marker!r}, expected \"headroom-core\"", file=sys.stderr)
-    sys.exit(1)
-print(f"verify OK: hello()={marker!r}, DiffCompressor={DiffCompressor!r}")
+print(f"verify OK: DiffCompressor={DiffCompressor!r}, SmartCrusher={SmartCrusher!r}")
 ' || fail "import verification failed (see above)"
 
 log "headroom._core build + install + verify: OK"