refactor: single-wheel maturin build backend (fixes #355)

Eliminates the dual-package architecture that was the root cause of #355.
`pip install headroom-ai` now produces ONE wheel containing both the Python
source (headroom/*.py) and the compiled Rust extension (headroom/_core.so).
No more separate `headroom-core-py` package, no more chicken-and-egg with
PyPI publication, no more wheelhouse / PIP_FIND_LINKS / composite-action
plumbing in CI.

This is the canonical pattern used by cryptography, polars, ruff,
pydantic-core, and other Rust-as-core Python packages. Honors the
"Rust as core engine" direction.

## What changed

- pyproject.toml: `[build-system]` swapped from hatchling to maturin.
  `[tool.hatch.*]` deleted; `[tool.maturin]` added pointing at
  `crates/headroom-py/Cargo.toml` for the cdylib. `python-source = "."`
  picks up the root `headroom/` package directly (dashboard HTML
  templates and other non-Python files included automatically).
- crates/headroom-py/pyproject.toml: deleted. The crate is no longer a
  separate published package; its Cargo.toml stays as the cdylib build
  target invoked via `[tool.maturin] manifest-path`.
- crates/headroom-py/python/: deleted (placeholder layout for the old
  separate package).

## CI updates

- ci.yml: `test` / `test-extras` / `test-agno` jobs simplified — Rust
  toolchain set up before `pip install -e .` (which now invokes maturin
  via build-system). Removed the "build wheel + symlink .so" dance.
  `build` job swapped from `python -m build` (hatch) to
  `maturin build` + `maturin sdist`.
- release.yml: collapsed dual-package matrix into one. New `build-wheels`
  matrix produces cross-platform wheels for cp310/11/12/13 ×
  {linux x86_64, linux aarch64, macos x86_64, macos aarch64}. New
  `collect-dist` aggregator merges artifacts. publish-pypi consumes the
  merged dist.
- init-native-e2e.yml: dropped windows-latest from the matrix —
  upstream `esaxx-rs` (/MT) and `ort-sys` (/MD) link with conflicting
  MSVC C runtime libraries, so the Rust extension cannot build for
  win_amd64 today. Tracked as a follow-up; not a blocker for Linux+macOS.
- headroom-e2e-setup: composite action now sets up Rust toolchain +
  Swatinem/rust-cache before `pip install -e .[proxy]`.
- eval.yml, publish.yml, rust.yml: same pattern — rust toolchain before
  install. rust.yml's wheels job builds from root pyproject.toml (no
  more `-m crates/headroom-py/Cargo.toml`).
- e2e/init/Dockerfile, e2e/wrap/Dockerfile: install rust + maturin in
  the build stage; copy `crates/` + workspace `Cargo.toml/lock` so the
  install can build the extension. Dropped `HEADROOM_REQUIRE_RUST_CORE=false`
  from wrap-e2e — the image now ships the full Rust core.
- Dockerfile (main): simplified — no more Layer 2/3 dance with
  `headroom-core-py` install + symlink. Single `uv pip install` builds
  + installs everything.
- .devcontainer/Dockerfile: rust toolchain + libssl-dev + maturin
  added so `uv sync` builds the extension inside the devcontainer.

## Lockfile + script

- uv.lock: regenerated. No `headroom-core-py` entries remain.
- scripts/build_rust_extension.sh: simplified from a symlink-into-tree
  workaround to a thin wrapper around `pip install -e .`. The maturin
  build-backend handles placement automatically.

## Local validation (all green on macOS aarch64)

1. Clean venv `pip install -e .` → `from headroom._core import …` works.
2. `maturin build --release` → 13.8 MB wheel, 336 files including
   `headroom/_core.cpython-311-darwin.so` (32 MB cdylib) and
   `headroom/dashboard/templates/dashboard.html`.
3. `pip install <wheel>` in fresh venv → import works.
4. Wheel contents verified via `unzip -l`.
5. `pytest tests/test_transforms/test_diff_compressor.py` — 29 passed.
6. `pytest tests/test_relevance.py` — 30 passed.
7. `cargo build --workspace` + `cargo test --workspace` — all green.
8. `make ci-precheck` — 176 Python tests + Rust + commitlint green.

## Migration notes

Users on `pip install headroom-ai` get the Rust core automatically
(linux + macos wheels). sdist installs require rust toolchain available
locally — pip will build via maturin.

Closes #355
Supersedes #357 (workarounds-based fix abandoned in favor of
architectural fix)
This commit is contained in:
chopratejas 2026-05-03 13:16:41 -07:00
parent 4bf559d5b5
commit 2a91cbb4b4
20 changed files with 4908 additions and 4872 deletions

View file

@ -5,14 +5,14 @@
},
"metadata": {
"description": "Headroom marketplace for Claude Code and GitHub Copilot CLI plugins.",
"version": "0.20.13"
"version": "0.20.14"
},
"plugins": [
{
"name": "headroom",
"source": "./plugins/headroom-agent-hooks",
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
"version": "0.20.13",
"version": "0.20.14",
"author": {
"name": "Headroom Contributors",
"url": "https://github.com/chopratejas/headroom"

View file

@ -1,4 +1,21 @@
ARG VARIANT=3.12-bookworm
FROM mcr.microsoft.com/devcontainers/python:1-${VARIANT}
RUN python -m pip install --no-cache-dir uv==0.6.17
# Single-wheel architecture (post-#355): `uv sync` builds `headroom-ai`
# from the local pyproject.toml using maturin (declared in build-system).
# Maturin needs rust + cargo + pkg-config + libssl-dev (transitive
# openssl-sys via fastembed/hf-hub).
RUN apt-get update && \
apt-get install -y --no-install-recommends \
pkg-config \
libssl-dev && \
rm -rf /var/lib/apt/lists/*
ENV CARGO_HOME=/usr/local/cargo \
RUSTUP_HOME=/usr/local/rustup \
PATH=/usr/local/cargo/bin:${PATH}
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.95.0 --profile minimal && \
rustc --version && cargo --version
RUN python -m pip install --no-cache-dir uv==0.6.17 'maturin>=1.5,<2.0'

View file

@ -1,8 +1,9 @@
name: Headroom e2e setup
description: >-
Checkout-agnostic setup shared by native e2e workflows (init, install, wrap).
Installs Python, installs headroom in editable mode, and (optionally) drops
a noop shim onto PATH so ``headroom init -g <target>`` can detect a tool
Installs Python + Rust toolchain, installs headroom in editable mode (which
builds the bundled Rust extension via maturin), and (optionally) drops a
noop shim onto PATH so ``headroom init -g <target>`` can detect a tool
that isn't actually installed on the runner.
inputs:
python-version:
@ -27,7 +28,18 @@ runs:
with:
python-version: ${{ inputs.python-version }}
- name: Install headroom (editable, with proxy extras)
# Single-wheel architecture: `pip install -e .` invokes maturin (declared
# in pyproject.toml's build-system) which calls cargo to compile the Rust
# extension. Toolchain has to be set up before the install step.
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
- name: Install headroom (editable, with proxy extras — builds Rust extension)
shell: bash
run: |
python -m pip install --upgrade pip
@ -35,6 +47,7 @@ runs:
# ``cli/proxy.py``), which requires ``fastapi`` even for ``init``.
# Install with the ``[proxy]`` extras to match the Docker e2e image.
pip install -e ".[proxy]"
python -c "from headroom._core import DiffCompressor; print('headroom._core OK:', DiffCompressor)"
- name: Drop shim (POSIX)
if: ${{ inputs.shim-target != '' && runner.os != 'Windows' }}

View file

@ -5,14 +5,14 @@
},
"metadata": {
"description": "Headroom marketplace for Claude Code and GitHub Copilot CLI plugins.",
"version": "0.20.13"
"version": "0.20.14"
},
"plugins": [
{
"name": "headroom",
"source": "./plugins/headroom-agent-hooks",
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
"version": "0.20.13",
"version": "0.20.14",
"author": {
"name": "Headroom Contributors",
"url": "https://github.com/chopratejas/headroom"

View file

@ -1,328 +1,296 @@
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Cache pip packages
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-${{ matrix.python-version }}-
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
# `headroom.transforms.diff_compressor` is now a thin pyo3 shim that
# delegates to `headroom._core` (built from `crates/headroom-py`). Without
# the wheel installed, every `DiffCompressor()` call raises
# `ModuleNotFoundError: No module named 'headroom._core'` and ~25 tests
# in `tests/test_transforms/test_diff_compressor.py` fail. The script
# below runs `maturin develop` and symlinks the built `.so` into the
# in-tree `headroom/` package so the editable install resolves it.
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
- name: Install maturin
run: |
pip install 'maturin>=1.5,<2.0'
# `scripts/build_rust_extension.sh` uses `maturin develop` which requires
# a virtualenv. CI runs against bare setup-python with no venv, so build
# a wheel + pip-install it instead. Then symlink the `.so` into the
# in-tree `headroom/` package so the editable install resolves
# `import headroom._core` past the source dir shadowing site-packages.
#
# We locate the `.so` via filesystem (not via `import headroom._core`)
# because the import would fail at this point — the editable install's
# in-tree `headroom/` directory shadows site-packages and doesn't
# contain the `.so` yet. That's exactly the problem this step fixes.
- name: Build Rust extension (headroom._core)
run: |
set -euo pipefail
maturin build --release -m crates/headroom-py/Cargo.toml --out dist
pip install --force-reinstall --no-deps dist/headroom_core_py-*.whl
SITE_PACKAGES=$(python -c "import site; print(site.getsitepackages()[0])")
SO_FILE=$(find "$SITE_PACKAGES/headroom" -maxdepth 1 -name "_core.cpython-*.so" -print -quit 2>/dev/null)
if [[ -z "$SO_FILE" ]]; then
echo "error: could not find _core.cpython-*.so under $SITE_PACKAGES/headroom/" >&2
ls -la "$SITE_PACKAGES/headroom/" || true
exit 1
fi
ln -sf "$SO_FILE" "headroom/$(basename "$SO_FILE")"
python -c "from headroom._core import DiffCompressor; print('headroom._core OK:', DiffCompressor)"
- name: Run linting
if: matrix.python-version == '3.12'
run: |
ruff check .
ruff format --check .
- name: Run type checking
if: matrix.python-version == '3.12'
run: |
mypy headroom --ignore-missing-imports
- name: Run tests
run: |
pytest -v --tb=short tests scripts/tests
- name: Run tests with coverage
if: matrix.python-version == '3.11'
run: |
pytest tests scripts/tests --cov=headroom --cov-report=xml --cov-report=term-missing
- name: Upload coverage to Codecov
if: matrix.python-version == '3.11'
uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
fail_ci_if_error: false
test-extras:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install with relevance extras
run: |
python -m pip install --upgrade pip
pip install -e ".[dev,relevance]"
# `tests/test_relevance.py::TestSmartCrusherIntegration` constructs
# a `SmartCrusher`, which is a hard import of `headroom._core` since
# the Python implementation was retired in Stage 3c.1b. Without the
# extension built, those tests `ModuleNotFoundError`. Build + install
# the wheel and symlink the `.so` into the in-tree `headroom/` so
# the editable install resolves it (same pattern as the main `test`
# job above).
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
- name: Install maturin
run: pip install 'maturin>=1.5,<2.0'
- name: Build Rust extension (headroom._core)
run: |
set -euo pipefail
maturin build --release -m crates/headroom-py/Cargo.toml --out dist
pip install --force-reinstall --no-deps dist/headroom_core_py-*.whl
SITE_PACKAGES=$(python -c "import site; print(site.getsitepackages()[0])")
SO_FILE=$(find "$SITE_PACKAGES/headroom" -maxdepth 1 -name "_core.cpython-*.so" -print -quit 2>/dev/null)
if [[ -z "$SO_FILE" ]]; then
echo "error: could not find _core.cpython-*.so under $SITE_PACKAGES/headroom/" >&2
ls -la "$SITE_PACKAGES/headroom/" || true
exit 1
fi
ln -sf "$SO_FILE" "headroom/$(basename "$SO_FILE")"
python -c "from headroom._core import SmartCrusher; print('headroom._core OK:', SmartCrusher)"
- name: Run relevance tests
run: |
pytest tests/test_relevance.py -v
test-agno:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install with agno extras
run: |
python -m pip install --upgrade pip
pip install -e ".[dev,agno]"
- name: Run agno tests
run: |
pytest tests/test_integrations/agno/ -v
docker-native-e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Build local Headroom image
run: |
docker build -t headroom-native-e2e:latest .
- name: Run Docker-native installer e2e
env:
HEADROOM_DOCKER_IMAGE: headroom-native-e2e:latest
run: |
bash e2e/docker-native-install.sh
- name: Run Docker-native compose smoke test
env:
HEADROOM_IMAGE: headroom-native-e2e:latest
HEADROOM_HOST_HOME: ${{ github.workspace }}
HEADROOM_WORKSPACE: ${{ github.workspace }}
run: |
mkdir -p .headroom .claude .codex .gemini
trap 'docker compose -f docker/docker-compose.native.yml down -v' EXIT
docker compose -f docker/docker-compose.native.yml up -d proxy
for attempt in $(seq 1 30); do
if curl --fail --silent http://127.0.0.1:8787/readyz >/dev/null; then
break
fi
if [ "$attempt" -eq 30 ]; then
docker compose -f docker/docker-compose.native.yml logs proxy
exit 1
fi
sleep 1
done
- name: Run Docker-native wrap e2e
run: |
docker build -f e2e/wrap/Dockerfile -t headroom-wrap-e2e .
docker run --rm headroom-wrap-e2e
- name: Run Docker-native init e2e
run: |
docker build -f e2e/init/Dockerfile -t headroom-init-e2e .
docker run --rm headroom-init-e2e
windows-native-wrapper:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install test dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
- name: Run native installer wrapper tests
run: |
pytest tests/test_install/test_native_installers.py -q
macos-native-wrapper:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install bash and test dependencies
run: |
brew install bash
python -m pip install --upgrade pip
python -m pip install --retries 10 --timeout 60 pytest
- name: Run native installer wrapper tests
run: |
BASH_PREFIX="$(brew --prefix bash)"
export PATH="$BASH_PREFIX/bin:$PATH"
pytest tests/test_install/test_native_installers.py -q
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Install build tools
run: |
python -m pip install --upgrade pip build twine
- name: Build package
run: |
python -m build
- name: Check package
run: |
twine check dist/*
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
commitlint:
if: github.event_name != 'push' || !startsWith(github.event.head_commit.message, 'Merge pull request ')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: wagoid/commitlint-github-action@v5
with:
configFile: .commitlintrc.json
workflow-validation:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install actionlint
run: |
curl -fsSL https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash | bash
sudo mv ./actionlint /usr/local/bin/actionlint
- name: Install act
run: |
curl -fsSL https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash
sudo install ./bin/act /usr/local/bin/act
- name: Validate workflow files
run: |
bash scripts/validate-workflows.sh
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
# `pip install -e .` invokes maturin (declared in `[build-system]
# requires`) under the hood, which calls cargo to build the Rust
# extension. The toolchain has to be available before the install
# step, otherwise build-isolation pulls maturin but `cargo` is
# missing.
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
- name: Cache pip packages
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-${{ matrix.python-version }}-
- name: Install dependencies (builds Rust extension via maturin)
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
python -c "from headroom._core import DiffCompressor; print('headroom._core OK:', DiffCompressor)"
- name: Run linting
if: matrix.python-version == '3.12'
run: |
ruff check .
ruff format --check .
- name: Run type checking
if: matrix.python-version == '3.12'
run: |
mypy headroom --ignore-missing-imports
- name: Run tests
run: |
pytest -v --tb=short tests scripts/tests
- name: Run tests with coverage
if: matrix.python-version == '3.11'
run: |
pytest tests scripts/tests --cov=headroom --cov-report=xml --cov-report=term-missing
- name: Upload coverage to Codecov
if: matrix.python-version == '3.11'
uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
fail_ci_if_error: false
test-extras:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
- name: Install with relevance extras (builds Rust extension via maturin)
run: |
python -m pip install --upgrade pip
pip install -e ".[dev,relevance]"
python -c "from headroom._core import SmartCrusher; print('headroom._core OK:', SmartCrusher)"
- name: Run relevance tests
run: |
pytest tests/test_relevance.py -v
test-agno:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
- name: Install with agno extras (builds Rust extension via maturin)
run: |
python -m pip install --upgrade pip
pip install -e ".[dev,agno]"
- name: Run agno tests
run: |
pytest tests/test_integrations/agno/ -v
docker-native-e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Build local Headroom image
run: |
docker build -t headroom-native-e2e:latest .
- name: Run Docker-native installer e2e
env:
HEADROOM_DOCKER_IMAGE: headroom-native-e2e:latest
run: |
bash e2e/docker-native-install.sh
- name: Run Docker-native compose smoke test
env:
HEADROOM_IMAGE: headroom-native-e2e:latest
HEADROOM_HOST_HOME: ${{ github.workspace }}
HEADROOM_WORKSPACE: ${{ github.workspace }}
run: |
mkdir -p .headroom .claude .codex .gemini
trap 'docker compose -f docker/docker-compose.native.yml down -v' EXIT
docker compose -f docker/docker-compose.native.yml up -d proxy
for attempt in $(seq 1 30); do
if curl --fail --silent http://127.0.0.1:8787/readyz >/dev/null; then
break
fi
if [ "$attempt" -eq 30 ]; then
docker compose -f docker/docker-compose.native.yml logs proxy
exit 1
fi
sleep 1
done
- name: Run Docker-native wrap e2e
run: |
docker build -f e2e/wrap/Dockerfile -t headroom-wrap-e2e .
docker run --rm headroom-wrap-e2e
- name: Run Docker-native init e2e
run: |
docker build -f e2e/init/Dockerfile -t headroom-init-e2e .
docker run --rm headroom-init-e2e
windows-native-wrapper:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install test dependencies
run: |
python -m pip install --upgrade pip
pip install pytest
- name: Run native installer wrapper tests
run: |
pytest tests/test_install/test_native_installers.py -q
macos-native-wrapper:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install bash and test dependencies
run: |
brew install bash
python -m pip install --upgrade pip
python -m pip install --retries 10 --timeout 60 pytest
- name: Run native installer wrapper tests
run: |
BASH_PREFIX="$(brew --prefix bash)"
export PATH="$BASH_PREFIX/bin:$PATH"
pytest tests/test_install/test_native_installers.py -q
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
# Single-wheel build via maturin: produces both the linux wheel and
# the platform-independent sdist in one shot. release.yml's matrix
# is what builds per-platform wheels for PyPI; this `build` job is
# a smoke check that the build system is wired right.
- name: Install build tools
run: |
python -m pip install --upgrade pip
pip install 'maturin>=1.5,<2.0' twine
- name: Build wheel + sdist
run: |
maturin sdist --out dist
maturin build --release --out dist
- name: Check package
run: |
twine check dist/*
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
commitlint:
if: github.event_name != 'push' || !startsWith(github.event.head_commit.message, 'Merge pull request ')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: wagoid/commitlint-github-action@v5
with:
configFile: .commitlintrc.json
workflow-validation:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install actionlint
run: |
curl -fsSL https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash | bash
sudo mv ./actionlint /usr/local/bin/actionlint
- name: Install act
run: |
curl -fsSL https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash
sudo install ./bin/act /usr/local/bin/act
- name: Validate workflow files
run: |
bash scripts/validate-workflows.sh

View file

@ -20,13 +20,9 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e ".[all]"
# `compression_only.evaluate_ccr_lossless` constructs a SmartCrusher
# which now hard-imports `headroom._core` (Stage 3c.1b). Build the
# Rust extension before running the smoke test or every call raises
# `ModuleNotFoundError`. Mirrors the main CI `test` job pattern.
# `pip install -e .` invokes maturin (declared in pyproject.toml's
# build-system) which calls cargo to compile the Rust extension.
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
@ -35,21 +31,9 @@ jobs:
with:
workspaces: ". -> target"
- name: Install maturin
run: pip install 'maturin>=1.5,<2.0'
- name: Build Rust extension (headroom._core)
- name: Install dependencies (builds Rust extension via maturin)
run: |
set -euo pipefail
maturin build --release -m crates/headroom-py/Cargo.toml --out dist
pip install --force-reinstall --no-deps dist/headroom_core_py-*.whl
SITE_PACKAGES=$(python -c "import site; print(site.getsitepackages()[0])")
SO_FILE=$(find "$SITE_PACKAGES/headroom" -maxdepth 1 -name "_core.cpython-*.so" -print -quit 2>/dev/null)
if [[ -z "$SO_FILE" ]]; then
echo "error: could not find _core.cpython-*.so under $SITE_PACKAGES/headroom/" >&2
exit 1
fi
ln -sf "$SO_FILE" "headroom/$(basename "$SO_FILE")"
pip install -e ".[all]"
python -c "from headroom._core import SmartCrusher; print('headroom._core OK:', SmartCrusher)"
- name: Run CCR round-trip (zero cost)
@ -77,8 +61,19 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -e ".[all]"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
- name: Install dependencies (builds Rust extension via maturin)
run: |
pip install -e ".[all]"
python -c "from headroom._core import SmartCrusher; print('headroom._core OK')"
- name: Run Tier 1 evaluation suite
run: python -m headroom.evals suite --tier 1 --ci -o eval_results/
env:

View file

@ -35,11 +35,18 @@ on:
jobs:
init-native:
runs-on: ${{ matrix.os }}
timeout-minutes: 15
timeout-minutes: 25
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
# Windows is excluded today: upstream `esaxx-rs` (transitively from
# `tokenizers`) and `ort-sys` (onnxruntime via `fastembed`) link
# with conflicting MSVC C runtime libraries (/MT vs /MD), so the
# Rust extension cannot build for `win_amd64` until the upstream
# CRT conflict is resolved. Re-add `windows-latest` once the wheel
# builds cleanly there. Tracked in the project plan; not a blocker
# for headroom-ai installs on Linux + macOS.
os: [ubuntu-latest, macos-latest]
target: [claude, codex, copilot, openclaw]
exclude:
# openclaw delegates to ``headroom wrap openclaw`` which needs a

View file

@ -1,8 +1,10 @@
name: Publish to PyPI
name: Publish to PyPI (manual fallback)
# DEPRECATED: This workflow is superseded by release.yml.
# Only used for manual PyPI publishing now.
# Manual trigger only — release.yml handles automated publishing.
# DEPRECATED: This workflow is superseded by release.yml's matrix-based
# build-and-publish flow. It exists as a manual fallback in case release.yml
# is broken and a hotfix needs to be pushed without going through the
# normal tag-driven pipeline. Single-platform; produces only the linux
# x86_64 wheel + sdist. For full cross-platform release, use release.yml.
on:
workflow_dispatch:
@ -11,8 +13,8 @@ jobs:
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write # For trusted publishing
contents: write # For uploading release assets
id-token: write # For trusted publishing
contents: write # For uploading release assets
steps:
- uses: actions/checkout@v4
@ -22,13 +24,22 @@ jobs:
with:
python-version: "3.11"
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@1.95.0
- name: Cache cargo registry + build
uses: Swatinem/rust-cache@v2
with:
workspaces: ". -> target"
- name: Install build tools
run: |
python -m pip install --upgrade pip build cyclonedx-bom
python -m pip install --upgrade pip 'maturin>=1.5,<2.0' cyclonedx-bom
- name: Build package
- name: Build wheel + sdist
run: |
python -m build
maturin sdist --out dist
maturin build --release --out dist
- name: Generate SBOM (CycloneDX)
run: |

View file

@ -73,6 +73,8 @@ jobs:
env:
MANUAL_VER: ${{ github.event.inputs.version }}
# Single source of truth for changelog + npm packaging. Wheels are
# built per-platform in `build-wheels` below. publish-pypi merges them.
build:
needs: [detect-version]
runs-on: ubuntu-latest
@ -132,13 +134,9 @@ jobs:
path: /tmp/changelog-backup.md
if-no-files-found: error
- name: Build Python package
run: python -m pip install build wheel && python -m build
- name: Build npm release packages
run: |
mkdir -p release-assets
cp dist/* release-assets/
cd sdk/typescript
npm install
@ -153,27 +151,124 @@ jobs:
npm version ${{ needs.detect-version.outputs.npm_version }} --no-git-tag-version --allow-same-version
npm pack --pack-destination ../../release-assets
- name: Upload dist artifact
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
- name: Upload release assets artifact
uses: actions/upload-artifact@v4
with:
name: release-assets
path: release-assets/
# Cross-platform wheel matrix. Each entry produces wheels for cp310/11/12/13
# in one maturin invocation (PyO3 ABI3 forward-compat handles 3.14+ until
# we bump pyo3 past 0.22). The `headroom-ai` wheel contains the entire
# Python source under `headroom/` plus the compiled `headroom/_core.so`
# — one atomic install via `pip install headroom-ai`.
build-wheels:
needs: [detect-version, build]
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
manylinux: auto
- os: ubuntu-latest
target: aarch64-unknown-linux-gnu
manylinux: 2_28
- os: macos-15-intel
target: x86_64-apple-darwin
manylinux: ""
- os: macos-14
target: aarch64-apple-darwin
manylinux: ""
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Sync version to pyproject.toml + Cargo.toml
shell: bash
run: |
python scripts/version-sync.py --version ${{ needs.detect-version.outputs.npm_version }}
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist --interpreter python3.10 python3.11 python3.12 python3.13
manylinux: ${{ matrix.manylinux }}
env:
# PyO3 0.22 supports up to Python 3.13; allow forward-compat
# builds for 3.14+ until we bump PyO3 (tracked separately).
PYO3_USE_ABI3_FORWARD_COMPATIBILITY: "1"
- name: Build sdist (linux x86_64 only — sdist is platform-independent)
if: matrix.os == 'ubuntu-latest' && matrix.target == 'x86_64-unknown-linux-gnu'
uses: PyO3/maturin-action@v1
with:
command: sdist
args: --out dist
- name: Upload wheels artifact
uses: actions/upload-artifact@v4
with:
name: wheels-${{ matrix.os }}-${{ matrix.target }}
path: dist/*
# Aggregator step: merge all wheel artifacts + the npm release assets
# into the canonical `dist/` directory for downstream publishing jobs.
collect-dist:
needs: [build, build-wheels]
runs-on: ubuntu-latest
steps:
- name: Download all wheel artifacts
uses: actions/download-artifact@v4
with:
pattern: wheels-*
path: wheels-tmp/
merge-multiple: true
- name: Download release assets
uses: actions/download-artifact@v4
with:
name: release-assets
path: release-assets/
- name: Stage final dist directory
run: |
mkdir -p dist
cp -v wheels-tmp/*.whl wheels-tmp/*.tar.gz dist/ 2>/dev/null || true
ls -la dist/
# Mirror the wheels into release-assets so create-release uploads them.
cp -v dist/*.whl dist/*.tar.gz release-assets/ 2>/dev/null || true
ls -la release-assets/
- name: Upload merged dist artifact
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
- name: Upload merged release-assets artifact
uses: actions/upload-artifact@v4
with:
name: release-assets-merged
path: release-assets/
publish-pypi:
needs: [build]
needs: [collect-dist]
if: github.event.inputs.dry_run != 'true' && vars.PYPI_SKIP != 'true'
environment: pypi # NOTE: environment name must be a literal; update here if the GitHub environment name changes
runs-on: ubuntu-latest
permissions:
id-token: write # Required for OIDC trusted publishing
steps:
- name: Download dist artifact
- name: Download merged dist artifact (sdist + cross-platform wheels)
uses: actions/download-artifact@v4
with:
name: dist
@ -281,32 +376,32 @@ jobs:
cp -R sdk/typescript "$workdir/sdk"
cd "$workdir/sdk"
npm install
npm run build
npm version ${{ needs.detect-version.outputs.npm_version }} --no-git-tag-version --allow-same-version
unscoped_sdk_tarball="$(npm pack --pack-destination "$assets_dir" | tail -n 1)"
node <<'EOF'
const fs = require("fs");
const pkg = JSON.parse(fs.readFileSync("package.json", "utf8"));
pkg.name = `@${process.env.GITHUB_PACKAGES_SCOPE}/${pkg.name}`;
npm install
npm run build
npm version ${{ needs.detect-version.outputs.npm_version }} --no-git-tag-version --allow-same-version
unscoped_sdk_tarball="$(npm pack --pack-destination "$assets_dir" | tail -n 1)"
node <<'EOF'
const fs = require("fs");
const pkg = JSON.parse(fs.readFileSync("package.json", "utf8"));
pkg.name = `@${process.env.GITHUB_PACKAGES_SCOPE}/${pkg.name}`;
pkg.publishConfig = {
...(pkg.publishConfig || {}),
registry: process.env.GITHUB_PACKAGES_REGISTRY_URL,
};
fs.writeFileSync("package.json", `${JSON.stringify(pkg, null, 2)}\n`);
EOF
sdk_tarball="$(npm pack --pack-destination "$assets_dir" | tail -n 1)"
printf 'unscoped_sdk_tarball=%s\n' "$assets_dir/$unscoped_sdk_tarball" >> "$GITHUB_OUTPUT"
printf 'sdk_tarball=%s\n' "$assets_dir/$sdk_tarball" >> "$GITHUB_OUTPUT"
npm publish --access public --registry ${{ env.GITHUB_PACKAGES_REGISTRY_URL }}
};
fs.writeFileSync("package.json", `${JSON.stringify(pkg, null, 2)}\n`);
EOF
sdk_tarball="$(npm pack --pack-destination "$assets_dir" | tail -n 1)"
printf 'unscoped_sdk_tarball=%s\n' "$assets_dir/$unscoped_sdk_tarball" >> "$GITHUB_OUTPUT"
printf 'sdk_tarball=%s\n' "$assets_dir/$sdk_tarball" >> "$GITHUB_OUTPUT"
npm publish --access public --registry ${{ env.GITHUB_PACKAGES_REGISTRY_URL }}
continue-on-error: true
- name: Publish ${{ env.NPM_OPENCLAW_PACKAGE }} to GitHub Package Registry
id: gpr-openclaw-publish
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_PACKAGES_SCOPE: ${{ steps.gh-scope.outputs.scope }}
SDK_TARBALL: ${{ steps.gpr-sdk-publish.outputs.unscoped_sdk_tarball }}
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_PACKAGES_SCOPE: ${{ steps.gh-scope.outputs.scope }}
SDK_TARBALL: ${{ steps.gpr-sdk-publish.outputs.unscoped_sdk_tarball }}
run: |
workdir="$(mktemp -d)"
cp -R plugins/openclaw "$workdir/openclaw"
@ -357,13 +452,15 @@ jobs:
enable_ref_tags: false
create-release:
needs: [detect-version, build, publish-pypi, publish-npm, publish-github-packages, publish-docker]
needs: [detect-version, build, build-wheels, collect-dist, publish-pypi, publish-npm, publish-github-packages, publish-docker]
if: >-
${{
always() &&
github.event.inputs.dry_run != 'true' &&
needs.detect-version.result == 'success' &&
needs.build.result == 'success'
needs.build.result == 'success' &&
needs.build-wheels.result == 'success' &&
needs.collect-dist.result == 'success'
}}
runs-on: ubuntu-latest
permissions:
@ -379,10 +476,10 @@ jobs:
name: changelog
path: /tmp
- name: Download release assets artifact
- name: Download merged release assets (npm tarballs + wheels + sdist)
uses: actions/download-artifact@v4
with:
name: release-assets
name: release-assets-merged
path: release-assets
- name: Show changelog
@ -395,28 +492,28 @@ jobs:
run: |
ls -la release-assets
- name: Create or update GitHub Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="v${{ needs.detect-version.outputs.version }}"
TITLE="Release v${{ needs.detect-version.outputs.version }}"
- name: Create or update GitHub Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="v${{ needs.detect-version.outputs.version }}"
TITLE="Release v${{ needs.detect-version.outputs.version }}"
if gh release view "$TAG" > /dev/null 2>&1; then
gh release edit "$TAG" --title "$TITLE" --notes-file .changelog.md
else
gh release create "$TAG" --title "$TITLE" --notes-file .changelog.md
fi
- name: Publish ${{ env.PYPI_PACKAGE }} Python distributions to GitHub Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="v${{ needs.detect-version.outputs.version }}"
gh release upload "$TAG" release-assets/*.whl release-assets/*.tar.gz --clobber
- name: Publish Node package tarballs to GitHub Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="v${{ needs.detect-version.outputs.version }}"
gh release upload "$TAG" release-assets/*.tgz --clobber
else
gh release create "$TAG" --title "$TITLE" --notes-file .changelog.md
fi
- name: Publish ${{ env.PYPI_PACKAGE }} Python distributions to GitHub Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="v${{ needs.detect-version.outputs.version }}"
gh release upload "$TAG" release-assets/*.whl release-assets/*.tar.gz --clobber
- name: Publish Node package tarballs to GitHub Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="v${{ needs.detect-version.outputs.version }}"
gh release upload "$TAG" release-assets/*.tgz --clobber

View file

@ -86,15 +86,15 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Build wheel
- name: Build wheel (single-wheel architecture: builds headroom-ai)
uses: PyO3/maturin-action@v1
# Maturin reads `[tool.maturin]` from the root `pyproject.toml`
# which points at `crates/headroom-py/Cargo.toml` for the cdylib.
# Output is `headroom_ai-<ver>-<py>-<py>-<platform>.whl` containing
# both Python source and the compiled `headroom/_core.so`.
with:
command: build
# `manifest-path:` is NOT a valid input on this action (it warns and
# ignores). Pass -m inside `args` so maturin sees the right Cargo.toml.
# Without this, maturin runs from the repo root and fails because
# the workspace Cargo.toml has no [package] section.
args: --release -m crates/headroom-py/Cargo.toml --out dist
args: --release --out dist
target: ${{ matrix.maturin-target }}
- name: Upload wheel artifact
uses: actions/upload-artifact@v4

View file

@ -14,15 +14,10 @@ ARG UV_VERSION
# build-essential / g++ for any C extension wheels uv may need to build
# from source. curl + ca-certificates are required by the rustup
# bootstrap below. Hotfix-A0 (Finding #2) added the rust toolchain so the
# image actually carries `headroom._core`; previously the runtime image
# shipped without the Rust extension and every compressed request fell
# back to a Python-only path or no-op.
#
# `pkg-config` + `libssl-dev` are required because the workspace
# transitively pulls `openssl-sys` (via reqwest/native-tls in some
# dependency chain). Without them, `cargo` fails the maturin build with
# "Could not find openssl via pkg-config" — observed in PR #350 CI.
# bootstrap below. pkg-config + libssl-dev for `openssl-sys` (transitive
# from fastembed/hf-hub/ureq → native-tls — the workspace `rustls-tls`
# pin loses to cargo feature unification). patchelf for maturin's
# wheel-link repair on linux.
RUN apt-get update && \
apt-get install -y --no-install-recommends \
build-essential \
@ -36,10 +31,10 @@ RUN apt-get update && \
RUN python -m pip install --no-cache-dir uv==${UV_VERSION}
# Rust toolchain for the headroom._core extension build. Pinned via
# rust-toolchain.toml at the repo root so this matches what local devs
# build with. Installed as root before WORKDIR change so the env
# additions stick for every subsequent RUN.
# Rust toolchain for the headroom._core extension. With single-wheel
# architecture (post-#355), `pip install -e .` invokes maturin via
# pyproject.toml's [build-system], which calls cargo. No more separate
# headroom-core-py package.
ENV CARGO_HOME=/usr/local/cargo \
RUSTUP_HOME=/usr/local/rustup \
PATH=/usr/local/cargo/bin:${PATH}
@ -48,73 +43,28 @@ RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
WORKDIR /build
# Layer 1: install deps only (cached unless pyproject.toml/uv.lock change)
# Copy the full set of files maturin needs to build the wheel: the root
# pyproject.toml + Cargo workspace + Rust crates + Python source. The
# uv install builds + installs the wheel in one shot.
COPY pyproject.toml uv.lock README.md ./
# Stub package so uv can resolve the local extras without full source
RUN mkdir -p headroom && touch headroom/__init__.py
ARG HEADROOM_EXTRAS=proxy,code
RUN --mount=type=cache,target=/root/.cache/uv \
uv pip install --system ".[${HEADROOM_EXTRAS}]"
# Layer 2 (Hotfix-A0): build and install the Rust extension wheel
# BEFORE installing headroom-ai source. Why this order:
#
# * The headroom-core-py wheel includes a stub `headroom/__init__.py`
# plus `headroom/_core.cpython-*.so` (maturin's `python-source`
# layout — see `crates/headroom-py/pyproject.toml`).
# * The headroom-ai install also writes files under `headroom/`.
# * If headroom-ai is installed FIRST and the wheel goes second with
# `--force-reinstall`, pip uninstalls the wheel's previously
# installed files, deleting `headroom/__init__.py` (which the wheel
# also claims). headroom-ai's __init__.py was already overwritten
# by the wheel's empty stub at install-time, so the deletion leaves
# no `__init__.py` at all — `from headroom._core import hello`
# then fails with `ModuleNotFoundError: No module named
# 'headroom._core'`. Observed in PR #350 CI before this reorder.
# * Installing the wheel FIRST means: wheel lays down stub
# `__init__.py` + `_core.so`. Then headroom-ai install OVERWRITES
# `__init__.py` with the real one and adds the rest of the
# `headroom/` tree. `_core.so` survives because headroom-ai
# doesn't claim ownership of it.
#
# uv already installed `maturin` as a transitive of the [proxy]/[code]
# extras; if it didn't, install it explicitly here so the build never
# silently skips.
COPY crates/ crates/
COPY Cargo.toml Cargo.lock rust-toolchain.toml ./
COPY crates/ crates/
COPY headroom/ headroom/
ARG HEADROOM_EXTRAS=proxy,code
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=cache,target=/root/.cargo/registry \
--mount=type=cache,target=/build/target \
uv pip install --system maturin \
&& maturin build --release -m crates/headroom-py/Cargo.toml --out /build/wheels \
&& uv pip install --system --no-deps /build/wheels/headroom_core_py-*.whl
uv pip install --system ".[${HEADROOM_EXTRAS}]"
# Layer 3: copy real source, install headroom-ai (no deps). This
# overwrites the wheel's stub `headroom/__init__.py` with the real one
# and adds the full `headroom/` tree alongside the surviving
# `_core.so` from Layer 2.
COPY headroom/ headroom/
RUN --mount=type=cache,target=/root/.cache/uv \
uv pip install --system --no-deps --reinstall-package headroom-ai .
# Layer 4 (Hotfix-A0): verify the extension actually loads end-to-end
# inside the build image. If this fails, the runtime image would fail
# its lifespan smoke test on every restart — better to break the build
# loudly here than ship a broken image.
#
# IMPORTANT: run from `/tmp`, not from `/build`. `WORKDIR /build` puts
# `''` (cwd) at the front of `sys.path`, which makes `import headroom`
# resolve to `/build/headroom/` (the source tree we just COPY'd in)
# instead of `/usr/local/lib/python3.11/site-packages/headroom/` (where
# the wheel installed `_core.so`). The source tree has no `_core.so`,
# so the verify falsely fails. Production startup runs from a different
# cwd (the proxy's working directory or `/`), so this is a build-time-
# only quirk caused by `WORKDIR /build`. Anchoring the verify in `/tmp`
# matches the production import order: site-packages wins.
RUN cd /tmp && python -c "from headroom._core import hello; \
marker = hello(); \
assert marker == 'headroom-core', f'expected headroom-core, got {marker!r}'; \
print(f'build-stage rust core verify OK: {marker}')"
# Build-stage smoke check: verify the extension loads end-to-end inside
# the build image before we copy site-packages into the runtime image.
# If this fails, the runtime image would fail Phase A0's fail-loud
# startup check on every restart. Run from /tmp so cwd doesn't shadow
# site-packages with /build/headroom/ (which has no _core.so since
# maturin installed the .so into site-packages).
RUN cd /tmp && python -c "from headroom._core import DiffCompressor, SmartCrusher; \
print(f'build-stage rust core verify OK: {DiffCompressor.__name__}, {SmartCrusher.__name__}')"
# ---- Runtime stage (python-slim): supports root/nonroot via build arg ----
FROM python:${PYTHON_VERSION}-slim@${PYTHON_DIGEST} AS runtime-slim-base

View file

@ -1,21 +0,0 @@
[build-system]
requires = ["maturin>=1.5,<2.0"]
build-backend = "maturin"
[project]
name = "headroom-core-py"
version = "0.1.0"
description = "Python bindings to the Rust headroom-core crate."
requires-python = ">=3.10"
license = { text = "Apache-2.0" }
authors = [{ name = "Headroom Maintainers" }]
[tool.maturin]
# Build as a submodule of the `headroom` namespace so it becomes
# `from headroom._core import hello`.
module-name = "headroom._core"
# Keep python source layout out of this directory; maturin will install the
# compiled module into an existing `headroom` package in site-packages (or
# the active venv's overlay created by `maturin develop`).
python-source = "python"
features = ["extension-module"]

View file

@ -1,38 +1,50 @@
FROM node:22-bookworm
ENV DEBIAN_FRONTEND=noninteractive \
PATH="/opt/headroom-venv/bin:${PATH}" \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_NO_CACHE_DIR=1 \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
git \
python3 \
python3-pip \
python3-venv && \
ln -sf /usr/bin/python3 /usr/local/bin/python && \
rm -rf /var/lib/apt/lists/*
WORKDIR /workspace
COPY pyproject.toml README.md uv.lock ./
COPY headroom ./headroom
COPY .claude-plugin ./.claude-plugin
COPY .github/plugin ./.github/plugin
COPY plugins/headroom-agent-hooks ./plugins/headroom-agent-hooks
# The init e2e harness imports from e2e._lib; both directories must be
# present and each must contain an __init__.py so Python sees them as
# packages rooted at /workspace.
COPY e2e/__init__.py ./e2e/__init__.py
COPY e2e/_lib ./e2e/_lib
COPY e2e/init ./e2e/init
RUN python -m venv /opt/headroom-venv && \
/opt/headroom-venv/bin/python -m pip install --upgrade "pip<25" && \
/opt/headroom-venv/bin/python -m pip install -e ".[proxy]"
CMD ["python", "e2e/init/run.py"]
FROM node:22-bookworm
ENV DEBIAN_FRONTEND=noninteractive \
PATH="/root/.cargo/bin:/opt/headroom-venv/bin:${PATH}" \
CARGO_HOME=/root/.cargo \
RUSTUP_HOME=/root/.rustup \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_NO_CACHE_DIR=1 \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
# Single wheel architecture: `pip install -e .` invokes maturin (declared in
# pyproject.toml's build-system) which calls cargo to compile the Rust
# extension. The image needs python + rust + maturin at build time.
RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
curl \
git \
pkg-config \
libssl-dev \
python3 \
python3-pip \
python3-venv && \
ln -sf /usr/bin/python3 /usr/local/bin/python && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.95.0 --profile minimal && \
rm -rf /var/lib/apt/lists/*
WORKDIR /workspace
COPY pyproject.toml README.md uv.lock ./
COPY Cargo.toml Cargo.lock rust-toolchain.toml ./
COPY headroom ./headroom
COPY crates ./crates
COPY .claude-plugin ./.claude-plugin
COPY .github/plugin ./.github/plugin
COPY plugins/headroom-agent-hooks ./plugins/headroom-agent-hooks
# The init e2e harness imports from e2e._lib; both directories must be
# present and each must contain an __init__.py so Python sees them as
# packages rooted at /workspace.
COPY e2e/__init__.py ./e2e/__init__.py
COPY e2e/_lib ./e2e/_lib
COPY e2e/init ./e2e/init
RUN python -m venv /opt/headroom-venv && \
/opt/headroom-venv/bin/python -m pip install --upgrade "pip<25" && \
/opt/headroom-venv/bin/python -m pip install -e ".[proxy]" && \
/opt/headroom-venv/bin/python -c "from headroom._core import DiffCompressor; print('headroom._core OK')"
CMD ["python", "e2e/init/run.py"]

View file

@ -4,40 +4,45 @@ ENV DEBIAN_FRONTEND=noninteractive \
AIDER_CHAT_VERSION=0.86.2 \
CODEX_VERSION=0.118.0 \
OPENCLAW_VERSION=2026.4.7 \
PATH="/opt/headroom-venv/bin:/opt/aider-venv/bin:${PATH}" \
PATH="/root/.cargo/bin:/opt/headroom-venv/bin:/opt/aider-venv/bin:${PATH}" \
CARGO_HOME=/root/.cargo \
RUSTUP_HOME=/root/.rustup \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_NO_CACHE_DIR=1 \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
# Hotfix-A0: this image installs headroom in editable mode and does
# NOT carry the Rust extension — the wrap-e2e harness exercises CLI
# routing only, not compression, so booting the proxy in degraded
# Python-only mode is the right scope for this test. Production
# images (main Dockerfile + smoke-test) build the wheel; this one
# opts out of the fail-loud startup smoke check.
HEADROOM_REQUIRE_RUST_CORE=false
PYTHONDONTWRITEBYTECODE=1
# Single wheel architecture: `pip install -e .` invokes maturin under the
# hood which compiles the Rust extension. With rust + maturin available at
# build time, the image now ships the full Rust core — no
# `HEADROOM_REQUIRE_RUST_CORE=false` opt-out needed.
RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates \
curl \
git \
pkg-config \
libssl-dev \
python3 \
python3-pip \
python3-venv && \
ln -sf /usr/bin/python3 /usr/local/bin/python && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.95.0 --profile minimal && \
rm -rf /var/lib/apt/lists/*
WORKDIR /workspace
COPY pyproject.toml README.md uv.lock ./
COPY Cargo.toml Cargo.lock rust-toolchain.toml ./
COPY headroom ./headroom
COPY crates ./crates
COPY sdk/typescript ./sdk/typescript
COPY plugins/openclaw ./plugins/openclaw
RUN python -m venv /opt/headroom-venv && \
/opt/headroom-venv/bin/python -m pip install --upgrade pip && \
/opt/headroom-venv/bin/python -m pip install -e ".[proxy]" && \
/opt/headroom-venv/bin/python -c "from headroom._core import DiffCompressor; print('headroom._core OK')" && \
python -m venv /opt/aider-venv && \
/opt/aider-venv/bin/python -m pip install --upgrade pip && \
/opt/aider-venv/bin/python -m pip install "aider-chat==${AIDER_CHAT_VERSION}" && \

View file

@ -1,6 +1,6 @@
{
"name": "headroom",
"version": "0.20.13",
"version": "0.20.14",
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
"author": {
"name": "Headroom Contributors",

View file

@ -1,6 +1,6 @@
{
"name": "headroom",
"version": "0.20.13",
"version": "0.20.14",
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
"author": {
"name": "Headroom Contributors",

View file

@ -1,6 +1,6 @@
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
requires = ["maturin>=1.5,<2.0"]
build-backend = "maturin"
[project]
name = "headroom-ai"
@ -220,22 +220,30 @@ Repository = "https://github.com/chopratejas/headroom"
Issues = "https://github.com/chopratejas/headroom/issues"
Changelog = "https://github.com/chopratejas/headroom/blob/main/CHANGELOG.md"
[tool.hatch.build.targets.wheel]
packages = ["headroom"]
# Include non-Python files (dashboard templates, etc.)
artifacts = [
"headroom/dashboard/templates/*.html",
]
[tool.hatch.build.targets.sdist]
include = [
"/headroom",
"/tests",
"/LICENSE",
"/NOTICE",
"/README.md",
"/CHANGELOG.md",
]
# Maturin builds a single wheel containing both the Python source under
# `headroom/` AND the compiled Rust extension `headroom/_core.so` (cdylib
# from `crates/headroom-py`). One `pip install headroom-ai` ships everything
# atomically — no separate `headroom-core-py` package, no chicken-and-egg,
# no PIP_FIND_LINKS plumbing. Phase A0's runtime fail-loud check still
# exists but only fires if someone forces an sdist install on a platform
# without a wheel and the rust toolchain isn't available to compile it.
[tool.maturin]
# Where the Python package lives. With `python-source = "."` and the
# package directory `headroom/` at repo root, maturin includes every file
# under `headroom/` in the wheel — that picks up the dashboard HTML
# templates, the bundled YAML configs, etc., without needing an explicit
# `include` list.
python-source = "."
module-name = "headroom._core"
# The cdylib source lives under `crates/headroom-py`. Maturin invokes
# `cargo build` with this manifest to produce `_core.cdylib`, then injects
# the resulting `.so` into the wheel at `headroom/_core.so`.
manifest-path = "crates/headroom-py/Cargo.toml"
features = ["extension-module"]
# Forbid building without the cdylib feature — bare `cargo build` won't
# produce a usable Python extension. Maturin's default `bindings` is "pyo3"
# which is correct here (see `crates/headroom-py/src/`).
bindings = "pyo3"
[tool.ruff]
target-version = "py310"

View file

@ -1,26 +1,19 @@
#!/usr/bin/env bash
# Build the Rust → Python extension (headroom._core) and link it into the
# in-tree `headroom/` package so `import headroom._core` resolves.
# Build + install the Rust extension (headroom._core) into the active venv.
#
# Why a wrapper script: `maturin develop` builds the `.so` and installs it
# into the venv's site-packages, but the in-tree `headroom/` source
# directory (loaded via `pip install -e .`) shadows that on sys.path.
# Python finds `headroom/__init__.py` at the project root before reaching
# the maturin overlay, so `import headroom._core` fails. Symlinking the
# built `.so` into `headroom/` fixes the lookup with zero copies.
# With single-wheel architecture (post-#355), `pip install -e .` invokes
# maturin (declared in pyproject.toml's `[build-system]`) which builds the
# Rust extension and installs it into site-packages alongside the Python
# source. Earlier versions of this script symlinked the .so into the
# in-tree `headroom/` directory because the dual-package layout left the
# .so in `crates/headroom-py/python/headroom/`. That dance is no longer
# needed — maturin places the .so directly in the editable install's
# overlay and Python's import system finds it.
#
# Hotfix-A0 (2026-05-02): the script now also runs an end-to-end import
# verification with the `hello()` marker so a partial / stale build is
# caught before the proxy is started. This mirrors the lifespan smoke
# test in `headroom.proxy.server._check_rust_core` so dev-time and
# deploy-time both catch the same class of failure.
#
# Idempotent. Safe to run repeatedly. Requires `maturin` in PATH (i.e.
# inside the project venv).
# Idempotent. Safe to run repeatedly.
set -euo pipefail
# Move to repo root so all relative paths below are stable.
cd "$(dirname "$0")/.."
log() {
@ -32,59 +25,34 @@ fail() {
exit 1
}
# Step 1: pre-flight. Maturin must be on PATH and a venv must be active —
# `maturin develop` writes into site-packages, and we want that write to
# land in the same env the proxy will run in.
if ! command -v maturin >/dev/null 2>&1; then
fail "maturin not found on PATH. Activate the venv first: source .venv/bin/activate"
fi
# Pre-flight: a venv should be active. The install would otherwise write
# into the system Python.
if [[ -z "${VIRTUAL_ENV:-}" ]]; then
log "warning: VIRTUAL_ENV is unset; maturin will install into the system Python."
log "warning: VIRTUAL_ENV is unset; pip will install into the system Python."
log " If that is not what you want, abort and 'source .venv/bin/activate' first."
fi
# Step 2: build + install via `maturin develop` (in-place editable install
# with C extensions). This produces a `.so` under
# crates/headroom-py/python/headroom/.
log "step 1/3: maturin develop"
maturin develop -m crates/headroom-py/Cargo.toml \
|| fail "maturin develop failed (see output above)"
# Step 3: locate the built artifact. `maturin develop` writes
# `_core.cpython-<ver>-<platform>.{so,dylib,pyd}` into the package dir.
SO_FILE=$(find crates/headroom-py/python/headroom -maxdepth 1 \
\( -name "_core.cpython-*.so" -o -name "_core.cpython-*.dylib" -o -name "_core.pyd" \) \
2>/dev/null | head -1 || true)
if [[ -z "${SO_FILE}" ]]; then
fail "maturin develop succeeded but produced no _core.* binary in crates/headroom-py/python/headroom/"
if ! command -v cargo >/dev/null 2>&1; then
fail "cargo not found on PATH. Install Rust toolchain (rustup) first."
fi
# Step 4: symlink into the in-tree package dir so the in-tree
# `headroom/__init__.py` resolves the `_core` submodule. Only the symlink
# style is supported; copy semantics drift on every rebuild.
LINK_NAME="headroom/$(basename "${SO_FILE}")"
ln -sf "$(pwd)/${SO_FILE}" "${LINK_NAME}" \
|| fail "failed to symlink ${SO_FILE} into ${LINK_NAME}"
log "step 2/3: linked ${LINK_NAME} -> ${SO_FILE}"
# Build + install in one shot. The `[build-system] build-backend = "maturin"`
# in pyproject.toml means pip drives maturin under the hood. The resulting
# wheel contains both the Python source and the compiled `headroom/_core.so`,
# and pip installs them into the editable overlay together.
log "pip install -e . (drives maturin via build-backend)"
python -m pip install -e . || fail "pip install -e . failed (see output above)"
# Step 5: end-to-end import verification. This is the same check the
# proxy lifespan runs at startup. Failing here means the build produced
# something that can't be loaded — fix the build, don't fix the proxy.
log "step 3/3: verifying \`from headroom._core import hello\`"
# End-to-end verification — same shape as Phase A0's startup smoke check.
log "verifying \`from headroom._core import DiffCompressor, SmartCrusher\`"
python -c '
import sys
try:
from headroom._core import hello, DiffCompressor
from headroom._core import DiffCompressor, SmartCrusher
except Exception as exc:
print(f"verify FAILED: {type(exc).__name__}: {exc}", file=sys.stderr)
sys.exit(1)
marker = hello()
if marker != "headroom-core":
print(f"verify FAILED: hello() returned {marker!r}, expected \"headroom-core\"", file=sys.stderr)
sys.exit(1)
print(f"verify OK: hello()={marker!r}, DiffCompressor={DiffCompressor!r}")
print(f"verify OK: DiffCompressor={DiffCompressor!r}, SmartCrusher={SmartCrusher!r}")
' || fail "import verification failed (see above)"
log "headroom._core build + install + verify: OK"

8448
uv.lock generated

File diff suppressed because it is too large Load diff