The @1.95.0 git ref of dtolnay/rust-toolchain shipped action code
that errors on ubuntu-latest with:
failed to install component: 'clippy-preview-x86_64-unknown-linux-gnu',
detected conflict: 'bin/cargo-clippy'
The runner's pre-installed Rust ships cargo-clippy at $HOME/.cargo/bin,
and the older action code's rustup invocation hits a path conflict
when adding the clippy-preview component for 1.95.0.
The @stable ref of the action has the fix; pass toolchain: 1.95.0 as
input so the version stays pinned. rust-toolchain.toml continues to
be the source of truth for the version (used by cargo's
auto-detection); this keeps the action's install in sync.
The action was set to @stable, which installs whatever the latest
stable is (1.95.0 right now). Then maturin invokes cargo, which reads
rust-toolchain.toml and re-resolves to "1.95.0 + clippy + rustfmt".
rustup treats stable and 1.95.0 as distinct toolchain identities and
refuses the second install with:
failed to install component 'clippy-preview-x86_64-unknown-linux-gnu',
detected conflict: 'bin/cargo-clippy'
This was intermittent across the matrix (only test (3.10) tripped on
the most recent run; others got lucky on cache state). Pinning the
action ref to 1.95.0 makes both sides ask for the exact same toolchain
identity, so the second install is a no-op and the conflict can't fire.
Bump procedure stays the same: when rust-toolchain.toml's channel
changes, update these refs in lock-step.
Plugin manifests auto-bumped 0.11.0 -> 0.13.2 by sync-plugin-versions
hook (unrelated to the workflow fix).
The cosign signing step passed bake metadata via env var:
env:
BAKE_META: ${{ steps.bake.outputs.metadata }}
run: echo "$BAKE_META" | jq ...
For large bake targets (code-nonroot, runtime-code-nonroot) the
metadata JSON is large enough that combined argv+env at bash spawn
exceeds Linux ARG_MAX (~128 KiB on ubuntu-latest), so bash dies with
E2BIG before the script even runs.
Switch to writing metadata into a heredoc-backed temp file, then read
it via jq file input. Heredocs put the JSON in the script body itself,
which bash reads from a temp file (no ARG_MAX limit), bypassing the
env-size ceiling entirely.
Module: .github/workflows/docker.yml
Five gates broke on the 2026-04-27 push of the smart_crusher branch.
Each is fixed below; the second half adds a `make ci-precheck` target
(plus an installable git pre-push hook) so the same dance never happens
again.
Failures fixed:
1. cargo fmt — 22 files had formatting drift introduced over the
stage 3c.1 work. `cargo fmt --all` reformatted them; no semantic
changes. `cargo test --workspace` still green (388 + supporting).
2. wheels job (macOS x86_64) — `fastembed -> ort -> ort-sys` does not
publish prebuilt ONNX Runtime binaries for `x86_64-apple-darwin`.
Removed that target from `.github/workflows/rust.yml`'s wheels
matrix. Apple Silicon (`aarch64-apple-darwin`) covers macOS
distribution; Intel macOS users can build from source. The matrix
now has 2 targets: linux x86_64 + macOS aarch64.
3. test-extras (relevance.py) — `tests/test_relevance.py::TestSmartCrusherIntegration`
constructs a `SmartCrusher`, which hard-imports `headroom._core`
since the python implementation was retired in stage 3c.1b. The
test-extras job didn't build the rust extension. Added the same
`maturin build + symlink` block the main `test` job uses.
4. smoke-test (eval.yml) — same root cause:
`compression_only.evaluate_ccr_lossless` instantiates a SmartCrusher.
Same fix: build the rust extension before the smoke test runs.
5. commitlint — three rules tripped:
- `subject-case` rejects PascalCase identifiers in subjects, but
the project deliberately names classes (SmartCrusher, HfTokenizer,
ContentRouter, DiffCompressor) in commit subjects. Disabled.
- `footer-leading-blank` is a warning that the wagoid action turns
into a CI failure; lines like `Module: foo.rs` in our bodies
match the conventional footer pattern and trip it. Disabled.
- `type-enum` doesn't include `parity`, but the project ships
parity-test infrastructure as its own concern (separate from
`test:`); added `parity` to the allowed types.
Pre-push verification — the prevention half:
`make ci-precheck` runs all of the above CI gates locally:
- `ci-precheck-rust`: cargo fmt --check + clippy + test --workspace.
- `ci-precheck-python`: builds the rust extension via maturin, then
runs the smart_crusher-affected python test files (185 tests across
test_transforms/, test_relevance*, test_ccr, test_acceptance,
test_critical_fixes, test_quality_retention).
- `ci-precheck-commitlint`: `npx commitlint --from origin/main --to
HEAD` against the same config CI uses. Skipped silently if npx is
not on PATH (install Node 18+ to enable).
`make install-git-hooks` (or `scripts/install-git-hooks.sh`) installs
a git pre-push hook that runs `make ci-precheck` automatically.
Bypass with `--no-verify` only when truly needed.
When new CI gates land in `.github/workflows/`, mirror them into a
`make ci-precheck-*` target. The Makefile is the local mirror of the
CI configuration; keeping them in sync is a load-bearing invariant.
Verification: `make ci-precheck` runs green on this commit.
The previous version of this step did `python -c "import headroom._core"`
to find the wheel's installed `.so` path. That failed in CI:
ModuleNotFoundError: No module named 'headroom._core'
— exactly the chicken-and-egg this step exists to fix. The editable
install (`pip install -e .`) puts the in-tree `headroom/` source dir
ahead of site-packages on `sys.path`. So `import headroom` finds the
in-tree dir (which doesn't yet have the `.so`), then
`import headroom._core` fails to find the submodule. The symlink we're
about to create is what makes the import work — but we can't import
to discover the symlink target before creating it.
Locate the `.so` via filesystem instead: read site-packages from
`site.getsitepackages()[0]`, glob for `_core.cpython-*.so` under
`<site-packages>/headroom/`, and symlink that into the in-tree dir.
The smoke test (`from headroom._core import DiffCompressor`) runs
*after* the symlink and confirms end-to-end resolution.
Also added `set -euo pipefail` and a sanity check with `ls -la` of the
site-packages dir if the glob comes up empty, so future failures
diagnose themselves.
`maturin develop` requires a virtualenv (it errors with "Couldn't find a
virtualenv or conda environment"). CI's setup-python provides a bare
system Python without a venv, so the dev script's build path doesn't
work there.
This switches CI to build a release wheel via `maturin build` and
install it with `pip install --force-reinstall --no-deps`. Then symlink
the installed `.so` into the in-tree `headroom/` package so the
editable install resolves `import headroom._core` past the source-dir
shadowing of site-packages.
`scripts/build_rust_extension.sh` is unchanged — it stays optimized for
local dev (where there IS a venv).
The python `DiffCompressor` was retired in this PR's stage-3b commit;
the public class now delegates to `headroom._core` (built from
`crates/headroom-py`). Without the wheel installed in CI, every test
that constructs a `DiffCompressor` fails with `ModuleNotFoundError:
No module named 'headroom._core'`.
This adds a build step to the main `test` job in `ci.yml` that:
1. Installs the stable Rust toolchain (`dtolnay/rust-toolchain@stable`).
2. Caches the cargo registry and build output (`Swatinem/rust-cache@v2`).
3. Installs maturin.
4. Runs `scripts/build_rust_extension.sh`, which calls `maturin develop`
and symlinks the built `.so` into the in-tree `headroom/` package
so the editable install resolves `import headroom._core`.
Only the main `test` job needs this — `test-extras` and `test-agno`
run narrow subsets that don't construct `DiffCompressor`. The existing
`rust.yml` workflow continues to handle wheel builds for distribution
and `cargo test` for the Rust workspace.
GitHub Actions deprecated the macos-13 runner label. The validate-workflows
actionlint step in CI fails because macos-13 is no longer in the available
labels list. macos-15-intel is the current x86_64 macOS runner.
(Bumped from macos-14 to macos-15 for arm64 was unnecessary; macos-14 is
still valid and we keep it for cache-warmth.)
cargo fmt --check failed in CI: import order in proxy.rs (cfg(test)
attributes before/after non-attr imports) and a few line-wrapping
nits in e2e_real.rs. Ran cargo fmt --all to fix.
maturin-action@v1 does not have a 'manifest-path' input — the action
warned 'Unexpected input(s) manifest-path' and proceeded to invoke
maturin from the repo root, which sees the workspace Cargo.toml with
no [package] section and bails. Move -m crates/headroom-py/Cargo.toml
back inside the 'args' string.
Adds 0.10.7-ab46594 (root) and 0.10.7-<variant>-<sha> (variants) so
images can be referenced by an exact version+commit pair without
relying on the moving variant or :latest tags.
CodeQL alert #61 (CWE-275, actions/missing-workflow-permissions):
add explicit `permissions: contents: read` to the rust workflow root.
Defaults the GITHUB_TOKEN to read-only across all jobs, so even if the
repo policy changes, this workflow stays at least-privilege. No job in
this workflow needs write — wheels/audit/parity all read-only.
Add real end-to-end test suite at tests/e2e_real.rs gated behind
HEADROOM_E2E=1. Spawns the actual Python Headroom proxy as a subprocess,
runs the Rust proxy in-process in front of it, and exercises:
- health endpoints across the full chain
- Anthropic non-streaming (real API call)
- Anthropic streaming SSE (real API call) with chunk-level validation
- OpenAI non-streaming (real API call)
- X-Request-Id generation and pass-through
Adds tokio-process feature for Command/Child usage. Loads .env at the
repo root for API keys (does not log values). Tests skip cleanly when
HEADROOM_E2E is unset, so cargo test stays fast.
When the docker workflow is triggered directly by release.published
(rather than via workflow_call from the Release parent), inputs.enable_ref_tags
is null and produced an empty enable= attribute that the metadata-action
rejected. Default to true on non-release triggers and skip ref/pr tags
on release events where they don't apply anyway.
- Replace full-sha image tags with type=sha,format=short (7-char) so the
primary package versions list stops accumulating long sha-only entries.
- Route cosign signatures into a sibling GHCR package via
COSIGN_REPOSITORY=<image>-signatures, so the main image's package
version list stays clean. GHCR does not yet implement the OCI 1.1
Distribution Referrers API (community discussion #163029, June 2025),
so legacy signature mode is used here -- OCI 1.1 mode would force the
signature manifest's subject into the same repo as the image and
override COSIGN_REPOSITORY. Verifiers must export the same
COSIGN_REPOSITORY value when running 'cosign verify'.
- Add a promote-latest job that runs after the variant matrix and
re-pushes the :latest tag pointing at the root image with a unique
index annotation. This forces a fresh manifest digest, generating a
new GHCR package version with current timestamp so :latest sits at
the top of the version listing instead of whichever variant happened
to finish last.
maturin>=1.5 requires -m to point to Cargo.toml, not pyproject.toml.
Fixes wheel build job failure in CI (all three matrix targets).
Also switches to manifest-path: action param for cleaner workflow syntax.
Applies same fix to Makefile build-wheel and develop targets.
Two fixes for the init-native-e2e matrix surfaced on PR #256:
1. Composite action installed `headroom` without extras, but
`headroom/cli/__init__.py` eagerly imports `proxy.server` (via
`cli/proxy.py`), which requires `fastapi`. All 6 POSIX jobs hit
`ModuleNotFoundError: No module named 'fastapi'` before `init` ran.
Fix: install `-e .[proxy]` to match the Docker e2e image.
2. On Windows, shims are `.cmd` files and Git Bash's `which` cannot
resolve them (exact-match only). Python's `shutil.which` (used by
`headroom init`) honors PATHEXT and finds the shim fine, but the
pre-flight `which` step failed first. Fix: use `Get-Command` via
`pwsh` for the Windows verification step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Existing Docker init-e2e runs on ubuntu only. Platform-specific bugs
(Windows path separators in written hook commands, PowerShell-vs-bash
matcher strings, macOS keychain prompts, shutil.which PATHEXT quirks)
slip past it. Add a matrix workflow that drops a noop shim for each
target agent and runs ``headroom init -g <target>`` on each of the
three supported OSes, then asserts the settings file was written to
the platform-correct location.
Matrix: [ubuntu-latest, macos-latest, windows-latest] x [claude,
codex, copilot]. ``openclaw`` is excluded because it delegates to
``headroom wrap openclaw`` which needs a real OpenClaw CLI and can't
be stubbed with a noop shim; the Docker suite already covers its
negative path.
Common setup (Python install, editable headroom install, shim drop,
PATH wiring) is factored into a composite action at
.github/actions/headroom-e2e-setup so follow-up per-command workflows
(install-native-e2e, wrap-native-e2e) can be near-copies that only
supply their matrix and assertion blocks. The composite action uses
the cross-platform shim scripts from e2e/_lib/make_shim.{sh,ps1} that
landed with the harness refactor.
Scoped trigger: pull_request touching init code OR the harness, plus
pushes to main and manual dispatch. This avoids burning CI minutes on
every push to unrelated feature branches while still gating every PR
that could regress init behavior.
Not verified locally: Windows runner behavior. Reviewer should watch
the first matrix run on PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The openclaw plugin depends on headroom-ai, but during the release build
the version-sync script updates the dependency to the new version (e.g.
^0.6.7) which hasn't been published to npm yet. Fix by installing the
locally-packed SDK tarball first so the dependency is already satisfied
when npm install runs for remaining packages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The version-sync.py script already sets the target version in package.json
before npm version runs. When npm version receives the same version that's
already in package.json, it exits with "Version not changed" (exit code 1),
breaking the build job. Adding --allow-same-version makes npm version a
no-op when the version matches, fixing the release pipeline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bring the branch onto upstream/main so GitHub validates the same merged
compression code locally, and install act into PATH for the
workflow-validation job.
This keeps the compact-JSON compression regression fixed while making
the shared validation script pass in CI.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix workflow validation failures by wiring detect-version outputs into all
release publish jobs, renaming the GitHub Packages skip variable to a
valid Actions variable name, and adjusting the macOS PATH export for
actionlint.
Also make min_tokens_to_compress use token counting instead of whitespace
splits so compact JSON tool outputs still compress after merging the
latest main branch changes, and add a regression test for that path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a workflow-validation CI job that installs actionlint and act,
checks the release and Docker workflows against checked-in event
fixtures, and shares the same validation script developers can run
locally.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Call the Docker workflow from the release pipeline so Docker publishes in
the same run, build npm tarballs alongside Python distributions, and
attach those artifacts to the GitHub release page.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Derive the exact Docker image version from the release tag or manual
workflow input, sync versioned files in the build workspace before the
image build, and publish an explicit matching image tag.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Determine the release bump from all unreleased commits since the previous
release tag and apply the highest required semantic version increment.
This keeps feat commits at a minor bump unless a breaking change requires
major, even when later patch-level commits are present.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the inline release version math with a tested helper that normalizes legacy four-part tags and computes a single semantic version for packages and GitHub releases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Grant the release job contents write permission so GitHub releases can be created, and add the missing docs/overrides directory required by MkDocs deployment.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- publish-pypi: add permissions: id-token: write for OIDC trusted publishing
- publish-npm: add npm run build before npm publish for both packages
- publish-github-packages: add npm run build, use --registry for GPR
- version-sync: add update_openclaw_package_json to sync headroom-ai dep range
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The release workflow now uses a loop-free algorithm:
- pyproject.toml is the canonical source of truth (never committed by workflow)
- Git tags use v{canonical}.{height} format (e.g. v0.5.25.3)
- npm publishes use 3-part semver bumped from canonical
- No commit step eliminates infinite release loops
- paths-ignore reduces unnecessary workflow triggers
Also:
- Add .releaseetadata to .gitignore
- Separate npm_version output for semver-compatible npm publishing
- create-release no longer blocks on publish jobs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
