mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-08-27 14:17:10 -04:00
6f2c0a8400
# Root cause of the wheel-build cascade We have shipped 5 release-pipeline hot-fixes in 12 hours, each addressing a different symptom of the same architectural problem: 1. PR #363 — npm artifact downloads + tried `yum openssl-devel` 2. PR #367 — vendored OpenSSL in `headroom-proxy` + dropped Intel mac 3. PR #369 — Debian-cross perl install (`perl` not `libipc-cmd-perl`) 4. PR #370 — moved `openssl/vendored` from headroom-proxy to headroom-py 5. (this PR) — ELIMINATE OpenSSL entirely Each fix exposed a different missing system package or feature flag in a different build surface (manylinux x86_64 vs aarch64-cross-Debian vs macOS Intel vs e2e/wrap Dockerfile vs e2e/init Dockerfile vs main Dockerfile vs devcontainer). We were playing whack-a-mole because every Cargo dep change to the OpenSSL surface required matching system-package updates in 6+ different Dockerfiles and workflows, and the PR-level CI didn't exercise all of them. # Why this PR is the structural fix `fastembed` exposes clean rustls feature flags: - `hf-hub-rustls-tls` (replaces default `hf-hub-native-tls`) - `ort-download-binaries-rustls-tls` (replaces default `…native-tls`) By disabling fastembed's default features and enabling the rustls variants explicitly, we remove `native-tls` (and therefore `openssl-sys`, `openssl`, `openssl-src`, perl modules, OpenSSL build-time deps, vendored OpenSSL ~30s build cost) from the entire workspace dep tree. Verified locally: $ cargo tree -p headroom-py -i openssl-sys error: package ID specification `openssl-sys` did not match any packages $ cargo tree -p headroom-py -i native-tls error: package ID specification `native-tls` did not match any packages $ cargo build --release -p headroom-py Finished `release` profile [optimized] target(s) in 25.57s (Down from 1m+ with vendored OpenSSL.) # Cleanups enabled by this change - crates/headroom-py/Cargo.toml — dropped the `openssl/vendored` workaround from PR #370. - crates/headroom-proxy/Cargo.toml — same dep removed. - e2e/wrap/Dockerfile — dropped `yum install openssl-devel pkgconfig perl-IPC-Cmd`. Comment retained explaining why. - e2e/init/Dockerfile — same. - Dockerfile (main) — dropped `pkg-config libssl-dev` from apt-get. - .devcontainer/Dockerfile — dropped `pkg-config libssl-dev`. - .github/workflows/release.yml — removed the entire before-script-linux block (perl install probe + multi-package-manager dispatch + fail-loud assertion). No longer needed. # Regression gate Three new structural tests in tests/test_release_workflows.py: - test_no_openssl_sys_in_wheel_build_tree — runs `cargo tree -p <crate> -i openssl-sys` for headroom-py / headroom-proxy / headroom-core. If openssl-sys reappears (a future native-tls enabler creeping in via a new dep), this fails AT PR TIME with an actionable message. - test_no_native_tls_in_wheel_build_tree — same shape, native-tls is the proximate cause. - test_fastembed_uses_rustls_features — checks the Cargo.toml so a future "let me bump fastembed and forget the features" doesn't silently re-introduce OpenSSL. Plus two cleanup gates: - test_dockerfiles_no_longer_install_openssl_devel - test_release_yml_does_not_install_openssl_or_perl_for_wheels All 13 release-workflow tests pass. `make ci-precheck` PASSED. # What this teaches us about rollouts (per user's ultrathink ask) The 5-fix cascade exposed three meta-problems: 1. PR checks don't block merges. PR #370 had docker-init-e2e, docker-wrap-e2e, docker-native-e2e all FAILED yet got merged. Branch protection should require these checks. Operator action needed (cannot fix in code). 2. Local validation is misleading. `cargo build -p headroom-py` from the workspace root used the workspace lockfile and looked green; CI did fresh resolution against headroom-py's manifest alone where the feature wasn't enabled. Lesson: verify structural invariants with `cargo tree -e features` before trusting that a build "works." 3. 6+ build surfaces with independent system-dep state. Every Cargo change required matching updates in 6 places. The structural answer (this PR) is to NOT depend on system OpenSSL at all. Where structural fixes are not possible, the answer is a single shared scripts/install-rust-build-deps.sh — but with this PR there's nothing left to install.
79 lines
3.6 KiB
Docker
79 lines
3.6 KiB
Docker
# ─── Stage 1: build the headroom-ai wheel in manylinux ─────────────────────
|
|
# See e2e/wrap/Dockerfile for the full rationale. tl;dr: building from
|
|
# source inside `node:22-bookworm` produced a `_core.so` referencing
|
|
# `__isoc23_strtoll` (glibc 2.38+) that the same image's runtime libc
|
|
# couldn't resolve. Building inside manylinux_2_28 (AlmaLinux 8, glibc
|
|
# 2.28 baseline) yields a wheel portable to any glibc 2.28+ runtime.
|
|
FROM quay.io/pypa/manylinux_2_28_x86_64 AS builder
|
|
|
|
# No OpenSSL system deps required (rustls-everywhere refactor). See
|
|
# e2e/wrap/Dockerfile for the full rationale.
|
|
|
|
ENV CARGO_HOME=/usr/local/cargo \
|
|
RUSTUP_HOME=/usr/local/rustup \
|
|
PATH=/usr/local/cargo/bin:/opt/python/cp311-cp311/bin:${PATH}
|
|
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
|
|
| sh -s -- -y --no-modify-path --profile minimal -c rustfmt -c clippy --default-toolchain 1.95.0
|
|
|
|
WORKDIR /build
|
|
|
|
COPY pyproject.toml uv.lock README.md ./
|
|
COPY Cargo.toml Cargo.lock rust-toolchain.toml ./
|
|
COPY crates/ crates/
|
|
COPY headroom/ headroom/
|
|
|
|
ENV PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1
|
|
# Build for Python 3.11 to match Stage 2's `python:3.11-slim` runtime.
|
|
RUN /opt/python/cp311-cp311/bin/pip install 'maturin>=1.5,<2.0' && \
|
|
/opt/python/cp311-cp311/bin/maturin build --release --out /dist --interpreter python3.11
|
|
|
|
# ─── Stage 2: python runtime ───────────────────────────────────────────────
|
|
# `python:3.11-slim` (now trixie, glibc 2.41) — see e2e/wrap/Dockerfile
|
|
# for why we need glibc ≥ 2.38 (the wheel's `_core.so` references C23
|
|
# symbols that bookworm's 2.36 doesn't export). The init e2e harness
|
|
# only needs python (no node) — `headroom init -g <target>` is Python.
|
|
FROM python:3.11-slim
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive \
|
|
PATH="/opt/headroom-venv/bin:${PATH}" \
|
|
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
|
PIP_NO_CACHE_DIR=1 \
|
|
PYTHONUNBUFFERED=1 \
|
|
PYTHONDONTWRITEBYTECODE=1 \
|
|
# Single-wheel refactor side effect: `headroom` is now installed
|
|
# from a wheel into site-packages, so `__file__.parents[2]` no
|
|
# longer points at the repo root (it points at site-packages).
|
|
# `_marketplace_source()` falls back to `chopratejas/headroom`
|
|
# (the GitHub remote) instead of the local `/workspace` path the
|
|
# `seq_claude_local` e2e assertion expects. Pin the source via
|
|
# the existing override env var so the local `/workspace`
|
|
# marketplace.json (COPY'd below) is used.
|
|
HEADROOM_MARKETPLACE_SOURCE=/workspace
|
|
|
|
RUN apt-get update && \
|
|
apt-get install -y --no-install-recommends \
|
|
ca-certificates \
|
|
git && \
|
|
rm -rf /var/lib/apt/lists/*
|
|
|
|
WORKDIR /workspace
|
|
|
|
COPY --from=builder /dist/*.whl /tmp/wheels/
|
|
|
|
# Init e2e harness imports from e2e._lib + .claude-plugin assets.
|
|
# DO NOT copy `headroom/` or `pyproject.toml` from the workspace —
|
|
# the installed wheel must be authoritative. The harness packages
|
|
# (e2e/_lib/, .claude-plugin/, plugins/) don't shadow `headroom`.
|
|
COPY .claude-plugin ./.claude-plugin
|
|
COPY .github/plugin ./.github/plugin
|
|
COPY plugins/headroom-agent-hooks ./plugins/headroom-agent-hooks
|
|
COPY e2e/__init__.py ./e2e/__init__.py
|
|
COPY e2e/_lib ./e2e/_lib
|
|
COPY e2e/init ./e2e/init
|
|
|
|
RUN python -m venv /opt/headroom-venv && \
|
|
/opt/headroom-venv/bin/python -m pip install --upgrade pip && \
|
|
/opt/headroom-venv/bin/python -m pip install "$(ls /tmp/wheels/headroom_ai-*.whl)[proxy]" && \
|
|
/opt/headroom-venv/bin/python -c "from headroom._core import DiffCompressor; print('headroom._core OK')"
|
|
|
|
CMD ["python", "e2e/init/run.py"]
|