headroom/Dockerfile
chopratejas 6f2c0a8400 fix(ci): rustls-everywhere — eliminate openssl-sys from build tree
# Root cause of the wheel-build cascade

We have shipped 5 release-pipeline hot-fixes in 12 hours, each
addressing a different symptom of the same architectural problem:

1. PR #363 — npm artifact downloads + tried `yum openssl-devel`
2. PR #367 — vendored OpenSSL in `headroom-proxy` + dropped Intel mac
3. PR #369 — Debian-cross perl install (`perl` not `libipc-cmd-perl`)
4. PR #370 — moved `openssl/vendored` from headroom-proxy to headroom-py
5. (this PR) — ELIMINATE OpenSSL entirely

Each fix exposed a different missing system package or feature flag in
a different build surface (manylinux x86_64 vs aarch64-cross-Debian vs
macOS Intel vs e2e/wrap Dockerfile vs e2e/init Dockerfile vs main
Dockerfile vs devcontainer). We were playing whack-a-mole because every
Cargo dep change to the OpenSSL surface required matching system-package
updates in 6+ different Dockerfiles and workflows, and the PR-level CI
didn't exercise all of them.

# Why this PR is the structural fix

`fastembed` exposes clean rustls feature flags:
- `hf-hub-rustls-tls`               (replaces default `hf-hub-native-tls`)
- `ort-download-binaries-rustls-tls` (replaces default `…native-tls`)

By disabling fastembed's default features and enabling the rustls
variants explicitly, we remove `native-tls` (and therefore `openssl-sys`,
`openssl`, `openssl-src`, perl modules, OpenSSL build-time deps,
vendored OpenSSL ~30s build cost) from the entire workspace dep tree.

Verified locally:

    $ cargo tree -p headroom-py -i openssl-sys
    error: package ID specification `openssl-sys` did not match any packages

    $ cargo tree -p headroom-py -i native-tls
    error: package ID specification `native-tls` did not match any packages

    $ cargo build --release -p headroom-py
    Finished `release` profile [optimized] target(s) in 25.57s

(Down from 1m+ with vendored OpenSSL.)

# Cleanups enabled by this change

- crates/headroom-py/Cargo.toml — dropped the `openssl/vendored`
  workaround from PR #370.
- crates/headroom-proxy/Cargo.toml — same dep removed.
- e2e/wrap/Dockerfile — dropped `yum install openssl-devel pkgconfig
  perl-IPC-Cmd`. Comment retained explaining why.
- e2e/init/Dockerfile — same.
- Dockerfile (main) — dropped `pkg-config libssl-dev` from apt-get.
- .devcontainer/Dockerfile — dropped `pkg-config libssl-dev`.
- .github/workflows/release.yml — removed the entire before-script-linux
  block (perl install probe + multi-package-manager dispatch + fail-loud
  assertion). No longer needed.

# Regression gate

Three new structural tests in tests/test_release_workflows.py:

- test_no_openssl_sys_in_wheel_build_tree — runs `cargo tree -p <crate>
  -i openssl-sys` for headroom-py / headroom-proxy / headroom-core. If
  openssl-sys reappears (a future native-tls enabler creeping in via a
  new dep), this fails AT PR TIME with an actionable message.
- test_no_native_tls_in_wheel_build_tree — same shape, native-tls is
  the proximate cause.
- test_fastembed_uses_rustls_features — checks the Cargo.toml so a
  future "let me bump fastembed and forget the features" doesn't
  silently re-introduce OpenSSL.

Plus two cleanup gates:
- test_dockerfiles_no_longer_install_openssl_devel
- test_release_yml_does_not_install_openssl_or_perl_for_wheels

All 13 release-workflow tests pass. `make ci-precheck` PASSED.

# What this teaches us about rollouts (per user's ultrathink ask)

The 5-fix cascade exposed three meta-problems:

1. PR checks don't block merges. PR #370 had docker-init-e2e,
   docker-wrap-e2e, docker-native-e2e all FAILED yet got merged.
   Branch protection should require these checks. Operator action
   needed (cannot fix in code).

2. Local validation is misleading. `cargo build -p headroom-py` from
   the workspace root used the workspace lockfile and looked green;
   CI did fresh resolution against headroom-py's manifest alone where
   the feature wasn't enabled. Lesson: verify structural invariants
   with `cargo tree -e features` before trusting that a build "works."

3. 6+ build surfaces with independent system-dep state. Every Cargo
   change required matching updates in 6 places. The structural answer
   (this PR) is to NOT depend on system OpenSSL at all. Where structural
   fixes are not possible, the answer is a single shared
   scripts/install-rust-build-deps.sh — but with this PR there's
   nothing left to install.
2026-05-03 23:26:04 -07:00

129 lines
4.8 KiB
Docker

ARG PYTHON_VERSION=3.11
ARG UV_VERSION=0.6.17
# Pinned 2026-04-15. Update via Dependabot or: docker pull python:3.11-slim
ARG PYTHON_DIGEST=sha256:233de06753d30d120b1a3ce359d8d3be8bda78524cd8f520c99883bfe33964cf
# Pinned 2026-04-15. Update via Dependabot or: docker pull gcr.io/distroless/python3-debian13
ARG DISTROLESS_DIGEST=sha256:ed3a4beb46f8f8baac068743ba1b1f95ea3f793422129cf6dd23967f779b6018
ARG DISTROLESS_IMAGE=gcr.io/distroless/python3-debian13
ARG PYTHON_SITE_PACKAGES=/usr/local/lib/python${PYTHON_VERSION}/site-packages
# ---- Build stage: compile native extensions, build wheel ----
FROM python:${PYTHON_VERSION}-slim@${PYTHON_DIGEST} AS builder
ARG UV_VERSION
# build-essential / g++ for any C extension wheels uv may need to build
# from source. curl + ca-certificates are required by the rustup
# bootstrap below. patchelf for maturin's wheel-link repair on linux.
# No OpenSSL system deps required: the rustls-everywhere refactor
# eliminated `openssl-sys` from our build tree by switching fastembed
# to `hf-hub-rustls-tls` + `ort-download-binaries-rustls-tls`.
RUN apt-get update && \
apt-get install -y --no-install-recommends \
build-essential \
g++ \
curl \
ca-certificates \
patchelf \
&& rm -rf /var/lib/apt/lists/*
RUN python -m pip install --no-cache-dir uv==${UV_VERSION}
# Rust toolchain for the headroom._core extension. With single-wheel
# architecture (post-#355), `pip install -e .` invokes maturin via
# pyproject.toml's [build-system], which calls cargo. No more separate
# headroom-core-py package.
ENV CARGO_HOME=/usr/local/cargo \
RUSTUP_HOME=/usr/local/rustup \
PATH=/usr/local/cargo/bin:${PATH}
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
| sh -s -- -y --no-modify-path --profile minimal -c rustfmt -c clippy --default-toolchain 1.95.0
WORKDIR /build
# Copy the full set of files maturin needs to build the wheel: the root
# pyproject.toml + Cargo workspace + Rust crates + Python source. The
# uv install builds + installs the wheel in one shot.
COPY pyproject.toml uv.lock README.md ./
COPY Cargo.toml Cargo.lock rust-toolchain.toml ./
COPY crates/ crates/
COPY headroom/ headroom/
ARG HEADROOM_EXTRAS=proxy,code
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=cache,target=/root/.cargo/registry \
--mount=type=cache,target=/build/target \
uv pip install --system ".[${HEADROOM_EXTRAS}]"
# Build-stage smoke check: verify the extension loads end-to-end inside
# the build image before we copy site-packages into the runtime image.
# If this fails, the runtime image would fail Phase A0's fail-loud
# startup check on every restart. Run from /tmp so cwd doesn't shadow
# site-packages with /build/headroom/ (which has no _core.so since
# maturin installed the .so into site-packages).
RUN cd /tmp && python -c "from headroom._core import DiffCompressor, SmartCrusher; \
print(f'build-stage rust core verify OK: {DiffCompressor.__name__}, {SmartCrusher.__name__}')"
# ---- Runtime stage (python-slim): supports root/nonroot via build arg ----
FROM python:${PYTHON_VERSION}-slim@${PYTHON_DIGEST} AS runtime-slim-base
ARG RUNTIME_USER=nonroot
ARG PYTHON_SITE_PACKAGES
RUN apt-get update && \
apt-get install -y --no-install-recommends curl && \
rm -rf /var/lib/apt/lists/*
COPY --from=builder ${PYTHON_SITE_PACKAGES} ${PYTHON_SITE_PACKAGES}
COPY --from=builder /usr/local/bin/headroom /usr/local/bin/headroom
RUN mkdir -p /home/nonroot /data && \
if [ "$RUNTIME_USER" = "nonroot" ]; then \
groupadd --gid 1000 nonroot && \
useradd --uid 1000 --gid nonroot --create-home nonroot && \
mkdir -p /home/nonroot/.headroom && \
chown -R nonroot:nonroot /data /home/nonroot; \
else \
mkdir -p /root/.headroom; \
fi
USER ${RUNTIME_USER}
WORKDIR /home/nonroot
ENV HEADROOM_HOST=0.0.0.0 \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
EXPOSE 8787
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
CMD ["curl", "--fail", "--silent", "http://127.0.0.1:8787/readyz"]
ENTRYPOINT ["headroom", "proxy"]
CMD ["--host", "0.0.0.0", "--port", "8787"]
FROM ${DISTROLESS_IMAGE}@${DISTROLESS_DIGEST} AS runtime-slim
ARG RUNTIME_USER=nonroot
ARG PYTHON_SITE_PACKAGES
COPY --from=builder ${PYTHON_SITE_PACKAGES} ${PYTHON_SITE_PACKAGES}
USER ${RUNTIME_USER}
WORKDIR /app
ENV HEADROOM_HOST=0.0.0.0 \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONPATH=${PYTHON_SITE_PACKAGES}
EXPOSE 8787
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
CMD ["python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8787/readyz', timeout=5)"]
ENTRYPOINT ["python3", "-m", "headroom.cli", "proxy"]
CMD ["--host", "0.0.0.0", "--port", "8787"]
# Default published image remains python-slim runtime
FROM runtime-slim-base AS runtime