From 5da978b131a853aa5c20c6a36de699ed4503a073 Mon Sep 17 00:00:00 2001 From: Amos Date: Thu, 30 Jul 2026 14:09:11 +0100 Subject: [PATCH] ci: make arm64 best-effort, add :edge, keep the build cache warm MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three separate problems with the pipeline, all surfaced by shipping the iw4x launcher fixes: 1. arm64 blocked every release. It has no upstream iw4x-launcher binary so it compiles from source, and it currently fails on upstream's stale pregenerated ODB sources (guard on ODB_VERSION 20551 / odb 2.6.0-b.51 while their own manifest asks for libodb >= 2.6.0- and odb 2.6.0 final is now published). That is not pinnable from here: bpkg reads the manifests from the remote repo, so patching a clone changes nothing. Mark arm64 optional + continue-on-error so amd64 still publishes, and have merge warn loudly instead of silently shipping a partial manifest. amd64 stays required — no amd64 digest, no publish. 2. Nothing was testable until arm64 finished. Publish :edge straight from the amd64 job. Single-platform on purpose, documented as such. 3. Cold builds kept recurring. The GHA cache evicts after 7 idle days, so any gap longer than a fortnight meant recompiling the build2 toolchain (~20 min vs ~3 min). A weekly cron keeps it alive. merge now downloads digests without merge-multiple so one subdirectory per platform survives, which is what lets it detect coverage; the manifest is assembled with find rather than a cwd glob to match. Docs: correct the stale Tags section (v2-layout branch is long gone, main publishes both :latest and :v2), document the arm64 situation and the upstream root cause, and note in the README that IW4x has no bundled config seed — the container refuses to start until you supply one, so point at iw4x/iw4-server-configs and flag its commented-out sv_maprotation. Also record which iw4x files the launcher owns, so nobody stages them in the read-only gamefiles mount. --- .github/workflows/docker-publish.yml | 64 +++++++++++++++++++++++++--- CLAUDE.md | 31 +++++++++++--- README.md | 19 ++++++++- 3 files changed, 101 insertions(+), 13 deletions(-) diff --git a/.github/workflows/docker-publish.yml b/.github/workflows/docker-publish.yml index 3a9c9e8..03aba33 100644 --- a/.github/workflows/docker-publish.yml +++ b/.github/workflows/docker-publish.yml @@ -7,14 +7,23 @@ name: Build and Push Docker Image # 1. `build` matrix runs each platform on its native runner in parallel. # Pushes a single-platform image by digest only (no human tag). # Saves the digest as a workflow artifact. -# 2. `merge` downloads both digest artifacts, generates the tag list once, -# and uses `docker buildx imagetools create` to stitch the per-platform -# digests into a single multi-arch manifest published as :latest, :v2, +# 2. `merge` downloads whichever digest artifacts exist, generates the tag +# list once, and uses `docker buildx imagetools create` to stitch the +# per-platform digests into a single manifest published as :latest, :v2, # and any release/PR/sha-derived tags. # # Why: native arm64 runners avoid the 60-90 min QEMU emulation cost while # still producing a standard multi-arch manifest so users pull :latest and # get the right platform automatically. +# +# arm64 is best-effort (`optional: true` + `continue-on-error`). It has no +# upstream iw4x-launcher binary so it compiles from source, which leaves it +# exposed to upstream build breakage we cannot pin around. When it fails, +# amd64 still publishes and `merge` emits a warning rather than blocking the +# release. amd64 is required: no amd64 digest means no publish at all. +# +# :edge is published straight from the amd64 job for fast iteration, so +# testing never waits on arm64. It is single-platform by design. on: push: @@ -27,6 +36,11 @@ on: - 'EXAMPLE-docker-compose.yml' release: types: [published] + # The GHA cache evicts after 7 idle days, and a cold arm64 job has to compile + # the build2 toolchain and the launcher from source (~20 min vs ~3 min warm). + # A weekly run keeps the cache alive so it never goes cold in practice. + schedule: + - cron: '0 4 * * 1' workflow_dispatch: env: @@ -46,11 +60,18 @@ jobs: runs-on: ubuntu-latest dockerfile: Dockerfile arch: amd64 + optional: false - platform: linux/arm64 runs-on: ubuntu-24.04-arm dockerfile: Dockerfile.arm64 arch: arm64 + # Best-effort: arm64 has no upstream iw4x-launcher binary, so it + # compiles from source and is exposed to upstream build breakage + # we cannot pin around (see README "Architecture support"). A + # failure here must not block publishing the amd64 image. + optional: true runs-on: ${{ matrix.runs-on }} + continue-on-error: ${{ matrix.optional }} permissions: contents: read @@ -86,6 +107,17 @@ jobs: provenance: false sbom: false + # Publish a tagged amd64-only image straight from this job so iterating + # never waits on the (much slower) arm64 build. :edge is deliberately + # single-platform — pulling it on an arm64 host fails with a platform + # mismatch. :latest and :v2 remain the multi-arch tags. + - name: Tag amd64-only :edge for fast iteration + if: matrix.arch == 'amd64' + run: | + docker buildx imagetools create \ + -t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:edge \ + ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build.outputs.digest }} + - name: Export digest run: | mkdir -p /tmp/digests @@ -110,12 +142,31 @@ jobs: packages: write steps: + # Not merge-multiple: keeping one subdirectory per artifact is what lets + # the next step tell which platforms actually built. - name: Download digests uses: actions/download-artifact@v4 with: path: /tmp/digests pattern: digests-* - merge-multiple: true + + - name: Check platform coverage + run: | + if [ ! -d /tmp/digests/digests-amd64 ]; then + echo "::error::amd64 digest missing — refusing to publish." + exit 1 + fi + if [ -d /tmp/digests/digests-arm64 ]; then + echo "Publishing multi-arch: amd64 + arm64." + echo "Published **amd64 + arm64**." >> "$GITHUB_STEP_SUMMARY" + else + echo "::warning::arm64 unavailable — :latest/:v2 will be amd64-only. See README 'Architecture support'." + { + echo "### :warning: amd64-only publish" + echo "The arm64 build did not produce a digest, so \`:latest\`/\`:v2\` carry amd64 only." + echo "Pulling them on an arm64 host will fail with a platform mismatch." + } >> "$GITHUB_STEP_SUMMARY" + fi - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 @@ -139,11 +190,12 @@ jobs: type=ref,event=pr type=sha + # Digests now live one level down (/tmp/digests/digests-/), + # so collect them with find rather than a cwd glob. - name: Create manifest list and push - working-directory: /tmp/digests run: | docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ - $(printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:%s ' *) + $(find /tmp/digests -type f -printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:%f ') - name: Inspect image run: | diff --git a/CLAUDE.md b/CLAUDE.md index 0517cc9..ac31c83 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -29,13 +29,32 @@ There are no automated tests or linters. The CI pipeline (`.github/workflows/doc ## Tags -- `ghcr.io/ayymoss/plutainer:v2` — built from `v2-layout` branch. New volume layout + unified `PLUTAINER_*` env vars. Opt-in. CI workflow tags it only on pushes to `v2-layout`; never promotes to `:latest`. -- `ghcr.io/ayymoss/plutainer:latest` — built from `main`. Deprecated v1 layout. No further v2 work merges here; bug-only updates if any. +The `v2-layout` branch is gone — v2 was merged into `main`, so **`main` is v2** and publishes both tags. Anything describing `:latest` as "the deprecated v1 layout" is stale. -CI logic in `.github/workflows/docker-publish.yml`: -- `type=raw,value=latest,enable={{is_default_branch}}` — only on main. -- `type=raw,value=v2,enable=${{ github.ref == 'refs/heads/v2-layout' }}` — only on v2-layout. -- Both branches also get `:sha-`. Branches stay completely separated. +- `ghcr.io/ayymoss/plutainer:latest` and `:v2` — both built from `main`, both multi-arch, identical content. `:v2` is kept so existing v2 users' compose files keep working. +- `ghcr.io/ayymoss/plutainer:edge` — **amd64 only**, published directly from the amd64 build job so it lands in ~3 min without waiting on arm64. For iterative testing. Pulling it on arm64 fails with a platform mismatch. +- `:sha-` on every build; `:` on releases; `:pr-` on PRs. + +Tag logic (`metadata-action` in the `merge` job): +- `type=raw,value=latest,enable={{is_default_branch}}` +- `type=raw,value=v2,enable={{is_default_branch}}` +- `type=ref,event=tag`, `type=ref,event=pr`, `type=sha` + +## Architecture support + +amd64 is required; **arm64 is best-effort** (`optional: true` + `continue-on-error` in the build matrix). If arm64 fails, amd64 still publishes and `merge` emits a `::warning::` plus a job-summary note; only a missing *amd64* digest blocks a publish. + +The asymmetry is upstream's: `iw4x/launcher` ships release binaries for `x86_64-linux` and `x86_64-windows` only, so `Dockerfile` grabs the prebuilt binary while `Dockerfile.arm64` compiles it (and the whole build2 toolchain) from source. + +**Known arm64 breakage.** The source build currently fails with: + +``` +launcher/pregenerated/launcher/cache/cache-types-odb.hxx:13:2: error: #error ODB runtime version mismatch +``` + +The launcher's checked-in *pregenerated* ODB sources guard on `ODB_VERSION != 20551UL` (odb 2.6.0-b.51), but its `manifest` declares `depends: libodb >= 2.6.0-` and its `repositories.manifest` takes odb from `codesynthesis-com/odb.git#master`. odb 2.6.0 final has since been published, so build2 correctly resolves 2.6.0 (`20600`) and the stale generated sources reject it. + +Not pinnable from our side: `bpkg` fetches the manifests from the *remote* repo, so patching a local clone has no effect, and the constraint lives in upstream's own manifest. It needs upstream to regenerate their ODB sources (or pin odb). Do not "fix" this by rewriting the version guard — the generated code may not be ABI-compatible with 2.6.0. ## Architecture diff --git a/README.md b/README.md index 017fe79..32d34d9 100644 --- a/README.md +++ b/README.md @@ -5,11 +5,14 @@ This repository contains the necessary files to build and run dedicated game ser The container is available on GitHub Container Registry: `ghcr.io/ayymoss/plutainer:latest` > **Tag layout:** -> - `:latest` (and the `:v2` alias) — current Plutainer v2. New volume layout, unified `PLUTAINER_*` environment variables. Built from `main`. +> - `:latest` (and the `:v2` alias) — current Plutainer v2. New volume layout, unified `PLUTAINER_*` environment variables. Built from `main`. Multi-arch. +> - `:edge` — **amd64 only.** Published straight from the amd64 build job so it appears within a few minutes instead of waiting on the slower arm64 build. Useful for testing a fresh commit; pulling it on an arm64 host fails with a platform mismatch. > - `:v1-final` (and the `:v1` alias) — frozen snapshot of the old v1 image. Legacy `PLUTO_*`/`IW4X_*`/`ALTER_*` env vars, flat `app/gamefiles/` + `app/plutonium/` layout. Shows a deprecation banner on every start. No further updates, fixes, or security patches. > > **Upgrading from v1?** See [MIGRATION.md](MIGRATION.md) — covers the env var rename, the volume migration command (one `docker run`), and how to pin `:v1-final` if you want to defer the migration. +> **Architecture support:** `linux/amd64` is the primary target. `linux/arm64` is built best-effort — upstream `iw4x/launcher` publishes `x86_64` binaries only, so the arm64 image compiles it from source and is occasionally broken by upstream changes. When that happens `:latest`/`:v2` are published **amd64-only** rather than being held back, and the build log carries a warning. If you run arm64, check `docker manifest inspect ghcr.io/ayymoss/plutainer:latest` before upgrading. + ## Overview The primary goal of this Docker image is to simplify the setup and management of dedicated servers for the following games: @@ -31,6 +34,10 @@ Before you can use this Docker image, you will need to have the base game files You will also need to have Docker and Docker Compose installed on your system. +The gamefiles mount is read-only and should contain only the base game's own files. Anything the updaters can fetch is written into the `app/` volume instead, so don't stage it in the mount. + +For **IW4x** specifically, the mount needs just the stock MW2 install — `main/`, `zone/english/`, `zone/dlc/`, `binkw32.dll`, `localization.txt`, `mss32.dll`. The `iw4x-launcher` fetches everything else into `app/runtime/gamefiles/` on first start (roughly 1–2 GB, kept across container recreation): `iw4x.exe`, `iw4x.dll`, `zonebuilder.exe`, the `iw4x/` asset directory including its `.iwd` archives, all of `zone/patch/` and `zone/zonebuilder/`, and the extra DLC fastfiles. `zone/patch/` and `zone/zonebuilder/` are owned entirely by the launcher — copies of those in your gamefiles mount are ignored, so a slimmed-down server install is fine. Client-only assets (`main/video/`, `logo.bmp`, `splash.bmp`) are never used and can be removed; [`mxve/shrink-iw4x`](https://github.com/mxve/shrink-iw4x) does this properly and ships a Linux binary. + ## Getting Started: `docker-compose.yml` Instead of using a long `docker run` command, it is highly recommended to use `docker-compose` to manage your server. See [EXAMPLE-docker-compose.yml](EXAMPLE-docker-compose.yml) for complete examples. @@ -171,6 +178,16 @@ Top-level `*.cfg` files from each seed bundle land in `app/configs/` (flat). Oth | Plutonium IW5 | [xerxes-at/IW5ServerConfig](https://github.com/xerxes-at/IW5ServerConfig) | | Alterware T7x | [Dss0/t7-server-config](https://github.com/Dss0/t7-server-config) (includes `t7x/lobby_scripts/` required for `sv_lobby_mode`) | +> **IW4x is not in this table — it has no bundled seed.** Nothing is copied into `app/configs/` for `PLUTAINER_GAME=iw4x`, so the container will refuse to start with a "config file not found" error until you put one there yourself. The upstream default is [`iw4x/iw4-server-configs`](https://github.com/iw4x/iw4-server-configs); grab `userraw/server.cfg` from it into `app/configs/` and set `PLUTAINER_CONFIG_FILE=server.cfg`: +> +> ```bash +> mkdir -p ./iw4x-1/configs +> curl -fsSL -o ./iw4x-1/configs/server.cfg \ +> https://raw.githubusercontent.com/iw4x/iw4-server-configs/main/userraw/server.cfg +> ``` +> +> Note that config ships `sv_maprotation` commented out, so `+map_rotate` has nothing to load — set one, or set `PLUTAINER_MAP_ROTATE=false` and drive map selection from a playlist. Remember to set `rcon_password` too, since the healthcheck and `rcon-cli` both read it. + To opt out — for example if you manage configs entirely yourself and don't want any default files appearing in your bind mount — set `PLUTAINER_SKIP_SEED=true`. The seed snapshot is frozen at image build time. Pulling a newer image only seeds files that don't yet exist in your bind mount, so the upstream repo never silently overwrites your edits.