ci: prebuild the Arch environment as a container image

All six nightly jobs ran the same pacman transaction before doing any work.
The toolchain now lives in an image published to the Gitea registry as
git.bjphoster.com/aur/autoupdate:latest, rebuilt monthly since Arch is rolling
and a month-old image is a month behind.

The build job itself runs on plain ubuntu-latest -- it only drives docker. Only
the image is Arch.

Based on archlinux:base rather than base-devel: the autoupdater parses
PKGBUILDs and never compiles, so the toolchain is dead weight, 822 MB against
1.48 GB. nodejs is included because actions/checkout is a JavaScript action the
runner executes with node from inside the image.

Slimming to base initially dropped diffutils, and apply_pkgrel_rule used
diff -q. A missing binary makes the `if` fail, which reads as "changed", so
pkgrel ratcheted upward on every run -- the same silent-wrong-default class as
the empty .SRCINFO this rewrite was about, and invisible to the host test run.
The comparison is now a shell string equality with no external dependency, and
is verified against an image with no diff at all. require_tools() additionally
fails on startup for any missing binary rather than midway through.
This commit is contained in:
2026-07-26 23:17:37 +02:00
parent 4fecc4bd2c
commit 4defd478dc
7 changed files with 199 additions and 18 deletions

4
.dockerignore Normal file
View File

@@ -0,0 +1,4 @@
# The image is a toolchain only -- the dockerfile COPYs nothing, and the
# scripts arrive via actions/checkout at run time. Keep the build context empty.
*
!dockerfile

View File

@@ -0,0 +1,67 @@
---
name: Build Autoupdater Image
concurrency:
group: build-image
cancel-in-progress: false
on:
schedule:
# Monthly. Arch is a rolling release, so a month-old image is a month
# behind on packages; :latest is overwritten each time.
- cron: "0 3 1 * *"
push:
branches:
- main
paths:
- dockerfile
- .dockerignore
- .gitea/workflows/build-image.yaml
workflow_dispatch:
defaults:
run:
shell: bash
jobs:
build:
# No Arch needed here -- this only drives docker. The *image* is Arch.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Log in to the Gitea container registry
env:
GIT_HOST: ${{ vars.GIT_HOST }}
REGISTRY_USER: ${{ vars.REGISTRY_USER }}
GIT_TOKEN: ${{ secrets.GIT_TOKEN }}
run: |
set -euo pipefail
# NB: this is the Gitea *username*, not vars.GIT_NAME -- that one is
# the display name used for git commit authorship and contains spaces.
printf '%s' "${GIT_TOKEN}" \
| docker login "${GIT_HOST}" \
--username "${REGISTRY_USER}" \
--password-stdin
- name: Build and push
env:
GIT_HOST: ${{ vars.GIT_HOST }}
run: |
set -euo pipefail
# The repo is `.autoupdate`, but OCI names cannot start with a dot.
image="${GIT_HOST}/aur/autoupdate"
# --pull so a monthly rebuild actually picks up a new archlinux base
# rather than reusing a cached one.
docker build --pull --tag "${image}:latest" .
docker push "${image}:latest"
echo "pushed ${image}:latest"
- name: Log out
if: always()
env:
GIT_HOST: ${{ vars.GIT_HOST }}
run: docker logout "${GIT_HOST}" || true

View File

@@ -37,8 +37,14 @@ jobs:
# emits an empty .SRCINFO, which the AUR then rejects -- that is what broke # emits an empty .SRCINFO, which the AUR then rejects -- that is what broke
# reflex-appimage, nethlink-appimage and pman-helper. regen_srcinfo() in # reflex-appimage, nethlink-appimage and pman-helper. regen_srcinfo() in
# common.sh fails loudly if this ever regresses. # common.sh fails loudly if this ever regresses.
#
# Prebuilt by build-image.yaml (see dockerfile) so the toolchain is not
# reinstalled in all six jobs every night.
container: container:
image: archlinux:base-devel image: git.bjphoster.com/aur/autoupdate:latest
credentials:
username: ${{ vars.REGISTRY_USER }}
password: ${{ secrets.GIT_TOKEN }}
strategy: strategy:
# One broken package must not cancel the rest of the matrix. # One broken package must not cancel the rest of the matrix.
@@ -53,15 +59,10 @@ jobs:
- tsparams - tsparams
steps: steps:
# Must come BEFORE actions/checkout. That action is JavaScript, and the # No bootstrap step: node, git, openssh, jq, curl and the makepkg
# archlinux image ships no node, so checkout dies with # toolchain all come from the prebuilt image above. node in particular
# `exec: "node": executable file not found in $PATH`. A `run:` step needs # has to be present before this action runs, since actions/checkout is
# no checkout, so the toolchain can be installed first. # JavaScript and the runner executes it with `node` from the image.
- name: Bootstrap container
run: |
pacman -Syu --noconfirm --needed \
nodejs git openssh jq curl pacman-contrib namcap fuse2 python
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Configure git and SSH - name: Configure git and SSH
@@ -91,11 +92,11 @@ jobs:
EOF EOF
chmod 600 ~/.ssh/config chmod 600 ~/.ssh/config
# makepkg refuses to run as root, and CI containers are root by default. # The `builder` user is baked into the image; this only hands it the
- name: Create build user # workspace and the credentials written by the previous step.
- name: Hand over to the build user
run: | run: |
set -euo pipefail set -euo pipefail
useradd --create-home --shell /bin/bash builder
chown -R builder:builder "${GITHUB_WORKSPACE}" chown -R builder:builder "${GITHUB_WORKSPACE}"
cp -r /root/.ssh /root/.gitconfig /root/.git-credentials /home/builder/ 2>/dev/null || true cp -r /root/.ssh /root/.gitconfig /root/.git-credentials /home/builder/ 2>/dev/null || true
chown -R builder:builder /home/builder chown -R builder:builder /home/builder

View File

@@ -103,9 +103,48 @@ The trap the previous implementation fell into was giving that same silent
Needs a genuine Arch `makepkg`. Ubuntu's build environment emits an empty Needs a genuine Arch `makepkg`. Ubuntu's build environment emits an empty
`.SRCINFO`, which the AUR rejects — this silently broke three packages. The CI `.SRCINFO`, which the AUR rejects — this silently broke three packages. The CI
job therefore runs in an `archlinux:base-devel` container as a non-root user job therefore runs in an Arch container as a non-root user (`makepkg` refuses to
(`makepkg` refuses to run as root), and `regen_srcinfo` in `common.sh` fails run as root), and `regen_srcinfo` in `common.sh` fails loudly if the output is
loudly if the output is ever empty again. ever empty again.
`autoupdate.sh` calls `require_tools` on startup, so a missing binary fails
immediately instead of taking a silently wrong branch further in.
## Container image
`dockerfile` builds the environment the nightly job runs in, published to the
Gitea registry as `git.bjphoster.com/aur/autoupdate:latest`. Without it, all six
matrix jobs would run a full `pacman` transaction every night.
`.gitea/workflows/build-image.yaml` rebuilds and overwrites `:latest` monthly
(Arch is rolling, so a month-old image is a month behind), on any push touching
the `dockerfile`, and on demand via `workflow_dispatch`. That job runs on plain
`ubuntu-latest` — it only drives `docker`; the *image* is the Arch part.
Based on `archlinux:base`, not `base-devel`: the autoupdater only ever parses
PKGBUILDs and never compiles, so the toolchain is dead weight — 822 MB against
1.48 GB. Add it back if a real build or verify step is ever introduced.
Building locally:
```sh
docker build -t git.bjphoster.com/aur/autoupdate:latest .
docker run --rm -v "$PWD:/src:ro" git.bjphoster.com/aur/autoupdate:latest \
bash -c 'cp -r /src /w && chown -R builder:builder /w && cd /w &&
runuser -u builder -- ./test.sh'
```
Running the suite inside the image is worth doing after any `dockerfile` change.
Trimming to `archlinux:base` originally dropped `diffutils`, and the missing
`diff` made every run look like a packaging change and ratchet `pkgrel` upward —
the host tests passed throughout. `pkgrel` no longer depends on `diff` at all,
but the lesson stands.
### Registry credentials
Needs `vars.REGISTRY_USER` — your Gitea **username**, not `vars.GIT_NAME`, which
is the display name used for commit authorship and contains spaces. The token is
the existing `secrets.GIT_TOKEN`, which needs `write:package` scope.
## Notes on checksums ## Notes on checksums

View File

@@ -24,6 +24,7 @@ _here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "${_here}/common.sh" source "${_here}/common.sh"
require_noninteractive require_noninteractive
require_tools
GIT_HOST="${GIT_HOST:-git.bjphoster.com}" GIT_HOST="${GIT_HOST:-git.bjphoster.com}"
# Where the package repos live, and how the upstream version is resolved. # Where the package repos live, and how the upstream version is resolved.
@@ -191,8 +192,15 @@ apply_pkgrel_rule() {
# packaging content. An edit to a local source lands here too, because its # packaging content. An edit to a local source lands here too, because its
# sha256sums entry was just regenerated. A README-only edit does not touch # sha256sums entry was just regenerated. A README-only edit does not touch
# the PKGBUILD and so correctly does not bump. # the PKGBUILD and so correctly does not bump.
if diff -q <(grep -v '^pkgrel=' PKGBUILD) \ #
<(grep -v '^pkgrel=' "$aur_pkgbuild") >/dev/null 2>&1; then # Deliberately a shell string comparison rather than diff(1): diff lives in
# diffutils, which archlinux:base does not ship, and a missing binary makes
# the `if` fail, which reads as "changed" and ratchets pkgrel upward on every
# single run. Same silent-wrong-default class of bug as the empty .SRCINFO.
local local_body aur_body
local_body="$(grep -v '^pkgrel=' PKGBUILD)"
aur_body="$(grep -v '^pkgrel=' "$aur_pkgbuild")"
if [ "$local_body" = "$aur_body" ]; then
return 0 # identical packaging; nothing to do return 0 # identical packaging; nothing to do
fi fi

View File

@@ -17,6 +17,19 @@ die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; }
# The old push.sh fell back to an interactive `read` when no version was passed. # The old push.sh fell back to an interactive `read` when no version was passed.
# In CI that reads EOF and yields an empty version, which is how pman-helper # In CI that reads EOF and yields an empty version, which is how pman-helper
# committed pkgver="" and the sha256 of a zero-byte download. # committed pkgver="" and the sha256 of a zero-byte download.
# Fail up front on a missing tool rather than midway through, where an absent
# binary tends to be read as "condition false" and silently takes the wrong
# branch. `diff` going missing in a slimmer base image did exactly that: it made
# every run look like a packaging change and ratcheted pkgrel upward.
require_tools() {
local t missing=()
for t in git curl jq sha256sum awk sed grep makepkg; do
command -v "$t" >/dev/null 2>&1 || missing+=("$t")
done
[ "${#missing[@]}" -eq 0 ] || die "missing required tool(s): ${missing[*]}"
return 0
}
require_noninteractive() { require_noninteractive() {
[ -t 0 ] && die "refusing to prompt: this script must never depend on interactive input" [ -t 0 ] && die "refusing to prompt: this script must never depend on interactive input"
return 0 return 0

49
dockerfile Normal file
View File

@@ -0,0 +1,49 @@
# Build environment for the AUR autoupdater.
#
# Rebuilt monthly by .gitea/workflows/build-image.yaml and pushed as :latest, so
# the six nightly package jobs start with the toolchain already in place instead
# of each running a full pacman transaction first.
#
# Arch specifically: makepkg --printsrcinfo must be the genuine Arch tool.
# Ubuntu's build environment silently emits an empty .SRCINFO, which the AUR
# then rejects -- that is what broke three packages before this was rewritten.
# base, not base-devel: the autoupdater only ever *parses* PKGBUILDs
# (makepkg --printsrcinfo) and never compiles anything, so the toolchain in
# base-devel is dead weight -- 822 MB against 1.48 GB. If a real build or
# verify step is ever added here, this needs to go back to base-devel.
FROM archlinux:base
LABEL org.opencontainers.image.title="aur-autoupdate"
LABEL org.opencontainers.image.description="Arch build environment for the AUR package autoupdater"
LABEL org.opencontainers.image.source="https://git.bjphoster.com/aur/.autoupdate"
# nodejs actions/checkout is a JavaScript action; the runner executes it
# with `node` from inside this image, and base-devel has none.
# pacman-contrib updpkgsums/paccache and friends
# namcap PKGBUILD linting
# fuse2 AppImage extraction
# python test.sh's local HTTP server
RUN pacman -Syu --noconfirm --needed \
nodejs \
git \
openssh \
jq \
curl \
pacman-contrib \
namcap \
fuse2 \
python \
diffutils \
&& rm -rf /var/cache/pacman/pkg/*
# NB: not `pacman -Scc` -- under --noconfirm it takes the default answer, which
# is *no* for the cache prompt, so it drops sync databases (breaking pacman -Ss
# for debugging) without actually reclaiming the cache. The rm does the real work.
# makepkg refuses to run as root and CI containers are root by default, so the
# nightly job drops to this user. Baked in here rather than created per run.
RUN useradd --create-home --shell /bin/bash builder
# Sanity check: fail the image build rather than the nightly job if makepkg
# ever stops working in this environment.
RUN runuser -u builder -- bash -c 'command -v makepkg && makepkg --version | head -1'