You've already forked .autoupdate
ci: prebuild the Arch environment as a container image
Some checks failed
Build Autoupdater Image / build (push) Failing after 6s
Some checks failed
Build Autoupdater Image / build (push) Failing after 6s
All six nightly jobs ran the same pacman transaction before doing any work. The toolchain now lives in an image published to the Gitea registry as git.bjphoster.com/aur/autoupdate:latest, rebuilt monthly since Arch is rolling and a month-old image is a month behind. The build job itself runs on plain ubuntu-latest -- it only drives docker. Only the image is Arch. Based on archlinux:base rather than base-devel: the autoupdater parses PKGBUILDs and never compiles, so the toolchain is dead weight, 822 MB against 1.48 GB. nodejs is included because actions/checkout is a JavaScript action the runner executes with node from inside the image. Slimming to base initially dropped diffutils, and apply_pkgrel_rule used diff -q. A missing binary makes the `if` fail, which reads as "changed", so pkgrel ratcheted upward on every run -- the same silent-wrong-default class as the empty .SRCINFO this rewrite was about, and invisible to the host test run. The comparison is now a shell string equality with no external dependency, and is verified against an image with no diff at all. require_tools() additionally fails on startup for any missing binary rather than midway through. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
4
.dockerignore
Normal file
4
.dockerignore
Normal file
@@ -0,0 +1,4 @@
|
||||
# The image is a toolchain only -- the dockerfile COPYs nothing, and the
|
||||
# scripts arrive via actions/checkout at run time. Keep the build context empty.
|
||||
*
|
||||
!dockerfile
|
||||
67
.gitea/workflows/build-image.yaml
Normal file
67
.gitea/workflows/build-image.yaml
Normal file
@@ -0,0 +1,67 @@
|
||||
---
|
||||
name: Build Autoupdater Image
|
||||
|
||||
concurrency:
|
||||
group: build-image
|
||||
cancel-in-progress: false
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Monthly. Arch is a rolling release, so a month-old image is a month
|
||||
# behind on packages; :latest is overwritten each time.
|
||||
- cron: "0 3 1 * *"
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- dockerfile
|
||||
- .dockerignore
|
||||
- .gitea/workflows/build-image.yaml
|
||||
workflow_dispatch:
|
||||
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
jobs:
|
||||
build:
|
||||
# No Arch needed here -- this only drives docker. The *image* is Arch.
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Log in to the Gitea container registry
|
||||
env:
|
||||
GIT_HOST: ${{ vars.GIT_HOST }}
|
||||
REGISTRY_USER: ${{ vars.REGISTRY_USER }}
|
||||
GIT_TOKEN: ${{ secrets.GIT_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# NB: this is the Gitea *username*, not vars.GIT_NAME -- that one is
|
||||
# the display name used for git commit authorship and contains spaces.
|
||||
printf '%s' "${GIT_TOKEN}" \
|
||||
| docker login "${GIT_HOST}" \
|
||||
--username "${REGISTRY_USER}" \
|
||||
--password-stdin
|
||||
|
||||
- name: Build and push
|
||||
env:
|
||||
GIT_HOST: ${{ vars.GIT_HOST }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# The repo is `.autoupdate`, but OCI names cannot start with a dot.
|
||||
image="${GIT_HOST}/aur/autoupdate"
|
||||
|
||||
# --pull so a monthly rebuild actually picks up a new archlinux base
|
||||
# rather than reusing a cached one.
|
||||
docker build --pull --tag "${image}:latest" .
|
||||
docker push "${image}:latest"
|
||||
|
||||
echo "pushed ${image}:latest"
|
||||
|
||||
- name: Log out
|
||||
if: always()
|
||||
env:
|
||||
GIT_HOST: ${{ vars.GIT_HOST }}
|
||||
run: docker logout "${GIT_HOST}" || true
|
||||
@@ -37,8 +37,14 @@ jobs:
|
||||
# emits an empty .SRCINFO, which the AUR then rejects -- that is what broke
|
||||
# reflex-appimage, nethlink-appimage and pman-helper. regen_srcinfo() in
|
||||
# common.sh fails loudly if this ever regresses.
|
||||
#
|
||||
# Prebuilt by build-image.yaml (see dockerfile) so the toolchain is not
|
||||
# reinstalled in all six jobs every night.
|
||||
container:
|
||||
image: archlinux:base-devel
|
||||
image: git.bjphoster.com/aur/autoupdate:latest
|
||||
credentials:
|
||||
username: ${{ vars.REGISTRY_USER }}
|
||||
password: ${{ secrets.GIT_TOKEN }}
|
||||
|
||||
strategy:
|
||||
# One broken package must not cancel the rest of the matrix.
|
||||
@@ -53,15 +59,10 @@ jobs:
|
||||
- tsparams
|
||||
|
||||
steps:
|
||||
# Must come BEFORE actions/checkout. That action is JavaScript, and the
|
||||
# archlinux image ships no node, so checkout dies with
|
||||
# `exec: "node": executable file not found in $PATH`. A `run:` step needs
|
||||
# no checkout, so the toolchain can be installed first.
|
||||
- name: Bootstrap container
|
||||
run: |
|
||||
pacman -Syu --noconfirm --needed \
|
||||
nodejs git openssh jq curl pacman-contrib namcap fuse2 python
|
||||
|
||||
# No bootstrap step: node, git, openssh, jq, curl and the makepkg
|
||||
# toolchain all come from the prebuilt image above. node in particular
|
||||
# has to be present before this action runs, since actions/checkout is
|
||||
# JavaScript and the runner executes it with `node` from the image.
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Configure git and SSH
|
||||
@@ -91,11 +92,11 @@ jobs:
|
||||
EOF
|
||||
chmod 600 ~/.ssh/config
|
||||
|
||||
# makepkg refuses to run as root, and CI containers are root by default.
|
||||
- name: Create build user
|
||||
# The `builder` user is baked into the image; this only hands it the
|
||||
# workspace and the credentials written by the previous step.
|
||||
- name: Hand over to the build user
|
||||
run: |
|
||||
set -euo pipefail
|
||||
useradd --create-home --shell /bin/bash builder
|
||||
chown -R builder:builder "${GITHUB_WORKSPACE}"
|
||||
cp -r /root/.ssh /root/.gitconfig /root/.git-credentials /home/builder/ 2>/dev/null || true
|
||||
chown -R builder:builder /home/builder
|
||||
|
||||
45
README.md
45
README.md
@@ -103,9 +103,48 @@ The trap the previous implementation fell into was giving that same silent
|
||||
|
||||
Needs a genuine Arch `makepkg`. Ubuntu's build environment emits an empty
|
||||
`.SRCINFO`, which the AUR rejects — this silently broke three packages. The CI
|
||||
job therefore runs in an `archlinux:base-devel` container as a non-root user
|
||||
(`makepkg` refuses to run as root), and `regen_srcinfo` in `common.sh` fails
|
||||
loudly if the output is ever empty again.
|
||||
job therefore runs in an Arch container as a non-root user (`makepkg` refuses to
|
||||
run as root), and `regen_srcinfo` in `common.sh` fails loudly if the output is
|
||||
ever empty again.
|
||||
|
||||
`autoupdate.sh` calls `require_tools` on startup, so a missing binary fails
|
||||
immediately instead of taking a silently wrong branch further in.
|
||||
|
||||
## Container image
|
||||
|
||||
`dockerfile` builds the environment the nightly job runs in, published to the
|
||||
Gitea registry as `git.bjphoster.com/aur/autoupdate:latest`. Without it, all six
|
||||
matrix jobs would run a full `pacman` transaction every night.
|
||||
|
||||
`.gitea/workflows/build-image.yaml` rebuilds and overwrites `:latest` monthly
|
||||
(Arch is rolling, so a month-old image is a month behind), on any push touching
|
||||
the `dockerfile`, and on demand via `workflow_dispatch`. That job runs on plain
|
||||
`ubuntu-latest` — it only drives `docker`; the *image* is the Arch part.
|
||||
|
||||
Based on `archlinux:base`, not `base-devel`: the autoupdater only ever parses
|
||||
PKGBUILDs and never compiles, so the toolchain is dead weight — 822 MB against
|
||||
1.48 GB. Add it back if a real build or verify step is ever introduced.
|
||||
|
||||
Building locally:
|
||||
|
||||
```sh
|
||||
docker build -t git.bjphoster.com/aur/autoupdate:latest .
|
||||
docker run --rm -v "$PWD:/src:ro" git.bjphoster.com/aur/autoupdate:latest \
|
||||
bash -c 'cp -r /src /w && chown -R builder:builder /w && cd /w &&
|
||||
runuser -u builder -- ./test.sh'
|
||||
```
|
||||
|
||||
Running the suite inside the image is worth doing after any `dockerfile` change.
|
||||
Trimming to `archlinux:base` originally dropped `diffutils`, and the missing
|
||||
`diff` made every run look like a packaging change and ratchet `pkgrel` upward —
|
||||
the host tests passed throughout. `pkgrel` no longer depends on `diff` at all,
|
||||
but the lesson stands.
|
||||
|
||||
### Registry credentials
|
||||
|
||||
Needs `vars.REGISTRY_USER` — your Gitea **username**, not `vars.GIT_NAME`, which
|
||||
is the display name used for commit authorship and contains spaces. The token is
|
||||
the existing `secrets.GIT_TOKEN`, which needs `write:package` scope.
|
||||
|
||||
## Notes on checksums
|
||||
|
||||
|
||||
@@ -24,6 +24,7 @@ _here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
source "${_here}/common.sh"
|
||||
|
||||
require_noninteractive
|
||||
require_tools
|
||||
|
||||
GIT_HOST="${GIT_HOST:-git.bjphoster.com}"
|
||||
# Where the package repos live, and how the upstream version is resolved.
|
||||
@@ -191,8 +192,15 @@ apply_pkgrel_rule() {
|
||||
# packaging content. An edit to a local source lands here too, because its
|
||||
# sha256sums entry was just regenerated. A README-only edit does not touch
|
||||
# the PKGBUILD and so correctly does not bump.
|
||||
if diff -q <(grep -v '^pkgrel=' PKGBUILD) \
|
||||
<(grep -v '^pkgrel=' "$aur_pkgbuild") >/dev/null 2>&1; then
|
||||
#
|
||||
# Deliberately a shell string comparison rather than diff(1): diff lives in
|
||||
# diffutils, which archlinux:base does not ship, and a missing binary makes
|
||||
# the `if` fail, which reads as "changed" and ratchets pkgrel upward on every
|
||||
# single run. Same silent-wrong-default class of bug as the empty .SRCINFO.
|
||||
local local_body aur_body
|
||||
local_body="$(grep -v '^pkgrel=' PKGBUILD)"
|
||||
aur_body="$(grep -v '^pkgrel=' "$aur_pkgbuild")"
|
||||
if [ "$local_body" = "$aur_body" ]; then
|
||||
return 0 # identical packaging; nothing to do
|
||||
fi
|
||||
|
||||
|
||||
13
common.sh
13
common.sh
@@ -17,6 +17,19 @@ die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; }
|
||||
# The old push.sh fell back to an interactive `read` when no version was passed.
|
||||
# In CI that reads EOF and yields an empty version, which is how pman-helper
|
||||
# committed pkgver="" and the sha256 of a zero-byte download.
|
||||
# Fail up front on a missing tool rather than midway through, where an absent
|
||||
# binary tends to be read as "condition false" and silently takes the wrong
|
||||
# branch. `diff` going missing in a slimmer base image did exactly that: it made
|
||||
# every run look like a packaging change and ratcheted pkgrel upward.
|
||||
require_tools() {
|
||||
local t missing=()
|
||||
for t in git curl jq sha256sum awk sed grep makepkg; do
|
||||
command -v "$t" >/dev/null 2>&1 || missing+=("$t")
|
||||
done
|
||||
[ "${#missing[@]}" -eq 0 ] || die "missing required tool(s): ${missing[*]}"
|
||||
return 0
|
||||
}
|
||||
|
||||
require_noninteractive() {
|
||||
[ -t 0 ] && die "refusing to prompt: this script must never depend on interactive input"
|
||||
return 0
|
||||
|
||||
49
dockerfile
Normal file
49
dockerfile
Normal file
@@ -0,0 +1,49 @@
|
||||
# Build environment for the AUR autoupdater.
|
||||
#
|
||||
# Rebuilt monthly by .gitea/workflows/build-image.yaml and pushed as :latest, so
|
||||
# the six nightly package jobs start with the toolchain already in place instead
|
||||
# of each running a full pacman transaction first.
|
||||
#
|
||||
# Arch specifically: makepkg --printsrcinfo must be the genuine Arch tool.
|
||||
# Ubuntu's build environment silently emits an empty .SRCINFO, which the AUR
|
||||
# then rejects -- that is what broke three packages before this was rewritten.
|
||||
|
||||
# base, not base-devel: the autoupdater only ever *parses* PKGBUILDs
|
||||
# (makepkg --printsrcinfo) and never compiles anything, so the toolchain in
|
||||
# base-devel is dead weight -- 822 MB against 1.48 GB. If a real build or
|
||||
# verify step is ever added here, this needs to go back to base-devel.
|
||||
FROM archlinux:base
|
||||
|
||||
LABEL org.opencontainers.image.title="aur-autoupdate"
|
||||
LABEL org.opencontainers.image.description="Arch build environment for the AUR package autoupdater"
|
||||
LABEL org.opencontainers.image.source="https://git.bjphoster.com/aur/.autoupdate"
|
||||
|
||||
# nodejs actions/checkout is a JavaScript action; the runner executes it
|
||||
# with `node` from inside this image, and base-devel has none.
|
||||
# pacman-contrib updpkgsums/paccache and friends
|
||||
# namcap PKGBUILD linting
|
||||
# fuse2 AppImage extraction
|
||||
# python test.sh's local HTTP server
|
||||
RUN pacman -Syu --noconfirm --needed \
|
||||
nodejs \
|
||||
git \
|
||||
openssh \
|
||||
jq \
|
||||
curl \
|
||||
pacman-contrib \
|
||||
namcap \
|
||||
fuse2 \
|
||||
python \
|
||||
diffutils \
|
||||
&& rm -rf /var/cache/pacman/pkg/*
|
||||
# NB: not `pacman -Scc` -- under --noconfirm it takes the default answer, which
|
||||
# is *no* for the cache prompt, so it drops sync databases (breaking pacman -Ss
|
||||
# for debugging) without actually reclaiming the cache. The rm does the real work.
|
||||
|
||||
# makepkg refuses to run as root and CI containers are root by default, so the
|
||||
# nightly job drops to this user. Baked in here rather than created per run.
|
||||
RUN useradd --create-home --shell /bin/bash builder
|
||||
|
||||
# Sanity check: fail the image build rather than the nightly job if makepkg
|
||||
# ever stops working in this environment.
|
||||
RUN runuser -u builder -- bash -c 'command -v makepkg && makepkg --version | head -1'
|
||||
Reference in New Issue
Block a user