Bryan Joshua Pedini
a9f5601ec2
The six nightly workflows each cloned a package repo and ran that repo's own
push.sh. Two problems compounded into months of silent breakage.
push.sh pushed to origin before the AUR. When the AUR push failed, origin
already carried the new version, so every later run hit "same (old) version
specified", bare-exit 0'd, and the workflow went green while the AUR rotted.
reflex-appimage sat at 1.0.11 locally against 1.0.10 published; nethlink-appimage
was never submitted at all.
The AUR pushes were failing because .SRCINFO was empty -- 0 bytes in
reflex-appimage, 1 byte in nethlink-appimage and pman-helper, against 500-2000
in the healthy ones. The AUR rejects an invalid .SRCINFO. Both PKGBUILDs verify
clean under a real Arch makepkg, so this was the CI environment, and with no
set -e the script committed and pushed the truncated file regardless.
pman-helper failed independently: the workflows queried the Gitea endpoint
/repos/{owner}/{repo}/repo/tags, which 404s -- the correct path is /tags. The
empty result fell through to an interactive read that got EOF, and it committed
pkgver="" plus the sha256 of a zero-byte download.
The replacement is a reconciler rather than a version-bump script. A new
upstream release, a manual push to origin, and a push that failed last night
all take the same path, and the AUR reconcile runs on every pass -- so a failed
push simply retries. Sync is decided by comparing tree content against the AUR,
not by whether this run happened to find something new.
pkgrel is now derived from the AUR rather than from local history, which is
what makes it idempotent: a version differing from the published one resets it
to 1, a packaging change increments it, and once published the next run sees no
difference and does nothing. Editing a PKGBUILD no longer needs a manual bump.
Checksums are computed directly instead of via makepkg -g / updpkgsums. Those
generate every arch array in one pass into a single directory, so when two
arches rename to the same target the second finds the first one's file and
emits an identical, silently wrong sum -- verified against freetube-appimage.
Downloads go to content-addressed cache paths instead. Multi-arch works for
when it is needed; single-arch is the degenerate case of the same loop.
Guards for each observed failure: a real Arch makepkg in an archlinux container
as non-root, set -euo pipefail throughout, .SRCINFO validated against the
PKGBUILD before any commit, and a failed version lookup exiting 1 so it can
never again be mistaken for a quiet "no new version" night.
Package-specific detail moves out of this repo entirely, into an .autoupdate
manifest committed to each package repo. sha256 only; sha1sums and md5sums are
removed. deskflow-bin and freetube-appimage are dropped from automation, having
been deleted from the AUR.
Covered by test.sh: 37 assertions against local bare repos and a local HTTP
server, no network and never touching the real AUR. The retry, pkgrel
idempotence and multi-arch collision tests were each verified by mutation.
AUR Auto Update
Nightly reconciler that keeps the aur/* package repos and their AUR
counterparts in sync.
This is a reconciler, not a version-bump script. A new upstream release is only one of the things that can make a package diverge from the AUR; a manual push to origin is another, and a push that failed last night is a third. All three take the same path, and the AUR reconcile runs on every pass.
Layout
| file | role |
|---|---|
autoupdate.sh |
entry point; orchestrates one or more packages |
latest-version.sh |
resolves the latest upstream version (GitHub / Gitea) |
update-checksums.sh |
downloads sources, computes sha256, rewrites the arrays |
sync-aur.sh |
reconciles the repo against the AUR |
common.sh |
logging, guards, PKGBUILD parsing and array splicing |
test.sh |
hermetic test suite |
.gitea/workflows/update.yaml |
nightly matrix job |
Nothing here is package-specific. Everything that differs between packages
lives in a .autoupdate manifest committed to the package repo itself.
Per-package manifest
Each package repo carries a .autoupdate file:
srchost="github" # github | gitea
srcmntr="Sunhaiy" # upstream owner/org
srcname="Reflex" # upstream repo
verstrip="v" # tag prefix to strip ("app-v" for open-video-downloader-*)
gui="true" # extract .desktop + icon from the AppImage
desktop="reflex.desktop" # name *inside* the AppImage
icon="reflex.png"
It rides along to the AUR on push, as README.md and LICENSE already do.
Harmless: the AUR reads only PKGBUILD and .SRCINFO.
What a run does
- Clone the package repo, read its manifest.
- Fetch the AUR over HTTPS as the reference state. This happens before
anything else, because
pkgrelis derived from it. Read-only, so no SSH key is required for a dry run. - Resolve the upstream version. A failed lookup exits 1; "no new version" is not a failure.
- If the version moved, re-download and re-hash the remote sources.
- Always re-hash local sources (
.desktop, icons) and regenerate.SRCINFO. Local hashing needs no network, so this is free, and it is what lets a hand-edited repo self-correct. - Derive
pkgrel(below). - Validate
.SRCINFO; abort before committing if it is empty or disagrees with the PKGBUILD. - Commit only if something actually changed.
- Push to the AUR first, then origin, so a failed AUR push leaves origin clean.
pkgrel
X.Y.Z-N — everything before the dash is upstream, -N is packaging-only
revisions. You should never need to touch it by hand.
pkgrel is derived from the AUR, because that is what -N is numbered
against:
- version differs from the published one →
pkgrel = 1 - same version, packaging content differs →
pkgrel = AUR's + 1 - same version, packaging identical → unchanged
- you already bumped it yourself → respected, not bumped again
Using the AUR as the reference rather than local history is what makes this
idempotent: once the bump is published, the next night sees no difference and
does nothing. There is a test for exactly that (pkgrel stable on re-run).
An edit to a local source such as an icon counts as a packaging change, because
its sha256sums entry is regenerated first. A README-only edit does not.
Environment
| variable | effect |
|---|---|
NO_GIT_PUSH=1 |
dry run: do everything except push |
FORCE_REBUILD=1 |
bump pkgrel even with no detected change |
FORCE_REFRESH=1 |
re-download remote sources without a version change |
AUR_FORCE=1 |
force-push over diverged AUR history (see below) |
GIT_HOST |
git host holding the package repos |
GITHUB_TOKEN |
optional; avoids GitHub's 60/hour anonymous rate limit |
A rejected AUR push usually means someone pushed there directly. The run fails
loudly rather than overwriting it; inspect, then re-run with AUR_FORCE=1.
Exit codes
"No new upstream version" is the overwhelmingly common outcome and exits 0.
The trap the previous implementation fell into was giving that same silent
exit 0 to a lookup that had broken — which is how pman-helper shipped
pkgver="" unnoticed. A failed lookup now exits 1.
Requirements
Needs a genuine Arch makepkg. Ubuntu's build environment emits an empty
.SRCINFO, which the AUR rejects — this silently broke three packages. The CI
job therefore runs in an archlinux:base-devel container as a non-root user
(makepkg refuses to run as root), and regen_srcinfo in common.sh fails
loudly if the output is ever empty again.
Notes on checksums
sha256 only; sha1sums/md5sums are actively removed.
Checksums are computed directly rather than via makepkg -g / updpkgsums.
Those generate every arch array in one pass and download all arches into a
single directory, so when two arches rename to the same target
(foo.AppImage::…-amd64, foo.AppImage::…-arm64) the second finds the first
one's file already there and emits an identical, silently wrong checksum.
Overriding CARCH does not help. Downloads here go to content-addressed cache
paths instead, never to the :: rename target. Multi-arch is supported for
when a package needs it; single-arch is the degenerate case of the same loop.
Tests
./test.sh # all
./test.sh pkgrel # filter by name
Runs against local bare repos and a local HTTP server — no network, and it
never touches the real AUR. Requires a real makepkg.