aur/.autoupdate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Releases Activity
Files
4defd478dc2e419f5e2089a2e4d09df90f65c88b
.autoupdate/test.sh
Bryan Joshua Pedini a9f5601ec2 Replace per-package push scripts with a self-healing AUR reconciler 
The six nightly workflows each cloned a package repo and ran that repo's own
push.sh. Two problems compounded into months of silent breakage.

push.sh pushed to origin before the AUR. When the AUR push failed, origin
already carried the new version, so every later run hit "same (old) version
specified", bare-exit 0'd, and the workflow went green while the AUR rotted.
reflex-appimage sat at 1.0.11 locally against 1.0.10 published; nethlink-appimage
was never submitted at all.

The AUR pushes were failing because .SRCINFO was empty -- 0 bytes in
reflex-appimage, 1 byte in nethlink-appimage and pman-helper, against 500-2000
in the healthy ones. The AUR rejects an invalid .SRCINFO. Both PKGBUILDs verify
clean under a real Arch makepkg, so this was the CI environment, and with no
set -e the script committed and pushed the truncated file regardless.

pman-helper failed independently: the workflows queried the Gitea endpoint
/repos/{owner}/{repo}/repo/tags, which 404s -- the correct path is /tags. The
empty result fell through to an interactive read that got EOF, and it committed
pkgver="" plus the sha256 of a zero-byte download.

The replacement is a reconciler rather than a version-bump script. A new
upstream release, a manual push to origin, and a push that failed last night
all take the same path, and the AUR reconcile runs on every pass -- so a failed
push simply retries. Sync is decided by comparing tree content against the AUR,
not by whether this run happened to find something new.

pkgrel is now derived from the AUR rather than from local history, which is
what makes it idempotent: a version differing from the published one resets it
to 1, a packaging change increments it, and once published the next run sees no
difference and does nothing. Editing a PKGBUILD no longer needs a manual bump.

Checksums are computed directly instead of via makepkg -g / updpkgsums. Those
generate every arch array in one pass into a single directory, so when two
arches rename to the same target the second finds the first one's file and
emits an identical, silently wrong sum -- verified against freetube-appimage.
Downloads go to content-addressed cache paths instead. Multi-arch works for
when it is needed; single-arch is the degenerate case of the same loop.

Guards for each observed failure: a real Arch makepkg in an archlinux container
as non-root, set -euo pipefail throughout, .SRCINFO validated against the
PKGBUILD before any commit, and a failed version lookup exiting 1 so it can
never again be mistaken for a quiet "no new version" night.

Package-specific detail moves out of this repo entirely, into an .autoupdate
manifest committed to each package repo. sha256 only; sha1sums and md5sums are
removed. deskflow-bin and freetube-appimage are dropped from automation, having
been deleted from the AUR.

Covered by test.sh: 37 assertions against local bare repos and a local HTTP
server, no network and never touching the real AUR. The retry, pkgrel
idempotence and multi-arch collision tests were each verified by mutation.
2026-07-26 22:50:07 +02:00

442 lines
16 KiB
Bash
Executable File
Raw Blame History

#!/usr/bin/env bash
# Hermetic tests for the autoupdater.
#
# usage: ./test.sh [name-filter]
#
# Runs against local bare repos standing in for origin and the AUR, with a
# stubbed version lookup, so nothing here touches the network or the real AUR.
# Needs a genuine Arch makepkg (same requirement as the CI job).
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
FILTER="${1:-}"
PASS=0; FAIL=0; FAILED_NAMES=()
pass() { printf ' \033[32mok\033[0m %s\n' "$1"; PASS=$((PASS+1)); }
fail() { printf ' \033[31mFAIL\033[0m %s\n %s\n' "$1" "${2-}"; FAIL=$((FAIL+1)); FAILED_NAMES+=("$1"); }
assert_eq() { # expected actual label
if [ "$1" = "$2" ]; then pass "$3"; else fail "$3" "expected '$1', got '$2'"; fi
}
assert_ne() {
if [ "$1" != "$2" ]; then pass "$3"; else fail "$3" "expected something other than '$1'"; fi
}
# ---------------------------------------------------------------- sandbox ---
SANDBOX=""
setup() {
SANDBOX="$(mktemp -d)"
mkdir -p "$SANDBOX"/{origin,aur,work}
cat > "$SANDBOX/fake-version.sh" <<'EOS'
#!/usr/bin/env bash
# Stands in for latest-version.sh. FAKE_VERSION_FAIL=1 simulates a broken
# upstream query; FAKE_VERSION="" simulates one that returns nothing.
if [ -n "${FAKE_VERSION_FAIL:-}" ]; then
echo "ERROR: simulated upstream lookup failure" >&2
exit 1
fi
printf '%s\n' "${FAKE_VERSION-1.0.0}"
EOS
chmod +x "$SANDBOX/fake-version.sh"
# A local HTTP server so remote-source paths can be tested without network.
mkdir -p "$SANDBOX/www"
printf 'x86_64 binary payload\n' > "$SANDBOX/www/x86.bin"
printf 'aarch64 binary payload\n' > "$SANDBOX/www/arm.bin"
# -u: without it the "Serving HTTP on ... port N" line stays buffered and the
# port can never be read back.
( cd "$SANDBOX/www" && exec python3 -u -m http.server 0 --bind 127.0.0.1 ) \
> "$SANDBOX/www.log" 2>&1 &
WWW_PID=$!
for _ in $(seq 1 50); do
PORT="$(sed -n -E 's/.*http:\/\/127\.0\.0\.1:([0-9]+).*/\1/p' "$SANDBOX/www.log" | head -1)"
[ -n "$PORT" ] && break
sleep 0.1
done
[ -n "${PORT:-}" ] || { echo "could not start test http server" >&2; exit 1; }
export PKG_REPO_BASE="file://$SANDBOX/origin"
export AUR_HTTPS_BASE="file://$SANDBOX/aur"
export AUR_SSH_BASE="file://$SANDBOX/aur"
export LATEST_VERSION_CMD="$SANDBOX/fake-version.sh"
export GIT_AUTHOR_NAME=test GIT_AUTHOR_EMAIL=test@test
export GIT_COMMITTER_NAME=test GIT_COMMITTER_EMAIL=test@test
}
teardown() {
[ -n "${WWW_PID:-}" ] && kill "$WWW_PID" 2>/dev/null
[ -n "$SANDBOX" ] && rm -rf "$SANDBOX"
return 0
}
# A multi-arch package whose arches download DIFFERENT files that both rename to
# the SAME target -- freetube-appimage's exact shape, and the case that makes
# `makepkg -g` emit identical sums for both arches.
make_pkg_colliding_remote() {
local pkg="$1"
local src="$SANDBOX/build/$pkg"
rm -rf "$src"; mkdir -p "$src"
cat > "$src/PKGBUILD" <<EOF
pkgname="$pkg"
pkgver="1.0.0"
pkgrel="1"
pkgdesc="colliding multi-arch test package"
url="https://example.invalid/$pkg"
arch=("x86_64" "aarch64")
license=("MIT")
source_x86_64=("payload.bin::http://127.0.0.1:${PORT}/x86.bin")
sha256sums_x86_64=("SKIP")
source_aarch64=("payload.bin::http://127.0.0.1:${PORT}/arm.bin")
sha256sums_aarch64=("SKIP")
package() {
install -Dm644 "\${srcdir}/payload.bin" "\${pkgdir}/usr/share/$pkg/payload.bin"
}
EOF
cat > "$src/.autoupdate" <<'EOF'
srchost="gitea"
srcmntr="test"
srcname="test"
verstrip=""
gui="false"
EOF
( cd "$src" && "$HERE/update-checksums.sh" --remote . >/dev/null \
&& makepkg --printsrcinfo > .SRCINFO )
git init --quiet --bare --initial-branch=main "$SANDBOX/origin/$pkg"
git init --quiet --bare --initial-branch=master "$SANDBOX/aur/${pkg}.git"
( cd "$src" && git init --quiet --initial-branch=main && git add -A \
&& git commit --quiet -m "initial" \
&& git push --quiet "file://$SANDBOX/origin/$pkg" main \
&& git push --quiet "file://$SANDBOX/aur/${pkg}.git" main:master )
}
# Create a package repo (origin) plus its AUR counterpart.
# make_pkg <name> [multiarch]
make_pkg() {
local pkg="$1" multi="${2:-}"
local src="$SANDBOX/build/$pkg"
rm -rf "$src"; mkdir -p "$src"
echo "payload for $pkg" > "$src/local.txt"
if [ -n "$multi" ]; then
echo "x86 payload" > "$src/pay-x86_64.txt"
echo "arm payload" > "$src/pay-aarch64.txt"
cat > "$src/PKGBUILD" <<EOF
pkgname="$pkg"
pkgver="1.0.0"
pkgrel="1"
pkgdesc="multi-arch test package"
url="https://example.invalid/$pkg"
arch=("x86_64" "aarch64")
license=("MIT")
source=("local.txt")
sha256sums=("SKIP")
source_x86_64=("pay-x86_64.txt")
sha256sums_x86_64=("SKIP")
source_aarch64=("pay-aarch64.txt")
sha256sums_aarch64=("SKIP")
package() {
install -Dm644 "\${srcdir}/local.txt" "\${pkgdir}/usr/share/$pkg/local.txt"
}
EOF
else
cat > "$src/PKGBUILD" <<EOF
pkgname="$pkg"
pkgver="1.0.0"
pkgrel="1"
pkgdesc="test package"
url="https://example.invalid/$pkg"
arch=("any")
license=("MIT")
source=("local.txt")
sha256sums=("SKIP")
package() {
install -Dm644 "\${srcdir}/local.txt" "\${pkgdir}/usr/share/$pkg/local.txt"
}
EOF
fi
cat > "$src/.autoupdate" <<'EOF'
srchost="gitea"
srcmntr="test"
srcname="test"
verstrip=""
gui="false"
EOF
echo "readme" > "$src/README.md"
# Fix up the SKIP placeholders and generate a valid .SRCINFO.
( cd "$src" && "$HERE/update-checksums.sh" . >/dev/null \
&& makepkg --printsrcinfo > .SRCINFO )
git init --quiet --bare --initial-branch=main "$SANDBOX/origin/$pkg"
git init --quiet --bare --initial-branch=master "$SANDBOX/aur/${pkg}.git"
( cd "$src" && git init --quiet --initial-branch=main && git add -A \
&& git commit --quiet -m "initial" \
&& git push --quiet "file://$SANDBOX/origin/$pkg" main \
&& git push --quiet "file://$SANDBOX/aur/${pkg}.git" main:master )
}
# Run the updater. Any extra env goes in front of the call by the caller.
run_update() {
local pkg="$1"; shift
WORKDIR="$SANDBOX/work/$(date +%s%N)" \
"$HERE/autoupdate.sh" "$pkg" "$@" >"$SANDBOX/last.log" 2>&1
}
# Read a field from a bare repo's tip.
field_at() { # <bare-repo> <ref> <var>
git -C "$1" show "$2:PKGBUILD" 2>/dev/null \
| sed -n -E "s/^$3=[\"']?([^\"']*)[\"']?$/\1/p" | head -n1
}
aur_field() { field_at "$SANDBOX/aur/$1.git" master "$2"; }
origin_field() { field_at "$SANDBOX/origin/$1" main "$2"; }
aur_head() { git -C "$SANDBOX/aur/$1.git" rev-parse master 2>/dev/null; }
should_run() { [ -z "$FILTER" ] || [[ "$1" == *"$FILTER"* ]]; }
# ------------------------------------------------------------------ tests ---
test_new_version() {
make_pkg p1
FAKE_VERSION=2.0.0 run_update p1
assert_eq "0" "$?" "new version: exits 0"
assert_eq "2.0.0" "$(aur_field p1 pkgver)" "new version: pkgver reaches the AUR"
assert_eq "1" "$(aur_field p1 pkgrel)" "new version: pkgrel resets to 1"
}
test_noop_is_quiet_and_green() {
make_pkg p2
FAKE_VERSION=1.0.0 run_update p2
local rc=$? before after
before="$(aur_head p2)"
FAKE_VERSION=1.0.0 run_update p2
after="$(aur_head p2)"
assert_eq "0" "$rc" "no-op: exits 0"
assert_eq "$before" "$after" "no-op: second run pushes nothing"
grep -q "AUR already in sync" "$SANDBOX/last.log" \
&& pass "no-op: reports already in sync" \
|| fail "no-op: reports already in sync" "$(tail -3 "$SANDBOX/last.log")"
}
# The headline regression: origin ahead of the AUR, nothing new upstream.
# The old script aborted with "same (old) version" and exit 0, forever.
# Reproduces the exact reported failure: a previous run published to origin but
# its AUR push failed. Origin is therefore fully consistent -- correct sums,
# correct .SRCINFO, pkgrel already bumped -- so this run computes NO local
# changes at all. The only thing wrong is that the AUR is behind. The old
# script hit "same (old) version specified", bare-exit 0'd, and never retried.
#
# The pkgrel is pre-bumped deliberately: otherwise the pkgrel rule would itself
# produce a local change and the run would push for that reason instead,
# leaving the actual retry path untested.
test_failed_push_is_retried() {
make_pkg p3
local clone="$SANDBOX/manual-p3"
git clone --quiet "file://$SANDBOX/origin/p3" "$clone"
( cd "$clone" \
&& sed -i 's/^pkgdesc=.*/pkgdesc="changed out of band"/;s/^pkgrel=.*/pkgrel="2"/' PKGBUILD \
&& makepkg --printsrcinfo > .SRCINFO \
&& git commit --quiet -am "published to origin; AUR push failed" \
&& git push --quiet origin main )
assert_ne "changed out of band" "$(aur_field p3 pkgdesc)" "retry: AUR starts out of date"
FAKE_VERSION=1.0.0 run_update p3
local rc=$?
assert_eq "0" "$rc" "retry: exits 0"
grep -q "no local changes" "$SANDBOX/last.log" \
&& pass "retry: run produced no local changes (exercises the reconcile path)" \
|| fail "retry: run produced no local changes" \
"the run committed something, so this is not the retry path: $(tail -5 "$SANDBOX/last.log")"
assert_eq "changed out of band" "$(aur_field p3 pkgdesc)" \
"retry: reconciles to the AUR with nothing new locally or upstream"
}
test_manual_edit_bumps_pkgrel() {
make_pkg p4
local clone="$SANDBOX/manual-p4"
git clone --quiet "file://$SANDBOX/origin/p4" "$clone"
( cd "$clone" && sed -i 's/^pkgdesc=.*/pkgdesc="fixed package() bug"/' PKGBUILD \
&& makepkg --printsrcinfo > .SRCINFO \
&& git commit --quiet -am "fix package()" && git push --quiet origin main )
FAKE_VERSION=1.0.0 run_update p4
assert_eq "2" "$(aur_field p4 pkgrel)" "manual edit: pkgrel auto-bumped 1 -> 2"
# The important one: re-running must not ratchet pkgrel upward.
FAKE_VERSION=1.0.0 run_update p4
assert_eq "2" "$(aur_field p4 pkgrel)" "manual edit: pkgrel stable on re-run (idempotent)"
FAKE_VERSION=1.0.0 run_update p4
assert_eq "2" "$(aur_field p4 pkgrel)" "manual edit: pkgrel still stable on third run"
}
test_hand_bumped_pkgrel_respected() {
make_pkg p5
local clone="$SANDBOX/manual-p5"
git clone --quiet "file://$SANDBOX/origin/p5" "$clone"
( cd "$clone" && sed -i 's/^pkgdesc=.*/pkgdesc="edited"/;s/^pkgrel=.*/pkgrel="7"/' PKGBUILD \
&& makepkg --printsrcinfo > .SRCINFO \
&& git commit --quiet -am "edit + own pkgrel" && git push --quiet origin main )
FAKE_VERSION=1.0.0 run_update p5
assert_eq "7" "$(aur_field p5 pkgrel)" "hand-bumped pkgrel is left alone"
}
test_readme_only_edit() {
make_pkg p6
local clone="$SANDBOX/manual-p6"
git clone --quiet "file://$SANDBOX/origin/p6" "$clone"
( cd "$clone" && echo "more docs" >> README.md \
&& git commit --quiet -am "docs" && git push --quiet origin main )
FAKE_VERSION=1.0.0 run_update p6
local rc=$? # must be captured before any other command clobbers $?
assert_eq "0" "$rc" "README-only edit: still exits 0"
assert_eq "1" "$(aur_field p6 pkgrel)" "README-only edit: no pkgrel bump"
assert_eq "more docs" "$(git -C "$SANDBOX/aur/p6.git" show master:README.md | tail -1)" \
"README-only edit: still reconciled to the AUR"
}
test_local_source_edit_bumps_pkgrel() {
make_pkg p7
local clone="$SANDBOX/manual-p7"
git clone --quiet "file://$SANDBOX/origin/p7" "$clone"
( cd "$clone" && echo "changed payload" > local.txt \
&& git commit --quiet -am "edit local source" && git push --quiet origin main )
FAKE_VERSION=1.0.0 run_update p7
assert_eq "2" "$(aur_field p7 pkgrel)" "local source edit: re-hashed and pkgrel bumped"
}
test_force_rebuild() {
make_pkg p8
FAKE_VERSION=1.0.0 run_update p8
FORCE_REBUILD=1 FAKE_VERSION=1.0.0 run_update p8
assert_eq "2" "$(aur_field p8 pkgrel)" "FORCE_REBUILD: pkgrel +1 with no other change"
}
test_lookup_failure_is_loud() {
make_pkg p9
local before; before="$(aur_head p9)"
FAKE_VERSION_FAIL=1 run_update p9
assert_eq "1" "$?" "broken upstream lookup: exits 1, not mistaken for 'no new version'"
assert_eq "$before" "$(aur_head p9)" "broken upstream lookup: nothing pushed"
}
test_empty_version_aborts_before_commit() {
make_pkg p10
local before; before="$(aur_head p10)"
FAKE_VERSION="" run_update p10
assert_eq "1" "$?" "empty version: exits 1"
assert_eq "$before" "$(aur_head p10)" "empty version: aborts before committing"
assert_eq "1.0.0" "$(origin_field p10 pkgver)" "empty version: origin left unpoisoned"
}
# reflex-appimage's real state: local ahead of the AUR carrying a stale pkgrel
# that the old script never reset.
test_stale_pkgrel_repaired() {
make_pkg p11
local clone="$SANDBOX/manual-p11"
git clone --quiet "file://$SANDBOX/origin/p11" "$clone"
( cd "$clone" && sed -i 's/^pkgver=.*/pkgver="1.5.0"/;s/^pkgrel=.*/pkgrel="4"/' PKGBUILD \
&& makepkg --printsrcinfo > .SRCINFO \
&& git commit --quiet -am "version bumped without resetting pkgrel" \
&& git push --quiet origin main )
FAKE_VERSION=1.5.0 run_update p11
assert_eq "1.5.0" "$(aur_field p11 pkgver)" "stale pkgrel: version published"
assert_eq "1" "$(aur_field p11 pkgrel)" "stale pkgrel: reset to 1 against the AUR"
}
test_multiarch_distinct_sums() {
make_pkg p12 multi
FAKE_VERSION=1.0.0 run_update p12
local x a
x="$(git -C "$SANDBOX/aur/p12.git" show master:PKGBUILD \
| sed -n '/^sha256sums_x86_64=/,/)/p' | grep -oE '[0-9a-f]{64}')"
a="$(git -C "$SANDBOX/aur/p12.git" show master:PKGBUILD \
| sed -n '/^sha256sums_aarch64=/,/)/p' | grep -oE '[0-9a-f]{64}')"
assert_eq "$(sha256sum "$SANDBOX/build/p12/pay-x86_64.txt" | awk '{print $1}')" "$x" \
"multi-arch: x86_64 sum correct"
assert_eq "$(sha256sum "$SANDBOX/build/p12/pay-aarch64.txt" | awk '{print $1}')" "$a" \
"multi-arch: aarch64 sum correct"
assert_ne "$x" "$a" "multi-arch: arches get distinct sums (the makepkg -g failure mode)"
}
# The regression that rules out `makepkg -g`: two arches downloading different
# files that rename to the same target must not end up sharing a checksum.
test_multiarch_colliding_downloads() {
make_pkg_colliding_remote p15
FAKE_VERSION=1.0.0 FORCE_REFRESH=1 run_update p15
local pb x a
pb="$(git -C "$SANDBOX/aur/p15.git" show master:PKGBUILD)"
x="$(printf '%s' "$pb" | sed -n '/^sha256sums_x86_64=/,/)/p' | grep -oE '[0-9a-f]{64}')"
a="$(printf '%s' "$pb" | sed -n '/^sha256sums_aarch64=/,/)/p' | grep -oE '[0-9a-f]{64}')"
assert_eq "$(sha256sum "$SANDBOX/www/x86.bin" | awk '{print $1}')" "$x" \
"colliding downloads: x86_64 hashed its own file"
assert_eq "$(sha256sum "$SANDBOX/www/arm.bin" | awk '{print $1}')" "$a" \
"colliding downloads: aarch64 hashed its own file, not x86_64's"
assert_ne "$x" "$a" "colliding downloads: sums differ despite the shared :: target"
}
test_srcinfo_guard() {
make_pkg p13
local clone="$SANDBOX/manual-p13"
git clone --quiet "file://$SANDBOX/origin/p13" "$clone"
# pkgver that makepkg rejects -> .SRCINFO cannot be generated.
( cd "$clone" && sed -i 's/^pkgdesc=.*/pkgdesc="x"/' PKGBUILD \
&& git commit --quiet -am "touch" && git push --quiet origin main )
local before; before="$(aur_head p13)"
FAKE_VERSION="not a version!" run_update p13
assert_eq "1" "$?" ".SRCINFO guard: invalid version rejected"
assert_eq "$before" "$(aur_head p13)" ".SRCINFO guard: nothing reached the AUR"
}
test_no_git_push_dry_run() {
make_pkg p14
local before_aur before_origin
before_aur="$(aur_head p14)"
before_origin="$(git -C "$SANDBOX/origin/p14" rev-parse main)"
NO_GIT_PUSH=1 FAKE_VERSION=3.0.0 run_update p14
assert_eq "0" "$?" "NO_GIT_PUSH: exits 0"
assert_eq "$before_aur" "$(aur_head p14)" "NO_GIT_PUSH: AUR untouched"
assert_eq "$before_origin" "$(git -C "$SANDBOX/origin/p14" rev-parse main)" \
"NO_GIT_PUSH: origin untouched"
}
# ------------------------------------------------------------------- main ---
trap teardown EXIT
setup
for t in $(declare -F | awk '{print $3}' | grep '^test_' | sort); do
should_run "$t" || continue
printf '\n\033[1m%s\033[0m\n' "$t"
"$t"
done
printf '\n────────────────────────\n'
printf 'passed: %d failed: %d\n' "$PASS" "$FAIL"
if [ "$FAIL" -gt 0 ]; then
printf 'failing: %s\n' "${FAILED_NAMES[*]}"
exit 1
fi
Reference in New Issue View Git Blame Copy Permalink