Files
.autoupdate/autoupdate.sh
Bryan Joshua Pedini a9f5601ec2 Replace per-package push scripts with a self-healing AUR reconciler
The six nightly workflows each cloned a package repo and ran that repo's own
push.sh. Two problems compounded into months of silent breakage.

push.sh pushed to origin before the AUR. When the AUR push failed, origin
already carried the new version, so every later run hit "same (old) version
specified", bare-exit 0'd, and the workflow went green while the AUR rotted.
reflex-appimage sat at 1.0.11 locally against 1.0.10 published; nethlink-appimage
was never submitted at all.

The AUR pushes were failing because .SRCINFO was empty -- 0 bytes in
reflex-appimage, 1 byte in nethlink-appimage and pman-helper, against 500-2000
in the healthy ones. The AUR rejects an invalid .SRCINFO. Both PKGBUILDs verify
clean under a real Arch makepkg, so this was the CI environment, and with no
set -e the script committed and pushed the truncated file regardless.

pman-helper failed independently: the workflows queried the Gitea endpoint
/repos/{owner}/{repo}/repo/tags, which 404s -- the correct path is /tags. The
empty result fell through to an interactive read that got EOF, and it committed
pkgver="" plus the sha256 of a zero-byte download.

The replacement is a reconciler rather than a version-bump script. A new
upstream release, a manual push to origin, and a push that failed last night
all take the same path, and the AUR reconcile runs on every pass -- so a failed
push simply retries. Sync is decided by comparing tree content against the AUR,
not by whether this run happened to find something new.

pkgrel is now derived from the AUR rather than from local history, which is
what makes it idempotent: a version differing from the published one resets it
to 1, a packaging change increments it, and once published the next run sees no
difference and does nothing. Editing a PKGBUILD no longer needs a manual bump.

Checksums are computed directly instead of via makepkg -g / updpkgsums. Those
generate every arch array in one pass into a single directory, so when two
arches rename to the same target the second finds the first one's file and
emits an identical, silently wrong sum -- verified against freetube-appimage.
Downloads go to content-addressed cache paths instead. Multi-arch works for
when it is needed; single-arch is the degenerate case of the same loop.

Guards for each observed failure: a real Arch makepkg in an archlinux container
as non-root, set -euo pipefail throughout, .SRCINFO validated against the
PKGBUILD before any commit, and a failed version lookup exiting 1 so it can
never again be mistaken for a quiet "no new version" night.

Package-specific detail moves out of this repo entirely, into an .autoupdate
manifest committed to each package repo. sha256 only; sha1sums and md5sums are
removed. deskflow-bin and freetube-appimage are dropped from automation, having
been deleted from the AUR.

Covered by test.sh: 37 assertions against local bare repos and a local HTTP
server, no network and never touching the real AUR. The retry, pkgrel
idempotence and multi-arch collision tests were each verified by mutation.
2026-07-26 22:50:07 +02:00

267 lines
9.6 KiB
Bash
Executable File

#!/usr/bin/env bash
# Nightly AUR package reconciler.
#
# usage: autoupdate.sh <package> [...]
#
# env:
# GIT_HOST git host holding the package repos (default git.bjphoster.com)
# WORKDIR where to clone; a temp dir by default
# NO_GIT_PUSH=1 do everything except push (dry run)
# FORCE_REBUILD=1 bump pkgrel even with no detected change
# FORCE_REFRESH=1 re-download remote sources even without a version change
# AUR_FORCE=1 force-push over diverged AUR history
#
# This is a RECONCILER, not a version-bump script. A new upstream release is
# only one of the things that can make a package diverge from the AUR; a manual
# push to origin is another, and a failed push last night is a third. All three
# funnel through the same path, and the AUR reconcile at the end runs
# unconditionally.
set -euo pipefail
_here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=common.sh
source "${_here}/common.sh"
require_noninteractive
GIT_HOST="${GIT_HOST:-git.bjphoster.com}"
# Where the package repos live, and how the upstream version is resolved.
# Both are overridable so the test suite can run hermetically against local
# bare repos; the defaults are the production values.
PKG_REPO_BASE="${PKG_REPO_BASE:-https://${GIT_HOST}/aur}"
LATEST_VERSION_CMD="${LATEST_VERSION_CMD:-${_here}/latest-version.sh}"
[ "$#" -ge 1 ] || die "usage: autoupdate.sh <package> [...]"
WORKDIR="${WORKDIR:-$(mktemp -d)}"
mkdir -p "$WORKDIR"
update_one() {
local pkg="$1"
local dir="${WORKDIR}/${pkg}"
log "=== ${pkg} ==="
if [ -d "$dir/.git" ]; then
git -C "$dir" fetch --quiet origin
git -C "$dir" reset --quiet --hard origin/HEAD
else
rm -rf "$dir"
git clone --quiet "${PKG_REPO_BASE}/${pkg}" "$dir" \
|| die "could not clone ${pkg} from ${PKG_REPO_BASE}"
fi
cd "$dir"
[ -f PKGBUILD ] || die "${pkg} has no PKGBUILD"
[ -f .autoupdate ] || die "${pkg} has no .autoupdate manifest"
# shellcheck disable=SC1091
source ./.autoupdate
# Per-run download cache, shared between the AppImage extraction below and
# update-checksums.sh so a 100 MB AppImage is fetched at most once.
export DL_CACHE="${WORKDIR}/.cache"
mkdir -p "$DL_CACHE"
# --- AUR reference state -------------------------------------------------
# Fetched up front: it is the reference for the pkgrel decision, not just a
# push target. Comparing against the AUR rather than against local history is
# what makes the pkgrel bump idempotent -- once pushed, the AUR matches and
# the next run finds nothing to do.
# Read-only HTTPS: establishing the reference state needs no SSH key, so
# dry runs work anywhere. Only sync-aur.sh's push is authenticated.
local aur_pkgbuild="" aur_pkgrel="" aur_pkgver="" aur_name
aur_name="$(pkgbuild_expand PKGBUILD 'printf %s "$pkgname"')"
git remote remove aur-ro >/dev/null 2>&1 || true
git remote add aur-ro "${AUR_HTTPS_BASE}/${aur_name}.git"
if git fetch --quiet aur-ro master 2>/dev/null; then
aur_pkgbuild="${WORKDIR}/${pkg}.aur.PKGBUILD"
if git show FETCH_HEAD:PKGBUILD > "$aur_pkgbuild" 2>/dev/null; then
aur_pkgrel="$(pkgbuild_get pkgrel "$aur_pkgbuild")"
aur_pkgver="$(pkgbuild_get pkgver "$aur_pkgbuild")"
info "AUR has ${aur_pkgver}-${aur_pkgrel}"
else
aur_pkgbuild=""
fi
else
info "not on the AUR yet"
fi
# --- upstream version ----------------------------------------------------
# A failure in here exits non-zero and is NOT mistaken for "no new version".
local oldver newver
oldver="$(pkgbuild_get pkgver)"
newver="$("$LATEST_VERSION_CMD" .)"
info "upstream ${newver}, local ${oldver:-<empty>}"
local version_changed=0
if [ "$newver" != "$oldver" ]; then
version_changed=1
log "new upstream version ${oldver:-<empty>} -> ${newver}"
pkgbuild_set pkgver "$newver"
# pkgrel is not touched here; apply_pkgrel_rule derives it from the AUR.
fi
# --- AppImage assets -----------------------------------------------------
if [ "${gui:-false}" = "true" ] && [ "$version_changed" -eq 1 ]; then
extract_appimage_assets
fi
# --- checksums + .SRCINFO ------------------------------------------------
# Local sources are always re-hashed (free, no network) so a hand-edited
# .desktop or icon self-corrects. Remote sources are only re-fetched when the
# version moved, or on demand.
local cs_args=()
if [ "$version_changed" -eq 1 ] || [ -n "${FORCE_REFRESH:-}" ]; then
cs_args+=(--remote)
fi
"${_here}/update-checksums.sh" "${cs_args[@]}" .
regen_srcinfo .
# --- pkgrel ---------------------------------------------------------------
apply_pkgrel_rule
# --- commit ---------------------------------------------------------------
git add -A
if git diff --cached --quiet; then
# Nothing changed. Deliberately do NOT exit here: the AUR may still be out
# of sync from a previous failed push, and an unguarded `git commit` with
# nothing staged would exit non-zero and, under `set -e`, abort before the
# reconcile.
log "no local changes for ${pkg}"
else
local ver rel
ver="$(pkgbuild_get pkgver)"; rel="$(pkgbuild_get pkgrel)"
git commit --quiet -m "Updated ${pkg} to ${ver}-${rel}"
log "committed ${pkg} ${ver}-${rel}"
fi
# --- publish --------------------------------------------------------------
# AUR first: if it rejects the push, origin is left unpoisoned and the whole
# thing is recomputed cleanly on the next run.
"${_here}/sync-aur.sh" .
if [ -n "${NO_GIT_PUSH:-}" ]; then
log "NO_GIT_PUSH set: not pushing ${pkg} to origin"
else
git push --quiet origin HEAD
fi
}
# Derive pkgrel so that fixing a package() bug and pushing needs no manual bump.
apply_pkgrel_rule() {
local cur_ver cur_rel
cur_ver="$(pkgbuild_get pkgver)"
cur_rel="$(pkgbuild_get pkgrel)"
# Nothing on the AUR to compare against (new package): push as-is.
if [ -z "${aur_pkgbuild:-}" ]; then
return 0
fi
# pkgrel is derived purely from the AUR reference, because the AUR is what
# `-N` is numbered against. If the version we are about to publish differs
# from the published one, this is a new upstream version as far as the AUR is
# concerned and the packaging revision restarts at 1 -- whether the bump
# happened in this run or in an earlier one that failed to push. That second
# case is real: reflex-appimage sat at 1.0.11-4 locally against 1.0.10-4 on
# the AUR, because the old script only reset pkgrel under FORCE_REBUILD.
if [ "$cur_ver" != "${aur_pkgver:-}" ]; then
if [ "$cur_rel" != "1" ]; then
pkgbuild_set pkgrel "1"
log "new version relative to the AUR (${aur_pkgver} -> ${cur_ver}); pkgrel ${cur_rel} -> 1"
regen_srcinfo .
fi
return 0
fi
local base_rel="${aur_pkgrel:-0}"
[[ "$base_rel" =~ ^[0-9]+$ ]] || base_rel=0
if [ -n "${FORCE_REBUILD:-}" ]; then
pkgbuild_set pkgrel "$(( base_rel + 1 ))"
log "FORCE_REBUILD: pkgrel -> $(( base_rel + 1 ))"
regen_srcinfo .
return 0
fi
# Compare with the pkgrel line stripped, so the decision is about substantive
# packaging content. An edit to a local source lands here too, because its
# sha256sums entry was just regenerated. A README-only edit does not touch
# the PKGBUILD and so correctly does not bump.
if diff -q <(grep -v '^pkgrel=' PKGBUILD) \
<(grep -v '^pkgrel=' "$aur_pkgbuild") >/dev/null 2>&1; then
return 0 # identical packaging; nothing to do
fi
if [ "$cur_rel" = "$aur_pkgrel" ]; then
pkgbuild_set pkgrel "$(( base_rel + 1 ))"
log "packaging changed without a pkgrel bump; pkgrel -> $(( base_rel + 1 ))"
regen_srcinfo .
else
info "pkgrel already moved by hand (${aur_pkgrel} -> ${cur_rel}); leaving it"
fi
}
# Pull the .desktop and icon out of the AppImage so they can be shipped as
# local sources. Unchanged in substance from the old push.sh.
extract_appimage_assets() {
local host_arr="source_$(uname -m)" entry loc img newdesktop newicon
local candidates=()
# A multi-arch AppImage is extracted once, against the host arch's image.
if pkgbuild_has_array PKGBUILD "$host_arr"; then
mapfile -t candidates < <(pkgbuild_array PKGBUILD "$host_arr")
fi
if [ "${#candidates[@]}" -eq 0 ]; then
mapfile -t candidates < <(pkgbuild_array PKGBUILD source)
fi
img=""
for entry in "${candidates[@]}"; do
loc="$(source_loc "$entry")"
if printf '%s' "${entry,,}" | grep -q "appimage" && is_remote "$entry"; then
img="$(download_cached "$loc")"
break
fi
done
[ -n "$img" ] || { warn "gui=true but no AppImage source found; skipping asset extraction"; return 0; }
local pkgbase
pkgbase="$(pkgbuild_expand PKGBUILD 'printf %s "${_pkgname:-$pkgname}"')"
newdesktop="${pkgbase}.desktop"
newicon="${pkgbase}.png"
local tmp; tmp="$(mktemp -d)"
cp "$img" "${tmp}/app.AppImage"
chmod +x "${tmp}/app.AppImage"
( cd "$tmp" && ./app.AppImage --appimage-extract >/dev/null 2>&1 ) \
|| { warn "could not extract ${pkgbase} AppImage; keeping existing assets"; rm -rf "$tmp"; return 0; }
if [ -f "${tmp}/squashfs-root/${desktop}" ]; then
cp "${tmp}/squashfs-root/${desktop}" "$newdesktop"
sed -i "/^X-AppImage-/d;/^$/d" "$newdesktop"
sed -i "s|^Exec=.*|Exec=${pkgbase} %U|" "$newdesktop"
info "extracted ${newdesktop}"
else
warn "${desktop} not found inside the AppImage"
fi
if [ -f "${tmp}/squashfs-root/${icon}" ]; then
cp "${tmp}/squashfs-root/${icon}" "$newicon"
info "extracted ${newicon}"
else
warn "${icon} not found inside the AppImage"
fi
rm -rf "$(readlink -f "${tmp}/squashfs-root" 2>/dev/null || true)" "$tmp"
}
rc=0
for pkg in "$@"; do
( update_one "$pkg" ) || { rc=1; warn "${pkg} FAILED"; }
done
exit "$rc"