You've already forked .autoupdate
All six nightly jobs ran the same pacman transaction before doing any work. The toolchain now lives in an image published to the Gitea registry as git.bjphoster.com/aur/autoupdate:latest, rebuilt monthly since Arch is rolling and a month-old image is a month behind. The build job itself runs on plain ubuntu-latest -- it only drives docker. Only the image is Arch. Based on archlinux:base rather than base-devel: the autoupdater parses PKGBUILDs and never compiles, so the toolchain is dead weight, 822 MB against 1.48 GB. nodejs is included because actions/checkout is a JavaScript action the runner executes with node from inside the image. Slimming to base initially dropped diffutils, and apply_pkgrel_rule used diff -q. A missing binary makes the `if` fail, which reads as "changed", so pkgrel ratcheted upward on every run -- the same silent-wrong-default class as the empty .SRCINFO this rewrite was about, and invisible to the host test run. The comparison is now a shell string equality with no external dependency, and is verified against an image with no diff at all. require_tools() additionally fails on startup for any missing binary rather than midway through.
171 lines
7.2 KiB
Markdown
171 lines
7.2 KiB
Markdown
# AUR Auto Update
|
|
|
|
Nightly reconciler that keeps the `aur/*` package repos and their AUR
|
|
counterparts in sync.
|
|
|
|
This is a **reconciler**, not a version-bump script. A new upstream release is
|
|
only one of the things that can make a package diverge from the AUR; a manual
|
|
push to origin is another, and a push that failed last night is a third. All
|
|
three take the same path, and the AUR reconcile runs on every pass.
|
|
|
|
## Layout
|
|
|
|
| file | role |
|
|
|---|---|
|
|
| `autoupdate.sh` | entry point; orchestrates one or more packages |
|
|
| `latest-version.sh` | resolves the latest upstream version (GitHub / Gitea) |
|
|
| `update-checksums.sh` | downloads sources, computes sha256, rewrites the arrays |
|
|
| `sync-aur.sh` | reconciles the repo against the AUR |
|
|
| `common.sh` | logging, guards, PKGBUILD parsing and array splicing |
|
|
| `test.sh` | hermetic test suite |
|
|
| `.gitea/workflows/update.yaml` | nightly matrix job |
|
|
|
|
Nothing here is package-specific. Everything that differs between packages
|
|
lives in a `.autoupdate` manifest committed to the package repo itself.
|
|
|
|
## Per-package manifest
|
|
|
|
Each package repo carries a `.autoupdate` file:
|
|
|
|
```sh
|
|
srchost="github" # github | gitea
|
|
srcmntr="Sunhaiy" # upstream owner/org
|
|
srcname="Reflex" # upstream repo
|
|
verstrip="v" # tag prefix to strip ("app-v" for open-video-downloader-*)
|
|
gui="true" # extract .desktop + icon from the AppImage
|
|
desktop="reflex.desktop" # name *inside* the AppImage
|
|
icon="reflex.png"
|
|
```
|
|
|
|
It rides along to the AUR on push, as `README.md` and `LICENSE` already do.
|
|
Harmless: the AUR reads only `PKGBUILD` and `.SRCINFO`.
|
|
|
|
## What a run does
|
|
|
|
1. Clone the package repo, read its manifest.
|
|
2. Fetch the AUR over **HTTPS** as the reference state. This happens before
|
|
anything else, because `pkgrel` is derived from it. Read-only, so no SSH key
|
|
is required for a dry run.
|
|
3. Resolve the upstream version. A *failed lookup* exits 1; "no new version" is
|
|
not a failure.
|
|
4. If the version moved, re-download and re-hash the remote sources.
|
|
5. Always re-hash local sources (`.desktop`, icons) and regenerate `.SRCINFO`.
|
|
Local hashing needs no network, so this is free, and it is what lets a
|
|
hand-edited repo self-correct.
|
|
6. Derive `pkgrel` (below).
|
|
7. Validate `.SRCINFO`; abort **before** committing if it is empty or disagrees
|
|
with the PKGBUILD.
|
|
8. Commit only if something actually changed.
|
|
9. Push to the AUR first, then origin, so a failed AUR push leaves origin clean.
|
|
|
|
## pkgrel
|
|
|
|
`X.Y.Z-N` — everything before the dash is upstream, `-N` is packaging-only
|
|
revisions. You should never need to touch it by hand.
|
|
|
|
`pkgrel` is derived from **the AUR**, because that is what `-N` is numbered
|
|
against:
|
|
|
|
- version differs from the published one → `pkgrel = 1`
|
|
- same version, packaging content differs → `pkgrel = AUR's + 1`
|
|
- same version, packaging identical → unchanged
|
|
- you already bumped it yourself → respected, not bumped again
|
|
|
|
Using the AUR as the reference rather than local history is what makes this
|
|
idempotent: once the bump is published, the next night sees no difference and
|
|
does nothing. There is a test for exactly that (`pkgrel stable on re-run`).
|
|
|
|
An edit to a local source such as an icon counts as a packaging change, because
|
|
its `sha256sums` entry is regenerated first. A README-only edit does not.
|
|
|
|
## Environment
|
|
|
|
| variable | effect |
|
|
|---|---|
|
|
| `NO_GIT_PUSH=1` | dry run: do everything except push |
|
|
| `FORCE_REBUILD=1` | bump `pkgrel` even with no detected change |
|
|
| `FORCE_REFRESH=1` | re-download remote sources without a version change |
|
|
| `AUR_FORCE=1` | force-push over diverged AUR history (see below) |
|
|
| `GIT_HOST` | git host holding the package repos |
|
|
| `GITHUB_TOKEN` | optional; avoids GitHub's 60/hour anonymous rate limit |
|
|
|
|
A rejected AUR push usually means someone pushed there directly. The run fails
|
|
loudly rather than overwriting it; inspect, then re-run with `AUR_FORCE=1`.
|
|
|
|
## Exit codes
|
|
|
|
"No new upstream version" is the overwhelmingly common outcome and exits **0**.
|
|
The trap the previous implementation fell into was giving that same silent
|
|
`exit 0` to a lookup that had *broken* — which is how `pman-helper` shipped
|
|
`pkgver=""` unnoticed. A failed lookup now exits **1**.
|
|
|
|
## Requirements
|
|
|
|
Needs a genuine Arch `makepkg`. Ubuntu's build environment emits an empty
|
|
`.SRCINFO`, which the AUR rejects — this silently broke three packages. The CI
|
|
job therefore runs in an Arch container as a non-root user (`makepkg` refuses to
|
|
run as root), and `regen_srcinfo` in `common.sh` fails loudly if the output is
|
|
ever empty again.
|
|
|
|
`autoupdate.sh` calls `require_tools` on startup, so a missing binary fails
|
|
immediately instead of taking a silently wrong branch further in.
|
|
|
|
## Container image
|
|
|
|
`dockerfile` builds the environment the nightly job runs in, published to the
|
|
Gitea registry as `git.bjphoster.com/aur/autoupdate:latest`. Without it, all six
|
|
matrix jobs would run a full `pacman` transaction every night.
|
|
|
|
`.gitea/workflows/build-image.yaml` rebuilds and overwrites `:latest` monthly
|
|
(Arch is rolling, so a month-old image is a month behind), on any push touching
|
|
the `dockerfile`, and on demand via `workflow_dispatch`. That job runs on plain
|
|
`ubuntu-latest` — it only drives `docker`; the *image* is the Arch part.
|
|
|
|
Based on `archlinux:base`, not `base-devel`: the autoupdater only ever parses
|
|
PKGBUILDs and never compiles, so the toolchain is dead weight — 822 MB against
|
|
1.48 GB. Add it back if a real build or verify step is ever introduced.
|
|
|
|
Building locally:
|
|
|
|
```sh
|
|
docker build -t git.bjphoster.com/aur/autoupdate:latest .
|
|
docker run --rm -v "$PWD:/src:ro" git.bjphoster.com/aur/autoupdate:latest \
|
|
bash -c 'cp -r /src /w && chown -R builder:builder /w && cd /w &&
|
|
runuser -u builder -- ./test.sh'
|
|
```
|
|
|
|
Running the suite inside the image is worth doing after any `dockerfile` change.
|
|
Trimming to `archlinux:base` originally dropped `diffutils`, and the missing
|
|
`diff` made every run look like a packaging change and ratchet `pkgrel` upward —
|
|
the host tests passed throughout. `pkgrel` no longer depends on `diff` at all,
|
|
but the lesson stands.
|
|
|
|
### Registry credentials
|
|
|
|
Needs `vars.REGISTRY_USER` — your Gitea **username**, not `vars.GIT_NAME`, which
|
|
is the display name used for commit authorship and contains spaces. The token is
|
|
the existing `secrets.GIT_TOKEN`, which needs `write:package` scope.
|
|
|
|
## Notes on checksums
|
|
|
|
sha256 only; `sha1sums`/`md5sums` are actively removed.
|
|
|
|
Checksums are computed directly rather than via `makepkg -g` / `updpkgsums`.
|
|
Those generate every arch array in one pass and download all arches into a
|
|
single directory, so when two arches rename to the same target
|
|
(`foo.AppImage::…-amd64`, `foo.AppImage::…-arm64`) the second finds the first
|
|
one's file already there and emits an identical, silently wrong checksum.
|
|
Overriding `CARCH` does not help. Downloads here go to content-addressed cache
|
|
paths instead, never to the `::` rename target. Multi-arch is supported for
|
|
when a package needs it; single-arch is the degenerate case of the same loop.
|
|
|
|
## Tests
|
|
|
|
```sh
|
|
./test.sh # all
|
|
./test.sh pkgrel # filter by name
|
|
```
|
|
|
|
Runs against local bare repos and a local HTTP server — no network, and it
|
|
never touches the real AUR. Requires a real `makepkg`.
|