208 lines
9.3 KiB
Markdown
208 lines
9.3 KiB
Markdown
# RPM Auto Update
|
|
|
|
Nightly reconciler that keeps the `rpm/*` package repos and the `rpm` org's
|
|
Gitea package registry in sync with their upstream releases.
|
|
|
|
It mirrors [`aur/.autoupdate`](https://git.bjphoster.com/aur/.autoupdate): a new
|
|
upstream tag, a spec edited by hand and pushed to origin, and an upload that
|
|
failed last night all take the same path. On every pass the version-release in
|
|
the spec is compared with what the registry holds, and the registry is brought
|
|
in step.
|
|
|
|
## Layout
|
|
|
|
| file | role |
|
|
|---|---|
|
|
| `autoupdate.sh` | entry point: bump, build, upload, push, one package per process |
|
|
| `latest-version.sh` | resolves the latest upstream version (verbatim from `aur/.autoupdate`) |
|
|
| `common.sh` | logging, guards, version compare, tarball fetch, registry API |
|
|
| `dockerfile` | the Fedora toolchain image the nightly job runs in |
|
|
| `.gitea/workflows/update.yaml` | nightly matrix job |
|
|
| `.gitea/workflows/build-image.yaml` | monthly image rebuild |
|
|
|
|
Nothing here is package-specific. Everything that differs between packages
|
|
lives in the package repo itself.
|
|
|
|
## Package repo requirements
|
|
|
|
`rpm/<pkg>` carries:
|
|
|
|
- `<pkg>.spec` with literal `Version:` and `Release: N%{?dist}` lines and a
|
|
`%changelog` section. `Source0` should be the Gitea archive of the tag,
|
|
`https://git.bjphoster.com/source/<srcname>/archive/%{version}.tar.gz`,
|
|
unpacked with `%autosetup -n <srcname>` (the archive's top-level directory is
|
|
`<srcname>/`).
|
|
- `.autoupdate`, the same manifest the AUR packages use:
|
|
|
|
```sh
|
|
srchost="gitea" # gitea | github
|
|
srcmntr="source" # upstream owner/org
|
|
srcname="skeptical-updater" # upstream repo
|
|
verstrip="" # tag prefix to strip (tags are bare semver)
|
|
gui="false" # ignored here; AUR AppImage extraction only
|
|
```
|
|
|
|
Start a new package at a placeholder such as `0.0.1`: the first nightly run
|
|
bumps it to the newest tag.
|
|
|
|
## What a run does
|
|
|
|
1. Clone `https://git.bjphoster.com/rpm/<pkg>` and read `.autoupdate`.
|
|
2. Resolve the newest upstream tag (`latest-version.sh`: Gitea tags API,
|
|
`sort -V`). A *failed lookup* exits 1; "no new version" is not a failure.
|
|
3. If the tag is newer than `Version:`, rewrite the spec: `Version:` set to the
|
|
tag, `Release:` back to `1%{?dist}`, and a `%changelog` entry prepended:
|
|
|
|
```
|
|
* Sat Oct 10 2026 <GIT_NAME> <<GIT_EMAIL>> - X-1
|
|
- Updated to X
|
|
```
|
|
|
|
The rest of the file stays byte-identical, and `rpmspec` must read back
|
|
exactly `X-1` before anything is committed. The commit is
|
|
`Updated <pkg> to X-1`.
|
|
4. Ask the registry whether it already holds that version-release.
|
|
5. If it does not, download `https://git.bjphoster.com/source/<srcname>/archive/<X>.tar.gz`,
|
|
build with `rpmbuild -bb --define 'dist %{nil}'` as the unprivileged
|
|
`builder` user, run `rpmlint` (advisory, never fatal) and upload every built
|
|
`.rpm` to the registry.
|
|
6. Push to origin, last. The registry is the distribution channel and goes
|
|
first, as the AUR does in `aur/.autoupdate`: a failed build or upload leaves
|
|
origin untouched, and the next night recomputes everything.
|
|
|
|
Building with `dist` undefined makes the noarch package `<pkg>-X-1.noarch.rpm`
|
|
rather than `.fc43.noarch.rpm`, so the one file serves every RPM distribution
|
|
that points at the registry.
|
|
|
|
## Versions
|
|
|
|
`X-N`: everything before the dash is upstream, `-N` is the packaging revision
|
|
(`Release:`).
|
|
|
|
- newer upstream tag: `X-1`
|
|
- `force_rebuild` with no new tag: `X-(N+1)`, changelog `- Rebuilt X`
|
|
- an older tag (a tag deleted upstream) never downgrades the package
|
|
- a hand edit that bumps `Release:` and is pushed to origin is published on the
|
|
next run, because the registry lacks that version-release
|
|
|
|
## Environment and inputs
|
|
|
|
| workflow input | variable | effect |
|
|
|---|---|---|
|
|
| `package` | | run one matrix entry only (blank = all) |
|
|
| `force_rebuild` | `FORCE_REBUILD=1` | bump `Release` even with no new upstream version; replaces a registry copy of that exact version-release if one exists |
|
|
| `force_refresh` | `FORCE_REFRESH=1` | re-download the source and rebuild even when nothing changed (a check; uploads only what is missing) |
|
|
| `no_git_push` | `NO_GIT_PUSH=1` | dry run: build, but neither upload nor push to origin |
|
|
| | `NO_UPLOAD=1` / `--no-upload` | never contact the registry: no lookup, no upload; builds only when this run bumped the version |
|
|
| | `GIT_HOST` | git host and registry (default `git.bjphoster.com`) |
|
|
| | `GIT_NAME`, `GIT_EMAIL` | `%changelog` and commit identity (default: `git config`) |
|
|
| | `GIT_TOKEN` | registry token (default: the git credential helper, as in CI) |
|
|
| | `WORKDIR` | where to clone and build (default: a temp dir) |
|
|
| | `AUTOUPDATE_TEST_VERSION` | local test mode: use this as the upstream version |
|
|
| | `AUTOUPDATE_TEST_TARBALL` | local test mode: build from this local tarball |
|
|
| | `PKG_REPO_BASE` | clone package repos from here (default `https://$GIT_HOST/rpm`) |
|
|
|
|
Either `AUTOUPDATE_TEST_*` variable forces `NO_GIT_PUSH=1` and `NO_UPLOAD=1`,
|
|
so a test run can never publish anything.
|
|
|
|
## Setup
|
|
|
|
Required on the `rpm` org (Settings -> Actions -> Secrets / Variables), or on
|
|
the `.autoupdate` repo itself:
|
|
|
|
| kind | name | value |
|
|
|---|---|---|
|
|
| secret | `GIT_TOKEN` | access token with **write:repository** (clone and push `rpm/*`) and **write:package** (registry upload, image push, image pull) |
|
|
| variable | `GIT_NAME` | display name for commits and `%changelog`, e.g. `Bryan Joshua Pedini` |
|
|
| variable | `GIT_EMAIL` | email for commits and `%changelog` |
|
|
| variable | `REGISTRY_USER` | your Gitea **username** for the container registry login (not `GIT_NAME`, which has spaces) |
|
|
| variable | `GIT_HOST` | optional; defaults to `git.bjphoster.com` |
|
|
|
|
Also:
|
|
|
|
- enable **Actions** and **Packages** on `rpm/.autoupdate` (repo Settings ->
|
|
Advanced Settings), and make sure a runner with the `ubuntu-latest` label
|
|
serves the `rpm` org (an instance-wide runner does);
|
|
- the `rpm` org's registry (`/api/packages/rpm/...`) needs no switch of its
|
|
own: it answers whenever packages are enabled on the instance. Enabling
|
|
Packages on the package repos only adds the Packages tab and lets you link
|
|
the registry entry to the repo;
|
|
- build the image once by hand (Actions -> Build Autoupdater Image -> Run
|
|
workflow) before the first nightly run, since `update.yaml` pulls it;
|
|
- list every package in the `matrix.package` of `.gitea/workflows/update.yaml`.
|
|
A package repo that is not listed there is never processed.
|
|
|
|
The token is read back by `autoupdate.sh` through `git credential fill`, from the
|
|
`~/.git-credentials` the workflow writes, and handed to `curl` on stdin; it is
|
|
never on a command line or in a log.
|
|
|
|
## Running by hand
|
|
|
|
Actions -> Update RPM Packages -> Run workflow. Inputs: `package` (blank for
|
|
the whole matrix), `force_rebuild`, `force_refresh`, `no_git_push`.
|
|
|
|
## Installing packages
|
|
|
|
Fedora, RHEL and derivatives:
|
|
|
|
```sh
|
|
# dnf4 (EL 8/9/10; needs dnf-plugins-core)
|
|
sudo dnf config-manager --add-repo https://git.bjphoster.com/api/packages/rpm/rpm.repo
|
|
# dnf5 (Fedora 41+)
|
|
sudo dnf config-manager addrepo --from-repofile=https://git.bjphoster.com/api/packages/rpm/rpm.repo
|
|
|
|
sudo dnf install skeptical-updater
|
|
```
|
|
|
|
openSUSE:
|
|
|
|
```sh
|
|
sudo zypper addrepo https://git.bjphoster.com/api/packages/rpm/rpm.repo
|
|
sudo zypper refresh # accept the repository key when asked
|
|
sudo zypper install skeptical-updater
|
|
```
|
|
|
|
The `.repo` file enables `gpgcheck` against
|
|
`https://git.bjphoster.com/api/packages/rpm/rpm/repository.key`; Gitea signs
|
|
each package with that key on upload, so nothing is signed here.
|
|
|
|
## Container image
|
|
|
|
`dockerfile` builds the environment the nightly job runs in, published to the
|
|
Gitea registry as `git.bjphoster.com/rpm/autoupdate:latest`:
|
|
`fedora:latest` with `rpm-build`, `rpmdevtools`, `rpmlint`, `git`, `curl`,
|
|
`jq`, `util-linux` (`runuser`), `nodejs` (actions/checkout is a JavaScript
|
|
action) and a `builder` user.
|
|
|
|
`.gitea/workflows/build-image.yaml` rebuilds and overwrites `:latest` monthly,
|
|
on any push touching the `dockerfile`, and on demand. It needs the same
|
|
`REGISTRY_USER` and `GIT_TOKEN`.
|
|
|
|
## Testing locally
|
|
|
|
A dry run against a throwaway copy of the real package repo, with a forced
|
|
upstream version and a local tarball built exactly like Gitea's archive, and no
|
|
network at all. The test variables imply `NO_GIT_PUSH=1` and `NO_UPLOAD=1`.
|
|
|
|
```sh
|
|
docker build -t git.bjphoster.com/rpm/autoupdate:latest .
|
|
|
|
t="$(mktemp -d)"
|
|
git clone --quiet --bare https://git.bjphoster.com/rpm/skeptical-updater "$t/origin/skeptical-updater"
|
|
git -C ~/projects/sources/skeptical-updater archive --format=tar \
|
|
--prefix=skeptical-updater/ HEAD | gzip -n > "$t/src.tar.gz"
|
|
|
|
docker run --rm --network none -v "$PWD:/src:ro" -v "$t:/t:ro" \
|
|
git.bjphoster.com/rpm/autoupdate:latest \
|
|
bash -c 'cp -r /src /w && cp -r /t /tt && chown -R builder:builder /w /tt && cd /w &&
|
|
runuser -u builder -- env GIT_NAME="Your Name" GIT_EMAIL=you@example.com \
|
|
PKG_REPO_BASE=file:///tt/origin WORKDIR=/tt/work \
|
|
AUTOUPDATE_TEST_VERSION=0.1.0 AUTOUPDATE_TEST_TARBALL=/tt/src.tar.gz \
|
|
./autoupdate.sh skeptical-updater &&
|
|
runuser -u builder -- git -C /tt/work/skeptical-updater show --stat HEAD'
|
|
```
|
|
|
|
The built package lands in `/tt/work/.rpmbuild/skeptical-updater/RPMS/noarch/`
|
|
inside the container. Lint the scripts with
|
|
`docker run --rm -v "$PWD:/mnt:ro" -w /mnt koalaman/shellcheck:stable -x *.sh`.
|