deb/.autoupdate
1
0 Fork 0
Code Issues Pull Requests Actions Packages Releases Activity

feat: add the nightly deb reconciler

Browse Source
This commit is contained in:
bryanpedini committed 2026-10-10 11:09:04 +02:00
1 parent 4f730b914e
commit f340028aae
5 files changed
+786 -1
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+206 -1
View File
@@ -1 +1,206 @@
# DEB Auto Update Workflows
# DEB Auto Update
Nightly reconciler that keeps the `deb/*` package repos and the `deb` org's
Gitea package registry in sync with their upstream releases.
It mirrors [`aur/.autoupdate`](https://git.bjphoster.com/aur/.autoupdate): a new
upstream tag, a `debian/` edited by hand and pushed to origin, and an upload
that failed last night all take the same path. On every pass the version in
`debian/changelog` is compared with what the registry holds, and the registry
is brought in step.
## Layout
| file | role |
|---|---|
| `autoupdate.sh` | entry point: bump, build, upload, push, one package per process |
| `latest-version.sh` | resolves the latest upstream version (verbatim from `aur/.autoupdate`) |
| `common.sh` | logging, guards, version compare, tarball fetch, registry API |
| `dockerfile` | the Debian toolchain image the nightly job runs in |
| `.gitea/workflows/update.yaml` | nightly matrix job |
| `.gitea/workflows/build-image.yaml` | monthly image rebuild |
Nothing here is package-specific. Everything that differs between packages
lives in the package repo itself.
## Package repo requirements
`deb/<pkg>` carries:
- `debian/` (at the top of the repo): `changelog` whose top version is
`<upstream>-<integer revision>`, `control`, `rules`, `source/format`, and so
on. It is dropped into the unpacked upstream tarball at build time, replacing
any `debian/` upstream might ship.
- `.autoupdate`, the same manifest the AUR packages use:
```sh
srchost="gitea" # gitea | github
srcmntr="source" # upstream owner/org
srcname="skeptical-updater" # upstream repo
verstrip="" # tag prefix to strip (tags are bare semver)
gui="false" # ignored here; AUR AppImage extraction only
```
Start a new package at a placeholder such as `0.0.1-1`: the first nightly run
bumps it to the newest tag.
## What a run does
1. Clone `https://git.bjphoster.com/deb/<pkg>` and read `.autoupdate`.
2. Resolve the newest upstream tag (`latest-version.sh`: Gitea tags API,
`sort -V`). A *failed lookup* exits 1; "no new version" is not a failure.
3. If the tag is newer than the changelog's upstream version, prepend a stanza
in exactly the format `dch --newversion X-1 --distribution stable` writes:
```
<source> (X-1) stable; urgency=medium
* Updated to X
-- <GIT_NAME> <<GIT_EMAIL>> Sat, 10 Oct 2026 00:00:00 +0000
```
Nothing below it changes, and `dpkg-parsechangelog` must read back exactly
`X-1` and parse every stanza without a warning before anything is
committed. The commit is `Updated <pkg> to X-1`.
4. Ask the registry whether it already holds that version.
5. If it does not, download `https://git.bjphoster.com/source/<srcname>/archive/<X>.tar.gz`,
place it as `<source>_<X>.orig.tar.gz` next to the unpacked tree, build with
`dpkg-buildpackage -us -uc -b` as the unprivileged `builder` user, run
`lintian` (advisory, never fatal) and upload every built `.deb` to the
`stable` distribution, `main` component.
6. Push to origin, last. The registry is the distribution channel and goes
first, as the AUR does in `aur/.autoupdate`: a failed build or upload leaves
origin untouched, and the next night recomputes everything.
The image is Debian bookworm: a package built against the older dpkg and
debhelper installs on bookworm, trixie and Ubuntu alike.
## Versions
`X-N`: everything before the dash is upstream, `-N` is the Debian revision.
- newer upstream tag: `X-1`
- `force_rebuild` with no new tag: `X-(N+1)`, changelog `* Rebuilt X`
- an older tag (a tag deleted upstream) never downgrades the package
- a hand edit that adds a changelog stanza and is pushed to origin is published
on the next run, because the registry lacks that version
- an epoch (`1:X-N`) is carried over unchanged
## Environment and inputs
| workflow input | variable | effect |
|---|---|---|
| `package` | | run one matrix entry only (blank = all) |
| `force_rebuild` | `FORCE_REBUILD=1` | bump the Debian revision even with no new upstream version; replaces a registry copy of that exact version if one exists |
| `force_refresh` | `FORCE_REFRESH=1` | re-download the source and rebuild even when nothing changed (a check; uploads only what is missing) |
| `no_git_push` | `NO_GIT_PUSH=1` | dry run: build, but neither upload nor push to origin |
| | `NO_UPLOAD=1` / `--no-upload` | never contact the registry: no lookup, no upload; builds only when this run bumped the version |
| | `DEB_DISTRIBUTION` | registry distribution and changelog distribution (default `stable`) |
| | `DEB_COMPONENT` | registry component (default `main`) |
| | `GIT_HOST` | git host and registry (default `git.bjphoster.com`) |
| | `GIT_NAME`, `GIT_EMAIL` | changelog and commit identity (default: `git config`) |
| | `GIT_TOKEN` | registry token (default: the git credential helper, as in CI) |
| | `WORKDIR` | where to clone and build (default: a temp dir) |
| | `AUTOUPDATE_TEST_VERSION` | local test mode: use this as the upstream version |
| | `AUTOUPDATE_TEST_TARBALL` | local test mode: build from this local tarball |
| | `PKG_REPO_BASE` | clone package repos from here (default `https://$GIT_HOST/deb`) |
Either `AUTOUPDATE_TEST_*` variable forces `NO_GIT_PUSH=1` and `NO_UPLOAD=1`,
so a test run can never publish anything.
## Setup
Required on the `deb` org (Settings -> Actions -> Secrets / Variables), or on
the `.autoupdate` repo itself:
| kind | name | value |
|---|---|---|
| secret | `GIT_TOKEN` | access token with **write:repository** (clone and push `deb/*`) and **write:package** (registry upload, image push, image pull) |
| variable | `GIT_NAME` | display name for commits and the changelog, e.g. `Bryan Joshua Pedini` |
| variable | `GIT_EMAIL` | email for commits and the changelog |
| variable | `REGISTRY_USER` | your Gitea **username** for the container registry login (not `GIT_NAME`, which has spaces) |
| variable | `GIT_HOST` | optional; defaults to `git.bjphoster.com` |
Also:
- enable **Actions** and **Packages** on `deb/.autoupdate` (repo Settings ->
Advanced Settings), and make sure a runner with the `ubuntu-latest` label
serves the `deb` org (an instance-wide runner does);
- the `deb` org's registry (`/api/packages/deb/...`) needs no switch of its
own: it answers whenever packages are enabled on the instance. Enabling
Packages on the package repos only adds the Packages tab and lets you link
the registry entry to the repo;
- build the image once by hand (Actions -> Build Autoupdater Image -> Run
workflow) before the first nightly run, since `update.yaml` pulls it;
- list every package in the `matrix.package` of `.gitea/workflows/update.yaml`.
A package repo that is not listed there is never processed.
The token is read back by `autoupdate.sh` through `git credential fill`, from the
`~/.git-credentials` the workflow writes, and handed to `curl` on stdin; it is
never on a command line or in a log.
## Running by hand
Actions -> Update Debian Packages -> Run workflow. Inputs: `package` (blank for
the whole matrix), `force_rebuild`, `force_refresh`, `no_git_push`.
## Installing packages
Debian, Ubuntu and derivatives:
```sh
sudo install -d -m 0755 /etc/apt/keyrings
sudo curl -fsSL https://git.bjphoster.com/api/packages/deb/debian/repository.key \
-o /etc/apt/keyrings/git.bjphoster.com-deb.asc
echo "deb [signed-by=/etc/apt/keyrings/git.bjphoster.com-deb.asc] https://git.bjphoster.com/api/packages/deb/debian stable main" \
| sudo tee /etc/apt/sources.list.d/git.bjphoster.com-deb.list
sudo apt update
sudo apt install skeptical-updater
```
The bare source line is
`deb https://git.bjphoster.com/api/packages/deb/debian stable main`; Gitea signs
the repository's `Release` files with
`https://git.bjphoster.com/api/packages/deb/debian/repository.key`.
## Container image
`dockerfile` builds the environment the nightly job runs in, published to the
Gitea registry as `git.bjphoster.com/deb/autoupdate:latest`:
`debian:bookworm-slim` with `dpkg-dev`, `debhelper`, `build-essential`,
`fakeroot`, `devscripts`, `lintian`, `git`, `curl`, `jq`, `ca-certificates`,
`nodejs` (actions/checkout is a JavaScript action) and a `builder` user.
`.gitea/workflows/build-image.yaml` rebuilds and overwrites `:latest` monthly,
on any push touching the `dockerfile`, and on demand. It needs the same
`REGISTRY_USER` and `GIT_TOKEN`.
## Testing locally
A dry run against a throwaway copy of the real package repo, with a forced
upstream version and a local tarball built exactly like Gitea's archive, and no
network at all. The test variables imply `NO_GIT_PUSH=1` and `NO_UPLOAD=1`.
```sh
docker build -t git.bjphoster.com/deb/autoupdate:latest .
t="$(mktemp -d)"
git clone --quiet --bare https://git.bjphoster.com/deb/skeptical-updater "$t/origin/skeptical-updater"
git -C ~/projects/sources/skeptical-updater archive --format=tar \
--prefix=skeptical-updater/ HEAD | gzip -n > "$t/src.tar.gz"
docker run --rm --network none -v "$PWD:/src:ro" -v "$t:/t:ro" \
git.bjphoster.com/deb/autoupdate:latest \
bash -c 'cp -r /src /w && cp -r /t /tt && chown -R builder:builder /w /tt && cd /w &&
runuser -u builder -- env GIT_NAME="Your Name" GIT_EMAIL=you@example.com \
PKG_REPO_BASE=file:///tt/origin WORKDIR=/tt/work \
AUTOUPDATE_TEST_VERSION=0.1.0 AUTOUPDATE_TEST_TARBALL=/tt/src.tar.gz \
./autoupdate.sh skeptical-updater &&
runuser -u builder -- git -C /tt/work/skeptical-updater show --stat HEAD'
```
The built package lands in `/tt/work/.debbuild/skeptical-updater/` inside the
container. Lint the scripts with
`docker run --rm -v "$PWD:/mnt:ro" -w /mnt koalaman/shellcheck:stable -x *.sh`.