304 lines
21 KiB
Markdown
304 lines
21 KiB
Markdown
# DPSAPI - Docker PS API
|
||
|
||
`dpsapi` is a small Go HTTP service exposing one authenticated REST endpoint, `GET /api/v1/containers`, which returns the Docker containers that are currently running (the equivalent of `docker ps`), read directly from the Docker Engine API on `/var/run/docker.sock`.
|
||
|
||
The same binary manages the API tokens and reads the audit log: with no arguments it runs the HTTP server in the foreground; with `token ...` arguments it performs a token operation against the SQLite database and exits; with `audit ...` arguments it prints audit rows and exits; `help` prints the usage text.
|
||
|
||
## Build and test
|
||
|
||
The makefile reads its build variables from `.vars` (create it with `./initialize.sh`, which copies `vars.example`).
|
||
|
||
| Command | Effect |
|
||
|---|---|
|
||
| `make build` | Static local binary `./dpsapi` (`CGO_ENABLED=0`, version and commit embedded) |
|
||
| `make test` | `go test ./...` |
|
||
| `make test-race` | `go test -race ./...` (needs cgo / a C compiler) |
|
||
| `make docker` | Builds the image `${CONTAINER_ORG}/${CONTAINER_IMAGE}:${APP_VERSION}` |
|
||
| `make test-e2e` | `make build`, then the Playwright suite in `e2e/` against the local binary (`--project=host`) |
|
||
| `make docker-e2e` | `make docker`, then the Playwright suite in `e2e/` against the image (`--project=image`, image reference exported as `DPSAPI_IMAGE`) |
|
||
| `make dockerpush` | Pushes the image |
|
||
| `make dockerrun` | Runs the image with the Docker socket and `./data` mounted |
|
||
| `make run` | `go run .` inside the Go builder image with the Docker socket and `./data` mounted |
|
||
| `make clean` | Removes `./dpsapi` and this project's images (`${CONTAINER_ORG}/${CONTAINER_IMAGE}:*` and dangling images labelled `dpsapi.project=${REPO_ORG}/${REPO_NAME}`) |
|
||
| `make version` (default) | Interactive: asks for a version, tags it in git, runs `make docker`, optionally pushes |
|
||
|
||
`APP_VERSION` defaults to `latest` and can be set from the environment (`APP_VERSION=1.0.0 make docker`). `COMMIT_ID` is the short `HEAD` hash, or `undefined` when the repository has no commits.
|
||
|
||
The image is built in two stages: `golang:1.27.1-alpine` compiles a static binary, and the final `scratch` stage contains only `/app` (no shell, no Docker CLI, no certificate bundle).
|
||
|
||
## Configuration
|
||
|
||
Configuration comes only from environment variables. It is validated before anything else happens, in both server and CLI mode.
|
||
|
||
| Variable | Required | Default | Accepted values |
|
||
|---|---|---|---|
|
||
| `DB_PATH` | yes | none | Path of the SQLite database file. Relative paths are resolved against the working directory. The parent directory must exist and be writable (SQLite also creates `-wal`/`-shm` files next to the database). |
|
||
| `HTTP_ADDR` | no | `0.0.0.0` | A literal IPv4 or IPv6 address (no hostnames, brackets, ports or zones). An IPv4 address listens on IPv4 only; an IPv6 address such as `::` listens dual-stack where the OS allows it. |
|
||
| `HTTP_PORT` | no | `80` | Decimal integer `1`–`65535`, ASCII digits only (no sign, no whitespace; leading zeros are accepted). |
|
||
| `LOG_LEVEL` | no | `INFO` | `ERROR`, `WARN`, `INFO`, `DEBUG`, `TRACE`, ASCII case-insensitive, exact match. |
|
||
|
||
A variable that is set but empty is an error for all four variables. Variables are checked in the order `DB_PATH`, `HTTP_ADDR`, `HTTP_PORT`, `LOG_LEVEL`; the first problem is reported. Configuration errors take precedence over command-line usage errors.
|
||
|
||
Configuration error messages (exit code `2`):
|
||
|
||
```text
|
||
dpsapi: configuration error: DB_PATH is required
|
||
dpsapi: configuration error: <VAR> must not be empty
|
||
dpsapi: configuration error: invalid HTTP_ADDR "<value>": must be an IPv4 or IPv6 address
|
||
dpsapi: configuration error: invalid HTTP_PORT "<value>": must be an integer between 1 and 65535
|
||
dpsapi: configuration error: invalid LOG_LEVEL "<value>": must be one of ERROR, WARN, INFO, DEBUG, TRACE
|
||
```
|
||
|
||
`<value>` is Go-quoted, so control characters are escaped.
|
||
|
||
## Database schema
|
||
|
||
The database is created on first use and the schema is applied idempotently (`CREATE TABLE IF NOT EXISTS`) every time the server or a CLI command opens it. Foreign-key enforcement, WAL journaling and a 5 s busy timeout are enabled on every connection, so the server and CLI commands can use the same file concurrently.
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS tokens (
|
||
token TEXT PRIMARY KEY NOT NULL,
|
||
description TEXT NOT NULL DEFAULT '',
|
||
active INTEGER NOT NULL CHECK (active IN (0, 1)),
|
||
created_at TEXT NOT NULL,
|
||
deleted_at TEXT NULL
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS operations (
|
||
token TEXT NOT NULL,
|
||
origin TEXT NOT NULL,
|
||
operation TEXT NOT NULL,
|
||
time TEXT NOT NULL,
|
||
FOREIGN KEY (token) REFERENCES tokens(token)
|
||
);
|
||
```
|
||
|
||
- All timestamps are UTC RFC3339 with nanoseconds as produced by Go's `time.RFC3339Nano` (trailing zeros of the fraction are dropped, e.g. `2026-10-05T20:03:15.9065Z`, `2026-01-01T00:00:00Z`).
|
||
- `deleted_at` is `NULL` while a token is active.
|
||
- Deleting a token sets `active = 0` and `deleted_at` in one transaction; the row is kept so audit rows keep referencing it.
|
||
- `operations` gets exactly one row per authenticated `GET /api/v1/containers` request, with `operation = 'GET containers'` and `origin` = the IP address of the TCP peer. CLI commands never write to `operations`; `audit` only reads it.
|
||
|
||
## Server
|
||
|
||
```sh
|
||
DB_PATH=/data/dpsapi.db ./dpsapi
|
||
```
|
||
|
||
The server listens on `${HTTP_ADDR}:${HTTP_PORT}` in the foreground. The Docker client is created lazily: a missing or inaccessible socket does not prevent startup, it makes each authenticated request fail with `500`.
|
||
|
||
On `SIGINT` or `SIGTERM` the server stops accepting connections, waits up to 15 s for in-flight requests, closes the database and exits `0`. If in-flight requests do not finish in time the exit code is `1` (`dpsapi: http server shutdown: context deadline exceeded`). A listen failure exits `1`, e.g.:
|
||
|
||
```text
|
||
dpsapi: listen tcp4 127.0.0.1:8080: bind: address already in use
|
||
```
|
||
|
||
HTTP server timeouts: read header 10 s, read 30 s, write 30 s, idle 120 s. Each Docker Engine call is bounded by 10 s.
|
||
|
||
## CLI
|
||
|
||
```sh
|
||
dpsapi help
|
||
dpsapi token list [all]
|
||
dpsapi token new [description]
|
||
dpsapi token delete <token>
|
||
dpsapi audit [all | n | n<unit>]
|
||
```
|
||
|
||
### Table output
|
||
|
||
`token list` and `audit` print a header line followed by one line per row, as a column-aligned table padded with spaces (Go's `text/tabwriter` with a padding of 2 and no tabs in the output):
|
||
|
||
- Every column except the last is as wide as its widest cell, header included, plus 2. Cell lengths are counted in Unicode code points (runes): not bytes, and not terminal display width (a CJK character or an emoji counts as 1).
|
||
- Each cell of those columns is followed by spaces up to the column width. The cell of the last column is written as-is, never padded, so no line has trailing whitespace. Every line ends with `\n`.
|
||
- Widths are computed per invocation over the printed lines only, so they change with the selected rows.
|
||
- With no rows only the header line is printed, each header cell padded by its own width plus 2 (`TOKEN DESCRIPTION ACTIVE CREATED_AT DELETED_AT`, `TIME TOKEN ORIGIN OPERATION`).
|
||
|
||
### `help`
|
||
|
||
Prints the usage text (shown below) to stdout and exits `0`; nothing is written to stderr except `DEBUG`/`TRACE` log records when `LOG_LEVEL` asks for them. The database is not opened or created. The configuration is still validated first, so a configuration error wins over `help`. `help` takes no arguments (`help <anything>` is a usage error) and is case-sensitive: `Help`, `-h` and `--help` are unknown commands.
|
||
|
||
### `token new [description]`
|
||
|
||
Generates a token (32 bytes from `crypto/rand`, base64url without padding: 43 characters from `[A-Za-z0-9_-]`), stores it as active, and prints exactly the token followed by a newline on stdout. The description is optional (default empty string) and is stored verbatim, byte for byte. Quote descriptions containing spaces; more than one argument is a usage error.
|
||
|
||
### `token list [all]`
|
||
|
||
`token list` prints the active tokens only; `token list all` prints every token, active and inactive. Both print a padded table (see [Table output](#table-output)) with one line per token, ordered by creation time then token. `token list all`:
|
||
|
||
```text
|
||
TOKEN DESCRIPTION ACTIVE CREATED_AT DELETED_AT
|
||
whEsbopp2tB2xXN7IC3LBnHhVge2F_-Gf910hIDL2pQ "smoke test" false 2026-10-05T20:03:15.894338161Z 2026-10-05T20:03:15.94012Z
|
||
S_muyryYKV3fegqvOSlEbHJ8ZvHz_s5tPov3dkY49Bo "" true 2026-10-05T20:05:01.042950671Z -
|
||
```
|
||
|
||
`token list` on the same database:
|
||
|
||
```text
|
||
TOKEN DESCRIPTION ACTIVE CREATED_AT DELETED_AT
|
||
S_muyryYKV3fegqvOSlEbHJ8ZvHz_s5tPov3dkY49Bo "" true 2026-10-05T20:05:01.042950671Z -
|
||
```
|
||
|
||
- `all` is the only accepted argument and is case-sensitive: any other argument (`ALL`, `--all`, an empty string, ...) is the usage error `unknown token list argument`; two or more arguments are the usage error `token list takes at most one argument`.
|
||
- `TOKEN` is the full 43-character token, so with at least one row the column is always 45 characters wide.
|
||
- `DESCRIPTION` is rendered with Go's `strconv.Quote`, so every row is exactly one line whatever the description contains (`"line1\nline2"`, `"tab\there"`, `"\xff"`). Printable non-ASCII characters are kept as-is and count as one character each for the padding.
|
||
- `ACTIVE` is `true` or `false` (always `true` without `all`).
|
||
- `DELETED_AT` is `-` while the token is active.
|
||
- With no (matching) tokens only the header line is printed.
|
||
|
||
### `token delete <token>`
|
||
|
||
Deactivates the token. Prints nothing on success. The token argument is matched byte-exact (case-sensitive, no trimming).
|
||
|
||
### `audit [all | n | n<unit>]`
|
||
|
||
Prints audit rows (the `operations` table) as a padded table (see [Table output](#table-output)) with the columns `TIME`, `TOKEN`, `ORIGIN` and `OPERATION`, one line per row, oldest first and newest last (like `tail`):
|
||
|
||
```text
|
||
TIME TOKEN ORIGIN OPERATION
|
||
2026-10-05T22:36:01.897010677Z hr9709HRvMYCsbHSwvWqqLJGUFLyjJtL8VTiQ91k1uU 127.0.0.1 GET containers
|
||
2026-10-05T22:36:01.910494251Z hMsqw73OrD3LiWZAm5B0yAoASaph5Kn3gqAQEhUfX0s 127.0.0.1 GET containers
|
||
```
|
||
|
||
| Invocation | Rows printed |
|
||
|---|---|
|
||
| `audit` | The latest 10 rows |
|
||
| `audit <n>` | The latest `n` rows (all rows if fewer exist) |
|
||
| `audit <n><unit>` | Every row whose time is at or after now − `n` × unit (the boundary is inclusive). Units: `s` seconds, `m` minutes, `h` hours, `d` days (24 h), `w` weeks (7 d). |
|
||
| `audit all` | Every row |
|
||
|
||
- `n` is ASCII decimal digits only and at least `1`; leading zeros are accepted (`010` = `10`, `01h` = `1h`). Unit letters are lowercase only.
|
||
- The largest accepted values are `9223372036854775807` rows and a window of about 292 years (`9223372036s`, `153722867m`, `2562047h`, `106751d`, `15250w`).
|
||
- `all` is the only accepted word, and only in lowercase: `ALL`, `All`, `--all`, `alls`, `all1`, `1all` or `all` surrounded by whitespace are not accepted.
|
||
- Anything else is a usage error (`invalid audit argument`, or `audit takes at most one argument` for two or more arguments, including `audit all <anything>`): `0` (also `0h`), signs, whitespace, decimals or other number syntaxes, an unknown or uppercase unit, a unit without a number, any word other than lowercase `all`, an empty argument, or a value above those limits.
|
||
- "Latest" and the printed order use the stored timestamp as a point in time (not its text), with rows stored at the same instant kept in insertion order. The window has no upper bound: a row with a time later than now (clock skew) is included.
|
||
- `TIME` and `TOKEN` are printed exactly as stored. `TOKEN` is the full token value, as in `token list`, so the token can be matched to its row there. Treat the output as sensitive. `ORIGIN` is the TCP peer IP address and `OPERATION` is always `GET containers` (it is the last column, so its space is never followed by padding). `TIME` values have variable length (the fraction is stored without trailing zeros), so the `TIME` column width depends on the printed rows.
|
||
- With no matching rows only the header line is printed (exit `0`).
|
||
- The command only reads: it never writes to `tokens` or `operations`. Like every command, it creates the database and the empty schema if `DB_PATH` does not exist yet.
|
||
- A row whose stored time cannot be parsed makes the command fail with exit `1` (`dpsapi: read operation row: parse stored timestamp: ...`) rather than print the rows in the wrong order.
|
||
|
||
### Exit codes and output
|
||
|
||
| Exit code | Meaning |
|
||
|---|---|
|
||
| `0` | Success (including graceful server shutdown and `help`) |
|
||
| `1` | Any runtime failure: database errors (including an unparseable stored audit time), `token not found`, `token already inactive`, listen failure, server failure, shutdown failure |
|
||
| `2` | Usage errors and configuration errors |
|
||
|
||
- Command results go to stdout (for `help`, the usage text); nothing else is ever written to stdout.
|
||
- Errors are a single line on stderr prefixed with `dpsapi: `; it is the last line written to stderr, except for usage errors, where the usage text follows it. The supplied token or audit argument is never echoed.
|
||
- Usage errors also print the usage text to stderr, after the error line. Usage errors do not open or create the database; neither does `help`.
|
||
|
||
```text
|
||
dpsapi: token not found
|
||
dpsapi: token already inactive
|
||
dpsapi: usage error: unknown command
|
||
dpsapi: usage error: help takes no arguments
|
||
dpsapi: usage error: missing token subcommand
|
||
dpsapi: usage error: unknown token subcommand
|
||
dpsapi: usage error: token list takes at most one argument
|
||
dpsapi: usage error: unknown token list argument
|
||
dpsapi: usage error: token new takes at most one argument
|
||
dpsapi: usage error: token delete requires exactly one argument
|
||
dpsapi: usage error: audit takes at most one argument
|
||
dpsapi: usage error: invalid audit argument
|
||
```
|
||
|
||
Usage text (printed to stderr after a usage error and to stdout by `help`, byte for byte the same):
|
||
|
||
```text
|
||
usage:
|
||
dpsapi start the HTTP server
|
||
dpsapi help print this usage text
|
||
dpsapi token list [all] list the active tokens, or every token with "all"
|
||
dpsapi token new [description] create a token and print it
|
||
dpsapi token delete <token> deactivate a token
|
||
dpsapi audit print the latest 10 audit rows
|
||
dpsapi audit <n> print the latest n audit rows
|
||
dpsapi audit <n><unit> print the audit rows of the last n units (s, m, h, d, w)
|
||
dpsapi audit all print every audit row
|
||
```
|
||
|
||
Commands, subcommands and the `all` arguments (of `token list` and `audit`) are case-sensitive. Use `dpsapi help` to print the usage text: there is no `-h`/`--help` flag (`-h`, `--help` and `Help` are unknown commands).
|
||
|
||
## HTTP API
|
||
|
||
```http
|
||
GET /api/v1/containers
|
||
Authorization: Bearer <token>
|
||
```
|
||
|
||
Success: `200 OK`, `Content-Type: application/json`, body = JSON array of Docker Engine container-summary objects (the `GET /containers/json` objects, `Id`, `Names`, `Image`, `State`, ...) for running containers only; `[]` when none are running. Bodies have an explicit `Content-Length` and no trailing newline.
|
||
|
||
| Status | Body | When |
|
||
|---|---|---|
|
||
| `401` | `{"error":"unauthorized"}` | Missing/empty/malformed `Authorization`, more than one `Authorization` header, unknown token, inactive token. Also sets `WWW-Authenticate: Bearer`. |
|
||
| `404` | `{"error":"not found"}` | Any path other than exactly `/api/v1/containers` (including `/api/v1/containers/`, `//api/v1/containers`, `/api/v1/./containers`, different case). Paths are not cleaned and never redirected. |
|
||
| `405` | `{"error":"method not allowed"}` | Any method other than `GET` (including `HEAD`) on `/api/v1/containers`. Also sets `Allow: GET`. |
|
||
| `500` | `{"error":"internal server error"}` | Token lookup failed, audit insert failed, Docker Engine request failed, or response serialization failed. |
|
||
|
||
All error responses have `Content-Type: application/json`. Internal error details are only logged.
|
||
|
||
Authentication rules:
|
||
|
||
- Exactly one `Authorization` header of the form `Bearer` + one space + token. The scheme is ASCII case-insensitive (`bearer`, `BEARER`); the token is compared byte-exact against the database (case-sensitive, no trimming: an extra space before the token makes it a different, unknown token). Leading and trailing whitespace of the whole header value is removed by HTTP parsing itself (RFC 9110), so `Bearer <token> ` authenticates.
|
||
- Processing order: authentication, then the audit row is inserted, then Docker is queried. An authenticated request is therefore audited even when Docker fails; unauthenticated, `404` and `405` requests are never audited.
|
||
- The audited origin is the IP address of the TCP connection. `X-Forwarded-For`, `Forwarded`, `X-Real-IP` and similar headers are ignored.
|
||
- Query strings are ignored (`/api/v1/containers?x=1` is the same endpoint). Percent-encoded unreserved characters are decoded before matching (`/api/v1/%63ontainers` matches); a percent-encoded slash is not a path separator (`/api/v1%2Fcontainers` is `404`).
|
||
|
||
Requests rejected by Go's HTTP server before they reach the application (malformed request lines, headers larger than 1 MiB → `431`, `OPTIONS *`) get Go's default plain-text responses and are not logged.
|
||
|
||
Example:
|
||
|
||
```sh
|
||
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8080/api/v1/containers
|
||
```
|
||
|
||
## Logging
|
||
|
||
Logs are written to stderr with Go's `log/slog` text handler, one record per line, timestamps in UTC with millisecond precision:
|
||
|
||
```text
|
||
time=2026-10-05T20:03:15.908Z level=INFO msg="this request has been made" method=GET path=/api/v1/containers status=200 duration=1.994499ms origin=127.0.0.1
|
||
```
|
||
|
||
Levels, from most to least severe: `ERROR`, `WARN`, `INFO`, `DEBUG`, `TRACE`. `LOG_LEVEL` sets the minimum emitted level.
|
||
|
||
- `ERROR`: request failures (`token lookup failed`, `audit insert failed`, `docker request failed`, `response serialization failed`). With no errors the server is silent at this level.
|
||
- `WARN`: messages from Go's HTTP server (e.g. TLS/accept problems).
|
||
- `INFO`: `server listening` (with `addr`, `version`, `commit`), `server stopped`, and for every request reaching the application, including `401`/`404`/`405`, the message `this request has been made` with `method`, `path`, `status`, `duration` (Go duration format) and `origin`. Successful CLI commands log nothing at `INFO`.
|
||
- `DEBUG`: configuration resolution, dispatch (`mode=server|cli`, `cli dispatch command="token ..."` (`command="token list"` with or without `all`), `cli dispatch command=audit limit=<n>|window=<duration>|all=true`, `cli dispatch command=help`), database open/schema initialization/close, route registration, Docker client creation, authentication result class (`result=missing_header|empty_header|multiple_headers|bad_scheme|empty_token|unknown_token|inactive_token|lookup_error|ok`), audit insertion, Docker request start/finish, token created/deactivated, audit window start (`audit window resolved since=...`), audit result count (`audit rows listed rows=<n>`), shutdown.
|
||
- `TRACE`: `function entry`/`function exit` records (`func=...`) for the application functions on the execution path, and row-level database records (`row inserted`, `row read`, `row updated`, `row not found`, `row already inactive`, `container listed`). `audit` logs a `row read` record for every `operations` row it reads, including rows outside the selection.
|
||
|
||
Token secrets are never logged. Records that concern a token carry `token_id`, the first 8 hex characters of the token's SHA-256 digest. The `Authorization` header and raw request headers are never logged. Descriptions are logged at `TRACE`. Full tokens appear only on stdout, from `token new`, `token list [all]` and `audit`.
|
||
|
||
## Docker
|
||
|
||
The container must have the Docker socket mounted; the application talks to `unix:///var/run/docker.sock` directly and never runs a `docker` executable. The SQLite database must live on a persistent mount.
|
||
|
||
```sh
|
||
docker run -d --name dpsapi \
|
||
-p 127.0.0.1:8080:80 \
|
||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||
-v /srv/dpsapi:/data \
|
||
-e DB_PATH=/data/dpsapi.db \
|
||
git.bjphoster.com/source/dpsapi:latest
|
||
```
|
||
|
||
Optional: `-e HTTP_ADDR=0.0.0.0 -e HTTP_PORT=80 -e LOG_LEVEL=INFO`.
|
||
|
||
The process runs as root inside the container so it can open the socket and write to the data directory.
|
||
|
||
Token management and audit inspection inside the running container (`docker exec` inherits the container's environment, so `DB_PATH` and `LOG_LEVEL` apply):
|
||
|
||
```sh
|
||
docker exec dpsapi /app help
|
||
docker exec dpsapi /app token new "ci pipeline"
|
||
docker exec dpsapi /app token list
|
||
docker exec dpsapi /app token list all
|
||
docker exec dpsapi /app token delete <token>
|
||
docker exec dpsapi /app audit
|
||
docker exec dpsapi /app audit 50
|
||
docker exec dpsapi /app audit 1d
|
||
docker exec dpsapi /app audit all
|
||
```
|