dpsapi/e2e

dpsapi E2E suite (Playwright)

API/process-only Playwright suite (no browsers) exercising the compiled dpsapi binary and the final image.

Running

make test-e2e                      # make build, then --project=host
make docker-e2e                    # make docker, then --project=image with DPSAPI_IMAGE set
cd e2e && npm ci
npx playwright test --project=host
DPSAPI_IMAGE=git.bjphoster.com/source/dpsapi:dev npx playwright test --project=image
npx playwright test --project=host tests/host/auth.spec.ts -g 'duplicated'

Variable	Meaning
DPSAPI_BIN	Binary under test (relative paths resolve against e2e/). Default ../dpsapi; only the default is (re)built by host-setup via make build.
DPSAPI_E2E_SKIP_BUILD=1	Skip that build (set by make test-e2e, which already built).
DPSAPI_IMAGE	Image under test for --project=image (required there).
DPSAPI_E2E_SEED	Run seed for all PRNG streams (default dpsapi-e2e-default-seed-1, printed at start).
DPSAPI_E2E_WORKERS	Worker count (default 4).
DPSAPI_E2E_ALPINE	Image for test/helper containers (default alpine:3.22, must exist locally).

Prerequisites (checked by the setup projects, failing fast with [harness:prerequisite]): curl, unprivileged unshare -Urm with bind/tmpfs mounts, a reachable Docker daemon, alpine:3.22, and the binary (host) or the image (image).

Projects: host-setup → host (tests/host/), image-setup → image (tests/image/). Global setup assigns a run id and scratch dir /tmp/dpsapi-e2e-<runId>/; global teardown fails the run (non-zero exit) if containers/networks/volumes labelled dpsapi.e2e=<runId>, processes carrying DPSAPI_E2E_PROC=<runId>, or workspaces remain, after cleaning them up.

Reports: reports/html/ (HTML), reports/results.json, per-test files in test-results/.

Writing tests

Import everything from the harness, never directly from @playwright/test:

import { test, expect, expectErrorResponse, assertDbInvariants } from '../../harness';

test('inactive token → 401', async ({ cli, startServer, http }) => {
  const token = await cli.tokenNew('demo');
  await cli.tokenDelete(token);
  const server = await startServer({ logLevel: 'DEBUG' });       // FakeDocker (empty) by default
  expectErrorResponse(await http.get(server.url(), { token }), 401);
  expect((await server.stop()).exitCode).toBe(0);                // every server must be stopped
  assertDbInvariants(server.dbPath, { operationCount: 0 });
});

Fixtures (harness/fixtures.ts)

Fixture	Type	Notes
ws	Workspace	Fresh temp dir; ws.dbPath (<dir>/dpsapi.db, not created) is the default DB_PATH. path(), mkdir(), writeFile(). Removed after the test (root-owned files via an alpine container).
cli	CliRunner	Host binary, no shell, argv byte-exact. run(args, {env, cwd, timeoutMs, input, allowSignal}) → CliResult; shell(script, opts) runs /bin/sh -c with $DPSAPI_BIN; tokenNew(desc?) → token, `tokenList(scope = 'active'
startServer	(opts?) => Promise<ServerHandle>	See below.
newFakeDocker	() => Promise<FakeDocker>	Extra fake Engine on a workspace socket; closed automatically.
http	HttpClient	fetch(url, {method, headers, token, data, timeoutMs}) / get() (Playwright request, no redirects) → HttpResult; curl(CurlOptions) → CurlResult; raw(RawHttpOptions) → RawResult. All recorded.
docker	RealDocker	Real Engine containers named dpsapi-e2e-<runId>-<testKey>-<suffix>, labelled dpsapi.e2e=<runId>, dpsapi.e2e.test=<testKey>: run({suffix, image, command, labels, env, init, network, extraArgs}), runMany(n, optsFor, concurrency), stop(c, sec), start, remove, inspect, listOwn({all}), runningIds(), cleanup(). Everything labelled for the test is force-removed at teardown.
image	ImageRunner	$DPSAPI_IMAGE: start(ImageStartOptions) → ImageContainer; run(args, {env, dataDir, docker, user, timeoutMs, extraArgs}) (one-shot docker run --rm, CliResult with via: 'docker-run'); extractBinary() → host path of /app; snapshotDataDir(dir); removeDataDir(dir); containerName(suffix).
secrets	SecretRegistry	register, registerAllIn, redact, findIn, assertNoSecretsIn(text, where). Tokens printed by token new, token list [all] and audit [all | n | n<unit>] via cli/image (printsTokens(args)) are registered automatically; help prints none.
rng	Rng	Deterministic per test (seed = run seed + title path, annotated as seed). next/int/bool/pick/shuffle/bytes/string/fork.
fuzz	FuzzRecorder	fuzz.record(case) stores a JSON-safe snapshot taken at that moment (BigInts, e.g. audit selections, become decimal strings) and returns the case unchanged; fuzz.recordLive(obj) for a case object filled in afterwards; fuzz.seed, fuzz.rng; cases attached as fuzz-cases.json on failure (or fuzz.attach()). Use safeStringify(v) (not JSON.stringify) for labels built from cases.
recorder, testKey, e2e	internals	e2e.allocatePort() (parallel-safe free port), e2e.trackedDbs.add(path) (snapshot on failure).
scanLogsForSecrets	option, default true	test.use({ scanLogsForSecrets: false }) for tests that put registered tokens into descriptions on purpose.

startServer(opts) / ServerHandle (harness/server.ts)

Options: logLevel (default 'INFO', null = unset), addr (default '127.0.0.1', null = unset), port (default allocated, null = unset → 80, only with privateNetwork), dbPath, docker (FakeDocker | 'real' | 'inaccessible' (EACCES) | 'missing' (ENOENT) | {bindSource}; default a new empty FakeDocker), env, privateNetwork, waitReady (default true), readyTimeoutMs, label, binary.

Non-real backends run the binary as unshare -Urm sh -c 'mount --bind <src> /var/run/docker.sock && exec <bin>' (same pid, host network). Readiness polls OPTIONS * over a raw socket: answered by Go itself, so it creates no app log record and no audit row.

Handle: baseURL, url(path = '/api/v1/containers') (path verbatim), port, addr, connectHost, dbPath, env, fakeDocker, pid, generation, stdout, stderr, logs() → LogSet, allOutput(), isRunning(), exit, waitReady(), waitForLog(query, ms), signal(sig), stop(sig = 'SIGTERM', {timeoutMs, allowSignalExit}) → ExitResult {exitCode, signal, durationMs, stdout, stderr} (hang → HarnessError('hang') after SIGKILL; death by signal → HarnessError('signal')), kill(), waitForExit(ms), restart() (SIGTERM, requires exit 0, same port/DB/backend).

FakeDocker (harness/fake-docker.ts)

Fake Engine API on a unix socket: HEAD/GET /_ping, GET [/vX]/containers/json. Live switches: empty(), many(n, rng?), setContainers(list), error(status, message)/error500(), flaky(k) (next k list calls → 500), destroy(), hang(), badJson(body), passthrough(upstream), delay(ms). Counters: stats {ping, list, other, total}, requests[] {method, url, path, query, at}, listCalls(), resetStats(). Summaries: makeContainerSummary(i, rng?, overrides), makeContainerSummaries(n, rng?). Never unlink the socket while a server uses it (bind mounts pin the inode) — switch modes instead.

ImageContainer (harness/image.ts)

image.start({logLevel, addr, containerPort, dbPath, env, docker: 'real' | 'none' | FakeDocker | {socketPath} (default new FakeDocker), dataDir, user, publish, extraArgs, args, waitReady, readyTimeoutMs, suffix}). Mounts the socket at /var/run/docker.sock, dataDir at /data (DB_PATH=/data/dpsapi.db), publishes 127.0.0.1:<random>. Handle: name, id, hostPort, baseURL, url(), dataDir, containerDbPath, fakeDocker, waitReady(), exec(args, {env, user, timeoutMs}) → CliResult (via: 'docker-exec'; exit ≥125 → HarnessError), execRaw(argv) → ProcResult, logs() → {stdout, stderr}, parsedLogs(), inspect(), isRunning(), stop({signal?, timeoutSec}) → {exitCode, oomKilled, stdout, stderr} (137 = killed after grace period), waitForExit(), remove(), snapshotDb() → host path of a host-owned copy of the (root-owned) DB + WAL. Reuse dataDir to replace a container on the same data.

HTTP helpers (harness/http.ts, harness/assertions.ts)

• HttpResult / CurlResult: status, headers (lower-case, repeats joined by , ), headersArray, body, bodyBuffer, json(), durationMs; curl adds exitCode (7 refused, 52 empty reply…), stderr, writeOut, argv. Curl options: url, method, headers: string[] (raw -H lines, repeatable; 'Name;' = empty value), token, data, pathAsIs, head, httpVersion, timeoutMs, allowTimeout, extraArgs.
• RawResult.outcome: 'response' | 'closed' | 'reset' | 'refused' | 'timeout' (timeout only with allowTimeout, otherwise HarnessError); responses[]/response (status, headers pairs, header(n), headerAll(n), body, bodyText), raw, closedByServer. Options: host, port, data (string | Buffer | chunks), chunkDelayMs, timeoutMs, expectResponses, waitForClose, endAfterWrite, head. buildRawRequest({method, target, headers, body, host, version, close}).
• expectErrorResponse(res, 401|404|405|500, {head}) (exact body, Content-Type/Length, WWW-Authenticate: Bearer only on 401, Allow: GET only on 405), expectContainersResponse(res) → array, expectNoInternalDetails(body), view(res), ERROR_BODIES.

CLI / DB / logs

• CliResult = ProcResult (stdout, stderr, stdoutBuffer, stderrBuffer, exitCode, signal, timedOut, durationMs, command, argv) + args, via.
• expectCliSuccess(r, {stdout?, stderr?}); expectCliFailure(r, {exitCode, message?, usage?, exactStderr = true, stdout = ''}) — first throws HarnessError on signal/timeout; message is the text after dpsapi: ; usage text expected for usage error: messages. expectExitedNormally(r), errorMessage(r), USAGE_TEXT (printed on stderr after every usage error and on stdout by help), USAGE_ERROR_MESSAGES (the complete list).
• Tables (token list [all], audit): padTable(lines: string[][]) reproduces the app's text/tabwriter layout exactly (every column but the last padded to 2 + its widest cell in runes — runeLength(s) = Array.from(s).length — over header and rows of that invocation; last cell unpadded; no trailing blanks; \n after every line; header only when there are no rows). TOKEN_LIST_COLUMNS, AUDIT_COLUMNS, EMPTY_TOKEN_LIST, EMPTY_AUDIT (header-only outputs), formatTokenList(entries) / formatAuditOutput(rows) (exact expected stdout for known rows; prefer expect(r.stdout).toBe(...) with them), and the strict parsers parseTokenList(stdout) (TOKEN = first 43 chars, then a Go-quoted DESCRIPTION parsed as a quoted literal — it may contain spaces and \" — then ACTIVE, CREATED_AT, DELETED_AT) / parseAuditOutput(stdout) (TIME, TOKEN, ORIGIN, then OPERATION = rest of the line), which also require the output to re-render byte-exactly through padTable. Rows carry the raw padded line; widths depend on the printed rows, so compare rows by cells, not lines, across invocations.
• Oracles (harness/personas.ts): cliOutcome(args, model) → help (exit 0, usage text on stdout) | list with scope: 'active' | 'all' (token list / token list all; any other single argument → unknown token list argument, two or more → token list takes at most one argument) | new | delete | audit (selection: {all: true} for audit all — exact, lowercase, one argument —, {limit} for no argument / n, {windowNs} for n<unit>) | error; auditArgsOutcome(args), selectAuditRows(ops, selection, nowNs?) / expectedAuditStdout(ops, selection, nowNs?) (all: every row, oldest first — parsed time, then rowid; nowNs only for windows), expectAuditWindow(...), expectedTokenListStdout(dbPath, scope) (exact stdout from the stored rows), StateModel.verifyList(rows, label, scope = 'active'), HELP_NEAR_MISSES / LIST_ALL_NEAR_MISSES / AUDIT_ALL_ARGV_NEAR_MISSES (full argv lists) and AUDIT_ALL_NEAR_MISSES / AUDIT_ALL_EXTRA_ARGS (audit arguments: ALL, All, all1, 1all, all, alls, --all… → invalid audit argument; all x… → audit takes at most one argument) for argv generators; fuzzAuditArgs(rng) also yields all and its near misses.
• goQuote, goUnquote, goUnquoteBytes (Go strconv compatible), shellQuote.
• DB (read-only node:sqlite): readTokens(db) (descriptionHex = exact bytes), readToken, readOperations, countRows, readSchema, withDb, fileDigest, dbSnapshot, RFC3339NANO_UTC_RE, rfc3339ToNanos, msToNanos, and assertDbInvariants(db, {tokenCount?, operationCount?, auditBeforeDeletion?}) → {tokens, activeTokens, inactiveTokens, operations}: integrity_check, foreign_key_check, exact schema, storage classes, token format, active∈{0,1}, deleted_at NULL iff active, strict RFC3339Nano UTC timestamps, deleted_at ≥ created_at, operation = GET containers, origin is an IP, audit time ≥ token created_at.
• Logs: parseLogs(text) → LogSet (records, unparsed, filter/find/count/has(query), levels(), messages(level?), requests()); LogQuery {level, msg (string|RegExp), attrs {k: string|number|boolean|RegExp|fn}}; LOG_LEVELS, visibleLevels(min), levelRank, LOG_TIME_RE; tokenId(token) = the token_id the app logs.
• Misc: runProcess(cmd, argv, {env, cwd, timeoutMs, input, allowTimeout, allowSignal}), toolEnv(), dockerCli(args, {allowFailure}), listByLabel, mapLimit, pollUntil, sleep, occupyPort, isPortFree, HarnessError/isHarnessError.

Conventions

• Files: tests/host/<area>.spec.ts, tests/image/<area>.spec.ts; persona suites tests/host/persona-<name>.spec.ts. One concern per test; use test.step for case tables.
• Exit codes are always explicit: expect(r.exitCode).toBe(0) / expectCliFailure(r, {exitCode: 1|2, ...}); every server is stopped by the test and its exit code asserted.
• Infrastructure vs application failures: timeouts, unexpected signals, hangs, spawn/docker failures, unparseable output throw HarnessError ([harness:<kind>]) — never catch them to satisfy a negative assertion.
• Isolation: everything mutable lives in the test's workspace, FakeDocker, ports and Docker resources keyed by testKey. Never assert on global container counts — only on the suite's names/labels. Never touch resources without the dpsapi-e2e- prefix.
• Cleanup/leaks: teardown kills+fails on running servers or any live process spawned by the worker, removes labelled containers and the workspace; global teardown fails the run on any leftover.
• Secrets: never console.log tokens (worker stdout is redacted anyway). Artifacts and the report redact registered tokens and any token-shaped 43-char string as [REDACTED:<token_id>]. Server logs, CLI stderr and container logs are scanned for registered tokens at teardown.
• Artifacts on failure: per server generation stderr/stdout/exit, container logs, cli-invocations.json, http-exchanges.json, fake-docker-N.json, db-snapshot-N.txt (integrity + rows), fuzz-cases.json (with seeds), harness-problems.txt, harness-notes.txt.
• DB: read-only inspection only; write to the app DB from tests only when a test explicitly simulates lock contention (open your own DatabaseSync for that).
• Timing: wait on observable conditions (waitReady, waitForLog, pollUntil, fake.listCalls()), never fixed sleeps.

CLI command coverage

Command	Main specs
help (stdout = usage text, exit 0, no DB; config errors win; DEBUG/TRACE log records on stderr only)	tests/host/config-usage.spec.ts, logging.spec.ts, tests/image/image-contract.spec.ts (docker exec … /app help, docker run … help)
token list (active only) / token list all (every token), padded tables, unknown token list argument / token list takes at most one argument	tests/host/cli-tokens.spec.ts, config-usage.spec.ts, personas, tests/image/image-contract.spec.ts, multi-transport.spec.ts, image-lifecycle.spec.ts
audit [all | n | n<unit>] (selection forms, units, limits, ordering, window boundary, read-only, usage errors, log levels, unparseable time)	tests/host/cli-audit.spec.ts, population.spec.ts, personas, tests/image/image-contract.spec.ts (docker exec … /app audit, /app audit all), multi-transport.spec.ts, image-lifecycle.spec.ts
audit all (every row oldest first — parsed time, ties in insertion order —; header only when empty; exact padded table with more than 10 rows; read-only; ALL/All/all1/1all/ all/alls/--all → invalid audit argument, all x → audit takes at most one argument, never echoed, no database; DEBUG cli dispatch command=audit all=true)	tests/host/cli-audit.spec.ts (audit all describe; also in the read-only, log-level, unparseable-time, population and usage tables), config-usage.spec.ts, harness-selfcheck.spec.ts, personas (fuzzer, operator session, cross abuse, stupid user), tests/image/image-contract.spec.ts, multi-transport.spec.ts
Usage-error table (every message, exact stderr, no DB)	tests/host/config-usage.spec.ts (checked against USAGE_ERROR_MESSAGES and cliOutcome)

The argv fuzzers (persona-fuzzer-user, persona-cross-abuse, persona-operator-session) generate help, token list all, audit all and their near misses (token list ALL, help help, token list all all, audit ALL, audit all x, audit --all, …), check every outcome with cliOutcome, and require minimum counts of each (e.g. the fuzzer forces audit all / an audit all near miss every 30th case; the operator session has an audit-all action).