test(e2e): add playwright e2e harness

Process and API-only Playwright harness: binary and image setup, server/CLI runners, fake Docker Engine socket, read-only database inspection, secret redaction in reports, seeded fuzzing, leak detection and harness self-checks.
This commit is contained in:
bryanpedini committed 2026-10-06 10:52:25 +02:00
1 parent a02eef13a7
commit fe0cd4bf6d
33 files changed
+6370

No files matched your search

+3
View File
@@ -0,0 +1,3 @@
node_modules/
test-results/
reports/
+120
View File
@@ -0,0 +1,120 @@
# dpsapi E2E suite (Playwright)
API/process-only Playwright suite (no browsers) exercising the compiled `dpsapi` binary and the final image.
## Running
```sh
make test-e2e # make build, then --project=host
make docker-e2e # make docker, then --project=image with DPSAPI_IMAGE set
cd e2e && npm ci
npx playwright test --project=host
DPSAPI_IMAGE=git.bjphoster.com/source/dpsapi:dev npx playwright test --project=image
npx playwright test --project=host tests/host/auth.spec.ts -g 'duplicated'
```
| Variable | Meaning |
|---|---|
| `DPSAPI_BIN` | Binary under test (relative paths resolve against `e2e/`). Default `../dpsapi`; only the default is (re)built by `host-setup` via `make build`. |
| `DPSAPI_E2E_SKIP_BUILD=1` | Skip that build (set by `make test-e2e`, which already built). |
| `DPSAPI_IMAGE` | Image under test for `--project=image` (required there). |
| `DPSAPI_E2E_SEED` | Run seed for all PRNG streams (default `dpsapi-e2e-default-seed-1`, printed at start). |
| `DPSAPI_E2E_WORKERS` | Worker count (default 4). |
| `DPSAPI_E2E_ALPINE` | Image for test/helper containers (default `alpine:3.22`, must exist locally). |
Prerequisites (checked by the setup projects, failing fast with `[harness:prerequisite]`): `curl`, unprivileged `unshare -Urm` with bind/tmpfs mounts, a reachable Docker daemon, `alpine:3.22`, and the binary (host) or the image (image).
Projects: `host-setup` → `host` (`tests/host/`), `image-setup` → `image` (`tests/image/`). Global setup assigns a run id and scratch dir `/tmp/dpsapi-e2e-<runId>/`; global teardown fails the run (non-zero exit) if containers/networks/volumes labelled `dpsapi.e2e=<runId>`, processes carrying `DPSAPI_E2E_PROC=<runId>`, or workspaces remain, after cleaning them up.
Reports: `reports/html/` (HTML), `reports/results.json`, per-test files in `test-results/`.
## Writing tests
Import everything from the harness, never directly from `@playwright/test`:
```ts
import { test, expect, expectErrorResponse, assertDbInvariants } from '../../harness';
test('inactive token → 401', async ({ cli, startServer, http }) => {
const token = await cli.tokenNew('demo');
await cli.tokenDelete(token);
const server = await startServer({ logLevel: 'DEBUG' }); // FakeDocker (empty) by default
expectErrorResponse(await http.get(server.url(), { token }), 401);
expect((await server.stop()).exitCode).toBe(0); // every server must be stopped
assertDbInvariants(server.dbPath, { operationCount: 0 });
});
```
### Fixtures (`harness/fixtures.ts`)
| Fixture | Type | Notes |
|---|---|---|
| `ws` | `Workspace` | Fresh temp dir; `ws.dbPath` (`<dir>/dpsapi.db`, not created) is the default `DB_PATH`. `path()`, `mkdir()`, `writeFile()`. Removed after the test (root-owned files via an alpine container). |
| `cli` | `CliRunner` | Host binary, **no shell**, argv byte-exact. `run(args, {env, cwd, timeoutMs, input, allowSignal})` → `CliResult`; `shell(script, opts)` runs `/bin/sh -c` with `$DPSAPI_BIN`; `tokenNew(desc?)` → token, `tokenList(scope = 'active' | 'all', opts?)` → `TokenListRow[]` (`token list`: active tokens only, asserted; `'all'` runs `token list all`), `tokenListAll(opts?)`, `tokenDelete(tok)`, `audit(arg? \| args[], opts?)` → `AuditRow[]` (`audit()` = latest 10, `audit('all')` = every row, `audit('25')`, `audit('1h')`), `help(opts?)` (stdout exactly `USAGE_TEXT`) (all assert exit 0 and, unless `LOG_LEVEL` is overridden, empty stderr); `env(over)`; `withBinary(path)`. Env overrides: `undefined` removes a variable. Default env is only `PATH`, `DB_PATH`, leak marker. |
| `startServer` | `(opts?) => Promise<ServerHandle>` | See below. |
| `newFakeDocker` | `() => Promise<FakeDocker>` | Extra fake Engine on a workspace socket; closed automatically. |
| `http` | `HttpClient` | `fetch(url, {method, headers, token, data, timeoutMs})` / `get()` (Playwright request, no redirects) → `HttpResult`; `curl(CurlOptions)` → `CurlResult`; `raw(RawHttpOptions)` → `RawResult`. All recorded. |
| `docker` | `RealDocker` | Real Engine containers named `dpsapi-e2e-<runId>-<testKey>-<suffix>`, labelled `dpsapi.e2e=<runId>`, `dpsapi.e2e.test=<testKey>`: `run({suffix, image, command, labels, env, init, network, extraArgs})`, `runMany(n, optsFor, concurrency)`, `stop(c, sec)`, `start`, `remove`, `inspect`, `listOwn({all})`, `runningIds()`, `cleanup()`. Everything labelled for the test is force-removed at teardown. |
| `image` | `ImageRunner` | `$DPSAPI_IMAGE`: `start(ImageStartOptions)` → `ImageContainer`; `run(args, {env, dataDir, docker, user, timeoutMs, extraArgs})` (one-shot `docker run --rm`, `CliResult` with `via: 'docker-run'`); `extractBinary()` → host path of `/app`; `snapshotDataDir(dir)`; `removeDataDir(dir)`; `containerName(suffix)`. |
| `secrets` | `SecretRegistry` | `register`, `registerAllIn`, `redact`, `findIn`, `assertNoSecretsIn(text, where)`. Tokens printed by `token new`, `token list [all]` and `audit [all \| n \| n<unit>]` via `cli`/`image` (`printsTokens(args)`) are registered automatically; `help` prints none. |
| `rng` | `Rng` | Deterministic per test (seed = run seed + title path, annotated as `seed`). `next/int/bool/pick/shuffle/bytes/string/fork`. |
| `fuzz` | `FuzzRecorder` | `fuzz.record(case)` stores a JSON-safe snapshot taken at that moment (BigInts, e.g. audit selections, become decimal strings) and returns the case unchanged; `fuzz.recordLive(obj)` for a case object filled in afterwards; `fuzz.seed`, `fuzz.rng`; cases attached as `fuzz-cases.json` on failure (or `fuzz.attach()`). Use `safeStringify(v)` (not `JSON.stringify`) for labels built from cases. |
| `recorder`, `testKey`, `e2e` | internals | `e2e.allocatePort()` (parallel-safe free port), `e2e.trackedDbs.add(path)` (snapshot on failure). |
| `scanLogsForSecrets` | option, default `true` | `test.use({ scanLogsForSecrets: false })` for tests that put registered tokens into descriptions on purpose. |
### `startServer(opts)` / `ServerHandle` (`harness/server.ts`)
Options: `logLevel` (default `'INFO'`, `null` = unset), `addr` (default `'127.0.0.1'`, `null` = unset), `port` (default allocated, `null` = unset → 80, only with `privateNetwork`), `dbPath`, `docker` (`FakeDocker` | `'real'` | `'inaccessible'` (EACCES) | `'missing'` (ENOENT) | `{bindSource}`; default a new empty FakeDocker), `env`, `privateNetwork`, `waitReady` (default true), `readyTimeoutMs`, `label`, `binary`.
Non-`real` backends run the binary as `unshare -Urm sh -c 'mount --bind <src> /var/run/docker.sock && exec <bin>'` (same pid, host network). Readiness polls `OPTIONS *` over a raw socket: answered by Go itself, so it creates no app log record and no audit row.
Handle: `baseURL`, `url(path = '/api/v1/containers')` (path verbatim), `port`, `addr`, `connectHost`, `dbPath`, `env`, `fakeDocker`, `pid`, `generation`, `stdout`, `stderr`, `logs()` → `LogSet`, `allOutput()`, `isRunning()`, `exit`, `waitReady()`, `waitForLog(query, ms)`, `signal(sig)`, `stop(sig = 'SIGTERM', {timeoutMs, allowSignalExit})` → `ExitResult {exitCode, signal, durationMs, stdout, stderr}` (hang → `HarnessError('hang')` after SIGKILL; death by signal → `HarnessError('signal')`), `kill()`, `waitForExit(ms)`, `restart()` (SIGTERM, requires exit 0, same port/DB/backend).
### `FakeDocker` (`harness/fake-docker.ts`)
Fake Engine API on a unix socket: `HEAD/GET /_ping`, `GET [/vX]/containers/json`. Live switches: `empty()`, `many(n, rng?)`, `setContainers(list)`, `error(status, message)`/`error500()`, `flaky(k)` (next k list calls → 500), `destroy()`, `hang()`, `badJson(body)`, `passthrough(upstream)`, `delay(ms)`. Counters: `stats {ping, list, other, total}`, `requests[] {method, url, path, query, at}`, `listCalls()`, `resetStats()`. Summaries: `makeContainerSummary(i, rng?, overrides)`, `makeContainerSummaries(n, rng?)`. Never unlink the socket while a server uses it (bind mounts pin the inode) — switch modes instead.
### `ImageContainer` (`harness/image.ts`)
`image.start({logLevel, addr, containerPort, dbPath, env, docker: 'real' | 'none' | FakeDocker | {socketPath} (default new FakeDocker), dataDir, user, publish, extraArgs, args, waitReady, readyTimeoutMs, suffix})`. Mounts the socket at `/var/run/docker.sock`, `dataDir` at `/data` (`DB_PATH=/data/dpsapi.db`), publishes `127.0.0.1:<random>`. Handle: `name`, `id`, `hostPort`, `baseURL`, `url()`, `dataDir`, `containerDbPath`, `fakeDocker`, `waitReady()`, `exec(args, {env, user, timeoutMs})` → `CliResult` (`via: 'docker-exec'`; exit ≥125 → `HarnessError`), `execRaw(argv)` → `ProcResult`, `logs()` → `{stdout, stderr}`, `parsedLogs()`, `inspect()`, `isRunning()`, `stop({signal?, timeoutSec})` → `{exitCode, oomKilled, stdout, stderr}` (137 = killed after grace period), `waitForExit()`, `remove()`, `snapshotDb()` → host path of a host-owned copy of the (root-owned) DB + WAL. Reuse `dataDir` to replace a container on the same data.
### HTTP helpers (`harness/http.ts`, `harness/assertions.ts`)
- `HttpResult` / `CurlResult`: `status`, `headers` (lower-case, repeats joined by `, `), `headersArray`, `body`, `bodyBuffer`, `json()`, `durationMs`; curl adds `exitCode` (7 refused, 52 empty reply…), `stderr`, `writeOut`, `argv`. Curl options: `url, method, headers: string[]` (raw `-H` lines, repeatable; `'Name;'` = empty value), `token, data, pathAsIs, head, httpVersion, timeoutMs, allowTimeout, extraArgs`.
- `RawResult.outcome`: `'response' | 'closed' | 'reset' | 'refused' | 'timeout'` (timeout only with `allowTimeout`, otherwise `HarnessError`); `responses[]`/`response` (`status`, `headers` pairs, `header(n)`, `headerAll(n)`, `body`, `bodyText`), `raw`, `closedByServer`. Options: `host, port, data (string | Buffer | chunks), chunkDelayMs, timeoutMs, expectResponses, waitForClose, endAfterWrite, head`. `buildRawRequest({method, target, headers, body, host, version, close})`.
- `expectErrorResponse(res, 401|404|405|500, {head})` (exact body, Content-Type/Length, `WWW-Authenticate: Bearer` only on 401, `Allow: GET` only on 405), `expectContainersResponse(res)` → array, `expectNoInternalDetails(body)`, `view(res)`, `ERROR_BODIES`.
### CLI / DB / logs
- `CliResult` = `ProcResult` (`stdout, stderr, stdoutBuffer, stderrBuffer, exitCode, signal, timedOut, durationMs, command, argv`) + `args`, `via`.
- `expectCliSuccess(r, {stdout?, stderr?})`; `expectCliFailure(r, {exitCode, message?, usage?, exactStderr = true, stdout = ''})` — first throws `HarnessError` on signal/timeout; `message` is the text after `dpsapi: `; usage text expected for `usage error:` messages. `expectExitedNormally(r)`, `errorMessage(r)`, `USAGE_TEXT` (printed on stderr after every usage error and on stdout by `help`), `USAGE_ERROR_MESSAGES` (the complete list).
- Tables (`token list [all]`, `audit`): `padTable(lines: string[][])` reproduces the app's `text/tabwriter` layout exactly (every column but the last padded to 2 + its widest cell in runes — `runeLength(s)` = `Array.from(s).length` — over header and rows of that invocation; last cell unpadded; no trailing blanks; `\n` after every line; header only when there are no rows). `TOKEN_LIST_COLUMNS`, `AUDIT_COLUMNS`, `EMPTY_TOKEN_LIST`, `EMPTY_AUDIT` (header-only outputs), `formatTokenList(entries)` / `formatAuditOutput(rows)` (exact expected stdout for known rows; prefer `expect(r.stdout).toBe(...)` with them), and the strict parsers `parseTokenList(stdout)` (TOKEN = first 43 chars, then a Go-quoted DESCRIPTION parsed as a quoted literal — it may contain spaces and `\"` — then ACTIVE, CREATED_AT, DELETED_AT) / `parseAuditOutput(stdout)` (TIME, TOKEN, ORIGIN, then OPERATION = rest of the line), which also require the output to re-render byte-exactly through `padTable`. Rows carry the raw padded `line`; widths depend on the printed rows, so compare rows by cells, not lines, across invocations.
- Oracles (`harness/personas.ts`): `cliOutcome(args, model)` → `help` (exit 0, usage text on stdout) | `list` with `scope: 'active' | 'all'` (`token list` / `token list all`; any other single argument → `unknown token list argument`, two or more → `token list takes at most one argument`) | `new` | `delete` | `audit` (`selection`: `{all: true}` for `audit all` — exact, lowercase, one argument —, `{limit}` for no argument / `n`, `{windowNs}` for `n<unit>`) | `error`; `auditArgsOutcome(args)`, `selectAuditRows(ops, selection, nowNs?)` / `expectedAuditStdout(ops, selection, nowNs?)` (`all`: every row, oldest first — parsed time, then rowid; `nowNs` only for windows), `expectAuditWindow(...)`, `expectedTokenListStdout(dbPath, scope)` (exact stdout from the stored rows), `StateModel.verifyList(rows, label, scope = 'active')`, `HELP_NEAR_MISSES` / `LIST_ALL_NEAR_MISSES` / `AUDIT_ALL_ARGV_NEAR_MISSES` (full argv lists) and `AUDIT_ALL_NEAR_MISSES` / `AUDIT_ALL_EXTRA_ARGS` (`audit` arguments: `ALL`, `All`, `all1`, `1all`, ` all`, `alls`, `--all`… → `invalid audit argument`; `all x`… → `audit takes at most one argument`) for argv generators; `fuzzAuditArgs(rng)` also yields `all` and its near misses.
- `goQuote`, `goUnquote`, `goUnquoteBytes` (Go `strconv` compatible), `shellQuote`.
- DB (read-only `node:sqlite`): `readTokens(db)` (`descriptionHex` = exact bytes), `readToken`, `readOperations`, `countRows`, `readSchema`, `withDb`, `fileDigest`, `dbSnapshot`, `RFC3339NANO_UTC_RE`, `rfc3339ToNanos`, `msToNanos`, and `assertDbInvariants(db, {tokenCount?, operationCount?, auditBeforeDeletion?})` → `{tokens, activeTokens, inactiveTokens, operations}`: integrity_check, foreign_key_check, exact schema, storage classes, token format, active∈{0,1}, deleted_at NULL iff active, strict RFC3339Nano UTC timestamps, deleted_at ≥ created_at, operation = `GET containers`, origin is an IP, audit time ≥ token created_at.
- Logs: `parseLogs(text)` → `LogSet` (`records`, `unparsed`, `filter/find/count/has(query)`, `levels()`, `messages(level?)`, `requests()`); `LogQuery {level, msg (string|RegExp), attrs {k: string|number|boolean|RegExp|fn}}`; `LOG_LEVELS`, `visibleLevels(min)`, `levelRank`, `LOG_TIME_RE`; `tokenId(token)` = the `token_id` the app logs.
- Misc: `runProcess(cmd, argv, {env, cwd, timeoutMs, input, allowTimeout, allowSignal})`, `toolEnv()`, `dockerCli(args, {allowFailure})`, `listByLabel`, `mapLimit`, `pollUntil`, `sleep`, `occupyPort`, `isPortFree`, `HarnessError`/`isHarnessError`.
## Conventions
- **Files**: `tests/host/<area>.spec.ts`, `tests/image/<area>.spec.ts`; persona suites `tests/host/persona-<name>.spec.ts`. One concern per `test`; use `test.step` for case tables.
- **Exit codes are always explicit**: `expect(r.exitCode).toBe(0)` / `expectCliFailure(r, {exitCode: 1|2, ...})`; every server is stopped by the test and its exit code asserted.
- **Infrastructure vs application failures**: timeouts, unexpected signals, hangs, spawn/docker failures, unparseable output throw `HarnessError` (`[harness:<kind>]`) — never catch them to satisfy a negative assertion.
- **Isolation**: everything mutable lives in the test's workspace, FakeDocker, ports and Docker resources keyed by `testKey`. Never assert on global container counts — only on the suite's names/labels. Never touch resources without the `dpsapi-e2e-` prefix.
- **Cleanup/leaks**: teardown kills+fails on running servers or any live process spawned by the worker, removes labelled containers and the workspace; global teardown fails the run on any leftover.
- **Secrets**: never `console.log` tokens (worker stdout is redacted anyway). Artifacts and the report redact registered tokens and any token-shaped 43-char string as `[REDACTED:<token_id>]`. Server logs, CLI stderr and container logs are scanned for registered tokens at teardown.
- **Artifacts on failure**: per server generation `stderr`/`stdout`/exit, container logs, `cli-invocations.json`, `http-exchanges.json`, `fake-docker-N.json`, `db-snapshot-N.txt` (integrity + rows), `fuzz-cases.json` (with seeds), `harness-problems.txt`, `harness-notes.txt`.
- **DB**: read-only inspection only; write to the app DB from tests only when a test explicitly simulates lock contention (open your own `DatabaseSync` for that).
- **Timing**: wait on observable conditions (`waitReady`, `waitForLog`, `pollUntil`, `fake.listCalls()`), never fixed sleeps.
## CLI command coverage
| Command | Main specs |
|---|---|
| `help` (stdout = usage text, exit 0, no DB; config errors win; DEBUG/TRACE log records on stderr only) | `tests/host/config-usage.spec.ts`, `logging.spec.ts`, `tests/image/image-contract.spec.ts` (`docker exec … /app help`, `docker run … help`) |
| `token list` (active only) / `token list all` (every token), padded tables, `unknown token list argument` / `token list takes at most one argument` | `tests/host/cli-tokens.spec.ts`, `config-usage.spec.ts`, personas, `tests/image/image-contract.spec.ts`, `multi-transport.spec.ts`, `image-lifecycle.spec.ts` |
| `audit [all \| n \| n<unit>]` (selection forms, units, limits, ordering, window boundary, read-only, usage errors, log levels, unparseable time) | `tests/host/cli-audit.spec.ts`, `population.spec.ts`, personas, `tests/image/image-contract.spec.ts` (`docker exec … /app audit`, `/app audit all`), `multi-transport.spec.ts`, `image-lifecycle.spec.ts` |
| `audit all` (every row oldest first — parsed time, ties in insertion order —; header only when empty; exact padded table with more than 10 rows; read-only; `ALL`/`All`/`all1`/`1all`/` all`/`alls`/`--all` → `invalid audit argument`, `all x` → `audit takes at most one argument`, never echoed, no database; DEBUG `cli dispatch command=audit all=true`) | `tests/host/cli-audit.spec.ts` (`audit all` describe; also in the read-only, log-level, unparseable-time, population and usage tables), `config-usage.spec.ts`, `harness-selfcheck.spec.ts`, personas (fuzzer, operator session, cross abuse, stupid user), `tests/image/image-contract.spec.ts`, `multi-transport.spec.ts` |
| Usage-error table (every message, exact stderr, no DB) | `tests/host/config-usage.spec.ts` (checked against `USAGE_ERROR_MESSAGES` and `cliOutcome`) |
The argv fuzzers (`persona-fuzzer-user`, `persona-cross-abuse`, `persona-operator-session`) generate `help`, `token list all`, `audit all` and their near misses (`token list ALL`, `help help`, `token list all all`, `audit ALL`, `audit all x`, `audit --all`, …), check every outcome with `cliOutcome`, and require minimum counts of each (e.g. the fuzzer forces `audit all` / an `audit all` near miss every 30th case; the operator session has an `audit-all` action).
+98
View File
@@ -0,0 +1,98 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Shared assertions on the HTTP contract of dpsapi.
*/
import { expect } from '@playwright/test';
import type { CurlResult, HttpResult, RawHttpResponse } from './http';
/** Exact error bodies of the application. */
export const ERROR_BODIES = {
401: '{"error":"unauthorized"}',
404: '{"error":"not found"}',
405: '{"error":"method not allowed"}',
500: '{"error":"internal server error"}',
} as const;
export type AppErrorStatus = keyof typeof ERROR_BODIES;
/** Client-independent response view. */
export interface ResponseView {
status: number;
/** Lower-cased names; repeated headers joined with ', '. */
headers: Record<string, string>;
body: string;
}
/** Normalizes Playwright / curl / raw responses into a ResponseView. */
export function view(r: HttpResult | CurlResult | RawHttpResponse | ResponseView): ResponseView {
if ('bodyText' in r) {
const headers: Record<string, string> = {};
for (const [k, v] of r.headers) {
const l = k.toLowerCase();
headers[l] = l in headers ? `${headers[l]}, ${v}` : v;
}
return { status: r.status, headers, body: r.bodyText };
}
return { status: r.status, headers: r.headers, body: r.body };
}
/** Substrings that must never appear in an HTTP response body. */
export const INTERNAL_DETAIL_MARKERS = [
'sql',
'sqlite',
'docker.sock',
'/var/run',
'unix',
'dial',
'goroutine',
'panic',
'runtime error',
'fake engine',
'connection refused',
'permission denied',
'no such file',
'context deadline',
'list containers',
'token_id',
];
/** Fails if the body contains any internal-detail marker (case-insensitive). */
export function expectNoInternalDetails(body: string): void {
const low = body.toLowerCase();
expect(INTERNAL_DETAIL_MARKERS.filter((m) => low.includes(m)), `internal details leaked in body ${JSON.stringify(body.slice(0, 300))}`).toEqual([]);
}
/**
* Asserts one of the application's JSON error responses exactly: status, body, Content-Type,
* Content-Length, WWW-Authenticate (only on 401: `Bearer`) and Allow (only on 405: `GET`).
*/
export function expectErrorResponse(res: HttpResult | CurlResult | RawHttpResponse | ResponseView, status: AppErrorStatus, { head = false }: { head?: boolean } = {}): void {
const r = view(res);
const body = ERROR_BODIES[status];
expect(r.status, `status (body ${JSON.stringify(r.body.slice(0, 200))})`).toBe(status);
expect(r.body, 'error body').toBe(head ? '' : body);
expect(r.headers['content-type'], 'content-type').toBe('application/json');
expect(r.headers['content-length'], 'content-length').toBe(String(body.length));
if (status === 401) expect(r.headers['www-authenticate'], 'www-authenticate').toBe('Bearer');
else expect(r.headers['www-authenticate'], 'www-authenticate').toBeUndefined();
if (status === 405) expect(r.headers['allow'], 'allow').toBe('GET');
else expect(r.headers['allow'], 'allow').toBeUndefined();
expect(r.headers['location'], 'no redirect').toBeUndefined();
}
/**
* Asserts a successful container listing: 200, application/json, exact Content-Length, a JSON
* array without trailing newline. Returns the parsed array.
*/
export function expectContainersResponse<T = Record<string, unknown>>(res: HttpResult | CurlResult | RawHttpResponse | ResponseView): T[] {
const r = view(res);
expect(r.status, `status (body ${JSON.stringify(r.body.slice(0, 200))})`).toBe(200);
expect(r.headers['content-type'], 'content-type').toBe('application/json');
expect(r.headers['content-length'], 'content-length').toBe(String(Buffer.byteLength(r.body)));
expect(r.body.endsWith('\n'), 'no trailing newline').toBe(false);
const parsed = JSON.parse(r.body) as unknown;
expect(Array.isArray(parsed), 'body is a JSON array').toBe(true);
return parsed as T[];
}
+518
View File
@@ -0,0 +1,518 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Running the compiled binary as a CLI, parsing its output, and asserting outcomes.
*/
import { expect } from '@playwright/test';
import { RFC3339NANO_UTC_RE } from './db';
import { binaryPath, PROC_MARKER_ENV, runId } from './env';
import { HarnessError } from './errors';
import { goQuote, goUnquoteBytes } from './gostrings';
import { runProcess, type ProcResult } from './proc';
import type { CliVia, Recorder } from './recorder';
import { redactText, TOKEN_RE, type SecretRegistry } from './secrets';
import type { Workspace } from './workspace';
export interface CliResult extends ProcResult {
/** Application arguments (after the binary / `docker exec <c> /app`), or [script] for shell runs. */
args: string[];
via: CliVia;
}
export interface CliOptions {
/** Environment overrides on top of the default app env; `undefined` removes a variable. */
env?: Record<string, string | undefined>;
cwd?: string;
/** Default 30 s. Server-mode invocations expected to fail fast should pass ~10 s. */
timeoutMs?: number;
input?: string | Buffer;
/** Do not throw HarnessError on signal termination (only for tests that deliver signals). */
allowSignal?: boolean;
}
/** Usage text: printed to stderr after every usage error, and to stdout by `help`. */
export const USAGE_TEXT =
'usage:\n' +
' dpsapi start the HTTP server\n' +
' dpsapi help print this usage text\n' +
' dpsapi token list [all] list the active tokens, or every token with "all"\n' +
' dpsapi token new [description] create a token and print it\n' +
' dpsapi token delete <token> deactivate a token\n' +
' dpsapi audit print the latest 10 audit rows\n' +
' dpsapi audit <n> print the latest n audit rows\n' +
' dpsapi audit <n><unit> print the audit rows of the last n units (s, m, h, d, w)\n' +
' dpsapi audit all print every audit row\n';
/** Every usage error message (text after `dpsapi: usage error: `) the CLI can print. */
export const USAGE_ERROR_MESSAGES = [
'unknown command',
'help takes no arguments',
'missing token subcommand',
'unknown token subcommand',
'token list takes at most one argument',
'unknown token list argument',
'token new takes at most one argument',
'token delete requires exactly one argument',
'audit takes at most one argument',
'invalid audit argument',
] as const;
// ---------------------------------------------------------------------------------------------
// Padded tables (Go text/tabwriter: minwidth 0, tabwidth 0, padding 2, padchar ' ', flags 0)
// ---------------------------------------------------------------------------------------------
/** Length in Unicode code points (what tabwriter counts: runes, not bytes, not display width). */
export function runeLength(s: string): number {
return Array.from(s).length;
}
/**
* Renders lines of cells exactly like the app's tabwriter: every column but the last is padded
* with spaces to (2 + the longest cell of that column over all lines, in runes); the last cell is
* written as-is; every line ends with '\n'. All lines must have the same number of cells.
*/
export function padTable(lines: ReadonlyArray<readonly string[]>): string {
if (lines.length === 0) return '';
const n = lines[0].length;
for (const l of lines) if (l.length !== n) throw new Error(`padTable: every line needs ${n} cells, got ${l.length}`);
const widths = Array.from({ length: Math.max(0, n - 1) }, (_, c) => 2 + Math.max(...lines.map((l) => runeLength(l[c]))));
return lines.map((l) => l.map((cell, c) => (c < n - 1 ? cell + ' '.repeat(widths[c] - runeLength(cell)) : cell)).join('') + '\n').join('');
}
/** Header cells of `token list` / `token list all`. */
export const TOKEN_LIST_COLUMNS = ['TOKEN', 'DESCRIPTION', 'ACTIVE', 'CREATED_AT', 'DELETED_AT'] as const;
/** Header cells of `audit`. */
export const AUDIT_COLUMNS = ['TIME', 'TOKEN', 'ORIGIN', 'OPERATION'] as const;
/** Exact stdout of `token list [all]` with no rows (the header padded by its own widths). */
export const EMPTY_TOKEN_LIST = padTable([TOKEN_LIST_COLUMNS]);
/** Exact stdout of `audit` with no rows. */
export const EMPTY_AUDIT = padTable([AUDIT_COLUMNS]);
/**
* True for commands whose successful stdout carries full token values (`token new`,
* `token list [all]`, `audit [all | n | n<unit>]`): the runners register every token-shaped string of their stdout
* as a secret. (`help` prints only the usage text.)
*/
export function printsTokens(args: readonly string[]): boolean {
return (args[0] === 'token' && (args[1] === 'new' || args[1] === 'list')) || args[0] === 'audit';
}
/** Error-message rendering of (part of) an output: clipped, token-shaped strings redacted. */
function show(s: string | undefined): string {
const v = s ?? '<missing>';
return JSON.stringify(redactText(v.length > 200 ? v.slice(0, 200) + '…' : v, []));
}
/** Splits table stdout into lines (must end with a newline; at least the header). */
function tableLines(stdout: string, what: string): string[] {
if (!stdout.endsWith('\n')) throw new Error(`${what} output does not end with a newline: ${show(stdout.slice(-120))}`);
return stdout.slice(0, -1).split('\n');
}
/** The re-rendered table must equal the output byte for byte (padding, widths, no trailing blanks). */
function expectRerenders(stdout: string, cells: string[][], what: string): void {
const again = padTable(cells);
if (again === stdout) return;
const a = stdout.split('\n');
const b = again.split('\n');
const i = a.findIndex((l, k) => l !== b[k]);
throw new Error(`${what} output is not the padded table of its own cells (first differing line ${i}): got ${show(a[i])}, expected ${show(b[i])}`);
}
export interface AuditRow {
/** TIME exactly as printed (= as stored). */
time: string;
/** Full token value. */
token: string;
origin: string;
operation: string;
/** The raw (padded) line. */
line: string;
}
/**
* Parses `audit` stdout strictly: trailing newline, the header, rows of TIME / TOKEN / ORIGIN
* (no blanks inside, separated by runs of spaces) and OPERATION (the rest of the line), TIME in
* strict UTC RFC3339Nano, TOKEN 43-char base64url; finally the whole output must re-render byte
* for byte as the padded table of the parsed cells. Throws on any deviation (malformed output is
* a test failure). Token values are redacted in the error message.
*/
export function parseAuditOutput(stdout: string): AuditRow[] {
const lines = tableLines(stdout, 'audit');
const header = lines[0].split(/ +/);
if (header.join('\t') !== AUDIT_COLUMNS.join('\t')) throw new Error(`unexpected audit header: ${show(lines[0])}`);
const rows = lines.slice(1).map((line, i) => {
const m = /^(\S+) +(\S+) +(\S+) +(\S.*)$/.exec(line);
if (!m) throw new Error(`audit row ${i + 1} does not have 4 space-separated cells: ${show(line)}`);
const [, time, token, origin, operation] = m;
if (!RFC3339NANO_UTC_RE.test(time)) throw new Error(`audit row ${i + 1}: TIME ${show(time)} is not strict UTC RFC3339Nano`);
if (!TOKEN_RE.test(token)) throw new Error(`audit row ${i + 1}: TOKEN is not a 43-character base64url token: ${show(token)}`);
return { time, token, origin, operation, line };
});
expectRerenders(stdout, [[...AUDIT_COLUMNS], ...rows.map((r) => [r.time, r.token, r.origin, r.operation])], 'audit');
return rows;
}
/** The exact `audit` stdout for the given rows (padded header + one line per row, in the given order). */
export function formatAuditOutput(rows: ReadonlyArray<{ time: string; token: string; origin: string; operation: string }>): string {
return padTable([[...AUDIT_COLUMNS], ...rows.map((r) => [r.time, r.token, r.origin, r.operation])]);
}
export interface TokenListRow {
token: string;
/** DESCRIPTION column exactly as printed (Go-quoted). */
descriptionQuoted: string;
/** Unquoted description bytes. */
descriptionBytes: Buffer;
/** Unquoted description as UTF-8 text. */
description: string;
active: boolean;
createdAt: string;
/** null when the column is `-`. */
deletedAt: string | null;
/** The raw (padded) line. */
line: string;
}
/** Index just after the closing quote of the Go double-quoted literal starting at s[start] (-1 if unterminated). */
function endOfQuoted(s: string, start: number): number {
if (s[start] !== '"') return -1;
for (let i = start + 1; i < s.length; i++) {
if (s[i] === '\\') i++;
else if (s[i] === '"') return i + 1;
}
return -1;
}
/**
* Parses `token list` / `token list all` stdout strictly: trailing newline, the header, rows of
* TOKEN (the first 43 characters, base64url), spaces, a Go-quoted DESCRIPTION (parsed as a quoted
* literal: it may contain spaces and escaped quotes), then ACTIVE (`true`/`false`), CREATED_AT
* (strict RFC3339Nano UTC) and DELETED_AT (`-` or a timestamp) separated by runs of spaces;
* finally the whole output must re-render byte for byte as the padded table of the parsed cells.
* Throws on any deviation: malformed output is a test failure.
*/
export function parseTokenList(stdout: string): TokenListRow[] {
const lines = tableLines(stdout, 'token list');
const header = lines[0].split(/ +/);
if (header.join('\t') !== TOKEN_LIST_COLUMNS.join('\t')) throw new Error(`unexpected token list header: ${show(lines[0])}`);
const rows = lines.slice(1).map((line, i): TokenListRow => {
const fail = (why: string): never => {
throw new Error(`token list row ${i + 1}: ${why}: ${show(line)}`);
};
const token = line.slice(0, 43);
if (!TOKEN_RE.test(token)) fail('TOKEN is not 43 base64url characters');
const q = /^ +/.exec(line.slice(43));
if (!q) fail('no spaces after TOKEN');
const qStart = 43 + q![0].length;
const qEnd = endOfQuoted(line, qStart);
if (qEnd < 0) fail('DESCRIPTION is not a terminated Go-quoted string');
const descriptionQuoted = line.slice(qStart, qEnd);
let descriptionBytes: Buffer;
try {
descriptionBytes = goUnquoteBytes(descriptionQuoted);
} catch (e) {
return fail(`DESCRIPTION is not a valid Go-quoted string (${(e as Error).message})`);
}
const rest = /^ +(true|false) +(\S+) +(\S+)$/.exec(line.slice(qEnd));
if (!rest) fail('expected ACTIVE, CREATED_AT and DELETED_AT after DESCRIPTION');
const [, active, createdAt, deletedAt] = rest!;
if (!RFC3339NANO_UTC_RE.test(createdAt)) fail(`CREATED_AT ${show(createdAt)} is not strict UTC RFC3339Nano`);
if (deletedAt !== '-' && !RFC3339NANO_UTC_RE.test(deletedAt)) fail(`DELETED_AT ${show(deletedAt)} is neither "-" nor strict UTC RFC3339Nano`);
return {
token,
descriptionQuoted,
descriptionBytes,
description: descriptionBytes.toString('utf8'),
active: active === 'true',
createdAt,
deletedAt: deletedAt === '-' ? null : deletedAt,
line,
};
});
expectRerenders(stdout, [[...TOKEN_LIST_COLUMNS], ...rows.map((r) => [r.token, r.descriptionQuoted, String(r.active), r.createdAt, r.deletedAt ?? '-'])], 'token list');
return rows;
}
/** One row as `token list` prints it; `description` is the exact stored text/bytes (Go-quoted here). */
export interface TokenListEntry {
token: string;
description: string | Buffer;
active: boolean;
createdAt: string;
deletedAt: string | null;
}
/** The exact `token list [all]` stdout for the given rows (padded header + one line per row, in the given order). */
export function formatTokenList(rows: ReadonlyArray<TokenListEntry>): string {
return padTable([[...TOKEN_LIST_COLUMNS], ...rows.map((r) => [r.token, goQuote(r.description), String(r.active), r.createdAt, r.deletedAt ?? '-'])]);
}
/** Throws HarnessError unless the process exited on its own with an exit status. */
export function expectExitedNormally(r: ProcResult, label = r.command): void {
if (r.timedOut) throw new HarnessError('timeout', `${label} timed out (killed by the harness)`);
if (r.signal !== null || r.exitCode === null) throw new HarnessError('signal', `${label} terminated by signal ${r.signal}`);
}
export interface CliSuccessExpectation {
/** Exact stdout or pattern. */
stdout?: string | RegExp;
/** Exact stderr or pattern ('' = must be empty). */
stderr?: string | RegExp;
}
/** Asserts a normal exit with status 0 (and optional exact stdout/stderr). */
export function expectCliSuccess(r: ProcResult, exp: CliSuccessExpectation = {}): void {
expectExitedNormally(r);
expect(r.exitCode, `exit code (stderr: ${r.stderr.slice(-500)})`).toBe(0);
if (exp.stdout !== undefined) {
if (typeof exp.stdout === 'string') expect(r.stdout, 'stdout').toBe(exp.stdout);
else expect(r.stdout, 'stdout').toMatch(exp.stdout);
}
if (exp.stderr !== undefined) {
if (typeof exp.stderr === 'string') expect(r.stderr, 'stderr').toBe(exp.stderr);
else expect(r.stderr, 'stderr').toMatch(exp.stderr);
}
}
export interface CliFailureExpectation {
/** Required: the exact expected exit status (1 runtime failure, 2 usage/configuration). */
exitCode: number;
/** Text after `dpsapi: ` on the error line (exact string or pattern). */
message?: string | RegExp;
/** Usage text expected after the error line. Default: message is a string starting with 'usage error:'. */
usage?: boolean;
/**
* true (default): stderr must be exactly the error line (+ usage text) — right for the default
* INFO level. false: only the error line (+ usage) at the end is checked (DEBUG/TRACE runs).
*/
exactStderr?: boolean;
/** Expected stdout (default ''). */
stdout?: string;
}
/**
* Asserts an expected application failure. First requires a normal exit (a signal/timeout is a
* HarnessError, never an expected failure), then exit status, stdout, and the stderr error line.
*/
export function expectCliFailure(r: ProcResult, exp: CliFailureExpectation): void {
expectExitedNormally(r);
expect(r.exitCode, `exit code (stderr: ${r.stderr.slice(-500)})`).toBe(exp.exitCode);
expect(r.stdout, 'stdout').toBe(exp.stdout ?? '');
const usage = exp.usage ?? (typeof exp.message === 'string' && exp.message.startsWith('usage error:'));
const suffix = usage ? USAGE_TEXT : '';
expect(r.stderr.endsWith('\n' + suffix) || r.stderr.endsWith(suffix), 'stderr ends with usage text').toBe(true);
const head = suffix ? r.stderr.slice(0, r.stderr.length - suffix.length) : r.stderr;
const lines = head.split('\n');
if (lines[lines.length - 1] === '') lines.pop();
const errorLine = lines[lines.length - 1] ?? '';
expect(errorLine.startsWith('dpsapi: '), `error line ${JSON.stringify(errorLine)} starts with "dpsapi: "`).toBe(true);
const message = errorLine.slice('dpsapi: '.length);
if (exp.message !== undefined) {
if (typeof exp.message === 'string') expect(message, 'error message').toBe(exp.message);
else expect(message, 'error message').toMatch(exp.message);
}
if (exp.exactStderr ?? true) expect(lines.length, `stderr has only the error line:\n${r.stderr}`).toBe(1);
}
/** Last line of stderr starting with `dpsapi: `, without the prefix (undefined when none). */
export function errorMessage(r: ProcResult): string | undefined {
const line = r.stderr
.split('\n')
.filter((l) => l.startsWith('dpsapi: '))
.pop();
return line?.slice('dpsapi: '.length);
}
export interface CliContext {
workspace: Workspace;
secrets: SecretRegistry;
recorder: Recorder;
}
/** Default environment of every app process: PATH, DB_PATH (workspace default) and the leak marker. */
export function baseAppEnv(dbPath: string): Record<string, string> {
return { PATH: process.env.PATH || '/usr/local/bin:/usr/bin:/bin', DB_PATH: dbPath, [PROC_MARKER_ENV]: runId() };
}
/** Applies overrides; `undefined` deletes. */
export function mergeEnv(base: Record<string, string>, over: Record<string, string | undefined> = {}): Record<string, string> {
const env = { ...base };
for (const [k, v] of Object.entries(over)) {
if (v === undefined) delete env[k];
else env[k] = v;
}
return env;
}
/** Env subset recorded in artifacts (no PATH / marker). */
export function recordableEnv(env: Record<string, string | undefined>): Record<string, string> {
const out: Record<string, string> = {};
for (const [k, v] of Object.entries(env)) if (k !== 'PATH' && k !== PROC_MARKER_ENV && v !== undefined) out[k] = v;
return out;
}
/** Runs the host binary as a CLI. Bound to a test's workspace/secrets/recorder by the `cli` fixture. */
export class CliRunner {
/** Absolute path of the binary this runner executes. */
readonly binary: string;
constructor(
private readonly ctx: CliContext,
binary: string = binaryPath(),
) {
this.binary = binary;
}
/** Same workspace/secrets/recorder, different binary (e.g. /app extracted from the image). */
withBinary(binary: string): CliRunner {
return new CliRunner(this.ctx, binary);
}
/** Default environment (PATH, DB_PATH=<workspace>/dpsapi.db, leak marker) with overrides applied. */
env(over?: Record<string, string | undefined>): Record<string, string> {
return mergeEnv(baseAppEnv(this.ctx.workspace.dbPath), over);
}
/**
* Runs `<binary> ...args` without a shell: every argument reaches the program byte-exact.
* Throws HarnessError on timeout/signal/spawn failure. Never throws for non-zero exit codes.
*/
async run(args: string[], opts: CliOptions = {}): Promise<CliResult> {
const env = this.env(opts.env);
const r = await runProcess(this.binary, args, {
env,
cwd: opts.cwd ?? this.ctx.workspace.dir,
timeoutMs: opts.timeoutMs,
input: opts.input,
allowSignal: opts.allowSignal,
label: `dpsapi ${args.slice(0, 2).join(' ')}`,
}).catch((e) => {
this.recordError('argv', this.binary, args, env, e);
throw e;
});
const res: CliResult = { ...r, args: [...args], via: 'argv' };
if (printsTokens(args)) this.ctx.secrets.registerAllIn(r.stdout);
this.record(res, env);
return res;
}
/**
* Runs `script` with /bin/sh -c (for tests that exercise shell quoting). The environment has
* DPSAPI_BIN (the binary path) plus the default app env, so scripts use `"$DPSAPI_BIN" token new ...`.
* Token-shaped strings printed on stdout are registered as secrets.
*/
async shell(script: string, opts: CliOptions = {}): Promise<CliResult> {
const env = { ...this.env(opts.env), DPSAPI_BIN: this.binary };
const r = await runProcess('/bin/sh', ['-c', script], {
env,
cwd: opts.cwd ?? this.ctx.workspace.dir,
timeoutMs: opts.timeoutMs,
input: opts.input,
allowSignal: opts.allowSignal,
label: 'sh -c <script>',
}).catch((e) => {
this.recordError('shell', '/bin/sh', [script], env, e);
throw e;
});
const res: CliResult = { ...r, args: [script], via: 'shell' };
this.record(res, env);
this.ctx.secrets.registerAllIn(r.stdout);
return res;
}
/**
* `token new [description]`: asserts success, a single 43-char base64url line on stdout and
* (unless LOG_LEVEL is overridden) empty stderr. Returns the (registered) token.
*/
async tokenNew(description?: string, opts: CliOptions = {}): Promise<string> {
const r = await this.run(description === undefined ? ['token', 'new'] : ['token', 'new', description], opts);
expectCliSuccess(r);
expect(r.stdout, 'token new stdout').toMatch(/^[A-Za-z0-9_-]{43}\n$/);
if (!opts.env?.LOG_LEVEL) expect(r.stderr, 'token new stderr').toBe('');
const token = r.stdout.slice(0, -1);
expect(TOKEN_RE.test(token)).toBe(true);
return this.ctx.secrets.register(token);
}
/**
* `token list` (active tokens only) or, with 'all', `token list all` (every token): asserts
* success (and empty stderr unless LOG_LEVEL is overridden) and returns the strictly parsed rows.
* The printed tokens are registered as secrets.
*/
async tokenList(scope: 'active' | 'all' = 'active', opts: CliOptions = {}): Promise<TokenListRow[]> {
const args = scope === 'all' ? ['token', 'list', 'all'] : ['token', 'list'];
const r = await this.run(args, opts);
expectCliSuccess(r);
if (!opts.env?.LOG_LEVEL) expect(r.stderr, `${args.join(' ')} stderr`).toBe('');
const rows = parseTokenList(r.stdout);
if (scope === 'active') expect(rows.filter((x) => !x.active || x.deletedAt !== null).map((x) => x.line.slice(43)), 'token list prints only active tokens').toEqual([]);
return rows;
}
/** `token list all`: every token, active and inactive (see tokenList). */
async tokenListAll(opts: CliOptions = {}): Promise<TokenListRow[]> {
return this.tokenList('all', opts);
}
/** `help`: asserts exit 0, stdout exactly the usage text and (unless LOG_LEVEL is overridden) empty stderr. */
async help(opts: CliOptions = {}): Promise<CliResult> {
const r = await this.run(['help'], opts);
expectCliSuccess(r, { stdout: USAGE_TEXT });
if (!opts.env?.LOG_LEVEL) expect(r.stderr, 'help stderr').toBe('');
return r;
}
/**
* `audit [arg]` (a string = one argument, e.g. `'all'`, `'10'`, `'1h'`; an array = the arguments verbatim): asserts success
* (and empty stderr unless LOG_LEVEL is overridden) and returns the strictly parsed rows.
* The printed tokens are registered as secrets.
*/
async audit(arg?: string | string[], opts: CliOptions = {}): Promise<AuditRow[]> {
const r = await this.run(['audit', ...(arg === undefined ? [] : typeof arg === 'string' ? [arg] : arg)], opts);
expectCliSuccess(r);
if (!opts.env?.LOG_LEVEL) expect(r.stderr, 'audit stderr').toBe('');
return parseAuditOutput(r.stdout);
}
/** `token delete <token>`: asserts success with empty stdout (and stderr unless LOG_LEVEL is overridden). */
async tokenDelete(token: string, opts: CliOptions = {}): Promise<CliResult> {
const r = await this.run(['token', 'delete', token], opts);
expectCliSuccess(r, { stdout: '' });
if (!opts.env?.LOG_LEVEL) expect(r.stderr, 'token delete stderr').toBe('');
return r;
}
private record(r: CliResult, env: Record<string, string>): void {
this.ctx.recorder.addCli({
via: r.via,
command: r.command,
args: r.args,
env: recordableEnv(env),
exitCode: r.exitCode,
signal: r.signal,
timedOut: r.timedOut,
durationMs: Math.round(r.durationMs),
stdout: r.stdout,
stderr: r.stderr,
});
}
private recordError(via: CliVia, command: string, args: string[], env: Record<string, string>, e: unknown): void {
const d = (e as HarnessError).details as ProcResult | undefined;
if (d?.stdout && (via === 'shell' || printsTokens(args))) this.ctx.secrets.registerAllIn(d.stdout);
this.ctx.recorder.addCli({
via,
command,
args,
env: recordableEnv(env),
exitCode: d?.exitCode ?? null,
signal: d?.signal ?? null,
timedOut: d?.timedOut ?? false,
durationMs: Math.round(d?.durationMs ?? 0),
stdout: d?.stdout ?? '',
stderr: `${d?.stderr ?? ''}\n[harness] ${(e as Error).message}`,
});
}
}
+257
View File
@@ -0,0 +1,257 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Read-only SQLite inspection of the application database (node:sqlite).
*
* Tests never write to the application database through this module.
*/
import crypto from 'node:crypto';
import fs from 'node:fs';
import net from 'node:net';
import { DatabaseSync } from 'node:sqlite';
import { expect } from '@playwright/test';
import { HarnessError } from './errors';
import { TOKEN_RE } from './secrets';
export interface TokenRow {
token: string;
/** Description decoded as UTF-8 (lossy for invalid bytes; use descriptionHex for exact bytes). */
description: string;
/** Exact stored bytes of the description, upper-case hex (SQLite hex()). */
descriptionHex: string;
active: number;
created_at: string;
deleted_at: string | null;
}
export interface OperationRow {
rowid: number;
token: string;
origin: string;
operation: string;
time: string;
}
/**
* Strict UTC RFC3339Nano as produced by Go's time.RFC3339Nano: optional fraction of 1–9 digits
* without trailing zeros, always `Z`.
*/
export const RFC3339NANO_UTC_RE = /^\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])T([01]\d|2[0-3]):[0-5]\d:[0-5]\d(\.\d{0,8}[1-9])?Z$/;
/** Nanoseconds since the Unix epoch of a strict RFC3339Nano UTC timestamp. */
export function rfc3339ToNanos(ts: string): bigint {
const m = /^(.*T\d{2}:\d{2}:\d{2})(?:\.(\d{1,9}))?Z$/.exec(ts);
if (!m) throw new Error(`not an RFC3339 UTC timestamp: ${ts}`);
const ms = Date.parse(m[1] + 'Z');
if (Number.isNaN(ms)) throw new Error(`invalid timestamp: ${ts}`);
return BigInt(ms) * 1_000_000n + BigInt((m[2] ?? '').padEnd(9, '0') || '0');
}
/** Nanoseconds since the epoch for a JS Date / epoch-ms value. */
export function msToNanos(ms: number): bigint {
return BigInt(Math.floor(ms)) * 1_000_000n;
}
/** Opens the database read-only, runs fn, always closes. */
export function withDb<T>(dbPath: string, fn: (db: DatabaseSync) => T): T {
if (!fs.existsSync(dbPath)) throw new HarnessError('infrastructure', `database file does not exist: ${dbPath}`);
let db: DatabaseSync;
try {
db = new DatabaseSync(dbPath, { readOnly: true, timeout: 5000 });
} catch (e) {
throw new HarnessError('infrastructure', `cannot open ${dbPath} read-only: ${(e as Error).message}`);
}
try {
return fn(db);
} finally {
db.close();
}
}
export function readTokens(dbPath: string): TokenRow[] {
return withDb(dbPath, (db) =>
db
.prepare(`SELECT token, CAST(description AS TEXT) AS description, hex(description) AS descriptionHex, active, created_at, deleted_at FROM tokens ORDER BY rowid`)
.all()
.map((r) => ({ ...(r as unknown as TokenRow), active: Number((r as { active: unknown }).active) })),
);
}
export function readToken(dbPath: string, token: string): TokenRow | undefined {
return readTokens(dbPath).find((t) => t.token === token);
}
export function readOperations(dbPath: string): OperationRow[] {
return withDb(dbPath, (db) => db.prepare(`SELECT rowid, token, origin, operation, time FROM operations ORDER BY rowid`).all() as unknown as OperationRow[]);
}
export function countRows(dbPath: string, table: 'tokens' | 'operations'): number {
return withDb(dbPath, (db) => Number((db.prepare(`SELECT count(*) AS n FROM ${table}`).get() as { n: number }).n));
}
/** Expected `PRAGMA table_info` of both tables. */
export const EXPECTED_COLUMNS = {
tokens: [
{ name: 'token', type: 'TEXT', notnull: 1, dflt_value: null, pk: 1 },
{ name: 'description', type: 'TEXT', notnull: 1, dflt_value: "''", pk: 0 },
{ name: 'active', type: 'INTEGER', notnull: 1, dflt_value: null, pk: 0 },
{ name: 'created_at', type: 'TEXT', notnull: 1, dflt_value: null, pk: 0 },
{ name: 'deleted_at', type: 'TEXT', notnull: 0, dflt_value: null, pk: 0 },
],
operations: [
{ name: 'token', type: 'TEXT', notnull: 1, dflt_value: null, pk: 0 },
{ name: 'origin', type: 'TEXT', notnull: 1, dflt_value: null, pk: 0 },
{ name: 'operation', type: 'TEXT', notnull: 1, dflt_value: null, pk: 0 },
{ name: 'time', type: 'TEXT', notnull: 1, dflt_value: null, pk: 0 },
],
} as const;
export interface DbSchemaInfo {
tables: string[];
columns: Record<string, Array<{ name: string; type: string; notnull: number; dflt_value: string | null; pk: number }>>;
foreignKeys: Array<{ table: string; from: string; to: string }>;
indexes: string[];
}
export function readSchema(dbPath: string): DbSchemaInfo {
return withDb(dbPath, (db) => {
const tables = (db.prepare(`SELECT name FROM sqlite_master WHERE type='table' ORDER BY name`).all() as Array<{ name: string }>).map((r) => r.name);
const columns: DbSchemaInfo['columns'] = {};
for (const t of tables) {
columns[t] = (db.prepare(`PRAGMA table_info(${JSON.stringify(t)})`).all() as Array<Record<string, unknown>>).map((c) => ({
name: String(c.name),
type: String(c.type),
notnull: Number(c.notnull),
dflt_value: c.dflt_value === null ? null : String(c.dflt_value),
pk: Number(c.pk),
}));
}
const foreignKeys = (db.prepare(`PRAGMA foreign_key_list(operations)`).all() as Array<Record<string, unknown>>).map((f) => ({
table: String(f.table),
from: String(f.from),
to: String(f.to),
}));
const indexes = (db.prepare(`SELECT name FROM sqlite_master WHERE type='index' ORDER BY name`).all() as Array<{ name: string }>).map((r) => r.name);
return { tables, columns, foreignKeys, indexes };
});
}
export interface DbInvariantOptions {
/** Also require every audit row time to be <= its token's deleted_at (false by default: requests racing deletion may legitimately audit after). */
auditBeforeDeletion?: boolean;
/** Expected exact number of token rows. */
tokenCount?: number;
/** Expected exact number of operations rows. */
operationCount?: number;
}
export interface DbInvariantReport {
tokens: number;
activeTokens: number;
inactiveTokens: number;
operations: number;
}
/**
* Asserts every structural invariant of the application database (Playwright expect, so a
* violation is a test failure with a precise message):
* integrity_check = ok; foreign_key_check empty; exact schema (tables, columns, FK); tokens
* well-formed (43-char base64url, typeof text/integer), active ∈ {0,1}, deleted_at NULL iff
* active = 1; created_at/deleted_at/time strict RFC3339Nano UTC; deleted_at >= created_at;
* every operations.token exists in tokens; operation always 'GET containers'; origin is an IP
* literal; time >= the token's created_at.
*/
export function assertDbInvariants(dbPath: string, opts: DbInvariantOptions = {}): DbInvariantReport {
expect(fs.existsSync(dbPath), `database ${dbPath} exists`).toBe(true);
const { integrity, fkViolations, typeErrors } = withDb(dbPath, (db) => ({
integrity: (db.prepare('PRAGMA integrity_check').all() as Array<Record<string, unknown>>).map((r) => Object.values(r)[0]),
fkViolations: db.prepare('PRAGMA foreign_key_check').all(),
typeErrors: db
.prepare(
`SELECT 'tokens' AS t, rowid FROM tokens WHERE typeof(token) <> 'text' OR typeof(description) <> 'text' OR typeof(active) <> 'integer'
OR typeof(created_at) <> 'text' OR (deleted_at IS NOT NULL AND typeof(deleted_at) <> 'text')
UNION ALL
SELECT 'operations', rowid FROM operations WHERE typeof(token) <> 'text' OR typeof(origin) <> 'text' OR typeof(operation) <> 'text' OR typeof(time) <> 'text'`,
)
.all(),
}));
expect(integrity, 'PRAGMA integrity_check').toEqual(['ok']);
expect(fkViolations, 'PRAGMA foreign_key_check').toEqual([]);
expect(typeErrors, 'column storage classes').toEqual([]);
const schema = readSchema(dbPath);
expect(schema.tables, 'tables').toEqual(['operations', 'tokens']);
expect(schema.columns.tokens, 'tokens columns').toEqual(EXPECTED_COLUMNS.tokens);
expect(schema.columns.operations, 'operations columns').toEqual(EXPECTED_COLUMNS.operations);
expect(schema.foreignKeys, 'operations foreign key').toEqual([{ table: 'tokens', from: 'token', to: 'token' }]);
expect(schema.indexes, 'indexes').toEqual(['sqlite_autoindex_tokens_1']);
const tokens = readTokens(dbPath);
const ops = readOperations(dbPath);
const byToken = new Map(tokens.map((t) => [t.token, t]));
const problems: string[] = [];
for (const t of tokens) {
const id = `token row ${t.token.slice(0, 4)}…`;
if (!TOKEN_RE.test(t.token)) problems.push(`${id}: token not 43-char base64url`);
if (t.active !== 0 && t.active !== 1) problems.push(`${id}: active=${t.active}`);
if (!RFC3339NANO_UTC_RE.test(t.created_at)) problems.push(`${id}: created_at ${JSON.stringify(t.created_at)} not strict RFC3339Nano UTC`);
if (t.active === 1 && t.deleted_at !== null) problems.push(`${id}: active but deleted_at=${t.deleted_at}`);
if (t.active === 0 && t.deleted_at === null) problems.push(`${id}: inactive but deleted_at NULL`);
if (t.deleted_at !== null) {
if (!RFC3339NANO_UTC_RE.test(t.deleted_at)) problems.push(`${id}: deleted_at ${JSON.stringify(t.deleted_at)} not strict RFC3339Nano UTC`);
else if (RFC3339NANO_UTC_RE.test(t.created_at) && rfc3339ToNanos(t.deleted_at) < rfc3339ToNanos(t.created_at)) problems.push(`${id}: deleted_at before created_at`);
}
}
for (const o of ops) {
const id = `operation rowid ${o.rowid}`;
const t = byToken.get(o.token);
if (!t) problems.push(`${id}: token not in tokens`);
if (o.operation !== 'GET containers') problems.push(`${id}: operation=${JSON.stringify(o.operation)}`);
if (net.isIP(o.origin) === 0) problems.push(`${id}: origin ${JSON.stringify(o.origin)} is not an IP literal`);
if (!RFC3339NANO_UTC_RE.test(o.time)) problems.push(`${id}: time ${JSON.stringify(o.time)} not strict RFC3339Nano UTC`);
else if (t && RFC3339NANO_UTC_RE.test(t.created_at)) {
if (rfc3339ToNanos(o.time) < rfc3339ToNanos(t.created_at)) problems.push(`${id}: audited before its token was created`);
if (opts.auditBeforeDeletion && t.deleted_at && rfc3339ToNanos(o.time) > rfc3339ToNanos(t.deleted_at)) problems.push(`${id}: audited after its token was deleted`);
}
}
expect(problems, 'database invariants').toEqual([]);
if (opts.tokenCount !== undefined) expect(tokens.length, 'token row count').toBe(opts.tokenCount);
if (opts.operationCount !== undefined) expect(ops.length, 'operations row count').toBe(opts.operationCount);
const active = tokens.filter((t) => t.active === 1).length;
return { tokens: tokens.length, activeTokens: active, inactiveTokens: tokens.length - active, operations: ops.length };
}
/** Exact byte image of the main database file (for "not modified" checks; checkpointed state only). */
export function fileDigest(path: string): string | undefined {
if (!fs.existsSync(path)) return undefined;
return crypto.createHash('sha256').update(fs.readFileSync(path)).digest('hex');
}
/**
* Textual snapshot for artifacts: integrity summary plus every row. Token values must be
* redacted by the caller (pass a redact function).
*/
export function dbSnapshot(dbPath: string, redact: (s: string) => string): string {
if (!fs.existsSync(dbPath)) return `${dbPath}: does not exist\n`;
try {
const lines: string[] = [`database: ${dbPath}`];
withDb(dbPath, (db) => {
lines.push(`integrity_check: ${JSON.stringify(db.prepare('PRAGMA integrity_check').all())}`);
lines.push(`foreign_key_check: ${JSON.stringify(db.prepare('PRAGMA foreign_key_check').all())}`);
lines.push(`schema:`);
for (const r of db.prepare(`SELECT type, name, sql FROM sqlite_master ORDER BY name`).all() as Array<Record<string, unknown>>) {
lines.push(` ${r.type} ${r.name}: ${String(r.sql ?? '').replace(/\s+/g, ' ')}`);
}
});
const tokens = readTokens(dbPath);
lines.push(`tokens (${tokens.length} rows): token | description(JSON) | active | created_at | deleted_at`);
for (const t of tokens.slice(0, 1000)) lines.push(` ${t.token} | ${JSON.stringify(t.description)} | ${t.active} | ${t.created_at} | ${t.deleted_at}`);
const ops = readOperations(dbPath);
lines.push(`operations (${ops.length} rows): rowid | token | origin | operation | time`);
for (const o of ops.slice(-1000)) lines.push(` ${o.rowid} | ${o.token} | ${o.origin} | ${o.operation} | ${o.time}`);
return redact(lines.join('\n') + '\n');
} catch (e) {
return `${dbPath}: snapshot failed: ${(e as Error).message}\n`;
}
}
+169
View File
@@ -0,0 +1,169 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Real Docker Engine helpers: uniquely named, labelled test containers with guaranteed cleanup.
*
* Every container is named `dpsapi-e2e-<runId>-<testKey>-<suffix>` and labelled
* `dpsapi.e2e=<runId>` + `dpsapi.e2e.test=<testKey>`. Assertions must only ever look at these
* names/labels — the host runs unrelated containers.
*/
import { ALPINE_IMAGE, LABEL_RUN, LABEL_TEST, RESOURCE_PREFIX, runId } from './env';
import { HarnessError } from './errors';
import { mapLimit, runProcess, toolEnv, type ProcResult } from './proc';
export interface DockerCliOptions {
/** Default 60 s. */
timeoutMs?: number;
/** Return non-zero results instead of throwing HarnessError('infrastructure'). */
allowFailure?: boolean;
input?: string | Buffer;
}
/** Runs the docker CLI. Non-zero exit is a HarnessError unless allowFailure. */
export async function dockerCli(args: string[], opts: DockerCliOptions = {}): Promise<ProcResult> {
const r = await runProcess('docker', args, { env: toolEnv(), timeoutMs: opts.timeoutMs ?? 60_000, input: opts.input, label: `docker ${args[0]}` });
if (r.exitCode !== 0 && !opts.allowFailure) {
throw new HarnessError('infrastructure', `docker ${args.join(' ')} failed (${r.exitCode}): ${r.stderr.trim()}`, r);
}
return r;
}
export interface TestContainer {
id: string;
name: string;
image: string;
labels: Record<string, string>;
command: string[];
}
export interface RunContainerOptions {
/** Name suffix (default c<n>); final name dpsapi-e2e-<runId>-<testKey>-<suffix>. */
suffix?: string;
/** Default alpine:3.22. */
image?: string;
/** Default ['sleep', '3600']. */
command?: string[];
/** Extra labels (the run/test labels are always added). */
labels?: Record<string, string>;
env?: Record<string, string>;
/** --init so SIGTERM stops sleep-style commands promptly (default true). */
init?: boolean;
/** Default 'none'. */
network?: string;
extraArgs?: string[];
}
export interface OwnContainer {
id: string;
name: string;
state: string;
labels: Record<string, string>;
}
let seq = 0;
export class RealDocker {
readonly runId = runId();
/** Prefix of every container name created through this instance. */
readonly namePrefix: string;
private readonly created = new Set<string>();
constructor(readonly testKey: string) {
this.namePrefix = `${RESOURCE_PREFIX}${this.runId}-${testKey}-`;
}
/** Full container name for a suffix. */
name(suffix: string): string {
return this.namePrefix + suffix;
}
/** Run + test labels (+ extra). */
labels(extra: Record<string, string> = {}): Record<string, string> {
return { [LABEL_RUN]: this.runId, [LABEL_TEST]: this.testKey, ...extra };
}
/** `docker run -d` a labelled container and return its identity. */
async run(opts: RunContainerOptions = {}): Promise<TestContainer> {
const suffix = opts.suffix ?? `c${++seq}`;
if (!/^[a-zA-Z0-9][a-zA-Z0-9_.-]*$/.test(suffix)) throw new HarnessError('infrastructure', `invalid container suffix ${suffix}`);
const name = this.name(suffix);
const image = opts.image ?? ALPINE_IMAGE;
const command = opts.command ?? ['sleep', '3600'];
const labels = this.labels(opts.labels);
const args = ['run', '-d', '--name', name, '--network', opts.network ?? 'none'];
if (opts.init ?? true) args.push('--init');
for (const [k, v] of Object.entries(labels)) args.push('--label', `${k}=${v}`);
for (const [k, v] of Object.entries(opts.env ?? {})) args.push('-e', `${k}=${v}`);
args.push(...(opts.extraArgs ?? []), image, ...command);
this.created.add(name);
const r = await dockerCli(args);
return { id: r.stdout.trim(), name, image, labels, command };
}
/** Starts n containers with bounded concurrency; optsFor(i) customizes each. */
runMany(n: number, optsFor: (i: number) => RunContainerOptions = () => ({}), concurrency = 6): Promise<TestContainer[]> {
return mapLimit(
Array.from({ length: n }, (_, i) => i),
concurrency,
(i) => this.run(optsFor(i)),
);
}
/** docker stop -t <timeoutSec> (default 2). */
async stop(c: TestContainer | string, timeoutSec = 2): Promise<void> {
await dockerCli(['stop', '-t', String(timeoutSec), typeof c === 'string' ? c : c.name]);
}
async start(c: TestContainer | string): Promise<void> {
await dockerCli(['start', typeof c === 'string' ? c : c.name]);
}
/** docker rm (-f by default). */
async remove(c: TestContainer | string, force = true): Promise<void> {
await dockerCli(['rm', ...(force ? ['-f'] : []), typeof c === 'string' ? c : c.name]);
}
async inspect(c: TestContainer | string): Promise<Record<string, unknown>> {
const r = await dockerCli(['inspect', typeof c === 'string' ? c : c.name]);
return (JSON.parse(r.stdout) as Array<Record<string, unknown>>)[0];
}
/** Containers (any state by default) carrying this test's label. */
async listOwn({ all = true }: { all?: boolean } = {}): Promise<OwnContainer[]> {
return listByLabel(`${LABEL_TEST}=${this.testKey}`, all);
}
/** Full IDs of this test's running containers (what `docker ps` would show). */
async runningIds(): Promise<string[]> {
return (await this.listOwn({ all: false })).map((c) => c.id);
}
/** Force-removes every container carrying this test's label (and any name created here). Returns removed names. */
async cleanup(): Promise<string[]> {
const own = await this.listOwn();
const names = new Set([...own.map((c) => c.name)]);
if (names.size === 0) return [];
await dockerCli(['rm', '-f', '-v', ...names], { allowFailure: true, timeoutMs: 120_000 });
const left = await this.listOwn();
if (left.length) throw new HarnessError('leak', `containers survived cleanup: ${left.map((c) => c.name).join(', ')}`);
return [...names];
}
}
/** docker ps [-a] --filter label=<label> (label 'k=v' or 'k'). */
export async function listByLabel(label: string, all = true): Promise<OwnContainer[]> {
const r = await dockerCli(['ps', ...(all ? ['-a'] : []), '--no-trunc', '--filter', `label=${label}`, '--format', '{{json .}}']);
return r.stdout
.split('\n')
.filter((l) => l.trim())
.map((l) => {
const j = JSON.parse(l) as { ID: string; Names: string; State: string; Labels: string };
const labels: Record<string, string> = {};
for (const kv of j.Labels ? j.Labels.split(',') : []) {
const i = kv.indexOf('=');
if (i > 0) labels[kv.slice(0, i)] = kv.slice(i + 1);
}
return { id: j.ID, name: j.Names, state: j.State, labels };
});
}
+80
View File
@@ -0,0 +1,80 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Run-level constants and environment resolution shared by the runner process,
* global setup/teardown and every worker.
*/
import path from 'node:path';
import { HarnessError } from './errors';
/** Directory containing playwright.config.ts. */
export const E2E_ROOT = path.resolve(__dirname, '..');
/** Repository root (parent of e2e/). */
export const REPO_ROOT = path.resolve(E2E_ROOT, '..');
/** Every Docker resource / temp directory created by the suite starts with this. */
export const RESOURCE_PREFIX = 'dpsapi-e2e-';
/** Docker label carrying the run id on every resource the suite creates. */
export const LABEL_RUN = 'dpsapi.e2e';
/** Docker label carrying the per-test key on every resource the suite creates. */
export const LABEL_TEST = 'dpsapi.e2e.test';
/**
* Environment variable set (to the run id) on every process the harness spawns.
* Global teardown scans /proc/<pid>/environ for it to find leaked processes.
* It is deliberately NOT present in the runner/worker environment.
*/
export const PROC_MARKER_ENV = 'DPSAPI_E2E_PROC';
/** Image used for throwaway/test containers. Must already be present locally. */
export const ALPINE_IMAGE = process.env.DPSAPI_E2E_ALPINE || 'alpine:3.22';
/** Fixed default seed used when DPSAPI_E2E_SEED is not set. */
export const DEFAULT_SEED = 'dpsapi-e2e-default-seed-1';
/**
* Absolute path of the binary under test: $DPSAPI_BIN (relative paths are resolved
* against the e2e/ directory) or ../dpsapi.
*/
export function binaryPath(): string {
return path.resolve(E2E_ROOT, process.env.DPSAPI_BIN || '../dpsapi');
}
/** True when DPSAPI_BIN was not given, i.e. the suite owns building ../dpsapi. */
export function binaryIsDefault(): boolean {
return !process.env.DPSAPI_BIN;
}
/** Image reference under test ($DPSAPI_IMAGE), or undefined. */
export function imageRef(): string | undefined {
return process.env.DPSAPI_IMAGE || undefined;
}
/** Run id assigned by global setup (short, [a-z0-9]). */
export function runId(): string {
const id = process.env.DPSAPI_E2E_RUN_ID;
if (!id) throw new HarnessError('infrastructure', 'DPSAPI_E2E_RUN_ID is not set: global setup did not run');
return id;
}
/** Per-run scratch root (workspaces, secret registry files). Created by global setup. */
export function runDir(): string {
const dir = process.env.DPSAPI_E2E_RUN_DIR;
if (!dir) throw new HarnessError('infrastructure', 'DPSAPI_E2E_RUN_DIR is not set: global setup did not run');
return dir;
}
/** Directory holding all per-test workspaces of this run. */
export function workspacesDir(): string {
return path.join(runDir(), 'ws');
}
/** Directory holding per-test secret registries (read by the redacting reporter). */
export function secretsDir(): string {
return path.join(runDir(), 'secrets');
}
/** Global PRNG seed: $DPSAPI_E2E_SEED or DEFAULT_SEED. */
export function globalSeed(): string {
return process.env.DPSAPI_E2E_SEED || DEFAULT_SEED;
}
+38
View File
@@ -0,0 +1,38 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Test-infrastructure errors.
*
* A HarnessError means "the harness could not obtain a trustworthy observation"
* (timeout, unexpected signal, hung process, spawn failure, leaked resource,
* missing prerequisite, unparseable protocol data). It is never a valid outcome
* for an expected-failure assertion: helpers throw it instead of returning a
* result, so a negative test can never pass because something crashed or hung.
*/
export type HarnessErrorKind =
| 'timeout' // a process / request / wait exceeded its deadline
| 'signal' // a process was terminated by a signal nobody asked for
| 'spawn' // a process could not be started
| 'hang' // a process did not exit after being asked to
| 'leak' // a resource (process, container, directory) outlived its owner
| 'prerequisite' // the environment is missing something the suite needs
| 'protocol' // a peer produced data the harness cannot parse
| 'transport' // an HTTP client failed below HTTP (refused, reset) where a response was required
| 'infrastructure'; // anything else that is the harness's fault, not the app's
export class HarnessError extends Error {
readonly kind: HarnessErrorKind;
readonly details?: unknown;
constructor(kind: HarnessErrorKind, message: string, details?: unknown) {
super(`[harness:${kind}] ${message}`);
this.name = 'HarnessError';
this.kind = kind;
this.details = details;
}
}
export function isHarnessError(e: unknown): e is HarnessError {
return e instanceof HarnessError;
}
+269
View File
@@ -0,0 +1,269 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* In-process fake Docker Engine API on a unix socket.
*
* Answers `HEAD/GET /_ping` and `GET [/vX.Y]/containers/json`; anything else gets a 404 Engine
* error. The behaviour of container listing is switchable live (the socket file is never
* recreated, so bind mounts that pinned its inode keep working).
*/
import fs from 'node:fs';
import http from 'node:http';
import type { Socket } from 'node:net';
import { HarnessError } from './errors';
import type { Rng } from './rng';
/** Subset of the Engine API container summary (GET /containers/json) used by the fake. */
export interface ContainerSummary {
Id: string;
Names: string[];
Image: string;
ImageID: string;
Command: string;
Created: number;
Ports: Array<{ IP?: string; PrivatePort: number; PublicPort?: number; Type: string }>;
Labels: Record<string, string>;
State: string;
Status: string;
HostConfig: { NetworkMode: string };
NetworkSettings: { Networks: Record<string, unknown> };
Mounts: unknown[];
[extra: string]: unknown;
}
/** Plausible running-container summary number i (deterministic unless an rng is given). */
export function makeContainerSummary(i: number, rng?: Rng, overrides: Partial<ContainerSummary> = {}): ContainerSummary {
const id = rng ? rng.bytes(32).toString('hex') : (i + 1).toString(16).padStart(64, '0');
const images = ['alpine:3.22', 'nginx:1.29', 'postgres:18', 'redis:8', 'busybox:1.37', 'example.org/team/app:v' + (i % 7)];
const image = rng ? rng.pick(images) : images[i % images.length];
return {
Id: id,
Names: [`/fake-${i}`],
Image: image,
ImageID: 'sha256:' + id,
Command: i % 3 === 0 ? 'sleep infinity' : i % 3 === 1 ? '/docker-entrypoint.sh nginx -g \'daemon off;\'' : 'sh -c "while :; do sleep 1; done"',
Created: 1_760_000_000 + i,
Ports: i % 4 === 0 ? [{ IP: '127.0.0.1', PrivatePort: 80, PublicPort: 30000 + (i % 1000), Type: 'tcp' }] : [],
Labels: { 'fake.index': String(i), 'com.example.tier': i % 2 ? 'web' : 'db' },
State: 'running',
Status: `Up ${1 + (i % 59)} minutes`,
HostConfig: { NetworkMode: 'bridge' },
NetworkSettings: { Networks: {} },
Mounts: [],
...overrides,
};
}
/** n plausible summaries: makeContainerSummary(0..n-1). */
export function makeContainerSummaries(n: number, rng?: Rng): ContainerSummary[] {
return Array.from({ length: n }, (_, i) => makeContainerSummary(i, rng));
}
/** Behaviour of `GET /containers/json`. */
export type FakeDockerMode =
| { kind: 'containers' } // 200 with `fake.containers`
| { kind: 'error'; status: number; body: string } // Engine error response
| { kind: 'destroy' } // drop the connection without a response
| { kind: 'hang' } // never answer (connection held until the fake closes)
| { kind: 'badjson'; body: string } // 200 with an unparseable body
| { kind: 'passthrough'; upstream: string }; // proxy everything to a real Engine socket
export interface FakeDockerRequest {
method: string;
url: string;
/** Path without the /vX.Y prefix and query. */
path: string;
query: string;
at: number;
}
export interface FakeDockerStats {
ping: number;
list: number;
other: number;
total: number;
}
export class FakeDocker {
/** Containers returned in 'containers' mode (and after flaky failures). */
containers: ContainerSummary[] = [];
mode: FakeDockerMode = { kind: 'containers' };
/** The next N list calls fail with 500 before `mode` applies ("temporarily failing"). */
failNext = 0;
/** Delay (ms) before answering list calls. */
delayMs = 0;
/** API version advertised by /_ping. */
apiVersion = '1.56';
readonly requests: FakeDockerRequest[] = [];
readonly stats: FakeDockerStats = { ping: 0, list: 0, other: 0, total: 0 };
private readonly server: http.Server;
private readonly sockets = new Set<Socket>();
private closed = false;
private constructor(readonly socketPath: string) {
this.server = http.createServer((req, res) => this.handle(req, res));
this.server.on('connection', (s) => {
this.sockets.add(s);
s.on('close', () => this.sockets.delete(s));
});
}
/** Starts a fake listening on socketPath (any stale file there is removed first). Starts in empty mode. */
static async start(socketPath: string): Promise<FakeDocker> {
if (Buffer.byteLength(socketPath) > 100) throw new HarnessError('infrastructure', `unix socket path too long: ${socketPath}`);
fs.rmSync(socketPath, { force: true });
const f = new FakeDocker(socketPath);
await new Promise<void>((resolve, reject) => {
f.server.once('error', reject);
f.server.listen(socketPath, () => resolve());
});
return f;
}
// ---- mode switches (live) ----
/** 200 `[]`. */
empty(): this {
return this.setContainers([]);
}
/** 200 with n generated summaries. */
many(n: number, rng?: Rng): this {
return this.setContainers(makeContainerSummaries(n, rng));
}
/** 200 with the given summaries. */
setContainers(list: ContainerSummary[]): this {
this.containers = list;
this.mode = { kind: 'containers' };
return this;
}
/** Engine error (default 500 `{"message":"fake engine failure"}`). */
error(status = 500, message = 'fake engine failure: /var/lib/docker internal detail'): this {
this.mode = { kind: 'error', status, body: JSON.stringify({ message }) };
return this;
}
/** Alias of error(500). */
error500(message?: string): this {
return this.error(500, message);
}
/** The next k list calls fail with 500, then the current mode applies again. */
flaky(k: number): this {
this.failNext = k;
return this;
}
destroy(): this {
this.mode = { kind: 'destroy' };
return this;
}
hang(): this {
this.mode = { kind: 'hang' };
return this;
}
badJson(body = '{"not":"an array"'): this {
this.mode = { kind: 'badjson', body };
return this;
}
passthrough(upstream = '/var/run/docker.sock'): this {
this.mode = { kind: 'passthrough', upstream };
return this;
}
delay(ms: number): this {
this.delayMs = ms;
return this;
}
/** Number of container-list calls received so far. */
listCalls(): number {
return this.stats.list;
}
resetStats(): void {
this.requests.length = 0;
this.stats.ping = this.stats.list = this.stats.other = this.stats.total = 0;
}
/** Closes the listener and drops every open connection (hung requests included). */
async close(): Promise<void> {
if (this.closed) return;
this.closed = true;
for (const s of this.sockets) s.destroy();
await new Promise<void>((r) => this.server.close(() => r()));
fs.rmSync(this.socketPath, { force: true });
}
private handle(req: http.IncomingMessage, res: http.ServerResponse): void {
const url = new URL(req.url ?? '/', 'http://fake');
const path = url.pathname.replace(/^\/v[0-9.]+(?=\/)/, '');
this.requests.push({ method: req.method ?? '', url: req.url ?? '', path, query: url.search, at: Date.now() });
this.stats.total++;
if (path === '/_ping') this.stats.ping++;
else if (path === '/containers/json') this.stats.list++;
else this.stats.other++;
const mode = this.mode;
if (mode.kind === 'passthrough') return this.proxy(req, res, mode.upstream);
if (path === '/_ping' && (req.method === 'GET' || req.method === 'HEAD')) {
res.writeHead(200, {
'Api-Version': this.apiVersion,
'Content-Type': 'text/plain; charset=utf-8',
'Docker-Experimental': 'false',
Ostype: 'linux',
Server: 'Docker/fake (linux)',
});
res.end(req.method === 'HEAD' ? undefined : 'OK');
return;
}
if (path === '/containers/json' && req.method === 'GET') {
const answer = () => this.answerList(res, req);
if (this.delayMs > 0 && mode.kind !== 'hang') setTimeout(answer, this.delayMs);
else answer();
return;
}
this.json(res, 404, { message: `page not found: ${req.method} ${path}` });
}
private answerList(res: http.ServerResponse, req: http.IncomingMessage): void {
if (this.closed) return;
if (this.failNext > 0) {
this.failNext--;
this.json(res, 500, { message: 'fake engine temporarily failing' });
return;
}
const mode = this.mode;
switch (mode.kind) {
case 'containers':
this.json(res, 200, this.containers);
return;
case 'error':
res.writeHead(mode.status, { 'Content-Type': 'application/json', 'Api-Version': this.apiVersion });
res.end(mode.body);
return;
case 'destroy':
req.socket.destroy();
return;
case 'hang':
return; // held until close()
case 'badjson':
res.writeHead(200, { 'Content-Type': 'application/json', 'Api-Version': this.apiVersion });
res.end(mode.body);
return;
case 'passthrough':
return; // handled earlier
}
}
private json(res: http.ServerResponse, status: number, body: unknown): void {
const data = JSON.stringify(body);
res.writeHead(status, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data), 'Api-Version': this.apiVersion });
res.end(data);
}
private proxy(req: http.IncomingMessage, res: http.ServerResponse, upstream: string): void {
const up = http.request({ socketPath: upstream, method: req.method, path: req.url, headers: req.headers }, (ur) => {
res.writeHead(ur.statusCode ?? 502, ur.headers);
ur.pipe(res);
});
up.on('error', () => req.socket.destroy());
req.pipe(up);
}
}
+323
View File
@@ -0,0 +1,323 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Playwright fixtures: per-test isolation, automatic cleanup, leak detection, secret scanning
* and failure artifacts. Import `test`/`expect` from 'harness' (index.ts), never from
* '@playwright/test', in spec files.
*/
import crypto from 'node:crypto';
import fs from 'node:fs';
import path from 'node:path';
import { test as base, expect, type TestInfo } from '@playwright/test';
import { CliRunner } from './cli';
import { dbSnapshot } from './db';
import { listByLabel, RealDocker, dockerCli } from './docker';
import { globalSeed, LABEL_TEST, secretsDir } from './env';
import { HarnessError } from './errors';
import { FakeDocker } from './fake-docker';
import { HttpClient } from './http';
import { ImageRunner, type ImageContainer } from './image';
import { killGroup, liveProcesses } from './proc';
import { jsonSafe, Recorder } from './recorder';
import { Rng, testSeed } from './rng';
import { redactText, SecretRegistry } from './secrets';
import { startServer, type ServerHandle, type StartServerOptions } from './server';
import { allocatePort, forceRemoveDir, Workspace } from './workspace';
/** Everything the harness tracks for one test. */
export interface E2EContext {
/** Short unique key of this test execution (8 hex); used in Docker names/labels and workspace names. */
testKey: string;
workspace: Workspace;
secrets: SecretRegistry;
recorder: Recorder;
servers: ServerHandle[];
fakes: FakeDocker[];
imageContainers: ImageContainer[];
/** Extra database files to snapshot on failure (workspace db and server dbs are automatic). */
trackedDbs: Set<string>;
/** Set when the `docker`/`image` fixtures were used (enables label-based container cleanup). */
usedDocker: boolean;
/** Seed of this test's rng/fuzz streams. */
seed: string;
/** Allocates a free TCP port (parallel-safe). */
allocatePort(): Promise<number>;
/** Starts a FakeDocker on a new socket in the workspace; closed automatically. */
newFakeDocker(): Promise<FakeDocker>;
}
/** Records fuzz cases; on failure they are attached (redacted) as fuzz-cases.json together with the seed. */
export class FuzzRecorder {
constructor(
/** Seed of this test's stream (reproduce with DPSAPI_E2E_SEED=<run seed>). */
readonly seed: string,
/** Deterministic generator for this test. */
readonly rng: Rng,
private readonly recorder: Recorder,
private readonly testInfo: TestInfo,
private readonly secrets: SecretRegistry,
) {}
/**
* Records one generated case and returns it unchanged. What is stored is a JSON-safe snapshot
* taken now (BigInts as decimal strings), so the fuzz-cases.json attachment always serializes;
* fields added to the case object later are not recorded — record the finished case, or use
* `recordLive` for a case object that is filled in while the step runs.
*/
record<T>(c: T): T {
this.recorder.fuzzCases.push(jsonSafe(c));
return c;
}
/** Records a mutable case object by reference (serialized with BigInt support when attached). */
recordLive<T extends object>(c: T): T {
this.recorder.fuzzCases.push(c);
return c;
}
get cases(): readonly unknown[] {
return this.recorder.fuzzCases;
}
/** Attaches the cases now (normally only done automatically on failure). */
async attach(name = 'fuzz-cases.json'): Promise<void> {
await this.testInfo.attach(name, {
body: JSON.stringify(this.secrets.redactValue({ seed: this.seed, runSeed: globalSeed(), cases: this.recorder.fuzzCases }), null, 2),
contentType: 'application/json',
});
}
}
export interface E2EOptions {
/** Fail the test if a registered token appears in any captured server log / CLI stderr / container log (default true). */
scanLogsForSecrets: boolean;
}
export interface E2EFixtures {
/** Per-test harness context (auto fixture; owns teardown, leak checks and artifacts). */
e2e: E2EContext;
/** Short unique key of this test execution. */
testKey: string;
/** Isolated temp directory; `ws.dbPath` is the default DB_PATH. Removed after the test. */
ws: Workspace;
secrets: SecretRegistry;
recorder: Recorder;
/** Host binary as CLI (no shell; `cli.shell()` for shell-quoting tests). */
cli: CliRunner;
/** Starts the host binary in server mode; must be stopped by the test (leak = failure). */
startServer: (opts?: StartServerOptions) => Promise<ServerHandle>;
/** Starts an additional FakeDocker (closed automatically). */
newFakeDocker: () => Promise<FakeDocker>;
/** Playwright request / curl / raw-socket HTTP clients, all recorded. */
http: HttpClient;
/** Real Docker Engine helpers scoped to this test (cleanup guaranteed). */
docker: RealDocker;
/** Image helpers ($DPSAPI_IMAGE); containers removed automatically. */
image: ImageRunner;
/** Deterministic PRNG for this test (seed annotated on the test). */
rng: Rng;
fuzz: FuzzRecorder;
}
/** Registry of the test currently running in this worker (tests run one at a time per worker). */
let currentSecrets: SecretRegistry | undefined;
let stdioWrapped = false;
/**
* Redacts registered secrets and token-shaped strings from everything the test process writes to
* stdout/stderr (console.log in tests), which reporters would otherwise stream verbatim.
*/
function installStdioRedaction(): void {
if (stdioWrapped) return;
stdioWrapped = true;
for (const stream of [process.stdout, process.stderr]) {
const original = stream.write.bind(stream) as (...a: unknown[]) => boolean;
stream.write = ((chunk: unknown, ...rest: unknown[]) => {
const text = typeof chunk === 'string' ? chunk : chunk instanceof Uint8Array ? Buffer.from(chunk).toString('utf8') : undefined;
return original(text === undefined ? chunk : redactText(text, currentSecrets?.all() ?? []), ...rest);
}) as typeof stream.write;
}
}
function keyFor(info: TestInfo): string {
return crypto.createHash('sha256').update(`${info.testId}:${info.repeatEachIndex}:${info.retry}:${process.pid}`).digest('hex').slice(0, 8);
}
export const test = base.extend<E2EFixtures & E2EOptions>({
scanLogsForSecrets: [true, { option: true }],
e2e: [
async ({ scanLogsForSecrets }, use, testInfo) => {
const testKey = keyFor(testInfo);
const workspace = Workspace.create(testKey);
const secrets = new SecretRegistry(path.join(secretsDir(), `${testKey}.jsonl`));
installStdioRedaction();
currentSecrets = secrets;
const recorder = new Recorder();
const seed = testSeed(globalSeed(), testInfo.titlePath);
let fakeSeq = 0;
const ctx: E2EContext = {
testKey,
workspace,
secrets,
recorder,
servers: [],
fakes: [],
imageContainers: [],
trackedDbs: new Set(),
usedDocker: false,
seed,
allocatePort: () => allocatePort(testInfo.parallelIndex),
newFakeDocker: async () => {
const f = await FakeDocker.start(workspace.path(`fd${++fakeSeq}.sock`));
ctx.fakes.push(f);
return f;
},
};
await use(ctx);
const problems: string[] = [];
const attachments: Array<{ name: string; body: string; contentType: string }> = [];
const red = (s: string) => secrets.redact(s);
// 1. Server processes must have been stopped by the test.
for (const s of ctx.servers) {
if (s.isRunning()) {
problems.push(`server ${s.label} (pid ${s.pid}) was still running at teardown — tests must stop every server they start`);
await s.kill().catch((e) => problems.push(`could not kill server ${s.label}: ${(e as Error).message}`));
}
}
// 2. Image containers: collect logs, then remove.
const containerLogs: Array<{ name: string; stdout: string; stderr: string }> = [];
for (const c of ctx.imageContainers) {
const l = await c.logs().catch(() => ({ stdout: '', stderr: '' }));
containerLogs.push({ name: c.name, ...l });
await c.remove().catch(() => undefined);
}
// 3. Fake Docker sockets.
for (const f of ctx.fakes) await f.close().catch(() => undefined);
// 4. Every container labelled for this test (test containers, image containers, helpers).
if (ctx.usedDocker) {
try {
const left = await listByLabel(`${LABEL_TEST}=${testKey}`, true);
if (left.length) await dockerCli(['rm', '-f', '-v', ...left.map((c) => c.name)], { allowFailure: true, timeoutMs: 120_000 });
const still = await listByLabel(`${LABEL_TEST}=${testKey}`, true);
if (still.length) problems.push(`containers survived cleanup: ${still.map((c) => c.name).join(', ')}`);
} catch (e) {
problems.push(`docker cleanup failed: ${(e as Error).message}`);
}
}
// 5. Secret scan of everything the application wrote to logs.
if (scanLogsForSecrets) {
const scan = (text: string, where: string) => {
const leaked = secrets.findIn(text);
if (leaked.length) problems.push(`token secret(s) ${leaked.join(', ')} found in ${where}`);
};
for (const s of ctx.servers) for (const g of s.allOutput()) {
scan(g.stderr, `server ${s.label} gen ${g.generation} stderr`);
scan(g.stdout, `server ${s.label} gen ${g.generation} stdout`);
}
for (const [i, c] of recorder.cli.entries()) scan(c.stderr, `stderr of CLI invocation #${i + 1} (${c.via} ${c.args.slice(0, 2).join(' ')})`);
for (const c of containerLogs) scan(c.stderr + c.stdout, `container ${c.name} logs`);
}
// 6. Failure artifacts (all redacted).
const failed = problems.length > 0 || testInfo.status !== testInfo.expectedStatus || testInfo.errors.length > 0;
// Building artifacts must never skip the cleanup below (a throw here used to leak the
// workspace): an artifact failure is reported as a teardown problem instead.
if (failed) {
try {
for (const s of ctx.servers) for (const g of s.allOutput()) {
attachments.push({ name: `server-${s.label}-gen${g.generation}.stderr.log`, body: red(g.stderr), contentType: 'text/plain' });
if (g.stdout) attachments.push({ name: `server-${s.label}-gen${g.generation}.stdout.log`, body: red(g.stdout), contentType: 'text/plain' });
attachments.push({ name: `server-${s.label}-gen${g.generation}.exit.json`, body: JSON.stringify({ pid: g.pid, exit: g.exit ?? 'still running at teardown', env: s.env.LOG_LEVEL ? { LOG_LEVEL: s.env.LOG_LEVEL, HTTP_ADDR: s.env.HTTP_ADDR, HTTP_PORT: s.env.HTTP_PORT } : {} }), contentType: 'application/json' });
}
for (const c of containerLogs) attachments.push({ name: `container-${c.name}.log`, body: red(`--- stdout\n${c.stdout}\n--- stderr\n${c.stderr}`), contentType: 'text/plain' });
if (recorder.cli.length) attachments.push({ name: 'cli-invocations.json', body: JSON.stringify(secrets.redactValue({ dropped: recorder.droppedCli, invocations: recorder.cli }), null, 2), contentType: 'application/json' });
if (recorder.http.length) attachments.push({ name: 'http-exchanges.json', body: JSON.stringify(secrets.redactValue({ dropped: recorder.droppedHttp, exchanges: recorder.http }), null, 2), contentType: 'application/json' });
if (recorder.fuzzCases.length) attachments.push({ name: 'fuzz-cases.json', body: JSON.stringify(secrets.redactValue({ seed, runSeed: globalSeed(), cases: recorder.fuzzCases }), null, 2), contentType: 'application/json' });
for (const [i, f] of ctx.fakes.entries()) attachments.push({ name: `fake-docker-${i + 1}.json`, body: JSON.stringify({ socket: f.socketPath, mode: f.mode, stats: f.stats, requests: f.requests.slice(-200) }, null, 2), contentType: 'application/json' });
const dbs = new Set<string>([workspace.dbPath, ...ctx.servers.map((s) => s.dbPath), ...ctx.trackedDbs]);
let n = 0;
for (const db of dbs) if (fs.existsSync(db)) attachments.push({ name: `db-snapshot-${++n}.txt`, body: dbSnapshot(db, red), contentType: 'text/plain' });
if (problems.length) attachments.push({ name: 'harness-problems.txt', body: red(problems.join('\n')), contentType: 'text/plain' });
if (recorder.notes.length) attachments.push({ name: 'harness-notes.txt', body: red(recorder.notes.join('\n')), contentType: 'text/plain' });
for (const a of attachments) await testInfo.attach(a.name, { body: a.body, contentType: a.contentType });
} catch (e) {
problems.push(`building the failure artifacts failed: ${(e as Error).message}`);
}
}
// 7. Workspace removal (root-owned files handled through a container).
try {
await forceRemoveDir(workspace.dir);
if (fs.existsSync(workspace.dir)) problems.push(`workspace ${workspace.dir} survived removal`);
} catch (e) {
problems.push(`workspace removal failed: ${(e as Error).message}`);
}
// 8. Any process this worker spawned that is still alive is a leak.
for (const p of liveProcesses()) {
problems.push(`process ${p.label} (pid ${p.pid}) still alive at teardown`);
killGroup(p.pid, 'SIGKILL');
}
if (problems.length) throw new HarnessError('leak', `teardown problems:\n- ${problems.map(red).join('\n- ')}`);
},
{ auto: true },
],
testKey: async ({ e2e }, use) => use(e2e.testKey),
ws: async ({ e2e }, use) => use(e2e.workspace),
secrets: async ({ e2e }, use) => use(e2e.secrets),
recorder: async ({ e2e }, use) => use(e2e.recorder),
cli: async ({ e2e }, use) => use(new CliRunner({ workspace: e2e.workspace, secrets: e2e.secrets, recorder: e2e.recorder })),
startServer: async ({ e2e }, use) => {
await use((opts?: StartServerOptions) =>
startServer({ workspace: e2e.workspace, recorder: e2e.recorder, allocatePort: e2e.allocatePort, newFakeDocker: e2e.newFakeDocker, servers: e2e.servers }, opts),
);
},
newFakeDocker: async ({ e2e }, use) => use(e2e.newFakeDocker),
http: async ({ e2e, request }, use) => use(new HttpClient(request, e2e.workspace, e2e.recorder)),
docker: async ({ e2e }, use) => {
e2e.usedDocker = true;
const d = new RealDocker(e2e.testKey);
await use(d);
// The e2e teardown removes everything carrying this test's label.
},
image: async ({ e2e }, use) => {
e2e.usedDocker = true;
await use(
new ImageRunner({
workspace: e2e.workspace,
recorder: e2e.recorder,
secrets: e2e.secrets,
testKey: e2e.testKey,
newFakeDocker: e2e.newFakeDocker,
containers: e2e.imageContainers,
}),
);
},
rng: async ({ e2e }, use, testInfo) => {
testInfo.annotations.push({ type: 'seed', description: `${e2e.seed} (run seed: ${globalSeed()})` });
await use(new Rng(e2e.seed));
},
fuzz: async ({ e2e, rng, secrets }, use, testInfo) => {
await use(new FuzzRecorder(e2e.seed, rng, e2e.recorder, testInfo, secrets));
},
});
export { expect };
+27
View File
@@ -0,0 +1,27 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Run-level setup (runner process): assigns the run id, creates the run scratch directory and
* resolves the seed. Values reach the workers through process.env.
* Project prerequisites (binary build/validation, image presence) are checked by the
* `host-setup` / `image-setup` projects (setup/*.setup.ts).
*/
import crypto from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { binaryPath, globalSeed, imageRef, RESOURCE_PREFIX } from './env';
export default async function globalSetup(): Promise<void> {
const id = crypto.randomBytes(4).toString('hex');
const dir = path.join(os.tmpdir(), `${RESOURCE_PREFIX}${id}`);
fs.mkdirSync(path.join(dir, 'ws'), { recursive: true, mode: 0o700 });
fs.mkdirSync(path.join(dir, 'secrets'), { recursive: true, mode: 0o700 });
process.env.DPSAPI_E2E_RUN_ID = id;
process.env.DPSAPI_E2E_RUN_DIR = dir;
process.env.DPSAPI_E2E_SEED = globalSeed();
console.log(
`[dpsapi-e2e] run id=${id} seed=${JSON.stringify(globalSeed())} binary=${binaryPath()} image=${imageRef() ?? '(DPSAPI_IMAGE unset)'} scratch=${dir}`,
);
}
+101
View File
@@ -0,0 +1,101 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Run-level leak detection (runner process). Throwing here makes `playwright test` exit non-zero
* (verified with Playwright 1.63), so any leak fails the build:
* - containers / networks / volumes labelled dpsapi.e2e=<runId>;
* - live processes whose environment carries DPSAPI_E2E_PROC=<runId>;
* - leftover per-test workspaces.
* Leaked resources are cleaned up before failing.
*/
import fs from 'node:fs';
import path from 'node:path';
import { spawnSync } from 'node:child_process';
import { LABEL_RUN, PROC_MARKER_ENV } from './env';
function docker(args: string[]): { code: number | null; out: string; err: string } {
const r = spawnSync('docker', args, { encoding: 'utf8', timeout: 120_000 });
return { code: r.status, out: r.stdout ?? '', err: r.stderr ?? '' };
}
function lines(s: string): string[] {
return s.split('\n').map((l) => l.trim()).filter(Boolean);
}
/** PIDs of live processes whose environ contains PROC_MARKER_ENV=<runId>. */
export function markedProcesses(runId: string): Array<{ pid: number; cmdline: string }> {
const marker = Buffer.from(`${PROC_MARKER_ENV}=${runId}\0`);
const out: Array<{ pid: number; cmdline: string }> = [];
for (const d of fs.readdirSync('/proc')) {
if (!/^\d+$/.test(d) || Number(d) === process.pid) continue;
try {
const env = fs.readFileSync(`/proc/${d}/environ`);
if (env.includes(marker)) {
const stat = fs.readFileSync(`/proc/${d}/stat`, 'utf8');
if (/^\d+ \(.*\) Z /.test(stat)) continue; // zombie: already exited, waiting to be reaped
out.push({ pid: Number(d), cmdline: fs.readFileSync(`/proc/${d}/cmdline`, 'utf8').split('\0').join(' ').trim() });
}
} catch {
/* gone or not ours */
}
}
return out;
}
export default async function globalTeardown(): Promise<void> {
const runId = process.env.DPSAPI_E2E_RUN_ID;
const runDir = process.env.DPSAPI_E2E_RUN_DIR;
if (!runId || !runDir) return;
const leaks: string[] = [];
// Containers
const ps = docker(['ps', '-a', '--filter', `label=${LABEL_RUN}=${runId}`, '--format', '{{.Names}}']);
if (ps.code !== 0) leaks.push(`cannot list containers: ${ps.err.trim()}`);
const containers = lines(ps.out);
if (containers.length) {
leaks.push(`containers left behind: ${containers.join(', ')}`);
docker(['rm', '-f', '-v', ...containers]);
}
// Networks and volumes (phase-2 suites may create them)
for (const kind of ['network', 'volume'] as const) {
const r = docker([kind, 'ls', '-q', '--filter', `label=${LABEL_RUN}=${runId}`]);
const ids = lines(r.out);
if (ids.length) {
leaks.push(`${kind}s left behind: ${ids.join(', ')}`);
docker([kind, 'rm', ...ids]);
}
}
// Processes
const procs = markedProcesses(runId);
if (procs.length) {
leaks.push(`processes left running: ${procs.map((p) => `${p.pid} (${p.cmdline.slice(0, 120)})`).join('; ')}`);
for (const p of procs) {
try {
process.kill(p.pid, 'SIGKILL');
} catch {
/* gone */
}
}
}
// Workspaces
const ws = path.join(runDir, 'ws');
const left = fs.existsSync(ws) ? fs.readdirSync(ws) : [];
if (left.length) leaks.push(`temporary workspaces left behind: ${left.join(', ')}`);
try {
fs.rmSync(runDir, { recursive: true, force: true });
} catch (e) {
// root-owned leftovers (image tests): remove through a throwaway container
const name = `dpsapi-e2e-${runId}-final-rm`;
docker(['run', '--rm', '--name', name, '--label', `${LABEL_RUN}=${runId}`, '--network', 'none', '-v', `${runDir}:/victim`, process.env.DPSAPI_E2E_ALPINE || 'alpine:3.22', 'sh', '-c', 'find /victim -mindepth 1 -delete']);
fs.rmSync(runDir, { recursive: true, force: true });
if (fs.existsSync(runDir)) leaks.push(`cannot remove run directory ${runDir}: ${(e as Error).message}`);
}
if (leaks.length) {
throw new Error(`[dpsapi-e2e] resource leaks detected for run ${runId} (cleaned up, failing the run):\n- ${leaks.join('\n- ')}`);
}
console.log(`[dpsapi-e2e] run ${runId}: no leaked containers, processes or workspaces`);
}
+168
View File
@@ -0,0 +1,168 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Go string quoting/unquoting compatible with strconv.Quote / strconv.Unquote,
* as used by `token list` (DESCRIPTION column), slog's text handler and the
* configuration error messages.
*/
/** Characters Go's strconv.IsPrint accepts: letters, marks, numbers, punctuation, symbols, and ASCII space. */
const PRINTABLE = /^[\p{L}\p{M}\p{N}\p{P}\p{S} ]$/u;
/**
* Go strconv.Quote of a byte string (Buffer) or a JS string (encoded as UTF-8).
* Note: relies on Node's Unicode tables; avoid code points newer than Go's tables in tests.
*/
export function goQuote(input: string | Buffer): string {
const bytes = typeof input === 'string' ? Buffer.from(input, 'utf8') : input;
let out = '"';
let i = 0;
while (i < bytes.length) {
const b = bytes[i];
if (b < 0x80) {
i++;
const c = String.fromCharCode(b);
switch (c) {
case '"':
out += '\\"';
continue;
case '\\':
out += '\\\\';
continue;
case '\x07':
out += '\\a';
continue;
case '\b':
out += '\\b';
continue;
case '\f':
out += '\\f';
continue;
case '\n':
out += '\\n';
continue;
case '\r':
out += '\\r';
continue;
case '\t':
out += '\\t';
continue;
case '\v':
out += '\\v';
continue;
}
if (b < 0x20 || b === 0x7f) out += '\\x' + hex2(b);
else out += c;
continue;
}
// Multi-byte: decode one UTF-8 sequence strictly
const cp = decodeUtf8At(bytes, i);
if (!cp) {
out += '\\x' + hex2(b);
i++;
continue;
}
const ch = String.fromCodePoint(cp.codePoint);
if (PRINTABLE.test(ch)) out += ch;
else if (cp.codePoint < 0x10000) out += '\\u' + cp.codePoint.toString(16).padStart(4, '0');
else out += '\\U' + cp.codePoint.toString(16).padStart(8, '0');
i += cp.length;
}
return out + '"';
}
function hex2(b: number): string {
return b.toString(16).padStart(2, '0');
}
/** Strict UTF-8 decode of the sequence at i (rejects overlongs, surrogates, > U+10FFFF). */
function decodeUtf8At(buf: Buffer, i: number): { codePoint: number; length: number } | undefined {
const b0 = buf[i];
let len: number;
let cp: number;
let min: number;
if (b0 >= 0xc2 && b0 <= 0xdf) {
len = 2;
cp = b0 & 0x1f;
min = 0x80;
} else if (b0 >= 0xe0 && b0 <= 0xef) {
len = 3;
cp = b0 & 0x0f;
min = 0x800;
} else if (b0 >= 0xf0 && b0 <= 0xf4) {
len = 4;
cp = b0 & 0x07;
min = 0x10000;
} else return undefined;
if (i + len > buf.length) return undefined;
for (let k = 1; k < len; k++) {
const b = buf[i + k];
if ((b & 0xc0) !== 0x80) return undefined;
cp = (cp << 6) | (b & 0x3f);
}
if (cp < min || cp > 0x10ffff || (cp >= 0xd800 && cp <= 0xdfff)) return undefined;
return { codePoint: cp, length: len };
}
/**
* Go strconv.Unquote for double-quoted strings. Returns the raw bytes (descriptions may
* contain invalid UTF-8). Throws on malformed input.
*/
export function goUnquoteBytes(quoted: string): Buffer {
if (quoted.length < 2 || quoted[0] !== '"' || quoted[quoted.length - 1] !== '"') {
throw new Error(`not a Go double-quoted string: ${JSON.stringify(quoted)}`);
}
const s = quoted.slice(1, -1);
const out: number[] = [];
const pushStr = (str: string) => out.push(...Buffer.from(str, 'utf8'));
for (let i = 0; i < s.length; ) {
const c = s[i];
if (c === '"') throw new Error(`unescaped quote in Go string: ${JSON.stringify(quoted)}`);
if (c !== '\\') {
const cp = s.codePointAt(i)!;
const ch = String.fromCodePoint(cp);
pushStr(ch);
i += ch.length;
continue;
}
const e = s[i + 1];
const simple: Record<string, number> = { a: 7, b: 8, f: 12, n: 10, r: 13, t: 9, v: 11, '\\': 92, '"': 34, "'": 39 };
if (e in simple) {
out.push(simple[e]);
i += 2;
} else if (e === 'x') {
out.push(parseHex(s.slice(i + 2, i + 4), quoted));
i += 4;
} else if (e === 'u') {
pushStr(String.fromCodePoint(parseHex(s.slice(i + 2, i + 6), quoted)));
i += 6;
} else if (e === 'U') {
pushStr(String.fromCodePoint(parseHex(s.slice(i + 2, i + 10), quoted)));
i += 10;
} else if (e >= '0' && e <= '7') {
const oct = s.slice(i + 1, i + 4);
if (!/^[0-7]{3}$/.test(oct)) throw new Error(`bad octal escape in ${JSON.stringify(quoted)}`);
out.push(parseInt(oct, 8));
i += 4;
} else {
throw new Error(`bad escape \\${e} in Go string ${JSON.stringify(quoted)}`);
}
}
return Buffer.from(out);
}
/** goUnquoteBytes decoded as UTF-8 (invalid sequences become U+FFFD). */
export function goUnquote(quoted: string): string {
return goUnquoteBytes(quoted).toString('utf8');
}
function parseHex(h: string, ctx: string): number {
if (!/^[0-9a-fA-F]+$/.test(h)) throw new Error(`bad hex escape in ${JSON.stringify(ctx)}`);
return parseInt(h, 16);
}
/** POSIX sh single-quoting: the result is always one shell word with the exact bytes of s. */
export function shellQuote(s: string): string {
return `'${s.split("'").join(`'\\''`)}'`;
}
+525
View File
@@ -0,0 +1,525 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* HTTP clients: Playwright APIRequestContext, real curl, and a raw TCP socket.
*/
import fs from 'node:fs';
import net from 'node:net';
import type { APIRequestContext } from '@playwright/test';
import { HarnessError } from './errors';
import { runProcess, toolEnv } from './proc';
import type { Recorder } from './recorder';
import type { Workspace } from './workspace';
// ---------------------------------------------------------------------------------------------
// Shared types
// ---------------------------------------------------------------------------------------------
export interface HeaderEntry {
name: string;
value: string;
}
export interface HttpResult {
client: 'playwright' | 'curl';
method: string;
url: string;
status: number;
statusText: string;
/** Lower-cased header names; repeated headers joined with ', '. */
headers: Record<string, string>;
/** Every header line in order, names as received. */
headersArray: HeaderEntry[];
body: string;
bodyBuffer: Buffer;
/** JSON.parse(body) (throws when the body is not JSON). */
json<T = unknown>(): T;
durationMs: number;
}
export function bearer(token: string): Record<string, string> {
return { Authorization: `Bearer ${token}` };
}
function headersRecord(arr: HeaderEntry[]): Record<string, string> {
const out: Record<string, string> = {};
for (const { name, value } of arr) {
const k = name.toLowerCase();
out[k] = k in out ? `${out[k]}, ${value}` : value;
}
return out;
}
function withJson<T extends { body: string }>(r: T): T & { json<J = unknown>(): J } {
return Object.assign(r, {
json<J = unknown>(): J {
return JSON.parse(r.body) as J;
},
});
}
// ---------------------------------------------------------------------------------------------
// Raw socket
// ---------------------------------------------------------------------------------------------
export interface RawHttpResponse {
httpVersion: string;
status: number;
statusText: string;
headers: Array<[string, string]>;
/** First value of a header (case-insensitive). */
header(name: string): string | undefined;
/** Every value of a header (case-insensitive). */
headerAll(name: string): string[];
body: Buffer;
bodyText: string;
}
/**
* - response: at least one complete response was parsed;
* - closed: the server closed the connection without a complete response;
* - reset: the connection was reset (ECONNRESET/EPIPE) without a complete response;
* - refused: nothing listening (ECONNREFUSED);
* - timeout: only returned when allowTimeout is set (otherwise HarnessError).
*/
export type RawOutcome = 'response' | 'closed' | 'reset' | 'refused' | 'timeout';
export interface RawResult {
outcome: RawOutcome;
responses: RawHttpResponse[];
/** responses[0] */
response?: RawHttpResponse;
/** Every byte received. */
raw: Buffer;
/** True when the server closed the connection (FIN) before the harness did. */
closedByServer: boolean;
error?: string;
durationMs: number;
}
export interface RawHttpOptions {
host: string;
port: number;
/** Bytes to send. An array is sent as separate writes `chunkDelayMs` apart. */
data: string | Buffer | Array<string | Buffer>;
chunkDelayMs?: number;
/** Default 10 s. */
timeoutMs?: number;
/** Resolve once this many complete responses were parsed (default 1). */
expectResponses?: number;
/** Keep reading until the server closes (default false). */
waitForClose?: boolean;
/** Half-close (FIN) after writing (default false). */
endAfterWrite?: boolean;
/** Return outcome 'timeout' instead of throwing HarnessError. */
allowTimeout?: boolean;
/** Treat responses as bodiless (HEAD). Default: data starts with "HEAD ". */
head?: boolean;
}
/** Parses as many complete HTTP/1.x responses as `buf` contains. `eof` completes a read-until-close body. */
export function parseHttpResponses(buf: Buffer, eof: boolean, head = false): { responses: RawHttpResponse[]; rest: Buffer } {
const responses: RawHttpResponse[] = [];
let off = 0;
while (off < buf.length) {
const hdrEnd = buf.indexOf('\r\n\r\n', off);
if (hdrEnd < 0) break;
const headText = buf.subarray(off, hdrEnd).toString('latin1');
const [statusLine, ...headerLines] = headText.split('\r\n');
const m = /^(HTTP\/\d(?:\.\d)?) (\d{3})(?: (.*))?$/.exec(statusLine);
if (!m) throw new HarnessError('protocol', `unparseable status line ${JSON.stringify(statusLine)}`);
const headers: Array<[string, string]> = headerLines.map((l) => {
const i = l.indexOf(':');
if (i < 0) throw new HarnessError('protocol', `unparseable header line ${JSON.stringify(l)}`);
return [l.slice(0, i), l.slice(i + 1).trim()];
});
const get = (n: string) => headers.filter(([k]) => k.toLowerCase() === n.toLowerCase()).map(([, v]) => v);
const status = Number(m[2]);
let bodyStart = hdrEnd + 4;
let body: Buffer;
let next: number;
const noBody = (head && responses.length === 0) || (status >= 100 && status < 200) || status === 204 || status === 304;
if (noBody) {
body = Buffer.alloc(0);
next = bodyStart;
} else if (get('transfer-encoding').some((v) => v.toLowerCase().includes('chunked'))) {
const chunks: Buffer[] = [];
let p = bodyStart;
let done = false;
while (true) {
const le = buf.indexOf('\r\n', p);
if (le < 0) break;
const size = parseInt(buf.subarray(p, le).toString('latin1').split(';')[0], 16);
if (Number.isNaN(size)) throw new HarnessError('protocol', 'bad chunk size');
if (size === 0) {
const end = buf.indexOf('\r\n\r\n', le);
if (end < 0 && buf.length < le + 4) break;
p = end < 0 ? le + 4 : end + 4;
done = true;
break;
}
if (buf.length < le + 2 + size + 2) break;
chunks.push(buf.subarray(le + 2, le + 2 + size));
p = le + 2 + size + 2;
}
if (!done) break;
body = Buffer.concat(chunks);
next = p;
} else if (get('content-length').length > 0) {
const len = Number(get('content-length')[0]);
if (buf.length < bodyStart + len) break;
body = buf.subarray(bodyStart, bodyStart + len);
next = bodyStart + len;
} else {
if (!eof) break;
body = buf.subarray(bodyStart);
next = buf.length;
}
responses.push({
httpVersion: m[1],
status,
statusText: m[3] ?? '',
headers,
header: (n) => get(n)[0],
headerAll: (n) => get(n),
body: Buffer.from(body),
bodyText: body.toString('utf8'),
});
off = next;
bodyStart = next;
}
return { responses, rest: buf.subarray(off) };
}
/** Sends raw bytes over TCP and parses the HTTP response(s). */
export function rawHttp(opts: RawHttpOptions): Promise<RawResult> {
const timeoutMs = opts.timeoutMs ?? 10_000;
const expectResponses = opts.expectResponses ?? 1;
const chunks = (Array.isArray(opts.data) ? opts.data : [opts.data]).map((c) => (typeof c === 'string' ? Buffer.from(c, 'latin1') : c));
const head = opts.head ?? chunks[0]?.subarray(0, 5).toString('latin1') === 'HEAD ';
const started = performance.now();
return new Promise<RawResult>((resolve, reject) => {
const received: Buffer[] = [];
let closedByServer = false;
let settled = false;
const sock = net.connect({ host: opts.host, port: opts.port });
const finish = (outcome: RawOutcome | 'auto', error?: string) => {
if (settled) return;
settled = true;
clearTimeout(timer);
sock.destroy();
const raw = Buffer.concat(received);
let responses: RawHttpResponse[] = [];
try {
responses = parseHttpResponses(raw, closedByServer || outcome !== 'auto', head).responses;
} catch (e) {
reject(e);
return;
}
let o: RawOutcome = outcome === 'auto' ? 'response' : outcome;
if (responses.length > 0 && (o === 'closed' || o === 'reset')) o = 'response';
if (o === 'response' && responses.length === 0) o = 'closed';
const durationMs = performance.now() - started;
if (o === 'timeout' && !opts.allowTimeout) {
reject(
new HarnessError('timeout', `raw HTTP to ${opts.host}:${opts.port}: no complete response within ${timeoutMs} ms (received ${raw.length} bytes: ${JSON.stringify(raw.subarray(0, 300).toString('latin1'))})`),
);
return;
}
resolve({ outcome: o, responses, response: responses[0], raw, closedByServer, error, durationMs });
};
const timer = setTimeout(() => finish('timeout'), timeoutMs);
sock.on('connect', async () => {
try {
for (let i = 0; i < chunks.length; i++) {
if (i > 0 && opts.chunkDelayMs) await new Promise((r) => setTimeout(r, opts.chunkDelayMs));
if (sock.destroyed) return;
sock.write(chunks[i]);
}
if (opts.endAfterWrite) sock.end();
} catch {
/* surfaced via 'error' */
}
});
sock.on('data', (d: Buffer) => {
received.push(d);
if (opts.waitForClose) return;
try {
const { responses } = parseHttpResponses(Buffer.concat(received), false, head);
if (responses.length >= expectResponses) finish('auto');
} catch (e) {
settled = true;
clearTimeout(timer);
sock.destroy();
reject(e);
}
});
sock.on('end', () => {
closedByServer = true;
finish('closed');
});
sock.on('error', (e: NodeJS.ErrnoException) => {
if (e.code === 'ECONNREFUSED') finish('refused', e.code);
else if (e.code === 'ECONNRESET' || e.code === 'EPIPE') {
closedByServer = true;
finish('reset', e.code);
} else {
if (settled) return;
settled = true;
clearTimeout(timer);
reject(new HarnessError('transport', `raw socket error to ${opts.host}:${opts.port}: ${e.message}`));
}
});
});
}
// ---------------------------------------------------------------------------------------------
// curl
// ---------------------------------------------------------------------------------------------
export interface CurlOptions {
url: string;
/** -X <method> (use head: true for HEAD so curl does not wait for a body). */
method?: string;
/** Raw header lines, repeatable, passed as `-H <line>` verbatim ('Name: value', 'Name;' = empty value, 'Name:' = remove). */
headers?: string[];
/** Bearer token shortcut: adds `-H 'Authorization: Bearer <token>'`. */
token?: string;
/** Request body via --data-binary @file (curl then defaults to POST unless method is set). */
data?: string | Buffer;
/** --path-as-is: send /./ and /../ segments unmodified. */
pathAsIs?: boolean;
/** -I (HEAD request). */
head?: boolean;
/** Default '1.1'. */
httpVersion?: '1.0' | '1.1';
/** curl --max-time in ms (default 15 s); exceeding it is a HarnessError unless allowTimeout. */
timeoutMs?: number;
allowTimeout?: boolean;
/** Extra raw arguments appended before the URL. */
extraArgs?: string[];
}
export interface CurlResult {
client: 'curl';
/** curl argv (body/header file paths are harness temp files). */
argv: string[];
/** curl's exit code (0 ok, 7 refused, 52 empty reply, 56 recv failure, ...). */
exitCode: number;
/** HTTP status (0 when no response). */
status: number;
httpVersion: string;
headers: Record<string, string>;
headersArray: HeaderEntry[];
body: string;
bodyBuffer: Buffer;
json<T = unknown>(): T;
stderr: string;
/** curl's own -w %{json} metadata (time_total etc.). */
writeOut: Record<string, unknown>;
durationMs: number;
}
let curlSeq = 0;
export async function runCurl(ws: Workspace, opts: CurlOptions): Promise<CurlResult> {
const n = ++curlSeq;
const bodyFile = ws.path(`.curl-${n}.body`);
const headFile = ws.path(`.curl-${n}.headers`);
const dataFile = ws.path(`.curl-${n}.data`);
const maxTimeMs = opts.timeoutMs ?? 15_000;
const argv = ['-sS', '--max-time', (maxTimeMs / 1000).toFixed(3), '-o', bodyFile, '-D', headFile, '-w', '%{json}'];
argv.push(opts.httpVersion === '1.0' ? '--http1.0' : '--http1.1');
if (opts.head) argv.push('-I');
if (opts.method) argv.push('-X', opts.method);
if (opts.token !== undefined) argv.push('-H', `Authorization: Bearer ${opts.token}`);
for (const h of opts.headers ?? []) argv.push('-H', h);
if (opts.data !== undefined) {
fs.writeFileSync(dataFile, opts.data);
argv.push('--data-binary', `@${dataFile}`);
}
if (opts.pathAsIs) argv.push('--path-as-is');
argv.push(...(opts.extraArgs ?? []), opts.url);
try {
const r = await runProcess('curl', argv, { env: toolEnv(), timeoutMs: maxTimeMs + 10_000, label: 'curl' });
let writeOut: Record<string, unknown> = {};
try {
writeOut = JSON.parse(r.stdout) as Record<string, unknown>;
} catch {
throw new HarnessError('protocol', `curl -w %{json} output unparseable: ${JSON.stringify(r.stdout.slice(0, 300))}`);
}
if (r.exitCode === 28 && !opts.allowTimeout) {
throw new HarnessError('timeout', `curl ${opts.url}: no response within ${maxTimeMs} ms (${r.stderr.trim()})`);
}
const bodyBuffer = fs.existsSync(bodyFile) ? fs.readFileSync(bodyFile) : Buffer.alloc(0);
const headText = fs.existsSync(headFile) ? fs.readFileSync(headFile, 'latin1') : '';
// -D may contain several blocks (1xx, redirects); keep the last
const blocks = headText.split(/\r\n\r\n/).filter((b) => b.trim() !== '');
const last = blocks[blocks.length - 1] ?? '';
const [statusLine, ...lines] = last.split('\r\n');
const sm = /^(HTTP\/[\d.]+) (\d{3})/.exec(statusLine ?? '');
const headersArray = lines.filter((l) => l.includes(':')).map((l) => ({ name: l.slice(0, l.indexOf(':')), value: l.slice(l.indexOf(':') + 1).trim() }));
// with -I curl writes the header block to the output file: a HEAD response has no body
const realBody = opts.head ? Buffer.alloc(0) : bodyBuffer;
const body = realBody.toString('utf8');
return withJson({
client: 'curl' as const,
argv,
exitCode: r.exitCode ?? -1,
status: Number(writeOut.http_code ?? (sm ? sm[2] : 0)),
httpVersion: sm?.[1] ?? '',
headers: headersRecord(headersArray),
headersArray,
body,
bodyBuffer: realBody,
stderr: r.stderr,
writeOut,
durationMs: r.durationMs,
});
} finally {
for (const f of [bodyFile, headFile, dataFile]) fs.rmSync(f, { force: true });
}
}
// ---------------------------------------------------------------------------------------------
// Client facade (bound to a test by the `http` fixture)
// ---------------------------------------------------------------------------------------------
export interface HttpRequestOptions {
method?: string;
headers?: Record<string, string>;
/** Bearer token shortcut (adds Authorization: Bearer <token>). */
token?: string;
data?: string | Buffer;
/** Default 30 s. Exceeding it is a HarnessError. */
timeoutMs?: number;
}
export class HttpClient {
constructor(
private readonly request: APIRequestContext,
private readonly ws: Workspace,
private readonly recorder: Recorder,
) {}
/**
* Playwright APIRequestContext request (no redirects followed, never throws on status).
* Transport failures (refused, reset, timeout) throw HarnessError('transport'/'timeout').
*/
async fetch(url: string, opts: HttpRequestOptions = {}): Promise<HttpResult> {
const method = opts.method ?? 'GET';
const headers = { ...(opts.token !== undefined ? bearer(opts.token) : {}), ...(opts.headers ?? {}) };
const started = performance.now();
const base = {
client: 'playwright' as const,
method,
url,
requestHeaders: Object.entries(headers) as Array<[string, string]>,
requestBodyBytes: opts.data === undefined ? 0 : Buffer.byteLength(opts.data),
};
let res;
try {
res = await this.request.fetch(url, { method, headers, data: opts.data, timeout: opts.timeoutMs ?? 30_000, maxRedirects: 0, failOnStatusCode: false });
} catch (e) {
const msg = (e as Error).message;
this.recorder.addHttp({ ...base, durationMs: performance.now() - started, outcome: msg });
if (/Timeout \d+ms exceeded|timed out/i.test(msg)) throw new HarnessError('timeout', `${method} ${url}: ${msg}`);
throw new HarnessError('transport', `${method} ${url}: ${msg}`);
}
const bodyBuffer = await res.body();
const durationMs = performance.now() - started;
const headersArray = res.headersArray();
const result = withJson({
client: 'playwright' as const,
method,
url,
status: res.status(),
statusText: res.statusText(),
headers: headersRecord(headersArray),
headersArray,
body: bodyBuffer.toString('utf8'),
bodyBuffer,
durationMs,
});
await res.dispose();
this.recorder.addHttp({
...base,
status: result.status,
responseHeaders: headersArray.map((h) => [h.name, h.value]),
bodyPreview: result.body,
durationMs,
});
return result;
}
get(url: string, opts: Omit<HttpRequestOptions, 'method'> = {}): Promise<HttpResult> {
return this.fetch(url, { ...opts, method: 'GET' });
}
/** Real curl process; see CurlOptions. */
async curl(opts: CurlOptions): Promise<CurlResult> {
const started = performance.now();
try {
const r = await runCurl(this.ws, opts);
this.recorder.addHttp({
client: 'curl',
method: opts.head ? 'HEAD' : (opts.method ?? (opts.data !== undefined ? 'POST' : 'GET')),
url: opts.url,
requestHeaders: [...(opts.token !== undefined ? [['Authorization', `Bearer ${opts.token}`] as [string, string]] : []), ...(opts.headers ?? []).map((h) => [h, ''] as [string, string])],
requestBodyBytes: opts.data === undefined ? 0 : Buffer.byteLength(opts.data),
status: r.status,
responseHeaders: r.headersArray.map((h) => [h.name, h.value]),
bodyPreview: r.body,
durationMs: r.durationMs,
curlExitCode: r.exitCode,
outcome: r.exitCode === 0 ? undefined : r.stderr.trim(),
});
return r;
} catch (e) {
this.recorder.addHttp({ client: 'curl', method: opts.method ?? 'GET', url: opts.url, requestHeaders: [], requestBodyBytes: 0, durationMs: performance.now() - started, outcome: (e as Error).message });
throw e;
}
}
/** Raw TCP request; see RawHttpOptions. */
async raw(opts: RawHttpOptions): Promise<RawResult> {
const first = (Array.isArray(opts.data) ? opts.data.map((c) => c.toString()).join('') : opts.data.toString()).split('\r\n')[0] ?? '';
const started = performance.now();
try {
const r = await rawHttp(opts);
this.recorder.addHttp({
client: 'raw',
method: first.split(' ')[0] ?? '',
url: `tcp://${opts.host}:${opts.port} ${first}`,
requestHeaders: [],
requestBodyBytes: Array.isArray(opts.data) ? opts.data.reduce((n, c) => n + Buffer.byteLength(c), 0) : Buffer.byteLength(opts.data),
status: r.response?.status,
responseHeaders: r.response?.headers,
bodyPreview: r.response?.bodyText,
durationMs: r.durationMs,
outcome: r.outcome,
});
return r;
} catch (e) {
this.recorder.addHttp({ client: 'raw', method: '', url: `tcp://${opts.host}:${opts.port} ${first}`, requestHeaders: [], requestBodyBytes: 0, durationMs: performance.now() - started, outcome: (e as Error).message });
throw e;
}
}
}
/** Builds a raw HTTP/1.1 request. Header lines are written verbatim (duplicates and odd spacing allowed). */
export function buildRawRequest(opts: { method?: string; target: string; headers?: Array<string | [string, string]>; body?: string; host?: string; version?: string; close?: boolean }): string {
const lines = [`${opts.method ?? 'GET'} ${opts.target} ${opts.version ?? 'HTTP/1.1'}`];
lines.push(`Host: ${opts.host ?? 'localhost'}`);
for (const h of opts.headers ?? []) lines.push(typeof h === 'string' ? h : `${h[0]}: ${h[1]}`);
if (opts.body !== undefined) lines.push(`Content-Length: ${Buffer.byteLength(opts.body)}`);
if (opts.close ?? true) lines.push('Connection: close');
return lines.join('\r\n') + '\r\n\r\n' + (opts.body ?? '');
}
/** host:port of a base URL like http://127.0.0.1:1234 or http://[::1]:1234. */
export function hostPortOf(baseURL: string): { host: string; port: number } {
const u = new URL(baseURL);
return { host: u.hostname.replace(/^\[|\]$/g, ''), port: Number(u.port) };
}
+354
View File
@@ -0,0 +1,354 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Container-image helpers (project `image`): run the final image as a uniquely named,
* labelled container with the Docker socket and a data directory mounted, drive the CLI
* through `docker exec`, read logs/exit codes, and inspect the root-owned database.
*/
import fs from 'node:fs';
import path from 'node:path';
import { ALPINE_IMAGE, imageRef, LABEL_RUN, LABEL_TEST, RESOURCE_PREFIX, runId } from './env';
import { HarnessError } from './errors';
import { FakeDocker } from './fake-docker';
import { buildRawRequest, rawHttp } from './http';
import { parseLogs, type LogSet } from './logs';
import { runProcess, sleep, toolEnv, type ProcResult } from './proc';
import { dockerCli } from './docker';
import { forceRemoveDir, type Workspace } from './workspace';
import { printsTokens, type CliResult } from './cli';
import type { Recorder } from './recorder';
import type { SecretRegistry } from './secrets';
/** Socket mounted at /var/run/docker.sock in the container. */
export type ImageDockerBackend = 'real' | 'none' | FakeDocker | { socketPath: string };
export interface ImageStartOptions {
/** LOG_LEVEL (default 'INFO'; null = unset). */
logLevel?: string | null;
/** HTTP_ADDR inside the container (default unset → 0.0.0.0). */
addr?: string | null;
/** HTTP_PORT inside the container (default unset → 80); the published port follows it. */
containerPort?: number;
/** DB_PATH inside the container (default '/data/dpsapi.db'; null = unset). */
dbPath?: string | null;
/** Extra env (undefined removes). */
env?: Record<string, string | undefined>;
/** Default: a new FakeDocker (empty) — pass 'real' for the host Engine. */
docker?: ImageDockerBackend;
/** Host directory mounted at /data (default: a new workspace dir). Reuse it to replace a container. */
dataDir?: string;
/** docker run --user. */
user?: string;
/** Publish the container port on 127.0.0.1:<random> (default true). */
publish?: boolean;
/** Extra `docker run` arguments (before the image). */
extraArgs?: string[];
/** Arguments after the image (default none → server mode). */
args?: string[];
/** Wait for HTTP readiness (default true when publishing and no args). */
waitReady?: boolean;
readyTimeoutMs?: number;
/** Name suffix (default img<n>). */
suffix?: string;
}
export interface ImageExecOptions {
/** `docker exec -e K=V` (undefined values are skipped). */
env?: Record<string, string | undefined>;
user?: string;
timeoutMs?: number;
}
export interface ContainerExit {
exitCode: number;
oomKilled: boolean;
stdout: string;
stderr: string;
}
export interface ImageContext {
workspace: Workspace;
recorder: Recorder;
secrets: SecretRegistry;
testKey: string;
newFakeDocker: () => Promise<FakeDocker>;
containers: ImageContainer[];
}
let seq = 0;
export class ImageContainer {
/** Published host port (0 when not published). */
hostPort = 0;
/** http://127.0.0.1:<hostPort> */
baseURL = '';
id = '';
constructor(
private readonly ctx: ImageContext,
readonly name: string,
readonly image: string,
/** Host directory mounted at /data (files inside are root-owned). */
readonly dataDir: string,
readonly options: ImageStartOptions,
readonly fakeDocker?: FakeDocker,
) {}
/** DB_PATH inside the container. */
get containerDbPath(): string | undefined {
return this.options.dbPath === null ? undefined : (this.options.dbPath ?? '/data/dpsapi.db');
}
url(p = '/api/v1/containers'): string {
return this.baseURL + p;
}
/** Polls `OPTIONS *` through the published port until the app answers (docker-proxy accepts early, so a real HTTP response is required). */
async waitReady(timeoutMs = this.options.readyTimeoutMs ?? 20_000): Promise<void> {
if (!this.hostPort) throw new HarnessError('infrastructure', `container ${this.name} has no published port`);
const deadline = performance.now() + timeoutMs;
let n = 0;
while (true) {
const r = await rawHttp({ host: '127.0.0.1', port: this.hostPort, data: buildRawRequest({ method: 'OPTIONS', target: '*' }), timeoutMs: 2000, allowTimeout: true }).catch(() => undefined);
if (r?.outcome === 'response') return;
if (++n % 10 === 0 && !(await this.isRunning())) {
const l = await this.logs();
throw new Error(`container ${this.name} exited during startup\n--- stderr\n${l.stderr}`);
}
if (performance.now() > deadline) {
const l = await this.logs();
throw new HarnessError('timeout', `container ${this.name} not ready on ${this.baseURL} within ${timeoutMs} ms\n--- stderr\n${l.stderr}`);
}
await sleep(50);
}
}
/**
* `docker exec [-e ..] <name> /app ...args` with the same result shape as CliRunner.run.
* Exit codes >= 125 (docker exec failures, signal deaths reported as 128+n) and daemon-refused execs are HarnessErrors.
*/
async exec(args: string[], opts: ImageExecOptions = {}): Promise<CliResult> {
const r = await this.execRaw(['/app', ...args], opts);
const res: CliResult = { ...r, args: [...args], via: 'docker-exec' };
if (printsTokens(args)) this.ctx.secrets.registerAllIn(r.stdout);
this.ctx.recorder.addCli({ via: 'docker-exec', command: `docker exec ${this.name} /app`, args, env: execEnv(opts.env), exitCode: r.exitCode, signal: r.signal, timedOut: r.timedOut, durationMs: Math.round(r.durationMs), stdout: r.stdout, stderr: r.stderr });
// The daemon can refuse an exec with exit 1 (e.g. unknown --user), which must not pass for an app failure.
const daemonRefused = r.exitCode !== 0 && /^(Error response from daemon:|OCI runtime exec failed)/.test(r.stderr + r.stdout);
if ((r.exitCode !== null && r.exitCode >= 125) || daemonRefused) {
throw new HarnessError('infrastructure', `docker exec ${this.name} /app ${args.slice(0, 2).join(' ')} failed at the docker level (exit ${r.exitCode}): ${(r.stderr + r.stdout).trim()}`, r);
}
return res;
}
/** `docker exec` of an arbitrary argv (e.g. ['sh'] to prove there is no shell). Never throws for exit codes. */
async execRaw(argv: string[], opts: ImageExecOptions = {}): Promise<ProcResult> {
const a = ['exec'];
if (opts.user) a.push('--user', opts.user);
for (const [k, v] of Object.entries(opts.env ?? {})) if (v !== undefined) a.push('-e', `${k}=${v}`);
a.push(this.name, ...argv);
return runProcess('docker', a, { env: toolEnv(), timeoutMs: opts.timeoutMs ?? 30_000, label: `docker exec ${argv[0]}` });
}
/** `docker logs`: the container's stdout and stderr separately. */
async logs(): Promise<{ stdout: string; stderr: string }> {
const r = await dockerCli(['logs', this.name], { allowFailure: true });
return { stdout: r.stdout, stderr: r.stderr };
}
/** Parsed application logs (container stderr). */
async parsedLogs(): Promise<LogSet> {
return parseLogs((await this.logs()).stderr);
}
async inspect(): Promise<Record<string, unknown>> {
const r = await dockerCli(['inspect', this.name]);
return (JSON.parse(r.stdout) as Array<Record<string, unknown>>)[0];
}
async isRunning(): Promise<boolean> {
const r = await dockerCli(['inspect', '-f', '{{.State.Running}}', this.name], { allowFailure: true });
return r.stdout.trim() === 'true';
}
/**
* Stops the container and returns the application's exit code.
* Default: `docker stop -t <timeoutSec>` (SIGTERM, then SIGKILL after the grace period).
* With `signal`: `docker kill -s <signal>` and wait up to timeoutSec.
* A SIGKILL by docker after the grace period shows as exitCode 137.
*/
async stop({ signal, timeoutSec = 20 }: { signal?: NodeJS.Signals; timeoutSec?: number } = {}): Promise<ContainerExit> {
if (signal) await dockerCli(['kill', '-s', signal, this.name]);
else await dockerCli(['stop', '-t', String(timeoutSec), this.name], { timeoutMs: (timeoutSec + 30) * 1000 });
return this.waitForExit((timeoutSec + 10) * 1000);
}
/** Waits for the container to exit (`docker wait`), then returns exit code and logs. */
async waitForExit(timeoutMs = 30_000): Promise<ContainerExit> {
const r = await runProcess('docker', ['wait', this.name], { env: toolEnv(), timeoutMs, label: 'docker wait' });
if (r.exitCode !== 0) throw new HarnessError('infrastructure', `docker wait ${this.name}: ${r.stderr}`);
const st = (await this.inspect()).State as { ExitCode: number; OOMKilled: boolean };
const l = await this.logs();
return { exitCode: st.ExitCode, oomKilled: st.OOMKilled, stdout: l.stdout, stderr: l.stderr };
}
/** docker rm -f -v. */
async remove(): Promise<void> {
await dockerCli(['rm', '-f', '-v', this.name], { allowFailure: true });
}
/**
* Copies the data directory (db + -wal, not -shm) into a new host-owned workspace directory and
* returns the path of the copied database file — inspect it with readTokens/assertDbInvariants.
* Works whether the app container is running, stopped or removed.
*/
async snapshotDb(): Promise<string> {
return snapshotDataDir(this.ctx, this.dataDir, path.basename(this.containerDbPath ?? '/data/dpsapi.db'));
}
}
function execEnv(env: Record<string, string | undefined> = {}): Record<string, string> {
const o: Record<string, string> = {};
for (const [k, v] of Object.entries(env)) if (v !== undefined) o[k] = v;
return o;
}
let snapSeq = 0;
/** Copies a (root-owned) data dir into a host-owned directory using a throwaway alpine container. Returns the copied db path. */
export async function snapshotDataDir(ctx: Pick<ImageContext, 'workspace' | 'testKey'>, dataDir: string, dbFile = 'dpsapi.db'): Promise<string> {
const out = ctx.workspace.mkdir(`db-snapshot-${++snapSeq}`);
const uid = process.getuid!();
const gid = process.getgid!();
const name = `${RESOURCE_PREFIX}${runId()}-${ctx.testKey}-snap${snapSeq}`;
await dockerCli([
'run', '--rm', '--name', name, '--label', `${LABEL_RUN}=${runId()}`, '--label', `${LABEL_TEST}=${ctx.testKey}`, '--network', 'none',
'-v', `${dataDir}:/data:ro`, '-v', `${out}:/out`, ALPINE_IMAGE,
'sh', '-c', `cp -a /data/. /out/ && rm -f /out/*-shm && chown -R ${uid}:${gid} /out && chmod -R u+rwX /out`,
]);
const p = path.join(out, dbFile);
if (!fs.existsSync(p)) throw new HarnessError('infrastructure', `no ${dbFile} in ${dataDir}`);
return p;
}
export class ImageRunner {
constructor(private readonly ctx: ImageContext) {}
/** The image under test ($DPSAPI_IMAGE). */
get image(): string {
const i = imageRef();
if (!i) throw new HarnessError('prerequisite', 'DPSAPI_IMAGE is not set (use `make docker-e2e` or export DPSAPI_IMAGE=<ref>)');
return i;
}
/** Name for a suite-owned container: dpsapi-e2e-<runId>-<testKey>-<suffix>. */
containerName(suffix: string): string {
return `${RESOURCE_PREFIX}${runId()}-${this.ctx.testKey}-${suffix}`;
}
/** docker run -d the image (server mode by default) and wait for readiness. */
async start(opts: ImageStartOptions = {}): Promise<ImageContainer> {
const image = this.image;
const suffix = opts.suffix ?? `img${++seq}`;
const name = this.containerName(suffix);
const dataDir = opts.dataDir ?? this.ctx.workspace.mkdir(`data-${suffix}`);
let fake: FakeDocker | undefined;
let sock: string | undefined;
const backend = opts.docker ?? (fake = await this.ctx.newFakeDocker());
if (backend === 'real') sock = '/var/run/docker.sock';
else if (backend instanceof FakeDocker) {
fake = backend;
sock = backend.socketPath;
} else if (backend !== 'none') sock = backend.socketPath;
if (sock && !fs.existsSync(sock)) throw new HarnessError('infrastructure', `socket ${sock} does not exist (docker would create a root-owned directory there)`);
const c = new ImageContainer(this.ctx, name, image, dataDir, opts, fake);
const containerPort = opts.containerPort ?? 80;
const args = ['run', '-d', '--name', name, '--label', `${LABEL_RUN}=${runId()}`, '--label', `${LABEL_TEST}=${this.ctx.testKey}`];
if (opts.publish ?? true) args.push('-p', `127.0.0.1::${containerPort}`);
if (sock) args.push('-v', `${sock}:/var/run/docker.sock`);
args.push('-v', `${dataDir}:/data`);
const env: Record<string, string | undefined> = {
DB_PATH: opts.dbPath === null ? undefined : (opts.dbPath ?? '/data/dpsapi.db'),
LOG_LEVEL: opts.logLevel === null ? undefined : (opts.logLevel ?? 'INFO'),
HTTP_ADDR: opts.addr ?? undefined,
HTTP_PORT: opts.containerPort !== undefined ? String(opts.containerPort) : undefined,
...(opts.env ?? {}),
};
for (const [k, v] of Object.entries(env)) if (v !== undefined) args.push('-e', `${k}=${v}`);
if (opts.user) args.push('--user', opts.user);
args.push(...(opts.extraArgs ?? []), image, ...(opts.args ?? []));
this.ctx.containers.push(c);
const r = await dockerCli(args);
c.id = r.stdout.trim();
this.ctx.recorder.note(`image container ${name} started: docker ${JSON.stringify(args)}`);
if (opts.publish ?? true) {
const p = await dockerCli(['port', name, `${containerPort}/tcp`], { allowFailure: true });
const m = /127\.0\.0\.1:(\d+)/.exec(p.stdout);
if (m) {
c.hostPort = Number(m[1]);
c.baseURL = `http://127.0.0.1:${c.hostPort}`;
} else if (opts.waitReady ?? !opts.args?.length) {
const l = await c.logs();
throw new Error(`container ${name} has no published port (exited?): ${p.stderr}\n--- stderr\n${l.stderr}`);
}
}
if ((opts.waitReady ?? !opts.args?.length) && (opts.publish ?? true)) await c.waitReady();
return c;
}
/**
* One-shot `docker run --rm` of the image with args (CLI mode / startup-failure checks).
* Returns the CliResult shape (via 'docker-run'); exit codes 125–127 are HarnessErrors.
*/
async run(
args: string[],
opts: { env?: Record<string, string | undefined>; dataDir?: string; docker?: ImageDockerBackend; user?: string; timeoutMs?: number; extraArgs?: string[]; suffix?: string } = {},
): Promise<CliResult> {
const name = this.containerName(opts.suffix ?? `run${++seq}`);
const dataDir = opts.dataDir ?? this.ctx.workspace.mkdir(`data-${name.slice(-12)}`);
const a = ['run', '--rm', '--name', name, '--label', `${LABEL_RUN}=${runId()}`, '--label', `${LABEL_TEST}=${this.ctx.testKey}`, '-v', `${dataDir}:/data`];
const backend = opts.docker ?? 'none';
if (backend === 'real') a.push('-v', '/var/run/docker.sock:/var/run/docker.sock');
else if (backend instanceof FakeDocker) a.push('-v', `${backend.socketPath}:/var/run/docker.sock`);
else if (backend !== 'none') a.push('-v', `${backend.socketPath}:/var/run/docker.sock`);
const env = { DB_PATH: '/data/dpsapi.db', ...(opts.env ?? {}) };
for (const [k, v] of Object.entries(env)) if (v !== undefined) a.push('-e', `${k}=${v}`);
if (opts.user) a.push('--user', opts.user);
a.push(...(opts.extraArgs ?? []), this.image, ...args);
const r = await runProcess('docker', a, { env: toolEnv(), timeoutMs: opts.timeoutMs ?? 60_000, label: `docker run ${args.slice(0, 2).join(' ')}` });
if (printsTokens(args)) this.ctx.secrets.registerAllIn(r.stdout);
this.ctx.recorder.addCli({ via: 'docker-run', command: `docker run ${this.image}`, args, env: execEnv(env), exitCode: r.exitCode, signal: r.signal, timedOut: r.timedOut, durationMs: Math.round(r.durationMs), stdout: r.stdout, stderr: r.stderr });
if (r.exitCode !== null && r.exitCode >= 125 && r.exitCode <= 127) {
throw new HarnessError('infrastructure', `docker run failed at the docker level (exit ${r.exitCode}): ${r.stderr.trim()}`, r);
}
return { ...r, args: [...args], via: 'docker-run' };
}
/**
* Copies /app out of the image (docker create + docker cp, no container is started) into the
* workspace and returns its host path — the static binary can then be driven on the host with
* cli.withBinary(path) / startServer({ binary: path }) for multi-transport comparisons.
*/
async extractBinary(): Promise<string> {
const name = this.containerName(`extract${++seq}`);
const dest = this.ctx.workspace.path(`app-from-image-${seq}`);
await dockerCli(['create', '--name', name, '--label', `${LABEL_RUN}=${runId()}`, '--label', `${LABEL_TEST}=${this.ctx.testKey}`, this.image]);
try {
await dockerCli(['cp', `${name}:/app`, dest]);
} finally {
await dockerCli(['rm', '-f', name], { allowFailure: true });
}
fs.chmodSync(dest, 0o755);
return dest;
}
/** Copies a data dir to a readable location; returns the db path (see ImageContainer.snapshotDb). */
snapshotDataDir(dataDir: string, dbFile = 'dpsapi.db'): Promise<string> {
return snapshotDataDir(this.ctx, dataDir, dbFile);
}
/** Deletes a (root-owned) data directory via a throwaway container. */
removeDataDir(dataDir: string): Promise<void> {
return forceRemoveDir(dataDir);
}
}
+112
View File
@@ -0,0 +1,112 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Public entry point of the dpsapi E2E harness. Spec files import everything from here:
*
* import { test, expect, expectCliFailure, assertDbInvariants } from '../../harness';
*/
export { test, expect, FuzzRecorder, type E2EContext, type E2EFixtures, type E2EOptions } from './fixtures';
export { HarnessError, isHarnessError, type HarnessErrorKind } from './errors';
export {
ALPINE_IMAGE,
DEFAULT_SEED,
E2E_ROOT,
LABEL_RUN,
LABEL_TEST,
PROC_MARKER_ENV,
REPO_ROOT,
RESOURCE_PREFIX,
binaryPath,
globalSeed,
imageRef,
runId,
} from './env';
export { DEFAULT_PROCESS_TIMEOUT_MS, mapLimit, pollUntil, runProcess, sleep, toolEnv, type ProcResult, type RunProcessOptions } from './proc';
export {
AUDIT_COLUMNS,
CliRunner,
EMPTY_AUDIT,
EMPTY_TOKEN_LIST,
TOKEN_LIST_COLUMNS,
USAGE_ERROR_MESSAGES,
USAGE_TEXT,
baseAppEnv,
errorMessage,
expectCliFailure,
expectCliSuccess,
expectExitedNormally,
formatAuditOutput,
formatTokenList,
mergeEnv,
padTable,
parseAuditOutput,
parseTokenList,
printsTokens,
runeLength,
type AuditRow,
type TokenListEntry,
type CliFailureExpectation,
type CliOptions,
type CliResult,
type CliSuccessExpectation,
type TokenListRow,
} from './cli';
export { ServerHandle, type DockerBackend, type ExitResult, type StartServerOptions } from './server';
export { FakeDocker, makeContainerSummaries, makeContainerSummary, type ContainerSummary, type FakeDockerMode, type FakeDockerRequest, type FakeDockerStats } from './fake-docker';
export {
HttpClient,
bearer,
buildRawRequest,
hostPortOf,
parseHttpResponses,
rawHttp,
runCurl,
type CurlOptions,
type CurlResult,
type HeaderEntry,
type HttpRequestOptions,
type HttpResult,
type RawHttpOptions,
type RawHttpResponse,
type RawOutcome,
type RawResult,
} from './http';
export { RealDocker, dockerCli, listByLabel, type DockerCliOptions, type OwnContainer, type RunContainerOptions, type TestContainer } from './docker';
export { ImageContainer, ImageRunner, snapshotDataDir, type ContainerExit, type ImageDockerBackend, type ImageExecOptions, type ImageStartOptions } from './image';
export {
EXPECTED_COLUMNS,
RFC3339NANO_UTC_RE,
assertDbInvariants,
countRows,
dbSnapshot,
fileDigest,
msToNanos,
readOperations,
readSchema,
readToken,
readTokens,
rfc3339ToNanos,
withDb,
type DbInvariantOptions,
type DbInvariantReport,
type DbSchemaInfo,
type OperationRow,
type TokenRow,
} from './db';
export { SecretRegistry, TOKEN_CANDIDATE_RE, TOKEN_RE, redactText, redactValue, tokenId } from './secrets';
export { LOG_LEVELS, LOG_TIME_RE, LogSet, levelRank, parseLogLine, parseLogs, visibleLevels, type LogLevel, type LogQuery, type LogRecord } from './logs';
export { goQuote, goUnquote, goUnquoteBytes, shellQuote } from './gostrings';
export { Rng, testSeed } from './rng';
export { Recorder, jsonSafe, safeStringify, type CliRecord, type HttpRecord } from './recorder';
export { Workspace, allocatePort, forceRemoveDir, isPortFree, occupyPort } from './workspace';
export {
ERROR_BODIES,
INTERNAL_DETAIL_MARKERS,
expectContainersResponse,
expectErrorResponse,
expectNoInternalDetails,
view,
type AppErrorStatus,
type ResponseView,
} from './assertions';
+166
View File
@@ -0,0 +1,166 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Parser and query helpers for dpsapi's log/slog text-handler output (stderr).
*
* Line format: `time=<RFC3339 ms UTC> level=<LEVEL> msg=<value> key=value ...` where a value
* is either bare (no spaces/quotes/=) or Go-quoted.
*/
import { goUnquote } from './gostrings';
export const LOG_LEVELS = ['TRACE', 'DEBUG', 'INFO', 'WARN', 'ERROR'] as const;
export type LogLevel = (typeof LOG_LEVELS)[number];
/** Severity rank: TRACE 0 … ERROR 4. */
export function levelRank(level: string): number {
const i = LOG_LEVELS.indexOf(level.toUpperCase() as LogLevel);
if (i < 0) throw new Error(`unknown log level ${level}`);
return i;
}
/** Levels that must be emitted when the minimum level is `min`. */
export function visibleLevels(min: LogLevel): LogLevel[] {
return LOG_LEVELS.filter((l) => levelRank(l) >= levelRank(min));
}
export interface LogRecord {
/** The `time` attribute (UTC, millisecond precision). */
time: string;
level: string;
msg: string;
/** All attributes except time/level/msg; the last value wins for duplicate keys. */
attrs: Record<string, string>;
/** Every key/value pair in order, including time/level/msg. */
pairs: Array<[string, string]>;
/** The raw line. */
raw: string;
/** 0-based line index in the stream. */
line: number;
}
export type AttrMatcher = string | number | boolean | RegExp | ((value: string | undefined) => boolean);
export interface LogQuery {
level?: string | string[];
msg?: string | RegExp;
/** Every listed attribute must match. Use `undefined`-checking functions for absence. */
attrs?: Record<string, AttrMatcher>;
}
/** Strict format of the slog `time` attribute as emitted by dpsapi (UTC, milliseconds). */
export const LOG_TIME_RE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
/** Parses one log line; returns undefined when the line is not a slog record. */
export function parseLogLine(raw: string, line = 0): LogRecord | undefined {
if (!raw.startsWith('time=')) return undefined;
const pairs: Array<[string, string]> = [];
let i = 0;
const n = raw.length;
const readToken = (stopAtEq: boolean): string | undefined => {
if (raw[i] === '"') {
// quoted: find closing quote honoring backslash escapes
let j = i + 1;
while (j < n && raw[j] !== '"') j += raw[j] === '\\' ? 2 : 1;
if (j >= n) return undefined;
const q = raw.slice(i, j + 1);
i = j + 1;
try {
return goUnquote(q);
} catch {
return undefined;
}
}
let j = i;
while (j < n && raw[j] !== ' ' && !(stopAtEq && raw[j] === '=')) j++;
const t = raw.slice(i, j);
i = j;
return t;
};
while (i < n) {
while (raw[i] === ' ') i++;
if (i >= n) break;
const key = readToken(true);
if (key === undefined || raw[i] !== '=') return undefined;
i++;
const value = readToken(false);
if (value === undefined) return undefined;
pairs.push([key, value]);
}
if (pairs.length < 3 || pairs[0][0] !== 'time' || pairs[1][0] !== 'level' || pairs[2][0] !== 'msg') return undefined;
const attrs: Record<string, string> = {};
for (const [k, v] of pairs.slice(3)) attrs[k] = v;
return { time: pairs[0][1], level: pairs[1][1], msg: pairs[2][1], attrs, pairs, raw, line };
}
export function matchesQuery(r: LogRecord, q: LogQuery): boolean {
if (q.level !== undefined) {
const levels = Array.isArray(q.level) ? q.level : [q.level];
if (!levels.map((l) => l.toUpperCase()).includes(r.level)) return false;
}
if (q.msg !== undefined) {
if (typeof q.msg === 'string' ? r.msg !== q.msg : !q.msg.test(r.msg)) return false;
}
for (const [k, m] of Object.entries(q.attrs ?? {})) {
const v = Object.prototype.hasOwnProperty.call(r.attrs, k) ? r.attrs[k] : undefined;
if (typeof m === 'function') {
if (!m(v)) return false;
} else if (m instanceof RegExp) {
if (v === undefined || !m.test(v)) return false;
} else if (v !== String(m)) return false;
}
return true;
}
export class LogSet {
readonly records: LogRecord[];
/** Lines that are not slog records (e.g. the final `dpsapi: ...` error line, usage text). */
readonly unparsed: Array<{ line: number; text: string }>;
constructor(text: string) {
this.records = [];
this.unparsed = [];
const lines = text.split('\n');
if (lines.length && lines[lines.length - 1] === '') lines.pop();
lines.forEach((l, idx) => {
const r = parseLogLine(l, idx);
if (r) this.records.push(r);
else this.unparsed.push({ line: idx, text: l });
});
}
filter(q: LogQuery = {}): LogRecord[] {
return this.records.filter((r) => matchesQuery(r, q));
}
find(q: LogQuery): LogRecord | undefined {
return this.records.find((r) => matchesQuery(r, q));
}
count(q: LogQuery = {}): number {
return this.filter(q).length;
}
has(q: LogQuery): boolean {
return this.find(q) !== undefined;
}
/** Distinct levels present. */
levels(): string[] {
return [...new Set(this.records.map((r) => r.level))];
}
/** Distinct messages present (optionally only at one level). */
messages(level?: string): string[] {
return [...new Set(this.records.filter((r) => !level || r.level === level.toUpperCase()).map((r) => r.msg))];
}
/** `this request has been made` records. */
requests(q: Omit<LogQuery, 'msg'> = {}): LogRecord[] {
return this.filter({ ...q, msg: 'this request has been made' });
}
}
export function parseLogs(text: string): LogSet {
return new LogSet(text);
}
File diff suppressed because it is too large. Load diff
+216
View File
@@ -0,0 +1,216 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Process spawning with mandatory timeouts, exact argv (never a shell unless asked),
* and leak tracking.
*/
import { spawn, type ChildProcess } from 'node:child_process';
import { HarnessError } from './errors';
import { PROC_MARKER_ENV, runId } from './env';
export interface ProcResult {
/** Executable that was run. */
command: string;
/** Arguments exactly as passed (not including the command). */
argv: string[];
stdout: string;
stderr: string;
stdoutBuffer: Buffer;
stderrBuffer: Buffer;
/** Exit status, or null when terminated by a signal. */
exitCode: number | null;
/** Terminating signal, or null on a normal exit. */
signal: NodeJS.Signals | null;
/** True when the harness killed the process because its deadline passed. */
timedOut: boolean;
durationMs: number;
}
export interface RunProcessOptions {
/** Complete environment of the child (nothing is inherited implicitly). */
env: NodeJS.ProcessEnv;
cwd?: string;
/** Hard deadline; the whole process group is SIGKILLed when it passes. Default 30 s. */
timeoutMs?: number;
/** Data written to stdin (stdin is /dev/null otherwise). */
input?: string | Buffer;
/** Return (instead of throwing HarnessError) when the deadline passes. */
allowTimeout?: boolean;
/** Return (instead of throwing HarnessError) when the process dies from a signal. */
allowSignal?: boolean;
/** Human label used in error messages. */
label?: string;
}
export const DEFAULT_PROCESS_TIMEOUT_MS = 30_000;
/** Processes spawned by this worker that have not been observed to exit yet. */
const live = new Map<number, { label: string; child: ChildProcess }>();
/** Snapshot of processes spawned by this worker that are still running. */
export function liveProcesses(): Array<{ pid: number; label: string }> {
return [...live.entries()].map(([pid, v]) => ({ pid, label: v.label }));
}
/** Registers a long-running child (e.g. a server) for leak tracking; returns an unregister fn. */
export function trackProcess(child: ChildProcess, label: string): void {
if (child.pid === undefined) return;
const pid = child.pid;
live.set(pid, { label, child });
child.once('exit', () => live.delete(pid));
child.once('error', () => live.delete(pid));
}
/** Kills the whole process group of a detached child (best effort). */
export function killGroup(pid: number, signal: NodeJS.Signals = 'SIGKILL'): void {
try {
process.kill(-pid, signal);
} catch {
try {
process.kill(pid, signal);
} catch {
/* already gone */
}
}
}
/** Environment for harness-spawned helper tools (docker, curl, sh): inherits the worker env plus the leak marker. */
export function toolEnv(extra: Record<string, string | undefined> = {}): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = { ...process.env, [PROC_MARKER_ENV]: runId() };
for (const [k, v] of Object.entries(extra)) {
if (v === undefined) delete env[k];
else env[k] = v;
}
return env;
}
/**
* Runs a command to completion. argv is passed verbatim (no shell).
* Throws HarnessError('spawn') if it cannot start, ('timeout') on deadline,
* ('signal') on signal termination, unless explicitly allowed.
*/
export function runProcess(command: string, argv: string[], opts: RunProcessOptions): Promise<ProcResult> {
const timeoutMs = opts.timeoutMs ?? DEFAULT_PROCESS_TIMEOUT_MS;
const label = opts.label ?? command;
const started = performance.now();
return new Promise<ProcResult>((resolve, reject) => {
let child: ChildProcess;
try {
child = spawn(command, argv, {
env: opts.env,
cwd: opts.cwd,
stdio: [opts.input === undefined ? 'ignore' : 'pipe', 'pipe', 'pipe'],
detached: true, // own process group: the deadline kills descendants too
shell: false,
});
} catch (e) {
reject(new HarnessError('spawn', `cannot spawn ${label}: ${(e as Error).message}`));
return;
}
trackProcess(child, label);
const out: Buffer[] = [];
const err: Buffer[] = [];
let timedOut = false;
let spawnError: Error | undefined;
child.stdout!.on('data', (d: Buffer) => out.push(d));
child.stderr!.on('data', (d: Buffer) => err.push(d));
if (opts.input !== undefined) {
child.stdin!.on('error', () => {
/* EPIPE when the child does not read stdin: irrelevant */
});
child.stdin!.end(opts.input);
}
const timer = setTimeout(() => {
timedOut = true;
if (child.pid !== undefined) killGroup(child.pid, 'SIGKILL');
}, timeoutMs);
child.on('error', (e) => {
spawnError = e;
});
child.on('close', (code, signal) => {
clearTimeout(timer);
// Also reap any descendants that kept running after the main process exited
if (child.pid !== undefined) killGroup(child.pid, 'SIGKILL');
const stdoutBuffer = Buffer.concat(out);
const stderrBuffer = Buffer.concat(err);
const result: ProcResult = {
command,
argv: [...argv],
stdout: stdoutBuffer.toString('utf8'),
stderr: stderrBuffer.toString('utf8'),
stdoutBuffer,
stderrBuffer,
exitCode: code,
signal,
timedOut,
durationMs: performance.now() - started,
};
if (spawnError) {
// Node reports spawn failures (ENOENT, E2BIG, EACCES) as a negative errno exit code
reject(new HarnessError('spawn', `cannot spawn ${label}: ${spawnError.message}`, result));
return;
}
if (timedOut && !opts.allowTimeout) {
reject(
new HarnessError(
'timeout',
`${label} did not finish within ${timeoutMs} ms and was killed\n--- stdout (tail)\n${tail(result.stdout)}\n--- stderr (tail)\n${tail(result.stderr)}`,
result,
),
);
return;
}
if (!timedOut && signal !== null && !opts.allowSignal) {
reject(
new HarnessError(
'signal',
`${label} was terminated by unexpected signal ${signal}\n--- stderr (tail)\n${tail(result.stderr)}`,
result,
),
);
return;
}
resolve(result);
});
});
}
/** Last n characters of a string, for error messages. */
export function tail(s: string, n = 2000): string {
return s.length <= n ? s : '…' + s.slice(-n);
}
/** Runs async tasks with bounded concurrency, preserving result order. */
export async function mapLimit<T, R>(items: readonly T[], limit: number, fn: (item: T, index: number) => Promise<R>): Promise<R[]> {
const results = new Array<R>(items.length);
let next = 0;
const workers = Array.from({ length: Math.max(1, Math.min(limit, items.length)) }, async () => {
while (true) {
const i = next++;
if (i >= items.length) return;
results[i] = await fn(items[i], i);
}
});
await Promise.all(workers);
return results;
}
/** Polls `probe` until it returns a non-undefined value or the deadline passes (HarnessError 'timeout'). */
export async function pollUntil<T>(
what: string,
probe: () => Promise<T | undefined> | T | undefined,
{ timeoutMs = 10_000, intervalMs = 25 }: { timeoutMs?: number; intervalMs?: number } = {},
): Promise<T> {
const deadline = performance.now() + timeoutMs;
while (true) {
const v = await probe();
if (v !== undefined) return v;
if (performance.now() > deadline) throw new HarnessError('timeout', `timed out after ${timeoutMs} ms waiting for ${what}`);
await sleep(intervalMs);
}
}
export function sleep(ms: number): Promise<void> {
return new Promise((r) => setTimeout(r, ms));
}
+98
View File
@@ -0,0 +1,98 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Per-test record of every external interaction, attached (redacted) to failed tests.
*/
export type CliVia = 'argv' | 'shell' | 'docker-exec' | 'docker-run';
export interface CliRecord {
at: string;
via: CliVia;
command: string;
args: string[];
/** Application-relevant environment (DB_PATH, HTTP_*, LOG_LEVEL and explicit overrides). */
env: Record<string, string>;
exitCode: number | null;
signal: string | null;
timedOut: boolean;
durationMs: number;
stdout: string;
stderr: string;
}
export type HttpClientKind = 'playwright' | 'curl' | 'raw';
export interface HttpRecord {
at: string;
client: HttpClientKind;
method: string;
url: string;
requestHeaders: Array<[string, string]>;
requestBodyBytes: number;
status?: number;
responseHeaders?: Array<[string, string]>;
bodyPreview?: string;
durationMs: number;
/** Transport-level outcome / error text when there was no (complete) response. */
outcome?: string;
curlExitCode?: number;
}
const MAX_TEXT = 16 * 1024;
/** Keeps at most the last MAX_TEXT characters. */
export function clip(s: string, max = MAX_TEXT): string {
return s.length <= max ? s : `…[${s.length - max} chars clipped]…` + s.slice(-max);
}
/**
* JSON.stringify replacer that renders BigInt values (e.g. audit selections: limits and windows in
* ns) as decimal strings instead of throwing "Do not know how to serialize a BigInt".
*/
export function bigintReplacer(_key: string, value: unknown): unknown {
return typeof value === 'bigint' ? value.toString() : value;
}
/** JSON.stringify that accepts BigInt values (rendered as decimal strings). */
export function safeStringify(value: unknown, space?: number): string {
return JSON.stringify(value, bigintReplacer, space);
}
/** A plain JSON value equal to `value` as serialized by safeStringify (a snapshot: later mutations do not show). */
export function jsonSafe(value: unknown): unknown {
const s = safeStringify(value);
return s === undefined ? undefined : JSON.parse(s);
}
const MAX_RECORDS = 2000;
export class Recorder {
readonly cli: CliRecord[] = [];
readonly http: HttpRecord[] = [];
readonly notes: string[] = [];
/** JSON-safe snapshots of the recorded fuzz cases (see FuzzRecorder.record). */
readonly fuzzCases: unknown[] = [];
droppedCli = 0;
droppedHttp = 0;
addCli(r: Omit<CliRecord, 'at'>): void {
if (this.cli.length >= MAX_RECORDS) {
this.cli.shift();
this.droppedCli++;
}
this.cli.push({ at: new Date().toISOString(), ...r, stdout: clip(r.stdout), stderr: clip(r.stderr) });
}
addHttp(r: Omit<HttpRecord, 'at'>): void {
if (this.http.length >= MAX_RECORDS) {
this.http.shift();
this.droppedHttp++;
}
this.http.push({ at: new Date().toISOString(), ...r, bodyPreview: r.bodyPreview === undefined ? undefined : clip(r.bodyPreview, 2048) });
}
note(text: string): void {
this.notes.push(`${new Date().toISOString()} ${text}`);
}
}
+62
View File
@@ -0,0 +1,62 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* First reporter in the chain: scrubs token secrets from every test result before the list,
* JSON and HTML reporters see it. It redacts (a) every token registered by the workers for this
* run (exact substring, catching tokens embedded in longer strings) and (b) every token-shaped
* 43-character base64url run. Covers error messages/stacks/snippets (also of steps), captured
* stdout/stderr, inline attachment bodies, and text attachment files written under the output
* directory (e.g. Playwright's error-context.md).
*/
import fs from 'node:fs';
import path from 'node:path';
import type { FullConfig, Reporter, TestCase, TestError, TestResult, TestStep } from '@playwright/test/reporter';
import { loadPersistedSecrets, redactText } from './secrets';
const TEXTUAL = /^(text\/|application\/(json|xml|x-ndjson))|markdown/;
export default class RedactingReporter implements Reporter {
private outputDirs: string[] = [];
printsToStdio(): boolean {
return false;
}
onBegin(config: FullConfig): void {
this.outputDirs = config.projects.map((p) => path.resolve(p.outputDir));
}
onTestEnd(_test: TestCase, result: TestResult): void {
const dir = process.env.DPSAPI_E2E_RUN_DIR;
const secrets = dir ? loadPersistedSecrets(path.join(dir, 'secrets')) : [];
const r = (s: string) => redactText(s, secrets);
const fixError = (e: TestError | undefined) => {
if (!e) return;
if (e.message) e.message = r(e.message);
if (e.stack) e.stack = r(e.stack);
if (e.value) e.value = r(e.value);
if (e.snippet) e.snippet = r(e.snippet);
if (e.cause) fixError(e.cause);
};
result.errors.forEach(fixError);
fixError(result.error);
const fixStep = (s: TestStep) => {
fixError(s.error);
s.title = r(s.title);
s.steps.forEach(fixStep);
};
result.steps.forEach(fixStep);
result.stdout = result.stdout.map((c) => (typeof c === 'string' ? r(c) : Buffer.from(r(c.toString('utf8')))));
result.stderr = result.stderr.map((c) => (typeof c === 'string' ? r(c) : Buffer.from(r(c.toString('utf8')))));
for (const a of result.attachments) {
if (!TEXTUAL.test(a.contentType)) continue;
if (a.body) a.body = Buffer.from(r(a.body.toString('utf8')));
if (a.path && this.outputDirs.some((d) => path.resolve(a.path!).startsWith(d + path.sep)) && fs.existsSync(a.path)) {
const text = fs.readFileSync(a.path, 'utf8');
const red = r(text);
if (red !== text) fs.writeFileSync(a.path, red);
}
}
}
}
+89
View File
@@ -0,0 +1,89 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Small deterministic PRNG (sfc32 seeded through a string hash) for reproducible fuzzing.
* The run seed comes from $DPSAPI_E2E_SEED (default DEFAULT_SEED); each test derives its own
* stream from the run seed and its title path, so results do not depend on scheduling.
*/
import crypto from 'node:crypto';
export class Rng {
private a: number;
private b: number;
private c: number;
private d: number;
/** `seed` is any string; equal seeds produce equal streams. */
constructor(readonly seed: string) {
const h = crypto.createHash('sha256').update(seed, 'utf8').digest();
this.a = h.readUInt32LE(0);
this.b = h.readUInt32LE(4);
this.c = h.readUInt32LE(8);
this.d = h.readUInt32LE(12);
for (let i = 0; i < 12; i++) this.nextUint32();
}
/** Independent child stream, e.g. rng.fork('descriptions'). */
fork(label: string): Rng {
return new Rng(`${this.seed}/${label}`);
}
nextUint32(): number {
const t = (((this.a + this.b) >>> 0) + this.d) >>> 0;
this.d = (this.d + 1) >>> 0;
this.a = this.b ^ (this.b >>> 9);
this.b = (this.c + (this.c << 3)) >>> 0;
this.c = ((this.c << 21) | (this.c >>> 11)) >>> 0;
this.c = (this.c + t) >>> 0;
return t;
}
/** Uniform float in [0, 1). */
next(): number {
return this.nextUint32() / 4294967296;
}
/** Uniform integer in [min, max] (inclusive). */
int(min: number, max: number): number {
return min + Math.floor(this.next() * (max - min + 1));
}
bool(pTrue = 0.5): boolean {
return this.next() < pTrue;
}
pick<T>(items: readonly T[]): T {
if (items.length === 0) throw new Error('pick from empty array');
return items[this.int(0, items.length - 1)];
}
/** Fisher–Yates shuffle (returns a new array). */
shuffle<T>(items: readonly T[]): T[] {
const a = [...items];
for (let i = a.length - 1; i > 0; i--) {
const j = this.int(0, i);
[a[i], a[j]] = [a[j], a[i]];
}
return a;
}
bytes(n: number): Buffer {
const b = Buffer.alloc(n);
for (let i = 0; i < n; i++) b[i] = this.nextUint32() & 0xff;
return b;
}
/** Random string of `length` characters drawn from `alphabet` (code points). */
string(length: number, alphabet = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'): string {
const chars = [...alphabet];
let s = '';
for (let i = 0; i < length; i++) s += chars[this.int(0, chars.length - 1)];
return s;
}
}
/** Seed of a test: `<run seed>::<title path joined by ' > '>`. */
export function testSeed(runSeed: string, titlePath: readonly string[]): string {
return `${runSeed}::${titlePath.join(' > ')}`;
}
+142
View File
@@ -0,0 +1,142 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Secret registry and redaction.
*
* Every token created during a test is registered (automatically for tokens printed by
* `token new` / `token list` through the harness CLI runners). Registered tokens are:
* - replaced by `[REDACTED:<token_id>]` in every artifact the harness attaches;
* - persisted to a per-test file read by the redacting reporter, which scrubs error
* messages, stdout/stderr and text attachments of the final report;
* - searched for in captured logs (`assertNoSecretsIn`), automatically at teardown.
*/
import crypto from 'node:crypto';
import fs from 'node:fs';
import path from 'node:path';
import { expect } from '@playwright/test';
/** Token format produced by `dpsapi token new`: 32 random bytes, base64url, no padding. */
export const TOKEN_RE = /^[A-Za-z0-9_-]{43}$/;
/** A token-shaped run of characters not adjacent to other token characters. */
export const TOKEN_CANDIDATE_RE = /(?<![A-Za-z0-9_-])[A-Za-z0-9_-]{43}(?![A-Za-z0-9_-])/g;
/** The identifier the application logs instead of a token: first 8 hex chars of SHA-256. */
export function tokenId(token: string): string {
return crypto.createHash('sha256').update(token, 'utf8').digest('hex').slice(0, 8);
}
export function redactionFor(secret: string): string {
return `[REDACTED:${tokenId(secret)}]`;
}
/**
* Replaces each secret (longest first) and, unless disabled, every token-shaped
* 43-character run, with a redaction marker.
*/
export function redactText(text: string, secrets: Iterable<string>, { patterns = true }: { patterns?: boolean } = {}): string {
let out = text;
const list = [...new Set(secrets)].filter((s) => s.length > 0).sort((a, b) => b.length - a.length);
for (const s of list) {
if (out.includes(s)) out = out.split(s).join(redactionFor(s));
}
if (patterns) out = out.replace(TOKEN_CANDIDATE_RE, (m) => redactionFor(m));
return out;
}
/** Recursively redacts every string inside a JSON-like value (BigInts become decimal strings, so the result always serializes). */
export function redactValue<T>(value: T, secrets: Iterable<string>): T {
const list = [...secrets];
const walk = (v: unknown): unknown => {
if (typeof v === 'string') return redactText(v, list);
if (typeof v === 'bigint') return v.toString(); // JSON has no BigInt: keep the value, as text
if (Array.isArray(v)) return v.map(walk);
if (Buffer.isBuffer(v)) return redactText(v.toString('utf8'), list);
if (v && typeof v === 'object') {
const o: Record<string, unknown> = {};
for (const [k, x] of Object.entries(v)) o[k] = walk(x);
return o;
}
return v;
};
return walk(value) as T;
}
export class SecretRegistry {
private readonly secrets = new Map<string, string>(); // secret -> label
constructor(private readonly persistFile?: string) {}
/** Registers a secret (idempotent). Returns it for chaining. */
register(secret: string, label = 'token'): string {
if (!secret || this.secrets.has(secret)) return secret;
this.secrets.set(secret, label);
if (this.persistFile) {
fs.mkdirSync(path.dirname(this.persistFile), { recursive: true, mode: 0o700 });
fs.appendFileSync(this.persistFile, JSON.stringify(secret) + '\n', { mode: 0o600 });
}
return secret;
}
/** Registers every token-shaped string found in text (e.g. `token list` output). */
registerAllIn(text: string, label = 'token'): string[] {
const found = text.match(TOKEN_CANDIDATE_RE) ?? [];
for (const t of found) this.register(t, label);
return found;
}
has(secret: string): boolean {
return this.secrets.has(secret);
}
all(): string[] {
return [...this.secrets.keys()];
}
get size(): number {
return this.secrets.size;
}
redact(text: string): string {
return redactText(text, this.secrets.keys());
}
redactValue<T>(value: T): T {
return redactValue(value, this.secrets.keys());
}
/** Secrets (redacted labels) found verbatim in text. */
findIn(text: string | Buffer): string[] {
const s = typeof text === 'string' ? text : text.toString('latin1');
const b = typeof text === 'string' ? undefined : text;
return this.all()
.filter((secret) => (b ? b.includes(Buffer.from(secret, 'utf8')) : s.includes(secret)))
.map((secret) => redactionFor(secret));
}
/** Fails the test if any registered secret occurs in text. `where` names the source for the message. */
assertNoSecretsIn(text: string | Buffer, where = 'text'): void {
const leaked = this.findIn(text);
expect(leaked, `registered token secret(s) found in ${where}`).toEqual([]);
}
}
/** Reads every secret persisted by workers for this run (used by the reporter). */
export function loadPersistedSecrets(dir: string): string[] {
const out = new Set<string>();
let files: string[] = [];
try {
files = fs.readdirSync(dir);
} catch {
return [];
}
for (const f of files) {
try {
for (const line of fs.readFileSync(path.join(dir, f), 'utf8').split('\n')) {
if (line.trim()) out.add(JSON.parse(line) as string);
}
} catch {
/* partially written line: ignore */
}
}
return [...out];
}
+354
View File
@@ -0,0 +1,354 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Running the compiled binary in server mode with a controlled Docker socket.
*/
import { spawn, type ChildProcess } from 'node:child_process';
import fs from 'node:fs';
import { binaryPath } from './env';
import { HarnessError } from './errors';
import { FakeDocker } from './fake-docker';
import { buildRawRequest, rawHttp } from './http';
import { parseLogs, type LogQuery, type LogRecord, type LogSet } from './logs';
import { killGroup, sleep, trackProcess } from './proc';
import { baseAppEnv, mergeEnv, recordableEnv } from './cli';
import type { Recorder } from './recorder';
import type { Workspace } from './workspace';
/**
* What the server sees at /var/run/docker.sock:
* - 'real': the host socket (binary runs directly, no namespaces);
* - FakeDocker: the fake's socket bind-mounted over the path (private mount namespace);
* - 'inaccessible': a root-owned unreadable file (/etc/shadow) bind-mounted over it → EACCES;
* - 'missing': an empty tmpfs over /run, so the path does not exist → ENOENT;
* - {bindSource}: any host path bind-mounted over it.
*/
export type DockerBackend = 'real' | 'inaccessible' | 'missing' | FakeDocker | { bindSource: string };
export interface StartServerOptions {
/** LOG_LEVEL; default 'INFO'; null leaves it unset (app default). */
logLevel?: string | null;
/** HTTP_ADDR; default '127.0.0.1'; null leaves it unset (app default 0.0.0.0). */
addr?: string | null;
/** HTTP_PORT; default: a freshly allocated free port; null leaves it unset (app default 80 — only usable with privateNetwork). */
port?: number | string | null;
/** DB_PATH; default the workspace's dpsapi.db. */
dbPath?: string;
/** Default: a new FakeDocker in empty mode (exposed as handle.fakeDocker). */
docker?: DockerBackend;
/** Extra environment overrides (undefined removes a variable). Applied last. */
env?: Record<string, string | undefined>;
/** Run in a private network namespace (loopback down; only log-based readiness). */
privateNetwork?: boolean;
/** Wait for HTTP readiness before returning (default true unless privateNetwork). */
waitReady?: boolean;
/** Default 10 s. */
readyTimeoutMs?: number;
/** Name used in artifacts. */
label?: string;
/** Binary to run (default $DPSAPI_BIN / ../dpsapi), e.g. /app extracted with image.extractBinary(). */
binary?: string;
}
export interface ExitResult {
exitCode: number | null;
signal: NodeJS.Signals | null;
/** Time from the stop request (or start, for waitForExit) to exit. */
durationMs: number;
stdout: string;
stderr: string;
}
export interface ServerContext {
workspace: Workspace;
recorder: Recorder;
allocatePort: () => Promise<number>;
newFakeDocker: () => Promise<FakeDocker>;
/** Registry of every handle started in the test (for teardown/artifacts). */
servers: ServerHandle[];
}
interface Generation {
child: ChildProcess;
pid: number;
stdout: string;
stderr: string;
startedAt: number;
exit?: { exitCode: number | null; signal: NodeJS.Signals | null; at: number };
closed: Promise<void>;
}
export class ServerHandle {
readonly label: string;
/** Effective HTTP_ADDR value (undefined when unset → app default 0.0.0.0). */
readonly addr: string | undefined;
/** Effective port (80 when HTTP_PORT is unset). */
readonly port: number;
/** Address the harness connects to (0.0.0.0 → 127.0.0.1, :: → ::1). */
readonly connectHost: string;
/** e.g. http://127.0.0.1:21003 or http://[::1]:21003 (no trailing slash). */
readonly baseURL: string;
readonly dbPath: string;
/** Present when the backend is a FakeDocker. */
readonly fakeDocker?: FakeDocker;
/** Environment of the server process. */
readonly env: Record<string, string>;
private readonly gens: Generation[] = [];
constructor(
private readonly ctx: ServerContext,
readonly options: StartServerOptions & { docker: DockerBackend },
env: Record<string, string>,
port: number,
) {
this.label = options.label ?? `server${ctx.servers.length + 1}`;
this.env = env;
this.addr = env.HTTP_ADDR;
this.port = port;
const a = this.addr ?? '0.0.0.0';
this.connectHost = a === '0.0.0.0' ? '127.0.0.1' : a === '::' ? '::1' : a;
this.baseURL = `http://${this.connectHost.includes(':') ? `[${this.connectHost}]` : this.connectHost}:${port}`;
this.dbPath = env.DB_PATH;
if (options.docker instanceof FakeDocker) this.fakeDocker = options.docker;
}
/** baseURL + path, path used verbatim (must start with '/'). */
url(path = '/api/v1/containers'): string {
return this.baseURL + path;
}
/** Process id of the current generation (the binary itself, also under unshare). */
get pid(): number {
return this.current().pid;
}
/** How many times the process was (re)started. */
get generation(): number {
return this.gens.length;
}
/** stdout of the current process generation. */
get stdout(): string {
return this.current().stdout;
}
/** stderr (logs) of the current process generation. */
get stderr(): string {
return this.current().stderr;
}
/** stderr of every generation, concatenated with separators (artifacts / secret scans). */
allOutput(): Array<{ generation: number; pid: number; stdout: string; stderr: string; exit?: Generation['exit'] }> {
return this.gens.map((g, i) => ({ generation: i + 1, pid: g.pid, stdout: g.stdout, stderr: g.stderr, exit: g.exit }));
}
/** Parsed logs of the current generation. */
logs(): LogSet {
return parseLogs(this.stderr);
}
isRunning(): boolean {
const g = this.gens[this.gens.length - 1];
return !!g && !g.exit;
}
/** Exit status of the current generation once it exited. */
get exit(): ExitResult | undefined {
const g = this.current();
return g.exit ? { exitCode: g.exit.exitCode, signal: g.exit.signal, durationMs: g.exit.at - g.startedAt, stdout: g.stdout, stderr: g.stderr } : undefined;
}
/** @internal spawns a new generation. */
async spawn(): Promise<void> {
const opts = this.options;
const bin = opts.binary ?? binaryPath();
let command: string;
let args: string[];
const backend = opts.docker;
if (backend === 'real' && !opts.privateNetwork) {
command = bin;
args = [];
} else {
const flags = opts.privateNetwork ? '-Urmn' : '-Urm';
let script: string;
let src = '';
if (backend === 'real') script = 'exec "$2"';
else if (backend === 'missing') script = 'mount -t tmpfs none /run && exec "$2"';
else {
src = backend === 'inaccessible' ? '/etc/shadow' : backend instanceof FakeDocker ? backend.socketPath : backend.bindSource;
script = 'mount --bind "$1" /var/run/docker.sock && exec "$2"';
}
command = 'unshare';
args = [flags, 'sh', '-c', script, 'sh', src, bin];
}
const child = spawn(command, args, { env: this.env, cwd: this.ctx.workspace.dir, stdio: ['ignore', 'pipe', 'pipe'], detached: true });
const spawnErr = await new Promise<Error | undefined>((resolve) => {
child.once('spawn', () => resolve(undefined));
child.once('error', (e) => resolve(e));
});
if (spawnErr || child.pid === undefined) throw new HarnessError('spawn', `cannot start server ${this.label}: ${spawnErr?.message}`);
trackProcess(child, `server ${this.label}`);
const gen: Generation = { child, pid: child.pid, stdout: '', stderr: '', startedAt: performance.now(), closed: Promise.resolve() };
const logFile = this.ctx.workspace.path(`${this.label}.gen${this.gens.length + 1}.stderr.log`);
child.stdout!.setEncoding('utf8');
child.stderr!.setEncoding('utf8');
child.stdout!.on('data', (d: string) => (gen.stdout += d));
child.stderr!.on('data', (d: string) => {
gen.stderr += d;
try {
fs.appendFileSync(logFile, d);
} catch {
/* workspace removed during teardown */
}
});
child.once('exit', (exitCode, signal) => {
gen.exit = { exitCode, signal, at: performance.now() };
});
gen.closed = new Promise<void>((resolve) => child.once('close', () => resolve()));
this.gens.push(gen);
this.ctx.recorder.note(`server ${this.label} gen ${this.gens.length} started pid=${child.pid} cmd=${command} ${JSON.stringify(args)} env=${JSON.stringify(recordableEnv(this.env))}`);
}
/**
* Waits until the server answers HTTP: polls `OPTIONS * HTTP/1.1` over a raw socket (answered by
* Go's server itself, so it creates no application log record and no audit row). Throws if the
* process exits first (with its stderr) or HarnessError('timeout').
*/
async waitReady(timeoutMs = this.options.readyTimeoutMs ?? 10_000): Promise<void> {
const deadline = performance.now() + timeoutMs;
while (true) {
const g = this.current();
if (g.exit) {
await g.closed;
throw new Error(`server ${this.label} exited during startup (code=${g.exit.exitCode}, signal=${g.exit.signal})\n--- stderr\n${g.stderr}`);
}
const r = await rawHttp({
host: this.connectHost,
port: this.port,
data: buildRawRequest({ method: 'OPTIONS', target: '*' }),
timeoutMs: 2000,
allowTimeout: true,
}).catch(() => undefined);
if (r?.outcome === 'response') return;
if (performance.now() > deadline) {
throw new HarnessError('timeout', `server ${this.label} not ready via HTTP on ${this.baseURL} within ${timeoutMs} ms\n--- stderr\n${g.stderr}`);
}
await sleep(20);
}
}
/** Waits until a log record matching q appears in the current generation's stderr. */
async waitForLog(q: LogQuery, timeoutMs = 10_000): Promise<LogRecord> {
const deadline = performance.now() + timeoutMs;
while (true) {
const r = this.logs().find(q);
if (r) return r;
if (this.current().exit) {
await this.current().closed;
const again = this.logs().find(q);
if (again) return again;
throw new Error(`server ${this.label} exited before logging ${JSON.stringify(q)}\n--- stderr\n${this.stderr}`);
}
if (performance.now() > deadline) throw new HarnessError('timeout', `log record ${JSON.stringify(q)} not seen within ${timeoutMs} ms`);
await sleep(20);
}
}
/** Sends a signal to the server process (does not wait). */
signal(sig: NodeJS.Signals): void {
const g = this.current();
if (g.exit) throw new Error(`server ${this.label} is not running`);
process.kill(g.pid, sig);
}
/**
* Sends `sig` (default SIGTERM) and waits for the process to exit and its output to be flushed.
* Throws HarnessError('hang') (after SIGKILL) if it does not exit within timeoutMs, and
* HarnessError('signal') if it died from a signal (dpsapi handles SIGTERM/SIGINT itself) unless
* allowSignalExit. Returns the exit status; assert `exitCode` explicitly.
*/
async stop(sig: NodeJS.Signals = 'SIGTERM', { timeoutMs = 20_000, allowSignalExit = false }: { timeoutMs?: number; allowSignalExit?: boolean } = {}): Promise<ExitResult> {
const g = this.current();
const t0 = performance.now();
if (!g.exit) process.kill(g.pid, sig);
const exited = await Promise.race([g.closed.then(() => true), sleep(timeoutMs).then(() => false)]);
if (!exited) {
killGroup(g.pid, 'SIGKILL');
await Promise.race([g.closed, sleep(5000)]);
throw new HarnessError('hang', `server ${this.label} (pid ${g.pid}) did not exit within ${timeoutMs} ms after ${sig}; killed\n--- stderr (tail)\n${g.stderr.slice(-3000)}`);
}
const res: ExitResult = { exitCode: g.exit!.exitCode, signal: g.exit!.signal, durationMs: g.exit!.at - t0, stdout: g.stdout, stderr: g.stderr };
this.ctx.recorder.note(`server ${this.label} gen ${this.gens.length} stopped with ${sig}: exitCode=${res.exitCode} signal=${res.signal}`);
if (res.signal !== null && !allowSignalExit) {
throw new HarnessError('signal', `server ${this.label} was terminated by signal ${res.signal} after ${sig} instead of exiting\n--- stderr (tail)\n${g.stderr.slice(-3000)}`);
}
return res;
}
/** SIGKILLs the process group and waits; never throws for the signal exit. */
async kill(): Promise<ExitResult> {
const g = this.current();
const t0 = performance.now();
if (!g.exit) killGroup(g.pid, 'SIGKILL');
await Promise.race([g.closed, sleep(10_000)]);
if (!g.exit) throw new HarnessError('hang', `server ${this.label} (pid ${g.pid}) survived SIGKILL`);
return { exitCode: g.exit.exitCode, signal: g.exit.signal, durationMs: g.exit.at - t0, stdout: g.stdout, stderr: g.stderr };
}
/** Waits for the process to exit on its own (e.g. startup failure). HarnessError('timeout') otherwise. */
async waitForExit(timeoutMs = 15_000): Promise<ExitResult> {
const g = this.current();
const exited = await Promise.race([g.closed.then(() => true), sleep(timeoutMs).then(() => false)]);
if (!exited) throw new HarnessError('timeout', `server ${this.label} still running after ${timeoutMs} ms\n--- stderr (tail)\n${g.stderr.slice(-3000)}`);
if (g.exit!.signal !== null) throw new HarnessError('signal', `server ${this.label} terminated by signal ${g.exit!.signal}`);
return { exitCode: g.exit!.exitCode, signal: g.exit!.signal, durationMs: g.exit!.at - g.startedAt, stdout: g.stdout, stderr: g.stderr };
}
/**
* Graceful restart against the same DB/port/backend: SIGTERM, require exit code 0, start a new
* process and wait for readiness. stdout/stderr/logs() then refer to the new generation.
*/
async restart(): Promise<void> {
if (this.isRunning()) {
const r = await this.stop('SIGTERM');
if (r.exitCode !== 0) throw new Error(`server ${this.label} exited with ${r.exitCode} on restart\n${r.stderr}`);
}
await this.spawn();
if (this.options.waitReady ?? !this.options.privateNetwork) await this.waitReady();
}
private current(): Generation {
const g = this.gens[this.gens.length - 1];
if (!g) throw new Error(`server ${this.label} was never started`);
return g;
}
}
/** Resolves options, spawns the server and (by default) waits for HTTP readiness. */
export async function startServer(ctx: ServerContext, opts: StartServerOptions = {}): Promise<ServerHandle> {
const docker = opts.docker ?? (await ctx.newFakeDocker());
let port: number;
let portEnv: string | undefined;
if (opts.port === null) {
port = 80;
portEnv = undefined;
} else if (opts.port === undefined) {
port = await ctx.allocatePort();
portEnv = String(port);
} else {
portEnv = String(opts.port);
port = Number(opts.port);
}
const env = mergeEnv(baseAppEnv(opts.dbPath ?? ctx.workspace.dbPath), {
HTTP_ADDR: opts.addr === null ? undefined : (opts.addr ?? '127.0.0.1'),
HTTP_PORT: portEnv,
LOG_LEVEL: opts.logLevel === null ? undefined : (opts.logLevel ?? 'INFO'),
...(opts.env ?? {}),
});
const handle = new ServerHandle(ctx, { ...opts, docker }, env, port);
ctx.servers.push(handle);
await handle.spawn();
if (opts.waitReady ?? !opts.privateNetwork) await handle.waitReady();
return handle;
}
+120
View File
@@ -0,0 +1,120 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Per-test temporary workspaces and TCP port allocation.
*/
import fs from 'node:fs';
import net from 'node:net';
import path from 'node:path';
import { ALPINE_IMAGE, LABEL_RUN, RESOURCE_PREFIX, runId, workspacesDir } from './env';
import { HarnessError } from './errors';
import { runProcess, toolEnv } from './proc';
export class Workspace {
/** Default SQLite path for the test (`<dir>/dpsapi.db`); nothing is created there by the harness. */
readonly dbPath: string;
constructor(readonly dir: string) {
this.dbPath = path.join(dir, 'dpsapi.db');
}
/** Creates a fresh workspace directory under the run's workspaces dir. */
static create(key: string): Workspace {
fs.mkdirSync(workspacesDir(), { recursive: true });
const dir = fs.mkdtempSync(path.join(workspacesDir(), `${key}-`));
return new Workspace(dir);
}
/** Absolute path inside the workspace (not created). */
path(...parts: string[]): string {
return path.join(this.dir, ...parts);
}
/** Creates (recursively) and returns a directory inside the workspace. */
mkdir(...parts: string[]): string {
const p = this.path(...parts);
fs.mkdirSync(p, { recursive: true });
return p;
}
writeFile(name: string, data: string | Buffer, mode?: number): string {
const p = this.path(name);
fs.mkdirSync(path.dirname(p), { recursive: true });
fs.writeFileSync(p, data, mode === undefined ? undefined : { mode });
return p;
}
/** Removes the workspace, falling back to a throwaway container for root-owned files. */
async remove(): Promise<void> {
await forceRemoveDir(this.dir);
}
}
/**
* Removes a directory tree. Files written by containers running as root cannot be removed by
* the host user; for those, the contents are deleted from a throwaway, labelled alpine container.
*/
export async function forceRemoveDir(dir: string): Promise<void> {
try {
fs.rmSync(dir, { recursive: true, force: true });
return;
} catch (e) {
const code = (e as NodeJS.ErrnoException).code;
if (code !== 'EACCES' && code !== 'EPERM' && code !== 'ENOTEMPTY') throw e;
}
const name = `${RESOURCE_PREFIX}${runId()}-rm-${Math.random().toString(36).slice(2, 10)}`;
const r = await runProcess(
'docker',
['run', '--rm', '--name', name, '--label', `${LABEL_RUN}=${runId()}`, '--network', 'none', '-v', `${dir}:/victim`, ALPINE_IMAGE, 'sh', '-c', 'find /victim -mindepth 1 -delete'],
{ env: toolEnv(), timeoutMs: 60_000, label: 'docker cleanup container' },
);
if (r.exitCode !== 0) throw new HarnessError('infrastructure', `cannot clean ${dir} via container: ${r.stderr}`);
fs.rmSync(dir, { recursive: true, force: true });
}
/** Lowest port used by the allocator; windows of PORT_WINDOW ports per parallel slot, below the ephemeral range. */
const PORT_BASE = 20_000;
const PORT_WINDOW = 500;
const PORT_SLOTS = 24; // 20000..31999, below Linux's default ephemeral range (32768+)
let portCursor = 0;
/** True when `port` can currently be bound on every local address (checked dual-stack). */
export function isPortFree(port: number): Promise<boolean> {
const tryListen = (host: string, ipv6Only: boolean) =>
new Promise<'free' | 'busy' | 'unsupported'>((resolve) => {
const srv = net.createServer();
srv.once('error', (e: NodeJS.ErrnoException) => {
resolve(e.code === 'EADDRINUSE' || e.code === 'EACCES' ? 'busy' : 'unsupported');
});
srv.listen({ host, port, ipv6Only, exclusive: true }, () => srv.close(() => resolve('free')));
});
return tryListen('::', false).then(async (r) => {
if (r === 'unsupported') return (await tryListen('0.0.0.0', false)) === 'free';
return r === 'free';
});
}
/**
* Allocates a currently free TCP port. Concurrent workers draw from disjoint windows keyed
* by their parallel index, so two running tests never get the same port from the allocator;
* the bind check additionally skips ports used by anything else on the host.
*/
export async function allocatePort(parallelIndex: number): Promise<number> {
const slot = parallelIndex % PORT_SLOTS;
for (let attempt = 0; attempt < PORT_WINDOW; attempt++) {
const port = PORT_BASE + slot * PORT_WINDOW + (portCursor++ % PORT_WINDOW);
if (await isPortFree(port)) return port;
}
throw new HarnessError('infrastructure', `no free port in window of parallel slot ${slot}`);
}
/** Holds a port busy (listening socket) until close() — for "address already in use" tests. */
export async function occupyPort(port: number, host = '127.0.0.1'): Promise<{ close: () => Promise<void> }> {
const srv = net.createServer((s) => s.destroy());
await new Promise<void>((resolve, reject) => {
srv.once('error', reject);
srv.listen({ host, port, exclusive: true }, () => resolve());
});
return { close: () => new Promise<void>((r) => srv.close(() => r())) };
}
+457
View File
@@ -0,0 +1,457 @@
{
"name": "dpsapi-e2e",
"version": "0.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "dpsapi-e2e",
"version": "0.0.0",
"devDependencies": {
"@playwright/test": "1.63.0",
"@types/node": "26.6.4",
"typescript": "7.0.2"
},
"engines": {
"node": ">=22.13"
}
},
"node_modules/@playwright/test": {
"version": "1.63.0",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.63.0.tgz",
"integrity": "sha512-oxMK4vllB9RK5NQ2l1pq1IfOf2AvnEuj/vYGDj0H2nMtmtZpKtCwt/l00GEO6xjGfpBNAvjovvYdCm50dRQkpQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.63.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/@types/node": {
"version": "26.6.4",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.6.4.tgz",
"integrity": "sha512-ldVPDCzj7fsaGZrLB0NuHuTvJcsNasysBAqMolr/cgxrLd1xbqxIr3XJiPnHHJUCxj5sNF1vnRj9aWnrVh5Jcg==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~8.9.0"
}
},
"node_modules/@typescript/typescript-aix-ppc64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-aix-ppc64/-/typescript-aix-ppc64-7.0.2.tgz",
"integrity": "sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-darwin-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-arm64/-/typescript-darwin-arm64-7.0.2.tgz",
"integrity": "sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-darwin-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-x64/-/typescript-darwin-x64-7.0.2.tgz",
"integrity": "sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-freebsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-arm64/-/typescript-freebsd-arm64-7.0.2.tgz",
"integrity": "sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-freebsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-x64/-/typescript-freebsd-x64-7.0.2.tgz",
"integrity": "sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-arm": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm/-/typescript-linux-arm-7.0.2.tgz",
"integrity": "sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ==",
"cpu": [
"arm"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm64/-/typescript-linux-arm64-7.0.2.tgz",
"integrity": "sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-loong64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-loong64/-/typescript-linux-loong64-7.0.2.tgz",
"integrity": "sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ==",
"cpu": [
"loong64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-mips64el": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-mips64el/-/typescript-linux-mips64el-7.0.2.tgz",
"integrity": "sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-ppc64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-ppc64/-/typescript-linux-ppc64-7.0.2.tgz",
"integrity": "sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-riscv64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-riscv64/-/typescript-linux-riscv64-7.0.2.tgz",
"integrity": "sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-s390x": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-s390x/-/typescript-linux-s390x-7.0.2.tgz",
"integrity": "sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw==",
"cpu": [
"s390x"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-x64/-/typescript-linux-x64-7.0.2.tgz",
"integrity": "sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-netbsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-arm64/-/typescript-netbsd-arm64-7.0.2.tgz",
"integrity": "sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-netbsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-x64/-/typescript-netbsd-x64-7.0.2.tgz",
"integrity": "sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-openbsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-arm64/-/typescript-openbsd-arm64-7.0.2.tgz",
"integrity": "sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-openbsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-x64/-/typescript-openbsd-x64-7.0.2.tgz",
"integrity": "sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-sunos-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-sunos-x64/-/typescript-sunos-x64-7.0.2.tgz",
"integrity": "sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-win32-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-win32-arm64/-/typescript-win32-arm64-7.0.2.tgz",
"integrity": "sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-win32-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-win32-x64/-/typescript-win32-x64-7.0.2.tgz",
"integrity": "sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/playwright": {
"version": "1.63.0",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.63.0.tgz",
"integrity": "sha512-+7ziBLidS4NaNCdt57SUDT+wYmmd5fmiQejUic/kb+YsYSCPyOOE9sebzMjNmQrsnNpDJqd4WHvV/8lfKfUDUg==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.63.0"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/playwright-core": {
"version": "1.63.0",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.63.0.tgz",
"integrity": "sha512-rYCsBF/M5HjUch52bbtVONEFjv6Xu8sm8h72dNlR5bzIE1fvC/bxgspzkjSfU+MweEMmPM8KJebG6nnyxo5mCg==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/typescript": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-7.0.2.tgz",
"integrity": "sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc"
},
"engines": {
"node": ">=16.20.0"
},
"optionalDependencies": {
"@typescript/typescript-aix-ppc64": "7.0.2",
"@typescript/typescript-darwin-arm64": "7.0.2",
"@typescript/typescript-darwin-x64": "7.0.2",
"@typescript/typescript-freebsd-arm64": "7.0.2",
"@typescript/typescript-freebsd-x64": "7.0.2",
"@typescript/typescript-linux-arm": "7.0.2",
"@typescript/typescript-linux-arm64": "7.0.2",
"@typescript/typescript-linux-loong64": "7.0.2",
"@typescript/typescript-linux-mips64el": "7.0.2",
"@typescript/typescript-linux-ppc64": "7.0.2",
"@typescript/typescript-linux-riscv64": "7.0.2",
"@typescript/typescript-linux-s390x": "7.0.2",
"@typescript/typescript-linux-x64": "7.0.2",
"@typescript/typescript-netbsd-arm64": "7.0.2",
"@typescript/typescript-netbsd-x64": "7.0.2",
"@typescript/typescript-openbsd-arm64": "7.0.2",
"@typescript/typescript-openbsd-x64": "7.0.2",
"@typescript/typescript-sunos-x64": "7.0.2",
"@typescript/typescript-win32-arm64": "7.0.2",
"@typescript/typescript-win32-x64": "7.0.2"
}
},
"node_modules/undici-types": {
"version": "8.9.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.9.0.tgz",
"integrity": "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==",
"dev": true,
"license": "MIT"
}
}
}
+20
View File
@@ -0,0 +1,20 @@
{
"name": "dpsapi-e2e",
"private": true,
"version": "0.0.0",
"description": "Playwright end-to-end suite for dpsapi",
"scripts": {
"test": "playwright test",
"test:host": "playwright test --project=host",
"test:image": "playwright test --project=image",
"typecheck": "tsc --noEmit"
},
"engines": {
"node": ">=22.13"
},
"devDependencies": {
"@playwright/test": "1.63.0",
"@types/node": "26.6.4",
"typescript": "7.0.2"
}
}
+40
View File
@@ -0,0 +1,40 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
import { defineConfig } from '@playwright/test';
/**
* dpsapi E2E suite (API/process only — no browsers).
*
* npx playwright test --project=host # compiled binary on the host ($DPSAPI_BIN, default ../dpsapi)
* npx playwright test --project=image # final container image ($DPSAPI_IMAGE)
*
* Setup projects (host-setup / image-setup) build/validate prerequisites and run automatically
* as dependencies. Global teardown fails the run on leaked containers/processes/workspaces.
*/
export default defineConfig({
testDir: './tests',
globalSetup: './harness/global-setup.ts',
globalTeardown: './harness/global-teardown.ts',
fullyParallel: true,
workers: Number(process.env.DPSAPI_E2E_WORKERS || 4),
retries: 0,
forbidOnly: true,
timeout: 60_000,
expect: { timeout: 10_000 },
reportSlowTests: null,
outputDir: './test-results',
reporter: [
['./harness/redacting-reporter.ts'],
['list'],
['json', { outputFile: './reports/results.json' }],
['html', { open: 'never', outputFolder: './reports/html' }],
],
use: { trace: 'off' },
projects: [
{ name: 'host-setup', testDir: './setup', testMatch: 'host.setup.ts' },
{ name: 'host', testDir: './tests/host', dependencies: ['host-setup'] },
{ name: 'image-setup', testDir: './setup', testMatch: 'image.setup.ts' },
{ name: 'image', testDir: './tests/image', dependencies: ['image-setup'] },
],
});
+44
View File
@@ -0,0 +1,44 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* `host-setup` project: build (when using the default ../dpsapi) and validate the production
* binary, and check every host prerequisite. All host tests depend on it.
*/
import fs from 'node:fs';
import { test as setup, expect } from '@playwright/test';
import { binaryIsDefault, binaryPath, REPO_ROOT } from '../harness/env';
import { HarnessError } from '../harness/errors';
import { requireCurl, requireDocker, requireUnshare, sh } from './prereq';
setup('host prerequisites: tools, namespaces, Docker, production binary', async () => {
setup.setTimeout(300_000);
requireCurl();
requireUnshare();
requireDocker();
const bin = binaryPath();
if (binaryIsDefault() && process.env.DPSAPI_E2E_SKIP_BUILD !== '1') {
// PLAN §12 item 1: build the production binary used by the suite (same recipe as `make build`).
const r = sh('make', ['-C', REPO_ROOT, 'build'], 280_000);
if (r.code !== 0) throw new HarnessError('prerequisite', `make build failed (${r.code}):\n${r.out}\n${r.err}`);
}
if (!fs.existsSync(bin)) throw new HarnessError('prerequisite', `binary ${bin} does not exist (run make build or set DPSAPI_BIN)`);
try {
fs.accessSync(bin, fs.constants.X_OK);
} catch {
throw new HarnessError('prerequisite', `binary ${bin} is not executable`);
}
const head = Buffer.alloc(4);
const fd = fs.openSync(bin, 'r');
fs.readSync(fd, head, 0, 4, 0);
fs.closeSync(fd);
if (!head.equals(Buffer.from([0x7f, 0x45, 0x4c, 0x46]))) throw new HarnessError('prerequisite', `binary ${bin} is not an ELF executable`);
// Smoke check: with an empty environment the binary must reject the configuration (exit 2).
const r = sh('env', ['-i', bin], 10_000);
if (r.code !== 2 || r.err !== 'dpsapi: configuration error: DB_PATH is required\n') {
throw new HarnessError('prerequisite', `binary ${bin} does not behave like dpsapi (env -i → exit ${r.code}, stderr ${JSON.stringify(r.err)})`);
}
expect(r.out).toBe('');
});
+20
View File
@@ -0,0 +1,20 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* `image-setup` project: check the image under test ($DPSAPI_IMAGE) and the tools the image
* suite needs. All image tests depend on it.
*/
import { test as setup } from '@playwright/test';
import { imageRef } from '../harness/env';
import { HarnessError } from '../harness/errors';
import { requireCurl, requireDocker, sh } from './prereq';
setup('image prerequisites: Docker, curl, image under test', async () => {
requireCurl();
requireDocker();
const image = imageRef();
if (!image) throw new HarnessError('prerequisite', 'DPSAPI_IMAGE is not set (run `make docker-e2e`, or export DPSAPI_IMAGE=<image ref>)');
const r = sh('docker', ['image', 'inspect', '--format', '{{.Id}}', image]);
if (r.code !== 0) throw new HarnessError('prerequisite', `image ${image} is not present locally (build it with \`make docker\`): ${r.err.trim()}`);
});
+40
View File
@@ -0,0 +1,40 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Prerequisite checks shared by the setup projects. Every failure is a HarnessError('prerequisite')
* with an actionable message, so a broken environment fails fast and is never mistaken for an
* application failure.
*/
import fs from 'node:fs';
import { spawnSync } from 'node:child_process';
import { ALPINE_IMAGE } from '../harness/env';
import { HarnessError } from '../harness/errors';
export function sh(cmd: string, args: string[], timeoutMs = 60_000): { code: number | null; out: string; err: string } {
const r = spawnSync(cmd, args, { encoding: 'utf8', timeout: timeoutMs });
if (r.error) return { code: null, out: '', err: r.error.message };
return { code: r.status, out: r.stdout ?? '', err: r.stderr ?? '' };
}
export function requireTool(name: string, args: string[], hint: string): void {
const r = sh(name, args, 20_000);
if (r.code !== 0) throw new HarnessError('prerequisite', `${name} is required (${hint}): \`${name} ${args.join(' ')}\` failed: ${r.err.trim() || r.out.trim()}`);
}
export function requireDocker(): void {
requireTool('docker', ['version', '--format', '{{.Server.Version}}'], 'Docker Engine reachable by the current user');
const r = sh('docker', ['image', 'inspect', '--format', '{{.Id}}', ALPINE_IMAGE]);
if (r.code !== 0) throw new HarnessError('prerequisite', `test image ${ALPINE_IMAGE} is not present locally (docker pull ${ALPINE_IMAGE}): ${r.err.trim()}`);
}
export function requireCurl(): void {
requireTool('curl', ['--version'], 'real curl client for the HTTP tests');
}
/** Unprivileged user+mount namespaces with a bind mount over /var/run/docker.sock (fake/inaccessible Docker sockets). */
export function requireUnshare(): void {
if (!fs.existsSync('/var/run/docker.sock')) throw new HarnessError('prerequisite', '/var/run/docker.sock must exist (bind-mount target for the fake Docker socket)');
const r = sh('unshare', ['-Urm', 'sh', '-c', 'mount --bind /etc/hostname /var/run/docker.sock && test -f /var/run/docker.sock && mount -t tmpfs none /run && test ! -e /var/run/docker.sock']);
if (r.code !== 0) throw new HarnessError('prerequisite', `unprivileged user/mount namespaces are required (unshare -Urm + bind mount + tmpfs): ${r.err.trim()}`);
}
+225
View File
@@ -0,0 +1,225 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// Copyright (C) 2026 Bryan Joshua Pedini
/**
* Self-checks of the harness guarantees the other suites rely on: infrastructure failures are
* HarnessErrors that can never satisfy an expected-failure assertion; parsers and redaction work.
*/
import {
test,
expect,
EMPTY_AUDIT,
EMPTY_TOKEN_LIST,
expectCliFailure,
expectContainersResponse,
formatAuditOutput,
formatTokenList,
padTable,
parseAuditOutput,
parseTokenList,
redactValue,
safeStringify,
goQuote,
goUnquote,
HarnessError,
isHarnessError,
parseLogLine,
redactText,
Rng,
runProcess,
toolEnv,
tokenId,
printsTokens,
USAGE_TEXT,
type ProcResult,
} from '../../harness';
test('timeouts, signals and spawn failures are HarnessErrors, never results', async ({ cli }) => {
const timeout = await runProcess('sleep', ['5'], { env: toolEnv(), timeoutMs: 200 }).catch((e) => e);
expect(isHarnessError(timeout) && timeout.kind).toBe('timeout');
const signal = await runProcess('sh', ['-c', 'kill -SEGV $$'], { env: toolEnv() }).catch((e) => e);
expect(isHarnessError(signal) && signal.kind).toBe('signal');
const spawn = await runProcess('/nonexistent/binary', [], { env: toolEnv() }).catch((e) => e);
expect(isHarnessError(spawn) && spawn.kind).toBe('spawn');
// A run that does not finish is killed and reported as a timeout, never as a failure exit
const reallyHung = await cli.shell('sleep 5', { timeoutMs: 200 }).catch((e) => e);
expect(isHarnessError(reallyHung) && reallyHung.kind).toBe('timeout');
// expectCliFailure refuses abnormal terminations even if the "exit code" would match
const fake: ProcResult = { command: 'x', argv: [], stdout: '', stderr: 'dpsapi: x\n', stdoutBuffer: Buffer.alloc(0), stderrBuffer: Buffer.alloc(0), exitCode: null, signal: 'SIGKILL', timedOut: false, durationMs: 1 };
expect(() => expectCliFailure(fake, { exitCode: 1 })).toThrow(HarnessError);
expect(() => expectCliFailure({ ...fake, signal: null, exitCode: 1, timedOut: true }, { exitCode: 1 })).toThrow(HarnessError);
});
test('Go quoting round-trips and matches strconv.Quote conventions', () => {
const cases: Array<[string | Buffer, string]> = [
['', '""'],
['plain', '"plain"'],
['a"b\\c', '"a\\"b\\\\c"'],
['\x07\b\f\n\r\t\v', '"\\a\\b\\f\\n\\r\\t\\v"'],
['\x00\x01\x1f\x7f', '"\\x00\\x01\\x1f\\x7f"'],
['é日🚀', '"é日🚀"'],
[' ​
\u0085', '"\\u00a0\\u200b\\u2028\\u0085"'],
['\u{E0001}', '"\\U000e0001"'],
[Buffer.from([0x61, 0xff, 0xc0, 0x80]), '"a\\xff\\xc0\\x80"'],
];
for (const [input, quoted] of cases) {
expect(goQuote(input)).toBe(quoted);
if (typeof input === 'string') expect(goUnquote(quoted)).toBe(input);
}
});
test('slog text parser handles quoting, escapes and non-record lines', () => {
const r = parseLogLine('time=2026-10-05T20:03:15.908Z level=INFO msg="this request has been made" method=GET path="/a b\\"c" status=200 empty="" eq="a=b"')!;
expect(r.level).toBe('INFO');
expect(r.msg).toBe('this request has been made');
expect(r.attrs).toEqual({ method: 'GET', path: '/a b"c', status: '200', empty: '', eq: 'a=b' });
expect(parseLogLine('dpsapi: token not found')).toBeUndefined();
expect(parseLogLine('usage:')).toBeUndefined();
});
test('redaction replaces registered secrets anywhere and token-shaped strings', ({ secrets }) => {
const tok = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQ';
expect(tok).toHaveLength(43);
secrets.register(tok);
expect(secrets.redact(`x${tok}y`)).toBe(`x[REDACTED:${tokenId(tok)}]y`);
expect(redactText(`Bearer ${'Z'.repeat(43)}`, [])).toBe(`Bearer [REDACTED:${tokenId('Z'.repeat(43))}]`);
expect(secrets.findIn(`... ${tok} ...`)).toHaveLength(1);
expect(secrets.findIn('nothing')).toEqual([]);
});
test('seeded PRNG is deterministic per seed and independent per fork', ({ rng }) => {
const a = new Rng('s');
const b = new Rng('s');
expect(Array.from({ length: 5 }, () => a.nextUint32())).toEqual(Array.from({ length: 5 }, () => b.nextUint32()));
expect(new Rng('s').fork('x').next()).not.toBe(new Rng('s').fork('y').next());
expect(rng.seed).toContain('seeded PRNG is deterministic');
expect(test.info().annotations.find((x) => x.type === 'seed')).toBeTruthy();
});
test('padTable reproduces the tabwriter layout of the spec examples byte for byte; the strict parsers round-trip and reject deviations', () => {
const t1 = 'whEsbopp2tB2xXN7IC3LBnHhVge2F_-Gf910hIDL2pQ';
const t2 = 'S_muyryYKV3fegqvOSlEbHJ8ZvHz_s5tPov3dkY49Bo';
const list =
'TOKEN DESCRIPTION ACTIVE CREATED_AT DELETED_AT\n' +
`${t1} "smoke test" false 2026-10-05T20:03:15.894338161Z 2026-10-05T20:03:15.94012Z\n` +
`${t2} "" true 2026-10-05T20:05:01.042950671Z -\n`;
const entries = [
{ token: t1, description: 'smoke test', active: false, createdAt: '2026-10-05T20:03:15.894338161Z', deletedAt: '2026-10-05T20:03:15.94012Z' },
{ token: t2, description: '', active: true, createdAt: '2026-10-05T20:05:01.042950671Z', deletedAt: null },
];
expect(formatTokenList(entries)).toBe(list);
expect(parseTokenList(list).map((r) => [r.token, r.description, r.active, r.createdAt, r.deletedAt])).toEqual(entries.map((e) => [e.token, e.description, e.active, e.createdAt, e.deletedAt]));
const audit = 'TIME TOKEN ORIGIN OPERATION\n' + `2026-10-05T20:03:16.001Z ${t1} 127.0.0.1 GET containers\n`;
expect(formatAuditOutput([{ time: '2026-10-05T20:03:16.001Z', token: t1, origin: '127.0.0.1', operation: 'GET containers' }])).toBe(audit);
expect(parseAuditOutput(audit).map((r) => [r.time, r.token, r.origin, r.operation])).toEqual([['2026-10-05T20:03:16.001Z', t1, '127.0.0.1', 'GET containers']]);
expect(EMPTY_TOKEN_LIST).toBe('TOKEN DESCRIPTION ACTIVE CREATED_AT DELETED_AT\n');
expect(EMPTY_AUDIT).toBe('TIME TOKEN ORIGIN OPERATION\n');
expect(parseTokenList(EMPTY_TOKEN_LIST)).toEqual([]);
expect(parseAuditOutput(EMPTY_AUDIT)).toEqual([]);
// widths count runes (code points), not UTF-16 units, bytes or display width; the last cell is never padded
// (an empty last cell leaves the padding of the cell before it, as tabwriter does; the app never prints one)
expect(padTable([['a', '🚀🚀', 'x y'], ['日本語', 'b', '']])).toBe('a 🚀🚀 x y\n日本語 b \n');
expect(padTable([['e\u0301', 'z'], ['ab', 'z']])).toBe('e\u0301 z\nab z\n');
// descriptions with runs of spaces and escaped quotes are parsed as Go-quoted strings, not split on blanks
const tricky = formatTokenList([{ token: t1, description: 'a "b" c\\', active: true, createdAt: '2026-10-05T20:05:01Z', deletedAt: null }]);
expect(parseTokenList(tricky)[0].description).toBe('a "b" c\\');
// deviations are rejected: tabs (old format), trailing blanks, wrong padding, unterminated quote, wrong header
const bad = [
'TOKEN\tDESCRIPTION\tACTIVE\tCREATED_AT\tDELETED_AT\n',
list.replace('DELETED_AT\n', 'DELETED_AT \n'),
list.replace('"smoke test" false', '"smoke test" false '),
list.replace(`${t2} ""`, `${t2} "`),
list.replace('ACTIVE', 'ACTIV'),
list.slice(0, -1),
list.replace(`${t2} `, `${t2.slice(1)} `),
];
for (const b of bad) expect(() => parseTokenList(b), JSON.stringify(b.slice(0, 60))).toThrow();
expect(() => parseAuditOutput(audit.replace('127.0.0.1 GET', '127.0.0.1 GET '))).toThrow();
expect(() => parseAuditOutput('TIME\tTOKEN\tORIGIN\tOPERATION\n')).toThrow();
});
test('BigInt values (audit selections) serialize in recorded fuzz cases and redacted artifacts', ({ fuzz }) => {
const sel = { kind: 'audit', exitCode: 0, selection: { windowNs: 3_600_000_000_000n } };
expect(() => JSON.stringify(sel)).toThrow(/BigInt/);
expect(safeStringify(sel)).toBe('{"kind":"audit","exitCode":0,"selection":{"windowNs":"3600000000000"}}');
expect(JSON.stringify(redactValue({ cases: [sel] }, []))).toBe('{"cases":[{"kind":"audit","exitCode":0,"selection":{"windowNs":"3600000000000"}}]}');
const c = fuzz.record({ i: 0, expected: sel });
expect(c.expected.selection.windowNs).toBe(3_600_000_000_000n);
expect(JSON.stringify(fuzz.cases)).toContain('"windowNs":"3600000000000"');
});
test('tokens printed by token list all and audit are registered as secrets by the runner (help registers nothing)', async ({ cli, secrets, startServer, http }) => {
// created and deleted behind the harness runners' back (plain runProcess): not registered yet
const outside = async (args: string[]) => {
const r = await runProcess(cli.binary, args, { env: cli.env() });
expect(r.exitCode, args.slice(0, 2).join(' ')).toBe(0);
return r.stdout.trim();
};
const listed = await outside(['token', 'new', 'only in list all']);
const audited = await outside(['token', 'new', 'only in audit']);
expect(secrets.has(listed) || secrets.has(audited)).toBe(false);
const server = await startServer();
expectContainersResponse(await http.get(server.url(), { token: audited }));
expect((await server.stop()).exitCode).toBe(0);
await outside(['token', 'delete', listed]);
await outside(['token', 'delete', audited]);
await cli.help();
await cli.tokenList(); // inactive tokens are not printed by plain token list
expect(secrets.has(listed) || secrets.has(audited), 'nothing printed, nothing registered').toBe(false);
await cli.tokenListAll();
expect(secrets.has(listed) && secrets.has(audited), 'token list all registers every printed token').toBe(true);
const third = await outside(['token', 'new', 'audit only']);
const s2 = await startServer();
expectContainersResponse(await http.get(s2.url(), { token: third }));
expect((await s2.stop()).exitCode).toBe(0);
await outside(['token', 'delete', third]);
expect(secrets.has(third)).toBe(false);
expect((await cli.audit()).map((r) => r.token)).toEqual([audited, third]);
expect(secrets.has(third), 'audit registers the printed tokens').toBe(true);
// `audit all` (via the runner's audit('all') and via plain run()) registers every printed token
const fourth = await outside(['token', 'new', 'audit all only']);
const s3 = await startServer();
expectContainersResponse(await http.get(s3.url(), { token: fourth }));
expect((await s3.stop()).exitCode).toBe(0);
await outside(['token', 'delete', fourth]);
expect(secrets.has(fourth)).toBe(false);
expect(printsTokens(['audit', 'all'])).toBe(true);
expect((await cli.audit('all')).map((r) => r.token)).toEqual([audited, third, fourth]);
expect(secrets.has(fourth), 'audit all registers the printed tokens').toBe(true);
const fifth = await outside(['token', 'new', 'audit all via run']);
const s4 = await startServer();
expectContainersResponse(await http.get(s4.url(), { token: fifth }));
expect((await s4.stop()).exitCode).toBe(0);
expect(secrets.has(fifth)).toBe(false);
expect((await cli.run(['audit', 'all'])).exitCode).toBe(0);
expect(secrets.has(fifth), 'run([audit, all]) registers the printed tokens').toBe(true);
});
test('usage text: one line per command form, descriptions aligned at offset 34, `audit all` last', () => {
const lines = USAGE_TEXT.split('\n');
expect(lines.shift()).toBe('usage:');
expect(lines.pop(), 'ends with a newline').toBe('');
expect(lines.map((l) => l.slice(0, 34).trimEnd())).toEqual([
' dpsapi',
' dpsapi help',
' dpsapi token list [all]',
' dpsapi token new [description]',
' dpsapi token delete <token>',
' dpsapi audit',
' dpsapi audit <n>',
' dpsapi audit <n><unit>',
' dpsapi audit all',
]);
for (const l of lines) expect(l[34] !== ' ' && l[33] === ' ', `description starts at offset 34: ${JSON.stringify(l)}`).toBe(true);
expect(lines.at(-1)).toBe(' dpsapi audit all' + ' '.repeat(16) + 'print every audit row');
});
+20
View File
@@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ES2023",
"lib": ["ES2023"],
"module": "preserve",
"moduleResolution": "bundler",
"types": ["node"],
"strict": true,
"noEmit": true,
"noUnusedLocals": true,
"noFallthroughCasesInSwitch": true,
"noImplicitOverride": true,
"skipLibCheck": true,
"esModuleInterop": true,
"isolatedModules": true,
"verbatimModuleSyntax": false,
"forceConsistentCasingInFileNames": true
},
"include": ["playwright.config.ts", "harness/**/*.ts", "setup/**/*.ts", "tests/**/*.ts"]
}