Console visual QA with Playwright

How to run automated visual QA of the web console (screenshots + console-error checks) using headless Chromium via Playwright. Written so an agent (or a human) can reproduce the whole loop from a clean checkout.

Prerequisites

• Node.js and npm (already required by the console build).

• Playwright Chromium, installed once:

  npx playwright install chromium

  The playwright npm package itself is resolved by npx on first use; the module lands under ~/.npm/_npx/<hash>/node_modules/playwright-core. Find the exact path with:

  find ~/.npm/_npx -path "*node_modules/playwright-core/index.js" | head -1

  (Alternatively npm i -D playwright-core inside console/ and import it from there.)

1. Build the console and start a server

# Build the static assets and stage them where the jar build picks it up
cd console && npm run build && cd ..
rm -rf lib/src/main/resources/console
cp -r console/dist lib/src/main/resources/console

# Minimal single-disk config
cat > /tmp/qa-config.yaml <<EOF
server:
  port: 8000
  baseUrl: "http://localhost:8000"
auth:
  accessKey: "rootroot"
  secretKey: "rootroot"
  region: "us-east-1"
storage:
  mode: simple
  path: /tmp/qa-data
EOF

./gradlew :tiny-s3-app:runTestServerSingleDisk \
  --args="--config /tmp/qa-config.yaml" &   # console at http://localhost:8000/console/

# Wait until ready
until [ "$(curl -s -o /dev/null -w '%{http_code}' http://localhost:8000/console/)" = "200" ]; do sleep 2; done

Note: runTestServerSingleDisk runs from compiled classes, so the console assets must be staged into lib/src/main/resources/console (as above) before Gradle compiles resources; if the server was already running, restart it to pick up new assets.

2. Generate traffic (so Dashboard/Metrics have data)

export AWS_ACCESS_KEY_ID=rootroot AWS_SECRET_ACCESS_KEY=rootroot AWS_DEFAULT_REGION=us-east-1
aws s3 mb s3://qa-bucket --endpoint-url http://localhost:8000
head -c 524288 /dev/urandom > /tmp/blob.bin
for i in 1 2 3 4 5; do aws s3 cp /tmp/blob.bin s3://qa-bucket/blob$i.bin --endpoint-url http://localhost:8000; done
for i in 1 2 3 4 5; do aws s3 cp s3://qa-bucket/blob$i.bin /tmp/got.bin --endpoint-url http://localhost:8000; done
# Failed logins for the auth-failure metrics
for i in 1 2 3; do curl -s -o /dev/null -H "Authorization: AWS4-HMAC-SHA256 Credential=bad/x" http://localhost:8000/qa-bucket; done

3. The QA script

Key trick: the console keeps credentials in sessionStorage under the key tinys3.console.creds, so seed them directly instead of driving the login form:

// qa.mjs — run with: node qa.mjs
import pkg from "<path-to>/node_modules/playwright-core/index.js"; // CommonJS: default import
const { chromium } = pkg;

const BASE = "http://localhost:8000/console";
const creds = {
  accessKeyId: "rootroot",
  secretAccessKey: "rootroot",
  region: "us-east-1",
  endpoint: "http://localhost:8000",
};
const pages = [
  ["dashboard", "/"],
  ["browser", "/browser/"],
  ["transfer", "/transfer/"],
  ["metrics", "/metrics/"],
  ["access", "/access/"],
  ["settings", "/settings/"],
];

const errors = [];
const browser = await chromium.launch();
const page = await (await browser.newContext({ viewport: { width: 1440, height: 900 } })).newPage();
page.on("console", (m) => m.type() === "error" && errors.push(m.text()));
page.on("pageerror", (e) => errors.push("pageerror: " + e.message));

await page.goto(`${BASE}/`, { waitUntil: "domcontentloaded" });
await page.evaluate((c) => sessionStorage.setItem("tinys3.console.creds", JSON.stringify(c)), creds);

for (const [name, path] of pages) {
  await page.goto(`${BASE}${path}`, { waitUntil: "networkidle" });
  await page.waitForTimeout(1200); // let charts/data settle
  await page.screenshot({ path: `qa-${name}.png`, fullPage: true });
}

await browser.close();
console.log("errors:", errors.length);
errors.forEach((e) => console.log("  " + e));

Interaction checks follow the same pattern — e.g. open a bucket and the object-details modal:

await page.goto(`${BASE}/browser/`, { waitUntil: "networkidle" });
await page.getByText("qa-bucket", { exact: true }).first().click();
await page.getByRole("button", { name: "blob1.bin" }).first().click(); // opens Object details modal
await page.setInputFiles("input[type=file]", "/tmp/blob.bin"); // triggers the upload dock

4. What to check

• Zero console/page errors on every route (the errors array must be empty; a few Failed to load resource: 404 entries can come from optional REST endpoints — investigate, they are usually a missing backend feature in the mode the server runs in).
• Screenshots: layout intact at 1440×900, charts populated, no unstyled flashes.
• Metrics page needs traffic first (step 2) or all charts are flat.

Pitfalls

• playwright-core is CommonJS — use import pkg from "..."; const { chromium } = pkg; (a named import { chromium } fails).
• The login page is at /console/ when no creds are in sessionStorage; seeding creds and navigating again skips it.
• Client-side routes are static-exported with trailing slashes (/browser/, not /browser).
• Kill the server with pkill -f "dev.totis.tinys3.Application" when done.

QA against the dev server (npm run dev)

The same scripts work against the standalone dev console — start the backend with CORS and the dev server (see console/README.md), then point BASE at the dev origin. Note the app keeps its /console basePath in dev:

const BASE = "http://localhost:8001/console";
const creds = { ...,  endpoint: "http://localhost:8000" }; // cross-origin: creds must name the API

Expect 404 console errors for /rest/api/v1/erasure/* when the backend runs in simple (single-disk) mode — the UI degrades gracefully; they are not a bug.