Browser Automation Test Data and State Hygiene

Browser tests become difficult to trust when a fixture owns data only by convention. A context can be clean while a shared account, storage snapshot, download directory, or server record remains mutable. State hygiene starts by naming every input, owner, observable result, and cleanup boundary before the test runs.

Use synthetic data with a declared owner

Create records through a documented test API or a small deterministic fixture. Give each record a scenario label rather than a person's name or email. Keep the fixture read-only and copy it into a worker-owned path when a test must modify it. Never use a real customer export, personal document, or credential as test input. See the fixture and isolation guide for resource scope.

The browser owns cookies, local storage, IndexedDB, pages, and contexts created by the runner. The application owns server sessions, queued jobs, and records it persists. CI owns artifact permissions and retention. Closing a context therefore proves only that browser-managed state was released; it does not prove that a server row or background job disappeared.

Test-data and state ownership matrix

Resource	Created by	Mutable owner	Observable result	Cleanup boundary
Synthetic account label	test data service	scenario worker	account heading or access result	service reset API
Browser context and cookies	fixture	fixture	clean-context assertion	context close in finally
Storage-state baseline	test repository	fixture copy	state-loaded route	retain baseline; delete copy
Download or trace directory	worker	worker	short receipt and file property	worker removes local path
Queued server job	application	application/service	documented status	service cancellation or expiry

The matrix prevents a common category error: local cleanup is not remote deletion. Record the owner and a short receipt, not cookies, tokens, or full page content.

Choose state scope deliberately

Use a new context when a scenario depends on cookies, permissions, service workers, or origin storage. A worker may reuse a browser process for cost, but it should create a private context and artifact directory for each attempt. Do not share one storageState output between parallel workers. Load an immutable baseline, write a worker-and-attempt-named copy, and mark partial output invalid.

Keep application mutations independent as well. Two contexts can still race on the same server record. Prefer separate synthetic records or a documented reset operation. If a shared mutation is unavoidable, serialize that one operation and identify its owner; an arbitrary delay does not establish ownership.

Failure fixture and cleanup receipt

Exercise the fixture with a synthetic page that withholds a readiness signal. The expected failure is bounded, names the missing signal, and still reports cleanup:

import os from 'node:os';
import path from 'node:path';
import { mkdtemp, rm } from 'node:fs/promises';
import { chromium } from 'playwright';
import { expect } from '@playwright/test';

const attemptDir = await mkdtemp(path.join(os.tmpdir(), 'state-hygiene-'));
const browser = await chromium.launch({ headless: true });
const context = await browser.newContext({ baseURL: 'http://fixture.test' });
const page = await context.newPage();
await page.route('/fixtures/without-ready-marker', route =>
  route.fulfill({ status: 200, contentType: 'text/html', body: '<main><p>Waiting</p></main>' })
);
let failure;
const remember = (label, error) => {
  const current = new Error(`${label}: ${error.message}`, { cause: error });
  failure = failure ? new AggregateError([failure, current], `${failure.message}; ${label} failed`) : current;
};
try {
  await page.goto('/fixtures/without-ready-marker');
  await expect(page.getByRole('status')).toHaveText('Ready', { timeout: 250 });
} catch (error) {
  remember('fixture assertion', error);
} finally {
  try {
    await context.close();
  } catch (error) {
    remember('context close', error);
  }
  try {
    await rm(attemptDir, { recursive: true, force: true });
  } catch (error) {
    remember('artifact cleanup', error);
  }
  try {
    await browser.close();
  } catch (error) {
    remember('browser close', error);
  }
}
if (failure) throw failure;

Preserve the first assertion or timeout error when cleanup also fails. A retry is a new context and a new attempt, not permission to reuse an unknown page. Retry only a read-only action or an operation with a documented status/idempotency check.

Decision table for state failures

Observation	Classification	Next action	Not proven
Baseline missing before launch	fixture	fail and repair the owned input	service availability
New context shows old cookie	browser state	discard context and inspect state loading	server session invalidation
Two workers overwrite a record	test data	allocate records or serialize mutation	browser isolation failure
Visible result succeeds, cleanup fails	infrastructure	retain both errors and quarantine artifacts	remote deletion
Retry passes after timeout	uncertain outcome	query supported status before retrying	first attempt was harmless

Keep evidence proportional

Capture scenario, browser/framework versions, worker label, URL category, visible assertion, and cleanup status. A screenshot or trace belongs in its own restricted path with a short retention period. A filename that contains the right worker label is not evidence that its bytes came from that worker.

Review state transitions.

Treat browser state as a sequence of transitions, not as a bag of values. At setup, record the baseline source and the context owner. During the journey, record only the visible checkpoint needed for the assertion. At teardown, record whether the context, page, temporary files, and application-owned resources reached their expected terminal state. This makes a failure understandable without retaining a replay of the entire session.

Account switching deserves a separate transition. Sign out through the application path, wait for its documented visible result, then create the next context from a different synthetic state. Clearing one origin's storage is not a substitute for ending a server session. A second tab, offline queue, or service worker can continue to show an old view. Assert the account heading or access result in the new context instead of asserting that a generic sign-in button exists.

State files also need version discipline. Record the browser and framework versions beside the state source, and reject a snapshot whose schema or creation attempt is unknown. Do not silently regenerate a missing state file during a test: that changes the input while hiding a fixture defect. A repair job may create a new baseline through the approved service, but it should produce a new receipt and owner review.

Protect secrets and retained artifacts.

Storage snapshots often contain cookies or origin data that can authenticate a synthetic account. Restrict their filesystem permissions, keep them outside ordinary logs, and expire worker copies after the run. A screenshot can expose page text, an accessibility tree can expose names, and a trace can include request metadata. Capture these only for an authorized synthetic page and publish the smallest artifact that answers the diagnostic question.

Use a short receipt as the normal test output. It can contain scenario label, attempt, browser release, result category, visible landmark, artifact path category, and cleanup status. It should not contain an authorization header, cookie value, full response body, personal document, or account discovery result. If a service owner needs a request identifier, store the identifier under the service's access policy rather than copying its content into CI output.

Retention is an ownership decision. The worker can remove its temporary directory, but CI may retain a copied trace under a separate policy. The application owner decides how long a synthetic server record remains. Document those separate clocks so a local finally block is not mistaken for complete data erasure. The download and upload hygiene guide provides a related artifact boundary.

Review retries and parallel workers.

Parallel execution changes the meaning of a filename and a record identifier. Include scenario, worker, and attempt in names, but do not include email addresses or customer identifiers. Fail when a target already exists if replacement would hide a collision. Atomic rename protects readers from a partial file, but it does not protect two workers that believe they own the same record. Ownership must be assigned before the first mutation.

When a timeout occurs, classify the action before retrying. A read-only navigation can usually start in a new context. A submission, upload, or cancellation may have reached the service even if the browser did not observe the response. Query a documented status endpoint or ask the service owner for evidence before repeating it. A later successful attempt does not erase the uncertainty of the first attempt.

Run a small matrix after browser or application changes: clean context, expired state, account switch, parallel artifact output, a server-side rejection, and forced teardown failure. Compare visible outcomes and cleanup receipts, not screenshots alone. Keep the same synthetic inputs where comparison is intended, and change one declared condition at a time. This narrows regressions without expanding the collection of data.

Make setup repeatable without hiding defects.

Repeatability comes from controlled inputs, not from a shared long-lived browser. Pin the route category, fixture schema, synthetic record shape, and browser configuration that the scenario actually needs. A test can then explain why its result changed: the application changed a visible behavior, a fixture changed an input, or the runner failed to establish the declared state. Avoid a broad suite setup that creates accounts, grants permissions, and leaves pages open for later tests. That setup may look convenient while making the first failing test impossible to attribute.

Keep setup work observable. A login fixture should report that a known synthetic state was loaded or that a documented sign-in completed; it should not print the session material used to get there. A data fixture should report the scenario label and reset result; it should not enumerate records in an environment to find one that happens to work. When the expected test input is absent, stop with a fixture failure. Creating an unplanned substitute can turn a useful failure into a result that cannot be reproduced.

Some state is intentionally external to the browser. For example, an application may use an email inbox simulator, a payment sandbox, or an asynchronous notification service. Give that dependency its own synthetic namespace and receipt. The browser test may observe a confirmation page, but it should not claim that every downstream delivery occurred. Use the service's documented test interface for facts that live there, and leave that interface's cleanup to its owner.

Separate client cleanup from service cleanup.

Closing a page releases page listeners and page-scoped handles. Closing a context releases the browser-managed state within that context. Neither action automatically removes a file that a test copied into a report directory, deletes an object uploaded to an application, or revokes a token held by another device. Place these operations in the order required by their owners: perform a documented application sign-out while the page is available, close the browser resources, then remove the local worker directory.

An application cleanup step can fail for a reason unrelated to the browser close. Retain both facts in the receipt. For example, a visible assertion can fail because a control never becomes ready, while context close can fail because the runner disconnects. Reporting only the later close loses the scenario evidence. Reporting only the assertion loses the fact that a resource might remain. A bounded aggregate error is sufficient; it need not include a page dump.

Do not broaden cleanup to compensate for uncertainty. A fixture must not scan for similarly named profiles, delete arbitrary directories, or close any context that happens to be open. Those actions can harm another worker and make ownership less clear. A recovery process can quarantine its own known attempt directory for later inspection when removal fails, provided that process uses a documented retention policy.

Design test data for recovery.

A synthetic record should be recognizable without looking like real user data. Use a scenario prefix, a generated run label, and values designed for the assertion. Keep the values small enough that screenshots and receipts remain low sensitivity. When a scenario requires a boundary value, alter one declared property at a time: for example, an expired synthetic state instead of a valid state, or a rejected fixture response instead of a successful one. That design lets a maintainer link the visible outcome to the input change.

Recovery should restore a known baseline rather than guess how to repair a partially used record. A service reset endpoint, a disposable namespace, or a record created per attempt can provide that baseline. The correct choice depends on the service contract, but the browser fixture should state which choice it requested and what it observed. It should not claim a universal reset guarantee, especially when an asynchronous task can outlive the UI journey.

When a cleanup receipt says incomplete, keep the scenario label available to the owner who can resolve it. Do not reuse that label automatically in the next attempt. A fresh label prevents a later test from treating residue as its own input. This also improves incident triage: a service owner can inspect one bounded synthetic namespace instead of a broad collection of unrelated records.

Use assertions that match the boundary.

An assertion should describe what a user or an authorized test can see at the browser boundary. A protected route may show an access result. An account switch may show the expected synthetic account heading. A data reset may show a documented empty state. These are useful results because they do not require extracting browser internals or collecting unrelated account details.

Do not infer a remote fact from a convenient local signal. An empty local-storage entry does not establish that a server session ended. A completed navigation does not establish that an asynchronous job ran. A file appearing in a worker directory does not establish that an application accepted its content. Pair browser assertions with service-owned checks only when the scenario requires that remote fact and the service provides an approved way to observe it.

The same boundary applies to diagnostics. Record the missing readiness signal, unexpected visible status, or documented error category. Keep the diagnostic stable across locale, browser release, and minor presentation changes where possible. Exact unowned error strings create fragile tests and may expose information that is not needed to decide the scenario.

For routine maintenance, review a failed receipt before changing test timing. Confirm that the scenario had a unique record, a unique context, and a private artifact directory. Then identify the first missing visible condition and the owner responsible for it. This order prevents an infrastructure symptom from becoming a permanent sleep or an unnecessary retry. It also gives the application owner a concise, reproducible request when the browser observation ends at the service boundary.

Document the expected retention outcome for each synthetic namespace. A temporary directory may disappear immediately, a trace may remain available for a short CI window, and a server record may expire later through a service policy. These distinct outcomes are normal when their owners are explicit. The test result should say which cleanup completed and which outcome requires a different owner, without implying that browser automation can control every system involved.

For routine maintenance, compare a failed receipt with the declared contract before changing timeouts. Verify the unique record, context, and artifact path first; then route the first missing visible condition to its actual owner. This prevents transient infrastructure symptoms from becoming permanent delays and keeps service questions reproducible.

BotBrowser capability and limitation

BotBrowser can provide isolated BrowserContexts with separate cookies, storage, and session state for authorized synthetic workflows. That capability helps verify that a fresh context does not inherit client-side state. See the BotBrowser multi-account isolation documentation.

BotBrowser does not replace Playwright or Selenium lifecycle management, application cleanup, server-session invalidation, secret handling, or a service's record-retention policy. It cannot guarantee that an expired cookie is accepted, cancel a queued job, or delete a provider record. Treat isolation as an input to the test and state the unobserved remote boundary in the receipt.

Before merging a fixture change, run a clean-context case, an expired-state case, and this forced readiness failure. Compare owner, worker path, visible result, and cleanup status across all three.

Sources

• Playwright: Best practices
• OWASP Web Security Testing Guide
• BotBrowser: Multi-account isolation