Web Platform Tests for Browser Feature Confidence

Web Platform Tests (WPT) are valuable conformance evidence, but a passing test is not a promise that every application journey will succeed. The WPT repository tests web-platform behavior under declared browser, origin, harness, and timing conditions. A product still needs runtime feature detection, permission handling, accessibility checks, and an application-level assertion. Treat confidence as a chain of bounded observations: a specification defines the behavior, a WPT test exercises a browser implementation, a local feature check decides whether the current page can try it, and an owned application test verifies what the user can see. This separation helps teams use standards evidence without turning it into a browser identity claim or a universal compatibility guarantee. It also keeps failures actionable: a conformance failure belongs to the browser matrix, while a missing permission message belongs to the page and a missing server acknowledgement belongs to the application.

Choose the WPT evidence

Start with the feature specification and the WPT repository entry for the behavior you need. The WPT repository contains test files, metadata, expected results, and harness conventions; the testharness API guide explains assertions, asynchronous completion, and cleanup. Select a test that matches the operation, document mode, origin, secure-context requirement, and permission state of your product. A nearby test for the same interface may exercise a different algorithm or a different global object and should not be treated as interchangeable. Record the test path, specification section, browser release, harness version, and result collection date. This makes a later comparison reproducible when the WPT tree or browser changes.

Read metadata and expected failures with the test itself. A test marked as tentative, flaky, or expected to fail is a signal to investigate, not a green result to hide. A test that passes in a local runner may still be skipped by the product matrix because the origin is insecure, the API is disabled by policy, or the test requires a user gesture. Keep expected failures visible and explain whether the product fallback is intentional. Do not copy a WPT test into production without checking its license, fixture assumptions, network dependencies, and cleanup behavior. Standards tests are shared evidence; application tests own product wording, data boundaries, and business completion.

Runtime capability checks

Use a local capability check immediately before the feature call. Test the method or constructor that the operation actually needs, then handle a thrown exception or rejected promise. A present property only means that the page can attempt the operation; it does not prove a secure context, permission grant, user gesture, service reachability, or successful application commit. Normalize the result into states such as missing, available, denied, failed, and completed. Keep the state vocabulary stable across browsers so dashboards and support reports compare behavior rather than browser labels. Avoid user-agent parsing and never collect unrelated browser properties to make a conformance test appear to pass.

The check should be cheap, side-effect free, and independently testable. For Clipboard, distinguish a missing object from a rejected write and from a visible copied status. For WebAuthn, distinguish an unsupported authenticator from a cancelled ceremony and from a server-side assertion failure. For File System Access, distinguish an unavailable picker from a user cancellation and from a failed write. For every state, define a keyboard-operable alternative and preserve entered data. A confidence report should state the declared origin, secure-context status, permission expectation, browser release, detected state, fallback, and visible outcome. It should not infer identity, device class, or intent from the capability result.

Confidence decision table

Observation	Decision	User-visible result	Evidence boundary
Matching WPT test passes in the declared matrix	include the browser in the candidate set	try the feature after runtime detection	conformance under declared conditions
WPT test is skipped, tentative, or expected to fail	investigate metadata and retain a fallback	explain that the optional path is unavailable	test coverage and risk
Runtime method is missing or context is blocked	use the alternative	keep the task available	current page capability
Runtime call rejects or permission is denied	classify once and stop hidden retries	show a recoverable message and alternative	browser runtime behavior
Browser call resolves	verify application acknowledgement	confirm only the visible result	browser plus owned application signal

Failure fixture

Use a synthetic page that creates its own status and fallback controls, stubs only the owned API surface, and removes the fixture nodes in every branch. Exercise a missing method and a rejected promise separately. The assertion should check the visible text and the fallback control, not an internal browser property. A deterministic fixture must not alter a fingerprint, send credentials, or claim that a production service is unavailable. Its purpose is to prove that the application keeps a useful path when a capability confidence signal is negative. Keep the URL, title, and data synthetic; preserve the first failure; and make cleanup run even when the assertion throws. The same fixture can run in a browser context used for a WPT-inspired smoke check, but the result remains application evidence rather than a replacement for WPT conformance.

async function runFeatureFixture() {
  const host = document.createElement('div');
  host.innerHTML = '<p id="status"></p><button id="fallback" hidden>Use the alternative</button>';
  document.body.append(host);
  const status = host.querySelector('#status');
  const fallback = host.querySelector('#fallback');
  try {
    const supported = typeof navigator.share === 'function';
    const state = supported
      ? await navigator
          .share({ title: 'Synthetic item', url: '/fixture' })
          .then(() => 'completed')
          .catch(() => 'failed')
      : 'missing';
    if (state === 'completed') status.textContent = 'Completed';
    else {
      fallback.hidden = false;
      status.textContent = 'Use the alternative';
    }
    console.assert(status.textContent.length > 0);
    console.assert(state === 'completed' || !fallback.hidden);
    return { state, visible: status.textContent };
  } finally {
    host.remove();
  }
}

Application verification

After the browser call, verify the signal that belongs to the application: a status announcement, a returned record identifier, an updated control, or another explicitly owned boundary. A resolved promise can mean that the browser accepted a request while the server is still processing it. A screenshot can prove that a message appeared while saying nothing about durable storage. Keep browser and application observations separate in logs and test reports. Assert that the first error is preserved, that a timeout is not mislabeled as an unsupported feature, and that the alternative remains usable. Accessibility is part of confidence: labels, focus, status roles, keyboard operation, and error recovery need their own assertions.

Confidence changes over time. Re-run the declared WPT subset when a browser release, specification, policy, secure-context requirement, or harness changes. Keep the test revision, browser build, origin, permissions, and expected-result metadata with the run. Remove a browser from a candidate set only after an owner reviews the evidence; do not silently turn a flaky test into a pass. A useful ledger records the WPT path, local feature state, fallback chosen, application outcome, and next review date. It should retain no profile, credential, location, or personal content. This gives release and support teams a compact explanation of what was tested and what was not.

BotBrowser capability and limitation

BotBrowser can provide controlled browser contexts for authorized WPT-inspired smoke journeys and repeatable feature-confidence checks. An isolated context lets a team run an owned synthetic page against a declared browser release set, compare visible states, and keep mutable session storage separate between attempts. That is useful for checking permission branches, secure-context setup, rejected promises, fallback controls, and cleanup. Record the context assumptions and fixture revision so the result can be reproduced. The result is one bounded browser observation and should be reported with the WPT path, runtime state, application assertion, and visible outcome.

BotBrowser does not certify the WPT suite, replace standards conformance infrastructure, grant permissions, make an insecure origin secure, or guarantee that an API exists on every origin. It cannot prove accessibility by itself, control upstream services, or turn a passing browser journey into evidence that a server record or business transaction succeeded. WPT maintainers own shared conformance tests; application teams own product assertions, fallback copy, and data cleanup. Treat BotBrowser isolation as evidence for repeatability, not as a substitute for the specification, WPT metadata, runtime detection, or production monitoring. Do not change a browser profile or collect unrelated signals to make a failed capability appear supported.

For related examples, see the browser API compatibility and fallback guide and the WebDriver BiDi automation guide. They show why standards evidence, browser behavior, and application completion should remain separate claims. Use the localized versions of these links in translated articles so readers can continue to a relevant technical example without leaving their language path.

BotBrowser can run authorized, repeatable browser-context checks for WPT-inspired feature journeys, while it cannot certify WPT conformance, grant permissions, or prove an application transaction. Keep that capability and limitation together in the test record so a passing context run is not mistaken for a universal browser guarantee. A controlled context is useful when a team needs to compare the same synthetic page across a declared browser release set. It can isolate storage, repeat a permission branch, and capture the status that a user would see after a rejected promise or unavailable method. It cannot decide whether the WPT metadata applies to a different origin, cannot make a policy-disabled feature available, and cannot replace a standards runner that owns the conformance result. The report should therefore name the browser build, context assumptions, fixture revision, WPT reference, runtime state, fallback action, and application acknowledgement. When those fields are present, a support engineer can reproduce the branch without inspecting a profile or collecting unrelated browser signals. When they are absent, the right conclusion is only that one journey ran, not that the feature is supported everywhere.

Confidence is strongest when every layer has a named owner and a small assertion. The specification owner identifies the algorithm and normative behavior. The WPT maintainer owns the shared test, metadata, expected result, and harness cleanup. The browser matrix owner chooses releases and investigates tentative or flaky outcomes. The application owner decides what a user-visible completion signal means and writes the fallback copy. The test author owns synthetic data, fixture isolation, timeout policy, and artifact cleanup. BotBrowser can make the context and release assumptions repeatable, but it does not merge those responsibilities or decide whether a business record is committed. A report should therefore show the WPT path and revision, the browser build, the origin and secure-context state, the permission expectation, the local capability result, the fallback branch, and the application assertion. Omitting one of these fields creates a tempting but unsupported conclusion. For example, a passing WPT test for a Web Share algorithm does not mean that a page can open a share sheet when it lacks a user gesture. A runtime method check does not mean that a user will grant permission. A successful browser promise does not mean that a server stored a report. Each statement can be true at its own boundary without proving the next statement.

Use negative evidence deliberately. A skipped WPT case can expose missing coverage; it should not be converted into a green dashboard cell. An expected failure can identify a release risk; it should not be hidden by changing the assertion. A rejected runtime promise can validate the fallback branch; it should not be reported as an upstream outage without an application-owned signal. A timeout can indicate a slow service, an unhandled permission prompt, or a fixture bug; classify it before changing the matrix. This discipline makes a confidence ledger useful during release review. The ledger can compare a previous browser build with a candidate build, show which WPT cases changed, and list the visible application result for each changed state. It can also record that a check was not run because the origin was intentionally HTTP-only or because a permission prompt requires a real user action. Honest “not tested” entries are more valuable than a fabricated pass because they direct the next engineering action without widening claims about users or devices.

When a feature graduates from experimental to ordinary use, keep the fallback until real application evidence supports removing it. Browser support tables evolve, but embedding policies, enterprise permissions, secure-context requirements, and assistive technology behavior can still vary. A small WPT subset plus a deterministic local fixture is easier to maintain than a large collection of user-agent branches. Review the subset after each browser or harness upgrade, and delete only tests whose product path has an explicit owner. Preserve the first failure and clean up all temporary nodes, files, and context state. This gives support staff a reproducible answer: which standard behavior was tested, what the current page detected, what the user saw, and what BotBrowser controlled. It avoids collecting raw profile content or inferring identity from a browser capability. The resulting confidence is bounded, explainable, and useful for choosing the next test or fallback change.

The practical confidence question is whether a person can complete the task under the declared conditions. A passing WPT assertion answers a narrower question about a web-platform algorithm. A runtime check answers whether this page can attempt the operation now. An application assertion answers whether the product displayed the intended result. A permission prompt, an insecure origin, an embedded policy, a disabled browser flag, a network timeout, and a server rejection can each interrupt the chain. Give each interruption a visible message and a useful next action. Keep the main task available when the optional path fails, preserve data already entered, and make the alternative operable by keyboard and assistive technology. This is why conformance evidence and product confidence should be linked but not conflated. The same evidence record can help release engineering compare builds, help support explain a customer report, and help accessibility review a fallback without claiming more than the test observed.

Keep a small confidence ledger rather than a large collection of raw logs. The ledger can link the WPT path, specification revision, browser build, context setup, runtime state, visible assertion, and fallback decision. It should state whether the test ran with a real permission prompt, a synthetic rejection, or a policy that intentionally blocked the feature. It should also state what was not tested: real hardware, upstream availability, account state, server durability, or broad assistive-technology coverage. This explicit boundary lets a reviewer approve the next test without reopening every browser trace.

For a release candidate, compare the candidate and previous browser using the same fixture and the same application endpoint. Preserve the first meaningful failure, report cleanup separately, and avoid converting an uncertain retry into a pass. A WPT confidence check is successful when it produces a reproducible observation and a clear product action, even when that action is to keep the fallback. The goal is dependable decisions, not a claim that every browser and origin behaves identically.

Document the owner and next review date for every exception. This prevents a tentative result from becoming permanent through neglect.

Name the visible fallback in the report so support can reproduce what a user was offered.

Keep this record readable to someone who did not run the browser. State the test boundary, the visible result, and the next action in plain language.

That final explanation is part of confidence because it prevents a narrow conformance result from being reused as an unsupported product promise.

An auditable result also records the test date and the next owner.

That small handoff keeps browser confidence current as releases change.

Sources

• Web Platform Tests repository
• WPT testharness API
• BotBrowser: Multi-account isolation