How to Test WebMCP Tools With Playwright

TL;DR: Playwright 1.64 can list and invoke tools that a website registers through the experimental WebMCP browser API. Treat those tools as a versioned application interface, not a shortcut around testing. Inventory every tool, pin its schema and annotations, check which roles can see it, send invalid inputs, separate read-only from consequential actions, test timeouts and retries, verify side effects exactly once, and confirm the durable result through an independent readback.

Playwright 1.64 added `page.webmcp` and `frame.webmcp`. They let a test list the tools a page registers through WebMCP and call those tools directly. Playwright MCP also offers page-defined tools to an agent with a webmcp_ prefix, while playwright-cli can list and call them from the command line.

That changes the QA surface. A website can now expose an application action as a structured tool as well as a visible interface. The action may be easier for an agent to discover and invoke, but it can still have the wrong schema, appear to the wrong user, execute twice, leave partial state, or report success before the result persists.

This is source-backed analysis, not an executed experiment.

What Playwright 1.64 Actually Added

The experimental Playwright WebMCP API has two methods:

• tools() returns the tools currently registered by a frame.

• callTool(name, input, options) invokes one registered tool and returns the result from its execute function.

Each listed tool can include a unique name, description, JSON input schema, and annotations. The documented annotations can identify a tool as read-only, mark its output as potentially containing untrusted third-party content, or flag it as consequential, such as a tool that places an order.

Playwright warns that the page provides the names, descriptions, schemas, annotations, and results. A test or agent must treat them as untrusted input.

The frame boundary matters too. page.webmcp covers only the main frame. A child frame exposes its own tools through frame.webmcp. A checkout tool registered inside an embedded payment frame will not necessarily appear in the main-frame inventory.

WebMCP is still experimental. Chromium requires the --enable-features=WebMCP launch argument. Firefox uses experimental preferences. Playwright documents no WebKit implementation.

Start With a Tool Inventory Test

Do not begin with a happy-path invocation. First freeze the interface the application intends to expose.

import { test, expect, chromium } from '@playwright/test';

test('WebMCP tool inventory matches the approved contract', async () => {
  const browser = await chromium.launch({
    args: ['--enable-features=WebMCP'],
  });
  const page = await browser.newPage();

  await page.goto('https://example.com/account');

  const tools = await page.webmcp.tools();
  const summary = tools.map(tool => ({
    name: tool.name,
    description: tool.description,
    inputSchema: tool.inputSchema,
    annotations: tool.annotations,
  }));

  expect(summary).toEqual([
    {
      name: 'update_profile',
      description: 'Update the signed-in user profile',
      inputSchema: {
        type: 'object',
        properties: {
          displayName: { type: 'string', minLength: 1 },
        },
        required: ['displayName'],
        additionalProperties: false,
      },
      annotations: {
        readOnly: false,
        consequential: true,
      },
    },
  ]);

  await browser.close();
});

The exact snapshot style is your choice. The important part is that an unexpected rename, missing required field, widened input, changed annotation, or newly exposed tool becomes a reviewed application change.

A passing inventory does not prove that the action works. It proves that the declared interface matches the approved contract for that build.

Use a WebMCP Tool Acceptance Contract

Record one contract for each tool and role.

The annotations are useful hints, not proof. A page can mark a tool as read-only while its implementation still changes state. Test the observed effect.

Test Discovery and Permission Boundaries

A tool should not merely reject unauthorized execution. It should follow the intended discovery policy.

For each relevant role, test:

1. the signed-out state;

2. a user without the feature or permission;

3. the lowest permitted role;

4. an administrative role;

5. an expired or stale session;

6. a child frame with a different origin or permission context, when applicable.

Ask two separate questions:

• Can this role discover the tool?

• Can this role invoke the tool successfully?

Those answers may differ by design. A tool might remain discoverable but return a clear authorization error. Another product may choose not to reveal the tool at all. The policy should be explicit and tested across releases.

Do not rely only on the description. If the tool says “update the current workspace,” verify which workspace identity the application resolves at execution time.

Send Invalid Inputs Before the Happy Path

A JSON Schema is useful only when its boundaries are exercised.

Test at least:

• a missing required field;

• the wrong type;

• an empty or overlong string;

• an unsupported enum value;

• an unexpected additional property;

• a value belonging to another account or workspace;

• text containing markup or third-party content when the result may be untrusted;

• a stale version or conflict condition.

The test should verify more than the returned error. Confirm that no durable state changed.

```ts const before = await readProfileFromApi();

await expect( page.webmcp.callTool('update_profile', { displayName: '', admin: true, }) ).rejects.toThrow();

const after = await readProfileFromApi(); expect(after).toEqual(before);

```

The exact error shape depends on the application. Playwright documents that callTool() returns a result with isError: true as-is, throws when the tool is not registered, and throws when the tool's execute function throws. Your contract should distinguish these paths.

Test Timeouts, Retries, and Side Effects Together

Both tools() and callTool() default to timeout: 0, which means no method-level timeout. Set an explicit { timeout: ... }, or configure Playwright's action or default timeout, so a stuck page execution function does not leave the test waiting without a bound.

The documented Playwright WebMCP contract exposes a timeout, but it does not expose a cancellation signal, transaction, or rollback primitive for the page's tool execution. A Playwright timeout ends the caller's wait. Do not treat that as proof that the application stopped or reversed the underlying work. Cancellation and rollback behavior must be designed and verified at the application or API layer.

Consider a place_order tool:

1. The call starts.

2. Playwright reaches its timeout.

3. The test retries.

4. The first operation completes late.

5. Two orders now exist.

A useful timeout test checks the system of record after the timeout before deciding whether a retry is safe. For consequential tools, require an idempotency key, operation ID, or another durable duplicate guard.

Test three outcomes separately:

• No effect: the timed-out operation never committed.

• Single late effect: the operation committed once after the caller stopped waiting.

• Partial effect: one record changed while a dependent action failed.

Then verify the retry policy for each outcome. “The second call returned success” is not enough.

Compare the Tool Result With a Durable Readback

A WebMCP result is the tool implementation's report. It should not be the only oracle for its own success.

For a profile update, reload the page and query the profile from a read-only API. For an order, inspect the order record and inventory reservation. For a file operation, verify the stored object, checksum, and application record. For a support action, verify the ticket status and message exactly once.

Use this sequence:

```text known start state → tool inventory and role check → call tool → capture returned result → wait for the application to settle → read durable state independently → compare allowed side effects → reset and rerun

```

The independent readback should not reuse the same transient message that the tool produced. If the tool returns “saved” and the test only asserts that response, the implementation is grading itself.

Test Contract Changes Across Application Builds

WebMCP tools are part of the application release surface. Compare the last accepted build with the candidate build.

Review changes to:

• names and descriptions;

• frame ownership;

• required and optional fields;

• enum values and bounds;

• annotations;

• role visibility;

• approval steps;

• error behavior;

• side effects;

• retry and idempotency rules;

• durable outcome checks.

A schema change can be breaking even when the visible interface looks unchanged. An annotation change can alter how an agent treats the action. A new tool can expand the permission surface without adding a new button.

The first review question should be simple: what changed in the tool contract, and was that change intended?

What Changes as Websites Expose More Actions to Agents

As websites expose more first-party actions directly to agents, QA will test the tool contract and permission surface alongside the visual interface; solving agents will consume structured tool definitions and failures before a human reviewer sees the browser path.

That means the failure report should identify the application build, tool name, frame, schema version, role, approval state, exact input, returned result, timeout or retry path, allowed side effects, observed side effects, and independent durable readback.

Inventory each exposed tool, pin its schema and owning application build, define role and approval boundaries, add invalid-input and timeout cases, verify side effects exactly once, and require an independent durable-state readback.

Connect a Failed Tool Call to the Visible Browser State

Samelogic does not expose or execute WebMCP tools. The application and Playwright test remain responsible for the tool contract, invocation, permissions, and outcome check.

When a permitted tester deliberately records a failing browser path, that recording can support review of the visible state around the invocation: the role and page state before the call, the user-visible response, and the first mismatch after the application settles. It does not replace schema tests or durable readback.

Teams evaluating browser-agent workflows can review the browser agent regression test pilot.

The practical rule is to test the website's tool surface like an application API: freeze the contract, attack its boundaries, control retries, and verify the result outside the tool's own response.

Sources

• Playwright 1.64 Release Notes

• Playwright WebMCP API

• Playwright CLI WebMCP Commands

• Playwright MCP WebMCP Support

• Applied Page Object to WebMCP Workflow