THE SHORT VERSION
AI agents need to know what they are looking at, not only what it looks like. A screenshot shows the visible result. Element context identifies the control and its state. The recorded sequence helps explain how you got there. Together, they give an investigation a more specific starting point.
Each layer answers a different question.
Element context is information about a UI element at a particular moment: its identity, surrounding structure, and relevant state. That could include a button’s role, accessible name, attributes, and a locator scoped to the right section.
Screenshot
Layout, visible text, overlap, and the visual symptom at that moment.
Limit Does not establish the underlying element, its programmatic state, or the earlier steps.
Element details
Relevant markup, role, accessible name, attributes, and a way to locate the target.
Limit A DOM match alone does not prove that a control can receive a click or that an action will succeed.
Recording or trace
The captured actions and changes before and after the symptom.
Limit Only covers what the tool recorded. A replay does not restore server state or prove a root cause.
Keep the screenshot. It can reveal clipped text, misplaced controls, and visual overlap that an element’s markup does not establish. For a canvas or other interface with little useful DOM structure, visual evidence may be essential.
Some agents already act through images and coordinates, as in Anthropic’s computer-use tools. Add DOM and accessibility details when they help resolve the question the image leaves open.
The same principle appears in Playwright’s Trace Viewer, which puts captured screenshots and DOM snapshots beside the actions from a test run.
Same appearance. Different state.
These two versions of “Export CSV” look identical. One is enabled; the other has a native disabled attribute. Switch states, try the button, and compare the element details below.
Usage report
September · All projects
Same styling in both states.
- Role
- button
- Accessible name
- Export CSV
- Section
- Usage report
disabledattribute- Absent
Try Export CSV. Then compare the disabled state.
Enabled. The outcome is still unverified.
This button can receive a click in the example. That tells you the handler ran—not that a real CSV was created.
<section aria-label="Usage report">
<button type="button" data-testid="export-csv">
Export CSV
</button>
</section>Next checkOn the real page, check that the expected export completes and contains the right data.
A local teaching example with fictional data. The buttons are deliberately styled alike to isolate the difference; a real interface should make its disabled state clear.
This example uses HTML’s native disabled attribute. By contrast, aria-disabled communicates a disabled state but does not itself prevent activation. Custom controls need behavior that matches their semantics.
Find it. Check it. Verify the result.
Element details narrow the question. They do not remove the need to check the current page. A target can move, be replaced, or become unavailable after a navigation or UI update.
- 01
Identify the intended control.
Use the section, role, and name to distinguish it from similar controls. Confirm the locator matches one intended element.
- 02
Check whether it can be used now.
Inspect the current state. A visible, enabled button may still be moving or covered by another element.
- 03
Check what the action actually did.
For an export, confirm the download and its contents. A click event or success message is only part of the evidence.
See a Playwright starting point
import { expect } from '@playwright/test';
// Run on your report page after the relevant setup.
const report = page.getByRole('region', {
name: 'Usage report', exact: true,
});
const exportButton = report.getByRole('button', {
name: 'Export CSV', exact: true,
});
await expect(exportButton).toHaveCount(1);
await expect(exportButton).toBeVisible();
await expect(exportButton).toBeEnabled();
// Click also checks stability and whether the target receives events.
await exportButton.click();
// Add an assertion for your app's actual export result.
// A successful click does not verify a file's contents.Adapt the names and setup to your app. This snippet checks the target and clicks it; it intentionally leaves the application-specific export assertion for you to add.
Playwright locators resolve against the current DOM. Its actionability checks cover conditions such as visibility, stability, receiving events, and being enabled before a click.
Say what you observed. Keep the cause open.
Finding disabled explains why the native button cannot be activated. It does not explain why the application disabled it. That could require the preceding steps, validation state, account permissions, or a relevant request and response.
The button is disabled.
The attribute is present. The native button does not activate. Its appearance has stayed the same.
Why did it enter that state?
The element alone cannot tell us. Review the transition and the rule that should enable the export.
Include the relevant moment from the recording, what you expected, and what happened. Keep the capture time and environment with the evidence. If a field or earlier step was not captured, say so instead of filling the gap with a guess.
Get a browser-agent handoff templateFROM THE EXAMPLE TO YOUR OWN APP
Keep the element with the evidence.
Record the steps, inspect the affected element, and add what you expected to happen. Give your teammate or coding assistant a specific place to begin.
Free extension for desktop Chrome. Try it before signing up.Inspect an element and review its selectors with our extension. Save and review the relevant evidence before sharing it. Available details depend on what was captured.
Using a coding assistant? Connect saved context through MCPA few common questions.
What is element context for an AI agent?
It is information that identifies a UI element and describes its structure and state at a particular moment. Useful details include its section, role, accessible name, relevant attributes, and a locator. Pair those details with the screenshot and the actions that led to the issue.
Can an AI agent work from screenshots alone?
Some agents use visual input to identify controls and interact through coordinates. Screenshots are useful for appearance and layout. When DOM or accessibility information is available, it can resolve questions that the pixels leave open, such as which repeated control is intended or whether a native button is disabled.
Does element context replace screenshots?
No. An element’s markup can look correct while its text is clipped or another element covers it. Canvas-based interfaces may expose little useful DOM structure. Keep the visual evidence and use the other available details to answer the specific question.
Is a CSS selector enough?
A selector can help locate an element. It does not explain the preceding actions, prove the element is ready for interaction, or confirm the result of a click. Check that it identifies the intended target in the current page state, then verify the application outcome separately.
What if the page has changed since the capture?
Treat the capture as evidence of an earlier run. Re-check the current page, target, and relevant setup before acting. Keep what was observed in the recording separate from what you have verified now.
How can Samelogic supply this context?
Use the extension to record the browser steps and inspect the affected element. Save the relevant evidence to a project and review it before sharing. Samelogic’s MCP tools give compatible coding assistants read-only access to authorized saved recordings and element details. Available fields depend on what was captured and saved; the connection does not control a live browser or verify a fix.
Sources & further reading
The comparison and example are authored for this guide. These references explain the browser and testing behavior behind them.