Find the cause.
Fix the flaky test.
Compare a failing run with a passing one. Work through timing, shared state, data, network, and locators to find your next useful check.
save.spec.ts| Step | Passing run | Failing run |
|---|---|---|
| Open settings | Page ready | Page ready |
| Click save | Clicked | Clicked |
| Check the result | Saved | Still saving |
This is where to look next.The click completed in both runs. Check the response and the state that follows.
A familiar failure.
A useful next check.
Choose the pattern closest to yours. Each one gives you evidence to gather and an explanation to test.
Several causes can produce the same symptom. Use these checks to test an explanation.
Check the state the test is waiting for.
A button can be clickable while the application is still loading or saving. Look at what the test expects immediately before and after the action.
Read the failed action’s log and DOM snapshots. Was the target ready, or did the expected result never appear?
Wait for an observable application state with a retrying assertion. A fixed sleep does not tell you whether that state is ready.
The failed assertion, the preceding action, and the UI state in both runs.
Try one condition-based wait or assertion, then repeat in the environment that failed.
Look for something another test leaves behind.
Playwright gives each test a fresh browser context by default. Shared accounts, server records, globals, and reused pages can still connect otherwise separate tests.
Compare the isolated run with the failing suite order. Check beforeAll hooks, reused pages, cleanup, and accounts used by multiple tests.
Compare the usual worker count with one worker as an experiment. A changed result is a clue about shared state or load, not proof of either.
The preceding test, worker count, shared account or record, and any cleanup that ran.
Give the test its own setup and data where needed, then verify with the original suite and worker count.
Make the starting conditions explicit.
A different role, expired record, or already-used fixture can change what the page allows. Compare the inputs before changing the interaction.
Check the account’s permissions and the record’s actual starting state. Look for expired, deleted, or reused data.
Give the case a known fixture and check setup completed successfully. Keep concurrent tests from editing the same record.
The fixture setup, role, relevant record state, and expected permission or feature state.
Repeat with freshly prepared data, then with the original failing conditions to test the explanation.
Follow the request through to the UI.
A timeout may begin with a delayed response, an unexpected payload, or a service error. The last failed assertion may only be the visible consequence.
Inspect the relevant request in the trace: did it start, what response arrived, and did its timing differ from the passing run?
Check the UI after the response. A successful status code alone does not prove that the expected content rendered.
The request, response status and relevant payload, plus the UI state that followed.
Use controlled responses when testing the UI independently; keep live-service coverage for behavior that depends on that service.
Confirm which element the test is finding.
A changed accessible name, reused test ID, or unstable class can change the target. Check the failing page state before replacing the selector.
Inspect the target in the failed trace. Check its accessible name, test ID, and the number of matching elements.
Narrow the locator to the intended region. Choosing the first match can hide an ambiguity that matters to the test.
The locator, intended target, match count, and the attribute or page structure that changed.
Choose an intentional locator contract and verify both the action and the expected outcome.
Keep the failure.
Learn from the trace.
Keep the failure visible
Repeat a focused test with retries disabled for the investigation. Keep the browser, environment, and data conditions that matter.
Compare the first difference
Open the HTML report and compare traces. Inspect the action, DOM state, console, and relevant network activity before the failure.
Verify one change
Test one explanation at a time, then repeat with fresh setup and the original suite conditions. Several passes are evidence, not a guarantee.
npx playwright test tests/checkout.spec.ts \
--retries=0 --repeat-each=5 \
--trace=on --reporter=html- 0 retries
- Keep each failed attempt visible.
- 5 repetitions
- Look for a pattern across runs.
- Traces + report
- Compare what happened before the failure.
Replace tests/checkout.spec.ts with your test file. Keep the failing browser and environment; tracing every attempt is for this focused investigation.
Keep the steps
that triggered it.
When you can reproduce the issue by hand, record the steps with Samelogic. Replay the sequence, inspect the affected element, and add a note for the receiving engineer.
The automated actions, snapshots, and network evidence from the test.
The manual browser path, inspected element, and your explanation of the issue.
- Open seat renewal
- Click “Add seat”
- Click “Add seat” again
- Calculate renewal
Three seats on screen.
Two in the quote.
Go deeper
where it matters.
Review the locator’s contract.
Compare role, test ID, and CSS matches when the page changes.
02Keep the steps behind the bug.
Explore a recording and inspect the element at the relevant moment.
03Understand why selectors break.
Take a closer look when the evidence points to a locator problem.
Good questions.
Clear answers.
What makes a Playwright test flaky?
A flaky test passes and fails without an intentional change to the behavior being tested. Common sources include timing, shared state, test data, network behavior, and locator changes. The symptom gives you a place to investigate; it does not identify the cause by itself.
Should I increase retries or timeouts?
Retries can keep a pipeline moving and help collect evidence, but a passing retry does not explain the original failure. Keep the failed attempt visible while investigating. Change a timeout when the expected operation legitimately needs longer, and check the condition you are waiting for before adding a fixed delay.
Why does it pass locally and fail in CI?
Compare browser and operating-system versions, worker count, service access, environment configuration, account permissions, and test data. Resource or timing differences can expose a problem that is hard to observe locally. Reproduce the relevant CI conditions and use the trace to decide what to check next.
Can Samelogic run or diagnose my Playwright suite?
Samelogic does not run your CI suite or automatically diagnose every failure. Use Playwright’s reports and traces for the automated run. If you can reproduce the issue manually, use the extension to record the browser steps, replay them, inspect an element, and share context with the receiving engineer.
Does the checklist analyze my trace?
No. Choose an observed pattern to see suggested checks and copy an investigation outline. Nothing is uploaded or analyzed. Run the checks against your own test and record what you find.
Can I try the extension before signing up?
Yes. Install the free extension in desktop Chrome and try recording and inspecting a browser flow. An account is needed to save recordings to a workspace and share them. Review the recording and its access settings before sharing.
Give the fix
a clear starting point.
Record the browser steps your teammate needs to investigate. Keep the relevant moment and element close to the issue.