Hark Handoff Raises the Browser-Agent Bar. Now Test The Failures.
On August 5, Brett Adcock introduced Hark Handoff as a computer-use agent built for everyday internet work: ordering food, shopping, booking reservations and flights, researching, and navigating sites that do not expose useful consumer APIs.
The headline result is substantial. The official Online-Mind2Web human-evaluation leaderboard currently lists Hark Handoff first with a 97.7% average success rate across 300 tasks on 136 live websites. The benchmark splits tasks into easy, medium, and hard groups based on the number of human steps required. Handoff is listed at 100% on easy tasks, 95% on medium tasks, and 100% on hard tasks.
Hark also reports that Handoff leads across three browser-use evaluations and, on average, outperforms GPT-5.4 by eight points and Opus 4.8 by two points. That broader comparison deserves a precise caveat: Hark says its WebTailBench and internal evaluations use a Hark-designed harness, while the Online-Mind2Web human-evaluation result is listed by the independent benchmark team.
That distinction does not weaken the release. It clarifies what the industry can verify today and what still needs external replication.
The larger signal is clear: browser agents are moving from demos that prove they can click through a page to products expected to complete consequential work on the live web.
The next benchmark frontier is not another success number by itself. It is a reliable answer to a harder question:
When a browser agent fails, can we prove why?
Hark is targeting the web as it actually exists
Hark's release post is unusually direct about the environment. It describes the internet as hostile to automation, full of bot blocking, pop-ups, banners, ads, dynamic interfaces, and session-specific friction.
That is the right framing.
A controlled demo can make the browser look like a sequence of buttons. Production browser automation is a negotiation among the agent, the browser, the website, identity systems, payment and inventory state, security controls, and the evaluator deciding whether the outcome counts.
Handoff runs each request in a dedicated virtual computer with a browser, file system, and terminal. It emits cursor and keyboard actions rather than depending on fixed integrations for a small set of sites. Hark says it uses reinforcement learning so the model can learn from real task successes and failures rather than relying on supervised demonstrations alone.
This matters because the category is no longer constrained to sites designed for automation. The product promise is precisely that the agent can operate where the interface is changing and the API is missing.
That broader reach also expands the failure surface.
A failed task is not one failure class
Suppose an agent is asked to book a flight and does not complete the purchase.
A benchmark can record failed. A production team needs to know which of these happened:
Planning failure: the agent chose the wrong route, date, fare, or sequence.
Perception or targeting failure: it misread the page or acted on the wrong control.
Browser-control failure: the click, scroll, upload, download, or navigation did not execute as intended.
Website failure: the page itself returned an error, rendered stale state, or broke during the workflow.
Identity or session failure: login expired, an account switch occurred, or an OTP challenge required the user.
Inventory or state change: the fare, table, item, or delivery window disappeared during the run.
Policy or security intervention: geo restrictions, rate limits, fraud controls, anti-bot systems, or permissions blocked the action.
Evaluator failure: the task succeeded, but the judge inspected the wrong state or lacked the evidence required to recognize it.
Safety stop: the system correctly refused to cross a purchase, messaging, or data boundary without confirmation.
These failures demand different responses.
Retraining the model will not fix an expired session. Retrying a website defect may create a duplicate order. Treating an OTP pause as agent incompetence punishes correct boundary handling. Calling an inventory change a website bug corrupts the reliability data. Declaring success from a confirmation screen without reading back the actual reservation can reward a false positive.
Pass or fail is the start of the diagnosis, not the diagnosis.
The evaluation unit should become a test artifact
Online-Mind2Web already points in a productive direction. Its version-two submission format bundles each step's action, thought, screenshot, and URL, while its guidance asks submitters to preserve factual actions and avoid contaminating the action history with agent narration. Its WebJudge design selects key screenshots and actions before judging the result.
The next step is to make the evaluation record useful not only for scoring but also for failure attribution.
For every consequential run, preserve a replayable test artifact with:
the task and success criteria defined before execution,
the agent, model, policy, account, and browser environment,
the starting website and session state,
every material action bound to its page, frame, and target,
relevant console, network, navigation, and permission events,
the first point where observed state contradicted expected state,
any human handoff, OTP request, or confirmation boundary,
the final side effect read back from the receiving system,
the evaluator's verdict and the evidence it used,
a specific failure class when the task does not succeed.
This record should be inspectable without replaying the whole internet session from memory. It should also survive changes to the live site, the agent policy, and the benchmark harness.
A benchmark result tells us how often an agent succeeded. A test artifact tells us what happened on the run and whether the verdict deserves trust.
Travel is an excellent reliability benchmark
Hark highlights flight and hotel booking as a target workflow. Travel is also one of the strongest verticals for testing production reliability.
A single travel task can combine:
logged-in and logged-out pricing,
geo-dependent inventory,
rapidly changing fares,
session expiration,
seat and room availability,
loyalty-account state,
payment authorization,
OTP or fraud challenges,
rate controls and bot defenses,
irreversible or costly final actions.
That mix makes failures both realistic and easy to misclassify.
A useful travel benchmark would score more than booking completion. It would ask whether the agent recognized stale inventory, paused at the correct confirmation boundary, avoided duplicate purchases, preserved the offered price and terms, and produced enough evidence to distinguish an agent mistake from a site or market-state change.
Ecommerce and restaurant reservations provide similar public test surfaces. Seller portals can add more complex account, catalog, and permissions state. Banking should come later, after the benchmark's safety boundaries, identity controls, and failure-classification accuracy are mature.
What this means for our industry
Hark Handoff is evidence that general browser use is becoming a serious product category. A verified 97.7% result on Online-Mind2Web's human-evaluation leaderboard raises expectations for what agents can do across the live web.
It also raises the cost of ambiguity.
As browser agents become more capable, a larger share of their remaining failures will sit at boundaries: identity, sessions, state changes, side effects, security controls, and evaluation. Those are precisely the places where retries and agent narration are least trustworthy.
Our industry now needs two layers working together:
a capable agent that can perceive, reason, and act across unpredictable websites;
an independent verification layer that can preserve the run, classify the failure, and prove the consequential result.
This is the direction we are building toward at Samelogic. Today, Samelogic turns deliberately initiated browser work into replayable test artifacts that preserve the transition a fresh-page reproduction skips. Programmatic agent capture and stronger attestation are the next layer, not claims we should pretend are already complete.
The browser-agent race is producing better operators. The next durable advantage will come from making every important run inspectable, diagnosable, and trustworthy.
If your team is already testing consequential browser workflows, try Samelogic on the Chrome Web Store and start preserving the path, not only the final screen.
Sources and further reading
Brett Adcock's Hark Handoff announcement thread:
Hark's research preview, including system design, use cases, benchmark claims, training, cost, and latency:
Official Online-Mind2Web human and automatic evaluation leaderboards:
https://huggingface.co/spaces/osunlp/Online_Mind2Web_Leaderboard
Online-Mind2Web benchmark repository, v2 trajectory schema, task coverage, and evaluation guidance: https://github.com/OSU-NLP-Group/Online-Mind2Web
Related workflows
Move from editorial context into the selector, Playwright, and bug-reproduction pages that turn exact UI evidence into action.
