Agents rely on web interfaces to complete tasks on the web. To compare task completion across web interfaces, we built the AIM Agentic Web Benchmark and ran 100 tasks through 7 interfaces, 5 times each, for 3,500 scored results.
Success rates
Bright Data CLI and Bright Data MCP completed 8.2 and 8.4 more tasks per 100 than Firecrawl MCP, the best of the other five.
Ten example tasks are on GitHub, and the methodology explains how we ran the benchmark.
Consistency across runs
Bright Data MCP has the best run with 52%. The gap between the best and worst run on Oxylabs MCP is only 5 tasks across 5 runs. It is the most consistent interface in the panel, followed by:
- Bright Data CLI: 7
- Nimble MCP: 7
- Nimble CLI: 9
- Firecrawl MCP: 10
- ZenRows MCP: 11
- Bright Data MCP: 15
Cost and speed
Bright Data MCP is the fastest interface at a median of 186 seconds per task, and all five MCP tools had lower medians than both CLI tools. On the same provider, the CLI took 1.9 times as long as the MCP server for Bright Data and 3.3 times as long for Nimble. Nimble CLI is the slowest at 830 seconds and retried 144 of its 500 tasks after timeouts, more than any other interface.
Total cost is the agent’s token spend plus the provider’s fee. Nimble MCP has the lowest total at $0.072 per task. For the two Bright Data interfaces and ZenRows MCP, which use a remote browser billed by bandwidth, provider fees make up 40 to 51 percent of the total. Firecrawl’s figure excludes per-minute browser sessions, so it is a lower bound.
Failure reasons
Bright Data CLI was blocked on average 17.6 times per 100 tasks, the fewest in the panel, while Firecrawl MCP, Nimble MCP and Oxylabs MCP were blocked 1.7 to 2 times as often.
Nimble CLI was blocked least among the five non-Bright Data interfaces at 18.2, but returned nothing on 28.4 tasks per 100, the highest non-delivery in the panel, which reflects hitting the 30-minute limit on 36.3 percent of its attempts (attempts exceed tasks because timed-out tasks were retried once).
Add as preferred sourceMethodology
Each interface ran on its own VPS. The agent was OpenClaw running MiniMax M3 through OpenRouter, without any parallel task runs. Every server had 2 vCPUs and 2GB of RAM.
The task set used was identical for all seven interfaces on all 5 runs.
Before each run, a probe fetched one low-cost page through the provider account, and the run refused to start if that fetch failed. Each attempt had 30 minutes.
A task that ran out of time or returned under 500 bytes of output got one retry, except the four state-changing tasks, which were never retried because a first attempt may already have modified a cart. The retry is the attempt that gets scored, and model cost and time come from it, even when it also hit the limit.
Versions used in this benchmark:
Example Tasks
Ten of the 100 tasks are published in full on GitHub, with the instruction the agent was given, the output format, the pass rules, and how each interface scored.
Dataset
The 100 tasks span 29 websites in six categories. We selected websites from SimilarWeb’s top 20 sites by traffic in each category on May 15, 2026, starting at the top of each list.
We skipped sites that serve a single non-US market, regional copies of a chosen site, messaging apps, sites that require a login, and sites with no public listings to search.
We prepared 2 to 5 tasks per website depending on how many actions the site supports. Ninety-six tasks retrieve information, and four change state on the remote site.
Evaluation
Each task and its pass conditions were written in English. The scorer reads the rule file and scores only when every rule that applies to it holds.
Blocked means the agent returned an error message instead of data, for example, that the provider could not reach the site or that a fee breakdown was not shown on the page.
Bypassed means the agent fetched the page directly rather than using the provider interface, typically via a direct request from the shell. We find these in the session logs. A bypassed task earns nothing and stays in the 100-task total.
“Not Determined” cells are those in which the scorer could not decide whether the answer was correct. These were counted neither correct nor blocked. This covers 8 to 18 tasks per interface per run, all from the same 28 tasks.
The same tasks, instructions, and model were used across all seven interfaces, and the rules were locked before the first run.
Cost
The chart adds two numbers. Model cost is what the agent spent on tokens, MiniMax M3 through OpenRouter at $0.30 per million input tokens, $1.20 per million output and $0.06 per million cache reads, counted from the scored attempt of each task.
Interface cost is what the provider charged for the same work, averaged over five runs:
- Bright Data: read from the account’s billing per zone per day. Scraping Browser is $8 per GB plus $0.10 per hour, Web Unlocker $1.50 per 1K requests, SERP $1.50 per 1K.
- ZenRows: from their panel, browser session bandwidth plus fetch credits as logged. The plan is EUR 129.99 per 1M credits; browser works out to about 40,500 credits per GB, and fetch averaged 14 credits per request after the JS render and premium proxy multipliers.
- Oxylabs: request counts from the agent transcripts at $1.15 per 1K results.
- Firecrawl: credits counted from the transcripts at $83 per 100K credits, 1 credit per scrape.
- Nimble: requests counted from the transcripts at $1.00 per 1K for extract and $1.10 per 1K for search.
Limitations
Nimble CLI ran out of time on 36.5 percent of its attempts.
All interfaces has different speeds, not all task start or finished at the same time. Some tasks depend on the date or on what is listed at the time, so on those, some providers did not see the same page. Pass rules were written to test the format and the retrieval, not a specific listing, so a changed page does not change the score.
Cite this research
Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.
@misc{dilmegani2026,
author = {Dilmegani, Cem and Kalelioğlu, Berk},
title = {{AIM Agentic Web Benchmark}},
year = {2026},
month = sep,
howpublished = {\url{https://aimultiple.com/agentic-web}},
note = {AIMultiple. Retrieved September 29, 2026}
}Results and timestamps of 16 data points. Download the summary data shown in this article's charts and tables as a ZIP file containing 2 CSV files.
Want the granular data behind it? Join Premium
Cem's work at AIMultiple has been cited by leading global publications including Business Insider, Forbes, Morning Brew, and Washington Post, global firms like Deloitte and HPE, NGOs like World Economic Forum, and supranational organizations like European Commission. [1], [2], [3], [4], [5]
Throughout his career, Cem served as a tech consultant, tech buyer and tech entrepreneur. He advised enterprises on their technology decisions at McKinsey & Company and Altman Solon for more than a decade. He also published a McKinsey report on digitalization.
He led technology strategy and procurement of a telco while reporting to the CEO. He has also led commercial growth of deep tech company Hypatos that reached a 7 digit annual recurring revenue and a 9 digit valuation from 0 within 2 years. Cem's work in Hypatos was covered by leading technology publications like TechCrunch and Business Insider.
Cem regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.
Be the first to comment
Your email address will not be published. All fields are required. Comments are left in their original language.