Premium
Services
Premium

Agents rely on web interfaces to complete web tasks. To compare task completion across web interfaces, we built the AIM Agentic Web Benchmark and ran 100 tasks through 8 interfaces, 5 times each, for 4,000 scored results.

Success rates

Successful retrieval rate, avg of five runs
Successful retrieval rate, avg of five runs
NameMetricSuccessful retrieval rate, avg of five runs
Nimble MCPTasks completed0.284
Zenrows MCPTasks completed0.29
Nimble CLITasks completed0.314
Oxylabs MCPTasks completed0.344
Firecrawl MCPTasks completed0.346
Zenrows CLITasks completed0.392
Bright Data CLITasks completed0.428
Bright Data MCPTasks completed0.43

Bright Data CLI and Bright Data MCP completed 3.6 and 3.8 more tasks per 100 than Zenrows CLI, the best of the other six.

Ten example tasks are on GitHub, and the methodology explains how we ran the benchmark.

Consistency across runs

Best and worst of five runs, same 100 tasks every time
NameBest runWorst run
Nimble MCP0.320.25
Zenrows MCP0.350.24
Nimble CLI0.360.27
Oxylabs MCP0.370.32
Firecrawl MCP0.380.28
Zenrows CLI0.430.35
Bright Data CLI0.460.39
Bright Data MCP0.520.37

Bright Data MCP has the best run with 52%. The gap between the best and worst run on Oxylabs MCP is only 5 tasks across 5 runs. It is the most consistent interface in the panel, followed by:

  • Bright Data CLI: 7
  • Nimble MCP: 7
  • Zenrows CLI: 8
  • Nimble CLI: 9
  • Firecrawl MCP: 10
  • Zenrows MCP: 11
  • Bright Data MCP: 15
Get our team to automate one of your business processes with AI agents, free of charge.
Automate a process

Cost and speed

Cost and speed against success rate
NameTasks completed out of 100Total cost per task (model plus interface fees, USD)Model costInterface costMetric
Bright Data CLI0.428357——Latency vs. Success Rate
Bright Data MCP0.43186——Latency vs. Success Rate
Firecrawl MCP0.346240——Latency vs. Success Rate
Nimble CLI0.314830——Latency vs. Success Rate
Oxylabs MCP0.344254——Latency vs. Success Rate
Zenrows MCP0.29306——Latency vs. Success Rate
Nimble MCP0.284253——Latency vs. Success Rate
Zenrows CLI0.392466.5——Latency vs. Success Rate
Bright Data CLI0.4280.45850.2750.1835Cost vs. Success Rate
Bright Data MCP0.430.3080.1520.156Cost vs. Success Rate
Firecrawl MCP0.3460.08420.080.0042Cost vs. Success Rate
Nimble CLI0.3140.27310.2640.0091Cost vs. Success Rate
Oxylabs MCP0.3440.10160.0890.0126Cost vs. Success Rate
Zenrows MCP0.290.33220.170.1622Cost vs. Success Rate
Nimble MCP0.2840.0720.0650.007Cost vs. Success Rate
Zenrows CLI0.3920.39820.27260.1255Cost vs. Success Rate

Bright Data MCP is the fastest interface at a median of 186 seconds per task, and all five MCP tools had lower medians than three CLI tools. On the same provider, the CLI took 1.5 times as long as the MCP server for Zenrows, 1.9 times for Bright Data, and 3.3 times for Nimble. Nimble CLI is the slowest at 830 seconds and retried 144 of its 500 tasks after timeouts, more than any other interface.

Total cost is the agent’s token spend plus the provider’s fee. Nimble MCP has the lowest total at $0.072 per task. For the two Bright Data interfaces and Zenrows MCP, which use a remote browser billed by bandwidth, provider fees make up 40 to 51 percent of the total. Firecrawl’s figure excludes per-minute browser sessions, so it is a lower bound.

Failure reasons

Average outcome mix per 100 tasks, all runs pooled
NameCompletedBlockedNo answerWrong answerNDBypassed
Nimble MCP28.435.4914.411.81
Zenrows MCP292921.610.29.40.8
Nimble CLI31.418.228.411.291.8
Oxylabs MCP34.430.47.411142.8
Firecrawl MCP34.630.212.21210.80.2
Zenrows CLI39.2284.612.814.60.8
Bright Data CLI42.817.69.613.215.61.2
Bright Data MCP4322.45.61216.80.2

Bright Data CLI was blocked on average 17.6 times per 100 tasks, the fewest in the panel, while Firecrawl MCP, Nimble MCP and Oxylabs MCP were blocked 1.7 to 2 times as often.

Nimble CLI was blocked least among the six non-Bright Data interfaces at 18.2, but returned nothing on 28.4 tasks per 100, the highest non-delivery in the panel, which reflects hitting the 30-minute limit on 36.3 percent of its attempts (attempts exceed tasks because timed-out tasks were retried once).

Zenrows CLI and Zenrows MCP were blocked at about the same rate, but the CLI completed more tasks successfully.

Don’t miss our benchmarks and data-driven insights. The button opens Google; selecting AIMultiple confirms that you wish to see AIMultiple more often in Google search results.
GoogleAdd as preferred source

Methodology

Each interface ran on its own VPS. The agent was OpenClaw running MiniMax M3 through OpenRouter, without any parallel task runs. Every server had 2 vCPUs and 2GB of RAM.

The task set used was identical for all eight interfaces on all 5 runs. 

Before each run, a probe fetched one low-cost page through the provider account, and the run refused to start if that fetch failed. Each attempt had 30 minutes.

A task that ran out of time or returned under 500 bytes of output got one retry, except the four state-changing tasks, which were never retried because a first attempt may already have modified a cart. The retry is the attempt that gets scored, and model cost and time come from it, even when it also hit the limit.

Every installed client except Zenrows CLI was on the latest release as of July 27, 2026. Zenrows CLI was first published on July 31, 2026. Versions used in this benchmark:

Example Tasks

Ten of the 100 tasks are published in full on GitHub, with the instruction the agent was given, the output format, the pass rules, and how each interface scored.

Dataset

The 100 tasks span 29 websites in six categories. We selected websites from SimilarWeb’s top 20 sites by traffic in each category on May 15, 2026, starting at the top of each list.

We skipped sites that serve a single non-US market, regional copies of a chosen site, messaging apps, sites that require a login, and sites with no public listings to search.

We prepared 2 to 5 tasks per website depending on how many actions the site supports. Ninety-six tasks retrieve information, and four change state on the remote site.

Evaluation

Each task and its pass conditions were written in English. The scorer reads the rule file and scores only when every rule that applies to it holds.

Blocked means the agent returned an error message instead of data, for example, that the provider could not reach the site or that a fee breakdown was not shown on the page.

Some providers rejected requests to certain sites. Those tasks stay in the 100-task total for every interface.

  • Zenrows: Ticketmaster and Instagram (7 tasks), were refused under its usage policy regardless of the settings.1
  • Oxylabs: Ticketmaster, AXS and SeatGeek, and StubHub in one run, rejected as “not supported”. Oxylabs lists ticketing among the targets it restricts unless the account passes a verification check.2
  • Firecrawl: LinkedIn, Reddit, TikTok, X and Instagram, answered with “we do not support this site”.

Bypassed means the agent fetched the page directly rather than using the provider interface, typically via a direct request from the shell. We find these in the session logs. A bypassed task earns nothing and stays in the 100-task total.

“Not Determined” cells are those in which the scorer could not decide whether the answer was correct. These were counted neither correct nor blocked. This covers 8 to 18 tasks per interface per run, all from the same 28 tasks.

We used the same tasks, instructions, and model across all eight interfaces, and locked the rules before the first run.

Cost

The chart adds two numbers. Model cost is what the agent spent on tokens, MiniMax M3 through OpenRouter at $0.30 per million input tokens, $1.20 per million output and $0.06 per million cache reads, counted from the scored attempt of each task.

Interface cost is what the provider charged for the same work, averaged over five runs:

  • Bright Data: usage read from the account’s billing per zone per day, priced at $8 per GB plus $0.10 per hour for Scraping Browser, $1.50 per 1K requests for Web Unlocker and $1.50 per 1K for SERP.
  • Zenrows: from their panel, browser session bandwidth plus fetch credits as logged. The plan is EUR 129.99 per 1M credits; browser works out to about 40,500 credits per GB, and fetch averages 14 credits per request after the JS render and premium proxy multipliers.
  • Oxylabs: request counts from the agent transcripts at $1.15 per 1K results.
  • Firecrawl: credits counted from the transcripts at $83 per 100K credits, 1 credit per scrape.
  • Nimble: requests counted from the transcripts at $1.00 per 1K for extract and $1.10 per 1K for search.

Limitations

Nimble CLI ran out of time on 36.3 percent of its attempts.

All interfaces have different speeds, not all tasks start or finish at the same time. Some tasks depend on the date or what is listed at the time, so some providers did not see the same page. Pass rules were written to test the format and the retrieval, not a specific listing, so a changed page does not change the score.

Cite this benchmark

Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.

Cem Dilmegani and Berk Kalelioğlu (2026) - "AIM Agentic Web Benchmark". Published online at AIMultiple.com. Retrieved October 15, 2026, from: https://aimultiple.com/agentic-web [Online Resource]

Dilmegani, C., & Kalelioğlu, B. (2026, October 15). AIM Agentic Web Benchmark. AIMultiple. https://aimultiple.com/agentic-web

@misc{dilmegani2026,
  author = {Dilmegani, Cem and Kalelioğlu, Berk},
  title  = {{AIM Agentic Web Benchmark}},
  year   = {2026},
  month  = oct,
  howpublished    = {\url{https://aimultiple.com/agentic-web}},
  note   = {AIMultiple. Retrieved October 15, 2026}
}
Download all data

Results and timestamps of 64 data points. Download the summary data shown in this article's charts and tables as a ZIP file containing 7 CSV files and a README.

Last updated: November 13, 2026
Download

Want the granular data behind it? Join Premium

Cem Dilmegani
Cem Dilmegani
Principal Analyst
Cem has been the principal analyst at AIMultiple since 2017.

Cem's work at AIMultiple has been cited by leading global publications including Business Insider, Forbes, Morning Brew, and Washington Post, global firms like Deloitte and HPE, NGOs like World Economic Forum, and supranational organizations like European Commission. [1], [2], [3], [4], [5]

Throughout his career, Cem served as a tech consultant, tech buyer and tech entrepreneur. He advised enterprises on their technology decisions at McKinsey & Company and Altman Solon for more than a decade. He also published a McKinsey report on digitalization.

He led technology strategy and procurement of a telco while reporting to the CEO. He has also led commercial growth of deep tech company Hypatos that reached a 7 digit annual recurring revenue and a 9 digit valuation from 0 within 2 years. Cem's work in Hypatos was covered by leading technology publications like TechCrunch and Business Insider.

Cem regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.
View Full Profile
Technically reviewed by
Berk Kalelioğlu
Berk Kalelioğlu
AI Researcher
Berk is an AI Researcher at AIMultiple's benchmark team, focusing on agentic AI, machine learning, and large and small language models (LLMs & SLMs).
View Full Profile

Be the first to comment

Your email address will not be published. All fields are required. Comments are left in their original language.

0/450

We follow ethical norms & our process for objectivity. Subscribers of AIMultiple's benchmarking services in Agentic Web include Bright Data, Oxylabs, ZenRows.