Agents rely on web interfaces to complete tasks on the web. To measure how interface choice affects task completion, we built the AIM Agentic Web Benchmark and attempted 3.500 tasks (100 tasks completed via 7 web interfaces across 5 runs).
Success rates
A Bright Data interface led every run. The Bright Data CLI retrieved 8 more tasks than the best non-Bright Data interface, which is Firecrawl MCP.
Investigate 10 example tasks on GitHub and see how we run this benchmark on methodology
Consistency across runs
Bright Data MCP has the best run with %52. The gap between the best and worst run on Oxylabs MCP is only 5 tasks across 5 runs. It is the most consistent interface in the panel, followed by:
- Bright Data CLI: 7
- Nimble MCP: 7
- Nimble CLI: 9
- Firecrawl MCP: 10
- ZenRows MCP: 11
- Bright Data MCP: 15
Cost and speed
The cost axis adds the agent’s token spend to what the provider charged for the same task. On time the MCP tools beat both CLI tools with no overlap. Working with CLI means the agent shells out, reads text back, and often retries the syntax, and every extra turn carries the whole conversation forward again. Cache reads dominate the cost, so the figure mostly counts how many turns an interface needs before the agent can answer.
Nimble MCP is the lowest cost and the fastest interface in the benchmark at $0,065 per task, with Firecrawl MCP next at $0.08. Bright Data MCP is the fastest, at a median of 186 seconds per task. Nimble CLI had more retries than other tools because of timeouts. A task that runs out of time gets a second attempt, and only that second attempt is scored. Cost and latency come from that same scored attempt, which can itself have hit the limit.
Failure reasons
Bright Data CLI was blocked on average 17.6 times per 100 tasks, the fewest in the panel, while Firecrawl MCP, Nimble MCP and Oxylabs MCP were blocked almost twice as often.
Nimble CLI was blocked least among the four non-Bright-Data interfaces at 18.2, but returned nothing on 28.4 tasks per 100, the highest non-delivery in the panel, which reflects hitting the 30-minute limit on 36.5 percent of its attempts (attempts exceed tasks because timed-out tasks were retried once).
Tercih edilen kaynak olarak ekleMethodology
Each interface ran on its own VPS. The agent was OpenClaw running MiniMax M3 through OpenRouter, without any parallel task runs. Every server had 2 vCPUs and 2GB of RAM.
The task set used was identical for all seven interfaces on all 5 runs.
Each attempt had 30 minutes. A task that ran out of time got one retry, except the four state-changing tasks, which were never retried. Before each run a probe fetched one low-cost page through the provider account, and the run refused to start if that fetch failed.
Versions used in this benchmark;
Example Tasks
Ten of the 100 tasks are published in full at GitHub, with the instruction the agent was given, the output format, the pass rules, and how each interface scored.
Dataset
The 100 tasks span 29 websites in these categories: e-commerce (24 tasks), travel (21), real estate (17), tickets (15), social (15) and jobs (8). We prepared 2 to 5 tasks per website depending on how many actions the site supports. Ninety-six tasks retrieve information, and four change state on the remote site and are never retried because a first attempt may already have modified a cart.
Evaluation
Each task and its pass conditions were written in English. The scorer reads the rule file and scores only when every rule that applies to it holds.
Blocked means the provider could not reach the site and said so. The answer came back as an error message instead of data.
Bypassed means the agent fetched the page directly rather than using the provider interface, typically via a direct request from the shell. We find these in the session logs. A bypassed task earns nothing and stays in the 100-task total.
“Not Determined” cells are those in which the scorer could not decide whether the answer was correct. These were counted neither correct nor blocked. This covers 8 to 18 tasks per interface per run, all from the same 27 tasks.
The same tasks, instructions, and model were used across all seven interfaces, and the rules were locked before the first run.
Cost
The chart adds two numbers. Model cost is what the agent spent on tokens, MiniMax M3 through OpenRouter at $0,30 per million input tokens, $1,20 per million output and $0,06 per million cache reads, counted from the scored attempt of each task.
Interface cost is what the provider charged for the same work, averaged over five runs:
- Bright Data: read from the account’s billing per zone per day. Scraping Browser is $8 per GB plus $0,10 per hour, Web Unlocker $1,50 per 1K requests, SERP $1,50 per 1K.
- ZenRows: from their panel, browser session bandwidth plus fetch credits as logged. The plan is EUR 129.99 per 1M credits; browser works out to about 40.500 credits per GB, and fetch averaged 14 credits per request after the JS render and premium proxy multipliers.
- Oxylabs: request counts from the agent transcripts at $1,15 per 1K results.
- Firecrawl: credits counted from the transcripts at $83 per 100K credits, 1 credit per scrape.
- Nimble: requests counted from the transcripts at $1,00 per 1K for extract and $1,10 per 1K for search.
Limitations
Nimble CLI ran out of time on 36.5 percent of its attempts.
All interfaces has different speeds, not all task start or finished at the same time. Some tasks depend on the date or on what is listed at the time, so on those, some providers did not see the same page. Pass rules were written to test the format and the retrieval, not a specific listing, so a changed page does not change the score.
Bu araştırmayı kaynak gösterin
Yayınlayacağınız yere uygun formatı seçin. Bağlantılı sürümü CMS'inize yapıştırmak, geri bağlantıyı korur.
@misc{dilmegani2026,
author = {Dilmegani, Cem and Kalelioğlu, Berk},
title = {{AIM Agentic Web Benchmark}},
year = {2026},
month = sep,
howpublished = {\url{https://aimultiple.com/agentic-web}},
note = {AIMultiple. Erişim tarihi: 21 Eylül 2026}
}
Yorum yapan ilk kişi olun
E-posta adresiniz yayınlanmayacak. Tüm alanlar gereklidir. Yorumlar orijinal dilinde bırakılır.