Agents rely on web interfaces to complete tasks on the web. To measure how interface choice affects task completion, we built the AIM Agentic Web Benchmark and attempted 3.500 tasks (100 tasks completed via 7 web interfaces across 5 runs).
Success rates
A Bright Data interface led every run. The Bright Data CLI retrieved 8 more tasks than the best non-Bright Data interface, which is Firecrawl MCP.
Investigate 10 example tasks on GitHub and see how we run this benchmark on methodology
Consistency across runs
Bright Data MCP has the best run with 52 %. The gap between the best and worst run on Oxylabs MCP is only 5 tasks across 5 runs. It is the most consistent interface in the panel, followed by:
- Bright Data CLI: 7
- Nimble MCP: 7
- Nimble CLI: 9
- Firecrawl MCP: 10
- ZenRows MCP: 11
- Bright Data MCP: 15
Cost and speed
The cost axis adds the agent’s token spend to what the provider charged for the same task. On time the MCP tools beat both CLI tools with no overlap. Working with CLI means the agent shells out, reads text back, and often retries the syntax, and every extra turn carries the whole conversation forward again. Cache reads dominate the cost, so the figure mostly counts how many turns an interface needs before the agent can answer.
Nimble MCP is the lowest cost and the fastest interface in the benchmark at $0,065 per task, with Firecrawl MCP next at $0.08. Bright Data MCP is the fastest, at a median of 186 seconds per task. Nimble CLI had more retries than other tools because of timeouts. A task that runs out of time gets a second attempt, and only that second attempt is scored. Cost and latency come from that same scored attempt, which can itself have hit the limit.
Failure reasons
Bright Data CLI was blocked on average 17.6 times per 100 tasks, the fewest in the panel, while Firecrawl MCP, Nimble MCP and Oxylabs MCP were blocked almost twice as often.
Nimble CLI was blocked least among the four non-Bright-Data interfaces at 18.2, but returned nothing on 28.4 tasks per 100, the highest non-delivery in the panel, which reflects hitting the 30-minute limit on 36.5 percent of its attempts (attempts exceed tasks because timed-out tasks were retried once).
Añadir como fuente preferidaMethodology
Each interface ran on its own VPS. The agent was OpenClaw running MiniMax M3 through OpenRouter, without any parallel task runs. Every server had 2 vCPUs and 2GB of RAM.
The task set used was identical for all seven interfaces on all 5 runs.
Each attempt had 30 minutes. A task that ran out of time got one retry, except the four state-changing tasks, which were never retried. Before each run a probe fetched one low-cost page through the provider account, and the run refused to start if that fetch failed.
Versions used in this benchmark;
Example Tasks
Ten of the 100 tasks are published in full at GitHub, with the instruction the agent was given, the output format, the pass rules, and how each interface scored.
Dataset
The 100 tasks span 29 websites in these categories: e-commerce (24 tasks), travel (21), real estate (17), tickets (15), social (15) and jobs (8). We prepared 2 to 5 tasks per website depending on how many actions the site supports. Ninety-six tasks retrieve information, and four change state on the remote site and are never retried because a first attempt may already have modified a cart.
Evaluation
Each task and its pass conditions were written in English. The scorer reads the rule file and scores only when every rule that applies to it holds.
Blocked means the provider could not reach the site and said so. The answer came back as an error message instead of data.
Bypassed means the agent fetched the page directly rather than using the provider interface, typically via a direct request from the shell. We find these in the session logs. A bypassed task earns nothing and stays in the 100-task total.
“Not Determined” cells are those in which the scorer could not decide whether the answer was correct. These were counted neither correct nor blocked. This covers 8 to 18 tasks per interface per run, all from the same 27 tasks.
The same tasks, instructions, and model were used across all seven interfaces, and the rules were locked before the first run.
Cost
The chart adds two numbers. Model cost is what the agent spent on tokens, MiniMax M3 through OpenRouter at $0,30 per million input tokens, $1,20 per million output and $0,06 per million cache reads, counted from the scored attempt of each task.
Interface cost is what the provider charged for the same work, averaged over five runs:
- Bright Data: read from the account’s billing per zone per day. Scraping Browser is $8 per GB plus $0,10 per hour, Web Unlocker $1,50 per 1K requests, SERP $1,50 per 1K.
- ZenRows: from their panel, browser session bandwidth plus fetch credits as logged. The plan is EUR 129.99 per 1M credits; browser works out to about 40.500 credits per GB, and fetch averaged 14 credits per request after the JS render and premium proxy multipliers.
- Oxylabs: request counts from the agent transcripts at $1,15 per 1K results.
- Firecrawl: credits counted from the transcripts at $83 per 100K credits, 1 credit per scrape.
- Nimble: requests counted from the transcripts at $1,00 per 1K for extract and $1,10 per 1K for search.
Limitations
Nimble CLI ran out of time on 36.5 percent of its attempts.
All interfaces has different speeds, not all task start or finished at the same time. Some tasks depend on the date or on what is listed at the time, so on those, some providers did not see the same page. Pass rules were written to test the format and the retrieval, not a specific listing, so a changed page does not change the score.
Cita esta investigación
Elige el formato que se ajuste al lugar donde vas a publicar. Pegar la versión con enlace en tu CMS conserva el enlace de retroceso.
@misc{dilmegani2026,
author = {Dilmegani, Cem and Kalelioğlu, Berk},
title = {{AIM Agentic Web Benchmark}},
year = {2026},
month = sep,
howpublished = {\url{https://aimultiple.com/agentic-web}},
note = {AIMultiple. Recuperado el 21 de septiembre de 2026}
}
El trabajo de Cem en AIMultiple ha sido citado por publicaciones líderes mundiales como Business Insider, Forbes, Morning Brew y Washington Post, empresas globales como Deloitte y HPE, ONG como World Economic Forum y organizaciones supranacionales como European Commission. [1], [2], [3], [4], [5]
A lo largo de su carrera, Cem trabajó como consultor tecnológico, comprador de tecnología y emprendedor tecnológico. Asesoró a empresas en sus decisiones tecnológicas en McKinsey & Company y Altman Solon durante más de una década. También publicó un informe de McKinsey sobre digitalización.
Dirigió la estrategia tecnológica y las adquisiciones de una empresa de telecomunicaciones reportando al CEO. También lideró el crecimiento comercial de la empresa de deep tech Hypatos, que alcanzó unos ingresos recurrentes anuales de 7 dígitos y una valoración de 9 dígitos desde 0 en 2 años. El trabajo de Cem en Hypatos fue cubierto por publicaciones tecnológicas líderes como TechCrunch y Business Insider.
Cem participa habitualmente en conferencias internacionales de tecnología. Se graduó en Bogazici University como ingeniero informático y tiene un MBA de Columbia Business School.
Sé el primero en comentar
Tu dirección de correo electrónico no será publicada. Todos los campos son obligatorios. Los comentarios se dejan en su idioma original.