E-Commerce Scraper: 6 Providers Benchmarked

We benchmarked six web data providers across 100 e-commerce domains, fetching 65,000 product and search pages each at 5 to 5,000 concurrent requests.

Response time and success rate

Averaged across concurrency levels, Decodo recorded the fastest median response, about 7 seconds, at a 59% success rate, while Bright Data reached the highest success rate (76%) at a 16-second median. Apify recorded the highest median response time at 46 seconds.

Response time is reported as the median (P50), the typical request, and the tail (P90), the slowest 10% of requests.

See our e-commerce benchmark methodology for how we tested.

E-commerce scraper success rate by concurrency

Bright Data recorded the highest overall success rate. At 5,000 parallel requests, Bright Data held 71% and Nimble 56%.

Concurrency is the number of requests sent at the same time. The 5,000-request level stresses a provider’s rate limits and infrastructure.

Success rate on product and search pages

Every provider scored higher on product (detail) pages than on search (listing) pages, by 5 points (Zyte) to 15 points (Decodo).

• Search (listing) pages return many items from a query or category, often paginated and rendered dynamically, which makes consistent extraction harder.

• Product (detail) pages show a single item with structured fields such as title, price, SKU, and images.

Review of the e-commerce scrapers

Bright Data provides a Web Scraper API paired with a no-code interface, along with a dedicated e-commerce scraper and over 1,000 ready-made scrapers covering marketplaces such as Amazon, eBay, AliExpress, Walmart, and Target. The combination of API access and a no-code control panel makes it usable for both engineering and non-technical teams. In our benchmark, Bright Data recorded the highest overall success rate and was one of the few providers to hold its rate under heavy concurrent load.

Performance:

• Highest overall success rate: 78% across 100 domains.
• Held its rate under load: one of two providers to sustain success at 5,000 concurrent requests (71%).

Oxylabs extracts real-time product, search, and pricing data from e-commerce marketplaces such as Amazon and eBay, returning raw HTML or structured JSON, with a Custom Parser for site-specific extraction rules and OxyCopilot for parsing returned data automatically. In our benchmark, it performed more reliably on product detail pages than on search listing pages.

Performance:

• Strong on product pages: 76% on product (detail) pages and 93% on Walmart.
• Search pages: 58%, an 18-point gap from its product-page rate.
• AI parsing: OxyCopilot parses returned data automatically, without manual coding.

Decodo offers a general-purpose Web Scraping API with 100+ ready-made templates, including an Amazon scraper that returns structured product data such as prices, ASINs, and offers across regional Amazon marketplaces. In our benchmark, it recorded the fastest median response time among the tested providers.

Performance:

• Fastest responses: about a 5-second median response time.
• Entry price: $0.88 per 1,000 requests.
• Coverage varied by site: 96% on Amazon, 20% on Target.

Zyte API is a web scraping API that combines ban handling, headless rendering, and AI extraction capabilities to structure raw HTML into typed data, such as product names, prices, and ratings, from retail product pages. It is accessible over REST and integrates with the open-source Scrapy framework, which Zyte maintains, along with Scrapy Cloud hosting. In our benchmark, it recorded the second-highest overall success rate and stayed consistent across major marketplaces and under elevated concurrency.

Performance:

• Second-highest overall success rate: 73% across 100 domains.
• Consistent across major marketplaces: Amazon 95%, Walmart 97%, Target 96%, eBay 98%.
• Smallest product-search gap: 74% on product pages and 70% on search pages, the narrowest gap between the two of any provider.

Apify runs web scraping through its Store of community-built Actors, including an Amazon Product Scraper that extracts data from a supplied Amazon URL, covering reviews, prices, descriptions, and ASINs, with configurable input options and structured export formats. Custom Actors can return large result sets, with the Amazon scraper documented to return over 100,000 results depending on the input. In our benchmark, it performed strongest on major marketplaces such as Amazon and Best Buy at standard concurrency.

Performance:

• Standard-load success: 72% overall, 96% on Amazon, and 92% on Best Buy.
• Flexibility: custom “Actors” can return 100,000+ results, backed by a large community library.
• Response time and load: 46-second median response time.

Nimble offers a general-purpose web scraping API alongside a dedicated e-commerce API that returns AI- and NLP-structured JSON and supports major marketplaces, including Amazon, Walmart, and Google Shopping. The API includes built-in residential proxies and geo-targeting down to the country, state, city, and ZIP-code level. In our benchmark, Nimble held its success rate under heavy concurrent load better than every provider except Bright Data.

Performance:

• Second-most resilient under load: held 56% success at 5,000 concurrent requests, behind Bright Data.
• Fast: about a 9-second median response time.
• Domain-specific strength: 100% on AliExpress and 90% on Best Buy, 25% on Amazon.

Dedicated scraper coverage by provider

Providers differ in how they extract data. Some ship a dedicated, pre-built scraper for each marketplace that returns the site’s structured fields; others apply one universal engine to any site. Bright Data offered the broadest dedicated coverage across the 100 domains, followed by Oxylabs. Zyte and Apify use a single universal extraction engine rather than a per-marketplace library.

For each provider, the metadata fields column counts the distinct data attributes in a parsed JSON product record, such as price, brand, rating, SKU, or image. The value is the average across the product pages that returned a parsed product record, and it excludes HTTP headers, request metadata, and any field that holds the raw page HTML.

Bright Data’s dedicated scrapers return product-page data; Oxylabs and Decodo offer dedicated scrapers for search and listing pages rather than product pages. Coverage reflects each provider’s catalog at the time of the benchmark.

E-commerce scraper pricing

*Bright Data pricing is per record (one extracted item, e.g. one product or one review), not per request. On search pages, a single request can return multiple records.

Don’t miss our benchmarks and data-driven insights. The button opens Google; selecting AIMultiple confirms that you wish to see AIMultiple more often in Google search results.

Add as preferred source

E-commerce benchmark methodology

Domain selection

We selected the top 15 countries by GDP and excluded Russia and China. For each country, we took the top 5 domains from the last three months of SimilarWeb’s e-commerce category. We added the top 5 global websites in consumer electronics and in grocery. For the United States, which ranked first, we also took the top 20 retail-category domains from Semrush, since public Semrush listings show 20 sites where SimilarWeb shows 5.

We then included the global top 20 retail category and top 20 apparel/fashion category for January 2026. We excluded domains serving countries outside the selected set, sites that required authentication to browse, and sites that required a country selection on the landing page before browsing. We excluded regional paths of global domains (for example, etsy.com/fr) but kept regional sites that had a separate domain. Where an exclusion left a gap in a country’s top 5, we filled it from Semrush’s top 20 e-commerce list for that country. To reach 100 domains, we added 10 selections from top e-commerce marketplaces.

Two single-brand domains, apple.com and consumer.huawei.com, returned a few hundred distinct products even after we crawled every category, below the per-domain target. We replaced them with two high-traffic domains by SimilarWeb traffic, bol.com and argos.co.uk.

URL collection

For each domain, we collected both product and listing URLs, retaining enough spare URLs that every run and load test draws from a fresh set. We discovered every URL without fetching the target page through any benchmarked provider, so those pages remain unused before the benchmark itself.

For product URLs, we used three sources in sequence, stopping when a domain reached its target:

• The site’s own product sitemap.
• A crawl of the site’s own category pages (from the homepage down through category and sub-category pages) when the sitemap was missing or too small, collecting product links while fetching category pages but never the product page.
• Google site-restricted search when a domain was still short.

Every URL was matched against a per-domain product pattern to drop non-product pages, and any domain still below target was flagged for manual review or replaced with a backup.

For search URLs, we generated queries from each domain’s own catalog rather than an external word list. The keywords came from the site’s category names and product slugs, so they are in the site’s own language (a Japanese or Korean site is queried in its own language, with no separate translation step), and were optionally generalized to plain category terms such as “running shoes”. We ran the queries against the site’s search endpoint, or its category pages where no keyword search exists, and kept the queries that returned several distinct products. All URLs were then normalized to one locale per domain, stripped of tracking parameters, and de-duplicated.

URL validation

Because the benchmark fetches every URL with every provider, validating the full set would consume the unused URLs, so we validated a sample of about a dozen of each page type per domain, roughly 2,500 URLs. We confirmed that product pages were live and showed a price, and that listing pages were live and returned several distinct products rather than an empty page or a single product. We removed staging and QA subdomains and mixed locales, and excluded every sampled URL from the delivered set.

Benchmark execution

Each provider fetched the same fresh URLs, with an equal number of pages drawn from every domain on each run. Where a provider offered a dedicated e-commerce scraper for a domain, we used it; otherwise, we used the provider’s general web unblocker. All providers ran from the same server location, so the location did not give any provider an advantage.

To measure behavior under load rather than best-case behavior, we ran the full set at three concurrency levels of 5, 100, and 5,000 parallel requests. An HTTP 200 status did not count as a success on its own. For every URL, we defined the expected data in advance, then checked the returned page against it: a response counted as a success only when its content matched that ground truth through a CSS selector or a structured (JSON) field carrying the target data (a price and product fields on a product page, several distinct products on a listing page).

A block page, a captcha, or an empty shell scored zero even with a 200 status. Every chart in this article uses this content-verified success rate. We logged the HTTP status and the end-to-end response time alongside it, and report response time as the median (P50) and tail (P90) of successful requests.

Only Bright Data and Nimble sustained a full run at the 5,000-request level; the other four providers were limited by account-level concurrency or credit ceilings at that load, not by scraping capability, so they are not shown at that concurrency.

Is e-commerce scraping legal?

It is legal to collect public information such as product names, prices, and stock status, as long as you do not bypass logins or access private data.

However, new AI rules will apply to some platforms. For instance, starting in 2026, eBay changed its User Agreement to ban “LLM-driven bots” and “buy-for-me agents” from using its platform without written permission. This update responds to the rise of “agentic commerce.”¹

While general scraping is still legal, using AI agents to interact with marketplaces is now subject to stricter rules.

FAQs

Cite this benchmark

Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.

@misc{dogan2026,
  author = {Dogan, Sedat and Sarı, Ekrem},
  title  = {{E-Commerce Scraper: 6 Providers Benchmarked}},
  year   = {2026},
  month  = jul,
  howpublished    = {\url{https://aimultiple.com/ecommerce-scraper}},
  note   = {AIMultiple. Retrieved July 13, 2026}
}

Reference Links

1.

Security Measure | eBay

Sedat Dogan

CTO

Follow On

Sedat is a technology and information security leader with experience in software development, web data collection and cybersecurity. Sedat:
- Has ⁠20 years of experience as a white-hat hacker and development guru, with extensive expertise in programming languages and server architectures.
- Is an advisor to C-level executives and board members of corporations with high-traffic and mission-critical technology operations like payment infrastructure.
- ⁠Has extensive business acumen alongside his technical expertise.

View Full Profile

Technically reviewed by

Ekrem Sarı

AI Researcher

Follow On

Ekrem is an AI Researcher and Data Analyst at AIMultiple. He designs and runs hands-on benchmarks for AI and LLM systems.

View Full Profile

Be the first to comment

Your email address will not be published. All fields are required. Comments are left in their original language.

Name

Email Address

Comment

0/450

Post Comment