We benchmarked best five web scraping providers on Zillow, one of the top real estate domains, running over 1,250 scrape requests across all providers. Each provider received an identical set of property listing URLs and was evaluated on completion time, success rate, and the number of structured data fields returned per listing.
Zillow scraping benchmark
We also analyzed completion time specifically for tasks where all providers succeeded. The results were nearly identical to the overall completion times, indicating that failed requests did not significantly inflate the average completion time for any provider.
You can read our benchmark methodology section for more details on the testing process.
Output format by provider
- ✅ = supported, returns HTML
- ✅ ✅ = supported, returns structured data
Available metadata fields by provider
Since Apify was the only provider returning structured JSON output, it was the only one with measurable metadata fields. The remaining providers returned rendered HTML, which was parsed locally using CSS selectors and therefore did not produce comparable metadata counts.
Zillow scraping benchmark results
Apify achieved a 100% success rate on Zillow with a completion time of around 17 seconds. It was the only provider returning structured JSON output, which also made it the only one with measurable metadata fields, averaging 185 data points per listing.
Nimble was the fastest provider in the benchmark, completing requests in roughly 12 seconds on average. Its success rate came in at 92%, which kept it in the upper half of the comparison. The combination of speed and reliability made it one of the more balanced results on this domain.
Decodo posted a 97% success rate, the second highest after Apify. However, its completion time was noticeably higher at around 51 seconds, placing it among the slower providers. It achieved this result using a general-purpose scraping configuration, without any Zillow-specific setup.
Zyte reached a 92% success rate with a completion time of roughly 46 seconds. It sat in a similar range to Nimble on success but was considerably slower. The result was steady but did not stand out in either direction.
Oxylabs had the lowest success rate in this comparison at 89%, and also recorded the highest average completion time at around 59 seconds. It used a dedicated Zillow scraping source, but this did not translate into a speed or reliability advantage on this particular dataset.
Benchmark methodology
We tested five web scraping providers (Apify, Decodo, Oxylabs, Nimble, Zyte) on zillow.com.
Dataset
We prepared 250 product page URLs from Zillow. Product pages are individual property listings with details like address, price, beds, baths, square footage, and property history.
All URLs were verified as accessible before the benchmark. URLs followed the standard Zillow detail page format (/homedetails/…/zpid/).
Shared configuration
All providers received identical URLs from the same dataset and were tested under the same conditions:
- Sequential execution: one request at a time, no parallel requests
- Delay between requests: 2 seconds
- Rate limit handling: 30-second wait with up to 3 retries on HTTP 429
- Submission timeout: 300 seconds
- Execution timeout: 600 seconds
- Each URL was tested once per provider
Provider configurations
Apify
Apify used the maxcopell/zillow-detail-scraper actor, which returns structured JSON with parsed fields. No CSS selector parsing was needed. Actor runs were polled at 1-second intervals until status reached SUCCEEDED.
Decodo
Decodo used the Universal Scraper API (target: universal, headless: html), which returns JavaScript-rendered HTML. The response was parsed locally with CSS selectors. All requests included a desktop User-Agent header.
Oxylabs
Oxylabs used the Realtime API with source: zillow and render: html, which returns JavaScript-rendered HTML. The response was parsed locally with CSS selectors.
Nimbleway
Nimble used the Extract API with render: true and driver: vx10 (stealth headless browser). The response was parsed locally with CSS selectors. No domain-specific configuration was applied.
Zyte
Zyte used the Extract API with browserHtml: true, which returns JavaScript-rendered HTML via a headless Chromium browser. The response was parsed locally with CSS selectors. No domain-specific configuration was applied.
Validation
HTTP status check
Before validation, the provider’s HTTP response code is checked first. Responses with status codes between 200-399 and 404 are considered successful submissions and proceed to the validation phase. Any other status code (400, 403, 500, 550, etc.) is treated as a failed submission, and the test is immediately marked as failed without entering the validation phase.
Validation rules
Tests that pass the HTTP status check are validated in the following order:
- 404 detection: If the page content or API error indicates the page no longer exists (“page not found”, “does not exist”, “dead_page”), the test is marked as valid. The provider correctly identified an unavailable page.
- Data extraction (JSON API): For providers returning structured JSON, at least one data field must be present and non-empty, with a valid type depending on the field (string or integer). Fields checked include address, price, beds, baths, and square footage.
- Data extraction (HTML): For providers returning HTML, the response is parsed with Zillow-specific CSS selectors. If at least one selector matches and returns a non-empty value, the test passes.
- Page indicator (HTML only): If no data items were extracted but at least one of the predefined CSS selectors for Zillow matched an element on the page, the test is marked as valid. This confirms the page was rendered and loaded, even if no structured data items were found in the expected containers.
- If none of the above conditions are met, the test fails. Common failure reasons include captcha/bot challenge pages, insufficient JavaScript rendering, proxy connection errors, and crawler errors.
Metrics
Validation success rate: The percentage of tested URLs where the provider returned usable data, calculated as successful tests divided by total tests.
Completion time: The total time from sending the scrape request to receiving validated results, measured in seconds. For async providers, job completion status was polled at 1-second intervals. Reported as the arithmetic mean across all runs in a group.
Available metadata: The number of unique field names returned by the provider across all items in a response. Only applicable to JSON API responses.
Be the first to comment
Your email address will not be published. All fields are required.