Benchmarked the Best Real Estate Scraper APIs
We benchmarked six web scraping providers, across five major real estate domains. The test set included 1,500 property listing URLs, distributed across domains, with 250 URLs per site.
Real estate scraping benchmark results
Overall comparison across all runs
The charts below summarize the benchmark results across all tested providers and domains. They show the relationship between success rate and completion time:
Domain coverage by provider
- ✅ = supported, returns HTML
- ✅ ✅ = supported, returns structured data
- ❌ = Included in benchmark but 0% success
Apify does not offer dedicated actors for Realtor, Rightmove, and Realestate.au, so these domains were excluded from Apify’s benchmark.
See the methodology section for more details on the testing process.
Domain-level benchmark results
Completion time for successful tasks vs. overall success rate
Available metadata fields by provider
Pros & cons of the top 6 real estate scrapers
Bright Data was one of the more consistent providers in the benchmark. Its stronger results appeared on Zillow, Realestate.com.au, and Rightmove. On Zillow, Bright Data reached roughly 100% success, although it was slower than the fastest providers on that chart.
On Realestate.com.au, it again posted a high success rate, at around 95%, placing it among the stronger results on that domain. On Rightmove, it combined a high success rate with very low latency, making it one of the most efficient results in that chart.
The weaker result appeared on Redfin, where Bright Data was very fast but less reliable than on its stronger domains, with success around 78%. On Realtor, it remained competitive, with moderate latency and a success rate around 80%, but it did not lead the field there.
The provider performed well across the full set of real estate domains without showing a major collapse on any one site. It was not the top result on every chart, but it stayed in the stronger half of the comparison throughout.
Oxylabs showed a mixed performance profile across the real estate benchmark. Its results were strong in some domains but much weaker in others, making it one of the more domain-dependent providers in the set.
On Realtor and Rightmove, Oxylabs performed well. It combined relatively low latency with high success rates, placing it among the stronger providers on those domains. On Realestate.com.au, success rates were also high, although completion times were noticeably slower. This suggests that Oxylabs can maintain reliable extraction for some targets, even when it is not among the fastest options.
Oxylabs’ stronger results came on Realtor and Rightmove, while Zillow and Redfin were materially weaker. For teams evaluating real estate scraping providers, Oxylabs is a more domain-specific option than a broadly stable choice across all five sites.
On Zillow, Decodo achieved a high success rate of roughly 97%, but latency was also high at around 51 seconds, making it much slower than the stronger providers. On Rightmove, it again posted a usable success rate, around 89%, but was still slower than most of the field.
Decodo did not perform as evenly or as strongly as the leading providers in this comparison. It was able to complete extraction on some domains, but often at higher latency, and it struggled more noticeably on the harder sites.
On Zillow, Apify combined a very high success rate with relatively low latency. It reached roughly 100% success at around 18 seconds, which placed it among the strongest results on that chart.
On Redfin, the result is even more important. Redfin was one of the more difficult domains in the benchmark, and several providers saw a clear drop in success there.
Apify performed better than the rest of the field on that chart, reaching about 88% success at roughly 14 seconds. It was not the fastest provider on Redfin, but it delivered the strongest combination of reliability and latency.
Zyte’s stronger results appeared on Rightmove, Zillow, and Realestate.com.au. On Rightmove, Zyte reached one of the highest success rates in the benchmark, at roughly 93%, while keeping latency in a reasonable range.
On Zillow, it also maintained a solid success rate of around 92%, although completion times were slower than those of the faster providers on that chart.
On Realestate.com.au, Zyte was relatively fast and still achieved a usable success rate, though it did not reach the top group for reliability. The weaker results appeared on Redfin and especially on Realtor.
Nimble’s main strength was consistency. It remained within a workable range across all five real estate domains and did not show any domain where performance collapsed.
Nimble’s stronger results appeared on Zillow, Redfin, and Rightmove. On Zillow, Nimble combined low latency with a high success rate, placing it among the stronger results on that chart.
On Redfin, it again held up reasonably well, with success around 79% and faster completion times than several competitors. On Rightmove, Nimble was also strong, achieving roughly 92% success with moderate latency, keeping it close to the top group in that domain.
On Realestate.com.au, Nimble remained relatively fast, but its success was somewhat lower than that of the strongest providers on that chart. The same pattern appeared on Realtor, where it stayed competitive, but did not clearly separate itself from the middle of the field.
Real estate scraper methodology
We tested six web scraping providers (Apify, Bright Data, Decodo, Oxylabs, Nimble, Zyte) on five real estate domains: zillow.com, redfin.com, realtor.com, rightmove.co.uk, and realestate.com.au.
Dataset
We prepared 1,250 product page URLs across 5 domains (250 per domain). Product pages are individual property listings with details like address, price, beds, and baths.
The five domains were selected as the top global real estate websites based on SimilarWeb traffic rankings.
All URLs were verified as accessible before the benchmark. Invalid redirects, expired listings, and non-standard subdomains were removed or corrected during dataset preparation.
Shared configuration
All providers received identical URLs from the same dataset and were tested under the same conditions:
- Sequential execution: one request at a time, no parallel requests
- Delay between requests: 2 seconds
- Rate limit handling: 30-second wait with up to 3 retries on HTTP 429
- Submission timeout: 300 seconds
- Execution timeout: 600 seconds
- Each URL was tested once per provider
Provider configurations
Bright Data
Bright Data used two integration methods depending on the domain. For Zillow and Realestate AU, we used the Dataset API, which returns structured JSON with parsed fields. For Redfin, Realtor, and Rightmove, we used a web unblocker that returns rendered HTML, which we parsed locally with CSS selectors.
The Dataset API was polled via the /progress/{snapshot_id} endpoint at 1-second intervals until the status reached ‘ready’. Results were then fetched from the /snapshot/{snapshot_id} endpoint.
Apify
Apify used pre-built actors specific to each supported domain. For Zillow, we used maxcopell/zillow-detail-scraper. For Redfin, we used tri_angle/redfin-detail. Apify does not support Realtor, Rightmove, or Realestate AU in this benchmark.
Apify actors return structured JSON, so no CSS selector parsing was needed. Actor runs were polled at 1-second intervals until the status reached SUCCEEDED.
Decodo
Decodo used the Universal Scraper API (target: universal, headless: HTML) for Zillow and Redfin. For Realtor, Rightmove, and Realestate AU, we used the web unblocker with the X-SU-Headless: HTML header for JavaScript rendering. All requests included a desktop User-Agent header.
Oxylabs
Oxylabs used a dedicated source API for Zillow (source: zillow) with render: html. For Redfin, Realtor, Rightmove, and Realestate AU, we used the Web Unblocker proxy. Unblocker requests included a desktop User-Agent header. JavaScript rendering was not enabled on the unblocker.
Nimbleway
Nimbleway used the Extract API for all domains with render: true and driver: vx10 (stealth headless browser). All requests returned rendered HTML, which we parsed with CSS selectors. No domain-specific configuration was applied.
Zyte
Zyte used the Extract API for all domains with browser HTML: true, which returns JavaScript-rendered HTML via a headless Chromium browser. No domain-specific configuration was applied.
Scraping methods by domain
- JSON API: Provider returns structured JSON with parsed fields (address, price, beds)
- HTML API: Provider returns JavaScript-rendered HTML via its scraping API, which is parsed locally with CSS selectors.
- Unblocker: Provider routes the request through a proxy to the target site, returns raw HTML, and parses it locally with CSS selectors.
Validation
HTTP status check
Before validation, the provider’s HTTP response code is checked first. Responses with status codes between 200 and 399 and 404 are considered successful submissions and proceed to the validation phase. Any other status code (400, 403, 500, 550) is treated as a failed submission, and the test is immediately marked as failed without entering the validation phase.
Validation rules
Tests that pass the HTTP status check are validated in the following order:
- 404 detection: If the page content or API error indicates the page no longer exists (“page not found”, “does not exist”, “dead_page”), the test is marked as valid. The provider correctly identified an unavailable page.
- Data extraction (JSON API): For providers that return structured JSON, at least one data field must be present and non-empty, with the field type depending on the field (string or integer). Fields checked include address, price, beds, baths, title, name, rating, and reviews.
- Data extraction (HTML): For providers that return HTML, the response is parsed using domain-specific CSS selectors. If at least one selector matches and returns a non-empty value, the test passes.
- Page indicator (HTML only): If no data items were extracted, but at least one of the predefined CSS selectors for that domain matched an element on the page, the test is marked as valid. This confirms the page was rendered and loaded, even if no structured data items were found in the expected containers.
If none of the above conditions are met, the test fails. Common failure reasons include CAPTCHA/bot challenge pages, insufficient JavaScript rendering, proxy connection errors, and crawler errors.
Metrics
- Validation success rate: The percentage of tested URLs where the provider returned usable data, calculated as successful tests divided by total tests.
- Completion time: The total time from sending the scrape request to receiving validated results, measured in seconds. For async providers, job completion status was polled every 1 second. Reported as the arithmetic mean across all runs in a group.
- Available metadata: The number of unique field names returned by the provider across all items in a response. Only applicable to JSON API responses.
Be the first to comment
Your email address will not be published. All fields are required.