Best Expedia Scrapers: Bright Data, Oxylabs & Decodo

To compare how well web scraping tools handle Expedia’s CAPTCHA challenges, dynamic JavaScript rendering, and aggressive bot detection, we tested 5 leading web data scrapers across 2,500 requests and tracked each provider’s success rate and completion time.

Expedia scraping benchmark

For more details on our testing process, you can read our benchmark methodology.

Top 5 Expedia scraper APIs

Bright Data had the highest success rate in the Expedia benchmark at 99%, and also the fastest mean completion time at 12 seconds.

We sent Expedia URLs through Bright Data Web Unlocker zone and got back rendered HTML. The unblocker handled CAPTCHA challenges and bot detection on its own, no extra configuration needed.

Oxylabs sits in the middle with 85% success and a mean completion time of 25 seconds. Expedia URLs went through the Realtime Web Scraper API using the universal source with render: html for JavaScript execution. Most of the 75 failed requests returned HTTP 200 but with Expedia’s generic “Shop travel” template instead of the hotel detail page, which is a soft redirect rather than an outright block. A couple of others hit HTTP 408 timeouts from the realtime endpoint on heavier pages.

For Decodo, we used the Web Scraper API v2 with target: universal and headless: html to get JavaScript-rendered HTML back. Results came in close to Oxylabs: 78% success with a mean completion time of 27 seconds. The 109 unsuccessful requests almost all returned HTTP 200 but with HTML missing the hotel-page CSS selectors, which is the same soft-redirect pattern Oxylabs ran into. Expedia returns a different template instead of the actual hotel page.

For Zyte, we used the Extract API with browserHtml: true. Expedia’s hotel pages are heavily JavaScript-driven, so a plain HTTP request returns mostly empty markup. We needed Zyte to run each page through a real headless Chromium and wait until JavaScript built the hotel details before capturing the HTML. That wait is what pushed completion times up to a mean of around 67 seconds, the longest in the benchmark.

Zyte’s success rate landed at 95%. The 22 failures all returned HTTP 520 (“Website Ban”), which is what Zyte sends after several rotation attempts can’t return content from the target without hitting bot detection. We experimented with extra actions like waitForSelector to give the page more time, but in our earlier tests those extra waits actually increased the 520 rate, since the longer the browser stayed open on Expedia, the more bot signals it sent. We kept the simpler browserHtml: true setup for the final run.

Nimble had the lowest success rate at 23%, mostly because over half of the requests returned HTTP 500 (“can’t download the query response”) while the headless browser was rendering Expedia.

We configured the Extract API with browser rendering enabled and the vx10 stealth driver.

Expedia scraping challenges

Expedia is one of the harder large sites to scrape reliably, with strong bot detection, heavy client-side rendering, and a UI framework that overlaps across page types. Here are the specific issues we ran into during the Expedia scraping benchmark.

CAPTCHA and bot detection

Expedia returns an HTTP 429 with a Cloudflare-style challenge page on direct requests. Providers without a real headless browser and a clean proxy pool can’t get past it. In Expedia scraping benchmark, this is what Zyte’s 22 HTTP 520 “Website Ban” responses came from.

Soft redirects to a generic template

Expedia often returns HTTP 200 with a generic “Shop travel” page instead of the requested hotel detail. The response looks successful but the content is wrong. Validation counts it as a pass; we caught it by requiring hotel-specific CSS selectors to match.

Heavy JavaScript rendering

Hotel data only appears after JavaScript executes. Plain HTTP fetches return mostly empty markup. Zyte’s average of 67 seconds came from waiting for the full render to finish.

CSS class collisions

Expedia’s uitk- design system is used across the homepage, search, and hotel pages. A provider can land on the wrong page and still match a generic selector. We tightened validation to require at least one hotel-specific match.

To get up to date on enterprise AI and software, follow us:

Cem Dilmegani

Principal Analyst

Suivre

What data you can scrape from Expedia

None of the tested providers returned structured JSON for Expedia; every successful response came back as rendered HTML, which then had to be parsed locally.

From the public Expedia pages, the following types of data can be collected:

• Hotels: hotel name, ID, brand chain, full address, neighborhood, rating score, rating label, review count, individual reviews, descriptions, amenities, photos, check-in/check-out policies
• Pricing and availability: nightly rate, total price, currency, taxes, room types, availability for selected dates
• Flights: route details, airlines, departure and arrival times, fares, number of stops, layovers
• Car rentals: vehicle class, pickup and drop-off locations and times, daily rates, included mileage
• Vacation packages: bundled hotel + flight + car deals, total package price, included components
• Search and listing pages: ranked results per destination, filters, aggregated price ranges, sort order

Expedia scraping benchmark methodology

We benchmarked 5 web scraping providers on Expedia hotel page extraction, with each provider receiving the same list of 500 hotel detail URLs.

Selector setup

All providers returned HTML in this benchmark, so each response was processed through local CSS selectors targeting Expedia’s hotel-detail elements.

Timeout and rate limiting

The execution timeout was 10 minutes. If a provider returned HTTP 429, we waited 30 seconds and retried up to 3 times; anything past that was logged as a failure.

Validation rules

We applied three checks per request.

For submission, the provider had to return an HTTP code in the 200-399 band, or 404. For execution, async jobs had to finish before the timeout without errors; sync providers cleared this step automatically. For validation, the response had to surface at least one of the hotel title, rating score, or rating label as a non-empty value via the CSS selectors above.

When the status code was in the 201-399 band or 404, validation was auto-passed and CSS extraction was skipped, on the assumption that the provider had handled a non-200 response correctly (redirect, page-not-found, etc.). Only HTTP 200 responses went through CSS matching.

After the full run we did a follow-up check on every auto-passed request to make sure none of them were false positives. For each URL, we compared the auto-pass result against the other providers’ outcomes: if another provider had pulled real hotel data from the same URL while this one auto-passed without content, we would have flipped the auto-pass to a failure. In practice, no Expedia URL triggered the flip, since every auto-pass corresponded to a genuinely non-200 response and the dataset contained no 404 URLs.

A run only counted as a full success when submission, execution, and validation all cleared.

Metrics measured

Validation success rate captures how many URLs cleared all three checks.

End-to-end completion time is wall-clock time from sending the request to getting a response, in seconds. Both mean and median are reported.

FAQ

Nazlı Şipi

Chercheur en IA

Suivre

Nazlı est analyste de données chez AIMultiple. Elle possède une expérience préalable en analyse de données dans divers secteurs, où elle a travaillé à transformer des ensembles de données complexes en informations exploitables.

Voir le profil complet

Soyez le premier à commenter

Votre adresse courriel ne sera pas publiée. Tous les champs sont obligatoires.

Nom

Adresse email

Commentaire

0/450

Publier un commentaire