Best Twitter (X) Scrapers: Benchmarked

Five web scraping APIs each received the same 1,000 X URLs, 500 profiles and 500 posts, for 5,000 requests in total. We recorded the success rate, completion time and number of parsed metadata fields of every response.

Twitter (X) scrapers benchmark

See our benchmark methodology for more details.

✅ structured JSON, ✓ HTML only, ✕ not supported.

X scrapers pricing

What data can be scraped from Twitter

When scraping X, you get the page either as rendered HTML, which you parse with CSS selectors, or as structured JSON from a dedicated X scraper. In our benchmark only Bright Data returned JSON. Decodo, Oxylabs, Nimble and Zyte returned HTML, and we read the profile and post data from it with CSS selectors.

Bright Data’s X scrapers returned the fields below:

X profiles:

X posts:

The best Twitter scraping tools

Bright Data’s Twitter Scraper API returns X profiles and posts as structured JSON, so no HTML parsing is needed. In our benchmark it returned 24 fields per profile and 34 per post, including view, bookmark and quote counts, embedded media, parent and quoted post details and the verification type.

Bright Data offers four X scrapers:

• Posts, collect by URL: extracts a post’s data from its URL
• Posts, discover by profile URL: collects all post URLs and their data from a profile
• Profiles, collect by URL: extracts profile details from a profile URL
• Profiles, discover by user name: collects profiles by user name 

Bright Data also offers two ready-made X datasets, posts and profiles, with more than 152 million records in total and a $250 minimum order.

We used the profiles and posts scrapers, each collecting by URL. Bright Data succeeded on every profile and on 99.8% of posts; the one failed post had a like count that failed the value check. Profiles took 8.2 seconds and posts 7.0 seconds on average.

We tested Oxylabs on the Realtime endpoint of its Web API, formerly Web Scraper API, with the universal source and JavaScript rendering. It succeeded on 99.6% of both profiles and posts, at 18.3 and 17.4 seconds per request. Each response is a rendered page, and CSS selectors read the profile and post data out of it.

Decodo succeeded on 99.8% of X profiles and 99.6% of posts, with only three failed requests, and averaged 17.6 seconds per request. Our Decodo setup used the Web Scraping API’s generic endpoint, with headless rendering on and requests routed through the premium proxy pool. The response contains no parsed fields, so you write your own CSS selectors for the profile and post data.

Decodo has moved this Scraper API into its Web Data API, which groups /scrape with /search, /crawl and /map under a single key.

Zyte was the fastest provider on X at 6.5 seconds per request: 6.6 seconds for profiles and 6.4 for posts. It succeeded on 99.8% of profiles and 99.2% of posts. Zyte API returns the fully rendered page through browserHtml in a single call, so profile and post data have to be extracted with CSS selectors.

We ran Nimble’s Extract API with browser rendering and HTML output. Nimble succeeded on 21.0% of profiles and 26.2% of posts. In most failed responses the HTML held X’s page shell and scripts but no rendered profile or post, so the required fields were missing. Successful requests averaged 7.8 seconds.

Don’t miss our benchmarks and data-driven insights. The button opens Google; selecting AIMultiple confirms that you wish to see AIMultiple more often in Google search results.

Add as preferred source

Twitter scraper benchmark methodology

The X results are drawn from our social media scraping benchmark, which measured 7 providers on 6,500 pages across Instagram, Facebook, TikTok, X and LinkedIn. Five of those providers each received the same 1,000 X URLs: 500 profiles and 500 posts. Apify and SerpApi were not tested on X.

The social media scraping benchmark describes how we selected the URLs, which endpoint we used for each provider and which checks decide a successful response.

Cite this benchmark

Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.

@misc{sipi2026,
  author = {Şipi, Nazlı},
  title  = {{Best Twitter (X) Scrapers: Benchmarked}},
  year   = {2026},
  month  = oct,
  howpublished    = {\url{https://aimultiple.com/twitter-scraper}},
  note   = {AIMultiple. Retrieved October 22, 2026}
}

Changelog

21 updates

All21General18Methodology3

2026202520242023

1. Jun 24

   Replaced the benchmarked scraper lineup, dropping Apify, NetNut, PhantomBuster and Octoparse for Decodo, Oxylabs and Zyte.

2. Jun 10

   Added a search market share breakdown section for Twitter scrapers and a price comparison section.

3. May 12

   Added an X API pay-per-use pricing subsection to the developer guide section.

4. Feb 5

   Removed the "How to scrape Twitter data" section.

Nazlı Şipi

AI Researcher

Follow On

Nazlı is a data analyst at AIMultiple. She has prior experience in data analysis across various industries, where she worked on transforming complex datasets into actionable insights.

View Full Profile

Be the first to comment

Your email address will not be published. All fields are required. Comments are left in their original language.

Name

Email Address

Comment

0/450

Post Comment