Is Web Scraping Legal? Laws & Best Practices
Legal regulations have changed in the web scraping market. While litigation once focused on unauthorized access, new lawsuits related to AI training and technical workarounds are shaping acceptable practices.
Disclaimer: Our work is for informational purposes and not legal advice; please get professional legal advice for specific guidance.
Is web scraping legal?
Scraping publicly available data is often legal, but no general rule makes it legal everywhere. Legality depends on how, what, and why you scrape, and on the jurisdiction.
In the EU, providers of general-purpose AI models must respect rights reservations, such as machine-readable opt-outs, when collecting training data. These obligations under the AI Act have applied since August 2, 20251
Web scraping can be legal when you:
- Prioritize logged-out scraping: Scrape publicly available data from webpages accessible without a login, subscription, or payment.
- Avoid technical circumvention: Do not bypass CAPTCHAs, login walls, rate limits, or IP blocks, and respect the website’s terms of service, robots.txt file, and copyright.
- Align with commercial use policies: Ongoing lawsuits over AI training, such as Reddit’s suits against Anthropic and Perplexity, are testing contract, anti-circumvention, and copyright claims against AI developers.
- Comply with global privacy laws: Don’t collect personal or sensitive data, such as names or contact information, in a manner that violates privacy laws, including the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA).
For more on ethical data collection, check out our ethical & compliant web data benchmark.
Latest web scraping legal updates
Though web scraping can be legal, being scraped is not desired by companies. If these platforms can show that being scraped by a bot damages their infrastructure or operations, then that activity may be found illegal by the court.
Below are significant scraping lawsuits, mostly from the U.S. Outcomes vary: in some, courts sided with the scraped website; in others, with the scraper.
Reddit vs. Perplexity AI & scraping services
Court: U.S. District Court for the Southern District of New York
Timeline: October 2025 – Present (Active Case)
Reddit sued the AI search engine Perplexity AI and three major scraping/proxy providers (SerpApi, Oxylabs, AWMProxy) for industrial-scale data collection and bypassing technical barriers. 2
Conflict:
Reddit alleges that the defendants engaged in a “bank robbery-style” scheme to steal copyrighted content. Instead of entering into licensing agreements (like OpenAI and Google), Perplexity used specialized scraping tools to bypass Reddit’s defenses.
Legal arguments:
- Indirect scraping via Google: Defendants bypassed Reddit’s own blocks by scraping Reddit’s content directly from Google Search Results (SERPs).
- DMCA violations: Unlike previous “public data” cases (such as hiQ), Reddit is invoking the Digital Millennium Copyright Act (DMCA) Section 1201. They argue that the defendants didn’t “access” data, but purposefully bypassed “technological measures” (rate limits, captchas, and SearchGuard).
- Refusal to license: Reddit highlights that while other AI giants pay for data access, Perplexity increased its scraping volume 40-fold after receiving a cease-and-desist letter, choosing “circumvention over cooperation.”
Current status:
As of late 2025, the case is ongoing, and no final ruling has been issued.
Reddit vs. Anthropic
Court: Superior Court of California in San Francisco
Timeline: June 2025 – Present (Active Litigation)
Reddit sued Anthropic in June 2025, alleging that Anthropic accessed Reddit and used its content to train AI models without a licence, in breach of Reddit’s user agreement. Reddit’s claims are state-law claims, including breach of contract, unjust enrichment, and unfair competition, rather than copyright claims. Anthropic disputes the claims.
Current status:
As of late 2025, there has been no final court ruling. The case is currently in the pre-trial discovery phase. Anthropic has moved to have parts of the case dismissed, arguing that factual data is not copyrightable.
Linkedin vs hiQ Labs Case
Court: U.S. District Court / Ninth Circuit Court of Appeals
Timeline: 2017–2022
LinkedIn sued hiQ Labs, a data analytics company, for scraping publicly available profiles to conduct a professional skill analysis.3 Several courts, including the Supreme Court, reviewed the case:
- The Ninth Circuit initially sided with hiQ, holding that scraping publicly available profiles likely does not violate the Computer Fraud and Abuse Act (CFAA)4
- In 2021, the Supreme Court vacated that ruling and sent the case back for reconsideration in light of Van Buren v. United States. In 2022, the Ninth Circuit reaffirmed its view that scraping publicly accessible data likely is not access “without authorization” under the CFAA.
However, in late 2022 the district court held that hiQ had breached LinkedIn’s User Agreement, and the parties settled with a judgment against hiQ. The case shows that even where the CFAA does not apply, breaching a website’s terms can still create liability.
Meta vs Bright Data
Court: U.S. District Court for the Northern District of California
Timeline: 2023–2024
Case Type: Civil lawsuit involving breach of contract and unauthorized data scraping
In January 2023, Meta initiated a lawsuit against Bright Data, alleging that it had illegally extracted data from Meta’s Facebook and Instagram platforms. Interestingly, Bright Data contested Meta’s claims about its data scraping rights, leading both parties to court.
The court ruled in favor of Bright Data, finding insufficient evidence to show that Bright Data had scraped nonpublic data or accessed data while logged into user accounts. In February 2024, Meta decided to drop the remaining claims against Bright Data.5
Does Meta (Facebook/Instagram) prohibit all automated data collection?
If you’ve read the Instagram terms of use, you’ve likely seen the clause stating that ‘scraping by automated means is prohibited.’
However, the legal reality is more complex. In the landmark Meta v. Bright Data (2024) case, the court ruled that if you are scraping public data while logged out, Meta’s terms do not necessarily apply because you never signed a contract by logging in.
Many websites include a Facebook terms, automated data collection, scraping prohibited’ warning. But as seen in recent web scraping legal updates, courts are increasingly distinguishing between data behind a login wall and data available to the open web.
X Corp. vs Bright Data
Court: U.S. District Court for the Northern District of California
Timeline: 2023–ongoing
Case Type: Unauthorized data access under computer fraud statutes, intellectual property violations
In July 2023, X Corp. filed a lawsuit against Bright Data, alleging that Bright Data violated its terms of service by scraping and selling vast amounts of data from the X platform. 6The legal action in California was about Bright Data’s access to public data on Twitter.
In May 2024, the court dismissed X’s claims, holding that its state-law claims over the scraping of public posts were preempted by federal copyright law, since X does not own the content its users post.
eBay vs Bidder’s Edge Case
Court: United States District Court for the Northern District of California
Timeline: 1999–2000
Case type: Civil lawsuit for trespass to chattels, in which eBay accused Bidder’s Edge of unlawfully scraping its site using automated data collection bots.
Bidder’s Edge (BE), an online price comparison website, used web scraping tools to aggregate auction listings from various platforms, including eBay, without permission. eBay claimed that BE’s automated bots caused unauthorized use of its systems.
The court order was preventing Bidger’s Edge from scraping eBay content again. The main argument eBay won was that Bidger’s Edge was overloading their system, and that others following Bidger’s Edge could cause further harm to eBay’s system.
Facebook vs Power Ventures Case
Court: U.S. District Court for the Northern District of California
Later, it appealed to the U.S. Court of Appeals for the Ninth Circuit
Timeline: 2008–2017
Case Type: Civil lawsuit under the CFAA and California’s anti-hacking law, with Facebook alleging unauthorized access to its platform.
In 2008, Facebook sued Power Ventures, which aggregated users’ social network data and used their Facebook accounts to send promotional messages. In 2016, the Ninth Circuit held that Power violated the CFAA and California’s anti-hacking law by continuing to access Facebook after receiving a cease-and-desist letter and having its IP addresses blocked. Power Ventures was ordered to pay damages.7
Latest regulations on web scraping by country
United States
Legal Status: No single federal law bans web scraping, but several laws can apply.
Depending on how and what you scrape, the CFAA, the DMCA’s anti-circumvention provision (Section 1201), copyright law, state contract and trespass-to-chattels claims, and state privacy laws such as the CCPA may apply. The BOTS Act of 2016 specifically prohibits using bots to circumvent ticket-purchasing limits.8
European Union and the UK
Legal Status: No single EU law governs web scraping. Its legality depends on copyright, database rights, data protection law, and contract terms.
Copyright and text and data mining: The Directive on Copyright in the Digital Single Market (Directive (EU) 2019/790) introduced two text and data mining (TDM) exceptions. Article 3 allows research organisations and cultural heritage institutions to carry out TDM for scientific research on works they have lawful access to. Article 4 allows anyone with lawful access to carry out TDM for any purpose, including commercial use, unless the rightsholder has expressly reserved their rights. For content published online, the reservation must be made in an appropriate manner, such as machine-readable means. These exceptions cover reproductions made for TDM; they do not make all reproduction of publicly available content lawful. Because this is a directive, each member state has transposed it into national law, and details vary. 9
Database rights: The Database Directive (96/9/EC) gives database makers a sui generis right that can prohibit extracting or reusing all or a substantial part of a database, even when individual items are not protected by copyright.
Personal data: The GDPR applies to scraping personal data, even when the data is publicly available. It does not ban such scraping outright, but the scraper needs a lawful basis, often legitimate interests, which requires a balancing test. The scraper must also generally inform the people whose data it collects and limit collection to what is necessary.
AI training: Under the AI Act (Regulation (EU) 2024/1689), providers of general-purpose AI models must put in place a policy to comply with EU copyright law, including identifying and respecting rights reservations made under Article 4(3) of the DSM Directive (Article 53(1)(c)). 10
United Kingdom: The UK did not implement the DSM Directive. Its TDM exception (section 29A of the Copyright, Designs and Patents Act 1988) covers only non-commercial research by people with lawful access. The UK GDPR and the UK database right apply in parallel.
Dos and don’ts of legal and ethical web scraping
From a legal standpoint, one question businesses should ask themselves is whether their scraping acts harm the scraped website. If the scraping activity:
- It is too intense, which can interrupt the services of the scraped website
- The scraped data is used to duplicate the activity or service of that website, even though no regulations exist.
The website would have grounds to file a lawsuit against the scraper.
From an ethical standpoint, given that web scraping has many use cases and professional providers in the market, there is no shame in using it for business purposes. There are technical web scraping best practices that will ease the traffic load on the scraped website, such as:
- Using the website’s APIs rather than web scraping, when available.
- Scheduling crawls during off-peak hours.
- Caching pages to avoid repeated requests.
As long as you find a trusted web scraper to work with or make sure your technical resources consider these, you can defend your web scraping as ethical for your business purposes.
Dos:
- Scrape the data you need by defining the exact business case and customizing your web crawler technology accordingly. This will minimize your risk of exhausting the scraped website with unwanted traffic.
- Always read the terms of use of the scraped website. In addition to commercial terms of use, websites also have a robots.txt file that specifies permissions for the website’s content. Your web crawling solution or technical experts should help you comply with these permissions.
- Be transparent about your web scraping and be ready to explain your scraping process to assure others that your approach is legal and ethical.
Don’ts:
- Do not exhaust the scraped website too often and with too extensive pulls. This will also increase the likelihood that the scraped website will block your crawler.
- Do not collect personal data unless you have a lawful basis under applicable privacy laws such as the GDPR; robots.txt permissions do not provide one. If you do collect it, minimise and pseudonymise it.
- Do not expose the scraped data to the public. Make sure that it is stored securely, like your own company data. You never know what purposes it may be used for if it is leaked.
Organizations for ethical web scraping
Leading web data infrastructure companies have formed associations to align their industry and stakeholders on the ethical use of web scraping. These associations are:
- Alliance for Responsible Data Collection, which includes Bright Data and Common Crawl among other stakeholders.
- Ethical Web Data Collection Initiative (EWDCI), which includes Oxylabs, ProxyEmpire, Zyte, among others.
Is scraping data for AI training legal?
The legal status of data scraping depends on the type of data, its location, and the methods used to access it. Many relevant laws are being interpreted and established by courts.
In the United States, the Ninth Circuit held in hiQ v. LinkedIn that scraping data publicly accessible without a login likely does not violate the CFAA. That ruling relied in part on the Supreme Court’s narrow reading of “exceeds authorized access” in Van Buren v. United States (2021), a case that did not involve scraping. Separately, in Meta v. Bright Data, the court found that Meta’s terms did not bar logged-off scraping of public data, a contract ruling rather than a CFAA one.
Whether using scraped data to train AI models infringes copyright is a separate question that courts in the U.S. and elsewhere are still deciding.
Add as preferred sourceFAQs
If a website’s terms of service (ToS) explicitly prohibit scraping, accessing, or collecting data from that site through automated means, doing so may constitute a violation of those terms.
Violating terms of service alone is generally treated as a contract matter rather than a CFAA violation. However, continuing to access a site after permission has been explicitly revoked, for example through a cease-and-desist letter or IP blocking, can create CFAA risk, as in Facebook v. Power Ventures. You can contact the site owner to request permission or use official APIs.
Not by itself. Courts treat terms-of-service violations as a civil contract matter, not a criminal offense. However, a violation can support breach-of-contract claims and strengthen claims under other laws, particularly after explicit notice, such as a cease-and-desist notice.
Cite this research
Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.
@misc{karatas2026,
author = {Karatas, Gulbahar},
title = {{Is Web Scraping Legal? Laws & Best Practices}},
year = {2026},
month = oct,
howpublished = {\url{https://aimultiple.com/is-web-scraping-legal}},
note = {AIMultiple. Retrieved October 26, 2026}
}Changelog
2 updatesAdded an "Is scraping data for AI training legal?" section and EU AI-training scraping guidance to the FAQs.
Reference Links
Comments 1
Share Your Thoughts
Your email address will not be published. All fields are required. Comments are left in their original language.
Thank you for the great and well-written articles. Can you write an article explaining the limits and/ or usefulness of using a website’s APIs rather than web scraping, when available. Instagram & TikTok website APIs for example are limited to what type of data can be extracted. My understanding is that not everything can be scrapped using their websites API. Looking forward to your response. Thank you.