Share on LinkedInShare on LinkedIn

ARTICLE · 14 JULY 2025

Cloudflare's New Economic Model To Address AI Web Scraping

Frankfurt Kurnit Klein & Selz PC
Frankfurt Kurnit Klein & Selz PC
Contributor
Frankfurt Kurnit Klein & Selz PC

Frankfurt Kurnit Klein & Selz PC

Founded more than 40 years ago as a boutique law firm servicing the entertainment and arts...

View firm profile
Explore more from Frankfurt Kurnit Klein & Selz PC

Last week, Cloudflare released a private beta version of "pay-per-crawl," a technical solution designed to help websites using Cloudflare automatically...

United StatesTechnology

Last week, Cloudflare released a private beta version of "pay-per-crawl," a technical solution designed to help websites using Cloudflare automatically block known AI crawlers unless domain owners explicitly opt in. This solution offers publishers an off-the-shelf method to manage and collect revenue from web crawlers, which is especially important in the age of AI.

Background

The development and training of AI models requires substantial amounts of input data, much of which is collected through public sources. To collect this data, AI model and data providers rely on web crawlers to scrape and index internet content. Creators and publishers have increasingly shared concerns about how their content will be used and monetized, resulting in litigation and industry efforts.

We operate a free-to-view policy, asking only that you register in order to read all of our content. Please login or register to view the rest of this article.

www.fkks.com

This post first appeared in Frankfurt Kurnit's Focus on the Data blog (www.focusonthedata.com). It provides general coverage of its subject area. We provide it with the understanding that Frankfurt Kurnit Klein & Selz is not engaged herein in rendering legal advice, and shall not be liable for any damages resulting from any error, inaccuracy, or omission. Our attorneys practice law only in jurisdictions in which they are properly authorized to do so. We do not seek to represent clients in other jurisdictions.

Originally published by the New York Legal Ethics Reporter

www.fkks.com

This alert provides general coverage of its subject area. We provide it with the understanding that Frankfurt Kurnit Klein & Selz is not engaged herein in rendering legal advice, and shall not be liable for any damages resulting from any error, inaccuracy, or omission. Our attorneys practice law only in jurisdictions in which they are properly authorized to do so. We do not seek to represent clients in other jurisdictions.

www.fkks.com

This alert provides general coverage of its subject area. We provide it with the understanding that Frankfurt Kurnit Klein & Selz is not engaged herein in rendering legal advice, and shall not be liable for any damages resulting from any error, inaccuracy, or omission. Our attorneys practice law only in jurisdictions in which they are properly authorized to do so. We do not seek to represent clients in other jurisdictions.

See more popular content from