Cloudflare introduces a consent-based approach for AI crawlers, giving website operators more power over their content. Media companies and platforms hope it will create fairer terms for original content online.
Cloudflare will now, by default, prompt owners of new websites to block AI crawlers that try to access content without explicit permission. The company says it's trying to restore balance between content creators and AI companies.
Under the new system, AI companies can disclose if they're collecting data for training, inference, or search. Website operators can then decide whether to grant access.
Cloudflare argues that the current model - where AI systems scrape content automatically and use it without attribution or payment - is no longer acceptable.
"Original content is what makes the Internet one of the greatest inventions in the last century, and it's essential that creators continue making it," said CEO Matthew Prince in a press release.
