Developer Offer
Try ImaginePro API with 50 Free Credits
Build and ship AI-powered visuals with Midjourney, Flux, and more — free credits refresh every month.
Cloudflare Lets Creators Charge AI Bots For Content
The Creator Dilemma in the Age of AI
Imagine running a regional news website. You've spent years building an archive of valuable local content. Then, one day, an AI bot scrapes three years of your work in just a few minutes, giving you no payment and minimal referral traffic. As a small publisher, getting the attention of a large AI company for a licensing deal can feel impossible. You're left with two bad choices: block all AI crawlers and miss out on potential visibility, or let them take your content for free.
It's a stalemate that many content creators are facing today. But what if there was a third option?

Introducing AI Crawl Control
Cloudflare is stepping in to break this deadlock. We are excited to announce that our AI Audit tool has been rebranded as AI Crawl Control and is now generally available to all customers. This change reflects the tool's evolution from a simple monitoring dashboard to a powerful suite for gaining detailed insights and, more importantly, control over how AI systems access your content. The response from the market has been clear: creators need real agency, not just visibility. AI Crawl Control is designed to deliver exactly that.

A New Way to Communicate with HTTP 402
For too long, the choice has been binary: block or allow. Content creators told us they wanted to license their work but didn't know how to start the conversation with crawlers. They felt that blanket blocking was closing doors to potential revenue and needed a way to state their terms upfront.
To solve this, we are making it easier than ever to use the HTTP 402 "Payment Required" status code. Instead of simply blocking a bot, you can now send a 402 response with a custom message. This opens a direct line of communication, allowing you to say, "We're open for business, but let's talk terms first."
Here’s how it works: within the AI Crawl Control dashboard, paid Cloudflare customers can select specific bots and, instead of a hard block, send a 402 response. You can customize the message to include contact information or a link to your pricing page. For example: "To access this content, email licensing@yoursite.com or call 1-800-LICENSE," or "Premium content available via API at api.yoursite.com/pricing."
This simple but powerful tool transforms a technical roadblock into a business opportunity, creating a direct path from crawling to a commercial agreement.
How to Customize Your 402 Response
For users on a paid Cloudflare plan, customizing your bot responses is straightforward:
- Navigate to the AI Crawl Control dashboard.
- When you choose to block an individual crawler, you will see an option to send a 402 Payment Required status code.
- You can then write your custom message in the settings.
Here is an example of what the crawler would see in the response:

And here is where you can configure the message in your dashboard:

The Future of AI and Content Partnerships
This is just the beginning. We plan to enhance the 402 response to include structured data about content value, freshness, and licensing terms, giving AI companies even more information to make informed decisions. We are moving toward a future where creators are active participants in the development of AI, with clear paths to monetization.
The 402 response code is more than just a technical feature; it's a bridge between two industries that need each other. AI Crawl Control provides the essential infrastructure to help these partnerships flourish, ensuring that the creators whose work fuels AI development are fairly compensated.

Compare Plans & Pricing
Find the plan that matches your workload and unlock full access to ImaginePro.
| Plan | Price | Highlights |
|---|---|---|
| Standard | $8 / month |
|
| Premium | $20 / month |
|
Need custom terms? Talk to us to tailor credits, rate limits, or deployment options.
View All Pricing Details
