Cloudflare CEO: bots now outpace humans, and the web needs a new business model

Cloudflare cofounder and CEO Matthew Prince told The Verge’s Decoder podcast that automated traffic overtook human traffic this year and that the advertising model which funded the web for three decades cannot carry what is coming. In the interview with Nilay Patel he laid out the company’s plan to make AI crawlers pay.

Bots overtook humans

Prince said Cloudflare’s data showed automated traffic passing human traffic in May, and the company announced the milestone in June. He had predicted the second half of 2027 at Web Summit in November, then cut that to the first half of 2027 at South by Southwest in March. “I was stunned,” he said. Extrapolating, he expects automated traffic to reach 1,000 times human traffic within five years, with human traffic roughly flat, and admits he has been wrong in every prediction on this. Cloudflare Radar data published in June put bots at 57.4 percent of HTTP requests, against 42.6 percent from humans.

Pay to crawl

Advertising cannot finance that growth because bots do not click on ads. Prince framed it as a tragedy of the commons: an AI agent looking for a lunch spot scans every menu nearby, yet only one restaurant wins the customer. Cloudflare is pushing micropayments back onto the web, with a page view costing a thousandth of a penny and a premium on news or academic work. It is working with Coinbase and Stripe on reviving the HTTP 402 “payment required” protocol, with the money coming from what users pay for AI services, much as Spotify pools subscriptions: Spotify returned about $12 billion to the music industry last year, and Prince cites a Danish songwriter earning 40 million euros a year from queries Spotify cannot answer.

“Google Zero” and the September 15 defaults

From September 15, Cloudflare intends to flip the default for all free customers so Google’s AI crawler is blocked; if Google will not separate AI training from search indexing, Prince says it will be blocked across the board, and site owners can opt out. He calls the risk “Google Zero” and argues Google should not turn yesterday’s search monopoly into tomorrow’s AI monopoly. After a UK ruling, he says, Google committed to more crawler transparency. He notes Reddit’s content sells for at least seven times, by some measures 14 times, more per token than The New York Times’, because it has no substitute, and that his own paper, the Park Record in Park City, Utah, should earn more from AI licensing this year than from digital ads.

Inside Cloudflare

AI is reshaping Cloudflare too. In May the company cut 1,100 people, about 20 percent of its workforce, as Prince explained in a Wall Street Journal op-ed. Most cuts came from measurement roles, following Peter Drucker’s builders, sellers and measurers split: investor-relations documents that took about 20 people two weeks now take three minutes. Cloudflare, with 4,500 to 5,000 employees, keeps hiring builders and sellers and is flattening management.