I mean, with a company as large as cloudflare. I think they could /easily/ strong-arm this move by making blocking google crawlers a default setting on websites. The amount of traffic drop alone from that would make google think twice about the whole ordeal. And people who care about the google search indexers can turn them on again which will allow indexing again. but a default block would cause a lot of disruption google side and many people I don’t think would go in and fix the setting till later on down the road.
Cloudflare’s customers probably wouldn’t be on board with that. Google’s properties provide a tonne of traffic to businesses. Doing anything to put that in jeopardy would probably have many of Cloudflare’s customers looking for a new provider.
Google used to provide a ton of traffic, they hoard it all themselves now through AI and summaries of content. Eventually the balance of cost/benefit will shift and Google will suddenly see itself rejected from scraping, furthering the product deathspiral.
content is only 1 category of website
ecommerce drives all the advertising that funds content… it’s a much bigger market, and they don’t care about content scraping as long as you buy their product
If I had to choose my favourite corporation, it would be Cloudflare. They at least do something good.
You may want to read: https://robindev.substack.com/p/cloudflare-took-down-our-website
I still think they are good. Isolated incidents like this are going to happen when you are doing business at such scales.
