> The deal was always simple: search engines had permission to crawl sites because they were going to be sending users to those sites. If they're hitting your site half a million times for every one user they send to your site, all they're giving you is higher costs.
The one that is not closed behind paywalls or sign ups with atrocious conditions; the one that allows everybody to participate, not just rich people; the one for which listed NGOs are fighting, see, e.g., https://www.eff.org/work.
Agree 100%.