I want to make my own search engine, one day, with my own crawler.<p>There is an SEO-proof way to determine what the ranking of a site <i>should be</i> - penalise it for each advertisement, penalise further for delivering different content to the crawler[1], allow logged-in users to down-rank a site, etc.<p>Basically, a site starts off with a perfect score, then gets penalised for each violation, for each dark-pattern, for each anti-user decision they took.<p>This sort of ranking cannot be gamed by SEO spammers, because ... why get their site to the top of the rankings if they can't put any advertisements on it?<p>Even if they do manage it, their time in the sun will be brief.<p>[1] Random spot-checks with different user agents with different fingerprints should work.