Can I ask you what was going wrong in the past that caused your site to take such a long time for Google to crawl it - and what you did to bring the crawl time down so dramatically?<p>For reference, when some sites I was tracking spiked up over a few hundred MS crawl time I grew alarmed and resolved those issues.<p>Also, another good way to measure things is to look at 'Site Performance' in the Labs section of Webmaster Tools (right below Diagnostics used in this post). You'll get a graph that represents your site relative to the Internet along with improvement suggestions.<p>Also - while the conclusion seems accurate: faster load time, better ranking in perhaps 1% of cases is something Google has talked about - I think your reasoning is off.<p>You wrote: <i>.If a site can be crawled faster - and requires less resources to index, doesn’t it stand to reason that it will be rewarded with higher search rankings?</i><p>It's not resources, it is a matter of user experience. The faster your site loads, the happier Google's searchers who clicked over to you are:<p><i>Speeding up websites is important - not just to site owners, but to all Internet users. Faster sites create happy users and we've seen in our internal studies that when a site responds slowly, visitors spend less time there</i><p>See <a href="http://googlewebmastercentral.blogspot.com/2010/04/using-site-speed-in-web-search-ranking.html" rel="nofollow">http://googlewebmastercentral.blogspot.com/2010/04/using-sit...</a>
Google's page loading times seem kind of flaky. One of our sites' loading times has jumped between 1.5 and 10 seconds and back four times since they started graphing it.<p>But I have a cron job that tracks the loading times of random pages on our sites (from offsite, every 10 minutes). I get very consistent loading times (nowhere near that variance).<p>(Edit: they also suggest GZIP resources that are in fact already gzipped, at least they are according to the HTTP headers)
I think this post has a grain of truth behind it, though I don't agree with the reasoning about resource utilization.<p>I noticed that when one of my sites went from 300ms response times to 30ms, the crawler started indexing more pages per day, and would index deeper, which meant more of my pages in their index. The result was a healthy boost in organic search traffic due to more long tail search matches.