I used to work as a Search Quality Rater for Google. It was pretty interesting. It's worth noting that raters are not Google employees but contracted through third parties and working from home. Many of my colleagues were stay-at-home moms making a few extra bucks. This was a few years ago though so things may have changed.
OT, the top rated comment on the above article published on googlewebmastercentral.blogspot.com is :<p><i>Google just made public their latest version of the Quality Raters Guidelines (PDF linked from the blog post)</i><p>Which is just summarizing what the blog post is about (probably a G+ share that got plus'd a lot). Nothing against the commentator, it's rather their ranking/voting system that needs to be fixed.
The "your money or your life" web page stuff is interesting. They call these YMYL pages anything that can have large impact on the user: financial info pages, legal info pages, medical info pages, and so on. They say they have very high quality page rating standards for these pages. OK, but how?<p>Later in the document they talk about doing reputation research on the site. They say, "[F]or Page Quality rating, you must also look for outside, independent reputation information about the website."<p>So they <i>are</i> evaluating content. They are encouraging their reviewers to make a judgement. This strikes me as unreliable, maybe even a slippery slope for them.
Surprising to see so much human input. Essentially google took a leaf out of Yahoo!'s playbook after first crushing them using just algorithms. Is this proof that beyond a certain degree of algorithmic extraction you need a human hand to advance or is it evidence for a return to the old days of a more curated index? (Or are there other possibilities?)