I was an early adopter. In the beginning, there was actually a handful of us reviewing avatars and assigning ratings. Now it's just kind of left up to trust.<p>I don't think it was an Automattic project at first, so I doubt it's built on WordPress. But it's probably built on PHP and MySQL, with a <i>ton</i> of caching in front of it.<p>It wouldn't surprise me if it leverages WordPress.com's CDN.
What do you find confusing? It's owned by Wordpress guys, and people integrate it into their website. The backend stores a picture you set on the site, or one on other people's sites integrated with it. It then also keeps track of all your different accounts.