The Anatomy Of Search Technology: Blekko’s NoSQL Database

This is the same technology that is used by the webgrepper tool [<a href="http://blekko.com/webgrep" rel="nofollow">http://blekko.com/webgrep</a>] (a grep for the web pages' sources).<p>Disclaimer: I work at blekko and I developed the webgrepper.<p>As a side note, we have used this for various other purposes - some fun ones being, store a big music collection (to extract meta data via mapjob), citizenship test q&#38;a (to pick random questions), the 'joke of the day' (of course, this is our "hello world" example internally to new employees) ..etc.

Hey, great stuff! How do you implement high availability?

Isn't DynamoDB on SSDs? AWS alone might not be cost effective but, how does it look with the other services also offered?

Very good article in deed. I have always been amazed of how the search engines can query so huge data sets so quickly. This brings some light to it.<p>It would had been nice to see some examples of the query language they use, it if is comparable to other NoSql databases.

Can you say more about the choice of swarm algorithms instead of Paxos?

Awesome insights into what goes into building a search engine. Very impressive indeed.

does blekko DB partition data base on primary key or it just query all nodes everytime like elasticsearch?