This is the problem with most of the guys who go with MongoDB. Obviously, this person is very technical, so I am not flaming him nor accusing him, but this is my view of the rest of them who pick MongoDB without exactly having a clue as to why (hipsters) or when they should use a NoSQL db and when they shouldn't.<p>I do not hesitate to admit that I was a hipster sometime back too. I chose MongoDB for many of my projects and it went well, till it reached some kind of moderate scale where I realized it was a terrible choice going with a NoSQL db (Sometimes, I'd have to duplicate data because there were no Joins, etc). And that's when you start to realize, that NoSQL is not a pure-white solution. It is designed to satisfy very specific use-cases. Relational databases are really good enough for 99% of the use cases out there.<p>Unless otherwise you are COMPLETELY unable to design your schema in a relational database, you SHOULD NOT simply opt for a NoSQL database. The claimed NoSQL performance benefits will easily be outrun by a terribly designed schema, if you use the wrong db for the wrong scenario. Trust me, MySQL has had so much negativity because of these hipsters, but even something as basically relational as MySQL scales really reaaallllly well. Infact, many top guys still use MySQL till date, for a reason, in production.[1]<p>Next time you launch your start-up, spend some time carefully evaluating your db design decisions, as the wrong db for the wrong use-case could easily become the most expensive mistake of your startup.<p>[1]<a href="http://www.quora.com/Quora-Infrastructure/Why-does-Quora-use-MySQL-as-the-data-store-instead-of-NoSQLs-such-as-Cassandra-MongoDB-or-CouchDB" rel="nofollow">http://www.quora.com/Quora-Infrastructure/Why-does-Quora-use...</a>
I'm a little blown away by the total lack of technical understanding when it comes to MongoDB. This person is obviously very technical, so why would he have chosen MongoDB in the first place? It isn't like MongoDb's technical shortcomings are a secret. The description of how MongoDb does sharding and distributed queries should have immediately raised red flags to any who has even a modicum of CAP understanding. If it sounds complex, a total hack and impossible to manage it probably is.<p>For gosh sakes, even the 10gen guys admit MongoDB lost data a year(!) ago. [1] If a database lost data in a single server configuration, why would you trust it as a cluster?<p>1: <a href="http://www.dbms2.com/2011/04/04/the-mongodb-story/" rel="nofollow">http://www.dbms2.com/2011/04/04/the-mongodb-story/</a>
A few comments on the problems:<p>* CPU bottleneck. The mongod is no usually bound by CPU except for building indexes on existing data (which shouldn't really happen in production). The issue he's talking about is contention between the web server (or workers) and the mongos. This isn't anything unexpected. It's recommended to put the mongos onto the application server and then you scale this by adding CPUs initially but then by adding multiple application nodes and using a load balancer. Or perhaps splitting the mongos onto dedicated nodes?<p>* Virtualisation: VMs are notorious for having variable performance because you have the overhead of the hypervisor but more importantly, are sharing resources with others. We run performance critical apps on dedicated servers and reserve VMs for tools or things which aren't high throughput e.g. a MongoDB arbiter.<p>* EBS: This has had known performance issues for years. It's fine as a basic file store but should never be used for databases. PIOPs are the way around this but local instance storage is also an option.<p>* No transactions: MongoDB has never had them. This is known.<p>* Schemaless != no schema design. It makes it easy to play around but you still need to think through things carefully. See <a href="http://blog.serverdensity.com/mongodb-schema-design-pitfalls/" rel="nofollow">http://blog.serverdensity.com/mongodb-schema-design-pitfalls...</a><p>* No joins. Again, it has never had joins. This is known.
Completely off-topic (well that's my nickname) but I'm seeing on HN more and more <i>beautiful</i> slide decks, from a purely esthetical point of view. This deck has beautiful fonts and a beautiful color scheme, and it is nicely designed.<p>My question is: how are they made? Keynote, Powerpoint, HTML...?
Are they made with the help of a graphic designer? They look completely outside of the reach of the average technical developer. Or do they use a pre-made theme?
As I clicked forward in the slideshow, I kept expecting to find something to disagree with but it never happened. Mostly I nodded to myself.<p>I have a huge investment in MongoDB at this point, both financially and in equipment - over 200 machines dedicated in various separate clusters - and a pivot to another datastore at this point would be a significant re-engineering effort.<p>All the people talking about changing databases after the MVP have clearly never had to deal with a typical hockey stick growth profile and having to allocate engineering resources based on need - either making the product better, or wasting time changing your database and losing traction.<p>Anyway, I'm hoping posts like this can dissuade people from choosing MongoDB for anything destined for high throughput Enterprise-level production environments.<p>I just wish there was something I could easily put in to replace MongoDB, but none of the available options quite fit the same document store model, but make better use of available resources and provide much better performance.
In all of these discussions about MongoDb on HN, I always wonder why aren't people talking about the 1 % use cases for which it's good for. Maybe it is too obvious for most folks , but anyway I will just regurgitate the use cases Mongodb folks have documented officially. Here: <a href="http://docs.mongodb.org/manual/use-cases/" rel="nofollow">http://docs.mongodb.org/manual/use-cases/</a>. They have literally like three major use cases.
Well my 2 cents to the discussion.<p>We really tried to make Mongo work over the last 1.5 years. We decided to go for mongo mostly because of JSON and geospatial indexing.<p>We became to realize that for our business case it is not the right tool. That said, and I work with databases now for 20+ years, I don't see what problem Mongo solves.
My first thoughts when I read this:<p>1. It's easy to screw up a relational database. I've seen more than a few mature relational databases... Most of them have plenty of sins, near-crippling performance snafus and other horrible legacy. <i>Any</i> database that is big enough and growing enough is a beast to manage.<p>2. From the slides, I think this guy took "Schema-Less" as a cue to stuff completely arbitrary data into MongoDB. No wonder his indexes went crazy. You still need to think about the data you're storing & the relationships. You need a design whatever database you use.<p>3. Any relational databases I've seen at scale have a lot of flattening. I've seen intra-day transaction dbs that are completely flattened. If your next port of call is a highly normalised relational database, you're going to hit another wall fast enough.<p>4. Two Phase commit. Seriously. Forget it. I've spent half my career in financial institutions. Quick, fail-fast processing with a reconciliation process is <i>by far</i> the most common approach. 2PC actually slows you down and introduces another component that gets in the way. It's used very sparingly (and even then, usually causes a world of pain).
I think that some people misunderstand the primary purpose of Schema-less design. It's not about typing, it's about document flexibility. It's about getting rid of EAV tables (read: Magento) and storing document-specific information. Typing obviously comes into play, but is only half the topic.<p>If you are building a system where the schema is the same for all records, then you really shouldn't be using Schema-less design.
After 6 intensive months with MongoDB to build my MVP, I just love it.<p>For sure, it's not perfect. Lack of joins is a shame, but can be quite easily solved outside the database.<p>But the ease of use and speed of development are such a HUGE advantage. Not having to break my schema into normalized relations and define it in the DB saved me literally days of work.<p>I can imagine a case that 1 year from now, when our product will be more mature, we'll be leaving mongodb for another SQL or No-SQL database. Doesn't matter. The benefit that mongodb gives us now justifies the costs the may be incurred years from now.
These guys are lucky they didn't try Cassandra. That's really Mongo's problem: it's too close to a regular SQL solution. You have a sharded NoSQL data store that performs in-store filtering and sorting? You can run aggregation queries? Compound indexes? Amazing! Tell me more.<p>Moral of the story is unless you can justify a NoSQL datastore for your particular solution and you can live without joins, stick with a regular SQL db.
Not totally related to the article, but every technology has its use. I get really annoyed when people ask me why I don't use ruby or mongodb at my job. I hate this movement towards "blog technologists" that basically read something in a blog, maybe (probably not) try it out themselves for something small and assume it is the best for everyone and if you dont use it you are stupid.
What version of Mongo and pymongo is he using? The connection client looks old (not using MongoClient) and the compound index selection (or lack of with $and) doesn't exist in 2.4.1. I'm curious as to how many of issues have been resolved...
I just submitted a related article on Hyperdex which, although it's not Python, has a very good Python interface.<p><a href="https://news.ycombinator.com/item?id=5686973" rel="nofollow">https://news.ycombinator.com/item?id=5686973</a>
The question I think is fundamental but that I didn't see asked nor replied is this: if we had a data store that had full performance, full scalability, etc., would we design it as a relational database or not? Said otherwise: the debate about NoSQL, is it an optimisation issue or a design issue? (Keep in mind pg's article about what would be a language in hundred years)
Slide 56 states: "Schema vs. Schema-less is just a different version of dynamic typing vs. static typing."<p>Wrong.<p>If anything, it is a different version of weak typing vs. strong typing. That is totally orthogonal.
The problem with most of these databases is that they are trying to do too much.<p>Either the database is great for ad-hoc queries and flexibility or it's great for performance and scale.
Why the URL are not working from the SpeakerDeck document? Captain Obvious here: But isn't that the purpose of the Web? It's the WEB... make those links work!
Sounds like 90% of your problems could have been solved if you'd spent a day or two researching what you were about to build your entire company around.<p>Sounds like the "We Fail" slide might be the most accurate one there.