All: there was a big thread about this yesterday (<a href="https://news.ycombinator.com/item?id=23285249" rel="nofollow">https://news.ycombinator.com/item?id=23285249</a>) but because it didn't focus on the technical content, and because there were glitches with a previous submission of this report (described at <a href="https://news.ycombinator.com/item?id=23288120" rel="nofollow">https://news.ycombinator.com/item?id=23288120</a> and <a href="https://news.ycombinator.com/item?id=23287763" rel="nofollow">https://news.ycombinator.com/item?id=23287763</a> if anyone cares), we invited aphyr to repost this. Normally we downweight follow-up posts that have such close overlap with a recent discussion (<a href="https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=by%3Adang%20follow-up&sort=byDate&type=comment" rel="nofollow">https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...</a>), so the exception is probably worth explaining.
I also want to point their Node.js transactions API is wrong and looks like they have no idea how promises or async code work in JS.<p>In mongo, you have a `withTransaction(fn)` helper that passes a session parameter. Mongo can call this function mutliple times with the same session object.<p>This means that if you have an async function with reference to a session and a transaction gets retried - you very often get "part of one attempt + some parts of another" committed.<p>We had to write a ton of logic around their poor implementation and I was shocked to see the code underneath.<p>It was just such a stark contrast to products that I worked with before that generally "just worked" like postgres, elasticsearch or redis. Even tools people joke about a lot like mysql never gave me this sort of data corruption.<p>Edit: I was kind of angry when writing this so I didn't provide a source and I'm a bit surprised this go so many upvotes without a source (I guess this community is more trusting than I assumed :] ). Anyway for good measure and to behave the way I'd like others to when making such accusations here is where they pass the same session object to the transacton <a href="https://github.com/mongodb/node-mongodb-native/blob/e5b762c6d53afa967f24c26a1d1b6c921757c9c9/lib/sessions.js#L376" rel="nofollow">https://github.com/mongodb/node-mongodb-native/blob/e5b762c6...</a> (follow from withTransaction in that file) - I can add examples of code easily introducing the above mentioned bug if people are interested.
Hi folks! Author of the report here. If anyone has questions about detecting transactional anomalies, what those anomalies are in the first place, snapshot isolation, etc., I'm happy to answer as best I can.
Mongo has been <i>related to</i> "perpetual irritation" up to "major production issue" at all three of my last companies.<p>For as easy as it is to use jsonb in Postgres, or Redis, or RocksDB/SQLite, or whatever else depending on your use case - I can't find any reason to advocate its use these days. In my anecdotal experience, the success stories never happen, and nearly developer I know has an unpleasant experience they can share.<p>Big thanks to aphyr and the Jepsen suite (and unrelated blog posts like Hexing the Interview) for inspiring me to do thorough engineering.
aphyr, you are of great inspiration as an engineer and as a human.<p>Your attitude of "a tool I need doesn't exists, so I'll just go ahead and create it" blew my mind and changed me for the better.<p>I'm dedicating my next test framework to you. Thank you for everything.
This article reinforces my stance that bad defaults are a bug. Defaults should be set up with the least number of pitfalls and safety tradeoffs possible so that the system is as robust as it can be for the majority of its users, since the vast majority of them aren't going to change the defaults.<p>Sometimes you end up with bad defaults simply by accident but I feel like for MongoDB the morally correct choice would be to own up to past mistakes and change the defaults rather than maintain a dangerous status quo for "backwards compatibility", even if you end up looking worse in benchmarks as a result.
How many more years do we have to keep evaluating, studying, and reading about MongoDB's ongoing failures? It would appear this product has been a great burden on the community for many years.
> Clients observed a monotonically growing list of elements until [1 2 3 5 4 6 7], at which point the list reset to [], and started afresh with [8]. This could be an example of MongoDB rollbacks, which is a fancy way of saying “data loss”.<p>I hope they learned the lesson, don't fuck with aphyr.
I wanted to incorporate MongoDB into a C++ server at one point.<p>Their C/C++ client is literally unusable. I went to look into writing my own that actually worked and their network protocols are almost impossible to understand. BSON is a wreck and basically the whole thing discouraged me from ever trying to interact with that project again.
Aphyr is such a competent professional. What a relatively thorough and polite response to Mongo's inaccurate claims. "We also wish to thank MongoDB’s Maxime Beugnet for inspiration." is a nice touch.
The general mood I observed about MongoDB was that it used to be inconsistent and unreliable but they fixed most, if not all of those problems and they now have a stable product but bad word of mouth among developers. Personally, I've treated it as "legacy" and migrated everything that I had to touch since 2013 [0], and luckily (just read the article so hindsight 20/20 -- transaction running twice and seeing its own updates? holy...) never gave it another try.<p>[0]: <a href="https://news.ycombinator.com/item?id=6801970" rel="nofollow">https://news.ycombinator.com/item?id=6801970</a> (BTW: no, my dream of simple migration never materialized, but exporting and dumping data to Postgres JSONB columns and rewriting queries turned out to be neither buggy nor hard).
This is not directly related to this report or Jepsen, but since you're here I've got to ask: Aphyr, are there any recent papers/research in the realm of distributed databases which you're excited about?
Friendly question: did you update anything on the findings since <a href="https://news.ycombinator.com/item?id=23191439" rel="nofollow">https://news.ycombinator.com/item?id=23191439</a> ?
The main argument for using a documented-oriented database: <a href="https://martinfowler.com/bliki/AggregateOrientedDatabase.html" rel="nofollow">https://martinfowler.com/bliki/AggregateOrientedDatabase.htm...</a>
I suppose there are reasons why the defaults are the way they are. Can anyone comment on the implications, performance or otherwise, of bumping up the read/write concerns?
mongodb's business model, forever, has been to get developers to write code, be damned the fact that you can't support it reliably on a cloudy day.
At this point I think we might be going a bit overboard with title changes.<p>Now that it's just "MongoDB 4.2.6", the title makes me think that this is a release announcement, not an analysis of the software.<p>The first title (that specifically referenced a finding of the analysis) was best, imo. Mildly opinionated or whatever, but at least it quickly communicated the gist of the post. On the other hand:<p>"Jepsen: MongoDB 4.2.6" – not super helpful if you're not already familiar with the Jepsen body of work.<p>"MongoDB 4.2.6" – as stated above, sounds like a release announcement.<p>If you want a suggestion, maybe something like "Jepsen evaluation of MongoDB 4.2.6"? Not overly specific (/ negative) like the first title, but at least provides some slight amount of context.<p>@dang
I am tech lead for a project that revolves around multiple terabytes of trading data for one of top ten largest banks in the world. My team has three, 3-node, 3TB per node MongoDB clusters where we keep huge amount of documents (mostly immutable 1kB to 10kB in size).<p>Majority write/read concern is exactly so that you don't loose data and don't observe stuff that is going to be rolled back. It is important to understand this fact when you evaluate MongoDB for your solution. That it comes with additional downsides is hardly a surprise, otherwise there would be no reason to specify anything else than majority.<p>You just can't test lower levels of guarantees and then complain you did not get what higher levels of guarantees were designed to provide.<p>It is also obvious, when you use majority concern, that some of the nodes may accept the write but then have to roll back when the majority cannot acknowledge the write. It is obvious this may cause some of the writes to fail that would succeed should the write concern be configured to not require majority acknowledgment.<p>The article simply misses the mark by trying to create sensation where there is none to be found.<p>The MongoDB documentation explains the architecture and guarantees provided by MongoDB enough so that you should be able to understand various read/write concerns and that anything below majority does not guarantee much. This is a tradeoff which you are allowed to make provided you understand the consequences.