Holy crap, I am scared!<p>Please, please, please read the fine print and ensure you understand the design tradeoffs as well as your application's requirements before blindly using this.<p>The moment I heard multi-master I thought Paxos, Raft or maybe virtual synchrony. Hmm, nothing in the documentation. Maybe a new consensus protocol was written from scratch then? That should be interesting!<p>No, none of that either - this implementation completely disregards consistency and makes write conflicts the developer's problem.<p>From <a href="http://bdr-project.org/docs/stable/weak-coupled-multimaster.html" rel="nofollow">http://bdr-project.org/docs/stable/weak-coupled-multimaster....</a><p>* Applications using BDR are free to write to any node so long as they are careful to prevent or cope with conflicts<p>* There is no complex election of a new master if a node goes down or network problems arise. There is no wait for failover. Each node is always a master and always directly writeable.<p>* Applications can be partition-tolerant: the application can keep keep working even if it loses communication with some or all other nodes, then re-sync automatically when connectivity is restored. Loss of a critical VPN tunnel or WAN won't bring the entire store or satellite office to a halt.<p>Basically:<p>* Transactions are a lie<p>* Consistent reads are a lie<p>* Datasets will diverge during network partitioning<p>* Convergence is not guaranteed without a mechanism for resolving write conflicts<p>I am sure there are use-cases where the risk of this design is acceptable (or necessary), but ensure you have a plan for dealing with data inconsistencies!
Some info from 2nd Quadrant on what BDR is: <a href="https://2ndquadrant.com/en/resources/bdr/" rel="nofollow">https://2ndquadrant.com/en/resources/bdr/</a><p>Bi-Directional Replication for PostgreSQL (Postgres-BDR, or BDR) is the first open source multi-master replication system for PostgreSQL to reach full production status, developed by 2ndQuadrant and assisted by a keen user community. BDR is specifically designed for use in geographically distributed clusters, using highly efficient asynchronous logical replication, supporting anything from 2 to more than 48 nodes in a distributed database.
While indeed very exciting, it's important to note that this makes the BDR extension from 2ndquadrant compatible with stock Postgres. This does not include BDR shipping with core Postgres.<p>This continued improvement with the core code and extension APIs will make more and more extensions feasible which will mean more are able to plug-in and add value without things having to be committed to core. Though in time this is one that has a good chance of actually being in core much like pg_logical.
The title is misleading, the replication is not coming to stock Postgresql 9.6.<p>A replication extension got the patches it needs to run into Postgresql 9.6 so you can use the extension without patching Postgres.
As the developer who also manages the servers we deploy on, and not a full time PgDBA, things like multi-master replication scare the hell out of me. They really make me worry about what happens after downtime. And latency.<p>Could anyone here recommend good reading material for scaling out your first database on to multiple servers? How do I know which scheme is the best for me?
Some more information for anyone trying to understand this better: <a href="http://bdr-project.org/docs/stable/overview.html" rel="nofollow">http://bdr-project.org/docs/stable/overview.html</a><p>Sourcecode: <a href="https://github.com/2ndQuadrant/bdr" rel="nofollow">https://github.com/2ndQuadrant/bdr</a>
Here is a rationale for multi-master from James Hamilton's "On Designing and Deploying Internet-Scale Services".<p><i>Designing for automation, however, involves significant service-model constraints. For example, some of the large services today depend upon database systems with asynchronous replication to a secondary, back-up server. Failing over to the secondary after the primary isn't able to service requests loses some customer data due to replicating asynchronously. However, not failing over to the secondary leads to service downtime for those users whose data is stored on the failed database server. Automating the decision to fail over is hard in this case since its dependent upon human judgment and accurately estimating the amount of data loss compared to the likely length of the down time. A system designed for automation pays the latency and throughput cost of synchronous replication. And, having done that, failover becomes a simple decision: if the primary is down, route requests to the secondary. This approach is much more amenable to automation and is considerably less error prone.</i>
I look forward to when this lands on PostgreSQL 9.7 without the need for an extension. But more so when I can also include the Citus DB extension.<p>Running CitusDB with just 1 master made me nervous. They did talk about having multi-master replication as a belt and braces solution, but I don't know how far they got.<p>Thinking about this. Both being used may give you a 100% fully fault tolerant solution?
Please make sure you understand log replication and go through fire drills for the list of things that can go wrong with bi-directional replication. The last thing you'll want to do is deploy this into production and wing operations as you go.
BDR is a nice building block to multi-master PostgreSQL. I'm looking forward to parallel aggregates in 9.6 being added to core. Using the agg[0] extension for something as core as using more than one core per (aggregate function) query felt strange. (I wonder if the time has come to decouple connections from processes/threads in postgres, as well...)<p>[0]: <a href="http://www.cybertec.at/en/products/agg-parallel-aggregations-postgresql/" rel="nofollow">http://www.cybertec.at/en/products/agg-parallel-aggregations...</a>
Do all nodes need to be up 100% of the time? If not, how long can a node be down without replicating (perhaps because a server is under maintenance).<p>Does BDR have rules for primary key insertion conflicts?
I have a (perhaps odd) situation where identical data is already being written to multiple servers. Currently handling with a custom replication mechanism.