GoshawkDB: A distributed, transactional, fault-tolerant object store

&gt; The only requirement is that if F is the number of unreachable nodes you wish to be able to tolerate, then your minimum cluster size is 2*F + 1. &gt; Currently no other data store that I&#x27;m aware of offers this flexibility.<p>It isn&#x27;t true. There&#x27;re plenty of storages with same strong consistency model (tolerates F failures of 2F+1 nodes), among them are Cassandra with lightweight transactions, Riak with consistent buckets, CockroachDB and ZooKeeper.<p>&gt; However, GoshawkDB has a radically different architecture to most databases and its object-store design means that contention on objects can be dramatically reduced, allowing the performance impact of strong-serializability to be minimised.<p>Can you please describe the algorithm that you use for implementing distributed transactions?<p>&gt; just three network delays occur (and one fsync) before the outcome of the transaction is known to that node<p>3 round trips for per transaction is the result which is very similar to the Yabandeh-style transaction, see the Cockroach&#x27;s blog for details - <a href="http:&#x2F;&#x2F;www.cockroachlabs.com&#x2F;blog&#x2F;how-cockroachdb-distributes-atomic-transactions&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.cockroachlabs.com&#x2F;blog&#x2F;how-cockroachdb-distribute...</a><p>Do you use the same approach?

This is really cool! I guess the main competition in terms of similar purpose and guarantees is <a href="http:&#x2F;&#x2F;hyperdex.org&#x2F;" rel="nofollow">http:&#x2F;&#x2F;hyperdex.org&#x2F;</a>, although the add-on that enables transactions (quite critical...) is commercial.

<i>GoshawkDB is unique as it allows you to configure the tolerance to failure independently of the size of the GoshawkDB cluster. For example, you could have a cluster of 5 nodes and require that it continues to operate unless more than 2 nodes become unreachable. Or you could have a cluster of 20 nodes and require that it continues to operate unless more than 3 nodes become unreachable. The only requirement is that if F is the number of unreachable nodes you wish to be able to tolerate, then your minimum cluster size is 2</i>F + 1. You may of course choose to have a cluster size larger than this. Currently no other data store that I&#x27;m aware of offers this flexibility.<p>I fail to see the point of making your cluster unavailable before you&#x27;ve lost so many nodes that you no longer have a quorum. It seems odd to have a cluster that can handle e.g. 4 node failures, and take it offline after only 2. Why would anyone want a feature like that?

Why should I choose this over Ceph RADOS? [0][1]<p>0: Reliable Autonomic Distributed Object Store<p>1: <a href="http:&#x2F;&#x2F;docs.ceph.com&#x2F;docs&#x2F;master&#x2F;rados&#x2F;api&#x2F;librados-intro&#x2F;" rel="nofollow">http:&#x2F;&#x2F;docs.ceph.com&#x2F;docs&#x2F;master&#x2F;rados&#x2F;api&#x2F;librados-intro&#x2F;</a>

&gt; The trade-off is that in the case of failure (node failure or network partitions), operations may block until the failures have been resolved.<p>Curious how this works at scale. For example if a node is down and N requests start blocking, is there an upper bound to N? What happens when this is reached? Does N map to a goroutine? A file descriptor? Is there timeouts?<p>Seems like possibly a node going down essentially means the entire show going down if requests are filling up and not being resolved, you would think there may be a tipping point in which other requests may not be able to be served as all resources are blocked.<p>Possibly there is just a sane timeout and these requests also just fail, leaving the client to know something is amiss?

Paxos from the ground-up to offer strong transactional guarantees, similar to CockroachDB: <a href="http:&#x2F;&#x2F;www.cockroachlabs.com&#x2F;blog&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.cockroachlabs.com&#x2F;blog&#x2F;</a> CockroachDB is a KV store (that will be released with a SQL layer on top of it) whereas GoshawkDB is a Graph store. GoshawkDB puts data into client caches (much like Datomic) and even attempts transactions there.

For those interested in a bit more detail how GoshawkDB actually works, I&#x27;ve added a (slightly rushed) blog post on the topic, which covers transaction lifecycle. <a href="https:&#x2F;&#x2F;goshawkdb.io&#x2F;blog.html#20151224" rel="nofollow">https:&#x2F;&#x2F;goshawkdb.io&#x2F;blog.html#20151224</a>

On the site it is written that GoshawkDB doesn&#x27;t support membership change. Since you system is based on single decree paxos then you can take Raft&#x27;s joint consensus and apply to plain paxos. Probably it&#x27;s the simplest and easiest to change membership in paxos cluster.

I don&#x27;t see how it&#x27;s possible to manage transactions fully client-side while adhering to the consistency model that&#x27;s being claimed.<p>The best you&#x27;d be able to do seems like read-committed isolation.

Anyone bothered with author using term &quot;object store&quot;, when in fact it&#x27;s no match for ZODB or Db4o and only stores serialized documents, more similar to CouchDB or MongoDB?

Hey msackman,<p>Pretty cool looking stuff. Where&#x27;d you get the design for object&#x2F;graph structure? It is what we are using for our database, <a href="http:&#x2F;&#x2F;gunDB.io&#x2F;" rel="nofollow">http:&#x2F;&#x2F;gunDB.io&#x2F;</a> . Although we take the opposite approach to CAP than you, and have no support for serializability (I don&#x27;t think this belongs in the protocol but instead in the data, like git).<p>What&#x27;s you&#x27;re take on concurrency? How do you handle conflicts (I assume this is easy for you with a master)?<p>Would love to jump on a chat too, being DB guys. Shoot me an email mark@gunDB.io welcome arounds!

Performance?