Riak Adoption - We Have Some Work To Do

I can't help but notice you're asking almost exactly the wrong people. People on the riak-users mailing list are by definition the choir.<p>As someone who works somewhere that looked into riak, here was my first impression:<p><pre><code> [root@brandt ~]# yum install riak &#60;snip&#62; No package riak available. Error: Nothing to do</code></pre>

Three things:<p>(1) Embrace the fact that a lot of people want to partner up Riak with an in memory DB like Redis and provide documentation, articles, tools to make this easy.<p>(2) A public face of Basho at events like Scala-days and who engages the community. @rit for 10Gen does a really good job of this - check out his twitter and Github<p>(3) Do a gap analysis between the content on sites like the cassandra website - I keep feeling like my week reading Cassandra content and articles hammered home fundamentals about the how things were working under the hood more than Riak. Which gives me more confidence than a "it just works" mentality.

We chose Mongodb over Riak for a host of reasons, but after working with Mongo, there's times I definitely wish we had gone after Riak, but there's a laundry list of why it lost out (and where it was super strong as well). I took a look at the list of improvements and one of the biggest reasons we didn't pick Riak (Riak's APIs and model make it significantly difficult to use it and be able to mentor a mediocre developer to working with an existing system...I've got a whole list of details on where, how and why this lost out) doesn't seem to be on the list. There's standardizing the client APIs, which is nice, but wouldn't address a large number of the issues. On the other hand, a lot of the issues could be resolved by wrapping existing APIs (whether directly in Riak or in client APIs/the REST API). We decided not to create our own wrapper because of maintenance and training/documentation concerns.<p>There's really too much info to go into it here (we spent something like 4-6 months investigating different datastores with actual production data and load levels), but Mark, if you want to sit down and talk about it, or chat via email, feel free to drop me a line. My contact info should be in my profile.

One of the questions I haven't been able to track down that has prevented adoption by me is how efficient the caching around MapReduce queries are in Riak.<p>In CouchDB MapReduce queries are completely incremental; if only a little data changes, only the relevant parts need to be rerun through the MapReduce process.<p>Riak looks really promising as:<p>1. It allows chaining MapReduce functions together<p>2. It supports sharding<p>Looking into Riak, it looked like there was a cache for part of the MapReduce system, but I wasn't sure how that cache worked or if it would be enough for large datasets that only change in little increments.<p>Edit: formatting.

I tried for the second time a couple months ago and found the documentation to be incomplete and inconsistent. e.g. This has been on the Riak Fast Track for many months now: "There are various screencasts throughout the Riak Fast Track. At the time they were made, Basho was using Mercurial and Bitbucket for our version control system and development platform, respectively. We have since switched to Git and GitHub but the screencasts do not yet reflect this. Do not be alarmed. We will re-record these using Git/GitHub when time permits."

If Riak was available as an on-demand service on any cloud provider other than Joyent then I would have jumped on it a while ago. If I ever had anything large enough to warrant it then I'd build my own cluster, but it's just not worth the effort/cost on a smaller build.

Last time I checked the ruby driver for risk didn't properly support secondary indexes. Looking at <a href="https://github.com/basho/riak-ruby-client" rel="nofollow">https://github.com/basho/riak-ruby-client</a> and <a href="https://github.com/seancribbs/ripple" rel="nofollow">https://github.com/seancribbs/ripple</a>, I can't see it there today either.<p>I'm sure that I might be able to find it in the spec folder somewhere, but I think it's a great feature and putting it in the readme would help a lot of people

I think the Riak pages are not geared at the people making the selection, e.g., architect-level technologists. The wiki is full of marketing claims ('Riak is the most powerful open-source, distributed database you'll ever put into production') ('Riak is the most boring database you’ll ever run in production').<p>But then suddenly:<p>'curl -v -X PUT -d '{"bar":"baz"}' -H "Content-Type: application/json" \ -H "X-Riak-Vclock: a85hYGBgzGDKBVIszMk55zKYEhnzWBlKIniO8mUBAA==" \ <a href="http://127.0.0.1:8091/riak/test/doc?returnbody=true" rel="nofollow">http://127.0.0.1:8091/riak/test/doc?returnbody=true</a><p>now, I understand why that is the way that it is -- the Riak guys are simultaneously very proud of their product, and also extremely technical -- but it's missing the middle ground. The middle ground is where you explain the characteristics of the system in non-marketing terms, describe what it's good at and what one could reasonably expect it to do, and describe also where it fails horribly and what you should not try to use it for. Once that's been outlined, _then_ bring out X-Riak-Vclock.<p>By comparison, e.g., redis.io has simple pages describing every command in the system and, critically, the associated algorithmic complexity and discussion of likely issues and problems. It describes what performance expectations are likely to be achieved on commodity hardware. And it allows you to test out your thoughts in real time right there on the page.<p>Personally, I have very little idea how, e.g., Riak compares to Redis. And I built Erlang and Riak from source and did the tutorial. I don't have a sense for how many ops/sec Riak can manage, what the equivalent to sinterstore and sunion look like, what the minimal real architecture for a production box setup should be. And a lot of the tutorial fills me with The Fear.<p>Which brings me to the last point: The Fear of Riak is pretty strong, and that's because very few people are running erlang on purpose in production. A lot of developers (and certainly devops people) have a hard enough time with their existing stack, without bringing on an entirely alien software, logging, alerting, monitoring, managing, and developing stack, and trying to understand how to reason about it. And, even those developers who can work up the courage to dive into erlang will have to deal with the fact that they will be novices for quite a long time on an extremely technical product that is designed to be at the core of their world.

Task 1: Tell us what it is.<p>Task 2: Tell us why it's better than Redis.<p>Meanwhile, at least they spell "key/value" correctly (it's not "key-value").