Have read a bit of the intro material, but I'm still not grokking what makes Kafka fundamentally different from ActiveMQ / Apollo. Can anyone sum up where and why one might need Kafka?
Kafka is an integral part of Shopify's infrastructure. It's brilliant but under appreciated technology.<p>A full company, scalable event bus like this can totally revolutionize the way you build services.
This is the first release as an Apache top-level project and represents many months of hard work.<p>A few of the major improvements (from <a href="https://archive.apache.org/dist/kafka/0.8.0/RELEASE_NOTES.html" rel="nofollow">https://archive.apache.org/dist/kafka/0.8.0/RELEASE_NOTES.ht...</a>):<p><pre><code> * Intra-cluster replication support
* Support multiple data directories
* Many new internal metrics
* Time based log segment rollout
</code></pre>
Plus many bug fixes and other improvements.
I wrote about this release and Kafka in general in Chinese and gather some information perhaps useful for Chinese guys: <a href="http://geek.csdn.net/news/detail/3866" rel="nofollow">http://geek.csdn.net/news/detail/3866</a>
This has nothing to do with the software itself, but it bothers me: Why do you call a "high-throughput distributed messaging system" Kafka? Kafka's stories essentially describe the polar-opposite: crippled, ineffective, labyrinthine message systems that are exceedingly hierarchical in nature. They are also rather user-unfriendly, i.e. their users usually die horrible, lonely deaths. Am I just missing the in-joke here, and it's called Kafka <i>because</i> it is exactly the opposite of this, or did somebody overdo the hipster naming scheme?