This is very timely. We are just starting to work on a new centralized logging system using logstache->kafka->elasticsearch->kibana, we may also use kibana for some other tasks as well. Anyone have any experience using that setup? Pros/Cons?<p>We used graylog for a while but ran into some issues with back pressure on elastic search (hence kafka).
I heard comparisons with other technologies before, like RabbitMQ [1] [2] but I would love to see an feature/performance comparison between kafka and other similar solutions, specially with cloud based services like the new kid on the 'Google Pub/Sub' [3]<p>[1] <a href="http://www.quora.com/RabbitMQ-vs-Kafka-which-one-for-durable-messaging-with-good-query-features/answer/Stuart-Charlton-1" rel="nofollow">http://www.quora.com/RabbitMQ-vs-Kafka-which-one-for-durable...</a><p>[2] <a href="https://youtu.be/MA_3fPBFBtg?t=35m27s" rel="nofollow">https://youtu.be/MA_3fPBFBtg?t=35m27s</a><p>[3] <a href="https://cloud.google.com/pubsub" rel="nofollow">https://cloud.google.com/pubsub</a>