Running Apache Kafka at Scale

This is very timely. We are just starting to work on a new centralized logging system using logstache-&gt;kafka-&gt;elasticsearch-&gt;kibana, we may also use kibana for some other tasks as well. Anyone have any experience using that setup? Pros&#x2F;Cons?<p>We used graylog for a while but ran into some issues with back pressure on elastic search (hence kafka).

I heard comparisons with other technologies before, like RabbitMQ [1] [2] but I would love to see an feature&#x2F;performance comparison between kafka and other similar solutions, specially with cloud based services like the new kid on the &#x27;Google Pub&#x2F;Sub&#x27; [3]<p>[1] <a href="http://www.quora.com/RabbitMQ-vs-Kafka-which-one-for-durable-messaging-with-good-query-features/answer/Stuart-Charlton-1" rel="nofollow">http:&#x2F;&#x2F;www.quora.com&#x2F;RabbitMQ-vs-Kafka-which-one-for-durable...</a><p>[2] <a href="https://youtu.be/MA_3fPBFBtg?t=35m27s" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;MA_3fPBFBtg?t=35m27s</a><p>[3] <a href="https://cloud.google.com/pubsub" rel="nofollow">https:&#x2F;&#x2F;cloud.google.com&#x2F;pubsub</a>