BlazingMQ: High-performance open source message queuing system

That fan-out functionality is particularly interesting for anyone needing to implement the upstream part of a firehose API, which of course Bloomberg do.

They actually open sourced it!<p>I worked on this team very briefly. Great team working on some interesting tech.

I have been working on an engine to perform heavy computations that is distributed and my messaging choice has been gRPC + Flatbuffers + proxys and I was wondering if someone could name some reasons or use cases to chose message brokers vs a lower level approach like gRPC. Is it mostly because of synchronous&#x2F;asynchronous needs or there is something else such as latency or even ease of implementation?

&gt; Supports a unique multi-hop network topology leading to a distribution tree for each queue, thereby leading to network bandwidth savings for certain use cases.<p>This is very interesting. One of the benefits of Kafka is partition routing so I&#x27;m curious to learn how this may compare.

Bloomberg has some interesting code! <a href="https:&#x2F;&#x2F;github.com&#x2F;orgs&#x2F;bloomberg&#x2F;repositories">https:&#x2F;&#x2F;github.com&#x2F;orgs&#x2F;bloomberg&#x2F;repositories</a><p>Has anyone used comdb2? It seems like an awesome DB that doesn&#x27;t seem to get the level of love it should.

Marketing is all well and good. Let&#x27;s see the data of p99 latency and throughput numbers compared to rabbitmq, kafka, activemq, qpid, and hornetq.<p>Also, it&#x27;s worth identifying and setting aside use-cases for broker-less messaging where 0mq, nanomsg, etc. could be more appropriate.<p>Edit: There&#x27;s only published performance data of itself, which has little value: <a href="https:&#x2F;&#x2F;bloomberg.github.io&#x2F;blazingmq&#x2F;docs&#x2F;performance&#x2F;benchmarks&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;bloomberg.github.io&#x2F;blazingmq&#x2F;docs&#x2F;performance&#x2F;bench...</a>

What a phenomenal landing page. Thanks to the animations I have a good idea of what this does and how it does it. You barely need to know what message queuing is to understand it.

How does it compare to NATS.io?

A yet another MQ :-) I quickly checked the performance page and it looks much better than its competitors. Documentation is better as well.

Are they working on other API&#x27;s besides Java and C++?

I&#x27;m the developer of the Message-Oriented Async RPC library Mats3: <a href="https:&#x2F;&#x2F;mats3.io&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;mats3.io&#x2F;</a><p>Its current sole implementation is based on Java Messaging Service JMS API, and it is used in production for a rather large UCITS unit holder registry on Apace ActiveMQ, and all tests runs fine on Apache Artemis MQ.<p>Every time a new message broker comes along, I sit up in the chair and wonder a) about their performance (!), and b) whether they have a JMS client implementation, and c) whether Mats3 works with that! When I tested RabbitMQ&#x27;s JMS client library, I sadly found that there was rather many differences - things I thought was screamingly obvious, was not available. E.g. as basic function as redelivery: &quot;Normal&quot; MQs try to redeliver N times, typically with a backoff between each attempt, and then, when all N attempts fail, it puts the message on the DLQ. Rabbit instead tight-loops the delivery attempts until either the message goes through, or the sun burns out. To be able to use Rabbit, I would have to use the native API, and implement redelivery and DLQing myself, client side. Also, transactions.<p>I now wonder whether I should make a lower-level abstraction, so that the JMS implementation is converted to a &quot;base&quot; impl, and then the JMS, Rabbit, Bloomberg, NATS, ZeroMQ, Aeron, etc implementations was extensions, or &quot;plugins&quot;, or &quot;drivers&quot;, to that.

Awesome landing page and looks great. Documentation is very well done. Would love to see more clients (python) and a web u&#x2F;i.

Does anyone here do large scale interface stuff with many millions of messages? I do this in healthcare and would like to try applying my skills somewhere else...

Very interesting. Thank you for benchmarks. Thank you for sharing. In a zero-replication no-leader-election (or all-in-one-node) scenario what is min latency observed? It appears predominant delay is in ACKing to ensure durability, right?<p>C++ and Java client examples are very simple which is cool.<p>Would appreciate if there is hook to enable `SO_TIMESTAMPNS` on socket and retrieve `cmsg` to get accurate timestamping and permit client code benchmarking.

I find it curious that lots of people always drag in Kafka as comparison to message brokers. Their own architecture (log of messages, vs. delivery of messages), but in particular the system architecture they lead to when used (event sourcing, vs. &quot;react to events and commands&quot;), are extremely dissimilar.<p>Kafka has its uses, in particular for massive influx of events, e.g. in a large IoT system - I&#x27;d say the perfect example would be continuous measurements of tens of thousands of gauges and sensors on an oil rig or any other large production system, or e.g. the energy meters in every home.<p>But I would personally never use such a system as the inter-service communication layer for a multi-service architecture. It is WAY to heavy coupling. Event sourcing looks fantastic on the surface, but is a disaster for a decades-living system with tons of developers. REST&#x2F;gRPC is better, but async messaging really rocks.

it never ceases to amaze me how the software engineering profession has very smart folks reimplement the same conceptual primitives over and over again. I&#x27;m curious if hospitals (if that&#x27;s even analogous to a company) do similar things by having different processes to solve the same problem.<p>I imagine this is used for Bloomberg (the terminal) and not Bloomberg (the website)?<p>going back to the article - fantastic animations. I&#x27;m just as curious to how that was made as the queue itself.

Why is Bloomberg approving the open sourcing of this system?

&gt; This section explains the leader election algorithm at a high level. It is by no means exhaustive and deliberately avoids any formal specification or proof. Readers looking for an exhaustive explanation should refer to the Raft paper, which acts as a strong inspiration for BlazingMQ’s leader election algorithm.<p>So their own homegrown leader election algorithm?<p>&gt; BlazingMQ’s leader election and state machine replication differs from that of Raft in one way: in Rafts leader election, only the node having the most up-to-date log can become the leader. If a follower receives an election proposal from a node with stale view of the log, it will not support it. This ensures that the elected leader has up-to-date messages in the replicated stream, and simply needs to sync up any followers which are not up to date. A good thing about this choice is that messages always flow from leader to follower nodes.<p>&gt; BlazingMQ’s elector implementation relaxes this requirement. Any node in the cluster can become a leader, irrespective of its position in the log. This adds additional complexity in that a new leader needs to synchronize its state with the followers and that a follower node may need to send messages to the new leader if the latter is not up to date. However, this deviation from Raft and the custom synchronization protocol comes in handy because it allows BlazingMQ to avoid flushing (fsync) every message to disk. Readers familiar with Apache Kafka internals will see similarities between the two systems here.<p>&quot;a new leader needs to synchronize its state with the followers and that a follower node may need to send messages to the new leader if the latter is not up to date&quot;. I thought a hallmark of HA systems was fast failover? If I come to your house and knock on the door, but it takes you 10mins to get off the couch to open the door, it&#x27;s perfectly acceptable for me to claim you were &quot;unavailable&quot;. Pedants will argue the opposite.

It has a gorgeous landing page but I&#x27;m not really sure why this is better than any other MQ. Can someone more knowledgeable provide any insight into its comparative advantages?<p>Edit: I did see the comparisons page, but.. well, there&#x27;s more to life than Rabbit and Kafka!

I worked on this team for 3 years along the OGs behind BMQ, specifically on a closed-source middleware with very similar architecture. I am very happy that they finally open sourced one of their main projects!<p>They have very strict standards which are necessary given that C++ really is a minefield. We did a great job avoiding UB by following those coding standards. There is some verbosity that can be avoided but hey these are details.<p>Personally, I have moved to Rust since then and I am not looking back.

I checked the docs, but couldn’t find anything about mqtt compat?

Built in C++, awesome :)

From the &quot;leadership election&quot; snippet, this setup looks perilously close to the peer to peer propagation of transactions and blocks in a blockchain system such as Solana. I suppose the latency requirements are much tighter than 400ms though.<p>I know Solana does mutual TLS over QUIC authentication between peers and does bandwidth rate limiting via stake. Also, the topology is self organizing. Is it worth copying this tech into a message queue system?

I wonder how this compares to something like Aeron performance-wise - though Aeron is strictly messaging (not queueing).

Congrats on open sourcing this, but I am curious if anyone here has ever had an actual scale issue that required changing from some &quot;slower&quot; queuing system (like RabbitMQ, Kestrel, Kafka, etc) where you actually needed to change queuing systems?<p>Maybe I&#x27;m old school but 99% of the use cases I&#x27;ve worked with in my 20 year career could scale with using a MySQL database table acting as a queue and some scripts querying the table and doing work. I have worked with maybe a dozen queuing technogies (including super expensive cloud Amazon SQS) and unless your app needs sub second latency on processing messages at greater than 5k per second I just don&#x27;t see why using these sophisticated systems have benefits?<p>Hope this comment doesn&#x27;t come across cynical or dismissive. I love seeing new tech like this come out and I am not saying that there aren&#x27;t use cases I am sure there (maybe HFT?) but curious if anyone has case studies to share of legit queue systems that couldn&#x27;t scale.

As much as Bloomberg (the guy and company) creep me out, this is impressive.

Very interesting, but looking up the supported standards it just uses TCP. There is no support for AMQP or MQTT. How is interoperability with just some client libraries for Python, Java, ...

Interesting that there&#x27;s a comparison (albeit low on details) to Rabbit, but not MQTT? Anyone had experience with both MQTT and BlazingMQ able to compare&#x2F;contrast?

Is this implemented with any sort of thought to security?<p>I don&#x27;t see anything that implies as such, all FAQ and comparisons to other message queues say nothing of security (I skimmed and ctrl+f so I could have missed it). This doesn&#x27;t bode well for a network-based application written in a memory unsafe language.<p>edit: I&#x27;m surprised so many people are interpeting this as a weird cargo cult thing. Application security includes a lot of mundane and critical things like:<p>Authentication, authorization, message signing &#x2F; authentication (e.g. HMAC), encryption, secret&#x2F;key management, how the system handles updates, etc.<p>You can read a bit more about it here: <a href="https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Application_security" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Application_security</a>

Is BlazingMQ faster than ZeroMQ?

TLA 0.1%<p>Finally! Thank you!

how does this compare against other existing message queuing systems?<p>the github shows its codebase is in c++ with gcc-7.3, that means it might be using c++14 in the code, or even c++17, which is nice.

It doesn&#x27;t use UDP. So my tongue-in-cheek reply is, it doesn&#x27;t qualify to have &quot;blazing&quot; in the name. Hopefully that&#x27;s not just funny inside my own head. :)<p><a href="https:&#x2F;&#x2F;bloomberg.github.io&#x2F;blazingmq&#x2F;docs&#x2F;architecture&#x2F;network_transport&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;bloomberg.github.io&#x2F;blazingmq&#x2F;docs&#x2F;architecture&#x2F;netw...</a>

Which version of C++ is this built on?

Java and C++ clients only?

&gt; “Carefully architected and written in C++”<p>Amazing to see a non-Rust product posted to HN.

Off topic here but<p>&gt; Carefully architected and written in C++ from the ground up with no dependency on any external framework, BlazingMQ provides…<p>What a weird selling point.<p>There are pros and cons to having no dependencies. Not long ago, it was a common decision for C++ projects because dependency management was a mess. But that hasn’t been the case for a long time (arguably we have the opposite problem - there are so many ways of doing it: Conan, vcpkg, bazel, spack, raw cmake, nix, etc)<p>So what would the pros and cons be today and why is it a selling point?<p>For instance, a pro is that everything is bespoke to the task at hand, no hammering square pegs into round holes designed for a slightly different use case.<p>A con is that the entire attack surface is managed by one team. CVEs identified and solved on another project is a pretty good thing when you depend on that project.<p>The main reason I’m surprised, though, is that there are some no-brainer dependencies these days. Fmt, catch2&#x2F;gtest, metaprogramming libraries, etc.

It&#x27;s called BlazingMQ but not written in Rust?!

420 messages per second.

LavinMQ is much better.

I don&#x27;t doubt that it is fast but please don&#x27;t name your thing &lt;claim&gt;&lt;thing&gt;.

I don&#x27;t understand how you can make queueing high-performance. Queueing things is literally the opposite of making things fast.