For those who are wondering the difference between Kafka/Pulsar and Liftbridge (from the docs):<p>"Liftbridge was designed to bridge the gap between sophisticated but complex log-based messaging systems like Apache Kafka and Apache Pulsar and simpler, cloud-native solutions. There is no ZooKeeper or other unwieldy dependencies, no JVM, no complicated API or configuration, and client libraries are just gRPC. More importantly, Liftbridge aims to extend NATS with a durable, at-least-once delivery mechanism that upholds the NATS tenets of simplicity, performance, and scalability. Unlike NATS Streaming, it uses the core NATS protocol with optional extensions. This means it can be added to an existing NATS deployment to provide message durability with no code changes. The ultimate goal of Liftbridge is to provide a message-streaming solution with a focus on simplicity and usability."
This is very cool OP. Amazing work.<p>For anyone looking for something <i>similar</i>, we are adding streams to Postgres - <a href="https://github.com/supabase/realtime" rel="nofollow">https://github.com/supabase/realtime</a><p>It isn’t an exact comparison but it has some of the features you see here too - you can use wildcards listen to: all changes in your database, schema-level, table-level changes, or row-level. The server is built with Elixir, so you can have thousands of listeners connected via web sockets.<p>It’s not as lightweight as OP’s solution but PG is very familiar.
Author here. I started Liftbridge after working as a core committer on NATS and NATS Streaming (as well as my experience using Kafka in the past). In particular, implementing replication in NATS Streaming and drawing from that experience/lessons learned. Happy to answer any questions!
I've been watching this with some interest as the successor of NATS streaming. One question I have is how does it fair in potential-high-latency, geographically distributed situations? E.g. what if I had dozens of servers at the edge worldwide wanting to receive events from one another?
The project has basically jumped from v0.0.1 to v1.0.0 within 4-5 months ... but how does it fare with reliability, throughput, ordering and deliveries?<p>Is it PROD ready? are there any use-cases or project sizes this utility is NOT recommended for? i.e. sensitive events such as payment or ecomm related vs. web traffic noise logs