It's my pleasure to share my open-source project with the HN community:<p>KanthorQ - Messaging System Backed by PostgreSQL<p>Repo: <a href="https://github.com/kanthorlabs/kanthorq">https://github.com/kanthorlabs/kanthorq</a>
Docs: <a href="https://docs.kanthorlabs.com/kanthorq/" rel="nofollow">https://docs.kanthorlabs.com/kanthorq/</a><p>I designed, developed, and maintained it to address most of my use cases. Here are some key features:<p>- Transactional enqueuement: Helps avoid situations where you successfully enqueue a message, but your compute is lost, or the opposite scenario occurs.<p>- Message persistence: Messages are stored in the database indefinitely until explicitly deleted. This makes tracing, monitoring, and debugging much easier.<p>- Categorization by subjects: Simplifies message filtering and organization by subject.<p>If you're familiar with NATS.io, you'll find it easy to get started with KanthorQ. I was an avid user of NATS.io, and it inspired me greatly while designing KanthorQ.<p>Some example usecases<p>- Webhook gateway<p>- Batch jobs<p>- Notifications
The funny story behind KanthorQ is that it stemmed from a challenge I faced with another project: an open-source webhook gateway (<a href="https://github.com/kanthorlabs/kanthor">https://github.com/kanthorlabs/kanthor</a>). After completing that project and starting some benchmarks, I realized I was dealing with issues I had never encountered before.<p>When something went wrong, I had to scan through a list of events to identify the failed ones. However, doing that with traditional message brokers like NATS.io or RabbitMQ was challenging. I needed an additional layer to store the events and analyze the data. As a result, a simple system required two high-availability components to function properly.<p>Additionally, backup and restoration were not straightforward with traditional message brokers like NATS.io or RabbitMQ. It's difficult to back up and restore these systems—or at least, I couldn't find a reliable way to do it.<p>That's why I developed KanthorQ. Now, I have a message broker where data is stored in a database, making it easy to browse, analyze, back up, and restore.