Building a Distributed Log from Scratch: Sketching a New System

I am usually skeptical of links to blog posts people add to HN themselves because I'm skeptical of self-promotion more generally, but this series of posts is actually really insightful. I enjoyed reading it and have been passing it to some friends of mine.

This is interesting but the constant reference to NATS is more than a little irritating. It's one player among many in the space, and it's decisions aren't particularly notable. I would have loved to read something that was honestly "from first principles" rather than a pivot from the NATS architecture or whatever.

I am in the process of building a system like this, and a question I&#x27;ve been asked is, why the need for a message broker at all. It&#x27;s been a little difficult to justify once I really start to think about it.<p>Why not just set everything into essentially an S3 bucket, and then process off of it. What&#x27;s the benefit of a broker in between them really?