An update on Redis Streams development

I believe Redis has a great developer experience. It’s easy to get set up and use. When I see the work on Redis streams, I think its going bring a much better getting started experience for developers that want to start using evented architectures. This might be a turning point where we see more developers and applications utilizing those types of architectures. In the end, this might give Apache Kafka a run for its money, or not, who knows. I&#x27;ve tried using Apache Kafka and it can be a bear to set up given its dependencies and to administer.<p>Put another way, I think Kafka solved a problem for enterprises and was a tops down approach to the problem. Redis streams is a bottoms up approach to implementing evented architectures. Maybe there&#x27;s room for both products in the market?

I must admit, for the past couple months I&#x27;ve been digging for status info about Redis Streams. They would fit a use case we have perfectly, but we use cloud providers for Redis so manual compilation with modules isn&#x27;t possible.<p>Really pleased to see that I&#x27;m not the only one digging for info and that work is ongoing.

I wonder if this impacts the plan of releasing Disque as a plugin in Redis 4.2. I always thought that Disque could have a great impact in the field of job queues.

Are there any examples of good web APIs that offer something like a unified log as an abstraction? I&#x27;m not looking for systems, but actual companies that have some kind of &quot;streaming&quot; data feed where you can (re)connect to an endpoint and say &quot;give me everything from [logical] timestamp X&quot;. Ideally one where you stay connected and get longpoll SSE&#x2F;WebSockets&#x2F;MQTT-style streaming responses.<p>I kind of want the opposite of webhooks.

Great to see. We actually ended up using Redis for our event stream after trying Kafka and the rest. Using the list extension module multiple list pop functions lets us get to 2gbps of throughput on a single redis node with AOF persistence. Using streams would make things even simpler and faster.

Could someone ELI5 what&#x27;s a redis stream ? I thought that pub&#x2F;sub mecanisms were some kind of a stream already.

From the blog post, it sounds like the RDB format will break. Is there any means to upgrade from the current unstable RDB format to the v5 one? Obviously there are no consumer groups in unstable now, so in theory reading from a v4 RDB should not be hard..

Here&#x27;s a question from an old Redis hater (the note is important, since my question is going to be slightly biased - I disagree with a lot of core decisions behind Redis):<p>How is this going to be different from Kafka? And I don&#x27;t mean implementation details, because these are always fun read. Kafka is on the market for ~7 years, during which it has proven to be oh-so-fast and pretty durable.<p>Oh, and while I&#x27;m at it. Here&#x27;s another problem Redis geniousl added: a GIL. GIL is a great idea, but comes with huge tradeoffs. David Beazley spent years showing how many tricks you can play upon yourself with GIL.<p>So ... now you have Streams and GIL together. And you already have dicts (you call them hashmaps). I have a feeling you&#x27;re trying to implement Python. If so, it&#x27;s done. But come on, 3.6 is cool. And we&#x27;re kinda solving the GIL problem. With PyPy. Which will blow your mind.