A while ago I created Mercure: an open pub-sub protocol built on top of SSE that is a replacement for WebSockets-based solutions such as Pusher. Mercure is now used by hundreds of apps in production.<p>At the core of Mercure is the hub. It is a standalone component that maintains persistent SSE (HTTP) connections to the clients, and it exposes a very simple HTTP API that server apps and clients can use to publish. POSTed updates are broadcasted to all connected clients using SSE. This makes SSE usable even with technologies not able to maintain persistent connections such as PHP and many serverless providers.<p>Mercure also adds nice features to SSE such as a JWT-based authorization mechanism, the ability to subscribe to several topics using a single connection, events history, automatic state reconciliation in case of network issue…<p>I maintain an open-source hub written in Go (technically, a module for the Caddy web server) and a SaaS version is also available.<p>Docs and code are available on <a href="https://mercure.rocks" rel="nofollow">https://mercure.rocks</a>
It doesn’t mention the big drawback of SSE as spelled out in the MDN docs:<p>“Warning: When not used over HTTP/2, SSE suffers from a limitation to the maximum number of open connections, which can be especially painful when opening multiple tabs, as the limit is per browser and is set to a very low number (6).”
I utilized SSE when building automatic restart functionality[0] into Doppler's CLI. Our api server would send down an event whenever an application's secrets changed. The CLI would then fetch the latest secrets to inject into the application process. (I opted not to directly send the changed secrets via SSE as that would necessitate rechecking the access token that was used to establish the connection, lest we send changed secrets to a recently deauthorized client). I chose SSE over websockets because the latter required pulling in additional dependencies into our Golang application, and we truly only needed server->client communication.
One issue we ran into that hasn't been discussed is HTTP timeouts. Some load balancers close an HTTP connection after a certain timeout (e.g. 1 hour) to prevent connection exhaustion. You can usually extend this timeout, but it has to be explicitly configured. We also found that our server had to send intermittent "ping" events to prevent either Cloudflare or Google Cloud Load Balancing from closing the connection, though I don't remember how frequently these were sent. Otherwise, SSE worked great for our use case.<p>[0] <a href="https://docs.doppler.com/docs/automatic-restart">https://docs.doppler.com/docs/automatic-restart</a>
> Perceived Limitations: The unidirectional nature might seem restrictive, though it's often sufficient for many use cases<p>For my use cases the main limitations of SSE are:<p>1. Text-only, so if you want to do binary you need to do something like base64<p>2. Browser connection limits for HTTP/1.1, ie you can only have ~6 connections per domain[0]<p>Connection limits aren't a problem as long as you use HTTP/2+.<p>Even so, I don't think I would reach for SSE these days. For less latency-sensitive and data-use sensitive applications, I would just use long polling.<p>For things that are more performance-sensitive, I would probably use fetch with ReadableStream body responses. On the server side I would prefix each message with a 32bit integer (or maybe a variable length int of some sort) that gives the size of the message. This is far more flexible (by allowing binary data), and has less overhead compared to SSE, which requires 7 bytes ("data:" + "\n\n") of overhead for each message.<p>[0]: <a href="https://stackoverflow.com/a/985704" rel="nofollow">https://stackoverflow.com/a/985704</a>
One thing I dislike regards to SSE, which is not its fault but probably a side effect of the perceived simplicity: lots of developers do not actually use proper implementations and instead just parse the data chunks with regex, or something of the sorts! This is bad because SSE, for example, supports comments (": text") in streams, which most of those hand-rolled implementations don't support.<p>For example, my friend used an LLM proxy that sends keepalive/queue data as SSE comments (just for debugging mainly), but it didn't work for Gemini, because someone at Google decided to parse SSE with a regex:
<a href="https://github.com/google-gemini/generative-ai-js/blob/main/src/requests/stream-reader.ts#L28">https://github.com/google-gemini/generative-ai-js/blob/main/...</a> (and yes, if the regex doesn't match the complete line, the library will just throw an error)
<a href="https://data-star.dev" rel="nofollow">https://data-star.dev</a> is a hypermedia-oriented front end library built entirely around the idea of streaming hypermedia responses via SSE.<p>It was developed using Go & NATS as backend technologies, but works with any SSE implementation.<p>Worth checking out if you want to explore SSE and what can be achieved w/it more deeply. Here is an interview with the author:<p><a href="https://www.youtube.com/watch?v=HbTFlUqELVc" rel="nofollow">https://www.youtube.com/watch?v=HbTFlUqELVc</a>
The part where it says:<p>> SSE works seamlessly with existing HTTP infrastructure:<p>I'd be careful with that assumption. I have tried using SSE through some 3rd party load balancer at my work and it doesn't work that well. Because SSE is long-lived and doesn't normally close immediately, this load balancer will keep collecting and collecting bytes from the server and not forward it until server closes the connection, effectively making SSEs useless. I had to use WebSockets instead to get around this limitation with the load balancer.
I tried implementing SSE in a web project of mine recently, and was very surprised when my website totally stopped working when I had more than 6 tabs open.<p>It turns out, Firefox counts SSE connections against the 6 host max connections limit, and gives absolutely no useful feedback that it's blocking the subsequent requests due to this limit (I don't remember the precise error code and message anymore, but it left me very clueless for a while). It was only when I stared at the lack of corresponding server side logs that it clicked.<p>I don't know if this same problem happens with websockets or not.
OpenAI's own documentation makes note of how difficult it is to work with SSE and to just use their library instead. My team wrote our own parser for these streaming events from an OpenAI compatible LLM server. The streaming format is awful. The double newline block separator also shows up in a bunch of our text, making parsing a nightmare. The "data:" signifier is slightly better, but when working with scientific software, it still occurs too often. Instead we've had to rely on the totally-not-reliable fact that the server returns each as a separate packet and the receiving end can be set up to return each packet in the stream.<p>The suggestions I've found online for how to deal with the newline issue are to fold together consecutive newlines, but this loses formatting of some documents and otherwise means there is no way to transmit data verbatim. That might be fine for HTML or other text formats where newlines are pretty much optional, but it sucks for other data types.<p>I'm happy to have something like SSE but the protocol needs more time to cook.
They're underrated when they work™<p>Currently at work I'm having issues because
- Auth between an embedded app and javascript's EventSource is not working, so I have to resort to a Microsoft package which doesn't always work.
- Not every tunnel is fond of keep-alive (Cloudflare), so I had to switch to ngrok (until I found out they have a limit of 20k requests).<p>I know this isn't the protocol's fault, and I'm sure there's something I'm missing, but my god is it frustrating.
> SSE works seamlessly with existing HTTP infrastructure<p>This is false. SSE is not supported on many proxies, and isn't even supported on some common local proxy tooling.
SSE is not underrated. In fact it's being used by Open AI for streaming completions. It's just not always needed unlike the very obvious use cases for normal REST APIs and Websockets.<p>It was a pain to figure out how to get it to work in a ReactJS codebase I was working on then and from what I remember Axios didn't support it then so I had to use native fetch to get it to work.
I’ve never understood the use of SSE over ndjson. Builtin browser support for SSE might be nice, but it seems fairly easy to handle ndjson? For non-browser consumers ndjson is almost assuredly easier to handle. ndjson works over any transport from HTTP/0.9 to HTTP/3 to raw TCP or unix sockets or any reliable transport protocol.
> Automatic Reconnection<p>I wouldn't characterize this as "automatic", you have to do a lot of manual work to support reconnection in most cases. You need to have a meaningful "event id" that can be resumed on a new connection/host somewhere else with the Last-Event-Id header. The plumbing for this event id is the trivial part IMO. The hard part is the server-side data synchronization, which is left as an exercise for the reader.<p>Also, God help you if your SSE APIs have side effects. If the API call is involved in a sequence of side-effecting steps then you'll enter a world of pain by using SSE. Use regular HTTP calls or WebSockets. (Mostly b/c there's no cancellation ack, so retries are often racy.)
Great post. I discovered SSE when building a chatbot and found out it’s what OpenAI used rather than WebSockets. The batteries-included automatic reconnection is huge, and the format is surprisingly human readable.
Does anyone have a good trick for figuring out when the client side connection is closed? I just kill the connection on the server every N minutes and force the client to reconnect, but it's not exactly graceful.<p>Secondly, on iOS mobile, I've noticed that the EventSource seems to fall asleep at some point and not wake up when you switch back to the PWA. Does anyone know what's up with that?
I love the format of this. I can tuck it away and whenever I need to think about SSE, refer back to it, ah, okay, now I'm all caught up.<p>Also, surprised nobody's brought up PointCast[1] yet. Dotcom bubble, or ahead of their time?<p>(Aside: while looking for a good reference link for Pointcast, I found an All Things Considered episode[2] about it from 1996!)<p>[1] <a href="https://www.ecommerce-digest.com/early-dot-com-failure-case-studies.html#:~:text=Pointcast.Com" rel="nofollow">https://www.ecommerce-digest.com/early-dot-com-failure-case-...</a><p>[2] <a href="https://www.npr.org/1996/02/13/1042379/pointcast" rel="nofollow">https://www.npr.org/1996/02/13/1042379/pointcast</a>
They are handy for implementing simple ad-hoc hot reloading systems as well. E.g. you can have whatever file watcher you are using call an API when a file of interest changes that sends an event to listening clients on the frontend. You can also trigger an event after restarting the backend if you make an API change by triggering the event at boot time. Then you can just add a dev-only snippet to your base template that reloads the page or whatever. Better than nothing if your stack doesn't support it out of the box and doesn't take very much code or require adding any additional project dependencies. Not as sophisticated as React environments that will only reload a component that changed and only do a full page refresh if needed, but it still gives a nice, responsive feeling when paired with tools that recompile your backend when it changes.
No they're not.
They're limited to 6 clients per browser per domain on http/1.1
Which is important because nginx can't reverse proxy http/2 or higher, so you end up with very weird functionality, essentially you can't use nginx with SSE.<p><a href="https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#sect1" rel="nofollow">https://developer.mozilla.org/en-US/docs/Web/API/Server-sent...</a><p>Edit: I see someone already posted about that
I've had no idea they exist until I began to use APIs serving LLM outputs. They work pretty well for this purpose from my experience. An alternative to SSE is web sockets for this purpose I suppose
I had some trouble implementing when server has to wait for another endpoint (webhook) to feed output to browser . During request processing (thread1) had to store the sse context in native Java object which will be retrieved later when webhook(thread2) is called
But then with multiple service instance you wouldn't know which service had stored it so webhook had to publish something which others has to subscribe.
I built several internal tool to tail logs using SSE with Flask/FastAPI. Easy to implement and maintain.<p>For FastAPI if you want some hooks when client disconnects aka nginx 499 errors, follow this simple tip<p><a href="https://github.com/encode/starlette/discussions/1776#discussioncomment-3207518">https://github.com/encode/starlette/discussions/1776#discuss...</a>
I'm curious as to how everyone deals with HTTP/2 requirements between the backend servers and the load balancer? By default, HTTP/2 requires TLS which means either no SSL termination at the load balancer or a SSL cert generated per server with a different one for the front end load balancer. This all seems very inefficient.
Finding use cases for SSE and reading about others doing the same brings me great joy. Very easy to set up -- you just set 2 or 3 response headers and off you go.<p>I have a hard time imagining the tech's limits outside of testing scenarios so some of the examples brought up here are interesting
This is why SignalR in .NET land is so popular in that ecosystem. It automatically falls back to SSE if web sockets aren’t available. In essence giving totally transparent real time communication.
So it’s websockets, only instead of the Web server needing to handle the protocol upgrade, you just piggyback on HTTP with an in-band protocol.<p>I’m not sure this makes sense in 2024. Pretty much every web server supports websockets at this point, and so do all of the browsers. You can easily impose the constraint on your code that communication through a websocket is mono-directional. And the capability to broadcast a message to all subscribers is going to be deceptively complex, no matter how you broadcast it.