Back to basics: Why we chose long-polling over websockets

Long polling has some problems of its own.<p>Second Life has an HTTPS long polling channel between client and server. It&#x27;s used for some data that&#x27;s too bulky for the UDP connection, not too time sensitive, or needs encryption. This has caused much grief.<p>On the client side, the poller uses libcurl. Libcurl has timeouts. If the server has nothing to send for a while, libcurl times out. The client then makes the request again. This results in a race condition if the server wants to send something between timeout and next request. Messages get lost.<p>On top of that, the real server is front-ended by an Apache server. This just passes through relevant requests, blocking the endless flood of junk HTTP requests from scrapers, attacks, and search engines. Apache has a timeout, and may close a connection that&#x27;s in a long poll and not doing anything.<p>Additional trouble can come from middle boxes and proxy servers that don&#x27;t like long polling.<p>There are a lot of things out there that just don&#x27;t like holding an HTTP connection open. Years ago, a connection idle for a minute was fine. Today, hold a connection open for ten seconds without sending any data and something is likely to disconnect it.<p>The end result is an unreliable message channel. It has to have sequence numbers to detect duplicates, and can lose messages. For a long time, nobody had discovered that, and there were intermittent failures that were not understood.<p>In the original article, the chart section labelled &quot;loop&quot; doesn&#x27;t mention timeout handling. That&#x27;s not good. If you do long polling, you probably need to send something every few seconds to keep the connection alive. Not clear what a safe number is.

Articles like this make me happy to use Phoenix and LiveView every day. My app uses WebSockets and I don’t think about them at all.

Would there be any technical benefit to this over using server sent events (SSE)?<p>Both are similar in that they hold the HTTP connection open and have the benefit of being simply HTTP (the big plus here). SSE (at least to me) feels like it&#x27;s more suitable for some use cases where updates&#x2F;results could be streamed in.<p>A fitting use case might be where you&#x27;re monitoring all job IDs on behalf of a given client. Then you could move the job monitoring loop to the server side and continuously yield results to the client.

I think this article is tying a lot of unrelated decisions to &quot;Websocket&quot; vs &quot;Long-polling&quot; when they&#x27;re actually independent. A long-polling server could handle a websocket client with just a bit of extra work to handle keep-alive.<p>For the other direction, to support long-polling clients if your existing architecture is websockets which get data pushed to them by other parts of the system, just have two layers of servers: one which maintains the &quot;state&quot; of the connection, and then the HTTP server which receives the long polling request can connect to the server that has the connection state and wait for data that way.

Unrelated to the topic in the article…<p><pre><code> await new Promise(resolve =&gt; setTimeout(resolve, 500)); </code></pre> In Node.js context, it&#x27;s easier to:<p><pre><code> import { setTimeout } from &quot;node:timers&#x2F;promises&quot;; await setTimeout(500);</code></pre>

I like long polling, it’s easy to understand from start to finish and from client perspective it just works like a very slow connection. You have to keep track of retries and client-side cancelled connections to have one but only one (and the right one) of requests at hand to answer to.<p>One thing that seems clumsy in the code example is the loop that queries the data again and again. Would be nicer if the data update could also resolve the promise of the response directly.

Neither Server-Sent Events nor WebSockets have replaced all use cases of long polling reliably. The connection limit of SSE comes up a lot, even if you’re using HTTP&#x2F;2. WebSockets, on the other hand, are unreliable as hell in most environments. Also, WS is hard to debug, and many of our prod issues with WS couldn’t even be reproduced locally.<p>Detecting changes in the backend and propagating them to the right client is still an unsolved problem. Until then, long polling is surprisingly simple and a robust solution that works.

Since the article mentioned Postgres by name, isn&#x27;t this a case for using its asynchronous notification features? Servers can LISTEN to a channel and PG can TRIGGER and NOTIFY them when the data changes.<p>No polling needed, regardless of the frontend channel.

I don&#x27;t know how meaningful it is any more, but with long polling with a short timeout and a gracefully ended request (i.e. chunked encoding with an eof chunk sent rather than disconnection), the browser would always end up with one spare idle connection to the server, making subsequent HTTP requests for other parts of the UI far more likely to be snappier, even if the app has been left otherwise idle for half the day<p>I guess at least this trick is still meaningful where HTTP&#x2F;2 or QUIC aren&#x27;t in use

Refreshing to be reminded of a relatively simple alternative to websockets. For a short time, I worked at a now-defunct startup which had made the decision for websockets. It was an app that would often be used on holiday so testing was done on hotel and restaurant wifi. Websockets made that difficult.

What about HTTP&#x2F;2 Multiplexing, how does it hold up against long-polling and websockets?<p>I have only tried it briefly when we use gRPC: <a href="https:&#x2F;&#x2F;grpc.io&#x2F;docs&#x2F;what-is-grpc&#x2F;core-concepts&#x2F;#server-streaming-rpc" rel="nofollow">https:&#x2F;&#x2F;grpc.io&#x2F;docs&#x2F;what-is-grpc&#x2F;core-concepts&#x2F;#server-stre...</a><p>Here it&#x27;s easy to specify that a endpoint is a &quot;stream&quot;, and then the code-generation tool gives all tools really to just keep serving the client with multiple responses. It looks deceptively simple. We already have setup auth, logging and metrics for gRPC, so I hope it just works off of that maybe with minor adjustments. But I&#x27;m guessing you don&#x27;t need the gRPC layer to use HTTP&#x2F;2 Multiplexing?

The points mentioned against websockets are mostly fud, I&#x27;ve used websockets in production for a very heavy global data streaming application, and I would respond the following to the &quot;upsides&quot; of not using websockets:<p>&gt; Observability Remains Unchanged<p>Actually it doesn&#x27;t, many standard interesting metrics will break because long-polling is not a standard request either.<p>&gt; Authentication Simplicity<p>Sure, auth is different than with http, but not more difficult. You can easily pass a token.<p>&gt; Infrastructure Compatibility<p>I&#x27;m sure you can find firewalls out there where websockets are blocked, however for my use case I have never seen this reported. I think this is outdated, for sure you don&#x27;t need &quot;special proxy configurations or complex infrastructure setups&quot;.<p>&gt; Operational Simplicity<p>Restarts will drop any persistent connection, state can be both or neither in WS or in LP, it doesn&#x27;t matter what you use.<p>&gt; Client implementation<p>It mentions &quot;no special WebSocket libraries needed&quot; and also &quot;It works with any HTTP client&quot;. Guess what, websockets will work with any websocket client! Who knew!<p>Finally, in the conclusion:<p>&gt; For us, staying close to the metal with a simple HTTP long polling implementation was the right choice<p>Calling simple HTTP long polling &quot;close to the metal&quot; in comparison to websockets is weird. I wouldn&#x27;t be surprised if websockets scale much better and give much more control depending on the type of data, but that&#x27;s besides the point. If you want to use long polling because you prefer it, go ahead. Its a great way to stick to request&#x2F;response style semantics that web devs are familiar with. Its not necessary to regurgitate a bunch of random hearsay arguments that may influence people in the wrong way.<p>Try to actually leave the reader with some notion of when to use long polling vs when to use websockets, not a post-hoc justification of your decision based on generalized arguments that do not apply.

I would appreciate if the article spent more time actually discussing the benefits of websockets (and&#x2F;or more modern approaches to pushing data from server -&gt; browser) and why the team decided those benefits were not worth the purported downsides. I could see the same simplicity argument being applied to using unencrypted HTTP&#x2F;1.1 instead of HTTP&#x2F;2, or TCP Reno instead of CUBIC.<p>The section at the end talking about &quot;A Case for Websockets&quot; really only rehashes the arguments made in &quot;Hidden Benefits of Long-Polling&quot; stating that you need to reimplement these various mechanisms (or just use a library for it).<p>My experience in this space is from 2011, when websockets were just coming onto the scene. Tooling &#x2F; libraries were much more nascent, websockets had much lower penetration (we still had to support IE6 in those days!), and the API was far less stable prior to IETF standardization. But we still wanted to use them when possible, since they provided much better user experience (lower latency, etc) and lower server load.

I think we could have the best of both worlds. E.g.:<p><a href="https:&#x2F;&#x2F;socket.io&#x2F;docs&#x2F;v4&#x2F;how-it-works&#x2F;#upgrade-mechanism" rel="nofollow">https:&#x2F;&#x2F;socket.io&#x2F;docs&#x2F;v4&#x2F;how-it-works&#x2F;#upgrade-mechanism</a>

Long polling is my choice for simple, reliable and plug and play like interfaces. HTTP requests tend to be standard and simplify authentication as well. Systems with frequent but not constant updates are ideal. Text yes. Voice maybe not.<p>Personal Case Study: I built mobile apps which used Flowise assistants for RAG and found websockets compeletely out of line with the rest of my system and interactions. Suddenly I was fitting a round peg in a square hole. I switched to OpenAI assistants and their polling system felt completely &quot;natural&quot; to integrate.

Why not just use chunked encoding and get rid of extra requests.

I think they are mixing some problems here. They could probably have used their original setup with Postgres NOTIFY+triggers in stead of polling, and only have one &quot;pickup poller&quot; to catch any missed events&#x2F;jobs. In my opinion transaction medium should not be linked to how the data is manage internally, but I know from experience that this separation is often hard to achieve in practice.

The article does discuss a lot of mixed concepts. I would prefer one process polling new jobs&#x2F;state and one process handling http connections&#x2F;websockets. Hence no flooding the database and completely scalable from the client side. The database process pushes everything downstream via some queue while the other process&#x2F;server handles those and sends them to respective clients

The full schema isn’t listed, but the indices don’t make sense to me.<p>(id, cluster_id) sounds like it could &#x2F; should be the PK<p>If the jobs are cleared once they’ve succeeded, and presumably retried if they’ve failed or stalled, then the table should be quite small; so small, that a. The query planner is unlikely to use the partial index on (status) b. The bloat from the rapidity of DELETEs likely overshadows the live tuple size.

I implemented a long polling solution in desktop software over 20 years ago and it’s still working great. It can even be used as a tunnel to stream RDP sessions, through which YouTube can play without a hiccup. Big fan of long polling, though I admit I didn’t get a chance to try web sockets back then.

Half-OT:<p>What&#x27;s the most resource efficient way to push data to clients over HTTP?<p>I can send data to a server via HTTP request, I just need a way to notify a client about a change and would like to avoid polling for it.<p>I heard talk about SSE, WebSockets, and now long-polling.<p>Is there something else?<p>What requires the least resources on the server?

&gt;Our system handles hundreds of worker nodes constantly polling our PostgreSQL-backed control plane for new jobs<p>Does everybody poll their PosgreSQL to get new rows in real-time? This is really weird, there are trigger functions and notifications.

&gt; Corporate firewalls blocking WebSocket connections was one of our other worries. Some of our users are behind firewalls, and we don&#x27;t need the IT headache of getting them to open up WebSockets.<p>Don&#x27;t websockets look like ordinary https connections?

Can someone explain why TTL = 60s is a good choice? Why not more, or less?

yeah, authentication complexity with WebSockets is severely underappreciated. We ran into major RBAC headaches when clients needed to switch between different privilege contexts mid-session. Long polling with standard HTTP auth patterns eliminates this entire class of problems.

Another reason: there is a patent troll suing companies over usage of websockets.

Given long polling, I have never ever understood why websockets became a thing. I’ve never implemented them and never will, it’s a protocol extension where none is necessary.

Using websockets with graphql, I feel like a lot of the challenges are then already solved. From the post:<p>- Observability: WebSockets are more stateful, so you need to implement additional logging and monitoring for persistent connections: solved with graphql if the existing monitoring is already sufficient.<p>- Authentication: You need to implement a new authentication mechanism for incoming WebSocket connections: solved with graphql.<p>- Infrastructure: You need to configure your infrastructure to support WebSockets, including load balancers and firewalls: True, firewalls need to be updated.<p>- Operations: You need to manage WebSocket connections and reconnections, including handling connection timeouts and errors: normally already solved by the graphql library. For errors, it&#x27;s basically the same though.<p>- Client Implementation: You need to implement a client-side WebSocket library, including handling reconnections and state management: Just have to <i>use</i> a graphql library that comes with websocket support (I think most of them do) and configure it accordingly.