Why you can have millions of goroutines but only thousands of Java threads

Fwiw, the original implementation of threads in Java were &#x27;green&#x27; threads (it was project Green after all). These were implemented in SunOS (vs System V) using setjmp&#x2F;longjmp and an some auto-yield logic in the JVM (basically when you did certain things it would start that thing and yield your thread while it completed). When I started I had the most System V experience so I ported it over the Solaris to get to what was affectionately called the &#x27;hello hello hello world world world&#x27; stage. The choice to use the Solaris native threads over green threads was motivated by the work that had gone on in the Solaris kernel to do CPU scheduling efficiently. That gave Java multiprocessing for &#x27;free&#x27;. Except it exposed a bunch of bugs in the mutex &#x2F; synchronized code when you actually got parallel execution :-).<p>The magic, as the author points out, is dynamic stack allocation. If you disassociate a thread with a required stack size, then you are back to using just the amount of memory that your object heap and reference stores are using. GC gets to be trickier to of course.<p>This should nominally be a non-issue on 64 bit address machines as you only need <i>map</i> the pages that actually have data in them (minimum one page) but that still puts a huge load on the page tables. This is especially true with huge pages which are used to avoid tlb thrashing and to save on the number of levels you have to go through to get back the physical address of the page you are trying to access.<p>Its the kind of optimization problem that systems folks really like to dig into and optimize around.

It&#x27;s all about fixed resource allocations. The default stack size for user-land threads tends to be 1MB or 2MB, and there&#x27;s also (smaller) kernel stacks.<p>In implementations of languages like Scheme or Haskell where no stack may even be allocated, or where stacks might be allocated in chunks, the cost of a co-routine can be very small -- as small as the cost of a closure. If you take that to the limit and allocate every call frame on the heap, and if you make heavy use of closures&#x2F;continuations, then you end up with more GC pressure because instead of freeing every frame on return you have to free many of them via the GC.<p>In terms of ease of programming to deal with async events, the spectrum runs from threads on the one hand to callback hell on the other. Callback hell can be ameliorated by allowing lambdas and closures, but you still end up with indentation and&#x2F;or paren&#x2F;brace hell. Co-routines are somewhere in the middle of the spectrum, but closer to threads than to callback hell. Await is something closer to callback hell with nice syntax to make things easier on the programmer.<p>Ultimately it&#x27;s all about how to represent state.<p>In threaded programming program state is largely implicit in the call stack (including all the local variables). In callback hell program state is largely explicit in the form of the context argument that gets passed to the callback functions (or which they close over), with a little bit of state implicit in the extant event registrations.<p>Threaded programming is easier because the programmer doesn&#x27;t have to think about how to compactly represent program state, but the cost is higher resource consumption, especially memory. More memory consumption == more cache pressure == more cache misses == slower performance.<p>Callback hell is much harder on the programmer because it forces the programmer to be much more explicit about program state, but this also allows the programmer to better compress program state, thus using fewer resources, thus allowing the program to deal with many more clients at once and also be faster than threaded programming.<p>Everything in computer science in the async I&#x2F;O space in the last 30 years has been about striking the right balance between minimizing program state on the one hand and minimizing programmer pain on the other. Continuations, delineated continuations, await, and so on -- all are about finding that sweet spot.

I&#x27;m surprised Loom hasn&#x27;t been mentioned yet in this thread.<p>Proposal: <a href="http:&#x2F;&#x2F;cr.openjdk.java.net&#x2F;~rpressler&#x2F;loom&#x2F;Loom-Proposal.html" rel="nofollow">http:&#x2F;&#x2F;cr.openjdk.java.net&#x2F;~rpressler&#x2F;loom&#x2F;Loom-Proposal.htm...</a><p>It&#x27;s in the early phase, but there is a prototype currently.<p>The commits in the mailing list mention a recent zero copy continuation thawing if I&#x27;m not mistaken.<p><a href="http:&#x2F;&#x2F;mail.openjdk.java.net&#x2F;pipermail&#x2F;loom-dev&#x2F;2018-October&#x2F;thread.html" rel="nofollow">http:&#x2F;&#x2F;mail.openjdk.java.net&#x2F;pipermail&#x2F;loom-dev&#x2F;2018-October...</a><p>EDIT: I&#x27;m mistaken, it is about lazy stack walking (but I remember zero copy continuation thawing might be attempted)

&gt; In Apache, each request is handled by 1 OS thread, effectively capping Apache to thousands of concurrent connections<p>That&#x27;s wrong. There are a variety of different Apache mpm&#x27;s: <a href="https:&#x2F;&#x2F;httpd.apache.org&#x2F;docs&#x2F;2.4&#x2F;mpm.html" rel="nofollow">https:&#x2F;&#x2F;httpd.apache.org&#x2F;docs&#x2F;2.4&#x2F;mpm.html</a><p>In the footnotes, it also points out that Erlang uses a similar system, which is true, and worth a look as well.

In typical Java fashion, there is a project, Quasar, the adds true green thread support to the language. Done by instrumenting bytecode rather than JVM support, but the benchmarks are still excellent. I&#x27;m very curious whether the Java or Go implementation is faster. Java does have fast async support which is all you need to add green threads on a library level<p>Also in Java fashion, the support isn&#x27;t perfect, and requires some fiddling with JVM parameters (and annotations, obivously). It&#x27;s still really solid from my experience however, and a shame that it isn&#x27;t more widely known.<p>A partner project Comsat provides fairly comprehensive library support for databases and such. I&#x27;ve used it before with Vert.X and the performance was insane. Possibly faster than Go.<p>Another aside, there&#x27;s an active research project to add greaan threads to Javas core as well. Project Loom

Go may have more threads, but it is not better at concurrency. It lacks several key things that make Java much easier to scale up, even if Go is easier to get started with. On my wish list:<p>* Thread Local storage. A million Goroutines means nothing is they are all fighting over shared storage. Consider trying to implement Java&#x27;s LongAdder class, which does intelligent sharding based on contention.<p>* ConcurrentHashMap. sync.Map is horrendously contentious for writes. It&#x27;s not even possible to write one yourself because Go hides the built in hash function it generates for each type.<p>* Goroutine joining. This is between a rock and a hard place, because if you don&#x27;t wait till Goroutines are done, you risk leaks, and if you do wait, you need to put WaitGroups all over the place.<p>All those millions of goroutines don&#x27;t help if you don&#x27;t have the tools to coordinate them.

The article is off by an order of magnitude in one of the more headline grabbing comparisons. It states 2.5 million goroutines in a gigabyte of RAM, when it should be 250k at 4KB per stack. Still impressive on its difference, but less so.

I think it is so important to not arbitrarily throw words like &#x27;RAM&#x27; in such discussions, and use more unambiguous terminology such as - virtual address space and the resident set size (RSS).<p>On a 64-bit machine, with 1MB stack size (this is a claim on the virtual address space, not the physical memory), you can have millions of threads too (that you may hit a lower limit due to other OS level knobs is another matter).

The article conflates Java and JVM. Kotlin ( on the JVM) has coroutines and can easily run 100k at a time.

<i>&gt; Supporting truly massive concurrency requires another optimization: Only schedule a thread when you know it can do useful work!</i><p>The problem is the scheduler doesn&#x27;t know when a goroutine can do useful work. It thinks that if a new value arrives to a channel there is a good chance the goroutine will do something useful but that&#x27;s not the case in general. My favorite example is Game of Life (CA) implemented as a network of goroutines communicating their state changes through channels: you have to collect 8 updates from predecessors to update your state, meaning you&#x27;ll be scheduled to run 8 times before you finally make a real progress. Not good for scalability.

&gt; CPUs can only execute about one logical thread per core truly concurrently.1 An inherent side effect: If you have more threads than cores, threads must be paused to allow other threads to do work, then later resumed when it’s their turn again.<p>When would millions of goroutines be useful if you can only run ~4 threads in parallel?

joe damato claims the reason threads were historically slower than userspace threading is because of an accident of map_32bit in glibc<p>&gt; So I think I&#x27;ve identified the culprit and I believe the culprit is XFree86.<p>&gt; it took seven years after this change went in for someone to figure out all the different parts that were damaged and fix them later, right? So threads were broken for a really long time.<p><a href="https:&#x2F;&#x2F;www.deconstructconf.com&#x2F;2017&#x2F;joe-damato-all-programmers-must-learn-c-and-assembly" rel="nofollow">https:&#x2F;&#x2F;www.deconstructconf.com&#x2F;2017&#x2F;joe-damato-all-programm...</a>

What&#x27;s the use-case for many virtual threads? I thought the main case for using threads was performance, where you need to use them to fully utilize the machine, although rather you wouldn&#x27;t be using threads.<p>Another use-case for threads that I see is when you have work-queues that have different priorities, so you put each queue into its own thread with a matching priority. For example, 1 thread for responding to UI events, and 1 lower priority thread for all background work.<p>Anyways, none of these use-cases require many threads. What use-case am I missing that makes go-routines useful?<p>I guess doing something in a thread is a fail-safe way to make sure that &#x27;some time&#x27; is spent processing it, so for example for the UI tasks, it would be easy to just dump all handling of UI events in virtual threads.

<i>For use cases where a high degree of concurrency is required, it’s simply the only option.</i><p>You can get high concurrency in Java by avoiding the 1 thread per request&#x2F;connection model and using NIO with easy-to-use libraries like Netty.

This article is wrong and reflects what appears to be a widely-held belief (practically canon here on HN) that a thread has a fixed-size stack. This is not the case on every operating system! Linux, which I&#x27;ve heard is popular, initially commits one page for a thread stack and continues to add pages until that thread&#x27;s stack limit is hit. So this statement from which this article draws its conclusion is incorrect:<p>&quot;each OS thread has its own fixed-size stack.&quot;<p>100% wrong.

Comparing Java threads and goroutines doesn&#x27;t really work because Java&#x27;s thread model maps threads 1:1 to OS threads, whereas goroutines are simply continuations that can consume any OS thread. Java threads couple both the concerns of scheduling and task definition. Java Runnables address only the concern of task definition and are closer in nature to what goroutines are, but they still don&#x27;t possess the ability to suspend and yield their execution.<p>You can implement Go&#x27;s thread model (more generally known as green threads, coroutines, fibers, lightweight threads) on the JVM. This is precisely what Project Loom intends to do and what Scala effect systems already do. You construct a program in a type called IO that only describes a computation that can be suspended and resumed, unlike Java Runnables. These IO values are cooperatively scheduled to run on a fixed thread pool. Because these IO values just represent continuations, you can have millions of them cooperatively executing at the same time. And there is no context switch penalty for switching the continuation you are executing for another.

You can easily have millions of coroutines on the JVM with Kotlin coroutines: <a href="https:&#x2F;&#x2F;kotlinlang.org&#x2F;docs&#x2F;reference&#x2F;coroutines&#x2F;coroutines-guide.html" rel="nofollow">https:&#x2F;&#x2F;kotlinlang.org&#x2F;docs&#x2F;reference&#x2F;coroutines&#x2F;coroutines-...</a>

The article claims that aside from memory consumption the main advantage of the Go approach is that the scheduler is integrated with the channels concept and thus knows which goroutines are blocked and don&#x27;t need to be scheduled until some data shows up in their channel.<p>But don&#x27;t OS threads work like this as well, by being integrated with file descriptors among other things? If a thread is blocked on a read, the OS knows this and won&#x27;t keep scheduling that thread - right?<p>If so, the article&#x27;s argument about time wasted on scheduling doesn&#x27;t make sense.

This post is wrong. OS threads do not have a fixed-size stack in the 1:1 model. After all, if they did, then Go couldn&#x27;t do stack growth, because goroutines are implemented as a layer on top of native threads. There are language runtimes, such as SML&#x2F;NJ, that <i>heap allocate</i> all stack frames, and the kernel doesn&#x27;t complain.<p>Additionally, as others have pointed out, Java (and, for that matter, all native code on Linux) used to use the Go model and switched to 1:1.

gorutine&#x27;s direct counterpart in Java is Runnable, not thread.

Ok there&#x27;s a technical gaff somewhere... In my understanding. I regularly have hundreds of threads running in jvms with between 128mb-384mb of heap. Im guessing the stack isn&#x27;t kept on the heap?

I&#x27;m not really a java user but I vaguely recall that java started with green threads to deal with this issue long before you could just run an absurd number of threads in the OS without issues

Would it be possible&#x2F;efficient to write an OS in Go, and use its scheduler to schedule threads living in the implemented OS? What if you ran Go in that OS?

Kotlin coroutines can do it at Java&#x2F;JVM. goroutines and JVM threads are different things

Who creates 10k threads, really? The hypothesis seems like a strawman

This is why I like Hacker News. Thanks go to the author, good stuff!

Had anyone considered using virtualization instructions to assist with managing page tables in the jvm?

Creating millions of coroutines or &quot;goroutines&quot; isn&#x27;t really interesting by itself. I believe the real questions for me are:<p>1. is Go scheduler better than Linux scheduler if you have a thousand concurrent goroutines or threads?<p>2. is really creating goroutines that much faster than creating a native thread?<p>3. are gorutines relevant for long lived and computationally intensive tasks or just for short lived I&#x2F;O task?<p>4. What is the performance of using channels between goroutines compared to native threads?<p>tbh, I have read several of respected articles that criticize Golang goroutines in terms of performance and I am not really sure that Golang&#x27;s only virtue imho which is simple concurrency is performant at all

I imagine there are actual use cases for it but what are programs doing spinning up thousands of threads? I&#x27;ve never seen a program like that. Whatever happened to worker thread pools?

Good luck with hundreds of JVM threads.

There makes no sense to comparing goroutines with os threads, they are totally different things, this is like comparing apples with oranges, of course both of them can be used to solve the same problem, but they are totally different. Like others have mentioned, is well known that a thread-per-connection solution does not scale well, a quite better approach is having a pool of threads with fixed size and using an asynchronous non-blocking event driven model. Actually in our company we did some benchmark messuring golang (1.9 with well known http servers) and java 8 with Netty, and the latter always won.