I don't understand why there seems to be so much confusion as to how node works. Here's my mental model:<p>1. The basic part of the system is a single thread running an event loop. It has a queue of waiting events. Every time the event loop starts, it pops off all of the waiting events and then gives your code a chance to process them, one after the other. Once your code is done processing this batch of events, a new even loop starts. Any events that occurred while your code was running will now be processed.[1]<p>2. Some kinds of work take place outside the event loop. I/O tasks such as sockets and file reading/writing are good examples. In Node.js, I/O calls automatically spawn a new thread, which goes off and does whatever task you wanted it to do. When it finishes, it pushes an event onto the event loop's event queue, which will be handled by the event loop the next time it starts a new loop. Other things, such as TCP servers, run perpetually in their own thread, but will push new events onto the event queue whenever something interesting happens (receiving new data, etc.).<p>3. You, the user, cannot do work outside the event loop. You cannot spawn threads. That is something that the framework does automatically, and only for certain, built-in tasks like file I/O. Node might add support for arbitrary asynchronous work later on, but for the moment this is not possible.<p>4. Remember from #1 that a new event loop only starts when the previous event loop has completed. Since an event loop is essentially "the code you have written to process events that you care about", if any of that code takes a very long time to complete, then there won't be another event loop for a long time. Any new events that occur during that time (such as new HTTP requests) will not be processed until a new event loop occurs.<p>[1] Exception: events that <i>your code</i> emits are processed immediately.
I think there is some confusion/conflation with "blocking IO" and "blocking the event loop". If some requests are more computationally intensive than others, you will see slower response times for the less computationally intensive requests. This is similar to the classic "queueing" problem with some load balancing algorithms, where a fast request can get queued up behind a slow request on one worker process, only to miss out on another worker process becoming available. Even if you load balance with node, each node is processing multiple requests so you can still have this problem.<p>One thing to note is that it's not inefficient. You will be maximizing your CPU's and will be processing work as efficiently as possible, but users will experience some requests having a variable response time. This can happen any time you have a pool of generalized worker processes btw, it's not just a "node thing".<p>Solutions? If some requests do significantly more computation than others, you can split the computation in half using process.nextTick(). This will let smaller requests go faster in exchange for the slower request going slower (remember, we're already at max efficiency, so it's not like everything is going to get faster). This is better than sending the work out to a separate process unless the job is so big that the serialization/IPC cost is negligible. Otherwise you're adding extra work and making everything slower.<p>But really, having to manually chop up your algorithms is kind of a special case. Most requests in your app probably take the same order of magnitude in terms of computation time, or they are obvious outliers (pdf generation, video encoding, etc) that you would want to move to a separate process.
Well written post. I guess a lot of people don't understand the contexts under which node.js is useful. They treat it like a magic solution that will cure all their scaling issues. Node.js is useful for I/O bound apps which are not CPU intensive because then your event loop is nothing but a "manager" which manages multiple incoming requests, and hooks them upto the I/O in an asynchronous manner and when the I/O finishes, sends the result back. For such use cases, node.js is very efficient and scales well (in the sense that with just one node.js process, you can handle many requests).
how do experienced people compare one (or 8, by the number of cores) threaded node.js performance with java's asynchronous IO servlets?<p><a href="http://tomcat.apache.org/tomcat-7.0-doc/aio.html" rel="nofollow">http://tomcat.apache.org/tomcat-7.0-doc/aio.html</a>
Or you could examine the Twisted answer: Use Ampoule and defer long-running CPU-bound work to external processes which are bootstrapped into Python using your already-existing worker code. Node <i>does</i> have subprocess controls; I just checked the documentation. If somebody hasn't built an Ampoule for Node, then maybe somebody should get on that.<p>(Or just use Twisted and Ampoule. Seriously, guys.)