I love async, but I can't code like this

The response by Isaac filled me with rage as soon as I got to 'No one actually writes code like this'. The guy provides a real example from his application, describes how alternative solutions stack up, and how he'd like to solve the problem, and gets called out for presenting a 'strawman'. Sigh, newsgroups...<p>I personally share the original author's perspective on this and was disappointed to see more sophisticated methods of writing concurrent code (futures, coroutines, etc) dropped from Node in favor of raw callbacks. In my experience, chaining raw callbacks is too easy to get wrong for it to be considered a reliable way of writing all your asynchronous logic, even if it <i>is</i> a great primitive to build a framework on top of. In particular, the effort involved in propagating exceptions all the way through a callback chain is rather significant, and making a mistake in a single function undermines the reliability of the entire chain (usually resulting in work silently being dropped on the floor). I wish more of an effort was made to address this problem.

The new C# 5.0 proposes to bring asynchronous keywords, which I believe works like macros to turn imperative code into continuations.<p><pre><code> The “await” operator used twice in that method does not mean “this method now blocks the current thread until the asynchronous operation returns”. That would be making the asynchronous operation back into a synchronous operation, which is precisely what we are attempting to avoid. Rather, it means the opposite of that; it means “if the task we are awaiting has not yet completed then sign up the rest of this method as the continuation of that task, and then return to your caller immediately; the task will invoke the continuation when it completes.” </code></pre> <a href="http://blogs.msdn.com/b/ericlippert/archive/2010/10/29/asynchronous-programming-in-c-5-0-part-two-whence-await.aspx" rel="nofollow">http://blogs.msdn.com/b/ericlippert/archive/2010/10/29/async...</a><p><pre><code> async void ArchiveDocuments(List&#60;Url&#62; urls) { Task archive = null; for(int i = 0; i &#60; urls.Count; ++i) { var document = await FetchAsync(urls[i]); if (archive != null) await archive; archive = ArchiveAsync(document); } } </code></pre> Microsoft has been very lucky to have someone like Anders Hejlsberg, who has steadily brought a stream of features to an enterprise programming language.

`Bind` with named methods goes a long way to cleaning up nested callbacks. Here's a (quick+dirty; I know I'm missing some `do`'s in there) version of the same code in CoffeeScript:<p><pre><code> _ = fileMenu: -&#62; mainWindow.menu "File", (e,f) =&#62; this.onFile(e,f) onFile: (err,file) -&#62; if err then throw err file.openMenu (e,m) =&#62; this.onOpenMenu(e,m) onOpenMenu: (err,menu) -&#62; if err then throw err menu.item "Open", (e,i) =&#62; this.onOpen(e,i) onOpen: (err,item) -&#62; if err then throw err mainWindow.getChild type('Window'), (e,d) =&#62; this.onDialog(e,d) onDialog: (err,dialog) -&#62; if err then throw err #... _.fileMenu() </code></pre> ( `=&#62;` is CoffeeScript for a function bound to the current scope )

This is why I always find it sad that people are averse to learning about "abstract, impractical" concepts like monads. Monads are abstract, but they are far from impractical: they solve problems which you are already encountering, and even already identifying as problems.<p>The best solution would be to add monadic syntax to javascript. You would get CPS, coroutines, error propagation, early failure, and several other capabilities all using the same syntax. Shit like generators or coroutines are just ad-hoc special-cased versions.

Using async.js (<a href="http://eligrey.com/blog/post/pausing-javascript-with-async-js" rel="nofollow">http://eligrey.com/blog/post/pausing-javascript-with-async-j...</a>), you can do away with callbacks and still have asynchronous code by using the yield statement. The following is an example (taken from my blog post) of asynchronous code using async.js:<p><pre><code> yield to.request("/feedback", "POST", ( yield to.prompt("What are your impressions of async.js?") )); yield to.inform("Thanks for your feedback!"); // do more stuff here </code></pre> The equivalent code with callbacks is as follows:<p><pre><code> async.prompt( ["What are your impressions of async.js?"], function (response) { async.request( ["feedback", "POST", response], function () { async.inform( ["Thanks for your feedback!"], function () { // do more stuff here } ); } ); } ); </code></pre> It makes it very easy to handle DOM events, as demonstrated below.<p><pre><code> var clickEvent = yield myButton.next("click"); </code></pre> Full disclosure: I wrote async.js.

There have been some different attempts to expand JavaScript syntax to make this style of programming more elegant.<p>Narrative JavaScript <a href="http://www.neilmix.com/narrativejs/doc/" rel="nofollow">http://www.neilmix.com/narrativejs/doc/</a> has a single construct, the arrow which turns everything after the arrow into a closure which is passed as an argument.<p>StratifiedJS (<a href="http://www.neilmix.com/narrativejs/doc/" rel="nofollow">http://www.neilmix.com/narrativejs/doc/</a>) is more rich with a number of new keywords: hold, spawn, wait etc. for asynchronous programming.

It would be nice if people stop saying 'async' when they mean 'callbacks'. Twisted's inlineCallbacks, Corotwine, Gevent... are all plenty async, and there's not an explicit callback in sight. (I suppose you could argue that the yield in inlineCallbacks and Monocle delimits a callback. You wouldn't be wrong.)

Does anyone know if there are any plans for coroutines or similar mechanisms in the new versions of ecma/javascript? Python's async frameworks like monocle and diesel work around issues like that by returning another item from a generator and receiving the result again, which works quite well and makes the code look the same as the synchronous one.

Others have mentioned coroutines; just to avoid repeating a good conversation that already happened on all this, here's the link:<p><a href="http://news.ycombinator.com/item?id=1549023" rel="nofollow">http://news.ycombinator.com/item?id=1549023</a>

JavaScript needs coroutines. They would solve a lot of problems.

I really like JS and it has excellent closure support, but it's really obvious that heavy async programming like we're doing nowadays with AJAX is not what it was designed for.<p>The 'nested callback' problem is one I've encountered over and over. I've figured out a relatively nice way to work around it. The solution goes something like this.<p>- Take the individual actions that you want and put them in closures.<p>- These closures take 1 argument, which is always a callback function that takes no arguments (this is the convention you structure everything around)<p>- Put these closures in an array, in the order that you want them executed<p>- Pass the array to a function that executes them for you (in Haskell it's called a sequence)<p>I have a really quick and really dirty presentation about it here, <a href="http://www.moondust.dds.nl/text/async-presentation/" rel="nofollow">http://www.moondust.dds.nl/text/async-presentation/</a><p>Some code on github up here, <a href="https://github.com/michiel/asynchelper-js" rel="nofollow">https://github.com/michiel/asynchelper-js</a> - the sequencer.js function is verbose, if you read the next SO link I've included a three line example of implementing it.<p>And a stackoverflow answer using this code and some explanation here, <a href="http://stackoverflow.com/questions/4462605/jquery-ajax-each-callback-next-each-firing-before-ajax-completed/4462955#4462955" rel="nofollow">http://stackoverflow.com/questions/4462605/jquery-ajax-each-...</a><p>And another one as a gist on github here, <a href="https://gist.github.com/604730" rel="nofollow">https://gist.github.com/604730</a> (for rendering a massive table from JSON data).<p>There's a lot more to this sequencer pattern. You can nest sequences, you can have the sequencer run each callback using setTimeout(callback, 0) to release control back to the main loop and get out of 'script is doing too much' popups, etc.<p>Need to make an async call? Make a closure that performs the async call and invoke the callback argument in your async callback.<p>Need to add some random JS code to a sequence? Wrap it in a closure and call the callback when you're done. If the random JS code requires an async call, see the previous paragraph.<p>Error handling, of course, is still awful. You can extend the sequencer convention to add error closures as a second parameter, you can do everything inside the closures in try/catch blocks and finally always call the callback, etc. But that just tends to make a mess of things. So the examples just assume everything is going fine.<p>It's not an ideal solution, but it's native JS and works <i>everywhere</i>.<p>It allows you to structure your code into functional blocks and work on them separately instead of making everything one big nested callback mess.

Sorry I missed the discussion!<p>Just wanted to add a pointer to what I ended up using: <a href="https://github.com/sam-mccall/ship" rel="nofollow">https://github.com/sam-mccall/ship</a><p>It's lets you create pseudo-threads, you pretend you're calling a sync function, and get a promise back, which you can call methods on (giving more promises) etc.<p>It needs a rewrite to allow objects in different pseudo-threads to interact - i.e. the threads to cross. Currently it's just a queue of actions to perform which doesn't allow this.

This is one of the reasons I love CSP and languages with CSP-like concurrency like Go (and Erlang, although it is more CSP-inspired than CSP-like).<p>I really can't understand why anyone would want to use anything else for concurrent programming.

I can't help feeling that all of the examples contain too much copy and pasting. This is dealt with by taking a higher order approach to the problem. Which is what async.js is for.<p><pre><code> doSomething = (c) -&#62; async.waterfall [ (c) -&#62; mainWindow.menu "File", c (file, c) -&#62; file.openMenu c (menu, c) -&#62; menu.item "Open", c (item, c) -&#62; item.click c (c) -&#62; mainWindow.getChild type("Window"), c (dialog, c) -&#62; dialog.doSomething c null ] </code></pre> It's not perfect, but I think it's pretty readable and succinct.

Perhaps what we need is something like CJSR (Common JavaScript Runtime). Languages that compile to JavaScript generate standardized debugging information as JSON (CoffeeScript has been pushing for this). These languages can then use a shared set of in-browser debugging tools (JSDB?). The fact that Safari, Chrome, and FF all support common browser extension architectures as this makes this solution even more compelling.<p>Agressive source transformation becomes less of an issue.

I tend to use Node for specific tasks like running a process that may take from 0 to 20 seconds without having a thread block. Using it for discrete tasks that it's suited for and then doing the bulk of my programming on regular vanilla synchronous server processes has worked for me as a way to manage complexity.<p>Until a standard model of async emerges I think programming large web apps on node will feel a little like hiking with a stone in your shoe.

There is a huge gap for a server side implementation of JS and node is not the one the masses are asking for, despite all the hype surrounding it.<p>Just take python, ruby or php, and make JS on the server on par with them. Forget about creating servers, async stuff and coroutines, 90% of moms and pops websites don't need them.<p>There is a huge momentum behind SSJS but it is all being wasted waiting for node to be the mother of all js apps. It will never be.

I felt the same way, but thankfully Javascript lets you write really cool things that handle the flow of your code in ways that correspond to the way you think of it.<p>I encourage you to check out <a href="http://news.ycombinator.com/item?id=2101789" rel="nofollow">http://news.ycombinator.com/item?id=2101789</a> ... with those functions javascript will be pleasant again.

I think the most likely solution would come from a language extension to something like Coffeescript.<p>It could work similar to the async monad in F#:<p>#asnc x = () -&#62; var f = window.menu("file"); var m = f.openmenu(); m.click();<p>would unwind as:<p>window.menu("file",function(f) { f.open(function(m) { m.click(); }); });

isn't arrows <a href="http://www.cs.umd.edu/projects/PL/arrowlets/example-drag-and-drop.xhtml" rel="nofollow">http://www.cs.umd.edu/projects/PL/arrowlets/example-drag-and...</a> supposed to solve this problem.

Use F# instead<p><pre><code> let async_copy (in:Stream) (out:Stream) = asnyc { let buf = Array.create 8192 let rec copy = async { return! match in.ReadAsync(0,buf.Length,buf) with | 0 -&#62; () | x -&#62; do! out.WriteAsnyc(0,x,buf); return! copy; } return! copy; } </code></pre> There's an async copy routine, not much more overhead than a sync copy.<p><pre><code> let sync_copy (in:Stream) (out:Stream) = let buf = Array.create 8192 let rec copy = match in.Read(0,buf.Length,buf) with | 0 -&#62; () | x -&#62; out.Write(0,x,buf); copy copy</code></pre>

Is there a reason that events don't work in the situation? The async result handler fires an event 'getXFinished' and any function that needs to respond does so. When they are finished, they fire events. And additional functions are called. And so on.