The magic of asyncio explained

Yet another asyncio tutorial that shows you to run a few sleep tasks concurrently. Can we finally get one that shows how to do <i>real</i> stuff such like socket programming, wrapping non-async-compatible libraries and separating cpu-intensive blocking tasks to awaitable threads?

Python is my language of first choice, but I must say that I am not that thrilled how this multithreading ended up. There are many tutorials about the topic promising to explain how it works, usually in the form of &quot;simple introduction&quot;. But when one tries to implement something production-ready, with correct error handling etc., things starts to complicate pretty quickly; at least that was my experience. I don&#x27;t want to accuse anyone specifically, but most of the tutorials I saw seems to portrait it in a way, that it looks easier than it actually is.<p>Ultimately, my company decided, that instead of fighting with asyncio, certain projects will switch to Go.

This is essentially how modern JavaScript works, in particular with the addition of async&#x2F;await syntax [1] (which was originally from C#, I think), but it&#x27;s been possible with libraries like task.js, co, and Bluebird [2] since generator functions were available (either natively or via transpiling).<p>The main difference is in JavaScript the event loop is automatic and hidden, and asynchronous IO is the default, so it&#x27;s a bit harder to shoot yourself in the foot.<p>1. <a href="https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;docs&#x2F;Web&#x2F;JavaScript&#x2F;Reference&#x2F;Statements&#x2F;async_function" rel="nofollow">https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;docs&#x2F;Web&#x2F;JavaScript&#x2F;Refe...</a><p>2. <a href="https:&#x2F;&#x2F;github.com&#x2F;mozilla&#x2F;task.js" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;mozilla&#x2F;task.js</a> <a href="https:&#x2F;&#x2F;github.com&#x2F;tj&#x2F;co" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;tj&#x2F;co</a> <a href="http:&#x2F;&#x2F;bluebirdjs.com&#x2F;docs&#x2F;api&#x2F;promise.coroutine.html" rel="nofollow">http:&#x2F;&#x2F;bluebirdjs.com&#x2F;docs&#x2F;api&#x2F;promise.coroutine.html</a>

I’m still bummed that Python took this direction. Maybe introducing new keywords into the language for event loop concurrency was Python’s way of satisfying “explicit is better than implicit” but i can’s shake the feeling that callback passing and generator coroutines are a fad that is complex enough to occupy the imagination of a generation of programmers while offering little benefit compared to green threads.

quoting the article:<p>&gt; Concurrency is like having two threads running on a single core CPU.<p>&gt; Parallelism is like having two threads running simultaneously on different cores<p>&gt; It is important to note that parallelism implies concurrency but not the other way round.<p>Aurgh! I don&#x27;t think this attempted definition-by-simile is helpful, or even somewhat correct.<p>I much prefer yosefk&#x27;s way of framing things:<p>&gt; &gt; concurrent (noun): Archaic. a rival or competitor.<p>&gt; &gt; Two lines that do not intersect are called parallel lines.<p>...<p>&gt; Computation vs event handling<p>&gt; With event handling systems such as vending machines, telephony, web servers and banks, concurrency is inherent to the problem – you must resolve inevitable conflicts between unpredictable requests. Parallelism is a part of the solution - it speeds things up, but the root of the problem is concurrency.<p>&gt; With computational systems such as gift boxes, graphics, computer vision and scientific computing, concurrency is not a part of the problem – you compute an output from inputs known in advance, without any external events. Parallelism is where the problems start – it speeds things up, but it can introduce bugs.<p>...<p>&gt; concurrency is dealing with inevitable timing-related conflicts, parallelism is avoiding unnecessary conflicts<p>yosefk&#x27;s whole essay about this is great: <a href="https:&#x2F;&#x2F;yosefk.com&#x2F;blog&#x2F;parallelism-and-concurrency-need-different-tools.html" rel="nofollow">https:&#x2F;&#x2F;yosefk.com&#x2F;blog&#x2F;parallelism-and-concurrency-need-dif...</a>

I really liked this article. It&#x27;s by far the most concise explanation of asyncio in python that I&#x27;ve come across. Also, great use of little &quot;quoted&quot; statements throughout that encourage you to stop and really understand what was said before moving on (these probably have a special name).<p>Bravo!

A few years ago I wrote an BitTorrent client in Python 3.5 to get to know asyncio better.<p>Maybe those blogposts are still of use to somebody:<p>- <a href="http:&#x2F;&#x2F;markuseliasson.se&#x2F;article&#x2F;introduction-to-asyncio&#x2F;" rel="nofollow">http:&#x2F;&#x2F;markuseliasson.se&#x2F;article&#x2F;introduction-to-asyncio&#x2F;</a><p>- <a href="http:&#x2F;&#x2F;markuseliasson.se&#x2F;article&#x2F;bittorrent-in-python&#x2F;" rel="nofollow">http:&#x2F;&#x2F;markuseliasson.se&#x2F;article&#x2F;bittorrent-in-python&#x2F;</a>

Here is a comparison of `asyncio` (Python), `async` (Ruby) and Go: <a href="https:&#x2F;&#x2F;github.com&#x2F;socketry&#x2F;async-await&#x2F;tree&#x2F;master&#x2F;examples&#x2F;port_scanner" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;socketry&#x2F;async-await&#x2F;tree&#x2F;master&#x2F;examples...</a><p>I wrote a similar article but for Ruby: <a href="https:&#x2F;&#x2F;www.codeotaku.com&#x2F;journal&#x2F;2018-06&#x2F;asynchronous-ruby&#x2F;index" rel="nofollow">https:&#x2F;&#x2F;www.codeotaku.com&#x2F;journal&#x2F;2018-06&#x2F;asynchronous-ruby&#x2F;...</a><p>Yes, it&#x27;s a good model for many use cases.<p>One thing I wondered about Ruby, is it really necessary to have the `await` keyword?

It seems odd that Python ended up with asyncio when they had a clear and successful model they could’ve adopted from Stackless. It would have been more difficult to implement, but it would allow for the same benefits without requiring an asynchronous programming model, which would have reduced the total amount of effort involved in getting to having good concurrency in Python.

Wasn&#x27;t there a language where every call was async? Instead of async ... A&#x2F;returing Future[A] it did&#x2F;would return A from method calls.<p>If it didn&#x27;t exist, one can imagine one.<p>A.x = 3 would be wrapped in A.map(_.x = 3) etc. So you write code that would be executed when you finally await a value. No more red&#x2F;blue world. Would probably need coroutines instead of threads for executing.

Someone send this article to Armin Ronacher (creator of Flask, Jinja, etc.) so he can understand asyncio since he wrote a much longer and more detailed article explaining why he doesn&#x27;t understand asyncio (previous discussion at <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=12829759" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=12829759</a>).

Is this any different than in C#?

My first endeavor with asyncio felt worse than a beating with a wet rubber hose. But it was a character building experience to really get up close and personal with the asynchronous model and I can definitely see its advantages over imperative, as well as it&#x27;s always good to have another tool in the box.

Thinly veiled advertisement for a &quot;Intelligent Infrastructure Analytics - a Machine Learning driven approach for DevOps &amp; SREs of modern age&quot; company. Seems like hackernoon just published a native advertisement? Not surprisingly shady though, considering the &quot;buy crypto with credit card&quot; link in the top bar.