Guide to Concurrency in Python with Asyncio

Forgive me, but this is such a ball of mud. All of the &quot;easy&quot; introductions into these primitives are basically whitepaper length, with long digressions into the high-level vs the low-level and historical vs modern patterns. And nothing gets into error cases or problems with cooperative scheduling or debugging&#x2F;troubleshooting or where the GIL applies. In 5 years debugging the rats nest of concurrent python code that people are writing right now will make clear that go really got this right. So sad that python did not.<p>Edit: elsewhere on HN right now:<p><a href="https:&#x2F;&#x2F;nullprogram.com&#x2F;blog&#x2F;2020&#x2F;05&#x2F;24&#x2F;" rel="nofollow">https:&#x2F;&#x2F;nullprogram.com&#x2F;blog&#x2F;2020&#x2F;05&#x2F;24&#x2F;</a>

This looks great. I wish python would have made the event loop generally more transparent to the end user. Why didn&#x27;t they just use a single global loop? Sorta like javascript and golang. It would have been more pythonic too. Anyone doing heavier context switching could&#x27;ve had their own loop management.<p>Making the event loop self-managed added a ton of clunkiness in aio apis (use-your-own-loop, loop lifetime management, etc) and becomes mentally complex for newcomers. Theres also the issue that these huge aio frameworks rewriting the same TCP clients have emerged only differentiated by their loop management patterns.

Sorry, but after using Golang with its very simple and powerful goroutines and channels approach to concurrency, this seems like a convoluted mess. When I need concurrency, I certainly don&#x27;t think of Python as the right tool.<p>Python seems to have lost its way after the 2 to 3 shift and is no longer what I&#x27;d consider &quot;pythonic&quot;.

One thing I haven&#x27;t seen any blogs write about is that multiprocessing in 3.8.x uses `spawn()` and not `fork()`[0] on MacOS. Granted, most applications are not running on OS X Server, but an update that changes a low level API like that led to some issues where running code locally will fail when running it remotely will work. The following code will run anywhere except on MacOS on 3.8.x, where it crashes with `_pickle.PicklingError` as it tries to resolve the name `func` that is not in the new stack:<p><pre><code> import multiprocessing some_data = {1: &quot;one&quot;, 2: &quot;two&quot;} func = lambda: some_data.get(2) process = multiprocessing.Process(target=func) process.start() </code></pre> [0]: <a href="https:&#x2F;&#x2F;bugs.python.org&#x2F;issue33725" rel="nofollow">https:&#x2F;&#x2F;bugs.python.org&#x2F;issue33725</a>

To those trashing Python in favor of Golang: A compiled language with no OO is not a replacement for Python. Let&#x27;s talk about the complexity of putting code generators in all of your projects. I&#x27;ve seen real golang projects and the complexity gets moved into your repo.

Shameless plug for what is essentially my own much shorter intro to asyncio: <a href="https:&#x2F;&#x2F;charemza.name&#x2F;blog&#x2F;posts&#x2F;python&#x2F;asyncio&#x2F;I-like-python-asyncio&#x2F;" rel="nofollow">https:&#x2F;&#x2F;charemza.name&#x2F;blog&#x2F;posts&#x2F;python&#x2F;asyncio&#x2F;I-like-pytho...</a>

Or just use curio or trio and have an infinitely better time with async in python :)

There&#x27;s this really interesting blog post Im not remembering about nodejs and async concurrency style as compared to golangs one. I cannot remember the author&#x27;s name, but it goes into length as of comparing async functions and the differences between normal regular func and the async ones, and calling async func only on async scope and so on. Only I remembered I found it retweeted by Brad Fitz. I though maybe someone could guide me to it here

I&#x27;ve found this [1] screencast on the low-level principles of async in Python an amazing resource to understand async at a deep level.<p>[1] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=Y4Gt3Xjd7G8&amp;list=PLKLmIMXskJQ9WpioXF4LPJJ6Tp9hbDXg-&amp;index=2&amp;t=0s" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=Y4Gt3Xjd7G8&amp;list=PLKLmIMXskJ...</a>

I don&#x27;t see anything about I&#x2F;O in this guide.<p>Can anyone recommend a guide which explains how to actually do I&#x2F;O using asyncio: how to accept connections, how to write to and read from a socket?

Every time I see concurrency and python together, I’m immediately turned off by it. First of all, it probably won’t be a true “concurrency” due to GIL and by the point you are looking for “advanced” libraries that allow you to do whatever you are trying to do, maybe it’s time to switch languages.

Asyncio detractors have been drowned in a stream of assurances that, really, it&#x27;s not that bad.<p>Obviously it is, or this wouldn&#x27;t be post #800 that promises to finally make asyncio clear to newcomers.<p>Twisted sucks. It was a joke. Now it&#x27;s been tacitly blessed to the point you can sprinkle the `async` keyword all over the place. Good luck. I hope you don&#x27;t forget about any spots.<p>I truly think part of the reason Guido left is because he couldn&#x27;t defend this shit.