Python Concurrency Decorators

Another page you might be interested in bookmarking:<p><a href="https:&#x2F;&#x2F;wiki.python.org&#x2F;moin&#x2F;PythonDecoratorLibrary" rel="nofollow">https:&#x2F;&#x2F;wiki.python.org&#x2F;moin&#x2F;PythonDecoratorLibrary</a><p>A similar decorator to thread function calls for concurrency:<p><a href="https:&#x2F;&#x2F;wiki.python.org&#x2F;moin&#x2F;PythonDecoratorLibrary#Lazy_Thunkify" rel="nofollow">https:&#x2F;&#x2F;wiki.python.org&#x2F;moin&#x2F;PythonDecoratorLibrary#Lazy_Thu...</a>

So, for me, replacing imap with pool.imap is the easy part. The hard part is dealing with things like handling exceptions, catching keyboard interrupts, and so on. Does this module do anything to address these issues?

A couple of things offhand:<p>- It has Python 2 and 3 support<p>- It&#x27;s a wrapper for the Python built-in &quot;multiprocessing&quot; library<p>- It spreads out work over all cores (so the abstraction hides the ability to control the pool)<p>Seems like a great way to get your feet wet with multiprocessing in Python, but it likely has limited use in production...although certain infrastructures like resource limited containers might be able to accommodate it.

See paper at <a href="https:&#x2F;&#x2F;drive.google.com&#x2F;file&#x2F;d&#x2F;0B_olmC0u8E3gWTBmN3pydGxHdEE&#x2F;preview" rel="nofollow">https:&#x2F;&#x2F;drive.google.com&#x2F;file&#x2F;d&#x2F;0B_olmC0u8E3gWTBmN3pydGxHdEE...</a>

With Python 3.5 there is native support for concurrency by using the keywords await and async.<p>For simple usage if you are familiar with Go there is this library: <a href="https:&#x2F;&#x2F;github.com&#x2F;pothos&#x2F;awaitchannel" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pothos&#x2F;awaitchannel</a>

Of note is that the version of deco on PyPI[1] (0.2) is incompatible with Python 3.<p>There have been few commits to fix compatibility, but it&#x27;s not there yet.<p>1. <a href="https:&#x2F;&#x2F;pypi.python.org&#x2F;pypi&#x2F;deco" rel="nofollow">https:&#x2F;&#x2F;pypi.python.org&#x2F;pypi&#x2F;deco</a>

Perhaps a strange choice of the word &quot;synchronized&quot; when coming from Java, this typically implies a critical section. Here it seems to initialize a multiprocessing pool for use in the function labelled concurrent (perhaps)?

&quot;We have proposed DECO, a simplification of concurrent programming techniques targeted at programmers with little understanding of concurrent programming.&quot; (from paper)<p>gave me the chills

Related: <a href="https:&#x2F;&#x2F;github.com&#x2F;hcarvalhoalves&#x2F;python-pmap" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;hcarvalhoalves&#x2F;python-pmap</a>

this is such a good idea! the @synchronized decorator to collect the parallelized task at the end of a parent call is very very smart &amp; simple.

source looks surprising, for instance the decorators parse the decorated function code and build an AST (still have to find out why)

This reminds me of AppEngine&#x27;s ndb tasklets.

I&#x27;ve been looking for a way to replace functools.partial and pool.map with something that could cause me to make bad architectural decisions. This could be the ticket.

Let me sum up my feelings towards this: neat!

Then I can easily parallelize existing code without touching anything?

I love developing in Flask cause of the way decorators written.