An introduction to parallel programming using Python's multiprocessing module

I feel that a discussion of <a href="http://en.wikipedia.org/wiki/Amdahl&#x27;s_law" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Amdahl&#x27;s_law</a> should be mandatory when introducing parallel programming<p>Small sequential portions of an otherwise parallel algorithm can have huge effects on the overall running when trying to scale up.<p>&quot;parconc&quot; explains this while discussing a parallel version of k-means, talks about how things like granularity of data needs to be fine-tuned for parallel algos, and provides some nice visualizations into what the CPU&#x27;s are actually doing on a timeline: <a href="http://chimera.labs.oreilly.com/books/1230000000929/ch03.html#sec_par-kmeans-perf" rel="nofollow">http:&#x2F;&#x2F;chimera.labs.oreilly.com&#x2F;books&#x2F;1230000000929&#x2F;ch03.htm...</a><p>Overall I think multicore is a good tool to have in your toolbox, but it seems like there needs to be a lot of tuning and effort to get good rewards for the time invested.

<a href="https://docs.python.org/dev/library/concurrent.futures.html" rel="nofollow">https:&#x2F;&#x2F;docs.python.org&#x2F;dev&#x2F;library&#x2F;concurrent.futures.html</a>

I wrote <a href="https://github.com/abemassry/crazip" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;abemassry&#x2F;crazip</a> in python and it was challenging to find out how to do multiprocessing effectively, not sure how this would run if implemented in other laungaues.

This article contains a perfect example of why I don&#x27;t like to write or read comments in code. Comments are not compiled and thus allow for sloppy verbiage such as the following:<p><pre><code> # Exit the completed processes for p in processes: p.join() </code></pre> The comment should read something more like: &quot;Wait for all the subprocesses to exit&quot; but is that really any more helpful than just reading the code and seeing that join is called on each subprocess and connecting the dots from there?

Instead of calling the &#x27;else&#x27; condition of the for loop a &#x27;completion-else&#x27;, just call it the &#x27;nobreak&#x27; condition. Unlike &#x27;completion-else&#x27;, &#x27;nobreak&#x27; immediately describes when it will be executed.

Better starting point: <a href="https://medium.com/@thechriskiehl/parallelism-in-one-line-40e9b2b36148" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;@thechriskiehl&#x2F;parallelism-in-one-line-40...</a>

&gt; x_i = (point_x - row[:,np.newaxis]) &#x2F; (h)<p>TypeError: list indices must be integers, not tuple

Still multiprocessing is experimental phase for various use case :(

pretty poor language choice to teach parallel programming concepts

the author seems to be unaware of threading and event based models.<p>multiprocessing adds memory isolation through the CPU&#x27;s protected memory.