I use parallel all the time for embarrasingly parallel scientific computations on a cluster. It is very easy to use and elegant, and it's one of the programs I'm most grateful for.<p>Recently the developers fixed a major bug for me, that child jobs on other nodes would not be killed when parallel was killed. This was the only thing stopping me from recommending it to my labmates, now there's no reason not to use it!
I use GNU Parallel. I like it because its interface is simple - input is piping filenames to it, just like xargs, and output is nicely collated to the screen.<p>I used to use ppss, which does the core task just as well, but the interface is more complex.<p>I mostly use these tools to optimize large numbers of PNGs before deployment, using optipng, pngout, and/or my own lossypng. These programs take a while to run so using all my cores gets the job done a lot quicker.
The documentation of GNU parallel (<a href="https://www.gnu.org/software/parallel/man.html" rel="nofollow">https://www.gnu.org/software/parallel/man.html</a>) also contains a lot of nice examples on how to use parallel.
I use GNU Parallel with s3cmd to move big data sets in and out of S3. I can easily saturate any network connection. I was able to GET ~2TB from S3 onto a Gluster cluster in a little more than an hour by using GNU Parallel to spread the GETs across 8 instances. Incredibly powerful, easy to use tool.
Is parallel buggy or is it just me? For example if i have a list of ip addresses:<p><pre><code> $ cat ips.txt | sort | uniq -c | sort -rn
3 127.0.0.1
2 192.168.1.1
1 192.168.1.2
</code></pre>
Now i want to reformat the output of uniq -c, i want the count to the last column:<p><pre><code> $ cat ips.txt | sort | uniq -c | sort -rn | parallel --colsep ' ' echo {2} {1}
</code></pre>
But gives empty output.. what gives?
It only works if I double pipe it thru parallel like this:<p><pre><code> $ cat ips.txt | sort | uniq -c | sort -rn | \
parallel --trim lr echo | parallel --colsep ' ' echo {2} {1}
127.0.0.1 3
192.168.1.1 2
192.168.1.2 1</code></pre>
I wrote a similar program for windows.<p><a href="https://github.com/jftuga/Windows/tree/master/mp" rel="nofollow">https://github.com/jftuga/Windows/tree/master/mp</a><p>The only file you need to download is mp.exe. Source code is mp.au3.