TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Using xargs to do Parallel Processing

36 pointsby martinaglvover 12 years ago

6 comments

emillonover 12 years ago
Meet GNU Parallel :<p><a href="http://www.gnu.org/software/parallel/" rel="nofollow">http://www.gnu.org/software/parallel/</a>
评论 #4920486 未加载
评论 #4922145 未加载
webreacover 12 years ago
My most common use of xarsg -P is for compressing logs on remote hosts before downloading them: cat RemoteHosts | xargs -P 0 -n 1 -I Host rsh Host gzip -9 xxx/*.log The cool thing is that it waits until the completion of all the commands before returning.
评论 #4922191 未加载
评论 #4927263 未加载
Erwinover 12 years ago
If you are compressing a single large file, you may find "pigz" of interest -- it's a gzip that uses multiple threads pretty well.
PaulHouleover 12 years ago
Ever read the man page for xargs? It reads just like the perscribing information for VIOXX.
sneakover 12 years ago
Google "ppss".
Jyaifover 12 years ago
make