TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

File Compression in the Multi-Core Era

15 pointsby ajbatacabout 16 years ago

3 comments

jrockwayabout 16 years ago
This is interesting, but not for the reasons Jeff suggests. bzip uses all 8 cores for 21 minutes to produce a 986M file (or 2 minutes for 1092M), while 7zip doesn't use all 8 cores, and produces a file smaller than anything bzip can produce, in 5 minutes.<p>So it looks like 7zip is not just slightly better than bzip; it's <i>much</i> better. Ideally you can utilize all your cores by piping data from the DB directly into the compressor -- the compressor will use 2 cores (or whatever), and your database will use the rest.
评论 #498928 未加载
ruslanabout 16 years ago
Poor guy discovered multi-threading on SMP too late! I would fire such system administrator who does not undestand system in essence. Also the one who does not respect the history and does not understand the fact that old unix utilities are not (and in most cases cannot be!) multi-threaded.
artificerabout 16 years ago
Note that the bzip2 implementation he uses is 7zip's; the classic unix implementation does not make use of multiple cores. But, there is also pbzip2, which supposedly uses all available cores:<p><a href="http://compression.ca/pbzip2/" rel="nofollow">http://compression.ca/pbzip2/</a>