TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Building Better Compression Together with DivANS – Dropbox Tech Blog

24 点作者 apu将近 7 年前

5 条评论

phrh8将近 7 年前
How does DivANS compare to state of the art compression algorithms? Is Brotli widely acknowledged as the state of the art or you just compare to it because it is fast?<p>My understanding is Zstd was designed to be fast, not necessarily efficient, and the other algorithms listed (e.g. 7-zip, gzip) are quite ancient.
评论 #17352570 未加载
BeeOnRope将近 7 年前
It is important to note that the IR here isn&#x27;t exactly generic: it embeds the assumption of an LZ77 compressor.<p>Now LZ77 compressors have the advantage of being really popular, and many of the leading compressors at various points on the space&#x2F;time frontier use it, but there a large number of other interesting compression algorithms as well, which this IR cuts out.<p>A big downside of LZ77 is that it is simple enough that we had optimal parsing for it a long time ago, and it isn&#x27;t likely to be the subject of any big breakthough in the future: much of the interesting work will probably happen in non-LZ77 style compressors.<p>One of the interesting areas for LZ77, however, is coding-sensitive-parsing: i.e., basing your match-finding decisions on the actual code lengths to represent specific matches, literals runs, etc. Since these code lengths are usually dynamic, set by the entropy coding backend, you often want a back-and-forth type of approach between the two components: either multiple passes or two-way feedback.<p>The approach of generating IR once and then feeding it into the DivANS entropy coding backend seems to preclude this.
bitwave将近 7 年前
reminds me of zpaq [1]. It is an public domain compression tool, which save the decompression algorithm as IR into the compressed archive. It is written by Matt Mahoney, best known for the PAQ archiver series.<p>[1]: <a href="http:&#x2F;&#x2F;mattmahoney.net&#x2F;dc&#x2F;zpaq.html" rel="nofollow">http:&#x2F;&#x2F;mattmahoney.net&#x2F;dc&#x2F;zpaq.html</a>
评论 #17360589 未加载
ksoong2将近 7 年前
Nice article!<p>I don&#x27;t understand how probability tables can compress AND be deterministic. Intuitively, I would think it&#x27;s a trade-off between the two competing effect.
评论 #17352865 未加载
Machyume将近 7 年前
Cool approach. Was wondering if you could chime in a bit about the compression speeds observed? Thanks!<p>Will pass this to a few colleagues.