Minimalist Guide to Lossless Compression (2019)

&gt;&quot;Also, consider that if you can only read at 100 MB&#x2F;s off a mechanical drive but your CPU can decompress data at ~500 MB&#x2F;s then the mechanical drive is able to provide 5x the throughput you&#x27;d otherwise expect thanks to compression.&quot;<p>I&#x27;d not really thought of that aspect before... My old brain is hard-coded to save cpu cycles ... Time to change my ways :)

Lossless compression is apparently equivalent to general intelligence:<p><a href="http:&#x2F;&#x2F;mattmahoney.net&#x2F;dc&#x2F;rationale.html" rel="nofollow">http:&#x2F;&#x2F;mattmahoney.net&#x2F;dc&#x2F;rationale.html</a>

Doesn&#x27;t LZFSE stem from the Finite State Entropy library Yann Collet (LZ4, ZSTD) wrote as a base for ZSTD (together with HUFF0) and Apple just decided to use it before it was fully mature? So shouldn&#x27;t LZFSE be a predecessor to ZSTD in the Tree?

Could anyone explain why LZ77 is preferred by implementors versus LZ78?<p>It seems important for compressibility to prepare the data for maximum self-similarity, in addition to the LZ algorithms (as evidenced by the sort in this article). Could someone point towards a good modern summary of the approaches or heuristics?

&gt;Entropy, an Information Theory term coined by Claude Shannon in 1948, describes the minimum number of bits, on average, needed to encode a dataset.<p>Shannon didn&#x27;t coin the term entropy. He borrowed it from the analogous definition in thermodynamics.