Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Given that LZMA is the algorithm with the highest compression ratio and lowest speed of all commonly used algorithms, this isn't particularly impressive.

Why is it better than bzip2, or gzip? Both of these are 2-3x faster than LZMA (much more so for gzip --fast) but have lower compression ratios.

LZ4 would be the extreme example of an algorithm even faster than gzip (but with still lower compression ratio).



FWIW, from the "Large Text Compression Benchmark" linked from that page:

                Compression     Compressed size      Decompresser Total size   Time (ns/byte)
  Program           Options      enwik8      enwik9     size (zip)    enwik9+prog  Comp Decomp  Mem Alg Note

  lzham alpha 3 x64 -m4 -d29   24,954,329  206,393,809  155,282 x  206,549,091    595     9 4800 LZ77 45 
  gzip 1.3.5        -9         36,445,248  322,591,995   38,801 x  322,630,796    101    17  1.6 LZ77
  bzip2 1.0.2       -9         29,008,736  253,977,839   30,036 x  254,007,875    379   129    8 BWT
Salient point is the decompression time (ns/byte), which is 129 for bzip2, 17 for gzip, and... 9 (!) for lzham. So on that point it blows them out of the water, while still achieving higher compression rate on this type of input. As the original webpage implies, this is perfect for stuff like video games.


That changes things, it's a shame the website doesn't explicitly say that it's faster than gzip to decompress, it just says it's faster than LZMA, which isn't saying much.

Much better than lz4hc (7 ns/byte, but only compresses to 44MB):

lz4hc 0.9 44,182,558 392,102,544 43,617 x 392,146,161 65 7 14 LZ77 26


Is the "Mem" column saying that lzham used 4800 megabytes vs gzip using 1.6 megabytes? If that's so, it makes lzham much less attractive for console and mobile games. If that could be reduced to 8 while still being faster than gzip, then it would be very interesting.

For anyone publishing compression benchmarks: Please, please include memory usage in the report! Very few reports bother to do so at the moment. Meanwhile, many compressors take the assumption that since 16 gigs of RAM is cheap these days, that means using a few extra gigs to get a speedup is totally reasonable. It may be reasonable for you, but it makes the algo useless for me.


The mem column is memory used for compression.


A better summary is "compression ratios within 10% of LZMA, decompression faster than gzip".


LZMA is slow to compress, indeed, but it's rather fast for decompression (given the compression ratio).

While bzip2 is faster at compressing and has lower memory requirements, LZMA can beat it at decompressing speed by large margins.

This is of course nowhere near LZ4.


The experts at http://encode.ru are rather less dismissive than you.

The authors of PAQ, LZ4 and everything in between hang out on that board and talk compression.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: