Add ISA-L to the comparison

[rhpvorderman]

The Intel Storage Acceleration Library is an implementation of the DEFLATE algorithm written in assembly. As a result it is very fast. It can be used as a replacement for zlib.

Unlike libdeflate, ISA-L has a streaming implementation as well as an all-at-once implementation. Libdeflate only has an all-at-once implementation which makes it unsuitable for big compressed files of genomic data (FASTQ format) as these would need to be read into memory entirely.

It would be quite interesting to see how ISA-L compares to the rest of the compression programs. Especially since it is 100%-compatible with the zlib formats. This allows for smooth transition.

From my own experiments my ballpark estimates are that ISA-L is 6x faster when compressing data and 3x faster when decompressing data than zlib. So it would be a quite nice addition to this already impressive comparison of programs.

A disadvantage of adding it would be that it needs an assembler (NASM or YASM) besides a c-compiler since it is written in assembly with C-bindings provided. So it would complicate the build slightly.

[jinfeihan57]

This “DEFLATE” can be used only by x86 CPUs.
Am I right？

[rhpvorderman]

No. ARM64 is also supported. EDIT: And I have benchmarked it extensively on AMD processors at well. There are no Intel anti-competitive shennanigans going on.

[rhpvorderman]

@jinfeihan57 I have no problem with adding it myself when time allows. I am currently short for time in doing it. But I think it is an interesting comparison so I will try once time allows.

[jinfeihan57]

Welcome to provide PR.
Which API are you testing? Is that it? 94-114

[rhpvorderman]

Well those are the stateless functions. These are interesting, but the primary use case will be the streaming functions:
https://github.com/intel/isa-l/blob/bee5180a1517f8b5e70b02fcd66790c623536c5d/include/igzip_lib.h#L795 .
https://github.com/intel/isa-l/blob/bee5180a1517f8b5e70b02fcd66790c623536c5d/include/igzip_lib.h#L946

Streaming is essential when handling big data volumes. So testing these will be most representative for the performance I think.

[ghost]

Hello, we're very interest in adding ISA-L into lzbench, is there any plan/progress for this?