Adding lzfse compression

[jakirkham]

Seeing as zstd is present. Thought it would be worth adding lzfse as well. It belongs to the same class of compression algorithms (FSE).

ref: https://github.com/lzfse/lzfse

[FrancescAlted]

Yes, many codecs can be included in Blosc, but in order to avoid overlapping, I'd like to see which advantages would bring this over the existing codecs (seeing actual figures would be nice). Same applies to #187 .

[jakirkham]

This actually seems to be a pretty nice collection of data points on various compression algorithms on standard datasets.

ref: https://quixdb.github.io/squash-benchmark/

[FrancescAlted]

Yes, that's a nice reference. However, most of the codec comparisons are about textual data, whereas Blosc is more about binary data. What I'd like to see is whether a codec has a special advantage when used inside Blosc (with shuffle filters activated) and for binary data. 2017-02-23 6:51 GMT+01:00 jakirkham <notifications@github.com>:

…

This actually seems to be a pretty nice collection of data points on various compression algorithms on standard datasets. ref: https://quixdb.github.io/squash-benchmark/ — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#186 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATMmUzydA8Fi4m9JD4x0nXBnqXhctniks5rfR5ugaJpZM4MAiBz> .

-- Francesc Alted

[jakirkham]

Well, they have some things in there like X-Ray data, an MRI, etc. That said, I think we need to refine our question more if we want to find a more specific answer. Examples would be helpful too. Though it should be noted that doing a good comparison (especially of Blosc specific features) will also require some level of integration to make it feasible.