You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The documentation of the filter option is misleading:
Filter out files before extracting
That's not really true. When you use decompress (say, with decompress-tar or decompress-unzip), it fully decompresses the entire tarball into memory before applying the filter. If you're not using the output feature, the filter option is entirely equivalent to just using Array.filter yourself on the return value.
It would make a lot more sense to me if decompress plugins ran filter as soon as they started processing a file, and dropped data from filtered-out files instead of filling RAM with garbage.
I recognize that this is a backwards-compatible change (as currently filter callbacks have access to data) so probably this means that a new "pre-filter" option should be added.
#57 is a PR to add documentation, though I'd suggest fixing the issue instead of documenting it.
The text was updated successfully, but these errors were encountered:
glasser
added a commit
to glasser/decompress
that referenced
this issue
Jul 26, 2017
The documentation of the
filter
option is misleading:That's not really true. When you use decompress (say, with decompress-tar or decompress-unzip), it fully decompresses the entire tarball into memory before applying the filter. If you're not using the
output
feature, the filter option is entirely equivalent to just using Array.filter yourself on the return value.It would make a lot more sense to me if decompress plugins ran filter as soon as they started processing a file, and dropped data from filtered-out files instead of filling RAM with garbage.
I recognize that this is a backwards-compatible change (as currently
filter
callbacks have access todata
) so probably this means that a new "pre-filter" option should be added.#57 is a PR to add documentation, though I'd suggest fixing the issue instead of documenting it.
The text was updated successfully, but these errors were encountered: