Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No improvement over System.IO.Compression.ZipArchive #1

Open
A9G-Data-Droid opened this issue Oct 4, 2023 · 3 comments
Open

No improvement over System.IO.Compression.ZipArchive #1

A9G-Data-Droid opened this issue Oct 4, 2023 · 3 comments
Labels
question Further information is requested

Comments

@A9G-Data-Droid
Copy link

I am rapidly dealing with zip files in the range of 250MB. I am seeing no improvement over my old System.IO.Compression.ZipArchive method. After running a few times I am reliably seeing the same results, a bit slower.

System.IO.Compression.ZipArchive    Runtime: 00:00:01.6661219
PowerPlayZipper                     Runtime: 00:00:01.6874230
@kekyo
Copy link
Owner

kekyo commented Oct 5, 2023

Thanks for trying it out!

PowerPlayZipper speeds up the process by decompressing large numbers of files in parallel. Are you saying that the zip file you tried contains a single 250MB file?

@A9G-Data-Droid
Copy link
Author

That makes sense. Yeah, maybe this archive is not a good candidate for parallelization.

@kekyo
Copy link
Owner

kekyo commented Oct 6, 2023

In fact, it would be nice to be able to achieve parallel decompression of a single file. I am not familiar with compression algorithms themselves, so I am not clear, but to achieve parallel processing with a compression algorithm, I think we need to design a suitable algorithm from the beginning.

Perhaps "Deflate" is an old algorithm, so even if we add some thought to our implementation, we can't expect it to be fast.
And PowerPlayZipper depends on System.IO.Compression. So it will be rate-limiting in a single file.

@kekyo kekyo added the question Further information is requested label Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants