Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the compressor only produce the estimated number of bytes used during optimization #26

Open
lifthrasiir opened this issue Sep 24, 2021 · 0 comments
Assignees
Labels

Comments

@lifthrasiir
Copy link
Owner

The current wasm compressor produces an array of bitwise probabilities (count = input.length * inBits ) that is fed to ANSEncoder, but the optimization process only needs the number of bytes used so this can be massively simplified. The length of the first line is log_2(PROD_i { predictions[i] / 2^(precision + 1) }) plus the coding overhead, while the length of the second line is currently independent of the exact content of the first line.

I've experimentally implemented this and found that while it is indeed faster, the estimation was slightly off for an unknown reason---probably the coding overhead is not as insignificant. This can be a problem at the later stages of the optimization where each improvement is as small as a single byte. Whether this is relevant or not hasn't been yet investigated.

@lifthrasiir lifthrasiir self-assigned this Sep 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant