Avoid Clearing Tables Even When Changing CParams #1780
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
felixhandte
force-pushed
the
workspace-efficiency-3
branch
from
a9b766c
to
edb3ad0
Compare
The source matchState is potentially at a lower current index, which means that any extra table space not overwritten by the copy may now contain invalid indices. The simple solution is to unconditionally shrink the valid table area to just the area overwritten.
This led to a nasty edgecase, where index reduction for modes that don't use the h3 table would have a degenerate table (size 4) allocated and marked clean, but which would not be re-indexed.
terrelln
reviewed
Sep 11, 2019
Cyan4973
approved these changes
Sep 12, 2019
terrelln
reviewed
Sep 13, 2019
…rovide It Instead, explicitly declare the functions we use.
Cyan4973
approved these changes
Sep 16, 2019
terrelln
approved these changes
Sep 16, 2019
QrczakMK
added a commit
to google/riegeli
that referenced
this pull request
Nov 8, 2019
Apply other changes enabled by this update: Use `ZSTD_CCtx_setParameter(ZSTD_c_srcSizeHint)` for specifying `size_hint`, instead of `ZSTD_createCCtxParams()` + `ZSTD_getParams()` + `ZSTD_CCtxParams_init_advanced()` + `ZSTD_CCtx_setParametersUsingCCtxParams()`. This was introduced by facebook/zstd#1733 Do not use `ZSTD_CCtxKey`, recycle compressors without comparing compression options. This is fast after facebook/zstd#1780 Do not create the zstd compressor lazily. It is no longer needed because `size_hint` no longer participates in `ZSTD_CCtxKey`. PiperOrigin-RevId: 279266067
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This changeset builds on the work in #1712, and uses the new workspace abstraction introduced in that PR to perform more efficient context resets, even when the compression parameters change.
Overall this PR should produce compression speed wins for repeated compressions on contexts where the parameters vary (whether because the compression level changes, or the size of the input changes), and be otherwise perf neutral. The gains are very situational, and range from 0% to 4%. There are also outsize wins that this PR brings from fixing existing pathological cases, like constantly switching compression levels on streaming compression of small responses without a pledged src size, which range up 40x improvements.