Release ZStandard v1.4.9 #2515
Merged
Release ZStandard v1.4.9 #2515
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
making better usage of default build rules
to be compatible with scenarios such as `make -j allmost`
for better diagnosis
streamline zlibwrapper makefile
for some reasons, this test fails at _installing_ 32-bit dependencies using the exact same command that actually works in other tests !!? It's unclear why it fails repeateadly for this test only. Try another way to install dependencies to fix that.
try to fix 32-bit test on github actions
try to keep libzstd.a "as is" once created
This ensures the symbols aren't redefined, which would result in a compiler error. I was getting redefined symbols for _LARGEFILE64_SOURCE when building for 32-bit x86 Linux on an older CentOS release in a CI environment. With this change, I'm able to compile the single file library in this environment. Closes #2443.
while waiting for it to be fixed
Add ifndef guards for _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE
Generate skippable frame API
can't use address calculation with `void*`
remove flackey msan ossfuzz test
[license] Update year to 2021
* Add a test that runs without a pledgedSrcSize and with a dictionary. * Add github.tar data with uses the github dictionary while compressing github.tar, instead of each file individually.
Fixes #2442. 1. When creating a dictionary keep the same behavior as before. Assume the source size is 513 bytes when adjusting parameters. 2. When calling ZSTD_getCParams() or ZSTD_adjustCParams() keep the same behavior as before. 3. When attaching a dictionary keep the same behavior of ignoring the dictionary size. When streaming this will select the largest parameters and not adjust them down. But, the CDict will use the correctly sized parameters, which seems like the right tradeoff. 4. When not attaching a dictionary (either forced not to, or using a prefix dictionary) we select parameters based on the dictionary size + source size, and assume the source size is small, which is the same behavior as before. But, now we don't adjust the window log (and hash and chain log) down when the source size is unknown. When the source size is unknown all cdicts should attach, except when the user disables attaching, or `forceWindow` is used. This means that when streaming with a CDict we end up in the good case where we get small CDict parameters, and large source parameters. TODO: Add a streaming + dictionary regression test case.
Treat ZSTD_getCParams() and ZSTD_adjustCParams() in the same way we treat streaming compression. Choose parameters based on the dictionary size + source size, and assume the source size is small if unkown. But, don't shrink the window log down in ZSTD_adjustCParams_internal().
This commit addresses #2491. Note that a downside of this solution is that it is global: `umask()` affects all file creation calls in the process. I believe this is safe since `fileio.c` functions should only ever be used in the zstd binary, and these are (almost) the only files ever created by zstd, and AIUI they're only created in a single thread. So we can get away with messing with global state. Note that this doesn't change the permissions of files created by `dibio.c`. I'm not sure what those should be...
Use umask() to Constrain Created File Permissions
allows recovering from pointer overflow
* Fix compiler version regex, which was broken for multi-digit versions. * Fix compiler detection for gcc. * Disable `pointer-overflow` instead of `integer-overflow` for gcc versions newer than 8.0.0.
[fuzz] Fix compiler detection & update ubsan flags
fix ubsan test errors
These are replaced by the corresponding context resets. When converting resetCStream, CCtx_setPledgedSrcSize isn't called if the source size is "unknown". This helps reduce the reliance on "static only" symbols, as well as reducing the use of deprecated functions. Signed-off-by: Stephen Kitt <steve@sk2.org>
Stop using deprecated reset?Stream functions
update FreeBSD image to latest 12.2
9f327c0 changed the compression method for LDM, so the results are slightly different. I've re-tested LDM on some larger inputs and everything seems fine. These ratio changes just seem to be noise. There is generally a 0.01% swing in ratio, sometimes better sometimes worse, but never large.
[regression] Update results.csv
CMake: Enable only C for lib and programs projects
Use standard md5 tool on NetBSD.
Fixes the update from PR #2508. I had accidentally forgotten to rebuild the library, and the regression test suite isn't hooked up to the new fancy build system yet. I've double checked that the results are deterministic.
[regression] Update results.csv
This commit addresses #2509.
Addresses #2493. I think. I don't have a NetBSD system to test on.
Detect `..` in Paths Correctly
Avoid Using `stat -c` on NetBSD
Prepare Codebase for v1.4.9 Release
Cyan4973
approved these changes
Mar 2, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelog:
umask()to Constrain Created File Permissions (Use umask() to Constrain Created File Permissions #2495, @felixhandte)--output-dir-mirror's Rejection of..-Containing Paths (Detect..in Paths Correctly #2512, @felixhandte)-f/--forceis Passed (Allow Input From Console When-f/--forceis Passed #2466, @felixhandte)stat -con NetBSD (Avoid Usingstat -con NetBSD #2513, @felixhandte)libzstd(zstd CLI can now be linked to libzstd dynamic library #2457, removed internal library dependency from CLI #2454 @Cyan4973)configure_file()to Create the.pcFile (cmake: use configure_file() for creating the .pc file #2462, @lazka)_LARGEFILE_SOURCEand_LARGEFILE64_SOURCE(Add ifndef guards for _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE #2444, @indygreg)zlibwrapperMakefile (streamline zlibwrapper makefile #2437, @Cyan4973)recover_directoryProgram ([contrib][recovery] Add recovery_directory program #2473, @terrelln)