Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preparation for release v1.5.5 #3585

Merged
merged 109 commits into from
Apr 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
a5a2418
Introduce Variants of Some Functions that Take Optional File Descriptors
felixhandte Feb 3, 2023
f746c37
Use File Descriptor in Setting Stat on Output File
felixhandte Feb 6, 2023
610c8b9
initial commit
daniellerozenblit Feb 9, 2023
cc4e941
initial commit
daniellerozenblit Feb 9, 2023
db3e1de
initial commit
daniellerozenblit Feb 9, 2023
183a18a
meson: correctly specify the dependency relationship for playtests
eli-schwartz Feb 10, 2023
97ab0e2
meson: always build the zstd binary when tests are enabled
eli-schwartz Feb 10, 2023
f37b291
Github Action to generate Win64 artifacts
Cyan4973 Feb 10, 2023
43bc470
move shell to mingw
Cyan4973 Feb 10, 2023
5be3f19
simplify win64 artifact generation
Cyan4973 Feb 11, 2023
f8ae216
try a different compiler
Cyan4973 Feb 11, 2023
a7de1d9
Fix all MSVC warnings (#3495)
embg Feb 11, 2023
c78f434
Fix zstd-dll build missing dependencies (#3496)
yoniko Feb 12, 2023
6894746
Bump github/codeql-action from 2.2.1 to 2.2.4 (#3503)
dependabot[bot] Feb 13, 2023
4373c5a
mmap dicitonaries for large dictionaries in patch-from
daniellerozenblit Feb 13, 2023
30cb0a4
Merge pull request #3491 from facebook/win_artifacts
Cyan4973 Feb 13, 2023
8a189b1
refactor dictionary file stat
daniellerozenblit Feb 13, 2023
886de7b
Use correct types in LZMA comp/decomp (#3497)
Hello71 Feb 14, 2023
727d031
Make Github workflows permissions read-only by default (#3488)
yoniko Feb 14, 2023
6a86db1
CI workflow to test external compressors dependencies
yoniko Feb 14, 2023
2d8afd9
add manual flag to mmap dictionary
daniellerozenblit Feb 14, 2023
7da1c6d
fix cli-tests issues
daniellerozenblit Feb 14, 2023
d3d0b92
add make test for 32bit
daniellerozenblit Feb 15, 2023
d54ad3c
Fix Permissions on Publish Release Artifacts Job
felixhandte Feb 15, 2023
345ed63
Merge pull request #3509 from daniellerozenblit/fix-window-resize-test
daniellerozenblit Feb 15, 2023
3c50854
Merge pull request #3511 from felixhandte/fix-release-artifact-upload…
felixhandte Feb 15, 2023
1c42844
Merge pull request #3479 from felixhandte/faster-file-ops
felixhandte Feb 16, 2023
8420502
Don't require CMake 3.18 or later
kou Feb 14, 2023
4ebaf36
Merge pull request #3490 from eli-schwartz/meson-tests-noprograms
Cyan4973 Feb 16, 2023
547794e
Fix typos found by codespell
DimitriPapadopoulos Feb 18, 2023
832f559
clarify zstd specification for Huffman blocks
Cyan4973 Feb 19, 2023
4b9e3d1
When benchmarking two files with fullbench, the second file will not …
Feb 20, 2023
395a2c5
[bug-fix] Fix rare corruption bug affecting the block splitter
Feb 22, 2023
e1ab691
Merge pull request #3514 from facebook/spec_huffman
Cyan4973 Feb 23, 2023
bd86e24
Merge pull request #3513 from DimitriPapadopoulos/codespell
Cyan4973 Feb 27, 2023
1be9529
Bump github/codeql-action from 2.2.4 to 2.2.5 (#3518)
dependabot[bot] Mar 2, 2023
db7d7b6
Merge pull request #3516 from dloidolt/fullbench_2_files
Cyan4973 Mar 6, 2023
db79219
simplify BMK_syntheticTest()
Cyan4973 Mar 6, 2023
9efc148
minor: fixed zlib wrapper internal benchmark
Cyan4973 Mar 6, 2023
1e38e07
simplified BMK_benchFilesAdvanced()
Cyan4973 Mar 6, 2023
988ce61
Adds initialization of clevel to static cdict (#3525) (#3527)
yoniko Mar 7, 2023
6313a58
[linux-kernel] Fix assert definition
Mar 7, 2023
96e55c1
ability to disable mmap + struct to manage FIO dictionary
daniellerozenblit Mar 8, 2023
07a2a33
Add ZSTD_set{C,F,}Params() helper functions
Mar 7, 2023
64e8511
added clarifications for sizes of compressed huffman blocks and streams.
Cyan4973 Mar 8, 2023
70850eb
assert to ensure that dict buffer type is valid
daniellerozenblit Mar 9, 2023
b7080f4
Increase tests timeout
Mar 9, 2023
50e8f55
Fix Python 3.6 Incompatibility in CLI Tests
felixhandte Mar 9, 2023
c4c3e11
Avoid Calling `setvbuf()` on Null File Pointer
felixhandte Mar 9, 2023
957a0ae
Add CLI Test
felixhandte Mar 9, 2023
1ec5562
Pin Moar Action Dependencies
felixhandte Mar 9, 2023
c40c737
Clarify dstCapacity requirements
Mar 7, 2023
e0fc9fd
Merge pull request #3486 from daniellerozenblit/patch-from-low-memory…
daniellerozenblit Mar 9, 2023
6bedef8
Merge pull request #3538 from facebook/doc_huffman
Cyan4973 Mar 9, 2023
e769da1
Merge pull request #3526 from facebook/bench_zstd_api
Cyan4973 Mar 9, 2023
283c228
Merge pull request #3541 from felixhandte/fix-setvbuf-segfault
felixhandte Mar 9, 2023
cd94860
Also Pin Dockerfile Dependency Hashes
felixhandte Mar 9, 2023
d55a648
Merge pull request #3542 from felixhandte/pin-moar-action-deps
felixhandte Mar 10, 2023
1df9f36
Improved seekable format ingestion speed for small frame size
Cyan4973 Mar 10, 2023
134d332
Merge pull request #3544 from facebook/seek_faster
Cyan4973 Mar 10, 2023
33e3909
Reduce RowHash's tag space size by x2 (#3543)
yoniko Mar 10, 2023
dd8cb5a
added documentation for the seekable format
Cyan4973 Mar 10, 2023
a91e91d
[Bugfix] row hash tries to match position 0 (#3548)
yoniko Mar 13, 2023
e2965ed
Bump github/codeql-action from 2.2.5 to 2.2.6 (#3549)
dependabot[bot] Mar 13, 2023
9420bce
Add init once memory (#3528) (#3529)
yoniko Mar 13, 2023
91f4c23
Add salt into row hash (#3528 part 2) (#3533)
yoniko Mar 13, 2023
488e45f
Merge pull request #3547 from facebook/seekable_doc
Cyan4973 Mar 14, 2023
f4563d8
added documentation for LDM + dictionary compatibility
Cyan4973 Mar 15, 2023
53bad10
patch-from speed optimization (#3545)
daniellerozenblit Mar 15, 2023
fbd97f3
Deprecated bufferless and block level APIs
terrelln Jan 24, 2023
e220824
Merge pull request #3553 from facebook/ldm_dict
Cyan4973 Mar 16, 2023
29b8a3d
Fix a Python bytes/int mismatch in CLI tests
ppentchev Mar 18, 2023
3b001a3
Simplify line splitting in the CLI tests
ppentchev Mar 18, 2023
a3c3a38
[lazy] Skip over incompressible data
terrelln Mar 10, 2023
3e0550e
fix window update (#3556)
daniellerozenblit Mar 21, 2023
82cf603
Add instructions for building Universal2 on macOS via CMake
rickmark Mar 22, 2023
979b047
Disable linker flag detection on MSVC/ClangCL.
tru Mar 22, 2023
408bd1e
Add instructions for building Universal2 on macOS via CMake
rickmark Mar 22, 2023
ca799f8
Merge branch 'readme_cmake_fat' of github.com:facebook/zstd into read…
rickmark Mar 23, 2023
35c0c20
minor doc update on --rsyncable
Cyan4973 Mar 23, 2023
4cf9c7e
Bump actions/checkout from 3.3.0 to 3.5.0
dependabot[bot] Mar 27, 2023
191d229
Bump github/codeql-action from 2.2.6 to 2.2.8
dependabot[bot] Mar 27, 2023
167157d
Merge pull request #3572 from facebook/dependabot/github_actions/acti…
Cyan4973 Mar 27, 2023
1b8bddc
[contrib/pzstd] Detect and Select Maximum Available C++ Standard
felixhandte Mar 27, 2023
7306832
Merge pull request #3570 from facebook/rsync_doc
Cyan4973 Mar 27, 2023
c36d54f
Update README.md
Cyan4973 Mar 27, 2023
cbe0f0e
Switch Strategies: Only Set `-std=c++11` When Default is Older
felixhandte Mar 27, 2023
93da041
Merge pull request #3574 from felixhandte/pzstd-max-cpp-std
felixhandte Mar 27, 2023
abb3585
Merge pull request #3568 from facebook/readme_cmake_fat
Cyan4973 Mar 28, 2023
a810e1e
Provide an interface for fuzzing sequence producer plugins
embg Mar 14, 2023
57e1b45
Merge pull request #3551 from embg/seq_prod_fuzz
embg Mar 28, 2023
e6dccbf
Inline BIT_reloadDStream
zhuhan0 Mar 27, 2023
b558190
Remove clang-only branch hints from ZSTD_decodeSequence
zhuhan0 Mar 28, 2023
b2ad17a
mmap for windows (#3557)
daniellerozenblit Mar 28, 2023
262e553
Merge pull request #3573 from facebook/dependabot/github_actions/gith…
Cyan4973 Mar 28, 2023
871f3a4
Merge pull request #3569 from tru/linker_flag_fix
Cyan4973 Mar 29, 2023
0f77956
added a Clang-CL Windows test to CI
Cyan4973 Mar 29, 2023
618bf84
seekable_format: Prevent rereading frame when seeking forward
YoniGilad Feb 17, 2022
649a9c8
seekable_format: Add unit test for multiple decompress calls
YoniGilad Mar 22, 2022
5bf1359
fix decompression with -o writing into a block device
Cyan4973 Mar 31, 2023
bb7fbd5
Merge pull request #3576 from zhuhan0/dev
Cyan4973 Mar 31, 2023
c1024af
Merge pull request #3540 from dvoropaev/tests_timeout
Cyan4973 Mar 31, 2023
f33a406
Merge pull request #3579 from facebook/clangclwintest
Cyan4973 Mar 31, 2023
7b828aa
Merge pull request #3581 from facebook/seekable_readOpt
Cyan4973 Mar 31, 2023
14d0cd5
do not add invocation of UTIL_isRegularFile()
Cyan4973 Mar 31, 2023
fcaf06d
Check that `dest` is valid for decompression (#3555)
daniellerozenblit Apr 1, 2023
c45eddf
Merge pull request #3584 from facebook/fix_o_blockdev
Cyan4973 Apr 1, 2023
9f58241
updated version number to v1.5.5
Cyan4973 Mar 28, 2023
9b4833d
updated changelog
Cyan4973 Mar 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/images/primary/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM circleci/buildpack-deps:focal
FROM circleci/buildpack-deps@sha256:f6f10c11b7b8ccfd4f4a5b830c3256803604ce61292b60cb22e26b12f62b0e8c

RUN sudo dpkg --add-architecture i386
RUN sudo apt-get -y -qq update
Expand Down
74 changes: 50 additions & 24 deletions .github/workflows/dev-long-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@ on:
pull_request:
branches: [ dev, release, actionsTest ]

permissions: read-all

jobs:
make-all:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: make all
run: make all

Expand All @@ -24,52 +26,66 @@ jobs:
DEVNULLRIGHTS: 1
READFROMBLOCKDEVICE: 1
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: make test
run: make test

# lasts ~26mn
make-test-osx:
runs-on: macos-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: OS-X test
run: make test # make -c lib all doesn't work because of the fact that it's not a tty

# lasts ~24mn
make-test-32bit:
runs-on: ubuntu-latest
env:
DEVNULLRIGHTS: 1
READFROMBLOCKDEVICE: 1
steps:
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: make test
run: |
sudo apt-get -qqq update
make libc6install
CFLAGS="-m32" make test

no-intrinsics-fuzztest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: no intrinsics fuzztest
run: MOREFLAGS="-DZSTD_NO_INTRINSICS" make -C tests fuzztest

tsan-zstreamtest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: thread sanitizer zstreamtest
run: CC=clang ZSTREAM_TESTTIME=-T3mn make tsan-test-zstream

ubsan-zstreamtest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: undefined behavior sanitizer zstreamtest
run: CC=clang make uasan-test-zstream

# lasts ~15mn
tsan-fuzztest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: thread sanitizer fuzztest
run: CC=clang make tsan-fuzztest


big-tests-zstreamtest32:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: zstream tests in 32bit mode, with big tests
run: |
sudo apt-get -qqq update
Expand All @@ -80,7 +96,7 @@ jobs:
gcc-8-asan-ubsan-testzstd:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: gcc-8 + ASan + UBSan + Test Zstd
# See https://askubuntu.com/a/1428822
run: |
Expand All @@ -92,14 +108,14 @@ jobs:
clang-asan-ubsan-testzstd:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: clang + ASan + UBSan + Test Zstd
run: CC=clang make -j uasan-test-zstd </dev/null V=1

gcc-asan-ubsan-testzstd-32bit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: ASan + UBSan + Test Zstd, 32bit mode
run: |
sudo apt-get -qqq update
Expand All @@ -113,7 +129,7 @@ jobs:
gcc-8-asan-ubsan-fuzz:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: gcc-8 + ASan + UBSan + Fuzz Test
# See https://askubuntu.com/a/1428822
run: |
Expand All @@ -125,14 +141,14 @@ jobs:
clang-asan-ubsan-fuzz:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: clang + ASan + UBSan + Fuzz Test
run: CC=clang FUZZER_FLAGS="--long-tests" make clean uasan-fuzztest

gcc-asan-ubsan-fuzz32:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: ASan + UBSan + Fuzz Test 32bit
run: |
sudo apt-get -qqq update
Expand All @@ -142,7 +158,7 @@ jobs:
clang-asan-ubsan-fuzz32:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: clang + ASan + UBSan + Fuzz Test 32bit
run: |
sudo apt-get -qqq update
Expand All @@ -152,28 +168,38 @@ jobs:
asan-ubsan-regression:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: ASan + UBSan + Regression Test
run: make -j uasanregressiontest

clang-ubsan-regression:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: clang + ASan + UBSan + Regression Test
run: CC=clang make -j uasanregressiontest

msan-regression:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: MSan + Regression Test
run: make -j msanregressiontest

clang-msan-fuzz-unoptimized:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: clang + MSan + Fuzz Test
run: |
sudo apt-get -qqq update
sudo apt-get install clang
CC=clang MOREFLAGS="-O0" make clean msan-fuzztest

clang-msan-fuzz:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: clang + MSan + Fuzz Test
run: |
sudo apt-get -qqq update
Expand All @@ -184,7 +210,7 @@ jobs:
clang-msan-testzstd:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: clang + MSan + Test Zstd
run: |
sudo apt-get update
Expand All @@ -194,7 +220,7 @@ jobs:
armfuzz:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: Qemu ARM emulation + Fuzz Test
run: |
sudo apt-get -qqq update
Expand All @@ -204,7 +230,7 @@ jobs:
valgrind-fuzz-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- name: valgrind + fuzz test stack mode # ~ 7mn
shell: 'script -q -e -c "bash {0}"'
run: |
Expand All @@ -220,8 +246,8 @@ jobs:
run:
shell: msys2 {0}
steps:
- uses: actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c # tag=v3
- uses: msys2/setup-msys2@v2
- uses: actions/checkout@8f4b7f84864484a7bf31766abe9204da3cbe65b3 # tag=v3
- uses: msys2/setup-msys2@5beef6d11f48bba68b9eb503e3adc60b23c0cc36 # tag=v2
with:
msystem: MINGW64
install: make
Expand Down
Loading