Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add experimental blosc2 support #107

Merged
merged 172 commits into from
Apr 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
172 commits
Select commit Hold shift + click to select a range
0f54c2f
Add Silesia Corpus data files and bzip
milesgranger May 11, 2023
7f7dae0
Initial impl of blosc2 in libcramjam
milesgranger May 7, 2023
21ecfd3
Merge branch 'master' into blosc2
milesgranger Mar 16, 2024
2a91ddf
Move blosc2 into experimental [skip ci]
milesgranger Mar 16, 2024
c3f4181
Propagate use-system-blosc2 feature to python-cramjam
milesgranger Mar 19, 2024
1a615c8
Fix submodule init for experimental.blosc2
milesgranger Mar 19, 2024
c79d5f4
Use blosc2 from crates.io
milesgranger Mar 20, 2024
cd9d4b0
Fixup
milesgranger Mar 20, 2024
6d03e73
Try again
milesgranger Mar 20, 2024
8a3ee99
Try again
milesgranger Mar 20, 2024
1b25416
Fix segfault in blosc2
milesgranger Mar 20, 2024
d19759d
bleh
milesgranger Mar 20, 2024
8aa7f48
Merge remote-tracking branch 'origin/master' into blosc2
milesgranger Mar 21, 2024
4d19823
fixup
milesgranger Mar 21, 2024
38babb6
fixup
milesgranger Mar 21, 2024
1d1fec3
Fixup
milesgranger Mar 21, 2024
43feadd
fixup
milesgranger Mar 21, 2024
9b98c70
Remove target
milesgranger Mar 21, 2024
ffb8561
try ssh
milesgranger Mar 22, 2024
522416d
fixup
milesgranger Mar 22, 2024
4c4b8cc
reset to main
milesgranger Mar 22, 2024
d8a3780
micromamba
milesgranger Mar 22, 2024
979795a
micromamba
milesgranger Mar 22, 2024
b24a84d
fixup
milesgranger Mar 22, 2024
17ce5d6
fixup
milesgranger Apr 1, 2024
ec6362f
more tries
milesgranger Apr 1, 2024
2e36460
chkpt
milesgranger Apr 1, 2024
af063f5
fiupx
milesgranger Apr 1, 2024
319480b
fiupx
milesgranger Apr 1, 2024
89f87a8
fixup
milesgranger Apr 1, 2024
35dac2c
fixup
milesgranger Apr 2, 2024
36de12d
fixup
milesgranger Apr 2, 2024
c3a7c4c
fixup
milesgranger Apr 2, 2024
774b8af
fixup
milesgranger Apr 2, 2024
94b0745
fixup
milesgranger Apr 2, 2024
21866ff
Fixup - try updated blosc2 crate
milesgranger Apr 7, 2024
3b7cb37
Update git fetching in config.toml
milesgranger Apr 7, 2024
8115631
another attempt
milesgranger Apr 7, 2024
2f8015a
change back
milesgranger Apr 7, 2024
fc156d5
don't use cli
milesgranger Apr 7, 2024
f5362ef
fixup
milesgranger Apr 7, 2024
219aab4
use ssh
milesgranger Apr 7, 2024
7e90878
use ssh
milesgranger Apr 7, 2024
a749ce3
Fixup
milesgranger Apr 7, 2024
ea831cc
use https
milesgranger Apr 7, 2024
606ea5b
fixup
milesgranger Apr 7, 2024
0a7b269
Debug segfault in compress impl
milesgranger Apr 7, 2024
087999f
try delocate
milesgranger Apr 7, 2024
64779c1
another try
milesgranger Apr 7, 2024
1457e43
try just testing cramjam-python
milesgranger Apr 8, 2024
6d03507
fix universal also
milesgranger Apr 8, 2024
5c07b73
Try better parametrization
milesgranger Apr 8, 2024
586ee73
also try aarch64
milesgranger Apr 8, 2024
f1b46e5
fixup
milesgranger Apr 8, 2024
6767ba7
fixup
milesgranger Apr 8, 2024
77e59ba
Fixup
milesgranger Apr 8, 2024
e4126c4
fixup
milesgranger Apr 8, 2024
169cad2
Fixup
milesgranger Apr 8, 2024
b905a95
try setting rpath
milesgranger Apr 8, 2024
406a652
Fixup - rpath in python crate
milesgranger Apr 8, 2024
31d282d
Fixup - move profile to root
milesgranger Apr 8, 2024
4edad98
remove profile
milesgranger Apr 8, 2024
a5e21fd
use maturin directly maybe
milesgranger Apr 8, 2024
ea1cc61
Fix blosc2 imports
milesgranger Apr 8, 2024
1a47cfc
universal2 target and updated osx deployment target
milesgranger Apr 8, 2024
08b6f9d
xfail blosc2 compress_into cramjam.File output
milesgranger Apr 8, 2024
1fe4bb4
Allow dead_code for now in cli
milesgranger Apr 8, 2024
0d4f3c9
Try to add aarch64 and remaining parameters to matrix
milesgranger Apr 8, 2024
28864d0
Add cache
milesgranger Apr 8, 2024
a1009a2
Install cramjam for cramjam-cli tests
milesgranger Apr 8, 2024
db7ae26
Add aarch64 target
milesgranger Apr 8, 2024
da1bca5
Only install and run tests if universal2
milesgranger Apr 8, 2024
bdea0bd
Just run rust tests on arm64 target
milesgranger Apr 8, 2024
4645aae
Fix target vs arch
milesgranger Apr 8, 2024
ac63f9b
fixup - try static again
milesgranger Apr 8, 2024
0dffd0d
Remove static link feature
milesgranger Apr 8, 2024
df3f271
try static again and no install prefix
milesgranger Apr 8, 2024
62f4f91
Back to macos-13
milesgranger Apr 8, 2024
5b651f1
non-static link
milesgranger Apr 8, 2024
a4f59f3
Skippg the compress_into File tests for blosc2
milesgranger Apr 8, 2024
fbadfb5
Ensure wheel isn't using system blosc2
milesgranger Apr 9, 2024
2947b25
chkpt - try to add ubuntu-latest
milesgranger Apr 9, 2024
35be641
Fix matrix impl in workflow
milesgranger Apr 9, 2024
e118531
Skip testing on cross compiled wheels
milesgranger Apr 9, 2024
accaddd
fixup
milesgranger Apr 9, 2024
70a49c4
Missing clang
milesgranger Apr 9, 2024
f026a1f
use upload-artifact@v4
milesgranger Apr 9, 2024
1f62ca3
chkpt: start work on windows ci
milesgranger Apr 9, 2024
483884d
fixup - missing wheel build
milesgranger Apr 9, 2024
f766c70
Try setting python arch
milesgranger Apr 9, 2024
1f7a6a5
fixup - skip rust testing on x86
milesgranger Apr 9, 2024
3ad16ab
Try removing target-triple
milesgranger Apr 9, 2024
74af1ac
Fix missing conf.target for windows
milesgranger Apr 9, 2024
85bc94f
try maturin directly - prev fails w/ missing dll
milesgranger Apr 10, 2024
f14c781
Fix multiline cmd
milesgranger Apr 10, 2024
f626703
Fix target triple windows
milesgranger Apr 10, 2024
b88c3b1
Try fixing the path to wheel
milesgranger Apr 10, 2024
d6026ed
Add lib path
milesgranger Apr 10, 2024
78194b2
write back out to dist
milesgranger Apr 10, 2024
a284f68
Skip benchmarks
milesgranger Apr 10, 2024
0020f2d
Add pytest-benchmark
milesgranger Apr 10, 2024
e1b82c0
remove memory profiler use
milesgranger Apr 10, 2024
3a17443
Fix path
milesgranger Apr 10, 2024
70b806b
Prev working, uncomment ubuntu and mac again
milesgranger Apr 10, 2024
5e01462
fixup - build when skipping tests and single target for macos
milesgranger Apr 10, 2024
a43b1d6
Try disabling cache [WORKING]
milesgranger Apr 10, 2024
8220291
try add back cache
milesgranger Apr 10, 2024
6f6ef50
Remove windows sdist
milesgranger Apr 10, 2024
a5ca31a
Attempt everything
milesgranger Apr 10, 2024
d90fd84
fixup
milesgranger Apr 10, 2024
37678f3
Fixup
milesgranger Apr 10, 2024
1374caa
fixup
milesgranger Apr 10, 2024
285abd6
Fixup
milesgranger Apr 10, 2024
9e8573a
fixup
milesgranger Apr 10, 2024
3ba52ad
Fixup
milesgranger Apr 10, 2024
95c658a
fixup
milesgranger Apr 10, 2024
70badb4
fixup
milesgranger Apr 10, 2024
c24c91c
fixup
milesgranger Apr 10, 2024
15e9965
fixup
milesgranger Apr 10, 2024
33bbd26
fixup
milesgranger Apr 10, 2024
f0a4988
only use exclude
milesgranger Apr 10, 2024
198917b
fixup
milesgranger Apr 10, 2024
bb90a87
fixup
milesgranger Apr 10, 2024
a65cf92
try yum and apt
milesgranger Apr 10, 2024
77ec470
Slowly debug
milesgranger Apr 10, 2024
95e5472
Limit libdeflater <1.20
milesgranger Apr 10, 2024
577b2b6
Debug aarch64 ubuntu build
milesgranger Apr 10, 2024
032347b
fixup
milesgranger Apr 10, 2024
8a5d65b
fixup
milesgranger Apr 10, 2024
21eb9e5
Move blosc2/experimental into opt in feature
milesgranger Apr 10, 2024
c1a63c9
Run all
milesgranger Apr 10, 2024
3b044a7
Only enable blosc2 on x86_64 linux
milesgranger Apr 10, 2024
b5a5d2b
restrict libdeflate-sys due to gcc
milesgranger Apr 11, 2024
4527f1c
bypass blosc2 when not installed
milesgranger Apr 11, 2024
60274f1
updates
milesgranger Apr 11, 2024
96ef5af
try to fix features in env vars2
milesgranger Apr 11, 2024
2ab6068
fixup
milesgranger Apr 11, 2024
e45f2cf
Fixes
milesgranger Apr 11, 2024
efa4109
don't test musllinx wheels
milesgranger Apr 11, 2024
9a0a36d
Add features to cache key
milesgranger Apr 11, 2024
f10ed28
A few more fixes
milesgranger Apr 11, 2024
e60c0a0
Fixup - run everything
milesgranger Apr 11, 2024
869e6c7
fixes
milesgranger Apr 11, 2024
eca2049
Add experimental to cramjam-cli
milesgranger Apr 11, 2024
44ce491
rerun
milesgranger Apr 11, 2024
466991e
Update for blosc2-rs using cc not cmake crate
milesgranger Apr 11, 2024
76c5adb
Another go with updated blosc2-rs build
milesgranger Apr 11, 2024
d53e135
Merge remote-tracking branch 'origin/master' into blosc2
milesgranger Apr 11, 2024
059fb5a
try to switch to macos-14
milesgranger Apr 11, 2024
004087e
3.8 not on macos-14
milesgranger Apr 11, 2024
a832f53
macos-14 no python arch
milesgranger Apr 11, 2024
ebfb8c8
try downloading existing blosc2 for macos-13
milesgranger Apr 11, 2024
534e8d4
Fixes for updated blosc2-rs
milesgranger Apr 11, 2024
4d79c83
Fixup
milesgranger Apr 11, 2024
b02280b
Fixup
milesgranger Apr 11, 2024
91b3cb3
Fixup
milesgranger Apr 11, 2024
38d4bcd
Try new blosc2-rs
milesgranger Apr 12, 2024
c0a582d
Anotehr go
milesgranger Apr 12, 2024
a960041
set macos fallback lib path
milesgranger Apr 12, 2024
495d412
change out and fixed wheel dir for windows
milesgranger Apr 12, 2024
d4ddc2c
fixup
milesgranger Apr 12, 2024
b837472
fixup
milesgranger Apr 12, 2024
590cf76
Update windows repair logic and comment out cramjam-cli
milesgranger Apr 12, 2024
e373902
Previous worked except for windows i686 - try static as well now
milesgranger Apr 12, 2024
b3594b3
Add cache and try other arch for experimental
milesgranger Apr 12, 2024
8812025
Try updated blosc2
milesgranger Apr 12, 2024
590d645
Try updated blosc2
milesgranger Apr 12, 2024
7a0c8a1
Try updated blosc2
milesgranger Apr 12, 2024
9e26734
Try updated blosc2
milesgranger Apr 12, 2024
6e555c2
Fix CI for blosc2 integration (#153)
milesgranger Apr 15, 2024
28d1c7a
native test logic fix maybe
milesgranger Apr 15, 2024
df8ff4d
Other fixes
milesgranger Apr 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .cargo/config
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@ rustflags = [
"-C", "link-arg=dynamic_lookup",
]

[net]
git-fetch-with-cli = true
661 changes: 346 additions & 315 deletions .github/workflows/CI.yml

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ __pycache__/

.idea/
benchenv/
cramjam-python/_build


# Distribution / packaging
Expand Down
Loading
Loading