Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude large test fixtures from .crate file #105

Closed
pzmarzly opened this issue Sep 13, 2024 · 1 comment
Closed

Exclude large test fixtures from .crate file #105

pzmarzly opened this issue Sep 13, 2024 · 1 comment

Comments

@pzmarzly
Copy link

The .crate file published to crates.io unpacks to a large directory, since the .crate format (.tar.gz) does not understand sparse files.

$ curl https://crates.io/api/v1/crates/gpt/3.1.0/download -L -o gpt.crate
$ tar xf gpt.crate
$ du -hs gpt-3.1.0
11M     gpt-3.1.0
$ stat gpt-3.1.0/tests/fixtures/test.img
  File: gpt-3.1.0/tests/fixtures/test.img
  Size: 10485760        Blocks: 20480      IO Block: 4096   regular file

$ fallocate --dig-holes gpt-3.1.0/tests/fixtures/test.img
$ fallocate --dig-holes gpt-3.1.0/tests/fixtures/gpt-linux-disk-01.img
$ du -hs gpt-3.1.0
704K    gpt-3.1.0
$ stat gpt-3.1.0/tests/fixtures/test.img
  File: gpt-3.1.0/tests/fixtures/test.img
  Size: 10485760        Blocks: 24         IO Block: 4096   regular file

Would you mind excluding these files from the archive? Adding something like this to Cargo.toml should help:

[package]
exclude = ["tests/*"]
@soerenmeier
Copy link
Collaborator

Makes sense, will be removed in 4.0

facebook-github-bot pushed a commit to facebookincubator/antlir that referenced this issue Oct 1, 2024
Summary:
# This diff
Just after I upgraded to 3.1.0 in D62638795, upstream released 4.0.0, which has two changes we want:

1. It removes some large text fixtures that made source control angry (we reported it in Quyzi/gpt#105)
2. The library can now operate on GPT tables where only the primary or secondary header are still in place. Before, it always required both to be correct.

# Breaking changes in 4.0.0

- `GptConfig` no longer has an `initialized()` flag/setting. Instead, one now needs to either `open()` or `create()` to get a disk.
- It is no longer necessary to call `update_partitions` with an empty Map to initialise internal datastructures when creating a new partition table. The `create*` functions do this automatically.
- `update_guid` can no longer fail, no longer returns a `Result`

Test Plan:
```
x86 0 ~/fbsource ❯❯❯ fbcode/common/rust/tools/scripts/third-party-check.sh
+ buck2 build --materializations=none -c=fbcode.arch=x86_64 fbsource//third-party/rust/ci:
File changed: fbsource//third-party/rust/vendor/gpt-4.0.0/Cargo.lock
File changed: fbsource//third-party/rust/vendor/simple-bytes-0.2.14/src/offset.rs
File changed: fbsource//third-party/rust/vendor/simple-bytes-0.2.14/src/lib.rs
67 additional file change events
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-macos-arm64-fbsource-minimal (cfg:linux-x86_64-fbcode-platform010-clang15-asan-ubsan-dev#b6e7c038c1af5f6a)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-macos-x86_64-fbsource-minimal (cfg:linux-x86_64-fbcode-platform010-clang15-asan-ubsan-dev#b6e7c038c1af5f6a)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-windows-x86_64 (cfg:linux-x86_64-fbcode-platform010-clang15-asan-ubsan-dev#b6e7c038c1af5f6a)`
Buck UI: https://www.internalfb.com/buck2/64c63986-45eb-4b37-b89f-b3c5df1f7102
Network: Up: 13MiB  Down: 337MiB  (reSessionID-955063a3-cd36-4809-bbee-18889c8a6525)
Jobs completed: 55665. Time elapsed: 1:47.1s.
Cache hits: 87%. Commands: 1127 (cached: 982, remote: 135, local: 10)
BUILD SUCCEEDED
+ buck2 build --materializations=none -c=fbcode.arch=aarch64 fbsource//third-party/rust/ci:
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-macos-x86_64-fbsource-minimal (cfg:linux-arm64-fbcode-platform010-aarch64-clang15-asan-ubsan-dev#33aba96c43367a79)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-macos-arm64-fbsource-minimal (cfg:linux-arm64-fbcode-platform010-aarch64-clang15-asan-ubsan-dev#33aba96c43367a79)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-windows-x86_64 (cfg:linux-arm64-fbcode-platform010-aarch64-clang15-asan-ubsan-dev#33aba96c43367a79)`
Buck UI: https://www.internalfb.com/buck2/ac993bc5-8dcd-4f05-9f26-ea09767204f2
Network: Up: 110KiB  Down: 113MiB  (reSessionID-a8b841f6-74f3-46b5-8aad-375c45551d0c)
Jobs completed: 53624. Time elapsed: 21.3s.
Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)
BUILD SUCCEEDED
+ buck2 build --materializations=none fbcode//mode/mac fbsource//third-party/rust/ci:
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-windows-x86_64 (cfg:macos-x86_64-macosx-no-san#b75302e18601f507)`
Buck UI: https://www.internalfb.com/buck2/8407084f-e247-449d-8441-aa8fd479272e
Network: Up: 19MiB  Down: 95MiB  (reSessionID-84c362e9-597c-4797-b56a-3d3f333b3d88)
Jobs completed: 91130. Time elapsed: 1:53.9s.
Cache hits: 86%. Commands: 1133 (cached: 980, remote: 130, local: 23)
BUILD SUCCEEDED
+ buck2 build --materializations=none fbcode//mode/mac-arm64 fbsource//third-party/rust/ci:
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-windows-x86_64 (cfg:macos-arm64-macosx-no-san#e983b3505cd1723c)`
Buck UI: https://www.internalfb.com/buck2/caad1d96-c278-4eee-a483-c432abaef0b6
Network: Up: 6.8MiB  Down: 33MiB  (reSessionID-b46e4cb1-5f26-413c-b546-2106f82fabd1)
Jobs completed: 90725. Time elapsed: 1:03.3s.
Cache hits: 0%. Commands: 3 (cached: 0, remote: 3, local: 0)
BUILD SUCCEEDED
+ buck2 build --materializations=none fbcode//mode/win fbsource//third-party/rust/ci:
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-macos-x86_64-fbsource-minimal (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-macos-arm64-fbsource-minimal (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-wasm32-emscripten (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-android-x86_32-fbsource (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-android-arm32-fbsource (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-infra-asic-fw (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-android-x86_64-fbsource (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-iphoneos-iphonesimulator-arm64-local (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-android-arm64-fbsource (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-iphoneos-iphoneos-arm64-local (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-iphoneos-iphonesimulator-x86_64-local (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-powerpc-nfast (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-maccatalyst-arm64 (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-maccatalyst-x86_64 (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-linux-fbcode-wasm32-wasi (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-thumbv7em (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Skipping target incompatible node `fbsource//third-party/rust/ci:rust-third-party-mips64-liquidsec (cfg:windows-x86_64-msvc-clang12-no-san#4fdfdc6856bb52dc)`
Buck UI: https://www.internalfb.com/buck2/364f3a2e-198e-48f1-b94d-66aae327b45e
Network: Up: 594KiB  Down: 7.0MiB  (reSessionID-d24635b3-fa40-46bd-b1c7-c877a32cdaf0)
Jobs completed: 50305. Time elapsed: 1:01.6s.
Cache hits: 0%. Commands: 6 (cached: 0, remote: 6, local: 0)
BUILD SUCCEEDED
x86 0 ~/fbsource ❯❯❯
```

Reviewed By: Babar, pzmarzly

Differential Revision: D62759259

fbshipit-source-id: cda3bcdd4008ba3cfe380c1535e9a9130d9b7ae4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants