Warning when large binary files are included into the bundle #9058
Comments
mexus
added
the
C-feature-request
|
There are many people that have downloaded all the crates and can get us a size by file type histogram. |
|
I know it's not that representative, but just for the reference, here's a distribution of binary files in my local
So in my case, most of the binary files are less than 100 KiB (but there's a significant amount of them though), but at I've also got at least a couple of files that are larger than 1 MiB. I also wonder how many of them have been included deliberately though 🤔 |
ehuss
added
A-diagnostics
Report crate size on package and publish ### Motivation Fixes #11251. This adds a line like `Packaged 42 files, 727.0KiB (143.8KiB compressed)` to the output of `cargo package` and `cargo publish`. See the associated issue for more details. ### Test info I've updated associated tests to account for the new line in the output, including the file count where relevant. I've also added a few additional tests specifically to address the uncompressed and compressed file sizes. If you'd like to test this manually, simply run `cargo package` or `cargo publish` within a project of your choice. The new `Packaged` line will appear at the end of the stderr output, or directly before the `Uploaded` line, respectively. ### Review info This PR touches many test files to account for the additional line of stderr output. For convenience, I've split the PR into 4 commits. 1. contains the actual implementation 2. updates existing tests unrelated to `cargo package` and `cargo publish` 3. updates existing tests related to `cargo package` and `cargo publish`, including file counts where relevant 4. adds new tests specifically for the file sizes, which are generally not covered by existing tests ### Additional information The feature was discussed [on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/Report.20crate.20size.20when.20packaging) prior to implementation. Potential future extensions to explore include: - Report size per file when using `--verbose` - Compare to the size of the previously uploaded version to warn if the increase is sufficiently large - Consider designs that can draw more attention to the most important information - Warn if large binary files are included ([#9058](#9058))
weihanglo
added
the
S-needs-design
|
Note that with |
|
This should also complain about directories with Cargo is smart enough not to package |
Describe the problem you are trying to solve
It seems like some crate authors do not ask cargo to exclude pictures/pdfs/other-blobs by providing an appropriate
exclude = [ .. ]directive at theirCargo.tomls.And it also seems like most of the time it is not intentional, it's more like developers might simply forget that anything that is not ignored by default gets packages into the published
.cratefile.I'm not quite sure how common the situation is, though it might be happening quite often, since a deliberate action is required to exclude blobs from the package, and I wasn't able to find any advices to exclude "extra" files in the official guides.
Describe the solution you'd like
I suppose a warning when including binary large objects should do the trick, something like
Notes
It would be great if somebody's got an idea how to actually check the assumption that developers are unintentionally including extra blobs to their packages on crates.io. Currently I can only think of an article in the TWIR which explains the possible issue, but I'm looking for better ideas :)
The text was updated successfully, but these errors were encountered: