Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Walk directories in sorted order for reproducibility #517

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

dalcinl
Copy link

@dalcinl dalcinl commented Nov 10, 2024

Implement a new utility function wrapping os.walk with the following modifications:

  • recurse directories in sorted order
  • recurse top-level *.dist-info/ directories last
  • list filenames in sorted order
  • list top-level *.dist-info/RECORD files last

* recurse directories in sorted order
* recurse top-level *.dist-info/ directories last
* list filenames in sorted order
* list top-level *.dist-info/RECORD files last
@dalcinl
Copy link
Author

dalcinl commented Nov 10, 2024

There is an additional nit that would be trivial to address in this PR. Currently, auditwheel outputs zip files using the usual convention of storing directories as an entry of zero bytes with name ending in "/". This is exactly what the zip utility does in Linux, but it is not what the wheel package does (and causes trouble with namespace packages).

What we could do (with a one-line change in this PR) is to still store subdirectories, but only if they are empty. Again, this is not what zip or python -m zipfile do. However, it is closer to what wheel does, albeit without the issue related to namespace packages. There is a minor backward incompatibility in the sense that the output zip files would no longer have all directory entries, but only those directories that are empty. However, upon unzipping, there would be no difference in the output directory tree. BTW, the wheel file spec says nothing about empty directories.

@dalcinl
Copy link
Author

dalcinl commented Nov 10, 2024

This PR covers #507 and provides additional features (.dist-info directories last, .dist-info/RECORD files last)

src/auditwheel/tools.py Outdated Show resolved Hide resolved
Copy link

codecov bot commented Dec 8, 2024

Codecov Report

Attention: Patch coverage is 90.32258% with 3 lines in your changes missing coverage. Please review.

Project coverage is 91.85%. Comparing base (9394d48) to head (fbdd62e).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/auditwheel/tools.py 88.00% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #517      +/-   ##
==========================================
- Coverage   92.35%   91.85%   -0.50%     
==========================================
  Files          20       20              
  Lines        1268     1289      +21     
  Branches      244      250       +6     
==========================================
+ Hits         1171     1184      +13     
- Misses         56       61       +5     
- Partials       41       44       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Co-authored-by: Lisandro Dalcin <dalcinl@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants