Feature implementation for zip & unzip functionality #316
Conversation
… file. The zipIn functionality will create a new file `archive.zip` if the name of the zip file is absent
…n existing zip file
…irectory if the target directory is not present.
- Rename `zipIn` to `zip` - Remove `options` parameter and replace with individual flags i.e `appendToExisting`
- Include only given files during zipping (similar to -i option in zip) - Exclude given files during zipping (similar to -x option in zip) - Delete given files from the zip archieve (similar to -d option in zip) Tests added along with the implementation
- listing files without unzipping - excluding certain files/file regex during unzipping
|
@chaitanyawaikar the docs and impl look great overall. Some compilation issues in CI, and we need to have Scaladoc on |
|
Can we add an API |
|
Can we also add a test that filesystem permissions are preserved through a zip/unzip round trip. Is that even possible? We should also add a test demonstrating the semantics of mtimes: Do we preserve them on creation? Zero them out? Do they get reset to "now" on unzipping? I think we probably need to add flags with all these options, since they are useful in different scenarios (e.g. build tools typically want them zeroed out for reproducibility, but other scenarios don't) |
…ding unit tests.
|
@chaitanyawaikar we need to expose/test/document an API to allow the user to configure how mtimes are managed as part of the zipping process, as well as how filesystem permissions are managed as part of the zipping process (if possible) |
Hey @lihaoyi I tried to implement the I checked the project's dependencies and found that the we rely less on apache commons libraries and hence had this question if we are open to adding this dependency |
|
@chaitanyawaikar got it. For now, I think let's just provide an option to zero out mtimes during zipping, since that's what's necessary for most build tools as it affects the byte-contents of the zip file. Properly setting mtimes during unzipping I think we can skip, since in general most build tools already ignore those when computing content hashes. |
|
@lihaoyi The I get the following errors when I cross compile to scala native os We have two options in front of us -
For the second option, I would need some help to enable this feature only for certain platforms as I have never done it before |
…ipping of files.
|
@chaitanyawaikar let's do conditional compilation. Go ahead and push you're changes to the PR with the pull request validation for scala-native failing, I'll set up the conditional compilation and merge it |
…ods. Add scaladoc for `unzip.apply` method
|
Continuing this work in #317. @chaitanyawaikar thanks for your work, send me your international wire/bank transfer details over email and I will close out the bounty. Also do take a look at the updated PR to see what the code looks like now that it's been made more idiomatic |
Pulls in changes from #316 and #310 and cleans it up Mostly documented in the readme.adoc. Major APIs added: - `os.zip`: create or append to an existing zip file on disk - `os.zip.stream`: create a new zip file but write it to an `java.io.OutputStream` rather than a file on disk - `os.unzip`: unzip a zip file on disk into a folder on disk - `os.unzip.stream`: unzip a zip file from an `java.io.InputStream` into a folder on disk - `os.unzip.list`: list the contents of a zip file - `os.unzip.streamRaw`: low-level API used by `os.unzip.stream` and `os.unzip.list`, exposed in case users need it - `os.zip.open`: Opens a zip file as `java.nio.file.FileSystem` and gives you an `os.Path` you can use to work with it Hopefully these are APIs we can start using in Mill rather than `"zip"` subprocesses or ad-hoc helpers like `IO.unpackZip` Limitations: * Use of `java.nio.file.FileSystem` is only supported on JVM and not on Scala-Native, and so using `os.zip` to append to existing jar files or `os.zip.open` does not work on Scala-Native. * Also `os.zip` doesn't support creating/unpacking symlinks or preserving filesystem permissions in Zip files, because the underlying `java.util.zip.Zip*Stream` doesn't support them. Apache Commons Compress can work with them (https://commons.apache.org/proper/commons-compress/zip.html), but if we're sticking with std lib we don't have that * Bumps the version requirement to Java 11 and above, matching the direction of the rest of com-lihaoyi. Probably not strictly necessary, but we have to do it eventually and now is as good a time as ever with requests already bumped and Mill bumping soon in 0.12.0 --------- Co-authored-by: Chaitanya Waikar <chaitanyawaikar1993@gmail.com>
This PR addresses the issue and adds the following functionalities
zipandunzipfunctionsSummary of Changes:
Zip Functionality:
- Excluding files/folders that match specific patterns.
- Including only specified files.
- Deleting files that match patterns from an existing zip.
Unzip Functionality