torch extra requires for safetensors dep
#2371
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Wauplin
added a commit
that referenced
this pull request
Jul 11, 2024
* Use extended path on Windows when downloading to local dir Change the path of the local dir to an extended path by prepending "\\?\" to the absolute path, when the absolute path is longer than 255 characters on Windows. Also fixed a small typo. * Use extended path on Windows when downloading to local dir Change the path of the local dir to an extended path by prepending "\\?\" to the absolute path, when the absolute path is longer than 255 characters on Windows. Also fixed a small typo. * Move path handling to `get_local_download_paths()` for robustness On Windows we check the length of `lock_path` and if it is longer than 255 characters we prepend the `\\?\` prefix to all paths if it does not already exist. We only need to check the length of `lock_path` because it is guaranteed to be the longest path. * `safetensors[torch]` (#2371) * Fix token=False not respected in file download (#2386) * Fix token=False not respected in file download * lint * Handle shared layers in `save_torch_state_dict` + add `save_torch_model` (#2373) * Handle shared layers in save_torch_state_dict + save_torch_model + some helpers * fix pytest rerun * more reruns * Support `expand` parameter in `xxx_info` and `list_xxxs` (model/dataset/Space) (#2333) * First draft to support `expand` parameter for models * add expand support for dataset * add expand support for Space * Use extended path on Windows when downloading to local dir Change the path of the local dir to an extended path by prepending "\\?\" to the absolute path, when the absolute path is longer than 255 characters on Windows. Also fixed a small typo. * Move path handling to `get_local_download_paths()` for robustness On Windows we check the length of `lock_path` and if it is longer than 255 characters we prepend the `\\?\` prefix to all paths if it does not already exist. We only need to check the length of `lock_path` because it is guaranteed to be the longest path. * Use extended path on Windows when downloading to local dir Change the path of the local dir to an extended path by prepending "\\?\" to the absolute path, when the absolute path is longer than 255 characters on Windows. Also fixed a small typo. * Removed old path handling * Reorder path check; add tests * Skip test if opn Windows The test now shows up a `skipped` if executed on a non-Windows machine Co-authored-by: Lucain <lucainp@gmail.com> * Fix indentation for test_local_folder.py * Fix code style --------- Co-authored-by: Lucain <lucain@huggingface.co> Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> Co-authored-by: Lucain <lucainp@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change is needed, otherwise numpy won't be installed. See https://github.com/huggingface/safetensors/blob/079781fd0dc455ba0fe851e2b4507c33d0c0d407/bindings/python/pyproject.toml#L35