Improve ability to convert safetensors files. #1276
Conversation
… or pth files are available, attempt to load safetensors files
convert.py
Outdated
| @@ -1037,6 +1037,10 @@ def load_some_model(path: Path) -> ModelPlus: | |||
| if path.is_dir(): | |||
| globs = ["consolidated.00.pth", "pytorch_model-00001-of-*.bin", "*.pt"] | |||
| files = [file for glob in globs for file in path.glob(glob)] | |||
| if not files: | |||
| # Check if it's a set of safetensors files | |||
| globs = ["model-00001-of-*.safetensors"] | |||
There was a problem hiding this comment.
Why not just add "model-00001-of-*.safetensors" to the globs above?
There was a problem hiding this comment.
That would generally work as well. I was thinking to treat it the same as the ggml files, where it's lower priority than the pth files, in case both exist. If I simply add the safetensors pattern to the globs, and both are present, the convert script will abort, since it's unsure which one to use.
In any case, I'm happy to change it if you prefer.
There was a problem hiding this comment.
Hm, shouldn't the safetensors be preferred if anything?
There was a problem hiding this comment.
That sounds good to me.
I initially wanted to minimize the behavior change, but I've updated the PR to try with the safetensors first.
… safetensors aren't available
When loading a safetensors file, ignore the metadata header.
If no pt or pth files are available, attempt to load safetensors files.
Edit: This has been changed to try to load safetensors files first, and only load the pt/pth/bin file if those aren't available.