ImportError: cannot import name 'get_model_weights' from 'torchvision.models'

[yil8]

🐛 Describe the bug

Can't use torch.hub.load from torchvision==0.13.0, since hubconf.py from main branch is doing

from torchvision.models import get_model_weights, get_weight

which is different from torchvision==0.13.0 hubconf.py.

Error:

    model = torch.hub.load("pytorch/vision", self.model_name, **self.kwargs)
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/hub.py:540: in load
    model = _load_local(repo_or_dir, model, *args, **kwargs)
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/hub.py:566: in _load_local
    hub_module = _import_module(MODULE_HUBCONF, hubconf_path)
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/hub.py:89: in _import_module
    spec.loader.exec_module(module)
<frozen importlib._bootstrap_external>:850: in exec_module
    ???
<frozen importlib._bootstrap>:228: in _call_with_frames_removed
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    dependencies = ["torch"]
    
>   from torchvision.models import get_model_weights, get_weight
E   ImportError: cannot import name 'get_model_weights' from 'torchvision.models' (/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torchvision/models/__init__.py)

Relevant PR #6364

Versions

Collecting environment information...
PyTorch version: 1.12.1+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.9.12 (main, Apr 18 2022, 22:40:46) [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-107-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 11.4.48
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 2080 Ti
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 510.54
cuDNN version: Probably one of the following:
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn.so.8.2.2
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.2.2
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.2.2
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.2.2
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.2.2
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.2.2
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.2.2
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.971
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.2
[pip3] torch==1.12.1
[pip3] torchvision==0.13.1
[conda] Could not collect

[datumbox]

@yil8 This new API is not available on TorchVision v0.13. You are trying to access something that just became available on the nightly version from an older version. Could you try uninstalling the old version of TorchVision and letting TorchHub do the import it on the fly or installing the latest nightly? That should work. Let me know if your issue persists. The feature was just added so it might have rough edges.

If I misunderstood what you meant, could you please provide some additional information over how you make the invocation? Thanks!

[yil8]

@datumbox Yeah, thanks for following up. I actually didn't run into this issue within my local virtual environment. It was triggered by test cases through github ci action. Unfortunately, I'm not sure if I'm allowed to copy the full ci error message, but below is part of it:

pip install -e ".[all, dev]
Collecting torch>=1.12.0
  Downloading torch-1.12.1-cp39-cp39-manylinux1_x86_64.whl (776.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 776.4/776.4 MB 685.4 kB/s eta 0:00:00
Collecting torchvision>=0.13.0
  Downloading torchvision-0.13.1-cp39-cp39-manylinux1_x86_64.whl (19.1 MB)

    model = torch.hub.load("pytorch/vision", self.model_name, **self.kwargs)
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/hub.py:540: in load
    model = _load_local(repo_or_dir, model, *args, **kwargs)
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/hub.py:566: in _load_local
    hub_module = _import_module(MODULE_HUBCONF, hubconf_path)
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/hub.py:89: in _import_module
    spec.loader.exec_module(module)
<frozen importlib._bootstrap_external>:[85](https://github.com/xxxxxxx/xxxml/runs/7867774026?check_suite_focus=true#step:6:86)0: in exec_module
    ???
<frozen importlib._bootstrap>:228: in _call_with_frames_removed
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    dependencies = ["torch"]
    
>   from torchvision.models import get_model_weights, get_weight
E   ImportError: cannot import name 'get_model_weights' from 'torchvision.models' (/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torchvision/models/__init__.py)

../../../.cache/torch/hub/pytorch_vision_main/hubconf.py:4: ImportError
----------------------------- Captured stderr call -----------------------------
Downloading: "https://github.com/pytorch/vision/zipball/main" to /home/runner/.cache/torch/hub/main.zip
______________________ test_classification_nets ______________________

I also started to notice somehow the github ci action was using .cache/torch/hub/pytorch_vision_main/hubconf.py, where my local environment was properly using .cache/torch/hub/pytorch_vision_v0.11.3. Not sure if there is any way to fix that.

[datumbox]

@yil8 I'm trying to understand your setup. If you are already installing torchvision 0.13, is there are specific reason you are using TorchHub to import the models instead of importing them directly? From your logs, I would guess that somehow TorchHub downloads the latest available TorchVision main and this conflicts with the other installed version, leading to issues. I can't be sure because I'm don't know how the invocations look like on your system and how many versions do you actually end up installing concurrently.

@NicolasHug any thoughts on why TorchHub confuses the different versions?

[NicolasHug]

My assumption is that torchvision was already imported before the call to torch.hub.load()? In which case:

• the torchvision module is already present in sys.modules
• torchhub properly downloads the main branch of the repo according to the stderr (but to make sure, maybe use force_reload=True)
• torchhub tries to load that folder's hubconf.py file, where this line is found from torchvision.models import get_model_weights, get_weight
• In a normal scenario (i.e. without torchvision already present in sys.modules), this line will import torchvision from the local directory (i.e. the downloaded main branch), where get_model_weights and get_weight can be found
• But since torchvision is already present in sys.modules, this local import doesn't happen, and python tries to load get_[model]weight from the 0.13 torchvision package, where they don't exist.

[NicolasHug]

You can try something nasty like:

import sys
sys.modules.pop("torchvision")

before calling hub.load() - but you'll likely need to re-load the 0.13 torchvision package programmatically if you still need it later in the script.

This is related to pytorch/hub#243 (comment) and unfortunately there is no easy way to "fix" this limitation of torchhub. I gave it a go in the past but this is a hack on top of hacks pytorch/hub#247 (comment).

[yil8]

@NicolasHug @datumbox I found where was my bug, previously I was doing:

model = torch.hub.load("pytorch/vision", self.model_name, **self.kwargs)

Which did not specify torchvision version number. The issue was resolved after I changed the code to:

model = torch.hub.load("pytorch/vision:v0.13.0", self.model_name, **self.kwargs)