module 'torch.nn' has no attribute 'RMSNorm'

[Jun-Howie]

Environment: NVIDIA H20 96G
System: linux
cuda: nvcc -v 12.3
Installed qwen-vl-utils and git+https://github.com/huggingface/transformers as requested.
and the dependencies in requirements_web_demo.
The torch version is 2.3.1. Running web_demo_mm reports the following error logs

(Qwen2-VL) root@ee84c7c15b61:/xinglin-data/Qwen2-VL# pip install git+https://github.com/huggingface/transformers accelerate
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting git+https://github.com/huggingface/transformers
Cloning https://github.com/huggingface/transformers to /tmp/pip-req-build-z4ahrge6
Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers /tmp/pip-req-build-z4ahrge6
Resolved https://github.com/huggingface/transformers to commit c409cd81777fb27aadc043ed3d8339dbc020fb3b
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: accelerate in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (0.33.0)
Requirement already satisfied: filelock in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (3.15.4)
Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (0.24.6)
Requirement already satisfied: numpy>=1.17 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (1.26.4)
Requirement already satisfied: packaging>=20.0 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (24.1)
Requirement already satisfied: pyyaml>=5.1 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (6.0.2)
Requirement already satisfied: regex!=2019.12.17 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (2024.7.24)
Requirement already satisfied: requests in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (2.32.3)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (0.19.1)
Requirement already satisfied: safetensors>=0.4.1 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (0.4.4)
Requirement already satisfied: tqdm>=4.27 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from transformers==4.45.0.dev0) (4.66.5)
Requirement already satisfied: psutil in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from accelerate) (6.0.0)
Requirement already satisfied: torch>=1.10.0 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from accelerate) (2.3.1)
Requirement already satisfied: fsspec>=2023.5.0 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers==4.45.0.dev0) (2024.6.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers==4.45.0.dev0) (4.12.2)
Requirement already satisfied: sympy in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (1.13.2)
Requirement already satisfied: networkx in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.3)
Requirement already satisfied: jinja2 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.1.4)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (8.9.2.26)
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch>=1.10.0->accelerate)
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/37/6d/121efd7382d5b0284239f4ab1fc1590d86d34ed4a4a2fdb13b30ca8e5740/nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 15.9 MB/s eta 0:00:00
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.0.106)
Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2.20.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (12.1.105)
Requirement already satisfied: triton==2.3.1 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (2.3.1)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.10.0->accelerate) (12.6.68)
Requirement already satisfied: charset-normalizer<4,>=2 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from requests->transformers==4.45.0.dev0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from requests->transformers==4.45.0.dev0) (3.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from requests->transformers==4.45.0.dev0) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from requests->transformers==4.45.0.dev0) (2024.8.30)
Requirement already satisfied: MarkupSafe>=2.0 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.5)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /miniconda/envs/Qwen2-VL/lib/python3.10/site-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)
Installing collected packages: nvidia-cublas-cu12
Attempting uninstall: nvidia-cublas-cu12
Found existing installation: nvidia-cublas-cu12 12.3.4.1
Uninstalling nvidia-cublas-cu12-12.3.4.1:
Successfully uninstalled nvidia-cublas-cu12-12.3.4.1
Successfully installed nvidia-cublas-cu12-12.1.3.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
(Qwen2-VL) root@ee84c7c15b61:/xinglin-data/Qwen2-VL# python web_demo_mm.py
Traceback (most recent call last):
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1659, in _get_module
return importlib.import_module("." + module_name, self.name)
File "/miniconda/envs/Qwen2-VL/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/generation/utils.py", line 51, in
from ..pytorch_utils import isin_mps_friendly
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 27, in
ALL_LAYERNORM_LAYERS = [nn.LayerNorm, nn.RMSNorm]
AttributeError: module 'torch.nn' has no attribute 'RMSNorm'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1659, in _get_module
return importlib.import_module("." + module_name, self.name)
File "/miniconda/envs/Qwen2-VL/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py", line 41, in
from ...modeling_utils import PreTrainedModel
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/modeling_utils.py", line 46, in
from .generation import GenerationConfig, GenerationMixin
File "", line 1075, in _handle_fromlist
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1649, in getattr
module = self._get_module(self._class_to_module[name])
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1661, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
module 'torch.nn' has no attribute 'RMSNorm'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/root/xinglin-data/Qwen2-VL/web_demo_mm.py", line 14, in
from transformers import AutoProcessor, Qwen2VLForConditionalGeneration, TextIteratorStreamer
File "", line 1075, in _handle_fromlist
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1650, in getattr
value = getattr(module, name)
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1649, in getattr
module = self._get_module(self._class_to_module[name])
File "/miniconda/envs/Qwen2-VL/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1661, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.models.qwen2_vl.modeling_qwen2_vl because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
module 'torch.nn' has no attribute 'RMSNorm'
(Qwen2-VL) root@ee84c7c15b61:~/xinglin-data/Qwen2-VL#

[aigonna]

pip install torch==2.4, cause from torch=2.3

[jinze1994]

It seems to be a Transformers bug for torch<2.4 which was discussed on the issue. And you can try pip install torch==2.4.

[johnmai-dev]

--- a/requirements_web_demo.txt
+++ b/requirements_web_demo.txt
@@ -2,10 +2,11 @@
 gradio==4.42.0
 gradio_client==1.3.0
 qwen-vl-utils==0.0.2
-torch==2.3.1
-torchvision==0.18.1
+torch==2.4
+torchvision==0.19.0
 transformers==4.45.0.dev0
 transformers-stream-generator==0.0.4
+accelerate>=0.26.0

[Jun-Howie]

Reinstalled torch 2.4.0 Problem solved, thanks! @jinze1994 @aigonna @maiqingqiang

[Jun-Howie]

Reinstalled torch 2.4.0 Problem solved, thanks! @jinze1994 @aigonna @maiqingqiang

[CHNtentes]

latest autoawq requires torch==2.3.1, not sure if I should update pytorch...

[jun0403]

请问操作手机怎么设置query呢？