Concern on using VMamba as a general backbone in MMCV framework. #289
Comments
|
You can simply copy the file vmamba.py and its dependencies and use it as a plug-and-play module. If you want to integrate it into mmcv, you can refer to the manual of mmcv for how to build your own net and use it by just modifying the config.py. And then, insert the code in vmamba into it, and it'll work. |
|
I did it as plug-and-play like you said. Following mmcv, I first create a folder and put the source code of vmamba.py and all its dependency into that folders, including folder kernels/selective_scan and mamba2. However, when I run the command: note: This error originates from a subprocess, and is likely not a problem with pip. |
|
selective_scan only support cuda>=11.4, better use cuda>=11.8, that may be the problem. |
|
I don't understand how but after reinstalling the same software version as above in another pc with a different gpu, the above error was from a RTX4090, now I reproduce the above procedure on RTX 3060 and discovered that the backbone required |
|
I did try vmamba with mmcv1.5, but i rebuilt mmcv in cuda12.1 rather than rewrite selective scan. In fact，the original selective scan in mamba only supports cuda>=11.8，and I found that there're only small differences in some packages used in selective scan between cuda11.8 and cuda11.4，so I change the code to make it be compatible with cuda11.4. However cuda 11.1 has so many differences that it is hard to apply the code to it. But if you wanna try，I may help when I am available. |
|
Can you clarify more on how you rebuild mmcv and which steps have you followed. Which pytorch version did you use? And how was the performance? Because when I tried to rebuild mmcv 1.5.2 with cuda 12.1 and pytorch 2.1 I got a bunch of mismatch dependency errors? If possible, please provide me with a full list of packages frameworks and the exact steps you have done. It seems that mmcv 1.5.2 requires cuda 11.1, pytorch 1.9 and python 3.8 |
|
I say build but not install，which means that the you have to download the source code and build from source yourself. If you find mismatches，chech the version of cuda you use in CUDA_HOME and pytorch，make sure they are aligned. |
|
Which pytorch version did you use to rebuild mmcv 1.5 from source? when you want to build mmcv, you need to have pytorch and cuda already there. Or you mean building pytorch from source, also? Please give me the version of pytorch you have used to build the mmcv 1.5 you have mentioned. |
|
torch2.2 + cuda 12.1 |
|
Thank you for your reply. But as far as I know from mmcv docs, it is impossible to build mmcv 1.5 from source using torch 2.2 and cuda 12.1. It was strange that you can build, because when I use torch 2.1+cuda12.1 and followed the exact build process from: this link. It gave this error: |
|
I built mmcv==1.5.0 again with torch2.2+cu121, and it works. all the commands are git clone https://github.com/open-mmlab/mmcv/ && cd mmcv && git checkout v1.5.0
# modify in setup.py: "--std=c++14" -> "--std=c++17"
pip install ninja psutil
MMCV_WITH_OPS=1 python setup.py develop
python .dev_scripts/check_installation.py
|
|
Hello BaophanN, did you successfully replace backbone? How did you do it? |
|
Thank you author for your kind response. Following your instruction, I have successfully built mmcv 1.5.2. However, due to version conflict, I have problem installing mmdet3d==1.0.0rc6 due to mismatch version between pytorch 2.+ and cuda 12.1. In detail, while installing mmdet3d, I have problems with importlib_metadata, platformdir. Still cannot manage to fix this. |
Hi author, I intend to use VMambaT as a general backbone to replace Resnet. However, my code base is implemented in mmcv==1.5.2, mmdet 2.26.0. Is there a workaround so that I can integrate VMamba into the framework of mmcv likes Resnet? Unlike Resnet, VMamba has many components implemented not in Trition so this task may not be so trivial. Thank you
The text was updated successfully, but these errors were encountered: