Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

param,flops #44

Open
wzylove opened this issue Jul 16, 2024 · 4 comments
Open

param,flops #44

wzylove opened this issue Jul 16, 2024 · 4 comments

Comments

@wzylove
Copy link

wzylove commented Jul 16, 2024

Hello, author, first of all, thank you very much for your work. I recently wanted to make some improvements in lightweight. I used the following code, but I found that an error occurred when testing your code. Can you provide the code for testing flops and param?
image

@csguoh
Copy link
Owner

csguoh commented Jul 16, 2024

Hi, thx for your interest in this work.

The code for calculating MACs and #param is actually in this floder: /analysis/flops_param.py. However, it should be noted that the params obtained from some third-party liarbries, such as thop, may ignore certain parts of the model, due to the BUG of these liarbries. The bug can lead to inaccurate counts which is also revealed in original Mamba repo. To obtain the correct #param, you may use this line to obtain the params:

trainable_num = sum(p.numel() for p in model.parameters() if p.requires_grad)

The obtained results are larger than the reported one in the paper. We have recently planned to train another MambaIR-light version under the correct parameter counts setup. Stay tuned.

@zhousai-zs
Copy link

Hi, thx for your interest in this work.

The code for calculating MACs and #param is actually in this floder: /analysis/flops_param.py. However, it should be noted that the params obtained from some third-party liarbries, such as thop, may ignore certain parts of the model, due to the BUG of these liarbries. The bug can lead to inaccurate counts which is also revealed in original Mamba repo. To obtain the correct #param, you may use this line to obtain the params:

trainable_num = sum(p.numel() for p in model.parameters() if p.requires_grad)

The obtained results are larger than the reported one in the paper. We have recently planned to train another MambaIR-light version under the correct parameter counts setup. Stay tuned.

So, the FLOPs obtained from ./analysis/flops_param.py is right? I have try to calc the FLOPs of SS2D,and I reduce K sequence to 1 sequence, but gain the same FLOPs and Parameters. In my opinion, it is a wrong answer.

@csguoh
Copy link
Owner

csguoh commented Jul 18, 2024

Hi, the FLOPs also encounters the same problem when using some third-party libraries. To obtain the accurate FLOPs, you may refer this issue in the Vmamba repo, and register the missed params in the fvcore. We are working on the accurate calculation and will release the revised code soon.

@zhousai-zs
Copy link

Hi, the FLOPs also encounters the same problem when using some third-party libraries. To obtain the accurate FLOPs, you may refer this issue in the Vmamba repo, and register the missed params in the fvcore. We are working on the accurate calculation and will release the revised code soon.

OK, Thanks for your reply, and looking forward to your update about calculating FLOPs, Params and inference time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants