[Feature] Support FLOPs and Params calculation in MMDet3D

[Wuziyi616]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Fix issue #731. Support FLOPs counter for MMDet3D models.

Modification

Modify the get_flops.py script from MMDet. Add forward_dummy function in some detector classes.

BC-breaking (Optional)

No.

Use cases (Optional)

See the docs for details.

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[codecov]

Codecov Report

Merging #736 (ccf34b7) into master (2f95eb7) will decrease coverage by 0.01%.
The diff coverage is 22.22%.

❗ Current head ccf34b7 differs from pull request most recent head 0d651a5. Consider uploading reports for the commit 0d651a5 to get more accurate results

@@            Coverage Diff             @@
##           master     #736      +/-   ##
==========================================
- Coverage   48.91%   48.89%   -0.02%     
==========================================
  Files         208      208              
  Lines       15866    15874       +8     
  Branches     2546     2546              
==========================================
+ Hits         7761     7762       +1     
- Misses       7609     7616       +7     
  Partials      496      496              

Flag	Coverage Δ
unittests	48.89% <22.22%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmdet3d/models/detectors/single_stage.py	58.62% <12.50%> (-17.57%)	⬇️
mmdet3d/models/detectors/voxelnet.py	33.87% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f95eb7...0d651a5. Read the comment docs.

[Wuziyi616]

This PR aims at introducing FLOPs counter to MMDet3D and support some models. Note that:

1. Multi-modality models are not supported, because currently calc_flops utils in mmcv only supports single input testing.
2. Two-stage models are not supported, because this may involve adding forward_dummy to many RPN heads, which I think is too big for a PR.
3. Custom ops won't be counted. Not sure how much this will affect the results.
4. Point cloud model behavior is very different according to input points, such as sampling operations. Current flops counter in mmcv uses all ones input (torch.ones(shape)). Not sure how much this will affect the results.

[Wuziyi616]

Result on VoteNet ScanNet detection:

python tools/analysis_tools/get_flops.py \
configs/votenet/votenet_8x8_scannet-3d-18class.py \
--shape 40000 4 \
--modality point

VoteNet(
  0.954 M, 100.000% Params, 5.781 GFLOPs, 100.000% FLOPs,
  (backbone): PointNet2SASSG(
    0.642 M, 67.299% Params, 5.29 GFLOPs, 91.505% FLOPs,
    (SA_modules): ModuleList(
      0.247 M, 25.860% Params, 4.985 GFLOPs, 86.241% FLOPs,
      (0): PointSAModule(
        0.013 M, 1.369% Params, 1.745 GFLOPs, 30.183% FLOPs,
        (groupers): ModuleList(
          0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs,
          (0): QueryAndGroup(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        )
        (mlps): ModuleList(
          0.013 M, 1.369% Params, 1.745 GFLOPs, 30.183% FLOPs,

...

    (conv_pred): BaseConvBboxHead(
      0.046 M, 4.828% Params, 0.012 GFLOPs, 0.205% FLOPs,
      (shared_convs): Sequential(
        0.034 M, 3.516% Params, 0.009 GFLOPs, 0.150% FLOPs,
        (layer0): ConvModule(
          0.017 M, 1.758% Params, 0.004 GFLOPs, 0.075% FLOPs,
          (conv): Conv1d(0.017 M, 1.731% Params, 0.004 GFLOPs, 0.073% FLOPs, 128, 128, kernel_size=(1,), stride=(1,))
          (bn): BatchNorm1d(0.0 M, 0.027% Params, 0.0 GFLOPs, 0.001% FLOPs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (activate): ReLU(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.001% FLOPs, inplace=True)
        )
        (layer1): ConvModule(
          0.017 M, 1.758% Params, 0.004 GFLOPs, 0.075% FLOPs,
          (conv): Conv1d(0.017 M, 1.731% Params, 0.004 GFLOPs, 0.073% FLOPs, 128, 128, kernel_size=(1,), stride=(1,))
          (bn): BatchNorm1d(0.0 M, 0.027% Params, 0.0 GFLOPs, 0.001% FLOPs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (activate): ReLU(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.001% FLOPs, inplace=True)
        )
      )
      (conv_cls): Conv1d(0.003 M, 0.270% Params, 0.001 GFLOPs, 0.011% FLOPs, 128, 20, kernel_size=(1,), stride=(1,))
      (conv_reg): Conv1d(0.01 M, 1.041% Params, 0.003 GFLOPs, 0.044% FLOPs, 128, 77, kernel_size=(1,), stride=(1,))
    )
  )
)
==============================
Input shape: (40000, 4)
Flops: 5.78 GFLOPs
Params: 953.83 k
==============================

[Wuziyi616]

Result on FCOS3D Nusc mono3d detection:

python tools/analysis_tools/get_flops.py \
> configs/fcos3d/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d.py \
> --shape 1600 900 \
> --modality image

FCOSMono3D(
  32.798 M, 60.046% Params, 334.696 GFLOPs, 100.000% FLOPs,
  (backbone): ResNet(
    23.333 M, 42.718% Params, 142.378 GFLOPs, 42.540% FLOPs,
    (conv1): Conv2d(0.0 M, 0.000% Params, 3.387 GFLOPs, 1.012% FLOPs, 3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (bn1): BatchNorm2d(0.0 M, 0.000% Params, 0.046 GFLOPs, 0.014% FLOPs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu): ReLU(0.0 M, 0.000% Params, 0.023 GFLOPs, 0.007% FLOPs, inplace=True)
    (maxpool): MaxPool2d(0.0 M, 0.000% Params, 0.023 GFLOPs, 0.007% FLOPs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (layer1): ResLayer(
      0.0 M, 0.000% Params, 19.526 GFLOPs, 5.834% FLOPs,

...

(scales): ModuleList(
      0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs,
      (0): ModuleList(
        0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs,
        (0): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        (1): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        (2): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
      )                                                                                                                                                                                                                                            (1): ModuleList(                                                                                                                                                                                                                               0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs,                                                                                                                                                                                              (0): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )                                                                                                                                                                                 (1): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )                                                                                                                                                                                 (2): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )                                                                                                                                                                               )                                                                                                                                                                                                                                            (2): ModuleList(
        0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs,
        (0): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        (1): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        (2): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
      )
      (3): ModuleList(
        0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs,
        (0): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        (1): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        (2): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
      )
      (4): ModuleList(
        0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs,
        (0): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        (1): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
        (2): Scale(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
      )
    )
    (loss_centerness): CrossEntropyLoss(0.0 M, 0.000% Params, 0.0 GFLOPs, 0.000% FLOPs, )
  )
)
==============================
Input shape: (3, 1600, 900)
Flops: 334.7 GFLOPs
Params: 54.62 M
==============================