Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support PAConv operation #598

Merged
merged 10 commits into from
Jun 7, 2021
Merged

Conversation

Wuziyi616
Copy link
Contributor

@Wuziyi616 Wuziyi616 commented May 27, 2021

Implement PAConv layer. PAConv is similar to Conv2d or MLP, serving as basic building block of a model. The basic idea of PAConv is simple. PAConv dynamically generates convolutional kernels by assembling some weight kernels in its weight bank. Every PAConv has a weight bank which consists of several (typically 16) trainable weight kernels, and a ScoreNet which is implemented by MLPs. Given a point pair p_c is a center and p_1 is its neighbor (queried by KNN), it uses ScoreNet to predict scores for this point pair, and uses the score to assemble weight bank to form one kernel. Then, it runs convolution using the assembled kernel on the point pair and get output features.

PAConv has two versions of implementation, one is called PAConv and the other one is called PAConvCUDA. The differences are:

  • PAConv is easy to understand, and can run on both CPU as well as GPU. Its inputs are features already grouped by KNN, so it only needs to compute scores and assemble weights and get the output.
  • PAConvCUDA is tricky, and can only run on GPU because it utilizes a custom cuda op assign_score_withk. Its inputs are features NOT grouped by KNN, and queries KNN on the fly (hard to describe). This saves memory because we don't need to pre-compute KNN features which have very large tensor size (typically (B, C, npoint, K)).

The operation is somewhat complex and I am not sure if I should write its detailed process in the comment. Also, the original code is dirty, although I tried my best to add comments and made it clear, it may still not meet our standard. Please kindly help me improve it (e.g. some function's name).

@codecov
Copy link

codecov bot commented May 27, 2021

Codecov Report

Merging #598 (cd8730e) into master (20c0fbd) will increase coverage by 0.07%.
The diff coverage is 57.22%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #598      +/-   ##
==========================================
+ Coverage   50.92%   51.00%   +0.07%     
==========================================
  Files         197      201       +4     
  Lines       15056    15220     +164     
  Branches     2444     2467      +23     
==========================================
+ Hits         7668     7763      +95     
- Misses       6884     6939      +55     
- Partials      504      518      +14     
Flag Coverage Δ
unittests 51.00% <57.22%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmdet3d/ops/group_points/group_points.py 31.11% <12.50%> (-1.04%) ⬇️
mmdet3d/ops/paconv/assign_score.py 29.16% <29.16%> (ø)
mmdet3d/ops/paconv/utils.py 52.38% <52.38%> (ø)
mmdet3d/ops/paconv/paconv.py 66.05% <66.05%> (ø)
mmdet3d/ops/__init__.py 100.00% <100.00%> (ø)
mmdet3d/ops/paconv/__init__.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 20c0fbd...cd8730e. Read the comment docs.

@Wuziyi616 Wuziyi616 requested a review from Tai-Wang May 28, 2021 03:11
@Wuziyi616
Copy link
Contributor Author

Would be better if you read the paper before reviewing my code.

Copy link
Contributor Author

@Wuziyi616 Wuziyi616 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this PR is ready for review @Tai-Wang @ZwwWayne

@ZwwWayne ZwwWayne merged commit c33d4ec into open-mmlab:master Jun 7, 2021
@Wuziyi616 Wuziyi616 deleted the paconv_op branch June 7, 2021 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants