Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CUDA][PASS]Legalize tensorcore #7147

Merged
merged 17 commits into from
Jan 29, 2021
Merged

[CUDA][PASS]Legalize tensorcore #7147

merged 17 commits into from
Jan 29, 2021

Conversation

Meteorix
Copy link
Contributor

Add legalize pass: padding dense/conv2d/batch_matmul ops to legal shapes for using tensorcore on cuda target. To limit the overhead introduced by padding, we count the extra_flops and set the threshold to 2x, which is conservative compared to the speedup of tensorcore.

This pr is dependent on #7146 .

@jcf94 @merrymercy could you also help review this pr?

jwfromm
jwfromm previously requested changes Dec 23, 2020
View reviewed changes
Copy link
Contributor

@jwfromm jwfromm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution! I like the idea of padding to make other shapes work on tensorcore, but I think its important that you add some tests to confirm this features works as expected before we consider merging.

python/tvm/topi/cuda/conv2d_alter_op.py Outdated Show resolved Hide resolved
@jcf94 jcf94 mentioned this pull request Dec 24, 2020
Merged
@Meteorix
Merge branch 'main' into legalize_tensorcore_2
25dac28 
Conflicts:
	python/tvm/topi/nn/batch_matmul.py
	python/tvm/topi/nn/dense.py
@Meteorix
Copy link
Contributor Author

@jwfromm @jcf94 I've added some tests. Could you help review this PR again?

And the cpu ci failed, I am not sure how to fix it.

Laurawly
Laurawly reviewed Jan 25, 2021
View reviewed changes
python/tvm/topi/cuda/conv2d_alter_op.py Outdated Show resolved Hide resolved
tests/python/relay/test_pass_legalize_tensorcore.py Outdated
return entry if isinstance(expr, relay.Function) else entry.body


def test_legalize_conv2d(data_shape, kernel_shape, pad_shape, do_pad=True):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please reference test_legalize_pass.py for the CI issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@Meteorix
Copy link
Contributor Author

@Laurawly Thanks! The ci passed.

@Laurawly
Copy link
Contributor

@jwfromm @jcf94 Could you update your reviews?

jcf94
jcf94 approved these changes Jan 28, 2021
View reviewed changes
Copy link
Contributor

@jcf94 jcf94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Sorry for my late reply. This looks good to me.

@Laurawly Laurawly requested a review from jwfromm January 28, 2021 06:38
@Laurawly Laurawly dismissed jwfromm’s stale review January 29, 2021 02:17

Waiting for review updates.

jwfromm
jwfromm approved these changes Jan 29, 2021
View reviewed changes
Copy link
Contributor

@jwfromm jwfromm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the tests, this LGTM now.

@Laurawly
Copy link
Contributor

Thank you @Meteorix @jwfromm @jcf94!

@Laurawly Laurawly merged commit 3734d5f into apache:main Jan 29, 2021
alexwong pushed a commit to alexwong/tvm that referenced this pull request Feb 11, 2021
@Meteorix @alexwong
[CUDA][PASS]Legalize tensorcore (apache#7147)
2abb880 
* add pad_to_tensorcore & legalize for dense/bmm/conv2d

* fix pad & slice

* fix comments

* fix comments

* resolve conflict

* resolve conflict

* support only fp16

* add tests/python/relay/test_pass_legalize_tensorcore.py

* add tests for legalize tensorcore

* fix pylint

* fix pylint

* code format

* use_gpu test only; fix conv2d_alter_op

* fix tests params

* revert transform fix
electriclilies pushed a commit to electriclilies/tvm that referenced this pull request Feb 18, 2021
@Meteorix @electriclilies
[CUDA][PASS]Legalize tensorcore (apache#7147)
aa6a4b4 
* add pad_to_tensorcore & legalize for dense/bmm/conv2d

* fix pad & slice

* fix comments

* fix comments

* resolve conflict

* resolve conflict

* support only fp16

* add tests/python/relay/test_pass_legalize_tensorcore.py

* add tests for legalize tensorcore

* fix pylint

* fix pylint

* code format

* use_gpu test only; fix conv2d_alter_op

* fix tests params

* revert transform fix
Lokiiiiii pushed a commit to Lokiiiiii/tvm that referenced this pull request Mar 2, 2021
@Meteorix @Lokiiiiii
[CUDA][PASS]Legalize tensorcore (apache#7147)
0998457 
* add pad_to_tensorcore & legalize for dense/bmm/conv2d

* fix pad & slice

* fix comments

* fix comments

* resolve conflict

* resolve conflict

* support only fp16

* add tests/python/relay/test_pass_legalize_tensorcore.py

* add tests for legalize tensorcore

* fix pylint

* fix pylint

* code format

* use_gpu test only; fix conv2d_alter_op

* fix tests params

* revert transform fix
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Mar 2, 2021
@Meteorix @trevor-m
[CUDA][PASS]Legalize tensorcore (apache#7147)
13cbfcb 
* add pad_to_tensorcore & legalize for dense/bmm/conv2d

* fix pad & slice

* fix comments

* fix comments

* resolve conflict

* resolve conflict

* support only fp16

* add tests/python/relay/test_pass_legalize_tensorcore.py

* add tests for legalize tensorcore

* fix pylint

* fix pylint

* code format

* use_gpu test only; fix conv2d_alter_op

* fix tests params

* revert transform fix
@masahi masahi mentioned this pull request Sep 16, 2021
Merged
@junrushao junrushao mentioned this pull request Nov 1, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kongroo kongroo kongroo left review comments

@Laurawly Laurawly Laurawly left review comments

@jcf94 jcf94 jcf94 approved these changes

@jwfromm jwfromm jwfromm approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Meteorix @Laurawly @jwfromm @jcf94 @kongroo