Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mlir][tosa][linalg] Apply direct tosa -> linalg Conv2D lowering #68304

Merged

Conversation

FranklandJack
Copy link
Contributor

@FranklandJack FranklandJack commented Oct 5, 2023

TOSA defines the filter channel ordering for 2D convolution operation tosa.conv2d as [OC, KH, KW, IC]. The LinAlg dialect supports [F, H, W, C] and [H, W, C, F] orderings via the linalg.conv_2d_nhwc_fhwc and linalg.conv_2d_nhwc_hwcf operations respectively. Where F == OC, KH == H, KW == W and C == IC.

Currently tosa.conv2d is lowered to linalg.conv2d_nhwc_hwcf meaning we need to insert a transposition operation to permute the filter channels before they can be passed as weights to the linalg op, that is [F, H, W, C] -> [H, W, C, F]. An analogous transformation needs to be applied to the quantized operation that lowers to linalg.conv_2d_nhwc_hwcf_q.

This commit updates the TOSA->LinAlg lowering so that tosa.conv2d is lowered to linalg.conv2d_nhwc_fhwc removing the need for the introduction of a transposition operation and making the mapping 1-1. It also adds a linalg.conv_2d_nhwc_fhwc_q quantized operation to the LinAlg dialect so the same direct 1-1 mapping can be applied to the quantized variant.

This commit does not add any new lit tests but repurposes the current TosaToLinalgNamed tests by removing the checks for transpositions and updating the targeted LinAlg operations from linalg.conv2d_nhwc_hwcf to linalg.conv2d_nhwc_fhwc`.

@llvmbot llvmbot added mlir:linalg mlir:python MLIR Python bindings mlir labels Oct 5, 2023
TOSA defines the filter channel ordering for 2D convolution operation
`tosa.conv2d` as `[OC, KH, KW, IC]`. The LinAlg dialect supports `[F, H,
W, C]` and `[H, W, C, F]` orderings via the `linalg.conv_2d_nhwc_fhwc`
and `linalg.conv_2d_nhwc_hwcf` operations respectively. Where `F == OC`,
`KH == H`, `KW == W` and `C == IC`.

Currently `tosa.conv2d` is lowered to `linalg.conv2d_nhwc_hwcf` meaning
we need to insert a transposition operation to permute the filter
channels before they can be passed as weights to the linalg op, that is
`[F, H, W, C]` -> `[H, W, C, F]`. An analogous transformation needs to
be applied to the quantized operation that lowers to
`linalg.conv_2d_nhwc_hwcf_q`.

This commit updates the TOSA->LinAlg lowering so that `tosa.conv2d` is
lowered to `linalg.conv2d_nhwc_fhwc` removing the need for the
introduction of a transposition operation and making the mapping 1-1. It
also adds a `linalg.conv_2d_nhwc_fhwc_q` quantized operation to the
LinAlg dialect so the same direct 1-1 mapping can be applied to the
quantized variant.

This commit does not add any new lit tests but repurposes the current
TosaToLinalgNamed tests by removing the checks for transpositions and
updating the targeted LinAlg operations from `linalg.conv2d_nhwc_hwcf`
to linalg.conv2d_nhwc_fhwc`.

Signed-off-by: Jack Frankland <jack.frankland@arm.com>
@FranklandJack FranklandJack force-pushed the jacfra01/tosa-linalg-conv2d-nhwc+fhwc branch from 58be62d to 9e5e7d9 Compare October 5, 2023 13:08
@eric-k256 eric-k256 requested a review from NatashaKnk October 5, 2023 16:11
Copy link
Contributor

@NatashaKnk NatashaKnk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@eric-k256 eric-k256 merged commit e29a253 into llvm:main Oct 7, 2023
2 checks passed
stellaraccident added a commit to iree-org/llvm-project that referenced this pull request Oct 16, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 18, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 19, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 20, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 20, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 20, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 20, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 20, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 20, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
Groverkss pushed a commit to iree-org/llvm-project that referenced this pull request Oct 20, 2023
…ing (llvm#68304)"

This reverts commit e29a253.

Breaking TFLite mobilenet test. Needs triage.
bjacob added a commit that referenced this pull request Oct 27, 2023
… ops. (#70482)

Switching to FHWC happened in
#68304 and is fine in itself
but caused downstream performance regression
iree-org/iree#15296 (comment) , so
this PR makes this optional.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants