[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN #9531

ashutosh-arm · 2021-11-18T16:21:47Z

Support for Maxpool and AvgPool via CMSIS-NN.
Code generation for fully connected layer via CMSIS-NN

This PR depends on #9456.

Tracking issue #8646.

manupak

Similiar to other one. Seems like this contains contents from FC PR as well ?

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

manupak · 2021-12-02T15:57:22Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+    tvm::Array<PrimExpr> context_buffer_args = {tir::StringImm(context_buffer_name),
+                                                ToArg(context_buffer_size)};
+
+    scalar_args = tvm::runtime::Concat(context_buffer_args, scalar_args);


I would suggest to keep the declaration/initiatlization vectors that get concat closer to the concat.

ashutosh-arm · 2021-12-02T16:21:21Z

@manupa-arm I think we should merge this one as this contains both of those changes once I've finally pushed changes addressing your comments.

manupak · 2021-12-02T16:23:04Z

If we want to do that please close the other PR and adjust titles and descriptions of this PR to reflect its contents.

ashutosh-arm · 2021-12-02T16:27:18Z

If we want to do that please close the other PR and adjust titles and descriptions of this PR to reflect its contents.

Done

Change-Id: I67f65e69dfa2c6d31c2ea928d92a827df80bc51a

Change-Id: Ibf22250d961a683208faee362d1960ea266347e8

Change-Id: I0675c3248515c9ddc6f25393a3756f907c98fd46

Change-Id: I2e9ca3edec85991be394bbdb2d1739c3afc62d5e

Change-Id: I38fbde65016713956371fa27dbd2a36f844f5799

manupak

Broadly looks good! There are few minor comments related to clarity which I dont want to block the PR for another CI cycle. Would you be able to address them in a follow up ?

manupak · 2021-12-06T17:10:06Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

@@ -164,21 +156,15 @@ class RelayToTIRVisitor : public MixedModeMutator {
                                        ToArg(dilation_w),   ToArg(dilation_h),    ToArg(clip_min),
                                        ToArg(clip_max)};

-    // cmsis_nn_dims *input_dims (NHWC)
+    // layout NHWC


Sorry Ashutosh, I dont still follow what this comment mean. How is this related to the following line ?

This was to help understand that the following shape is in NHWC layout which will be passed on to CMSIS-NN API eventually.

I think might worth saying that explicitly :)

manupak · 2021-12-06T17:10:40Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc


-    // cmsis_nn_dims *filter_dims (OHWI for Conv2D and IHWO for depthwise)
+    // OHWI for Conv2D and IHWO for depthwise


Same here, where is the information OHWI is used ?

Same as above. Maybe I should explicitly say that this is for CMSIS-NN API?

I think the information is that the following line produces a tensor is that layout and subsequent access assume that ?

Yes, comment got displaced in the process of review and new edits 😆

manupak · 2021-12-06T17:12:41Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

@@ -194,7 +180,7 @@ class RelayToTIRVisitor : public MixedModeMutator {
    if (depth_multiplier != -1) {
      cmsisnn_api = "arm_depthwise_conv_wrapper_s8";
      Array<PrimExpr> depthwise_filter_shape{1, filter_shape[0], filter_shape[1], out_channels};


Is it possible to use something like
int kernel_pos_o = kernel_layout.find("O");
instead of the numbers ?

Yes, partly done above. Missed here.

manupak · 2021-12-06T17:15:36Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+                                                ToArg(context_buffer_size)};
+
+    scalar_args = tvm::runtime::Concat(context_buffer_args, scalar_args);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_input_shape);


[Clarity] I think we can bring the declaration "cmsisnn_input_shape" closer to here as it is not used before this

We cannot. Its being used to derive another size in the middle. Also, I would prefer to keep concatenation of arguments at one place. Locating all arguments at same place has helped me with debugs.

I think Im missing something here :). Isn't it defined in line 291 and is there a use in between ?
Anyway, if it helps debugging, lets have it this way.

Sorry, I was looking at a different operator. For some of the others, context buffer creation requires this shape. Just to be consistent, I didn't want to change the code layout.

manupak · 2021-12-06T17:16:08Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+
+    scalar_args = tvm::runtime::Concat(context_buffer_args, scalar_args);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_input_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_filter_shape);


[Clarity] I think we can bring the declaration "cmsisnn_filter_shape" closer to here as it is not used before this

Same as above.

manupak · 2021-12-06T17:16:27Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+    scalar_args = tvm::runtime::Concat(context_buffer_args, scalar_args);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_input_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_filter_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, bias_shape);


[Clarity] I think we can bring the declaration "bias_shape" closer to here as it is not used before this

Same as above.

manupak · 2021-12-06T17:16:45Z

src/relay/backend/contrib/cmsisnn/relay_to_tir.cc

+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_input_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_filter_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, bias_shape);
+    scalar_args = tvm::runtime::Concat(scalar_args, cmsisnn_output_shape);


[Clarity] I think we can bring the declaration "cmsisnn_output_shape" closer to here as it is not used before this

Same as above.

ashutosh-arm · 2021-12-06T17:23:32Z

Thanks for the reviews.
Sure @manupa-arm, I can create a follow up for this. Please feel free to merge.

ashutosh-arm · 2021-12-06T17:45:12Z

@manupa-arm Could you please put commit message as "Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN"

manupak · 2021-12-06T17:46:40Z

Thanks @ashutosh-arm ! This is merged now.

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

ashutosh-arm requested review from anijain2305, areusch, comaniac, icemelon, jroesch, junrushao, jwfromm, leandron, manupak, MarisaKirisame, mbaret, mbrookhart, merrymercy, slyubomirsky, tqchen, trevor-m, vinx13, wweic, yzhliu, zhiics and ZihengJiang as code owners November 18, 2021 16:21

ashutosh-arm changed the title ~~Pooling~~ Code generation for Pooling via CMSIS-NN Nov 18, 2021

ashutosh-arm changed the title ~~Code generation for Pooling via CMSIS-NN~~ [7/10] Code generation for Pooling via CMSIS-NN Nov 18, 2021

ashutosh-arm mentioned this pull request Nov 18, 2021

[RFC][Tracking Issue] CMSIS-NN Integration #8646

Closed

ashutosh-arm force-pushed the pooling branch from b06bb11 to 28bc3dd Compare December 1, 2021 09:49

manupak reviewed Dec 2, 2021

View reviewed changes

Code generation for fully connected layer via CMSIS-NN

093f3e1

Change-Id: I67f65e69dfa2c6d31c2ea928d92a827df80bc51a

ashutosh-arm added 3 commits December 3, 2021 10:33

Code generation for Pooling layers via CMSIS-NN

2ea9511

Change-Id: Ibf22250d961a683208faee362d1960ea266347e8

Skip tests for i386 CI runs

c4d7c89

Change-Id: I0675c3248515c9ddc6f25393a3756f907c98fd46

Addressed review comments: removed disconcerting comments

a51669a

Change-Id: I2e9ca3edec85991be394bbdb2d1739c3afc62d5e

ashutosh-arm force-pushed the pooling branch from 28bc3dd to a51669a Compare December 3, 2021 10:33

Dummy commit

a05b17d

Change-Id: I38fbde65016713956371fa27dbd2a36f844f5799

ashutosh-arm changed the title ~~[7/10] Code generation for Pooling via CMSIS-NN~~ [7/10] Code generation for Pooling and Fully Connected via CMSIS-NN Dec 6, 2021

manupak approved these changes Dec 6, 2021

View reviewed changes

manupak merged commit 3f1c931 into apache:main Dec 6, 2021

ashutosh-arm deleted the pooling branch December 7, 2021 10:05

ashutosh-arm mentioned this pull request Dec 10, 2021

Follow up from CMSIS-NN Pooling failure #9708

Merged

ylc pushed a commit to ylc/tvm that referenced this pull request Jan 7, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

8bb072a

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

yangulei pushed a commit to yangulei/tvm that referenced this pull request Jan 11, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

8de4263

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

yangulei pushed a commit to yangulei/tvm that referenced this pull request Jan 12, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

ed48b7c

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

ylc pushed a commit to ylc/tvm that referenced this pull request Jan 13, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

574390d

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

qsqqsqqsq-intellif pushed a commit to qsqqsqqsq-intellif/tvm that referenced this pull request Apr 29, 2022

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN (a…

de361b8

…pache#9531) Support for code generation of Maxpool, AvgPool and Fully Connected layers via CMSIS-NN

driazati mentioned this pull request Jul 14, 2022

TVM v0.9.0.rc0 Release Candidate Notes #12102

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN #9531

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN #9531

ashutosh-arm commented Nov 18, 2021 •

edited

Loading

manupak left a comment

manupak Dec 2, 2021

ashutosh-arm commented Dec 2, 2021

manupak commented Dec 2, 2021

ashutosh-arm commented Dec 2, 2021

manupak left a comment

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

manupak Dec 6, 2021

ashutosh-arm Dec 6, 2021

ashutosh-arm commented Dec 6, 2021

ashutosh-arm commented Dec 6, 2021

manupak commented Dec 6, 2021


		// cmsis_nn_dims *filter_dims (OHWI for Conv2D and IHWO for depthwise)
		// OHWI for Conv2D and IHWO for depthwise

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN #9531

[7/10] Code generation for Pooling and Fully Connected via CMSIS-NN #9531

Conversation

ashutosh-arm commented Nov 18, 2021 • edited Loading

manupak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ashutosh-arm commented Dec 2, 2021

manupak commented Dec 2, 2021

ashutosh-arm commented Dec 2, 2021

manupak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ashutosh-arm commented Dec 6, 2021

ashutosh-arm commented Dec 6, 2021

manupak commented Dec 6, 2021

ashutosh-arm commented Nov 18, 2021 •

edited

Loading