【PaddlePaddle Hackathon 3】Add Paddle group_norm operator #12329

Patrick-Star125 · 2022-07-27T15:23:52Z

Details:

add group_norm operation in Paddle front end

Reference:

Unit-test passed

openvino-dev-samples

Thanks for your contribution, could you provide a screenshot for unitest results?

openvino-dev-samples · 2022-07-29T06:06:33Z

src/frontends/paddle/src/op/group_norm.cpp

+
+    auto shape = ov::opset6::Constant::create(ngraph::element::i64, Shape{3}, {0, num_groups, -1});
+    auto reshape_input = std::make_shared<ov::opset6::Reshape>(data, shape);
+    auto scale_ = ov::opset6::Constant::create(dtype, {1}, {1.0 * num_groups});


please use the default_opset , not ov::opset6

openvino-dev-samples · 2022-07-29T06:13:58Z

src/frontends/paddle/src/op/group_norm.cpp

+    auto dtype = data.get_element_type();
+    auto num_groups = node.get_attribute<int32_t>("Groups");
+    auto epsilon = node.get_attribute<float>("Epsilon");
+    auto data_layout = node.get_attribute<std::string>("Data_layout");


better use the default value for 'get_attribute()' function in case it throw error for models which doesn't set this attributes

Patrick-Star125 · 2022-07-29T07:10:06Z

I faced this problem and I don't know how to fix it. @OpenVINO-dev-contest

I think this led to the error described below

I can't find the error log called "unable_build_paddle_models.txt".
It's made by wrong code? or compilation mistaken? I want to figure out the potential factors.

openvino-dev-samples · 2022-07-29T07:15:52Z

I faced this problem and I don't know how to fix it. @OpenVINO-dev-contest I think this led to the error described below

I can't find the error log called "unable_build_paddle_models.txt".

It's made by wrong code? or compilation mistaken? I want to figure out the potential factors.

Have you installed Paddle ? If you haven't Paddle environment, the model generation will not run.
Try run the model generation script separately.

Patrick-Star125 · 2022-07-29T09:41:36Z

Thank you, I fixed that problem by adding python path to the cmake command line parameters
But I ran into another tricky problem (oh..

I think it is some kind of type conversion error. But due to lacking of information, I still have no clue about how to deal with it.

openvino-dev-samples · 2022-07-29T10:05:55Z

Thank you, I fixed that problem by adding python path to the cmake command line parameters But I ran into another tricky problem (oh.. I think it is some kind of type conversion error. But due to lacking of information, I still have no clue about how to deal with it.

Did you try other OP's unitest ? will the same error happen?

Patrick-Star125 · 2022-07-30T03:05:57Z

most of other Op's unittests passed successfully
And I retried to recompile and test my code, it showed some different information this time.
It seems like there is some type conversion problem in my code indeed, but the information provided is quite fuzzy. I combined many OPs to implement group_norm function, is there any way I can locate this problematic conversion?

openvino-dev-samples · 2022-07-30T07:09:32Z

most of other Op's unittests passed successfully

And I retried to recompile and test my code, it showed some different information this time.

It seems like there is some type conversion problem in my code indeed, but the information provided is quite fuzzy. I combined many OPs to implement group_norm function, is there any way I can locate this problematic conversion?

most of other Op's unittests passed successfully.

And I retried to recompile and test my code, it showed some different information this time.

It seems like there is some type conversion problem in my code indeed, but the information provided is quite fuzzy. I combined many OPs to implement group_norm function, is there any way I can locate this problematic conversion?

Hi
I think the error is from variable cast from int to long. you may switch the type of some parameter

Patrick-Star125 · 2022-08-01T08:18:04Z

the CI of Code Style didn't passed. Is that means I should reformat my code with Google C++ standard?

openvino-dev-samples · 2022-08-01T08:23:25Z

the CI of Code Style didn't passed. Is that means I should reformat my code with Google C++ standard?

you can format your code (for VScode click "format selection") and retrigger the CI.

openvino-dev-samples · 2022-08-01T08:28:55Z

Since your mapping is complicated, could you help to explain why you select these OPs to build 'group_norm' ?
Can it be simplified? Do you have more reference sample, like ONNX2IR ? It will helpful for us to speed-up the review process.
Thank you

Patrick-Star125 · 2022-08-01T14:04:00Z

Hi, I want to illustrate my implementation point by point.

Why my mapping is so comlicated.
Unlike most of the other paddle mapping OPs, I didn't not find the function can be called directly, for example <default_opset::GroupNorm>. Therefore, I thought I need to implement this operator on my own, which lead to my mapping comlicated because it's computation process is not quite simple indeed.
Why select these OPs to build 'group_norm'.(Reference updated)
The main structure of my code is refer to the group norm implemented by openvino. In addition, I chang some of the OPs' structure and add some OPs inspired by onnx's implementation.
Whether it can be simplified.
Although this mapping is kind of comlicated, I think it's sematicaly clear(still I can add more annotation), and replace these OPs by other comprehensive operators may bring unnecessary coupling.

openvino-dev-samples · 2022-08-02T10:11:00Z

Hi, I want to illustrate my implementation point by point.

Why my mapping is so comlicated.
Unlike most of the other paddle mapping OPs, I didn't not find the function can be called directly, for example <default_opset::GroupNorm>. Therefore, I thought I need to implement this operator on my own, which lead to my mapping comlicated because it's computation process is not quite simple indeed.

Why select these OPs to build 'group_norm'.(Reference updated)
The main structure of my code is refer to the group norm implemented by openvino. In addition, I chang some of the OPs' structure and add some OPs inspired by onnx's implementation.

Whether it can be simplified.
Although this mapping is kind of comlicated, I think it's sematicaly clear(still I can add more annotation), and replace these OPs by other comprehensive operators may bring unnecessary coupling.

Thanks for your illustration. Could you help to put this link on your Reference which is the document for the OP you try to enable: https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/static/nn/group_norm_cn.html#group-norm

Thanks

Patrick-Star125 · 2022-08-03T02:15:41Z

Reference updated

Patrick-Star125 · 2022-08-05T08:28:31Z

@OpenVINO-dev-contest Could you please review?

openvino-dev-samples · 2022-08-05T08:45:17Z

@OpenVINO-dev-contest Could you please review?

Cloud you help to explain why you only delivery a single output mapping for this operation? and it seems has more one output ?
Thank you.
https://github.com/PaddlePaddle/Paddle/blob/release/2.3/python/paddle/fluid/layers/nn.py#L3790

Patrick-Star125 · 2022-08-06T05:37:06Z

@OpenVINO-dev-contest Hi, I think the placeholder of outputs ["Mean", "Variance"] are just code aligment with other normalization operators.

Unlike the other operators in same file, this python API only return "group_norm_out" instead of a collection of outputs.
the document of this op shows there is only single return indeed.

openvino-dev-samples · 2022-08-08T06:13:01Z

@OpenVINO-dev-contest Hi, I think the placeholder of outputs ["Mean", "Variance"] are just code aligment with other normalization operators.

Unlike the other operators in same file, this python API only return "group_norm_out" instead of a collection of outputs.

the document of this op shows there is only single return indeed.

Thanks for your illustration.

Patrick-Star125 · 2022-08-11T02:08:29Z

@OpenVINO-dev-contest could please you take a review?

openvino-dev-samples · 2022-08-12T02:34:24Z

@OpenVINO-dev-contest could please you take a review?

Hi, seems scale_attr and bias_attr can be in bool, will your test case include it ？
https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/static/nn/group_norm_cn.html#group-norm

Patrick-Star125 · 2022-08-12T09:10:54Z

update: 1.python code of unit test 2.screenshot for unitest results

yuxu42 · 2022-08-15T07:20:08Z

@luo-cheng2021 could you please take a review? Thanks!

luo-cheng2021

The mapping is quite complicated, could you please add some comments in the code to explain the ideas, thanks.

src/frontends/paddle/src/op/group_norm.cpp

ilyachur · 2022-09-05T04:02:24Z

src/frontends/paddle/src/op/group_norm.cpp

+    size_t rank_size = pshape.rank().get_length();
+    PADDLE_OP_CHECK(node, rank_size >= 2, "2-D and above tensors supported only");
+
+    if (data_layout == "NHWC") {


What if don't we have defined layout in the model?

In line 29, the default value of data_layout is "NCHW", which is same as Paddle doc description.

@ilyachur Some Paddle operations have this data_layout attribute, while some are not. For NHWC layout, it is okay to use this layout conversion for the op mapper, I think. We could apply a transformation from model perspective to eliminate the conversions between operations later.

Patrick-Star125 requested a review from a team as a code owner July 27, 2022 15:23

Patrick-Star125 mentioned this pull request Jul 27, 2022

【PaddlePaddle Hackathon 第三期】任务总览 PaddlePaddle/Paddle#43938

Closed

openvino-dev-samples reviewed Jul 29, 2022

View reviewed changes

yuxu42 requested a review from luo-cheng2021 August 15, 2022 07:20

add paddle group norm

c7a4981

luo-cheng2021 requested changes Aug 24, 2022

View reviewed changes

src/frontends/paddle/src/op/group_norm.cpp Outdated Show resolved Hide resolved

src/frontends/paddle/src/op/group_norm.cpp Outdated Show resolved Hide resolved

src/frontends/paddle/src/op/group_norm.cpp Outdated Show resolved Hide resolved

Merge remote-tracking branch 'open/master'

60fe811

Patrick-Star125 requested a review from a team as a code owner August 25, 2022 03:42

remove unecessary code

5acf012

remove unecessary code

7bbfd3f

Patrick-Star125 commented Aug 25, 2022

View reviewed changes

src/frontends/paddle/src/op/group_norm.cpp Show resolved Hide resolved

Patrick-Star125 added 2 commits August 27, 2022 15:08

Merge remote-tracking branch 'open/master'

dc33017

remove unecessary code

484e0f1

luo-cheng2021 approved these changes Aug 29, 2022

View reviewed changes

ceciliapeng2011 approved these changes Sep 5, 2022

View reviewed changes

ceciliapeng2011 added PaddlePaddle Hackathon a Intel and Baidu joint Hackathon event category: PDPD FE OpenVINO PaddlePaddle FrontEnd labels Sep 5, 2022

ilyachur reviewed Sep 5, 2022

View reviewed changes

ilyachur approved these changes Sep 13, 2022

View reviewed changes

ilyachur merged commit 089955f into openvinotoolkit:master Sep 13, 2022

ilyachur added this to the 2022.3 milestone Sep 13, 2022

ilya-lavrenov added the ExternalPR External contributor label Mar 4, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

【PaddlePaddle Hackathon 3】Add Paddle group_norm operator #12329

【PaddlePaddle Hackathon 3】Add Paddle group_norm operator #12329

Patrick-Star125 commented Jul 27, 2022 •

edited

Loading

openvino-dev-samples left a comment

openvino-dev-samples Jul 29, 2022

openvino-dev-samples Jul 29, 2022

Patrick-Star125 commented Jul 29, 2022

openvino-dev-samples commented Jul 29, 2022

Patrick-Star125 commented Jul 29, 2022

openvino-dev-samples commented Jul 29, 2022

Patrick-Star125 commented Jul 30, 2022

openvino-dev-samples commented Jul 30, 2022 •

edited

Loading

Patrick-Star125 commented Aug 1, 2022

openvino-dev-samples commented Aug 1, 2022

openvino-dev-samples commented Aug 1, 2022

Patrick-Star125 commented Aug 1, 2022 •

edited

Loading

openvino-dev-samples commented Aug 2, 2022

Patrick-Star125 commented Aug 3, 2022

Patrick-Star125 commented Aug 5, 2022

openvino-dev-samples commented Aug 5, 2022

Patrick-Star125 commented Aug 6, 2022 •

edited

Loading

openvino-dev-samples commented Aug 8, 2022

Patrick-Star125 commented Aug 11, 2022

openvino-dev-samples commented Aug 12, 2022

Patrick-Star125 commented Aug 12, 2022

yuxu42 commented Aug 15, 2022

luo-cheng2021 left a comment

ilyachur Sep 5, 2022

Patrick-Star125 Sep 6, 2022 •

edited

Loading

ceciliapeng2011 Sep 13, 2022

【PaddlePaddle Hackathon 3】Add Paddle group_norm operator #12329

【PaddlePaddle Hackathon 3】Add Paddle group_norm operator #12329

Conversation

Patrick-Star125 commented Jul 27, 2022 • edited Loading

Details:

Reference:

Unit-test passed

openvino-dev-samples left a comment

Choose a reason for hiding this comment

openvino-dev-samples Jul 29, 2022

Choose a reason for hiding this comment

openvino-dev-samples Jul 29, 2022

Choose a reason for hiding this comment

Patrick-Star125 commented Jul 29, 2022

openvino-dev-samples commented Jul 29, 2022

Patrick-Star125 commented Jul 29, 2022

openvino-dev-samples commented Jul 29, 2022

Patrick-Star125 commented Jul 30, 2022

openvino-dev-samples commented Jul 30, 2022 • edited Loading

Patrick-Star125 commented Aug 1, 2022

openvino-dev-samples commented Aug 1, 2022

openvino-dev-samples commented Aug 1, 2022

Patrick-Star125 commented Aug 1, 2022 • edited Loading

openvino-dev-samples commented Aug 2, 2022

Patrick-Star125 commented Aug 3, 2022

Patrick-Star125 commented Aug 5, 2022

openvino-dev-samples commented Aug 5, 2022

Patrick-Star125 commented Aug 6, 2022 • edited Loading

openvino-dev-samples commented Aug 8, 2022

Patrick-Star125 commented Aug 11, 2022

openvino-dev-samples commented Aug 12, 2022

Patrick-Star125 commented Aug 12, 2022

yuxu42 commented Aug 15, 2022

luo-cheng2021 left a comment

Choose a reason for hiding this comment

ilyachur Sep 5, 2022

Choose a reason for hiding this comment

Patrick-Star125 Sep 6, 2022 • edited Loading

Choose a reason for hiding this comment

ceciliapeng2011 Sep 13, 2022

Choose a reason for hiding this comment

Patrick-Star125 commented Jul 27, 2022 •

edited

Loading

openvino-dev-samples commented Jul 30, 2022 •

edited

Loading

Patrick-Star125 commented Aug 1, 2022 •

edited

Loading

Patrick-Star125 commented Aug 6, 2022 •

edited

Loading

Patrick-Star125 Sep 6, 2022 •

edited

Loading