[Hackathon NO.71] 为 Paddle-TRT 添加 pad3d 算子 #50986

AndSonder · 2023-02-27T13:53:11Z

PR types

Others

PR changes

Others

Describe

为 Paddle-TRT 添加 pad3d 算子
已通过单测，单测结果如下：

paddle-bot · 2023-02-27T13:53:16Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

zhangjun · 2023-02-28T07:23:40Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+    const std::vector<int> paddings =
+        PADDLE_GET_CONST(std::vector<int>, op_desc.GetAttr("paddings"));


需考虑Paddings作为tensor输入情形。同时还有其他属性value、mode、data_format, 详细OP定义参考 https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/pad3d_op.cc#L71-L110，实现参考https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/kernels/cpu/pad3d_kernel.cc

我参考了 pad_op.cc 的OP实现，两个API输入基本一样，但是 pad_op.cc 中并没有您上述的要求

针对value, 源代码中表明："The pad layer of TRT only support zero."

https://github.com/AndSonder/Paddle/blob/5e8a79e1cf6d1adc7139883398b23f075fd47d79/paddle/fluid/inference/tensorrt/op_teller.cc#L1688-L1695

对于 mode 和 data_format 我也没有看到 pad_op.cc 对其进行处理，请问需要都加上吗？ @zhangjun

这个题目的单测文件是你们提供，是否也需要添加上述的样例？

我参考了 pad_op.cc 的OP实现，两个API输入基本一样，但是 pad_op.cc 中并没有您上述的要求

针对value, 源代码中表明："The pad layer of TRT only support zero."

https://github.com/AndSonder/Paddle/blob/5e8a79e1cf6d1adc7139883398b23f075fd47d79/paddle/fluid/inference/tensorrt/op_teller.cc#L1688-L1695

对于 mode 和 data_format 我也没有看到 pad_op.cc 对其进行处理，请问需要都加上吗？ @zhangjun

这个题目的单测文件是你们提供，是否也需要添加上述的样例？

pad3d和pad有差异。新增加OP单测需要单独添加，已有的单测文件可供参考，输入、属性完全一样的情形下可复用。

我参考了 pad_op.cc 的OP实现，两个API输入基本一样，但是 pad_op.cc 中并没有您上述的要求
针对value, 源代码中表明："The pad layer of TRT only support zero."
https://github.com/AndSonder/Paddle/blob/5e8a79e1cf6d1adc7139883398b23f075fd47d79/paddle/fluid/inference/tensorrt/op_teller.cc#L1688-L1695
对于 mode 和 data_format 我也没有看到 pad_op.cc 对其进行处理，请问需要都加上吗？ @zhangjun
这个题目的单测文件是你们提供，是否也需要添加上述的样例？

pad3d和pad有差异。新增加OP单测需要单独添加，已有的单测文件可供参考，输入、属性完全一样的情形下可复用。

通过你们的这个单测是给出的题目要求

我参考了 pad_op.cc 的OP实现，两个API输入基本一样，但是 pad_op.cc 中并没有您上述的要求
针对value, 源代码中表明："The pad layer of TRT only support zero."
https://github.com/AndSonder/Paddle/blob/5e8a79e1cf6d1adc7139883398b23f075fd47d79/paddle/fluid/inference/tensorrt/op_teller.cc#L1688-L1695
对于 mode 和 data_format 我也没有看到 pad_op.cc 对其进行处理，请问需要都加上吗？ @zhangjun
这个题目的单测文件是你们提供，是否也需要添加上述的样例？

pad3d和pad有差异。新增加OP单测需要单独添加，已有的单测文件可供参考，输入、属性完全一样的情形下可复用。

我又仔细看了一下pad_op.cc 这个算子有很多局限的地方，这个任务是需要实现把所有属性都考虑到吗，但是你们官方给的单测代码并没要求那么多，现在的代码已经可以完美通过你们给的单测了 @weishengying @zhangjun

我参考了 pad_op.cc 的OP实现，两个API输入基本一样，但是 pad_op.cc 中并没有您上述的要求
针对value, 源代码中表明："The pad layer of TRT only support zero."
https://github.com/AndSonder/Paddle/blob/5e8a79e1cf6d1adc7139883398b23f075fd47d79/paddle/fluid/inference/tensorrt/op_teller.cc#L1688-L1695
对于 mode 和 data_format 我也没有看到 pad_op.cc 对其进行处理，请问需要都加上吗？ @zhangjun
这个题目的单测文件是你们提供，是否也需要添加上述的样例？

pad3d和pad有差异。新增加OP单测需要单独添加，已有的单测文件可供参考，输入、属性完全一样的情形下可复用。

我又仔细看了一下pad_op.cc 这个算子有很多局限的地方，这个任务是需要实现把所有属性都考虑到吗，但是你们官方给的单测代码并没要求那么多，现在的代码已经可以完美通过你们给的单测了 @weishengying @zhangjun

单测文件只是写法示例。核心是完备的功能实现，单测需要进行新的添加，现有单测可以部分复用或者完善。

我参考了 pad_op.cc 的OP实现，两个API输入基本一样，但是 pad_op.cc 中并没有您上述的要求
针对value, 源代码中表明："The pad layer of TRT only support zero."
https://github.com/AndSonder/Paddle/blob/5e8a79e1cf6d1adc7139883398b23f075fd47d79/paddle/fluid/inference/tensorrt/op_teller.cc#L1688-L1695
对于 mode 和 data_format 我也没有看到 pad_op.cc 对其进行处理，请问需要都加上吗？ @zhangjun
这个题目的单测文件是你们提供，是否也需要添加上述的样例？

pad3d和pad有差异。新增加OP单测需要单独添加，已有的单测文件可供参考，输入、属性完全一样的情形下可复用。

我又仔细看了一下pad_op.cc 这个算子有很多局限的地方，这个任务是需要实现把所有属性都考虑到吗，但是你们官方给的单测代码并没要求那么多，现在的代码已经可以完美通过你们给的单测了 @weishengying @zhangjun

单测文件只是写法示例。核心是完备的功能实现，单测需要进行新的添加，现有单测可以部分复用或者完善。

了解了

需考虑Paddings作为tensor输入情形。同时还有其他属性value、mode、data_format, 详细OP定义参考 https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/pad3d_op.cc#L71-L110，实现参考https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/phi/kernels/cpu/pad3d_kernel.cc

已添加Paddings作为tensor输入的情况，其他属性如value，mode 等添加了Tensorrt不支持的相关提示
单侧代码添加了对于Paddings为Tensor和list 情况下的测试案例

本地单侧通过

zhangjun · 2023-03-03T03:30:03Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+    auto* layer = TRT_ENGINE_ADD_LAYER(engine_,
+                                       PaddingNd,
+                                       *const_cast<nvinfer1::ITensor*>(input),
+                                       pre_pad,
+                                       post_pad);


这个好像只支持2d，可以用slice 可参考https://github.com/onnx/onnx-tensorrt/blob/fd119fec8565264add819f8edc801066116a32dd/builtin_op_importers.cpp#L3210

已按照要求更改，单测添加了不同填充的mode，通过本地单测

…into pad3d

zhangjun · 2023-03-05T16:19:49Z

paddle/fluid/inference/tensorrt/op_teller.cc

+#if !IS_TRT_VERSION_GE(8200)
+      VLOG(3) << "pad3d is not supported when TensorRT < 8.4";
+      return false;
+#endif


应该是 “< 8.2”

zhangjun · 2023-03-05T16:20:32Z

paddle/fluid/inference/tensorrt/op_teller.cc

+        if (mode != "constant" && mode != "reflect" && mode != "replicate") {
+          VLOG(3) << "The pad3d layer of TRT only support "


这里应该是或 ||

zhangjun · 2023-03-05T16:21:21Z

paddle/fluid/inference/tensorrt/op_teller.cc

+      std::vector<int64_t> shape;
+      auto* block = desc.Block();
+      if (block == nullptr) {
+        VLOG(3) << "The block desc is nullptr, we can't continue to analyze. "
+                   "Developers need to check whether block_desc is passed in "
+                   "the pass.";
+        return false;
+      }


这部分可以删掉

zhangjun · 2023-03-05T16:25:07Z

python/paddle/fluid/tests/unittests/ir/inference/test_trt_convert_pad3d.py

+        if len(attrs[0]['paddings']) == 6:
+            # for dynamic_shape


这里为什么要限制paddings长度为6

zhangjun · 2023-03-05T16:28:03Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+      value = PADDLE_GET_CONST(float, op_desc.GetAttr("value"));
+    }
+
+    std::string padding_mode;


直接给默认值constant

zhangjun · 2023-03-05T16:29:58Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+    nvinfer1::ITensor* pre_pad = vectorToTensor<int>(pre_pad_v);
+    nvinfer1::ITensor* post_pad = vectorToTensor<int>(post_pad_v);
+
+    std::vector<int> zeros_v(inputDim, 0);
+    auto const zeros = vectorToTensor<int>(zeros_v);


有一个Add1DConstantLayer API可以直接用

zhangjun · 2023-03-05T16:32:17Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+    } else if (padding_mode == "reflect") {
+      slice_layer->setMode(nvinfer1::SliceMode::kREFLECT);
+    } else if (padding_mode == "replicate") {
+      slice_layer->setMode(nvinfer1::SliceMode::kCLAMP);


这是TRT 8.2后添加的，这里Converter实现可以用宏定义包起来，参考one_hot实现。

已添加trt限制

zhangjun · 2023-03-05T16:38:19Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+      auto* paddings_v = scope.FindVar(op_desc.Input("Paddings")[0]);
+      auto* padding_t = paddings_v->GetMutable<phi::DenseTensor>();
+      phi::DenseTensor paddings_tensor;
+      paddings_tensor.Resize(padding_t->dims());
+      platform::CPUPlace cpu_place;
+      paddle::framework::TensorCopySync(
+          (*padding_t), cpu_place, &paddings_tensor);
+      auto* paddings_data =
+          paddings_tensor.mutable_data<int>(platform::CPUPlace());


有一个GetITensor的函数可以利用

这里写法是有问题的，动态shape下Paddings为tensor输入，这里是获取不到数据的，全部都需要使用Tensor形式操作

这里应该是可以获取到的，我参考了 batch norm 那个op的写法，数据是从 op_desc.Input("Paddings") 里面拿到的

zhangjun · 2023-03-08T07:47:53Z

paddle/fluid/inference/tensorrt/op_teller.cc

+      }
+      if (desc.HasAttr("mode")) {
+        std::string mode = PADDLE_GET_CONST(std::string, desc.GetAttr("mode"));
+        if (mode != "constant" || (mode != "reflect" && mode != "replicate")) {


这个写法有问题，应该是“如果全部都不等于”

zhangjun · 2023-03-08T07:49:09Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+      auto* paddings_v = scope.FindVar(op_desc.Input("Paddings")[0]);
+      auto* padding_t = paddings_v->GetMutable<phi::DenseTensor>();
+      phi::DenseTensor paddings_tensor;
+      paddings_tensor.Resize(padding_t->dims());
+      platform::CPUPlace cpu_place;
+      paddle::framework::TensorCopySync(
+          (*padding_t), cpu_place, &paddings_tensor);
+      auto* paddings_data =
+          paddings_tensor.mutable_data<int>(platform::CPUPlace());


这里写法是有问题的，动态shape下Paddings为tensor输入，这里是获取不到数据的，全部都需要使用Tensor形式操作

zhangjun · 2023-03-08T07:49:43Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+    for (int i = 0; i < inputDim; i += 2) {
+      pre_pad_v[i + 2] = paddings[pad_size - 2 - i];
+      post_pad_v[i + 2] = paddings[pad_size - 1 - i];
+    }


同上面提到问题，paddings不一定能获取到

… pad3d

… split the pre_pad and the post pad]

zhangjun

LGTM

zhangjun · 2023-03-19T17:18:35Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+    auto* input = engine_->GetITensor(op_desc.Input("X")[0]);
+
+    nvinfer1::ITensor* paddings;
+    if (op_desc.HasInput("Paddings")) {


需要同时满足 op_desc.Input("Paddings").size() > 0，否则会core dump

zhangjun · 2023-03-19T17:19:51Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

@@ -0,0 +1,184 @@
+/* Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.


顺便改成2023吧

zhangjun · 2023-03-19T17:20:36Z

paddle/fluid/inference/tensorrt/convert/pad3d_op.cc

+                  const framework::Scope& scope,
+                  bool test_mode) override {
+#if IS_TRT_VERSION_GE(8200)
+    VLOG(3) << "convert a transpose op to tensorrt tranpose layer";


改成pad3d

zhangjun

LGTM

update codes about pad3d

5e8a79e

paddle-bot bot added contributor External developers status: proposed labels Feb 27, 2023

AndSonder mentioned this pull request Feb 28, 2023

【PaddlePaddle Hackathon 第四期】任务总览 #50629

Closed

luotao1 assigned zhangjun, luotao1 and cloud2009 Feb 28, 2023

zhangjun reviewed Feb 28, 2023

View reviewed changes

AndSonder requested a review from zhangjun March 1, 2023 04:27

AndSonder added 5 commits March 2, 2023 10:21

add codes about Tensor type Padding

5d6bf44

update

443f608

更新单测文件

0c92e0e

format code style

66a3c42

update and to &&'

933e021

zhangjun reviewed Mar 3, 2023

View reviewed changes

AndSonder added 6 commits March 4, 2023 15:23

rewrite codes about pad3d

ed83ca0

add codes about converting paddle pad format to tensorrt pad format

01b6b4c

pull Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle …

f1563b6

…into pad3d

fix some errors

0deb315

指定trt版本范围

c246abb

修正dims初始化方式

89f0957

AndSonder requested a review from zhangjun March 5, 2023 04:37

AndSonder added 2 commits March 5, 2023 13:13

fix code style

90713f3

update test pad values

8a72aff

zhangjun reviewed Mar 5, 2023

View reviewed changes

AndSonder added 2 commits March 6, 2023 16:43

指定pad3d trt版本

924ce3f

更新单测文件范围

215472c

luotao1 assigned Ligoml Mar 6, 2023

Ligoml mentioned this pull request Mar 7, 2023

【PaddlePaddle Hackathon 第四期】任务总览 #51281

Closed

更新单测文件

dfcd101

AndSonder force-pushed the pad3d branch from 8cdbac7 to dfcd101 Compare March 7, 2023 13:15

zhangjun reviewed Mar 8, 2023

View reviewed changes

AndSonder added 9 commits March 8, 2023 16:21

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

eac5f71

… pad3d

update pad3d paddings convert codes

5e50733

update pad3d

ac1918d

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

40afeb7

… pad3d

add static mode support

dcf4210

update test file

b448e45

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

13a1c82

… pad3d

fix bugs about dynamic mode test codes

a3106bd

fix bug and add limite in op_teller

46a6218

AndSonder requested a review from zhangjun March 14, 2023 13:41

AndSonder added 3 commits March 14, 2023 22:35

use a new padding convert method[ITensor* padding with using Slice to…

458f538

… split the pre_pad and the post pad]

fix PADDLE_THROW grammaly error

c90c556

update test codes

af0d6f3

zhangjun previously approved these changes Mar 17, 2023

View reviewed changes

zhangjun self-requested a review March 19, 2023 16:48

zhangjun reviewed Mar 19, 2023

View reviewed changes

添加对于Tensor padding 的 size 判断

61363ee

AndSonder dismissed zhangjun’s stale review via 61363ee March 20, 2023 01:39

AndSonder requested a review from zhangjun March 20, 2023 01:41

zhangjun approved these changes Mar 20, 2023

View reviewed changes

luotao1 merged commit c36e3fd into PaddlePaddle:develop Mar 20, 2023

AndSonder deleted the pad3d branch April 23, 2024 13:57

		const std::vector<int> paddings =
		PADDLE_GET_CONST(std::vector<int>, op_desc.GetAttr("paddings"));

		if (mode != "constant" && mode != "reflect" && mode != "replicate") {
		VLOG(3) << "The pad3d layer of TRT only support "

		@@ -0,0 +1,184 @@
		/* Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.

[Hackathon NO.71] 为 Paddle-TRT 添加 pad3d 算子 #50986

[Hackathon NO.71] 为 Paddle-TRT 添加 pad3d 算子 #50986

Conversation

AndSonder commented Feb 27, 2023

PR types

PR changes

Describe

paddle-bot bot commented Feb 27, 2023

Choose a reason for hiding this comment

AndSonder Feb 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndSonder Feb 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhangjun left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhangjun left a comment

Choose a reason for hiding this comment

AndSonder Feb 28, 2023 •

edited

Loading

AndSonder Feb 28, 2023 •

edited

Loading