[Hackathon NO.73] 为 Paddle-TRT 添加 temporal_shift 算子 #51207
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
zhangjun
reviewed
Mar 6, 2023
Comment on lines
2582
to
2587
| if (op_type == "temporal_shift") { | ||
| #if !IS_TRT_VERSION_GE(8200) | ||
| VLOG(3) << "temporal_shift is not supported when TensorRT < 8.5.1"; | ||
| return false; | ||
| #endif | ||
| } |
There was a problem hiding this comment.
不支持静态shape可以在op teller里面说明
There was a problem hiding this comment.
在 paddle/fluid/framework/ir/trt_support_nhwc_pass.cc 中设置了保持该算子输入的维度，目前已支持静态shape输入
Comment on lines
86
to
91
| # # for static_shape | ||
| # clear_dynamic_shape() | ||
| # self.trt_param.precision = paddle_infer.PrecisionType.Float32 | ||
| # yield self.create_inference_config(), (1, 3), 1e-5 | ||
| # self.trt_param.precision = paddle_infer.PrecisionType.Half | ||
| # yield self.create_inference_config(), (1, 3), 1e-3 |
Comment on lines
50
to
59
| const int NT = input_dims.d[0]; | ||
| const int C = input_dims.d[1]; | ||
| const int H = input_dims.d[2]; | ||
| const int W = input_dims.d[3]; | ||
| const int N = NT / T; | ||
|
|
||
| // Reshape input to [N,T,C,H,W] | ||
| auto reshape_layer = TRT_ENGINE_ADD_LAYER(engine_, Shuffle, *input); | ||
| nvinfer1::Dims reshape_dims{5, {N, T, C, H, W}}; | ||
| reshape_layer->setReshapeDimensions(reshape_dims); |
There was a problem hiding this comment.
在 op 代码中添加了 Permute 的代码，单测代码已添加 nhwc 输入的情况
Comment on lines
113
to
114
| slice_layer->setInput(2, *size); | ||
| slice_layer->setMode(nvinfer1::SliceMode::kFILL); |
AndSonder
requested review from
zhangjun
and removed request for
xiaoxiaohehe001
zhangjun
reviewed
Mar 8, 2023
| def sample_program_configs(self): | ||
| def generate_input1(attrs): | ||
| T = attrs[0]["seg_num"] | ||
| return np.ones([3 * T, 10, 64, 64]).astype(np.float32) |
Comment on lines
91
to
101
| self.trt_param.precision = paddle_infer.PrecisionType.Float32 | ||
| yield self.create_inference_config(), (0, 3), 1e-5 | ||
| self.trt_param.precision = paddle_infer.PrecisionType.Half | ||
| yield self.create_inference_config(), (0, 3), 1e-3 | ||
|
|
||
| # for dynamic_shape | ||
| generate_dynamic_shape(attrs) | ||
| self.trt_param.precision = paddle_infer.PrecisionType.Float32 | ||
| yield self.create_inference_config(), (0, 3), 1e-5 | ||
| self.trt_param.precision = paddle_infer.PrecisionType.Half | ||
| yield self.create_inference_config(), (0, 3), 1e-3 |
There was a problem hiding this comment.
这里写法有问题，0， 3分别表示trt支持op数量与非trt op数量 参考其他单测增加版本判断
There was a problem hiding this comment.
其他值会报错，check 的时候提示 动态和静态 需要的值是一样的
There was a problem hiding this comment.
这种写法表示没有跑trt， 参考Paddle/python/paddle/fluid/tests/unittests/ir/inference/test_trt_convert_cast.py或者test_trt_convert_gelu.py里面的写法
There was a problem hiding this comment.
已修正，红框中的内容为 converter 中打印的信息，说明代码有运行到trt的部分
zhangjun
reviewed
Mar 12, 2023
Comment on lines
28
to
43
| compile_version = paddle_infer.get_trt_compile_version() | ||
| runtime_version = paddle_infer.get_trt_runtime_version() | ||
| if ( | ||
| compile_version[0] * 1000 | ||
| + compile_version[1] * 100 | ||
| + compile_version[2] * 10 | ||
| < 8200 | ||
| ): | ||
| return False | ||
| if ( | ||
| runtime_version[0] * 1000 | ||
| + runtime_version[1] * 100 | ||
| + runtime_version[2] * 10 | ||
| < 8200 | ||
| ): | ||
| return False |
There was a problem hiding this comment.
generate_trt_nodes_num 有对应限制，这里就可以去掉
zhangjun
reviewed
Mar 13, 2023
Comment on lines
17
to
25
| namespace paddle { | ||
| namespace framework { | ||
| class Scope; | ||
|
|
||
| namespace proto { | ||
| class OpDesc; | ||
| } // namespace proto | ||
| } // namespace framework | ||
| } // namespace paddle |
Comment on lines
1
to
2
| /* Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved. | ||
|
|
| bool test_mode) override { | ||
| #if IS_TRT_VERSION_GE(8200) | ||
|
|
||
| VLOG(3) << "convert a fluid temporal shift op to tensorrt temporal layer"; |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
Others
Describe
为 Paddle-TRT 添加 temporal_shift 算子，代码思路如下所示（参考Paddle文档）：
单测代码中添加 seg_num, shift_ratio 不同情况的测试，单测代码通过，本地单测结果如下图所示：