Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Hackathon No.235】 Adding support for PaddleSlim quantized model to T… #500

Merged
merged 1 commit into from
Apr 6, 2023
Merged

【Hackathon No.235】 Adding support for PaddleSlim quantized model to T… #500

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions rfcs/TVM/20230405_paddlehackathon4_235_support_quantized_model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# RFC of PaddlePaddle Hackathon 4 Task 235

## Solution name

Adding support for PaddleSlim quantized model to TVM PaddlePaddle front end.

## Description

At present, the PaddlePaddle Frontend in TVM already supports 100+ operators, but it still does not support the PaddleSlim quantization model. This solution will increase TVM's support for the PaddleSlim model and verify the quantified performance improvement.
Specifically, this solution plans to support the following two operators.

- dequantize_linear
- quantize_linear

## Workflow

1. Set up the TVM development environment.
1. For reference, investigate ONNX [quantization operator definition](https://github.com/onnx/onnx/blob/main/docs/Operators.md) and [quantization operator conversion](https://github.com/apache/tvm/blob/main/python/tvm/relay/frontend/onnx.py).
1. In frontend source [paddlepaddle.py](https://github.com/apache/tvm/blob/main/python/tvm/relay/frontend/paddlepaddle.py), implement the conversion of linear_quantize.
1. In the same file, implement the conversion of dequantize_linear.
1. Test with quantitative models.

## Results

TVM can support following PaddleSlim quantization model.

- [resnet50_vd_ptq](https://bj.bcebos.com/paddlehub/fastdeploy/resnet50_vd_ptq.tar)
- [mobilenetv1_ssld_ptq](https://bj.bcebos.com/paddlehub/fastdeploy/mobilenetv1_ssld_ptq.tar)
- [PP_LiteSeg_T_STDC1_cityscapes_without_argmax_infer_QAT_new](https://bj.bcebos.com/paddlehub/fastdeploy/PP_LiteSeg_T_STDC1_cityscapes_without_argmax_infer_QAT_new.tar)

## Project Timeline

2023/4/05 Submitting RFC
2023/4/20 Creating PR
2023/4/30 Merging PR

## Your experience in ML and DL

I'm a PPDE, and I completed three tasks in the first [PaddlePaddle Hackathon](https://github.com/PaddlePaddle/Paddle/issues/35940).
I have experience with model deployment.