Skip to content

Latest commit

 

History

History
executable file
·
64 lines (45 loc) · 3.66 KB

README.md

File metadata and controls

executable file
·
64 lines (45 loc) · 3.66 KB
Raw

English | 简体中文

FastDeploy Diffusion Model High-Performance Deployment

This document completes the high-performance deployment of the Diffusion model with ⚡️FastDeploy, based on DiffusionPipeline in project Diffusers designed by Huggingface.

Preperation for Deployment

This example needs the deployment model after exporting the training model. Here are two ways to obtain the deployment model:

  • Methods for model export. Please refer to Model Export to export deployment model.
  • Download the deployment model. To facilitate developers to test the example, we have pre-exported some of the Diffusion models, so you can just download models and test them quickly:
Model Scheduler
CompVis/stable-diffusion-v1-4 PNDM
runwayml/stable-diffusion-v1-5 EulerAncestral

Environment Dependency

In the example, the word splitter in CLIP model of PaddleNLP is required, so you need to run the following line to install the dependency.

pip install paddlenlp paddlepaddle-gpu

Quick Experience

We are ready to start testing after model deployment. Here we will specify the model directory as well as the inference engine backend, and run the infer.py script to complete the inference.

python infer.py --model_dir stable-diffusion-v1-4/ --scheduler "pndm" --backend paddle

The image file is fd_astronaut_rides_horse.png. An example of the generated image is as follows (the generated image is different each time, the example is for reference only):

fd_astronaut_rides_horse.png

If the stable-diffusion-v1-5 model is used, you can run these to complete the inference.

# Inference on GPU
python infer.py --model_dir stable-diffusion-v1-5/ --scheduler "euler_ancestral" --backend paddle

# Inference on KunlunXin XPU
python infer.py --model_dir stable-diffusion-v1-5/ --scheduler "euler_ancestral" --backend paddle-kunlunxin

Parameters

infer.py supports more command line parameters than the above example. The following is a description of each command line parameter.

Parameter Description
--model_dir Directory of the exported model.
--model_format Model format. Default is 'paddle', optional list: ['paddle', 'onnx'].
--backend Inference engine backend. Default ispaddle, optional list: ['onnx_runtime', 'paddle', 'paddle-kunlunxin'], when the model format is onnx, optional list is['onnx_runtime'].
--scheduler Scheduler in StableDiffusion model. Default is'pndm', optional list ['pndm', 'euler_ancestral']. The scheduler corresponding to the StableDiffusio model can be found in ppdiffuser model list.
--unet_model_prefix UNet model prefix, default is unet.
--vae_model_prefix VAE model prefix, defalut is vae_decoder.
--text_encoder_model_prefix TextEncoder model prefix, default is text_encoder.
--inference_steps Running times of UNet model, default is 100.
--image_path Path to the generated images, defalut is fd_astronaut_rides_horse.png.
--device_id gpu id. If device_id is -1, cpu is used for inference.
--use_fp16 Indicates if fp16 is used, default is False. Can be set to True when using tensorrt or paddle-tensorrt backend.