Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

webui support controlnet extension #948

Merged
merged 23 commits into from
Jun 22, 2024
Merged

webui support controlnet extension #948

merged 23 commits into from
Jun 22, 2024

Conversation

marigoold
Copy link
Contributor

only supported UNet inference with control, but controlnet model not supported now

@lijunliangTG
Copy link
Contributor

lijunliangTG commented Jun 19, 2024

分辨率在512*512的情况下
base模型为:AWPainting_v1.2.safetensors

  • 仅使用ControlNet Unit0
Control Type OneDiff 速度 原生速度 原图 output_img OneDiff_img
Canny 2.662 2.767s stock_mountain a_large_avalanchecannyFalse a_large_avalanchecannyTrue
Depth 2.535 2.545 s sk-b-dep a_large_avalancheDepthFalse a_large_avalancheDepthTrue
Segmentation 2.793 s 2.771 s sk-b-src a_large_avalancheSegmentationFalse a_large_avalancheSegmentationTrue
OpenPose 2.503 s 2.597 s an-pose a_large_avalancheopenposeTrue a_large_avalancheopenposeFalse
  • 同时使用ControlNet Unit0 和ControlNet Unit1
步骤 图片
先使用Canny生成图片,再同时用Canny+Openpose 生成图片,可以正常出图 image (3)
接着关闭Canny,然后仅开启Openpose 可以正常出图,可以正常出图 image (4)
接着关闭Openpose ,然后仅开启Canny可以正常出图,可以正常出图 image (5)
  • 同时使用ControlNet Unit0 和 ControlNet Unit1和ControlNet Unit2
步骤 图片
同时用Depth+Openpose+ Canny 生成图片,可以正常出图 image (9)
关闭Canny,然后仅开启Depth+Openpose可以正常出图 image (8)
关闭Openpose ,然后仅开启Depth+ Canny可以正常出图,可以正常出图 image (7)
接着关闭Canny ,然后仅开启Canny可以正常出图,可以正常出图 image (10)

@lijunliangTG
Copy link
Contributor

lijunliangTG commented Jun 19, 2024

分辨率在1024*1024的情况下
base模型为:AWPainting_v1.2.safetensors

  • 仅使用ControlNet Unit0
Control Type OneDiff 速度 原生速度 原图 output_img OneDiff_img
Canny 17.575 s 20.900s stock_mountain a_large_avalanchecontrol_v11p_sd15_cannyTrue a_large_avalanchecontrol_v11p_sd15_cannyFalse
Depth 16.420 s 19.198 s sk-b-dep a_large_avalanchecontrol_v11f1p_sd15_depthFalse a_large_avalanchecontrol_v11f1p_sd15_depthTrue
Segmentation 16.560 s 21.455 s sk-b-src a_large_avalanchecontrol_sd15_segFalse a_large_avalanchecontrol_sd15_segTrue
OpenPose 16.484 s 21.781 s an-pose a_large_avalanchecontrol_sd15_openposeFalse a_large_avalanchecontrol_sd15_openposeTrue

@lijunliangTG
Copy link
Contributor

lijunliangTG commented Jun 21, 2024

  • 分辨率在1024*1024的情况下
    base模型为:v1-5-pruned-emaonly.safetensors
    仅使用ControlNet Unit0
Control Type OneDiff 速度 原生速度 原图 output_img OneDiff_img
Canny 2.898 s 6.256s stock_mountain a_large_avalanchecontrol_v11p_sd15_cannyFalse a_large_avalanchecontrol_v11p_sd15_cannyTrue
Depth 2.645 s 6.198 s sk-b-dep a_large_avalanchecontrol_v11f1p_sd15_depthFalse a_large_avalanchecontrol_v11f1p_sd15_depthTrue
Segmentation 2.972 s 9.657 s sk-b-src a_large_avalanchecontrol_sd15_segFalse a_large_avalanchecontrol_sd15_segTrue
OpenPose 2.741 s 7.826 s an-pose a_large_avalanchecontrol_sd15_openposeFalse a_large_avalanchecontrol_sd15_openposeTrue
  • 分辨率在1024*1024的情况下
    base模型为:sd_xl_base_1.0.safetensors
    X代表没有找到相应的Control模型
  • 仅使用ControlNet Unit0
Control Type OneDiff 速度 原生速度 原图 output_img OneDiff_img
Canny X X
Depth X X
Segmentation X X
OpenPose X X

@lijunliangTG
Copy link
Contributor

1、直接用sd_xl_base_1.0.safetensors+onediff+controlnet-canny-sdx不能正常运行,报错如sdxl_onediff_error.txt
sdxl_onediff_error.txt

2、先使用sd_xl_base_1.0.safetensors+onediff,再sd_xl_base_1.0.safetensors+onediff+controlnet-canny-sdx,可以正常运行

sd_xl_base_1.0.safetensors+onediff sd_xl_base_1.0.safetensors+onediff+controlnet-canny-sdxl
image (14) image (15)

@lijunliangTG lijunliangTG self-requested a review June 21, 2024 06:24
@marigoold marigoold merged commit 102ee05 into main Jun 22, 2024
7 checks passed
@marigoold marigoold deleted the dev_wy_webui_controlnet branch June 22, 2024 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants