Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SD3 transformer部分的优化 #713

Merged
merged 71 commits into from
Sep 24, 2024
Merged
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
a6631e7
optimize SD3
chang-wenbin Aug 19, 2024
b0ea9ef
optimize SD3 transformer_SD3
chang-wenbin Aug 19, 2024
f06a61a
optimize SD3 transformer_SD3
chang-wenbin Aug 19, 2024
dcff90c
update SD3
chang-wenbin Aug 20, 2024
15c5e44
uodate triton &sim_SD3
chang-wenbin Aug 20, 2024
ab73a63
modify temb_silu && modify nvtx
chang-wenbin Aug 20, 2024
ed2b7b1
modify linear from fused_linear
chang-wenbin Aug 20, 2024
f4330d3
modify simplified_sd3
chang-wenbin Aug 20, 2024
cc1af0f
add split_concat triton kernel
chang-wenbin Aug 20, 2024
70e6b6e
modify split_concat triton kernel
chang-wenbin Aug 21, 2024
9543b11
update
chang-wenbin Aug 21, 2024
357b75a
update transformer_sd3
chang-wenbin Aug 21, 2024
f54bf84
update transformer_sd3
chang-wenbin Aug 21, 2024
3245b2f
update triton & simplified_sd3
chang-wenbin Aug 21, 2024
5516df6
update simplified_sd3
chang-wenbin Aug 22, 2024
874d5d7
update simplified_sd3
chang-wenbin Aug 22, 2024
111f4cd
delete context_pre_only=False
chang-wenbin Aug 22, 2024
18777b6
modify triton_optimize
chang-wenbin Aug 22, 2024
7a288e4
modify triton_optimize
chang-wenbin Aug 22, 2024
840b153
modify triton_optimize
chang-wenbin Aug 22, 2024
95c9e47
modify triton_fuse & Modifying performance issues affected by CUDA sy…
chang-wenbin Aug 22, 2024
84a9e7a
modify transformer_sd3 if optimize_prigin
chang-wenbin Aug 23, 2024
9dd918d
update vae triton_split
chang-wenbin Aug 23, 2024
3a0b7e1
vae T5 d2s & transformer forward d2s
chang-wenbin Aug 26, 2024
6d02d79
update demo
chang-wenbin Aug 26, 2024
5d81b44
update five model d2s
chang-wenbin Aug 26, 2024
4bab118
update SD3 clip T5 vae
chang-wenbin Aug 27, 2024
5a14a0f
update clip
chang-wenbin Aug 27, 2024
cd2ef01
uodate T5
chang-wenbin Aug 27, 2024
624168c
uodate T5
chang-wenbin Aug 27, 2024
b009b9f
update scheduling_flow_match_euler_discrete
chang-wenbin Aug 27, 2024
8caa10a
update normalization
chang-wenbin Aug 28, 2024
377629a
update normalization
chang-wenbin Aug 28, 2024
6863054
Merge remote-tracking branch 'upstream/develop' into SD3_PaddleMIX_819
chang-wenbin Aug 28, 2024
15fda4e
update SD3
chang-wenbin Aug 29, 2024
cb993c5
merge develop
chang-wenbin Aug 30, 2024
0e90eaf
update cutlass gemm&fast_gelu
chang-wenbin Sep 2, 2024
c5bb81f
update per-mmdit
chang-wenbin Sep 4, 2024
2c8cc85
merge develop
chang-wenbin Sep 4, 2024
499752a
update triton op split_concat
chang-wenbin Sep 4, 2024
1084f4a
update embeddings
chang-wenbin Sep 5, 2024
e3a5d7c
merge
chang-wenbin Sep 6, 2024
fa84559
recovery
chang-wenbin Sep 6, 2024
27c62f9
recovery
chang-wenbin Sep 6, 2024
951f7a6
merge
chang-wenbin Sep 6, 2024
9515323
update normalization
chang-wenbin Sep 10, 2024
d61e4cb
update dtype
chang-wenbin Sep 10, 2024
d961a4a
add SD3 doc
chang-wenbin Sep 10, 2024
ac1e139
merge develop
chang-wenbin Sep 18, 2024
48c66a6
update SD3 doc
chang-wenbin Sep 18, 2024
24c3c9e
add 'del transformer_blocks'
chang-wenbin Sep 19, 2024
422f33b
update SD3
chang-wenbin Sep 19, 2024
c43d84f
update SD3
chang-wenbin Sep 19, 2024
9d03624
update Notes
chang-wenbin Sep 19, 2024
ded06bf
add Notes
chang-wenbin Sep 19, 2024
d845da2
update demo
chang-wenbin Sep 19, 2024
db6aad1
update doc
chang-wenbin Sep 19, 2024
33f37ae
first commit
zhoutianzi666 Sep 19, 2024
74e0419
first commit
zhoutianzi666 Sep 19, 2024
c036878
commit
zhoutianzi666 Sep 19, 2024
d879bd8
commit
zhoutianzi666 Sep 19, 2024
e4367eb
commit
zhoutianzi666 Sep 19, 2024
8423701
commit
zhoutianzi666 Sep 19, 2024
b73f899
commit
zhoutianzi666 Sep 19, 2024
841e739
commit
zhoutianzi666 Sep 19, 2024
fd2c0d9
commit
zhoutianzi666 Sep 19, 2024
bf70669
commit
zhoutianzi666 Sep 19, 2024
7d8064c
commit
zhoutianzi666 Sep 19, 2024
e2e2879
remove inference_optimize_origin
zhoutianzi666 Sep 20, 2024
2aba4ec
commit
zhoutianzi666 Sep 20, 2024
b291a7e
commit
zhoutianzi666 Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
modify transformer_sd3 if optimize_prigin
chang-wenbin committed Aug 23, 2024

Verified

This commit was signed with the committer’s verified signature.
0x2142 Matt Schmitz
commit 84a9e7a3c02f3b8886edbfdc83a04a37dcba23f9
8 changes: 6 additions & 2 deletions ppdiffusers/ppdiffusers/models/transformer_sd3.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# Copyright (c) 2024 PaddlePaddle Authors. All Rights Reserved.
# Copyright 2023 The HuggingFace Team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
@@ -344,9 +343,14 @@ def forward(
encoder_hidden_states = self.context_embedder(encoder_hidden_states)

if self.inference_optimize:
hidden_states = self.simplified_sd3(
out = self.simplified_sd3(
hidden_states=hidden_states, encoder_hidden_states=encoder_hidden_states, temb=temb
)
# this is for paddle inference.
if isinstance(out, paddle.Tensor):
hidden_states = out
else:
hidden_states = out[1]
encoder_hidden_states = None

elif self.inference_optimize_origin: