Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEFT Integration for Text Encoder to handle multiple alphas/ranks, disable/enable adapters and support for multiple adapters #5147

Merged
merged 63 commits into from
Sep 27, 2023
Merged
Changes from 1 commit
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
ba24f2a
more fixes
younesbelkada Sep 14, 2023
c17634c
up
younesbelkada Sep 15, 2023
2a6e535
up
younesbelkada Sep 15, 2023
01f6d1d
style
younesbelkada Sep 15, 2023
5a150b2
add in setup
younesbelkada Sep 15, 2023
961e776
oops
younesbelkada Sep 15, 2023
cdbe739
more changes
younesbelkada Sep 15, 2023
691368b
v1 rzfactor CI
younesbelkada Sep 18, 2023
7918851
Apply suggestions from code review
younesbelkada Sep 18, 2023
14db139
few todos
younesbelkada Sep 18, 2023
c06c40b
Merge branch 'main' into peftpart-1
younesbelkada Sep 18, 2023
d56a14d
protect torch import
younesbelkada Sep 18, 2023
ec87c19
style
younesbelkada Sep 18, 2023
40a6028
fix fuse text encoder
younesbelkada Sep 18, 2023
0c62ef3
Merge remote-tracking branch 'upstream/main' into peftpart-1
younesbelkada Sep 18, 2023
c4295c9
Update src/diffusers/loaders.py
younesbelkada Sep 19, 2023
4162ddf
replace with `recurse_replace_peft_layers`
younesbelkada Sep 19, 2023
1d13f40
keep old modules for BC
younesbelkada Sep 19, 2023
78a860d
adjustments on `adjust_lora_scale_text_encoder`
younesbelkada Sep 19, 2023
78a01d5
Merge branch 'main' into peftpart-1
younesbelkada Sep 19, 2023
ecbc714
Merge remote-tracking branch 'upstream/main' into peftpart-1
younesbelkada Sep 19, 2023
9d650c9
Merge branch 'peftpart-1' of https://github.com/younesbelkada/diffuse…
younesbelkada Sep 19, 2023
6f1adcd
nit
younesbelkada Sep 19, 2023
f890906
move tests
younesbelkada Sep 19, 2023
f8e87f6
add conversion utils
younesbelkada Sep 19, 2023
3ba2d4e
Merge remote-tracking branch 'upstream/main' into peftpart-1
younesbelkada Sep 19, 2023
dc83fa0
remove unneeded methods
younesbelkada Sep 19, 2023
b83fcba
use class method instead
younesbelkada Sep 19, 2023
74e33a9
oops
younesbelkada Sep 19, 2023
9cb8563
use `base_version`
younesbelkada Sep 19, 2023
c90f85d
fix examples
younesbelkada Sep 19, 2023
40a4894
fix CI
younesbelkada Sep 19, 2023
ea05959
fix weird error with python 3.8
younesbelkada Sep 19, 2023
27e3da6
fix
younesbelkada Sep 19, 2023
3d7c567
better fix
younesbelkada Sep 19, 2023
d01a292
style
younesbelkada Sep 19, 2023
e836b14
Apply suggestions from code review
younesbelkada Sep 20, 2023
cb48405
Apply suggestions from code review
younesbelkada Sep 20, 2023
325462d
add comment
younesbelkada Sep 20, 2023
b412adc
Apply suggestions from code review
younesbelkada Sep 20, 2023
b72ef23
conv2d support for recurse remove
younesbelkada Sep 20, 2023
e072655
added docstrings
younesbelkada Sep 20, 2023
bd46ae9
more docstring
younesbelkada Sep 20, 2023
724b52b
add deprecate
younesbelkada Sep 20, 2023
5e6f343
revert
younesbelkada Sep 20, 2023
71650d4
try to fix merge conflicts
younesbelkada Sep 20, 2023
920333f
Merge remote-tracking branch 'upstream/main' into peftpart-1
younesbelkada Sep 20, 2023
0985d17
peft integration features for text encoder
pacman100 Sep 22, 2023
ece3b02
Merge branch 'main' into peftpart-1
pacman100 Sep 22, 2023
01a15cc
fix bug
pacman100 Sep 22, 2023
080db75
Merge branch 'main' into smangrul/peft-integration
pacman100 Sep 22, 2023
ffbac30
fix code quality
pacman100 Sep 22, 2023
916c31a
Apply suggestions from code review
pacman100 Sep 25, 2023
5de0f1b
fix bugs
pacman100 Sep 25, 2023
c32872e
Merge branch 'smangrul/peft-integration' of https://github.com/huggin…
pacman100 Sep 25, 2023
0acb58c
Apply suggestions from code review
pacman100 Sep 25, 2023
1ca4c62
address comments
pacman100 Sep 26, 2023
7c37788
fix code quality
pacman100 Sep 26, 2023
2fcf174
address comments
pacman100 Sep 26, 2023
a1f0128
address comments
pacman100 Sep 26, 2023
7b2ccff
Merge branch 'main' into smangrul/peft-integration
patrickvonplaten Sep 27, 2023
fd9bcfe
Apply suggestions from code review
patrickvonplaten Sep 27, 2023
9916ac6
find and replace
patrickvonplaten Sep 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix CI
younesbelkada committed Sep 19, 2023
commit 40a489457d0c3e5c86bdbccbbe152c4626a3317c
4 changes: 2 additions & 2 deletions src/diffusers/loaders.py
Original file line number Diff line number Diff line change
@@ -1547,7 +1547,7 @@ def load_lora_into_text_encoder(
rank = {}
text_encoder_lora_state_dict = convert_state_dict_to_diffusers(text_encoder_lora_state_dict)

if cls.use_peft_backend:
if cls.use_peft_backend is True:
# convert state dict
text_encoder_lora_state_dict = convert_state_dict_to_peft(text_encoder_lora_state_dict)

@@ -1583,7 +1583,7 @@ def load_lora_into_text_encoder(
k.replace(f"{prefix}.", ""): v for k, v in network_alphas.items() if k in alpha_keys
}

if cls.use_peft_backend:
if cls.use_peft_backend is True:
from peft import LoraConfig

lora_rank = list(rank.values())[0]