Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Quantization] Add quantization support for bitsandbytes #9213

Merged
merged 119 commits into from
Oct 21, 2024
Merged
Changes from 1 commit
Commits
Show all changes
119 commits
Select commit Hold shift + click to select a range
e634ff2
quantization config.
sayakpaul Aug 19, 2024
02a6dff
fix-copies
sayakpaul Aug 19, 2024
c385a2b
Merge branch 'main' into quantization-config
sayakpaul Aug 20, 2024
0355875
Merge branch 'main' into quantization-config
sayakpaul Aug 20, 2024
e41b494
Merge branch 'main' into quantization-config
sayakpaul Aug 20, 2024
dfb33eb
Merge branch 'main' into quantization-config
sayakpaul Aug 21, 2024
e492655
Merge branch 'main' into quantization-config
sayakpaul Aug 22, 2024
6e86cc0
fix
sayakpaul Aug 22, 2024
58a3d15
modules_to_not_convert
sayakpaul Aug 22, 2024
1d477f9
Merge branch 'main' into quantization-config
sayakpaul Aug 22, 2024
bd7f46d
Merge branch 'main' into quantization-config
sayakpaul Aug 23, 2024
d5d7bb6
Merge branch 'main' into quantization-config
sayakpaul Aug 28, 2024
44c8a75
Merge branch 'main' into quantization-config
sayakpaul Aug 28, 2024
6a0fcdc
add bitsandbytes utilities.
sayakpaul Aug 28, 2024
e4590fa
make progress.
sayakpaul Aug 28, 2024
77a1438
Merge branch 'main' into quantization-config
sayakpaul Aug 29, 2024
335ab6b
fixes
sayakpaul Aug 29, 2024
d44ef85
quality
sayakpaul Aug 29, 2024
210fa1e
up
sayakpaul Aug 29, 2024
f4feee1
up
sayakpaul Aug 29, 2024
e8c1722
Merge branch 'main' into quantization-config
sayakpaul Aug 29, 2024
7f86a71
Merge branch 'main' into quantization-config
sayakpaul Aug 29, 2024
ba671b6
minor
sayakpaul Aug 30, 2024
c1a9f13
up
sayakpaul Aug 30, 2024
4489c54
Merge branch 'main' into quantization-config
sayakpaul Aug 30, 2024
f2ca5e2
up
sayakpaul Aug 30, 2024
d6b8954
fix
sayakpaul Aug 30, 2024
45029e2
provide credits where due.
sayakpaul Aug 30, 2024
4eb468a
make configurations work.
sayakpaul Aug 30, 2024
939965d
fixes
sayakpaul Aug 30, 2024
8557166
Merge branch 'main' into quantization-config
sayakpaul Aug 30, 2024
d098d07
fix
sayakpaul Aug 30, 2024
c4a0074
update_missing_keys
sayakpaul Aug 30, 2024
ee45612
fix
sayakpaul Aug 30, 2024
b24c0a7
fix
sayakpaul Aug 31, 2024
473505c
make it work.
sayakpaul Aug 31, 2024
c795c82
fix
sayakpaul Aug 31, 2024
c1d5b96
Merge branch 'main' into quantization-config
sayakpaul Aug 31, 2024
af7caca
provide credits to transformers.
sayakpaul Aug 31, 2024
80967f5
empty commit
sayakpaul Sep 1, 2024
3bdf25a
handle to() better.
sayakpaul Sep 2, 2024
27415cc
tests
sayakpaul Sep 2, 2024
51cac09
change to bnb from bitsandbytes
sayakpaul Sep 2, 2024
15f3032
fix tests
sayakpaul Sep 2, 2024
77c9fdb
better safeguard.
sayakpaul Sep 2, 2024
ddc9f29
change merging status
sayakpaul Sep 2, 2024
44c4109
courtesy to transformers.
sayakpaul Sep 2, 2024
27666a8
move upper.
sayakpaul Sep 2, 2024
3464d83
better
sayakpaul Sep 2, 2024
b106124
Merge branch 'main' into quantization-config
sayakpaul Sep 2, 2024
330fa0a
Merge branch 'main' into quantization-config
sayakpaul Sep 2, 2024
abc8607
make the unused kwargs warning friendlier.
sayakpaul Sep 3, 2024
31725aa
harmonize changes with https://github.com/huggingface/transformers/pu…
sayakpaul Sep 3, 2024
e5938a6
style
sayakpaul Sep 3, 2024
444588f
trainin tests
sayakpaul Sep 3, 2024
d3360ce
Merge branch 'main' into quantization-config
sayakpaul Sep 3, 2024
d8b35f4
Merge branch 'main' into quantization-config
sayakpaul Sep 3, 2024
859f2d7
Merge branch 'main' into quantization-config
sayakpaul Sep 4, 2024
3b2d6e1
feedback part i.
sayakpaul Sep 4, 2024
5799954
Add Flux inpainting and Flux Img2Img (#9135)
Gothos Sep 4, 2024
8e4bd08
Revert "Add Flux inpainting and Flux Img2Img (#9135)"
sayakpaul Sep 6, 2024
835d4ad
tests
sayakpaul Sep 6, 2024
27075fe
don
sayakpaul Sep 6, 2024
5c00c1c
Merge branch 'main' into quantization-config
sayakpaul Sep 6, 2024
5d633a0
Merge branch 'main' into quantization-config
sayakpaul Sep 8, 2024
c381fe0
Apply suggestions from code review
sayakpaul Sep 10, 2024
3c92878
Merge branch 'main' into quantization-config
sayakpaul Sep 10, 2024
acdeb25
contribution guide.
sayakpaul Sep 11, 2024
aa295b7
Merge branch 'main' into quantization-config
sayakpaul Sep 11, 2024
7f7c9ce
Merge branch 'main' into quantization-config
sayakpaul Sep 15, 2024
55f96d8
Merge branch 'main' into quantization-config
sayakpaul Sep 15, 2024
b28cc65
changes
sayakpaul Sep 17, 2024
8328e86
Merge branch 'main' into quantization-config
sayakpaul Sep 17, 2024
9758942
empty
sayakpaul Sep 17, 2024
b1a9878
fix tests
sayakpaul Sep 17, 2024
971305b
harmonize with https://github.com/huggingface/transformers/pull/33546.
sayakpaul Sep 18, 2024
f41adf1
numpy_cosine_distance
sayakpaul Sep 19, 2024
0bcb88b
Merge branch 'main' into quantization-config
sayakpaul Sep 19, 2024
55b3696
Merge branch 'main' into quantization-config
sayakpaul Sep 20, 2024
4cb3a6d
Merge branch 'main' into quantization-config
sayakpaul Sep 23, 2024
8a03eae
Merge branch 'main' into quantization-config
sayakpaul Sep 24, 2024
53f0a92
Merge branch 'main' into quantization-config
sayakpaul Sep 26, 2024
6aab47c
Merge branch 'main' into quantization-config
sayakpaul Sep 27, 2024
9b9a610
resolved conflicts,
sayakpaul Sep 29, 2024
510d57a
Merge branch 'main' into quantization-config
sayakpaul Oct 10, 2024
555a5ae
config_dict modification.
sayakpaul Oct 10, 2024
da10365
remove if config comment.
sayakpaul Oct 10, 2024
71316a6
note for load_state_dict changes.
sayakpaul Oct 10, 2024
12f5c59
float8 check.
sayakpaul Oct 10, 2024
5e722cd
quantizer.
sayakpaul Oct 10, 2024
c78dd0c
raise an error for non-True low_cpu_mem_usage values when using quant.
sayakpaul Oct 10, 2024
af3ecea
low_cpu_mem_usage shenanigans when using fp32 modules.
sayakpaul Oct 10, 2024
a473d28
don't re-assign _pre_quantization_type.
sayakpaul Oct 10, 2024
870d74f
make comments clear.
sayakpaul Oct 10, 2024
3e6cfeb
remove comments.
sayakpaul Oct 10, 2024
673993c
handle mixed types better when moving to cpu.
sayakpaul Oct 10, 2024
0d5f2f7
add tests to check if we're throwing warning rightly.
sayakpaul Oct 10, 2024
3cb20fe
better check.
sayakpaul Oct 10, 2024
10940a9
fix 8bit test_quality.
sayakpaul Oct 10, 2024
c0a88ae
Merge branch 'main' into quantization-config
sayakpaul Oct 10, 2024
dcc5bc5
Merge branch 'main' into quantization-config
sayakpaul Oct 12, 2024
5e0b4eb
Merge branch 'main' into quantization-config
sayakpaul Oct 12, 2024
569dd96
Merge branch 'main' into quantization-config
sayakpaul Oct 13, 2024
8bdc846
Merge branch 'main' into quantization-config
sayakpaul Oct 15, 2024
ff8ddef
handle dtype more robustly.
sayakpaul Oct 15, 2024
de6394a
better message when keep_in_fp32_modules.
sayakpaul Oct 15, 2024
81bb48a
handle dtype casting.
sayakpaul Oct 15, 2024
c5e62ae
Merge branch 'main' into quantization-config
sayakpaul Oct 15, 2024
d023b40
Merge branch 'main' into quantization-config
sayakpaul Oct 16, 2024
a3d2655
Merge branch 'main' into quantization-config
sayakpaul Oct 16, 2024
700b0f3
Merge branch 'main' into quantization-config
sayakpaul Oct 18, 2024
0ae70fe
fix dtype checks in pipeline.
sayakpaul Oct 18, 2024
ecdf1d0
fix warning message.
sayakpaul Oct 18, 2024
aea3398
Update src/diffusers/models/modeling_utils.py
sayakpaul Oct 18, 2024
3a91974
Merge branch 'main' into quantization-config
sayakpaul Oct 18, 2024
5d8e844
Merge branch 'main' into quantization-config
sayakpaul Oct 19, 2024
501a6ba
mitigate the confusing cpu warning
sayakpaul Oct 19, 2024
1a931cb
Merge branch 'main' into quantization-config
sayakpaul Oct 21, 2024
2fa8fb9
Merge branch 'main' into quantization-config
sayakpaul Oct 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix tests
sayakpaul committed Sep 17, 2024

Verified

This commit was signed with the committer’s verified signature.
snyk-bot Snyk bot
commit b1a98787962cc6792d96bed8ff1463589ccc7d1b
316 changes: 0 additions & 316 deletions bnb_fixes.patch

This file was deleted.

4 changes: 2 additions & 2 deletions src/diffusers/quantizers/bitsandbytes/bnb_quantizer.py
Original file line number Diff line number Diff line change
@@ -270,7 +270,7 @@ def _process_model_before_weight_loading(
# Unlike `transformers`, we don't know if we should always keep certain modules in FP32
# in case of diffusion transformer models. For language models and others alike, `lm_head`
# and tied modules are usually kept in FP32.
self.modules_to_not_convert = list(filter(None.__ne__, self.modules_to_not_convert))
self.modules_to_not_convert = [module for module in self.modules_to_not_convert if module is not None]

model = replace_with_bnb_linear(
model, modules_to_not_convert=self.modules_to_not_convert, quantization_config=self.quantization_config
@@ -521,7 +521,7 @@ def _process_model_before_weight_loading(
# Unlike `transformers`, we don't know if we should always keep certain modules in FP32
# in case of diffusion transformer models. For language models and others alike, `lm_head`
# and tied modules are usually kept in FP32.
self.modules_to_not_convert = list(filter(None.__ne__, self.modules_to_not_convert))
self.modules_to_not_convert = [module for module in self.modules_to_not_convert if module is not None]

model = replace_with_bnb_linear(
model, modules_to_not_convert=self.modules_to_not_convert, quantization_config=self.quantization_config
Loading