Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support generating with fallback for short form audio in Whisper #30984

Merged
merged 72 commits into from
Jul 19, 2024
Merged

Support generating with fallback for short form audio in Whisper #30984

Changes from 18 commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
f752535
remove is_shortform
kamilakesbi May 22, 2024
d58d3fa
adapt _retrieve_max_frames_and_seek for short_form
kamilakesbi May 22, 2024
9eda3eb
return bos token in short and long form
kamilakesbi May 22, 2024
7a97483
add decoder_input_ids to short form audios
kamilakesbi May 23, 2024
ba36c8e
add eos token for short form
kamilakesbi May 23, 2024
d347633
handle short form token_timestamps
kamilakesbi May 23, 2024
7eddab3
no need to return scores
kamilakesbi May 23, 2024
07e7db3
add is_shortform conditions
kamilakesbi May 24, 2024
fe1da29
handle when max_new_tokens is None - short form
kamilakesbi May 24, 2024
24769d7
handle assistant decoding
kamilakesbi May 24, 2024
54ad952
fix
kamilakesbi May 24, 2024
1534c8e
handle return_dict_in_generate
kamilakesbi May 27, 2024
b37372e
handle split_by_batch for encoder_attentions attribute
kamilakesbi May 27, 2024
f58af54
handle num_beams>1
kamilakesbi May 27, 2024
9125277
handle num_return_sequences>1 in generate_with_fallback
kamilakesbi May 27, 2024
78252b1
handle num_return_sequences>1 with return_dict_in_generate=True
kamilakesbi May 27, 2024
0d0c720
raise error if max_new_tokens + decoder_inputs_ids > max_target_pos
kamilakesbi May 27, 2024
a47e9ba
fix
kamilakesbi May 28, 2024
78c3842
apply review suggestions
kamilakesbi May 29, 2024
779b741
fix
kamilakesbi May 29, 2024
2cc9813
Update src/transformers/models/whisper/generation_whisper.py
kamilakesbi May 29, 2024
51abdcb
Update src/transformers/models/whisper/generation_whisper.py
kamilakesbi May 29, 2024
accf48c
Update src/transformers/models/whisper/generation_whisper.py
kamilakesbi May 30, 2024
49faeb9
fix
kamilakesbi May 31, 2024
6c2d76d
logits for both short form and long form
kamilakesbi May 31, 2024
e2cd613
handle if logits_processor is None
kamilakesbi May 31, 2024
0ba7611
test
kamilakesbi May 31, 2024
146905f
apply review changes to num_return_sequences
kamilakesbi May 31, 2024
a10675f
add _expand_variables_for_generation
kamilakesbi May 31, 2024
fa8ddd7
remove short form commented section
kamilakesbi May 31, 2024
98884ec
update comments
kamilakesbi May 31, 2024
1082914
uncomment num_beams line in generate_with_fallback
kamilakesbi May 31, 2024
ae11682
update assistant decoding
kamilakesbi May 31, 2024
3f3ee08
handle return_segment with short form generation
kamilakesbi May 31, 2024
12465a3
up
kamilakesbi Jun 3, 2024
bff1397
fix output format is_shortform
kamilakesbi Jun 3, 2024
38aa42d
overwrite beam_sample test
kamilakesbi Jun 3, 2024
579d77d
update _set_return_timestamps
kamilakesbi Jun 5, 2024
91adcdf
apply review suggestions
kamilakesbi Jun 5, 2024
4b2266e
apply review suggestions
kamilakesbi Jun 5, 2024
38c1162
remove seek_outputs_short_form
kamilakesbi Jun 6, 2024
5625d66
fix _stack_split_outputs
kamilakesbi Jun 6, 2024
6f1bcdb
fix stack dim in _stack_split_outputs
kamilakesbi Jun 6, 2024
116974f
update tests
kamilakesbi Jun 6, 2024
63c6475
fix past_key_values + beam tests
kamilakesbi Jun 6, 2024
92927d1
fix
kamilakesbi Jun 6, 2024
3f5f173
clean _expand_variables_for_generation
kamilakesbi Jun 6, 2024
91a6f39
make style
kamilakesbi Jun 6, 2024
db2c289
fix slow tests
kamilakesbi Jun 6, 2024
967591e
make style
kamilakesbi Jun 6, 2024
0e323a9
Merge branch 'main' into fallback_short_form
kamilakesbi Jun 6, 2024
b8a5f74
max_length condition
kamilakesbi Jun 7, 2024
0f26442
make style
kamilakesbi Jun 7, 2024
d64b1f4
add slow tests for shortform fallback
kamilakesbi Jun 7, 2024
265a5d2
Update src/transformers/models/whisper/generation_whisper.py
kamilakesbi Jun 11, 2024
d6d4474
Update src/transformers/models/whisper/generation_whisper.py
kamilakesbi Jun 11, 2024
88e91a8
apply review changes
kamilakesbi Jun 11, 2024
65ff1dd
Update src/transformers/models/whisper/generation_whisper.py
kamilakesbi Jun 11, 2024
a18c6ce
up
kamilakesbi Jun 11, 2024
80c5bc6
fix slow tests
kamilakesbi Jun 11, 2024
51070fc
apply review suggestions
kamilakesbi Jul 16, 2024
73a9805
update test
kamilakesbi Jul 17, 2024
2e36400
Merge branch 'main' into fallback_short_form
kamilakesbi Jul 18, 2024
a96d600
make style
kamilakesbi Jul 18, 2024
299de5d
small fix
kamilakesbi Jul 18, 2024
17b9d47
fix
kamilakesbi Jul 18, 2024
3ef601e
fix test_new_cache_format
kamilakesbi Jul 18, 2024
d1cc8d3
fix past_key_values
kamilakesbi Jul 18, 2024
ed8cc34
fix
kamilakesbi Jul 18, 2024
6b7b3d6
make style
kamilakesbi Jul 18, 2024
b503c14
fix slow tests
kamilakesbi Jul 18, 2024
cb4201e
fix
kamilakesbi Jul 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading