Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support generating with fallback for short form audio in Whisper #30984

Merged
merged 72 commits into from
Jul 19, 2024
Merged

Support generating with fallback for short form audio in Whisper #30984

merged 72 commits into from
Jul 19, 2024

Commits on May 24, 2024

  1. remove is_shortform

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    f752535 View commit details
    Browse the repository at this point in the history

  2. adapt _retrieve_max_frames_and_seek for short_form

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    d58d3fa View commit details
    Browse the repository at this point in the history

  3. return bos token in short and long form

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    9eda3eb View commit details
    Browse the repository at this point in the history

  4. add decoder_input_ids to short form audios

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    7a97483 View commit details
    Browse the repository at this point in the history

  5. add eos token for short form

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    ba36c8e View commit details
    Browse the repository at this point in the history

  6. handle short form token_timestamps

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    d347633 View commit details
    Browse the repository at this point in the history

  7. no need to return scores

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    7eddab3 View commit details
    Browse the repository at this point in the history

  8. add is_shortform conditions

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    07e7db3 View commit details
    Browse the repository at this point in the history

  9. handle when max_new_tokens is None - short form

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    fe1da29 View commit details
    Browse the repository at this point in the history

  10. handle assistant decoding

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    24769d7 View commit details
    Browse the repository at this point in the history

  11. fix

    @kamilakesbi
    kamilakesbi committed May 24, 2024
    Configuration menu
    Copy the full SHA
    54ad952 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. handle return_dict_in_generate

    @kamilakesbi
    kamilakesbi committed May 27, 2024
    Configuration menu
    Copy the full SHA
    1534c8e View commit details
    Browse the repository at this point in the history

  2. handle split_by_batch for encoder_attentions attribute

    @kamilakesbi
    kamilakesbi committed May 27, 2024
    Configuration menu
    Copy the full SHA
    b37372e View commit details
    Browse the repository at this point in the history

  3. handle num_beams>1

    @kamilakesbi
    kamilakesbi committed May 27, 2024
    Configuration menu
    Copy the full SHA
    f58af54 View commit details
    Browse the repository at this point in the history

  4. handle num_return_sequences>1 in generate_with_fallback

    @kamilakesbi
    kamilakesbi committed May 27, 2024
    Configuration menu
    Copy the full SHA
    9125277 View commit details
    Browse the repository at this point in the history

  5. handle num_return_sequences>1 with return_dict_in_generate=True

    @kamilakesbi
    kamilakesbi committed May 27, 2024
    Configuration menu
    Copy the full SHA
    78252b1 View commit details
    Browse the repository at this point in the history

  6. raise error if max_new_tokens + decoder_inputs_ids > max_target_pos

    @kamilakesbi
    kamilakesbi committed May 27, 2024
    Configuration menu
    Copy the full SHA
    0d0c720 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. fix

    @kamilakesbi
    kamilakesbi committed May 28, 2024
    Configuration menu
    Copy the full SHA
    a47e9ba View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. apply review suggestions

    @kamilakesbi
    kamilakesbi committed May 29, 2024
    Configuration menu
    Copy the full SHA
    78c3842 View commit details
    Browse the repository at this point in the history

  2. fix

    @kamilakesbi
    kamilakesbi committed May 29, 2024
    Configuration menu
    Copy the full SHA
    779b741 View commit details
    Browse the repository at this point in the history

  3. Update src/transformers/models/whisper/generation_whisper.py 

    Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
    @kamilakesbi @sanchit-gandhi
    kamilakesbi and sanchit-gandhi authored May 29, 2024
    Configuration menu
    Copy the full SHA
    2cc9813 View commit details
    Browse the repository at this point in the history

  4. Update src/transformers/models/whisper/generation_whisper.py 

    Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
    @kamilakesbi @sanchit-gandhi
    kamilakesbi and sanchit-gandhi authored May 29, 2024
    Configuration menu
    Copy the full SHA
    51abdcb View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Update src/transformers/models/whisper/generation_whisper.py 

    Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
    @kamilakesbi @sanchit-gandhi
    kamilakesbi and sanchit-gandhi authored May 30, 2024
    Configuration menu
    Copy the full SHA
    accf48c View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. fix

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    49faeb9 View commit details
    Browse the repository at this point in the history

  2. logits for both short form and long form

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    6c2d76d View commit details
    Browse the repository at this point in the history

  3. handle if logits_processor is None

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    e2cd613 View commit details
    Browse the repository at this point in the history

  4. test

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    0ba7611 View commit details
    Browse the repository at this point in the history

  5. apply review changes to num_return_sequences

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    146905f View commit details
    Browse the repository at this point in the history

  6. add _expand_variables_for_generation

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    a10675f View commit details
    Browse the repository at this point in the history

  7. remove short form commented section

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    fa8ddd7 View commit details
    Browse the repository at this point in the history

  8. update comments

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    98884ec View commit details
    Browse the repository at this point in the history

  9. uncomment num_beams line in generate_with_fallback

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    1082914 View commit details
    Browse the repository at this point in the history

  10. update assistant decoding

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    ae11682 View commit details
    Browse the repository at this point in the history

  11. handle return_segment with short form generation

    @kamilakesbi
    kamilakesbi committed May 31, 2024
    Configuration menu
    Copy the full SHA
    3f3ee08 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. up

    @kamilakesbi
    kamilakesbi committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    12465a3 View commit details
    Browse the repository at this point in the history

  2. fix output format is_shortform

    @kamilakesbi
    kamilakesbi committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    bff1397 View commit details
    Browse the repository at this point in the history

  3. overwrite beam_sample test

    @kamilakesbi
    kamilakesbi committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    38aa42d View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. update _set_return_timestamps

    @kamilakesbi
    kamilakesbi committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    579d77d View commit details
    Browse the repository at this point in the history

  2. apply review suggestions

    @kamilakesbi
    kamilakesbi committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    91adcdf View commit details
    Browse the repository at this point in the history

  3. apply review suggestions

    @kamilakesbi
    kamilakesbi committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    4b2266e View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. remove seek_outputs_short_form

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    38c1162 View commit details
    Browse the repository at this point in the history

  2. fix _stack_split_outputs

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    5625d66 View commit details
    Browse the repository at this point in the history

  3. fix stack dim in _stack_split_outputs

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    6f1bcdb View commit details
    Browse the repository at this point in the history

  4. update tests

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    116974f View commit details
    Browse the repository at this point in the history

  5. fix past_key_values + beam tests

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    63c6475 View commit details
    Browse the repository at this point in the history

  6. fix

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    92927d1 View commit details
    Browse the repository at this point in the history

  7. clean _expand_variables_for_generation

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    3f5f173 View commit details
    Browse the repository at this point in the history

  8. make style

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    91a6f39 View commit details
    Browse the repository at this point in the history

  9. fix slow tests

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    db2c289 View commit details
    Browse the repository at this point in the history

  10. make style

    @kamilakesbi
    kamilakesbi committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    967591e View commit details
    Browse the repository at this point in the history

  11. Merge branch 'main' into fallback_short_form

    @kamilakesbi
    kamilakesbi authored Jun 6, 2024
    Configuration menu
    Copy the full SHA
    0e323a9 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. max_length condition

    @kamilakesbi
    kamilakesbi committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    b8a5f74 View commit details
    Browse the repository at this point in the history

  2. make style

    @kamilakesbi
    kamilakesbi committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    0f26442 View commit details
    Browse the repository at this point in the history

  3. add slow tests for shortform fallback

    @kamilakesbi
    kamilakesbi committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    d64b1f4 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Update src/transformers/models/whisper/generation_whisper.py 

    Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
    @kamilakesbi @sanchit-gandhi
    kamilakesbi and sanchit-gandhi authored Jun 11, 2024
    Configuration menu
    Copy the full SHA
    265a5d2 View commit details
    Browse the repository at this point in the history

  2. Update src/transformers/models/whisper/generation_whisper.py 

    Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
    @kamilakesbi @sanchit-gandhi
    kamilakesbi and sanchit-gandhi authored Jun 11, 2024
    Configuration menu
    Copy the full SHA
    d6d4474 View commit details
    Browse the repository at this point in the history

  3. apply review changes

    @kamilakesbi
    kamilakesbi committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    88e91a8 View commit details
    Browse the repository at this point in the history

  4. Update src/transformers/models/whisper/generation_whisper.py 

    Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
    @kamilakesbi @sanchit-gandhi
    kamilakesbi and sanchit-gandhi authored Jun 11, 2024
    Configuration menu
    Copy the full SHA
    65ff1dd View commit details
    Browse the repository at this point in the history

  5. up

    @kamilakesbi
    kamilakesbi committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    a18c6ce View commit details
    Browse the repository at this point in the history

  6. fix slow tests

    @kamilakesbi
    kamilakesbi committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    80c5bc6 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. apply review suggestions

    @kamilakesbi
    kamilakesbi committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    51070fc View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. update test

    @kamilakesbi
    kamilakesbi committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    73a9805 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. Merge branch 'main' into fallback_short_form

    @kamilakesbi
    kamilakesbi authored Jul 18, 2024
    Configuration menu
    Copy the full SHA
    2e36400 View commit details
    Browse the repository at this point in the history

  2. make style

    @kamilakesbi
    kamilakesbi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    a96d600 View commit details
    Browse the repository at this point in the history

  3. small fix

    @kamilakesbi
    kamilakesbi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    299de5d View commit details
    Browse the repository at this point in the history

  4. fix

    @kamilakesbi
    kamilakesbi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    17b9d47 View commit details
    Browse the repository at this point in the history

  5. fix test_new_cache_format

    @kamilakesbi
    kamilakesbi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    3ef601e View commit details
    Browse the repository at this point in the history

  6. fix past_key_values

    @kamilakesbi
    kamilakesbi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    d1cc8d3 View commit details
    Browse the repository at this point in the history

  7. fix

    @kamilakesbi
    kamilakesbi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    ed8cc34 View commit details
    Browse the repository at this point in the history

  8. make style

    @kamilakesbi
    kamilakesbi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    6b7b3d6 View commit details
    Browse the repository at this point in the history

  9. fix slow tests

    @kamilakesbi
    kamilakesbi committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    b503c14 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. fix

    @kamilakesbi
    kamilakesbi committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    cb4201e View commit details
    Browse the repository at this point in the history