[Wav2vec2 + LM Test] Improve wav2vec2 with lm tests and make torch version dependent for now #18749
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…to improve_wav2vec2_with_lm_tests
|
It fails consistently since PT 1.12. |
|
The documentation is not available anymore as the PR was closed or merged. |
|
Note after some more debugging the reason seems to be in ds = load_dataset("common_voice", "en", split="train", streaming=True)
ds = ds.cast_column("audio", datasets.Audio(sampling_rate=16_000))
ds_iter = iter(ds)
sample = next(ds_iter)
print(sample["audio"]["array"])yields different results between |
patrickvonplaten
changed the title
tests/models/wav2vec2_with_lm/test_processor_wav2vec2_with_lm.py
Outdated
Show resolved
Hide resolved
tests/models/wav2vec2_with_lm/test_processor_wav2vec2_with_lm.py
Outdated
Show resolved
Hide resolved
|
@ydshieh good for a second review |
patrickvonplaten
changed the title
ydshieh
approved these changes
Aug 24, 2022
There was a problem hiding this comment.
Thank you @patrickvonplaten . LGTM, but probably a small change to make it better.
|
|
||
| # TODO(Patrick): This if-else version statement should be removed once | ||
| # https://github.com/huggingface/datasets/issues/4889 is resolved | ||
| if version.parse(torch.__version__) >= version.parse("1.12.0"): |
There was a problem hiding this comment.
Maybe let's use what has been done in
#18460
?
There was a problem hiding this comment.
Yes definitely - thanks!
tests/models/wav2vec2_with_lm/test_processor_wav2vec2_with_lm.py
Outdated
Show resolved
Hide resolved
oneraghavan
pushed a commit
to oneraghavan/transformers
that referenced
this pull request
Sep 26, 2022
…rsion dependent for now (huggingface#18749) * add first generation tutorial * remove generation * make version dependent expected values * Apply suggestions from code review * Update tests/models/wav2vec2_with_lm/test_processor_wav2vec2_with_lm.py * fix typo
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Trying to correct potentially flaky test:
The test actually always passed for me locally.
I've now changed list to tensor as list comparison with float numbers seems brittle. @ydshieh - just to better understand, is this test consistently failing or was it flaky?