Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simpler and faster resampling #1087

Merged
merged 8 commits into from
Jan 19, 2021
Merged

Simpler and faster resampling #1087

merged 8 commits into from
Jan 19, 2021

Conversation

adefossez
Copy link
Contributor

This is the implementation for #1057
Implementation is taken from julius/resample.py and adapted for consistency with the previous implementation. See the issues linked above for a comparison in speed.
Output should be exactly the same as I changed the default parameters to match those of the previous implementation.

Copy link
Collaborator

@mthrok mthrok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. I made some comments.



def resample_waveform(waveform: Tensor,
orig_freq: float,
new_freq: float,
lowpass_filter_width: int = 6) -> Tensor:
lowpass_filter_width: int = 6,
rolloff: float = 0.99) -> Tensor:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my understanding, rolloff was the constant 0.99 in the original implementation.
If so, why not keep it that way? What's the necessity to export it as public API?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking it could be useful to power users (rolloff being always a trade off between limiting aliasing while not losing too much signal, and there is not perfect value). It is not so different from allowing the end user to change lowpass_filter_width, and it does not break backward compatibility. But if you prefer I can remove it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have renamed the parameter to lowpass_cutoff_ratio for clarity, if you agree on keeping it in the api :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have somewhat mixing feeling about adding the new parameter. But I am not DSP expert so I cannot judge. If you go strong on adding it, I do not have a strong counter argument other than keeping API same. However, the way you described, I was thinking it could be useful to power users sounds it's still the assumption, it would be nice if we had some evidence.

Is there a library that expose similar parameter? I know that lowpass_filter_width is originated from Kaldi.

torchaudio/compliance/kaldi.py Outdated Show resolved Hide resolved
@adefossez adefossez requested a review from mthrok December 15, 2020 13:07
@mthrok
Copy link
Collaborator

mthrok commented Dec 15, 2020

Looking at the CI jobs, the tests which were feeding float sampling rate are now failing.

Can I suggest to split this PR into the three PRs?

  1. The speed up part
  2. Changing sampling rate from float to int
  3. Adding parameter for lowpass_cutoff_ratio

Then, the speed up part should be merged easily, and for the others I can bring them up in our team meeting to get more feedback.

@adefossez
Copy link
Contributor Author

I'm wondering whether I should keep the assertion that the sample rates are indeed integers (even if typed as float). People might be using the current implementation with true floats, and in that case my code would throw an exception, while the previous one would not, and the change would not be backward compatible.

In the original code, a mixture of integer and float values was used. I don't think this is completely accurate as for instance resampling from 2 to 2.9999 is going to be quite different from resampling from 2 to 3. I could try to emulate this in the new implementation but I need a bit more time to make sure I am doing the same thing as the previous implementation.

I would need to add some form of unit tests because non integer change of sample rate are not covered in the tests. I could add a saved file from the current implementation as a test resource, but I'm afraid this would scare future contributors from trying to change that bit, while in fact the current implementation is doing something a bit weird and somewhat incorrect.

@vincentqb
Copy link
Contributor

vincentqb commented Dec 17, 2020

I'm wondering whether I should keep the assertion that the sample rates are indeed integers (even if typed as float). People might be using the current implementation with true floats, and in that case my code would throw an exception, while the previous one would not, and the change would not be backward compatible.

In the original code, a mixture of integer and float values was used. I don't think this is completely accurate as for instance resampling from 2 to 2.9999 is going to be quite different from resampling from 2 to 3. I could try to emulate this in the new implementation but I need a bit more time to make sure I am doing the same thing as the previous implementation.

I would need to add some form of unit tests because non integer change of sample rate are not covered in the tests. I could add a saved file from the current implementation as a test resource, but I'm afraid this would scare future contributors from trying to change that bit, while in fact the current implementation is doing something a bit weird and somewhat incorrect.

As mentioned in #891, we do wish to change sample rate to floats in torchaudio. Parts of the code that assume integer sample rate should check that condition to avoid silent correctness issue later.

I second @mthrok suggestion above to split out the change from int to float (and the new parameter).

@adefossez
Copy link
Contributor Author

@vincentqb for some reason I thought the old code was doing something weird if the sample rate were floats, which I was worried about, but actually it was equivalent to just converting those with int first, which is what the new code does too.
So I've removes here everything change but the change to the algorithm itself, and this is functionally equivalent to the previous implementation.

@adefossez
Copy link
Contributor Author

Ping @vincentqb @mthrok :) let me know if you are good with the current version, it is completely equivalent to the previous one.

Copy link
Contributor

@vincentqb vincentqb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, thanks!

torchaudio/compliance/kaldi.py Show resolved Hide resolved
@mthrok
Copy link
Collaborator

mthrok commented Jan 14, 2021

@adefossez Sorry for not getting back to you sooner. Let me come back to this the next week.

@mthrok
Copy link
Collaborator

mthrok commented Jan 19, 2021

@adefossez

Sorry for not getting back to you sooner. The PR looks good. Thanks!

@mthrok mthrok merged commit e43a8e7 into pytorch:master Jan 19, 2021
mthrok pushed a commit to mthrok/audio that referenced this pull request Feb 26, 2021
* Add TorchScript fork/join tutorial

* Add note about zipfile format in serialization tutorial

* Profiler recipe (pytorch#1019)

* Profiler recipe

Summary:
Adding a recipe for profiler

Test Plan:
make html-noplot

* [mobile] Mobile Perf Recipe

* Minor syntax edits to mobile perf recipe

* Remove built files

* [android] android native app recipe

* [mobile_perf][recipe] Add ChannelsLast recommendation

* Adding distributed pipeline parallel tutorial

* Add async execution tutorials

* Fix code block in pipeline tutorial

* Adding an Overview Page for PyTorch Distributed (pytorch#1056)

* Adding an Overview Page for PyTorch Distributed

* Let existing PT Distributed tutorials link to the overview page

* Add a link to AMP

* Address Comments

* Remove unnecessary dist.barrier()

* [Mobile Perf Recipe] Add the benchmarking part for iOS (pytorch#1055)

* [Mobile Perf Recipe] Add the benchmarking part for iOS

* [Mobile Perf Recipe] Add the benchmarking part for iOS

Co-authored-by: Jessica Lin <jplin@fb.com>

* RPC profiling recipe (pytorch#1068)

* Initial commit

* Update

* Complete most of recipe

* Add image

* Link image

* Remove extra file

* update

* Update

* update

* Push latest changes from master into release/1.6 (pytorch#1074)

* Update feature classification labels

* Update NVidia -> Nvidia

* Bring back default filename_pattern so that by default we run all galleries.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Add prototype_source directory

* Add prototype directory

* Add prototype

* Remove extra "done"

* Add REAME.txt

* Update for prototype instructions

* Update for prototype feature

* refine torchvision_tutorial doc for windows

* Update neural_style_tutorial.py (pytorch#1059)

Updated the mistake in the Loading Images Section.

* torch_script_custom_ops restructure (pytorch#1057)

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Port custom ops tutorial to new registration API, increase testability.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Kill some other occurrences of RegisterOperators

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Update README.md

* Make torch_script_custom_classes tutorial runnable

I also fixed some warnings in the tutorial, and fixed some minor bitrot
(e.g., torch::script::Module to torch::jit::Module)

I also added some missing quotes around some bash expansions.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Update torch_script_custom_classes to use TORCH_LIBRARY (pytorch#1062)

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Co-authored-by: Edward Z. Yang <ezyang@fb.com>
Co-authored-by: Yang Gu <yangu@microsoft.com>
Co-authored-by: Hritik Bhandari <bhandari.hritik@gmail.com>

* Tutorial for DDP + RPC (pytorch#1071)

* Update feature classification labels

* Update NVidia -> Nvidia

* Bring back default filename_pattern so that by default we run all galleries.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Tutorial for DDP + RPC.

Summary: Based on example from pytorch/examples#800

* Add to main section

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Added separate code file and used literalinclude

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Co-authored-by: Jessica Lin <jplin@fb.com>
Co-authored-by: Edward Z. Yang <ezyang@fb.com>
Co-authored-by: pritam <pritam.damania@fb.com>

* Make RPC profiling recipe into prototype tutorial (pytorch#1078)

* Add RPC tutorial

* Update to include recipes

* Add Graph Mode Dynamic Quant tutorial (pytorch#1065)

* Update feature classification labels

* Update NVidia -> Nvidia

* Bring back default filename_pattern so that by default we run all galleries.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Add prototype_source directory

* Add prototype directory

* Add prototype

* Remove extra "done"

* Add REAME.txt

* Update for prototype instructions

* Update for prototype feature

* refine torchvision_tutorial doc for windows

* Update neural_style_tutorial.py (pytorch#1059)

Updated the mistake in the Loading Images Section.

* torch_script_custom_ops restructure (pytorch#1057)

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Port custom ops tutorial to new registration API, increase testability.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Kill some other occurrences of RegisterOperators

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Update README.md

* Make torch_script_custom_classes tutorial runnable

I also fixed some warnings in the tutorial, and fixed some minor bitrot
(e.g., torch::script::Module to torch::jit::Module)

I also added some missing quotes around some bash expansions.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Update torch_script_custom_classes to use TORCH_LIBRARY (pytorch#1062)

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Add Graph Mode Dynamic Quant tutorial

Summary:
Tutorial to demonstrate graph mode dynamic quant on BERT model.
Currently not directly runnable as it requires to download glue dataset and fine-tuned model

Co-authored-by: Jessica Lin <jplin@fb.com>
Co-authored-by: Edward Z. Yang <ezyang@fb.com>
Co-authored-by: Yang Gu <yangu@microsoft.com>
Co-authored-by: Hritik Bhandari <bhandari.hritik@gmail.com>

* Add mobile recipes images

* Update mobile recipe index

* Remove RPC Profiling recipe from index

* 1.6 model freezing tutorial (pytorch#1077)

* Update feature classification labels

* Update NVidia -> Nvidia

* Bring back default filename_pattern so that by default we run all galleries.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Add prototype_source directory

* Add prototype directory

* Add prototype

* Remove extra "done"

* Add REAME.txt

* Update for prototype instructions

* Update for prototype feature

* refine torchvision_tutorial doc for windows

* Update neural_style_tutorial.py (pytorch#1059)

Updated the mistake in the Loading Images Section.

* torch_script_custom_ops restructure (pytorch#1057)

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Port custom ops tutorial to new registration API, increase testability.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Kill some other occurrences of RegisterOperators

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Update README.md

* Make torch_script_custom_classes tutorial runnable

I also fixed some warnings in the tutorial, and fixed some minor bitrot
(e.g., torch::script::Module to torch::jit::Module)

I also added some missing quotes around some bash expansions.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Update torch_script_custom_classes to use TORCH_LIBRARY (pytorch#1062)

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Add Model Freezing in TorchScript

Co-authored-by: Edward Z. Yang <ezyang@fb.com>
Co-authored-by: Yang Gu <yangu@microsoft.com>
Co-authored-by: Hritik Bhandari <bhandari.hritik@gmail.com>

* Update title

* Update recipes_index.rst

Touch for rebuild.

* Update dcgan_faces_tutorial.py

Update labels to be floats to work around torch.full inference change.

Co-authored-by: James Reed <jamesreed@fb.com>
Co-authored-by: ilia-cher <30845429+ilia-cher@users.noreply.github.com>
Co-authored-by: Ivan Kobzarev <ivankobzarev@fb.com>
Co-authored-by: Shen Li <shenli@devfair017.maas>
Co-authored-by: Shen Li <cs.shenli@gmail.com>
Co-authored-by: Tao Xu <taox@fb.com>
Co-authored-by: Rohan Varma <rvarm1@fb.com>
Co-authored-by: Edward Z. Yang <ezyang@fb.com>
Co-authored-by: Yang Gu <yangu@microsoft.com>
Co-authored-by: Hritik Bhandari <bhandari.hritik@gmail.com>
Co-authored-by: Pritam Damania <9958665+pritamdamania87@users.noreply.github.com>
Co-authored-by: pritam <pritam.damania@fb.com>
Co-authored-by: supriyar <supriyar@fb.com>
Co-authored-by: Brian Johnson <brianjo@fb.com>
Co-authored-by: gchanan <gchanan@fb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants