Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use dlopen for sox #3082

Closed
wants to merge 3 commits into from
Closed

Use dlopen for sox #3082

wants to merge 3 commits into from

Conversation

mthrok
Copy link
Collaborator

@mthrok mthrok commented Feb 21, 2023

This commit changes the way torchaudio integrates libsox.
Previously, libsox has been statically linked and shipped.
This commit changes it so that libsox will be dl-opened at runtime.

Users now need to install libsox separately. Conda users can do so by
conda install -c conda-forge sox libopus. Ubuntu users can use apt install libsox-fmt-all

The notable difference is that the supported formats are different from
one distribution to another, thus some formats will not be available unless users build libsox
with these formats enabled.

If this change causes an unrecoverable issue, one can use TORCHAUDIO_USE_SOX=0 to disable
libsox integration at runtime.

@mthrok mthrok force-pushed the dynamic-link-sox branch 4 times, most recently from f68dd6b to 2e330b5 Compare April 6, 2023 00:29
@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@mthrok mthrok force-pushed the dynamic-link-sox branch from 2e330b5 to 4ef769a Compare May 10, 2023 00:59
@pytorch-bot
Copy link

pytorch-bot bot commented May 10, 2023

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/audio/3082

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 3 Unrelated Failures

As of commit 88c3f02:

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base da21202:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@mthrok mthrok force-pushed the dynamic-link-sox branch 2 times, most recently from ba9a8ef to df19aa6 Compare May 10, 2023 20:33
@mthrok mthrok force-pushed the dynamic-link-sox branch 3 times, most recently from 1954488 to 87b376e Compare May 23, 2023 22:37
@mthrok mthrok force-pushed the dynamic-link-sox branch 5 times, most recently from 077fd6f to f46b7c5 Compare May 29, 2023 01:22
@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@mthrok mthrok force-pushed the dynamic-link-sox branch 10 times, most recently from 3b89487 to dae2b57 Compare June 2, 2023 00:47
@mthrok mthrok force-pushed the dynamic-link-sox branch 2 times, most recently from 92bda56 to bb14df9 Compare June 9, 2023 17:04
@mthrok mthrok removed the with-ssh label Jun 13, 2023
@mthrok mthrok force-pushed the dynamic-link-sox branch 2 times, most recently from 1f90dc2 to 2988fe9 Compare July 7, 2023 17:52
@mthrok mthrok force-pushed the dynamic-link-sox branch from 2988fe9 to b78f418 Compare July 25, 2023 02:10
mthrok added a commit to mthrok/audio that referenced this pull request Jul 25, 2023
In preparation for pytorch#3082

Disable those FFmpeg tests that depend on sox CLI. These tests need to be updated or removed so as not to use sox CLI.

Auto-skip some sox tests if decoder/encoder are not available
@mthrok mthrok force-pushed the dynamic-link-sox branch from aabd0d6 to b701c96 Compare July 25, 2023 15:26
mthrok added a commit to mthrok/audio that referenced this pull request Jul 25, 2023
In preparation for pytorch#3082

Disable those FFmpeg tests that depend on sox CLI. These tests need to be updated or removed so as not to use sox CLI.

Auto-skip some sox tests if decoder/encoder are not available
@mthrok mthrok force-pushed the dynamic-link-sox branch from b701c96 to 67b04ab Compare July 25, 2023 17:59
mthrok added a commit to mthrok/audio that referenced this pull request Jul 25, 2023
In preparation for pytorch#3082

Disable those FFmpeg tests that depend on sox CLI. These tests need to be updated or removed so as not to use sox CLI.

Auto-skip some sox tests if decoder/encoder are not available
mthrok added a commit to mthrok/audio that referenced this pull request Jul 25, 2023
In preparation for pytorch#3082

Disable those FFmpeg tests that depend on sox CLI. These tests need to be updated or removed so as not to use sox CLI.

Auto-skip some sox tests if decoder/encoder are not available
@mthrok mthrok force-pushed the dynamic-link-sox branch from 67b04ab to 7530973 Compare July 25, 2023 18:01
mthrok added a commit to mthrok/audio that referenced this pull request Jul 25, 2023
In preparation for pytorch#3082

Disable those FFmpeg tests that depend on sox CLI. These tests need to be updated or removed so as not to use sox CLI.

Auto-skip some sox tests if decoder/encoder are not available
@mthrok mthrok force-pushed the dynamic-link-sox branch from 7530973 to c412aa9 Compare July 25, 2023 19:07
mthrok added a commit to mthrok/audio that referenced this pull request Jul 25, 2023
In preparation for pytorch#3082

Disable those FFmpeg tests that depend on sox CLI. These tests need to be updated or removed so as not to use sox CLI.

Auto-skip some sox tests if decoder/encoder are not available
@mthrok mthrok force-pushed the dynamic-link-sox branch from c412aa9 to 9a78725 Compare July 25, 2023 19:52
facebook-github-bot pushed a commit that referenced this pull request Jul 25, 2023
Summary:
In preparation for #3082

Disable those FFmpeg tests that depend on sox CLI. These tests need to be updated or removed so as not to use sox CLI.

Auto-skip some sox tests if decoder/encoder are not available

Pull Request resolved: #3494

Differential Revision: D47761948

Pulled By: mthrok

fbshipit-source-id: 3a48d7f280f8376a48d223947dd41a7cdc8cbc30
@mthrok mthrok force-pushed the dynamic-link-sox branch 3 times, most recently from 1bcff9f to 52b18f1 Compare July 26, 2023 19:10
Since libsox and ffmpeg extensions now depend on external libraries,
their initialization processes might cause unrecoverable issue, such as
segfault.

This commit adds environment variable to disable them so that importing
torchaudio won't attempt to load these libraries.
@mthrok mthrok force-pushed the dynamic-link-sox branch from 52b18f1 to 4acc213 Compare July 26, 2023 19:42
@mthrok mthrok force-pushed the dynamic-link-sox branch from 4acc213 to 88c3f02 Compare July 26, 2023 19:46
@mthrok
Copy link
Collaborator Author

mthrok commented Jul 27, 2023

superseded by #3497, which is simpler

@mthrok mthrok closed this Jul 27, 2023
@mthrok mthrok deleted the dynamic-link-sox branch July 27, 2023 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants