Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build and use GPU-enabled FFmpeg in doc CI #3045

Closed
wants to merge 1 commit into from

Conversation

mthrok
Copy link
Collaborator

@mthrok mthrok commented Feb 8, 2023

This commit add the step to build FFmpeg with GPU decoder in build_doc job so that we can use GPU decoder/encoder in documentations.

@mthrok mthrok force-pushed the ffmpeg-gpu-in-ci branch 5 times, most recently from ca59f4c to 9e08117 Compare February 13, 2023 01:35
@mthrok mthrok changed the title test Build and use GPU-enabled FFmpeg in doc CI Mar 1, 2023
@pytorch-bot
Copy link

pytorch-bot bot commented May 6, 2023

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/audio/3045

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit bce5345:

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch pytorch deleted a comment from netlify bot May 6, 2023
@mthrok mthrok force-pushed the ffmpeg-gpu-in-ci branch 2 times, most recently from e820d55 to 56a2a3e Compare May 9, 2023 21:16
@mthrok mthrok closed this May 11, 2023
@mthrok mthrok reopened this May 11, 2023
mthrok added a commit to pytorch/test-infra that referenced this pull request May 11, 2023
This commit set Nvidia GPU driver capabilities to `all`.

Currently, attempting to use GPU video decoder/encoder with `linux_job`
fails
due to the corresponding driver libraries not exposed to Docker
container (though they exist in host)

Example failure:
https://github.com/pytorch/audio/actions/runs/4899270270/jobs/8749113374#step:10:2022

This is because, GPU is enabled with `--gpus all` flag, which set
default drive capability to `utility` and `compute`.
To run GPU video decoder/encoder, `video` needs to be added or we can
specify `all`.


https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/user-guide.html#driver-capabilities

According to
NVIDIA/nvidia-container-runtime#112 (comment),
nowadays it should be fine to set `all`.

This change is being tested on
pytorch/audio#3045, the job is still failing for
different reason, but
In
https://github.com/pytorch/audio/actions/runs/4930570795/jobs/8811634409#step:10:1339,
the failure mentioned above is no longer happening.
@mthrok mthrok force-pushed the ffmpeg-gpu-in-ci branch 4 times, most recently from 94e3495 to 8993920 Compare May 17, 2023 21:36
@mthrok mthrok marked this pull request as ready for review May 17, 2023 21:50
@facebook-github-bot
Copy link
Contributor

@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@mthrok merged this pull request in 0db5ab2.

@github-actions
Copy link

Hey @mthrok.
You merged this PR, but labels were not properly added. Please add a primary and secondary label (See https://github.com/pytorch/audio/blob/main/.github/process_commit.py).


Some guidance:

Use 'module: ops' for operations under 'torchaudio/{transforms, functional}', and ML-related components under 'torchaudio/csrc' (e.g. RNN-T loss).

Things in "examples" directory:

  • 'recipe' is applicable to training recipes under the 'examples' folder,
  • 'tutorial' is applicable to tutorials under the “examples/tutorials” folder
  • 'example' is applicable to everything else (e.g. C++ examples)
  • 'module: docs' is applicable to code documentations (not to tutorials).
    Regarding examples in code documentations, please also use 'module: docs'.

Please use 'other' tag only when you’re sure the changes are not much relevant to users, or when all other tags are not applicable. Try not to use it often, in order to minimize efforts required when we prepare release notes.


When preparing release notes, please make sure 'documentation' and 'tutorials' occur as the last sub-categories under each primary category like 'new feature', 'improvements' or 'prototype'.

Things related to build are by default excluded from the release note, except when it impacts users. For example:
* Drop support of Python 3.7.
* Add support of Python 3.X.
* Change the way a third party library is bound (so that user needs to install it separately).

@mthrok mthrok deleted the ffmpeg-gpu-in-ci branch May 19, 2023 14:50
mthrok added a commit to mthrok/audio that referenced this pull request May 19, 2023
Follow-up pytorch#3045
- Revert the removal of HW acceleration doc
- comment out FFmpeg CLI test run
@mthrok mthrok mentioned this pull request May 19, 2023
facebook-github-bot pushed a commit that referenced this pull request May 24, 2023
Summary:
Follow-up #3045
- Revert the removal of HW acceleration doc
- comment out FFmpeg CLI test run

Pull Request resolved: #3349

Reviewed By: nateanl

Differential Revision: D46121899

Pulled By: mthrok

fbshipit-source-id: dfc030a69f05addec73637cfb6a720c184e37323
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants