-
Notifications
You must be signed in to change notification settings - Fork 66
Description
🚀 The feature
It would be great if torchcodec dynamically loaded the nvcuvid
library instead of directly linking it, or at least have an option to dynamically load the nvcuvid
library instead of directly link it.
Motivation, pitch
First of all, thanks a lot for the great work on the new NVCUVID
backend (#910 and following PRs). I looked into that as part of the 0.8.0, and I noticed that currently we directly link the nvcuvid
library in
${NVCUVID_LIBRARY} |
This specific choice works great in most cases, but in some corner cases can be problematic (that problably only emerge in conda-forge packaging and hence why I stumbled on them), in particular:
- On some Linux NVIDIA system without
NVDEC
silicon (such as Tegra/Jetson Orin), thelibnvcuvid.so
library is not installed as part of the system driver. So, that always meant that torchcodec's GPU capabilities were not working, but at least before 0.8.0 it was possible to use a torchcodec CUDA build to actually doCPU
decoding, aslibnvcuvid.so
library was not required unless ffmpeg was actually requested to use_cuvid
decoder. After 0.8.0 instead, for a torchcodec CUDA build, it is required forlibnvcuvid.so
to be present in the system, otherwise thelibtorchcodec_core7.so
can't be loaded at all, preventing the use of torchcodec even just for CPU usage. - On Windows to actually link the
nvcuvid.dll
library, you also need thenvcuvid.lib
import library, that is not installed by the NVIDIA driver asnvcuvid.dll
, but it installed separately by the "Video Codec for application developers", that needs to be manually downloaded from https://developer.nvidia.com/nvidia-video-codec-sdk/download by accepting the " NVIDIA Software Developer License Agreement". This is different from Linux, there thelibnvcuvid.so
is the one installed in the system, but that can also be used when linking the library.
How does other projects like ffmpeg or dali deal with this? Instead of directly linking to nvcuvid
library, they dynamically load nvcuvid
, gracefully raising an error if the library can't be loaded, see:
- ffmpeg
- https://github.com/FFmpeg/FFmpeg/blob/20051ed3af4c8c355b8749d7b59593fc9a56bf80/libavcodec/nvdec.c#L211
- https://github.com/FFmpeg/nv-codec-headers/blob/876af32a202d0de83bd1d36fe74ee0f7fcf86b0d/include/ffnvcodec/dynlink_loader.h#L434 (interestingly, all the code under https://github.com/FFmpeg/nv-codec-headers is MIT licensed and so more easily reusable w.r.t. to ffmpeg itself that is LGPL)
- dali:
If there is interested for something like that I probably can also contribute in that direction, even if I can't provide a definite timeframe for this.