Trame/Async-ParaView/VTKStreaming H264 Encoding "OpenGL rendering is not on a CUDA device"

[jcorbettfrank]

Hi, I'm able to run the async paraview wavelets example: https://gitlab.kitware.com/async/async-paraview-examples/-/blob/main/async/python/wavelet.py.

However when I try to use H.264 encoding I see:

3: (   4.084s) [main thread     ]     vtkNvEncoderGL.cxx:145    ERR| CUDA driver API cuGLGetDevices_v2(&cudaDeviceCount, devices, 4, CU_GL_DEVICE_LIST_ALL) failed with error: CUDA_ERROR_UNKNOWN
3: (   4.084s) [main thread     ]     vtkNvEncoderGL.cxx:149    ERR| OpenGL rendering is not on a CUDA device.
3: (   4.084s) [main thread     ]    vtkVideoEncoder.cxx:277    ERR| Failed to initialize encoding context.
3: (   4.084s) [main thread     ]    vtkVideoEncoder.cxx:236    ERR| Failed to update encoder context for 480x480

I'm able to reproduce with VTKStreaming's tests (which is where I took the output from above; it's the same error though when running wavelets.py.

Running NvEncodeCxx-TestNvEncoderGLMapResource on two machines I see the above error.

Machine 1 (my dev laptop):

Thu Aug  3 16:46:43 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.105.01   Driver Version: 515.105.01   CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro RTX 3000     On   | 00000000:01:00.0  On |                  N/A |
| N/A   45C    P8    10W /  N/A |    210MiB /  6144MiB |     37%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1567      G   /usr/lib/xorg/Xorg                 30MiB |
|    0   N/A  N/A      3927      G   /usr/lib/xorg/Xorg                175MiB |
+-----------------------------------------------------------------------------+

Machine 2 (heavy server):

Thu Aug  3 16:48:33 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.86.01    Driver Version: 515.86.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro RTX 6000     Off  | 00000000:21:00.0 Off |                    0 |
| N/A   26C    P8    22W / 250W |   1028MiB / 23040MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Quadro RTX 6000     Off  | 00000000:81:00.0 Off |                    0 |
| N/A   26C    P8    23W / 250W |    554MiB / 23040MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

Looking at https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new, I see both GPUs are listed as supported:

I appreciate any help! If I am better off posting directly to VTKStreaming please let me know.

[jourdain]

You probably need to do ENV NVIDIA_DRIVER_CAPABILITIES=all within your docker.

[jcorbettfrank]

Hi @jspanchu, thanks for your response.
Here's the output; it definitely is different than yours. Posting here while I google/troubleshoot...

frank-jobot@cavali:~$ glxinfo -B
name of display: :1
display: :1  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
   Vendor: Intel (0x8086)
   Device: Mesa Intel(R) UHD Graphics (CML GT2) (0x9bc4)
   Version: 21.2.6
   Accelerated: yes
   Video memory: 3072MB
   Unified memory: yes
   Preferred profile: core (0x1)
   Max core profile version: 4.6
   Max compat profile version: 4.6
   Max GLES1 profile version: 1.1
   Max GLES[23] profile version: 3.2
OpenGL vendor string: Intel
OpenGL renderer string: Mesa Intel(R) UHD Graphics (CML GT2)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 21.2.6
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.6 (Compatibility Profile) Mesa 21.2.6
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.2 Mesa 21.2.6
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

[jspanchu]

This is a big problem. OpenGL is rendering on Intel GPU. You might need to use the correct display environment variable.

[jcorbettfrank]

Thanks, I will troubleshoot and post back when I eventually fix it

[jcorbettfrank]

To fix this I:

• uninstalled and re-installed nvidia drivers
• selected the latest non-"open package" nvidia driver from Ubuntu Software & Updates
• used nvidia-prime to set nvidia as the default

Now all VTKStreaming tests pass and I'm able to run the wavelets.py example with H.264 encoding.

Funnily enough, the JPG encoder feels much more responsive, even if it uses twice the bandwidth. Of course I'm running locally, so need to consider that...We are currently evaluating whether to incorporate the H.264 encoding into our application (vtkWebApplication [pushRender, and all that] + Angular + wslink) or not.

Thank you for your help @jspanchu and @jourdain!

[jspanchu]

Glad you fixed it!

Your reports of the responsiveness are unfortunately correct. The server side encoding with VTKStreaming is very rapid. However, the client-side decoding in the browser is not so rapid on some systems. I've seen it works best on Microsoft Edge. For client-side decoding, trame relies upon WebCodecs API. It is still being actively implemented in all browsers and firefox doesn't yet support it. So we can hope it improves after the WebCodecs implementations have stabilized!

[jcorbettfrank]

Awesome, thank you again!