hardware accelerated video decoding

[totaam]

Following #147, we should go one step further and offload x264 video decoding to the graphics card using VAAPI / libva

Some pointers:

• vaapi at freedesktop
• intel vaapi support

Why VAAPI? (lwn.net) Because this is the most widely compatible/available API, with glue libraries available for both vdpau (nvidia) and xvba (ati).
If really needed we could also talk to those directly, but that is best avoided.

[totaam]

2012-11-01 09:30:44: ahuillet commented

---

I don't think we should be using VAAPI directly for video decoding acceleration. libav is a pain, but it has a VAAPI backend, and it will probably be better to use it. Rationale: we'll need video decoding acceleration on Windows (possibly even more than on Linux). We'll want to do it through libva as well.

[totaam]

decoding example

[totaam]

For DXVA see #346

[totaam]

From the comments section on VA-API Video Acceleration On The Linux Desktop Is Nearly Ready For Chrome:

• Then there's the ability to output hardware decoded video to an X11 window or a pixmap, Radeon VAAPI does have that, but this is good mainly for direct display (like a dedicated video player), not for integration into a complex UI that is a web page. It's also tied to X, obviously, so not ideal also from this perspective.
• If you want to use the opengl video output with hardware decoding on Radeon, VDPAU is probably the better option.

[totaam]

Now that encoding via vaapi works (#451), can we use the same solution here? (no)

The data we feed into the decoder is relatively small (so no problem uploading it to the GPU), but the decoded data is big (full size pixel frames) and so it's a waste if we have to download it to CPU memory only to re-upload to the GPU for rendering on screen.
It probably negates most of the savings from decoding on the GPU - the server usually selects fast presets which aren't very taxing for a CPU software decoder.
Then there's also the extra decoder complexity (except on macos with h264_vda which is a drop in replacement for h264?) and usual hardware flakiness (lack of hardware for testing, differences in drivers / OS, etc).

Staying on the GPU, as we do encoding side with nvfbc #1317, will require direct use of GPU buffers (dmabuf / pixmaps on Linux, whatever else on other platforms).
This is harder, and will need to be done for each supported platform separately. PITA.

This is looking unlikely for 4.1