Skip to content

Gstreamer Video4Linux2 plugin patches

Jump to bottom
fduncanh edited this page Jan 25, 2024 · 378 revisions

Instructions for Raspberry Pi users about a patch for the GStreamer Video4Linux2 plugin that may be needed for hardware video decoding by the Broadcom GPU on the Pi. (For GStreamer < 1.22)

  • Since UxPlay 1.63, software video decoding (using -avdec option) also works quite well on Raspberry Pi when the -vsync option (now the default since UxPlay 1.64) is also used, so while using hardware decoding is still preferred, it is no longer absolutely essential.

  • Raspberry Pi OS (Bullseye) now supplies a patched plugin (but the patch here is more complete); Ubuntu 21.10, 22.04 LTS, 22.10 need patching. Latest Ubuntu 23.04 and Manjaro provide GStreamer-1.22, and do not need a patch.

  • IMPORTANT, kernel module bcm2835_codec.ko MUST be present for hardware video decoding. (Raspberry Pi OS, Ubuntu and Manjaro all have it, but if you are using a different distribution, you should check.) Look for it with "find /lib/modules -name bcm2835_codec.ko". There is no point in patching if the bcm2835_codec module is missing.

The kernel module bcm2835_codec is maintained by Raspberry Pi at https://github.com/raspberrypi/linux/tree/rpi-5.15.y/drivers/staging/vc04_services and is not in the mainline Linux kernel. It merely accesses the Broadcom firmware, which does the actual video decoding. Only Raspberry Pi OS and distributions available with Raspberry Pi Imager (Ubuntu, Manjaro) appear to have it. If you want to use this module, and your distribution does not supply it, you need to build and use a kernel with the Raspberry Pi kernel source: instructions for this are beyond the scope of this guide.

2022-07-29 patches (backports for GStreamer 1.18.4 and later, for use with Raspberry Pi's Broadcom GPU) updated:

v4l2_1_xx_x.diff3.txt now supplies a new GStreamer patch where Apple's color settings are recognized without using the "bt709" workaround (from GStreamer development branch). (Apple Airplay Mirror protocol uses an uncommon "full-range" variant of bt709 color standard for HDTV Broadcasts that was not previously recognized by the Video4Linux2 plugin.)

  • do NOT use UxPlay option -bt709 with these new patches.

  • GStreamer-1.22 used in Ubuntu-23.04 and current Manjaro does not need any patches. (Some GStreamer error messages (due to a minor bug in the Broadcom firmware accessed by the bcm2835 module) which were suppressed by the patch will be shown by GStreamer-1.22 and can be ignored)

v4l2_1_18_4_diff3.txt for Raspberry Pi OS Bullseye

wget https://github.com/FDH2/UxPlay/files/9196641/v4l2_1_18_4_diff3.txt

v4l2_1_18_5_diff3.txt for Ubuntu 21.10

wget https://github.com/FDH2/UxPlay/files/9196643/v4l2_1_18_5_diff3.txt

v4l2_1_18_6_diff3.txt

wget https://github.com/FDH2/UxPlay/files/9196647/v4l2_1_18_6_diff3.txt

v4l2_1_20_0_diff3.txt

wget https://github.com/FDH2/UxPlay/files/9196648/v4l2_1_20_0_diff3.txt

v4l2_1_20_1_diff3.txt for Ubuntu 22.04

wget https://github.com/FDH2/UxPlay/files/9196657/v4l2_1_20_1_diff3.txt

v4l2_1_20_2_diff3.txt

wget https://github.com/FDH2/UxPlay/files/9196660/v4l2_1_20_2_diff3.txt

v4l2_1_20_3_diff3.txt for Ubuntu 22.04.1 and 22.10.

wget https://github.com/FDH2/UxPlay/files/9196663/v4l2_1_20_3_diff3.txt

v4l2_1_20_4_diff3.txt

wget https://github.com/FDH2/UxPlay/files/9212951/v4l2_1_20_4_diff3.txt

v4l2_1_20_5_diff3.txt will also work with 1.20.6.

wget https://github.com/FDH2/UxPlay/files/10298147/v4l2_1_20_5_diff3.txt

Patching instructions for GStreamer 1.18.4 and later (backport from GStreamer 1.22, will not be needed for GStreamer-1.22 or later): tested on R Pi model 4B, also confirmed to work on model 3B+.

  • An alternative to patching is to completely replace your GStreamer with the latest development version built from source: see here However, it is much easier to just patch your distribution's GStreamer package.

  • Recipes for patching The GStreamer Video4Linux2 plugin from gstreamer1.0-plugins-good are given here for (1) GStreamer-1.18.4 as used on Debian Bullseye-based systems such as Rapberry Pi OS (2) GStreamer-1.18.5 systems such as Ubuntu 21.20, and (3) GStreamer-1.20.1 systems such as Ubuntu-22.04 (4) GStreamer-1.20.3 systems such as Ubuntu-22.04-1, as well as (5) the Arch-based GStreamer-1.20.4 system Manjaro-Arm-RPi4-22.10. (These are for use until your distribution provides these updates.)

  • Note: the patched version of libgstvideo4linux2.so that you will install here will get overwritten by re-installation or installation of updates of the gstreamer1.0-plugins-good package.

(Raspberry Pi 4 only) First check that your Raspberry Pi's firmware is up to date

The Video4Linux2 plugin works by using a kernel module to access the firmware for the Broadcom GPU which is written in the Pi's EEPROM chip. To check that the firmware is up to date, run

sudo rpi-eeprom-update

It this shows updates are available, run "sudo rpi-eeprom-update -a", and then reboot to install them.
After the reboot, run "sudo rpi-eeprom-update" again to verify that the firmware is now up to date.

Next, verify that you have a complete Video4linux2 plugin.

Use gst-inspect-1.0 video4linux2 (In Debian-based systems, this requires that gstreamer1.0-tools is installed):

pi@raspberrypi:~ $ gst-inspect-1.0 video4linux2
Plugin Details:
  Name                     video4linux2
  Description              elements for Video 4 Linux
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
  Version                  1.18.4
  License                  LGPL
  Source module            gst-plugins-good
  Source release date      2021-03-15
  Binary package           GStreamer Good Plug-ins source release
  Origin URL               Unknown package origin

  v4l2src: Video (video4linux2) Source
  v4l2sink: Video (video4linux2) Sink
  v4l2radio: Radio (video4linux2) Tuner
  v4l2deviceprovider: Video (video4linux2) Device Provider
  v4l2jpegdec: V4L2 JPEG Decoder
  v4l2h264dec: V4L2 H264 Decoder
  v4l2h264enc: V4L2 H.264 Encoder
  v4l2convert: V4L2 Video Converter
  v4l2video18convert: V4L2 Video Converter
  v4l2jpegenc: V4L2 JPEG Encoder

  10 features:
  +-- 9 elements
  +-- 1 device providers

UxPlay will need the v4l2h264dec and v4l2convert plugins, and v4l2h264dec needs to have fixes from gstreamer-1.21 development series backported to it. If none of the decoders, encoders and converters are shown, your system may be missing a required kernel module such as bcm2835_codec.ko.

Check for the Broadcom kernel drivers with

lsmod | grep bcm

This should show that the bcm2835_codec module is present.

  • Without the bcm2835_codec kernel module, and the video4linux2 decoders, hardware decoding of h264 video by UxPlay will not be available on the RPi. Correct this before proceeding. The most likely cause for a missing v4l2h264dec plugin is that firmware is not updated, gstreamer1.0-plugins-good is not installed, or that kernel modules that access the firmware are missing.

Debian-based systems (Raspberry Pi OS, Ubuntu): patch the deb package source.

Before you begin, install the GStreamer packages libgstreamer1.0-dev, libgstreamer-plugins-base1.0-dev, which are needed in the build process. (If you have already built UxPlay, these will be already be installed.) Also install gstreamer1.0-plugins-good.

GStreamer-1.18.4 (Debian-Bullseye-based Raspberry Pi OS.).

  • The current Raspberry Pi OS (Bullseye) now has a patched package gstreamer1.0-plugins-good-1.18.4-2+deb11u1+rpt1: you no longer need to patch gstreamer, but will need to use the -bt709 option with UxPlay-1.56 or later, as the patch used was (almost) equivalent to v4l2_1_18_4_diff2.txt. (There are still some issues with the patch in the current Raspberry Pi OS package, such as (if the -v4l2 or -rpi* options are used) a crash when the image on the client is rotated, so you may still wish to apply the patch given here.)

Download the patch: v4l2_1_18_4_diff3.txt; if you are doing this from the command line, use

wget https://github.com/FDH2/UxPlay/files/9196641/v4l2_1_18_4_diff3.txt

The first step is to download the gstreamer1.0-plugins-good source package: edit /etc/apt/sources.list to uncomment the deb-src entries (or duplicate your deb entries and change "deb" to "deb-src") and then run sudo apt update. Then do

sudo apt install devscripts
sudo apt build-dep gstreamer1.0-plugins-good

Next (in a directory where you will rebuild the package, and without using "sudo"), do

apt source gstreamer1.0-plugins-good
cd gst-plugins-good1.0-1.18.4
debuild

This rebuilds the unpatched package. When the building phase is finished, debuild seems to stall with "ninja: no work to do" while the post-build gstreamer tests are running: it's OK to just terminate debuild with ctrl-c at this point. (Some GStreamer tests that debuild runs will probably fail because some GStreamer components they need are not installed, and even if they all pass, debsign will fail as debuild finishes.)

  • If you want to be sure that the build succeeded do "cd obj*; ninja ; cd ..". The response should be: "ninja: no work to do".

Don't worry about debuild error messages and just continue to the next step:

patch -p1 < (path to the patch)/v4l2_1_18_4_diff3.txt
cd obj*          
ninja

The build system "ninja" should report something like "[24/24] Linking target sys/v4l2/libgstvideo4linux2.so" to show that the patched rebuild was successful. Then install the new patched version of libgstvideo4linux2.so by by stripping it (removing debug symbols) and overwriting the unpatched version that came with your distribution.

cd sys/v4l2
strip libgstvideo4linux2.so
sudo cp libgstvideo4linux2.so  (location)/gstreamer-1.0/

Here

(location) = /usr/lib/aarch64-linux-gnu  (64 bit version)
(location) = /usr/lib/arm-linux-gnueabihf  (32 bit version)

and obj* is either obj-aarch64-linux-gnu or obj-arm-linux-gnueabihf.

(It is possible you may need to reboot before the patch works.) Unless it is already running, you need to start avahi-daemon (sudo systemctl start avahi-daemon) before testing, and if the Pi is running a firewall, open up the network ports you will use with UxPlay's -p option.

  • If you are using the "Lite" version of Raspberry Pi OS without X11, use uxplay -vs kmssink . In "Desktop" versions just use uxplay (or uxplay -vs waylandsink if Wayland graphics are being used) (This assumes UxPlay-1.55 or later). The extra option -v4l2 may sometimes be helpful, as may be alternative -vs videosink choices on Desktop systems.
    (In latest UxPlay, these are also available combined with -v4l2 as -rpi, -rpigl, -rpifb and -rpiwl.)

In the test on a clean install of "Lite", sound through hdmi was got to work by installing pulseaudio (reboot after installing, and use raspi-config to select the sound output device).

  • NOTE: sometimes just "uxplay" with no options gives better performance than using the -v4l2 option. This seems to vary between different operating system distributions and their versions. Explore using/not using -v4l2 and choosing different videosinks.

GStreamer-1.20.1 (for Ubuntu 22.04 LTS "jammy")

Procedure is same as for RPi OS, except replace 1.18.4 by 1.20.1, 1_18_4 by 1_20_1, etc.

patch is: v4l2_1_20_1_diff3.txt.

wget https://github.com/FDH2/UxPlay/files/9196657/v4l2_1_20_1_diff3.txt

Use "settings" to set the sound output device. UxPlay options -v4l2 -vs glimagesink (or -rpigl) seem to work well.

  • Installation on Wayland-based Ubuntu 21.10 is similar, using the patch for 1.18.5. Here use -vs waylandsink.

GStreamer-1.20.3 (For Ubuntu 22.04.1, LTS "jammy update", and Ubuntu-22.10 "kinetic kudu")

Same as above, but replace 1.20.1 , 1_20_1 by 1.20.3 and 1_20_3.

patch is: v4l2_1_20_3_diff3.txt

wget https://github.com/FDH2/UxPlay/files/9196663/v4l2_1_20_3_diff3.txt

In this case, with Ubuntu-22.04.1, some post-recompilation testing steps in debuild also fail, before "debsign" is reached. This seems to be because Ubuntu 22.04.1 updated some of gstreamer to 1.20.3, but left the installed base packages at version 1.20.1. The failed tests do not involve video4linux2, so they don't matter. Ignore them and continue to the patching step. (3 tests failed on Ubuntu-22.10: again ignore them.)

Non-Debian systems (e.g., Manjaro-Arm 64bit Linux for RPi 4) with GStreamer-1.20.x: patch the official GStreamer source.

  • (No longer needed, as current Manjaro supplied GStreamer-1.22)

First make sure build tools such as git, gcc, cmake, and patch are installed. Also test for pkg-config with "which pkg-config": install it if it is not found (packages for pkg-config can also have names like "pkgconfig" or "pkgconf").

Before you begin, install the GStreamer packages needed for building UxPlay, plus the GStreamer "plugins-good" package.

The GStreamer source will be taken from the official GStreamer site. Patches can be found at the top of the this page.

On Manjaro (currently using GStreamer-1.20.5), (install package) = pacman -S. Use the patch v4l2_1_20_5_diff3.txt

wget https://github.com/FDH2/UxPlay/files/10298147/v4l2_1_20_5_diff3.txt

  1. (install package) meson flex bison
  2. git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git
  3. cd gstreamer
  4. git checkout 1.20.5
  5. cd subprojects/gst-plugins-good
  6. patch -p1 < (path to the patch)/v4l2_1_20_5_diff3.txt
  7. meson setup build
  8. cd build
  9. ninja
  10. cd sys/v4l2
  11. strip libgstvideo4linux.so
  12. sudo cp libgstvideo4linux.so (location)/gstreamer-1.0/

On Manjaro, (location) = /usr/lib/.

  • make sure that step 6 meson build succeeded before continuing to step 7 (it will fail if bison or flex are not installed)

To patch GStreamer 1.18.x , (example is 1.18.4: patches only available for 1.18.4 - 1.18.6), using gitlab.freedesktop.org source:

  1. (install package) meson flex bison
  2. git clone https://gitlab.freedesktop.org/gstreamer/gst-build.git
  3. cd gst-build
  4. git checkout 1.18.4
  5. meson setup build
  6. cd subprojects/gst-plugins-good
  7. patch -p1 < (path to patch)/v4l2_1_18_4_diff3.txt
  8. meson setup build
  9. cd build
  10. ninja
  11. cd sys/v4l2
  12. strip libgstv4l2videodec.so
  13. sudo cp libgstv4l2videodec.so (location)/gstreamer-1.0/
  • make sure that step 4 meson build succeeded before continuing to step 5 (it will fail if bison or flex are not installed)
Clone this wiki locally