V4L2 issues with ffmpeg

[aptalca]

Hi guys,

Just want to report that I've been doing some tests with hw encode on an rpi4 via both openmax and v4l2. Openmax seems to work as expected, without issues (encode only).

V4L2 on the other hand results in some issues and errors (also encode only).

The encoded video appears in shades of green, no other colors.

Here's a full ffmpeg log output: https://pastebin.com/h6Awbv9X
Here are the errors that occur:

[h264_v4l2m2m @ 0x2fc65e0] Failed to set number of B-frames
    Last message repeated 1 times
[h264_v4l2m2m @ 0x2fc65e0] Failed to set header mode
[h264_v4l2m2m @ 0x2fc65e0] Failed to set gop size
[h264_v4l2m2m @ 0x2fc65e0] h264 profile not found
[h264_v4l2m2m @ 0x2fc65e0] Encoder adjusted: qmin (0), qmax (51)
[h264_v4l2m2m @ 0x2fc65e0] Failed to set minimum video quantizer scale
[h264_v4l2m2m @ 0x2fc65e0] Failed to set maximum video quantizer scale

The helpful folks over at LibreELEC pointed me in the right direction (especially 6by9 from raspberry pi). The shades of green is due to jellyfin's ffmpeg missing this commit which is merged to master but is not in 4.2

I'll let you decide whether you want to patch it for the arm/aarch64 builds or not. But I thought you should know.

Thanks.

[Artiume]

Hey there, I made a PR for this. Unfortunately, your ffmpeg logs are gone, can i get them again?

[aptalca]

@Artiume
Here's a fresh log with the latest jellyfin stable, trying v4l2 for encode (getting green screen): https://pastebin.com/JSRzjXG6

By the way, in the previous version of jellyfin, openmax was working fine, but in the latest it is no longer working. I'm looking into it and will post logs in a bit.

[aptalca]

When I select openmax (only for encode, the decode ones are unchecked), playback never starts and results in a pop up error:

Playback Error
There was an issue with the client profiling and the server isn't sending a compatible media format.

Here's the log: https://pastebin.com/0hAFiwdB

[Artiume]

    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Stream #0:2(eng): Subtitle: subrip (default)
Only '-vf scale=trunc(min(max(iw\,ih*dar)\,720)/2)*2:trunc(ow/dar/2)*2' read, ignoring remaining -vf options: Use ',' to separate filters
Only '-af volume=2' read, ignoring remaining -af options: Use ',' to separate filters
Stream mapping:

  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_omx))

  Stream #0:1 -> #0:1 (ac3 (native) -> mp3 (libmp3lame))

Press [q] to stop, [?] for help

Updated to include mappings

[JustAMan]

@aptalca this cannot be the full log, it doesn't contain any error messages. Those that @Artiume quoted are only warnings.

[aptalca]

With openmax it did not even start playback, resulted in a pop up error in web gui (attempting playback in browser), and the log I posted is the entire transcode log.

I can try again, if there was a new release since

[cvaziri-cca]

I have confirmed the commit @aptalca pointed out does give us v4l2 playback in ARM without the green overlay. After compiling ffmpeg from the master branch and providing jellyfin with this ffmpeg build I am getting normal playback with x264 and x265, so far. My build and tests are with Pi 4 + Debian 10 (slim).

[aptalca]

Hey guys, finally got a chance to repeat my experiment with omx. With both the latest stable and nightly, it works for hw encode. Tested on rpi4 with raspbian buster.

And one bit of surprise. . . drum roll. . . I got hw decode of h264 working on rpi4 with mmal. I'll update the linuxserver image readme with instructions and will link here.

[Artiume]

Awesome.

For HEVC, I was reading into this ticket
popcornmix/omxplayer#714

The HEVC hardware uses custom pixel formats that ffmpeg doesn't understand. This isn't a problem for kodi as it can pass them as opaque buffers to mmal_render component for display.

We might have upstream issues blocking the hevc support :(

[aptalca]

Here's the updated instructions for mmal: https://github.com/linuxserver/docker-jellyfin#openmax-raspberry-pi
Basically passing /dev/vc-mem device allows the container necessary access for mmal. Also requires increasing gpu mem, which you guys advertise in your wiki anyway.

In jellyfin gui settings, when you enable h264 decoding with omx, it will attempt mmal for decode and it works.

@Artiume btw, we also added jellyfin's opencollective donation link to our github repo's sponsor button , as well the container logs. Sharing the FOSS love

[Artiume]

Thank you. It's awesome that we now have h264 decoding added to the Rpi4. It's turning into an indomitable little machine. I need to work out getting it working for our image and fix the docs. I need to change it to compose version 2 so I use group_add for the video group. Just haven't gotten time to work on it 😅

[aptalca]

@Artiume I did notice a bug, though (weird one) with the latest stable.

I set hw accel to OpenMax and enable H264 decoding. I play H264 media, everything's great, mmal decoding, omx encoding.

But once I play hevc media (native hevc decode, omx encode), the H264 decode option in openmax transcode settings gets disabled.

I can reproduce this every time.

EDIT: Tested to confirm that the disabling of H264 decoding happens right after hevc playback starts

[Artiume]

But once I play hevc media (native hevc decode, omx encode), the H264 decode option in openmax transcode settings gets disabled

@dkanada might be able to help here. That bug was supposed to be fixed

[ikaruswill]

+1, I face the same issue as @aptalca with OMX H264 hardware decoding being automatically disabled after HEVC playback, as referenced in the issue linked above.

[JustAMan]

What you're referring to is probably fixed in jellyfin/jellyfin#2809 and is a bug in Jellyfin, not its ffmpeg build.

[K5HV]

For anyone who is looking for solution for green output when using V4L2 M2M:
Use new FFmpeg (I've used git master version, newer then 4.2.3)
Use yuv420p as pixel format (-pix_fmt yuv420p). Latest FFmpeg should disallow using other formats with v4l2 anyway.

[RealCorebb]

For anyone who is looking for solution for green output when using V4L2 M2M:
Use new FFmpeg (I've used git master version, newer then 4.2.3)
Use yuv420p as pixel format (-pix_fmt yuv420p). Latest FFmpeg should disallow using other formats with v4l2 anyway.

Excuse me.How can i set the -pix_fmt yuv420p in Jellyfin?

[K5HV]

ffmpeg.zip
I'm using this script as starter for ffmpeg. It just do simple rewrite of nv21 to yuv420p and removes yadif fillter and start corrent ffmpeg.

Rename your ffmpeg binary to ffmpeg-ext (in my case: /usr/lib/jellyfin-ffmpeg/ffmpeg to /usr/lib/jellyfin-ffmpeg/ffmpeg-ext) and copy this script as ffmpeg (/usr/lib/jellyfin-ffmpeg/ffmpeg in my case). Don't forget about correct ownership and executable permission.

I'm also using binary from https://johnvansickle.com/ffmpeg/ as it seems to be a bit more faster than jellyfin one.

[Shadowghost]

Pixel format selection should be fixed in 10.8, v4l2 on rpi still has some limitations though, see #129