HW-Accelerated encoding to HEVC fails with QuickSync over VA-API #209
Comments
|
Can you try this command?
|
|
Hi, |
|
Thanks. The issue comes from the default b-frame settings of the |
|
@nyanmisaka Hi, I'm getting a similar error when I enable QSV and play some videos. Could you please check if it's the same bug or should I open a new one? Here is the log: |
|
@cuihaoleo Same issue as the OP. |
|
@cuihaoleo Can you try this command and see if it works? |
|
@nyanmisaka Your command seems to finish without error. Here is the output: |
|
It is invalid. @cuihaoleo Can you try this one again? |
|
@nyanmisaka This one ran for about 10 seconds but eventially died with the original error. Full outputThe last progress line before the error overwrote it was |
|
I saw you removed A strange finding when I played around the command. When I modified the command to avoid audio encoding (either For example, here is my modified command: And it ran through the whole video without any error: Log |
|
@cuihaoleo It seem the issue is more complicated than I was thinking about. Can you share the whole video so as I can reproduce it on my end? And please give more contexts about your iCPU model, Linux distro and kernel version. Thanks! |
|
@nyanmisaka I generated a smaller sample to save bandwidth and avoid illegal distribution with the following command: I hope this is enough to test the bug as I observed the same error when playing this Here are the video and jellyfin / ffmpeg logs: jellyfin-bug.tgz System information |
|
@cuihaoleo Thanks! I'll look into it. |
|
@cuihaoleo I can't reproduce the issue locally, with or without the modified ffmpeg cmd. hd630_fine.txt I suspect there's an issue when muxing the video and audio. |
|
Hi @nyanmisaka I tried First, I found the number of threads matters. When I set Full output: log_t12.log When I set it to 13 or more, the error occurs: Full output: log_t13.log Second, I built a version with debug information with your portable versions builder and run GDB on it to see what happened. The error occurs when NULL is returned from jellyfin-ffmpeg/libavutil/hwcontext_vaapi.c Lines 480 to 482 in ae2db3b Some GDB output (note that the line numbers can differ from code in the repo due to patches and some manual edits): |
|
@cuihaoleo Thanks for digging into it!
In most cases the threads of HW decoders have little effect on FPS scaling. You can try setting it to 1 or 2 to observe the FPS. VAAPI in FFmpeg currently does not support dynamic frame pool resizing, which means that you must set If it's handy, you can try this patch, it seems to be forgotten by upstream. |
|
@nyanmisaka I tried the "dynamically frame pool resizing" patch. It seems it only add one line of error before crash, without fixing anything: I also traced where the value 10 of jellyfin-ffmpeg/libavfilter/vaapi_vpp.c Line 192 in 52193be I don't find any clue about what this hardcoded value means. But I guess it's really about some obscure VAAPI VPP settings instead of decoders? Changing it to a higher value, for example, 16, works around the error on my side. |
|
@cuihaoleo Add |
|
@nyanmisaka
|
|
@cuihaoleo The legacy jellyfin-ffmpeg/libavfilter/vf_scale_qsv.c Line 195 in ae2db3b And I added extra 32 in a rare case on AMD VAAPI. Can you reproduce the issue in these two cases?
|
QSV HWA, OS native VAAPI decoder disabledI assume you mean to keep "Hardware acceleration" at "Intel QuickSync (QSV)" and unselect "Prefer OS native DXVA or VA-API hardware decoders" option in Jellyfin setting. The error went away under this configuration. Log: FFmpeg.Transcode-2023-02-15_18-03-25_3f4edc52ad8223cf98a2e25f8e3e9c9d_df38b53e.log VAAPI HWAI assume you mean to change "Hardware acceleration" to "Video Acceleration API (VAAPI)". The error was reproducible under this configuration. Log: FFmpeg.Transcode-2023-02-15_18-05-27_3f4edc52ad8223cf98a2e25f8e3e9c9d_3dd3d0de.log Modifying the command with |
|
@cuihaoleo Can you reproduce the issue when the subtitle burn-in is disabled? If not, we just need to figure out a suitable BTW how about 4k transcoding? We've been testing 1080p. |
|
@nyanmisaka I don't think I can turn off subtitle burn-in for PGSSUB. The minimum I can choose is burn "Only image formats" and that still includes PGSSUB. If I disable subtitle, Jellyfin will switch to direct stream and there will be no error. I don't have any 4K video with PGSSUB on my machine. Let me see if I can download or create one. |
|
I wrote a simple script to try a few different encoding parameters and search for the minimum The script#!/usr/bin/env python3
import sys
import os
from itertools import product
import subprocess
ENCODER_LIST = ["h264_qsv", "hevc_qsv"]
RESOLUTION_LIST = [(1280, 720), (1920, 1080), (3840, 2160)]
DURATION_LIST = ["00:01:00", "00:02:00", "00:04:00", "00:08:00"]
encoding_cmd = "ffmpeg -y -nostdin -hide_banner -loglevel error -hwaccel qsv -hwaccel_output_format qsv -canvas_size {width}x{height} -f lavfi -i testsrc=size={width}x{height},format=yuv420p -f lavfi -i anoisesrc -i {pgs_file} -map 0:v -map 1:a -map 2:s -codec:v {encoder} -preset veryfast -codec:a flac -codec:s copy -t {duration} {output}"
transcoding_cmd = "/usr/lib/jellyfin-ffmpeg/ffmpeg -nostdin -hide_banner -loglevel error -init_hw_device vaapi=va:,driver=iHD,kernel_driver=i915 -init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -canvas_size 1920x1080 -i file:{input} -autoscale 0 -map 0:0 -map 0:1 -codec:v:0 h264_qsv -filter_complex '[0:2]scale=flags=fast_bilinear,format=bgra,hwupload=derive_device=qsv:extra_hw_frames=64[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=nv12:extra_hw_frames={extra_hw_frames},hwmap=derive_device=qsv,format=qsv[main];[main][sub]overlay_qsv=eof_action=endall:shortest=1:repeatlast=0:w={width}:h={height}' -codec:a:0 libfdk_aac -f null -"
os.environ["AV_LOG_FORCE_NOCOLOR"] = '1'
pgs_file, = sys.argv[1:]
for encoder, (width, height), duration in product(ENCODER_LIST, RESOLUTION_LIST, DURATION_LIST):
filename = "test.mkv"
subprocess.run(encoding_cmd.format(encoder=encoder, width=width, height=height, duration=duration, output=filename, pgs_file=pgs_file), shell=True, check=True)
lo = 0 # known to not work
hi = 32 # known to work
while hi - lo > 1:
mid = (lo + hi) // 2
p = subprocess.run(transcoding_cmd.format(encoder=encoder, width=width, height=height, duration=duration, input=filename, extra_hw_frames=mid), shell=True)
if p.returncode == 0:
hi = mid
else:
lo = mid
print(f"{encoder} {width}x{height} {duration}: {hi}")The A few observations:
With a paticular PGS track, I observed as many as 22 And there is another PGS track with which I observed as few as 8 Given that I already found a few other transcoding parameters ( I'm going to simply disable "Prefer OS native DXVA or VA-API hardware decoders" as a workaround. But let me know if you want to do other tests. |
|
@cuihaoleo Could you try this patch in jellyfin-ffmpeg and see if it helps? This time dynamic frame pool is allowed in vaapi_vpp. https://patch-diff.githubusercontent.com/raw/intel-media-ci/ffmpeg/pull/628.patch |
|
I attempted to apply that patch, and I have a breakage still with Intel Arc A770 LE 16GB, transcoding MPEG-2 (DVD rip) to H.264: E: It works if I turn off subtitles. |
@kode54 This seems to be a different issue on DG2. Can you share a short clip of your video with this cli?
|
|
Here you go: ttps://f.losno.co/v/test-clip.mkv |
|
@kode54 I can reproduce the issue on Linux. It should be an upstream regression in ffmpeg. I'll report it to intel. Before they figure out a patch you can disable the "OS native VA-API decoder" as a workaround. |
|
@kode54 Here's the ticket. https://trac.ffmpeg.org/ticket/10274 |
Try with Then, for every invocation of Test with this command that implements these changes and test back: /usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M \
-init_hw_device vaapi=va:,driver=iHD,kernel_driver=i915 \
-init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel vaapi -hwaccel_output_format vaapi \
-autorotate 0 -canvas_size 720x480 \
-i file:"/mnt/storage/fat/Videos/TV/Castle/Season 2/11 - The Fifth Bullet.mkv" \
-autoscale 0 -map_metadata -1 -map_chapters -1 -threads:v 1 -map 0:0 -map 0:1 \
-codec:v:0 h264_qsv -preset 7 -look_ahead 0 -b:v 4774178 -maxrate 4774178 \
-bufsize 9548356 -profile:v:0 high -level 51 -g:v:0 90 -keyint_min:v:0 90 \
-filter_complex "[0:2]scale=flags=fast_bilinear,format=bgra,hwupload=derive_device=qsv:extra_hw_frames=64[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=nv12,hwmap=derive_device=qsv:extra_hw_frames=64,format=qsv[main];[main][sub]overlay_qsv=eof_action=endall:shortest=1:repeatlast=0:w=720:h=480" \
-start_at_zero -codec:a:0 libfdk_aac -ac 2 -ab 384000 -af "volume=2" \
-copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 \
-f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts \
-start_number 0 \
-hls_segment_filename "/var/lib/jellyfin/transcodes/14553cf18532518a3f1d74a788431c17%d.ts" \
-hls_playlist_type vod -hls_list_size 0 -y "/var/lib/jellyfin/transcodes/14553cf18532518a3f1d74a788431c17.m3u8" |
|
I appear to be affected by this, or at any rate, someone else using my server is, and says the video aborts towards the end of something they are watching. Their client gets transcoded HEVC with burned-in subtitles, and the server uses an Intel DG2 Arc A380 for recoding work. System information
ffmpeg log: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe The Bug
When transcoding a VP9 video stream to HEVC with QuickSync, enabling the "prefer native encoder" setting of Jellyfin (leading to using the VAAPI API for hardware acceleration) results in an "Error while filtering: Cannot allocate memory" error at the start of transcoding.
If that setting, however, is unchecked (leading to using native QuickSync API for hardware acceleration), transcoding works flawlessly.
I am not sure if different input formats lead to different results, I've only tested it for VP9 in a WebM container.
Steps To Reproduce
yt-dlp --audio-format opus https://www.youtube.com/watch\?v\=ksQqPHJZVo4) via AndroidTV client for HEVC transcodingExpected Behavior
Successful transcoding given that the hardware is actually capable of decoding VP9 and encoding HEVC
System (please complete the following information):
MediaInfo
FFmpeg Logs
Failed conversion:
Successful transcoding when disabling "Prefer System Native Hw Decoder" in Step 3:
Additional Context
I'm running my Jellyfin instance in an unprivileged Docker container.
HW-accelerated transcoding to H264 works as expected.
The text was updated successfully, but these errors were encountered: