Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix transcoding errors in readonly docker containers #3198

Merged
merged 1 commit into from
Oct 23, 2020

Conversation

gramakri
Copy link
Contributor

ffmpeg seems to create some temporary files in the cwd. When PeerTube
is run in a read-only docker container, this causes all transcoding
to fail. As a workaround, we set the cwd to the configured tmp dir.

@gramakri
Copy link
Contributor Author

gramakri commented Oct 16, 2020

The exact error is in the forum post here - https://forum.cloudron.io/topic/3414/transcode-after-import-fails (EROFS)

error[15-10-2020 09:32:42] Error in transcoding job.

{
  "stdout": "",
  "stderr": "ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers\n  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)\n  configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared\n  libavutil      55. 78.100 / 55. 78.100\n  libavcodec     57.107.100 / 57.107.100\n  libavformat    57. 83.100 / 57. 83.100\n  libavdevice    57. 10.100 / 57. 10.100\n  libavfilter     6.107.100 /  6.107.100\n  libavresample   3.  7.  0 /  3.  7.  0\n  libswscale      4.  8.100 /  4.  8.100\n  libswresample   2.  9.100 /  2.  9.100\n  libpostproc    54.  7.100 / 54.  7.100\nInput #0, mov,mp4,m4a,3gp,3g2,mj2, from '/app/data/storage/videos/881f46cb-4afb-4977-9fe2-7fe580d20fc0-480.mp4':\n  Metadata:\n    major_brand     : isom\n    minor_version   : 512\n    compatible_brands: isomiso2avc1mp41\n    encoder         : Lavf57.83.100\n  Duration: 00:00:27.05, start: 0.000000, bitrate: 799 kb/s\n    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 854x480 [SAR 1:1 DAR 427:240], 669 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default)\n    Metadata:\n      handler_name    : VideoHandler\n    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)\n    Metadata:\n      handler_name    : SoundHandler\n[hls @ 0x55601c225e40] Opening 'init.mp4' for writing\n[hls @ 0x55601c225e40] Failed to open segment '/app/data/storage/streaming-playlists/hls/881f46cb-4afb-4977-9fe2-7fe580d20fc0/881f46cb-4afb-4977-9fe2-7fe580d20fc0-480-fragmented.mp4'\nCould not write header for output file #0 (incorrect codec parameters ?): Read-only file system\nStream mapping:\n  Stream #0:0 -> #0:0 (copy)\n  Stream #0:1 -> #0:1 (copy)\n    Last message repeated 1 times\n"
}

error[15-10-2020 09:32:42] Cannot execute job 7 in queue video-transcoding.

{
  "payload": {
    "type": "hls",
    "videoUUID": "881f46cb-4afb-4977-9fe2-7fe580d20fc0",
    "resolution": 480,
    "copyCodecs": true
  },
  "err": {
    "stack": "Error: ffmpeg exited with code 1: \n    at ChildProcess.<anonymous> (/app/code/server/node_modules/fluent-ffmpeg/lib/processor.js:182:22)\n    at ChildProcess.emit (events.js:310:20)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)",
    "message": "ffmpeg exited with code 1: "
  }
}

error[15-10-2020 09:32:43] Error in transcoding job.

{
  "stdout": "",
  "stderr": "ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers\n  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)\n  configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared\n  libavutil      55. 78.100 / 55. 78.100\n  libavcodec     57.107.100 / 57.107.100\n  libavformat    57. 83.100 / 57. 83.100\n  libavdevice    57. 10.100 / 57. 10.100\n  libavfilter     6.107.100 /  6.107.100\n  libavresample   3.  7.  0 /  3.  7.  0\n  libswscale      4.  8.100 /  4.  8.100\n  libswresample   2.  9.100 /  2.  9.100\n  libpostproc    54.  7.100 / 54.  7.100\nInput #0, mov,mp4,m4a,3gp,3g2,mj2, from '/app/data/storage/videos/881f46cb-4afb-4977-9fe2-7fe580d20fc0-480.mp4':\n  Metadata:\n    major_brand     : isom\n    minor_version   : 512\n    compatible_brands: isomiso2avc1mp41\n    encoder         : Lavf57.83.100\n  Duration: 00:00:27.05, start: 0.000000, bitrate: 799 kb/s\n    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 854x480 [SAR 1:1 DAR 427:240], 669 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc (default)\n    Metadata:\n      handler_name    : VideoHandler\n    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)\n    Metadata:\n      handler_name    : SoundHandler\nStream mapping:\n  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))\n  Stream #0:1 -> #0:1 (aac (native) -> aac (native))\nPress [q] to stop, [?] for help\n[libx264 @ 0x560188acf240] using SAR=1281/1280\n[libx264 @ 0x560188acf240] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2\n[libx264 @ 0x560188acf240] profile High, level 3.1\n[libx264 @ 0x560188acf240] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=4 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=16 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=50 keyint_min=5 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=728 vbv_bufsize=1456 crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00\n[hls @ 0x560188aca9a0] Opening 'init.mp4' for writing\n[hls @ 0x560188aca9a0] Failed to open segment '/app/data/storage/streaming-playlists/hls/881f46cb-4afb-4977-9fe2-7fe580d20fc0/881f46cb-4afb-4977-9fe2-7fe580d20fc0-360-fragmented.mp4'\nCould not write header for output file #0 (incorrect codec parameters ?): Read-only file system\nError initializing output stream 0:0 -- \n[aac @ 0x560188af2620] Qavg: 13786.650\n[aac @ 0x560188af2620] 2 frames left in the queue on closing\nConversion failed!\n"
}

error[15-10-2020 09:32:43] Cannot execute job 8 in queue video-transcoding.

{
  "payload": {
    "type": "hls",
    "videoUUID": "881f46cb-4afb-4977-9fe2-7fe580d20fc0",
    "resolution": 360,
    "isPortraitMode": false,
    "copyCodecs": false
  },
  "err": {
    "stack": "Error: ffmpeg exited with code 1: Conversion failed!\n\n    at ChildProcess.<anonymous> (/app/code/server/node_modules/fluent-ffmpeg/lib/processor.js:182:22)\n    at ChildProcess.emit (events.js:310:20)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)",
    "message": "ffmpeg exited with code 1: Conversion failed!\n"
  }
}

@gramakri
Copy link
Contributor Author

I think this fails because we have to update the @types/fluent-ffmpeg to 2.1.14 (DefinitelyTyped/DefinitelyTyped#48421)

ffmpeg seems to create some temporary files in the cwd. When PeerTube
is run in a read-only docker container, this causes all transcoding
to fail. As a workaround, we set the cwd to the configured tmp dir.
@Chocobozzz
Copy link
Owner

Thanks @gramakri

@Chocobozzz Chocobozzz merged commit 313921b into Chocobozzz:develop Oct 23, 2020
@gramakri gramakri deleted the xcode_tmpdir branch October 23, 2020 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Docker 🐳 Deals with containerisation, a hellish nightmare for Chocobos Component: Transcoding
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants