Native support for codecs/formats (TS,H265,mpeg2++)

[mifi]

Due to losslesscut using chromium's html5 video tag which only supports certain container formats and codecs, we currently need to "convert to supported format" when trying to open exotic formats or codecs. This could be improved with the help of ffmpeg.

Workarounds

Compile custom electron

• https://github.com/electron/build-tools

Convert to supported format

Currently it's possible to cut those files by first using the File -> Convert to supported format (slow) function, however this is a re-encoding (although fast) of the whole file, just to be able to view it in the HTML5 video player (note that the original and cut versions will not be re-encoded and are still lossless.)

Improve current proxy/dummy code

Currently a <canvas> element is used, and ffmpeg will stream low-resolution jpeg images to that canvas.

Drawbacks:

• Slow, laggy, low resolution, low fps
• No audio

This could be improved:

• MediaSource API & https://www.electronjs.org/docs/latest/tutorial/message-ports
• Web Audio API: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API
• https://github.com/strukturag/libde265.js
• https://stackoverflow.com/questions/55376781/seeking-with-ffmpeg-at-the-end-of-an-rtsp-stream
• https://stackoverflow.com/questions/61672575/rtsp-to-webrtc-live-video-streaming
• https://stackoverflow.com/questions/41965793/rtsp-h-264-in-google-chrome-browser
• https://github.com/Streampunk/beamcoder
• https://github.com/GleasonK/WebRTC-YouTubeTogether
• https://phoboslab.org/log/2013/09/html5-live-video-streaming-via-websockets
• https://github.com/phoboslab/jsmpeg

Related

• H.265 video playback webtorrent/webtorrent-desktop#561
• Support more audio and video codecs webtorrent/webtorrent-desktop#1389

Electron (Chromium) might never support h265

It seems like Chrome and most other browsers are not going to be supporting H265 as it stands now, and since we are using Chromium (Electron), this will be a problem.
It might be possible to enable a flag to enable H265 decoding, but it requires some modding of electron.

• https://stackoverflow.com/questions/36382568/h-265-hevc-web-browser-support
• Building with an alternative libffmpeg electron/libchromiumcontent#198

Update: Chromium now supports h265 (hardware decoding only)

[killaz47]

heres the solution from the chrnoium developers
https://github.com/henrypp/chromium/blob/master/hevc_support.md
https://github.com/henrypp/chromium/blob/master/hevc_support_new.md

[killaz47]

Now that more and more cameras (including DJI Mavic 2) are starting to support H265, it would be nice to have support for this here as well.
Currently it's possible to cut those files by first using the File -> Convert to friendly format (slow) function, however this is a re-encoding (although fast) of the whole file, just to be able to view it in the HTML5 video player (note that the original and cut versions will not be re-encoded and are still lossless.)

It seems like Chrome and other browsers are not going to be supporting H265 as it stands now, and since we are using Chromium (Electron), this will be a problem.
It might be possible to enable a flag to enable H265 decoding, but it requires some modding of electron.

If anyone has some bright ideas, shout out!

• https://stackoverflow.com/questions/36382568/h-265-hevc-web-browser-support
• webtorrent/webtorrent-desktop#561
• electron/libchromiumcontent#198
• https://github.com/strukturag/libde265.js

heres the solution from the chrnoium developers
https://github.com/henrypp/chromium/blob/master/hevc_support.md
https://github.com/henrypp/chromium/blob/master/hevc_support_new.md

[mifi]

I'm not sure how to do this with electron. (LosslessCut uses Electron which uses Chromium.) Feel free to open an issue at electron to see if this can be integrated. I think it is not a trivial task, so I don't have time to look at this now.

[TechManiacHD]

Lately I swap my gear and see, that recording in HEVC format gives me better quality over H264 even with lower bitrate. I would love to switch to HEVC for all my projects although only Lossless Cut stops me from doing that - https://i.imgur.com/avu5AQE.png (after converting to friendly format).

Is there a chance this will work at some point or should I still sit on old format? Thanks.

[esamattis]

I'm seeing that too with GoPro 7 HEVC videos and fast convert. The slow one works.

[TechManiacHD]

You're right! It works with slow one (That quality xD). Enough for cutting no worries :)
Thanks, that will do for time being.

[mifi]

Great! the reason for the low quality is to make it as fast as remotely possible, and only to be used as a preview.

[antonioriva]

As LossLess-cut is build in electron do you ever considered to use mpv as player component? I've just found that a project to do that it's already there... https://github.com/Kagami/mpv.js/ with also some samples implementation

[mifi]

Thanks for the tip! I didn't know about mpv.js, but if it's easy to integrate then that's definitely an option.
Kagami/mpv.js#27

[mifi]

In 1.5.0 I have implemented a hack that lets us cut H265 videos, but scrubbing and playing only works at a very low fps and currently without audio.

[ThexXTURBOXx]

Getting this error when trying to merge 2 H.265 videos together. Is there something I can do about this?

[peterhighgrove]

Is it possible to add support for grabbing a frame at 4k/hevc to jpg even though the preview is not native?

[mifi]

Is it possible to add support for grabbing a frame at 4k/hevc to jpg even though the preview is not native?

I have implemented this now

[peterhighgrove]

Nice, thanks! Can't wait to test it in the next release!

[mifi]

Check out newest version now

[peterhighgrove]

Thanks, tested 3.17.9. It's working great:)

[ThexXTURBOXx]

@mifi Is there any news on supporting H265 natively?

[mifi]

No. I think it needs a lot of work because have to compile a custom electron with a custom chromium that uses a custom ffmpeg. Not something i will prioritize unless this work gets funded somehow, or if we can find the work of someone else who did this already.

[dpallen]

Thanks for the quick reply, but I resolved my issue after some more testing. My footage used 4:4:4 chroma subsampling. Using 4:2:0 HEVC footage works perfectly in llc 3.48.2 beta.

[mifi]

So to summarize, in order to detect if the player/device supports a file, (for example HEVC) we need to:

const mediaConfig = {
  type: 'file',
  video: {
    contentType : 'video/mp4;codecs="hev1.1.6.L120.90"',
    width: 1920,
    height: 1080,
    bitrate: 10000, 
    framerate: 30,
  }
}

const { supported } = await navigator.mediaCapabilities.decodingInfo(mediaConfig);
// result.supported indicates whether or not the video with given profile, width, and height can played well on the browser

The problem is to find the contentType parameter. It is described here: https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter but it varies by format/codec. Need to find a library where someone implemented this logic into a library for all formats/codecs. I would think that the output from ffprobe could be transformed into this codec parameter.

Potential implementations for each format/codec:

• AV1 spec implementation
• MP4

See also

• https://github.com/StaZhu/enable-chromium-hevc-hardware-decoding#mediacapabilities
• https://developer.mozilla.org/en-US/docs/Web/API/Navigator/mediaCapabilities

[ThaUnknown]

hate to bear the bad news, but that decoder implementation is incorrect, long story short? the color-space decoding is fucked,
I went into detail here: Alex313031/thorium#70

I probably still recomment uzing StaZhu's or Muril-o patches, as they both seem to do color-space conversion correctly, I'll see about merging StaZhu's patches into Muril-o's as they additionally support more codecs, not just HEVC

[mifi]

Not sure why that's a problem. the hevc decoder is only used for previewing. When exporting, the decoder will not be used

[Spaarpott]

@mifi

Not sure why that's a problem. the hevc decoder is only used for previewing. When exporting, the decoder will not be used

And when you save a frame as .jpeg?

[mifi]

That’s a point. We could have a setting for whether to use <video> tag or ffmpeg to capture frame.

[Spaarpott]

I'm not a programmer, so I wouldn't know. As long as the image colors are correct in a export whether it is a .jpeg or a movie file.

[mifi]

I'm not sure how to detect if the player shows black screen. Could someone who sees a black screen with a HEVC file, please try to

1. press "play",
2. check Developer Tools log and look for the warning line onVideoError
3. copy paste this whole line here?

[mifi]

next version will allow choosing ffmpeg or video-tag for capturing frame, as well as captured image quality

[mifi]

next version will allow disabling the hevc support (for people who have problem with blank screen)
it will also include a basic check for hevc support

[Fred-Vatin]

In my case, my hardware can not read H265. So if the support option is enabled, I have sound but blank screen. But if I disable the option, I have image but no sound.

I have no problem to read H265 in other players.

[Delphox]

I'm having a strange issue with HEVC. My graphics chip support decoding it, and most HEVC videos play fine on LosslessCut, however videos specifically recorded from OBS then converted into HEVC with ffmpeg (either with libx265 or hevc_amf) into an mp4 container will turn into a black screen on LosslessCut. They play fine everywhere else including chrome.

I've attached a problematic mp4 file so you can play around with it:

2023-06-30.18-30-01_hevc.mp4

A workaround for now is converting to HEVC into an mkv container instead then remuxing into an mp4, or if I already have a problematic mp4, remux into mkv then back into mp4.

[mifi]

hmm, this video doesn't play in chrome for me. audio only

[Spaarpott]

@mifi In my chrome version the file of Delphox plays fine. Chrome version: 114.0.5735.199
However, i don't hear any audio, nor in Chrome, nor in VLC, but it has a audio track.
It shows indeed a black screen in LosslessCut v3.54.
When I multiplex the file to MKV with mkvtoolnix, it opens in LosslessCut fine.

Screenshot of Chrome:

[Delphox]

It has an empty audio track yea, as I recorded it in OBS with no audio playing (the sample video had no audio track at all). Removing the audio track makes it play on LosslessCut, which is strange, and why I left the audio track on.

[mifi]

Ok I just upgraded my chrome from 111 to 114 and now I can also play the video. So it seems like it was a bug in chrome(chromium). Some time in the future once we upgrade electron to the latest version in LosslessCut, the video will probably just start working in losslesscut too.

[mifi]

coming up in a pr #1844 an improved playback of unsupported files, using the MediaSource API. this allows playback of virtually all formats/codecs with audio as well