Keyframe cut position accuracy problem (in both modes)

[bouscram]

I have noticed a problem which seems to be in ffmpeg, so I don't know if you can do anything, but here it is... (I used version 3.21.0 on macOS 10.15.4 for the tests)

I have a file I want to cut. So I have set the cutting point to a keyframe, as can be seen here:

When I use the "normal" cutting mode, the sound starts at the right time, but the video is blank until the next keyframe.
When I use the "keyframe" cutting mode, everything starts at the previous keyframe, adding a part of the commercial I want to remove, as if I had selected at this point:

In both cases, the command uses the same time as a stating point:

• normal mode:

ffmpeg -hide_banner -i '/Users/cram/Movies/Handbrake/Opération jupons.mp4' -ss '603.19755' -t '6922.98914' -c copy -map '0:0' -map '0:1' -map_metadata 0 -movflags use_metadata_tags -ignore_unknown -f mp4 -y '/Users/cram/Movies/Handbrake/Opération jupons-00.10.03.197-02.05.26.186.mp4'

• keyframe mode:

ffmpeg -hide_banner -ss '603.19755' -i '/Users/cram/Movies/Handbrake/Opération jupons.mp4' -t '6922.98914' -avoid_negative_ts make_zero -c copy -map '0:0' -map '0:1' -map_metadata 0 -movflags use_metadata_tags -ignore_unknown -f mp4 -y '/Users/cram/Movies/Handbrake/Opération jupons-00.10.03.197-02.05.26.186.mp4'

To support your analysis, I have prepared a zip file including the video from the beginning and both cut files: https://www.jottacloud.com/s/13733169b83ae2749a2a2924a699121bf92

[IsabellLehmann]

I have the same problem using version 3.21.0 on Windows. Instead of a blank frame, I still see the last frame from the last cut. Another interesting thing is, if I pause the video when the new frame should be there but is not and wait for some time, suddently the new frame appears. At least in some cases

[dbogdanov]

Same problem here. MacOS, Lossless Cut 2.4.0.

[bouscram]

I have found a sort of workaround. In "keyframe" cut mode, I have set the cut point about 2 second after the keyframe intended for the cut, as shown here:

With that setting, the video what cut at the keyframe before, as I wanted it. But it has worked only when the cut point was set far enough from the intended keyframe (I tried with 1 second and it didn't work). Based on other trials, it seems that the cut is performed at the previous, not the nearest keyframe as indicated in the settings panel

(You can also get something similar by using the "normal" cut mode, and setting the cut point some frames before the keyframe where you want to cut. It will add a short blank period before the video starts)

[mifi]

Yea, I've spent a lot of time trying to find a formula that works consistently, when it comes to where to cut to achieve an accurate cut, but I have not yet succeeded. Lossless cutting is not an exact science and will depend on the codec, keyframe types and a lot of factors. This is because of how ffmpeg works. So unfortunately for some formats and codecs you just have to trial and error because I don't know how to universally solve this. I will update the readme instructions.

[jberkus]

FWIW, I've just been bitten by this. I'm trying to remove a single keyframe from the beginning of a short video (that keyframe contains me coughing), and there's simply no way to do it with LLC. Unless I'm willing to trim off multiple keyframes (and lose content) I end up with a video that still includes the cough.

[mifi]

@jberkus one thing you could try is to download the latest GIT version from http://ffmpeg.org/download.html and then run it from the command line on your file

ffmpeg -ss "some time right after your first keyframe" -i your_file -c copy -o outfile.mkv

see if that works with the newest ffmpeg. If it works, then there is a chance that it will just start working in a future losslesscut version.

[cpiber]

One solution would be to insert keyframes as per https://superuser.com/a/908325, though I don't know how feasible this is for LLC.
It definitely works to re-encode a video (tried it with x264), haven't tried adding just that one keyframe where you want to cut.

[parasiteoflife]

I'm having a similar problem, and with several video files so far.
What I normally do for cutting is use Alt + LeftArrow/RightArrow so I end up in a keyframe then I go one (1) frame to the right using . (for the start) and one to the left using , (for the end) that way LossLessCut works as intended, if I go to the keyframe and set the start time there LossLessCut goes to the previous/next frame for the start and end of the segment. Then there are these video files that would be cut to a previous/next keyframe no matter what I do, and I don't know why.

I think LossLessCut should do what I first pointed out by default when the user is cutting using keyframe cut.

[mifi]

I get a lot of questions about this. I will refer other duplicates to this issue.

TLDR: With keyframe cut, ffmpeg will always cut at the keyframe before your selected from-cutpoint. Sometimes it will be the keyframe the cursor is currently sitting at, sometimes it will be the keyframe before that. Depends on codec, format, framerate etc. I've spent a lot of time trying to find a formula that works consistently, when it comes to where to cut to achieve an accurate cut, but I have not yet succeeded.

Lossless cutting is not an exact science and will depend on the codec, keyframe types and a lot of factors. This is because of how ffmpeg works. So unfortunately for some formats and codecs you just have to trial and error because I don't know how to universally solve this without breaking it for some codecs.

Potential fixes/workarounds:

• When calling ffmpeg, add a small time value (certain amount of frames) after the current cutpoint
  • add 1 frame (sometimes doesn't work)
  • add 2 frames (sometimes doesn't work)
  • cut in the middle between the keyframes (maybe works)
  • con: risk cutting at the next keyframe instead if keyframes are too close together, e.g. 1 keyframe at every frame
• Possibly some of these issues are caused by [Bugfix] Keyframe seeking rounds wrongly #585

See also discussion in #546 #516 #887

[fernandoherreradelasheras]

This can be solved with the minimum possible transcoding by:

1. transcode from the starting point to the frame before the next keyframe
2. extract the remaining part of the video
3. concatenate both videos

I have created a simple proof of concept in bash here:

smart-video-cutter.sh

[pldavid2]

Great work on this @fernandoherreradelasheras and @mifi
One question, wouldn't it be better to use show_packets instead of show_frames to find the correct keyframe? Remeber reading that on this comment, as not every I frame is an IDR frame:
#13 (comment)

[mifi]

@pldavid2 my code is using show_packets, see:

lossless-cut/src/ffmpeg.js

Line 104 in 11e1f81

const { stdout } = await runFfprobe(['-v', 'error', ...intervalsArgs, '-show_packets', '-select_streams', streamIndex, '-show_entries', 'packet=pts_time,flags', '-of', 'json', filePath]);

Let me know if it can be improved further. This is the function losslesscut has always used to read keyframes (also the ones shown in the timeline)

[pldavid2]

sorry my bad @mifi ! I only saw the proof of concept 🤦

I'll wait for the official release to test it 😄

[cyzs233]

Personally, I used a similar approach as bouscram's "keyframe cut" method. Except you don't have to cut 2 secs after the keyframe, 5 frames (press next_frame 5 times)will do the trick. I have cut 1000+ films using this tool with this "5 frame diviation principle" on both cut_begin and cut_end, and never ever encounter this "cut at the wrong keyframe" problem. (Of course, you have to press pre_frame on cut_ends)

But to cover every user case, I suggest we make it an option to allow users to tweak "frames of diviation from cutpoint" to their own needs. 4 frames of deviation is also fine, but I do get wrong cuts from time to time, thus increased it to 5 frames.

[mifi]

@cyzs233 interesting. I wonder where the "5 frames" comes from. Sometimes keyframes are also spaced less than 5 frames apart, e.g. in high action footage.

[cyzs233]

@cyzs233 interesting. I wonder where the "5 frames" comes from. Sometimes keyframes are also spaced less than 5 frames apart, e.g. in high action footage.

Indeed, tigher keyframe margins exist. IMO, from a user's perspective, the deterioration in those extreme cases usually can't be noticed by a user(merely a handful of frames differ from the desired position after all). If they want absolutely precise frame control, they would have used AviSynth/vapoursynth at the beginning instead of lossless-cut.

[Hark0nnen]

I have been cutting some HEVC MKVs, and after some trial and error found that to achieve a precise cut i have to use "normal cut" mode and set segment start point on a frame before the desired start key-frame, and for segment end i need to set it to 2 frames before the desired end frame

[mifi]

I just tried cutting a HEVC mkv and from my experience, 4 frames after the keyframe time seems to be correct. Doesn't matter if I disable all other tracks or not. If only we could find a way to determine this number...

[mifi]

next version will have a new function that allows auto-aligning all segments to keyframes with the press of a menu button.

[mifi]

next nightly build will have a new Export Option called "Shift all start times", it can be used to automatically shift all segment start times forward by one or more frames before cutting (up to 10 frames). This can be useful if the output video starts from the wrong (preceding) keyframe.

[ac615223s5]

why not set it to one frame before the next keyframe?

[ac615223s5]

even a 20 frame offset is insufficient for video at 240fps

[Dragodraki]

Here I am pointing this issue to be still present and a solution is needed ("Smart Cut" is not helpful).

It's really frustrating we only can decide between "keyframe cut" with parts of video we won't have or parts missing and "normal cut" which leads the output to having audio but no video before the first keyframe passed. "Smart Cut" should had been the solution to it, but the output shows a matrix-like repetition of the first secconds (the first iteration has artifcats in it), no matter the codec, the file type, the cut point and the audio/video export quality.
Example: My webcam Logitech Brio Stream 4K is one of the few that support 1080p with 60fps and not only 30 (you'll realize the difference, believe me). But with 60fps my videos have so few keyframes its almost impossible to perform cuts, some records have only 1 keyframe in 20 secconds!!!. As if you can imagine deciding of define a cut up to +/- 20 secconds is not sufficient. Of course its not the fault of LLC as other tools have those limits too.

Conclusion:
Please, please, please - fix "Smart Cut" as its not working in its current state (creates matrix-like repetitions) or add another solution so LosslessCut could add a keyframe for normal cuts (between keyframes) without distorting audio/video track.

---

[Another request: One general thing: The user should definitely be told by LLC that he would not be able to cut AND separate/merge/change segements in one session. Export options like "Chapter only" and "Separate files" are not very informative for beginners - I know but others might not. Even better would be the tool could handle both of them in the first place.]

UPDATE:
I didn't know about exact frame-cutting with re-encoding by tools like "Avidemux" were possible in the first place. Surely, you cannot make 100% sure that you might not have a tiny quality loss, but in my case this wasn't notificable.
Anyway, if you find a method that would allowing us to use exact frame cutting and smooth transitions when merging in LosslessCut would be urgently needed. But at least I have an alternative until then now.

[DGrv]

As a short answer to your concern because I had in the past the same: add keyframes to your videos before processing them in losslesscut.

Here some example of code to do this:

In shell:

keyframe () {
    # add keyframe to get 1 per second to all files in folder
    for i in *mp4; do
       fr=$(exiftool -n -T -VideoFrameRate -s3 $i)
       fr=$(calc -d "round($fr)")
       fr=$(echo $fr)
       nname="$(basename $i .mp4)___old_KF.mp4"
       mv "$i" "$nname"
       cecho -g "Add keyframes $i:"
       ffmpeg -stats -loglevel error -i "$nname" -vcodec libx264 -x264-params keyint=$fr:scenecut=0 -acodec copy "$i"
    done
}

If you look for something in windows via a batch file look this code here.

For more info the important cmd is:
ffmpeg -stats -loglevel error -i input.mp4 -vcodec libx264 -x264-params keyint=<number of frames btw keyframes>:scenecut=0 -acodec copy output.mp4

[Pombaroom]

Hi. I had the same problem with mkv files. When I pick starting I-frame and cut - I got a video that starts from previous I-frame from what I picked.
The solution for me is...
to disable "ffmpeg" avoid_negative_ts option. It is at the bottom of the list after you click "Export"

Not sure what it does and why it is enabled (I'm noob)... but disabling it helps.
But it messes with MKV fps info. Original fps is Constant 25fps, and after this operation it is shown as 2525.198 fps Variable. But playback is correct. If I disable Metadata in Export settings, then fps isn't shown at all (only Original one is shown) but still Variable.

[Dragodraki]

@DGrv
Thanks, but actually an implementation in LosslessCut would be nice to do so (maybe even automatically for every few secconds if not already, as it is needed in general). I already have five different programs for video editing and if being in a rush, fiddling with console commands would be heavy unnerving. But thanks for your code, maybe the dev could add this to LLC.

@Pombaroom
As afraid as I am, I already tried all possible combinations with ffmpeg ts in export options and all produced these errors for me.

[parasiteoflife]

There is a reason why LosslessCut is named like that, isn't it? Why do you all keep requesting lossy features? You should start looking for another program

[Dragodraki]

Yes, I forget to mention the requested solution should be lossles. Thanks for remembering me.

[DashingDave]

I get a massive discrepancy when I split adjacent segments into separate files. When playing the resulting files, segment 1 stops at the correct keyframe, but segment 2 starts 117 keyframes prior to the end of segment 1. (There's a more moderate gap between segments 2 and 3, with segment 3 starting 49 early, but seems like a lot when the export adjustment only allows for a +10 frame adjustment....)