Dropped scenes if using --copy with split-video

[ghost]

Bug/Issue Description:
With ffmpeg is fine but mkvmerge eats some split.

Required Information:
scenedetect -i 1.mkv detect-content split-video -c

clip.zip

BUG_REPORT.txt

Computing Environment:

• OS: Windows 10 (using official build)

[Breakthrough]

Hi @notagainbutyeah;

Interesting find - it looks like this works when using ffmpeg stream copying instead of mkvmerge, I was able to get it to work - you can do that with the following command:

scenedetect -i 1.mkv detect-content split-video -a "-c:v copy -c:a copy"

Does that command yield the correct output on your end?

If so, I think it now makes sense to default to using ffmpeg for stream copy mode, and only fallback to mkvmerge when ffmpeg cannot be found. This was originally tracked in #77, but this now seems like a bug instead of an improvement, so I'll need to merge the two.

Thanks!

[ghost]

Hello, I get wrong split from that.

The first split end with the beginning of the second. The second looks fine.

[Breakthrough]

Unfortunately when using the copy mode, precise splits are impossible in a lot of cases since it requires a keyframe to be at the exact split point. This behavior is described in both the ffmpeg and mkvmerge documentation. I will double check if there is an option for mkvmerge that can handle this particular case, but if you require precise splits, you should avoid using stream copying.

If performance is your main concern, you can tweak the ffmpeg arguments to reduce processing time at expense of file size (same with quality, can even make it lossless). I'll leave this open for now though as I'd like to look more into this sample video to see exactly the distribution of I/P/B frames that is causing this issue.

[ghost]

That was just an example. If I split a whole episode I get so many not split parts. Or are they wrong splits in the first place?
Here's another clip: 2.zip
But if they are wrong splits, they seems a bit long to me. I see cases where I have few seconds of delayed split.

Anyway when you take out a possible wrong split the bug of this thead arise.

The concer about transcode is quality. Also is there a way to ask ffmpeg to get the same quality? Without ending with nonsense size.

[Breakthrough]

I had a look with ffprobe and it looks like there is only a single keyframe (I-frame) in the clip you provided (the first frame) which likely leads to this behavior. In particular, when using P and B frames, they may be stored within a container out of order (e.g. frame 4 may come before frame 3 when decoding the video). This causes issues when doing stream copying, as extra frames will be present and cannot be removed without re-encoding the video.

Thus, with ffmpeg, extra frames are included when doing stream copying. This is implicitly done with mkvmerge as well, but the mkv container has additional metadata indicating when the stream should stop being presented. Using mkvmerge also doesn't output the 2nd scene since there are no keyframes after/on the starting timecode. ffmpeg ignores this and simply copies the P and B frames from the output, which is why the first few frames may appear corrupted when trying to view the video.

This issue is typically worse with animated videos, since they generally have a much higher amount of P and B frames to reduce the encoded file size.

The only solution that works for all types of input videos consistently is to not use -c/--copy with split-video.

You should be able to get pretty good results by specifying -hq when using split-video and specifying --preset if you need to improve speed.

If you truly do need lossless, then you can specify split-video -crf 0 -preset slow, although this is typically unnecessary since specifying -hq will result in visually identical output for most cases (-hq is the same as crf 17 - if quality is still an issue, try -crf 10 or lower, although file size will increase). There are additional tweaks you can use specifically for animated content if you look into the ffmpeg documentation (you can specify any arguments you want to ffmpeg by using the -a option).

Also note that in some cases OpenCV has already been known to drop some frames which can lead to incorrect timecodes - see #213 for details. Hope this helps, and thanks for the additional clip.

[Breakthrough]

So after re-visiting the output, my analysis above might be slightly incorrect re: ffmpeg's case - there, the last 3 frames is a sequence of P -> B -> P. The video should end on the B-frame according to where the split is detected, but this is not possible, since actually decoding the B frame requires both frames adjacent to it. Thus an extra frame from the next scene is required to be included.

Try doing it this way:

scenedetect.py -i 1.mkv detect-content split-video -a "-c:v copy" -f $VIDEO_NAME-$SCENE_NUMBER.mkv

Remember to escape the $'s if you're on bash, e.g.:

scenedetect.py -i 1.mkv detect-content split-video -a "-c:v copy" -f \$VIDEO_NAME-Scene-\$SCENE_NUMBER.mkv

Does this solve the issue for you? Edit: I think I was looking at the wrong file - this still has some off by one issues... Let me know though what you think.

Depending on if it's better/worse, I might rework #77 to just get rid of using mkvmerge entirely, and just use ffmpeg always for doing steam copying (or swap the default to ffmpeg, since clearly it gives better behavior out of the box). Either way, marking this as a bug now since clearly --copy doesn't do the right thing out of the box here, but can if it uses ffmpeg by default.

[ghost]

I don't understand what is the difference with this new command, I can tell that the audio part is missing.

I am not able to run this since in Windows cmd $ get not escaped with .

[Breakthrough]

@notagainbutyeah the only difference is the extension that gets passed to ffmpeg. If the output file name ends in .mkv, it forces the output container to be Matroska.

[Breakthrough]

The default for -c/--copy in v0.6 will be to use ffmpeg which should avoid this issue. See #77 for details.

Available in the v0.6 beta.