[3.x] Support WebM seeking operation #57744

John-Gdi · 2022-02-07T08:40:11Z

Support webm video seeking operation with minimal modification. Currently, it only support 3.x version since master 4.0 version has abandoned webm format support.

Modified files:
1.modules/webm/video_stream_webm.cpp
2.thirdparty/libsimplewebm/WebMDemuxer.cpp
3.thirdparty/libsimplewebm/WebMDemuxer.hpp

Kapture.2022-02-06.at.11.49.43.mp4

jamie-pate · 2022-03-03T17:47:45Z

see #14430

John-Gdi · 2022-03-09T09:44:36Z

see #14430

Thanks for your feedback, since I'm not an expert in video decoding, but I'll try my best to fix this issue.

John-Gdi · 2022-03-10T07:18:19Z

see #14430

@jamie-pate i have fix the issue, and please verify it on 1ede4ca. Thanks!

Atem1995 · 2022-03-11T17:55:07Z

@John-Gdi nice work！ Do you know how to fix VideoStreamWebm playing slowly in Html5? I really want to know where the problem is.

Calinou · 2022-03-11T19:00:04Z

@John-Gdi nice work！ Do you know how to fix VideoStreamWebm playing slowly in Html5? I really want to know where the problem is.

Video decoding will always perform poorly in HTML5 for two reasons:

yasm assembly optimizations cannot be used.
Decoding is performed on the CPU, via a pure C WebAssembly layer, which is slow.

There isn't a good way to work around this, as WebAssembly cannot make use of the browser's native video decoding functionality (which would allow for hardware-accelerated decoding). Theora might perform better as it's less demanding on the CPU, but I wouldn't expect it to be fantastic there either.

jamie-pate · 2022-03-11T23:25:50Z

thirdparty/libsimplewebm/VPXDecoder.cpp

@@ -62,7 +62,11 @@ VPXDecoder::VPXDecoder(const WebMDemuxer &demuxer, unsigned threads) :
 	}

 	m_ctx = new vpx_codec_ctx_t;
-	if (vpx_codec_dec_init(m_ctx, codecIface, &codecCfg, m_delay > 0 ? VPX_CODEC_USE_FRAME_THREADING : 0))
+
+	// disable multi thread for decoder to make some vp8 and vp9 video files work properly


See also here where chromium removed the flag: https://bugs.chromium.org/p/webm/issues/detail?id=1395

jamie-pate · 2022-03-12T17:23:44Z

@John-Gdi nice work！ Do you know how to fix VideoStreamWebm playing slowly in Html5? I really want to know where the problem is.

You would likely need a whole new way to play video in the html version.. using the built in video element to create a texture for webgl (like this https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/Tutorial/Animating_textures_in_WebGL ). godot's OS.native_video_play https://docs.godotengine.org/en/stable/classes/class_os.html#class-os-method-native-video-play call would be another way, but docs say they are only currently implemented for ios

You could possibly just call out to the browser with javascript to play the video fullscreen with the video element though

see #17495

Atem1995 · 2022-03-13T05:03:23Z

Oh, god. I can't handle this work now.(crying...... :< )

jamie-pate · 2022-03-14T23:01:34Z

make the attached vp8 and vp9 webm file plays properly <- this commit does not appear to create a significant performance regression, and I can confirm that the video which is broken in the reproduction plays correctly with up to 4x video streams being decoded concurrently. (probably need a better way to measure performance though than decoding 4 concurrent video streams)

perf.log i7-8750H
perf-u620-i7-8650U.log

When playing a file that was encoded with vp9 and did not fail to playback in the previous version of godot the frame rate was roughly the same for both commits, and only slightly slower than the godot-videodecoder plugin...

i7-8750H
GODOT=./libvp9-thread-godot.x11.opt.tools.64
Godot Engine v3.3.5.rc.custom_build.63ac434b3 - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder

videos:0 fps:7673 ts:10091
videos:1 fps:6419 ts:15090
videos:2 fps:4412 ts:20090
videos:3 fps:4037 ts:25090
videos:4 fps:3663 ts:30090
videos:4 fps:3837 ts:35090
videos:4 fps:3711 ts:40090
videos:3 fps:4109 ts:45090
videos:2 fps:4364 ts:50090
videos:1 fps:4635 ts:55090
videos:0 fps:4994 ts:60090

GODOT=./libvp9-nothread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.1ede4ca4b - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder
videos:0 fps:1 ts:1
videos:0 fps:9063 ts:5111
videos:0 fps:8985 ts:10112
videos:1 fps:4349 ts:15112
videos:2 fps:6185 ts:20112
videos:3 fps:3474 ts:25112
videos:4 fps:2253 ts:30112
videos:4 fps:2986 ts:35112
videos:4 fps:4589 ts:40112
videos:3 fps:3449 ts:45112
videos:2 fps:3934 ts:50112
videos:1 fps:4388 ts:55112
videos:0 fps:4907 ts:60112

i7-8650U

GODOT=./libvp9-nothread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.1ede4ca4b - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder
videos:0 fps:2945 ts:6353
videos:0 fps:2898 ts:11353
videos:1 fps:950 ts:16354
videos:2 fps:492 ts:21355
videos:3 fps:333 ts:26354
videos:4 fps:232 ts:31360
videos:4 fps:238 ts:36357
videos:4 fps:234 ts:41359
videos:3 fps:274 ts:46359
videos:2 fps:304 ts:51355
videos:1 fps:336 ts:56358
videos:0 fps:366 ts:61359

GODOT=./libvp9-thread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.0c58454da - https://godotengine.org
videos:0 fps:2834 ts:6338
videos:0 fps:2554 ts:11339
videos:1 fps:954 ts:16340
videos:2 fps:515 ts:21339
videos:3 fps:366 ts:26342
videos:4 fps:279 ts:31342
videos:4 fps:278 ts:36341
videos:4 fps:281 ts:41340
videos:3 fps:299 ts:46342
videos:2 fps:320 ts:51342
videos:1 fps:341 ts:56345
videos:0 fps:363 ts:61344

Complete name                            : char-waypoints-2022-01-27_20.08.38.webm
Format                                   : VP9
Width                                    : 1 434 pixels
Height                                   : 660 pixels

Complete name                            : vp8-encoded.webm
Format                                   : VP8
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels

Complete name                            : vp9-encoded.webm
Format                                   : VP9
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels

John-Gdi · 2022-03-16T07:32:01Z

make the attached vp8 and vp9 webm file plays properly <- this commit does not appear to create a significant performance regression, and I can confirm that the video which is broken in the reproduction plays correctly with up to 4x video streams being decoded concurrently. (probably need a better way to measure performance though than decoding 4 concurrent video streams)

perf.log i7-8750H perf-u620-i7-8650U.log

When playing a file that was encoded with vp9 and did not fail to playback in the previous version of godot the frame rate was roughly the same for both commits, and only slightly slower than the godot-videodecoder plugin...

i7-8750H
GODOT=./libvp9-thread-godot.x11.opt.tools.64
Godot Engine v3.3.5.rc.custom_build.63ac434b3 - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder

videos:0 fps:7673 ts:10091
videos:1 fps:6419 ts:15090
videos:2 fps:4412 ts:20090
videos:3 fps:4037 ts:25090
videos:4 fps:3663 ts:30090
videos:4 fps:3837 ts:35090
videos:4 fps:3711 ts:40090
videos:3 fps:4109 ts:45090
videos:2 fps:4364 ts:50090
videos:1 fps:4635 ts:55090
videos:0 fps:4994 ts:60090

GODOT=./libvp9-nothread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.1ede4ca4b - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder
videos:0 fps:1 ts:1
videos:0 fps:9063 ts:5111
videos:0 fps:8985 ts:10112
videos:1 fps:4349 ts:15112
videos:2 fps:6185 ts:20112
videos:3 fps:3474 ts:25112
videos:4 fps:2253 ts:30112
videos:4 fps:2986 ts:35112
videos:4 fps:4589 ts:40112
videos:3 fps:3449 ts:45112
videos:2 fps:3934 ts:50112
videos:1 fps:4388 ts:55112
videos:0 fps:4907 ts:60112

i7-8650U

GODOT=./libvp9-nothread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.1ede4ca4b - https://godotengine.org
Playing char-waypoints-2022-01-27_20.08.38.webm 4x with builtin decoder
videos:0 fps:2945 ts:6353
videos:0 fps:2898 ts:11353
videos:1 fps:950 ts:16354
videos:2 fps:492 ts:21355
videos:3 fps:333 ts:26354
videos:4 fps:232 ts:31360
videos:4 fps:238 ts:36357
videos:4 fps:234 ts:41359
videos:3 fps:274 ts:46359
videos:2 fps:304 ts:51355
videos:1 fps:336 ts:56358
videos:0 fps:366 ts:61359

GODOT=./libvp9-thread-godot.x11.opt.tools.64
Godot Engine v3.5.beta.custom_build.0c58454da - https://godotengine.org
videos:0 fps:2834 ts:6338
videos:0 fps:2554 ts:11339
videos:1 fps:954 ts:16340
videos:2 fps:515 ts:21339
videos:3 fps:366 ts:26342
videos:4 fps:279 ts:31342
videos:4 fps:278 ts:36341
videos:4 fps:281 ts:41340
videos:3 fps:299 ts:46342
videos:2 fps:320 ts:51342
videos:1 fps:341 ts:56345
videos:0 fps:363 ts:61344

Complete name                            : char-waypoints-2022-01-27_20.08.38.webm
Format                                   : VP9
Width                                    : 1 434 pixels
Height                                   : 660 pixels

Complete name                            : vp8-encoded.webm
Format                                   : VP8
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels

Complete name                            : vp9-encoded.webm
Format                                   : VP9
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels

Thanks for your test, and hope this pr can satisfy some pepole's needs

Atem1995 · 2022-04-07T04:40:52Z

@John-Gdi nice work！ Do you know how to fix VideoStreamWebm playing slowly in Html5? I really want to know where the problem is.

Video decoding will always perform poorly in HTML5 for two reasons:

yasm assembly optimizations cannot be used.

Decoding is performed on the CPU, via a pure C WebAssembly layer, which is slow.

There isn't a good way to work around this, as WebAssembly cannot make use of the browser's native video decoding functionality (which would allow for hardware-accelerated decoding). Theora might perform better as it's less demanding on the CPU, but I wouldn't expect it to be fantastic there either.
@Calinou
Can the technology in the link below make it possible to play webm videos smoothly?
https://github.com/GoogleChromeLabs/webm-wasm

jamie-pate · 2022-04-07T05:58:43Z

Can the technology in the link below make it possible to play webm videos smoothly?
https://github.com/GoogleChromeLabs/webm-wasm

"This is a proof-of-concept and not a production-grade library."

It looks like it's for encoding video files from raw frames... Also probably not real time

Atem1995 · 2022-04-07T06:09:31Z

@jamie-pate @Calinou
Mainly I want to be able to play webm video smoothly in HTML5, but can't find any technology to replace it, although Ogg Theora video does what I want, it's too low definition. I'm wondering if it's possible to pre-decode webm-formatted videos while packing. . .

akien-mga · 2022-04-07T07:13:40Z

Ogg Theora shouldn't be low definition, it depends on how you encoded the video, like any encoder.

Atem1995 · 2022-04-07T08:04:45Z

Is it because I set it wrong? The resolution of the video is not as good as the webm format.
@akien-mga

Calinou · 2022-04-07T17:28:47Z

Your video encoding bitrate is set to 6 Mb/s, which is quite low by today's standards, especially for Ogg Theora which is less efficient than H.264 or VP9. For comparison, 1080p60 H.264 video on YouTube is around 10 Mb/s, but even that is considered low quality nowadays (and that's for a more efficient format).

To make things worse, you're using the Realtime encoding speed preset, which favors speed over quality (for a given bitrate). Use a encoding slower preset to get better quality with the same bitrate – it doesn't affect decoding speed.

Atem1995 · 2022-04-08T03:30:13Z

Your video encoding bitrate is set to 6 Mb/s, which is quite low by today's standards, especially for Ogg Theora which is less efficient than H.264 or VP9. For comparison, 1080p60 H.264 video on YouTube is around 10 Mb/s, but even that is considered low quality nowadays (and that's for a more efficient format).

To make things worse, you're using the Realtime encoding speed preset, which favors speed over quality (for a given bitrate). Use a encoding slower preset to get better quality with the same bitrate – it doesn't affect decoding speed.

Great, after modifying the settings, the effect is much better now.

golddotasksquestions · 2022-08-05T22:33:30Z

The current lack of seek() for the VideoPlayer node is what makes this node pretty much useless for me. From my tests there is no way to work around this issue.

Also all previous proposals and issues have been closed.

I'm very happy to see someone is working on this, may I ask what is left for it to be merged? Are there any plans for ogv seek support as well?

Calinou · 2022-08-06T02:02:00Z

I'm very happy to see someone is working on this, may I ask what is left for it to be merged? Are there any plans for ogv seek support as well?

Please test the PR locally and report if it works 🙂

Are there any plans for ogv seek support as well?

This was being worked on in #32539, but it needs to be salvaged.

golddotasksquestions · 2022-08-06T21:16:26Z

@Calinou

Please test the PR locally and report if it works

I would love to, but the Windows artifacts are apparently already expired. And I don't feel competent enough to compile and build Godot with these changes.

YeldhamDev · 2022-08-15T17:08:23Z

I tested it on my end (GNU/Linux), and it doesn't appear to work. The video just goes back to the beginning.
I'm using this file for testing: https://filesamples.com/samples/video/webm/sample_960x400_ocean_with_audio.webm

Calinou · 2023-11-20T10:58:09Z

@John-Gdi Could you look into testing your PR with the file linked by YeldhamDev above?

Support WebM seeking operation on 3.x

0c58454

John-Gdi requested review from a team as code owners February 7, 2022 08:40

akien-mga changed the title ~~Reopen Support WebM seeking operation on 3.x~~ [3.x] Support WebM seeking operation Feb 7, 2022

akien-mga added enhancement topic:core labels Feb 7, 2022

akien-mga added this to the 3.5 milestone Feb 7, 2022

akien-mga added the topic:thirdparty label Feb 7, 2022

jamie-pate mentioned this pull request Mar 3, 2022

VideoStreamWebm VP9 video stops playing #58718

Open

make the attached vp8 and vp9 webm file plays properly

1ede4ca

John-Gdi mentioned this pull request Mar 11, 2022

VideoPlayer set_stream_position() doesn't work (seeking is not implemented) #14430

Closed

jamie-pate reviewed Mar 11, 2022

View reviewed changes

akien-mga mentioned this pull request Jul 2, 2022

Support WebM seeking operation #57690

Closed

akien-mga removed this from the 3.5 milestone Jul 2, 2022

akien-mga added this to the 3.x milestone Jul 2, 2022

akien-mga force-pushed the 3.x branch from beb5ae6 to abaa76f Compare August 25, 2022 12:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[3.x] Support WebM seeking operation #57744

[3.x] Support WebM seeking operation #57744

John-Gdi commented Feb 7, 2022

jamie-pate commented Mar 3, 2022

John-Gdi commented Mar 9, 2022

John-Gdi commented Mar 10, 2022

Atem1995 commented Mar 11, 2022

Calinou commented Mar 11, 2022 •

edited

Loading

jamie-pate Mar 11, 2022

jamie-pate commented Mar 12, 2022 •

edited

Loading

Atem1995 commented Mar 13, 2022

jamie-pate commented Mar 14, 2022 •

edited

Loading

John-Gdi commented Mar 16, 2022

Atem1995 commented Apr 7, 2022

jamie-pate commented Apr 7, 2022

Atem1995 commented Apr 7, 2022

akien-mga commented Apr 7, 2022

Atem1995 commented Apr 7, 2022

Calinou commented Apr 7, 2022 •

edited

Loading

Atem1995 commented Apr 8, 2022

golddotasksquestions commented Aug 5, 2022

Calinou commented Aug 6, 2022 •

edited

Loading

golddotasksquestions commented Aug 6, 2022 •

edited

Loading

YeldhamDev commented Aug 15, 2022

Calinou commented Nov 20, 2023

[3.x] Support WebM seeking operation #57744

Are you sure you want to change the base?

[3.x] Support WebM seeking operation #57744

Conversation

John-Gdi commented Feb 7, 2022

jamie-pate commented Mar 3, 2022

John-Gdi commented Mar 9, 2022

John-Gdi commented Mar 10, 2022

Atem1995 commented Mar 11, 2022

Calinou commented Mar 11, 2022 • edited Loading

jamie-pate Mar 11, 2022

Choose a reason for hiding this comment

jamie-pate commented Mar 12, 2022 • edited Loading

Atem1995 commented Mar 13, 2022

jamie-pate commented Mar 14, 2022 • edited Loading

John-Gdi commented Mar 16, 2022

Atem1995 commented Apr 7, 2022

jamie-pate commented Apr 7, 2022

Atem1995 commented Apr 7, 2022

akien-mga commented Apr 7, 2022

Atem1995 commented Apr 7, 2022

Calinou commented Apr 7, 2022 • edited Loading

Atem1995 commented Apr 8, 2022

golddotasksquestions commented Aug 5, 2022

Calinou commented Aug 6, 2022 • edited Loading

golddotasksquestions commented Aug 6, 2022 • edited Loading

YeldhamDev commented Aug 15, 2022

Calinou commented Nov 20, 2023

Calinou commented Mar 11, 2022 •

edited

Loading

jamie-pate commented Mar 12, 2022 •

edited

Loading

jamie-pate commented Mar 14, 2022 •

edited

Loading

Calinou commented Apr 7, 2022 •

edited

Loading

Calinou commented Aug 6, 2022 •

edited

Loading

golddotasksquestions commented Aug 6, 2022 •

edited

Loading