Videos do not render in Firefox on Android

[brianpeiris]

Description
Videos seem to play, but do not actually render in Firefox on Android. They just display as a black rectangle. This appears to be the case with all video formats.

To Reproduce
Steps to reproduce the behavior:

1. Use Firefox on Android
2. Create a room
3. Add a video to it, like https://brianpeiris.github.io/test-media/big-buck-bunny-360p-short.mp4
4. See that the video does not render

Expected behavior
Videos should render in Firefox on Android!

Hardware

• Device: Phone
• OS: Android
• Browser: Firefox

┆Issue is synchronized with this Jira Task

[takahirox]

Oh... this is a serious problem. I would take this ticket unless someone already started to tackle it.

[takahirox]

I confirmed the issue. I reproduced this problem on Android Firefox Nightly version 90.0a1 on Samsung Galaxy S8+. And also confirmed the videos play on Android Chrome.

[takahirox]

I did remote debugging and saw the CORS error on Android Firefox Nightly.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://hubs.mozilla.com/. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

But I also see this error Android Chrome, Desktop Chrome, and Desktop Firefox but videos play on them. I guess Android Firefox CORS is more restricted? I'll keep investigating more...

Update: I talked to @netpro2k about the CORS error but the error may be unrelated to video stuffs.

[takahirox]

TLDR: The current Android Firefox seems to have a problem to upload video (mp4, webm, and maybe other formats except for ogv) to texture with gl.texImage2D(). I will report it to bugzilla.

---

As I wrote in #4224 (comment), the CORS error doesn't seem to be related to the video play problem.

I investigated the problem again and found the following error messages in the console with remote debugging.

WebGL warning: tex(Sub)Image[23]D: Resource has no data (yet?). Uploading zeros. threejs-allocation-patches.js:9:8
WebGL warning: drawElementsInstanced: Tex image TEXTURE_2D level 0 is incurring lazy initialization. three.module.js:16782:5

I first thought the messages were originally caused by the CORS error but as I mentioned the CORS error doesn't seem to be related the video problem. So it seems to simply fail to upload the video data to texture with gl.texImage2D().

I sought simpler examples on which I can reproduce the problem and found the Three.js examples.

https://threejs.org/examples/webgl_video_kinect.html

In this example, I see the same texImage2D() errors. An interesting thing is a video plays in materials video example

https://threejs.org/examples/webgl_materials_video.html

The reason why the video plays seems the format is ogv. The former one uses mp4 and webm.

(The latter example requires a user gesture first while the former one doesn't, so I first thought the reason can be the user gesture but the former one still couldn't play the video even though I added a user gesture request.)

I tried to place ogv video in Hubs room and it plays on Android Firefox.

So my conclusion is the current Android Firefox seems to have a problem to upload video (mp4, webm, and maybe other formats except for ogv) to texture with gl.texImage2D(). I will report it to bugzilla tomorrow and also push Firefox team internally.

If it will take time to resolve this problem, we might want to convert videos to ogv for Android Firefox on the fly as workaround if possible.

[takahirox]

I reported the problem to Android Firefox mozilla-mobile/fenix#19399

[brianpeiris]

I can confirm that the webgl_materials_video example breaks in Firefox Android if you only include the sintel.mp4 video file.

[takahirox]

Yeah, I confirmed that too. ogv video texture seems to work fine even on the current Android Firefox so perhaps workaround we can do on our ends so far may be using ogv.

[brianpeiris]

Also confirmed the issue with this simpler webgl video test, just to make sure it wasn't specific to three.js. https://webglsamples.org/video/video.html

I think using ogv is not viable since it would not be feasible to convert video on the fly at scale.

[takahirox]

I made a minimal test reproducing the problem and shared it with Android Firefox and Geckoview teams.

https://takahirox.github.io/texImage2D-gl-test/index.html

it would not be feasible to convert video on the fly at scale.

Yeah, it isn't probably a good option. But I couldn't come up with any other workarounds for now.

The best solution is the problem will be fixed on Android Firefox end. I will try to work closely to Android Firefox and Geckoview teams.

[brianpeiris]

Sounds great. Thanks Takahiro!

[brianpeiris]

The Fenix bug was migrated to https://bugzilla.mozilla.org/show_bug.cgi?id=1709726

[brianpeiris]

I actually cannot reproduce this anymore on my Galaxy S21 with Firefox 99.

I'm not sure why it's working in Hubs, because some of the test cases here still fail to work in Firefox. So there is probably still a bug in Firefox, but we seemed to have worked around it somehow. Perhaps a three.js update fixed it for us.

I'll close this, because I'm pretty sure Hubs is not affected anymore, but feel free to re-open someone else can repro.