Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

strange canvas output if graphical is true on chrome for android #521

Closed
philolt opened this issue Oct 20, 2019 · 9 comments
Closed

strange canvas output if graphical is true on chrome for android #521

philolt opened this issue Oct 20, 2019 · 9 comments
Assignees
@robertleeplummerjr

Comments

@philolt
Copy link

philolt commented Oct 20, 2019

What is wrong?

I tried to run your yellifish canvas direct draw example on my mobile phone (Samsung Galaxy S7) with latest Android updates. If CPU mode is enabled everything works like expected (slow). If I enable the GPU the output is shuffled. It seems like the texture coordinates are shifted.
expected (cpu):
screenshot_1
observed (gpu):
screenshot

I tried another example where an color gradient is drawn directly to the canvas. Same behavior on this specific device.
expected (cpu):
screenshot_2
observed (gpu):
screenshot_3

Where does it happen?

Chrome for Android on my Samsung Galaxy S7. Didn't try another Android Device. Can try it tomorrow. Linux Desktop works like expected.

How do we replicate the issue?

If you have such a device, you can use the Yellifish Video example. Or you can use the second example below:

<!DOCTYPE html>
<meta charset="utf-8">

<body>
<script src="js/gpu.js"></script>
<script>
const gpu = new GPU({mode: "gpu"});
const render = gpu.createKernel(function(x) {
  this.color(this.thread.x/500, this.thread.y/500, x[0]);
})
  .setOutput([500, 500])
  .setGraphical(true)
render([0.4])
const canvas = render.canvas;
document.getElementsByTagName('body')[0].appendChild(canvas);
</script>

How important is this (1-5)?

5 (my subjective importance) because I cannot use gpu.js for my project with this behaviour. I want to apply video effects to the device camera. Without this behaviour the performance of gpu.js would be great for this project. It is an awesome library! Thanks for that! :)

Expected behavior (i.e. solution)

There seems to be an anomaly in the canvas output of this (maybe more) devices. It is not the input because of the second test (computed output). It seems like there is a special planar color space in the output texture, so that the uv-mappings doesn't apply to this device (only a slight guess). I have tried other THREE.js video texture examples on this device to proof it is possible and no hardware related problem. The examples i found worked like expected.

Other Comments

Thank you for this great library!
@robertleeplummerjr
Copy link
Member

I believe this is the highest priority. I'm away right now but curious if using useLegacyEncoder set to true has any effect.

@robertleeplummerjr robertleeplummerjr self-assigned this Oct 20, 2019
@philolt
Copy link
Author

philolt commented Oct 20, 2019

Hi,
thanks for the fast response. Just ran:

<!DOCTYPE html>
<meta charset="utf-8">

<body>
<script src="js/gpu.js"></script>
<script>
const gpu = new GPU({mode: "gpu"});
const render = gpu.createKernel(function(x) {
  this.color(this.thread.x/500, this.thread.y/500, x[0]);
})
  .setOutput([500, 500])
  .setGraphical(true)
  .setUseLegacyEncoder(true)
render([0.4])
const canvas = render.canvas;
document.getElementsByTagName('body')[0].appendChild(canvas);
</script>

Produces the exact same image like above.
Best regards!

@philolt
Copy link
Author

philolt commented Oct 21, 2019 •
edited
Loading

Hi,
Just tried the code on 3 different android devices. None of these devices displays the output correctly. First one was a Huawei P20 Pro (Android 9). The output was the same like that one on the Galaxy. On the other devices the output was different.
Blackview A10 Android 7.0:
screenshot_5

Samsung Galaxy S4 Android 5.0.1:
screenshot_4

I also tried an Iphone Xs. The results were the same like the Galaxy and the Huawei.
Hope the results helps.
Best regards!

@RobinKa
Copy link

RobinKa commented Oct 21, 2019 •
edited
Loading

Try setting the tactic to performance (requires a newer version of GPU.js from github (eg. develop or master branch), the one on npm is too old). That made my gradient output smooth on iPad 6 and Nexus 6. Any other setting such as the default "balanced" produces artifacts.

@philolt
Copy link
Author

philolt commented Oct 21, 2019 •
edited
Loading

Yay, that's working! Huawei, Samsung and Iphone is now okay. The other device (Blackview A10 Android 7.0) doesn't work. Is it possible to determine in which specific cases these settings should be applied?
Thank you!

@RobinKa
Copy link

RobinKa commented Oct 21, 2019 •
edited
Loading

https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_best_practices mentions that highp (which performance will use) doesn't work on older devices, maybe that's why. It also mentions using getShaderPrecisionFormat is possible to check for support.

// Edit: Perhaps it is better to use integers rather than floats where possible (although I have no idea how to specify types with this library)? I'd assume these accuracy issues would then disappear.

@robertleeplummerjr
Copy link
Member

I'll have a good chunk today I can look into this and resolve it, fyi.

@robertleeplummerjr robertleeplummerjr mentioned this issue Nov 9, 2019
Open
@robertleeplummerjr
Copy link
Member

This has to be related: mapbox/webgl-wind#12

@robertleeplummerjr
Copy link
Member

Have a fix, will have this added later today.

ammyk9 pushed a commit to ammyk9/gpu.js that referenced this issue Aug 8, 2024
@robertleeplummerjr @ammyk9
feat: Added features and fixes for the following issues:
6f66395 
...it kind of snowballed from some needs
Fixes gpujs#521 - If `tactic` is not set, check precision allowed from WebGL, and automatically change based off needs, otherwise use value from `tactic`.
Fixes gpujs#535 - Internally check if texture from argument is the same as output, if so, clone this texture, and then clean it up after the kernel runs.
Fixes gpujs#536 - Normalize all declarations to non-destructured, and then parse
Fixes gpujs#537 - Change logic
Fixes gpujs#538 - Found the GL script that would work, and reduced the methods to use it
Fixes gpujs#539 - Found a better way of testing random, and this gives me an error for 1 in 10 runs, acceptable

Some refactoring for less duplicate code and documentation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@robertleeplummerjr robertleeplummerjr

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@robertleeplummerjr @RobinKa @philolt