Use Skija (Java bindings to Skia) as a renderer

[processing-bot]

Created by: suyashcjoshi

Feature Request

I came across this awesome graphics library (shiny new from folks who build IntelliJ) that can perhaps be used to replace the current graphics library.

Have a look at the project README for examples - https://github.com/JetBrains/skija/ They mentions support for the following formats:

• Bitmap (Canvas)
• OpenGL
• Metal (example pending)
• Vulkan (in future)

[processing-bot]

In GitLab by @micycle1 on Nov 20, 2020, 13:33

This looks quite interesting, but it's possibly more comparable to Processing itself, than as a rendering mode for Processing.

[processing-bot]

Created by: benfry

This looks like a promising project, and potentially in better shape than the current mess in Java2D or JavaFX.

The path for implementing this would first be someone creating a Processing Library that makes use of it, and then over time, improving that to the point where it could be included as part of the core libraries.

(Edited for clarification)

[processing-bot]

In GitLab by @micycle1 on Nov 20, 2020, 23:43

I've just made a proof-of-concept here which plugs in to Processing's OPENGL mode, allowing calls to Skija bindings to draw in to the Processing sketch.

The next step would be to wrap up Skija bindings into some sort of PGraphicsSkia to create a Processing Skia renderer whose API matches Processing's existing renderers.

[processing-bot]

Created by: suyashcjoshi

Awesome..can't wait to try myself!

[processing-bot]

Created by: tonsky

it's possibly more comparable to Processing itself, than as a rendering mode for Processing

Hi, I am author of Skija :) I would say Skija is more like java.awt.Graphics2D or javafx.scene.canvas.GraphicsContext, and I think it might make a great rendering backend for Processing 4. Hard Skija dependencies are Java 11 and OpenGL context (optionally, it can render into bitmap too, but slower).

Happy to clarify further or dive into details. Feel free to ask anything or let me know if you need something supported, I will be happy to help.

[processing-bot]

In GitLab by @micycle1 on Dec 1, 2020, 13:25

@tonsky

Ok, here's a question: I was testing performance when rendering circles and ellipses across Processing's extant OPENGL mode (P2D), Skija (using the JOGL stage created by Processing, via processing-skia) and Processing's JavaFX renderer.

Performance is more-or-less the same between P2D and Skija (while JavaFX was much better). Is this what you'd expect? Alsom would Skija see performance gains by using a different backend (lwjgl for example)?

[processing-bot]

Created by: tonsky

Is this what you'd expect?

No, it should be on par with JavaFX at least, hopefully faster (at least it was when I was testing with clipping arbitrary shapes and shadows).

Can you give a rough estimate of what your scene looked like? How many circles, how big, were they transparent, that sort of thing?

Also, what machine did you test it on? Did it have a graphics card?

Alsom would Skija see performance gains by using a different backend (lwjgl for example)?

Probably not, as long as it’s OpenGL it shouldn’t matter. Metal might show different results (better or worse).

[processing-bot]

In GitLab by @micycle1 on Dec 1, 2020, 23:08

Can you give a rough estimate of what your scene looked like? How many circles, how big, were they transparent, that sort of thing?

10000 circles; radius = 4; this paint: Paint().setColor4f(new Color4f(0, 0, 0)).setAntiAlias(true); Windows 10; RX 5700

~55fps P2D; ~70fps Skija; ~300fps FX2D

[processing-bot]

Created by: tonsky

I am getting 130 fps on Skija (macOS, MacBook Pro (15-inch, 2019), Radeon Pro 560X 4 GB). 150 if I render to sRGB instead of P3.

For FX I am getting ~220 fps in sRGB, no idea how to turn on P3.

I guess FX is faster at drawing circles ¯\_(ツ)_/¯. Will do more thorough testing with text, gradients, shadows, clips this weekend.

I pushed code here: https://github.com/JetBrains/skija/blob/master/examples/lwjgl/src/main/java/org/jetbrains/skija/examples/lwjgl/BenchCirclesScene.java

and here: https://github.com/tonsky/java-graphics-benchmark/blob/master/src/fx/BenchCirclesDemo.java

[processing-bot]

Created by: tonsky

I finally wrote some benchmarks to compare Skija and JavaFX more closely.

In short: on some tasks, JavaFX is faster (drawing primitives, gradients, filling simple English text), while terribly slow on others (clips, shadows). I also encountered couple of problems (working with text, vsync, scaling, etc).

I guess there’s no right answer ¯\_(ツ)_/¯

Full details & how to repro here https://github.com/tonsky/java-graphics-benchmark#benchmarks

[processing-bot]

In GitLab by @micycle1 on Dec 18, 2020, 18:47

Cool. Any idea why Skia is so slow with rendering primitives by comparison? Is it overhead or something else?

[processing-bot]

Created by: tonsky

There’s some overhead in how JNI in Skija. JavaFX does batching for JNI calls. After that, on OpenGL side, I have no idea what’s going on.

[processing-bot]

Created by: benfry

Thanks for sharing your thoughts, @tonsky. Very helpful context.

[processing-bot]

Created by: benfry

For anyone interested in doing this, the route would be to create a renderer (ala PGraphicsJavaFX or PGraphicsOpenGL and others) that instead draws using Skijia.

Starting with PGraphicsJava2D would probably be easiest. You'd also need to create a PSurface object that works with Skijia to handle windows and mouse/keyboard input (or pass those through to a modified PSurfaceAWT if Skijia is set up to play nicely with AWT).

[processing-bot]

Created by: benfry

Closing this one for now since it's better handled in a third party (contributed) Library, at least to start. Could be fun but not something I have time for at the moment.