Benchmarking

[limzykenneth]

Hi, just want to reach out here about benchmarking the performance of this library against p5.js. I'm currently maintaining a benchmarking repo for p5.js here (it is extracted from the core repo a short while ago and partly rewritten) and would like to hear if you are interested in using it for benchmarking performance.

It is a slow work in progress and I only work on it when I have time left over but I have some plans for it. The setup uses benchmark.js + Karma and runs on headless Firefox and Chrome. To achieve consistency and comparability it is also run on a CI environment before the result is published at https://limzykenneth.github.io/p5-benchmark/

I originally work on it to have something I can benchmark my WebAssembly implementation of some p5.js functions so the ability to benchmark different libraries are already in consideration. If you are interested in being included, let me know so and also which areas/functions you want to be benchmarked against the main library. 😄

[LingDong-]

Hi @limzykenneth,

Thanks! I’d definitely be interested in using your benchmark! So far I’ve only tested q5’s performance by running it on my laptop browser and watching the FPS, and I only covered a couple functions that I know are significantly faster — so I think a more comprehensive and professional benchmarking is going to be very helpful for understanding, and potentially fixing and/or improving performance for q5 :)

I think maybe we can start with the ones listed in the “Benchmarks” section in the README first? Eventually I think the more that’s covered the better, but I’m not familiar with benchmark.js or Karma yet so I’d like to learn by starting with a small subset.

BTW, I too have experimented a bit with the idea of implementing some p5 functions in WebAssembly. I was writing the assembly directly by hand, having implemented noise() and randomGaussian() here. I discovered that though my wasm implementations run some 5-10x times faster then JS when called small number of times (1-1000), the JavaScript JIT is probably smart enough to optimize on the fly so that when calling millions of times in a tight loop the performance becomes too similar. So I ended up not including it in q5 :P

[limzykenneth]

I still need to solve a few logistical problems first but I'll include benchmarks for q5 once I'm done.

p5.wasm is mostly for me to practice Rust with. It tries to be idiomatic Rust and use Rust standard library or simple crates when available. My tests so far shown that performance wise it tends to be slow when native JS functions are available and marginally faster when complex algorithm is required so overall isn't that promising in terms of seeking maximum performance.

[limzykenneth]

I'm ready to add some q5 functions to the benchmark now, can you share the test code you used for your test so I know I'm testing the right thing?

[LingDong-]

Hi @limzykenneth,

Great, thanks! I've organized my tests in this glitch demo: https://glitch.com/edit/#!/q5-bench

In the project there's a list of sketch01.js sketch02.js... Those are the individual tests, uncomment one of their links in index.html to try it. p5/q5 library can also be toggled in index.html by similarly comment/uncommenting the cdn link. (Sorry I didn't make a GUI)

The tests cover the items I listed in "Benchmark" section of the repo. All the filter() tests are in the same file, they can be comment/uncommented to be tested individually.

Please let me know if there're any issues/questions! Thank you very much!!

[limzykenneth]

The benchmark is now up at https://limzykenneth.github.io/p5-benchmark/. I'm still to add comparison functionality but the benchmark tests are running smoothly now.