feat: Benchmarks

[ianpurvis]

Adds support for building and running benchmarks.

Bullet 2.82 benchmarks can be run with:

$ node builds/benchmarks.js
$ node builds/benchmarks.wasm.js

A new benchmark tool bammo can be run with:

$ node builds/bammo.js
$ node builds/bammo.wasm.js 

Non-emscripten versions of each tool can also be built from inside the bammo folder:

$ cd bammo
$ cmake -B native
$ cmake --build native
$ ./native/benchmarks
$ ./native/bammo

[ianpurvis]

@kripken This was a quite a journey... per your recommendation in #322, I started working with the existing Bullet 2.82 benchmarks. They were not hard to get running in ammo master, but I couldn't build them against newer Bullet. It turns out that cli benchmarking has been very broken in Bullet since v2.83, when the gui example browser was introduced.

This PR is the result of fixing headless examples in Bullet 2.89, adding a new google benchmark cli, and then porting everything back to Bullet 2.82 and ammo master. My thought is that once this is merged, the remaining fixes for Bullet >= 2.83 benchmarking can be added to the chore/bullet-2.89 branch. Then we can benchmark each branche with the new cli and compare with the google benchmark compare.py tool.

I am calling this new cli bammo, short for benchmark ammo. I took care to match the bullet 2.82 benchmark functionality as closely as possible. It runs each benchmark for 200 frames, sampling physics step duration every 25 frames. I guess there is more useful data to be had, but hopefully this provides some continuity to the old benchmarks. One nice addition is FPS....

$ node builds/bammo.wasm.js --benchmark_counters_tabular=true
failed to open /proc/cpuinfo
2020-09-16T21:18:03-05:00
Running /Users/ian/Work/ianpurvis/ammo.js/builds/bammo.wasm.js
Run on (-1 X 1000.07 MHz CPU )
-------------------------------------------------------------------------------------------------------
Benchmark                            Time             CPU   Iterations        fps     frames      start
-------------------------------------------------------------------------------------------------------
3000 Boxes/iterations:1           5.46 ms         5.44 ms            1  183.715/s          1          0
3000 Boxes/iterations:1           3.93 ms         3.91 ms            1  255.432/s          1         25
3000 Boxes/iterations:1           5.02 ms         5.02 ms            1  199.315/s          1         50
3000 Boxes/iterations:1           5.24 ms         5.23 ms            1  191.358/s          1         75
3000 Boxes/iterations:1           7.87 ms         7.87 ms            1  127.125/s          1        100
3000 Boxes/iterations:1           12.1 ms         12.1 ms            1  82.4542/s          1        125
3000 Boxes/iterations:1           14.9 ms         14.9 ms            1  67.1437/s          1        150
3000 Boxes/iterations:1           19.5 ms         19.5 ms            1   51.393/s          1        175
3000 Boxes                        2003 ms         2003 ms            1  99.8686/s        200          0
1000 Stack/iterations:1           8.19 ms         8.19 ms            1  122.173/s          1          0
1000 Stack/iterations:1           13.0 ms         13.0 ms            1  77.2172/s          1         25
1000 Stack/iterations:1           14.3 ms         14.3 ms            1  69.7976/s          1         50
1000 Stack/iterations:1           14.5 ms         14.5 ms            1  69.1041/s          1         75
1000 Stack/iterations:1           14.1 ms         14.1 ms            1  70.7752/s          1        100
1000 Stack/iterations:1           14.0 ms         14.0 ms            1  71.4107/s          1        125
1000 Stack/iterations:1           14.0 ms         14.0 ms            1  71.6023/s          1        150
1000 Stack/iterations:1           14.0 ms         14.0 ms            1  71.5597/s          1        175
1000 Stack                        2754 ms         2754 ms            1  72.6148/s        200          0
Ragdolls/iterations:1             6.67 ms         6.65 ms            1  150.481/s          1          0
Ragdolls/iterations:1             5.47 ms         5.48 ms            1  182.642/s          1         25
Ragdolls/iterations:1             5.49 ms         5.48 ms            1  182.502/s          1         50
Ragdolls/iterations:1             5.76 ms         5.75 ms            1  173.955/s          1         75
Ragdolls/iterations:1             6.23 ms         6.22 ms            1  160.794/s          1        100
Ragdolls/iterations:1             6.66 ms         6.65 ms            1  150.375/s          1        125
Ragdolls/iterations:1             7.10 ms         7.09 ms            1  141.044/s          1        150
Ragdolls/iterations:1             7.98 ms         7.97 ms            1  125.464/s          1        175
Ragdolls                          1426 ms         1426 ms            1  140.292/s        200          0
Convex Stack/iterations:1         1.32 ms         1.31 ms            1  760.579/s          1          0
Convex Stack/iterations:1         1.12 ms         1.12 ms            1  896.561/s          1         25
Convex Stack/iterations:1         4.25 ms         4.25 ms            1  235.466/s          1         50
Convex Stack/iterations:1         11.7 ms         11.7 ms            1  85.5411/s          1         75
Convex Stack/iterations:1         17.5 ms         17.4 ms            1  57.3236/s          1        100
Convex Stack/iterations:1         18.0 ms         18.0 ms            1  55.6565/s          1        125
Convex Stack/iterations:1         18.1 ms         18.1 ms            1  55.3589/s          1        150
Convex Stack/iterations:1         20.7 ms         20.7 ms            1  48.2936/s          1        175
Convex Stack                      2630 ms         2630 ms            1  76.0345/s        200          0
Prim vs Mesh/iterations:1         1.87 ms         1.86 ms            1  537.228/s          1          0
Prim vs Mesh/iterations:1         1.37 ms         1.36 ms            1  734.915/s          1         25
Prim vs Mesh/iterations:1         1.24 ms         1.23 ms            1  810.056/s          1         50
Prim vs Mesh/iterations:1         1.27 ms         1.26 ms            1  795.866/s          1         75
Prim vs Mesh/iterations:1         1.44 ms         1.43 ms            1  701.269/s          1        100
Prim vs Mesh/iterations:1         1.84 ms         1.83 ms            1  547.825/s          1        125
Prim vs Mesh/iterations:1         3.00 ms         3.00 ms            1  333.549/s          1        150
Prim vs Mesh/iterations:1         4.05 ms         4.05 ms            1  247.141/s          1        175
Prim vs Mesh                       435 ms          435 ms            2  459.314/s        200          0
Convex vs Mesh/iterations:1       2.78 ms         2.76 ms            1   362.47/s          1          0
Convex vs Mesh/iterations:1       1.12 ms         1.12 ms            1  896.032/s          1         25
Convex vs Mesh/iterations:1       1.12 ms         1.11 ms            1  897.461/s          1         50
Convex vs Mesh/iterations:1       1.21 ms         1.19 ms            1  841.296/s          1         75
Convex vs Mesh/iterations:1       2.32 ms         2.31 ms            1   432.05/s          1        100
Convex vs Mesh/iterations:1       2.36 ms         2.35 ms            1  426.054/s          1        125
Convex vs Mesh/iterations:1       4.12 ms         4.12 ms            1  242.863/s          1        150
Convex vs Mesh/iterations:1       8.92 ms         8.91 ms            1  112.195/s          1        175
Convex vs Mesh                     581 ms          581 ms            1  344.088/s        200          0
Raycast/iterations:1              5.18 ms         5.17 ms            1  193.512/s          1          0
Raycast/iterations:1              3.87 ms         3.86 ms            1  258.979/s          1         25
Raycast/iterations:1              4.26 ms         4.25 ms            1  235.051/s          1         50
25500 rays in 147 ms 147 147 147.000000
Raycast/iterations:1              4.50 ms         4.49 ms            1  222.725/s          1         75
25500 rays in 140 ms 140 140 140.000000
Raycast/iterations:1              4.56 ms         4.55 ms            1  219.819/s          1        100
25500 rays in 151 ms 151 151 151.000000
25500 rays in 172 ms 151 172 161.500000
Raycast/iterations:1              5.80 ms         5.84 ms            1  171.305/s          1        125
25500 rays in 148 ms 148 148 148.000000
25500 rays in 173 ms 148 173 160.500000
Raycast/iterations:1              8.84 ms         8.83 ms            1   113.23/s          1        150
25500 rays in 156 ms 156 156 156.000000
25500 rays in 175 ms 156 175 165.500000
25500 rays in 175 ms 156 175 168.666672
Raycast/iterations:1              10.2 ms         10.2 ms            1  97.6976/s          1        175
25500 rays in 156 ms 156 156 156.000000
25500 rays in 179 ms 156 179 167.500000
25500 rays in 176 ms 156 179 170.333328
Raycast                           1313 ms         1313 ms            1  152.364/s        200          0
blinky:ammo.js ian$ 

[kripken]

Wow, very impressive!

What are the build instructions for builds/bammo.js? (Are they not from the bammo/ subdir that the non-emscripten version uses?)

[ianpurvis]

I tried to keep emscripten targets in the top level CMakeLists.txt (not sure if that's the best approach, tho). So builds/bammo.js and builds/bammo.wasm.js are automatically built when you build all. Just make sure to bump up TOTAL_MEMORY for the raycast benchmark:

$ cmake -B builds -D CLOSURE=1 -D TOTAL_MEMORY=134217728
$ cmake --build builds

I think I will submit the native code to upstream bullet. Maybe if Erwin accepts it we can revise this PR to provide that stuff via the bullet subtree.

[ianpurvis]

Upstream PR here: bulletphysics/bullet3#3074

[kripken]

Just make sure to bump up TOTAL_MEMORY for the raycast benchmark:

I think the bammo target in CMakeLists sets the size of memory, so why is this needed?

[ianpurvis]

I think the bammo target in CMakeLists sets the size of memory, so why is this needed?

Ah- no good reason, just avoiding the change to the CMakeLists... shall we up it?

[kripken]

Hmm, I guess I don't feel strongly either way. As you prefer.