benchmarks: add guido-fuzzer-find-1

[axic]

Depends on fix from wasm3: wasm3/wasm3#131

[axic]

fizzy/instantiate/guido-fuzzer-find-1     3872441 ns      3872605 ns           10
 wabt/instantiate/guido-fuzzer-find-1        8.42 ns         8.42 ns      1484906
wasm3/instantiate/guido-fuzzer-find-1        8.27 ns         8.27 ns      1692268
/home/builder/.hunter/_Base/135567a/1e4f2f8/f950746/Build/benchmark/Source/src/benchmark_runner.cc:120: RunInThread: Check `st.iterations() >= st.max_iterations' failed. Benchmark returned before State::KeepRunning() returned false!

@chfast any idea what this means?

[chfast]

@chfast any idea what this means?

This is effect of state.SkipWithError() and some internal benchark library assert. Fixed upstream: google/benchmark@c078337.

To fix it here I would need to refactor our benchmarking code.

[axic]

It is not important right now, given this tests a bomb on the parser and execute should trap, so it is not good for the benchmarks I guess.

[axic]

@chfast could we somehow merge this in a partially-failing way so that we expect wasm3 to fail?

[chfast]

@chfast could we somehow merge this in a partially-failing way so that we expect wasm3 to fail?

If you can find a regex pattern to exclude the failing benchmark in --benchmark_filter=. But I have issues with using negative-lookahead regex there. Not even sure which regex lib is libbenchmark built with.

[axic]

Current rough numbers:

fizzy/parse/guido-fuzzer-find-1                    314 us          247 us         2241 rate=72.0597M/s size=17.833k
 wabt/parse/guido-fuzzer-find-1                   8961 us         4023 us          183 rate=4.43242M/s size=17.833k
wasm3/parse/guido-fuzzer-find-1                   2.36 us         1.25 us       611696 rate=14.2176G/s size=17.833k
fizzy/instantiate/guido-fuzzer-find-1              583 us          346 us         2406
 wabt/instantiate/guido-fuzzer-find-1             5024 us         3983 us          153
wasm3/instantiate/guido-fuzzer-find-1             31.1 us         28.9 us        22654
fizzy/execute/guido-fuzzer-find-1/main      ERROR OCCURRED: 'Trapped'
 wabt/execute/guido-fuzzer-find-1/main      ERROR OCCURRED: 'Trapped'
wasm3/execute/guido-fuzzer-find-1/main      ERROR OCCURRED: 'Function "main" not found'

[axic]

If we disable execution, it could be included. It should help us see if parsing slowed down significantly.

[chfast]

Can it be executed now?
It should go to micro/, not the default set. We don't want to use it for PGO.

[axic]

Can it be executed now?

No, that's why I removed the .inputs. It never will be possible to "execute" it as it results in trap. Check the README I included.

[chfast]

Can it be executed now?

No, that's why I removed the .inputs. It never will be possible to "execute" it as it results in trap. Check the README I included.

That's fine if not executed at all.

[codecov]

Codecov Report

Merging #234 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #234   +/-   ##
=======================================
  Coverage   98.24%   98.24%           
=======================================
  Files          62       62           
  Lines        9048     9048           
=======================================
  Hits         8889     8889           
  Misses        159      159