WASM Microbenchmarks Not Working

[LoopedBard3]

Wasm Microbenchmarks are still failing after the BDN update. Two spots that pop out are a seeming failure with building, although another build of seemingly the same location succeeds after; and an inability to run a specific 'main.js' file from the browser-wasm/AppBundle folder. Respectively:

// start dotnet build -c Release /p:DebugType=portable -bl:benchmarkdotnet.binlog --no-restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1 /p:Deterministic=true /p:Optimize=true /p:NuGetPackageRoot="/home/helixbot/work/A50E08AE/p/performance/artifacts/packages" in /home/helixbot/work/A50E08AE/p/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/f55d017a-085a-41d6-a734-6abcdf4c2743
[2022/02/08 12:59:15][INFO] // command took 14.99s and exited with 1

and

[2022/02/08 13:14:47][INFO] // Execute: /home/helixbot/.jsvu/v8 --expose_wasm main.js -- --run f55d017a-085a-41d6-a734-6abcdf4c2743.dll --benchmarkName Burgers.Test2 --job "PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Arguments=/p:DebugType=portable,-bl:benchmarkdotnet.binlog, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1" --benchmarkId 0  in /home/helixbot/work/A50E08AE/p/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/f55d017a-085a-41d6-a734-6abcdf4c2743/bin/net7.0/browser-wasm/AppBundle
[2022/02/08 13:14:47][INFO] Failed to set up high priority. Make sure you have the right permissions. Message: Permission denied
[2022/02/08 13:14:47][INFO] Error reading 'main.js'
[2022/02/08 13:14:47][INFO] No Workload Results were obtained from the run.

Upon further investigation, it looks like the main.js file got renamed to test-main.js. Example ls result in AppBundle:

dotnet.js  dotnet.timezones.blat  dotnet.wasm  icudt.dat  index.html  managed  mono-config.json  run-v8.sh  supportFiles  test-main.js

@naricc @SamMonoRT

[SamMonoRT]

@pavelsavara @lewing - please take a look

[pavelsavara]

Yes test-main.js is right thing to do for Net7.0 but still main.js for Net6.0.
Does this repo target both ?

[naricc]

This should be resovled with this change to BDN: dotnet/BenchmarkDotNet#1916

Are it looks like the update to pull the new version of BDN merged: #2245

So why is this still broken with the same error message? Did we get the wrong BDN version?

[LoopedBard3]

@pavelsavara This run specifies to target net7.0. The inner build command is:

dotnet build /home/helixbot/work/AA7B0915/p/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net7.0 --no-restore 

giving this output

[2022/02/09 08:14:09][INFO] Microsoft (R) Build Engine version 17.2.0-preview-22104-01+09bdfae16 for .NET
[2022/02/09 08:14:09][INFO] Copyright (C) Microsoft Corporation. All rights reserved.
[2022/02/09 08:14:09][INFO] 
[2022/02/09 08:14:10][INFO]   Reporting -> /home/helixbot/work/AA7B0915/p/performance/artifacts/bin/Reporting/Release/netstandard2.0/Reporting.dll
[2022/02/09 08:14:11][INFO]   BenchmarkDotNet.Extensions -> /home/helixbot/work/AA7B0915/p/performance/artifacts/bin/BenchmarkDotNet.Extensions/Release/netstandard2.0/BenchmarkDotNet.Extensions.dll
[2022/02/09 08:14:14][INFO]   MicroBenchmarks -> /home/helixbot/work/AA7B0915/p/performance/artifacts/bin/MicroBenchmarks/Release/net7.0/MicroBenchmarks.dll
[2022/02/09 08:14:14][INFO] 
[2022/02/09 08:14:14][INFO] Build succeeded.
[2022/02/09 08:14:14][INFO]     0 Warning(s)
[2022/02/09 08:14:14][INFO]     0 Error(s)
[2022/02/09 08:14:14][INFO] 
[2022/02/09 08:14:14][INFO] Time Elapsed 00:00:05.25

[naricc]

I retrieved the command it is running the microbenchmakrs with from the logs:

dotnet run --project /home/helixbot/work/B9C40A56/p/performance/src/benchmarks/micro/MicroBenchmarks.csproj --configuration Release --framework net7.0 --no-restore --no-build -- --anyCategories Libraries Runtime "" --category-exclusion-filter NoInterpreter NoWASM NoMono --wasmEngine /home/helixbot/.jsvu/v8 --runtimeSrcDir /home/helixbot/work/B9C40A56/p/dotnet-wasm --aotcompilermode wasm --buildTimeout 3600 "" --partition-count 30 --partition-index 0

Trying this command locally (with modifications. for the paths), and it seems to run fine. So I am not sure where the error is coming in. Are these runs happening with the very latest Microbenchmarks.csproj?

I will retrieve the helix payload and see what I can get out of it.

[lewing]

dotnet/BenchmarkDotNet#1926 should help

[lewing]

The bdn change has merged, what are the next steps?

[naricc]

@LoopedBard3 @DrewScoggins Can one of you update the BDN version the microbenchmarks pull, and upload the new package to the repository?

[LoopedBard3]

PR to update the version is up: #2270.

[LoopedBard3]

It looks like even after the update, the main.js file is the one being looked for.

[lewing]

It looks like even after the update, the main.js file is the one being looked for.

Looking at dotnet/BenchmarkDotNet#1916 my guess is that
string main_js = runtime.RuntimeMoniker < RuntimeMoniker.WasmNet70 ? "main.js" : "test-main.js";
Is picking the wrong js file when RuntimeMoniker.Wasm is being used instead of RuntimeMoniker.WasmNet70

https://github.com/dotnet/BenchmarkDotNet/blob/beb543a6714623a6644adb3fbfc492820b888a14/src/BenchmarkDotNet.Annotations/Jobs/RuntimeMoniker.cs#L141-L156