[Wasm][Perf] Wasm Wasm BDN perf tests are failing to build with top level stack overflow. #98334
Assignees
Labels
Comments
ghost
added
the
|
Tagging subscribers to this area: @directhex Issue DetailsDescriptionWasm BenchmarkDotNet performance tests are failing to build and run in performance pipeline. Top level error is a Stack Overflow but that seems to be a downstream issue rather than the primary issue. Reproduction StepsGeneral steps:
Working on testing another easier repro. Expected behaviorBenchmarkDotNet builds and runs the tests successfully as demonstrated in the last successful run https://dev.azure.com/dnceng/internal/_build/results?buildId=2374634&view=results (...wasm wasm micro...) Actual behaviorFull logs can be found in ... wasm wasm micro ... job in run https://dev.azure.com/dnceng/internal/_build/results?buildId=2375008&view=results Regression?Last known working runtime githash was a79c62d and first broken githash was ed9f475. Compare link: a79c62d...ed9f475 Known WorkaroundsNo response Configurationjsvu: Found specific V8 version: v12.0.267. Other informationCompare with commits that were merged between working and failing build: a79c62d...ed9f475
|
|
Added an alternative set of instructions to maybe help get a repro using the performance repos benchmarks_local.py to install dotnet, clone the correct runtime, build wasm, and try to run the BDN tests. I was able to replicate the error using this once some fixes were added (they are part of my branch that I linked in the steps). |
|
We just changed the pinvoke table generator, so I'm certain that broke it. Will look into it once I can repro. |
|
Your repro steps make it impossible for me to use a local build of the runtime, it seems like? Is there a way for me to adapt them to work like this old incantation that used a runtime from my local checkout: This worked the last time I had to iterate on the benchmarks. |
|
Please see comment below as I initially wasn't aware wasm could be directly used with the benchmarks_ci.py script. If you already have the artifacts built you can definitely just use the benchmarks_ci.py command instead. The benchmarks_local.py mostly acts as a wrapper around benchmarks_ci.py and builds the runtime. It is also possible to have benchmarks_local.py build a local repo if you want to skip manually doing the build steps between local changes. benchmarks_local.py should also print out the benchmarks_ci.py command for a given run on a line starting with "Running single benchmarks_ci.py for RunType.WasmInterpreter ...".
If you want to try benchmarks_local.py with a local repo of the runtime, the command line would look something like: |
|
It also looks like you may be able to call the command line you have directly, though you will have to update the --wasmDataDir to point to /src/mono/browser instead of /src/mono/wasm. I wasn't aware that for wasm runs you could bypass having to build runtime like that. |
|
Build is failing, any idea why? |
|
Can you try running again now that it has been some time? We occasionally hit errors similar to this when using a dotnet sdk that tries to pull in things not published yet. I was able to successfully build with sdk version: 9.0.100-preview.2.24113.9 from https://github.com/dotnet/installer?tab=readme-ov-file#use-dotnet---help-to-see-available-commands-or-visit-httpsakamsdotnet-cli which I downloaded yesterday. |
|
I was able to manually build the autogenerated benchmarks csproj by doing this: It's still not clear to me why I can't build using the normal ci script, it's still failing to find NETCore.App.Ref 9.0.0-dev when the script tries to build. Using the current rev of #98250 it doesn't hang, but the build fails due to a pinvoke-related issue. I will try to fix that so that once the PR lands, it should be possible to build benchmarks again. I'm still not sure why it broke. |
|
#98250 should fix this now that it's landed in main. if it doesn't, let me know and I'll dig in immediately. |
|
Hm... we now seem to be hitting: |
|
I have been unable to repro this locally with the local tests running successfully. This means that this issue is not related to this change and that the fix for the issue seems to have worked 👍. I will do some more digging on this new failure and open a new issue if appropriate. |
|
This seems to have been properly fixed as outlined in dotnet/performance#3984. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Description
Wasm BenchmarkDotNet performance tests are failing to build and run in performance pipeline. Top level error is a Stack Overflow but that seems to be a downstream issue rather than the primary issue.
Reproduction Steps
General steps:
build.sh -ci -arch wasm -os browser -s mono+libs+host+packs -c Release /p:AotHostArchitecture=x64 /p:AotHostOS=linuxdotnet-latestdirectory in/artifacts/bin:./dotnet.sh build -p:TargetOS=browser -p:TargetArchitecture=wasm /nr:false /p:TreatWarningsAsErrors=true /p:Configuration=Release /p:ContinuousIntegrationBuild=true /t:InstallWorkloadUsingArtifacts ./src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csprojpython3 <performance>/scripts/benchmarks_ci.py --csproj <performance>/src/benchmarks/micro/MicroBenchmarks.csproj --incremental no --architecture x64 -f net9.0 --dotnet-versions 9.0.100-alpha.1.24070.3 --wasm --bdn-artifacts ./artifacts/BenchmarkDotNet.Artifacts --bdn-arguments='--anyCategories Libraries Runtime --category-exclusion-filter NoInterpreter NoWASM NoMono --wasmEngine ./.jsvu/bin/v8-12.0.267 "--wasmArgs=--expose_wasm --module" --wasmDataDir <performance>/artifacts/bin/wasm/wasm-data --logBuildOutput --generateBinLog'(replace with path to respective repo)Using dotnet/performance scripts/benchmarks_local.py (on Linux):
sudo python3 benchmarks_local.py --commits ed9f475dde7f3cb0e05a7613e82d69bd1a42ce0e --repo-storage-path "/home/<user>/runtimes" --run-types WasmInterpreter --filter *Span.IndexerBench.CoveredIndex2* *WriteReadAsync* --wasm-engine-path /home/<user>/.jsvu/engines/v8/v8Expected behavior
BenchmarkDotNet builds and runs the tests successfully as demonstrated in the last successful run https://dev.azure.com/dnceng/internal/_build/results?buildId=2374634&view=results (...wasm wasm micro...)
Actual behavior
Full logs can be found in ... wasm wasm micro ... job in run https://dev.azure.com/dnceng/internal/_build/results?buildId=2375008&view=results
Regression?
Last known working runtime githash was a79c62d and first broken githash was ed9f475. Compare link: a79c62d...ed9f475
Known Workarounds
No response
Configuration
jsvu: Found specific V8 version: v12.0.267.
Other information
Compare with commits that were merged between working and failing build: a79c62d...ed9f475
The text was updated successfully, but these errors were encountered: