Add Integration Test for WASM

[caaavik-msft]

I'm making this PR a draft since I was not able to get it to a condition where it is good to merge in, however I think it will be a useful start for someone who is more familiar with WASM to complete it. There also would need to be some changes to the CI machines to install the wasm-tools workload and the v8 engine so that wasm can run.

This PR also includes the changes from #2531 but I think that change should go in separately and can be removed from this PR. I only include it as I needed it to test out that everything is working. I can confirm though that the issue that caused WASM runs to fail was reproducible by this integration test.

The test doesn't pass yet on my system though, but I believe it is failing because of the stuff in test-main.js which I copied from the dotnet/runtime repository and not because of any issues with BDN itself. I wasn't able to figure out how to create a minimal test-main.js to use for this integration test and just used the one from the dotnet/runtime repository, but I think maybe it relies on some v8 features or specific versions that I wasn't able to figure out. Here is the error message I get when it tries to run the js:

TypeError: dotnet.withVirtualWorkingDirectory(...).withEnvironmentVariables(...).withDiagnosticTracing(...).withExitOnUnhandledError(...).withExitCodeLogging(...).withElementOnExit(...).withInteropCleanupOnExit(...).withAssertAfterExit(...).withDumpThreadsOnNonZeroExit is not a function

[adamsitnik]

@caaavik-msft Thank you for working on this! It would be great to have the coverage and finally stop randomly breaking WASM support.

There also would need to be some changes to the CI machines to install the wasm-tools workload and the v8 engine so that wasm can run.

We are using GH actions, so this should be doable. Here you can take a look to see how we are using npm to install some other js stuff:

BenchmarkDotNet/.github/workflows/run-tests.yaml

Lines 113 to 118 in 0a41e16

	- uses: actions/setup-node@v3
	name: Setup node
	with:
	node-version: "18"
	- name: Install cSpell
	run: npm install -g cspell@8.0.0

Our WASM doc is outdated, but it can be helpful too: https://benchmarkdotnet.org/articles/configs/toolchains.html#wasm

[caaavik-msft]

@radical @lewing Are one of you able to assist with writing a minimal test-main.js to be used in a WASM BDN Integration Test that is simple enough we don't have to worry about using specific versions of v8. I took the one from the runtime repo but I think it might depend on the version defined in the ChromeVersion.props file. Once we get this change in we can prevent regressions like dotnet/performance#3984 from occurring again.

[radical]

cc @pavelsavara @maraf

[timcassell]

#2531 was merged, so you can rebase and remove those changes.

[pavelsavara]

Are one of you able to assist with writing a minimal test-main.js

test-main.js is not great template and it also says so in the header.

I don't know what kind of setup this perf test requires, but here is simple V8 sample

Also this is public API you want to use to configure it.

[caaavik-msft]

@pavelsavara it seems that script is sufficient to get things working without any more customisation, thanks!

@adamsitnik @AndreyAkinshin CI is fully working now setting up everything needed for the wasm integration test to run. However, I am not sure if maybe that stuff better belongs inside the build scripts rather than in the github actions yaml. Particularly the part where we install the wasm-tools workload. Let me know what you think

[adamsitnik]

It looks great to me @caaavik-msft, big thanks for preventing future WASM breaks!

[adamsitnik]

However, I am not sure if maybe that stuff better belongs inside the build scripts rather than in the github actions yaml. Particularly the part where we install the wasm-tools workload. Let me know what you think

IMO if it turns out to be a problem we can improve it later.

@AndreyAkinshin thoughts?

[caaavik-msft]

Thanks to @ilonatommy who sent me the changes needed to add this functionality to the BenchmarkDotNet.Build project, I think those concerns should be addressed now and we should be ok to merge this

[AndreyAkinshin]

@caaavik-msft thank you very much for WasmTests support! Great job with extending build.cmd! I like this approach: it's consistent with the other build workflows and provides a simple way to configure the local environment. It was pretty easy to run it locally.

P.S. I have added additional docs about local setup.