[wasm] Run `Wasm.Build.Tests` against workloads #54451

radical · 2021-06-19T06:30:39Z

Currently we run Wasm.Build.Tests on helix, by setting various paths to reference files from artifacts. This PR adds another mode, we try to test against workload packs.

For this:

we need the nugets for the various workload packs, and the manifest
manifest id, path to those nugets
and a SDK version to test

Steps:

The specified SDK is installed in artifacts/bin/dotnet-workload
the specified manifest is installed from the built nugets, and any depends-on manifests in the json.
dotnet workload install is used to install the packs, with the built nugets path being used as one the nuget sourcs

Once this setup is done, then the tests are run in an environment such that they use dotnet from above, and try to resolve packs from there.

We still want to test without packs, the case of using EMSDK_PATH directly, for example, in the library tests. So, we now run Wasm.Build.Tests for Workloads, and the regular EMSDK_PATH.
Two workload specific tests were added in a recent PR ([wasm] Update Wasm.Build.Tests to build with net6.0 #54936)
simple test for building blazorwasm template project with AOT
a test to validate that the files in UnixFilePermissions.xml for the packs actually exist on disk

.. and these are enabled here.

To use this, the project sets TestUsingWorkloads=true, and specifies the workload with:

    <WorkloadIdForTesting Include="wasm-tools"
                           Name="microsoft.net.workload.mono.toolchain"
                           ManifestName="Microsoft.NET.Workload.Mono.ToolChain"
                           Version="$(PackageVersion)"
                           VersionBand="$(SdkBandVersion)" />

Notes:

This also fixes using the correct runtime pack based on the version in the manifest json.
The sdk version is specified with <SdkVersionForWorkloadTesting>6.0.100-preview.7.21326.4</SdkVersionForWorkloadTesting> in eng/Versions.props
Known issues: I'm explicitly invoking Pack on the aotcross pkgproj

TODO:

Run the blazorwasm test with playwright
Add sdk-with-no-packs case also

ghost · 2021-06-29T00:07:29Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Currently we run Wasm.Build.Tests on helix, by setting various paths to reference files from artifacts. This PR adds another mode, we try to test against workload packs.

For this:

we need the nugets for the various workload packs, and the manifest
manifest id, path to those nugets
and a SDK version to test

The specified SDK is installed in artifacts/bin/dotnet-workload. Then we use InstallWorkload to "install" the packs for this SDK. This does not use dotnet workload install, instead it:

extracts the WorkloadManifest.json from the manifest nuget
Given the manifest id, gets the list of packs
For the packs with nugets in the artifacts dir, it unpacks them into the correct locations
For the remaining ones (Emscripten in this case), it restores them from nuget

Once this setup is done, then the tests are run in an environment such that they use dotnet from above, and try to resolve packs from there.

Known issues:

I wasn't able to build the cross compiler package, so for this PR, I'm copying over the files explicitly

We still want to test without packs, the case of using EMSDK_PATH directly, for example, in the library tests. So, we now run Wasm.Build.Tests for Workloads, and the regular EMSDK_PATH.

Tests:

a simple test is added for building blazorwasm template project with AOT
a test to validate that the files in UnixFilePermissions.xml for the packs actually exist on disk

Notes:

This also fixes using the correct runtime pack based on the version in the manifest json.
And it changes, and fixes building test projects with net6.0
The sdk version is specified with <SdkVersionForWorkloadTesting>6.0.100-preview.7.21326.4</SdkVersionForWorkloadTesting> in eng/Versions.props

Author:	radical
Assignees:	-
Labels:	`arch-wasm`, `area-Build-mono`
Milestone:	-

radical · 2021-06-29T00:07:48Z

cc @lewing @steveisok @radekdoulik @akoeplinger RFC!

lewing · 2021-06-29T23:45:50Z

src/mono/nuget/Microsoft.NET.Runtime.WebAssembly.Sdk/Sdk/AutoImport.props.in

+<Project>
+  <PropertyGroup>
+    <WasmNativeWorkload>true</WasmNativeWorkload>
+    <UseMonoRuntime>true</UseMonoRuntime>


This file is always included once the workload is installed, you should stick any new properties in a conditionally included group.

But the right place to still all these is in Sdk.props I think

src/mono/nuget/Microsoft.NET.Runtime.WebAssembly.Sdk/Sdk/AutoImport.props.in

src/mono/nuget/Microsoft.NET.Workload.Mono.Toolchain.Manifest/WorkloadManifest.targets.in

radical · 2021-07-08T08:11:10Z

Just doing some final tidbits, but this should be good for reviewing.

src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.props

radical · 2021-07-12T18:57:14Z

/azp run

azure-pipelines · 2021-07-12T18:57:19Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

radical · 2021-07-12T18:58:03Z

/azp run runtime,runtime-staging

azure-pipelines · 2021-07-12T18:58:25Z

Azure Pipelines successfully started running 2 pipeline(s).

…ation fails

akoeplinger · 2021-07-13T15:16:41Z

src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.props

@@ -1,4 +1,8 @@
 <Project>
+  <PropertyGroup>
+    <UseMonoRuntime Condition="'$(RuntimeIdentifier)' == 'Browser-wasm'">true</UseMonoRuntime>


I think/hope this is no longer needed after my #55258

Your PR changes the InTree, and LocalBuild targets, which don't get used for the workloads.

I think we could move it to an Sdk.props in the WebAssembly sdk though, that way it will only get picked up when that workload is in use.

Follow up PR? :) And if we are moving it to wasm sdk, then might as well move it to wasm props, then it won't be needed in InTree/LocalBuild either.

…ng corrupted by concurrent access

radical · 2021-07-14T00:04:52Z

Windows x86 failure is #55611

lewing

Discussed some follow up with @radical but since we're already have a pending pr building off this the changes can happen there

That will be handled in dotnet#54451

* [wasm] Run browser tests on helix/windows * Build just wasm/browsertests on helix/windows * Use $(ChromiumRevision) in windows links * Fix conditions * Set PATH differently * Use backslash in PATH on windows * Try different version of chromium * Pass scenario and browser host to build And set browser path on windows to be able to start chrome from xharness * Try to get more info from the helix workitems * Fix dir separator, add broser path * Create WasmBuildSupportDir * Revert "Try to get more info from the helix workitems" This reverts commit 8807434. * Put the dir cmds back, fix mkdir call * More debug info * Bump xharness * Bump xharness again With darc this time * StressLogAnalyzer didn't print the number of messages correctly if it exceeded the int range (2 billion). (#54832) Fix is to just use 64 bit ints instead. * Found a race condition where the LOH flag on a segment is set too late. This gives another thread the chance to allocate in a fresh LOH region that doesn't have the LOH flag set just yet and trip over an assert in Object::ValidateInner. (#54839) The fix is simply to set the flag in get_new_region before the region is put on the list for the LOH generation. * Try to show the chrome logs * Use different path for chrome logs * Use newer image with font for chrome The chrome was crashing, because it didn't find any sans-serif font. * Increase timeouts * Disable tests which timeout * Remove debug calls * Put back normal scenario * Do not set scenario in build args * Add browser sample exclusion * Restore the platform matrix * Remove the wasm build test changes That will be handled in #54451 * Remove duplicate exclusion * Suggested property name change * Fix last merge * Simplify condition We don't pass Scenario anymore * Include chrome and chromedriver in the payload Co-authored-by: Peter Sollich <petersol@microsoft.com>

dotnet-issue-labeler bot added the area-Build-mono label Jun 19, 2021

radical force-pushed the wasm-native-libs-workloads branch 2 times, most recently from 3a62ae1 to 81373d9 Compare June 22, 2021 07:29

radical mentioned this pull request Jun 22, 2021

Workload install copying across mounts error dotnet/sdk#18450

Closed

karelz mentioned this pull request Jun 24, 2021

Connect_DualMode_DnsConnect_RetrievedEndPoints_Success - Assert ::1 #54677

Closed

steveisok mentioned this pull request Jun 25, 2021

[mono] new Sdk that selects mono runtime components #54432

Merged

runfoapp bot mentioned this pull request Jun 25, 2021

Test failure System.Threading.Tests.PeriodicTimerTests.WaitForNextTickAsync_CanceledWaitThenWaitAgain_Succeeds #54713

Closed

radical force-pushed the wasm-native-libs-workloads branch from 6d43522 to 9165c39 Compare June 26, 2021 01:18

radical changed the title ~~[IGNORE] [wip] wasm native libs, and testing wbt with workloads~~ [wasm] Run Wasm.Build.Tests against workload Jun 28, 2021

radical added the arch-wasm WebAssembly architecture label Jun 29, 2021

radical changed the title ~~[wasm] Run Wasm.Build.Tests against workload~~ [wasm] Run Wasm.Build.Tests against workloads Jun 29, 2021

lewing reviewed Jun 30, 2021

View reviewed changes

radical force-pushed the wasm-native-libs-workloads branch from d42f53c to b7dfd99 Compare June 30, 2021 20:32

radical mentioned this pull request Jul 1, 2021

[IGNORE][NO MERGE] Wasm workloads testing #55022

Closed

radical force-pushed the wasm-native-libs-workloads branch 3 times, most recently from 9326d56 to 76cd866 Compare July 8, 2021 05:56

radical marked this pull request as ready for review July 8, 2021 08:10

radical requested a review from marek-safar as a code owner July 8, 2021 08:10

radical requested a review from akoeplinger July 8, 2021 22:34

radical force-pushed the wasm-native-libs-workloads branch from da822a5 to 6ca6281 Compare July 11, 2021 08:59

steveisok reviewed Jul 11, 2021

View reviewed changes

src/mono/nuget/Microsoft.NET.Runtime.MonoTargets.Sdk/Sdk/Sdk.props Show resolved Hide resolved

radical added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Jul 12, 2021

radical added 4 commits July 12, 2021 18:35

Add missing EnvironmentVariables.cs

2282814

Add doc for workloads testing

0f13abc

cleanup

0a6532a

Log directories in the dotnet sdk-manifests, and packs dir if install…

df94014

…ation fails

radical force-pushed the wasm-native-libs-workloads branch from b73db09 to df94014 Compare July 12, 2021 23:49

radical added 2 commits July 12, 2021 20:44

Fix typo

dd92d28

fix typo

5d897b8

radical removed the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Jul 13, 2021

radical added 4 commits July 13, 2021 00:11

Fix skipping tests on windows

1dbae80

Add a prefix to helix jobs for wasm.build.tests

cc38268

re-enable all the jobs

364dc80

cleanup

d7d8b27

akoeplinger reviewed Jul 13, 2021

View reviewed changes

radical added 4 commits July 13, 2021 13:10

Merge remote-tracking branch 'origin/main' into wasm-workloads-staging2

7b8a4ae

lock access to the stringbuilder for output, to prevent it from getti…

236b644

…ng corrupted by concurrent access

Add a separate build job for wasm.build.tests

31d41b9

dummy commit in mono, to trigger the builds

32fa9c4

radical mentioned this pull request Jul 14, 2021

System.Linq.Expressions.Tests crashed with Fatal error. 0xC0000005 #55611

Closed

Lock around ToString to ensure the handlers have flushed

b9f99a8

lewing approved these changes Jul 14, 2021

View reviewed changes

radekdoulik added a commit to radekdoulik/runtime that referenced this pull request Jul 14, 2021

Remove the wasm build test changes

c833f5d

That will be handled in dotnet#54451

radekdoulik mentioned this pull request Jul 14, 2021

[wasm] Run browser tests on helix/windows #52699

Merged

radical merged commit d953229 into dotnet:main Jul 14, 2021

radical deleted the wasm-native-libs-workloads branch July 14, 2021 14:47

ManickaP mentioned this pull request Jul 20, 2021

[QUIC] Remove AppContext switch from S.N.Quic #56027

Merged

karelz mentioned this pull request Jul 20, 2021

Failing SendReceive_MemoryArrayTask.SendAsync_ConcurrentDispose_SucceedsOrThrowsAppropriateException test #55085

Closed

karelz mentioned this pull request Jul 28, 2021

[WinHttpHandler] Long running test: ResponseHeadersRead_SynchronizationContextNotUsedByHandler on Win7/Win8 #54034

Closed

ghost locked as resolved and limited conversation to collaborators Aug 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wasm] Run `Wasm.Build.Tests` against workloads #54451

[wasm] Run `Wasm.Build.Tests` against workloads #54451

radical commented Jun 19, 2021 •

edited

Loading

ghost commented Jun 29, 2021

radical commented Jun 29, 2021 •

edited

Loading

lewing Jun 29, 2021

lewing Jun 30, 2021

radical commented Jul 8, 2021

radical commented Jul 12, 2021

azure-pipelines bot commented Jul 12, 2021

radical commented Jul 12, 2021

azure-pipelines bot commented Jul 12, 2021

akoeplinger Jul 13, 2021

radical Jul 13, 2021

lewing Jul 13, 2021

radical Jul 14, 2021

radical commented Jul 14, 2021

lewing left a comment

[wasm] Run Wasm.Build.Tests against workloads #54451

[wasm] Run Wasm.Build.Tests against workloads #54451

Conversation

radical commented Jun 19, 2021 • edited Loading

ghost commented Jun 29, 2021

radical commented Jun 29, 2021 • edited Loading

lewing Jun 29, 2021

Choose a reason for hiding this comment

lewing Jun 30, 2021

Choose a reason for hiding this comment

radical commented Jul 8, 2021

radical commented Jul 12, 2021

azure-pipelines bot commented Jul 12, 2021

radical commented Jul 12, 2021

azure-pipelines bot commented Jul 12, 2021

akoeplinger Jul 13, 2021

Choose a reason for hiding this comment

radical Jul 13, 2021

Choose a reason for hiding this comment

lewing Jul 13, 2021

Choose a reason for hiding this comment

radical Jul 14, 2021

Choose a reason for hiding this comment

radical commented Jul 14, 2021

lewing left a comment

Choose a reason for hiding this comment

[wasm] Run `Wasm.Build.Tests` against workloads #54451

[wasm] Run `Wasm.Build.Tests` against workloads #54451

radical commented Jun 19, 2021 •

edited

Loading

radical commented Jun 29, 2021 •

edited

Loading