Blazor WASM Template tests are too long

[BrennanConroy]

The Templates.Blazor.WebAssembly.Tests project is taking 37+ minutes on Helix. This is dangerously close to the 40 minute timeout we have on Helix and in fact the tests were hitting this timeout before we split the Blazor.Server parts out into it's own project in: #42382. Unfortunately this only saved ~4.5 minutes.

I suggest the Blazor team takes a look at this test project and either trims down the test variants, or splits the project up even further to reduce the time each test leg takes.

[BrennanConroy]

An example where it hit the 40 minute timeout today
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-aspnetcore-refs-pull-42464-merge-e6065bc2374f4845a7/Templates.Blazor.WebAssembly.Tests--net7.0/1/console.4991686d.log?helixlogtype=result

[halter73]

And more:

https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-aspnetcore-refs-pull-42704-merge-ed526c9c570e4a699a/Templates.Blazor.WebAssembly.Tests--net7.0/1/console.49356bbc.log?helixlogtype=result
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-aspnetcore-refs-pull-42704-merge-97eb92965ff74a6fa7/Templates.Blazor.WebAssembly.Tests--net7.0/1/console.31d0b82b.log?helixlogtype=result

https://dev.azure.com/dnceng/public/_build/results?buildId=1883911&view=results

[TanayParikh]

The issue seems to be particularly acute on release/7.0-preview7. @cerasamson was running into it as well and was experimenting with an increased helix timeout (as a temporary unblocker): 80a0577

I'm planning on looking into this in the RC1 timeframe to figure out how we can shorten the run-times overall.

[dougbu]

I'm planning on looking into this in the RC1 timeframe to figure out how we can shorten the run-times overall.

Kusto says these work items run far longer than other template tests over the last 2 weeks. Rebalancing tests between projects may be enough.

FriendlyName	QueueName	min_Duration	max_Duration
Templates.Blazor.Server.Tests--net7.0	osx.1100.amd64.open	00:03:31	00:05:01
Templates.Blazor.Server.Tests--net7.0	ubuntu.1804.amd64.open	00:03:50	00:04:26
Templates.Blazor.Server.Tests--net7.0	windows.11.amd64.client.open	00:04:55	00:06:37
Templates.Blazor.WebAssembly.Tests--net7.0	osx.1100.amd64.open	00:22:09	00:32:56
Templates.Blazor.WebAssembly.Tests--net7.0	ubuntu.1804.amd64.open	00:31:38	00:36:17
Templates.Blazor.WebAssembly.Tests--net7.0	windows.11.amd64.client.open	00:36:14	00:40:15
Templates.Mvc.Tests--net7.0	osx.1100.amd64.open	00:10:51	00:14:47
Templates.Mvc.Tests--net7.0	ubuntu.1804.amd64.open	00:15:29	00:20:51
Templates.Mvc.Tests--net7.0	windows.11.amd64.client.open	00:23:27	00:31:15
Templates.Tests--net7.0	osx.1100.amd64.open	00:05:18	00:07:32
Templates.Tests--net7.0	ubuntu.1804.amd64.open	00:05:33	00:10:30
Templates.Tests--net7.0	windows.11.amd64.client.open	00:06:51	00:09:16

Query

Jobs
| where Queued > ago(14d)
    and Repository == "dotnet/aspnetcore"
    and Branch == "refs/heads/main"
    and Properties has "aspnetcore-ci"
| join kind=inner WorkItems on JobId
| where FriendlyName has "Templates" and FailCount == 0 and PassOnRetryCount == 0
| project FriendlyName, QueueName, QueueAlias, Duration = Finished1 - Started1
| summarize format_timespan(min(Duration), 'hh:mm:ss'),
    format_timespan(max(Duration), 'hh:mm:ss')
    by FriendlyName, QueueName
| sort by FriendlyName asc, QueueName asc

[TanayParikh]

Recently, there have been several new tests in the space:

• Empty Blazor Server Project Template #42079
• Add blazorwasm Template Baseline Tests #42420
• Empty Blazor WebAssembly Project Template #42215

I wonder if we can track the increased execution times to those tests. It's a bit odd WASM specifically takes so long...

[dougbu]

@TanayParikh only #42420 went in after #42382. We could probably look at data for the WASM work items since 23 June (when #42382 went in) by date. @DamianEdwards might already have the right query 😀

[TanayParikh]

we split the Blazor.Server parts out into it's own project in: #42382. Unfortunately this only saved ~4.5 minutes.

This is in line with Doug's table above where the Server tests are pretty quick:

FriendlyName	QueueName	min_Duration	max_Duration
Templates.Blazor.Server.Tests--net7.0	osx.1100.amd64.open	00:03:31	00:05:01
Templates.Blazor.Server.Tests--net7.0	ubuntu.1804.amd64.open	00:03:50	00:04:26
Templates.Blazor.Server.Tests--net7.0	windows.11.amd64.client.open	00:04:55	00:06:37

Created #42879 to split out the auth tests which seems like a pretty clear "split line" for the Blazor WASM Tests.