Re-enable SB tests #46055

mthalman · 2025-01-16T17:55:02Z

Tests that are re-enabled:

DotNetWatchTests.WatchTests
SymbolsTests.VerifySdkSymbols

Tests that are failing due to issues

WebScenarioTests.VerifyScenario (self-contained): Prevent Microsoft.AspNetCore.App.Internal.Assets from being restored when not needed #46165
DotNetFormatTests.FormatProject: Microsoft.Build.Locator excluded from MSBuild.BuildHost.deps.json file when building with .NET 10 SDK roslyn#76797

mthalman · 2025-01-16T18:55:57Z

Current test failures:

DotNetWatchTests.WatchTests

Assert.True() Failure
Expected: True
Actual:   False

 at Microsoft.DotNet.SourceBuild.Tests.DotNetWatchTests.WatchTests() in /_/test/Microsoft.DotNet.SourceBuild.Tests/DotNetWatchTests.cs:line 35
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

WebScenarioTests.VerifyScenario

Runtime packs were retrieved from NuGet instead of the SDK

   at Microsoft.DotNet.SourceBuild.Tests.WebScenarioTests.VerifyRuntimePacksForSelfContained(String projectPath) in /_/test/Microsoft.DotNet.SourceBuild.Tests/WebScenarioTests.cs:line 55
   at Microsoft.DotNet.SourceBuild.Tests.TestScenario.Execute(DotNetHelper dotNetHelper) in /_/test/Microsoft.DotNet.SourceBuild.Tests/TestScenario.cs:line 75
   at Microsoft.DotNet.SourceBuild.Tests.WebScenarioTests.VerifyScenario(TestScenario scenario) in /_/test/Microsoft.DotNet.SourceBuild.Tests/WebScenarioTests.cs:line 25
   at InvokeStub_WebScenarioTests.VerifyScenario(Object, Span`1)
   at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

DotNetFormatTests.FormatProject

System.InvalidOperationException : Failed to execute /__w/1/vmr/artifacts/obj/extracted-dotnet-sdk/dotnet format /__w/1/vmr/artifacts/bin/Microsoft.DotNet.SourceBuild.Tests/Release/projects-202501161845028438/FormatProject/FormatProject.csproj
Exit code: 1


Unhandled exception: Microsoft.CodeAnalysis.MSBuild.RemoteInvocationException: An exception of type System.IO.FileNotFoundException was thrown: Could not load file or assembly 'Microsoft.Build.Locator, Version=1.6.10.0, Culture=neutral, PublicKeyToken=9dff12846e04bfbd'. The system cannot find the file specified.


at Microsoft.CodeAnalysis.MSBuild.RpcClient.InvokeCoreAsync(Int32 targetObject, String methodName, List1 parameters, Type expectedReturnType, CancellationToken cancellationToken)    at Microsoft.CodeAnalysis.MSBuild.RpcClient.InvokeAsync[T](Int32 targetObject, String methodName, List1 parameters, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.MSBuild.RemoteBuildHost.LoadProjectFileAsync(String projectFilePath, String languageName, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.Worker.DoOperationAndReportProgressAsync[TResult](ProjectLoadOperation operation, String projectPath, String targetFramework, Func1 doFunc)    at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.Worker.LoadProjectFileInfosAsync(String projectPath, DiagnosticReportingOptions reportingOptions, CancellationToken cancellationToken)    at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.Worker.LoadProjectInfosFromPathAsync(String projectPath, DiagnosticReportingOptions reportingOptions, CancellationToken cancellationToken)    at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.Worker.LoadAsync(CancellationToken cancellationToken)    at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.LoadProjectInfoAsync(String projectFilePath, ProjectMap projectMap, IProgress1 progress, ILogger msbuildLogger, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.LoadProjectInfoAsync(String projectFilePath, ProjectMap projectMap, IProgress1 progress, ILogger msbuildLogger, CancellationToken cancellationToken)    at Microsoft.CodeAnalysis.MSBuild.MSBuildWorkspace.OpenProjectAsync(String projectFilePath, ILogger msbuildLogger, IProgress1 progress, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.Tools.Workspaces.MSBuildWorkspaceLoader.LoadAsync(String solutionOrProjectPath, WorkspaceType workspaceType, String binaryLogPath, Boolean logWorkspaceWarnings, ILogger logger, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.Tools.CodeFormatter.OpenMSBuildWorkspaceAsync(String solutionOrProjectPath, WorkspaceType workspaceType, Boolean noRestore, Boolean requiresSemantics, String binaryLogPath, Boolean logWorkspaceWarnings, ILogger logger, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.Tools.CodeFormatter.FormatWorkspaceAsync(FormatOptions formatOptions, ILogger logger, CancellationToken cancellationToken, String binaryLogPath)
at Microsoft.CodeAnalysis.Tools.FormatCommandCommon.FormatAsync(FormatOptions formatOptions, ILogger`1 logger, CancellationToken cancellationToken)
at Microsoft.CodeAnalysis.Tools.Commands.RootFormatCommand.FormatCommandDefaultHandler.InvokeAsync(ParseResult parseResult, CancellationToken cancellationToken)
at System.CommandLine.Invocation.InvocationPipeline.InvokeAsync(ParseResult parseResult, CancellationToken cancellationToken)

SymbolsTests.VerifySdkSymbols

Assert.True() Failure
Expected: True
Actual:   False

   at Microsoft.DotNet.SourceBuild.Tests.SymbolsTests.VerifySdkSymbols() in /_/test/Microsoft.DotNet.SourceBuild.Tests/SymbolsTests.cs:line 57
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

mthalman · 2025-01-16T20:09:18Z

@NikolaMilosavljevic - Could you help investigate why the symbols test is failing? It looks like all of the DLLs within the SDK are being reported as having no PDBs.

NikolaMilosavljevic · 2025-01-17T02:17:23Z

@NikolaMilosavljevic - Could you help investigate why the symbols test is failing? It looks like all of the DLLs within the SDK are being reported as having no PDBs.

Pushed a fix for symbols tests - 196f148

tmds · 2025-01-17T10:34:35Z

DotNetWatchTests.WatchTests

sdk/src/SourceBuild/content/test/Microsoft.DotNet.SourceBuild.Tests/DotNetWatchTests.cs

Line 39 in 2f9c4fe

const string waitingString = "Waiting for changes";

should be changed to:

const string waitingString = "Waiting for a file to change before restarting";

WebScenarioTests.VerifyScenario

VerifyRuntimePacksForSelfContained is detecting the Microsoft.AspNetCore.App.Internal.Assets package that was added in #44997. Probably this one can be ignored by VerifyRuntimePacksForSelfContained.

DotNetFormatTests.FormatProject

The BuildHost-netcore (that gets started by dotnet-format) is failing to load Microsoft.Build.Locator. From the .deps.json it looks like something is wrong when the project gets built. .NET 9 vs .NET 10 .deps.json:

@@ -1,22 +1,17 @@
 {
   "runtimeTarget": {
-    "name": ".NETCoreApp,Version=v9.0",
+    "name": ".NETCoreApp,Version=v10.0",
     "signature": ""
   },
   "compilationOptions": {},
   "targets": {
-    ".NETCoreApp,Version=v9.0": {
-      "Microsoft.CodeAnalysis.Workspaces.MSBuild.BuildHost/4.12.0-3.24570.6": {
+    ".NETCoreApp,Version=v10.0": {
+      "Microsoft.CodeAnalysis.Workspaces.MSBuild.BuildHost/4.14.0-1.25064.9": {
         "dependencies": {
-          "Microsoft.Build": "17.3.4",
           "Microsoft.Build.Framework": "17.3.4",
-          "Microsoft.Build.Locator": "1.6.10",
-          "Microsoft.Build.Tasks.Core": "17.3.4",
-          "Microsoft.Build.Utilities.Core": "17.3.4",
-          "Microsoft.DotNet.XliffTasks": "9.0.0-beta.24516.2",
           "Newtonsoft.Json": "13.0.3",
-          "System.Collections.Immutable": "9.0.0",
-          "System.CommandLine": "2.0.0-beta4.24326.1"
+          "System.Collections.Immutable": "10.0.0-alpha.1.25063.12",
+          "System.CommandLine": "2.0.0-beta4.24529.1"
         },
         "runtime": {
           "Microsoft.CodeAnalysis.Workspaces.MSBuild.BuildHost.dll": {}
@@ -63,61 +58,9 @@
           }
         }
       },

mthalman · 2025-01-17T15:37:46Z

VerifyRuntimePacksForSelfContained is detecting the Microsoft.AspNetCore.App.Internal.Assets package that was added in #44997. Probably this one can be ignored by VerifyRuntimePacksForSelfContained.

@tmds - Can you explain why it's safe to ignore that one?

tmds · 2025-01-17T15:53:52Z

Can you explain why it's safe to ignore that one?

My limited understanding is that this functionality is different than the ASP.NET Core runtime, which is what the test is checking for. It would be good to understand what it is exactly and how this is going to be delivered eventually.

mthalman · 2025-01-17T16:09:09Z

Can you explain why it's safe to ignore that one?

My limited understanding is that this functionality is different than the ASP.NET Core runtime, which is what the test is checking for. It would be good to understand what it is exactly and how this is going to be delivered eventually.

@MackinnonBuck - Can you provide some more context around the Microsoft.AspNetCore.App.Internal.Assets package and its inclusion with the SDK changes in #44997? We have a test that is failing due to this assembly. The test runs a self-contained publish of a web project. The intent of the test is to verify that no packages get restored from a NuGet feed and instead are available directly from the SDK. But that doesn't happen with the Microsoft.AspNetCore.App.Internal.Assets package.

MackinnonBuck · 2025-01-17T18:10:34Z

That package contains JavaScript files that get served as part of each Blazor app to provide core framework functionality. Prior to the introduction of this package, these files were embedded resources in framework DLLs, but this meant they couldn't participate in the app's build process the same way as other assets defined by the app (build time compression and fingerprinting, etc.).

Furthermore, NodeJS is required to build these files, but we didn't want NodeJS to be a dependency of source build. Our initial solution to this was to check in the built JS assets, but this created a number of issues, including security concerns.

So, we recently introduced this package to implicitly bring those files into the customer's project, and it's excluded from source build due to its NodeJS dependency.

cc @javiercn, in case there's more context on the source-build problem you think should be shared.

tmds · 2025-01-17T18:23:11Z

Will this package be shipped with the SDK, or will it be provided through nuget.org? Is this the same mechanism for the source-built and the Microsoft-built SDKs?

MackinnonBuck · 2025-01-17T21:29:28Z

I believe this package was just going to be provided on nuget.org. The intention was to make this package work in a similar way to Microsoft.NET.Sdk.WebAssembly.Pack. But looking at my local dotnet SDK installation, I see that Microsoft.NET.Sdk.WebAssembly.Pack is included in the library-packs folder, whereas Microsoft.AspNetCore.App.Internal.Assets is not. Do we need to somehow include Microsoft.AspNetCore.App.Internal.Assets in the SDK as well? TBH this is an area I'm not very familiar with.

MackinnonBuck · 2025-01-17T21:34:47Z

@maraf, do you know how Microsoft.NET.Sdk.WebAssembly.Pack ships? It gets included in the SDK somehow, right? Do you know how we can make this happen with Microsoft.AspNetCore.App.Internal.Assets, which was added in #44997?

maraf · 2025-01-20T07:51:02Z

@maraf, do you know how Microsoft.NET.Sdk.WebAssembly.Pack ships? It gets included in the SDK somehow, right? Do you know how we can make this happen with Microsoft.AspNetCore.App.Internal.Assets, which was added in #44997?

The Microsoft.NET.Sdk.WebAssembly.Pack is included in the SDK (library-packs) only if user installs workload (wasm-tools). If the user don't have workload installed or the version doesn't match, it restores from configured nuget feed (nuget.org). Using library-packs is just an optimisation in this case.

tmds · 2025-01-20T11:59:58Z

I believe this package was just going to be provided on nuget.org.

The test that is acquiring the package does a regular webapp. Afaik it doesn't need any Blazor assets.

If a source-built SDK is expected to download this package from nuget.org, it's highly desired to not have to acquire it for applications that have no need for it. Otherwise this introduces a non source-buildable (build-time) dependency for every ASP.NET Core app built with a source-built SDK.

maraf · 2025-01-20T14:28:26Z

We probably need to find a way to determine if Blazor will be used in AspNetCore app. Currently it's hooked by usage of Razor SDK https://github.com/dotnet/sdk/blob/main/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets#L139

mthalman · 2025-01-21T14:12:47Z

@maraf - I've logged an issue for this: #46165

MichaelSimons

Would you mind updating the PR description to include a summary? It would be useful to capture which tests were fixed/enabled and which remain. Also will you open tracking issues to re-enable the remain two tests once the associated product issues are resolved? TIA

mthalman · 2025-01-21T16:33:58Z

Created dotnet/source-build#4835 for format test.
And dotnet/source-build#4578 already exists for the self-contained web scenario test.

tmds · 2025-01-22T06:46:51Z

I believe this package was just going to be provided on nuget.org.

@MackinnonBuck is it set up for this package to be published for the first preview(s)?
Without the package, (most) ASP.NET 10 projects won't be usable with the SDK (because they can't be restored).

MackinnonBuck · 2025-01-22T16:59:01Z

@tmds yes, the package will get published on nuget.org for the first preview.

Re-enable SB tests

407f38b

dotnet-issue-labeler bot added Area-Infrastructure untriaged Request triage from a team member labels Jan 16, 2025

mthalman mentioned this pull request Jan 16, 2025

Reenable PublishSelfContained scenario test dotnet/source-build#4578

Open

Fix for symbols tests

196f148

This was referenced Jan 17, 2025

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Open

Failure on Helix tests - Tests timing out #40074

Open

mthalman added 2 commits January 17, 2025 10:24

fix dotnet watch test

c61dbd2

Fix self-contained test

12fcbf9

mthalman mentioned this pull request Jan 17, 2025

Microsoft.Build.Locator excluded from MSBuild.BuildHost.deps.json file when building with .NET 10 SDK dotnet/roslyn#76797

Open

disable format test

3462aa8

This was referenced Jan 18, 2025

slow macOS - "##[error]The job running on agent Azure Pipelines 9 ran longer than the maximum time of 60 minutes." dotnet/dnceng#1883

Open

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

mthalman mentioned this pull request Jan 21, 2025

Prevent Microsoft.AspNetCore.App.Internal.Assets from being restored when not needed #46165

Open

add link to issue

b5ac60b

Merge branch 'main' into issue4578

9518327

mthalman marked this pull request as ready for review January 21, 2025 14:15

mthalman requested review from a team as code owners January 21, 2025 14:15

MichaelSimons reviewed Jan 21, 2025

View reviewed changes

MichaelSimons approved these changes Jan 21, 2025

View reviewed changes

mthalman merged commit c559923 into dotnet:main Jan 21, 2025
34 of 38 checks passed

mthalman deleted the issue4578 branch January 21, 2025 16:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Re-enable SB tests #46055

Re-enable SB tests #46055

mthalman commented Jan 16, 2025 •

edited

Loading

mthalman commented Jan 16, 2025

mthalman commented Jan 16, 2025

NikolaMilosavljevic commented Jan 17, 2025

tmds commented Jan 17, 2025 •

edited

Loading

mthalman commented Jan 17, 2025

tmds commented Jan 17, 2025

mthalman commented Jan 17, 2025

MackinnonBuck commented Jan 17, 2025

tmds commented Jan 17, 2025 •

edited

Loading

MackinnonBuck commented Jan 17, 2025

MackinnonBuck commented Jan 17, 2025

maraf commented Jan 20, 2025 •

edited

Loading

tmds commented Jan 20, 2025

maraf commented Jan 20, 2025

mthalman commented Jan 21, 2025

MichaelSimons left a comment

mthalman commented Jan 21, 2025

tmds commented Jan 22, 2025

MackinnonBuck commented Jan 22, 2025

Re-enable SB tests #46055

Re-enable SB tests #46055

Conversation

mthalman commented Jan 16, 2025 • edited Loading

mthalman commented Jan 16, 2025

DotNetWatchTests.WatchTests

WebScenarioTests.VerifyScenario

DotNetFormatTests.FormatProject

SymbolsTests.VerifySdkSymbols

mthalman commented Jan 16, 2025

NikolaMilosavljevic commented Jan 17, 2025

tmds commented Jan 17, 2025 • edited Loading

mthalman commented Jan 17, 2025

tmds commented Jan 17, 2025

mthalman commented Jan 17, 2025

MackinnonBuck commented Jan 17, 2025

tmds commented Jan 17, 2025 • edited Loading

MackinnonBuck commented Jan 17, 2025

MackinnonBuck commented Jan 17, 2025

maraf commented Jan 20, 2025 • edited Loading

tmds commented Jan 20, 2025

maraf commented Jan 20, 2025

mthalman commented Jan 21, 2025

MichaelSimons left a comment

Choose a reason for hiding this comment

mthalman commented Jan 21, 2025

tmds commented Jan 22, 2025

MackinnonBuck commented Jan 22, 2025

mthalman commented Jan 16, 2025 •

edited

Loading

tmds commented Jan 17, 2025 •

edited

Loading

tmds commented Jan 17, 2025 •

edited

Loading

maraf commented Jan 20, 2025 •

edited

Loading