coreclr build failure: Unable to find package Microsoft.AspNetCore.App.Runtime.win-arm #185

BruceForstall · 2019-11-21T19:31:05Z

In a clean runtime clone, I run src\coreclr\build.cmd skipbuildpackages (using skipbuildpackages to get around #181) and after the native test components build, the build fails with:

BUILDTEST: Restoring CoreCLR product from packages
  [11:27:25.30] Restoring all packages...
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: Unable to find package Microsoft.AspNetCore.App.Runtime.win-arm with version (= 5.0.0-alpha1.19470.6) [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 178 version(s) in dotnet-core [ Nearest version: 5.0.0-alpha1.19472.2 ] [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 11 version(s) in nuget.org [ Nearest version: 3.1.0-preview3.19555.2 ] [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-corefxlab [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-coreclr [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-eng [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet5-transport [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: Unable to find package Microsoft.AspNetCore.App.Runtime.win-x64 with version (= 5.0.0-alpha1.19470.6) [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 175 version(s) in dotnet-core [ Nearest version: 5.0.0-alpha1.19472.2 ] [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 12 version(s) in nuget.org [ Nearest version: 3.1.0-preview3.19555.2 ] [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-corefxlab [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-coreclr [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-eng [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet5-transport [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: Unable to find package Microsoft.AspNetCore.App.Runtime.win-x86 with version (= 5.0.0-alpha1.19470.6) [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 151 version(s) in dotnet-core [ Nearest version: 5.0.0-alpha1.19472.2 ] [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 12 version(s) in nuget.org [ Nearest version: 3.1.0-preview3.19555.2 ] [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-corefxlab [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-coreclr [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet-eng [f:\gh\runtime2\src\coreclr\tests\build.proj]
f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj : error NU1102: - Found 0 version(s) in dotnet5-transport [f:\gh\runtime2\src\coreclr\tests\build.proj]
    Restore failed in 803.73 ms for f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj.
f:\gh\runtime2\src\coreclr\tests\build.proj(56,5): error MSB3073: The command ""C:\Program Files\dotnet\dotnet.exe" restore f:\gh\runtime2\src\coreclr\tests\src\Common\test_dependencies\test_dependencies.csproj  /p:SetTFMForRestore=true /p:__BuildOS=Windows_NT /p:__BuildArch=x64 /p:__BuildType=Debug " exited with code 1.

@dotnet/runtime-infrastructure

The text was updated successfully, but these errors were encountered:

jkoritzinsky · 2019-11-21T19:37:42Z

I've seen this in the CoreCLR repo when I've had a 5.0.0 SDK installed on my machine when trying to build. Never quite figured out the underlying issue, but uninstalling the 5.0.0 SDK, deleting the artifacts folder, and deleting the .dotnet folder if it exists usually fixed it for me.

ViktorHofer · 2019-11-21T19:39:08Z

Why do we depend on Microsoft.AspNetCore.App.Runtime?

BruceForstall · 2019-11-21T19:42:23Z

fwiw, I do have dotnet SDK version 5.0.100-alpha1-014888 on my machine

jkoritzinsky · 2019-11-21T19:47:36Z

We don't actually depend on it AFAIK. If I remember from my last investigation, for some reason the 5.0.100-* SDK had been grabbed at some point (restore time maybe) and left artifacts around about which FrameworkReferences it had installed. When publishing crossgen2, the 3.0.100 SDK can't find those FrameworkReferences (rightly so) and gets confused (as shown above).

safern · 2019-11-21T19:57:42Z

I believe this is because the preview SDKs reference all KnownFrameworkReferences directly. Actually if you look at the docs in the dogfooding of the SDK they suggest adding some restore sources.

https://github.com/dotnet/core-sdk#installers-and-binaries

What we can do is remove the AspNetCore and WIndows.Desktop KnownFrameworkReferences.

runtime/src/libraries/pkg/test/frameworkSettings/netcoreapp5.0/settings.targets

Lines 32 to 33 in 6492892

    
           <KnownFrameworkReference Remove="Microsoft.AspNetCore.App" /> 
        
           <KnownFrameworkReference Remove="Microsoft.WindowsDesktop.App" />

Also, note that it is the recommendation that you set DOTNET_MULTILEVEL_LOOKUP=0 in your environment globally when building our repos, so that it actually builds using the dotnet specified in the global.json.

BruceForstall · 2019-11-21T21:55:48Z

Also, note that it is the recommendation that you set DOTNET_MULTILEVEL_LOOKUP=0 in your environment globally when building our repos

Shouldn't our build scripts do that? E.g., build.cmd, build-test.cmd, etc.?

safern · 2019-11-21T23:04:32Z

Yeah, they should. It seems like build.cmd does as it calls msbuild.ps1 to restore S.P.CoreLib and that goes through tools.ps1

https://github.com/dotnet/runtime/blob/master/eng/common/tools.ps1#L107

So then there something else wrong here that we should investigate.

@ericstj might have some ideas.

ericstj · 2019-11-21T23:35:16Z

/cc @dsplaisted I believe this behavior is expected. The CLI will download these if you don't have them installed and build a standalone app. @safern pointed out how we turn this off for our package testing.

BruceForstall · 2019-11-22T16:56:13Z

@safern I notice in dotnet.cmd we have:

:: Don't resolve runtime, shared framework, or SDK from other locations to ensure build determinism
set DOTNET_MULTILEVEL_LOOKUP=0

:: Disable first run since we do not need all ASP.NET packages restored.
set DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1

So that path to invoking dotnet CLI sets it also.

So what is the fix?

ericstj · 2019-11-22T17:32:57Z

If you don't want the CLI to download these, you need to remove the KnownFrameworkReference items.

BruceForstall · 2019-11-22T17:57:13Z

@ericstj I guess I don't understand: why should my global SDK install have any impact whatsoever on my product build?

And do you mean remove these lines from src\libraries\pkg\test\frameworkSettings\netcoreapp5.0\settings.targets?

<KnownFrameworkReference Remove="Microsoft.AspNetCore.App" />
<KnownFrameworkReference Remove="Microsoft.WindowsDesktop.App" />

because when you write "If you don't want the CLI to download these, you need to remove the KnownFrameworkReference items", it looks to me like those lines, already using "Remove=" syntax, is already going to "remove the KnownFrameworkReference items" -- but I really have no idea what I'm looking at or talking about here.

ericstj · 2019-11-22T18:12:53Z

No, I mean, if you want to restore a self-contained project like https://github.com/dotnet/runtime/blob/master/src/coreclr/tests/src/Common/test_dependencies/test_dependencies.csproj you need to remove the KnownFrameworkReference items yourself, just like we do in our own package testing to avoid the same problem. You'll probably hit this whenever you use a 3.0 or later SDK to restore a self-contained netcoreapp3.0 project, so even if it wasn't something you had to worry about before because you were using an old SDK, you should address it. The fix would be to add those remove items to that test_dependencies.csproj or some common targets file as appropriate. Specifically, copy the following and dump it in that project.

  <ItemGroup>
    <KnownFrameworkReference Remove="Microsoft.AspNetCore.App" />
    <KnownFrameworkReference Remove="Microsoft.WindowsDesktop.App" />
  </ItemGroup>

The fact that your build happens to be using the machine wide SDK is because you're launching the restore here:

runtime/src/coreclr/build-test.cmd

Lines 295 to 300 in 82d1049

    
           powershell -NoProfile -ExecutionPolicy ByPass -NoLogo -Command "%__RepoRootDir%\eng\common\msbuild.ps1" %__ArcadeScriptArgs%^ 
        
             %__ProjectDir%\tests\build.proj -warnAsError:0 /t:BatchRestorePackages /nodeReuse:false^ 
        
             /p:RestoreDefaultOptimizationDataPackage=false /p:PortableBuild=true^ 
        
             /p:UsePartialNGENOptimization=false /maxcpucount^ 
        
             %__SkipFXRestoreArg%^ 
        
             !__Logging! %__CommonMSBuildArgs% %__PriorityArg% %__UnprocessedBuildArgs%

Which uses the dotnet found by the arcade tools:

runtime/eng/common/tools.ps1

Line 397 in 82d1049

    
           $buildTool = @{ Path = Join-Path $dotnetRoot "dotnet.exe"; Command = "msbuild"; Tool = "dotnet"; Framework = "netcoreapp2.1" }

I believe it is expected that the arcade tools choose either the local install or machine wide, depending on what satisfies the version listed in global.json.

ericstj · 2019-11-22T18:18:57Z

Also, the reason why this might appear to come and go is that the restore will only be necessary once. After the SDK successfully downloads those runtime packs, it will no longer complain. The version that it needs ends up being tied to the version of the SDK in the global.json. The right fix is to disable this part of the SDK's download using the snippet I shared above. You might do it here, just in case you restore any other projects in this way: https://github.com/dotnet/runtime/blob/64928927e67f48ade7cbb3fc972c3901a5f5696f/src/coreclr/tests/src/Common/Directory.Build.targets

jkoritzinsky · 2019-11-22T18:19:50Z

The restore for the projects under coreclr/tests/src/Common actually gets invoked here, which is why it might have problems:

runtime/src/coreclr/tests/build.proj

Lines 50 to 57 in 82d1049

    
           <Target Name="RestorePackage"> 
        
             <PropertyGroup> 
        
               <_ConfigurationProperties>/p:__BuildOS=$(__BuildOS) /p:__BuildArch=$(__BuildArch) /p:__BuildType=$(__BuildType)</_ConfigurationProperties> 
        
               <DotnetRestoreCommand Condition="'$(__DistroRid)' == ''">"$(DotNetTool)" restore $(RestoreProj) $(PackageVersionArg) /p:SetTFMForRestore=true $(_ConfigurationProperties) $(__SkipFXRestoreArg)</DotnetRestoreCommand> 
        
               <DotnetRestoreCommand Condition="'$(__DistroRid)' != ''">"$(DotNetTool)" restore -r $(__DistroRid) $(RestoreProj) $(PackageVersionArg) /p:SetTFMForRestore=true $(_ConfigurationProperties) $(__SkipFXRestoreArg)</DotnetRestoreCommand> 
        
             </PropertyGroup> 
        
             <Exec Command="$(DotnetRestoreCommand)"/> 
        
           </Target>

BruceForstall · 2019-11-22T18:29:35Z

I believe it is expected that the arcade tools choose either the local install or machine wide, depending on what satisfies the version listed in global.json.

This seems to be the root of the problem; why would we ever want the machine-wide install?

tools.ps1 has:

# True to attempt using .NET Core already that meets requirements specified in global.json
# installed on the machine instead of downloading one.
[bool]$useInstalledDotNetCli = if (Test-Path variable:useInstalledDotNetCli) { $useInstalledDotNetCli } else { $true }

shouldn't we force useInstalledDotNetCli to false in the build system? It looks like I don't have a ".dotnet" directory, so Arcade (?) decided to always use whatever global version I have installed?

safern · 2019-11-22T18:34:47Z

shouldn't we force useInstalledDotNetCli to false in the build system? It looks like I don't have a ".dotnet" directory, so Arcade (?) decided to always use whatever global version I have installed?

We specify the version of the dotnet that we want in global.json. So if arcade finds that version installed machine wide, in order to have a faster build, it doesn't install it under .dotnet, it just uses the global one.

However, we also specify in the global.json to allow rolling forward of the sdk for a major version change, so that's why it is picking the new SDK in your machine. This is to allow people building on VS and having a newer SDK installed (without forcing them to install the exact version we depend, machine wide).

runtime/global.json

Lines 3 to 5 in 82d1049

    
           "version": "3.0.100", 
        
           "allowPrerelease": true, 
        
           "rollForward": "major"

I think it makes sense to add the lines I shared to remove KnownFrameworkReferences into the common tests Directory.Build.targets

ericstj · 2019-11-22T18:39:27Z

I also want to point out that machine wide vs local is red herring. That has nothing to do with the error you are seeing, other than the fact that the version of the SDK determines the version of the runtime packs it tries to install.

ericstj · 2019-11-22T19:35:47Z

After @BruceForstall and @safern and I chatted offline we agreed to do the following:

@BruceForstall will add the KnownFrameworkReference Remove workaround.
@ericstj will file an issue on SDK resolver.
@safern will follow up to consider modification to dotnet/runtime's global.json.

As part of looking into 2, I found that the SDK resolver should actually prefer the closest match: https://github.com/dotnet/core-setup/blob/27063871b42e5c4ee80b8953ebafecb98381d39e/src/corehost/cli/fxr/sdk_resolver.cpp#L376-L400

When I tried this I did observe that behavior. This was a recent change dotnet/core-setup#6953 (/cc @peterhuene @vitek-karas), so its possible the sdkresolver bits you had did not have this fix. Looking at the version of the SDK you had it is 5.0.0-alpha1.19470.6. I believe this build would have fallen in the window where the 5.0 SDK was lagging behind 3.0 and 3.1, I bet if you update to a newer 5.0 machine wide SDK you'll get @peterhuene's feature that handles rollForward and the issue will also go away.

Net result: 2 and 3 should not be needed here. If other folks hit this for early builds of the 5.0 SDK then update to the latest 5.0 SDK to get a fix.

…#225) Fixes #185

Merge runtime/master into feature/NativeAOT

BruceForstall added the area-Infrastructure-coreclr label Nov 21, 2019

Dotnet-GitSync-Bot added the untriaged New issue has not been triaged by the area owner label Nov 21, 2019

BruceForstall mentioned this issue Nov 22, 2019

Fix test build error when .NET Core SDK 5 is installed on the machine #225

Merged

ViktorHofer closed this as completed in #225 Nov 25, 2019

ViktorHofer pushed a commit that referenced this issue Nov 25, 2019

Fix test build error when .NET Core SDK 5 is installed on the machine (…

4842a7e

…#225) Fixes #185

MichalStrehovsky pushed a commit to MichalStrehovsky/runtime that referenced this issue Nov 10, 2020

Merge pull request dotnet#185 from safern/MergeRuntime

dac76cc

Merge runtime/master into feature/NativeAOT

ghost locked as resolved and limited conversation to collaborators Dec 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

coreclr build failure: Unable to find package Microsoft.AspNetCore.App.Runtime.win-arm #185

coreclr build failure: Unable to find package Microsoft.AspNetCore.App.Runtime.win-arm #185

BruceForstall commented Nov 21, 2019

jkoritzinsky commented Nov 21, 2019

ViktorHofer commented Nov 21, 2019

BruceForstall commented Nov 21, 2019 •

edited

Loading

jkoritzinsky commented Nov 21, 2019

safern commented Nov 21, 2019

BruceForstall commented Nov 21, 2019

safern commented Nov 21, 2019

ericstj commented Nov 21, 2019

BruceForstall commented Nov 22, 2019

ericstj commented Nov 22, 2019

BruceForstall commented Nov 22, 2019

ericstj commented Nov 22, 2019

ericstj commented Nov 22, 2019

jkoritzinsky commented Nov 22, 2019

BruceForstall commented Nov 22, 2019

safern commented Nov 22, 2019

ericstj commented Nov 22, 2019

ericstj commented Nov 22, 2019

coreclr build failure: Unable to find package Microsoft.AspNetCore.App.Runtime.win-arm #185

coreclr build failure: Unable to find package Microsoft.AspNetCore.App.Runtime.win-arm #185

Comments

BruceForstall commented Nov 21, 2019

jkoritzinsky commented Nov 21, 2019

ViktorHofer commented Nov 21, 2019

BruceForstall commented Nov 21, 2019 • edited Loading

jkoritzinsky commented Nov 21, 2019

safern commented Nov 21, 2019

BruceForstall commented Nov 21, 2019

safern commented Nov 21, 2019

ericstj commented Nov 21, 2019

BruceForstall commented Nov 22, 2019

ericstj commented Nov 22, 2019

BruceForstall commented Nov 22, 2019

ericstj commented Nov 22, 2019

ericstj commented Nov 22, 2019

jkoritzinsky commented Nov 22, 2019

BruceForstall commented Nov 22, 2019

safern commented Nov 22, 2019

ericstj commented Nov 22, 2019

ericstj commented Nov 22, 2019

BruceForstall commented Nov 21, 2019 •

edited

Loading