[Bug]: under certain conditions package restore fails with ProjectNotNominatedException when NuGetPackage is not initialized in VS

[crusader-mike]

NuGet Product Used

MSBuild.exe

Product Version

VS 2019 v16.11.5

Worked before?

No response

Impact

It's more difficult to complete my work

Repro Steps & Context

I have a single C++ project that I have modified (following instructions here) to provide PackageReference support:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup Label="ProjectConfigurations">
    ...
    <ProjectCapability Include="PackageReferences" />
  </ItemGroup>
  <PropertyGroup Label="Globals">
    ...
    <NuGetTargetMoniker Condition="'$(NuGetTargetMoniker)' == ''">native,Version=v0.0</NuGetTargetMoniker>
    <RuntimeIdentifiers Condition="'$(RuntimeIdentifiers)' == ''">win;win-x86;win-x64;win-arm;win-arm64</RuntimeIdentifiers>
  </PropertyGroup>
  <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
  ...
  <ItemGroup>
    <PackageReference Include="..." Version="..."/>
  </ItemGroup>
  <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
</Project>

everything works fine (nuget restore, etc). But if you open solution in (fresh instance of) VS2019, right-click project and select Build -- it produces this in Build output window:

1>------ Build started: Project: CommonLib, Configuration: Debug x64 ------
NuGet package restore failed. Please see Error List window for detailed warnings and errors.
...

and build fails (sometimes not) due to missing header files/etc (that are located in referenced package). Package itself is already restored and sits in %home%\.nuget\packages.

Running with detailed build output produces:

Build started...
1>------ Up-To-Date check: Project: Common\CommonLib\CommonLib.vcxproj, Configuration: Debug x64 ------
1>Project is not up-to-date: last build was unsuccessful.
1>------ Build started: Project: CommonLib, Configuration: Debug x64 ------
NuGet package restore failed. Please see Error List window for detailed warnings and errors.
Error occurred while restoring NuGet packages: NuGet.PackageManagement.VisualStudio.Exceptions.ProjectNotNominatedException: The operation failed as details for project CommonLib could not be loaded.
   at NuGet.PackageManagement.VisualStudio.CpsPackageReferenceProject.GetPackageSpecsAndAdditionalMessagesAsync(DependencyGraphCacheContext context)
   at NuGet.PackageManagement.DependencyGraphRestoreUtility.<GetSolutionRestoreSpecAndAdditionalMessages>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.SolutionRestoreManager.SolutionRestoreJob.<RestorePackageSpecProjectsAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at NuGet.SolutionRestoreManager.SolutionRestoreJob.<RestoreAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at NuGet.SolutionRestoreManager.SolutionRestoreJob.<RestoreAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.SolutionRestoreManager.SolutionRestoreJob.<ExecuteAsync>d__21.MoveNext()

which points to this line -- it looks like NuGet maintains some sort of cache with project-related info and at this point it is expected that cache is already populated. I also noticed that this function uses project full path (while few other places use project full name) as a lookup key. It is my first time looking at NuGet source code, so I can't really comment on what is going on.

In cases when error message is generated and build fails detailed logs has this line buried in the output:

1>Target "ResolveNuGetPackageAssets" skipped, due to false condition; ('$(ResolveNuGetPackages)' == 'true' and exists('$(ProjectLockFile)')) was evaluated as ('true' == 'true' and exists('obj\project.assets.json')).

which suggests that at this stage project.assets.json was not created at that moment yet. Rerunning build works fine (no message and build succeeds).

There seem to be some race condition somewhere because there are basically three possible behaviors:

• all works fine
• NuGet package restore failed message and build works fine
• NuGet package restore failed message and build fails (usually after I do Clean, restart VS and click Build again)

I am pushing the envelope here since C++ projects do not officially support PackageReference. I don't expect a bugfix, but I am hoping there is a workaround for this problem -- maybe there is a directive to add to .vcxproj that will ensure package restore stage is delayed until _projectSystemCache is populated?

Maybe an MSBuild construct that will create a dependency between certain tasks that will guarantee population of said cache before it is used?

(if I click Restore NuGet Packages before building my project -- all works fine)

Verbose Logs

No response

[crusader-mike]

Note: simply opening Package Manager Console (or having it open by default) is enough to make the problem go away.
(same if you open Manage NuGet Packages window at least once. Basically initializing in-built NuGet extension in any way causes problem to go away)

All I need is to figure out a way to get VS2019 to perform related steps (that NuGet.SolutionRestoreManager.SolutionRestoreJob needs to succeed) before kicking off build...

[zivkan]

As you mentioned C++ projects don't officially support PackageReference. Therefore we unfortunately can't commit resources to investigating this.

[crusader-mike]

@zivkan Is there any way to force initialization of NuGetPackage in VS? Not asking you to "commit resources" -- maybe you know it from the top of your head.

[zivkan]

If you open the Package Manager Console, it will load. VS lazy-loads tool windows, so if you restart VS and see PMC as a "background" tab, PMC won't be loaded, so you'll need to click on it every time you restart VS. Alternatively, open Tools->Options and open either of the NuGet options pages, but this is much slower than opening PMC.

Depending on how motivated you are, you could create your own VS extension that uses ProvideAutoLoadAttribute to either load on VS open, or on solution open, and all your extension does it to tell the VS shell to load the NuGet package.

[crusader-mike]

@zivkan This is actually an interesting idea -- I'll see if I can put this VS extension together. I know very little about NuGet integration with VS and I really appreciate your help, thank you.

I guess, when you press Build in VS it kicks off MSBuild process that restores NuGet packages and this somehow ties into NuGetPackage (which at this point may or may not be initialized yet). I wonder if I could inject a build target (e.g. via Directory.Build.targets at the root of my repo) that will delay MSBuild until NuGetPackage finishes initialization.

If it is not a bother for you -- can you direct me to some resource where I can read on how this (VS <--> NuGet) integration works, please?

[zivkan]

Restore in Visual Studio is not via MSBuild, which is the reason why all this is a problem. We can provide much faster no-op restore (up to date check, when no package or project references changed since the last time restore/build ran), not to mention VS specific credential providers for private feeds, avoid re-reading and parsing the nuget.config multiple times, by not participating in MSBuild. Instead NuGet calls VS and project system APIs directly, caching some information and when VS gives us the pre-build event, run restore our restore code directly, blocking VS from starting MSBuild until restore finishes. This is also the reason why build will run even if restore fails, because it's only hooking into the pre-build event that existed before NUGet did, VS doesn't really understand NuGet as a part of the build.

This is the only docs I know of that very briefly explains how NuGet talks to project systems in VS: https://github.com/NuGet/NuGet.Client/blob/dev/docs/nuget-project-types.md

Frankly, I don't understand why C++ PackageReference projects would work after the NuGetPackage was loaded by VS, but I've got too many other things to work on, so I'm afraid I'm just not motivated to debug this to understand.

[crusader-mike]

@zivkan Huh... This could explain why I don't have this problem in another solution (that in addition to C++ project also contains few .csproj). Maybe VS calls you only if certain conditions are met (like presence of officially supported project type in solution)...

Thank you for the link, I'll read it tonight.

Frankly, I don't understand why C++ PackageReference projects would work after the NuGetPackage was loaded by VS

It certainly works in my case (solution with ~120 projects, mix of C++, C++/CLI and C#). NuGet UI works too. All thanks to magic of ProjectCapability and AssetTargetFallback. The only real problem I have right now is these messages:

NU1201: Project CommonLibInterop is not compatible with net462 (.NETFramework,Version=v4.6.2) / win-x64. Project CommonLibInterop supports: native (native,Version=v0.0)

Basically whenever C# project refers to C++/CLI project (directly or transitively via reference to another C# project) -- this gets generated during restore. As of now I address it with global <AssetTargetFallback>$(AssetTargetFallback);native</AssetTargetFallback>, but would really love to find a way to declare CLI/C++ projects as "supporting .NETFramework,Version=v4.6.2". So far, no amount of TargetFrameworkMoniker / NuGetTargetMoniker / SetTargetFramework / etc magic convinced NuGet that this project supports .Net too. Simply not enough knowledge about NuGet/MsBuild -- I feel like a blind kitten. :) If you know a proper way to do it -- please, help.