Getting "GenerateDepsFile" task failed unexpectedly with KeyNotFoundException when publishing project

[nguyenquyhy]

I am getting "GenerateDepsFile" task failed unexpectedly with KeyNotFoundException when publishing my projects.

Below are the conditions that trigger the issue:

• .NET Core 3.0 Preview 4
• WindowsAzure.Storage 9.3.3 is in a .NET Standard library

This can be fixed by either going back to Preview 3 or add WindowsAzure.Storage directly into the .NET Core project.

Error in build log:

C:\Program Files\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): Error MSB4018: The "GenerateDepsFile" task failed unexpectedly.
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Microsoft.NET.Build.Tasks.DependencyContextBuilder2.CalculateExcludedLibraries() in /_/src/Tasks/Microsoft.NET.Build.Tasks/DependencyContextBuilder2.cs:line 769
   at Microsoft.NET.Build.Tasks.DependencyContextBuilder2.Build() in /_/src/Tasks/Microsoft.NET.Build.Tasks/DependencyContextBuilder2.cs:line 215
   at Microsoft.NET.Build.Tasks.GenerateDepsFile.WriteDepsFileNew(String depsFilePath) in /_/src/Tasks/Microsoft.NET.Build.Tasks/GenerateDepsFile.cs:line 248
   at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() in /_/src/Tasks/Microsoft.NET.Build.Tasks/GenerateDepsFile.cs:line 284
   at Microsoft.NET.Build.Tasks.TaskBase.Execute() in /_/src/Tasks/Common/TaskBase.cs:line 35
   at Microsoft.NET.Build.Tasks.GenerateDepsFile.Execute() in /_/src/Tasks/Microsoft.NET.Build.Tasks/GenerateDepsFile.cs:line 262
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()

Sample repository: https://github.com/nguyenquyhy/IssueWindowsAzureStoragePublish

[DenSmoke]

Same error on Azure Pipelines, but with 'NETStandard.Library':

##[section]Starting: dotnet publish
==============================================================================
Task         : .NET Core
Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command. For package commands, supports NuGet.org and authenticated feeds like Package Management and MyGet.
Version      : 2.150.1
Author       : Microsoft Corporation
Help         : [More Information](https://go.microsoft.com/fwlink/?linkid=832194)
==============================================================================
[command]C:\windows\system32\chcp.com 65001
Active code page: 65001
[command]C:\hostedtoolcache\windows\dotnet\dotnet.exe publish d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj -c Release -r linux-x64 --self-contained
Microsoft (R) Build Engine version 16.1.54-preview+gd004974104 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 37.56 ms for d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj.
  Restore completed in 316.6 ms for d:\a\1\s\Libs\DpLib\DpLib.csproj.
  Restore completed in 269.87 ms for d:\a\1\s\Libs\TechWiz.Lib\TechWiz.Lib.csproj.
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(151,5): message NETSDK1057: You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(151,5): message NETSDK1057: You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview [d:\a\1\s\Libs\TechWiz.Lib\TechWiz.Lib.csproj]
  DpLib -> d:\a\1\s\Libs\DpLib\bin\Release\netstandard2.0\DpLib.dll
  TechWiz.Lib -> d:\a\1\s\Libs\TechWiz.Lib\bin\Release\netstandard2.0\TechWiz.Lib.dll
  TechWiz.Server -> d:\a\1\s\Apps\TechWiz.Server\bin\Release\netcoreapp3.0\linux-x64\TechWiz.Server.dll
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018: The "GenerateDepsFile" task failed unexpectedly. [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018: System.Collections.Generic.KeyNotFoundException: The given key 'NETStandard.Library' was not present in the dictionary. [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at System.Collections.Generic.Dictionary`2.get_Item(TKey key) [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at Microsoft.NET.Build.Tasks.DependencyContextBuilder2.CalculateExcludedLibraries() in /_/src/Tasks/Microsoft.NET.Build.Tasks/DependencyContextBuilder2.cs:line 769 [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at Microsoft.NET.Build.Tasks.DependencyContextBuilder2.Build() in /_/src/Tasks/Microsoft.NET.Build.Tasks/DependencyContextBuilder2.cs:line 217 [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.WriteDepsFileNew(String depsFilePath) in /_/src/Tasks/Microsoft.NET.Build.Tasks/GenerateDepsFile.cs:line 248 [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() in /_/src/Tasks/Microsoft.NET.Build.Tasks/GenerateDepsFile.cs:line 284 [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() in /_/src/Tasks/Common/TaskBase.cs:line 35 [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.Execute() in /_/src/Tasks/Microsoft.NET.Build.Tasks/GenerateDepsFile.cs:line 262 [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
C:\hostedtoolcache\windows\dotnet\sdk\3.0.100-preview4-011223\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Publish.targets(801,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj]
##[error]Error: The process 'C:\hostedtoolcache\windows\dotnet\dotnet.exe' failed with exit code 1
##[error]Dotnet command failed with non-zero exit code on the following projects : d:\a\1\s\Apps\TechWiz.Server\TechWiz.Server.csproj
##[section]Finishing: dotnet publish```

[Zoxive]

I'm getting this as well, and its blocking me from updating my dev project to preview 4. (I'm currently using preview 3 and have some bugs that are fixed with 4...)

In my case it seems to be blowing up with the dictionary key name being the name of a reference inside a build.prop file.

[Zoxive]

@dsplaisted will this be addressed for preview 5?

[dsplaisted]

I'm not sure if this has been fixed for preview 5 or not. I will take a look. However, you can work around the issue by setting the following MSBuild property:

<DepsFileGenerationMode>old</DepsFileGenerationMode>

[dsplaisted]

I've investigated and figured out what this is. With the repro provided (https://github.com/nguyenquyhy/IssueWindowsAzureStoragePublish), this will not fail. However, the bug still exists and could occur in other scenarios.

The repro for preview 5 is to publish the console app with the following 2 projects:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="WindowsAzure.Storage" Version="9.3.3" />
  </ItemGroup>
  
</Project>

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\ClassLibrary\ClassLibrary.csproj" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="12.0.2" Publish="false" />
  </ItemGroup>

</Project>

What happens is that the class library references a package that depends on NETStandard.Library 1.6.1. However, the implementation of building libraries targeting .NET Standard 2.0 involves a reference to NETStandard.Library 2.0.3 with PrivateAssets="all", which prevents that reference from flowing to the console app. So the console app ends up without any library for NETStandard.Library in its assets file, even though there are other libraries in the assets file that list NETStandard.Library as a dependency.

In the "new" deps file generation code, this causes an exception in the logic that processes the packages to exclude from publish. This logic is only run if there are package references where Publish="false".

The original repro doesn't fail on preview 5, because the Publish="false" package that was triggering the logic to be run was the app host pack reference (which has PrivateAssets="all" which implies Publish="false"), and in preview 5 the app host pack is included as part of the SDK installation, so we don't have to add the PrivateAssets="all" PackageReference to acquire it.

[kingboyk]

I was getting this error when attempting to publish a Linux-x64 package using VS 2019 Preview and the latest nightly build of .NET Core 3.0 Preview 6 ('latest' as in downloaded and installed today, 12 May). The <DepsFileGenerationMode>old</DepsFileGenerationMode> workaround does fix it, however.

[stuarth-inova]

I know this is closed - but I am having this problem now on a Xamarin project, all of a sudden, and installing the latest is not a great option at the moment. Where do I put this code?

<DepsFileGenerationMode>old</DepsFileGenerationMode>

As in what file(s) to put this into?

[nielscup]

Same here, all newly created Xamarin.Forms projects have this issue. Where do we put this config?

[krishnarajbb]

I had .net standard project built on .net standard 2.0. Yesterday I updated my Visual Studio 2019 to 16.3.0 to leverage .Net Core 3.0. But all of sudden build of project targeting on standard 2.0 started throwing "GenerateDepsFile" task failed unexpectedly error.

I was able to resolve this issue by changing the target framework to 2.1.