Issues with .NET Standard 2.0 with .NET Framework & NuGet

[terrajobst]

Summary

We've designed .NET Standard & our tooling so that projects targeting .NET Framework 4.6.1 can consume NuGet packages & projects targeting .NET Standard 2.0 or earlier. Unfortunately, we've seen a few issues around that scenario. The purpose of this document is to summarize the issues, outline our plan on addressing them, and providing workarounds you can deploy with today's state of our tooling.

Symptoms and root cause

The primary symptom is that applications crash with a FileLoadException or a FileNotFoundException. Another symptom is warnings at build time regarding assembly versions. This is due to one or both of the following issues:

1. Missing binding redirects
2. Missing binaries that come from indirect NuGet packages

Missing binding redirects

.NET Standard 1.x was based around contracts. Many of these contracts shipped with .NET Framework 4.5 and later. However, different versions of .NET Framework picked up different versions of these contracts, as by-design of contract versioning. As a side effect of marking .NET Framework 4.6.1 as implementing .NET Standard 2.0, some projects will now start picking up binaries built for .NET Standard 1.5 and 1.6 (as opposed to previously where .NET Framework 4.6.1 was considered as implementing .NET Standard 1.4). This results in mismatches of the assembly versions between what was shipped in .NET Framework and what was part of .NET Standard 1.5/1.6.

This can be addressed by binding redirects. As writing them by hand sucks, we added an Automatic Binding Redirect Generation feature in .NET Framework 4.5.1. This feature is opt-in. Unfortunately, it's not enabled based on target framework, but by which target framework was selected when the project was created (as the feature is turned on via an MSBuild property that is conditionally emitted by the template). In practice, this means it's mostly off if you often upgrade existing projects, rather than creating new ones.

Missing binaries

There are two primary flavors of NuGet: packages.config and PackageReference.

• With packages.config, each project has a config file with a flattened graph of all the NuGet dependencies. The project file in turn has direct links to all the assets. The assets are selected at install time. None of this includes indirect NuGet references coming from referenced projects.

• With PackageReference each project contains MSBuild PackageReference items. The project file contains no references to any assets as the assets are selected at build time. Package restore will compute the graph of all packages, including indirect NuGet references coming from referenced projects.

The default for .NET Framework projects is packages.config. This ensures more compatibility because PackageReference doesn't support all the features that packages.config did, for example, PowerShell install scripts and content.

The only supported mode for SDK-style projects (.NET Core/.NET Standard) is PackageReference. This means that a .NET Framework project referencing a .NET Standard project ends up crossing the streams between two different NuGet models. When the .NET Standard project references NuGet packages that the .NET Framework project doesn't reference, the application ends up missing all binaries coming from those packages.

Why has this worked before? Because with packages.config, all dependencies are copied to each project's output folder. MSBuild copies them up from there. With PackageReference, we don't copy the binaries because it relies on the consuming project to see its dependencies and extract the proper asset itself. This allows the consuming project to pick up the right assets for packages that use bait & switch (which many of the .NET packages must do).

Plan

The plan is to address these issues moving forward as follows:

1. Converge on PackageReference for all project types, including .NET Framework. The short-term plan for (1) is to start blocking project-to-project references in Visual Studio 15.4 that will end up crossing the streams between packages.config and PackageReference. This block is UI only; you can still edit the reference by editing the project by hand. The error message will instruct you to switch the .NET Framework project to PackageReference if you want to reference a .NET Standard project. Referencing .NET Standard binaries or NuGet packages will not require this, it's only about project-to-project references. In later releases, we plan on providing a converter. The challenge is that packages.config has features we can't offer for PackagReference across the board, in particular PowerShell install scripts and content. We'll need good guidance and mitigations, if applicable.

2. Ensure binding redirects are on by default. Short term, this means we need to fix our target files to make sure we turn on automatic binding redirect generation. However, binding redirects don't work well in all scenarios, when there is no application project (like unit tests or add-ins). We need to work on a plan to bring the regular “higher wins” binding policy without binding redirects. This needs a proposal and proper vetting, but it seems we've reached the point where this is necessary.

Workarounds

Regular .NET Framework projects

1. Enable automatic binding redirects in the root .NET Framework application.
2. Make sure your root application project doesn't use packages.config but uses PackageReference for NuGet packages
   • If you currently don't have packages.config, simply add <RestoreProjectStyle>PackageReference</RestoreProjectStyle> to your project file
   • If you currently do have a packages.config, convert the contents to packages references in the project file. The syntax is like this:
     • <PackageReference Include="package-id" Version="package-version" />

ASP.NET web applications and web sites

1. Web applications and web sites don't support automatic binding redirect generation. In order to resolve binding conflicts, you need to double click the warning in the error list and Visual Studio will add them to your web.config file.
2. In web application projects, you should enable PackageReference like mentioned above. In web sites, you cannot use PackageReference as there is no project file. In that case, you need to install all NuGet packages into your web site that any of the direct or indirect project references depend on.

Unit tests projects

By default, binding redirects aren't added to class library projects. This is problematic for unit testing projects as they are essentially like apps. So in addition to what's outlined in automatic binding redirects you also need to specify GenerateBindingRedirectsOutputType:

<PropertyGroup>
    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
    <GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
</PropertyGroup>

[mungojam]

Thanks for this, things do seem a little unpolished, so it's useful to have some clarity.

Other things that currently make it painful for us to mix .net framework with .net standard are a FileNotFoundException that will make it difficult for us to migrate our internal packages in a safe way: NuGet/Home#5715 and the lack of visibility of indirect packages when switching to PackageReferences: UserVoice to convert all projects to new common project format

[mungojam]

Is it correct that PackageReference is not supported for anything other than .net core, .net standard and UWP? See the documentation:

At present, package references are supported in Visual Studio 2017 only, for .NET Core projects, .NET Standard projects, and UWP projects targeting Windows 10 Build 15063 (Creators Update).

Hopefully that will change in future as it is an option in Visual Studio and is the workaround that you describe.

[0x53A]

Hi @terrajobst, is there a document detailing which scenarios are supported for .NET Framework 4.6.1 and .NET Standard 1.6?

I created a net461 old-sdk project in VS2017.3 and added a .NET Standard 1.6 NuGet. The build automagically copied the required shims to the output, totalling 100 files. ✅

If I open the exact same project in VS2015 and build, the output contains only 4 files, so the shims are missing. ❌

---

Where do the shims come from, anyway? I was following some earlier discussions, and I know at first they were deployed via a NuGet package. But later discussions talked about some machine-wide installation. What was the final result? What do we need to deploy to a build server to enable this scenario?

[terrajobst]

@mungojam

Is it correct that PackageReference is not supported for anything other than .net core, .net standard and UWP?

You can use PackageReference in .NET Framework projects, but as I outlined there is a loss in certain features. For instance, if you install packages that rely on PowerShell scripts (for instance, EF6) they will no longer work. Also, packages that require content (i.e. source files that are copied to the consuming project) they will also no work.

@0x53A

Which NuGet package are you consuming? For .NET Standard 1.x, the package is supposed to depend on NETStandard.Library which will bring in the shims.

[0x53A]

@terrajobst repro is here: https://github.com/0x53A/n16test

The nupkg was created calling "dotnet pack" on
https://github.com/0x53A/n16test/blob/75fcfdc0d540531c30b805e74bb239e42d13ffea/ns16lib/ns16lib.csproj#L1-L7.

The resulting package looks like this:

To reproduce my issue:

1. open https://github.com/0x53A/n16test/blob/master/net461exe/net461exe/net461exe.sln in VS 2017.3 and build.
2. note that /bin/ contains 100 files.
3. clean /bin/, re-open in VS 2015.3 and build
4. note that /bin/ contains only 4 files.

[FransBouma]

In vs2015, in a csproj targeting v4.6.2, I wanted to add a reference to Entity Framework Core 2.0, but that failed with this error:

Attempting to gather dependency information for package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0' with respect to project 'EF Core 2\NWEFCore2.Persistence', targeting '.NETFramework,Version=v4.6.2'
Gathering dependency information took 0,87 ms
Attempting to resolve dependencies for package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0' with DependencyBehavior 'Lowest'
Resolving dependency information took 0 ms
Resolving actions to install package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0'
Resolved actions to install package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0'
Retrieving package 'Microsoft.EntityFrameworkCore.SqlServer 2.0.0' from 'nuget.org'.
Install failed. Rolling back...
Package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0' does not exist in project 'NWEFCore2.Persistence'
Package 'Microsoft.EntityFrameworkCore.SqlServer.2.0.0' does not exist in folder 'C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.3\UnitTests\EntityFramework\packages'
Executing nuget actions took 62,07 ms
install-package : Could not install package 'Microsoft.EntityFrameworkCore.SqlServer 2.0.0'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.6.2', but the package does not contain any assembly references or content fi
les that are compatible with that framework. For more information, contact the package author.
At line:1 char:1
+ install-package Microsoft.EntityFrameworkCore.SqlServer
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Install-Package], Exception
    + FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PackageManagement.PowerShellCmdlets.InstallPackageCommand

Likely a bug in the Nuget client in 2015 (but I am under the assumption I have the latest version installed), but IMHO this should simply work as .NET 4.6.2 supports .NET standard 2.0

(edit) This does work in vs2017 15.3.x. That's not an excuse to not fix this! Many devs are on 2015 and have no plans to move to 2017 (yet) and as .net standard 2.0 is supported on .net 4.6.2, they should be able to expect this to just work.

I don't know if this is the right location to file this (likely not!), if not please point me to the repo where I should file this, thanks.

[isaacabraham]

cc @forki I wonder what impact this has on Paket vis-a-vis automatic binding redirects (Paket has the option to generate BRs on demand based on the dependency graph).

@terrajobst If there's a BR already in the app.config file and you turn on the automatic BR redirection, who wins?

[forki]

@isaacabraham I have no idea. this will bring interesting new problems to us.

[FransBouma]

And another problem:

Using EF Core 2.0 in a .NET 4.6.2 project: as soon as you create a DbContext derived type, you'll get this error:

System.IO.FileLoadException : Could not load file or assembly 'System.ValueTuple, Version=0.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
  Exception doesn't have a stacktrace

Likely caused by the same issue as the ValueTuple containing assembly isn't there. This is in vs2017 15.3.x btw.

Brb, throwing some heavy furniture against the wall screaming.

[0x53A]

@FransBouma This looks familiar: #476 (comment)

[matthid]

How about dropping this scenario and make it work "cleanly" with a future version instead of trying to get with the head through the wall?

[dasMulli]

Note that the binding redirection documentation is missing the critical

<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>

property which is crucial to making "classic" test projects work (as well as libraries loaded via reflection).

[FransBouma]

@0x53A yes, I think that's similar. The bindingredirect however kills nunit for some reason (the test is now simply not seen by it and not run).

@dasMulli Thanks for that, but I am at a loss where to add that... adding it to the bindingredirect nor the csproj etc. does make nunit work again.

And here I am hoping that after weeks of fighting with MS pre-alpha quality tooling (Where's the QA testing? We paid for this software!), this week would go smoothly. Nope

(edit) fsck it, converting to new csproj format, perhaps that gives some results.

(edit) for ref: dotnet/sdk#1070 adding the 2 redirects made nunit work again. With the new csproj format the bindingredirect isn't needed, the valuetuple issue isn't popping up, but the attribute referred by @dasMulli are required to make things work.

Man... one could start wondering what 'alpha' quality means these days if 'RTM' is this quality.

[aliostad]

On the announcement, the "primary symptom" link is broken (via @forki )

[dasMulli]

@FransBouma I wasn't directly referring to your problem but rather pointing out that the official documentation and announcement issue lacks this information (cc @terrajobst).

For VS 2015, there is an additional extension to install for .NET Standard 2.0 support - When using NuGet 3.6.0, it will print out the link. However, I've tried it but it is currently broken anyway for most scenarios: dotnet/sdk#1539

[FransBouma]

@dasMulli oh sorry about that, but in any case it helped me a lot, so thanks for your input :)
Yeah I think 2015 is a lost cause at the moment. Thanks for referring to the nuget link, I didn't know I had to install a separate extension. In my 2015 installation it didn't mention that.

[mariusGundersen]

When you say

simply add PackageReference to your project file

I'm assuming this means the csproj has to use the VS2017 style csproj, as in, it needs to start with <Project Sdk="Microsoft.NET.Sdk"> and it needs to be built using msbuild 15 or the dotnet cli, right? Only working in Visual Studio is not much use for everyone (and I hope this is the majority) who use CI/CD pipelines. So this means that we can't really use netstandard nugets without converting the csproj file to vs2017 style.

[dasMulli]

@mariusGundersen all csproj format ("classic" + "sdk-style") can use PackageReference items since VS 2017 15.2. NuGet tries to detect if there is any <PackageReference> item in the project or if there is a packages.config in the project to determine which "restore style" is to be used.

However, if neither of those are present, the <RestoreProjectStyle> can be set to PackageReference to enable all features of projects using this style. This then enables transitive dependencies from referenced projects also using PackageReference. The "sdk-style" (<Project Sdk="Microsoft.NET.Sdk">) projects already set this by default.

[mariusGundersen]

So I need to install the latest nuget client on the build server to be able to use <PackageReference>? And I guess that as long as the project file is still classic style it needs the <Reference> and <HintPath> to work?

[terrajobst]

@0x53A

Thanks for the repo, we'll take a look.

@FransBouma

Make sure you have the latest NuGet client for VS 2015. It looks your client doesn't know that .NET Framework 4.6.1 supports .NET Standard 2.0. If the error persist after upgrading, I'd file a bug in the NuGet org (https://github.com/NuGet/Home/issues/new).

@isaacabraham and @forki

I wonder what impact this has on Paket vis-a-vis automatic binding redirects (Paket has the option to generate BRs on demand based on the dependency graph).

My recommendation is: don't add BR via a package manager. We've worked with the NuGet folks to disable their BR generation as well. It should be generated during build, because that's the only place where all the necessary context is available. If anything, I'd consider making Paket add the AutoGenerateBindingRedirects property to the project file if it isn't set already.

Eventually, I'd like the desktop binder to do the right thing here, but we'll see whether we can actually pull this off...

If there's a BR already in the app.config file and you turn on the automatic BR redirection, who wins?

The automatic binding redirect feature will attempt to merge them. Generally speaking, manual BRs will be honored and win.

@matthid

How about dropping this scenario and make it work "cleanly" with a future version instead of trying to get with the head through the wall?

What would you do differently from what I outlined in the plan above? To me, the plan is precisely what you're asking for.

@dasMulli

Excellent point. Fixed.

@mariusGundersen

When you say

simply add PackageReference to your project file

I'm assuming this means the csproj has to use the VS2017 style csproj

No, you can use non-SDK style projects with PackageReference.

[scottsauber]

@terrajobst when you say "No, you can use non-SDK style projects with PackageReference" does this mean I can just delete my packages.config and then in my .csproj remove the hint paths and replace with PackageReferences and it'll just work (provided I'm on the correct NuGet version)?

Or is there some other magic you need to do to use PackageReference with non-SDK style projects?

[forki]

Actually we often set it to make sure nuget/build does not mess with what we intended (or at least to know when that happens). As double-entry bookkeeping if you will. Now this usecase is basically broken - or in some sense you are basically removing all use cases of the redirects if you make the automatic all the time. Am 04.09.2017 9:48 nachm. schrieb "Scott Sauber" <notifications@github.com>:

…

@terrajobst <https://github.com/terrajobst> when you say "No, you can use non-SDK style projects with PackageReference" does this mean I can just delete my packages.config and remove the hint paths and replace with PackageReference and it'll just work (provided I'm on the correct NuGet version)? Or is there some other magic you need to do to use PackageReference with non-SDK style projects? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#481 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNOBR7y8_ejww5Pg3tcM878YaubHQks5sfFQPgaJpZM4PKtHI> .

[matthid]

@terrajobst

How about dropping this scenario and make it work "cleanly" with a future version instead of trying to get with the head through the wall?

What would you do differently from what I outlined in the plan above? To me, the plan is precisely what you're asking for.

I appreciate the response. Well yes you are dropping packages.config support. But I meant dropping the compatibility of net461 with netstandard20 from the compatibility table and remove all the special handling around this. This would be the honest way of saying: "Well the runtime did not ship with the proper support for netstandard20 but a future version will". What are the reasons which justify the amount of pain you put everyone through and the amount of effort it generates everywhere (not only internally in Microsoft but also externally like in Paket). There must be really important reasons to have this single compatibility connection but I have no clue what they are.

Could this been fixed with net462 or net463? Maybe I'm missing something here.

[terrajobst]

@scottsauber

when you say "No, you can use non-SDK style projects with PackageReference" does this mean I can just delete my packages.config and then in my .csproj remove the hint paths and replace with PackageReferences and it'll just work (provided I'm on the correct NuGet version)?

Two options: remove the packages.config and convert all package references by hand as explained above. You can also control the default for .NET Framework works via Tools | Options | Package Manager | General.

@matthid

But I meant dropping the compatibility of net461 with netstandard20 from the compatibility table and remove all the special handling around this.

Believe me, no one on my team is keen on on the way the net461 support story has unfolded. In a sense, what you're asking has already been implemented: .NET Framework 4.7.1 will ship with full built-in support of .NET Standard 2.0. The challenge is that (1) it hasn't been shipped yet and (2) many of our customers will not be able to upgrade immediately because the target environment is outside their control. The option was to support it the best we can or can it altogether. Yes, this unfortunately requires jumping through hoops but at least there is something app developers can do. If you fully control the environment, by all means, jump to .NET Framework 4.7.1.

[terrajobst]

@forki

Actually we often set it to make sure nuget/build does not mess with what we intended (or at least to know when that happens). As double-entry bookkeeping if you will. Now this usecase is basically broken - or in some sense you are basically removing all use cases of the redirects if you make the automatic all the time.

The problem with double bookkeeping is that automatic binding redirect generation has to assume the project owner knows best and has to honor the BRs that are present. I don't know Paket well enough, but with NuGet we had the issue that NuGet assumed that the same package & version will have consistent assembly versions and not generate redirects based on the referenced assemblies but based on whether package versions differed. If Paket gets it right a 100% then there is arguably nothing against it, but I'd also argue nothing in favor of it either. Outside of .NET Framework you already rely on the binding behavior that higher wins. A generated app.config with the right redirects will do the same, so I don't see a reason why a checked in copy with the project is helping. I just think it causes noise and adds potential for conflicts...

[forki]

Binding redirects where the only way to protect against the issue that nuget can have multiple versions of the same package in the repository and to make sure the right one was selected. Without explicit redirects this is gone and the flaw in the package manager model is still there. @isaac_abraham I think the issue is not really related to paket since we only added them when the user asked for it. Or at least I can't really see the implications yet. But one thing is for sure - all these changes are not feeling sound yet. But that's probably only my personal feeling. Am 05.09.2017 1:33 vorm. schrieb "Immo Landwerth" <notifications@github.com

…

: @forki <https://github.com/forki> Actually we often set it to make sure nuget/build does not mess with what we intended (or at least to know when that happens). As double-entry bookkeeping if you will. Now this usecase is basically broken - or in some sense you are basically removing all use cases of the redirects if you make the automatic all the time. The problem with double bookkeeping is that Automatic Binding Redirect generation has to assume the project owner knows best and has to honor the BRs that are present. I don't know Paket well enough, but with NuGet we had the issue that NuGet assumed that the same package & version will have consistent assembly versions and not generate redirects based on the referenced assemblies but based on whether package versions differed. If Paket gets is right a 100% then there is arguably no reason nothing against it, but I'd also argue nothing in favor of it either. Outside of .NET Framework, you already rely on the binding behavior that higher wins. A generated app.config with the right redirects will do the same, so I don't see a reason why a checked in copy with the project is helping. I just think it causes noise and potential for conflicts... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#481 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNIDuw8N8Nn9S0egMUPdu9bfDds8Vks5sfIjfgaJpZM4PKtHI> .

[isaacabraham]

If what @terrajobst is saying is that simply adding that single element to a vbproj, fsproj or csproj magically fixes all binding redirects, all the time, then I'm not averse to using that instead and deprecating Paket support for BRs. But only if this works consistently and can be applied to e.g. net452 projects running on old MSBuild project format.

[forki]

magically fixes all binding redirects

think about what that means. It means there are effectively no binding redirects anymore.
So in some sense the feature is gone.

[hpavon]

Could you please make this part idiot proof?

2. Make sure your root application project doesn't use packages.config but uses PackageReference for NuGet packages
   o If you currently don't have packages.config, simply add <RestoreProjectStyle>PackageReference</RestoreProjectStyle> to your project file
   o If you currently do have a packages.config, convert the contents to packages references in the project file. The syntax is like this:
   <PackageReference Include="package-id" Version="package-version" />

I am new to .Net and I already screwed my project to the point of having to start all over –all my bad obviously– trying to fix this issue. So I don't want to have to do it again just because I'm figuring out what to touch in which file. Neither my boss wants.

What I'd like to know is:

• Do I have to convert the contents in my csproj files?
• This PackageReference tag, where do I have to place it? What does it substitute?
• Where do I get package-id and package-version from? Is it one entry for each one in my packages.config file. In that case the reference to System.Net.Http that is bothering me is not even there.

Thanks in advance,

Héctor

[joperezr]

This might help you get some context: https://docs.microsoft.com/en-us/nuget/consume-packages/migrate-packages-config-to-package-reference

Basically what you want is this, if in Visual Studio you see something similar to the following:

Then you have NuGet package dependencies, and you are using the legacy packages.config technology. The link I provided starting this comment will tell you how to migrate (which is basically rightclicking this file and telling VS to do the migration for you)

If that is not the case, and you edit your .csproj file and see something like this:

Then you also have NuGet package dependencies, but you are already using the new technology (meaning PackageReference) so there is nothing left for you to do 👍

Finally, if you didn't see a packages.config, and you also couldn't find the tag PackageReference in your .csproj, then that means that you don't have NuGet dependencies, but you may have project dependencies which might have package dependencies, so you should add a property to your project to ensure that this happens. In order to do that, go to your .csproj again, and add this code immediatly after the first import on your project:

After doing this, make sure that from the developer command prompt you call at least once the following command msbuild yourProject.csproj /t:Restore in order to make sure that we calculate all package dependencies for you.

Let me know if you hit any problems after following that.

[hpavon]

Thanks @joperezr I really appreciate

[compil3]

I can't get an older project that worked fine to work again, I still get the exception regardless of anything I've tried. I may just convert this from a .net Console app to .Net Core console and be done with it.