Framework assemblies are no longer added to .nuspec when using the new .csproj project system

[kevinchalet]

Moved from #4412 (comment):

---

In the marvelous world of project.json, specifying a frameworkAssemblies node was enough to tell NuGet to add the corresponding entries in the .nuspec file:

{
  "frameworks": {
    "net451": {
      "frameworkAssemblies": {
        "System.IdentityModel": "4.0.0.0",
        "System.Xml": "4.0.0.0"
      }
    }
  }
}

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    // ...
    <frameworkAssemblies>
      <frameworkAssembly assemblyName="System.IdentityModel" targetFramework=".NETFramework4.5.1" />
      <frameworkAssembly assemblyName="System.Xml" targetFramework=".NETFramework4.5.1" />
    </frameworkAssemblies>
  </metadata>
</package>

With .csproj, this is no longer true when using Reference:

<Project Sdk="Microsoft.NET.Sdk">

  <ItemGroup>
    <Reference Include="System.IdentityModel" />
    <Reference Include="System.Xml" />
  </ItemGroup>

</Project>

This breaking change has obviously important implications, as framework assemblies are no longer brought transitively, which means that the final user of the package has to reference the framework assemblies in his own app.

[rohit21agrawal]

@dsplaisted Can <Reference> in Net Core projects point to a random DLL on disk which is not a framework assembly?

[dsplaisted]

It will be able to in the 2.0 tools.

[rohit21agrawal]

@dsplaisted will there be a way to distinguish what References come from the framework and what come from a DLL on disk? Is there a design doc for this so we can adapt pack accordingly?

[dsplaisted]

Here's the PR where this was added to the SDK: dotnet/sdk#876

It turns out we mentioned that nuget pack would need to handle this but never filed an issue.

The logic to figure out where the references actually come from is very complicated and involves the ResolveAssemblyReferences task. I doubt there is a current spec for it.

It might be OK to just see whether the ItemSpec has a .dll or .exe extension, and if so assume it's a file reference, otherwise assume it's a framework reference. Also, .NET Core and .NET Standard projects wouldn't support framework references anyway.

@rainersigwald, @nguerrera, or @eerhardt may be able to provide more feedback.

[NickCraver]

This is a pretty big blocker to find out about at the end of a port. Is there a workaround? I'm porting major libraries here (Dapper, MiniProfiler, StackExchange.Redis, etc.) and this is a big step backwards for the consumer, having to reference framework libraries manually that just worked on the old system.

From a user point of view, they're better off with me not moving to the VS 2017 project system - what can we do so this isn't the case?

[dsplaisted]

One work around would be to continue to use a nuspec, I think.

[davidfowl]

@dsplaisted that's a pretty big hammer of a workaround.

[rohit21agrawal]

A better workaround in my opinion would be to use the GenerateNuspec task to get a nuspec without the framework assemblies in your obj folder, then write a custom task that takes in this nuspec and adds the framework assemblies (can use any xml reader writer for this), and then pack the resulting nuspec.

I know its not the most convenient thing but its not terribly hard either. I can come up with a small example of how to do this if you think this might help. Let me know.

[NickCraver]

Are we expecting every library author that writes a package that supports .NET full framework (the vast majority of packages) to do this in under to use the new tooling for the next 3 minor releases? Am I the only one that sees this as a large blocker that really needs to be addressed in NuGet 4.1 (or hopefully, sooner)?

Let me be as blunt as possible: We aren't porting our projects to the new .csproj system, and this issue is why.

[rohit21agrawal]

Cc: @rrelyea for your urgent triaging attention

[NickCraver]

Is there any hope of getting this fixed soon? We're ready to merge in .csproj ports on several major NuGet packages with millions of downloads and we're blocked by only this issue. This is incredibly frustrating.

Remember to port to .csproj we had to abandon project.json all together and move our projects over. From the moment we do that, the move itself starts to decay if unmerged. We have to pause all project-related PRs and changes, or make them on both branches and merge all along the way (which users and PRs won't do). To have to hang on merging .csproj formats or indefinitely not release on NuGet are terrible options and they're costing time and frustration. I'm personally up to 6 blocked projects now, and other maintainers at Stack are hitting the same.

Please up the priority on this, it needs a fix very soon.

[rohit21agrawal]

@NickCraver to get you a workaround as fast as we can, can you help me understand how are you packing projects? msbuild or dotnet or VS?

[NickCraver]

@rohit21agrawal I have steps and an easy repro for you, an example library is MiniProfiler (current commit link). You only need to run this:

.\build.ps1 -VersionSuffix "alpha1234"

You can see everything in the simple build script, but here's the relevant bits:

dotnet msbuild "/t:Restore;Pack" "/p:Configuration=Release" "/p:VersionSuffix=$VersionSuffix" "/p:PackageOutputPath=$packageOutputFolder" "/p:CI=true"

Note that I have to currently use /t:Restore;Pack instead of dotnet restore and dotnet pack because #4337 is another bug with pre-release package references in play.

cc @onovotny who also noticed this break

[m0sa]

The packaging in this case is done by the PackTask from the Nuget.Client repo:

The build already walks all the framework targets to get it's output files per target framework. The same way it could get the required <Reference> items per target framework. But currently, even if it did that, there's no way of passing those to the PackTask, as you can see in the latest code for it on the ˙dev˙ branch , there's no way of passing frameworkAssembly references to it.

[m0sa]

I've dug around a bit more, and there is an AssemblyReference task property that gets populated from the _Reference item group, but it doesn't seem to do anything, e.g.:

I added this task to the .csproj

  <Target Name="_ReferencesTest" BeforeTargets="GenerateNuspec">
    <ItemGroup>
      <_References Include="System.Data" />
      <_References Include="System.Xml" />
    </ItemGroup>
  </Target>

Those were passed in to the PackTask:

But no changes to the nuspec. The AssemblyReferences property doesn't seem to be used anywhere yet.

[mgravell]

Context: I have changes I want to deploy in multiple fairly high usage libs: dapper, protobuf-net, stackexchange.redis, etc (including multiple MS PRs that I'm sure they'd love to see deployed). Being "bleeding edge, feel the pain", I've updated from csproj to project.json back to csproj. And the thing that makes me not want to push to nuget: this glitch, because it will actively hurt my users.

[NickCraver]

@rohit21agrawal @rrelyea Is there any update here? This is a HUGE pain for us. We'll soon have to throw away the ports of projects we've done or face severe merge headaches, and we don't know which route is worse.

We have an entire pipeline of new tooling in play that works from start to finish (even if it's a workaround in place) up until the final packaging. We can't release the libraries generated by the tooling to our users; that'd be actively harmful to the community. Why isn't this a higher priority? There's no way this can linger for months, unless we simply don't care about any library authors moving to VS 2017 until the Update 3 timeframe.

This is the worst kind of bug, one developers very likely don't even know they're causing. I highly doubt we're the only ones affected - there's just no way, and I've already heard from other authors. I think we're just some of the few who noticed.

Please, please treat this as higher priority than months away. It is so insanely frustrating to be blocked by only this issue on so many large projects. We'd take any fix here, including any incredibly hacky workaround.

[rohit21agrawal]

@NickCraver I am working on this issue, I will update this thread very soon.

[kevinchalet]

We'd take any fix here, including any incredibly hacky workaround.

Personally, I opted for the "custom .nuspec approach". It's not ideal, but it works:

• https://github.com/aspnet-contrib/AspNet.Security.OpenIdConnect.Server/blob/dev/src/Owin.Security.OpenIdConnect.Server/Owin.Security.OpenIdConnect.Server.csproj#L32-L61
• https://github.com/aspnet-contrib/AspNet.Security.OpenIdConnect.Server/blob/dev/src/Owin.Security.OpenIdConnect.Server/Owin.Security.OpenIdConnect.Server.nuspec#L24-L27

[rohit21agrawal]

as of now it ships as part of both VS and .NET Core.

[bording]

@rohit21agrawal I've tried packing with Visual Studio, and also using msbuild.exe /t:pack from the command line. I can't currently use dotnet pack on this project because it uses GitVersion, which doesn't yet have build targets compatible with .NET Core.

The file version of "C:\Program Files (x86)\Microsoft Visual Studio\Preview\Professional\MSBuild\Sdks\NuGet.Build.Tasks.Pack\Desktop\NuGet.Build.Tasks.Pack.dll" is 4.3.0.4087.

[rohit21agrawal]

@bording that version definitely has the fix. can you provide me a sample of the project that repros this for you?

[bording]

@rohit21agrawal I've been seeing this while working on https://github.com/Particular/NServiceBus, and the develop branch now has netcoreapp2.0 targets.

As an example, this project file has framework assembly references that are not included by default in the new project system, but the package produced does not reference them.

[rohit21agrawal]

I just tried packing the project file you provided with the version of msbuild that shipped with 15.3 Preview 3 and i got the following assembly references in my output nuspec:

I did have to remove netcoreapp2.0 as target framework because restore wasn't able to resolve some package required for that particular TFM.

<frameworkAssemblies>
      <frameworkAssembly assemblyName="Microsoft.CSharp" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Configuration" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Core" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Data" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Drawing" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.IO.Compression.FileSystem" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Numerics" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Runtime.Serialization" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Transactions" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Xml" targetFramework=".NETFramework4.5.2" />
      <frameworkAssembly assemblyName="System.Xml.Linq" targetFramework=".NETFramework4.5.2" />
    </frameworkAssemblies>

[bording]

@rohit21agrawal I'm definitely not seeing that when I build it. What package could you not resolve for netcoreapp2.0? They should all be publicly available.

[bording]

When I pack that project here is the nuspec that gets generated in the obj folder:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    <id>NServiceBus.AcceptanceTesting</id>
    <version>7.0.0-alpha0030</version>
    <authors>NServiceBus Ltd</authors>
    <owners>NServiceBus Ltd</owners>
    <requireLicenseAcceptance>true</requireLicenseAcceptance>
    <licenseUrl>http://particular.net/LicenseAgreement</licenseUrl>
    <projectUrl>http://particular.net/</projectUrl>
    <iconUrl>http://s3.amazonaws.com/nuget.images/NServiceBus_32.png</iconUrl>
    <description>Acceptance testing framework for NServiceBus endpoints. This is an unsupported package.</description>
    <copyright>Copyright 2010-2017 NServiceBus. All rights reserved</copyright>
    <tags>nservicebus servicebus msmq cqrs publish subscribe</tags>
    <dependencies>
      <group targetFramework=".NETFramework4.5.2">
        <dependency id="NServiceBus" version="1.0.0" exclude="Build,Analyzers" />
      </group>
      <group targetFramework=".NETCoreApp2.0">
        <dependency id="NServiceBus" version="1.0.0" exclude="Build,Analyzers" />
      </group>
    </dependencies>
  </metadata>
  <files>
    <file src="C:\Code\NServiceBus\binaries\net452\NServiceBus.AcceptanceTesting.dll" target="lib\net452\NServiceBus.AcceptanceTesting.dll" />
    <file src="C:\Code\NServiceBus\binaries\netcoreapp2.0\NServiceBus.AcceptanceTesting.dll" target="lib\netcoreapp2.0\NServiceBus.AcceptanceTesting.dll" />
    <file src="C:\Code\NServiceBus\binaries\net452\NServiceBus.AcceptanceTesting.pdb" target="lib\net452\NServiceBus.AcceptanceTesting.pdb" />
    <file src="C:\Code\NServiceBus\binaries\netcoreapp2.0\NServiceBus.AcceptanceTesting.pdb" target="lib\netcoreapp2.0\NServiceBus.AcceptanceTesting.pdb" />
  </files>
</package>

[clairernovotny]

@bording silly question, but how are you invoking your build when you're trying this? Are you sure you're using the msbuild from the VS Preview directory and not the main VS 2017 stable directory? (just verifying) :)

[bording]

@onovotny I'm running it from a VM that has only ever had the preview VS installed, so there isn't another MSBuild to conflict.

To double check, here's the output from which msbuild:

C:\Program Files (x86)\Microsoft Visual Studio\Preview\Professional\MSBuild\15.0\Bin\MSBuild.exe

[bording]

I just double checked to make sure I'm running the latest Preview release, and the Installer thinks I'm updated. Here's what I'm seeing in the About box:

I've also have 15.3 Preview 3 installed on our CI build agent, and it is not producing packages with framework assembly references either.

[bording]

@rohit21agrawal I also just tried removing the netcoreapp2.0 target from the AcceptanceTesting project to see if that was somehow causing the problem, but it just produces the following:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    <id>NServiceBus.AcceptanceTesting</id>
    <version>7.0.0-fix-source-packa0001</version>
    <authors>NServiceBus Ltd</authors>
    <owners>NServiceBus Ltd</owners>
    <requireLicenseAcceptance>true</requireLicenseAcceptance>
    <licenseUrl>http://particular.net/LicenseAgreement</licenseUrl>
    <projectUrl>http://particular.net/</projectUrl>
    <iconUrl>http://s3.amazonaws.com/nuget.images/NServiceBus_32.png</iconUrl>
    <description>Acceptance testing framework for NServiceBus endpoints. This is an unsupported package.</description>
    <copyright>Copyright 2010-2017 NServiceBus. All rights reserved</copyright>
    <tags>nservicebus servicebus msmq cqrs publish subscribe</tags>
    <dependencies>
      <group targetFramework=".NETFramework4.5.2">
        <dependency id="NServiceBus" version="7.0.0-fix-source-packa0001" exclude="Build,Analyzers" />
      </group>
    </dependencies>
  </metadata>
  <files>
    <file src="C:\Code\NServiceBus\binaries\net452\NServiceBus.AcceptanceTesting.dll" target="lib\net452\NServiceBus.AcceptanceTesting.dll" />
    <file src="C:\Code\NServiceBus\binaries\net452\NServiceBus.AcceptanceTesting.pdb" target="lib\NServiceBus.AcceptanceTesting.pdb" />
  </files>
</package>

[rohit21agrawal]

can you do a diag build+pack and attach the log file?

[bording]

@rohit21agrawal What would be the specific command you'd like me to invoke? Just enable diagnostics level logging?

[rohit21agrawal]

Msbuild /t:pack /v:diag

[bording]

@rohit21agrawal Here you go:
log.zip

[rohit21agrawal]

Using "PackTask" task from assembly "C:\Program Files\dotnet\sdk\2.0.0-preview1-005977\Sdks\NuGet.Build.Tasks.Pack\buildCrossTargeting\..\Desktop\NuGet.Build.Tasks.Pack.dll".

This is what i see in the log file. So the msbuild you are using is resolving the dotnet core sdk to 2.0.0-preview1-005977 which does not have the fix.

[bording]

@rohit21agrawal Is it supposed to do that? Is there a way for me to prevent that from happening?

[rohit21agrawal]

@bording I actually don't know that for sure. Tagging @nguerrera since he might be the best person to answer this.

[rohit21agrawal]

but for now, you can always install the latest CLI from https://github.com/dotnet/cli/tree/release/2.0.0-preview2 and that should fix your problem.

[bording]

I had been wanting to wait for the official preview 2 bits before installing them, but if that's the only way to work around the issue, I guess I should give them a try now.

[clairernovotny]

The way to stop it is to use a global.json and pin the SDK version to 1.0.4. That's the one VS 2017 ships "in box".

[bording]

@onovotny Will that still work if I'm actually targeting netcoreapp2.0, though?

[clairernovotny]

@bording nope. If you need netcoreapp2.0, then you'll need the latest SDK from the preview2 branch then. FWIW, I have that working on AppVeyor w/o issues:

https://github.com/paulcbetts/refit/blob/master/appveyor.yml#L7-L9

[bording]

Sounds like I'll be grabbing some preview2 bits then!

It does seem odd to me that MSBuild would ship with a version of the PackTask assembly that won't actually be used though, even when targeting just net452.

[dasMulli]

Another workaround would be to use the NuGet.Build.Tasks.Pack nuget package that was published to nuget.org yesterday:

<Project>
  <PropertyGroup>
    <NuGetBuildTasksPackTargets>junk-value-to-avoid-conflicts</NuGetBuildTasksPackTargets>
  </PropertyGroup>
  <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />

  <!-- All your project's other content here -->

  <ItemGroup>
    <PackageReference Include="NuGet.Build.Tasks.Pack" Version="4.3.0-preview3-4168" PrivateAssets="All" />
  </ItemGroup>
  <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
</Project>

VS' MSBuild now carries an SDK resolver that uses the installed CLI. AFAIK mono's msbuild will do the same.