PackageReference support in VS 2017

[clairernovotny]

Projects in VS 2017 no longer use ps1 scripts for install/uninstall. They are not called.

Instead, a package can provide a \build\<tfm>\packagename.targets file that will automatically get imported. In this case, it'd be \build\net20\DllExport.targets.

[3F]

today I checked PackageReference and...

a package can provide a \build<tfm>\packagename.targets file that will automatically get imported

nope, no any .targets if we use PackageReference, just references:

  <ItemGroup>
    <PackageReference Include="DllExport">
      <Version>0.0.1</Version>
    </PackageReference>
  </ItemGroup>

However if we'll use Packages.config it will:

  <Import Project="..\packages\DllExport.1.5.2\build\net20\DllExport.targets" Condition="Exists('..\packages\DllExport.1.5.2\build\net20\DllExport.targets')" />
  <Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
    <PropertyGroup>
      <ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them.  For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
    </PropertyGroup>
    <Error Condition="!Exists('..\packages\DllExport.1.5.2\build\net20\DllExport.targets')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\DllExport.1.5.2\build\net20\DllExport.targets'))" />
  </Target>

I already used \build feature in my other packages like old regXwild

\build\native\regXwild.targets
\build\net40\regXwild.targets

that package can be installed for both type of projects at the same time for .NET CLR and native (unmanaged) like C++ etc.

but for PackageReference I don't know exactly about support - is it really supported ? If you talk about it above ...

where to look documentation for \build\... .targets + PackageReference feature ?

[3F]

hmm, PackageReference uses global cache for each received package, thus we have problem for current impl. of the ddNS feature #2 :(

but in general, it also requires full review for the new distribution and configuration because of removed install/uninstall scripts. Primarily we have a tool for service of user projects, and the \build targets are helpful partially only.

It looks like decision with solution-level >( Guys from NuGet team seems don't understand at all that we can have tools instead of libraries to service of any projects, libraries, the build processes etc. :(

well, need to review all this

[clairernovotny]

@3F I'm happy to help get this working if you want. Not sure what the underlying issues you're having with NuGet are that's creating the need for #38?

[3F]

@onovotny,

I'm happy to help get this working if you want.

:) Good ! More than one way for distribution would be useful anyway.

Actually I already finished the new distribution & manager that described in #38

The short illustration of our new future manager:

• https://www.youtube.com/watch?v=okPThdWDZMM (please note, in this video I use local server)

In total, it contains now lightweight but powerful manager (100% adapted for our project) via new https://github.com/3F/MvsSln & https://github.com/3F/GetNuTool

I hope to release beta on the next week.

Not sure what the underlying issues you're having with NuGet

The main problem here - we are not a library, we are tool. We already discussed about this here NuGet/Home#1521 (comment)

But mainly NuGet focused now for libraries only. The PackageReference it exactly strict way to manage all remote libraries in classical References manner. This is really good to view all remote libraries as an local thing, that I think, but we also have other tools for service projects and libraries like this https://www.nuget.org/packages/vsSBE.CI.MSBuild/ and yes, like our DllExport tool.

The second problem the non-controlled global caching at all. I don't see any features (API etc.) to control this. Through new our manager we can:

<Reference Include="DllExport, Version=1.6.0.26424, Culture=neutral, PublicKeyToken=8337224c9ad9e356">
  <HintPath>..\packages\DllExport.1.6.0-beta\gcache\{0415A0AA-76E4-43A4-BDC3-D115A6781C7D}\DllExport.dll</HintPath>
  <Private>False</Private>
</Reference>

We also need information between each installation/removing or any API to manage our binaries and internal tools like IL Assembler etc. !
(only init.ps1 with caching between projects & \build targets - this is what we have finally today)

Well, I think you may continue dev_1.6 branch for adding alternatives via standard nuget ways (but only as additions to our new manager - anyway, it can be easily combined via commands to Wizard).

dev_1.6 already contains \build\net20\DllExport.targets as simple caller to show information about our new distribution. Only information because this does not work when used PackageReference option: no any .targets if we use PackageReference, just references (#36 (comment)) i.e., as I see it does not give any guarantee.

ah yes, you can also contact with me via skype or email if you need private discussion.

[clairernovotny]

Happy to continue via email -- mine is oren at novotny dot org if you want shoot me a note.

I have written many tools that use NuGet and package ref to do similar things. The trick is usually providing the right targets in the build directory and ensuring that paths are all setup via variables relative to some known location in the project.

Here's an example of an older tool (no longer needed): https://github.com/onovotny/ReferenceGenerator. It runs purely as a build event to manipulate things during the build. There are no library references to it.

For things where you need persistent local binaries, the easiest thing to do is to use the object directory variables like we do in Refit: https://github.com/paulcbetts/refit/blob/master/Refit/targets/refit.targets

In that case, we write to $(IntermediateOutputPath)\SomeFile.cs and add it to the compile. You can easily add references that way too, so you could easily have a targets that generates a temp file and adds it as a reference.

I'll admit that I don't fully follow the flow of your toolchain, but I love that it works! See this thread for what I've been doing with it: https://twitter.com/onovotny/status/893960433119481856 :) The repo with the tooling using it is here https://github.com/onovotny/KeyVaultSignToolWrapper

[3F]

hmm, I think we are talking about different things :)

I also have a few nuget packages that exactly uses \build features: you can also look 7z.Libs, and I remind you about old regXwild (contains targets for native unmanaged C++ projects and managed .NET via Conari). All this works without .ps1 scripts, only targets to configure Link.AdditionalDependencies, ClCompile.PreprocessorDefinitions etc.

The trick is usually providing the right targets in the build directory and ensuring that paths are all setup via variables

I don't see something like this at all when I use PackageReference option:

<Import Project="..\packages\DllExport.1.6.0-beta\build\net20\DllExport.targets" Condition="Exists('..\packages\DllExport.1.6.0-beta\build\net20\DllExport.targets')" />
<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
  ...  
</Target>

Absolutely no any new Import or Target section in project files ! Therefore my question above was about is it really supported by this, when used PackageReference (it works with first option only, see above) ? Or NuGet before adding Import and Target sections into project files, also will try to check a correct logic of this loaded targets o_O ? if so, it should be leader among omg-projects >(

• \build\net20\DllExport.targets o_O

... as a build event to manipulate things during the build.

For something like this, I use vssbe that's being developed from 2013 and still perfectly for some crazy things with VS or msbuild :) This project also fully controlled by this. Anyone please note this.

I'll admit that I don't fully follow the flow of your toolchain, but I love that it works!

Like a music for my ears ... that all works fine :p thanks for using :)

[clairernovotny]

I don't see something like this at all when I use PackageReference option:

That's because NuGet includes them automatically. NuGet does not touch the project file anymore. That doesn't mean they're not included :)

If you look at the obj directory, you'll note that there are a few generated props/targets on restore...and they'll include your \build\net20\DLLExport.targets automatically. The goal has been to simplify the project file by making NuGet and the tooling do more behind the scenes.

[3F]

@onovotny I see, that's what I was looking for :) thanks

<ProjectFileName>.csproj.nuget.g.targets:

<ImportGroup Condition=" '$(ExcludeRestorePackageImports)' != 'true' ">
  <Import Project="$(NuGetPackageRoot)dllexport\1.6.0-beta\build\net20\DllExport.targets" Condition="Exists('$(NuGetPackageRoot)dllexport\1.6.0-beta\build\net20\DllExport.targets')" />
</ImportGroup>

Good ! our Wizard already may control caching gcache\{0415A0AA-76E4-43A4-BDC3-D115A6781C7D}\DllExport.dll thus it really can be combined easily.

... do more behind the scenes.

by the way, the vssbe above was created initially with the same thoughts :) it serves all solution data without any modifications of projects files.

[3F]

@onovotny

Unfortunately it still contains a lot of problems for our project. Let me explain...

In the morning I was thinking about same scheme like external .net.dllexport + \build feature:

<PropertyGroup Condition=" '$(ProjectGuid)' == '{36e28b5f-b075-4de7-85e5-d97d1be73f88}' ">
  <DllExportNamespace>System.Runtime.InteropServices</DllExportNamespace>
  ...
</PropertyGroup>
<PropertyGroup Condition=" '$(ProjectGuid)' == '{0415A0AA-76E4-43A4-BDC3-D115A6781C7D}' ">
  <DllExportNamespace>net.r_eg.DllExport</DllExportNamespace>
  ...
</PropertyGroup>
<ItemGroup>
  <Reference Include="DllExport, Version=1.6.0.26424, Culture=neutral, PublicKeyToken=8337224c9ad9e356">
    <HintPath>$(DllExportInstalledPath)\gcache\$(ProjectGuid)\DllExport.dll</HintPath>
    <Private>False</Private>
  </Reference>
</ItemGroup>
....

Looks good to avoid reloading environment outside EnvDTE logic etc., this I thought until I saw a hard coded logic for this type of references:

failed.

PackageReference section does not affects for this at all:

<PackageReference Include="DllExport">
  <Version>1.6.0-beta</Version>
</PackageReference>

It will work only when project.assets.json will be removed or changed with some incompatible data to destroy link on internal nuget processing of this.

That is, summary:

• We can't use DllExport.dll from lib because it's not prepared for projects.
• We can't avoid distribution of library outside lib folder because we need to provide anything (see related solution-level problem)
• We also can't simply replace (via MvsSln etc.) this new added PackageReference from project file, because it actually does not affect on this. Also $(ExcludeRestorePackageImports) = false etc.
• Seems we also have no any option to control path to libraries in PackageReference section https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files
• Another interesting, PackageReference seems also does not support like this <Private>False</Private> i.e. we need to additional processing for remove this from output folder etc.

I'm still see many problems and inconveniences with this, i.e. I have some ideas but ... I want much more transparent logic for all this.

Please note again, DllExport.dll library it's marker only that's actually does not contains any logic at all. We need this only for Cecil processing in attribute manner.

Well, I think I will improve only preparing of our manager via std. nuget (we can standardize via \build as I see) + probably I'll implement .net.dllexport feature, need to think.

But the transparent scheme is still as the removing package from official nuget clients, then to continue through our manager. Because I'm watching that NuGet project still is inconvenient for something like this project or this https://www.nuget.org/packages/vsSBE.CI.MSBuild/ etc.

Did I miss something ? some API for control behavior of PackageReference section or project.assets.json etc. ?

[clairernovotny]

Couple things... To prevent PackageReference content from going to the output, you can add a PrivateAssets attribute;

<PackageReference Include="SomeNuget" Version="3.0" PrivateAssets="all" />
That will prevent any lib content from making it to the output directory. There is an open issue to have NuGet automatically do this for packages that are marked as a developmentDependency, but it doesn't do it yet :(

Also keep in mind that ProjectGuid doesn't exist anymore....

What I would suggest is to not have anything in the lib directory for this. In the build targets, you can have a condition that either supplies a Reference to a particular dll (the temporary façade), or some generated one depending on what you need. Not sure I understand what the gcache version does or why the DllExportNamespace would change based on a guid?

[3F]

What I would suggest is to not have anything in the lib directory for this.

It's not possible anymore :) i.e. the package will not be installed at all if not exists any library in lib folder.
Try to install: https://www.nuget.org/packages/vsSBE.CI.MSBuild/ - after 2015 we use GetNuTool. Because actually CIM requires only solution-level and does not provide any lib for projects.

Not sure I understand what the gcache version does or why the DllExportNamespace would change based on a guid?

For example, this is required by ddNS feature - #2 (e.g.: #47 )

Also keep in mind that ProjectGuid doesn't exist anymore....

That is, it will not be generated by VS when created new project o_O Can I see some official information etc. about this changes ?

[clairernovotny]

I don't have a link to the docs handy, but look at the "SDK style projects" and all of the clean-up they have been doing. Here's an example of an exe:
https://github.com/onovotny/KeyVaultSignToolWrapper/blob/master/KeyVaultSignToolWrapper/KeyVaultSignToolWrapper.csproj

I can say with certainty that it's possible to have packages that install but do not have a lib folder :) Newer clients have no problem with just a \build item, but to satisfy the older clients, you can use an empty file called _._ in the lib directory. So you could have lib\net20\_._ as an empty file.

The idea here is that all of the logic is handled by your props and targets and you're just telling NuGet to install it and wire-it-up.

Not sure what the real value is of dynamic namespaces? Most tools like this (including Roslyn based things), just pick a namespace and go with it. Only you can answer if it's worth the effort :) If the dll is a marker and gets removed, one option that may be easier could be code generation. Dynamically write a DllExport.cs in the IntermediateOutputPath directory and add that to the Compile You can do this on a BeforeTargets=CoreCompile step. Then you don't need a two-pass compile. The attribute would be included in the Compile and you can put it in whatever namespace you want (based on a property). Would that work?

[3F]

hmm, the interesting solution to forget about identifiers of projects at all <_<

I thought that ProjectGuid just will be encapsulated internally and still will be available as RO data, but seems no -_-

Maybe you know, what about CurrentSolutionConfigurationContents for SDK-based projects ?

Well, it doesn't matter for DllExport project, because we can also use other internal identifier for specific configured project. But for vssbe project it does :) thanks for the information.

So you could have lib\net20_._ as an empty file.

Looks like a some alternatives, need to check, but I'm scared to use something undocumented from NuGet project. And I'm worry about availability for active std.nuget clients, and so on. How to say... I don't want new surprises from their projects. But I also see that an project now is more standardized in recent time.., hmm

Does exist some official/authoritative information about this ?

The attribute would be included in the Compile and you can put it in whatever namespace you want (based on a property). Would that work?

No :) if I understand correctly your thought. DllExport.dll it's frontend as a marker for user space, and this attribute should be available at runtime for usage 'as is'. Today, as I know, roslyn does not provide any mechanism to reimplement this at runtime. I mean, of course we can fully generate IL via CodeDom compiler + reflection, etc. but we can't reimplement this like ValueTuples in C# 7 :) e.g.: https://github.com/3F/sandbox/blob/master/csharp/_core_features/Tuple_csharp7/ValueTuple.cs this is possible because it's already implemented by compiler to final implementation later. ValueTuple<...> etc. as you can see from my sandbox. Therefore, preprocessing with modifications of the final binaries (that's already referenced to project) is the most easy way. Well, it's my thoughts only, so anyone may also improve this way if it's needed.

upd.

project.assets.json:

  "DllExport/<version> {
    "type": "package",
    "compile": {
      "lib/net20/_._": {}
    },
    "runtime": {
      "lib/net20/_._": {}
    },
    "build": {
      "build/net20/DllExport.targets": {}
    }
  }

ugly hack, but yes, seems it works.

need to think -_- any reliable and transparent way, because...

[3F]

@onovotny Actually, I left integration with PackageReference because of lot of inconveniences and ambiguities like with 'References' inside VS IDE from my post above.

It means that I have no plans for this today until I see some API or some other transparent way for support our important features.

However, you or anyone else may try to integrate this with our new Wizard. I'll review your suggestions later.

Yesterday has been released v1.6 beta2:

• Latest manager: https://3F.github.io/DllExport/releases/latest/manager/

[clairernovotny]

I'd be happy to try, but I'm not sure I entirely understand the gap between what PackageReference does and what you need?

[3F]

Where L1 points to final communications with our manager.

we also need to avoid this:

[clairernovotny]

There really should not be another package manager that pulls in files....I don't understand why this can't be self-contained like it is now? Sent from a mobile device, please excuse brevity and typos

…

________________________________ From: Denis Kuzmin <notifications@github.com> Sent: Sunday, August 20, 2017 12:21:38 PM To: 3F/DllExport Cc: Oren Novotny; Mention Subject: Re: [3F/DllExport] PackageReference support in VS 2017 (#36) [dllexport_manager_and_nuget]<https://user-images.githubusercontent.com/3246589/29496502-84e62e0c-85dc-11e7-9ecf-2c95adc61b35.png> Where L1 points to final communications with our manager. we also need to avoid this: [dllexport_dblref]<https://user-images.githubusercontent.com/3246589/29496464-ec03ff0c-85db-11e7-9e58-62b84f98185f.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#36 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ABXHVFTA0j6GfW2bfZffzQFCXw5XVB21ks5saF0SgaJpZM4NSY58>.

[3F]

@onovotny how you will control, for example, DllExport.dll metalibrary via PackageReference ? and also how you will execute our Wizard/Configurator ?

For better support of our tool, now we provide embeddable manager, but it still doesn't matter, because we need to support our Configurator, and some other features, anyway.

I mean, even without manager, you should communicate with our Wizard/Configurator.

[clairernovotny]

All the Wizard should do is add some properties to the MSBuild file, no? To invoke the wizard, you could easily have a target so a user would do msbuild /t:InvokeDllExportConfig and it'd pop-up a UI. Most of the time, it's not explicitly needed. There shouldn't be any "communication" required at install time, just setting some properties for the build to work.

PackageReference would control the dllexport.dll library by adding targets. It can be either DesignTime, or BuildTime, or both.

[3F]

To invoke the wizard, you could easily have a target so a user would do msbuild /t:InvokeDllExportConfig and it'd pop-up a UI.
There shouldn't be any "communication" required at install time, just setting some properties for the build to work.

:) If I correctly understand your thought, yes, it's not so important when it should be configured at install time or when build is started:

Just try to install 1.6.0-beta2 package via PackageReference option and you will see same behaviour. We already execute some UI only when build is started.

Good ! we have the opportunity to configure all what we need :)
no, what about DllExport.dll ?

we already discussed about this above, btw, I updated structure \gcache\metalib\<namespace>\ and added stub _._ as you noticed about this "feature" (that was the one of the reason in the chain of the bugs for beta-1 release :p - #38 (comment)).

so, how we'll manage 2 or more different DllExport.dll ? I mean, does nuget provide some options to control path to library (see project.assets.json), how we'll update/replace this for PackageReference installation ?

Also, how about removing ? we have some configured data for specific projects, therefore we also need to remove this after removing our package.

In general, I have some ideas, but...

[3F]

1.7 includes the following changes:

* CHANGED: New installation behavior through official NuGet.
           init.ps1 also adds optional copying of package data for our manager.

           Please note: This is not nuget support!
                        It just replaces known info-form that was in 1.6.x.

This, by the way, partially follows the first draft logic above #36 (comment)

And I still think this is a good point for project like this.
Especially after .NET Core support with optional rebasing, and other patching.

But anyone can open PR for actual review together!

@onovotny,

Let me know if anything else.

Thanks for the suggestion,