VS 2017 doesn't execute install.ps1 from packages if used as PackageReference

[monkey-dsc]

Hi,

maybe a duplicate Topic, I don't know... But none of the topics here solved my Issue.

I need to install a Nuget-Package, Glass.Mapper.Sc -Version 4.4.0.199, the Package has a Dependency to Glass.Mapper.Sc.Core -Version 4.4.0.199 which evaluates with install.ps1 which Sitecore Version is present in the project for backward compatibility. If a Sitecore.Kernel package is present, e.g. Sitecore.Kernel.NoReferences -Version 8.2.170407, it will install Glass.Mapper.Sc.dll, if also Sitecore MVC Version is present in the project, eg. Sitecore.Mvc.NoReferences -Version 8.2.170407, it will install Glass.Mapper.Sc.Mvc.dll (also described in the readme.txt in the package...).

Sitecore does have its own NuGet feed so you have to add https://sitecore.myget.org/F/sc-packages/api/v3/index.json to your nuget package sources.

NuGet Product used: Package Manager in VS 2017

NuGet Version is 3.5+

Visual Studio 2017

OS Version: Win10

Worked before in VS 2015

Repro:

1. Create a new class library in VS 2017 (.Net Framework)
2. Format project to the new csproj format as described here (Startover: Class Library)
3. Install Sitecore.Kernel.NoReferences -Version 8.2.170407 NuGet package
4. Install Sitecore.Mvc.NoReferences -Version 8.2.170407 NuGet package
5. Install Glass.Mapper.Sc-Version 4.4.0.199 NuGet package

If everything works as expected, Glass.Mapper.Sc.dll and Glass.Mapper.Sc.Mvc.dll should be added to the Project. But it doesn't.

The evaluation of the Sitecore version works fine in 2015, since 2017 it doesn't work anymore :(
We are very dependend on this package, because we want to build lots of nuget packages which will have a dependency on this...

Am I doing something wrong? I could also provide a Solution with a single Interface where its not working to install the package properly.

Regards
Dirk

[mishra14]

@monkey-dsc The help page for Glass.Mapper.Sc is http://www.glass.lu/Mapper/Sc/HowTo/1-Install

It seems that you also need to add a reference to System.Web.Mvs.dll. Can you please try adding reference to that before installing Glass.Mapper.Sc?

If you are still facing the same issue, please let us know.

[monkey-dsc]

Hi @mishra14 ,

I have setup a little sample project where you can see this behaviour. A reference to System.Web.Mvc is also present and you can just start adding Glass.Mapper.Sc to the project.

Regards
Dirk
Company.Foundation.PresentationSettings.Core.zip

[jainaashish]

Format project to the new csproj format as described here (Startover: Class Library)

install.ps1 doesn't execute with PackageReference format. Blogpost about PackageReference https://blog.nuget.org/20170316/NuGet-now-fully-integrated-into-MSBuild.html also talks about this.

There is already a tracking bug #5963 for this issue. And our preliminary thought is to have msbuild targets instead of install.ps1 scripts to achieve the same behavior. So once we've all the guidelines then we'll ask package authors to update their packages to remove install or uninstall ps1 scripts and add targets instead.

[monkey-dsc]

Wow hard stuff... @jainaashish that's a really really everything breaking change... 👎

[jainaashish]

It has always been like this with transitive world, be it project.json or PackageReference. It was never a good idea to have these install.ps1 and uninstall.ps1 scripts at the first place which are even more troublesome to maintain since users do all sort of weird stuff through these scripts and they provide a loophole in the system. Besides, since most of these scripts have hard dependency on dte project, so they only works inside visual studio, and couldn't be supported via command line.

So we don't want to carry the same design issues of packages.config to PackageReference and which is what we're trying to fix here. And MSBuild seems to the right approach for all these kind of stuff. We understand it's not going to be an easy process but we're gradually working on it and making progress. We also plan to work with package authors to make this transition as smooth as possible.

[RussKie]

So once we've all the guidelines then we'll ask package authors to update their packages to remove install or uninstall ps1 scripts and add targets instead.

It has been 8 month since you were going to create guidelines, still nothing...

We are migrating a large SDK-style solution which is heavily dependent on install.ps1. Because netstandards and .netcore projects use exclusively PackageReference we are essentially blocked...

[augustoproiete]

Because netstandards and .netcore projects use exclusively PackageReference we are essentially blocked...

Same here. Really looking forward to having a supported alternative to install.ps1 guys!

[jainaashish]

@RussKie @caioproiete can you guys please share specific details about the requirement of install.ps1 with your packages? If you guys can't share details in public then drop me an email at asja @ Microsoft .com with all the details or possibly a sample solution to depict what were you trying to achieve with these scripts.

[augustoproiete]

Hey @jainaashish here are some examples:

Configuration templates

• I need to add files to the user's project when installing a nuget package. These files are "template files" containing configuration settings (i.e. a starter template), which the user is expected to make changes to it / adapt. PackageReference dropped support for content and introduced contentFiles that are immutable which I cannot use because I need mutable files.

• I need to dynamically rename a file being added to the project. The nuget package has "template.ext" and as the package is installed, I need to rename it to something like `$(ProjectName)-v1.ext". You can see an example here.

• I need to perform XDT transformations on configuration files. This was supported in the packages.config model, but it's not supported with PackageReference

Code templates

• I need to add CSharp files to the user's project with code transformation (.pp) when installing a nuget package. Again, the user is expected to change the file, so contentFiles is not an option. See LibLog as an example.

Assembly settings

• Some assemblies that are added as reference to the project need to be marked as CopyLocal=false. I can do with PowerShell on the packages.config model. You can see an example here.

Cleanup on uninstall

• Need to do some clean-up when the user uninstalls a nuget package. That can include removing and/or renaming files from the project that we're uninstalling the package from. You can see an example here.

[augustoproiete]

@jainaashish One more scenario I've just remembered:

Changing project settings via DTS upon installation of NuGet package

• I'm providing to users a framework that allows them to develop Add-Ins for Excel. That means that when they debug the add-in, Visual Studio must run EXCEL.exe and attach to it, so that I can debug.

This means that I need to update properties in the .csproj.user file such as StartAction, StartProgram, and StartArguments which are settings for that particular user (given that different people use different versions of Excel, etc.).

e.g.

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/ ...">
  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|AnyCPU'">
    <StartAction>Program</StartAction>
    <StartProgram>C:\Program Files\Microsoft Office\Office15\EXCEL.EXE</StartProgram>
    <StartArguments>"MyExcelAddIn-AddIn.xll"</StartArguments>
  </PropertyGroup>
</Project>

I do that via PowerShell through install.ps1

This is not something that can be added to a .targets files, first because it's a per-user setting, and not global to the team sharing the solution, and second because Visual Studio requires that these properties must be inside the .csproj or csproj.user... Defining them in a .targets file that gets imported has no effect.

[RussKie]

I am part of a team developing a Enterprise RAD framework (in many ways it is similar to .NET SDK).
Our SDK consists of numerous packages, and whenever a developer installs or updates a package in their project, the package must make necessary changes to project's files, such as:

• apply XDT or XSLT transformations to a web.config files (including Debug and Release), and/or
• create or modify configurations in web.config files (including Debug and Release), and/or
• create a project's manifest file (a vital xml file in our case) or apply XSLT to it, and/or
• apply XSLT to project's manifest file, and/or
• run code transformations (.pp), and/or
• uninstall deprecated Nuget packages, etc

All of these are install-time operations. I've looked at .targets, but they seem to be run-time operations, and thus can't meet our requirements.

[david-m-forster]

To add to @RussKie 's comments (I work with him),

We originally used the nuget built-in xdt capability to transform web.*.config as well as custom xml files when a package was added. A given consumer may install one or many nuget packages depending on their business requirements. i.e different nuget packages enabled different related capabilities. So each package may add/remove XML sections or update existing sections.

The problem with xdt is that it would end up completely corrupting the targeted resource (like web.config say) as each nuget package would end up interfering with the changes made by other packages. This lead to 100s of our engineers being confused and led to a support nightmare attempting to help them get their config sorted out.

We then innovated by using the install.ps1 script to kick off an XSLT transformation. XSLT, although somewhat out-of-vogue these days is incredibly powerful so we could selectively transform, add or remove items without interfering with other nuget installs (examples like deprecating old app settings or even renaming them). Further, the XSLT files could be unit tested to ensure they were robust under many different installation scenarios. we even have integration tests that apply multiple package transformations in one go.

So now we want to move everything over to .netcore but if we do we will be seriously regressing when it comes to transformations of config files.

[BrunoJuchli]

@jainaashish
Another example is https://github.com/Fody/Fody
It's a weaver with several plugins like https://github.com/Fody/PropertyChanged
Generally, the sequence of the weaver-plugins can matter, plus some have configuration options.
The sequence and the configuration were done in an XML, FodyWeavers.xml
For convenience, when you installed a weaver-plugin, it updated the FodyWeavers.xml and added an entry for itself.
When relevant, the user still had to edit the FodyWeavers.xml manually to fix weaver-plugin sequence and -configuration. However, it was more convenient as he didn't have to look everything up in the Wiki.

Fody has been migrated to PackageReference, and now the user has to create the xml manually and also add entries to it manually. It's possible that he'll forget to create the entry upon installation (how would he know he has to create them, in the first place?).

This is not a huge deal, and compared to other broken use cases something of little importance.
It might help with showing a pattern (or patterns), though.

[jainaashish]

Thanks everyone, keep pouring these examples. These will certainly help us figure out right strategy going forward.

We'll analyze all these examples and then decides the next course of action. For time being, I'm reopening this issue for further tracking.

@anangaur

[augustoproiete]

@mnmr:

While performing all kinds of setup magic can be helpful in reducing first-time friction, it is equally frustrating to work with when you don’t use the package defaults

I hear you and I've experienced this pain myself too at times. Still, you're describing issues caused by package authors not doing a good job in terms of experience. I also experienced the opposite, where packages install and clean-up properly and reliably every time... The NuGet install.ps1 feature is not the problem here. Just because some authors don't do a good job, doesn't mean a similar feature shouldn't exist.

Just an idea, but it seems that a command-line argument to prevent scripts from running could ease most of the pain you describe. e.g. dotnet add package mypackage --no-scripts

[augustoproiete]

@mnmr:

Abusing the package manager as a trigger for all sorts of "setup magic" is just not the right solution

I respectfully disagree. Moving the "setup magic" to global dotnet tools doesn't change any of the "setup magic" necessary to consume certain packages... It just moves "the magic" somewhere else, makes it harder for everyone by adding extra steps for both consumers and developers (that now have more documentation to write, and more things to maintain, manage, and deploy)...

The main end-goal of NuGet should be to reduce friction for developers to re-use shared components. Asking users to download project templates separately, run additional scripts manually, and run global tools, is the opposite of reducing friction.

I'd love to get to a place where I can install a NuGet package that will automatically add new project templates related to the package (available only for that solution) and perform any kind of setup needed to get me going immediately.

[augustoproiete]

@mnmr:

A package manager should download packages, not run them. Running arbitrary powershell downloaded from the internet is dangerous at best

Please stop saying that because it's just silly... So "Running arbitrary powershell downloaded from the internet is dangerous" but executing arbitrary dotnet global tools downloaded from the internet is not dangerous?" 😃

In fact, if you were truly concerned about security, you wouldn't be using NuGet at all... Any NuGet package today can run arbitrary code on your machine. Not having the install.ps1 doesn't make it "safe" to install NuGet packages...

If you have a minute, install this NuGet package in any project (.NET Core or .NET Framework) and see it of yourself:

---

To be clear, I don't think bringing back install.ps1 is the answer, but I would like to see a strategy that bridges the gap created over two years ago that doesn't involve asking the user to do everything by hand, or jumping through hoops.

[mnmr]

Please forgive me for not quoting you all properly; consolidating your comments into topics helps keep the discussion noise down.

Frustration-Free Packaging
I don't buy the premise that the only solution to low-friction is the ability to execute arbitrary powershell code. If your package is that complicated to use/setup, most likely people will consult your web site anyway. Provide guidance, tools or scripts to help people get started.

Consider the example of Chocolatey. You are asked to open a shell and paste a single line of code. That's hardly friction.

Manual execution also enables customization (you can ask the user for things), which allows for much richer setup experiences.

Security
I absolutely agree that NuGet is currently insecure, but purposefully adding or restoring features that are known to be insecure only ensures that it will never become secure. An insecure package manager is a threat to the whole eco-system and we should stick to requesting features that do not compromise security.

And yes, I do believe that it is more insecure to have code executing automatically (as opposed to being explicitly run). For example, I may have started VS with admin privileges and forgotten about this when later adding a package.

Possible Solutions
I'd support a declarative solution, where package authors could specify certain actions to be performed, as long as the code performing those actions is part of NuGet itself. This would limit the scope of what can be done, thus providing both convenience and security. Even a limited set of actions would probably cover most use cases, so also not a herculean task to implement.

The ability to enable/disable install scripts could work. However, any solution that also requires VS changes (in this case the package manager UI) is much less likely to be adopted (or so I presume).

Using dedicated Setup packages (e.g. MyFancyCMS.Setup.nupkg) that only have the scripts would also go a long way, by making the choice to run something more explicit (and enabling opt-out by not adding the package). However, not all users would know about such a convention and I'm therefore not sure how viable it would be.

[BrunoJuchli]

@mnmr
It could also be designed to require consent to require install/uninstall scripts.
This could also be used to perform opt-out (i'm not sure how useful that is and if there should be a distinction between

• "oh this runs scripts! Might not be safe, I'd rather not install it"
• "I'd rather not run the script but install the package anyway"

I'd support a declarative solution, where package authors could specify certain actions to be performed

Certainly safer. But will be less flexible while costing a whole lot more to implement & maintain. Right?

At any rate, a good overview of the previous use cases is necessary to decide which of those are better addressed with different designs/tools (chocolatey, Dotnet tools,...) or are really best addressed by the package installation/uninstallation process.

[DerekFoulk]

@StingyJack I just don't think all of this "setup magic" belongs in a package manager; whether the feed is private/trusted does not significantly impact this.

Abusing the package manager as a trigger for all sorts of "setup magic" is just not the right solution to the problem. I fully understand the desire to have certain scenarios be easy and friction free, but we're talking once-per-project here.

If your org needs something that can do more than fetch packages, consider creating project templates, global dotnet tools (e.g. to inject stuff into a project after it has been created), custom scripts (you could add pre-build actions to your custom project template to ensure stuff has been added), use chocolatey or some other app installation system, etc.. There are literally a thousand ways to do this, and none of the good solutions involve abusing the package manager.

I just want Fody/PropertyChanged to automatically add FodyWeavers.xml again.

Extra Credit:

• Let me add and transform files when using PackageReference- like Web.config...
• Let me add files to a project, that the user can modify, in .NET Standard

We're all adults that are using source control. If a package modifies files that you don't want it to, you'll see the change before committing them (because you thoroughly review your 'git diff' before commiting 😉).

[jlmarrntx]

Have there been any updates on this topic? I assume ps1 files are still a no go...?

[danielmeza]

@jlmarrntx this is tracked here #5963

[zkat]

Thanks for all the discussion. We're not going to do this because it's a security risk.

[StingyJack]

@zkat It's not a security risk for private repositories, which is exactly where we usually need special "enterprise" tooling.

Nobody wants to have to do a bunch of stuff manually after installing a package, especially when it can be done automatically. It lowers the value of having a package.

There are other gaps in nuget that the ps1 solves. Maybe you should be fixing them instead of painting us into a corner.

It's not even a security risk for nuget.org, because you already scan packages for malware, etc.

[augustoproiete]

@zkat Is there an alternative plan that mitigates the security risk and, at the same time, allows for library authors to set up things upon package installation (and clean-up upon uninstall)?

Also, is there a plan to also block running MSBuild scripts bundled in the packages? Blocking the execution of PowerShell scripts doesn't mean package authors can't execute arbitrary code in the user's machine.

[IanKemp]

Thanks for all the discussion. We're not going to do this because it's a security risk.

Not acceptable. You cannot simply decide to remove a well-known, well-understood, and often necessary function without providing an alternative.

At least be honest by saying "we can't be arsed to implement this for Core" instead of hiding behind the already-debunked "security risk" excuse.

[gulbanana]

This has nothing to do with .NET Core. The feature is gone if you use PackageReference in old-style .NET Framework csproj files too, and that's a good thing.

[JorgeEAM]

Completely agree with the people who think that remove this functionality is a BIG mistake.

As it has been said, What about private repositories? And products as Azure DevOps and private organizations with private access and private repositories?

Don't hide yourself behind the excuse about security risk. Simply say, "We don't want to make it possible". The thread would be already closed.

[nhatkhai]

Thanks for all the discussion. We're not going to do this because it's a security risk.

Wow - another B.S. reason to step backward on technology improve my live.

I'm thinking start using nuget... But well I now cannot create a package that just run without my user install MICROSOFT access Engine, and FTDI driver.

Instead of me make one install/uninstall thing script for those and well tested, EACH of my nuget user would INVENT their own way for automation..... WOW.

We are shift security "blame" into the nuget user instead. I guest this is what it call shift left movement. It is the similar story on python with wheel package, but good that they do not just OBSOLETING the egg package which allow customize installation."pypa/packaging-problems#64

[nhatkhai]

Seem like software fork forget that software ran on hardware. There are packages that doing hardware and software too. Just unzip stuff would never work..... unless I could unzip the drivers into Windows OS too....

[nhatkhai]

Why don't you MICROSOFT keep the install,uninstall alone in .nuget. And introduce a .nugetx for only unzip concept only?

[nhatkhai]

@martinrrm, Shift the install.ps1 out side do not solving the security problem at all. At the end of the day what need to be run in install.ps1 would be run some where else in some other method.
I think if we want to improve security, we should instead lead install.ps1 run and introduce a while-list servers set. So each company may need a difference set of while-list servers to meet their own security definition. And sure a secure certification server similar like Debian apt: https://wiki.debian.org/SecureApt would do....
Then MS nuget server can also introduce muliti level of secure with some set of re-run script check list .....

[StingyJack]

@nhatkhai - The nuget team effectively stopped listening to the user community a few years ago. If it improves your mental health by responding in this thread, by all means keep doing so. Just dont expect anyone there to listen to you.

[nhatkhai]

where should I find open mind place?

[nhatkhai]

https://news.ycombinator.com/item?id=26087064
HOPE THIS Start some body engine again.

[Luensche]

Any news on alternatives? I need to add files to solution folder (like db scripts) on installing or updating packages.

[nhatkhai]

I recently found Conda package/environment manager - not idea, but workable solution as of right now. It can work with NuGet, and other package manger as a sub-level (call inside Conda pacakge).

So ideally everyone use Conda as the top-level for install package, it would cover all the DESIGNED limitation that NuGet, and other package managers got.

[nbarnwell]

Personally I made do with adding some powershell functions to the init.ps1, and adding a readme that tells people to open the Package Manager console and invoking those functions.

[nhatkhai]

I do that too. But it won't work for:

• Automation such as CI/CD
• If your package is not a 1st level of dependency such as A use B, and B use your. Developer of A would have a hard time to realize 1 package need to manual run a script out of 100 other package they used.

[nbarnwell]

• It works perfectly well in CI/CD, because your install.ps1 shouldn't need to do anything at build time, and you can use msbuild targets to do that.
• Even when a dependency is not a direct one, any readme files will be opened in VS when the package is installed for the first time (in fact I suspect the behaviour mimics that of init.ps1 in terms of when it happens). That's how you tell people they need to run some script.

Don't get me wrong, I don't really like it either. On mobile phones we are asked whether we give apps certain permissions, so it's not crazy to think we could be asked for a given package "Do you want to allow this package to run scripts at install time?", but the init.ps1 workaround does what I needed, and it's all we're going to get, as you can tell from Microsoft's response to this whole conversation. It's their game, their rules. The only way to win is not to play.

I do that too. But it won't work for:

• Automation such as CI/CD
• If your package is not a 1st level of dependency such as A use B, and B use your. Developer of A would have a hard time to realize 1 package need to manual run a script out of 100 other package they used.