Enable server side deprecation of obsolete/legacy NuGet Packages

[daveaglick]

Update -12/21/2018 by anangaur: Spec

---

Original comment by @daveaglick:

As NuGet gets older, I've noticed a pattern where some packages have a description along the lines of "This package is old and shouldn't be used, use XYZ instead". It seems reasonable that as projects age, their packages may change names, split, combine, etc. over time. In fact, I'm faced with doing this for a number of packages myself soon.

I propose a feature where we can mark packages as deprecated or obsolete. This would be different from marking them as unlisted in that they would still be available on the gallery and would still show up in search results (perhaps with an additional GUI element to indicate the deprecated state). It would also be nice to specify which package(s) replace them and show that in the gallery and also when installing a deprecated package.

[yishaigalatzer]

@rrelyea this issue was logged before as far as I recall, please dedup

[daveaglick]

@yishaigalatzer @rrelyea I searched the GitHub repo and didn't find anything similar - perhaps it was in the old CodePlex site (I didn't search there), or maybe I just missed it.

[yishaigalatzer]

Its a good ask, I just recall seeing it before :) You might be right, we just need to spend a few more minutes trying to find the dup

[lilith]

I cannot find the duplicate. I also cannot find any issue which offers a workflow for dealing with package vulnerabilities. Or information on whether floating versions are supported in .nuspec files.

Oldest-possible + no deprecation/version flagging = scary context for security process. Hoping I'm just missing something obvious.

[Richiban]

I agree; this would be a seriously useful feature.

In particular I would also like to be able to deprecate specific versions of a package to indicate to anyone trying to install / restore this package that it no longer works (this situation normally arises when you write a client for an API and then the API itself gets deprecated or breaking changes are made). In this case the user should get a warning prompting them to upgrade their version.

[rjso]

It's a real shame this feature does not exist. I can come up with multiple reasons why we would like to discontinue a certain project and at least give a warning to whoever is using, in a similar fashion to the deprecated attribute in C#. Is this being considered? From my naivety, I suppose this is more of a strategic decision for the nuget team as opposed to amount of code needed.

[ChrisMaddock]

I was surprise to hear that chocolatey now has a deprecation feature, which NuGet still doesn't. We'd also have use for this feature - we're currently maintaining duplicate packages, as we don't feel we can remove old packages without causing mass disruption to our users. And automatic upgrade path to a new package, like choco has would be a great solution for this.

[CharliePoole]

I think chocolatey's "feature" is more a matter of convention. I might be wrong, but I"m pretty sure you can do the same thing in nuget:

1. Change title to say "Deprecated something" (so people notice it)
2. Advance the version - perhaps some convention like x.y.9999.
3. Remove files and add a reference to the new package

That only works if you choose to no longer have the old package available. I think this issue is about keeping the package but also deprecating it and having nuget know it's deprecated. I don't think that's possible using a convention - you need an extra bit somewhere.

[PeteX]

If this feature gets implemented, please can we also have a flag saying, 'This package has a vulnerability.'

Our deployed applications have to be reviewed periodically to make sure they are still secure. Part of this process involves checking dependencies to make sure none of them have released security patches. At the moment it's almost impossible to distinguish between feature releases, releases that fix normal bugs, and security releases. This wastes a lot of time for us, and presumably for other people who are trying to take security seriously.

[anangaur]

Seems like a good feature to have as NuGet.org grows and packages become obsolete. I am trying to summarize the scenarios for deprecation here. Please add to the list (and I will add to this list so that it would be easier to design a feature that solves the most of the issues):

1. I released a pre-release version of the package but I would like developers to move to stable version that I released later
2. I released a package version that had serious bugs in it and I would like to recommend developers using this version to move to the latest version. I would also like the packages taking dependency on this version of my package to move off this version to a newer version.
3. I had released a package that made use of currently deprecated/discontinued set of APIs and hence I would like developers to move to a newer version of this package.
4. I have a newer package (with different ID) on NuGet.org that should now be used instead of the given package and I would like developers to somehow not taken dependency on this package.

[PeteX]

@anangaur By far the most important thing for me is 'This package has a vulnerability, so users need to upgrade to maintain their systems' security.'

The big prize here, IMO, is being able to construct an automated system that tells you about important package upgrades you should be doing. If you're not bothered about automating things, you don't need to build anything extra. You can just release a new version of the package, and put something in the release notes saying why.

Incidentally, I think people uploading a new version of a package to Nuget should be asked whether there are security fixes, and forced to say 'yes' or 'no'. That's the only way to make sure that this essential information is captured, because otherwise some people will fix security exposures while not bothering to set the flag (perhaps because they just don't know about it).

[StefanBertels]

I miss "deprecated" flag, too. Maybe this issue isn't already implemented because we make scope too large?
"vulnerable" might be technically similar (flag), but this flag is not that easy.

"deprecated" is quite simple: If something is deprecated without follow-up this means "no longer maintained / think about removing it". With a follow-up this means "please upgrade by switching to package xyz".

If you search for deprecated packages, you should have to enable some switch (similar to "include prerelease"). If you restore deprecated packages a warning should occur.

It's not easy to define semantics (why is it deprecated) and we should let users decide how to handle the warning/flag. Maybe package is useless (incompatible API), maybe not, maybe partially.

Security is more difficult: Who defines "is vulnerable"? Package owner, only? There are different levels of vulnerability. Is it only a problem when used in ASP.NET (world accessable)? Only a special feature has a flaw? If you have some openssh like library which support unsecure methode (RC4) is this a vulnerability of the package? Are unflagged packages secure?

I see limited use for a "vulnerable" flag. Only useful aspect I could see: This might open a way to let package owners (or NuGet.Org) send security warnings to users. A common (existing) alternative for many projects is a security-announcement mailing list...

[anangaur]

"deprecated" is quite simple: If something is deprecated without follow-up this means "no longer maintained / think about removing it". With a follow-up this means "please upgrade by switching to package xyz".

Today one can de-list the package and such a package has the following information:

Summarizing the missing pieces:

1. Ability to label an unlisted package with special tag/reason with separate meanings:

• Deprecated: When a version of the package is deprecated by the author and the author recommends either not using the package+version or using a newer non-deprecated version.
• Vulnerable: When the package version contains a security vulnerability and the author recommends not using the package and instead a newer patched version is recommended.
  • Authors should optionally be able to provide a CVE number
• Legacy: When the package is no longer maintained by the author and author may have published another package (ID) instead.
  * Each of the above takes an optional recommended package.

2. Client warning when an unlisted package is installed/restored with one of the reasons specified above. If an additional recommended package+version is specified, clients should be able to show that information as well.

[PeteX]

@StefanBertels I would settle for a vulnerable flag that is yes or no, and set by the package owner. Our approach is that we update any vulnerable package, rather than trying to determine whether the vulnerability applies to us. It generally ends up being quicker, and it reduces the likelihood of making errors which allow vulnerable code to stay in production.

For people who don't do that, a vulnerable marker would still be useful because it would flag up the package for investigation. One way to make things a bit smoother, I suppose, would be a way for someone to specify on the client side that they have investigated CVE-1234 and decided that it's not relevant to them. Conceptually it would be a bit like suppressing a warning in some source code.

The reason people aren't screaming for a vulnerable flag, I think, is that most people just ignore security until it bites them. They deploy code with packages that were believed secure at the time, and then they just forget about it. This is very poor practice and people who do it should be shouted at by the tooling. :)

[StefanBertels]

@PeteX:
I agree, but what motivates package owners not to be ignorant?

(Package) users don't care, too. What happens if you "scare" your users with security warnings?

There is a long way to go.

[petertiedemann]

@anangaur I am very interested in the Legacy option. Without it, changing package ids (Which is frequently tied to assembly namespaces) become a major pain for users. I would very much like to see NuGet automatically ask the user whether to upgrade to the new package id, and then apply that change without having to mess around with manually uninstall/install.

[dotMorten]

. Unless you have a fixed and regular release cycle,

Perhaps not common for many open source projects, but I'd think most companies do (including Microsoft), since most customers require it.

[PeteX]

This looks good, thank you, and it will save a lot of time auditing our deployed applications. A few comments:

• A command to audit all packages in a project is described as out of scope, but later in the document it talks about the "dotnet audit" command which seems to do just that. Am I missing something?

• Will it be possible to add a "vulnerable" flag to a package that is already deprecated for some other reason? This seems quite useful. I might keep using a "legacy" package if it works for my use case, but I still want to know if a vulnerability is found.

• NPM seem to have a proactive process rather than just relying on package authors. You can report vulnerable packages and NPM themselves then manage the process, so reports don't get lost if the package maintainers are not responsive. You describe this sort of thing as out of scope, and I realise you can't do everything at once, but something like this seems highly beneficial. A lot of the smaller packages are just spare time projects and personal commitments may mean that the author is focused elsewhere at the time when the vulnerability report comes in.

These are just a few thoughts but overall it looks very good and will be a big help keeping our systems secure.

[anangaur]

@PeteX, Thanks for the comments. My response:

A command to audit all packages in a project is described as out of scope, but later in the document it talks about the "dotnet audit" command which seems to do just that. Am I missing something?

This was a typo and has been fixed.

Will it be possible to add a "vulnerable" flag to a package that is already deprecated for some other reason? This seems quite useful. I might keep using a "legacy" package if it works for my use case, but I still want to know if a vulnerability is found.

Yes. Multiple deprecation reasons can be selected. Refer to the **3. Select Reason(s) storyboard screenshot in the publisher experience.

NPM seem to have a proactive process rather than just relying on package authors. You can report vulnerable packages and NPM themselves then manage the process, so reports don't get lost if the package maintainers are not responsive...

Deprecation is a publisher based experience. With this feature, we will enable publishers to deprecate packages so that the deprecation warning reaches the consumers of the package. And one of the reasons for deprecation is security vulnerability. NPM audit or the likes is consumer based experience that does not rely on publisher providing this information and rightly so. We are having discussions on similar functionality but it may take some time to get there. And when we implement a similar feature, it should absolutely be more proactive.

[bruno-garcia]

@dotMorten - this is great feedback. My hypothesis was that you would deprecate all package versions for legacy reason. Seems like thats not quite right. We can definitely do away with the warning.

This feature is interesting for sentry.io to mark a whole package (SharpRaven, all versions) as legacy and point the users to a new package (Sentry). So the original assumption, at least in my case does make sense.

[anangaur]

@bruno-garcia That's my hypothesis too. And we want to warn you if you haven't selected all versions. But we will not stop you if you don't want to select all versions.

[dotMorten]

@bruno-garcia @anangaur I never meant to say your scenario isn't valid. It was just that my scenario wasn't covered :-)

[bruno-garcia]

@dotMorten you're right. I made my comment in an attempt to keep both use cases instead of replacing one with the other.

[anangaur]

With the latest proposal, both scenarios are covered :). It warns but allows you to continue deprecating a subset of package versions.

[dotMorten]

It warns but allows you to continue deprecating a subset of package versions.

I don't get why it warns? There's absolutely nothing wrong about deprecating only old or insecure versions. Warning people is essentially telling them that you really shouldn't be doing this.

[dealproc]

Is there a way to combine this with a tool like DevAudit for the vulnerabilities? I know this means forcing folks to file the vulnerabilities, but maybe there is a way where if vulnerabilities are entered via nuget.org as part of uploading the nupkg, the Nuget.org site could post into the cve.mitre.org system?

[OkGoDoIt]

It appears that this feature is currently live? From reading the issue and spec document, I was under the impression that deprecation functionality has not yet rolled out, but just now I was able to set it on a package. Is there a page tracking the current state of this feature?

[anangaur]

@OkGoDoIt The feature is not yet live but in active development. A glitch in the roll out process made the current not-fully-implemented feature live, inadvertently, for some time.

We will announce when the feature will be fully functional and live. Apologies for the false-positive.

[mattiasnordqvist]

I swear I had seen it somewhere! :D

[anangaur]

We are separating out the experience for Deprecating the packages and reporting vulnerabilities. This is because as we made progress on the spec and implementation, it felt more and more like we were merging separate experiences.

• Deprecation is a package publisher oriented feature that allows package publishers to inform the consumers to take off dependency of a given package.
• For vulnerabilities, its more a consumption oriented feature and cannot depend solely on the data provided by package publishers.

I will be forking and updating the spec soon.

[anangaur]

The spec has been updated to reflect the Deprecation feature and Vulnerability aspect has been moved out to a separate feature: [Security] Flag vulnerable packages #8087

[xavierdecoster]

Section CLI consumer experience of the spec has been updated.

[rrelyea]

Initial support for this is shipping in NuGet 5.3. Closing this issue to reflect it in release notes.
Further work will be tracked using other issues.

[blackboxlogic]

Here's the documentation for package deprecation.