Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dotnet update package for NuGet packages. #11812

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
179 changes: 179 additions & 0 deletions proposed/2022/DotnetUpdatePackage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
# Title

- [Jon Douglas](https://github.com/JonDouglas)
- Start Date (2022-05-11)
- [#4103](https://github.com/NuGet/Home/issues/4103)

## Summary

<!-- One-paragraph description of the proposal. -->
Staying current on the latest package version is a major challenge for modern .NET developers. Developers would like an update all experience in the .NET CLI similar to what they can do in modern IDEs like Visual Studio and checking all packages to then be updated in bulk. This feature will support the bulk updating of eligible `<PackageReference>` elements throughout their project files and solutions.

`dotnet update package` can automatically update any outdated package(s) to the latest version depending on the update strategy provided by the user.

## Motivation

<!-- Why are we doing this? What pain points does this solve? What is the expected outcome? -->
To rid developers of "DLL Hell" in managing modern .NET dependencies, we would like to implement this feature to rid developers of daily pain of managing their dependencies and staying current with the latest versions of packages to maintain a secure supply chain and new package feature functionality.

## Explanation

### Functional explanation

<!-- Explain the proposal as if it were already implemented and you're teaching it to another person. -->
<!-- Introduce new concepts, functional designs with real life examples, and low-fidelity mockups or pseudocode to show how this proposal would look. -->
This command will update dependencies to the latest version respecting the semver constraints of your package and its transitive dependencies. It will also install any missing packages if an newer version brings them in. It will take into account lock files and central package management as locations that exist in addition to common definitions of `<PackageReference>` in project file(s).

To ensure compatibility with existing package definitions in various files, we should not overwrite or simplify XML elements, but rather only update package version attributes.

![](../../meta/resources/dotnetupdatepackage/dotnetupdatepackage.png)

```
dotnet update [<PROJECT>|<SOLUTION>] package <PACKAGE_NAME>
[-f|--framework <FRAMEWORK>] [--interactive]
[-n|--no-restore] [--package-directory <PACKAGE_DIRECTORY>]
[--prerelease] [-s|--source <SOURCE>] [-v|--version <VERSION>]
[--dry-run] [--highest-minor] [--highest-patch]

dotnet update package -h|--help
```

#### Arguments

- PROJECT | SOLUTION
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there way to specify a directory of projects? i.e. I want to update all my packages in my \test directory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to limit the scope. So let's start with project or solution. Updating the world would be amazing, but very complex I think!

Copy link
Contributor

@chrisraygill chrisraygill May 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it? This design already necessitates the ability to "find a project" within the current directory when the <PROJECT> isn't specified. Might be oversimplifying, but isn't the functionality I'm describing just a loop to do that for all projects recursively within a directory?

Conceivably, one of the biggest users of this feature will be VS Code .NET devs who might not have a solution file to target. However, we know that many users prefer to keep all of their packages across all projects at the latest stable version.

It also adds a nice algorithmic component for an intern 😉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's possible. For example if you ran the command on the repo folder, it should work recursively.

Copy link
Contributor

@chrisraygill chrisraygill May 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on this line:

The project or solution file to operate on. If not specified, the command searches the current directory for one. If more than one solution or project is found, an error is thrown.

The current design will error out if you run it on the repo folder because multiple projects will be found. My expectation would be that it would run recursively/ apply to all found projects.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change this language then!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How many folders deep would you search for projects?
All of it?

I think deciding the target of a command should be something we involve the SDK team on given that package is a first class noun in dotnet.exe.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@baronfel Any idea how we can get more eyes from SDK team on these types of things? :)

Copy link

@baronfel baronfel May 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can unfortunately can't tag @dotnet/dotnet-cli, so I'll poke a few here: @dsplaisted, @joeloff, @gkulin

I'd generally be in favor of globs/patterns for more SDK operations - I know a similar capability has been requested for many of the MSBuild-driving commands in the past. It's common to use globs for this kind of thing - **/*Test.*proj to grab every test project in a solution recursively, for example.


The project or solution file to operate on. If not specified, the command searches the current directory for one. If more than one solution or project is found, an error is thrown.

- PACKAGE_NAME

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

going to throw out a crazy idea here that might consolidate a few flags/options.

PACKAGE_NAME can be

  • pattern as described below, or
  • a list of 'package with version spec's, where 'package with version spec' means a string of the format {PACKAGE_NAME}@{npm-style version expression}

The NPM(and also gem/paket/etc)-style expressions allow you to easily express bounds for the update operation on a package-by package basis, unambiguously, in a way that PACKAGENAME -version VERSION can't scale to. It also removes the need for --higest-minor, --highest-patch, and --prerelease, as those can be specified with appropriate syntax in the correct positions.

This does come with a downside of a) documenting these and b) writing code to parse them, however.


The package reference to add. Package name wildcards should be supported and update a subset of a glob-style pattern package name. i.e. `Microsoft.*` would update Microsoft packages. The commonly used symbols to support are:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know if there's an ask for this at this point?
Would a list of package ids be preferable instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The primary ask is updating all packages. The secondary ask is having control of updating. A list of package IDs can be preferable for sure. This shorthand might be helpful in our ecosystem given prefix/source mapping.


- `*` - zero or more characters.
- `?` - single occurrence of any character.
- `.` - literal "." character

#### Options
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any plan to include a --compatible or have it automatically update packages to the latest compatible version if the package latest stable version doesn't support the TFM?

It's referenced in this comment: #4103 (comment)

It's probably beyond the scope an initial release, but a maybe a cool idea 🤷‍♂️

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet. We don't have compat client tooling yet outside of one field. Once we can use the server side stuff, perhaps we can light up this.


- -f|--framework <FRAMEWORK>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This becomes tricky quickly in the solution case.

For example, what if a project in a solution does not target that exact framework?


Adds a package reference only when targeting a specific framework.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably meant to say update.


- -?|-h|--help

Prints out a description of how to use the command.

- --interactive

Allows the command to stop and wait for user input or action. For example, to complete authentication.

- -n|--no-restore

Adds a package reference without performing a restore preview and compatibility check.

- --package-directory <PACKAGE_DIRECTORY>

The directory where to restore the packages. The default package restore location is %userprofile%\.nuget\packages on Windows and ~/.nuget/packages on macOS and Linux. For more information, see Managing the global packages, cache, and temp folders in NuGet.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can not add this.
I know add package has it, but I'm wondering how commonly it's used if at all.

Specifying the global packages folder makes more scene in commandline restore scenarios that add/update imo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly we don't have much of a signal of how much this is used. Does this still make sense for update scenario?


- --prerelease

Allows prerelease packages to be installed. Available since .NET Core 5 SDK
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Copied from add package? :D Probably remove the last part.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes for consistency sake!


- -s|--source <SOURCE>

The URI of the NuGet package source to use during the restore operation.

- -v|--version <VERSION>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you specify version for a glob of packages as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to lock down the update strategies first I think. In general, I don't think so unless people have a compelling reason to do this!

Copy link
Contributor

@chrisraygill chrisraygill May 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember having a conversation with @nkolev92 about how some packages need to have their version kept in sync, particularly in ASP.NET Core world. I believe a lot of those packages also share prefixes like these:

image

Azure packages are also versioned similarly:

image

In these cases dotnet projectA update package Azure.Storage.* 12.10.0 might be pretty handy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should be supported with current spec right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, unless explicitly disallowed. I just wanted to clarify the intention 🙂

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be preferrable to specify a list of ids instead?


Version of the package. See NuGet package versioning.

- --dry-run
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this display the changes in transitive packages as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Top-level for now since those would be the only "affected" packages per-say. But let's keep this open for more perspectives.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, top-level only by default but have a --verbosity option to include more info.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should start with top level ones, but we can consider adding it later.

It's probably worth considering whether we do deltas or maybe just display the new list of packages.


Displays what would be updated, but doesn't actually do the operation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default is major?

Wonder if we need an option that takes parameters instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open to perspectives! Right now yes, major default with two flags for minor and patch.


- --highest-minor

Considers only the packages with a matching major version number when searching for updated packages.

- --highest-patch

Considers only the packages with a matching major and minor version numbers when searching for updated packages.

#### Examples

- Updates all NuGet packages in the current working directory.

```
dotnet update package
```

- Updates a specific NuGet package in the current working directory.

```
dotnet update package <PACKAGE_NAME>
```

- Updates a glob of NuGet package(s) in the current working directory.

```
dotnet update package Microsoft.*
```

- Displays what would be updated in the update operation for the current working directory.

```
dotnet update package --dry-run
```

#### Exit Codes
Copy link
Contributor

@chrisraygill chrisraygill May 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it all or nothing? Is it possible for some packages to update while others fail?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still thinking through it. Any suggestions?

Copy link
Contributor

@chrisraygill chrisraygill May 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, it shouldn't be all or nothing by default - but may want to provide an option for it if users want to take advantage of the exit code - but that's a secondary need.

If I have a project with an older framework where the latest version of half my packages still support my TFM, but the latest version of the other half don't, an all or nothing design would prevent me from bulk updating altogether.

Based on feedback about version preferences from the HaTS survey, I think the typical user will likely want to update all compatible packages to the latest stable version, even if other will fail. I think it's probably a fairly small subset that will take advantage of the exit code for automation purposes.

Maybe --atomic or --disable-partial would make sense as an option for all or nothing 🤷‍♂️

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think update by tfm compatibility is a special scenario.
It's one of those few scenarios where a failure is that exact package only.

The trickier one is what if the latest versions causes resolution conflicts, such as downgrades etc.
It becomes a user decision what they'd want to do then, so something semi interactive.
--no-restore is a an option that skips validations.
Right now the most common failures at, NU1605, NU1105, NU1201 and NU1104.

Back to the compat scenario, I think it's the one that we're most likely to solve. Worth noting that we'd have performance/implementation challenges for any source that isn't nuget.org.
Right now, search by tfm is a search thing only, we'd need to validate the perf issues.

tldr; I feel the pain of the customers, I run into some of these myself, but solving this in the first iteration might be a reach.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another "fun" case when updating by tfm compatibility that I run in to is that dotnet-ef supports something like dotnet core 2.1 in the latest version so if you update everything to the latest compatible version the EF Core packages might get out-of-sync. I really whish to be to update all packages to the latest compatible version but it's not as easy as it might seem.


- 0 - NuGet succeeded in update.
- 1 - NuGet failed to update.

### Technical explanation

<!-- Explain the proposal in sufficient detail with implementation details, interaction models, and clarification of corner cases. -->

## Drawbacks

<!-- Why should we not do this? -->

While `dotnet add package` provides some of this functionality, it doesn't give much control, ease-of-use, or discoverability to a simple update command.

## Rationale and alternatives

<!-- Why is this the best design compared to other designs? -->
<!-- What other designs have been considered and why weren't they chosen? -->
<!-- What is the impact of not doing this? -->

There currently doesn't exist a .NET CLI equivalent of this command and the impact of not doing this can lead to small papercuts, general dissatisfaction compared to other package management experiences and extensive workarounds instead of a common command that provides a solution.

While there has been one design in the past, it was not extensive enough to gather feedback from the community and surrounding teams on whether the direction is sound to pursue.

## Prior Art

<!-- What prior art, both good and bad are related to this proposal? -->
<!-- Do other features exist in other ecosystems and what experience have their community had? -->
<!-- What lessons from other communities can we learn from? -->
<!-- Are there any resources that are relevant to this proposal? -->

- [NuGet update](https://docs.microsoft.com/en-us/nuget/reference/cli-reference/cli-ref-update)
- [cargo update](https://doc.rust-lang.org/cargo/commands/cargo-update.html)
- [npm update](https://docs.npmjs.com/cli/v8/commands/npm-update)
- [RubyGems update](https://rubygems.org/gems/rubygems-update)
- [dart pub upgrade](https://dart.dev/tools/pub/cmd/pub-upgrade)
- [pip install -U](https://pip.pypa.io/en/stable/cli/pip_install/)

## Unresolved Questions

<!-- What parts of the proposal do you expect to resolve before this gets accepted? -->
<!-- What parts of the proposal need to be resolved before the proposal is stabilized? -->
<!-- What related issues would you consider out of scope for this proposal but can be addressed in the future? -->
- Should this command use `dotnet add package` under the hood for the scenarios that make sense such as finding the latest version of a specific package name?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's a technical question that'll likely be accounted for by the implementer.
These 2 commands will share a bunch of common functionality.

- What verb makes most sense? `update`, `upgrade`, `refresh`, etc? Update seems the most consistent for package managers such as cargo, npm, rubygems, and NuGet. Upgrade seems the most consistent for package managers such as pip.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth noting that "update" is consistent with the current PMUI terminology.

- Should `--highest-minor` and `--highest-patch` be replaced with a single `--dependency-version <version>` parameter to account for more scenarios like `Lowest(default)` and `Highest`?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Lowest vs Highest is a larger feature than the update command itself.
it makes it difficult for me to suggest a name based on what that feature may turn out to be.


## Future Possibilities

<!-- What future possibilities can you think of that this proposal would help with? -->
- Any CI/CD environment can leverage this CLI command as an easy way to update dependencies or check for them with exit codes.
- Any project can leverage `dotnet update package` and build on-top of it in the ecosystem.