PackRef does not show transitive dependencies - install tab & update tab behavior?

[akamyshanov]

When using "legacy"-style .csproj project files we have a separate packages.config file where all dependencies are listed, including transitive ones. This allows a use case when one installs a package with dependencies and then decides which transitive dependencies can be manually updated. So, the benefits are:

• Dependencies are easily identifiable due to presence of a flat list
• Fine-grain control over all dependency versions

There are drawdowns of course: the flat list can blow up to a huge one.

E.g., after installing Autofac.WebApi2.Owin from NuGet, we have a picture like this:

Transitive dependencies which are clearly viewable can be manually updated very easily.

When using the new Sdk-style .csproj projects NuGet references are added as <PackageReference/> to the project file itself and transitive dependencies are resolved by MSBuild automatically:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net462</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Autofac.WebApi2.Owin" Version="4.0.0" />
  </ItemGroup>
</Project>

So, to update transitive dependencies, one would have to

1. Identify them (e.g. via obj/project.assets.json)
2. Add all of them explicitly to the project
3. Perform updates

And this has to be done after each update and for every (!) transitive dependency in the project which is nearly impossible.

This seems like a "regression" to me.

The best possible resolution would be to show transitive dependency updates in NuGet GUI or via CLI. If the user decides to update one of the dependencies, it would get explicitly added to the project. (similar to #3159)

Maybe there is some workaround to achieve the specified behaviour?

[davidfowl]

I'm curious, why do you need this? Once you update the transitive dependency do you expect it to be hoisted into your project? Packages can remove dependencies in future versions so why does it matter that the dependency in transitive in the first place?

Why not just add the ones you care about explicitly updating manually to your project?

[akamyshanov]

@davidfowl
Thank you for the answer.

why do you need this?

• Bug/security fixes in the dependencies
• performance improvements
• other reasons

Once you update the transitive dependency do you expect it to be hoisted into your project?

When I explicitly update the dependency, I would like this fact to be somehow respected by NuGet and build tools, yes.

Packages can remove dependencies in future versions so why does it matter that the dependency in transitive in the first place?

Sorry, I do not fully understand your question. If I haven't updated a transitive dependency, it would be removed as it is now. If I have updated it, it should probably stay in the project but it's open for discussion.

My key point here is lack of control from the developer. In my opinion it is important to see the full list of dependencies a project references. I understand that this list can be quite big or depend on the target framework.

• With packages.config I could see all the dependencies and their updates. I may or may not want to update them, but I do see them and can make the decision based on the change in version, release notes, etc.
• Currently with <PackageReference/> the dependencies are a black box with almost zero control from the developer. The behaviour is quite similar to NPM which has always been a headache for frontend developers.

Currently it's not possible to obtain the list of dependencies and to update them in a human-friendly manner (NuGet GUI, CLI).

In my opinion, the most simple solution would be to add a <PackageReference/> to my project in case of a manual update of a particular package. Maybe, the element can be marked that it's just an update and not a direct reference by the project, so it can be removed if there are no packages depending on it.

[davidfowl]

Bug/security fixes in the dependencies
performance improvements
other reasons

Security fixes is a fair concern but honestly this is a rabbit hole, if you have a deep dependency chain and your other dependencies don't update, it's on you to directly reference and update those dependencies transitively. If those transitive dependencies are removed in a future version and you're still using it directly nothing is going to tell you to remove it. In the worst case what you'll end up with is the flattened graph

I'm not sure what you mean by performance improvements means.

I can't think of a single package manager that supports transitive dependencies that treats those transitive dependencies as if they were added top level (showing updates and allowing direct manipulation).

In my opinion, the most simple solution would be to add a to my project in case of a manual update of a particular package. Maybe, the element can be marked that it's just an update and not a direct reference by the project, so it can be removed if there are no packages depending on it.

I agree with the first part but not the rest. Once a dependency is hoisted to the top level, it's treated differently (like any other top level dependency) and you're responsible for updating it. IMO, the default experience should be the way it is today and you can have an option (checkbox in the ui or a command line switch) that shows updates for all transitive dependencies. Clicking update on one of these hoists it to a top level dependency and from then on you're on your own maintaining it like a top level dependency.

[akamyshanov]

if you have a deep dependency chain and your other dependencies don't update, it's on you to directly reference and update those dependencies transitively.

Correct, I just want a human-friendly way to find and perform these updates as it has been with packages.config.

If those transitive dependencies are removed in a future version and you're still using it directly nothing is going to tell you to remove it. In the worst case what you'll end up with is the flattened graph

Yes, correct.

I'm not sure what you mean by performance improvements means.

I mean performance improvements in the updates. But it doesn't really matter, what matters is that reasons may be different.

I can't think of a single package manager that supports transitive dependencies that treats those transitive dependencies as if they were added top level (showing updates and allowing direct manipulation).

NuGet working with packages.config? :) (yes, I know that it just adds all dependencies as top-level ones)

I agree with the first part but not the rest. Once a dependency is hoisted to the top level, it's treated differently (like any other top level dependency) and you're responsible for updating it.

Fair enough.

IMO, the default experience should be the way it is today and you can have an option (checkbox in the ui or a command line switch) that shows updates for all transitive dependencies. Clicking update on one of these hoists it to a top level dependency and from then on you're on your own maintaining it like a top level dependency.

Sure, that was my idea in the first place, the "improvement" was just a proposal.

To sum up, IMO additional tab(s) in the NuGet UI with transitive dependencies and updates (and similar CLI switches) would bring back workflows that are similar to NuGet's behaviour with packages.config that some developers (including me) are so used to. Enabling such a thing can be implemented with a global switch in the settings and/or a project-level setting.

[davidfowl]

As @akamyshanov pointed out, what if there's a critical security update in the update? Now you can say, "well, you can add Autofac back as a top level" but that then entirely defeats the point of having transitive dependencies.

Ideally, you'd want the things that depend on the thing with the security update to update as well right? That way you don't end up copying lots of leaf nodes to each project. Neither approach scales IMO, either way you'll end up with top level dependencies.

Ideally what we need is a mechanism to mark that the 2.1 update of Package B is a "TransitiveUpdate" and that if no top level projects or their dependencies depend on PackageB, it should be removed.

Lets take the scenario you mentioned, you start with this project:

Autofac.Owin 4.0.0 -> Autofac 4.0.0 -> System.Magic 1.0.0

<PackageReference Include="Autofac.Owin" Version="4.0.0" />

Time goes by and there's a new version of Autofac 4.0.1 but there's no new Autofac.Owin. So you use this fancy new feature we just invented:

<PackageReference Include="Autofac.Owin" Version="4.0.0" />
<PackageReference Include="Autofac" Version="4.0.1" Transitive="true" />

Then there's another security update for System.Magic 1.0.3:

<PackageReference Include="Autofac.Owin" Version="4.0.0" />
<PackageReference Include="Autofac" Version="4.0.1" Transitive="true" />
<PackageReference Include="System.Magic" Version="1.0.3" Transitive="true" />

Adding this new reference type to avoid the dangling reference problem doesn't buy you much IMO. Instead, maybe a prune command could be introduced to do the same operation (removed things that aren't used).

[rrelyea]

Interesting customer feedback from people who have migrated from Packages.Config to PackageReference. @rohit21agrawal, @nkolev92, @jainaashish - please add this user experience different to our PC->PR list of issues, so we can consider as a whole.

[EamonNerbonne]

I think the security hotfix concept is a distraction here. Almost all updates aren't hotfixes; the vast majority of packages you use you will not use in a fashion that is typically exploitable - things that are exposed to the outside world are rare, so typically you'll be left with libs that don't check input for hostile content (but then again, it's not always the case this should be the libs responsibility in the first place).

There's a tradeoff here: do you want the normal workflow to be as smooth as possible (and thus not cluttered with indirect package references that you can really only mess up)? Or do you want to force everyone to explicitly decide about updates, so that potentially security relevant updates (those rare cases!) aren't missed? And don't forget that semver isn't (and cannot really) be enforced: so by updating indirect dependencies, even minor version, you can break your app. It's not even all that unlikely; you merely need to depend on some implementation detail the library does not consider relevant (e.g. the type of return value of some method is unchanged, but the order unspecified and now different!) but you, perhaps unintentionally, depend on.

I think explicitly choosing for simplicity is wise here; the new-style package references with implicit transitive dependencies were the right choice, and that remains so.

If indeed the risk of missing security updates is real - and i'm not saying it isn't - then perhaps there are other approaches to dealing with that with fewer downsides. E.g., it would be cool if a package (version) could be marked obsolete or even explicitly insecure; something like the suggestions made in #2867 sounds more targeted, and really even better for security too.

[akamyshanov]

@EamonNerbonne Everything you're saying is correct, however security updates are just an example. It's not the reason I'd like to focus on but the choice.

do you want the normal workflow to be as smooth as possible (and thus not cluttered with indirect package references that you can really only mess up)?

Yes, of course, smooth workflows are cool :)

Or do you want to force everyone to explicitly decide about updates, so that potentially security relevant updates (those rare cases!) aren't missed?

Absolutely not, I do not want anyone forcing anybody their workflows, that's the point. As stated earlier, this can be an optional switch that enables separate tabs in the UI.

I think explicitly choosing for simplicity is wise here; the new-style package references with implicit transitive dependencies were the right choice, and that remains so.

Not arguing that it's the right choice, I do believe that it really is for a majority of developers.

I just want a way to be able to view all transitive references as a flat list (without going through the tree manually) and decide if I want to update them.

[markusschaber]

Apart from the security issues example discussed above, we have a different use case:

We need this transitive dependency list for license compliance. If we don't know all the indirect dependencies, we don't know which licenses we have to obey (and which packages we should avoid because one of their transitive dependencies comes with a license which is not acceptable in the current project).

Thus, we don't care that much about adding transient references, but we strongly need a way to list all (direct and indirect) dependencies of a project / solution - best if it also contains license indicator.

[anangaur]

Related #5553

[gordon-matt]

I have a good reason for needing to be able to update transitive dependencies. In my case, I am using the Peachpie PHP compiler for .NET Core. I ran into an issue, which left me stuck unable to update all the dependencies, because 1 of the dependencies was a transitive one and COULD NOT be referenced directly in the main project, as it is the SDK and makes changes to the project (like removing the "Dependencies" folder. You can see an image of the problem here: https://ibb.co/nvix2H.

So I need to be able to update that transitive SDK dependency WITHOUT adding it to the main project.

[jainaashish]

@gordon-matt you can't update transitive dependency without making it top level. This issue is all about making it convenient to show all the transitive dependencies coming along with your top-level and allows you to update any of those transitive by making it as top level too.

Your issue is very specific to your file manager library and it's dependency on peachpie. You'd need to keep releasing new version in order to consume updated peachpie SDK.

[gordon-matt]

@jainaashish Thank you for your response. The point is there are valid reasons (even if they are rare) for wanting to be able to update transitive dependencies without making them top-level. How hard could it be to enable such a thing? Maybe just an additional "Update" option in the right-click context menu when clicking on that transitive dependency in the project. Is this really not possible?

[anangaur]

It boils down to the information that NuGet/project has about these dependencies. With PackageReference, the only information that the project has is the direct dependencies. The transitive dependencies are computed at the time of restore.

In order to persist the transitive dependency's version, the only way possible today will be capturing the information as PackageReference in the project file that will make it direct/top-level dependency. Persisting the transitive dependency information will be a huge change.

We are planning a couple of features to improve PackageReference experiences as captured as part of this Epic issue. I guess Centrally managing package dependencies will solve this problem but it will also change the way you manage package dependencies. We would love hear your feedback.

[nkolev92]

As @anangaur said, there are ways to address your scenarios and plans for improvements for the same.

@gordon-matt
NuGet resolution is rather elaborate, for reference look at our docs

The versions in direct dependencies are the most important ones and NuGet will always attempt to restore those exact one if available, while for the transitive ones, they're dynamic and NuGet would attempt to find the best matching ones, satisfying all the conditions in the direct references (2 different direct package can depend on 2 different version of the same package etc).

Adding a specific version for a transitive dependencies would introduce a lot of complexity, and likely lead to problems trying to resolve the graph. It effectively elevates a transitive dependency to same importance as a direct one (which is what we're recommending be done anyways)
So this is a rather significant undertaking.

[pantonis]

I have just updated to PackageReference. I have one question thought. How can I view the transitive dependencies of a nuget package? Solution explorer just shows the top-level dependencies. Is there any way of viewing them. Maybe adding an expand option?

Thanks

[anangaur]

It already has an expand option for SDK style projects (.NET Core).

[pantonis]

What about .NET?

[mungojam]

For .net projects you can try the project converter that I've contributed to and used : https://github.com/hvanbakel/CsprojToVs2017 or wait for Microsoft's version which they announced at Build conference

[EmilAlipiev]

@mungojam is it not same as right click on packages.config and select "migrate from packages.config to projectreferences" option?

[mungojam]

@EmilAlipiev no, that just converts the way nuget packages are handled. It doesn't convert the project format so the legacy project loader is still used and you don't get essential features like the transitive dependency tree.

[chrisraygill]

Hi @alex-jitbit! Providing a solution to this problem is priority for us that we plan to begin working on soon.

@JonDouglas wrote a design spec to show transitive packages in the VS Package Manager UI some time ago: https://github.com/NuGet/Home/blob/dev/proposed/2020/Transitive-Dependencies.md

There are still some design questions we want to address before we begin implementation, but the existing spec should a give a solid idea of the direction we plan to go in. Any feedback you have would be appreciated 🙂

The best way to view and manage transitive dependencies today is with the dotnet CLI. You can use dotnet list package --include-transitive to view transitive dependencies and versions. From there, you can directly install a transitive dependency to the desired version with the dotnet add package command.

Let me know if that helps!

[Pilchard123]

This MS devblog seems relevant to the discussion here.

Introducing Transitive Dependencies in Visual Studio

[JonDouglas]

Hi friends,

🎉 We recently released an experiment in Visual Studio 17.3 regarding this feature. 🎉

https://devblogs.microsoft.com/nuget/introducing-transitive-dependencies-in-visual-studio/

While you all may not be included in the experiment group today, we believe this work concludes the spirit of this issue. If we find no major bugs or perf issues in our experiments, we will release it to everyone in the next major release.

For further feedback on enhancing this experience or the existing dotnet list package --include-transitive, please file a new issue for us to track.

Thus we are closing this issue as completed!

[alex-jitbit]

It's been almost 2 years and I still don't see dependencies in Visual Studio

[cremor]

@JonDouglas Has this been removed from Visual Studio again? I remember that I saw the transitive packages in the NuGet package manager in Visual Studio some time ago, but now (with VS version 17.5.5) I don't see it any more.

[JonDouglas]

@cremor We're gearing up to release it for everyone as we ran into a few performance regressions we had to resolve. /cc @jebriede

[cremor]

@JonDouglas I've just updated to VS 17.6 and still don't see the transitive dependencies. When will this be released? Is there an (open) issue that I could follow?

edit: I can now see the transitive dependencies in VS 17.6.3

[Halofreak1990]

@JonDouglas I've just updated to VS 17.6 and still don't see the transitive dependencies. When will this be released? Is there an (open) issue that I could follow?

edit: I can now see the transitive dependencies in VS 17.6.3

Is there something specific that needs to be done to enable this? I'm using VS 17.12.0 Preview 2.1 and I don't see transitive dependencies in the NuGet package manager

[JonDouglas]

@JonDouglas I've just updated to VS 17.6 and still don't see the transitive dependencies. When will this be released? Is there an (open) issue that I could follow?
edit: I can now see the transitive dependencies in VS 17.6.3

Is there something specific that needs to be done to enable this? I'm using VS 17.12.0 Preview 2.1 and I don't see transitive dependencies in the NuGet package manager

Can you provide a screenshot?

[cremor]

@Halofreak1990 Are you looking at the NuGet Package Manager UI for one project? That should show transitive dependencies.
But the NuGet Package Manager UI for a whole solution will not show them.

[Halofreak1990]

@cremor, no, I was looking at the solution-level NuGet packages. I can indeed see the transitive packages on the project level. Thanks.

[JonDouglas]

@Halofreak1990 Are you looking at the NuGet Package Manager UI for one project? That should show transitive dependencies. But the NuGet Package Manager UI for a whole solution will not show them.

This now exists in 17.12 Previews I believe(not sure exact release atm). We added solution level and have a blog coming soon.

[nkolev92]

17.12 preview 3 is where the transitive dependencies on the solution level PM UI will go live.
Please wait for another week or so and update then.