NuGetPackageManager.InstallPackageAsync doesn't respect IgnoreFailedSources

[AndreyAkinshin]

If I have an unavailable feed in the NuGet.Config (e.g., http://localhost:9000/guestAuth/app/nuget/v1/FeedService.svc/), I can't install a NuGet package via NuGetPackageManager.InstallPackageAsync.
I tried to set ResolutionContext.SourceCacheContext.IgnoreFailedSources = true, but it doesn't help because NuGet.PackageManagement.ResolverGather.InitializeResourcesAsync doesn't check this flag and just rethrow an exception:

11:56:27|RIDER|ERROR| Exception 'System.AggregateException' thrown when trying to add source 'http://localhost:9000/guestAuth/app/nuget/v1/FeedService.svc/'. Please verify all your online package sources are available.
System.InvalidOperationException: Exception 'System.AggregateException' thrown when trying to add source 'http://localhost:9000/guestAuth/app/nuget/v1/FeedService.svc/'. Please verify all your online package sources are available. ---> System.AggregateException: One or more errors occurred. ---> NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source http://localhost:9000/guestAuth/app/nuget/v1/FeedService.svc/. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request ---> System.Net.WebException: Error: ConnectFailure (Connection refused) ---> System.Net.Sockets.SocketException: Connection refused
  at NuGet.Protocol.ServerWarningLogHandler+<SendAsync>d__1.MoveNext () [0x0007f] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.StsAuthenticationHandler+<SendAsync>d__9.MoveNext () [0x000bc] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.HttpSourceAuthenticationHandler+<SendAsync>d__9.MoveNext () [0x000c1] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.TimeoutUtility+<StartWithTimeout>d__0`1[T].MoveNext () [0x001bd] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.HttpRetryHandler+<SendAsync>d__0.MoveNext () [0x004a0] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.HttpSource+<GetThrottledResponse>d__20.MoveNext () [0x001d6] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.HttpSource+<ProcessResponseAsync>d__18`1[T].MoveNext () [0x000c5] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.ODataServiceDocumentResourceV2Provider+<TryCreate>d__8.MoveNext () [0x0023f] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.Core.Types.SourceRepository+<GetResourceAsync>d__16`1[T].MoveNext () [0x000b3] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.DependencyInfoResourceV2FeedProvider+<TryCreate>d__1.MoveNext () [0x000ee] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.Protocol.Core.Types.SourceRepository+<GetResourceAsync>d__16`1[T].MoveNext () [0x000b3] in <0cc41fb8f1d742228e77b6939c0d63b7>:0 
  at NuGet.PackageManagement.ResolverGather+<>c__DisplayClass31_1+<<InitializeResourcesAsync>b__0>d.MoveNext () [0x00072] in <747438c4536f4d6cb8a05bba4076e466>:0 
  at NuGet.PackageManagement.ResolverGather+<InitializeResourcesAsync>d__31.MoveNext () [0x003d8] in <747438c4536f4d6cb8a05bba4076e466>:0 
  at NuGet.PackageManagement.ResolverGather+<InitializeResourcesAsync>d__31.MoveNext () [0x00519] in <747438c4536f4d6cb8a05bba4076e466>:0 
  at NuGet.PackageManagement.ResolverGather+<GatherAsync>d__21.MoveNext () [0x000ae] in <747438c4536f4d6cb8a05bba4076e466>:0 
  at NuGet.PackageManagement.ResolverGather+<GatherAsync>d__20.MoveNext () [0x0006b] in <747438c4536f4d6cb8a05bba4076e466>:0 
  at NuGet.PackageManagement.NuGetPackageManager+<PreviewInstallPackageAsync>d__68.MoveNext () [0x00a07] in <747438c4536f4d6cb8a05bba4076e466>:0 
  at NuGet.PackageManagement.NuGetPackageManager+<InstallPackageAsync>d__48.MoveNext () [0x0009b] in <747438c4536f4d6cb8a05bba4076e466>:0 
  at NuGet.PackageManagement.NuGetPackageManager+<InstallPackageAsync>d__47.MoveNext () [0x000b2] in <747438c4536f4d6cb8a05bba4076e466>:0 

It also affects different NuGet clients. For example, I can't install a package in Visual Studio (15.7.4) while restore is working without any problems.

NuGet product used: NuGet.PackageManagement
NuGet version: 4.8.0
OS version: reproduced on Windows, Linux, macOS

[jainaashish]

IgnoreFailedSources is not enabled inside VS. Corresponding tracking issue# #5643

Please vote/comment there and follow-up for future update to that issue.

[AndreyAkinshin]

@jainaashish, my original concern was about broken NuGet API: it take an instance of ResolutionContext, but ignores the IgnoreFailedSources flag. Could you introduce a way to ignore failed sources in a NuGetPackageManager.InstallPackageAsync call?

[kskrygan]

@jainaashish @karann-msft @rrelyea Hi folks!
Could you please explain why you are closing this ticket as duplicate?
It seems to be clear that this issue has nothing to do with VS, it's a pure NuGet API problem.

[jainaashish]

NuGetPackageManager.InstallPackageAsync isn't a NuGet public api published and supported across versions. Here is the list of NuGet public APIs for reference, everything else may seem public (since the api visibility and the fact that anyone can take dependency on any package) but aren't really published APIs. And we're not going to explicitly add support for IgnoreFailedSources in this method.

But the same method InstallPackageAsync is used when we install a package from UI so when we enable IgnoreFailedSources through UI, it will also plumb it through that api, which is why I closed this as duplicate.

[AndreyAkinshin]

@jainaashish,
In the official documentation, there is a section about NuGet Client SDK: https://docs.microsoft.com/en-us/nuget/reference/nuget-client-sdk
It says:

The NuGet Client SDK refers to a group of .NET libraries centered around NuGet.Client, Nuget.Packaging, and NuGet.Protocol.

However, you wrote:

Here is the list of NuGet public APIs for reference, everything else may seem public (since the api visibility and the fact that anyone can take dependency on any package) but aren't really published APIs.

Only the NuGet.VisualStudio package includes APIs from your list (because the title of the list is "NuGet API in Visual Studio"). Do I understand correctly that other packages from the NuGet Client SDK (like NuGet.Packaging or NuGet.Protocol) don't contain public API at all?

[jainaashish]

Those libraries NuGet.client, NuGet.Packaging and NuGet.Protocol are part of NuGet SDK to support NuGet IVS apis and NuGet Web apis. So these libraries provide required model classes, utilities, or resources to consume NuGet public apis. There are no documented public apis out of these libraries but we try to make sure that these libraries are backward compatible.

[AndreyAkinshin]

part of NuGet SDK to support NuGet IVS apis and NuGet Web apis

Well, you publish these libraries on nuget.org, and they are available for everyone. There are a lot of different applications which use the NuGet Client SDK. I'm trying to understand your policy about these applications. Am I right that you are not going to make any changes which will be useful for non-Microsoft NuGet clients (all clients except Visual Studio or NuGet Gallery)?

[jainaashish]

Again you're misreading it. I only meant that those libraries have supported infrastructure for existing IVS apis and web apis. But any consumer is free to consume them or any other NuGet library as and when it fits their requirement. But they aren't publicly supported APIs.

Making a change in non-public apis or adding new ones always depends on the individual cases, we've done it in past depending on how valuable those changes are for the whole community. But it's also not possible to take every single request into consideration given the time n resource constraints.

Having said that, It's all open source and if there are enough community feedback to support this request, then we'll be happy to take a contribution. Or you always have an option to fork and have your own copy of NuGet (but that will add an overhead to maintain it at your own).

[AndreyAkinshin]

Having said that, It's all open source and if there are enough community feedback to support this request, then we'll be happy to take a contribution

@jainaashish, here is another issue from me: #3324 It has 72 👍 , dozens of comments, and even a blog post about this issue. You had an awesome solution for this problem in NuGet 2.x. However, you don't want to reimplement it in NuGet 4.x.

I (and some other developers) will be happy to implement a proper fix, but you ignore all our suggestions. How much community feedback do you need to accept such a PR?

[AndreyAkinshin]

@jainaashish still waiting for a response from you. How much community feedback do you need to accept a contribution?

[jainaashish]

I'm not the right person to answer that question. @rrelyea @anangaur might be able to throw some light there. But just a note about the original issue, there hasn't been any other complaint after yours and neither there has been any activity on the issue# #5643

[AndreyAkinshin]

@jainaashish,

But just a note about the original issue, there hasn't been any other complaint after yours and neither there has been any activity on the issue# #5643

I got such complaints from .NET developers every week. Is it OK if ask people who suffer from this problem to write comments on the issue #5643?

[anangaur]

@AndreyAkinshin voting #5463 up would definitely help us prioritize the issue against others.