restore fails hard by default if one source is unavailable/invalid/down

[livarcocc]

From @trondhindenes on July 15, 2017 23:16

Steps to reproduce

dotnet --version: 1.0.4
Given a nuget config with multiple feeds, the default (nuget.org) + another custom feed:
When trying to install a package which is present in the public feed, the install fails if not all feeds have the package or respond.:

in this example, the private feed doesn't respond at all:
dotnet add package App.Metrics

output:

info : Adding PackageReference for package 'App.Metrics' into project 'C:\Users\trond\Documents\projects\appmetrics-test\appmetrics-test.csproj'.
log  : Restoring packages for C:\Users\trond\Documents\projects\appmetrics-test\appmetrics-test.csproj...
info :   CACHE https://api.nuget.org/v3-flatcontainer/app.metrics/index.json
info :   CACHE https://api.nuget.org/v3-flatcontainer/app.metrics/1.3.0/app.metrics.1.3.0.nupkg
info :   GET https://nuget.myprivaterepo.no/api/FindPackagesById()?id='App.Metrics'
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://nuget.myprivaterepo.no/api/FindPackagesById()?id='App.Metrics''.
log  : An error occurred while sending the request.
log  :   The operation timed out
info :   GET https://nuget.myprivaterepo.no/api/FindPackagesById()?id='App.Metrics'

The process doesn't fail either, it just hangs there it seems.

In this example, the private feed is up, but doesn't have the package:

info : Adding PackageReference for package 'App.Metrics' into project 'C:\Users\trond\Documents\projects\appmetrics-test\appmetrics-test.csproj'.
log  : Restoring packages for C:\Users\trond\Documents\projects\appmetrics-test\appmetrics-test.csproj...
info :   CACHE https://api.nuget.org/v3-flatcontainer/app.metrics/index.json
info :   CACHE https://api.nuget.org/v3-flatcontainer/app.metrics/1.3.0/app.metrics.1.3.0.nupkg
info :   GET https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics'
info :   NotFound https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics' 1359ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics''.
log  : Response status code does not indicate success: 404 (Not Found).
info :   GET https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics'
info :   NotFound https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics' 505ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics''.
log  : Response status code does not indicate success: 404 (Not Found).
info :   GET https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics'
info :   NotFound https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics' 166ms
error: Failed to retrieve information about 'App.Metrics' from remote source 'https://nuget.myprivatefeed.no/api/FindPackagesById()?id='App.Metrics''.
error:   Response status code does not indicate success: 404 (Not Found).

Expected behavior

Package install should succeed if at least one of the feeds is able to satisfy the request, regardless of the status of other feeds. In both of the examples above the package should get installed, maybe possible with a warning for unreachable feeds.

Actual behavior

see description

Environment data

.NET Command Line Tools (1.0.4)

Product Information:
 Version:            1.0.4
 Commit SHA-1 hash:  af1e6684fd

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.15063
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.4

Copied from original issue: dotnet/cli#7171

[anangaur]

Today NuGet looks for availability of all the feeds when trying to install or add a new package and this is an existing issue we plan to look at. Its not necessary for the package to be present in all the feeds but they(feeds) should be accessible. So it would be good to check if the feeds are reachable. If not, its a known issue.

Additional 2 questions if the feeds are already accessible:

1. Is the feed an authenticated one? If yes, how is the creds being passed?

2. The description says that the package in question is not present on the private feed. So if UI: Action changes to updating previous version after installing #1 is not the case then is it fair to say that nuget works fine when the package is present in the feed?

[trondhindenes]

I'm not sure I understand. Like I described above, it is my experience that the package has to be accessible in all feeds (example 2) - which I consider a bug

NVM, I understand now. No, my private feed is not authenticated. As you can see from the logs, it responds with a 404, not a 401 which would normally be the case if there was authentication involved.

Your question nr 2: Yes, it works fine if the package is present in all feeds. Like my logs above show, nuget does not work fine if there are multiple feeds and the package is only available in one of them.

Also note that this issue was sent here from the dotnet/cli repo where I originally posted the issue. I have no idea if the bug is in nuget or in the dotnet cli, but it's my experience that "regular" nuget on .Net classic is much more forgiving than "dotnet add package" on .Net core. So if this is indeed a .net core cli issue and not a nuget issue, please let me know and I'll ask the cli maintainers to move it back.

[trondhindenes]

Another issue also worth noting is that the process in example number 2 is extremely slow caused by all the retries. I don't see the point in retrying a 404. The package isn't there. Move on.

[anangaur]

Your question nr 2: Yes, it works fine if the package is present in all feeds. Like my logs above show, nuget does not work fine if there are multiple feeds and the package is only available in one of them.

This seems weird. As mentioned before, we do have issues when any of the feeds in nuget.config is unreachable.

#5033 refers to similar issue. I am adding this to the list of issues with authenticated or in general multi-feeds category of issues tracked by #4949

[rohit21agrawal]

CC: @mishra14

[mishra14]

cc: @emgarten as this is a restore issue/design choice.

[emgarten]

This is by design, the option to ignore failing sources on restore exists to allow using only the available sources. Ignoring failing sources by default is dangerous since a source could go down and a restore could pass and end up with different packages.

For the add command I think having the same behavior as restore makes sense, it would be unexpected if add succeeded and then restore failed right after.

[anangaur]

@emgarten The issue reported here is need for the package to present on all the feeds which IMO shouldn't be the case and if it is, then this is a bug

[emgarten]

@trondhindenes do you have a repro project and feeds that demonstrate this?

A package only needs to be in a single feed, but all feeds need to be available for restore.

[trondhindenes]

Yeah, I have looked into this a bit more. Will post repro steps/details when back in the office