[Bug]: Custom kernel breaks nuget

[Killfrra]

NuGet Product Used

dotnet.exe

Product Version

.NET SDK Version: 6.0.302 Commit: c857713418

Worked before?

.NET SDK Version: 6.0.302 Commit: c857713418

Impact

It's more difficult to complete my work

Repro Steps & Context

It would be nice to be able to disable the sending of unnecessary information through the UserAgent.

...
///For Unix this currently comes from "uname -srv"
...
    RuntimeInformation.OSDescription

-- from dotnet/sdk/blob/main/src/Cli/dotnet/Telemetry/TelemetryCommonProperties.cs

In addition to what is shown in the "verbose logs", performing dotnet restore in a project that already has a PackageReference in .csproj gives an error:
error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.

Verbose Logs

$ ls
$ dotnet new console
$ dotnet add package Newtonsoft.Json
  Determining projects to restore...
  Writing /tmp/tmpUqFyzk.tmp
info : Adding PackageReference for package 'Newtonsoft.Json' into project '/home/Killfrra/Documents/test/test.csproj'.
error: Unable to load the service index for source https://api.nuget.org/v3/index.json.
error:   The format of value '(Linux 5.18.0-14.2-liquorix-amd64 #1 ZEN SMP PREEMPT_DYNAMIC liquorix 5.18-16ubuntu1~jammy (2022-0)' is invalid.


Usage: NuGet.CommandLine.XPlat.dll package add [options]

Options:
  -h|--help               Show help information
  --force-english-output  Forces the application to run using an invariant, English-based culture.
  --package               Id of the package to be added.
  --version               Version of the package to be added.
  -d|--dg-file            Path to the dependency graph file to be used to restore preview and compatibility check.
  -p|--project            Path to the project file.
  -f|--framework          Frameworks for which the package reference should be added.
  -n|--no-restore         Do not perform restore preview and compatibility check. The added package reference will be unconditional.
  -s|--source             Specifies NuGet package sources to use during the restore.
  --package-directory     Directory to restore packages in.
  --interactive           Allow the command to block and require manual action for operations like authentication.
  --prerelease            Allows prerelease packages to be installed.

$ uname -srv
Linux 5.18.0-14.2-liquorix-amd64 #1 ZEN SMP PREEMPT_DYNAMIC liquorix 5.18-16ubuntu1~jammy (2022-0
$

[heng-liu]

Hi @Killfrra , may I know if you have run restore successfully from source nuget.org on the same machine before?
It seems the Unable to load the service index for source https://api.nuget.org/v3/index.json. is the root cause.
Could you check if you're able to access to the source on this machine?

BTW, could you try to add a package from a local source (which contains the package you'd like to install, so it will definitely run successfully)? And check if the error: The format of value '(Linux 5.18.0-14.2-liquorix-amd64 #1 ZEN SMP PREEMPT_DYNAMIC liquorix 5.18-16ubuntu1~jammy (2022-0)' is invalid. still there?
You may run dotnet add package newtonsoft.json --source <local source path>
For local source, you may refer to https://docs.microsoft.com/en-us/nuget/hosting-packages/local-feeds

[Killfrra]

on the same machine before?

Yes

able to access to the source on this machine?

The URL opens fine in the browser

from a local source

Unfortunately, I can't do it right now.
But rebooting with a generic kernel helped.

[nkolev92]

@Killfrra I'm not certain how to intepret your last comment.

Are any you still facing any issues?

[Killfrra]

Yes, I still have this problem. I found a workaround but I want nuget

• not to break if the kernel has an unexpected name string
• to not pass the kernel string to the server

[nkolev92]

We can consider that.

As an immediate solution please consider changing the output of what the OSDescription returns to a format that wouldn't break the headers.

[rajvalentino]

would be great if you share the workaround for this issue.

[Nihlus]

I'm also encountering this issue, and it's making the dotnet nuget tooling near completely unusable. What workaround did you find for the problem, save booting from a generic kernel?

[Killfrra]

It's awkward and wild, but yes, booting with generic kernel

[Nihlus]

That's a shame. I was hoping there was some other workaround.

[cbaal83]

Hi, is there any news on this? Any workaround?
Thanks!

[zivkan]

I'm unsure where exactly .NET's RuntimeInformation.OSDescription, or uname -srv gets the strings from, but the problem from the original comment in this issue is that there's a ( without a matching ). Therefore, if by chance the string is coming from some config file (something under /etc/?), then you can edit it to match the parenthesis. If you're compiling your own kernel, or otherwise have control over the kernel string, make sure you have matched parenthesis.

This will let you mitigate until the above pull request is merged and shipping to customers.

[zivkan]

The fix should be released in NuGet 6.5, which will be in the .NET 7.0.200 SDK. Hopefully it'll be in the nightly build SDKs within a week.

[cbaal83]

Hi zivkan,
unfortunately it's still broken in dotnet-sdk-7.0.300-preview.23108.17-x64.
Do i need todo anything special in order to get your fix?

Best,
//CB

[zivkan]

Due to major CI issues in January, I missed the deadline for NuGet 6.5, so it'll be in NuGet 6.6. I don't know how .NET SDK preview numbers work (what date does 23108.17 correspond to?). Anyway, if you run dotnet nuget, it will print NuGet's version number on the first line. If it says 6.6.0.1 or higher, it should be included (and therefore my fix doesn't work for your scenario?). If it's a lower number, you'll need to update to a newer preview.

[baronfel]

what does does correspond to?

We just merged some docs about this in the PR guidelines for the SDK - @zivkan wanna take a look and see if that helps?

[zivkan]

So, it looks like it's from yesterday, the 8th of February. So it certainly has a 6.6 build of NuGet.

@cbaal83 can you please run this and tell me what the program output is?

using System.Net.Http.Headers;
using System.Runtime.InteropServices;

var os = RuntimeInformation.OSDescription;
os = os.Replace("(", "\\(").Replace(")", "\\)");
Console.WriteLine("OSDescription: " + os);

ProductInfoHeaderValue userAgent = new("(" + os + ")");
Console.WriteLine("User-Agent: " + userAgent);

[cbaal83]

Unfortunatly, i can't run this.
When i try to build, following message occurs:
: error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.

Unfortunately i can't easily switch to another kernel.

Does the Preview-Package install a system-wide version of nuget? Because the latest and greatest i was able to find online was "6.4.0.123", which is a version iam running using mono....
Not sure how that is handled by the SDK. Some internal nuget-client?
Best,
//CB

[zivkan]

The original post in this issue shows they got 2 lines of error, the first was the "Unable to load the service index", and the second one explained why "The format of value ...". Do you not get a second error, explaining why the service index couldn't be downloaded? Can you try restoring at a higher verbosity level? dotnet restore -v:n

Although the code sample doesn't need any packages, so I would have hoped that dotnet new console ; dotnet run should work without downloading anything extra.

NuGet 6.4 maps to the .NET SDK 7.0.1xx, so it's not expected for the 7.0.300 preview to have it. .NET SDK 7.0.2xx will have NuGet 6.5, and .NET SDK 7.0.300 will have NuGet 6.6. We all increment versions are the same pace, it's just we have different versioning schemes.

However, it also appears that the dotnet/sdk is not getting NuGet's 6.6 updates in its release/7.0.3xx branch: https://github.com/dotnet/sdk/blob/31945c5e5c1e85e289efd3d222fd9f76b42a3293/eng/Version.Details.xml#L108

@baronfel do you know if sdk needs to update its darc subscriptions to get the vs17.6 channel into the 7.0.3xx branch?

The first NuGet 6.6 insertion into sdk's main branch (8.0.1xx) was 8 days ago: dotnet/sdk@bc5bc9a

[baronfel]

@zivkan can you create the appropriate subscription for the release/7.0.300 SDK? we're not sure which would be the appropriate source branch from y'all's end. ignore this - we just figured it out and took care of it.

[cbaal83]

@zivkan
i tried your suggestion:

christian@christian-desktop:~/dev/csharp$ mkdir test
christian@christian-desktop:~/dev/csharp$ cd test/
christian@christian-desktop:~/dev/csharp/test$ dotnet new console
The template "Console App" was created successfully.
Processing post-creation actions...
Restoring /home/christian/dev/csharp/test/test.csproj:
  Determining projects to restore...
/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
  Failed to restore /home/christian/dev/csharp/test/test.csproj (in 89 ms).
Restore failed.
Post action failed.
Manual instructions: Run 'dotnet restore'

Best,
//CB

//EDIT:

Sorry, forget about the verbose restore:

christian@christian-desktop:~/dev/csharp/test$ dotnet restore -v:n
Build started 2/9/2023 9:09:37 PM.
     1>Project "/home/christian/dev/csharp/test/test.csproj" on node 1 (Restore target(s)).
     1>_GetAllRestoreProjectPathItems:
         Determining projects to restore...
       Restore:
         X.509 certificate chain validation will use the fallback certificate bundle at '/usr/share/dotnet/sdk/7.0.300-preview.23108.17/trustedroots/codesignctl.pem'.
         X.509 certificate chain validation will use the fallback certificate bundle at '/usr/share/dotnet/sdk/7.0.300-preview.23108.17/trustedroots/timestampctl.pem'.
         Restoring packages for /home/christian/dev/csharp/test/test.csproj...
     1>/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
     1>/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
     1>/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
         Writing assets file to disk. Path: /home/christian/dev/csharp/test/obj/project.assets.json
         Failed to restore /home/christian/dev/csharp/test/test.csproj (in 88 ms).
         
         NuGet Config files used:
             /home/christian/.nuget/NuGet/NuGet.Config
             /home/christian/.nuget/NuGet/config/Godot.Offline.Config
         
         Feeds used:
             https://api.nuget.org/v3/index.json
     1>Done Building Project "/home/christian/dev/csharp/test/test.csproj" (Restore target(s)) -- FAILED.

Build FAILED.

       "/home/christian/dev/csharp/test/test.csproj" (Restore target) (1) ->
       (Restore target) -> 
         /home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
         /home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
         /home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.

    0 Warning(s)
    3 Error(s)

Time Elapsed 00:00:00.56

[zivkan]

@cbaal83 sorry about all the back-and-forth. Could you also try setting the environment variable NUGET_SHOW_STACK to true. Hopefully that will show more details. As a side note, we (NuGet team) should improve the default NU1301 error, because when it's unable to load the service index, it really needs to show why, as the current message is not actionable (and I'm disappointed that normal or verbose verbosity doesn't show the details).

[cbaal83]

@zivkan

christian@christian-desktop:~/dev/csharp/test$ echo $NUGET_SHOW_STACK
true
christian@christian-desktop:~/dev/csharp/test$ dotnet restore -v:n
Build started 2/10/2023 8:00:11 PM.
     1>Project "/home/christian/dev/csharp/test/test.csproj" on node 1 (Restore target(s)).
     1>_GetAllRestoreProjectPathItems:
         Determining projects to restore...
       Restore:
         X.509 certificate chain validation will use the fallback certificate bundle at '/usr/share/dotnet/sdk/7.0.300-preview.23108.17/trustedroots/codesignctl.pem'.
         X.509 certificate chain validation will use the fallback certificate bundle at '/usr/share/dotnet/sdk/7.0.300-preview.23108.17/trustedroots/timestampctl.pem'.
         Restoring packages for /home/christian/dev/csharp/test/test.csproj...
     1>/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
     1>/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
     1>/home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
         Writing assets file to disk. Path: /home/christian/dev/csharp/test/obj/project.assets.json
         Failed to restore /home/christian/dev/csharp/test/test.csproj (in 114 ms).
         
         NuGet Config files used:
             /home/christian/.nuget/NuGet/NuGet.Config
         
         Feeds used:
             https://api.nuget.org/v3/index.json
     1>Done Building Project "/home/christian/dev/csharp/test/test.csproj" (Restore target(s)) -- FAILED.

Build FAILED.

       "/home/christian/dev/csharp/test/test.csproj" (Restore target) (1) ->
       (Restore target) -> 
         /home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
         /home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.
         /home/christian/dev/csharp/test/test.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.

    0 Warning(s)
    3 Error(s)

Time Elapsed 00:00:00.54

I can't see any difference. Have i dont something wrong? :D

Best,
//CB

[zivkan]

Have i dont something wrong?

No, I don't know why the error message isn't reporting any additional information. The problem getting NuGet updates in the .NET SDK 7.0.3xx builds has been resolved, so if you try a new nightly build of the SDK, hopefully it will work. If you continue to have problems, then it's almost certainly unrelated to the customer kernel, so needs to be handled in a different issue.

[cbaal83]

Ha, i just installed dotnet-sdk-7.0.300-preview.23112.4-x64.deb.
This is the result:

christian@christian-desktop:~/dev/csharp/test$ dotnet restore -v:n

Welcome to .NET 7.0!
---------------------
SDK Version: 7.0.300-preview.23112.4

Telemetry
---------
The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only).
Learn about HTTPS: https://aka.ms/dotnet-https
----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
--------------------------------------------------------------------------------------
Build started 2/13/2023 9:40:18 AM.
     1>Project "/home/christian/dev/csharp/test/test.csproj" on node 1 (Restore target(s)).
     1>_GetAllRestoreProjectPathItems:
         Determining projects to restore...
       Restore:
         X.509 certificate chain validation will use the fallback certificate bundle at '/usr/share/dotnet/sdk/7.0.300-preview.23112.4/trustedroots/codesignctl.pem'.
         X.509 certificate chain validation will use the fallback certificate bundle at '/usr/share/dotnet/sdk/7.0.300-preview.23112.4/trustedroots/timestampctl.pem'.
         Restoring packages for /home/christian/dev/csharp/test/test.csproj...
           GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.host.linux-x64/index.json
           GET https://api.nuget.org/v3-flatcontainer/microsoft.aspnetcore.app.ref/index.json
           GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.ref/index.json
           OK https://api.nuget.org/v3-flatcontainer/microsoft.aspnetcore.app.ref/index.json 300ms
           OK https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.host.linux-x64/index.json 468ms
           OK https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.ref/index.json 646ms
           GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.host.linux-x64/7.0.0/microsoft.netcore.app.host.linux-x64.7.0.0.nupkg
           GET https://api.nuget.org/v3-flatcontainer/microsoft.aspnetcore.app.ref/7.0.0/microsoft.aspnetcore.app.ref.7.0.0.nupkg
           GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.ref/7.0.0/microsoft.netcore.app.ref.7.0.0.nupkg
           OK https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.host.linux-x64/7.0.0/microsoft.netcore.app.host.linux-x64.7.0.0.nupkg 42ms
           OK https://api.nuget.org/v3-flatcontainer/microsoft.aspnetcore.app.ref/7.0.0/microsoft.aspnetcore.app.ref.7.0.0.nupkg 42ms
           OK https://api.nuget.org/v3-flatcontainer/microsoft.netcore.app.ref/7.0.0/microsoft.netcore.app.ref.7.0.0.nupkg 45ms
         Installed Microsoft.NETCore.App.Host.linux-x64 7.0.0 from https://api.nuget.org/v3/index.json with content hash rUEzak/A4xsZEgyMFzHLhkFIm9RAkm2wxqCCx0bgAysgaevUPx/cSjH2BiZkhGV0D+hKJfea/nL9Z7oHfzP8Aw==.
         Installed Microsoft.AspNetCore.App.Ref 7.0.0 from https://api.nuget.org/v3/index.json with content hash 0MeX3St0WJvIJ42rG19xL9B+NHzxYalgM7/ujZ6dj9JybTOUfA+mf+h1kQQK5XiWqQI9a92iJgbPUGi+cgruig==.
         Installed Microsoft.NETCore.App.Ref 7.0.0 from https://api.nuget.org/v3/index.json with content hash CrX2bAyttzhfwaNRZ/swlW5PBVxW7IK2V6GHPj6LpWiYIhSxR/OoMcFNRIRRT7TefARlb5dciQ4nh28J8LcKmg==.
         Generating MSBuild file /home/christian/dev/csharp/test/obj/test.csproj.nuget.g.props.
         Writing assets file to disk. Path: /home/christian/dev/csharp/test/obj/project.assets.json
         Restored /home/christian/dev/csharp/test/test.csproj (in 4.49 sec).
         
         NuGet Config files used:
             /home/christian/.nuget/NuGet/NuGet.Config
         
         Feeds used:
             https://api.nuget.org/v3/index.json
     1>Done Building Project "/home/christian/dev/csharp/test/test.csproj" (Restore target(s)).

Build succeeded.

Looks like it's finally working again!!

Thank you very much.
Best,
//CB