NuGet cannot restore from HTTPS sources that have SSL certificate problems

[joelverhagen]

Idea

We should add a way for users to bypass certain HTTPS certificate errors. If the user has a source that does not have a valid certificate chain, they should still have some way of getting NuGet to interact with this source.

"Inspirations"

npm and yarn have this:

npm config set strict-ssl false
yarn config set strict-ssl false

Maven has some extra knobs and switches:

-Dmaven.wagon.http.ssl.allowall=true
-Dmaven.wagon.http.ssl.insecure=true
-Dmaven.wagon.http.ssl.ignore.validity.dates=true

PHP's Composer

{
    "config": {
        "disable-tls": true,
        "secure-http": false
    }
}

curl has it:

--no-check-certificate

Specific Scenario

nuget.org server side runs functional tests against the staging slot of DEV, INT, and PROD that use nuget.exe. The DNS for this staging slot is some random blah.cloudapp.net. The certificate used is associated with the production DNS (e.g. *.nuget.org). This means nuget.exe fails SSL checks when running against this staging slot.

Proposal

Add a NuGet.config trustedCertificates section with standard <add> and <remove> tags. The value should be the SHA1 fingerprint (a.k.a. thumbprint) of the certificate. This allows developers to specifically opt-in into the tenuous state of SSL validation failures. The risk is mitigated since the developer must opt into it and even then only on a case-by-case basic.

Any certificate added to the trustedCertificates set is excluded from SSL validations -- I am thinking ALL validations exposed to use via the .NET API (untrusted root, name mistmatch, etc).

To improve discovery, we can add a log which writes out the fingerprint of the certificate that failed SSL validation at the error level.

We should also consider adding an environment variable to set this value. I believe npm has this and this sibling environment variable would match other NuGet.config keys, such as the location of the user packages directory.

Implementation

HttpClientHandler in System.Net.Http 4.3.0 (maybe earlier) has a way to set the certification validation check. This allows an app to configure the certificate validation on a per-request basis, instead of a per-AppDomain basis previously provided by ServicePointManager static properties.

With this approach, we should be able to implement the settings for all of our entry points: nuget.exe, .NET CLI, and (most tricky) Visual Studio.

The only possible problem is TFM compatibility with the System.Net.Http package. This requires investigation.

Issues

• Add a disableTLSCertificateVerification property #12996
• Disable TLS certificate validation when disableTLSCertificateValidation is set in the config file. #13023

[scottbommarito]

Is there any danger for us to add an "allow all SSL certs" configuration much like the other package managers? I understand that it is a risky operation that somebody shouldn't do, but given that so many others do it, I could imagine somebody being frustrated that we don't allow the functionality.

[joelverhagen]

Good question. I would rather be reactive to that change based on community requirements. This feature request is already driving by an "external" requirement (server team). I think a broader option is unnecessary and even undesired for us. If a customer wants that broader SSL validation opt-out, they should speak up 😄.

[gavinbeatty]

For the problem of corporate firewalls which MITM traffic with their own CA, a solution other package managers/apps use is to add a config option to use a specific CA bundle, or respect the environment variables SSL_CERT_DIR or SSL_CERT_FILE. Examples:

• ~/.curlc: cacert = /path/to/ca
• ~/.wgetrc: ca_certificate=/path/to/ca
• ~/.pip/pip.conf:

  [global]
  cert = /path/to/ca
  

• rustup: respects SSL_CERT_FILE
• git: http.sslCAInfo

I think this suggestion might be orthogonal to yours, @joelverhagen. I don't know how common it is for certs to be broken -- in my case, I just want NuGet to recognize the unknown cert that is valid.

[joelverhagen]

@gavinbeatty wouldn't the Windows-y solution here be to install the corporate CA's certificate as a trusted root on that machine?

[KyleWiering]

@joelverhagen Correct, that is the way to remedy the issue of a self signed cert not being recognized. as @gavinbeatty suggests, most other utilities that connect through ssl provide an optional path to a cert bundle - as they do not know how to reference the Windows Certificate store.

• Newer composer uses PHP.ini openssl.cafile=path/to/cafile

The suggestion really is an approach to getting stuff working quickly in a local environment - and bringing nuget more in-line with other package managers. The downside to such ability, as @scottbommarito suggests, is that it can be used to turn off TLS inappropriately.

Not being able to temporarily ignore a cert signature is a frustration. Admittedly, I come from a more open source / composer / npm background - and internal proxy package hosts with self signed certs used congruously with the approved external package hosts is commonplace.

[pcewing]

Just a note, I came across this issue after trying to restore packages with NuGet on Ubuntu via the Rider IDE. It failed despite having trusted certificates installed via the ca-certificates package. Having the option to either bypass SSL or specify trusted certs via environment variables would be great!

[joelverhagen]

@pcewing, what would the ideal configuration be for you? I see three options so far:

1. Allow (trust) all SSL certificates. This seems like @scottbommarito's suggestion.

2. Allow a "cafile" option where you can point at a file containing one or more trusted certs. Presumably these are trusted roots not just specific certificates. This seems to be @gavinbeatty and @KyleWiering's suggestion.

   1. Maybe supporting well known environment variables SSL_CERT_DIR and SSL_CERT_FILE.

3. Allow trusting of specific certificate fingerprints (thumbprints) via NuGet config. This does not imply trusting a root just trusting specific SSL certificates. This is my suggestion.

I think one or more of these could be supported at the same time. I want to get a sense of what covers different people's scenarios. Note although I am on the NuGet team, this feature is by no means a commitment -- we're in the brainstorming phase here 😄!

[pcewing]

@joelverhagen I think a combination of 1 with something like 2 or 3 would be great. Another package manager I frequently use for Erlang/Elixir is Hex, and they added support for a HEX_UNSAFE_HTTPS environment variable that can be set when SSL should be bypassed altogether.

Personally I think option 2 sounds better as setting environment variables is already a standard step when configuring build/CI machines and it's easy for an IT department to do globally on developer workstations. Option 3 seems more useful in the case where a repository specifically depends on trusted certs, so every developer who clones that repo automatically gets the Nuget.config pointing to them.

Either way, just having a mechanism would be nice. With .NET core being adopted and development becoming more common on other operating systems, it will probably become a more frequent request. And don't worry, I totally understand! :)

[gavinbeatty]

There are many solutions that might work:

• if the software looks in the Windows cert store, install the cert if you have permissions.
• if that doesn't work out (unfortunately it is quite common for non-Microsoft open source software to not look in the Windows cert store at all):
  • ignore all cert checks (option 1).
  • ignore specific cert checks (option 3).
  • specify an alternative cert (option 2).

I'm assuming NuGet does the right thing and looks in the Windows cert store, but I still slightly prefer option 2, the alternative cert file/dir (my suggestion).

• From memory, I think there are "personal" and "system" cert stores in Windows and it's not obvious which to choose.
• No permissions required.
• Actually checks certs.
• Everybody else is doing it. I think it's likely that people other than me have already been searching Stack Overflow for "how to specify ssl cert for nuget", etc., simply because that's the goto solution in other domains. It would be nice if the answer was a straightforward "we have that in nuget too".

[foresightyoue]

@joelverhagen I have the nuget problem by CA certificates as below,could it be fixed by engored the CA certificates?if so,how to set the nuget config with egnored the CA certificates?Thanks.

user@a1adaf3:/projects$ dotnet restore
log : Restoring packages for /projects/CSharpDemo/project.json...
error: Unable to load the service index for source https://api.nuget.org/v3/index.json.
error: An error occurred while sending the request.
error: Peer certificate cannot be authenticated with given CA certificates

[joelverhagen]

@hwx405562, today NuGet only supports using the built-in set of CA certificates. This behavior is defined by the implementation of .NET Core's HTTP stack. On Windows, this is the system's trusted CAs from the Windows certificate store. I am not sure about the specifics of macOS and Linux.

This issue is mainly focusing on designing a new feature. Could you file a new issue with your question, including OS version, .NET CLI version, repro steps, etc? We can focus on your particular issue on more depth with a seperate GitHub issue.
https://github.com/NuGet/Home/issues/new

[gavinbeatty]

It sounds like @pcewing and I would both be very pleased with environment variables, but @joelverhagen, what are your thoughts?

[anaselhajjaji]

In my side, I forked the nuget and modified the source code to accept all the certificates (it's for internal use in company).
https://github.com/anaselhajjaji/nuget/tree/db4c607f1b326074da583e9a13bae47932d963f4/nuget-binary

[zhili1208]

close as dupe of #5773

[emmellee]

@zhili1208, @joelverhagen, @rrelyea, could I persuade you and your nuget team members to take a renewed interest in support for ssl certificates? My organization wishes to create nuget packages for release to our internal feed provided by our TFS Server. The DoD relies heavily on the use of smartcards and CA certs and, like any DoD webserver, our TFS Server requires mutual ssl authentication. Unfortunately, we are unable to get past this and duplicate issue #5773. Any help your team can provide towards resolving this issue would be greatly appreciated. The TFS/VSTS teams have offered their resources towards resolving their piece of the puzzle (reference issue #5773). Thank you.

[marco-carvalho]

right now I can't develop with dotnet 6/7 using docker inside my organization because of this
really wish there was a way to disable it like npm/yarn

edit: in my company the problem was kaspersky, we put "*.nuget.org" in the whitelist and it's working

[Octarines]

I want to throw my hat on the pile for this. Working in a corporate environment using zScaler (which uses MITM certs) and cannot locally run a docker build because the solution utilises a custom nuget feed.

[marco-carvalho]

in my company the problem was kaspersky, we put "*.nuget.org" in the whitelist and it's working

[karpikpl]

I want to throw my hat on the pile for this. Working in a corporate environment using zScaler (which uses MITM certs) and cannot locally run a docker build because the solution utilises a custom nuget feed.

You can add this to Docker layer that executes dotnet restore

RUN openssl s_client -showcerts -connect nuget.org:443 </dev/null | sed -n -e '/-.BEGIN/,/-.END/ p' > /usr/local/share/ca-certificates/ZscalerRootCertificate-2048-SHA256.crt && \
	update-ca-certificates

[Octarines]

I want to throw my hat on the pile for this. Working in a corporate environment using zScaler (which uses MITM certs) and cannot locally run a docker build because the solution utilises a custom nuget feed.

You can add this to Docker layer that executes dotnet restore

RUN openssl s_client -showcerts -connect nuget.org:443 </dev/null | sed -n -e '/-.BEGIN/,/-.END/ p' > /usr/local/share/ca-certificates/ZscalerRootCertificate-2048-SHA256.crt && \
	update-ca-certificates

I've tried this (as well as a number of similar approaches) to no avail.

[hvojdani]

I think nuget cli needs an option to set certificate for each source, so people can deploy their nuget servers by self signed cert.
on push or get if server cert was different from what user specified cert, throw an error. its logical and secure way that most of developers are ok with that.

[noeltupas]

I want to throw my hat on the pile for this. Working in a corporate environment using zScaler (which uses MITM certs) and cannot locally run a docker build because the solution utilises a custom nuget feed.

You can add this to Docker layer that executes dotnet restore

RUN openssl s_client -showcerts -connect nuget.org:443 </dev/null | sed -n -e '/-.BEGIN/,/-.END/ p' > /usr/local/share/ca-certificates/ZscalerRootCertificate-2048-SHA256.crt && \
	update-ca-certificates

I've tried this (as well as a number of similar approaches) to no avail.

I tried our cert for bypassing Zscaler. No luck either.

[Octarines]

I want to throw my hat on the pile for this. Working in a corporate environment using zScaler (which uses MITM certs) and cannot locally run a docker build because the solution utilises a custom nuget feed.

I finally found my solution and it ended up being a combination of a lot of things. The last hurdle was because my corporate certificate was in der format rather than pem (which seems to be needed for the debian base which is used by the dotnet/sdk image). The use of the crt suffix seems important to debian as well as my original certificate has the cer suffix. The code I added to the DOCKERFILE before the restore step is as follows:

ADD "{my-root-ca.cer}" "/temp/my-root-ca.crt"  
RUN openssl x509 -inform der -in "/temp/my-root-ca.crt" -out "/usr/local/share/ca-certificates/my-root-ca.crt"  
RUN update-ca-certificates

Where {my-root-ca.cer} is the path and filename of your corporate root CA cert.
In my case I am using ADD rather than COPY because I neatened things up to use the certificate from a url webstore but it works just the same with a local copy of the cert.

FYI @noeltupas

[JonDouglas]

Hi friends,

I've added an initial proposal that includes functionality to disable/ignore HTTPS certificate validation. Please feel free to read through it and provide your feedback & upvote if you believe it is headed in the right direction.

#12542

[KarenArzumanyan]

This would be a much needed option.
Especially important for CI/CD
For example, when using docker-compose build inside dron-agent (docker into docker).

[robindegen]

With the plans to drop http, I feel it's extra important to allow validation skipping. We have internal facing servers and I don't want to pay for proper certificates, and if I'm going to self sign I might as well just disable the check since all security is out the window already. Perhaps dropping http should be reconsidered. Not everything is internet facing. Perhaps simply support and allow http from known internal ips 192.168, 10.x etc. Enforcing https in this case is a major hassle with no security benefits.

[Kalyxt]

Just run into this issue, I would like to host my own packages on gitea with self-signed cert but there is no workaround currently or I overlooked something ?

[Nigusu-Allehu]

We have added disableTLSCertificateValidation attribute that can be added to a nuget source in a NuGet config file which would allow access to sources with an invalid TLS certificate as follows

<!-- Disables certification validation on a specific https source -->
<packageSources>
    <add key="Contoso" value="https://contoso.com/packages/" disableTLSCertificateValidation="true" />
</packageSources>

[WeihanLi]

@Nigusu-Allehu which NuGet client version or dotnet sdk version could we expect? seemed not available now

[Nigusu-Allehu]

NuGet 6.11 and dotnet 8.0.400

[tmds]

disableTLSCertificateValidation="true"

On Linux (your machine/CI/...), a secure alternative is getting the CA certificate in a directory and setting the SSL_CERT_DIR envvar to include the system certificates and your CA directory, something like: SSL_CERT_DIR=/etc/ssl/certs:/dir_with_your_ca_cert.