Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL RemoteCertificateNameMismatch on MacOS Catalina #666

Closed
TingluoHuang opened this issue Dec 7, 2019 · 16 comments
Closed

SSL RemoteCertificateNameMismatch on MacOS Catalina #666

TingluoHuang opened this issue Dec 7, 2019 · 16 comments

Comments

@TingluoHuang
Copy link

I am using Proxyman to decrypt https traffic from my self-contained netcore 3.0 console app on macOS. Like using Fiddler on Windows.

I am getting System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. with inner exception System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure when the proxy is on.

I added a customize ServerCertificateCustomValidationCallback to dump out the ssl error, request url and the certificate.

sslPolicyErrors: RemoteCertificateNameMismatch
HttpRequestUri: https://pipelines.actions.githubusercontent.com/WoxlUxJHrKEzIp4Nz3YmrmLlZBonrmj9xCJ1lrzcJ9ZsD1Tnw7/_apis/connectionData?connectOptions=1&lastChangeId=-1&lastChangeId64=-1
Certificate:
[Version]
  V3

[Subject]
  OU=https://proxyman.io, CN=*.actions.githubusercontent.com, O="GitHub, Inc.", L=San Francisco, C=US
  Simple Name: *.actions.githubusercontent.com
  DNS Name: *.actions.githubusercontent.com

[Issuer]
  OU=https://proxyman.io, CN="Proxyman CA (1 Nov 2019, htl-mac.local)", O=Proxyman Ltd, L=Singapore, C=SG
  Simple Name: Proxyman CA (1 Nov 2019, htl-mac.local)
  DNS Name: Proxyman CA (1 Nov 2019, htl-mac.local)

[Serial Number]
  00EE6265BFC8F6A251

[Not Before]
  11/1/2019 2:15:21 PM

[Not After]
  2/3/2022 1:15:21 PM

[Thumbprint]
  D1657538605625A0D41B7195CCF80806682374DA

[Signature Algorithm]
  sha256RSA(1.2.840.113549.1.1.11)

[Public Key]
  Algorithm: RSA
  Length: 2048
  Key Blob: 3082010A0282010100E2788EC7C9FF6705B22295EB4782FAE49DC259B7A0C6C80A655BF6AC330E4449BEC11D2835AF4D2C855D39B0E0472A72500B38AA44D64F6250581DBCC937153306976B00EBAAEA864C08D5F4CAC13D445DF5FE2F18DC528FA01B8C82CA85899E1064DED6905A867032C38CE18B6B5E01870533C5B53F56F6D081D3FBADC2FC8DA5843FBC4D6E1146A7E0E2791802167EF09D890288E13876B8C0FDDE8FAF3719981745BB983C6393A569CF768146DB3F0E8264D5E0084EAD2471D6131E9C5FFC6DE0C53FE7566323746716809D4245C3B585220B9B3378418131BBE92DFAE8CFB511AF7CCEA12312A80F5BAB5F52FF39C76D4B5EA7BCAC6215DEDD7F935EB8D90203010001
  Parameters: 0500

[Extensions]
* X509v3 Key Usage(2.5.29.15):
  030204F0
* (2.5.29.17):
  DNS:*.actions.githubusercontent.com, DNS:actions.githubusercontent.com

The request url's authority seems match with the cert's CN.
If i export DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER=0 to force my app use the old curl http handler, the SSL error goes away.
So I am not sure why I am getting SSL error on when use SocketHttpHandler.

@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added area-System.Net.Http untriaged New issue has not been triaged by the area owner labels Dec 7, 2019
@davidsh davidsh added the os-mac-os-x macOS aka OSX label Dec 7, 2019
@davidsh
Copy link
Contributor

davidsh commented Dec 7, 2019

@wfurt

@wfurt
Copy link
Member

wfurt commented Dec 8, 2019

Do you use environment variables or do you set proxy explicitly in via API? Short code fragment would be nice for clarity how to reproduce it.

@wfurt
Copy link
Member

wfurt commented Dec 8, 2019

And do you have LANG=en_US.UTF-8 or do you have locale set to anything else?

@TingluoHuang
Copy link
Author

@wfurt

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
  </PropertyGroup>

</Project>
using System;
using System.Net;
using System.Net.Http;
using System.Net.Security;

namespace proxytest
{
    public class WebProxy : IWebProxy
    {
        public ICredentials Credentials { get; set; }

        public Uri GetProxy(Uri destination)
        {
            return new Uri("http://127.0.0.1:9090");
        }

        public bool IsBypassed(Uri host)
        {
            return false;
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");


            using (HttpClientHandler clientHandler = new HttpClientHandler())
            {
                clientHandler.Proxy = new WebProxy();
                clientHandler.ServerCertificateCustomValidationCallback = (sender, certificate, chain, sslPolicyErrors) =>
                {
                    if (sslPolicyErrors == SslPolicyErrors.None)
                    {
                        return true;
                    }
                    else
                    {
                        Console.WriteLine($"SSL ERROR:{sslPolicyErrors.ToString()}");
                        Console.WriteLine(sender.RequestUri.AbsoluteUri);
                        Console.WriteLine(certificate.ToString(true));

                        return false;
                    }
                };
                using (var httpClient = new HttpClient(clientHandler))
                {
                    var result = httpClient.GetStringAsync("https://github.com").GetAwaiter().GetResult();
                    Console.WriteLine(result);
                }
            }
        }
    }
}
ting@htl-mac proxytest % printenv|sort
HOME=/Users/ting
LANG=en_US.UTF-8
LOGNAME=ting
LaunchInstanceID=42DF0CC6-7E63-405A-B233-8A6E7C29719A
OLDPWD=/Users/ting/Desktop
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/usr/local/share/dotnet:~/.dotnet/tools
PWD=/Users/ting/Desktop/proxytest
SECURITYSESSIONID=186a6
SHELL=/bin/zsh
SHLVL=1
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.CEawhAXvQp/Listeners
TERM=xterm-256color
TERM_PROGRAM=Apple_Terminal
TERM_PROGRAM_VERSION=433
TERM_SESSION_ID=A8A7F4C6-E8A2-4C32-9954-A6D7295908A4
TMPDIR=/var/folders/h1/nkrscyxx6858j3wz7dvzcq5r0000gn/T/
USER=ting
XPC_FLAGS=0x0
XPC_SERVICE_NAME=0
_=/usr/bin/printenv
ting@htl-mac proxytest % dotnet run                                            
SSL ERROR:RemoteCertificateNameMismatch
https://github.com/
[Version]
  V3

[Subject]
  OU=https://proxyman.io, CN=github.com, O="GitHub, Inc.", L=San Francisco, C=US
  Simple Name: github.com
  DNS Name: github.com

[Issuer]
  OU=https://proxyman.io, CN="Proxyman CA (1 Nov 2019, htl-mac.local)", O=Proxyman Ltd, L=Singapore, C=SG
  Simple Name: Proxyman CA (1 Nov 2019, htl-mac.local)
  DNS Name: Proxyman CA (1 Nov 2019, htl-mac.local)

[Serial Number]
  00EE6265BFC8F6A254

[Not Before]
  12/8/2019 9:26:39 AM

[Not After]
  3/12/2022 9:26:39 AM

[Thumbprint]
  70396C36BA73798B784EA2941BF05BB230634EF7

[Signature Algorithm]
  sha256RSA(1.2.840.113549.1.1.11)

[Public Key]
  Algorithm: RSA
  Length: 2048
  Key Blob: 3082010A0282010100FD073FCE3DF3A256AF6AAD4DBDA5F21672355C386C408AC7FABE8AC1BA86D950DC414102CE24718C45F5971DDB62EEF5421D165B449832B3DF5A23626307FB258123114C8F96B367D7420FF4C90CDEFFBD42B0F4F3DE7D0B3123960F26DDD93407FC527360FDD25009B0277EDDB69DF3D5ACDE0706ABAA95E8E13C135F3B5137FD61762BC2CFCB319A96FD15E221BAED3C9610E078E8CEA89AE6627A5A455B78FE2CED65DFC32EC382BBBD6D75425BF59C7E3CE6D7503F2AFBE327C6A07B08EAD125CE318B7A8709ECC6CED2D79311794338E6A55F95E5F6CBAEEBFE801559655E40AD3ACD003E5FD7DD698BB2C01F386D287DF24C4143E94EDC34F52219CC110203010001
  Parameters: 0500

[Extensions]
* X509v3 Key Usage(2.5.29.15):
  030204F0
* (2.5.29.17):
  DNS:github.com, DNS:www.github.com

Unhandled exception. System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.

@wfurt
Copy link
Member

wfurt commented Dec 9, 2019

I was not able to reproduce it @TingluoHuang
I did run Wireshark to make sure request goes trough Proxman:

CONNECT github.com:443 HTTP/1.1
Host: github.com:443

HTTP/1.1 200 Connection Established

...........]...*...q..1..._.......B......S...,...,.+.$.#.
.	...0./.(.'...........=.<.5./.
...L.....
..
github.com.
.................
.........................................Y...U..]....mf<%t5uuk.....x......<...[. {1..[..Q...[......U.......*&.R...0..
............................0...0.........	..A...J.o0
.	*.H..
.....0..1.0	..U....SG1.0...U...	Singapore1.0...U.
..Proxyman Ltd1/0-..U...&Proxyman CA (9 Dec 2019, macik2.local)1.0...U....https://proxyman.io0..
191209194709Z.
220313194709Z0o1.0	..U....US1.0...U...
San Francisco1.0...U.
..GitHub, Inc.1.0...U...
github.com1.0...U....https://proxyman.io0.."0
.	*.H..
..........0..

and I do see decrypted response details in Proxyman GUI.
tested on 10.14.6

cc: @bartonjs in case he has some ideas.

@bartonjs
Copy link
Member

bartonjs commented Dec 9, 2019

We let the OS determine whether or not the hostname matches (https://github.com/dotnet/corefx/blob/release/3.0/src/Native/Unix/System.Security.Cryptography.Native.Apple/pal_ssl.c#L357-L462).

Maybe something else is going on that's making things return false, but essentially we and Safari should have the same answer.

@davidsh davidsh added needs more info and removed untriaged New issue has not been triaged by the area owner labels Dec 9, 2019
@wfurt
Copy link
Member

wfurt commented Dec 10, 2019

On that note, Proxyman changes OS setting so it should be easy to test.

@TingluoHuang
Copy link
Author

I tried Safari, it works fine with Proxyman, SSL traffic in Safari get decrypted correctly.
Any other information I can collect that would help investigation? BTW I am on 10.15.1

@wfurt
Copy link
Member

wfurt commented Dec 10, 2019

I can reproduce it on Catalina. I will take a look.

@wfurt
Copy link
Member

wfurt commented Dec 11, 2019

It looks like we get ksectrustresultrecoverabletrustfailure:
https://developer.apple.com/documentation/security/sectrustresulttype/ksectrustresultrecoverabletrustfailure?language=objc
The CA certificate does not have OSCP or CRL distribution list. It should not matter as we set revocation to NoCheck. But it seems like something changes at OS crypto layer.
As @bartonjs mentioned, it is OS telling us not to trust that certificate. The name mismatch may be artifact of mapping OS errors.

@TingluoHuang
Copy link
Author

Since Safari works fine with Proxyman, i assume there might be some different between Safari and Netcore ask OS to validate SSL cert, maybe some validation option difference.

@wfurt
Copy link
Member

wfurt commented Dec 13, 2019

I'm still not sure what exactly is going on as Catalina sources are not available yet.
Internally this fails with -2147408889 CSSMERR_APPLETP_INVALID_EXTENDED_KEY_USAGE (Invalid Extended Key Usage for policy). It should not matter as the certificate generated by Proxyman does not have any EKU.

@wfurt wfurt changed the title SSL RemoteCertificateNameMismatch when use sockethttphandler in netcore 3.0 SSL RemoteCertificateNameMismatch on MacOS Catalina Dec 13, 2019
@vcsjones
Copy link
Member

@wfurt

I think perhaps Apple started requiring the EKU as per the CAB/F baseline requirements in 10.15. Apple has a support article on this here: https://support.apple.com/en-us/HT210176

Additionally, all TLS server certificates issued after July 1, 2019 (as indicated in the NotBefore field of the certificate) must follow these guidelines:

TLS server certificates must contain an ExtendedKeyUsage (EKU) extension containing the id-kp-serverAuth OID.

@TingluoHuang
Copy link
Author

Proxyman fixed their CA cert and server certs when decrypt https traffic.
This issue can be resolved now.
Thanks @wfurt

@bartonjs
Copy link
Member

bartonjs commented Jan 4, 2020

Glad things are working for you now 😄.

@bartonjs bartonjs closed this as completed Jan 4, 2020
@danmoseley
Copy link
Member

Was there more info we could have included in the exception that might have made this quicker to understand (like CSSMERR_APPLETP_INVALID_EXTENDED_KEY_USAGE)?

@karelz karelz added this to the 5.0 milestone Feb 19, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

8 participants