Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't depend on Windows' SSL implementation #290

Closed
ice6 opened this issue May 14, 2019 · 16 comments
Closed

Don't depend on Windows' SSL implementation #290

ice6 opened this issue May 14, 2019 · 16 comments
Labels
area:http Issue with HttpListener and related types enhancement wontfix
Milestone

Comments

@ice6
Copy link

ice6 commented May 14, 2019

Is your feature request related to a problem? Please describe.
I'm always frustrated when I want to expose https, netsh is really a mess. I embed a golang reverse proxy server to resolve this problem in production.

Describe the solution you'd like
Is there a better way to do this in CSharp?

@rdeago rdeago added this to the x.0.0 milestone May 15, 2019
@rdeago
Copy link
Collaborator

rdeago commented May 15, 2019

Unfortunately there seems to be no way around it. Not unless we stop using Microsoft's and Mono's HttpListeners. Maybe not even then.

Even µHttpSharp, that handles the HTTP protocol on its own, relies on SslStream to handle encryption over TCP. Not a problem for them, since apparently it is a Windows-only library.

.NET Core has a fully managed implementation of HttpListener and related classes. It normally gets compiled only for other operating systems, but there's a build option to compile it for Windows too, so maybe we could use that instead (or in addition to) current HttpListener options.

From a quick look, the .NET Core managed implementation seems to use OpenSSL. This brings up interesting questions about dependencies and deployment. I think a serious discussion about this is better left for after the release of EmbedIO v3.

@geoperez
Copy link
Member

We can check also how Kestrel is handling this.

@rdeago
Copy link
Collaborator

rdeago commented May 15, 2019

You're right @geoperez, I forgot to mention Kestrel!

But alas, it uses SslStream too.

@rdeago rdeago changed the title Is there a better and just work solution for TLS? Don't depend on Windows' SSL implementation May 15, 2019
@geoperez
Copy link
Member

But how is Kestrel binding to the address:port. I only see how it loads the certificate.

BTW this is an interesting way to load a default cert:

https://github.com/aspnet/AspNetCore/blob/e4fbd598b56c5582335b427dedfb68495d1b8489/src/Servers/Kestrel/Core/src/KestrelServerOptions.cs#L145

@rdeago
Copy link
Collaborator

rdeago commented May 16, 2019

The topic of address / port binding deserves a conversation of its own, but in short Kestrel does not use HttpListener; instead, it is meant to replace HttpListener entirely.

Besides, no matter how we load a certificate, SslStream under Windows uses SSPI. Therefore, Kestrel applications need NETSH just like us mere mortals. 😄

Maybe, and that's a big maybe, we could try to build the Unix implementation of SslStream, which uses OpenSSL, and see if it can work under other operating systems. That's apt to be both fun and painful IMHO.

@rdeago
Copy link
Collaborator

rdeago commented Jun 5, 2019

For future reference: a managed wrapper for OpenSSL exists. It requires the unmanaged OpenSSL libraries, of course, and it seems to be for .NET 2.0, but it may be a starting point.

@rdeago
Copy link
Collaborator

rdeago commented Jun 27, 2019

Mono also has managed TLS support using Google's BoringSSL. The latter is, in Google's own words, an OpenSSL fork "not intended for general use". I guess there has to be a reason why Mono uses it, other than thinking different without being Apple. Anyway, one more bit of food for thought, and one more pile of third-party code to look into as soon as we have both the time and courage.

@Genteure
Copy link

Besides, no matter how we load a certificate, SslStream under Windows uses SSPI. Therefore, Kestrel applications need NETSH just like us mere mortals. 😄

I don't think so? Isn't SslStream itself is sufficient to provide SSL/TLS encryption, at any platform, with the possibility to do SNI and client certificate?

The GitHub search link shows only three result (at least as of today, GitHub only search on master branch), and they have nothing to do with Kestrel and SSL.
image


µHttpSharp

is a .NET Framework only project.
HTTPS implemented using SslStream.

System.Net.HttpListener

Both .NET Framework and .NET Core use httpapi.dll/http.h on Windows.
HTTPS implemented by httpapi.dll, certificate is configured by netsh http.

On .NET Core it use a managed implementation, HTTPS implemented by SslStream. It doesn't support SNI despite SslStream can do it.

Reference:
.NET Framework (From ReferenceSource)
image
.NET Core Windows
.NET Core Other and call to SslStream

Kestrel

Kestrel is using SslStream, and it support SNI.

netsh

Netsh commands for HTTP writes:

You can use the Netsh commands for HTTP context to query and configure HTTP.sys settings and parameters.

I don't think it have anything to do with SSPI / SslStream on Windows.

SslStream

It use OpenSSL on Unix, AppleCrypto on OSX, SSPI on Windows.
But that's .NET internal implementation, we don't need to care about that as long as it provide a uniform behavior.


EmbedIO currently do support HTTPS on systems other than Windows if using HttpListenerMode.EmbedIO, since it use SslStream.

// v2

new WebServer(new string[]{"https://example.com/"}, RoutingStrategy.Regex, new Unosquare.Net.HttpListener(new System.Security.Cryptography.X509Certificates.X509Certificate2("cert.pfx")))

I suggest we just use SslStream, it has all the feature we need, it's in .NET Standard, it doesn't need any native dependency (at least from our point of view).
As for System.Net.HttpListener, it can stay with netsh. Users of EmbedIO can decide whether they want use EmbedIO's internal server to have HTTPS anywhere, or use System.Net.HttpListener and have to deal with netsh commands. (Or we remove support for System.Net.HttpListener altogether, though I'm not sure if anyone have any reason must to use it.)

@Genteure
Copy link

Genteure commented Jun 27, 2019

During test I also find out currently if SSL handshake failed for some reason, a common reason is client doesn't trust server's certificate, entire WebServer crashes.

proof of concept code:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes
openssl pkcs12 -inkey key.pem -in cert.pem -export -out cert.pfx
// Program.cs
using System;
using System.Net;
using System.Threading.Tasks;
using Unosquare.Labs.EmbedIO;
using Unosquare.Labs.EmbedIO.Constants;
using Unosquare.Labs.EmbedIO.Modules;

namespace sslstreamtest
{
    class Program
    {
        static async Task Main(string[] args)
        {
            var hl = new Unosquare.Net.HttpListener(new System.Security.Cryptography.X509Certificates.X509Certificate2("cert.pfx"));
            var server = new WebServer(new string[]{"https://*:8899/"}, RoutingStrategy.Regex, hl);
            await server.RunAsync();
        }
    }
}

Request with curl without -k

curl https://localhost:8899/test

EmbedIO 2.9.1 on Ubuntu 18.04 LTS with .NET Core 2.1 throws:

Unhandled Exception: System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> Interop+OpenSsl+SslException: SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL. ---> Interop+Crypto+OpenSslCryptographicException: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
   --- End of inner exception stack trace ---
   at Interop.OpenSsl.DoSslHandshake(SafeSslHandle context, Byte[] recvBuf, Int32 recvOffset, Int32 recvCount, Byte[]& sendBuf, Int32& sendCount)
   at System.Net.Security.SslStreamPal.HandshakeInternal(SafeFreeCredentials credential, SafeDeleteContext& context, SecurityBuffer inputBuffer, SecurityBuffer outputBuffer, SslAuthenticationOptions sslAuthenticationOptions)
   --- End of inner exception stack trace ---
   at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
   at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslState.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Security.SslState.ThrowIfExceptional()
   at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsServer(IAsyncResult asyncResult)
   at System.Net.Security.SslStream.<>c.<AuthenticateAsServerAsync>b__48_1(IAsyncResult iar)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at Unosquare.Net.HttpConnection..ctor(Socket sock, EndPointListener epl, X509Certificate cert)
   at Unosquare.Net.EndPointListener.ProcessAccept(SocketAsyncEventArgs args)
   at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted(SocketAsyncEventArgs e)
   at System.Net.Sockets.SocketAsyncEventArgs.ExecutionCallback(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationAsyncSuccess(Int32 bytesTransferred, SocketFlags flags)
   at System.Net.Sockets.SocketAsyncEventArgs.CompletionCallback(Int32 bytesTransferred, SocketFlags flags, SocketError socketError)
   at System.Net.Sockets.SocketAsyncEventArgs.AcceptCompletionCallback(IntPtr acceptedFileDescriptor, Byte[] socketAddress, Int32 socketAddressSize, SocketError socketError)
   at System.Net.Sockets.SocketAsyncContext.AcceptOperation.InvokeCallback(Boolean allowPooling)
   at System.Net.Sockets.SocketAsyncContext.OperationQueue`1.ProcessAsyncOperation(TOperation op)
   at System.Net.Sockets.SocketAsyncContext.OperationQueue`1.<>c.<.cctor>b__18_0(Object op)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

Request with curl with -k

curl https://localhost:8899/test -kv
*   Trying ::1...
* TCP_NODELAY set
* connect to ::1 port 8899 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8899 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=Test EmbedIO Cert
*  start date: Jun 27 19:11:02 2019 GMT
*  expire date: Jun 26 19:11:02 2020 GMT
*  issuer: CN=Test EmbedIO Cert
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /test HTTP/1.1
> Host: localhost:8899
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Content-Type: text/html; charset=utf-8
< Server: embedio/2.0
< Date: Thu, 27 Jun 2019 19:26:23 GMT
< Content-Length: 63
< Keep-Alive: timeout=15,max=100
< 
* Connection #0 to host localhost left intact
<html><head></head><body><h1>404 - Not Found</h1></body></html>

@rdeago
Copy link
Collaborator

rdeago commented Jun 28, 2019

@Genteure the problem with SslStream is that it uses SSPI!

SSPI (which http.sys and httpapi.dll use under the hood, or implement themselves, I don't know and it's not relevant here) is why we have to use netsh, which is a major pain in the posterior (admin rights required, different command line according to Windows language...)

The idea was to try and recompile .NET COre's or Mono's TLS support for Linux to make it work on other systems (primarily Windows). Since OpenSSL has Windows binaries available (not sure about BoringSSL but I'd say yes, since it's used by Chrome) it should be possible to support secure connections without relying on SSPI.

@Genteure
Copy link

Can you write a example code to elaborate why SSPI would need to use netsh?
I can't find any result on Google with SSPI and netsh together that don't involve "httpapi" "httpsys" "Windows Web Services".

Here's my POC, it runs on .NET Core Windows, without netsh.
Tested with openssl as client: openssl s_client -connect 127.0.0.1:8877

static async Task Main(string[] args)
{
    var serverCert = new X509Certificate2("cert.pfx");

    var sock = new Socket(SocketType.Stream, ProtocolType.Tcp);
    sock.Bind(new IPEndPoint(IPAddress.Any, 8877));
    sock.Listen(8);

    while (true)
    {
        try
        {
            Socket accepted = await sock.AcceptAsync();
            var ns = new NetworkStream(accepted, true);
            var ss = new SslStream(ns);

            var options = new SslServerAuthenticationOptions
            {
                CertificateRevocationCheckMode = X509RevocationMode.NoCheck,
                ClientCertificateRequired = false,
                RemoteCertificateValidationCallback =
                (object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) =>
                {
                    // check client certificate
                    return true;
                },
                ServerCertificateSelectionCallback =
                (object sender, string hostName) =>
                {
                    // check hostName
                    return serverCert;
                }
            };

            await ss.AuthenticateAsServerAsync(options, default);

            ss.Close();
            ss.Dispose();

        }
        catch (Exception ex)
        {
            Console.WriteLine(ex);
        }
    }
}

@stale
Copy link

stale bot commented Aug 27, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Aug 27, 2019
@rdeago rdeago added area:http Issue with HttpListener and related types and removed wontfix labels Sep 2, 2019
@vhnatyk
Copy link

vhnatyk commented Oct 6, 2019

Sorry - seems quite obvious but nevertheless I would like to make it clear that as of today (october 2019) it doesn't support https on mobile platforms?

@geoperez
Copy link
Member

Nop, the support to HTTPS right now is only with Windows.

@TomCJones
Copy link

ssl is part of windows (kerberos) auth - it belongs to the security team not the network team. If you want to work beyond windows you will need to grab some other code - or perhaps put the code into platform specific modules.

@stale
Copy link

stale bot commented Dec 30, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Dec 30, 2019
@stale stale bot closed this as completed Jan 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:http Issue with HttpListener and related types enhancement wontfix
Projects
None yet
Development

No branches or pull requests

6 participants