Improvements to HTTPS

[danroth27]

Introduction

• HTTPS enforcement is becoming increasingly strict on the web
• GDPR requires use of HTTPS
• Developing with HTTPS can help prevent related issues in production
• Setting up HTTPS with Kestrel today requires substantial code and custom configuration

Scenarios

• Kestrel with HTTPS during development
• Kestrel with HTTPS as production edge server
• IIS Express with HTTPS during development
• IIS with HTTPS in production
• IIS with HTTPS in development
• Development with HTTPS inside of Docker
• In production with HTTPS inside of Docker

Goals

• Minimal/no configuration for development and production
• Single point for configuring HTTPS related features
• Redirect incomming HTTP requests to HTTPS
• Configure HTTP Strict Transport Security Protocol (HSTS)

Technical challenges

• Port and certificate discovery. (development/production) (edge server / behind proxy).
• Port and certificate configuration. (development/production) (edge server / behind proxy).
• Certificate management (mostly development).
• Certificate trust (different across platforms).

User experience

Kestrel with HTTPS during development

• New project created from command-line or VS setup by default for HTTPS
• Listens on both an HTTP and HTTPS address (as specified in launchSettings.json)
• Uses ASP.NET Core HTTPS development certificate by default
  • Certificate setup by .NET Core SDK first run experience or by VS when running the project
  • Command-line tool for manually creating and trusting the ASP.NET Core HTTPS development certificate
• Project setup to redirect to HTTPS (port specified using env var)
• Setup with HSTS support (not used in Development environment)

Kestrel wtih HTTPS as a production edge server

• Production environment specific config required to specify listening addresses and production HTTPS certificate
  • Configure Kestrel endpoints and certificates using default config schema (setup by default web host builder, takes precedence over server URLs):

    {
      "Kestrel": {
        "Endpoints": {
          "HTTP": { "Url": "http://*:6000" },
          "HTTPS": {
            "Url": "https://*:6443",
            "Certificate": {
              "Path": "testCert.pfx",
              "Password": "testPassword"
            }
          }
        }
      }
    }

• Absent any configuration, by default Kestrel listens on http://localhost:5000 and https://localhost:5001 if the default HTTPS certificate is available (i.e. on a dev machine with the .NET Core SDK and the ASP.NET Core HTTPS development certificate installed)
  • The default HTTPS certificate is used for any HTTPS addresses configured as server URLs
  • TODO: Consider providing a way to change the default certificate
• HTTPS redirect to first HTTPS server address (typically 443)
• HSTS enabled in production

HTTPS redirection

• New HTTPS redirection extension method (app.UseHttpsRedirection()) simplifies redirecting to HTTPS by simplifying the configuration of the HTTPS port
  • Specify the port and redirect status code via options
  • Specify the port via ASPNETCORE_HTTPS_PORT environment variable if not specified via options
• By default redirects to HTTPS server address if the server is listening on only one (throw otherwise), otherwise port 443

Enabling HSTS

• New HSTS middleware (app.UseHsts())
• Only active on HTTPS requests
• HSTS options for max age, subdomains, and Chrome's HSTS preload list
• Typically used only in non-dev scenarios

Development from the command-line first run experience

• Run "dotnet new razor" or "dotnet new mvc"

• The .NET Core SDK installs the ASP.NET Core HTTPS development certificate as part of the first run experience

  • Installation of the HTTPS development certificate can be disabled by setting the DOTNET_SKIP_HTTPS_DEVELOPMENT_CERTIFICATE environment variable to true

• As part of the first run experience, a message is displayed with instructions for trusting the development certificate:

  Installed the ASP.NET Core HTTPS development certificate. To trust the ASP.NET Core HTTPS development certificate run "dotnet dev-certs https --trust"
  

• Run "dotnet dev-certs https --trust" to trust the ASP.NET Core HTTPS development certificate

  • Windows: The certificate is added to the user's trusted root store. Windows displays a UI prompt to confirm the action.
  • Mac: The certificate is added to the user's keychain.
  • Linux: The tool displays an error message: "The --trust option is not supported on this platform. For details on establishing certificate trust manually see https://go.microsoft.com/fwlink/?linkid=848054."

• Run "dotnet run"

Project template updates

• launchSettings.json is generated by default, including from the command-line:

  {
    "iisSettings": {
      "windowsAuthentication": false,
      "anonymousAuthentication": true,
      "iisExpress": {
        "applicationUrl": "http://localhost:11199/",
        "sslPort": 44333
      }
    },
    "profiles": {
      "IIS Express": {
        "commandName": "IISExpress",
        "launchBrowser": true,
        "environmentVariables": {
          "ASPNETCORE_ENVIRONMENT": "Development"
        }
      },
      "WebApplication38": {
        "commandName": "Project",
        "launchBrowser": true,
        "environmentVariables": {
          "ASPNETCORE_ENVIRONMENT": "Development",
        },
        "applicationUrl": "https://localhost:44333;http://localhost:11199"
      }
    }
  }

  • The applicationUrl and sslPort properties result in the ASPNETCORE_URLS and ASPNETCORE_HTTPS_PORT environment variables being set
  • The HTTP and HTTPS ports for all launch profiles should match
  • IIS Express HTTPS port must be in the range 44300-44399

• Startup.cs updated to enable HSTS and HTTPS redirect:

  public void Configure(IApplicationBuilder app, IHostingEnvironment env)
  {
      if (env.IsDevelopment())
      {
          app.UseDeveloperExceptionPage();
      }
      else
      {
          app.UseExceptionHandler("/Home/Error");
          app.UseHsts();
      }
  
      app.UseHttpsRedirection();
      app.UseStaticFiles();
  
      app.UseMvc(routes =>
      {
          routes.MapRoute(
              name: "default",
              template: "{controller=Home}/{action=Index}/{id?}");
      });
  }

Certificate missing error

• If an HTTPS server URL is configured, but a certificate is not available an exception with the following error message is thrown:

  An HTTPS server URL is configured, but no HTTPS certificate is available. To install the ASP.NET Core HTTPS development certificate run "dotnet dev-certs https". For details on configuring HTTPS in production see https://go.microsoft.com/fwlink/?linkid=848054.
  

IIS Express with HTTPS during development

• No change to IIS Express HTTPS setup - same as exists today
• Uses IIS Express Development Certificate
• VS sets up env var to specify HTTPS port for HTTPS redirection middleware

IIS with HTTPS in production

• No change to IIS HTTPS setup - same as exists today

Development with HTTPS in Docker

• Docker tools export the ASP.NET Core HTTPS development certificate as a PFX file and mounts the path to the exported certificate
• Docker tools stores the certificate password using the user secrets manager and mounts the path to the user secrets file
• Project Docker Compose overrides file includes environment variables for configuring HTTPS using the exported certificate and for the specifying the HTTPS port. Also port mappings for both HTTP and HTTPS that match launchSettings.json.

In production with HTTPS in Docker

• Production Docker compose file includes environment variables for configuring the production certificate and ports.

HTTPS in Docker details

Runtime

• ASP.NET Core on the docker container will be configured through the new Kestrel configuration system.
• The configuration will be passed as environment variables on the docker-compose.override.yml files and as user secrets mapped from the hosting container.
• We will use an HTTPS certificate that will be provided in a PFX file along with its password (via user secrets).

Tooling

• The docker tools will use the 'dev-certs' tool to generate/trust/export the certificate to use
  for HTTPS into a well-known location and with a password of their choosing.
• The docker tools will use the 'user-secrets' tool to place the password for the certificate in the user secrets
  file for the project under the right key.
• On first run, the docker tooling will perform the above actions to ensure that the certificate is provisioned/trusted/exported to
  the right folder location and that the password used to export the certificate is on the user secrets for the application.

Dockerfile changes

• In addition to exposing port 80 we will also export port 443 for HTTPS connections.

Docker compose changes

• We will add environment variables to configure the endpoints kestrel will listen on and to indicated the path where the certificate will
  be made available inside the container.
• We will define the external ports where we want the container to be listening on and those will match the ones in launchSettings.json
• We will map two paths from the host machine into the container.
  • We will map the user secrets folder on the host machine to the user secrets folder on the container.
  • We will map the folder where we've exported the HTTPS certificate into the folder where we expect the certificate to be found in the container.

Sample docker-compose.overrride.yml

version: '3.0'

services:
  dockerhttps:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
      # The name of the variable below does not reflect the actual name
      - ASPNETCORE_URLS=https://*:443;http://*:80
      - ASPNETCORE_HTTPS_PORT=44349
    ports:
      # Both of these ports have been taken from launchSettings.json in the application.
      - "51217:80"
      - "44349:443"
    volumes:
      - ${APPDATA}/Microsoft/UserSecrets/:/root/.microsoft/usersecrets
      - ${APPDATA}/ASP.NET/Https:/root/.aspnet/https/
    
networks:
  default:
    external:
      name: nat

HTTPS project

[danroth27]

@javiercn @jkotalik @Tratcher

[DamianEdwards]

Closing this as only bug-level items left if any.