feat: fetch TLS client hello message from HTTP.SYS #60806

DeagleGross · 2025-03-07T12:41:01Z

Expose API to get the TLS client hello message from http.sys

A new callback added to HttpSysOptions should invoke a callback with the raw bytes of TLS client hello message, and it should be happening once per connection.

namespace Microsoft.AspNetCore.Server.HttpSys;

public class HttpSysOptions
{
   Action<IFeatureCollection, ReadOnlySpan<byte>> ClientHelloBytesCallback { get; set; }
}

Fixes #60805

Copilot

PR Overview

This PR adds an API to fetch the TLS client hello message by introducing a new interface (ITlsAccessFeature) along with sample code changes and necessary updates in the underlying HttpSys and native interop code. Key changes include:

Introducing ITlsAccessFeature and its implementation to expose raw TLS client hello message bytes.
Updating the HttpSys request processing to retrieve, log, and expose the TLS client hello message.
Adding support in HttpApi interop and configuring HTTP service settings to enable caching of the client hello message.

Reviewed Changes

File	Description
src/Servers/HttpSys/samples/TlsFeaturesObserve/Program.cs	Updated sample to invoke new TLS client hello message caching configuration.
src/Servers/HttpSys/samples/TlsFeaturesObserve/Startup.cs	Sample updated to use ITlsAccessFeature for fetching TLS client hello bytes.
src/Servers/Connections.Abstractions/src/Features/ITlsAccessFeature.cs	New API interface for accessing TLS client hello message bytes.
src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs	Integrated methods to retrieve and parse TLS client hello message bytes.
src/Servers/HttpSys/src/NativeInterop/HttpApi.cs	Updated interop definitions to include HttpSetServiceConfiguration.
Others	Minor updates in logging, error codes, and feature collection to support the new API.

Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (4)

src/Servers/HttpSys/samples/TlsFeaturesObserve/Program.cs:11

[nitpick] The namespace 'TlsFeatureObserve' is inconsistent with the folder name 'TlsFeaturesObserve'. Consider aligning the namespace with the folder structure to improve clarity.

namespace TlsFeatureObserve;

src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs:245

[nitpick] A magic number (11) is used inline for the TLS client hello property. Consider using a named constant to enhance readability and maintainability.

11 /* HTTP_REQUEST_PROPERTY.HttpRequestPropertyTlsClientHello  */

src/Servers/HttpSys/src/LoggerEventIds.cs:62

[nitpick] The logger event name 'TlsClientHelloParseError' is defined here but the LoggerMessage in RequestContext.Log.cs uses 'TlsClientHelloRetrieveError'. Ensure consistent naming for clarity.

public const int TlsClientHelloParseError = 55;

src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs:227

[nitpick] The implementation of 'GetTlsClientHelloMessageBytes()' appears similar to other copies in the code. Consider refactoring the duplicate logic into a shared utility method to improve maintainability.

internal unsafe byte[]? GetTlsClientHelloMessageBytes()

DeagleGross · 2025-04-07T17:16:39Z

/backport to release/8.0

github-actions · 2025-04-07T17:16:50Z

Started backporting to release/8.0: https://github.com/dotnet/aspnetcore/actions/runs/14315380132

github-actions · 2025-04-07T17:17:13Z

@DeagleGross an error occurred while backporting to "release/8.0", please check the run log for details!

Error: @DeagleGross is not a repo collaborator, backporting is not allowed. If you're a collaborator please make sure your dotnet team membership visibility is set to Public on https://github.com/orgs/dotnet/people?query=DeagleGross

BrennanConroy · 2025-04-07T17:17:32Z

/backport to release/8.0

github-actions · 2025-04-07T17:17:46Z

Started backporting to release/8.0: https://github.com/dotnet/aspnetcore/actions/runs/14315397681

github-actions · 2025-04-07T17:18:12Z

@BrennanConroy backporting to "release/8.0" failed, the patch most likely resulted in conflicts:

$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: setup for tls clinet hello exposure
.git/rebase-apply/patch:249: trailing whitespace.
        // use an array of bytes instead of the sockaddr structure 
.git/rebase-apply/patch:362: trailing whitespace.
    /// Client certificate is not to be verified for revocation. 
.git/rebase-apply/patch:367: trailing whitespace.
    /// Only cached certificate is to be used the revocation check. 
warning: 3 lines add whitespace errors.
Using index info to reconstruct a base tree...
M	AspNetCore.sln
M	src/Servers/HttpSys/HttpSysServer.slnf
A	src/Servers/HttpSys/src/NativeInterop/ErrorCodes.cs
M	src/Servers/HttpSys/src/NativeInterop/HttpApi.cs
M	src/Servers/HttpSys/src/RequestProcessing/Request.cs
M	src/Servers/HttpSys/src/RequestProcessing/RequestContext.FeatureCollection.cs
M	src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs
Falling back to patching base and 3-way merge...
Auto-merging src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs
CONFLICT (content): Merge conflict in src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs
Auto-merging src/Servers/HttpSys/src/RequestProcessing/RequestContext.FeatureCollection.cs
CONFLICT (content): Merge conflict in src/Servers/HttpSys/src/RequestProcessing/RequestContext.FeatureCollection.cs
Auto-merging src/Servers/HttpSys/src/RequestProcessing/Request.cs
CONFLICT (content): Merge conflict in src/Servers/HttpSys/src/RequestProcessing/Request.cs
Auto-merging src/Servers/HttpSys/src/NativeInterop/HttpApi.cs
CONFLICT (content): Merge conflict in src/Servers/HttpSys/src/NativeInterop/HttpApi.cs
CONFLICT (modify/delete): src/Servers/HttpSys/src/NativeInterop/ErrorCodes.cs deleted in HEAD and modified in setup for tls clinet hello exposure. Version setup for tls clinet hello exposure of src/Servers/HttpSys/src/NativeInterop/ErrorCodes.cs left in tree.
Auto-merging src/Servers/HttpSys/HttpSysServer.slnf
Auto-merging AspNetCore.sln
CONFLICT (content): Merge conflict in AspNetCore.sln
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 setup for tls clinet hello exposure
Error: The process '/usr/bin/git' failed with exit code 128

Please backport manually!

src/Servers/HttpSys/src/HttpSysOptions.cs

src/Servers/HttpSys/src/NativeInterop/DisconnectListener.cs

src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs

src/Servers/HttpSys/src/RequestProcessing/RequestContext.Log.cs

BrennanConroy · 2025-04-07T20:54:06Z

src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs

+        uint* bytesReturned = &bytesReturnedValue;
+        uint statusCode;
+
+        var requestId = PinsReleased ? Request.RequestId : RequestId;


Any reason we don't always use Request.RequestId?

other interop API in this file is used at a time when NativeRequest is available and this one is used later when NativeRequest is already released, meaning we should be using different requestId. I thought making an implementation more generic would help in usage, because it is a very simple check, but helps avoid confusion.

src/Servers/HttpSys/src/RequestProcessing/RequestContext.cs

src/Servers/HttpSys/src/RequestProcessing/TlsListener.cs

markalward · 2025-04-08T03:31:23Z

src/Servers/HttpSys/src/RequestProcessing/TlsListener.cs

+
+    internal void InvokeTlsClientHelloCallback(IFeatureCollection features, Request request)
+    {
+        if (_connectionIdsTlsClientHelloCallbackInvokedMap.TryGetValue(request.UConnectionId, out _))


I think UConnectionId varies per request on HTTP/2, see comment here: https://source.dot.net/#Microsoft.AspNetCore.Server.HttpSys/RequestProcessing/Request.cs,48.

The DisconnectToken that's used to fire the ConnectionClosed() callback behaves the same way - it calls HttpWaitForDisconnectEx(), which returns at the end of every request for HTTP/2, not at the end of the connection.

Ye, I also thought about it. I will re-do now tracking RawConnectionId instead

So since I dont see a way to follow on any http.sys notification/subscription around connection lifetime (RawConnectionId, not UConnectionId), I changed approach to TTL based (basically keep it in cache while connection alive, if no requests come in some interval (10 minutes as hardcoded value), then we remove the connectionId from cache).

I am going to sync with @BrennanConroy to discuss it offline

davidfowl · 2025-04-08T04:28:50Z

src/Servers/HttpSys/samples/TlsFeaturesObserve/Program.cs

+using TlsFeatureObserve;
+using TlsFeaturesObserve.HttpSys;
+
+HttpSysConfigurator.ConfigureCacheTlsClientHello();


What is this?

Fetching TLS client hello feature from HTTP.SYS works only if a pre-configuration was done for ssl cert binding and netsh ssl config beforehand. User would need to set service configuration via HttpSetServiceConfiguration with HTTP_SERVICE_CONFIG_SSL_FLAG_ENABLE_CACHE_CLIENT_HELLO flag.

That is and should not be a part of ASP.NET app, but I added that for "sample app" for simplicity of reproduction.

src/Servers/HttpSys/src/RequestProcessing/TlsListener.cs

Old

src/Servers/HttpSys/src/RequestProcessing/TlsListener.cs

src/Servers/HttpSys/src/HttpSysListener.cs

DeagleGross added 3 commits March 6, 2025 16:10

setup for tls clinet hello exposure

d7953f6

correctly retry access

aa3ae32

last minute changes

840cdac

DeagleGross self-assigned this Mar 7, 2025

Copilot bot review requested due to automatic review settings March 7, 2025 12:41

DeagleGross requested review from halter73, BrennanConroy, JamesNK and mgravell as code owners March 7, 2025 12:41

dotnet-issue-labeler bot added the area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions label Mar 7, 2025

Copilot AI reviewed Mar 7, 2025

View reviewed changes

fix warnings

c9431ec

dotnet-policy-service bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Mar 15, 2025

DeagleGross changed the title ~~feat: create API to fetch TLS client hello message~~ feat: fetch TLS client hello message from HTTP.SYS Mar 20, 2025

DeagleGross mentioned this pull request Mar 20, 2025

[API Proposal]: expose TLS client hello message in SslStream dotnet/runtime#113729

Open

DeagleGross force-pushed the dmkorolev/ja4 branch from 880651a to c9431ec Compare April 4, 2025 15:04

DeagleGross added 7 commits April 7, 2025 13:04

hook up tls client hello callback

1d0f0d8

fix warnings & publish API

3c4d6fc

minimal

0c31882

only go via callback if options has callback set; remove unused

5d7d187

a bit refactor

8cf4cea

merge main

6f5cfaf

update in slnx !

7566aec

build-analysis bot mentioned this pull request Apr 7, 2025

System.TimeoutException : The operation has timed out. dotnet/dnceng#5279

Open

3 tasks

PR review

946440a

BrennanConroy previously approved these changes Apr 7, 2025

View reviewed changes

markalward reviewed Apr 8, 2025

View reviewed changes

src/Servers/HttpSys/src/RequestProcessing/TlsListener.cs Outdated Show resolved Hide resolved

markalward reviewed Apr 8, 2025

View reviewed changes

davidfowl reviewed Apr 8, 2025

View reviewed changes

DeagleGross added 3 commits April 8, 2025 12:07

Merge branch 'main' into dmkorolev/ja4

dc8ae7e

address PR comments x1

0ad12ca

TTL & evict approach

e9c7ead

BrennanConroy reviewed Apr 8, 2025

View reviewed changes

src/Servers/HttpSys/src/RequestProcessing/TlsListener.cs Outdated Show resolved Hide resolved

src/Servers/HttpSys/src/RequestProcessing/TlsListener.cs Show resolved Hide resolved

src/Servers/HttpSys/src/RequestProcessing/TlsListener.cs Outdated Show resolved Hide resolved

DeagleGross added 2 commits April 8, 2025 22:41

address comments 1

e170454

periodic timer

b830fca

BrennanConroy reviewed Apr 8, 2025

View reviewed changes

DeagleGross added 3 commits April 9, 2025 00:26

address comments x3

e1bdba4

TryAdd

033e211

make a static field (just in case)

23bfa1b

DeagleGross enabled auto-merge (squash) April 8, 2025 22:36

Cache updates

c69797f

build-analysis bot mentioned this pull request Apr 11, 2025

System.OperationCanceledException : The operation was canceled. dotnet/dnceng#5278

Open

3 tasks

davidfowl reviewed Apr 11, 2025

View reviewed changes

src/Servers/HttpSys/src/HttpSysListener.cs Outdated Show resolved Hide resolved

BrennanConroy added 4 commits April 11, 2025 15:48

test

886de4f

whitespace

3fe7871

clear array

8dc51f8

appcontext

54a9c75

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: fetch TLS client hello message from HTTP.SYS #60806

feat: fetch TLS client hello message from HTTP.SYS #60806

DeagleGross commented Mar 7, 2025 •

edited

Loading

Copilot AI left a comment

DeagleGross commented Apr 7, 2025

github-actions bot commented Apr 7, 2025

github-actions bot commented Apr 7, 2025

BrennanConroy commented Apr 7, 2025

github-actions bot commented Apr 7, 2025

github-actions bot commented Apr 7, 2025

BrennanConroy Apr 7, 2025

DeagleGross Apr 8, 2025

markalward Apr 8, 2025

DeagleGross Apr 8, 2025

DeagleGross Apr 8, 2025

davidfowl Apr 8, 2025

DeagleGross Apr 8, 2025

feat: fetch TLS client hello message from HTTP.SYS #60806

Are you sure you want to change the base?

feat: fetch TLS client hello message from HTTP.SYS #60806

Conversation

DeagleGross commented Mar 7, 2025 • edited Loading

Expose API to get the TLS client hello message from http.sys

Copilot AI left a comment

Choose a reason for hiding this comment

PR Overview

Reviewed Changes

DeagleGross commented Apr 7, 2025

github-actions bot commented Apr 7, 2025

github-actions bot commented Apr 7, 2025

BrennanConroy commented Apr 7, 2025

github-actions bot commented Apr 7, 2025

github-actions bot commented Apr 7, 2025

BrennanConroy Apr 7, 2025

Choose a reason for hiding this comment

DeagleGross Apr 8, 2025

Choose a reason for hiding this comment

markalward Apr 8, 2025

Choose a reason for hiding this comment

DeagleGross Apr 8, 2025

Choose a reason for hiding this comment

DeagleGross Apr 8, 2025

Choose a reason for hiding this comment

davidfowl Apr 8, 2025

Choose a reason for hiding this comment

DeagleGross Apr 8, 2025

Choose a reason for hiding this comment

DeagleGross commented Mar 7, 2025 •

edited

Loading