proxy-authentication header missing with https

[maddo7]

I want to create a mitm proxy that can only be access by providing correct credentials, e.g. I check if proxy-authentication has the correct value:

(async () => {
    const mockttp = require('mockttp');

    // Create a proxy server with a self-signed HTTPS CA certificate:
    const https = await mockttp.generateCACertificate();
    const server = mockttp.getLocal({ https });

server.forAnyRequest().thenCallback((request) => {
    return {
        status: 200,
        // Return a JSON response with an incrementing counter:
        json: request
    };
});
    await server.start(8080);

    // Print out the server details:
    const caFingerprint = mockttp.generateSPKIFingerprint(https.cert)
    console.log(`Server running on port ${server.port}`);
    console.log(`CA cert fingerprint ${caFingerprint}`);
})(); // (Run in an async wrapper so we can use top-level await everywhere)

With http it works flawlessly, the proxy-authorization header is present:

curl -k -v --proxy "user:pass@127.0.0.1:8080" http://www.google.com

{
   "id":"8978f1a3-8a4f-4395-b0dc-0cf8929e760a",
   "matchedRuleId":"5a1bc167-7e34-4b0d-9f51-f8e49015b349",
   "protocol":"http",
   "httpVersion":"1.1",
   "method":"GET",
   "url":"http://www.google.com/",
   "path":"/",
   "remoteIpAddress":"::ffff:127.0.0.1",
   "remotePort":32932,
   "headers":{
      "host":"www.google.com",
      "proxy-authorization":"Basic dXNlcjpwYXNz",
      "user-agent":"curl/7.83.1",
      "accept":"*/*",
      "proxy-connection":"Keep-Alive"
   },
   "rawHeaders":[
      [
         "Host",
         "www.google.com"
      ],
      [
         "Proxy-Authorization",
         "Basic dXNlcjpwYXNz"
      ],
      [
         "User-Agent",
         "curl/7.83.1"
      ],
      [
         "Accept",
         "*/*"
      ],
      [
         "Proxy-Connection",
         "Keep-Alive"
      ]
   ],
   "tags":[
      
   ],
   "timingEvents":{
      "startTime":1663860475270,
      "startTimestamp":7655.8840999901295,
      "bodyReceivedTimestamp":7656.588100001216
   },
   "body":{
      "buffer":{
         "type":"Buffer",
         "data":[
            
         ]
      }
   }
}

Now the problem is that if it runs through https, the proxy-authorization disappears:

curl -k -v --proxy "user:pass@127.0.0.1:8080" https://www.google.com
{
   "id":"dd9f61c9-8ecb-4f94-87aa-095fd2f40da6",
   "matchedRuleId":"5a1bc167-7e34-4b0d-9f51-f8e49015b349",
   "protocol":"https",
   "httpVersion":"1.1",
   "method":"GET",
   "url":"https://www.google.com/",
   "path":"/",
   "remoteIpAddress":"::ffff:127.0.0.1",
   "remotePort":34557,
   "headers":{
      "host":"www.google.com",
      "user-agent":"curl/7.83.1",
      "accept":"*/*"
   },
   "rawHeaders":[
      [
         "Host",
         "www.google.com"
      ],
      [
         "User-Agent",
         "curl/7.83.1"
      ],
      [
         "Accept",
         "*/*"
      ]
   ],
   "tags":[
      
   ],
   "timingEvents":{
      "startTime":1663860737403,
      "startTimestamp":269786.7910999954,
      "bodyReceivedTimestamp":269787.29159998894
   },
   "body":{
      "buffer":{
         "type":"Buffer",
         "data":[
            
         ]
      }
   }
}

Is there anything I'm unaware of that causes this behaviour?

[pimterry]

Hi @maddo7, this is a good question. This happens because there's two different ways of doing HTTP proxying.

The first way is that the client sends a request to the proxy like GET http://example.com/abc - i.e. it sends the entire request to the proxy, and the proxy parses that, extracts the target server (example.com) and forwards the request there. In this case, all headers are included in that one request. This is probably what's happening with your HTTP proxying.

The second way is that the client sends a CONNECT request to the proxy like CONNECT example.com:80, asking for a tunnel a remote server, and then it sends a separate request like GET /abc inside that tunnel, talking directly to the remote server. In this case, proxy-specific headers only appear on the first request. Wikipedia has some more details: https://en.wikipedia.org/wiki/HTTP_tunnel.

Currently, in that second case, you can't interact with the outer tunneling request or see that data via Mockttp at all. Mockttp simply unwraps and discards all layers of CONNECT tunnelling, and only considers the final request to the end server. Extending this is a bit more complicated than it sounds, because requests aren't even 1-1, for example a client can CONNECT through the proxy once, and then send many independent requests inside the resulting tunnel to the remote server (or even none at all, if it changes its mind). You can even CONNECT to create a tunnel to a different proxy server, and then CONNECT there too, at unlimited depth, using different authentication headers for each step.

For this specific use case of authenticated tunneling though, I think it's probably possible to handle this though - we could add a proxyAuth option like getLocal({ https, proxyAuth: { username: '...', password: '...' } }), and then enforce that on incoming requests before we start normal final-request processing. PRs very welcome if that's something you'd be interested in.