Feature request: add support for Digest authentication for scrape targets

[Tailzip]

Proposal

Add support for Digest authentification (in addition to already existing Basic authentication) for scrape targets.

[LeviHarrison]

Transferring to the appropriate repo.

[LeviHarrison]

I'm not familiar with Digest authentication, but given that it's not included in the Go standard http library, I think it might be too niche to support.

cc @roidelapluie

[roidelapluie]

Indeed. Digest requires an extra roundtrip and does not bring added value. If you want to secure the authorization please use basic auth with tls.

[muety]

Would love to have digest auth support, because wanting to scrape targets that only support such.

[anarcat]

i've found myself in this situation that i needed http digest auth for prometheus, then re-checked what htdigest auth is and, oh dear, that thing is just plain horrible.

the server keeps a copy of the user:realm:password MD5 hash on the server and that is basically what's used by the client to authenticate as a secret. The server sends a nonce, but that's it: the client replies with the MD5(MD5(username:realm:password):nonce:MD5(method:digestURI)) hash, which can be easily replayed by someone who has a copy of the server side password store.

essentially, those are plain-text passwords, it's bonkers.

i actually discourage the Prometheus folks from implementing this, maybe it will give people good ideas if they see Prom does not implement this.

the only universe where this makes sense is in plain HTTP where the wire is constantly surveilled. then there is an advantage because only the latter hash travels over the wire. but those days are long gone...

[muety]

I agree that if Prometheus was a web framework or something, it probably should intentionally not support digest auth to prevent people from implementing a digest auth-based login in the first place.

However, since Prometheus is a monitoring system and people use it to run against already existing systems, which they often times can't control, I think Prometheus should offer as many options as possible (following Postel's Law in some sense).

In my particular use case, I want to monitor a system that only support digest auth. But I can't use Prometheus to so so, unless somebody eventually merges #553.

[anarcat]

On 2024-09-12 23:39:36, Ferdinand Mütsch wrote: I agree that if Prometheus was a web framework or something, it probably should intentionally _not_ support digest auth to prevent people from implementing a digest auth-based login in the first place. However, since Prometheus is a monitoring system and people use it to run against _already existing_ systems, which they often times can't control, I think Prometheus should offer as many options as possible (following [Postel's Law](https://en.wikipedia.org/wiki/Robustness_principle) in some sense). In my particular use case, I want to monitor a system that _only_ support digest auth. But I can't use Prometheus to so so, unless somebody eventually merges #553.

And there were many comments made on that merge request that question whether or not even that code is a good idea. At this point, I would argue the robustness principle would say we should *not* implement this. There are *lots* of authentication systems out there, and you need to figure out a way to work around those to make your monitoring work with Prometheus. I am afraid that if we open the door to legacy systems such as this one, it will make an argument for other people to request more such things. See also: https://www.robustperception.io/prometheus-security-authentication-authorization-and-encryption/ ... which lists, as examples, " Digest access, bearer tokens, Kerberos, GSSAPI, OpenID, SSL client certificates, Radius and PAM ", just for the authentication layer. Out of those, I believe Prometheus supports SSL and bearer tokens. I think that's fair.

[SuperQ]

I'm going to leave it up to the maintainers, but I recommend this be closed as not planned.

[roidelapluie]

I don't think we should supprt it, but you could have a proxy do it:

import base64
import hashlib
from http.server import BaseHTTPRequestHandler, HTTPServer
import http.client
from urllib.parse import urlparse


class ProxyHTTPRequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        # Parse the requested URL
        parsed_url = urlparse(self.path)
        target_host = parsed_url.netloc
        target_path = parsed_url.path or '/'
        
        # Create a connection to the target server
        conn = http.client.HTTPConnection(target_host)

        # Check for Basic Authentication header
        auth_header = self.headers.get('Authorization')
        if auth_header and auth_header.startswith('Basic '):
            # Decode Basic Auth credentials
            auth_type, encoded_credentials = auth_header.split(' ', 1)
            decoded_credentials = base64.b64decode(encoded_credentials).decode('utf-8')
            username, password = decoded_credentials.split(':', 1)

            # Remove Basic Auth header for the forwarded request
            del self.headers['Authorization']

            # Make an initial request to get the Digest challenge
            conn.request('GET', target_path, headers=self.headers)
            initial_resp = conn.getresponse()

            if initial_resp.status == 401 and 'WWW-Authenticate' in initial_resp.headers:
                # Extract Digest challenge from the server response
                digest_challenge = initial_resp.headers['WWW-Authenticate']
                challenge_params = self.parse_digest_challenge(digest_challenge)

                # Generate Digest Authentication header
                digest_auth_header = self.generate_digest_auth_header(username, password, 'GET', target_path, challenge_params)

                # Forward the request again with Digest Authentication
                self.headers['Authorization'] = digest_auth_header
                conn.request('GET', target_path, headers=self.headers)
                response = conn.getresponse()

                # Send response back to the client
                self.send_response(response.status)
                for key, value in response.getheaders():
                    self.send_header(key, value)
                self.end_headers()
                self.wfile.write(response.read())
                return

        # Forward the request if no authentication is required
        conn.request('GET', target_path, headers=self.headers)
        response = conn.getresponse()
        self.send_response(response.status)
        for key, value in response.getheaders():
            self.send_header(key, value)
        self.end_headers()
        self.wfile.write(response.read())

    def parse_digest_challenge(self, header):
        """Parse the Digest challenge header and return a dictionary of parameters."""
        challenge = {}
        for param in header.split(","):
            key, value = param.strip().split("=", 1)
            challenge[key.strip()] = value.strip('"')
        return challenge

    def generate_digest_auth_header(self, username, password, method, uri, challenge):
        """Generate the Digest Authentication header."""
        ha1 = hashlib.md5(f"{username}:{challenge['realm']}:{password}".encode('utf-8')).hexdigest()
        ha2 = hashlib.md5(f"{method}:{uri}".encode('utf-8')).hexdigest()
        response = hashlib.md5(f"{ha1}:{challenge['nonce']}:00000001:xyz:{challenge['qop']}:{ha2}".encode('utf-8')).hexdigest()

        return (f'Digest username="{username}", realm="{challenge["realm"]}", nonce="{challenge["nonce"]}", uri="{uri}", '
                f'qop={challenge["qop"]}, nc=00000001, cnonce="xyz", response="{response}", opaque="{challenge.get("opaque", "")}"')


def run(server_class=HTTPServer, handler_class=ProxyHTTPRequestHandler, port=8080):
    server_address = ('', port)
    httpd = server_class(server_address, handler_class)
    print(f"Starting proxy server on port {port}")
    httpd.serve_forever()


if __name__ == '__main__':
    run()