Handling of private key in Certificate API

[av-commits]

@jaredhendrickson13 thx for your great work

I'm working on CSR handling and your framework is great. Functionality is done, im try now the test framework.

I struggle a bit with the fact that I try to protect the private key by dealing with CSR and the read certificate service just sends it even if not required. It would be great to have a dedicate service to get the private key and a certificate service with more details e.g. Issuer, start and end date. I know that the UI is working the same but I still stink it's not the right way to act with private keys on a critical device.

If you agree or at least think its an acceptable approch how would you adress it? It would be a breaking change, would you do a new version, add something to URI e.g. "?safe" or creative names for new URIs?

Regards
Andrin

[jaredhendrickson13]

I'm definitely not opposed to adding protections for the private key and other sensitive fields. I'd be in favor of having a mechanism built into the framework that allows potentially sensitive fields to be scrubbed from the API response. Personally I'd approach this by adding a couple things:

1: Add a new API setting to enable or disable scrubbing sensitive data returned by the API. Maybe have this setting enabled by default, but allow admins to disable it if they really need to read the private key or any other potentially sensitive field.
2: Allow APIEndpoints to specify sensitive fields by name as a new class property and add a new class method that takes the unfiltered API response and scrubs the sensitive field(s) from the response; if the API setting mentioned previously is enabled.

Such a feature could be added in a minor update. PRs are certainly welcome for this or a new feature request can be opened.

[av-commits]

Thanks for your time.

I would include in the PR a change for GET /api/v1/system/certificate: remove prv (private key) and add all in the web ui available fields (code for all this information should be available in pfsense, details below).
I would also add a new method POST /api/v1/system/certificate-export to access the private key (same as web ui, details below).

The API setting "scrubbing sensitive data" disabled would allow to get "prv" in GET /api/v1/system/certificate (that would allow to be backward compatible)? Or would it apply more with POST /api/v1/system/certificate-export?

Currently Certificate Update (PUT) says (comment in code) that refid and descr is optional and prv and crt are mandatory. I would change that to refid is mandatory and all the other fields descr, prv and crt are optional and updated if available. Currently its required to have the prv to update the descr or add the crt in case of csr (I think that is not needed). Remove dedicated field is not allowed for descr and not usefull for csr or prv.

GET /api/v1/system/certificate
remove prv if available and API settings correct

add field

• certtype

  • certificate-signing-request
  • self-signed
  • certificate-referenced-ca
  • certificate
  • private-key-only
  • unknown

• keyavailable

  • true
  • false

• subject
  Subject DN

• issuer
  Issuer DN

• validfrom
  e.g. 2012-04-23T18:25:43.511Z

• validto
  e.g. 2012-04-23T18:25:43.511Z

• inuse
  where is it in use on pfsense array of enum value: ["webConfigurator","OpenVPN Server","IPsec Tunnel"]

• isservercert
  true if it is a server certificate

• iscacert
  true if it is a ca certificate

• totallifetime
  Total lifetime in days

• lifetimeremaining
  days until expiration

• keytypee
  e.g. RSA

• keylen
  e.g. 2048

• ecname
  e.g. prime256v1 (in case of keytype ECDSA)

• sigtype
  e.g. RSA-SHA256

• altnames
  List of alternative names
  I have seen that test contains following: [{"dns": "test-altname.example.com"},{"ip": "1.1.1.1"},{"uri": "http://example.com/example/uri"},{"email": "example@example.com"}]
  Multiple dns alt name would look like e.g. [{"dns":"t1.exa.com","dns":"t1.exa.com"}]

• keyusage
  Key usage e.g. Digital Signature, Key Encipherment

• extendedkeyusage
  Extended key usage e.g. TLS Web Server Authentication

• serial
  certificate serial given my ca

• subjectkeyid
  Subject Key ID

• authoritykeyid
  Authority Key ID, Key ID of signing CA

• hash
  certificate hash

POST /api/v1/system/certificate-export

Parameters:

• refid: e.g. 64bfb2e64a33d
• exportformat: pem, key-pem or pkcs12
• exportencryption: openssl default (if needed add legacy formats later)
• password: password key will be encrypted with (will be ignored for pem, pkcs12 no password means encryption with empty string --> problems on MacOS https://stackoverflow.com/questions/27497723/export-a-pkcs12-file-without-an-export-password, key-pem no password means no encryption) pfsense standard capabilities are used

Return
Standard with status, code, return, message and data

Data contains all parameters except password and field export with the base64 encoded data of cert, key or both (pkcs12)

Example response:

{
  "status": "ok",
  "code": 200,
  "return": 0,
  "message": "Success",
  "data": [
    {
      "refid": "64dd4420d07a8",
      "descr": "webConfigurator default (64dd4420d07a8)",
      "type": "server",
      "crt": "LS0tLS...",
      "keyavailable": true,
      "subject": "O=pfSense webConfigurator Self-Signed Certificate, CN=pfSense-64dd4420d07a8",
      "issuer": "O=pfSense webConfigurator Self-Signed Certificate, CN=pfSense-64dd4420d07a8",
      "certtype": "self-signed",
      "iscacert": false,
      "isservercert": true,
      "isrevoked": false,
      "validfrom": "2023-08-16T23:48:17+02:00",
      "validto": "2024-09-17T23:48:17+02:00",
      "serial": "9031587196325656075",
      "sigtype": "RSA-SHA256",
      "altnames": [
        {
          "dns": "pfSense-64dd4420d07a8"
        }
      ],
      "keyusage": [
        "Digital Signature",
        "Key Encipherment"
      ],
      "extendedkeyusage": [
        "TLS Web Server Authentication",
        "TLS Web Client Authentication",
        "IP Security IKE Intermediate"
      ],
      "hash": "20904889",
      "subjectkeyid": "EB:9F:FA:1A:B2:96:79:7C:F0:98:4C:A9:F3:3C:70:7D:82:F3:DD:8A",
      "authoritykeyid": "keyid:EB:9F:FA:1A:B2:96:79:7C:F0:98:4C:A9:F3:3C:70:7D:82:F3:DD:8A\nDirName:\/O=pfSense webConfigurator Self-Signed Certificate\/CN=pfSense-64dd4420d07a8\nserial:7D:56:A4:B4:22:F4:F6:0B\n",
      "totallifetime": 398,
      "lifetimeremaining": 391,
      "keytype": "RSA",
      "keylen": 2048
    },
    {
      "refid": "64e3194fde7e3",
      "descr": "ubs",
      "crt": "LS0tLS...",
      "keyavailable": false,
      "subject": "serialNumber=CHE-101.329.561, jurisdictionST=Zürich, jurisdictionC=CH, businessCategory=Private Organization, O=UBS AG, L=Zurich, CN=ebanking-ch.ubs.com, C=CH",
      "issuer": "O=DigiCert Inc, CN=DigiCert EV RSA CA G2, C=US",
      "certtype": "certificate",
      "iscacert": false,
      "isservercert": true,
      "isrevoked": false,
      "validfrom": "2023-01-26T01:00:00+01:00",
      "validto": "2024-02-27T00:59:59+01:00",
      "serial": "2760289667963346329227612742816209184",
      "sigtype": "RSA-SHA256",
      "altnames": [
        {
          "dns": "ebanking-ch.ubs.com"
        },
        {
          "dns": "ebanking-ch1.ubs.com"
        },
        {
          "dns": "ebanking-ch2.ubs.com"
        },
        {
          "dns": "ebanking-ch3.ubs.com"
        },
        {
          "dns": "ebanking-ch4.ubs.com"
        }
      ],
      "keyusage": [
        "Digital Signature",
        "Key Encipherment"
      ],
      "extendedkeyusage": [
        "TLS Web Server Authentication",
        "TLS Web Client Authentication"
      ],
      "hash": "564c0a41",
      "subjectkeyid": "95:3D:3A:6C:2B:13:E3:12:DE:9F:90:1D:2F:E9:7F:B9:03:9A:36:91",
      "authoritykeyid": "keyid:6A:4E:50:BF:98:68:9D:5B:7B:20:75:D4:59:01:79:48:66:92:32:06\n",
      "totallifetime": 396,
      "lifetimeremaining": 187
    },
    {
      "refid": "64e3196347abc",
      "descr": "csr",
      "csr": "LS0tLS...",
      "keyavailable": true,
      "certtype": "certificate-signing-request",
      "subject": "CN=test-csr",
      "keytype": "RSA",
      "keylen": 2048
    },
    {
      "refid": "64e33f5965640",
      "descr": "test ec",
      "csr": "LS0tLS...",
      "keyavailable": true,
      "certtype": "certificate-signing-request",
      "subject": "CN=test ec",
      "keytype": "ECDSA",
      "ecname": "prime256v1"
    },
    {
      "refid": "64e4c259e03a3",
      "descr": "API CSR 1",
      "type": "server",
      "crt": "LS0tLS...",
      "caref": "64e34172aa476",
      "keyavailable": true,
      "subject": "CN=test-csr, C=DE",
      "issuer": "ST=Some-State, O=Internet Widgits Pty Ltd, CN=myca, C=CH",
      "certtype": "certificate-referenced-ca",
      "iscacert": false,
      "isservercert": true,
      "isrevoked": false,
      "validfrom": "2023-08-23T01:46:09+02:00",
      "validto": "2023-09-02T01:46:09+02:00",
      "serial": "0x561EFABF5D260A139D6B3541329687B43A732889",
      "sigtype": "RSA-SHA256",
      "altnames": [
        {
          "dns": "test-altname1.example.com"
        },
        {
          "dns": "test-altname2.example.com"
        },
        {
          "dns": "test-altname3.example.com"
        },
        {
          "ip": "10.10.10.10"
        },
        {
          "ip": "10.10.10.11"
        }
      ],
      "keyusage": [
        "Digital Signature",
        "Key Encipherment"
      ],
      "extendedkeyusage": [
        "TLS Web Server Authentication",
        "TLS Web Client Authentication",
        "IP Security IKE Intermediate"
      ],
      "hash": "5f8b0d58",
      "subjectkeyid": "9A:05:AA:AB:4F:1B:02:92:7C:12:38:CF:0A:63:C6:89:B8:AD:E4:15",
      "totallifetime": 10,
      "lifetimeremaining": 9,
      "keytype": "RSA",
      "keylen": 2048
    },
    {
      "refid": "64e4c25b12c7c",
      "descr": "webConfigurator default (64e4c25b12c7c)",
      "type": "server",
      "crt": "LS0tLS...",
      "keyavailable": true,
      "subject": "O=pfSense webConfigurator Self-Signed Certificate, CN=pfSense-64e4c25b12c7c",
      "issuer": "O=pfSense webConfigurator Self-Signed Certificate, CN=pfSense-64e4c25b12c7c",
      "certtype": "self-signed",
      "iscacert": false,
      "isservercert": true,
      "isrevoked": false,
      "inuse": [
        "webConfigurator"
      ],
      "validfrom": "2023-08-22T16:12:43+02:00",
      "validto": "2024-09-23T16:12:43+02:00",
      "serial": "701819409534630913",
      "sigtype": "RSA-SHA256",
      "altnames": [
        {
          "dns": "pfSense-64e4c25b12c7c"
        }
      ],
      "keyusage": [
        "Digital Signature",
        "Key Encipherment"
      ],
      "extendedkeyusage": [
        "TLS Web Server Authentication",
        "TLS Web Client Authentication",
        "IP Security IKE Intermediate"
      ],
      "hash": "a9bc56d7",
      "subjectkeyid": "4D:F1:0D:91:25:4E:01:6A:45:60:46:5D:74:3B:24:B8:70:47:38:75",
      "authoritykeyid": "keyid:4D:F1:0D:91:25:4E:01:6A:45:60:46:5D:74:3B:24:B8:70:47:38:75\nDirName:\/O=pfSense webConfigurator Self-Signed Certificate\/CN=pfSense-64e4c25b12c7c\nserial:09:BD:5D:08:D9:C9:3C:01\n",
      "totallifetime": 398,
      "lifetimeremaining": 397,
      "keytype": "RSA",
      "keylen": 2048
    },
    {
      "refid": "64e4cbba709dd",
      "descr": "test csr 234234",
      "csr": "LS0tLS...",
      "keyavailable": true,
      "certtype": "certificate-signing-request",
      "subject": "CN=test123",
      "keytype": "RSA",
      "keylen": 2048
    }
  ]
}

[av-commits]

Commit 1 (av-commits@1374131)

• Certificates details
• Remove private key return in Read, Updatet and Create
• Support create and process of CSR
• Update tests and documentation for listed functionality

Commit 2 (av-commits@b9a9c21)

• Signing a csr with pfsense CA
• Read Certificate: support inuse for packages
• Feature 'scrubbing sensitive data'
• Support pkcs12 and pem encrypted key in create
• Update tests and documentation for listed functionality

Open Points / Questions

• I did a first try for 'scrubbing sensitive data', is that what you spoke about?
• Is it correct to add python modules to "/requirements.txt"? (modules installed with pip install)

TODO (complete UI functionality)

• Export key and certs (started with, details below)
• Renew certificate
• Import pkcs12 (Update)

Export Certs and Keys

The UI supports download of p12 or pem files as content type application/octet-stream. I think it would be good to support this also. I did following solution to reach that:

APIResponse.inc

Add a new response type (16)

 16 => [
            "status" => "ok",
            "code" => 200,
            "return" => $id,
            "message" => "Download file of type application/octet-stream",
        ],

Model

Decide if the response should be process as json response or binary content: return id 0 or 16.
Add fields like file name and data content.
Currently I did it with a dedicated request field and I did not use Accept Header. Maybe I need to do a small SPA to see what the best way is: if it is possible to send e.g. a Accept header in a AJAX request.

APIEndpoint.inc

Extend function content_type_encode to support return id 16

    public function content_type_encode($data) {
        # TODO: in future if additional response Content-Types need to be supported we can add them here
        # TODO: and control the requested content-type using the Accept HTTP header
        # Encode response as JSON if content type is application/json

        if ($data["return"] == 16) {
            $size = strlen($data["data"]["content"]);
            $name = $data["data"]["filename"];
            header("Content-Type: application/octet-stream", true);
            header("Content-Length: {$size}");
            header("Content-Disposition: attachment; filename=" . urlencode($name));
            return $data["data"]["content"];
        } else {
            header("Content-Type: application/json", true);
            return json_encode($data);
        }
    }

Is that the correct way to solve this kind of need?