-
Notifications
You must be signed in to change notification settings - Fork 107
TLS SSL Certificates
Where possible, our TLS certificates are managed automatically by our platform. In the BSP environments, certificates are managed manually.
DNS Certification Authority Authorization (CAA) records tell Certificate Authorities whether or not they are authorized to issue certificates for a given domain and its subdomains. This helps prevent abuse where a certificate might be issued for data.gov without our knowledge.
To see the existing CAA records, check DNS.
$ dig CAA data.gov
Any domain served from BSP would use the GSA certificate authority/issuer, currently DigiCert. But some domains might be issued from other providers. Following the principle of least privilege, subdomain records outside of BSP should have their own CAA records.
| Hosting | Certificate Authority | Domains |
|---|---|---|
| API* | Let's Encrypt | api.data.gov |
| Federalist | Let's Encrypt | federation.data.gov, resources.data.gov, sdg.data.gov, strategy.data.gov |
| AWS | AWS ACM | ssb.data.gov |
| BSP | DigiCert | data.gov (everything else) |
* Data.gov delegates api.data.gov domain to the #api-data-gov team.
When requesting a CAA record, it should have no flags (0) and the issuer tag.
data.gov CAA 0 issuer "digicert.com"
No action needed for Federalist sites. Federalist manages the certificates, we’re only responsible for the content.
Note: This section mostly applies to the BSP environment. For AWS Sandbox environments, public certificates are automatically provisioned through AWS and host certificates are self-signed.
For Data.gov, we use GSA provisioned certificates in the BSP environment. Site certificates cover "public" network facing components like load balancers or CDNs. Host certificates are installed on each host to provide transport encryption between hosts within the network, which is required for security compliance.
Production site certificates are external GSA certificates while non-production/staging site certificates are internal GSA certificates. All host certificates are internal GSA certificates.
The process for getting (or updating) a certificate is as follows:
- Create a new key and CSR
- Open a ticket in Service Now and attach the CSR
- Install the certificate and key
- Expiration reminder
Each environment will need a site certificate and a host certificate. Wildcard certificates are not allowed by GSA, so we include all hosts/domains via Subject Alternate Names (SANs).
Note: For site certificates, keep in mind that we have CAA records to only authorize specific Certificate Authorities to issue certificates for data.gov domains.
data.gov contains a few scripts to automate the process and reduce the chance for mistakes or missing a host.
gen-csr.sh reads Subject Alternative Names (SANs) from stdin to create a single certificate. Files are generated in gen-csr-out, including the key and CSR.
There should be a text file next to gen-csr.sh for production and staging.
$ cd ansible
$ bin/gen-csr.sh data.gov < bin/production-sans.txt
ansible/gen-csr-out/data.gov.csr will be attached to the Service Now request. Upload the .key file to Google Drive and save it for later. It will be installed to the load balancers and CDNs once the certificate is provisioned by GSA. Use Google Drive's file versioning feature to update the existing file rather than creating a new one (right-click and select Manage Versions).
You can inspect the CSR with openssl.
$ openssl req -text -noout -in gen-csr-out/data.gov.csr
Host certificates get the SANs from the Ansible inventory. Run the gen-csr.sh with the inventory name and feed the hostnames from ansible-inventory to stdin.
$ cd ansible
$ inventory=staging
$ bin/gen-csr.sh $inventory < <(ansible-inventory -i "inventories/$inventory" --list | jq -r '.[].hosts[]?' | sort | uniq)
ansible/gen-csr-out/${inventory}.csr will be attached to the Service Now request. Save the .key file to the Ansible Vault.
You can inspect the CSR with openssl.
$ openssl req -text -noout -in gen-csr-out/$inventory.csr
Open a Service Now ticket for GSA SSL certificate request. Fill out the required information. When asked if this is a new certificate or a replacement, select new certificate.
For production site certificates, you want to request an external certificate. For all other certificates, you want to request an internal certificate.
You will be filling in a lot of redundant information that is included in the CSR. You can inspect the CSR for this information:
$ openssl req -text -noout -in gen-csr-out/data.gov.csr
Some additional fields not in the CSR:
| Field | Value |
|---|---|
| Enhanced Key Usage | Server Authentication (1.3.6.1.5.5.7.3.1) |
| Responsible SSO | Federal Acquisition Services (Q) |
| FISMA System | Data.gov |
| Application Team | FCS Team Approval |
Submit the form. Once the request is created you can attach the CSR. Note: On May 20th, 2020 it was not accepting .csr extensions, so I appended .txt before I could upload the CSR.
Provisioning the certificate may take a few days. Once provisioned, the certificate will be attached to the Service Now ticket. The GSA certificate is in PEM format and should be installed with the key generated in step 1.
For site certificates, upload the certificate and key to Google Drive.
Host certificate and keys should be added to the Ansible inventory as default_tls_host_certificate and default_tls_host_key, respectively.
For production site certificates:
- Install certificate and key in the CloudFront CDN from the opp prod account.
- Open issue with FCS to install certificate and key for NetScaler and Palo Alto firewall.
Site certificate and keys are installed manually to CDNs and firewalls. You will need to open an FCS ticket to get these installed.
For any certificate changes, you will need to involve the GSA SecOps team. Add a note saying FCS, please reach out to the GSA SecOps team to coordinate timing. Should also raise for awareness on the FCS Google Chat, that Team, ticket number __ is a certificate change and will require coordination with GSA SecOps. Please let us know the timing that is decided upon so that we can be ready to assist if necessary.
For staging site certificates:
There are no CDN's or WAF's for staging. So, you will need to open a ticket with BSP to install the cert on Netscaler. Follow these steps:
Create a generic request using these values:
-
Short Description: Install certificate on Netscaler -
Justification for request: [Current certificate is expired] OR [SANs list is modified]. Please install new certificate on Netscaler. Needed for data.gov. Urls are:insert all urls here-> You can get this list from filestaging-sans.txtin the data.gov repository. -
Attachments: Attach both the key file you generated and certificate file you received from previous steps. Service Now doesn't accept these file types. To get around this you can use zip to create a zip file before attaching. -
Add comments: In addition to install the new certificate on Netscaler, please coordinate with GSA Secops to have it installed on Palo Alto appliance due to recent decryption policy change on the Firewalls.
The certificates can be verified by browsing to the site and looking at the Not After date of the certificate for the page or on the command line using openssl.
echo | openssl s_client -4 -servername catalog.data.gov -connect catalog.data.gov:443 2>/dev/null | openssl x509 -noout -dates
We're currently manually tracking certificate expiration. Check when the new certificate is due to expire. Add a reminder to the data.gov calendar 60 days before the expiration date.