feat: encrypt sensitive columns

[hf]

Adds support for encrypting sensitive columns like the MFA secret and password hash.

The goal with this encryption mechanism is to add yet another layer of security on top of the database permissions provided by Postgres. In the event that the database leaks or is accessed by malicious users or the database permissions are incorrectly defined, the encryption key would also be required to inspect this sensitive data.

Encryption is done using AES-GCM-256. Strings that are encrypted are converted into a JSON string with this shape:

{
  "key_id": "key identifier used for encryption",
  "alg": "aes-gcm-hkdf",
  "nonce": "GCM 12 byte nonce",
  "data": "Base64 standard encoding of the ciphertext"
}

As AES-GCM must not be used more than 2^32 times with a single symmetric key, and this is not that much -- imagine serving 100m users -- then this means that all users can only add 42 passwords or MFA verification factors before running into this hard limit. To fix this, a symmetric key is derived using HKDF with SHA256 such that the symmetric key is used together with the object ID (for passwords - the user ID, for TOTP secrets - the factor ID). This way there's a separate AES-GCM key per object, and additionally gives the security property that a malicious actor with write permissions to the database cannot swap passwords / TOTP secrets from Malice's account to Target's account. They would need to also change the UUIDs of these objects, which is likely to be hard.

To turn on encryption the following configs need to be added:

GOTRUE_SECURITY_DB_ENCRYPTION_ENCRYPT=true -- that turns on encryption for new objects.
GOTRUE_SECURITY_DB_ENCRYPTION_ENCRYPTION_KEY_ID=key-id -- ID of the encryption key, allowing to rotate keys easily.
GOTRUE_SECURITY_DB_ENCRYPTION_ENCRYPTION_KEY=key -- Base64 URL encoding of a 256 bit AES key

Once encryption has been turned on, in order to have the rows be readable for ever this config must be provided with all past and future keys:

GOTRUE_SECURITY_DB_ENCRYPTION_DECRYPTION_KEYS=key-id:key -- A map of key IDs and Base64 URL key encodings of the keys.

To retire keys, you should just move the old key to the decryption keys map, and advertise the new encryption key ID. On each successful sign in with password, or any MFA verification attempt, the latest key will be used to re-encrypt the column. This also applies for the non-encrypted-to-encrypted case.

[github-actions]

Pull Request Test Coverage Report for Build 9366388885

Details

• 100 of 151 (66.23%) changed or added relevant lines in 8 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.05%) to 57.638%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
internal/api/admin.go	1	2	50.0%
internal/api/mfa.go	14	18	77.78%
internal/models/user.go	16	20	80.0%
internal/conf/configuration.go	15	31	48.39%
internal/crypto/crypto.go	48	74	64.86%

Totals
Change from base Build 9361623090:	0.05%
Covered Lines:	8591
Relevant Lines:	14905

---

💛 - Coveralls

[J0]

LGTM, thanks for putting this together

[kangmingtay]

@hf Some questions:

... such that the symmetric key is used together with the object ID (for passwords - the user ID, for TOTP secrets - the factor ID). This way there's a separate AES-GCM key per object, ...

just to clarify my understanding, the encryption key of an object is derived from the symmetric key + the object id?

To retire keys, you should just move the old key to the decryption keys map, and advertise the new encryption key ID.

The old key will already be in the decryption keys map because we had to use it previously to decrypt the objects right? So we actually just need to update the encryption key and encryption key id?

the latest key will be used to re-encrypt the column.

This gives me the impression that GOTRUE_SECURITY_DB_ENCRYPTION_ENCRYPTION_KEY is an array of keys but it is actually just 1 key right?

[hf]

... such that the symmetric key is used together with the object ID (for passwords - the user ID, for TOTP secrets - the factor ID). This way there's a separate AES-GCM key per object, ...

just to clarify my understanding, the encryption key of an object is derived from the symmetric key + the object id?

Yes, the ..._ENCRYPTION_KEY is really a key-derivation-key. The encryption key for a specific row is derived from this KDK + the row's UUID.

To retire keys, you should just move the old key to the decryption keys map, and advertise the new encryption key ID.

The old key will already be in the decryption keys map because we had to use it previously to decrypt the objects right? So we actually just need to update the encryption key and encryption key id?

Yeah, I'll change the wording I see how it's confusing.

the latest key will be used to re-encrypt the column.

This gives me the impression that GOTRUE_SECURITY_DB_ENCRYPTION_ENCRYPTION_KEY is an array of keys but it is actually just 1 key right?

Yes, it's just one key.

Addressed the comments.