gpg.mozilla.org is broken (probably for good)

[ajvb]

gpg.mozilla.org is busted at the moment and it is likely that it will stay this way. Mozilla has been wanting to retire it for a while, and it looks like it might happen here quickly.

Currently, this keyserver is hardcoded as the default for sops, and can only be changed using an env var (SOPS_GPG_KEYSERVER).

I think that in the short term, it might be best to switch to a different SKS server, with hkps.pool.sks-keyservers.net being the most likely candidate in my mind.

In the longer term, I'd want to consider some combination of:

1. Switching to age as our recommended default - Add support for age. #688
2. Supporting Keybase and recommending it as the default - Support key fetching from keybase.io #200

I don't think we should remove SKS-based key fetching, but we should consider moving away from it. The SKS keyserver network is not doing to well - https://code.firstlook.media/the-death-of-sks-pgp-keyservers-and-how-first-look-media-is-handling-it

[ajvb]

@autrilla What do you think about all of this?

[abeluck]

FWIW, one datapoint, we use sops extensively with PGP keys and have never used the keyserver fetching feature in sops. We always distribute team member's keys as part of onboarding. We're a team of less than 10, so this works.

[autrilla]

@autrilla What do you think about all of this?

I definitely agree with 1.

2 seems to be something a significant amount of people want, but #200 is also about PGP, and I don't think we should recommend using PGP keys, whether it's from Keybase or elsewhere.

I honestly don't remember why we added key fetching in the first place. I'm not sure I've personally used it. We could switch to hkps.pool.sks-keyservers.net and add a deprecation notice that's shown whenever it's used successfully for retrieving a key.

[ajvb]

2 seems to be something a significant amount of people want, but #200 is also about PGP, and I don't think we should recommend using PGP keys, whether it's from Keybase or elsewhere.

Agreed.

I honestly don't remember why we added key fetching in the first place. I'm not sure I've personally used it. We could switch to hkps.pool.sks-keyservers.net and add a deprecation notice that's shown whenever it's used successfully for retrieving a key.

As a Mozilla employee, it was definitely a nice feature with gpg.mozilla.org, as most folks had their key up there already. But after sleeping on this, I'm not really sure I want to point sops at a different SKS server.

In looking into various SKS servers, a lot of them seem to be sort of unmaintained or strange. For example, hkps.pool.sks-keyservers.net and keys.gnupg.net have a self-signed cert for HTTPS. I'm sure there is some historical reason for this, but I don't really want to deal with it if our real goal is to move away from PGP as the recommended default.

Alongside that, I don't really want to have to deal with this issue again if something goes wrong with some other SKS server.

I'm sure that similar to @abeluck there are a lot of users that don't even use this feature.

So yeah, I'd vote to either remove the feature completely or to switch to an SKS server (that supports functional HTTPS, like https://keyserver.ubuntu.com/) for a short period with a deprecation notice.

[ajvb]

@autrilla Lol nevermind on the Ubuntu SKS server...

$ for x in $(seq 1 5); \
  do http -h 'https://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0xFBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4' \
  | grep 'HTTP' && sleep 1; done
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
HTTP/1.1 200 OK

[ajvb]

In reading through SKS Keyserver Network Under Attack, the recommended keyserver for now seems to be https://keys.openpgp.org. It is not using the same SKS software, so using it will require dropping support for SOPS_GPG_KEYSERVER.

[bootc]

Perhaps one way around this would be to have an option to read keys from a file, or to include the key itself in the .sops.yaml file, and avoid having to fetch keys from anywhere?