-
Notifications
You must be signed in to change notification settings - Fork 687
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New source creation uses Sequoia #6799
Comments
What is coming in this PR:
Honestly I've spent most of the time getting tests to pass, including:
Except some tests are failing with |
The missing session key issue seems to be with encrypting a message for multiple recipients. Here's a pure-Rust test case that fails: #[test]
fn test_multiple_recipients() {
// Generate 2 keys
let (public_key1, secret_key1, _fingerprint1) = generate_source_key_pair(
"correcthorsebatterystaple",
"foo1@example.org",
)
.unwrap();
let (public_key2, secret_key2, _fingerprint2) = generate_source_key_pair(
"correcthorsebatterystaple",
"foo2@example.org",
)
.unwrap();
let tmp = NamedTempFile::new().unwrap();
// Encrypt a message
encrypt_message(
vec![public_key1, public_key2],
"Rust is great 🦀".to_string(),
tmp.path().to_path_buf(),
)
.unwrap();
let ciphertext = std::fs::read_to_string(tmp.path()).unwrap();
// Verify ciphertext looks like an encrypted message
assert!(ciphertext.starts_with("-----BEGIN PGP MESSAGE-----\n"));
// Decrypt as key 1
let plaintext = decrypt(
ciphertext.clone().into_bytes(),
secret_key1,
"correcthorsebatterystaple".to_string(),
)
.unwrap();
// Verify message is what we put in originally
assert_eq!("Rust is great 🦀", &plaintext);
// Decrypt as key 2
let plaintext = decrypt(
ciphertext.into_bytes(),
secret_key2,
"correcthorsebatterystaple".to_string(),
)
.unwrap(); // <-- fails here
// Verify message is what we put in originally
assert_eq!("Rust is great 🦀", &plaintext);
} |
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
In summary, this change implements: * Newly created sources have a Sequoia-generated key pair stored in the database, not in the GPG keyring. * All message and file encryption functions are handled by Sequoia. The journalist public key is read off disk instead of the GPG keyring. * Decryption of journalist messages for legacy GPG sources is still done through pretty_bad_protocol since the secret key is still in the GPG keyring. == EncryptionManager == The journalist public key is now read out of the SECUREDROP_DATA_ROOT folder instead of from the keyring, so we no longer need to know the specific fingerprint. Key generation is invoked directly in source_user.create_source(), so generate_source_key_pair() is removed, though it lives on in tests that continue to verify GPG-based source behavior. Since we can easily get a source's public key out of the GPG keyring, all of the encryption functions now use Sequoia. Functionally this is inefficient since we need to pull the key out of GPG and then pass it to Sequoia, but the next phase of the transition will move the public keys into the database, which will make this more efficient. == Missing keypairs == It's generally wrong to mutate state during GET requests, so move the step where we'd generate a keypair on /lookup requests to right after a successful login. This necessitated refactoring the flow of the login function to exit early in all failure cases so it's more obvious the new code only runs for a successful request. And this nicely leaves us a spot to fit in the migration of the secret key out of GPG (coming soon). == Tests == We're no longer able to mock the key length of generated keys since it's hardcoded in Rust, so the tests might overall be slower. We could probably pregenerate a set of keys and cycle through them over the entire test run, but that's left for a follow-up. The journalist public key is now copied into each test's unique data_root. It is no longer present in the GPG keyring. A number of tests also imported the journalist secret key, decrypted a submission, and then deleted the key from the keyring. A new utils.decrypt_as_journalist() helper function uses Sequoia to do all of that in a much simpler way. Some EncryptionManager tests were duplicated to use the create_legacy_gpg_key() helper, which generates a GPG key pair and deletes the Sequoia-generated keys to mimic what a pre-Sequoia source creation would be like. Tests that are explicitly testing GPG-based functionality should have `gpg` in their name, so it's easy to run them directly with `-k gpg`. Fixes #6799.
As part of our work to migrate to Sequoia, the first step is to have new source creation generate a PGP key pair using Sequoia, and then use those keys for encryption/decryption.
This work will include:
Most of this work already exists in the oxidize branch
The text was updated successfully, but these errors were encountered: