Use Sequoia for newly created sources and all encryption

In summary, this change implements:
* Newly created sources have a Sequoia-generated key pair stored in the
database, not in the GPG keyring.
* All message and file encryption functions are handled by Sequoia. The
journalist public key is read off disk instead of the GPG keyring.
* Decryption of journalist messages for legacy GPG sources is still done
through pretty_bad_protocol since the secret key is still in the GPG
keyring.

== EncryptionManager ==

The journalist public key is now read out of the SECUREDROP_DATA_ROOT
folder instead of from the keyring, so we no longer need to know the
specific fingerprint.

Key generation is invoked directly in source_user.create_source(), so
generate_source_key_pair() is removed, though it lives on in tests that
continue to verify GPG-based source behavior.

Since we can easily get a source's public key out of the GPG keyring,
all of the encryption functions now use Sequoia. Functionally this is
inefficient since we need to pull the key out of GPG and then pass it to
Sequoia, but the next phase of the transition will move the public keys
into the database, which will make this more efficient.

== Missing keypairs ==

It's generally wrong to mutate state during GET requests, so move the
step where we'd generate a keypair on /lookup requests to right after a
successful login. This necessitated refactoring the flow of the login
function to exit early in all failure cases so it's more obvious the new
code only runs for a successful request.

And this nicely leaves us a spot to fit in the migration of the secret
key out of GPG (coming soon).

== Tests ==

We're no longer able to mock the key length of generated keys since it's
hardcoded in Rust, so the tests might overall be slower. We could
probably pregenerate a set of keys and cycle through them over the
entire test run, but that's left for a follow-up.

The journalist public key is now copied into each test's unique
data_root. It is no longer present in the GPG keyring. A number of tests
also imported the journalist secret key, decrypted a submission, and
then deleted the key from the keyring. A new
utils.decrypt_as_journalist() helper function uses Sequoia to do all of
that in a much simpler way.

Some EncryptionManager tests were duplicated to use the
create_legacy_gpg_key() helper, which generates a GPG key pair and
deletes the Sequoia-generated keys to mimic what a pre-Sequoia source
creation would be like. Tests that are explicitly testing GPG-based
functionality should have `gpg` in their name, so it's easy to run them
directly with `-k gpg`.

Fixes #6799.

securedrop/debian/securedrop-app-code.postinst

@@ -177,7 +177,9 @@ export_journalist_public_key() {
         # Based on sdconfig.py, this should work with very old config.py files.
         fingerprint=$(cd /var/www/securedrop; python3 -c "import config; print(config.JOURNALIST_KEY)")
         # Set up journalist.pub as root/www-data 640 before writing to it.
-        install --mode=640 --owner=root --group=www-data $journalist_pub
+        touch $journalist_pub
+        chown root:www-data $journalist_pub
+        chmod 640 $journalist_pub
         # FIXME: we should verify the exported key matches the fingerprint we wanted.
         # shellcheck disable=SC2024
         sudo -u www-data gpg2 --homedir=/var/lib/securedrop/keys --export --armor "$fingerprint" > $journalist_pub

securedrop/encryption.py

@@ -1,16 +1,18 @@
 import os
 import re
 import typing
-from datetime import date
-from io import BytesIO, StringIO
+from io import BytesIO
 from pathlib import Path
-from typing import Dict, List, Optional
+from typing import BinaryIO, Dict, Optional
 
 import pretty_bad_protocol as gnupg
 from redis import Redis
 from sdconfig import SecureDropConfig
 
+import redwood
+
 if typing.TYPE_CHECKING:
+    from models import Source
     from source_user import SourceUser
 
 # To fix https://github.com/freedomofpress/securedrop/issues/78
@@ -33,28 +35,20 @@ class GpgDecryptError(Exception):
 
 
 class EncryptionManager:
-
-    GPG_KEY_TYPE = "RSA"
-    GPG_KEY_LENGTH = 4096
-
-    # All reply keypairs will be "created" on the same day SecureDrop (then
-    # Strongbox) was publicly released for the first time.
-    # https://www.newyorker.com/news/news-desk/strongbox-and-aaron-swartz
-    DEFAULT_KEY_CREATION_DATE = date(2013, 5, 14)
-
-    # '0' is the magic value that tells GPG's batch key generation not
-    # to set an expiration date.
-    DEFAULT_KEY_EXPIRATION_DATE = "0"
+    """EncryptionManager provides a high-level interface for each PGP operation we do"""
 
     REDIS_FINGERPRINT_HASH = "sd/crypto-util/fingerprints"
     REDIS_KEY_HASH = "sd/crypto-util/keys"
 
-    SOURCE_KEY_NAME = "Source Key"
     SOURCE_KEY_UID_RE = re.compile(r"(Source|Autogenerated) Key <[-A-Za-z0-9+/=_]+>")
 
-    def __init__(self, gpg_key_dir: Path, journalist_key_fingerprint: str) -> None:
+    def __init__(self, gpg_key_dir: Path, journalist_pub_key: Path) -> None:
         self._gpg_key_dir = gpg_key_dir
-        self._journalist_key_fingerprint = journalist_key_fingerprint
+        self.journalist_pub_key = journalist_pub_key
+        if not self.journalist_pub_key.exists():
+            raise RuntimeError(
+                f"The journalist public key does not exist at {self.journalist_pub_key}"
+            )
         self._redis = Redis(decode_responses=True)
 
         # Instantiate the "main" GPG binary
@@ -71,43 +65,29 @@ def __init__(self, gpg_key_dir: Path, journalist_key_fingerprint: str) -> None:
             binary="gpg2", homedir=str(self._gpg_key_dir), options=["--yes", "--trust-model direct"]
         )
 
-        # Ensure that the journalist public key has been previously imported in GPG
-        try:
-            self.get_journalist_public_key()
-        except GpgKeyNotFoundError:
-            raise OSError(
-                f"The journalist public key with fingerprint {journalist_key_fingerprint}"
-                f" has not been imported into GPG."
-            )
-
     @classmethod
     def get_default(cls) -> "EncryptionManager":
         global _default_encryption_mgr
         if _default_encryption_mgr is None:
             config = SecureDropConfig.get_current()
             _default_encryption_mgr = cls(
                 gpg_key_dir=config.GPG_KEY_DIR,
-                journalist_key_fingerprint=config.JOURNALIST_KEY,
+                journalist_pub_key=(config.SECUREDROP_DATA_ROOT / "journalist.pub"),
             )
         return _default_encryption_mgr
 
-    def generate_source_key_pair(self, source_user: "SourceUser") -> None:
-        gen_key_input = self._gpg.gen_key_input(
-            passphrase=source_user.gpg_secret,
-            name_email=source_user.filesystem_id,
-            key_type=self.GPG_KEY_TYPE,
-            key_length=self.GPG_KEY_LENGTH,
-            name_real=self.SOURCE_KEY_NAME,
-            creation_date=self.DEFAULT_KEY_CREATION_DATE.isoformat(),
-            expire_date=self.DEFAULT_KEY_EXPIRATION_DATE,
-        )
-        new_key = self._gpg.gen_key(gen_key_input)
-
-        # Store the newly-created key's fingerprint in Redis for faster lookups
-        self._save_key_fingerprint_to_redis(source_user.filesystem_id, str(new_key))
-
     def delete_source_key_pair(self, source_filesystem_id: str) -> None:
-        source_key_fingerprint = self.get_source_key_fingerprint(source_filesystem_id)
+        """
+        Try to delete the source's key from the filesystem.  If it's not found, either:
+        (a) it doesn't exist or
+        (b) the source is Sequoia-based and has its key stored in Source.pgp_public_key,
+            which will be deleted when the Source instance itself is deleted.
+        """
+        try:
+            source_key_fingerprint = self.get_source_key_fingerprint(source_filesystem_id)
+        except GpgKeyNotFoundError:
+            # If the source is entirely Sequoia-based, there is nothing to delete
+            return
 
         # The subkeys keyword argument deletes both secret and public keys
         self._gpg_for_key_deletion.delete_keys(source_key_fingerprint, secret=True, subkeys=True)
@@ -116,7 +96,7 @@ def delete_source_key_pair(self, source_filesystem_id: str) -> None:
         self._redis.hdel(self.REDIS_FINGERPRINT_HASH, source_filesystem_id)
 
     def get_journalist_public_key(self) -> str:
-        return self._get_public_key(self._journalist_key_fingerprint)
+        return self.journalist_pub_key.read_text()
 
     def get_source_public_key(self, source_filesystem_id: str) -> str:
         source_key_fingerprint = self.get_source_key_fingerprint(source_filesystem_id)
@@ -134,63 +114,55 @@ def get_source_key_fingerprint(self, source_filesystem_id: str) -> str:
         return source_key_fingerprint
 
     def encrypt_source_message(self, message_in: str, encrypted_message_path_out: Path) -> None:
-        message_as_stream = StringIO(message_in)
-        self._encrypt(
+        redwood.encrypt_message(
             # A submission is only encrypted for the journalist key
-            using_keys_with_fingerprints=[self._journalist_key_fingerprint],
-            plaintext_in=message_as_stream,
-            ciphertext_path_out=encrypted_message_path_out,
+            recipients=[self.get_journalist_public_key()],
+            plaintext=message_in,
+            destination=encrypted_message_path_out,
         )
 
-    def encrypt_source_file(self, file_in: typing.IO, encrypted_file_path_out: Path) -> None:
-        self._encrypt(
+    def encrypt_source_file(self, file_in: BinaryIO, encrypted_file_path_out: Path) -> None:
+        redwood.encrypt_stream(
             # A submission is only encrypted for the journalist key
-            using_keys_with_fingerprints=[self._journalist_key_fingerprint],
-            plaintext_in=file_in,
-            ciphertext_path_out=encrypted_file_path_out,
+            recipients=[self.get_journalist_public_key()],
+            plaintext=file_in,
+            destination=encrypted_file_path_out,
         )
 
     def encrypt_journalist_reply(
-        self, for_source_with_filesystem_id: str, reply_in: str, encrypted_reply_path_out: Path
+        self, for_source: "Source", reply_in: str, encrypted_reply_path_out: Path
     ) -> None:
-        source_key_fingerprint = self.get_source_key_fingerprint(for_source_with_filesystem_id)
-        reply_as_stream = StringIO(reply_in)
-        self._encrypt(
+        redwood.encrypt_message(
             # A reply is encrypted for both the journalist key and the source key
-            using_keys_with_fingerprints=[source_key_fingerprint, self._journalist_key_fingerprint],
-            plaintext_in=reply_as_stream,
-            ciphertext_path_out=encrypted_reply_path_out,
+            recipients=[for_source.public_key, self.get_journalist_public_key()],
+            plaintext=reply_in,
+            destination=encrypted_reply_path_out,
         )
 
-    def decrypt_journalist_reply(self, for_source_user: "SourceUser", ciphertext_in: bytes) -> str:
+    def decrypt_journalist_reply(
+        self, for_source_user: "SourceUser", for_source: "Source", ciphertext_in: bytes
+    ) -> str:
+        """
+        Decrypt a reply sent by a journalist.
+
+        Note that `for_source_user` and `for_source` must point to the same source
+        """
+        if for_source_user.filesystem_id != for_source.filesystem_id:
+            raise ValueError("for_source_user and for_source filesystem_id's did not match!")
+        if for_source.pgp_secret_key is not None:
+            return redwood.decrypt(
+                ciphertext_in,
+                secret_key=for_source.pgp_secret_key,
+                passphrase=for_source_user.gpg_secret,
+            ).decode()
+        # TODO: Migrate gpg-stored secret keys to database storage for Sequoia
         ciphertext_as_stream = BytesIO(ciphertext_in)
         out = self._gpg.decrypt_file(ciphertext_as_stream, passphrase=for_source_user.gpg_secret)
         if not out.ok:
             raise GpgDecryptError(out.stderr)
 
         return out.data.decode("utf-8")
 
-    def _encrypt(
-        self,
-        using_keys_with_fingerprints: List[str],
-        plaintext_in: typing.IO,
-        ciphertext_path_out: Path,
-    ) -> None:
-        # Remove any spaces from provided fingerprints GPG outputs fingerprints
-        # with spaces for readability, but requires the spaces to be removed
-        # when using fingerprints to specify recipients.
-        sanitized_key_fingerprints = [fpr.replace(" ", "") for fpr in using_keys_with_fingerprints]
-
-        out = self._gpg.encrypt(
-            plaintext_in,
-            *sanitized_key_fingerprints,
-            output=str(ciphertext_path_out),
-            always_trust=True,
-            armor=False,
-        )
-        if not out.ok:
-            raise GpgEncryptError(out.stderr)
-
     def _get_source_key_details(self, source_filesystem_id: str) -> Dict[str, str]:
         for key in self._gpg.list_keys():
             for uid in key["uids"]:

securedrop/journalist_app/main.py

@@ -143,7 +143,7 @@ def reply() -> werkzeug.Response:
             g.source.interaction_count, g.source.journalist_filename
         )
         EncryptionManager.get_default().encrypt_journalist_reply(
-            for_source_with_filesystem_id=g.filesystem_id,
+            for_source=g.source,
             reply_in=form.message.data,
             encrypted_reply_path_out=Path(Storage.get_default().path(g.filesystem_id, filename)),
         )

securedrop/loaddata.py

@@ -221,7 +221,7 @@ def add_reply(
     record_source_interaction(source)
     fname = f"{source.interaction_count}-{source.journalist_filename}-reply.gpg"
     EncryptionManager.get_default().encrypt_journalist_reply(
-        for_source_with_filesystem_id=source.filesystem_id,
+        for_source=source,
         reply_in=next(replies),
         encrypted_reply_path_out=Path(Storage.get_default().path(source.filesystem_id, fname)),
     )
@@ -252,9 +252,6 @@ def add_source() -> Tuple[Source, str]:
     source = source_user.get_db_record()
     db.session.commit()
 
-    # Generate source key
-    EncryptionManager.get_default().generate_source_key_pair(source_user)
-
     return source, codename
 
 

securedrop/source_app/main.py

@@ -7,7 +7,7 @@
 import store
 import werkzeug
 from db import db
-from encryption import EncryptionManager, GpgKeyNotFoundError
+from encryption import EncryptionManager
 from flask import (
     Blueprint,
     abort,
@@ -43,6 +43,8 @@
 )
 from store import Storage
 
+import redwood
+
 
 def make_blueprint(config: SecureDropConfig) -> Blueprint:
     view = Blueprint("main", __name__)
@@ -161,7 +163,9 @@ def lookup(logged_in_source: SourceUser) -> str:
                 with open(reply_path, "rb") as f:
                     contents = f.read()
                 decrypted_reply = EncryptionManager.get_default().decrypt_journalist_reply(
-                    for_source_user=logged_in_source, ciphertext_in=contents
+                    for_source_user=logged_in_source,
+                    for_source=logged_in_source_in_db,
+                    ciphertext_in=contents,
                 )
                 reply.decrypted = decrypted_reply
             except UnicodeDecodeError:
@@ -175,13 +179,6 @@ def lookup(logged_in_source: SourceUser) -> str:
         # Sort the replies by date
         replies.sort(key=operator.attrgetter("date"), reverse=True)
 
-        # If not done yet, generate a keypair to encrypt replies from the journalist
-        encryption_mgr = EncryptionManager.get_default()
-        try:
-            encryption_mgr.get_source_public_key(logged_in_source.filesystem_id)
-        except GpgKeyNotFoundError:
-            encryption_mgr.generate_source_key_pair(logged_in_source)
-
         return render_template(
             "lookup.html",
             is_user_logged_in=True,
@@ -356,20 +353,33 @@ def batch_delete(logged_in_source: SourceUser) -> werkzeug.Response:
     @view.route("/login", methods=("GET", "POST"))
     def login() -> Union[str, werkzeug.Response]:
         form = LoginForm()
-        if form.validate_on_submit():
-            try:
-                SessionManager.log_user_in(
-                    db_session=db.session,
-                    supplied_passphrase=DicewarePassphrase(request.form["codename"].strip()),
-                )
-            except InvalidPassphraseError:
-                current_app.logger.info("Login failed for invalid codename")
-                flash_msg("error", None, gettext("Sorry, that is not a recognized codename."))
-            else:
-                # Success: a valid passphrase was supplied
-                return redirect(url_for(".lookup", from_login="1"))
-
-        return render_template("login.html", form=form)
+        if not form.validate_on_submit():
+            return render_template("login.html", form=form)
+        try:
+            source_user = SessionManager.log_user_in(
+                db_session=db.session,
+                supplied_passphrase=DicewarePassphrase(request.form["codename"].strip()),
+            )
+        except InvalidPassphraseError:
+            current_app.logger.info("Login failed for invalid codename")
+            flash_msg("error", None, gettext("Sorry, that is not a recognized codename."))
+            return render_template("login.html", form=form)
+        # Success: a valid passphrase was supplied and the source was logged-in
+        source = source_user.get_db_record()
+        if source.fingerprint is None:
+            # This legacy source didn't have a PGP keypair generated yet,
+            # do it now.
+            public_key, secret_key, fingerprint = redwood.generate_source_key_pair(
+                source_user.gpg_secret, source_user.filesystem_id
+            )
+            source.pgp_public_key = public_key
+            source.pgp_secret_key = secret_key
+            source.pgp_fingerprint = fingerprint
+            db.session.add(source)
+            db.session.commit()
+        # TODO: Migrate GPG secret key here too
+
+        return redirect(url_for(".lookup", from_login="1"))
 
     @view.route("/logout")
     def logout() -> Union[str, werkzeug.Response]:

securedrop/source_user.py

@@ -12,6 +12,8 @@
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.orm import Session
 
+import redwood
+
 if TYPE_CHECKING:
     from passphrases import DicewarePassphrase
     from store import Storage
@@ -99,9 +101,18 @@ def create_source_user(
         # Could not generate a designation that is not already used
         raise SourceDesignationCollisionError()
 
+    # Generate PGP keys
+    public_key, secret_key, fingerprint = redwood.generate_source_key_pair(
+        gpg_secret, filesystem_id
+    )
+
     # Store the source in the DB
     source_db_record = models.Source(
-        filesystem_id=filesystem_id, journalist_designation=valid_designation
+        filesystem_id=filesystem_id,
+        journalist_designation=valid_designation,
+        public_key=public_key,
+        secret_key=secret_key,
+        fingerprint=fingerprint,
     )
     db_session.add(source_db_record)
     try:

securedrop/store.py

@@ -8,7 +8,7 @@
 from hashlib import sha256
 from pathlib import Path
 from tempfile import _TemporaryFileWrapper
-from typing import IO, List, Optional, Type, Union
+from typing import BinaryIO, List, Optional, Type, Union
 
 import rm
 from encryption import EncryptionManager
@@ -301,7 +301,7 @@ def save_file_submission(
         count: int,
         journalist_filename: str,
         filename: Optional[str],
-        stream: "IO[bytes]",
+        stream: BinaryIO,
     ) -> str:
 
         if filename is not None: