feat: Updates to the AWS Encryption SDK.

This change includes fixes for issues that were reported by Thai Duong from Google's Security team, and
for issues that were identified by AWS Cryptography.

See: https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/migration.html

Author: farleyb-amazon

CHANGELOG.rst

@@ -2,6 +2,11 @@
 Changelog
 *********
 
+1.7.0 -- 2020-09-24
+===================
+
+TODO
+
 1.4.1 -- 2019-09-20
 ===================
 

README.rst

@@ -92,76 +92,121 @@ uses a key derivation function, the data key is used to generate the key that di
 *****
 Usage
 *****
-To use this client, you (the caller) must provide an instance of either a master key provider
-or a CMM. The examples in this readme use the ``KMSMasterKeyProvider`` class.
 
-KMSMasterKeyProvider
-====================
-Because the ``KMSMasterKeyProvider`` uses the `boto3 SDK`_ to interact with `AWS KMS`_, it requires AWS Credentials.
-To provide these credentials, use the `standard means by which boto3 locates credentials`_ or provide a
-pre-existing instance of a ``botocore session`` to the ``KMSMasterKeyProvider``.
-This latter option can be useful if you have an alternate way to store your AWS credentials or
-you want to reuse an existing instance of a botocore session in order to decrease startup costs.
+EncryptionSDKClient
+===================
+To use this module, you (the caller) must first create an instance of the ``EncryptionSDKClient`` class.
+The constructor to this class requires a single keyword argument, ``commitment_policy``. There is
+currently only one valid value for this argument: ``FORBID_ENCRYPT_ALLOW_DECRYPT``.
 
 .. code:: python
 
     import aws_encryption_sdk
-    import botocore.session
+    from aws_encryption_sdk.identifiers import CommitmentPolicy
+
+
+    client = aws_encryption_sdk.EncryptionSDKClient(
+        commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT
+    )
+
 
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider()
+You must then create an instance of either a master key provider or a CMM. The examples in this
+readme use the ``StrictAwsKmsMasterKeyProvider`` class.
 
-    existing_botocore_session = botocore.session.Session()
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(botocore_session=existing_botocore_session)
 
+StrictAwsKmsMasterKeyProvider
+=============================
+A ``StrictAwsKmsMasterKeyProvider`` is configured with an explicit list of AWS KMS
+CMKs with which to encrypt and decrypt data. On encryption, it encrypts the plaintext with all
+configured CMKs. On decryption, it only attempts to decrypt ciphertexts that have been wrapped
+with one of the configured CMKs.
 
-You can pre-load the ``KMSMasterKeyProvider`` with one or more CMKs.
-To encrypt data, you must configure the ``KMSMasterKeyProvider`` with as least one CMK.
-If you configure the the ``KMSMasterKeyProvider`` with multiple CMKs, the `final message`_
+Because the ``StrictAwsKmsMasterKeyProvider`` uses the `boto3 SDK`_ to interact with `AWS KMS`_,
+it requires AWS Credentials.
+To provide these credentials, use the `standard means by which boto3 locates credentials`_ or provide a
+pre-existing instance of a ``botocore session`` to the ``StrictAwsKmsMasterKeyProvider``.
+This latter option can be useful if you have an alternate way to store your AWS credentials or
+you want to reuse an existing instance of a botocore session in order to decrease startup costs.
+
+To create a ``StrictAwsKmsMasterKeyProvider`` you must provide one or more CMKs.
+If you configure the the ``StrictAwsKmsMasterKeyProvider`` with multiple CMKs, the `final message`_
 will include a copy of the data key encrypted by each configured CMK.
 
 .. code:: python
 
     import aws_encryption_sdk
 
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
+    kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
         'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
         'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
     ])
 
-You can add CMKs from multiple regions to the ``KMSMasterKeyProvider``.
+You can add CMKs from multiple regions to the ``StrictAwsKmsMasterKeyProvider``.
 
 .. code:: python
 
     import aws_encryption_sdk
 
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
+    kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
         'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
         'arn:aws:kms:us-west-2:3333333333333:key/33333333-3333-3333-3333-333333333333',
         'arn:aws:kms:ap-northeast-1:4444444444444:key/44444444-4444-4444-4444-444444444444'
     ])
 
 
+DiscoveryAwsKmsMasterKeyProvider
+================================
+We recommend using a ``StrictAwsKmsMasterKeyProvider`` in order to ensure that you can only
+encrypt and decrypt data using the AWS KMS CMKs you expect. However, if you are unable to
+explicitly identify the AWS KMS CMKs that should be used for decryption, you can instead
+use a ``DiscoveryAwsKmsMasterKeyProvider`` for decryption operations. This provider
+attempts decryption of any ciphertexts as long as they match a ``DiscoveryFilter`` that
+you configure. A ``DiscoveryFilter`` consists of a list of AWS account ids and an AWS
+partition.
+
+.. code:: python
+
+    import aws_encryption_sdk
+    from aws_encryption_sdk.key_providers.kms import DiscoveryFilter
+
+    discovery_filter = DiscoveryFilter(
+        account_ids=['222222222222', '333333333333'],
+        partition='aws'
+    )
+    kms_key_provider = aws_encryption_sdk.DiscoveryAwsKmsMasterKeyProvider(
+        discovery_filter=discovery_filter
+    )
+
+If you do not want to filter the set of allowed accounts, you can also omit the ``discovery_filter`` argument.
+
+Note that a ``DiscoveryAwsKmsMasterKeyProvider`` cannot be used for encryption operations.
+
 Encryption and Decryption
 =========================
-After you create an instance of a ``MasterKeyProvider``, you can use either of the two
-high-level ``encrypt``/``decrypt`` functions to encrypt and decrypt your data.
+After you create an instance of an ``EncryptionSDKClient`` and a ``MasterKeyProvider``, you can use either of
+the client's two ``encrypt``/``decrypt`` functions to encrypt and decrypt your data.
 
 .. code:: python
 
     import aws_encryption_sdk
+    from aws_encryption_sdk.identifiers import CommitmentPolicy
 
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
+    client = aws_encryption_sdk.EncryptionSDKClient(
+        commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT
+    )
+
+    kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
         'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
         'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
     ])
     my_plaintext = b'This is some super secret data!  Yup, sure is!'
 
-    my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
+    my_ciphertext, encryptor_header = client.encrypt(
         source=my_plaintext,
         key_provider=kms_key_provider
     )
 
-    decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt(
+    decrypted_plaintext, decryptor_header = client.decrypt(
         source=my_ciphertext,
         key_provider=kms_key_provider
     )
@@ -174,14 +219,19 @@ You can provide an `encryption context`_: a form of additional authenticating in
 .. code:: python
 
     import aws_encryption_sdk
+    from aws_encryption_sdk.identifiers import CommitmentPolicy
+
+    client = aws_encryption_sdk.EncryptionSDKClient(
+        commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT
+    )
 
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
+    kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
         'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
         'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
     ])
     my_plaintext = b'This is some super secret data!  Yup, sure is!'
 
-    my_ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
+    my_ciphertext, encryptor_header = client.encrypt(
         source=my_plaintext,
         key_provider=kms_key_provider,
         encryption_context={
@@ -190,7 +240,7 @@ You can provide an `encryption context`_: a form of additional authenticating in
         }
     )
 
-    decrypted_plaintext, decryptor_header = aws_encryption_sdk.decrypt(
+    decrypted_plaintext, decryptor_header = client.decrypt(
         source=my_ciphertext,
         key_provider=kms_key_provider
     )
@@ -209,17 +259,22 @@ offering context manager and iteration support.
 .. code:: python
 
     import aws_encryption_sdk
+    from aws_encryption_sdk.identifiers import CommitmentPolicy
     import filecmp
 
-    kms_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[
+    client = aws_encryption_sdk.EncryptionSDKClient(
+        commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT
+    )
+
+    kms_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[
         'arn:aws:kms:us-east-1:2222222222222:key/22222222-2222-2222-2222-222222222222',
         'arn:aws:kms:us-east-1:3333333333333:key/33333333-3333-3333-3333-333333333333'
     ])
     plaintext_filename = 'my-secret-data.dat'
     ciphertext_filename = 'my-encrypted-data.ct'
 
     with open(plaintext_filename, 'rb') as pt_file, open(ciphertext_filename, 'wb') as ct_file:
-        with aws_encryption_sdk.stream(
+        with client.stream(
             mode='e',
             source=pt_file,
             key_provider=kms_key_provider
@@ -230,7 +285,7 @@ offering context manager and iteration support.
     new_plaintext_filename = 'my-decrypted-data.dat'
 
     with open(ciphertext_filename, 'rb') as ct_file, open(new_plaintext_filename, 'wb') as pt_file:
-        with aws_encryption_sdk.stream(
+        with client.stream(
             mode='d',
             source=ct_file,
             key_provider=kms_key_provider

decrypt_oracle/test/integration/integration_test_utils.py

@@ -77,8 +77,7 @@ def kms_master_key_provider(cache: Optional[bool] = True):
         return _KMS_MKP
 
     cmk_arn = get_cmk_arn()
-    _kms_master_key_provider = KMSMasterKeyProvider()
-    _kms_master_key_provider.add_master_key(cmk_arn)
+    _kms_master_key_provider = KMSMasterKeyProvider(key_ids=[cmk_arn])
 
     if cache:
         _KMS_MKP = _kms_master_key_provider

examples/src/basic_encryption.py

@@ -12,6 +12,7 @@
 # language governing permissions and limitations under the License.
 """Example showing basic encryption and decryption of a value already in memory."""
 import aws_encryption_sdk
+from aws_encryption_sdk import CommitmentPolicy
 
 
 def cycle_string(key_arn, source_plaintext, botocore_session=None):
@@ -22,17 +23,20 @@ def cycle_string(key_arn, source_plaintext, botocore_session=None):
     :param botocore_session: existing botocore session instance
     :type botocore_session: botocore.session.Session
     """
+    # Set up an encryption client with an explicit commitment policy
+    client = aws_encryption_sdk.EncryptionSDKClient(commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT)
+
     # Create a KMS master key provider
     kms_kwargs = dict(key_ids=[key_arn])
     if botocore_session is not None:
         kms_kwargs["botocore_session"] = botocore_session
-    master_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(**kms_kwargs)
+    master_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(**kms_kwargs)
 
     # Encrypt the plaintext source data
-    ciphertext, encryptor_header = aws_encryption_sdk.encrypt(source=source_plaintext, key_provider=master_key_provider)
+    ciphertext, encryptor_header = client.encrypt(source=source_plaintext, key_provider=master_key_provider)
 
     # Decrypt the ciphertext
-    cycled_plaintext, decrypted_header = aws_encryption_sdk.decrypt(source=ciphertext, key_provider=master_key_provider)
+    cycled_plaintext, decrypted_header = client.decrypt(source=ciphertext, key_provider=master_key_provider)
 
     # Verify that the "cycled" (encrypted, then decrypted) plaintext is identical to the source plaintext
     assert cycled_plaintext == source_plaintext

examples/src/basic_file_encryption_with_multiple_providers.py

@@ -22,7 +22,7 @@
 from cryptography.hazmat.primitives.asymmetric import rsa
 
 import aws_encryption_sdk
-from aws_encryption_sdk.identifiers import EncryptionKeyType, WrappingAlgorithm
+from aws_encryption_sdk.identifiers import CommitmentPolicy, EncryptionKeyType, WrappingAlgorithm
 from aws_encryption_sdk.internal.crypto.wrapping_keys import WrappingKey
 from aws_encryption_sdk.key_providers.raw import RawMasterKeyProvider
 
@@ -76,11 +76,14 @@ def cycle_file(key_arn, source_plaintext_filename, botocore_session=None):
     cycled_kms_plaintext_filename = source_plaintext_filename + ".kms.decrypted"
     cycled_static_plaintext_filename = source_plaintext_filename + ".static.decrypted"
 
+    # Set up an encryption client with an explicit commitment policy
+    client = aws_encryption_sdk.EncryptionSDKClient(commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT)
+
     # Create a KMS master key provider
     kms_kwargs = dict(key_ids=[key_arn])
     if botocore_session is not None:
         kms_kwargs["botocore_session"] = botocore_session
-    kms_master_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(**kms_kwargs)
+    kms_master_key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(**kms_kwargs)
 
     # Create a static master key provider and add a master key to it
     static_key_id = os.urandom(8)
@@ -94,23 +97,21 @@ def cycle_file(key_arn, source_plaintext_filename, botocore_session=None):
 
     # Encrypt plaintext with both AWS KMS and static master keys
     with open(source_plaintext_filename, "rb") as plaintext, open(ciphertext_filename, "wb") as ciphertext:
-        with aws_encryption_sdk.stream(source=plaintext, mode="e", key_provider=kms_master_key_provider) as encryptor:
+        with client.stream(source=plaintext, mode="e", key_provider=kms_master_key_provider) as encryptor:
             for chunk in encryptor:
                 ciphertext.write(chunk)
 
     # Decrypt the ciphertext with only the AWS KMS master key
     with open(ciphertext_filename, "rb") as ciphertext, open(cycled_kms_plaintext_filename, "wb") as plaintext:
-        with aws_encryption_sdk.stream(
-            source=ciphertext, mode="d", key_provider=aws_encryption_sdk.KMSMasterKeyProvider(**kms_kwargs)
+        with client.stream(
+            source=ciphertext, mode="d", key_provider=aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(**kms_kwargs)
         ) as kms_decryptor:
             for chunk in kms_decryptor:
                 plaintext.write(chunk)
 
     # Decrypt the ciphertext with only the static master key
     with open(ciphertext_filename, "rb") as ciphertext, open(cycled_static_plaintext_filename, "wb") as plaintext:
-        with aws_encryption_sdk.stream(
-            source=ciphertext, mode="d", key_provider=static_master_key_provider
-        ) as static_decryptor:
+        with client.stream(source=ciphertext, mode="d", key_provider=static_master_key_provider) as static_decryptor:
             for chunk in static_decryptor:
                 plaintext.write(chunk)
 

examples/src/basic_file_encryption_with_raw_key_provider.py

@@ -15,7 +15,7 @@
 import os
 
 import aws_encryption_sdk
-from aws_encryption_sdk.identifiers import EncryptionKeyType, WrappingAlgorithm
+from aws_encryption_sdk.identifiers import CommitmentPolicy, EncryptionKeyType, WrappingAlgorithm
 from aws_encryption_sdk.internal.crypto.wrapping_keys import WrappingKey
 from aws_encryption_sdk.key_providers.raw import RawMasterKeyProvider
 
@@ -53,6 +53,9 @@ def cycle_file(source_plaintext_filename):
 
     :param str source_plaintext_filename: Filename of file to encrypt
     """
+    # Set up an encryption client with an explicit commitment policy
+    client = aws_encryption_sdk.EncryptionSDKClient(commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT)
+
     # Create a static random master key provider
     key_id = os.urandom(8)
     master_key_provider = StaticRandomMasterKeyProvider()
@@ -63,13 +66,13 @@ def cycle_file(source_plaintext_filename):
 
     # Encrypt the plaintext source data
     with open(source_plaintext_filename, "rb") as plaintext, open(ciphertext_filename, "wb") as ciphertext:
-        with aws_encryption_sdk.stream(mode="e", source=plaintext, key_provider=master_key_provider) as encryptor:
+        with client.stream(mode="e", source=plaintext, key_provider=master_key_provider) as encryptor:
             for chunk in encryptor:
                 ciphertext.write(chunk)
 
     # Decrypt the ciphertext
     with open(ciphertext_filename, "rb") as ciphertext, open(cycled_plaintext_filename, "wb") as plaintext:
-        with aws_encryption_sdk.stream(mode="d", source=ciphertext, key_provider=master_key_provider) as decryptor:
+        with client.stream(mode="d", source=ciphertext, key_provider=master_key_provider) as decryptor:
             for chunk in decryptor:
                 plaintext.write(chunk)
 

examples/src/data_key_caching_basic.py

@@ -12,6 +12,7 @@
 # language governing permissions and limitations under the License.
 """Example of encryption with data key caching."""
 import aws_encryption_sdk
+from aws_encryption_sdk import CommitmentPolicy
 
 
 def encrypt_with_caching(kms_cmk_arn, max_age_in_cache, cache_capacity):
@@ -31,8 +32,11 @@ def encrypt_with_caching(kms_cmk_arn, max_age_in_cache, cache_capacity):
     # Create an encryption context
     encryption_context = {"purpose": "test"}
 
+    # Set up an encryption client with an explicit commitment policy
+    client = aws_encryption_sdk.EncryptionSDKClient(commitment_policy=CommitmentPolicy.FORBID_ENCRYPT_ALLOW_DECRYPT)
+
     # Create a master key provider for the KMS customer master key (CMK)
-    key_provider = aws_encryption_sdk.KMSMasterKeyProvider(key_ids=[kms_cmk_arn])
+    key_provider = aws_encryption_sdk.StrictAwsKmsMasterKeyProvider(key_ids=[kms_cmk_arn])
 
     # Create a local cache
     cache = aws_encryption_sdk.LocalCryptoMaterialsCache(cache_capacity)
@@ -48,7 +52,7 @@ def encrypt_with_caching(kms_cmk_arn, max_age_in_cache, cache_capacity):
     # When the call to encrypt data specifies a caching CMM,
     # the encryption operation uses the data key cache specified
     # in the caching CMM
-    encrypted_message, _header = aws_encryption_sdk.encrypt(
+    encrypted_message, _header = client.encrypt(
         source=my_data, materials_manager=caching_cmm, encryption_context=encryption_context
     )