Merging develop to sam-pipelines #3055

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

c2tarun merged 13 commits into sam-pipelines from develop

Jul 14, 2021

appveyor.yml

-Original file line number
+Diff line change
@@ Expand Up / @@ -159,7 +159,7 @@ for: @@
           # Pre-dev Tests
           - "pip install -e \".[pre-dev]\""
           - "pylint --rcfile .pylintrc samcli"
           # Dev Tests
           - "pip install -e \".[dev]\""
           - "pytest --cov samcli --cov-report term-missing --cov-fail-under 94 tests/unit"
@@ Expand All / @@ -170,7 +170,11 @@ for: @@
           # Runs only in Linux, logging docker hub when running canary and docker cred is available
           - sh: "
             if [[ -n $BY_CANARY ]] && [[ -n $DOCKER_USER ]] && [[ -n $DOCKER_PASS ]];
-              then echo Logging in Docker Hub; echo $DOCKER_PASS | docker login --username $DOCKER_USER --password-stdin;
+              then echo Logging in Docker Hub; echo $DOCKER_PASS | docker login --username $DOCKER_USER --password-stdin registry-1.docker.io;
+            fi"
+          - sh: "
+            if [[ -n $BY_CANARY ]];
+              then echo Logging in Public ECR; aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws;
             fi"
           - sh: "pytest -vv tests/integration"
           - sh: "pytest -vv tests/regression"
@@ Expand Down @@

requirements/base.txt

-Original file line number
+Diff line change
@@ Expand Up / @@ -6,7 +6,7 @@ boto3~=1.14 @@
     jmespath~=0.10.0
     PyYAML~=5.3
     cookiecutter~=1.7.2
-    aws-sam-translator==1.36.0
+    aws-sam-translator==1.37.0
     #docker minor version updates can include breaking changes. Auto update micro version only.
     docker~=4.2.0
     dateparser~=0.7
@@ Expand Down @@

requirements/reproducible-linux.txt

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -1,5 +1,5 @@
  
    #

    # This file is autogenerated by pip-compile

    # This file is autogenerated by pip-compile with python 3.7

    # To update, run:

    #

    #    pip-compile --allow-unsafe --generate-hashes --output-file=requirements/reproducible-linux.txt

    @@ -17,10 +17,10 @@ aws-lambda-builders==1.4.0 \
  
        --hash=sha256:5d4e4ecb3d3290f0eec1f62b7b0d9d6b91160ae71447d95899eede392d05f75f \

        --hash=sha256:d32f79cf67b189a7598793f69797f284b2eb9a9fada562175b1e854187f95aed

        # via aws-sam-cli (setup.py)

    aws-sam-translator==1.36.0 \

        --hash=sha256:4195ae8196f04803e7f0384a2b5ccd8c2b06ce0d8dc408aa1f1ce96c23bcf39d \

        --hash=sha256:f7d51b661fe1f5613a882f4733d1c92eff4dac36a076eafd18031d209b178695 \

        --hash=sha256:fa1b990d9329d19052e7b91cf0b19371ed9d31a529054b616005884cd662b584

    aws-sam-translator==1.37.0 \

        --hash=sha256:12cbf4af9e95acf73dabfbc44af990dc1e880f35697bb8c04f31b3bb90ab5526 \

        --hash=sha256:26e4866627e4284afc367bee2bd04d3cf23cecc8ff879b419457715a738395a9 \

        --hash=sha256:6884d942a815450637bac48e297996df2dacc27077d25ced09d8e9ce1f6a585c

        # via aws-sam-cli (setup.py)

    binaryornot==0.4.4 \

        --hash=sha256:359501dfc9d40632edc9fac890e19542db1a287bbcfa58175b66658392018061 \

    @@ -88,17 +88,17 @@ itsdangerous==1.1.0 \
  
        --hash=sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19 \

        --hash=sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749

        # via flask

    jinja2-time==0.2.0 \

        --hash=sha256:d14eaa4d315e7688daa4969f616f226614350c48730bfa1692d2caebd8c90d40 \

        --hash=sha256:d3eab6605e3ec8b7a0863df09cc1d23714908fa61aa6986a845c20ba488b4efa

        # via cookiecutter

    jinja2==2.11.3 \

        --hash=sha256:03e47ad063331dd6a3f04a43eddca8a966a26ba0c5b7207a9a9e4e08f1b29419 \

        --hash=sha256:a6d58433de0ae800347cab1fa3043cebbabe8baa9d29e668f1c768cb87a333c6

        # via

        #   cookiecutter

        #   flask

        #   jinja2-time

    jinja2-time==0.2.0 \

        --hash=sha256:d14eaa4d315e7688daa4969f616f226614350c48730bfa1692d2caebd8c90d40 \

        --hash=sha256:d3eab6605e3ec8b7a0863df09cc1d23714908fa61aa6986a845c20ba488b4efa

        # via cookiecutter

    jmespath==0.10.0 \

        --hash=sha256:b85d0567b8666149a93172712e68920734333c0ce7e89b78b3e987f71e5ed4f9 \

        --hash=sha256:cdf6525904cc597730141d61b36f2e4b8ecc257c420fa2f4549bac2c2d0cb72f

samcli/__init__.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -2,4 +2,4 @@ @@
     SAM CLI version
     """
-    __version__ = "1.24.1"
+    __version__ = "1.26.0"

samcli/commands/deploy/command.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -153,7 +153,9 @@ @@
         "--resolve-s3",
         required=False,
         is_flag=True,
-        help="Automatically resolve s3 bucket for non-guided deployments."
+        help="Automatically resolve s3 bucket for non-guided deployments. "
+        "Enabling this option will also create a managed default s3 bucket for you. "
+        "If you do not provide a --s3-bucket value, the managed bucket will be used. "
         "Do not use --s3-guided parameter with this option.",
     )
     @metadata_override_option
@@ Expand Down @@

samcli/commands/deploy/guided_context.py

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -316,7 +316,7 @@ def prompt_image_repository(self, stacks: List[Stack]):
  
                        if isinstance(self.image_repositories, dict)

                        else "" or self.image_repository,

                    )

                    if not is_ecr_url(image_repositories.get(resource_id)):

                    if resource_id not in image_repositories or not is_ecr_url(str(image_repositories[resource_id])):

                        raise GuidedDeployFailedError(

                            f"Invalid Image Repository ECR URI: {image_repositories.get(resource_id)}"

                        )

samcli/commands/package/command.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -121,7 +121,9 @@ def resources_and_properties_help_string(): @@
             exc_set=PackageResolveS3AndS3SetError,
             exc_not_set=PackageResolveS3AndS3NotSetError,
         ),
-        help="Automatically resolve s3 bucket for non-guided deployments."
+        help="Automatically resolve s3 bucket for non-guided deployments. "
+        "Enabling this option will also create a managed default s3 bucket for you. "
+        "If you do not provide a --s3-bucket value, the managed bucket will be used. "
         "Do not use --s3-guided parameter with this option.",
     )
     @metadata_override_option
@@ Expand Down @@

samcli/commands/package/exceptions.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -124,7 +124,8 @@ class BucketNotSpecifiedError(UserException): @@
         def __init__(self, **kwargs):
             self.kwargs = kwargs
-            message_fmt = "\nS3 Bucket not specified, use --s3-bucket to specify a bucket name or run sam deploy --guided"
+            message_fmt = "\nS3 Bucket not specified, use --s3-bucket to specify a bucket name, or use --resolve-s3 \
+    to create a managed default bucket, or run sam deploy --guided"
             super().__init__(message=message_fmt.format(**self.kwargs))
@@ Expand Down @@

samcli/lib/bootstrap/bootstrap.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -73,6 +73,9 @@ def _get_stack_template(): @@
                           - "/*"
                     Principal:
                       Service: serverlessrepo.amazonaws.com
+                    Condition:
+                      StringEquals:
+                        aws:SourceAccount: !Ref AWS::AccountId
         Outputs:
           SourceBucket:
@@ Expand Down @@

samcli/lib/build/build_strategy.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -5,6 +5,7 @@ @@
     import pathlib
     import shutil
     from abc import abstractmethod, ABC
+    from copy import deepcopy
     from typing import Callable, Dict, List, Any, Optional, cast
     from samcli.commands.build.exceptions import MissingBuildMethodException
@@ Expand Down Expand Up @@
             LOG.debug("Building to following folder %s", single_build_dir)
+            # we should create a copy and pass it down, otherwise additional env vars like LAMBDA_BUILDERS_LOG_LEVEL
+            # will make cache invalid all the time
+            container_env_vars = deepcopy(build_definition.env_vars)
             # when a function is passed here, it is ZIP function, codeuri and runtime are not None
             result = self._build_function(
                 build_definition.get_function_name(),
@@ Expand All @@
                 build_definition.get_handler_name(),
                 single_build_dir,
                 build_definition.metadata,
-                build_definition.env_vars,
+                container_env_vars,
             )
             function_build_results[single_full_path] = result
@@ Expand Down @@

samcli/lib/package/ecr_utils.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -6,5 +6,5 @@ @@
     from samcli.lib.package.regexpr import ECR_URL
-    def is_ecr_url(url):
+    def is_ecr_url(url: str) -> bool:
         return bool(re.match(ECR_URL, url)) if url else False

samcli/lib/package/s3_uploader.py

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -85,7 +85,7 @@ def upload(self, file_name: str, remote_path: str) -> str:
  
            # Check if a file with same data exists

            if not self.force_upload and self.file_exists(remote_path):

                LOG.debug("File with same data is already exists at %s. " "Skipping upload", remote_path)

                LOG.info("File with same data already exists at %s, skipping upload", remote_path)

                return self.make_url(remote_path)

            try:

samcli/lib/providers/sam_base_provider.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -10,6 +10,8 @@ @@
     from samcli.lib.intrinsic_resolver.intrinsics_symbol_table import IntrinsicsSymbolTable
     from samcli.lib.samlib.resource_metadata_normalizer import ResourceMetadataNormalizer
     from samcli.lib.samlib.wrapper import SamTranslatorWrapper
+    from samcli.lib.package.ecr_utils import is_ecr_url
     LOG = logging.getLogger(__name__)
@@ Expand All / @@ -34,6 +36,11 @@ class SamBaseProvider: @@
             SERVERLESS_LAYER: "ContentUri",
         }
+        IMAGE_PROPERTY_KEYS = {
+            LAMBDA_FUNCTION: "Code",
+            SERVERLESS_FUNCTION: "ImageUri",
+        }
         def get(self, name: str) -> Optional[Any]:
             """
             Given name of the function, this method must return the Function object
@@ Expand Down Expand Up @@
                 isinstance(location, str) and location.startswith("s3://")
             )
+        @staticmethod
+        def _is_ecr_uri(location: Optional[Union[str, Dict]]) -> bool:
+            """
+            the input could be:
+            - ImageUri of Serverless::Function
+            - Code of Lambda::Function
+            """
+            return location is not None and is_ecr_url(
+                str(location.get("ImageUri", "")) if isinstance(location, dict) else location
+            )
         @staticmethod
         def _warn_code_extraction(resource_type: str, resource_name: str, code_property: str) -> None:
             LOG.warning(
@@ Expand All @@
                 code_property,
             )
+        @staticmethod
+        def _warn_imageuri_extraction(resource_type: str, resource_name: str, image_property: str) -> None:
+            LOG.warning(
+                "The resource %s '%s' has specified ECR registry image for %s. "
+                "It will not be built and SAM CLI does not support invoking it locally.",
+                resource_type,
+                resource_name,
+                image_property,
+            )
         @staticmethod
         def _extract_lambda_function_imageuri(resource_properties: Dict, code_property_key: str) -> Optional[str]:
             """
@@ Expand Down @@

samcli/lib/providers/sam_function_provider.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -130,13 +130,28 @@ def _extract_functions( @@
                         resource_properties["Metadata"] = resource_metadata
                     if resource_type in [SamFunctionProvider.SERVERLESS_FUNCTION, SamFunctionProvider.LAMBDA_FUNCTION]:
+                        resource_package_type = resource_properties.get("PackageType", ZIP)
                         code_property_key = SamBaseProvider.CODE_PROPERTY_KEYS[resource_type]
-                        if SamBaseProvider._is_s3_location(resource_properties.get(code_property_key)):
+                        image_property_key = SamBaseProvider.IMAGE_PROPERTY_KEYS[resource_type]
+                        if resource_package_type == ZIP and SamBaseProvider._is_s3_location(
+                            resource_properties.get(code_property_key)
+                        ):
                             # CodeUri can be a dictionary of S3 Bucket/Key or a S3 URI, neither of which are supported
                             if not ignore_code_extraction_warnings:
                                 SamFunctionProvider._warn_code_extraction(resource_type, name, code_property_key)
                             continue
+                        if resource_package_type == IMAGE and SamBaseProvider._is_ecr_uri(
+                            resource_properties.get(image_property_key)
+                        ):
+                            # ImageUri can be an ECR uri, which is not supported
+                            if not ignore_code_extraction_warnings:
+                                SamFunctionProvider._warn_imageuri_extraction(resource_type, name, image_property_key)
+                            continue
                     if resource_type == SamFunctionProvider.SERVERLESS_FUNCTION:
                         layers = SamFunctionProvider._parse_layer_info(
                             stack,
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merging develop to sam-pipelines #3055

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!