Referencing Layer by parameter arn worked in Sam 0.19.0, after that it broke

[bmschwa]

Description

Had been using the most recent aws-sam-cli to build our projects. Version 0.21.0 & 0.20.0 have an issue when we refer to a layer param whereas version 0.19.0 did not.

Steps to reproduce

Using the following template with the command sam build --use-container --skip-pull-image --debug --template layer_ref_bug.yaml caused the error in versions 0.21, but not in 0.19.0

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  Demo Template; can I reproduce this issue?
  
Parameters:
  aLayer:
    Description: Some Layer
    Type: String
    Default: arn:aws:lambda:us-east-1:764866452798:layer:chrome-aws-lambda:4
Resources:
  aFunctionResource:
    Type: AWS::Serverless::Function
    Properties:
      Timeout: 900
      CodeUri: .
      Handler: resource.handler
      Runtime: python2.7
      Layers:
        - Ref: aLayer

Observed

(from a code deploy build)

Traceback (most recent call last): 
  File "/usr/local/bin/sam", line 10, in <module> 
    sys.exit(cli()) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__ 
    return self.main(*args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main 
    rv = self.invoke(ctx) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke 
    return _process_result(sub_ctx.command.invoke(sub_ctx)) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke 
    return ctx.invoke(self.callback, **ctx.params) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke 
    return callback(*args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func 
    return ctx.invoke(f, obj, *args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke 
    return callback(*args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 94, in wrapped 
    raise exception  # pylint: disable=raising-bad-type 
  File "/usr/local/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 65, in wrapped 
    return_value = func(*args, **kwargs) 
  File "/usr/local/lib/python3.7/site-packages/samcli/commands/build/command.py", line 105, in cli 
    skip_pull_image, parameter_overrides, mode)  # pragma: no cover 
  File "/usr/local/lib/python3.7/site-packages/samcli/commands/build/command.py", line 138, in do_cli 
    mode=mode) as ctx: 
  File "/usr/local/lib/python3.7/site-packages/samcli/commands/build/build_context.py", line 65, in __enter__ 
    self._function_provider = SamFunctionProvider(self._template_dict, self._parameter_overrides) 
  File "/usr/local/lib/python3.7/site-packages/samcli/commands/local/lib/sam_function_provider.py", line 52, in __init__ 
    self.functions = self._extract_functions(self.resources) 
  File "/usr/local/lib/python3.7/site-packages/samcli/commands/local/lib/sam_function_provider.py", line 100, in _extract_functions 
    layers = SamFunctionProvider._parse_layer_info(resource_properties.get("Layers", []), resources) 
  File "/usr/local/lib/python3.7/site-packages/samcli/commands/local/lib/sam_function_provider.py", line 263, in _parse_layer_info 
    raise InvalidLayerReference() 
samcli.commands.local.lib.exceptions.InvalidLayerReference: Layer References need to be of type 'AWS::Serverless::LayerVersion' or 'AWS::Lambda::LayerVersion' 
 

Expected result

Working with old version, not with the new one

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Windows 10 (using powershell)
2. Python 3.7.4
   3a. sam --version: (when it worked): SAM CLI, version 0.19.0
   3b, sam --version: (when it didn't work): SAM CLI, version 0.21.0

[sriram-mv]

We had our integration tests pass: https://github.com/awslabs/aws-sam-cli/blob/develop/tests/integration/testdata/invoke/layers/layer-template.yml#L5 (But we over-ride our parameters with parameter over-rides), do you see this if you override parameters?

[daveykane]

When I try referencing by a param I get the following error:

Error: Layer References need to be of type 'AWS::Serverless::LayerVersion' or 'AWS::Lambda::LayerVersion'

[bmschwa]

We had our integration tests pass: https://github.com/awslabs/aws-sam-cli/blob/develop/tests/integration/testdata/invoke/layers/layer-template.yml#L5 (But we over-ride our parameters with parameter over-rides), do you see this if you override parameters?

Sorry for the delay. I drafted a response, and guess I never hit send. Even when I use the parameter overrides, I see the same thing.

I tried using !Sub & Fn::Sub, suggested in 1182, but also did not resolve the issue.

[jfuss]

@bmschwa I took the same template above and it built without any issues. The main difference looks to be you are on windows and I am on Mac, but I don't think OS should have anything to do with this. How have you installed SAM CLI? Do you have multiple versions installed by chance?

You are able to repro with the template above right (locally not in CodeBuild)? Want to make sure we have good repro steps, otherwise this is nearly impossible to know why a path is getting executed when it shouldn't.

[daveykane]

@jfuss I am seeing the same issue and I'm running on Mac, I have SAM CLI 0.21.0 installed via brew. I also had an issue when using !Sub, it didn't error but it also did not download the Layer.

I tried a lot of different options but could only get the Layer loading with SAM CLI 2 ways:

1. Hard code the Arn
2. Reference a local LayerVersion resource but the contentUri had to be a directory not a zip (which was an issue for me due to the size limits deployment the stack) and I believe that has been raised here Layer not working locally due to not supporting zip file #947.

[jfuss]

@daveykane Can you provide a template and any additional details needed to repro?

For the second point above, that is a completely different thing than this and would prefer to keep this issue scoped to the initial issue raised.

[daveykane]

@jfuss I have a very large template so I've put together an example which is consistently reproducing the error for me. This is consistently throws Error: Layer References need to be of type 'AWS::Serverless::LayerVersion' or 'AWS::Lambda::LayerVersion' for me when starting the template with sam local start-api

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Layer Test
Parameters:
  LayerArn:
    Default: 'arn:aws:lambda:us-east-1:764866452798:layer:chrome-aws-lambda:4'
    Description: Layer Arn
    Type: String
Resources:
  FunctionWithLayer:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: .
      FunctionName: function-with-layer
      Handler: index.handler
      Runtime: nodejs10.x
      MemorySize: 512
      Timeout: 240
      Layers:
        - Ref: LayerArn
      VpcConfig:
        SecurityGroupIds:
        - Fn::ImportValue: !Sub ${VpcStackName}-AppSG
        SubnetIds:
        - Fn::ImportValue: !Sub ${VpcStackName}-AppSubnet1
        - Fn::ImportValue: !Sub ${VpcStackName}-AppSubnet2

Strangely if I commented out / remove the VpcConfig from this example the error disappears. Are you able to reproduce? If I remove the VpcConfig from my main template the error remains so I would guess there is some other setting, as well as the VpcConfig, in there which also causing an issue?

[jfuss]

@daveykane So what looks to be happening is the following (I stepped through this template with SAM CLI in debug mode):

• We run our 'resolve intrinsics' class on the template.
• Ref: LayerArn does get resolved to the default Parameter value
• We try to resolve the VpcConfig Property, which fails because we cannot resolve Fn::ImportValue.
• The FunctionWithLayer Resource remains unchanged
• We try and parse the values of Layers, which needs to be a arn and fail because we see a Ref instead.

We currently do not fail the resolving of intrinsics if we cannot resolve but I think leaving the whole resource unchanged is a bug. We should just set VpcConfig back to the original value and keep moving on, instead of the Resource. This also explains why our integ tests pass, because we don't have any additional values that are not resolvable.

Not sure how quick of a fix this is. There is lots of recursion I need to wrap my head around first.

[daveykane]

@jfuss that all makes sense, thanks for debugging. I have a workaround for now and will keep an eye out for the fix. Happy to do any testing you might need help with when a fix is ready.