SAM deploy doesn't set environment variables

[chrisfosterelli]

Description

When using AWS SAM for local development, I can introduce environment variables by setting them in the template with no value, and then defining them in my environment. However, when I go to deploy, the environment variables do not appear to be inserted by sam package or sam deploy, and I get the following error on deploy:

Failed to create the changeset: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state Status: FAILED. Reason: [/Resources/GeneratePlan/Type/Environment/Variables/SECRET_ACCESS_KEY] 'null' values are not allowed in templates

Where SECRET_ACCESS_KEY is one of my environment variables. I cannot find any documentation detailing how to deploy projects with environment variables, either by having them defined in my environment or providing them in an alternate config.

I don't want to add the environment variables to my template.yml directly because this is stored in Git, and I don't want to edit them manually into the packaged.yml file each time between the package and deploy steps as that's cumbersome.

I haven't seen any steps in the documentation or similar issues, so I presume this is either an edge case bug or I am just missing something simple (in which case I might file this as a documentation bug) 😄

Steps to reproduce

The following config is a minimal excerpt from mine:

AWSTemplateFormatVersion: '2010-09-09'

Transform: AWS::Serverless-2016-10-31

Globals:

  Function:
    Environment:
      Variables:
        SECRET_ACCESS_KEY:

Resources:

  AnswerChallenge:
    Type: "AWS::Serverless::Function"
    Properties:
      Runtime: nodejs8.10
      Handler: build/functions/answerChallenge.default
      CodeUri: ./
      Policies: AmazonDynamoDBFullAccess
      Events:
        GetRequest:
          Type: Api
          Properties:
            Path: /event
            Method: get

Observed result

When I run sam start-api with SECRET_ACCESS_KEY defined, the lambda works as expected. When I attempt to deploy with sam package and sam deploy, I receive an error about undefined variables.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: MacOS
2. sam --version: 0.11

[jsonmaur]

Same issue here. sam package/sam deploy should accept the --env-vars flag.

My current workaround is to use CF parameters, since sam package/sam deploy are aliases for cloudformation package/cloudformation deploy.

template.yml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  SomeFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: some-function/
      Handler: index.handler
      Runtime: nodejs8.10
      Environment:
        Variables:
          SOME_VAR: !Ref SomeVar
Parameters:
  SomeVar:
    Type: String

$ sam package --template-file template.yml --s3-bucket BUCKET_NAME --output-template-file packaged.yml

$ sam deploy --template-file packaged.yml --stack-name STACK_NAME --capabilities CAPABILITY_IAM --parameter-overrides SomeVar=123

[chrisfosterelli]

@jsonmaur Thanks for sharing a workaround! Do you know if that is compatible with local development (i.e. start-api and start-lambda)?

[jsonmaur]

Works fine for me when I run sam local invoke --env-vars env.json. --env-vars seems to override the parameters.

[Marclev78]

Is there any response to this? I'm seeing the same thing.

sam package creates a YAML file containing my environment variables as expected, but after running sam deploy I can see that the Lambda configuration in the AWS console hasn't been updated to include them.

Trying to execute the Lambda then fails with the predictable "environment variable not defined" result.

[ivanmarques]

Hi!

Any updates on this issue?

Thanks!

[whereisaaron]

Wow does SAM make setting environment variables complicated!

@jsonmaur's workaround seems the best, using the (undocumented in sam docs) --parameter-overrides argument to sam deploy using the Name1=Value1 Name2=Value2 syntax.

sam build documents a --parameter-overrides argument, but it uses the the JSON shorthand syntax: ParameterKey=KeyPairName, ParameterValue=MyKey ParameterKey=InstanceType, ParameterValue=t1.micro .

So sam and cloudformation have multiple same-named arguments with entirely incompatible value syntax? Not helpful!

I tried using the sam build --parameter-overrides option but the parameters don't see to go into the build, and the values you specify for build are lost when you deploy. So not even sure what that is for?

[chrisfosterelli]

Yeah the sam local start-api also uses the mismatched format in --parameter-overrides, which is frustrating! Having to use two different config formats between local dev and prod deploy is not great... I had tried the --env-vars solution but had issues with the different cases expected between uppercase and snake case. This is what I had to do to get some resemblance of consistency:

I store my config file as a JSON file, such as config.json:

{
  "Region": "us-west-1",
  "Secret": "my-real-secret",
  [...]
}

And define them as parameters in the template.yaml:

Parameters:
  Secret:
    Type: String
  [...]

[... later ...]

Globals:
  Function:
    Environment:
      Variables:
        SECRET: !Ref Secret
        [...]

Then for development I have the following in package.json:

"scripts": {
  "api": "sam local start-api --region us-west-2 --parameter-overrides \"$(jq -r -j 'to_entries[] | \"ParameterKey=\\(.key),ParameterValue=\\(.value) \"' config.json)\"",
}

So I can run npm run api to get the api server going with the config file.

Then for deployment our CI runs:

> sam package --template-file ./template.yaml --s3-bucket bucketname --output-template-file ./packaged.yaml
> sam deploy --template-file ./packaged.yaml --stack-name stackname --capabilities CAPABILITY_IAM --parameter-overrides $(jq -r 'to_entries[] | "\(.key)=\(.value)"' config.json)

Surprising amount of work to get sane environment variables in SAM... I think this needs some developer experience TLC or maybe I'm missing something blatantly obvious 😅

[L-F-Escobar]

this is an issue. My Environment.variables just disappear upon command sam deploy --template-file packaged.yaml --region us-west-2 --capabilities CAPABILITY_IAM --stack-name lambda-stack-test

  LambdaStackFunction:
Type: AWS::Serverless::Function
Properties:
  FunctionName: LambdaStack-Local
  Timeout: 15
  Description: Local stack use to test -> deployed version
  CodeUri: lambda_stack/
  Handler: main.handler
  Role: !GetAtt LambdaExecutionRole.Arn
  Runtime: python3.7
  Layers:
    - arn:aws:lambda:us-west-xxxxxxxxxxxxxxx
    - arn:aws:lambda:us-west-xxxxxxxxxxxxxxx
Environment:
  variables:
    S3_ENDPOINT: https://s3.us-west-2.amazonaws.com
    SQS_ENDPOINT: https://sqs.us-west-2.amazonaws.com
    REGION: us-west-2
    TIMEZONE: US/Pacific

[whereisaaron]

@LEscobar-Driver capital ‘V’ in Variables

[L-F-Escobar]

@whereisaaron does nothing at all. Still the same

[awood45]

For local testing, you can override env variables in a few ways, including the --env-vars parameter. However, in the deploy step, you're just passing along the packaged template. For that, you must define all environment variables, either directly or as a parameter.

If this doesn't cover your needs, it could help to explain more about what you're trying to do.