How to add versioning to continuous deployment model

[jabalsad]

Hi,

Uncertain where best to propose this question, but this seems like a good place to start. I've been playing with automatically deploying lambda functions using this serverless application model + cloudformation, based off this document [1]. One thing I've noticed is that with every change set that code pipeline creates in cloudformation, I don't get a new version of the lambda function. How do you propose I add some kind of versioning (perhaps correlated with the CodeUri) to the cloudformation change set?

I appreciate your input on the matter.

[1] http://docs.aws.amazon.com/lambda/latest/dg/automating-deployment.html

[sanathkr]

We are aware of this issue. SAM doesn't support Lambda function versioning yet.

[stuartstevenson]

I'm having the same issue after finallybgetting my pipeline working. Should we be able to use cfn native lambda support to version lambdas as a workaround?

[sanathkr]

@stuartstevenson yes you can use CFN native lambda resource as a workaround

[stuartstevenson]

Thank you for the response @sanathkr. I have just tested out a CFN template with a function, version and alias resource and when I updated the function the version remained the same. I realise now I have made some large and incorrect assumptions, namely that CFN has the same feature set and behaviour of the API, and the lambda version was doing some magic under the hood.

I'm pondering if it would not be too much effort to create a custom resource to check the state of the codesha against the last version, and if it varies then update

[sanathkr]

A Lambda version is an immutable resource which is created by AWS::Lambda::Version CFN resource type. To create a new Lambda version, you need to add a new AWS::Lambda::Version resource to your CFN template with a different resource name.

This behavior makes it hard for SAM to automatically create versions for you. SAM template should somehow automatically store all earlier version resources

[stuartstevenson]

This article published on the 20th is a nice example of how to build a full CD pipeline using SAM https://aws.amazon.com/blogs/compute/continuous-deployment-for-serverless-applications/. Interestingly it doesn't use versions or aliases. I've noticed that Serverless works in a similar way of creating an api and lambda per 'stage'. Is the aim with SAM to use Versions and Aliases instead of duplicating resources?

[sanathkr]

Versions & Aliases are powerful tools to help you build a production service on Lambda. SAM certainly wants to explore the possibility of supporting it

[lafiosca]

@jabalsad I ran into the same difficulty and resolved it by implementing custom actions in CodePipeline for versioning and aliasing the Lambda Functions that are created/updated by my CloudFormation changeset executions. I published the functions on GitHub but have not yet had time to add documentation or write up a blog post about them. In case they help you in the meantime:

https://github.com/qblu/codepipeline-lambda-versioner
https://github.com/qblu/codepipeline-lambda-aliaser

They both use this parent package which provides a generic way to build CodePipeline custom actions as Node functions:

https://github.com/qblu/codepipeline-custom-action

[jbankston]

@sanathkr I recently ran into this same issue. Is there an updated ETA on when versioning and aliases will be addressed in SAM? Thanks!

[aparnamane]

@sanathkr I am also looking versioning and alias in SAM, wondering does these functionalities are supported by SAM now

[lafiosca]

Seems like SAM could implement updating Lambda Versions and Aliases the same way it handles ApiGateway Deployments, by generating a new unique logical id each time.

[gabpaiz3]

Any ETA on the feature-request?

[sanathkr]

@lafiosca But that means earlier versions will be deleted, which is undesirable.

I don't have an ETA yet. We are working with CloudFormation to find a workaround to the problem I described earlier.

[dinvlad]

@sanathkr why would the old versions need to be deleted? Seems like we just publish a new version and then re-point the API resource to use these. I really like the approach described in https://stackoverflow.com/a/41455577/2962507 (1st option), similar to what @lafiosca described.

I.e. we DON'T use the inconvenient AWS::Lambda::Version resource. Rather, during transform:

1. we generate a "template version" parameter and set its default value to a new random GUID.
2. add a global function (one per template) that takes a function Arn and publishes a new version of that function. It also exports the new Version so that it can be used elsewhere in the template.
3. for each serverless function, we add a custom resource whose service token points to (2), and takes in the function Arn as well as the template version from (1). The reason to specify the latter is such that the custom resource is updated (and thus a new function version is published) every time the stack is updated.

Elsewhere in the template, we can refer both to the function Arn attribute, as well as the new Version attribute (which can actually be set to VersionArn for convenience). This way, we can support both the "old" behavior where we just point to function Arn (corresponding to the $LATEST version), and the "new" behavior where we refer to the versioned Arn. When the user uses the "new" behavior (where they point to VersionArn instead of Arn), API Gateway resource switches to the new version in production only at the very last step, when the stage is updated. Prior to that moment, all of our functions still point to the "old" version and continue working coherently during the stack update (my understanding is, currently SAM replaces the code for all of the functions first, and then updates the stage, which may introduce temporary incosistencies).

Are there any show stoppers for this? Seems like a simple enough approach that is both directly implementable using existing CloudFormation capabilities (e.g. custom resources), and doesn't break existing behavior.

[lafiosca]

@sanathkr I haven't thought this through fully yet, but would DeletionPolicy: Retain help with that?

[sanathkr]

@dinvlad Yeah, custom resources will solve the "I want to always point to latest version" problem. But if you want to track and point to multiple versions, the template definition becomes less intuitive. Instead of GUID, we can use Lambda function's CodeSHA if you want to auto-publish on every change.

[dinvlad]

Agreed, a code hash would be preferable. I actually ended up calculating and passing it to my template as a parameter.

[jbankston]

+1. The SHA-256 code hash is how others like Terraform calculate changes.

[ffxsam]

It's clunky, but I managed to solve this issue this way:

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Resources:
  SomeFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      FunctionName: !Sub "${AWS::StackName}-someFunc"
      Handler: "index.handler"
      CodeUri: "."
      Runtime: "nodejs6.10"
      MemorySize: 256
      Timeout: 5
      Environment:
        Variables:
          ANOTHER_FUNC_VERSION: !Sub "${AnotherFunctionV2.Version}"
      Policies:
      - AWSLambdaBasicExecutionRole
  AnotherFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      FunctionName: !Sub "${AWS::StackName}-anotherFunc"
      Handler: "magic.handler"
      CodeUri: "."
      Runtime: "nodejs6.10"
      MemorySize: 256
      Timeout: 5
      Policies:
      - AWSLambdaBasicExecutionRole
  AnotherFunctionV2:
    Type: "AWS::Lambda::Version"
    Properties:
      FunctionName: !Ref AnotherFunction

So in this scenario, SomeFunction might have some code that invokes AnotherFunction using a specific version. Any time I update the AnotherFunc code, I rename the AWS::Lambda::Version resource to the next version up, e.g. AnotherFunctionV3, then package and deploy.

[sanathkr]

This is now available in SAM. Checkout: https://github.com/awslabs/serverless-application-model/releases/tag/1.3.0

[ffxsam]

@sanathkr Thank you (and the team) for your hard work on this!! This is absolutely invaluable.

PS: Hope you didn't actually relocate to Alaska.. 😉

[sanathkr]

Lol we didn't :)

[lafiosca]

Kudos! Looking forward to exploring this stuff in the coming days!

[quarryman]

Guys, thanx for releasing versioning with SAM.

Can I have some help with what is the better way to add policies allowing Lambda versioning to corresponding role?

Simple adding AutoPublishAlias: live to function results in deploy fail due to role limitations to ListAliasesByFunction and other permissions to support versioning.

Do I get it right that SAM will not manage it automatically?
What is the better way to add this policy with SAM then assuming I have next CodeStar template for a function
Role: Fn::ImportValue: !Join ['-', [!Ref 'ProjectId', !Ref 'AWS::Region', 'LambdaTrustRole']]

[lmunro]

nicely done @sanathkr!

[Jun711]

@quarryman if u manually attach these roles to your CloudFormationWorker role, it will be able to work.
lambda:ListVersionsByFunction
lambda:CreateAlias
lambda:UpdateAlias
lambda:PublishVersion
lambda:DeleteAlias

[qmg-akara]

Hi, is it possible to set Lambda version DeletionPolicy to Delete when using AutoPublishAlias: live? It's now Retain in transformed template:

"FunctionVersione123abc456": {
  "Type": "AWS::Lambda::Version",
  "DeletionPolicy": "Retain",
  "Properties": {
    "FunctionName": {
      "Ref": "Function"
    }
  }
},

Something like VersionDeletionPolicy: Delete would be great.
Thanks!