feat(parser): support for CloudFormation Custom Resources

[ran-isenberg]

Issue number: #2295

Summary

Changes

• added 3 parser models: create/update/delete custom resource event source,
• added 3 json test files,
• added tests and added documentation of models.

User experience

from aws_lambda_powertools.utilities.parser import event_parser
from aws_lambda_powertools.utilities.parser.models import (
    CloudFormationCustomResourceCreateModel,
    CloudFormationCustomResourceDeleteModel,
    CloudFormationCustomResourceUpdateModel,
)
from aws_lambda_powertools.utilities.typing import LambdaContext


@event_parser(model=CloudFormationCustomResourceCreateModel)
def handle_create_custom_resource(event: CustomResourceCreateModel, _: LambdaContext):
    assert event.request_type == "Create"

@event_parser(model=CloudFormationCustomResourceUpdateModel)
def handle_update_custom_resource(event: CustomResourceUpdateModel, _: LambdaContext):
    assert event.request_type == "Update"

@event_parser(model=CloudFormationCustomResourceDeleteModel)
def handle_delete_custom_resource(event: CustomResourceDeleteModel, _: LambdaContext):
    assert event.request_type == "Delete"

Using extended model:

class MyModel(BaseModel):
    MyProps: str


class MyCustomResource(CloudFormationCustomResourceCreateModel):
    resource_properties: MyModel = Field(..., alias="ResourceProperties")


@event_parser(model=MyCustomResource)
def handle_create_custom_resource_extended_model(event: MyCustomResource, _: LambdaContext):
    assert event.request_type == "Create"
    assert event.resource_properties.MyProps == "ss"

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

No. **RFC issue number**:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[ran-isenberg]

@heitorlessa @leandrodamascena pr is ready. let me know what you think #2335

[ran-isenberg]

@heitorlessa @leandrodamascena fixed the code for older python support

[codecov-commenter]

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (ad6a85b) 97.16% compared to head (6be00f2) 97.16%.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #2335   +/-   ##
========================================
  Coverage    97.16%   97.16%           
========================================
  Files          154      155    +1     
  Lines         7047     7067   +20     
  Branches       515      515           
========================================
+ Hits          6847     6867   +20     
  Misses         157      157           
  Partials        43       43           

Impacted Files	Coverage Δ
...bda_powertools/utilities/parser/models/__init__.py	100.00% <100.00%> (ø)
...es/parser/models/cloudformation_custom_resource.py	100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[leandrodamascena]

Hi @ran-isenberg! Thanks for sending the PR and sorry for the late reply, I was attending a meetup in another country.

I'll look at it tomorrow; if all goes well, we can include it in the next release.

[heitorlessa]

Assigning to Leandro as he shared he's taking it @leandrodamascena

[leandrodamascena]

Hi @ran-isenberg! We need to make some updates before we merge this.

[leandrodamascena]

Hi @ran-isenberg! We need to make some updates before we merge this.

[ran-isenberg]

@leandrodamascena I've pushed your suggestions

[heitorlessa]

hey @ran-isenberg could you update the PR body with the new name so we don't forget?

Having a look at the rest now

[ran-isenberg]

hey @ran-isenberg could you update the PR body with the new name so we don't forget?

Having a look at the rest now

It seems older python code doesnt like the '|' instead of Union. what do you suggest?
I've updated the PR body

[heitorlessa]

You can import from future: from __future__ import annotations. That should work. Pydantic has support to handle postpone evaluation of annotations.

https://peps.python.org/pep-0563/

[ran-isenberg]

You can import from future: from __future__ import annotations. That should work. Pydantic has support to handle postpone evaluation of annotations.

https://peps.python.org/pep-0563/

Yes, I added that import, still no go. In other models we use Optional and Union which I think are more readable than the pipes '|'.
Not sure how to continue here.

[heitorlessa]

let me have a look ;) just finishing my last meeting of the day \o/

[heitorlessa]

I see what's happening, you've just hit a bug in the grammar, and they fixed it in 3.10 only :/ https://bugs.python.org/issue42233

I'll revert to Union and make any necessary changes to get this merged.

[ran-isenberg]

I see what's happening, you've just hit a bug in the grammar, and they fixed it in 3.10 only :/ https://bugs.python.org/issue42233

I'll revert to Union and make any necessary changes to get this merged.

oh wow, that's a nasty one.

[heitorlessa]

Ready to merge - these are the changes I've made. Waiting for CI to complete then merge.

---

• Rename test events to prefix CloudFormation (we wouldn't know otherwise in the future)
• Revert union type as | to Union due to bug not backported
• Docs: improved Model description
  • Before: Lambda Event Source payload for AWS CloudFormation 'create' custom resource
  • After: Lambda Event Source payload for AWS CloudFormation CREATE operation
• Rename test name to match model name (cloudformation_custom_resource)
• Fix Model type in resource_properties to allow model override (Type[BaseModel] -> BaseModel). Type[BaseModel] would be for envelopes only
• Refactor tests as unit tests - keeps tests focused on Model validation, less boilerplate, and allows us to update test data in the future
  • Get rid of event_parser, handle_ functions, and keep contained within the Model we're testing
  • Rename test to match our standard
  • Replace hard coded value with pointer to allow value changing in the future

Leandro is on PTO, heitorlessa is taking over.

[heitorlessa]

Thanks a lot @ran-isenberg for the contribution, much appreciated! The major change I've made that's worth pointing out is to refactor those tests as unit tests - here's the difference in a nutshell.

BEFORE

@event_parser(model=CloudFormationCustomResourceDeleteModel)
def handle_delete_custom_resource(event: CloudFormationCustomResourceDeleteModel, _: LambdaContext):
    assert event.request_type == "Delete"
    assert event.request_id == "xxxxx-d2a0-4dfb-ab1f-xxxxxx"
    assert event.service_token == "arn:aws:lambda:us-east-1:xxx:function:xxxx-CrbuiltinfunctionidProvi-2vKAalSppmKe"
    assert (
        str(event.response_url)
        == "https://cloudformation-custom-resource-response-useast1.s3.amazonaws.com/7F%7Cb1f50fdfc25f3b"
    )
    assert event.stack_id == "arn:aws:cloudformation:us-east-1:xxxx:stack/xxxx/271845b0-f2e8-11ed-90ac-0eeb25b8ae21"
    assert event.logical_resource_id == "xxxxxxxxx"
    assert event.resource_type == "Custom::MyType"
    assert event.resource_properties == {
        "ServiceToken": "arn:aws:lambda:us-east-1:xxxxx:function:xxxxx",
        "MyProps": "ss",
    }

def test_delete_trigger_event():
    event_dict = load_event("cloudformationCustomResourceDelete.json")
    handle_delete_custom_resource(event_dict, LambdaContext())

AFTER

def test_cloudformation_custom_resource_delete_event():
    raw_event = load_event("cloudformationCustomResourceDelete.json")
    model = CloudFormationCustomResourceDeleteModel(**raw_event)

    assert model.request_type == raw_event["RequestType"]
    assert model.request_id == raw_event["RequestId"]
    assert model.service_token == raw_event["ServiceToken"]
    assert str(model.response_url) == raw_event["ResponseURL"]
    assert model.stack_id == raw_event["StackId"]
    assert model.logical_resource_id == raw_event["LogicalResourceId"]
    assert model.resource_type == raw_event["ResourceType"]
    assert model.resource_properties == raw_event["ResourceProperties"]

This now focuses on the Model validation per se, no hard code data allowing us to have a schedule routine to generate real event to spot regressions in the future, and reduce boilerplate.

If we use the same idea and try to test the Create test with a Custom model, we now only have to test the model per se:

def test_cloudformation_custom_resource_create_event_custom_model():
    class MyModel(BaseModel):
        MyProps: str

    class MyCustomResource(CloudFormationCustomResourceCreateModel):
        resource_properties: MyModel = Field(..., alias="ResourceProperties")

    raw_event = load_event("cloudformationCustomResourceCreate.json")
    model = MyCustomResource(**raw_event)

    assert model.resource_properties.MyProps == raw_event["ResourceProperties"].get("MyProps")

[ran-isenberg]

@heitorlessa nice one, makes more sense as a unit test