Feature request: support DynamoDB image values to dict with built-in Python types

[shanab]

Use case

When using DynamoDBRecord class, the values for old_image and new_image are represented as Dict[str, AttributeValue]. This representation is sufficient when the data parsed is simple and does not contain nested maps/lists.
But when the data parsed contains nested maps/lists, then it becomes hard to get to the data that AttributeValue represents.
One would need to recursively parse the AttributeValue objects in the nested dicts to get to the underlying data.

Solution/User Experience

Using a deserializer similar to the TypeDeserializer provided by boto3 (link to code), one would be able to convert image to a Python dict where the values would be built-in Python types.

Alternative solutions

No response

Acknowledgment

• This feature request meets Lambda Powertools Tenets
• Should this be considered in other Lambda Powertools languages? i.e. Java, TypeScript

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.

[heitorlessa]

Thanks a lot for taking the time to raise it Ahmed. I can’t wait to see this PR.

…

On Sun, 7 Aug 2022 at 20:20, boring-cyborg[bot] ***@***.***> wrote: Thanks for opening your first issue here! We'll come back to you as soon as we can. — Reply to this email directly, view it on GitHub <#1432 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZPQBFDAIJWNCR57BOXDSTVX75ARANCNFSM5523LQSA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>

[ran-isenberg]

@shanab
did you try using the parser? you can specify models for the images and it all gets parsed nicely.
https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/parser/#built-in-envelopes
There's a DynamoDBStreamEnvelope class.

It seems like the data classes features is trying to implement parsing features IMO.

[shanab]

@ran-isenberg the problem in our case was that the service was not using Pydantic, and we did not want to introduce it only to parse the DynamoDB record.
I agree that there is an overlap between data classes and Parser, but IMO data classes should still allow traversing the fields of a record.
Also looking at the data classes docs, one of the key features mentioned is "Helper functions for decoding/deserializing nested fields"

[heitorlessa]

hey @shanab Ahmed, checking in to see if you still have the bandwidth to contribute as you already had the code working.

Thank you!

[shanab]

@heitorlessa yes, I should have time this week to have a first draft. Sorry for leaving this open until now!

[jplock]

What about using boto3's TypeDeserializer ?

[shanab]

What about using boto3's TypeDeserializer ?

That can be an option, but to my knowledge then one would not be able to do the following:

@event_source(data_class=DynamoDBStreamEvent)
def lambda_handler(event: DynamoDBStreamEvent, context):
    pass

because you'd need the raw Python dictionary for NewImage and OldImage to be able to use TypeDeserializer, and the event won't expose that.