Feature request: `AppConfigProvider` to return the last valid value when the API returns empty value on subsequent calls

[dreamorosi]

Use case

When retrieving a configuration profile from AppConfig, customers need to be mindful of the poll interval and to the fact that AppConfig returns empty responses for every subsequent call made after a successful retrieval and before the remote config has changed.

To provide a better DX, and to align with the implementation made in Powertools for Python, we should store the last valid configuration and return it instead of returning a response if users call the API again and the main cache was expired.

Solution/User Experience

Whenever a valid configuration profile is retrieve, store the value in the provider. Then, if the API is polled before the configuration has been refreshed (which would return an empty response), return the stored value instead of the empty response.

Alternative solutions

No response

Acknowledgment

• This feature request meets Lambda Powertools Tenets
• Should this be considered in other Lambda Powertools languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[brnkrygs]

Started a fork to work on this, will PR as soon as I can!

[brnkrygs]

Looking at this closer (and trying a differing maxAge algorithm), it looks like expiration is behaving as expected.

Notably, I only saw this behavior for the AppConfigProvider. I am starting to suspect the behavior of GetLatestConfiguration again.

Specifically, if I issue multiple calls within the cache age, I get the expected results.

If I issue a call after the maxAge is finished (e.g. > 5 seconds by default), but within the age of the AppConfig session, I get empty results.

We may need additional caching logic inside the AppConfig provider to handle the empty GetLatestConfiguration response as documented here.

After a new cold start, I get values again.

[brnkrygs]

More testing is showing that Subsequent calls are pulling the (empty) value from cache at the right time, and falling back to AppConfig at the right time.

Investigating strategies for an internal secondary cache inside the AppConfigProvider. Curious how the python team is handling this... do they leave it up to clients to force a refresh if the value is empty?

Looks like they keep a local cache inside the AppConfig provider:
https://github.com/awslabs/aws-lambda-powertools-python/blob/develop/aws_lambda_powertools/utilities/parameters/appconfig.py#L121

Side note... I'm a little concerned they don't scope that by the name parameter being passed to _get. Depending on the lifecycle of Provider instances, is this pattern at risk of having different configurations clobber each other?

[dreamorosi]

I think the clobbering behavior you're observing is exacerbated by the fact that the cache is short lived, and so each call always overwrite the previous value no matter what.

If we fix the cache behavior then it's up to users to tune the maxAge value in accordance with their expected polling frequency, while being aware of how AppConfig works (aka it returns an empty value when queried too often and the value hasn't changed).

As I mentioned in the PR review, I'm happy to make this behavior more explicit in the docs but I don't think we should mask this with another cache layer that always returns data when AppConfig stays empty. This to me, is an anti pattern as customers might be relying on the previously known behavior from AppConfig and handling empty values as "config has not changed". If we start returning data at each call, we might cause them to unnecessarily refresh their app/service for no reason.