feat(parameters): Allow settings boto3.client() arguments

[gshpychka]

Is your feature request related to a problem? Please describe.
I'm using VPC endpoints with private DNS disabled, so I have to pass endpoint_url to boto3.client to use the endpoint-specific host name. I am not able to use parameters because this is not supported.

Describe the solution you'd like
Perhaps allowing to pass additional keyword arguments to the client method.

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.

[gshpychka]

It would be even more amazing if we could go one step further and allow to set these endpoint URLs via environment variables, so that I could use VPC endpoints with no code modification.

[ran-isenberg]

Hey, to my understanding this is already supported.
Take a look at https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/parameters/#passing-additional-sdk-arguments
or https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/parameters/#customizing-boto-configuration

[gshpychka]

Hey, to my understanding this is already supported. Take a look at https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/parameters/#passing-additional-sdk-arguments or https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/parameters/#customizing-boto-configuration

The former only allows me to pass arguments to the client's methods, and the latter allows me to modify the session, but none of these allow me to modify the client initialization.

[ran-isenberg]

look at the appconfig example
config = Config(region_name="us-west-1")
appconf_provider = parameters.AppConfigProvider(environment="my_env", application="my_app", config=config)

config is sent to the client initalization and points it to a specific region.

[gshpychka]

look at the appconfig example config = Config(region_name="us-west-1") appconf_provider = parameters.AppConfigProvider(environment="my_env", application="my_app", config=config)

config is sent to the client initalization and points it to a specific region.

Right, but the endpoint_url cannot be set with the Config object, it has to be passed as a separate argument to the client.

[gshpychka]

@ran-isenberg can you confirm that I'm not missing anything here?

[ran-isenberg]

@gshpychka I'm not familiar with this field BUT to my understand, unless you can set it via the config param:
https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
then you can't do.

I think you are right.
What do you use endpoint_url for if i may ask?

[gshpychka]

@gshpychka I'm not familiar with this field BUT to my understand, unless you can set it via the config param: https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html then you can't do.

I think you are right. What do you use endpoint_url for if i may ask?

Yes, exactly. As stated in the issue description, I'm using VPC endpoints with private DNS disabled, so I have to pass endpoint_url to boto3.client to use the endpoint-specific host name.

[ran-isenberg]

@heitorlessa I think we should allow people to send qwargs to the init function, not just config.

[heitorlessa]

Thanks everyone! The more I look into the Parameters utility the more I find we should've accepted a whole boto Client only. It would simplify our logic to maintain it, and also make it easier for customers to stub for unit testing, use snapshot testing with placebo, or in this case pass a different endpoint_url.

We won't be able to add this now due to our pause and bandwidth. I do believe there are other customers who will eventually be blocked by this too, so if anyone could create a PR to accept a Client I'll happily work with you to get it merged as an exception.

Note: I'm following this issue for boto to eventually add support to endpoint_url via environment variables as it's a common ask.

[ran-isenberg]

@heitorlessa A boto client in addition to the session or instead?

[heitorlessa]

It’d have to be in addition to not break customers using the boto3_session argument. If they pass a session, we create a client from it. If they pass a client, we use that instead.

…

On Tue, 5 Apr 2022 at 11:37, Ran Isenberg ***@***.***> wrote: @heitorlessa <https://github.com/heitorlessa> A boto client in addition to the session or instead? — Reply to this email directly, view it on GitHub <#1079 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZPQBBXCOAXVQ4RSFYBHZ3VDQCWVANCNFSM5Q7X65TQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ran-isenberg]

@heitorlessa FYI #1096

[heitorlessa]

Releasing it first thing tomorrow morning @gshpychka

[github-actions]

This is now released under 1.26.0 version!

[heitorlessa]

@gshpychka this is available on PyPi and SAR already - a new Lambda Layer will be available in a few hours. Docs are here: https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/parameters/#bring-your-own-boto-client

Please do feel free to reopen if this has not solved your problem. We took liberty to add for others to address a related testing challenge with Parameters too.