Allow get_aws_credentials to assume_role_with_web_identity

[Qmando]

This change will allow the caller of get_aws_credentials to use assume_role_with_web_identity with a simple argument. This is ultimately to allow paasta spark-run to be used from within a Pod Identity context. The previous workaround was to create a AWS config file with a profile that did the same thing.

Pairs with Yelp/paasta#3787

Manual test, after setting ENV variables with a real token:
https://fluffy.yelpcorp.com/i/9nt6FK4T2dlSScKdVF53371flGRPS05F.html

[danielpops]

This case seems like it should raise instead of returning the (None, None, None) tuple and logging a warning. If you want it to use web_identity and it doesn't find the token or the role ARN, it will likely make it more difficult to diagnose why permission errors are happening (e.g. whomever is calling this may end up just defaulting to whatever standard resolution like the instance profile

[88manpreet]

We are soon going to migration our spark jobs onto driver on k8s. Which means:

• Tron jobs: Spark driver would run on k8s pod (not batch box)
• Adhoc jobs: Spark driver for now keeps running on given host. But in future it would be launched on k8s pod

Would the functional behavior of using the above flags (assume-aws-role or aws-profile or use-web-identity) remain the same?
cc'ing: @CaptainSame who can help you provide context and testing for driver on k8s.

[Qmando]

We are soon going to migration our spark jobs onto driver on k8s. Which means:

• Tron jobs: Spark driver would run on k8s pod (not batch box)
• Adhoc jobs: Spark driver for now keeps running on given host. But in future it would be launched on k8s pod

Would the functional behavior of using the above flags (assume-aws-role or aws-profile or use-web-identity) remain the same?

Seems like there's potentially two things that need to happen:

• include paasta_tools::spark_role_assumer in kube node puppet module
• Spark driver pod needs to mount /nail/etc/spark_role_assumer/
• Maybe a file permission change so the pod can access the file

[CaptainSame]

Is this meant to solve the adhoc paasta spark-run use-case where AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN env variables are present? Does the already implemented --assume-aws-role method have some limitations? Can you please explain the use-cases

[CaptainSame]

In our latest release with Spark drivers running on k8s, we have removed the usage of paasta spark-run for Spark batches running on tron in favor of a cleaner spark submit (The role assumption shenanigans are being done using a KUBECONFIG file). So I am not sure where this use-case fits

[nurdann]

Is this meant to solve the adhoc paasta spark-run use-case where AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN env variables are present? Does the already implemented --assume-aws-role method have some limitations? Can you please explain the use-cases

This is specifically to solve the issue of spark-run not picking up pod identity when running Jenkins pipeline. See for more details https://jira.yelpcorp.com/browse/SEC-18599

[CaptainSame]

Sure. This looks good to me