Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix(virtual-cluster): add missing Kubernetes RBAC permission
We are getting this error when executing an EMR job using the role and the role bindings installed in this module. Command: ``` aws emr-containers start-job-run \ --virtual-cluster-id "virtual-cluster-id" \ --name=test_emr \ --region "xxxxx" \ --execution-role-arn "arn:aws:iam::0000000000:role/execution-role" \ --release-label emr-7.0.0-latest \ --job-driver '{ "sparkSubmitJobDriver":{ "entryPoint": "s3://path_to_jar/jar_name.jar", "entryPointArguments":["argument_name_1","argument_1"], "sparkSubmitParameters": "--class MapperEntrypoint --conf spark.executor.instances=2 --conf spark.executor.memory=2G --conf spark.executor.cores=1 --conf spark.driver.cores=1" } }' \ --configuration-overrides '{ "applicationConfiguration": [ { "classification": "spark-defaults", "properties": { "spark.sql.extensions": "io.delta.sql.DeltaSparkSessionExtension", "spark.sql.catalog.spark_catalog":"org.apache.spark.sql.delta.catalog.DeltaCatalog", "spark.jars.packages": "com.amazonaws:aws-java-sdk-s3:1.12.401,com.amazonaws:aws-java-sdk-secretsmanager:1.12.401,org.apache.spark:spark-sql-kafka-0-10_2.12:3.3.2,io.delta:delta-spark_2.12:3.0.0" } } ], "monitoringConfiguration": { "persistentAppUI": "ENABLED", "s3MonitoringConfiguration": { "logUri": "s3://path_to_logs" } } }' ``` Error: ``` io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: DELETE at: https://kubernetes.default.svc/api/v1/namespaces/emr-production/persistentvolumeclaims?labelSelector=spark-app-selector%3Dspark-000000033c4hs4rueii. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. persistentvolumeclaims is forbidden: User "system:serviceaccount:emr-production:emr-containers-sa-spark-driver-702351764633-2paf8mkf46wdp1726c2315tio0bk0ult5j2i3dp304fcwq27hb2touevq1l3xggvbl1" cannot deletecollection resource "persistentvolumeclaims" in API group "" in the namespace "emr-production". ```
- Loading branch information