Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CloudBatchSubmitJobOperator does not forward the project_id in the hook #40559

Closed
2 tasks done
guillaumeblaquiere opened this issue Jul 2, 2024 · 2 comments
Closed
2 tasks done
Assignees
Labels
area:providers good first issue kind:bug This is a clearly a bug provider:google Google (including GCP) related issues

Comments

@guillaumeblaquiere
Copy link
Contributor

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

All, even in the latest, there isn't the correct line of code.

Apache Airflow version

2.6.3

Operating System

composer-2.6.0

Deployment

Google Cloud Composer

Deployment details

Composer 2 version
composer-2.6.0-airflow-2.6.3

What happened

When using the CloudBatchSubmitJobOperator, the project_id is a required parameter. However, this parameter is ignored (not added in the hook submit_batch_job). Therefore, the runtime project (where Composer is deployed) is always taken into account and never the target project (where the real workload must be done)

What you think should happen instead

The project_id provided in the operator should be forwarded to the subsequent hook. All the other operation (get, list, delete) have this feature well implemented. Only the submit has this issue

How to reproduce

Create a Cloud Composer in a project, Configure a CloudBatchSubmitJobOperator, with an explicit project_id referencing an other project. The Cloud Batch deployment is made in the Cloud Composer project, and not in the other project

Anything else

I will submit the fix

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@guillaumeblaquiere guillaumeblaquiere added area:providers kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Jul 2, 2024
@dosubot dosubot bot added the provider:google Google (including GCP) related issues label Jul 2, 2024
@shahar1 shahar1 removed the needs-triage label for new issues that we didn't triage yet label Jul 2, 2024
@tsafacjo
Copy link

can I pick it ?

@eladkal
Copy link
Contributor

eladkal commented Aug 31, 2024

can I pick it ?

It was already fixed by #40560

@eladkal eladkal closed this as completed Aug 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers good first issue kind:bug This is a clearly a bug provider:google Google (including GCP) related issues
Projects
None yet
Development

No branches or pull requests

4 participants