Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQueryInsertJobOperator broken #24017

Closed
2 tasks done
DrStriky opened this issue May 30, 2022 · 2 comments
Closed
2 tasks done

BigQueryInsertJobOperator broken #24017

DrStriky opened this issue May 30, 2022 · 2 comments
Labels
duplicate Issue that is duplicated

Comments

@DrStriky
Copy link

DrStriky commented May 30, 2022

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

7.0.0

Apache Airflow version

2.2.5

Operating System

google composer

Deployment

Composer

Deployment details

No response

What happened

BigQueryInsertJobOperator can not be used to run SQL query without specifying destination table explicitly in the operator

"query": {
                "query": f"""
                          INSERT INTO `{{{{params.project}}}}-{os.getenv('COMPOSER_ENVIRONMENT_STAGE')}.{{{{params.dataset}}}}.{{{{params.table_order_objects}}}}`
                          SELECT
                              *
                          FROM
                              `{{{{params.project}}}}-{os.getenv('COMPOSER_ENVIRONMENT_STAGE')}.{{{{params.dataset}}}}.{{{{params.table_order_objects}}}}_SZ`;
                          TRUNCATE TABLE `{{{{params.project}}}}-{os.getenv('COMPOSER_ENVIRONMENT_STAGE')}.{{{{params.dataset}}}}.{{{{params.table_order_objects}}}}_SZ`;
                          """,
                "useLegacySql": False,
            }

Traceback (most recent call last):
  File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/operators/bigquery.py", line 2269, in execute
    table = job.to_api_repr()["configuration"]["query"]["destinationTable"]

What you think should happen instead

To run a SQL a destination is not necessary to be specified

How to reproduce

Airflow 2.2.5 and airflow provider 7.0.0, BigQueryInsertJobOperator without specifying destinationTable

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@DrStriky DrStriky added area:providers kind:bug This is a clearly a bug labels May 30, 2022
@boring-cyborg
Copy link

boring-cyborg bot commented May 30, 2022

Thanks for opening your first issue here! Be sure to follow the issue template!

@eladkal
Copy link
Contributor

eladkal commented May 30, 2022

Duplicate #23826

@eladkal eladkal closed this as not planned Won't fix, can't repro, duplicate, stale May 30, 2022
@eladkal eladkal added duplicate Issue that is duplicated and removed kind:bug This is a clearly a bug area:providers labels May 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate Issue that is duplicated
Projects
None yet
Development

No branches or pull requests

2 participants