-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Closed
Labels
area:providersgood first issuekind:bugThis is a clearly a bugThis is a clearly a bugprovider:common-sql
Description
Apache Airflow Provider(s)
common-sql
Versions of Apache Airflow Providers
apache-airflow = {extras = ["mysql", "postgres"], version = "^3.0.0"}
Running on python 3.11
Apache Airflow version
3.0.0
Operating System
Debian GNU/Linux 12 (bookworm)
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
What happened
The operator try to load data from the destination_hook instead of the source one.
It only appends when the parameter page_size is None.
What you think should happen instead
The source hook should be used [here ](
airflow/providers/common/sql/src/airflow/providers/common/sql/operators/generic_transfer.py
Line 165 in ed06d99
| results = self.destination_hook.get_records(self.sql) |
How to reproduce
use a operator as following operator use should lead to an error as far as the wrong hook is used
transfer_op = GenericTransfer(
task_id='transfer',
sql='SELECT * FROM source_table',
source_conn_id='source',
destination_conn_id='destination',
destination_table='target_table',
)Anything else
change proposal here
replace :
results = self.destination_hook.get_records(self.sql)by:
results = self.source_hook.get_records(self.sql)Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
area:providersgood first issuekind:bugThis is a clearly a bugThis is a clearly a bugprovider:common-sql