Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic import MySqlHook/PostgresHook in CloudSQLDatabaseHook #41009

Merged
merged 1 commit into from
Jul 26, 2024

Conversation

jhongy1994
Copy link
Contributor

closes: #40175

CloudSQLDatabaseHook: Dynamic imports MySqlHook/PostgresHook within get_database_hook method

Changes:

  • Implement dynamic imports within the get_database_hook method to import hooks only when needed.
  • Change the return type from PostgresHook | MySqlHook to BaseHook.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@boring-cyborg boring-cyborg bot added area:providers provider:google Google (including GCP) related issues labels Jul 24, 2024
@jhongy1994 jhongy1994 force-pushed the fix/unnecessary-imports-cloudsql branch from c978cc1 to 74548fd Compare July 24, 2024 18:24
Copy link
Contributor

@moiseenkov moiseenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good, but what's the reason for placing imports inside methods? Could we just keep them on the top?

@jhongy1994 jhongy1994 force-pushed the fix/unnecessary-imports-cloudsql branch from 74548fd to 8113c00 Compare July 25, 2024 14:19
@jhongy1994
Copy link
Contributor Author

jhongy1994 commented Jul 25, 2024

@moiseenkov Thank you for reviewing!

The issue #40175 : ModuleNotFoundError (ModuleNotFoundError: No module named 'airflow.providers.mysql') occurs when calling the cloud_sql Operator (e.g. CloudSQLExportInstanceOperator) even when mysql is not used.

By placing the module import inside the method, we ensure that the module is imported only when it is actually needed, thereby avoiding the import error when the module is not required.

The first screenshot shows what happens when the import is placed at the top, without the mysql module.
image

The second screenshot shows what happens when the import is placed inside the method.
image

@jhongy1994 jhongy1994 force-pushed the fix/unnecessary-imports-cloudsql branch from 8113c00 to 77f2246 Compare July 25, 2024 14:53
@jhongy1994 jhongy1994 force-pushed the fix/unnecessary-imports-cloudsql branch from 77f2246 to 3d9f4a3 Compare July 25, 2024 15:09
@jhongy1994 jhongy1994 force-pushed the fix/unnecessary-imports-cloudsql branch from 3d9f4a3 to 194d596 Compare July 25, 2024 15:32
Copy link
Contributor

@shahar1 shahar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to Apache AIrflow, it looks good!
I got a small question before merging :)

@shahar1 shahar1 merged commit 6e3bc84 into apache:main Jul 26, 2024
6 checks passed
Copy link

boring-cyborg bot commented Jul 26, 2024

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Google provider : Unnecessary imports for CloudSQL operators
3 participants