Skip to content

Conversation

@dabla
Copy link
Contributor

@dabla dabla commented Aug 30, 2025

Fix sync to async issue when getting connection for SQLExecuteQueryTrigger. This PR fixes the same issue that was addressed for the MSGraphAsyncOperator.

closes: #54350


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@dabla
Copy link
Contributor Author

dabla commented Sep 2, 2025

@eladkal I've added unit test

@dabla dabla marked this pull request as draft September 2, 2025 10:13
@dabla dabla marked this pull request as ready for review September 2, 2025 12:30
@dabla
Copy link
Contributor Author

dabla commented Sep 2, 2025

@eladkal Do you prefer using asyncio.to_thread instead of sync_to_async from asgiref?

@dabla dabla requested a review from eladkal September 2, 2025 12:32
@potiuk
Copy link
Member

potiuk commented Sep 6, 2025

@eladkal Do you prefer using asyncio.to_thread instead of sync_to_async from asgiref?

I believe it's good - we are using sync_to_async in other places already.

@eladkal
Copy link
Contributor

eladkal commented Sep 10, 2025

@eladkal Do you prefer using asyncio.to_thread instead of sync_to_async from asgiref?

I believe it's good - we are using sync_to_async in other places already.

I am away from laptop so cant take a look at the code.

As long as we are not forcing installing async packages to the users who use only sync approch that is good enough for me.

@dabla
Copy link
Contributor Author

dabla commented Sep 12, 2025

@eladkal Do you prefer using asyncio.to_thread instead of sync_to_async from asgiref?

I believe it's good - we are using sync_to_async in other places already.

I am away from laptop so cant take a look at the code.

As long as we are not forcing installing async packages to the users who use only sync approch that is good enough for me.

Once providers are used in Airflow 3.1, they wont need the asgiref anymore as they would be able to use the async aget_connection method, thus the module is also only import if needed for fallback, otherwise it's not. As we have to keep providers backward compatible with Airflow 2, we need this fallback functionality.

@potiuk
Copy link
Member

potiuk commented Oct 11, 2025

Looks good. Side comment (as a potential follow-up) - maybe we should add common.compat for aget_connection and use it in all places instead of importing from asgiref.sync import sync_to_async -> that would also help with removing this compat when we have >= 3.1

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Nov 26, 2025
@dabla
Copy link
Contributor Author

dabla commented Nov 26, 2025

aget_connection

Do you mean a "compat" module in common provider which is then re-used accross all providers? This would be neat but would tightly couple other providers with common provider? Or do you mean just add it in version_compat of each provider, in this case sql provider?

@dabla dabla requested a review from potiuk November 26, 2025 08:08
@github-actions github-actions bot removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Nov 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MSGraphAsyncOperator fails to call API

3 participants