-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Made get_conn method async for KiotaRequestAdapterHook to fix AsyncToSync error #54598
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Made get_conn method async for KiotaRequestAdapterHook to fix AsyncToSync error #54598
Conversation
…x "An error occurred: You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly." in Airflow 3
providers/microsoft/azure/tests/unit/microsoft/azure/hooks/test_msgraph.py
Outdated
Show resolved
Hide resolved
|
@dabla thanks overall LGTM, some tests needs to be fixed. BTW this is complete breaking change for AF2 users or anyone using these hooks methods , because Offcourse this change must for this operator to work with AF3. |
|
cc: @eladkal FYI. |
…ype and test_throw_failed_responses_with_application_json_content_type
We could but normally this is an "internal" method, as user would normally not interact with it when using operator of even the hook as there you have the run method. WDYT as get_conn is expected to be async now. |
This is not good. I don't think we can break Airflow 2.10 comparability for this do we? |
I don't think it's broken for Airflow 2 as this is "internal kitchen stuff", I can test it to make sure in Airflow 2. |
If they directly interact with get_conn then yes it's would be broken, but that would be a bad practise normally you should use the run method. I can add a new get_async_conn method and keep the original for backward compatibility. |
I'll made get_conn synced again and create new method get_async_conn and deprecate the former one. |
I'm not into the details of this change. If this works on AF2 and we have reasonable claim to classify it as bug fix then this is fine. If mitigation steps or special instructions into what was changed are needed then we need a log entry to specify this change (a key indication to know if this is required is: if we expect that as a result of releasing this change workflows will behave differently) |
…t_conn method for backward compatibility
Well I just kept original method and added new async one like other providers do, it wasn't big of a deal after all, and so we are certain we don't break anything, @gopidesupavan was right by mentioning this. |
Cool |
|
Will adapt PowerBI related triggers also so it's aligned with MSGraph as they use the same code. |
|
@eladkal Should be ready now |
providers/microsoft/azure/src/airflow/providers/microsoft/azure/triggers/msgraph.py
Show resolved
Hide resolved
providers/microsoft/azure/src/airflow/providers/microsoft/azure/triggers/powerbi.py
Show resolved
Hide resolved
|
Thanks for the update, instead of removing the properties its better to throw depreciation warning. my only concern is all these are public facing methods or properties, not how users interact with these so its better with deprecation warning. WDYT? |
I've re-added get_conn with deprecation. |
eladkal
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
cc @gopidesupavan
gopidesupavan
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lovely thanks @dabla :)
…Sync error (apache#54598) * refactor: Made get_conn method of KiotaRequestAdapterHook async to fix "An error occurred: You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly." in Airflow 3 * refactor: Fixed static checks * refactor: Re-added missing import HttpxRequestAdapter * refactor: Reformatted imports * refactor: Fixed imports * refactor: Added missing AirflowProviderDeprecationWarning * refactor: Fixed test_serialize * refactor: Fixed test_throw_failed_responses_with_text_plain_content_type and test_throw_failed_responses_with_application_json_content_type * refactor: Renamed async get_conn method to get_async_conn in KiotaRequestAdapterHook * refactor: Introduced async get_async_conn and keep original synced get_conn method for backward compatibility * refactor: Introduced async get_async_conn and keep original synced get_conn method for backward compatibility * refactor: Refactor PowerBI triggers like MSGraph * refactor: Generate class names dynamically in PowerBI triggers * refactor: Fixed some mypy issues in PowerBITrigger * refactor: Fixed test_serialize * refactor: Reformatted test_get_conn * refactor: Added docstring to BasePowerBITrigger * refactor: Replaced DeprecationWarning with AirflowProviderDeprecationWarning * refactor: Fixed patching of 2 remaining tests * refactor: Re-added deprecated get_conn methods in triggers * refactor: Fixed static checks
This is a fix for "An error occurred: You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly." in Airflow 3, as get_conn should be async.
closes: #54598
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.