DatabricksSubmitRunOperator with deferrable=True doesn’t work in Airflow 3

[FrankPortman]

Apache Airflow Provider(s)

databricks

Versions of Apache Airflow Providers

Latest for databricks

Apache Airflow version

3.0.4

Operating System

Any

Deployment

Astronomer

Deployment details

N/A

What happened

RuntimeError: You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly.

Looks to be the same issue as #53447

What you think should happen instead

It should work. This codepath being completely incompatible with Airflow 3 is not documented or anticipated.

How to reproduce

Try to use DSRO in deferrable mode.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[vikrantkumar-max]

i like to work on this issue

[jroachgolf84]

@vikrantkumar-max, please feel free to open up a PR for this issue. Thanks!

[BasPH]

FYI in-progress PR: #55568

[FrankPortman]

@BasPH iiuc that PR will directly fix this, so we can link the ticket to the PR and make sure nobody else is working on it.

[FrankPortman]

@BasPH are you still working on that PR?

I see #55799 and #55812 are related?

[FrankPortman]

@kaxil is #55812 expected to fix this issue?

[kaxil]

#55799 should. Proper fix for Databricks in #55568