issue-6044: Updating Databricks API endpoints to appropriate versions

[jroachgolf84]

Description

As part of #59217, certain endpoints we improperly updated to version 2.2 when they, in fact, did not support that endpoint. This was the case for the .../sql/endpoints, which are called out in #60444. Each of the Databricks endpoints were researched to find the latest supported version. From there, the provider was appropriately updated.

• closes: Databricks endpoint does not exist api/2.2/sql/endpoints #60444
• related: issue-59189: Updating Databricks provider to point to 2.2/jobs/... endpoint #59217

Testing

The appropriate unit-tests have been updated. These were run using the command below.

breeze testing providers-tests --test-type "Providers[databricks]"

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

[rodigu]

watching this issue.
for now, are there any temporary fixes for it (e.g. pip installing a different version of the databricks operator)?

[jroachgolf84]

I believe that you should be able to roll back to a previous version of the apache-airflow-providers-databricks package.

[jroachgolf84]

@eladkal, @jscheffl, do you mind taking a peek at this PR?

[jscheffl]

Thanks for the (partly) reversion.

[aru-trackunit]

Thank you @jroachgolf84 for solving it quickly

[rodigu]

I believe that you should be able to roll back to a previous version of the apache-airflow-providers-databricks package.

rolling back to 7.8.1 with pip install --upgrade apache-airflow-providers-databricks==7.8.1 works.

thank you, @jroachgolf84 !