Add cancel_previous_run to DatabricksRunNowOperator

[SubhamSinghal]

Closes: #38695

Adds cancel_previous_runs parameter which will cancel all running jobs before submitting a new one.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[Lee-W]

Overall, it looks great! could you please add a test case for this? Thanks

[SubhamSinghal]

@Lee-W Added UTs, PTAL

[Lee-W]

Could you please update the doc here as well? I forget to mention it yesterday 🤦‍♂️

[Lee-W]

I just noticed the test is failing. We might need your help to resolve it. Thanks!

[subham611]

@Lee-W Where can I see test failure logs?

[Lee-W]

@Lee-W Where can I see test failure logs?

Scroll down a bit. You should be able to find it here. I just approved the CI run. You can click the detail

[subham611]

Fixed UTs and tested locally

[Lee-W]

Thanks for the prompt fixings! There's one last static check failure and an improvement not directly related to this PR. You can check that locally with breeze static-check

breeze doc

[subham611]

@Lee-W All tests passed.

[pankajkoti]

cc: @alexott if you could please offer a review here if and when you have time :)

[SubhamSinghal]

@potiuk Please review this PR.

[potiuk]

@potiuk Please review this PR.

Why tagging me? @SubhamSinghal ? What's the reasoning of doing it ?

I think there are enough reviewers. generally please avoid tagging people directly unless you know for sure they will be insterested. Remember that people here participate when they have time and decide to spend their time on some issues - not because someone else decides they should do it. There are cases where people know and agreed upfront to be tagged, but tagging people like that out of the blue is generally a bad idea and try to avoid it, unless you really need it (and know people are interested).

[potiuk]

If you just want to get your PR merged faster, then it's a very selfish reason to tap into time and attention of others when YOU want it and not when THEY have time for it.

[Lee-W]

As many have approved this, I plan to merge it today or tomorrow. Please let me know if anyone want to take a deeper look 🙂

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.