Make Databricks operators' json parameter compatible with XComs, Jinja expression values #40471
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
boraberke
changed the title
potiuk
reviewed
Jun 28, 2024
potiuk
approved these changes
Jun 28, 2024
potiuk
approved these changes
Jul 2, 2024
52 tasks
Closed
2 tasks
pankajkoti
pushed a commit
that referenced
this pull request
Jul 18, 2024
pankajkoti
added a commit
to astronomer/airflow
that referenced
this pull request
Jul 26, 2024
…ms, Jinja expression values (apache#40471)" This reverts commit 4fb2140.
pankajkoti
added a commit
to astronomer/airflow
that referenced
this pull request
Jul 26, 2024
…ms, Jinja expression values (apache#40471)" This reverts commit 4fb2140.
romsharon98
pushed a commit
to romsharon98/airflow
that referenced
this pull request
Jul 26, 2024
…a expression values (apache#40471)
romsharon98
pushed a commit
to romsharon98/airflow
that referenced
this pull request
Jul 26, 2024
This PR fixes the many named parameters that was templated and was broken with apache#40471. The following operators are affected: DatabricksCreateJobsOperator DatabricksSubmitRunOperator DatabricksRunNowOperator closes: apache#40788
pankajkoti
added a commit
that referenced
this pull request
Jul 27, 2024
* Revert "Fix named parameters templating in Databricks operators (#40864)" This reverts commit cfe1d53. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (#40471)" This reverts commit 4fb2140. This reverts PR #40864 and PR #40471. Previously, PR #40471 was contributed to address issue #35433. However, that contribution gave rise to another issue #40788. Next #40788 was being attempted to be resolved in PR #40864. However, with the second PR, it appears that the previous old issue #35433 has [resurfaced](#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue #35433 and peacefully address it by taking the needed time.
molcay
pushed a commit
to VladaZakharova/airflow
that referenced
this pull request
Aug 19, 2024
…e#41050) * Revert "Fix named parameters templating in Databricks operators (apache#40864)" This reverts commit cfe1d53. * Revert "Make Databricks operators' json parameter compatible with XComs, Jinja expression values (apache#40471)" This reverts commit 4fb2140. This reverts PR apache#40864 and PR apache#40471. Previously, PR apache#40471 was contributed to address issue apache#35433. However, that contribution gave rise to another issue apache#40788. Next apache#40788 was being attempted to be resolved in PR apache#40864. However, with the second PR, it appears that the previous old issue apache#35433 has [resurfaced](apache#40864 (comment)). So, at the moment, the case is that we have 2 PRs on top of the existing implementation eventually having nil effect and the previous issues persists. I believe it is better to revert those 2 PRs, reopen the earlier issue apache#35433 and peacefully address it by taking the needed time.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR moves the assignment operations on the
jsontemplated field to theexecutefunction in the following operators:closes: #35433
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.