-
Notifications
You must be signed in to change notification settings - Fork 14.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move BaseOperatorLink
into the separate module
#35032
Conversation
The new packages should be added here: https://airflow.apache.org/docs/apache-airflow/stable/public-airflow-interface.html Also - are we going to update the providers with importing the new one ? Otherwise all operators will raise deprecation warning ? |
Community providers should not raise warnings, because all of them use lazy import from airflow.models |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. @uranusjr ?
Right. Then we don't even have to do the try/Except. BUT that made me think. Maybe this calls for NOT removing the provider pre-commit @Taragolis ? There is a reason why it is like that and if we remove the pre-commit, then providers MIGHT start diverging where they are importing it from. WDYT? |
Try/except might required only if #34600 completed. Some ideas why it might be a good idea (or maybe not)
We could ban Lines 151 to 153 in 303a329
|
Yep. We could. That would work. |
a240beb
to
ece538e
Compare
@@ -569,31 +569,6 @@ repos: | |||
entry: ./scripts/ci/pre_commit/pre_commit_sync_init_decorator.py | |||
pass_filenames: false | |||
files: ^airflow/models/dag\.py$|^airflow/(?:decorators|utils)/task_group\.py$ | |||
- id: check-base-operator-usage |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’m not sure if this should be removed outright. Instead we may want to check the two classes are imported from respectively correct modules.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for clarify which behavior we expect here
Option 1:
Outside of core airflow.models.baseoperator.BaseOperator
and airflow.models.baseoperatorlink.BaseOperatorLink
should be imported from airflow.models
Option 2:
Allow to use airflow.models.baseoperator.BaseOperator
and airflow.models.baseoperatorlink.BaseOperatorLink
Ban import airflow.models.baseoperator.BaseOperatorLink
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In core: Allow only airflow.models.baseoperator.BaseOperator
and airflow.models.baseoperatorlink.BaseOperatorLink
Outside core: Allow only airflow.models.BaseOperator
and airflow.models.BaseOperatorLink
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@uranusjr - Are you good with this one getting merged?
ece538e
to
7f5c6ea
Compare
7f5c6ea
to
1eb7aeb
Compare
static checks are failing |
1eb7aeb
to
99ec51f
Compare
99ec51f
to
d997655
Compare
It's all green and Jarek has approved. No reply from Tzu-ping in a few weeks but it does appear to me that his concerns were addressed. I think we can merge it?? |
Let's rebase first and have a look is is still working |
d997655
to
4d3ab0e
Compare
* Move `BaseOperatorLink` into the separate module * Add `airflow.models.baseoperatorlink` as part of Public Interface of Airflow * Ban `airflow.models.baseoperator.BaseOperatorLink` usage in codebase * Return back check-base-operator-usage pre-commit hooks
* Move `BaseOperatorLink` into the separate module * Add `airflow.models.baseoperatorlink` as part of Public Interface of Airflow * Ban `airflow.models.baseoperator.BaseOperatorLink` usage in codebase * Return back check-base-operator-usage pre-commit hooks
Move
BaseOperatorLink
fromairflow.models.baseoperator
into the separate module this might slightly reduce import in case of creation extra.In additional remove
check-base-operator-usage
pre-commit checks which are prevents direct use ofairflow.models.baseoperator.BaseOperatorLink
and forced contributors to useairflow.models.BaseOperatorLink
in providers and other parts^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.