-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Limit Python prek hooks to use Python 3.10 #58439
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
9db8f6e to
63c9887
Compare
63c9887 to
2f82106
Compare
2f82106 to
1deb31b
Compare
|
Can we go for py 3.13 instead for all these? |
No. Because our Python code base supports 3.10 so we have to go for lowest common denominator. |
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. Fixes: apache#56048
1deb31b to
2429ad9
Compare
For example the whole reason why things started to fail for the user in #56048 (look closely) - is that the code we were using in regular Airflow was using some things that were removed in 3.14. Now - if we allow people to run their check on 3.13 - and we have some code that does not work on 3.10 - it will work for them but it will not be python 3.10 compliant. So absolutely safest choice is to have everyone use the same as the lowest denominator for Airflow code - Python 3.10. Even if Python 3.13 works a bit faster and has better UX for developers (it does) - the "works for me" problem is far bigger than any of the gains. In this case - optimisation goal is not for this to be "better" - but to be "the same for everyone" (and compatible with the code we write for Airflow). |
|
And it has almost (besides a little speed) almost 0 impact for anyone to be perfectly honest, so I am not sure what we would gain from 3.13 support here:
So frankly, I do not see practically any benefit of having 3.13 there.. But maybe I missed something? |
|
Nope, nothing more than "py 3.13 runs 5-50% faster" but I agree with the reasons you listed, it's not worth it. (I was just thinking "most of the scripts are not really tied to what version of python we support, they just do things like read files or check AST etc" but most != all. |
jscheffl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the constructive and not heated discussion. I fully agree to arguments and outcome! LGTM.
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. (cherry picked from commit d31e578) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Fixes: #56048
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. (cherry picked from commit d31e578) Fixes: #56048 Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Heated ? Have you seen heated discussion between us 😄 ? |
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. (cherry picked from commit d31e578) Fixes: #56048 Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. (cherry picked from commit d31e578) Fixes: #56048 Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. (cherry picked from commit d31e578) Fixes: #56048 Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. Fixes: apache#56048
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. (cherry picked from commit d31e578) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Fixes: apache#56048
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. (cherry picked from commit d31e578) Fixes: #56048 Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
| # under the License. | ||
| # /// script | ||
| # requires-python = ">=3.10" | ||
| # # requires-python = ">=3.10,<3.11" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Noticed another one similar to #58439 (comment)
| # # requires-python = ">=3.10,<3.11" | |
| # requires-python = ">=3.10,<3.11" |
@potiuk I think this wasn't intentional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR #59047
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. Fixes: apache#56048
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14. Actually in order to have good stability, we are fine to limiting EVERYONE to use Python 3.10 for prek. There are no real benefits from using a different Python version (except maybe a little speed but prek-hooks are generally rather fast anyway) and we can definitely benefit from better stability and having exactly the same output for prek hooks by everyone. Uv will handle installation of the right python version automatically. Fixes: apache#56048
We have inline project metadata in the prek hooks, and that means that each of the scripts will use their own venv to roon. However Airflow code and libraries does not support Python 3.14 yet (and will not for quite a while - because our dependencies will need time to catch-up. In the meantime we have to limit prek hooks to not install Python 3.14.
Actually in order to have good stability, we are fine to limiting
EVERYONE to use Python 3.10 for prek. There are no real benefits from
using a different Python version (except maybe a little speed but
prek-hooks are generally rather fast anyway) and we can definitely
benefit from better stability and having exactly the same output
for prek hooks by everyone. Uv will handle installation of the
right python version automatically.
Fixes: #56048
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.