Build python from source for CI #52265
Conversation
Builds python from source, also installs golang from official distribution. Does both of these for the ci image only.
boring-cyborg
bot
added
area:dev-tools
backport-to-v3-1-test
potiuk
left a comment
potiuk
left a comment
There was a problem hiding this comment.
Looks great and is IMHO "fast enough" - enough to merge and try it even now in CI image
Adds support for using the airflow api to fetch the newest python patch version available for specific major_minor pair
potiuk
left a comment
potiuk
left a comment
There was a problem hiding this comment.
Really nice - with one caveat only - that we should add golang in a similar fashion as uv
|
VEEERY NICE! |
|
once we play with it in CI - we can do the next step and apply it to PROD images :) Looks really, really good - thanks to that our PROD image will have a chance to catch-up way faster with some 3rd-party CVEs. |
|
@aritra24 |
|
Versions are hard 🙃 🥲 |
Dependencies are harder 🙃 |
gopidesupavan
left a comment
gopidesupavan
left a comment
There was a problem hiding this comment.
@aritra24 Great work :) just one small nit everything fine. Thanks for doing this.
Indeed :) |
Updated python fetch request during upgrade to use github token and fixed the regex
|
Is the redis test failure something we're seeing in other prs as well? I don't think I've touched anything to break redis 🤔 |
Yeah - we have a flaky redis tests that we need to address. |
|
Let me merge it now - we can always iterate on details ;) |
Backport failed to create: v3-0-test. View the failure log Run details
You can attempt to backport this manually by running: cherry_picker d06a27e v3-0-testThis should apply the commit to the v3-0-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continue |
* Draft: Build python from source Builds python from source, also installs golang from official distribution. Does both of these for the ci image only. * Updates path * Adds version upgrade check for python version Adds support for using the airflow api to fetch the newest python patch version available for specific major_minor pair * Updated to use args in dockerfile for python * Added support for golang upgrade * Fixed go version sorting in pre_commit install * Added github token usage and fixed version regex Updated python fetch request during upgrade to use github token and fixed the regex * Updated dockerfile.ci file (cherry picked from commit d06a27e)
This reverts commit d06a27e.
|
Hmmm.. We need to revert .. I found some issues with other versions of python :( |
* Draft: Build python from source Builds python from source, also installs golang from official distribution. Does both of these for the ci image only. * Updates path * Adds version upgrade check for python version Adds support for using the airflow api to fetch the newest python patch version available for specific major_minor pair * Updated to use args in dockerfile for python * Added support for golang upgrade * Fixed go version sorting in pre_commit install * Added github token usage and fixed version regex Updated python fetch request during upgrade to use github token and fixed the regex * Updated dockerfile.ci file
This reverts commit d06a27e.
3 participants
closes: #52106
Builds python from source, also installs
golang from official distribution. Does
both of these for the ci image only.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.