Add RHEL support to the python feature #830

jdputschadi · 2024-02-01T20:48:31Z

This PR adds RHEL support to the python feature.

CentOS 7, RHEL/Alma/Rocky Linux 8 & 9, Mariner, and Fedora should all work. Tests for all of these are included.

Note:

Currently the python feature only supports Debian/Ubuntu-based distributions with the apt based systems, after this change those systems will continue to work as will RedHat systems. There is user visible change for systems that are not explicity supported: they will get an error message stating "Linux distro XXXX not supported."

This is instead of "odd" messages about not being able to find apt-get on unsupported systems.

resolves #829

jdputschadi · 2024-02-01T21:26:47Z

I see the cause of the error in the "checks" and will update the submitted code to not fail.

samruddhikhandale · 2024-02-01T22:12:26Z

FYI - #577 is a known issue and tests against bookworm are currently failing. No pressure in fixing that, but it would be amazing if you could. ✨

… on Mariner systems

jdputschadi · 2024-02-02T05:00:54Z

Commit 4007701 should fix the previously failing checks/tests. If so, the python feature portions of #577 would also be resolved by this PR.

… assuming "python" will work

jdputschadi · 2024-02-02T05:30:53Z

Needed f20275f to finish addressing #577 for this the python feature.

jdputschadi · 2024-02-02T17:36:32Z

Darn it I see new failures -- serves me right for only testing the previous failures. I'll get the issue resolved.

managed. If so, add "--break-system-packages" to the pip install flags. This does not really breack system packages due to the setting of PYTHONUSERBASE during the install of pipx, but does get us past checks for installing python packages into the system python install.

samruddhikhandale · 2024-02-02T21:30:38Z

I don't see an option to run the tests, closing and opening this PR (attempt at fixing it)

samruddhikhandale · 2024-02-02T21:31:33Z

I don't see an option to run the tests, closing and opening this PR (attempt at fixing it)

Didn't help 😓

Not sure if ^ needs to be fixed.

jdputschadi · 2024-02-02T21:34:16Z

Samruddhi:

commit dad4f8d works for the base tests:

ubunto:focal
ubuntu:jammy
debian:11
debian:12
mcr.microsoft.com/devcontainers/base:debian
mcr.microsoft.com/devcontainers/base:ubunto

It should work on all the scenarios. I'm in the process of confirming that.

jdputschadi · 2024-02-02T21:38:18Z

Looks like I need to merge in a recent commit.

That may take a while, especially if I re-run tests on my end before pushing the result of the mrege.

jdputschadi · 2024-02-05T04:55:33Z

Commit 13a7f30 resolves test failures and passes all tests on my laptop. This should solve the known issues with the python feature outlined in #577.

jdputschadi · 2024-02-05T21:57:52Z

@samruddhikhandale

I don't understand the failure. Commit f9f1e7f contains code, from commit 4007701 , that is required for Mariner and also prevents the Mariner test case from failing due to lack of a "tk-devel" RPM package in Mariner:

        # Mariner does not have tk-devel package available, RedHat ubi8 and ubi9 do not have tk-devel
        if [ ${ID} != "mariner" ] && [ ${ID} != "rhel" ]; then
            REQUIRED_PKGS="${REQUIRED_PKGS} \
                tk-devel"
        fi

However the test log shows the mariner test failing for that reason (from the 0_test-scenarios\ \(python\).txt file in the downloaded logs):

2024-02-01T21:35:10.6118401Z 17.57 Running transaction
2024-02-01T21:35:10.6119075Z 17.59 Installing/Updating: mpfr-4.1.0-2.cm2.x86_64
2024-02-01T21:35:10.6119870Z 17.61 Installing/Updating: gawk-5.1.1-1.cm2.x86_64
2024-02-01T21:35:10.6127810Z 17.76 Running yum check-update ...
2024-02-01T21:35:10.6128493Z 17.77 Loaded plugin: tdnfrepogpgcheck
2024-02-01T21:35:10.6129084Z 17.87 Package findutils is already installed.
2024-02-01T21:35:10.6129689Z 17.87 Package gnupg2 is already installed.
2024-02-01T21:35:10.6130403Z 17.87 tk-devel package not found or not installed
2024-02-01T21:35:10.6131028Z 17.87 Error(1011) : No matching packages
2024-02-01T21:35:10.6132684Z 17.88 ERROR: Feature "Python" (Unknown) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/python for help troubleshooting this error.

The most recent test run acts as if it was run based of a commit associated with this PR that was earlier than 4007701.

I don't know how to get the tests to rerun. I suppose I could make an in-consequential change (add white-space somewhere) and push that change, but that doesn't seem right.

I'll follow any advice/direction you provide on this one.

samruddhikhandale · 2024-02-05T22:04:54Z

I don't know how to get the tests to rerun.

I triggered a rerun for failed action runs. A bit busy now, however, will take a peek at reviewing sometime later this afternoon.

samruddhikhandale · 2024-02-05T23:25:02Z

The tests failed this time due to 👇 As this test adds quite a bit of new tests, we are out of disk space. I think we'd need to configure powerful ACTIONS, i'd look into it.

2024-02-05T22:55:40.5897864Z [2024-02-05T22:55:40.589Z] #14 126.8 ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device

samruddhikhandale · 2024-02-05T23:28:10Z

@jdputschadi Can you update https://github.com/devcontainers/features/blob/main/.github/workflows/test-pr.yaml#L7 to 👇. Let's see if it fixes it.

runs-on: devcontainer-image-builder-ubuntu

samruddhikhandale

Super cool, left some thoughts. Thank you so much!!!

src/python/devcontainer-feature.json

src/python/install.sh

jdputschadi · 2024-02-06T20:01:45Z

@jdputschadi Can you update https://github.com/devcontainers/features/blob/main/.github/workflows/test-pr.yaml#L7 to 👇. Let's see if it fixes it.
runs-on: devcontainer-image-builder-ubuntu

@samruddhikhandale : this line is in multiple places in the test-pr.yaml file. Do you want all of them changed?

samruddhikhandale · 2024-02-06T20:48:31Z

https://github.com/devcontainers/features/blob/main/.github/workflows/test-pr.yaml#L7

Good point, let's only update for https://github.com/devcontainers/features/blob/main/.github/workflows/test-pr.yaml#L70 now.

jdputschadi · 2024-02-06T21:24:51Z

@samruddhikhandale : Before I push a commit addressing the above notes and conversations, I have one overarching question:

Do you want the packages option from #768 included in the push (i.e. merge the change into one PR).

samruddhikhandale · 2024-02-06T21:29:18Z

@samruddhikhandale : Before I push a commit addressing the above notes and conversations, I have one overarching question:

Do you want the packages option from #768 included in the push (i.e. merge the change into one PR).

I was mostly hinting at replacing toolsToInstall with packages, but I understand that it's not possible.

So feel free to include/exclude based on your convenience and confidence. This PR is already big, we can merge that into a separate one.

jdputschadi · 2024-02-06T21:56:40Z

@samruddhikhandale : I think I've got the next push ready and it should address all your input. I'm running all tests locally to make sure they pass before executing the push.

The packages option will be left to the other PR.

jdputschadi · 2024-02-07T14:16:59Z

Commit baf1568 addresses input from @samruddhikhandale and passes all tests locally.

samruddhikhandale

We really appreciate all the efforts you take for the contribution! ✨

Left some minor comments.

.github/workflows/test-pr.yaml

src/python/install.sh

test/python/scenarios.json

src/python/install.sh

samruddhikhandale

Looks wonderful, thank you so much! Appreciate it 👏

samruddhikhandale · 2024-02-09T17:59:20Z

The packages option will be left to the other PR.

@jdputschadi (Not to pressure you) Were you planning on submitting a PR? because I'd like to soon merge this new option due to devcontainers/images#856.

If you are not planning, then let me know. Looking forward to avoid duplicate efforts, thanks!

jdputschadi · 2024-02-09T18:11:49Z

The packages option will be left to the other PR.

@jeffwilcox (Not to pressure you) Were you planning on submitting a PR? because I'd like to soon merge this new option due to devcontainers/images#856.

If you are not planning, then let me know. Looking forward to avoid duplicate efforts, thanks!

Possibly the wrong Jeff? The "other PR" I was referring to was #768. I was indicating that I would not include the "packages" feature that Alexander was submitting in #768 and leaving that work to him.

samruddhikhandale · 2024-02-09T18:15:53Z

Possibly the wrong Jeff?

Oops!

The "other PR" I was referring to was #768. I was indicating that I would not include the "packages" feature that Alexander was submitting in #768 and leaving that work to him.

Great, thanks for confirming!

Will merge this PR as soon as the tests are green.

Jeff Putsch added 9 commits December 18, 2023 14:32

install gpg2.22 on centos 7 when installing python

fc6cd0e

RHEL support, exisiting tests pass, new RHEL tests pass.

5e79d61

Merge branch 'main' into python-feature-rhel-support

31a5f7a

add tests, cleanup install organization

4f77b37

update testing to include RHEL tests

a1fd5b2

update testing to include RHEL tests

6fc48e3

undo addition of installing additional pip modules

11414e2

Merge branch 'main' into python-feature-rhel-support

f0c136e

Merge branch 'main' into python-feature-rhel-support

0feb316

jdputschadi requested a review from a team as a code owner February 1, 2024 20:48

fix errors installing os-provided Python on recent Debian systems and…

4007701

… on Mariner systems

adjust to properly use newly installed python (PYTHON_SRC) instead of…

f20275f

… assuming "python" will work

samruddhikhandale closed this Feb 2, 2024

samruddhikhandale reopened this Feb 2, 2024

Jeff Putsch and others added 3 commits February 2, 2024 13:47

merge from main

50c7d12

update check for managed python install. pass all tests.

13a7f30

Merge branch 'main' into python-feature-rhel-support

f9f1e7f

samruddhikhandale requested changes Feb 5, 2024

View reviewed changes

add "packages" option from PR devcontainers#768

dffe200

Jeff Putsch added 2 commits February 6, 2024 14:02

remove "packages" option

1473900

Address PR feedback, passes all tests locally.

baf1568

samruddhikhandale requested changes Feb 7, 2024

View reviewed changes

Jeff Putsch and others added 2 commits February 7, 2024 20:46

fix install error on centos

6c1335a

Merge branch 'main' into python-feature-rhel-support

12d2e62

samruddhikhandale approved these changes Feb 9, 2024

View reviewed changes

samruddhikhandale linked an issue Feb 9, 2024 that may be closed by this pull request

[debian bookworm] Python Feature fails to install with 'error: externally-managed-environment' #577

Closed

samruddhikhandale merged commit 30e03b6 into devcontainers:main Feb 9, 2024
11 checks passed

jdputschadi deleted the python-feature-rhel-support branch February 10, 2024 23:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add RHEL support to the python feature #830

Add RHEL support to the python feature #830

jdputschadi commented Feb 1, 2024

jdputschadi commented Feb 1, 2024

samruddhikhandale commented Feb 1, 2024 •

edited

Loading

jdputschadi commented Feb 2, 2024 •

edited

Loading

jdputschadi commented Feb 2, 2024

jdputschadi commented Feb 2, 2024

samruddhikhandale commented Feb 2, 2024

samruddhikhandale commented Feb 2, 2024

jdputschadi commented Feb 2, 2024

jdputschadi commented Feb 2, 2024

jdputschadi commented Feb 5, 2024 •

edited

Loading

jdputschadi commented Feb 5, 2024 •

edited

Loading

samruddhikhandale commented Feb 5, 2024

samruddhikhandale commented Feb 5, 2024

samruddhikhandale commented Feb 5, 2024

samruddhikhandale left a comment

jdputschadi commented Feb 6, 2024

samruddhikhandale commented Feb 6, 2024

jdputschadi commented Feb 6, 2024

samruddhikhandale commented Feb 6, 2024

jdputschadi commented Feb 6, 2024

jdputschadi commented Feb 7, 2024

samruddhikhandale left a comment

samruddhikhandale left a comment

samruddhikhandale commented Feb 9, 2024 •

edited

Loading

jdputschadi commented Feb 9, 2024

samruddhikhandale commented Feb 9, 2024

Add RHEL support to the python feature #830

Add RHEL support to the python feature #830

Conversation

jdputschadi commented Feb 1, 2024

jdputschadi commented Feb 1, 2024

samruddhikhandale commented Feb 1, 2024 • edited Loading

jdputschadi commented Feb 2, 2024 • edited Loading

jdputschadi commented Feb 2, 2024

jdputschadi commented Feb 2, 2024

samruddhikhandale commented Feb 2, 2024

samruddhikhandale commented Feb 2, 2024

jdputschadi commented Feb 2, 2024

jdputschadi commented Feb 2, 2024

jdputschadi commented Feb 5, 2024 • edited Loading

jdputschadi commented Feb 5, 2024 • edited Loading

samruddhikhandale commented Feb 5, 2024

samruddhikhandale commented Feb 5, 2024

samruddhikhandale commented Feb 5, 2024

samruddhikhandale left a comment

Choose a reason for hiding this comment

jdputschadi commented Feb 6, 2024

samruddhikhandale commented Feb 6, 2024

jdputschadi commented Feb 6, 2024

samruddhikhandale commented Feb 6, 2024

jdputschadi commented Feb 6, 2024

jdputschadi commented Feb 7, 2024

samruddhikhandale left a comment

Choose a reason for hiding this comment

samruddhikhandale left a comment

Choose a reason for hiding this comment

samruddhikhandale commented Feb 9, 2024 • edited Loading

jdputschadi commented Feb 9, 2024

samruddhikhandale commented Feb 9, 2024

samruddhikhandale commented Feb 1, 2024 •

edited

Loading

jdputschadi commented Feb 2, 2024 •

edited

Loading

jdputschadi commented Feb 5, 2024 •

edited

Loading

jdputschadi commented Feb 5, 2024 •

edited

Loading

samruddhikhandale commented Feb 9, 2024 •

edited

Loading