Support container provisioner in toolbox #3228

thrix · 2024-09-19T22:36:43Z

For Fedora Silverblue users it is common to run podman via flatpak-spawn --host which runs podman on the host system itself. This requires to pass the toolbox container name when running podman cp to correctly copy stuff from the toolbox container, where tmt is installed to the provisioned container.

Fixes #1020

Pull Request Checklist

implement the feature
extend the test coverage
include a release note

thrix · 2024-09-19T22:37:37Z

@jkonecny12 this is the change I talked about, with it tmt container provisioner works well :)

tmt/steps/provision/podman.py

janhavlin · 2024-09-23T08:58:53Z

I'm getting this error, any idea? The directory /var/tmp/tmt/run-135/testing-farm/sanity/provision/default-0 exists only in toolbox it seems.

❯ tmt --version
tmt version: 1.37.0.dev20+gce8a3c6e

❯ tmt run -a provision -h container -i fedora:40 plan -n /testing-farm/sanity
/var/tmp/tmt/run-135

/testing-farm/sanity
    discover
        how: fmf
        directory: /var/home/jhavlin/work/tests
        tests: /testing-farm/script
        summary: 1 test selected
    provision
        queued provision.provision task #1: default-0
        
        provision.provision task #1: default-0
        how: container
        image: fedora:40
        fail: Command 'podman pull -q fedora:40' returned 1.
        fail: Command 'podman pull -q fedora:40' returned 1.
        fail: Command 'podman pull -q fedora:40' returned 1.
        fail: Command 'podman pull -q fedora:40' returned 1.
        fail: Command 'podman pull -q fedora:40' returned 1.
        fail: Retries of 'Pulling 'fedora:40' image' unsuccessful.
    finish
        warn: Nothing to finish, no guests provisioned.

plan failed

The exception was caused by 1 earlier exceptions

Cause number 1:

    provision step failed

    The exception was caused by 1 earlier exceptions

    Cause number 1:

        Retries of 'Pulling 'fedora:40' image' unsuccessful.

        The exception was caused by 5 earlier exceptions

        Cause number 1:

            Command 'podman pull -q fedora:40' returned 1.

            stderr (1/1 lines)
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            Portal call failed: Failed to start command: Failed to change to directory “/var/tmp/tmt/run-135/testing-farm/sanity/provision/default-0” (No such file or directory)
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...

thrix · 2024-09-24T12:06:58Z

@janhavlin this is something we should document, add this to your environment (I have it in .bashrc):

export TMT_WORKDIR_ROOT=$HOME/.local/share/tmt

janhavlin · 2024-09-24T12:12:29Z

Thanks, I got it working!

jkonecny12 · 2024-10-16T12:24:49Z

@janhavlin this is something we should document, add this to your environment (I have it in .bashrc):
export TMT_WORKDIR_ROOT=$HOME/.local/share/tmt

Couldn't we have this resolved by default correctly? Requesting users to set somewhere something to make things work seems like an issue generator and unhappy users.

thrix · 2024-10-16T13:42:28Z

@janhavlin this is something we should document, add this to your environment (I have it in .bashrc):
export TMT_WORKDIR_ROOT=$HOME/.local/share/tmt
Couldn't we have this resolved by default correctly? Requesting users to set somewhere something to make things work seems like an issue generator and unhappy users.

@jkonecny12 I agree it is sad, but we will need to have a discussion with tmt folks :)

thrix · 2024-10-17T11:57:52Z

So we agreed we will go with this default if we detect running in toolbox :)

thrix · 2024-12-10T16:39:11Z

FTR the workdir autodetection will be another patch but it won't be in 1.40.

For Fedora Silverblue users it is common to run podman via `flatpak-spawn --host` which runs podman on the host system itself. This requires to pass the toolbox container name when running `podman cp` to correctly copy stuff from the toolbox container, where `tmt` is installed to the provisioned container. Fixes #1020 Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

psss

The change itself looks good. The test setup looks really super-complex and thus, I'm afraid, a bit fragile. Added a few comments though.

tests/provision/container/toolbox/test.sh

Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

thrix · 2025-01-29T15:09:01Z

The newly added test failed because of a missing toolbox package. Should be fixed by e58860d.

I simplified, so I hope this concern is now addressed

thrix · 2025-01-29T17:01:09Z

/packit build

psss

Looks much better now, thanks! Though it seems that tests are red. You've mentioned that el9 should be disabled, right? Plus there's seems to be an avc failure on fedora-41+.

Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

thrix · 2025-01-29T19:24:12Z

Looks much better now, thanks! Though it seems that tests are red. You've mentioned that el9 should be disabled, right? Plus there's seems to be an avc failure on fedora-41+.

yes, disabling avc check and centos stream 9, I tried to make it work, but it is a mess

tests/provision/container/toolbox/test.sh

thrix · 2025-01-30T06:09:35Z

/packit build

thrix · 2025-01-30T11:13:11Z

Issues here are unrelated, merging

For Fedora Silverblue users it is common to run podman via `flatpak-spawn --host` which runs podman on the host system itself. This requires to pass the toolbox container name when running `podman cp` to correctly copy stuff from the toolbox container, where `tmt` is installed to the provisioned container. Fixes #1020 Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

happz reviewed Sep 20, 2024

View reviewed changes

tmt/steps/provision/podman.py Outdated Show resolved Hide resolved

happz reviewed Sep 20, 2024

View reviewed changes

tmt/steps/provision/podman.py Outdated Show resolved Hide resolved

thrix added this to the 1.37 milestone Sep 20, 2024

thrix modified the milestones: 1.37, 1.38 Oct 1, 2024

thrix modified the milestones: 1.38, 1.39 Oct 25, 2024

thrix modified the milestones: 1.39, 1.40 Nov 5, 2024

bajertom modified the milestones: 1.40, 1.41 Dec 3, 2024

happz modified the milestones: 1.41, 1.40 Dec 10, 2024

thrix force-pushed the support-container-executor-toolbox branch from ce8a3c6 to 2579e3c Compare December 10, 2024 16:38

thrix marked this pull request as ready for review December 10, 2024 16:39

thrix requested review from psss, lukaszachy and janhavlin as code owners December 10, 2024 16:39

thrix force-pushed the support-container-executor-toolbox branch from 2579e3c to 3a12526 Compare December 10, 2024 16:39

psss added the ci | full test Pull request is ready for the full test execution label Dec 12, 2024

psss force-pushed the support-container-executor-toolbox branch 2 times, most recently from a2df6da to e58860d Compare December 12, 2024 09:29

skycastlelily approved these changes Jan 14, 2025

View reviewed changes

thrix force-pushed the support-container-executor-toolbox branch from e58860d to a2f97ba Compare January 27, 2025 19:41

thrix requested a review from martinhoyer as a code owner January 27, 2025 19:41

thrix force-pushed the support-container-executor-toolbox branch 2 times, most recently from 8944e90 to 34d2311 Compare January 27, 2025 21:53

thrix added the status | ready for merge The only missing piece is to do the rebase the current 'main' and let the CI finish. label Jan 27, 2025

thrix force-pushed the support-container-executor-toolbox branch from 34d2311 to 8664605 Compare January 28, 2025 12:48

thrix added 2 commits January 29, 2025 13:51

Fix tests in CI

ed8281a

Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

thrix force-pushed the support-container-executor-toolbox branch from 4121741 to ed8281a Compare January 29, 2025 12:51

psss reviewed Jan 29, 2025

View reviewed changes

tests/provision/container/toolbox/test.sh Outdated Show resolved Hide resolved

tests/provision/container/toolbox/test.sh Outdated Show resolved Hide resolved

Simplify the test, use make develop to install additional deps

35b1540

Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

thrix requested a review from psss January 29, 2025 14:56

Fix dnf install call

28678ab

Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

thrix changed the base branch from main to fix-container-tests January 29, 2025 17:00

psss approved these changes Jan 29, 2025

View reviewed changes

thrix force-pushed the fix-container-tests branch from 6f30463 to 9c96137 Compare January 29, 2025 19:12

Disable on non-Fedora distros, expect avc failures

4630ce1

Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

thrix commented Jan 29, 2025

View reviewed changes

tests/provision/container/toolbox/test.sh Outdated Show resolved Hide resolved

thrix added 2 commits January 29, 2025 20:26

Update tests/provision/container/toolbox/test.sh

59a1b1f

Ignore AVC only on Fedora-41 and later

02287e9

Base automatically changed from fix-container-tests to main January 30, 2025 09:38

thrix merged commit 0f4eac8 into main Jan 30, 2025
15 of 17 checks passed

thrix deleted the support-container-executor-toolbox branch January 30, 2025 11:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support container provisioner in toolbox #3228

Support container provisioner in toolbox #3228

thrix commented Sep 19, 2024 •

edited

Loading

thrix commented Sep 19, 2024

janhavlin commented Sep 23, 2024 •

edited

Loading

thrix commented Sep 24, 2024

janhavlin commented Sep 24, 2024

jkonecny12 commented Oct 16, 2024

thrix commented Oct 16, 2024

thrix commented Oct 17, 2024

thrix commented Dec 10, 2024

psss left a comment

thrix commented Jan 29, 2025

thrix commented Jan 29, 2025

psss left a comment

thrix commented Jan 29, 2025

thrix commented Jan 30, 2025

thrix commented Jan 30, 2025

Support container provisioner in toolbox #3228

Support container provisioner in toolbox #3228

Conversation

thrix commented Sep 19, 2024 • edited Loading

thrix commented Sep 19, 2024

janhavlin commented Sep 23, 2024 • edited Loading

thrix commented Sep 24, 2024

janhavlin commented Sep 24, 2024

jkonecny12 commented Oct 16, 2024

thrix commented Oct 16, 2024

thrix commented Oct 17, 2024

thrix commented Dec 10, 2024

psss left a comment

Choose a reason for hiding this comment

thrix commented Jan 29, 2025

thrix commented Jan 29, 2025

psss left a comment

Choose a reason for hiding this comment

thrix commented Jan 29, 2025

thrix commented Jan 30, 2025

thrix commented Jan 30, 2025

thrix commented Sep 19, 2024 •

edited

Loading

janhavlin commented Sep 23, 2024 •

edited

Loading