Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get rid of Linux Podman Machine requirement and instead run escalated privileged CLI command. #623

Closed
cdrage opened this issue Jul 11, 2024 · 6 comments · Fixed by #684
Closed
Assignees
Labels
kind/feature 💡 Issue for requesting a new feature

Comments

@cdrage
Copy link
Contributor

cdrage commented Jul 11, 2024

Is your feature request related to a problem? Please describe

Podman Machine is a requirement in order to run this extension which causes inconveniences for Linux users.

It requires us create a separate podman machine, use it, as well as make sure that it also has the correct image too.

There have been numerous cases where Podman Machine isn't detected (it is not officially supported and a hidden feature of PD) and requires PD to be restarted to "use" the podman machine.

Why this is possible now:

  • BIB does not require us to "pull" the image (can pass in directory) now
  • We already have functions to "generate" the podman CLI code that would be ran.

There are also other cases:

  • Much faster, runs local not on VM
  • Podman Machine on Linux isn't "officially" supported and there may be bugs
  • Does not align with bib / osbuild team with regards to their development / they do not test on podman machine
  • Issues arise with cumbersome steps for setup.

Describe the solution you'd like

Instead, our extension could use the exec command / process command of PD extension to run a privileged command (https://github.com/containers/podman-desktop/blob/main/packages/extension-api/src/extension-api.d.ts#L4114) and run the following example command:

sudo podman run \
  --name httpd-bootc-image-builder \
  --tty \
  --privileged \
  --security-opt label=type:unconfined_t \
  -v /home/testuser/bootc/test123:/output/ \
  -v $HOME/.local/share/containers/storage:/var/lib/containers/storage \
  --label bootc.image.builder=true \
  quay.io/centos-bootc/bootc-image-builder:latest-1720185748 \
  quay.io/bootc-extension/httpd:latest \
  --output \
  /output/ \
  --local \
  --type \
  raw \
  --target-arch \
  amd64

Note that this is running the SUDO command but uses $HOME/.local/share/containers/storage/. for the storage. This means that the user can use podman rootless like normal, but just for building, it will use SUDO.

Describe alternatives you've considered

No response

Additional context

No response

@cgwalters
Copy link
Contributor

So, I have a pretty strong opinion that this project should not be implementing its own logic here in the medium/long term but should just be a GUI around a CLI (and maybe that CLI exposes a language API).

There's a lot of related threads on the podman-bootc CLI for this, see: containers/podman-bootc#9

I feel strongly enough about this that I think this issue should be closed as a duplicate of #166

@vrothberg
Copy link
Contributor

I agree with @cgwalters. If we get podman-bootc into a state where it can run on Linux without a machine and get the bootc-extension to use podman-bootc, it's a huge win.

Requires some organization and good planning along with commitments to meet deadlines.

@cdrage
Copy link
Contributor Author

cdrage commented Jul 11, 2024

So, I have a pretty strong opinion that this project should not be implementing its own logic here in the medium/long term but should just be a GUI around a CLI (and maybe that CLI exposes a language API).

There's a lot of related threads on the podman-bootc CLI for this, see: containers/podman-bootc#9

I feel strongly enough about this that I think this issue should be closed as a duplicate of #166

@vrothberg @cgwalters

For this issue I'm referring to the building image fix / removing the podman machine requirement as Linux builds aren't working right now. Not with regards to running the resulting image in a VM.

From my understanding https://github.com/containers/podman-bootc helps (at the moment) with regards to spinning up the VM / using the resulting built image. From my understanding there's no other features implemented yet (issues opened for them though regarding bib).

I agree that we should get this in medium/long-term, but this issue should focus on the short-term fix of linux builds.

EDIT: I see containers/podman-bootc#58 which is awesome and a lot more new PR's. But I believe again that a temporary fix should go in under we switch 100% to podman-bootc for all build logic across mac / windows / linux.

@deboer-tim
Copy link
Contributor

I would prefer it as a wrapper around a cli too, but I think it would depend on the timeline? i.e. when would podman-bootc support windows and be the preferred approach enough so that pages like https://osbuild.org/docs/bootc/ would change? If that was in the pipe/'soonish' then we should just wait and focus on that, but if not we'll likely need to do something in the meantime.

@cdrage cdrage moved this to 📋 Backlog in Podman Desktop Planning Jul 11, 2024
@cdrage cdrage self-assigned this Jul 11, 2024
@vrothberg
Copy link
Contributor

@germag WDYT?

@cdrage
Copy link
Contributor Author

cdrage commented Jul 12, 2024

Another caveat too is how Podman Machine on Podman Desktop is by default root / rootful on macOS and Windows when creating the machine, but non-root on Linux (since it requires no podman machine). From the PR: containers/podman-bootc#58 we will be hardcoding: /var/lib/containers/storage (https://github.com/containers/podman-bootc/pull/58/files#diff-b617e9b9c374b3eb34e1995a6010fadc91cfaa59baba743b40bec8163eec0ed0R158). But bootc-image-builder requires root to function due to filesystem requirements.

What the current workflow looks like:

macOS & Windows:

  1. Podman Desktop is started rootful
  2. User builds or pulls bootc image within PD (root)
  3. Uses this extension or podman-bootc (already root)
  4. Passes fine since everything is root

But the current cycle for Linux is:

  1. Podman Desktop is started on Linux (non-root)
  2. User builds or pulls bootc image within PD (non-root)
  3. Build using this extension or podman-bootc (requires root)
  4. Error: User unable to use the image that they built / pulled since it was built in non-root environment and saved to ~/.local/share/containers/storage
  5. Now requires opening a podman machine / using podman-bootc
  6. Rebuild / pull image
  7. Build again
  8. Error: Run into qemu issues (see Does not work on Linux with Podman Machine (errors with cp: error writing '/output/image/./disk.raw') osbuild/bootc-image-builder#540)

What I'm proposing for Linux is as a short-term fix until we are at 100% feature parity with bib for podman-bootc is:

  1. Podman Desktop is started on Linux (non-root)
  2. User builds or pulls bootc image within PD (non-root)
  3. Extension runs privileged podman CLI command with ~/.local/share/containers/storage passed in, which uses the non-root images they had built (requires root). We already have this code ready-to-go in: https://github.com/containers/podman-desktop/blob/main/packages/extension-api/src/extension-api.d.ts#L4114
  4. Passes

TLDR; Linux builds are failing at the moment, due to very old Podman versions on different machines (openSUSE tumbleweed, fedora 40, etc.) using flakey podman machine's and we should fix them with this solution until Linux support is better.

@cdrage cdrage moved this from 📋 Backlog to 🚧 In Progress in Podman Desktop Planning Jul 23, 2024
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 30, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 30, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
@cdrage cdrage moved this from 🚧 In Progress to 🚥 In Review in Podman Desktop Planning Jul 30, 2024
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 31, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 31, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 31, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 31, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 31, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 31, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 31, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Jul 31, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Aug 1, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
cdrage added a commit to cdrage/podman-desktop-extension-bootc that referenced this issue Aug 1, 2024
### What does this PR do?

* Switches to using native podman building for Linux rather than using
  podman machine
* Tested against using a manifest as well as a normal image.
* Uses CLI commands the equivalant of doing `sudo podman run`. PD does
  not support running / viewing / using sudo root connections. So we use
  the CLI instead
* Uses CLI commands for saving the image / importing as well. The
  reasoning is that importing requires `sudo` / privileged and
  retrieving via image ID does not work for saving via the API.

### Screenshot / video of UI

<!-- If this PR is changing UI, please include
screenshots or screencasts showing the difference -->

### What issues does this PR fix or reference?

<!-- Include any related issues from Podman Desktop
repository (or from another issue tracker). -->

Closes podman-desktop#623

### How to test this PR?

<!-- Please explain steps to reproduce -->

1. Try on Linux (Fedora 40 or above)
2. Go to build and it should ask for credentials after a few moments of
   building
3. Successful image build

Signed-off-by: Charlie Drage <charlie@charliedrage.com>
@github-project-automation github-project-automation bot moved this from 🚥 In Review to ✔️ Done in Podman Desktop Planning Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature 💡 Issue for requesting a new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants