Rebase RHEL CoreOS 4.6 on Fedora CoreOS Live ISO/installer #210
Conversation
openshift-ci-robot
added
the
size/M
cgwalters
force-pushed
the
rhcos-live-iso
branch
from
f50fb96 to
5d1479c
Compare
openshift-ci-robot
added
the
approved
|
I've asked folks to review this so we get a few more eyes. |
| This allows administrators to run arbitrary code before and after the installation process by providing Ignition that wraps the installer. For example, higher level tooling like Foreman and Ironic can easily `curl` a URL to report provisioning success, or even pull down and launch a container before/after. | ||
|
|
||
| An additional goal is to allow administrators to more easily run RHCOS as a "live" system for *interactive* experiementation and testing. For example, an administrator could run the system from a USB stick to validate the network interface names, etc. | ||
|
|
There was a problem hiding this comment.
Another motivation that might be worth mentioning is the new installer isn't written in shell script but in a compiled safe language, which should reduce defect count and make security vulnerabilities harder to slip in.
|
|
||
| Add a basic (virtualized) test case for the CoreOS installer. | ||
|
|
||
| Ship the new `coreos-install` binary at https://mirror.openshift.com and as an RPM in the OpenShift channel. |
enhancements/rhcos/liveisoinstall.md
Outdated
|
|
||
| ### Risks and Mitigations | ||
|
|
||
| It's unknown exactly how difficult it will be for customers who are automating OpenShift installations to do the conversion, in particular for customers who want to be able to install both 4.4 and 4.5 for example. |
There was a problem hiding this comment.
The OpenShift bare metal IPI process relies on having pre-prepared disk images that can be written to the system without having to interact with an installer process. Does this enhancement propose eliminating that?
There was a problem hiding this comment.
No, we still have -metal images suitable for dd.
More broadly speaking, this is basically replacing the "dd in initramfs" with "dd in a full live OS".
|
This has been open for a little over a month. If possible, anyone who wants to provide feedback please do so in the next couple of working days 🙏 |
|
is there a reason this is not built into the openshift-installer? Shipping yet another binary seems like a were going to create a confusing situation for what tool to use and where. |
Fair question, though note we do use The point about it being confusing is valid. There are a number of things with "installer" in the name today used in OCP. Hopefully no new ones will be added/needed 😄. |
|
|
||
| ## Motivation | ||
|
|
||
| This allows administrators to run arbitrary code before and after the installation process by providing Ignition that wraps the installer. For example, higher level tooling like Foreman and Ironic can easily `curl` a URL to report provisioning success, or even pull down and launch a container before/after. |
There was a problem hiding this comment.
As a product (OpenShift) don't we want to discourage this (modifying ignition). As a product, we may use an underlying technology (from upstream) to accomplish a goal, but not neccarly expose that to the downstream products end-users. (this is how I view ignition - cool upstream tech, that we treat as a black box, inside of the installer)
If we allow this (arbitrary modification of ignition) how will we keep 'ignition' from becoming the next 'ansible' (issue for OpenShift); that makes the infinite number of configuration permutations, impossible for us to comprehend / track; and thus can't ensure a smooth/simple install experience? Is there a way to gate/validate the ignition we get to make sure its safe and does not conflict with our install processes, befor the installer starts?
There was a problem hiding this comment.
We have to support appending to the openshift-installer generated Ignition today for machine-specific configuration like /etc/hostname and networking etc.
If we allow this (arbitrary modification of ignition) how will we keep 'ignition' from becoming the next 'ansible' (issue for OpenShift); that makes the infinite number of configuration permutations
Today MachineConfig via the MCO already allows executing arbitrary code as root via systemd units - day 1 or day 2. And it's the same thing with Ignition.
Is there a way to gate/validate the ignition we get to make sure its safe and does not conflict with our install processes, befor the installer starts?
I can't really think of a way to do that other than to provide a higher level set of knobs, you could imagine something a bit like https://github.com/coreos/fcct for provisioning time, something like a file:
postInstallURL: http://example.com/provisioning/blah
that openshift-install could perhaps accept and generate the Ignition for the machine install.
There was a problem hiding this comment.
Note that the Ignition config described in this paragraph is not the config applied to the installed system. It's a second, independent Ignition config that only applies to the live system which runs the installer itself.
|
Any other comments on this enhancement? |
For openshift/enhancements#210 I think we will need at least some transitional period where RHCOS ships both. Even ignoring customers, the way our CI works strongly encourages "ratcheting" changes in a compatible way, i.e.: - Introduce new thing, ship it - Port consumers to new thing - Remove old thing And honestly I think this is a good model, because it forces us to experience some of the pain that customers might feel here. The basic idea here is: - Install as `/usr/libexec/coreos-installer.legacy` so there's no chance of conflict with the real one - Test for the presence of the "stamp file' we drop in our live initramfs so we know to defer to the real root's coreos-installer This would be even more robust if we changed coreos-assembler to inject an explicit stamp file into the legacy initramfs (and ideally, we don't ship legacy installer in the initramfs but that's a bit trickier to do cleanly)
For openshift/enhancements#210 I think we will need at least some transitional period where RHCOS ships both. Even ignoring customers, the way our CI works strongly encourages "ratcheting" changes in a compatible way, i.e.: - Introduce new thing, ship it - Port consumers to new thing - Remove old thing And honestly I think this is a good model, because it forces us to experience some of the pain that customers might feel here. The basic idea here is: - Install as `/usr/libexec/coreos-installer.legacy` so there's no chance of conflict with the real one - Test for the presence of the "stamp file' we drop in our legacy initramfs so we know to defer to the real root's coreos-installer; see coreos/coreos-assembler#1389
For openshift/enhancements#210 I think we will need at least some transitional period where RHCOS ships both. Even ignoring customers, the way our CI works strongly encourages "ratcheting" changes in a compatible way, i.e.: - Introduce new thing, ship it - Port consumers to new thing - Remove old thing And honestly I think this is a good model, because it forces us to experience some of the pain that customers might feel here. The basic idea here is: - Install as `/usr/libexec/coreos-installer.legacy` so there's no chance of conflict with the real one - Test for the presence of the "stamp file' we drop in our legacy initramfs so we know to defer to the real root's coreos-installer; see coreos/coreos-assembler#1389
|
Also tangentially related https://github.com/chuckersjp/coreos-iso-maker |
RHEL CoreOS is built from RHEL components, but uses configuration and tooling from Fedora CoreOS. They share many things such as the installer code, etc. This enhancement tracks rebasing RHCOS 4.5 on the latest FCOS code, in particular the installer.
|
OK updated this to note the current plan is to ship both installers: coreos/coreos-installer#220 |
|
OK we've done the work to ship this code, let's make it official! |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ashcrow, cgwalters, jlebon The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
cgwalters
changed the title
RHEL CoreOS is built from RHEL components, but uses
configuration and tooling from Fedora CoreOS. They share
many things such as the installer code, etc.
This enhancement tracks rebasing RHCOS 4.5 on the latest FCOS code,
in particular the installer.