From 7b98b7710c8e810407f11fddb9646783991bcadb Mon Sep 17 00:00:00 2001 From: Ben Nemec Date: Mon, 28 Jun 2021 16:54:43 -0500 Subject: [PATCH 1/4] Baremetal IPI Network Configuration for Day-1 Describe user-facing API for day-1 network customizations in the IPI workflow, with particular focus on baremetal where such configuration is a common requirement. Co-authored-by: Steve Hardy Co-authored-by: Zane Bitter Co-authored-by: Dan Kenigsberg --- .../baremetal-ipi-network-configuration.md | 281 ++++++++++++++++++ 1 file changed, 281 insertions(+) create mode 100644 enhancements/network/baremetal-ipi-network-configuration.md diff --git a/enhancements/network/baremetal-ipi-network-configuration.md b/enhancements/network/baremetal-ipi-network-configuration.md new file mode 100644 index 0000000000..2de1bd32b1 --- /dev/null +++ b/enhancements/network/baremetal-ipi-network-configuration.md @@ -0,0 +1,281 @@ +--- +title: baremetal-ipi-network-configuration +authors: +- "@cybertron" +- "@hardys" +- "@zaneb" +reviewers: +- "@kirankt" +- "@dtantsur" +- "@zaneb" +approvers: +- "@trozet" +creation-date: 2021-05-21 +last-updated: 2021-07-07 +status: provisional|implementable + +see-also: +- "/enhancements/host-network-configuration.md" +- "/enhancements/machine-config/mco-network-configuration.md" +- "/enhancements/machine-config/rhcos/static-networking-enhancements.md" +--- + +# Baremetal IPI Network Configuration + +Describe user-facing API for day-1 network customizations in the IPI workflow, +with particular focus on baremetal where such configuration is a common +requirement. + +## Release Signoff Checklist + +- [ ] Enhancement is `implementable` +- [ ] Design details are appropriately documented from clear requirements +- [ ] Test plan is defined +- [ ] Operational readiness criteria is defined +- [ ] Graduation criteria for dev preview, tech preview, GA +- [ ] User-facing documentation is created in [openshift-docs](https://github.com/openshift/openshift-docs/) + +## Summary + +Currently in the IPI flow, there is no way to provide day-1 network configuration +which is a common requirement, particularly for baremetal users. We can build +on the [UPI static networking enhancements](rhcos/static-networking-enhancements.md) +to enable such configuration in the IPI flow. + +## Motivation + +Since the introduction of baremetal IPI, a very common user request is how +to configure day-1 networking, and in particular the following cases which are not currently possible: + +* Deploy with OpenShift Machine network on a tagged (non-default) VLAN +* Deploy with OpenShift Machine network using static IPs (no DHCP) + +In both cases, this configuration cannot be achieved via DHCP so some +means of providing the configuration to the OS is required. + +In the UPI flow this is achieved by consuming user-provided NetworkManager +keyfiles, as an input to `coreos-install --copy-network`, but there is +no corresponding user interface at the openshift-install level. + +Additionally, there are other networking configurations that would be useful +to configure via the same mechanism, even though it may be possible to +accomplish them in another way. For example: + +* Deploy with OpenShift Machine network on a bond +* Deploy with OpenShift Machine network on a bridge +* Configure attributes of network interfaces such as bonding policies and MTUs + +The proposed solutions should all be flexible enough to support these use +cases, but it is worth noting in case an alternative with a narrower scope +would be put forward. + +### Goals + +* Define API for day-1 network customizations +* Enable common on-premise network configurations (bond+vlan, static ips) via IPI + +Initially these configurations will be one per host. If there is time, an +additional goal would be to provide a mechanism to apply a single config to all +nodes of a particular type. For example, one config for all masters and another +config for all workers. + +### Non-Goals + +* Platforms other than `baremetal`, although the aim is a solution which could be applied to other platforms in future if needed. +* Enabling kubernetes-nmstate by default for day-2 networking is discussed via +[another proposal](https://github.com/openshift/enhancements/pull/747) +* Provide a consistent (ideally common) user API for deployment and post-deployment configuration. Getting agreement on a common API for day-1 and day-2 has stalled due to lack of consensus around enabling kubernetes-nmstate APIs (which are the only API for day-2 currently) by default +* Configuration of the provisioning network. Users who don't want DHCP in their +deployment can use virtual media, and users who want explicit control over the +addresses used for provisioning can make the provisioning network unmanaged and +deploy their own DHCP infrastructure. + +## Proposal + +### User Stories + +#### Story 1 + +As a baremetal IPI user, I want to deploy via PXE and achieve a highly +available Machine Network configuration in the most cost/space effective +way possible. + +This means using two top-of-rack switches, and 2 NICS per host, with the +default VLAN being used for provisioning traffic, then a bond+VLAN configuration +is required for the controlplane network. + +Currently this [is not possible](https://bugzilla.redhat.com/show_bug.cgi?id=1824331) +via the IPI flow, and existing ignition/MachineConfig APIs are not sufficient +due to the chicken/egg problem with accessing the MCS. + +#### Story 2 + +As an on-premise IPI user, I wish to use static IPs for my controlplane network, +for reasons of network ownership or concerns over reliability I can't use DHCP +and therefore need to provide a static configuration for my primary network. + +There is no way to provide [MachineSpecific Configuration in OpenShift](https://github.com/openshift/machine-config-operator/issues/1720) so I am +forced to use the UPI flow which is less automated and more prone to errors. + +### Risks and Mitigations + +In some existing workflows, kubernetes-nmstate is used to do network configuration on day-2. Using a different interface for day-1 introduces the potential for mismatches and configuration errors when making day-2 changes. It should be possible to mitigate this by generating the NetworkManager keyfiles using the nmstatecli, which will allow the same nmstate config to be used directly in the day-2 configuration. + +An added benefit of writing nmstate configurations instead of NetworkManager +keyfiles directly is that if/when a day-1 nmstate-based interface becomes +available the config files can be directly inserted into the new interface. +However, this is not a complete solution because it still requires manually copying +configuration to multiple locations which introduces the potential for errors. + +## Design Details + +In the IPI flow day-1 network configuration is required in 2 different cases: + +* Deployment of the controlplane hosts via terraform, using input provided to openshift-install +* Deployment of compute/worker hosts (during initial deployment and scale-out), via Machine API providers for each platform + +In the sections below we will describe the user-facing API that contains network configuration, and the proposed integration for each of these cases. + +### User-facing API + +RHCOS already provides a mechanism to specify NetworkManager keyfiles during deployment of a new node. We need to expose that functionality during the IPI install process. There are a couple of options on how to do that: + +* A new section in install-config. +* A secret that contains base64-encoded content for the keyfiles. + +These are not mutually exclusive. If we implement the install-config option, we will still need to persist the configuration in a secret so it can be used for day-2. In fact, the initial implementation should likely be based on secrets, and then install-config integration can be added on top of that to improve the user experience. This way the install-config work won't block implementation of the feature. + +The data provided by the user will need to have the following structure: +```yaml +[hostname]: + [interface name]: + [contents of NetworkManager key file for interface] + [second interface name]: + [contents of NetworkManager key file for second interface] + etc... +``` + +For example: +```yaml +openshift-master-0: + eth0: + [connection] + id=eth0 + uuid=18e0cec7-041c-4fb3-957c-a60c80dd9b85 + type=ethernet + etc... +openshift-master-1: + eth0: + [connection] + id=eth0 + etc... +``` + +In install-config this would look like: +```yaml +platform: + baremetal: + hosts: + - name: openshift-master-0 + networkConfig: + - interface: eth0 + data: | + [connection] + id=eth0 + uuid=18e0cec7-041c-4fb3-957c-a60c80dd9b85 + type=ethernet + etc... +``` + +Because the initial implementation will be baremetal-specific, we can put the +network configuration data into the baremetal host field, which will allow easy +mapping to the machine in question. + +### Processing user configuration + +#### Deployment of the controlplane hosts via terraform + +We will map the keyfiles to their appropriate BareMetalHost using the host field +of the baremetal install-config. The keyfiles will then be added to custom +images for each host built by Terraform and Ironic. + +Since different configuration may be needed for each host (for example, when +deploying with static IPs), a Secret per host will be created. A possible +future optimization is to use a single secret for scenarios such as VLANs +where multiple hosts can consume the same configuration, but the initial +implementation will have a 1:1 Secret:BareMetalHost mapping. + +#### Deployment of compute/worker hosts + +BareMetalHost resources for workers will be created with the Secret containing the network data referenced in the `preprovisioningNetworkData` field defined in the Metal³ [image builder integration design](https://github.com/metal3-io/metal3-docs/blob/master/design/baremetal-operator/image-builder-integration.md#custom-agent-image-controller). +This will cause the baremetal-operator to create a PreprovisioningImage CRD and wait for it to become available before booting the IPA image. + +An OpenShift-specific PreprovisioningImage controller will use the provided network data to build a CoreOS IPA image with the correct ignition configuration in place, and the baremetal-operator will then use this to boot the Host. + +If not overridden, the contents of the same network data secret will be passed to the Ironic custom deploy step for CoreOS that installs the node. + +### Test Plan + +Support will be added to [dev-scripts](https://github.com/openshift-metal3/dev-scripts) for deploying the baremetal network without DHCP enabled. A CI job will populate install-config with the appropriate network configuration and verify that deployment works properly. + +### Graduation Criteria + +We expect to support this immediately on the baremetal IPI platform. + +#### Dev Preview -> Tech Preview + +N/A + +#### Tech Preview -> GA + +N/A + +#### Removing a deprecated feature + +N/A + +### Upgrade / Downgrade Strategy + +There should be little impact on upgrades and downgrades. Nodes are deployed with network configuration baked into the image, which means it will remain over upgrades or downgrades. NetworkManager keyfiles are considered a stable interface so any version of NetworkManager should be able to parse them equally. + +Any additions or deprecations in the keyfile interface would need to be handled per the NetworkManager policy. + +### Version Skew Strategy + +As this feature targets day-1 configuration there should be no version skew. Day-2 operation will be handled by other components which are outside the scope of this document. + +## Implementation History + +4.9: Initial implementation + +## Drawbacks + +Using NetworkManager keyfiles as the user interface for network configuration makes day-2 configuration more problematic. The only existing mechanism for modifying keyfiles after initial deployment is machine-configs, which will trigger a reboot of all affected nodes when changes are made. It also provides no validation or rollback functionality. + +## Alternatives + +### Use Kubernetes-NMState NodeNetworkConfigurationPolicy custom resources + +If we were able to install the [NNCP CRD](https://nmstate.io/kubernetes-nmstate/user-guide/102-configuration) +at day-1 then we could use that as the configuration interface. This has the advantage of matching the configuration syntax and objects used for day-2 network configuration via the operator. + +This is currently blocked on a resolution to [Enable Kubernetes NMstate by default for selected platforms](https://github.com/openshift/enhancements/pull/747). Without NMState content available at day-1 we do not have any way to process the NMState configuration to a format usable in initial deployment. +While we hope to eventually come up with a mechanism to make NMState available on day-1, we needed another option that did not make use of NMState in order to deliver the feature on time. + +Once the keyfile implementation is done it should be simpler to layer NMState over that, so the two solutions are not exclusive of each other. + +### Provide raw nmstate data + +We could expose an interface to the user which accepts raw nmstate data (not a CR wrapper). This would mean a common DSL to the NodeNetworkConfigurationPolicy resource, but not directly reusing that API, so there is risk of drift between the two APIs. + +This does not solve the problem of the user needing to pass the same configuration in multiple formats, although it does ensure that the same underlying format is used, which somewhat reduces the opportunity for errors. + +#### Create a net-new NMState Wrapper CR + +The [assisted-service](https://github.com/openshift/assisted-service/blob/0b0e3677ae83799151d11f1267cbfa39bb0c6f2e/docs/hive-integration/crds/nmstate.yaml) has created a new NMState wrapper CR. + +We probably want to avoid a proliferation of different CR wrappers for nmstate +data, but one option would be to convert that (or something similar) into a common +OpenShift API, could such an API be a superset of NNCP e.g also used for day-2? + +This would mean we could at least use a common configuration format based on nmstate with minimal changes (or none if we make it _the_ way OpenShift users interact with nmstate), but unless the new API replaces NNCP there is still the risk of configuration drift between the day-1 and day-2 APIs. From 60e6ab7e86145284ebce9007e234f437f16efe51 Mon Sep 17 00:00:00 2001 From: Zane Bitter Date: Thu, 26 Aug 2021 17:27:11 -0400 Subject: [PATCH 2/4] Switch Day-1 networking format proposal to nmstate Use nmstate instead of NetworkManager keyfiles as the user interface for configuring pre-deploy baremetal host networking. --- .../baremetal-ipi-network-configuration.md | 63 +++++++++---------- 1 file changed, 28 insertions(+), 35 deletions(-) diff --git a/enhancements/network/baremetal-ipi-network-configuration.md b/enhancements/network/baremetal-ipi-network-configuration.md index 2de1bd32b1..bf12784c80 100644 --- a/enhancements/network/baremetal-ipi-network-configuration.md +++ b/enhancements/network/baremetal-ipi-network-configuration.md @@ -138,7 +138,7 @@ In the sections below we will describe the user-facing API that contains network ### User-facing API -RHCOS already provides a mechanism to specify NetworkManager keyfiles during deployment of a new node. We need to expose that functionality during the IPI install process. There are a couple of options on how to do that: +RHCOS already provides a mechanism to specify NetworkManager keyfiles during deployment of a new node. We need to expose that functionality during the IPI install process, but preferably using [nmstate](https://nmstate.io) files as the interface for a more user-friendly experience. There are a couple of options on how to do that: * A new section in install-config. * A secret that contains base64-encoded content for the keyfiles. @@ -147,28 +147,22 @@ These are not mutually exclusive. If we implement the install-config option, we The data provided by the user will need to have the following structure: ```yaml -[hostname]: - [interface name]: - [contents of NetworkManager key file for interface] - [second interface name]: - [contents of NetworkManager key file for second interface] - etc... +: +: +etc... ``` For example: ```yaml openshift-master-0: - eth0: - [connection] - id=eth0 - uuid=18e0cec7-041c-4fb3-957c-a60c80dd9b85 - type=ethernet - etc... + interfaces: + - name: eth0 + type: ethernet + etc... openshift-master-1: - eth0: - [connection] - id=eth0 - etc... + interfaces: + - name: eth0 + etc... ``` In install-config this would look like: @@ -178,13 +172,10 @@ platform: hosts: - name: openshift-master-0 networkConfig: - - interface: eth0 - data: | - [connection] - id=eth0 - uuid=18e0cec7-041c-4fb3-957c-a60c80dd9b85 - type=ethernet - etc... + interfaces: + - name: eth0 + type: ethernet + etc... ``` Because the initial implementation will be baremetal-specific, we can put the @@ -210,9 +201,9 @@ implementation will have a 1:1 Secret:BareMetalHost mapping. BareMetalHost resources for workers will be created with the Secret containing the network data referenced in the `preprovisioningNetworkData` field defined in the Metal³ [image builder integration design](https://github.com/metal3-io/metal3-docs/blob/master/design/baremetal-operator/image-builder-integration.md#custom-agent-image-controller). This will cause the baremetal-operator to create a PreprovisioningImage CRD and wait for it to become available before booting the IPA image. -An OpenShift-specific PreprovisioningImage controller will use the provided network data to build a CoreOS IPA image with the correct ignition configuration in place, and the baremetal-operator will then use this to boot the Host. +An OpenShift-specific PreprovisioningImage controller will use the provided network data to build a CoreOS IPA image with the correct ignition configuration in place. This will be accomplished by [converting the nmstate data](https://nmstate.io/features/gen_conf.html) from the Secret into NetworkManager keyfiles using `nmstatectl gc`. The baremetal-operator will then use this customised image to boot the Host into IPA. The network configuration will be retained when CoreOS is installed to disk during provisioning. -If not overridden, the contents of the same network data secret will be passed to the Ironic custom deploy step for CoreOS that installs the node. +If not overridden, the contents of the same network data secret will be passed to the Ironic custom deploy step for CoreOS that installs the node, though this will be ignored at least initially. ### Test Plan @@ -236,7 +227,7 @@ N/A ### Upgrade / Downgrade Strategy -There should be little impact on upgrades and downgrades. Nodes are deployed with network configuration baked into the image, which means it will remain over upgrades or downgrades. NetworkManager keyfiles are considered a stable interface so any version of NetworkManager should be able to parse them equally. +There should be little impact on upgrades and downgrades. Nodes are deployed with network configuration baked into the image, which means it will remain over upgrades or downgrades. NetworkManager keyfiles are considered a stable interface so any version of NetworkManager should be able to parse them equally. The same is true of nmstate files. Any additions or deprecations in the keyfile interface would need to be handled per the NetworkManager policy. @@ -262,13 +253,7 @@ at day-1 then we could use that as the configuration interface. This has the adv This is currently blocked on a resolution to [Enable Kubernetes NMstate by default for selected platforms](https://github.com/openshift/enhancements/pull/747). Without NMState content available at day-1 we do not have any way to process the NMState configuration to a format usable in initial deployment. While we hope to eventually come up with a mechanism to make NMState available on day-1, we needed another option that did not make use of NMState in order to deliver the feature on time. -Once the keyfile implementation is done it should be simpler to layer NMState over that, so the two solutions are not exclusive of each other. - -### Provide raw nmstate data - -We could expose an interface to the user which accepts raw nmstate data (not a CR wrapper). This would mean a common DSL to the NodeNetworkConfigurationPolicy resource, but not directly reusing that API, so there is risk of drift between the two APIs. - -This does not solve the problem of the user needing to pass the same configuration in multiple formats, although it does ensure that the same underlying format is used, which somewhat reduces the opportunity for errors. +In any event, we inevitably need to store the network config data for each node in a (separate) Secret to satisfy the PreprovisioningImage interface in Metal³, so the existence of the same data in a NNCP CRD is irrelevant. In future, once this is available we could provide a controller to keep them in sync. #### Create a net-new NMState Wrapper CR @@ -278,4 +263,12 @@ We probably want to avoid a proliferation of different CR wrappers for nmstate data, but one option would be to convert that (or something similar) into a common OpenShift API, could such an API be a superset of NNCP e.g also used for day-2? -This would mean we could at least use a common configuration format based on nmstate with minimal changes (or none if we make it _the_ way OpenShift users interact with nmstate), but unless the new API replaces NNCP there is still the risk of configuration drift between the day-1 and day-2 APIs. +This would mean we could at least use a common configuration format based on nmstate with minimal changes (or none if we make it _the_ way OpenShift users interact with nmstate), but unless the new API replaces NNCP there is still the risk of configuration drift between the day-1 and day-2 APIs. And we still need a Secret to generate the PreprovisioningImage. + +### Pass NetworkManager keyfiles directly + +NetworkManager keyfiles are already used (directly with CoreOS) for UPI and when doing network configuration in the Machine Config Operator (MCO). However, they are harder to read, harder to produce, and they don't store well in JSON. + +In addition, we hope to eventually base Day-2 networking configuration on nmstate NodeNetworkConfigurationPolicy (as KubeVirt already does). So using the nmstate format provides a path to a more consistent interface in the future than do keyfiles. + +If we were eventually decide never to use NNCP and instead try to configure Day-2 networking through the Machine Config Operator, then it might perhaps be better to use keyfiles as that is what MCO uses. From ab6244c4169905fa02d19fcfdbb1075aa84c28e4 Mon Sep 17 00:00:00 2001 From: Ben Nemec Date: Wed, 1 Sep 2021 15:29:35 -0500 Subject: [PATCH 3/4] Fix line length for linter --- enhancements/network/baremetal-ipi-network-configuration.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/enhancements/network/baremetal-ipi-network-configuration.md b/enhancements/network/baremetal-ipi-network-configuration.md index bf12784c80..1622e0c830 100644 --- a/enhancements/network/baremetal-ipi-network-configuration.md +++ b/enhancements/network/baremetal-ipi-network-configuration.md @@ -201,7 +201,8 @@ implementation will have a 1:1 Secret:BareMetalHost mapping. BareMetalHost resources for workers will be created with the Secret containing the network data referenced in the `preprovisioningNetworkData` field defined in the Metal³ [image builder integration design](https://github.com/metal3-io/metal3-docs/blob/master/design/baremetal-operator/image-builder-integration.md#custom-agent-image-controller). This will cause the baremetal-operator to create a PreprovisioningImage CRD and wait for it to become available before booting the IPA image. -An OpenShift-specific PreprovisioningImage controller will use the provided network data to build a CoreOS IPA image with the correct ignition configuration in place. This will be accomplished by [converting the nmstate data](https://nmstate.io/features/gen_conf.html) from the Secret into NetworkManager keyfiles using `nmstatectl gc`. The baremetal-operator will then use this customised image to boot the Host into IPA. The network configuration will be retained when CoreOS is installed to disk during provisioning. +An OpenShift-specific PreprovisioningImage controller will use the provided network data to build a CoreOS IPA image with the correct ignition configuration in place. This will be accomplished by [converting the nmstate data](https://nmstate.io/features/gen_conf.html) +from the Secret into NetworkManager keyfiles using `nmstatectl gc`. The baremetal-operator will then use this customised image to boot the Host into IPA. The network configuration will be retained when CoreOS is installed to disk during provisioning. If not overridden, the contents of the same network data secret will be passed to the Ironic custom deploy step for CoreOS that installs the node, though this will be ignored at least initially. From f33988f0af7c20d33f37261d2112976463609c24 Mon Sep 17 00:00:00 2001 From: Ben Nemec Date: Wed, 27 Oct 2021 11:34:14 -0500 Subject: [PATCH 4/4] Updates to reflect nmstate plan --- .../baremetal-ipi-network-configuration.md | 42 ++++++++++++------- 1 file changed, 28 insertions(+), 14 deletions(-) diff --git a/enhancements/network/baremetal-ipi-network-configuration.md b/enhancements/network/baremetal-ipi-network-configuration.md index 1622e0c830..e9c5ca9a89 100644 --- a/enhancements/network/baremetal-ipi-network-configuration.md +++ b/enhancements/network/baremetal-ipi-network-configuration.md @@ -10,9 +10,10 @@ reviewers: - "@zaneb" approvers: - "@trozet" +- "@staebler" creation-date: 2021-05-21 -last-updated: 2021-07-07 -status: provisional|implementable +last-updated: 2021-10-27 +status: implementable see-also: - "/enhancements/host-network-configuration.md" @@ -28,8 +29,8 @@ requirement. ## Release Signoff Checklist -- [ ] Enhancement is `implementable` -- [ ] Design details are appropriately documented from clear requirements +- [*] Enhancement is `implementable` +- [*] Design details are appropriately documented from clear requirements - [ ] Test plan is defined - [ ] Operational readiness criteria is defined - [ ] Graduation criteria for dev preview, tech preview, GA @@ -39,7 +40,7 @@ requirement. Currently in the IPI flow, there is no way to provide day-1 network configuration which is a common requirement, particularly for baremetal users. We can build -on the [UPI static networking enhancements](rhcos/static-networking-enhancements.md) +on the [UPI static networking enhancements](https://github.com/openshift/enhancements/blob/master/enhancements/rhcos/static-networking-enhancements.md) to enable such configuration in the IPI flow. ## Motivation @@ -117,15 +118,15 @@ and therefore need to provide a static configuration for my primary network. There is no way to provide [MachineSpecific Configuration in OpenShift](https://github.com/openshift/machine-config-operator/issues/1720) so I am forced to use the UPI flow which is less automated and more prone to errors. -### Risks and Mitigations +### API Extensions + +This does not modify the API of the cluster. -In some existing workflows, kubernetes-nmstate is used to do network configuration on day-2. Using a different interface for day-1 introduces the potential for mismatches and configuration errors when making day-2 changes. It should be possible to mitigate this by generating the NetworkManager keyfiles using the nmstatecli, which will allow the same nmstate config to be used directly in the day-2 configuration. +### Risks and Mitigations -An added benefit of writing nmstate configurations instead of NetworkManager -keyfiles directly is that if/when a day-1 nmstate-based interface becomes -available the config files can be directly inserted into the new interface. -However, this is not a complete solution because it still requires manually copying -configuration to multiple locations which introduces the potential for errors. +In some existing workflows, kubernetes-nmstate is used to do network configuration on day-2. Using a different interface for day-1 introduces the potential for mismatches and configuration errors when making day-2 changes. +However, this is mitigated by the fact that the exact same configuration data can be used for both interfaces. The nmstate configuration provided to the installer can be copied directly into a NodeNetworkConfigurationPolicy for kubernetes-nmstate. +While there's still the potential for user error, the process is much simpler and less error-prone than if completely different formats were used. ## Design Details @@ -143,7 +144,7 @@ RHCOS already provides a mechanism to specify NetworkManager keyfiles during dep * A new section in install-config. * A secret that contains base64-encoded content for the keyfiles. -These are not mutually exclusive. If we implement the install-config option, we will still need to persist the configuration in a secret so it can be used for day-2. In fact, the initial implementation should likely be based on secrets, and then install-config integration can be added on top of that to improve the user experience. This way the install-config work won't block implementation of the feature. +These are not mutually exclusive. If we implement the install-config option, we will still need to persist the configuration in a secret so it can be used for day-2. The data provided by the user will need to have the following structure: ```yaml @@ -236,13 +237,26 @@ Any additions or deprecations in the keyfile interface would need to be handled As this feature targets day-1 configuration there should be no version skew. Day-2 operation will be handled by other components which are outside the scope of this document. +### Operational Aspects of API Extensions + +NA + +#### Failure Modes + +Improper network configuration may cause deployment failures for some or all nodes in the cluster, depending on the nature of the misconfiguration. + +#### Support Procedures + +Because a networking failure is likely to make a node inaccessible, it may be necessary to access the failed node via its BMC (iDRAC, iLO, etc.) to determine why the network config failed. + ## Implementation History 4.9: Initial implementation ## Drawbacks -Using NetworkManager keyfiles as the user interface for network configuration makes day-2 configuration more problematic. The only existing mechanism for modifying keyfiles after initial deployment is machine-configs, which will trigger a reboot of all affected nodes when changes are made. It also provides no validation or rollback functionality. +Adds a dependency on NMState. However, NMState provides a strong backward compatibility +promise (much like NetworkManager itself), so this should be a stable interface. ## Alternatives