oVirt IPI enhancement document #61

rgolangh · 2019-10-06T16:15:50Z

Enhancement document to cover the addition of oVirt platform for IPI

Related PRs:

enhancements/platforms/ovirt-rhv-ipi.md

sdodson · 2019-10-07T15:08:12Z

/cc @crawford @abhinavdahiya

abhinavdahiya · 2019-10-07T18:42:53Z

enhancements/platforms/ovirt-rhv-ipi.md

+- essentially providing an internal cluster-level DNS service using mDNS and
+ coreDNS, and load-balancing using keepalived.
+
+Components involved:


what about the ingress-controller?

I'm adding a note to reference this in baremetal networking doc.
Also, I don't think I need to change something in ingress controller itself but
I do need t ask the user for the ingress IP and hence I added this in the PR the PlatformStatus:

type OvirtPlatformStatus struct { ... // ingressIP is an external IP which routes to the default ingress controller. // The IP is a suitable target of a wildcard DNS record used to resolve default route host names. IngressIP string `json:"ingressIP,omitempty"` }

abhinavdahiya · 2019-10-07T18:44:41Z

enhancements/platforms/ovirt-rhv-ipi.md

+of the installation: **url**, **user** and **password** of ovirt api. The installer \
+will validate it can communicate with the api, otherwise it will fail to proceed.\
+Next the installer will ask to choose the ovirt **cluster**, where the VMs will be\
+created, and next would be a *template* from that cluster for those VMs.\


Can you link or provide details around what is a template?

done

I would like to see these things detailed:

1. How does the boot-images for VMs work?

Will adding this document help? [coreos ignition support][https://ovirt.org/develop/release-management/features/virt/coreos-ignition-support.html]

2. How the ingress-controllers are setup, internally for cluster and how it would be exposed for users.

3. What is the storage setup.

As far as I understand this is day-two operation needed for the registry?

4. Does is turn on any cloud-provider code in k8s? If yes, how are credentials handled?

no, cloud provider is not needed.

Also the docs references Baremetal networking and I couldn't find link for that..

It is at the bottom

https://ovirt.org/develop/release-management/features/virt/coreos-ignition-support.html

404s

How does the boot-images for VMs work?

Where will you get the boot-images from? For bootstrap, control-plane and compute.
Is there anything special when providing large ignition files like bootstrap.ign

https://ovirt.org/develop/release-management/features/virt/coreos-ignition-support.html

404s

It works

How does the boot-images for VMs work?

Where will you get the boot-images from? For bootstrap, control-plane and compute.

For this stage the users will upload the RHCOS qcow image as a template. Later in the survey they will see it and choose it. I add a section to describe that.
I may change that in the future to be easier to use.

Is there anything special when providing large ignition files like bootstrap.ign

No. The boostrap.ign works the same ways for masters and workers, our DB doesn't have the limitation I think openstack had/has

For this stage the users will upload the RHCOS qcow image as a template. Later in the survey they will see it and choose it. I add a section to describe that.
I may change that in the future to be easier to use.

So users are allowed to pick a template other than vetted by the installer in IPI??

that is a departure from other platforms..

How does the installer does that? guide me here, and I'll asses how much work do I need to make it work like that.

on AWS, the installer picks the AMI from the embedded https://github.com/openshift/installer/blob/909f4f8409de9ffe561779bd9f46909a0a14da1d/data/data/rhcos.json#L2-L51 and then creates the machines.

on Azure, the installer picks the vhd URL from embedded https://github.com/openshift/installer/blob/909f4f8409de9ffe561779bd9f46909a0a14da1d/data/data/rhcos.json#L54 -> imports the VHD to percluster storage blob -> Creates an Image using the VHD and creates machines using the Image.

similarly on GCP.

openstack is also in the process to moving to this model, openshift/installer#2473

abhinavdahiya · 2019-10-07T18:45:52Z

enhancements/platforms/ovirt-rhv-ipi.md

+terraform, which automatically load `.auto.tfvars.json`. `terraform.tfstate` is\
+the actual declaration of the resources and with it terraform creates the VMs.\
+
+3. Bootstrap


This needs to take into account how cluser-etcd-operator fits into bootstrapping.

Can you elaborate?

The impact of the etcd-operator on the need publishing of the SRV records will probably mean that ovirt, baremetal and openstack platforms can have simpler bootstraping, but it is not yet understood how this is going to change. I'll add it as related enhancement.

abhinavdahiya · 2019-10-07T18:46:38Z

enhancements/platforms/ovirt-rhv-ipi.md

+The bootstrap VM has a huge ignition config set using terraform and is visible\
+in the `terraform.tfstate` file. oVirt boots that VM with that content and the\
+bootstraping begins when the `bootkube.service` systemd service starts.\
+The bootstrap job is to:


I would like these steps to outline what special is being done with oVirt rather the bootstrap process in general

Nothing special in fact. Do you have a link that explains the bootstrap process?

@abhinavdahiya ^

enhancements/platforms/ovirt-rhv-ipi.md

abhinavdahiya · 2019-10-07T18:53:45Z

I would like to see these things detailed:

How does the boot-images for VMs work?
How the ingress-controllers are setup, internally for cluster and how it would be exposed for users.
What is the storage setup.
Does is turn on any cloud-provider code in k8s? If yes, how are credentials handled?

Also the docs references Baremetal networking and I couldn't find link for that..

rgolangh · 2019-10-08T05:11:48Z

enhancements/platforms/ovirt-rhv-ipi.md

+
+- Storage
+
+    Recommended storage setup?


@mykaul I want to write down a word about recommended storage setup. Can you help? is there anything to take into account?

Assuming there's an oVirt CSI driver, then exposing some of the properties (like thin or preallocated provisioning) makes sense. Otherwise, it should be quite straightforward.

This work establishes an IPI installation on oVirt platform. Installation document: https://github.com/openshift/installer/tree/master/docs/user/oviret The enhancement document: openshift/enhancements#61 The installation uses a similar network infrastructure to baremetal. https://github.com/openshift/installer/blob/master/docs/design/baremetal/networking-infrastructure.md Signed-off-by: Roy Golan <rgolan@redhat.com>

This work establishes an IPI installation on oVirt platform. Installation document: https://github.com/openshift/installer/tree/master/docs/user/ovirt The enhancement document: openshift/enhancements#61 The installation uses a similar network infrastructure to baremetal. https://github.com/openshift/installer/blob/master/docs/design/baremetal/networking-infrastructure.md Signed-off-by: Roy Golan <rgolan@redhat.com>

This work is part of making oVirt a supported platform as mentioned in this the enhancement document: openshift/enhancements#61 oVirt's cluster-api-providr[1] is added to images.json and it returned for platform type 'ovirt'. [1] https://github.com/ovirt/cluster-api-provider-ovirt Related-to: openshift/installer#1948 Signed-off-by: Roy Golan <rgolan@redhat.com>

Publish ovirt/cluster-api-ptovider-ovirt image Enhancement: openshift/enhancements#61 Signed-off-by: Roy Golan <rgolan@redhat.com>

Enhancement: openshift/enhancements#61 Signed-off-by: Roy Golan <rgolan@redhat.com>

This work establishes an IPI installation on oVirt platform. Installation document: https://github.com/openshift/installer/tree/master/docs/user/ovirt The enhancement document: openshift/enhancements#61 The installation uses a similar network infrastructure to baremetal. https://github.com/openshift/installer/blob/master/docs/design/baremetal/networking-infrastructure.md Signed-off-by: Roy Golan <rgolan@redhat.com>

sdodson · 2020-02-27T15:53:18Z

@rgolangh should we go ahead and merge this? I don't know how much actual implementation has deviated from what's outlined here but we should make sure that the enhancement gets merged.

dougsland · 2020-06-09T18:44:05Z

@sdodson @abhinavdahiya looks like an old PR. I would consider merging (or abandon?) this document. The oVirt support is already working in the master tree with @rgolangh patches and also listed as one of the platform supported in the README.

dougsland · 2020-06-09T18:46:41Z

enhancements/platforms/ovirt-rhv-ipi.md

+will validate it can communicate with the api, otherwise it will fail to proceed.\
+Next the installer will ask to choose the ovirt **cluster**, where the VMs will be\
+created, and next would be a *template* from that cluster for those VMs.\
+*Note* that it is the user's responsibility to upload the relevant RHCOS template\


dougsland

LGTM, nit changes.

sdodson · 2020-06-09T19:32:13Z

/lgtm
/hold
@rgolangh As long as there are no major deviations between actual implementation and what's written up here go ahead and lift the hold so that we can merge this. If there are significant deviations lets get this updated and merged ASAP.

openshift-ci-robot · 2020-06-09T19:32:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rgolangh, sdodson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sdodson]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

russellb · 2020-08-13T15:24:21Z

@rgolangh can you take a look at this and ensure it reflects what has already been implemented? then we can just merge this

Signed-off-by: Roy Golan <rgolan@redhat.com>

rgolangh · 2020-10-14T14:45:01Z

@sdodson @russellb I updated the section of the template as noted here #61 (comment)

This enhancement needs more updates (CSI for example) but for now I'm good with merging it. I neglected it long enough.

openshift-bot · 2021-01-12T18:12:13Z

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

russellb · 2021-01-22T16:56:42Z

OK, I'll merge and follow-up PRs can update this as necessary. It should be in the repo now to reflect what was already done.

/hold cancel
/lgtm

mykaul · 2021-02-01T07:42:28Z

enhancements/platforms/ovirt-rhv-ipi.md

+This document describes how `oVirt` becomes a platform provider for Openshift. \
+`oVirt` is a virtualization platform and is similar to the `baremetal` platform \
+ provider it is lacking DNS and Load-Balancing services, but it has the advantage \
+if software-defining your data-center, utilizing existing hardware and making \ 


openshift-ci-robot requested review from deads2k and runcom October 6, 2019 16:16

openshift-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 6, 2019

rgolangh mentioned this pull request Oct 6, 2019

Add oVirt platform status openshift/api#451

Merged

sdodson reviewed Oct 7, 2019

View reviewed changes

enhancements/platforms/ovirt-rhv-ipi.md Show resolved Hide resolved

sdodson reviewed Oct 7, 2019

View reviewed changes

enhancements/platforms/ovirt-rhv-ipi.md Outdated Show resolved Hide resolved

openshift-ci-robot requested review from abhinavdahiya and crawford October 7, 2019 15:08

rgolangh force-pushed the master branch from 735eb5a to 849bd4a Compare October 7, 2019 18:35

rgolangh changed the title ~~oVirt IPI enhancment document~~ oVirt IPI enhancement document Oct 7, 2019

abhinavdahiya reviewed Oct 7, 2019

View reviewed changes

enhancements/platforms/ovirt-rhv-ipi.md Outdated Show resolved Hide resolved

abhinavdahiya reviewed Oct 7, 2019

View reviewed changes

enhancements/platforms/ovirt-rhv-ipi.md Outdated Show resolved Hide resolved

abhinavdahiya reviewed Oct 7, 2019

View reviewed changes

enhancements/platforms/ovirt-rhv-ipi.md Outdated Show resolved Hide resolved

rgolangh force-pushed the master branch from 849bd4a to c9c1ead Compare October 8, 2019 05:06

rgolangh commented Oct 8, 2019

View reviewed changes

This was referenced Oct 8, 2019

Add ovirt as a platform openshift/machine-config-operator#766

Closed

Add ovirt as a platform openshift/machine-config-operator#767

Merged

Add ovirt installer provider openshift/installer#1948

Merged

rgolangh mentioned this pull request Oct 21, 2019

Add intaller e2e tests for oVirt platform openshift/release#4340

Merged

rgolangh added a commit to rgolangh/release that referenced this pull request Jan 16, 2020

Publish cluster-api-provider-ovirt image

230a196

Publish ovirt/cluster-api-ptovider-ovirt image Enhancement: openshift/enhancements#61 Signed-off-by: Roy Golan <rgolan@redhat.com>

rgolangh added a commit to rgolangh/release that referenced this pull request Jan 16, 2020

Publish ovirt-csi-driver image

26b7e80

Enhancement: openshift/enhancements#61 Signed-off-by: Roy Golan <rgolan@redhat.com>

dougsland reviewed Jun 9, 2020

View reviewed changes

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 9, 2020

openshift-ci-robot assigned sdodson Jun 9, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 9, 2020

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 9, 2020

rgolangh added 2 commits October 14, 2020 17:34

oVirt IPI enhancement document

8564d23

Signed-off-by: Roy Golan <rgolan@redhat.com>

reffer to the installer overview doc

03a29f4

Signed-off-by: Roy Golan <rgolan@redhat.com>

rgolangh force-pushed the master branch from f09fdb1 to 03a29f4 Compare October 14, 2020 14:43

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Oct 14, 2020

openshift-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 12, 2021

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 22, 2021

openshift-ci-robot assigned russellb Jan 22, 2021

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 22, 2021

openshift-merge-robot merged commit 007853c into openshift:master Jan 22, 2021

mykaul reviewed Feb 1, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

oVirt IPI enhancement document #61

oVirt IPI enhancement document #61

rgolangh commented Oct 6, 2019 •

edited

Loading

sdodson commented Oct 7, 2019

abhinavdahiya Oct 7, 2019

rgolangh Oct 8, 2019

abhinavdahiya Oct 7, 2019

rgolangh Oct 8, 2019

abhinavdahiya Oct 10, 2019

rgolangh Oct 10, 2019

abhinavdahiya Oct 10, 2019

rgolangh Oct 10, 2019

abhinavdahiya Oct 10, 2019

abhinavdahiya Oct 7, 2019

rgolangh Oct 7, 2019

abhinavdahiya Oct 10, 2019

rgolangh Oct 17, 2019

abhinavdahiya Oct 7, 2019

rgolangh Oct 7, 2019

dougsland Jun 9, 2020

abhinavdahiya commented Oct 7, 2019

rgolangh Oct 8, 2019

mykaul Oct 8, 2019

sdodson commented Feb 27, 2020

dougsland commented Jun 9, 2020

dougsland Jun 9, 2020

dougsland left a comment

sdodson commented Jun 9, 2020 •

edited

Loading

openshift-ci-robot commented Jun 9, 2020

russellb commented Aug 13, 2020

rgolangh commented Oct 14, 2020

openshift-bot commented Jan 12, 2021

russellb commented Jan 22, 2021

mykaul Feb 1, 2021

oVirt IPI enhancement document #61

oVirt IPI enhancement document #61

Conversation

rgolangh commented Oct 6, 2019 • edited Loading

sdodson commented Oct 7, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abhinavdahiya commented Oct 7, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sdodson commented Feb 27, 2020

dougsland commented Jun 9, 2020

Choose a reason for hiding this comment

dougsland left a comment

Choose a reason for hiding this comment

sdodson commented Jun 9, 2020 • edited Loading

openshift-ci-robot commented Jun 9, 2020

russellb commented Aug 13, 2020

rgolangh commented Oct 14, 2020

openshift-bot commented Jan 12, 2021

russellb commented Jan 22, 2021

Choose a reason for hiding this comment

rgolangh commented Oct 6, 2019 •

edited

Loading

sdodson commented Jun 9, 2020 •

edited

Loading