Add support for repository rpms in HE setup

[kresss]

When setting up an ofline ovirt environment requires a repository to be setup. This change uses the existing ovirt.ovirt.repositories role to setup the repositories using a repositories RPM.

[kresss]

I need to clean up some naming in this before it is ready for a formal review. Just wanted to get it out there incase someone else hits the same issue.

[kresss]

I added a new variable to turn on this functionality. I also moved it into what I think will be a more acceptable location.

I think this is ready for someone to look at.

@arachmani If you have any comments or would like changes please let me know.

[arachmani]

ci add to whitelist

[arachmani]

ci test please

[arachmani]

@kresss Thank you for your contribution!
We need to support both upstream and downstream, which means we need to pass some vars to ovirt-repositories, so I think it's a bit complex to do that here.
For those cases we have these options https://github.com/oVirt/ovirt-ansible-collection/tree/master/roles/hosted_engine_setup#make-changes-in-the-engine-vm-during-the-deployment, so we can use hooks or connect to the localVM and configure it manually.
BTW, did you encounter any issue that this PR should fix? 
AFAICT, localVM already has the ovirt repos.

[kresss]

@arachmani Yes we are having issues installing RHV with a HE in an offline environment without satellite. We tracked it all down to the repos not being present. We have offline repos setup with all the packages. We created a ovirt repos rpm file and it all works well. I think we would still have some issues if we were using satellite in offline, but I don't have that setup to do the testing.

With the patch as it is submitted, you have to set the he_apply_repositories_role == true and set all of the ovirt_repositories* variables that you want. I should state that in the README line. I didn't remap all of the ovirt_repositories_* variables since I was applying the whole role. I saw in other areas where the hosted_engine_setup role calls only partial task files from other roles, but I didn't feel like that was needed in this case and could only add to confusion.

I noticed in the example for the engine_setup role that you applied the repositories role first. That is what lead me down this path. I didn't know about the hooks directory at the time.

In the end we expect that we would receive an RPM with this role from RH as part of our normal subscription. I would rather not have to install additional hooks into the collection structure after that before we use the roles. Having to modify the collections under /usr/share/ansible/collections is not preferred. That said, If you decide to not accept the PR, it sounds like that would be a more maintainable workaround than patching the plays.

Let me know if you would like to proceed.

Thank you,

Seth.

[jekader]

ci test please

[ovirt-infra]

Hello contributor, thanks for submitting a PR for this project!

I am the bot who triggers "standard-CI" builds for this project.
As a security measure, I will not run automated tests on PRs that are not from white-listed contributors.

In order to allow automated tests to run, please ask one of the project maintainers to review the code and then do one of the following:

1. Type ci test please on this PR to trigger automated tests for it.
2. Type ci add to whitelist on this PR to trigger automated tests for it and also add you to the contributor white-list so that your future PRs will be tested automatically. ( keep in mind this list might be overwritten if the job XML is refreshed, for permanent whitelisting, please follow ovirt dynamic plugin improvements #3 option )
3. If you are planning to contribute to more than one project, maybe it's better to ask them to add you to the project organization, so you'll be able to run tests for all the organization's projects.

[mnecas]

ci test please

[mnecas]

Please check out the changes around ansible.
As for the logic, if we need to import the role there, I'll keep it up to @arachmani

[mwperina]

+1

[mnecas]

LGTM

[kresss]

Should I squash the commits into a single commit for a merge?

[mnecas]

You don't need to do that, the Github can do it automatically when mering.
Waiting for @arachmani approval.

[arachmani]

@kresss for this kind of situation you have the he_offline_deployment parameter.
You can set he_offline_deployment to true, this will not update the packages and the deployment should complete successfully, after that, you can configure any repo and update the environment.
When I tried out this fix on oVirt (upstream), the deployment fails because we need to pass more parameters.
We also need to support RHV repo, in this case, we need to add a subscription etc.
Currently, I don't think we can merge it, since this fix leaves some loose ends.

[kresss]

@arachmani First I have to admit that I have not tried this with upstream ovirt, only RHV 4.4. I still think there is a gap here.

When we do an install with he_offline_deployment: true it fails out trying to install ovirt-engine. The ovirt-engine package is not pre-installed in the appliance vm from RH. I see how the he_offline_deployment maps to the engine_setup role's ovirt_engine_setup_offline variable. What I never found a solution to was how the ovirt-engine package would get installed without a repo to install it from. Looking into it further I see how the ovirt_engine_setup_offline is used. It only prevents updates to packages as the documenation states. It does not preclude the need to install new packages.

We are using RHEL & RHV. We can't use subscription mangement because we are in a disconnected network without a Satellite server. We do have copies of RH yum repos. Yes we have subscriptions ;)

As for more parameters, what are you refering too? If I am missing some extra parameter that makes the offline deployment work, please share.

[arachmani]

@arachmani First I have to admit that I have not tried this with upstream ovirt, only RHV 4.4. I still think there is a gap here.

When we do an install with he_offline_deployment: true it fails out trying to install ovirt-engine. The ovirt-engine package is not pre-installed in the appliance vm from RH. I see how the he_offline_deployment maps to the engine_setup role's ovirt_engine_setup_offline variable. What I never found a solution to was how the ovirt-engine package would get installed without a repo to install it from. Looking into it further I see how the ovirt_engine_setup_offline is used. It only prevents updates to packages as the documenation states. It does not preclude the need to install new packages.

The ovirt-engine package should be in the appliance.
Do you use OpenSCAP? if so, seems like you hit:
https://bugzilla.redhat.com/show_bug.cgi?id=1867158

We are using RHEL & RHV. We can't use subscription mangement because we are in a disconnected network without a Satellite server. We do have copies of RH yum repos. Yes we have subscriptions ;)

As for more parameters, what are you refering too? If I am missing some extra parameter that makes the offline deployment work, please share.

I need to check that, I'm not familiar with the repositories role.

[kresss]

Yes we are using OpenSCAP via he_apply_openscap_profile: true. It looks like the bug you referenced is related to our issues.

As for the extra varaibles for the repositories role, we are using these:

he_apply_openscap_profile: true
he_network_test: ping
he_offline_deployment: false
he_apply_repositories_role: true
ovirt_repositories_ovirt_release_rpm: http://XXXX.XXXXX/repos/XXXX_rhel8/Packages/ovirt-release-rpm-4.4-0001.29.el8.noarch.rpm
ovirt_repositories_disable_gpg_check: true
ovirt_repositories_ovirt_dnf_modules: ["pki-deps", "postgresql:12"]
he_additional_package_list: ["nss"]
ipv6_deployment: true

It is important to note that we built a custom ovirt-release-rpm which installs some repo files in /etc/yum.repos.d/

Also fair warning, even if you use the repositories role like I do with this PR and everything 'works', RHV HE still fails. Due to a FIPS bug. See: https://bugzilla.redhat.com/show_bug.cgi?id=1875363

@mwperina already submitted a patch for bz 1875363, but it hasn't been released out yet. he_additional_package_list: ["nss"] is a work around for the first issue in the bug, but we still need the patch for the second part.

[arachmani]

So, is this PR basically a workaround for the bugs we have mentioned and the fixes that didn't release yet?

[kresss]

@arachmani Trying to work around the open bugs is how I got started with this PR. I didn't know about the bugs at the time, but they have been discovered along the way. I am currently waiting for RHV 4.4.4.2 to drop so I can test everything again with the latest release of ovirt-ansible-collection. I will be happy if everything works at that point and this is not needed.

The question would then be, does this PR still have merit? The current HE-setup role enables users to install extra packages as part of the setup should it also enable them to configure the repositories for potential in-house packages? Or is it good enough to point those users down the hooks path?

[arachmani]

This PR covers a specific case when offline deployment is used with an internal repo.
In order to get it in, it should support both online and offline deployments plus oVirt and RHV, which make it somewhat complex (though possible).
IMO, we currently have all we need for a successful HE deployment, so the best solution here is to configure the repo and install the packages after deployment is finished or use the hooks option.

[kresss]

@arachmani

IMO, we currently have all we need for a successful HE deployment, so the best solution here is to configure the repo and install the packages after deployment is finished or use the hooks option.

If it worked, I wouldn't be here. Then again it is kind of Red Hat's thing to say things work but then say, we have a bug open for that. Once all the bugs are resolved with RHV 4.4.4.2, I will try this again. That probably will not be for you know a few more months, but who wanted to actually be able to use the product anyways. If needed I will use the hooks, but don't tell me that it works.

[sandrobonazzola]

@kresss can you please resolve conflicts?

[kresss]

@sandrobonazzola - I will add this to my list for Monday.

[sandrobonazzola]

@kresss we have 4.5 beta release coming in 4 days, if you resolve the conflicts we can try to get it in.

[mnecas]

The GitHub actions have a bit of problem nowadays and sometimes don't wanna start.
LGTM, @arachmani how about you?

[sandrobonazzola]

@kresss can you please rebase resolving conflicts?

[kresss]

@sandrobonazzola yeah I can get on this tomorrow.