Fixes #7104: Ensure qpidd is restarted before qpid config is run.

[ehelms]

On RHEL7, qpidd is not started fully before qpid-config is run due
to adding the candlepin certs to the NSS database needing to refresh
the qpidd service and qpid-config requiring that the qpidd service
be running when called.

[waldenraines]

I'm testing this one.

[waldenraines]

I get these errors when trying with the change from this PR:

 /Stage[main]/Foreman::Database/Foreman::Rake[apipie:cache]/Exec[foreman-rake-apipie:cache]: Failed to call refresh: /usr/sbin/foreman-rake apipie:cache returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[apipie:cache]/Exec[foreman-rake-apipie:cache]: /usr/sbin/foreman-rake apipie:cache returned 1 instead of one of [0]
 Could not start Service[foreman-tasks]: Execution of '/usr/bin/systemctl start foreman-tasks' returned 1: Job for foreman-tasks.service failed. See 'systemctl status foreman-tasks.service' and 'journalctl -xn' for details.
 /Stage[main]/Foreman::Plugin::Tasks/Service[foreman-tasks]/ensure: change from stopped to running failed: Could not start Service[foreman-tasks]: Execution of '/usr/bin/systemctl start foreman-tasks' returned 1: Job for foreman-tasks.service failed. See 'systemctl status foreman-tasks.service' and 'journalctl -xn' for details.
 /usr/share/foreman/script/foreman-config      -k ssl_ca_file -v '/etc/foreman/proxy_ca.pem'      -k ssl_certificate -v '/etc/foreman/client_cert.pem'      -k ssl_priv_key -v '/etc/foreman/client_key.pem' returned 1 instead of one of [0]
 /Stage[main]/Certs::Foreman/Exec[foreman_certs_config]/returns: change from notrun to 0 failed: /usr/share/foreman/script/foreman-config      -k ssl_ca_file -v '/etc/foreman/proxy_ca.pem'      -k ssl_certificate -v '/etc/foreman/client_cert.pem'      -k ssl_priv_key -v '/etc/foreman/client_key.pem' returned 1 instead of one of [0]

When trying with this change only:

diff --git a/manifests/candlepin.pp b/manifests/candlepin.pp
index 83d2314..22905a1 100644
--- a/manifests/candlepin.pp
+++ b/manifests/candlepin.pp
@@ -88,6 +88,7 @@ class certs::candlepin (
     exec { 'create candlepin qpid exchange':
       command => "qpid-config --ssl-certificate ${client_cert} --ssl-key ${client_key} -b 'amqps://${::fqdn}:5671' add exchange topic event --durable",
       unless  => "qpid-config --ssl-certificate ${client_cert} --ssl-key ${client_key} -b 'amqps://${::fqdn}:5671' exchanges event",
+      require => Service['qpidd'],
     } ~>
     exec { 'import CA into Candlepin truststore':
       command  => "keytool -import -v -keystore ${amqp_truststore} -storepass ${keystore_password} -alias ${certs::default_ca_name} -file ${ca_cert} -noprompt",

I get these errors:

 qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps://[hostname-redacted]:5671' add exchange topic event --durable returned 1 instead of one of [0]
 /Stage[main]/Certs::Candlepin/Exec[create candlepin qpid exchange]/returns: change from notrun to 0 failed: qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps://[hostname-redacted]:5671' add exchange topic event --durable returned 1 instead of one of [0]
 /Stage[main]/Candlepin::Service/Exec[cpinit]: Failed to call refresh: /usr/bin/wget --timeout=30 --tries=5 --retry-connrefused -qO- http://localhost:8080/candlepin/admin/init >/var/log/candlepin/cpinit.log 2>&1 && touch /var/lib/candlepin/cpinit_done returned 4 instead of one of [0]
 /Stage[main]/Candlepin::Service/Exec[cpinit]: /usr/bin/wget --timeout=30 --tries=5 --retry-connrefused -qO- http://localhost:8080/candlepin/admin/init >/var/log/candlepin/cpinit.log 2>&1 && touch /var/lib/candlepin/cpinit_done returned 4 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]

When trying with both changes, I get this:

/usr/share/foreman/script/foreman-config      -k ssl_ca_file -v '/etc/foreman/proxy_ca.pem'      -k ssl_certificate -v '/etc/foreman/client_cert.pem'      -k ssl_priv_key -v '/etc/foreman/client_key.pem' returned 1 instead of one of [0]
 /Stage[main]/Certs::Foreman/Exec[foreman_certs_config]/returns: change from notrun to 0 failed: /usr/share/foreman/script/foreman-config      -k ssl_ca_file -v '/etc/foreman/proxy_ca.pem'      -k ssl_certificate -v '/etc/foreman/client_cert.pem'      -k ssl_priv_key -v '/etc/foreman/client_key.pem' returned 1 instead of one of [0]

I have all three of these VMs still if you'd like to have a look.

[ehelms]

And without the changes, on the latest, do you see the error?

[waldenraines]

And without the changes, on the latest, do you see the error?

 /Stage[main]/Foreman::Database/Foreman::Rake[apipie:cache]/Exec[foreman-rake-apipie:cache]: Failed to call refresh: /usr/sbin/foreman-rake apipie:cache returned 1 instead of one of [0]
 /Stage[main]/Foreman::Database/Foreman::Rake[apipie:cache]/Exec[foreman-rake-apipie:cache]: /usr/sbin/foreman-rake apipie:cache returned 1 instead of one of [0]
 Could not start Service[foreman-tasks]: Execution of '/usr/bin/systemctl start foreman-tasks' returned 1: Job for foreman-tasks.service failed. See 'systemctl status foreman-tasks.service' and 'journalctl -xn' for details.
 /Stage[main]/Foreman::Plugin::Tasks/Service[foreman-tasks]/ensure: change from stopped to running failed: Could not start Service[foreman-tasks]: Execution of '/usr/bin/systemctl start foreman-tasks' returned 1: Job for foreman-tasks.service failed. See 'systemctl status foreman-tasks.service' and 'journalctl -xn' for details.
 /usr/share/foreman/script/foreman-config      -k ssl_ca_file -v '/etc/foreman/proxy_ca.pem'      -k ssl_certificate -v '/etc/foreman/client_cert.pem'      -k ssl_priv_key -v '/etc/foreman/client_key.pem' returned 1 instead of one of [0]
 /Stage[main]/Certs::Foreman/Exec[foreman_certs_config]/returns: change from notrun to 0 failed: /usr/share/foreman/script/foreman-config      -k ssl_ca_file -v '/etc/foreman/proxy_ca.pem'      -k ssl_certificate -v '/etc/foreman/client_cert.pem'      -k ssl_priv_key -v '/etc/foreman/client_key.pem' returned 1 instead of one of [0]

[iNecas]

The alternative patches to fix the issue with qpid not being ready:

• Fixes 7104 - make sure qpidd is really running after service starts puppet-service_wait#4
• Refs #7104 - ensure the qpidd is really running before configuring it #33

[waldenraines]

I'm closing this PR as I just ACK'd @iNecas' PRs.