Harbor registry fails to start when CA signed certificate added during OVA deployment

[mattstocum]

VIC Product version:

OVA vic-v1.2.0-rc3-ce9fca67.ova

Deployment details:

vCenter 6.5 VSA 6.5.0.5100

Steps to reproduce:

1. Deploy VIC from OVA.
2. During initial deployment, enter a CA signed certificate in section 4.2 and key in section 4.3 (Management Portal Configuration) of the vApp options.
3. Power on VM and wait for it to finish booting

Actual behavior:

Harbor registry (port 443) fails to start. Port 9443 does start properly using the correct certificate.

Expected behavior:

Harbor registry should start.

Workaround:

1. Shutdown VM
2. Clear sections 4.2 and 4.3 from vApp options
3. Start VM and verify that the registry is listening with a self-signed certificate
4. Shutdoown VM
5. Enter CA signed certificate and key in 4.2 and 4.3
6. Start VM and verify that the registry is listening with CA signed certificate

Notes:
I'm not sure if this is relevant, but the CA cert I have does have an intermediate certificate that I am not able to supply using the vApp options. The intermediate certificate is installed and trusted on the systems where I need to access VIC from, however.

[andrewtchin]

Hi @mattstocum thank you for your report
As an additional data point - did Admiral start with the correct certificate in this deployment? https://<appliance_ip>:8282

[mattstocum]

@andrewtchin Yes, Admiral started properly using the same certificate.

[andrewtchin]

@reasonerjt could you triage this?

[mattstocum]

On further inspection with the 1.2.0 release, I have some new information. harbor_startup.service is what's failing to start

systemctl status harbor_startup.service
● harbor_startup.service - Harbor Startup Configuration
   Loaded: loaded (/usr/lib/systemd/system/harbor_startup.service; disabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2017-09-13 17:58:07 UTC; 6min ago
     Docs: http://github.com/vmware/harbor
  Process: 797 ExecStartPre=/usr/bin/bash /etc/vmware/harbor/configure_harbor.sh (code=exited, status=1/FAILURE)

Sep 13 17:58:07 anchorage.lebow.drexel.edu systemd[1]: Starting Harbor Startup Configuration...
Sep 13 17:58:07 anchorage.lebow.drexel.edu bash[797]: cp: cannot stat '/data/admiral/cert/ca.crt': No such file or directory
Sep 13 17:58:07 anchorage.lebow.drexel.edu systemd[1]: harbor_startup.service: Control process exited, code=exited status=1
Sep 13 17:58:07 anchorage.lebow.drexel.edu systemd[1]: Failed to start Harbor Startup Configuration.
Sep 13 17:58:07 anchorage.lebow.drexel.edu systemd[1]: harbor_startup.service: Unit entered failed state.
Sep 13 17:58:07 anchorage.lebow.drexel.edu systemd[1]: harbor_startup.service: Failed with result 'exit-code'.

After further testing, it looks like the certificate needs to be omitted from Management Portal (port 8282) rather than the File Server. I guess the Admiral startup generates /data/admiral/cert/ca.crt on boot, and Harbor is checking for the existence of the file. I believe the proper Work Around steps would be to boot them VM once using self-signed certs for all services, shutdown, add CA signed certs to both locations, then boot the VM.

[andrewtchin]

Thanks for the additional info we'll take a look at this

[andrewtchin]

@mattstocum just to be clear - the issue you described in slack about vCenter redirecting back to Admiral's IP instead of hostname seems different than what you wrote in this issue. Are these 2 separate issues?

[mattstocum]

@andrewtchin yes, two separate issues. Sorry for the confusion. The redirection to the IP, I think would be better classified as a feature request, rather than a bug.

[andrewtchin]

Cool 797 is for the described issue, I'm opening another for the redirect issue

[andrewtchin]

The cause of this is that we don't put the ca cert in /data/admiral/cert/ca.crt so that file doesnt exist and thats why harbor fails to star