vSphere 7 – Certificates with VMCA as Subordinate

For enterprises that need fully trusted SSL certificates for the vSphere 7.0 environment, you have two basic options:

Full Custom Mode: Manually replace all certificates for vCenter and the ESXi hosts with your trusted certificates. Subordinate CA Mode: Use the built-in VMCA service as an official subordinate CA of your existing PKI infrastructure. After the initial configuration, automates the issuing of SSL certs for your vSphere environment. This is the method covered in this blog post. VMware offers two other certificate options: Fully Managed and hybrid mode, for a total of four certificate options. You can find out more about all of them in this VMware blog post.

In a high-security environment, it is very likely the security team will NOT let you configure the vCenter VMCA as a subordinate CA. So, you will be left with the full custom mode if you want 100% of the certificates trusted. However, if you are in a situation where you can configure the VMCA as a subordinate CA, this post is for you!

Note: Before you do this replacement in production, I strongly urge you to set up a test vCenter instance and run this entire procedure. Botched certificate replacements can lead to bad days. And another tip for a lab test is to snapshot the vCenter VM before you try the process. That way, you can easily roll back should things go south.

Enable SCP on the VCSAs

We will be transferring files back and forth from the VCSA, so we need to enable SCP. Run these steps if you don’t already have SCP enabled (it’s disabled by default).

shell.set --enable True
shell
chsh -s /bin/bash root

Generate the Certificate Signing Request (CSR)

/usr/lib/vmware-vmca/bin/certificate-manager

The Certificate manager created two files:

/tmp/vmca_issued_csr.csr
/tmp/vmca_issued_key.key

Download the VMCA Files

Signing the Subordinate Certificate

How you will be minting your Certificate is highly dependent on your PKI infrastructure. In my case, I’m running a two-tier Windows Server 2019 CA. So I’ll walk you through that process. The 2019 CA is configured exactly like I’ve written about in my Windows Server 2019 Two-Tier PKI CA series. Have a look at those three posts if you are in a lab and don’t already have a running CA.

Configuring the Subordinate Certificate Template

If you followed my Windows Server 2019 CA guide, you would need to authorize a new template type so that you can issue a subordinate CA certificate via the CA’s web interface.

Submit Certificate Request

Validating the VMCA Certificate

While going through this procedure in my lab, I ran into a certificate issue as described in VMware KB 71120: “ERROR certificate-manager ‘lstool get-site-id’ failed: 1” in the /log/vmware/vmcad/certificate-manager.log. The symptom of this was the VMCA replacement failing at 85% and being unable to roll back. Quite a sticky situation.

Per the KB, VMware does NOT support the Signature Algorithm RSASSA-PSS. I looked at my Certificate, and sure enough, that was my problem. So open the certnew.cer file in Explorer and verify that you are using another signature algorithm, such as sha256RSA. If you have a CA that is issuing certs with the RSASSA-PSS algorithm, check out my blog post on how to change that setting for Microsoft CAs. DO NOT PROCEED if RSASSA-PSS is present. The replacement procedure WILL FAIL. It would be great if VMware validated the certificates better before installing them to head off this issue.

Obtaining CA Certificate Chain

Configuring the VMCA

VMCA Certificate Validation

Renewing ESXi Certificates

Unfortunately, when you configure the VMCA to be a subordinate CA, the process does NOT automatically renew/replace the ESXi host certificates. And, there’s another little gotcha too. Suppose you manually renew the ESXi host certificate within 24 hours of configuring your VMCA as a subordinate. In that case, it will fail with an error 70034: A general system error occurred: Unable to get signed Certificate for the host: esxi_hostname. Error: Start Time Error (70034)

To work around this issue, VMware wrote KB 2123386, which involves modifying an existing vCenter 7 advanced settings. To change this setting:

Updating ESXi Machine Certificate

comments powered by Disqus