Tuesday, 16 February 2016

Certificate Status Alarm Triggered After Upgrading vCenter To 6.0

Written by Suhas Savkoor



So, recently I have received a handful of cases, where the certificate status alarm is triggered, when the existing 5.x environment is upgraded to 6.x. You can acknowledge and clear out the alarm, however after certain days, this alert is triggered back. When this alert is triggered, the vCenter health service enters a warning state. 

When you login to Web Client, click the certificate in the address bar and view certificate information, then you will notice that the there is quite some time for the certificate to expire. 

However, when checking the the vpxd logs for the vCenter:
vCenter Server vpxd log location:
C:\Program Data\VMware\vCenter Server\logs\vmware-vpx
You will see the below logging:
2015-07-17T09:05:18.767+02:00 warning vpxd[05124] [Originator@6876 sub=Main opID=CheckCertificateExpiry-312b91e5] [Vpxd::VecsUtil::CheckCertificatesFromStore] Certificate [Subject: CN=SMS-120924141331507,O=VMware] from store SMS will expire on 2014-09-24 12:13:31.000. 

2015-07-17T09:48:15.592+02:00 warning vpxd[03620] [Originator@6876 sub=[SSO][GroupcheckAdapter] opID=7413e525] [FindAllParentGroups] Maybe SSO Groupcheck is expired, trying to re-login. Exception: class Sso::Fault::NotAuthenticated::Exception(sso.fault.NotAuthenticated)

Here the Storage Monitoring Service (SMS) 5.5 certificate is still in the VECS (VMware Endpoint Certificate Store) and has expired. This certificate is no longer used in version vCenter Server 6.0.

How to remove it?

1. Login to the vCenter machine and open a command prompt in Administrative mode
2. Change the directory to:
cd "C:\Program Files\VMware\vCenter Server\vmafdd"
3. Run the below command to display the cert for SMS service.
vecs-cli entry list --store SMS --text | more
The output is something like below:

Alias : sms_self_signed
Entry type :    Private Key
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            01:37:e2:a8:83:a9
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: O=VMware, CN=SMS-120612174523625
        Validity
            Not Before: Jun 12 21:45:23 2011 GMT
            Not After : Jun 12 21:45:23 2014 GMT
        Subject: O=VMware, CN=SMS-120612174523625
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:d0:f9:5b:15:83:cc:5e:d9:cd:f1:05:1d:1b:54:
                    24:da:93:0f:2e:cb:d6:98:55:68:a3:ec:80:dc:1a:
                    0a:3a:c8:a0:96:bf:70:61:5a:50:3a:c1:a2:b8:6c:
                    4a:69:90:9f:eb:2c:ae:8c:6f:a7:63:c0:8d:60:a7:
                    41:85:04:23:67:5a:b8:50:d4:60:36:3f:a6:85:08:
                    56:ba:2c:be:38:ea:be:a1:49:0b:c5:7e:cd:4f:19:

Here the alias is: sms_self_signed

4. Make a note of the cert alias and run the below command to remove the cert:
vecs-cli entry delete --store SMS --alias <certificate_alias>
5. Restart the Web Client service. If there is an alert triggered prior to this procedure, clear it and monitor your environment if the alert re-appears.

99.99 percent, you are in the green zone sergeant!!