Thursday, 21 January 2016

Update Manager Service Crashes During A Scan Operation On An ESXi Host

Written by Suhas Savkoor

Today, I came across an issue with Update Manager while working on a support request. The environment, comprised of two vCenter in linked mode and each of them having their own Update Manager server. The update manager was installed on a machine of their own. The second vCenter was running well and good, however, on the first vCenter there was an issue with the "Scan" operation.

Whenever a baseline was attached to any of the hosts under this vCenter, and a scan operation was performed, the progress would go to 10 percent, stop there for few minutes and then the vSphere Update Manager service used to stop and crash causing the VUM to lose connectivity with the vCenter.

Upon on reviewing the logs for the failure, vmware-vum-server-log4cpp.log, I noticed the following:
Error accessing stagepath C:/ProgramData/VMware/VMware Update Manager/Data/host_upgrade_packages/esxi-upgrade-ryvdmfvtoz type 1 error 0/The operation completed
This means that the patch store and the DB are not in-sync

When I browse C:\ProgramData\VMware\VMware Update Manager\Data, I do not see the host_upgrade_packages folder, and the scan is failing because it is unable to find this folder.
The install directory may vary depending on your installation settings.

What can be done?

1. If you have your old update manager (Rarely happens), then you can copy paste this folder into this directory and the scan will work good!

If not, then we will have to re-initialize the update manager database.
**Re-initializing the database will clear out the database for update manager, which means, if you had any custom baselines and patches downloaded, they will be lost**

Steps to Re-initialize the VUM database:

1. First Login to SQL management studio hosting this update manager database. Expand Database > Right click the VUM database > All Tasks > Backup. Back this database to a disk.
2. Stop the Update Manager service from services.msc
3. Open a command prompt in elevated permission mode (Administrative mode) change the drive to the VUM installed disk drive and run the below command:
cd "C:\Program Files (x86)\VMware\Infrastructure\Update Manager\" 
4. Then run the below command to re-initialize the database:
vciInstallUtils.exe -O dbcreate -C . -L . 
(Both . should be used)

5. Once command has executed restart the Update Manager service.
6. Login to vCenter > Select ESXi host > Update Manager > Admin View
7. Under Configuration tab select Download Settings and download the patches again. If you want to add your custom baselines, then you can go ahead and do so
8. Go back to Compliance View and Attach and Scan, and this time the operation should succeed!