Thursday, 3 November 2016

VDP Status Error: Data Domain Storage Status Is Not Available

Today when I logged into my lab to do bit of testing on my vSphere Data Protection 6.1 appliance I noticed the following in the VDP plugin in Web Client:

It was stating, "The Data Domain storage status is not available. Unable to get system information
So, I logged into the vdp-configure page and switched to the Storage tab and noticed the similar error message.

So the first instinct was to go ahead and Edit the Data Domain Settings to try re-adding it back. But that failed too, with the below message.

When a VDP is configured to a Data Domain, these configuration error logging will be in ddrmaintlogs located under /usr/local/avamar/var/ddrmaintlogs

Here, I noticed the following:

Oct 29 01:54:27 vdp58 ddrmaint.bin[30277]: Error: get-system-info::body - DDR_Open failed:, DDR result code: 5040, desc: calling system(), returns nonzero
Oct 29 01:54:27 vdp58 ddrmaint.bin[30277]: Error: <4780>Datadomain get system info failed.
Oct 29 01:54:27 vdp58 ddrmaint.bin[30277]: Info: ============================= get-system-info finished in 62 seconds
Oct 29 01:54:27 vdp58 ddrmaint.bin[30277]: Info: ============================= get-system-info cmd finished =============================
Oct 29 01:55:06 vdp58 ddrmaint.bin[30570]: Warning: Calling DDR_OPEN returned result code:5040 message:calling system(), returns nonzero

Well, this says a part of the problem. It is unable to fetch the Data Domain System Information. So the next thing is to see vdr-configure logs located under /usr/local/avamar/var/vdr/server_logs
All operations done in the vdp-configure page will be logged under vdr-configure logs.

Here, the following was seen:

2016-10-29 01:54:28,564 INFO  [http-nio-8543-exec-9]-services.DdrService: Error Code='E30973' Message='Description: The file system is disabled on the Data Domain system. The file system must be enabled in order to perform backups and restores. Data: null Remedy: Enable the file system by running the 'filesys enable' command on the Data Domain system. Domain:  Publish Time: 0 Severity: PROCESS Source Software: MCS:DD Summary: The file system is disabled. Type: ERROR'

This is a much detailed Error. So, I logged into my Data Domain system using the "sysadmin" credentials and ran the below command to check the status of the filesystem:
# filesys status

The output was:
The filesystem is enabled and running.

The Data Domain was reporting the file-system is already up and running. Perhaps this was in a non responding / stale state. So, I re-enabled the file-system using:
# filesys enable

Post this, the data domain automatically connected to the VDP appliance and the right status was displayed.