Wednesday, 14 June 2017

VDP 6.1.4 Backup Failures For ESXi 5.1

When you backup a VM residing on ESXi 5.1 which does not share the storage with the ESXi hosting the VDP appliance, the backup will use nbd / nbdssl protocol. With VDP 6.1.4 these backups will fail.

In the backup log you will notice the following:

2017-06-14T09:21:07.869+05:00 avvcbimage Info <14700>: submitting pax container in-use block extents:
2017-06-14T09:21:07.899+05:00 avvcbimage Warning <16041>: VDDK:[NFC ERROR] NfcFssrvrProcessErrorMsg: received NFC error 2 from server: Illegal message during fssrvr session, id = 46
 2017-06-14T09:21:07.900+05:00 avvcbimage Info <16041>: VDDK:DISKLIB-LIB   : RWv failed ioId: #1 (290) (34) .
 2017-06-14T09:21:07.900+05:00 avvcbimage Info <16041>: VDDK:VixDiskLib: Detected DiskLib error 290 (NBD_ERR_GENERIC).
2017-06-14T09:21:07.900+05:00 avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_Read: Read 128 sectors at 0 failed. Error 1 (Unknown error) (DiskLib error 290: NBD_ERR_GENERIC) at 5178.

2017-06-14T09:21:07.900+05:00 avvcbimage Error <0000>: [IMG0008] VixDiskLib_Read() at offset 0 length 128 returned 1 (1) Unknown error
2017-06-14T09:21:07.900+05:00 avvcbimage Error <0000>: [IMG0008] VixDiskLib_Read() ([Prod-Datastore] VM-Prod/VM-Prod.vmdk) at offset 0 length 128 sectors returned (1) (1) Unknown error
2017-06-14T09:21:07.900+05:00 avvcbimage Info <9772>: Starting graceful (staged) termination, VixDiskLib_Read returned an error (wrap-up stage)
2017-06-14T09:21:07.901+05:00 avvcbimage Info <40654>: isExitOK()=157
2017-06-14T09:21:07.901+05:00 avvcbimage Info <16022>: Cancel detected(miscellaneous error), isExitOK(0).
2017-06-14T09:21:07.901+05:00 avvcbimage Info <9746>: bytes submitted was 0

You might also notice a back trace:

2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  INTERNAL ERROR: <0001> assert error (uapp::staging().isExitCancel() || list == __null || skipped_ntfs_chunks), /local/jenkins/workspace/client_CuttyS
ark_SLES11-64/abs2/work/src/avclient/filepipe.cpp line 101
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006b93b1
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006ba127
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006ba292
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006ba3fe
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006ba87d
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 000000000060cb74
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 000000000060d67d
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006d9379
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006dc9a1
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006dc200
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000006d9379
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000005e5af1
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 000000000056455a
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 0000000000442c07
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000004844d5
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00000000004844f9
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 000000000049a0bd
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 000000000070ac4c
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00007f8a8d282806
2017-06-14T09:21:08.322+05:00 [VcbImageBackupAssistThread]  | 00007f8a8bec79bd

This is because, in the newer release of VDP for backup over network, they have added couple of new compression algorithms. Zlib, FastLZ, SkipZ in the VDDK module. And ESXi 5.1 does not support this.

You could try making the VMs storage on ESXi 5.1 available to the VDP's ESXi host so a hotadd protocol can be used.

But the bottom line is, Please! Upgrade your ESXi from 5.1 (This is officially unsupported for 6.1.3 and 6.1.4 VDP)

Hope this helps.