Wednesday, 26 April 2017

VDP Error 1009 When Connecting To Web Client

When you try connecting to a specific VDP appliance from the web client, you will run into the generic Web Client Flash Error.

This error is generally encountered when there is an issue with VMware Tools on the VDP appliance. If you select the VDP VM and go to summary tab, you will notice that there are no tools running and there is no IP address for this VM.

The fix would be to uninstall VMware tools from VDP and re-install it back. To uninstall the VMware tools, run the script which is located under /usr/bin/
Once this script completes, the tools no longer exist on VDP. So next, you will have to re-install it back.

Right click the VM > Guest > Install or Upgrade VMtools. Then SSH into the appliance and run the below set of commands:
# mkdir /mnt/cdrom

# mount /dev/cdrom /mnt/cdrom

# cp /mnt/cdrom/VMwareTools-10.0.6-3560309.tar.gz /tmp
(This might vary depending on version of tools)

# cd /tmp

# tar -zxvf VMwareTools-10.0.6-3560309.tar.gz

# cd vmware-tools-distrib/
The VMtools install script has a ton of options if you go with manual install. The best is to choose default install by running the below command:

# ./ -d default

This will consider all the default settings and complete the tools install and you should see the VM tools status in the VM summary page as running and the appliance should connect successfully to the web client.

Hope this helps.

Tuesday, 25 April 2017

Unable To Reinstall SRM With Existing Database "The selected vCenter server does not match the one used in the previous installation"

There are scenarios where you will have to reinstall a vCenter site. When a vCenter site is reinstalled the SRM solution connected to it has to be reinstalled. The reinstall is usually done with an existing database is because you would like to save your protection groups and recovery plans without the need to recreate them all. However, this reinstall with existing database would fail with the following error:

The selected vCenter server does not match the one used in the previous installation

This is because, every vCenter has a vCenter UUID with it. In this case, the vCenter being used was 6.0 and SRM was 6.1.

If you look at the instance.cfg file. It is located under:

Installation directory/VMware/vCenterServer/vmware-vpx/cfg

In this instance.cfg you will have the vCenter UUID.

When SRM is registered to vCenter, this instanceUUid is saved in the SRM database. Now, when the vCenter is reinstalled, you will have a new instanceUuid which does not match the one stored in SRM Database and the installation fails.

So, you will have to manually update the SRM database with this instanceUuid prior to installation.

The table you need to look into SRM is pd_localsite. If you connect to your DB and view this table, you will see something as below. I am using Embedded Postgres for SRM, so the query would be:

select * from pd_localsite;

The output:
 db_id |    mo_id    | ref_count |            name             | vcaddress | vcport | vccertthumbprint |                 uuid                 |                        siteurl                        |            vcinstanceuuid            |    domain    |                                  sucertificate
     1 | DrLocalSite |         1 | vcenter-prod.happycow.local |           |      0 |                  | 0a063f18-a09c-4126-9556-db5bd30c37dd | https://psc-prod.happycow.local:443/lookupservice/sdk | c4c2202e-4acd-4b35-a2da-0947a3429658 | vmware.local | C:\Program Files\VMware\VMware vCenter Site Re
covery Manager\bin\
(1 row)

So the vcinstanceuuid needs to be matching your instanceUuid of instance.cfg 

Once this is done, proceed with the SRM installation and it will work just fine. 

Tuesday, 18 April 2017

VDP - Avmar Migration: Part-4: Migrating Data And Jobs From VDP To Avamar

VDP - Avmar Migration: Part-3: Avamar Client And Configuring To vCenter Server

In this part, we will replicate VDP backup data to Avamar and then migrate the jobs. First, to replicate the data from VDP to Avamar, we create a regular replication job. Let's have a look at this.

Connect to the VDP appliance from the web client and click the Restore tab. To give you a brief overview, here is the restore points for my VM-A

Click the Replication tab and click Replication job actions. and click Create New Replication Job.

You will be presented with the below wizard. I want to replicate my current Image backups, so the first option and click Next.

I will replicate the individual client, and select VM-A and then click Next.

Select which backup schedule you would like to replicate. This is as per your need and then click Next.

Enter the AVE FQDN/IP as the hostname, the port for replication is 29000. The username to be used is repluser and the password for repluser is the one you setup during the Avamar configuration. Click Verify Authentication. Once the connection test is completed successfully, click Next.

Select how frequently you need to replicate. This does not really matter here, because it is going to be one time replication, because post the migration source VDP will be in a unusable state. Click Next.

Select a retention policy for the replicated backup as per your requirement and click Next.

Provide a name to this replication job and then click Next and then Finish the wizard.

Once the job is created, select the job and click Replicate Now to begin the task.

Once the replication is completed, you can see the replicated data in the Avamar end.

You will still notice there is a red X for Backup Jobs as the jobs are not transferred. It is only the VDP backup data that is replicated to Avamar.

Next we will begin migrating the jobs. To do this, open a browser and go to:

Once on this page, click Avamar Migration and click Migrate.

Check the Terms and conditions to proceed further and click Next.

Provide the AVE FQDN, the root credentials would be for Avamar root and not the OS root password. Click Verify Authentication.

Once the authentication succeeds, click OK and then click Next.

You can select the backup jobs you need to migrate here. 

You can switch the tab to Backup Verification and select the required verification jobs and then click Next.

A validation will be performed and if everything is successful, click Next.

You will be prompted for migration. Click Yes to begin the migration.

You can then monitor the migration progress.

Then you can review the migration status. Click Close.

Now the status is, Jobs have been migrated and the appliance is disabled.

The same is seen in the VDP page in web client as well.

On the Avamar end, you can see the jobs have now been migrated successfully.

So, in the end, the VDP would be used only for performing restore operations with the existing data. And once new valid backups are performed by Avamar, you can discard VDP completely.

Hope this series was helpful. Let me know your thoughts in the comments.

VDP - Avmar Migration: Part-3: Avamar Client And Configuring To vCenter Server

VDP - Avmar Migration: Part-2: Configuring Avamar Virtual Edition 7.2

Previously we configured AVE services. Next we need to download the Avamar Client, install it and then configure a vCenter and deploy a proxy. Let's look at this step by step.

Let's start off by downloading the Avamar Client. To get this, go to the below URL:

You should see the below screen:

Click Downloads and you should see the below screen. First download the jre.exe and have it installed on the machine where you plan to run the Avamar Client. Then download AvamarConsoleMultipleWindows.exe

Run the installer, it's a pretty straightforward wizard to follow.

Once the client installation is done, open the Avamar client and you will be presented with the below login screen. The Server would be FQDN or IP of your AVE. The Username would be MCUser and the Password for the same with / Domain.

Once the login is successful, we will see the below Avamar management UI.

Notice there are no backup jobs, hence a red X for Backup Jobs tab. Now, we will have to configure vCenter to the AVE. Click Administration from the management UI and you will see a new window:

The vCenter client, should always be added to the / domain. So select the ave and then click Actions > Account Management and then click New Client.

You will see the following section. Select the Client Type as VMware vCenter. Provide the vCenter FQDN or IP for Client name. For username provide an admin user defined on the vCenter level. Disable vCenter Certificate parameter before performing this. You can find this step in this link here.
Click OK and the vCenter should be registered successfully to the AVE machine.

You will notice the vcenter-prod (my vCenter) under the ave domain added as a client.

Next, you will have to deploy a proxy, without this your migration pre-check will fail. In the same window, click VMware and Proxy Deployment Manager.

Here you can input your required setting and create a recommendation.

Click Apply and then click New Proxy and you will be presented with the below window. Enter a name for the proxy. The domain would be your /vcenter-fqdn/VirtualMachines. Provide an IP for the proxy, datastore and a network. Please note the proxy should be resolvable before performing the deployment. Click Save.

The proxy deployment will begin and you can monitor the progress here as well as the vCenter Recent Tasks pane.

This should be it. Until now, you have deployed AVE ovf, configured it, downloaded the client and configured it to a vCenter and added a proxy.

Next we will look at the migration of data and jobs from VDP to AVE.

VDP - Avmar Migration: Part-4: Migrating Data And Jobs From VDP To Avamar

Monday, 17 April 2017

VDP - Avmar Migration: Part-2: Configuring Avamar Virtual Edition 7.2

VDP - Avmar Migration: Part-1: Deploying Avamar Virtual Edition 7.2

In part 1 we saw how to deploy and setup the basic Avamar 7.2 machine. In this article, we will have the appliance configured for usage.

Open a browser and type in:

This should bring up the below screen and you can login with root as the user and changeme as the password.

Post the login you will see the below page. Here on the top left corner you will see a lock icon. Click this and enter the password as Supp0rtHarV1.  Once unlocked, click Install to begin the package installation.

The initialization will take a few minutes to complete and once done you will be presented with the below window. Here, you will need to fill all the fields with the red exclamation marks.

Under Server Settings, the Avamar Server Address should be the hostname and select an appropriate time zone.

The number of storage node I will be using is 0, just like how my VDP is setup. A single node server. This is the node you see when you run status.dpn

Fill in the remaining fields and click Continue on the bottom right. This should begin the package installation like seen below:

Once the configuration is complete you will see the below task. At this point, your AVE setup is complete.

Now you can SSH into the AVE machine with admin credentials and run the below command to verify system and service status.

admin@ave:~/>: dpnctl status
Identity added: /home/admin/.ssh/dpnid (/home/admin/.ssh/dpnid)
dpnctl: INFO: gsan status: up
dpnctl: INFO: MCS status: up.
dpnctl: INFO: emt status: up.
dpnctl: INFO: Backup scheduler status: up.
dpnctl: INFO: Maintenance windows scheduler status: suspended.
dpnctl: INFO: Unattended startup status: enabled.
dpnctl: INFO: avinstaller status: up.

admin@ave:~/>: status.dpn
Tue Apr 18 00:49:01 IST 2017  [AVE] Mon Apr 17 19:19:01 2017 UTC (Initialized Mon Apr 17 17:51:04 2017 UTC)
Node   IP Address     Version   State   Runlevel  Srvr+Root+User Dis Suspend Load UsedMB Errlen  %Full   Percent Full and Stripe Status by Disk
0.0   7.2.1-32  ONLINE fullaccess mhpu+0hpu+0hpu   1 false   0.49 5209    33104   0.0%   0%(onl:8  )  0%(onl:8  )  0%(onl:8  )
Srvr+Root+User Modes = migrate + hfswriteable + persistwriteable + useraccntwriteable

System ID: 1492451464@00:50:56:9A:52:F6

All reported states=(ONLINE), runlevels=(fullaccess), modes=(mhpu+0hpu+0hpu)
System-Status: ok
Access-Status: full

Last checkpoint: cp.20170417180550 finished Mon Apr 17 23:36:09 2017 after 00m 19s (OK)
No GC yet
No hfscheck yet

Maintenance windows scheduler capacity profile is active.
  WARNING: Scheduler is WAITING TO START until Wed Apr 19 20:30:00 2017 IST.
  Next backup window start time: Thu Apr 20 20:00:00 2017 IST
  Next maintenance window start time: Thu Apr 20 08:00:00 2017 IST

That should be it with the configuration part. Next, we will look into how to Configure AVE to vCenter Server. 

VDP - Avmar Migration: Part-3: Avamar Client And Configuring To vCenter Server

VDP - Avmar Migration: Part-1: Deploying Avamar Virtual Edition 7.2

Since VMware announced, the end of vSphere Data Protection, there is a choice to migrate existing deployment to EMC Avamar. More about EOA can be found in this link here.

In this article, we will be looking at deploying Avamar Virtual Edition 7.2. You can go ahead and download the required version of AVE from EMC download portal. The version I will be using is Avamar 7.2.1.

Login to Web Client or vSphere Client, Select the ESXi host where you want to deploy your AVE and select File > Deploy OVF. Browse the location for the AVE download and add the file. Click Next.

Review the details of the OVF template and click Next.

Accept the EULA and click Next.

Provide a name for this AVE virtual machine. Click Next.

If available and required select a resource pool in which you want to place this VM. Click Next.

Select the datastore where you want to deploy this. Remember the vmdk bundled with AVE is just the OVF, the data drives are configured later just like a VDP appliance. Click Next.

Select a disk provisioning type. Thick provision is recommended. Click Next.

Select a network where this AVE should be connected to. Click Next.

Review the changes and click Finish. Do not check Power On after deployment, because there are couple of steps to be done once the OVF deployment is completed.

Just like VDP, the AVE comes with 4 supported available backup storage. You can refer the below table to size your AVE accordingly.

Once you choose the deployment type, you will need to refer the below table to plan the drive sizes. Just like in VDP a 512GB deployment will have 3 drives of 256 GB each. The additional space is for the checkpoint maintenance overhead.

So the rule goes like:
Total size = GSAN capacity + 1/2 of GSAN capacity.

GSAN capacity would be the actual space for storing backup data.

So go ahead and add three disks manually (Depending on your AVE configuration) to this VM. Only Thick Provisioning is supported for AVE. I will be using Thin because of space constraints.

Once the drives are added, power on the AVE virtual machine.

The default login is root and changeme

First, we will have to configure network settings for the AVE machine. Post the login to AVE from VM console, run yast2 to begin the networking configuration. You will see a similar interface:

Select Network Devices and then Network Services to begin the network configuration wizard and you should see something similar:

You will need to Set IP Configuration in Overview, Hostname / DNS settings and Gateway under Routing.

Once the appliance is configured with network, restart the guest and then verify the network by ping and nslookup. If this works good proceed to Part 2 in the below link.

VDP - Avmar Migration: Part-2: Configuring Avamar Virtual Edition 7.2