Monday, 17 July 2017

Bash Script To Determine Backup Protocol

In vSphere Data Protection, you have couple of backup protocols. SAN mode, HotAdd, NBD and NBD over SSL. HotAdd is always the recommended protocol, as data handling and transfer is much faster than the rest. If your backups are running slow, then the first thing we will check is the backup protocol mode. Then we will move further to VDP load and finally the VMFS / Array performance.

If you have few VMs, you can easily find out the protocol type from the logs. However, if you have a ton of VMs and would like to determine the protocol, then you can use this script that I have written.
https://github.com/happycow92/shellscripts/blob/master/backup-protocol-type.sh

#!/bin/bash
clear
IFS=$(echo -en "\n\b")
echo "This script should be executed on a proxy machine"
echo "Checking current Machine......"
directory="/usr/local/vdr"
if [ ! -d "$directory" ]
then
printf "Current machine is Proxy machine"
else
printf "Current machine is VDP Server"
fi
echo && echo
sleep 2s
echo -e "--------------------------------------------------------"
echo -e "| Client Name | Backup Type | Proxy Used |"
echo -e "--------------------------------------------------------"
cd /usr/local/avamarclient/var
backupLogList=$(ls -lh | grep -i "vmimagew.log\|vmimagel.log" | awk '{for (i=1; i<=8; i++) $i=""; print $0}' | sed 's/^ *//')
for i in $backupLogList
do
clientName=$(cat $i | grep -i "<11982>" | awk '{print $NF}' | cut -d '/' -f 1)
protocolType=$(cat $i | grep -i "<9675>" | awk '{print $7}' | head -n 1)
proxyName=$(cat $i | grep -i "<11979>" | cut -d ',' -f 2)
if [ "$protocolType" == "hotadd" ]
then
protocol="hotadd"
elif [ "$protocolType" == "nbdssl" ]
then
protocol="nbdssl"
elif [ "$protocolType" == "nbd" ]
then
protocol="nbd"
else
protocol="SAN Mode"
fi
printf "| %-20s| %14s| %12s|\n" "$clientName" "$protocolType" "$proxyName"
done
echo && echo
Few things:
> The script must be always executed on a proxy machine. If your VDP is using internal proxy, then run it on the VDP machine itself.
> If you are using one or more External Proxy, then you need to run this on each of the proxy machines.
> Note, this will work on 6.x VDP and above.

I have added an IFS (Internal Field Separator) to handle spaces in backup job names. The rough version of script had issues handling spaces in job names.

It's a very lightweight script, takes seconds to execute and does not make any changes to your system.

Hope this helps.

Friday, 7 July 2017

Bash Script For Backup Details.

Earlier, I had written a script to count number of backups for each client in VDP. This was a very basic script and you can access it from the below link:
http://www.virtuallypeculiar.com/2017/06/bash-script-to-list-number-of-backups.html

I have added couple of more features to this script to make it more readable and more insightful.
The script along with counting number of backups, it tells about the size of the VM, the Type of OS, Is it a partial backup and also tells if the backup is on local VDP storage or a data domain. Along with this, the previous script did not account the option of Agent level backups. This script will take care of the agent level backup count as well.

Currently, I am setting up a replication, so the /REPLICATE domain can be included to count number of replicated restore points.

Edit: Change-log:
Added Feature of Listing Replicated backup (1-1 Replication only. Many to 1 or 1 to Many is not tested. You are welcome to test and report me with the working of it)
Added Feature of differentiating between backup and replication.
Added Feature of displaying last backup date for client.

The Complete Script can be accessed from my repository below:
https://github.com/happycow92/shellscripts/blob/master/list-backup.sh

The output would be similar to the below:


I will update the change log here.
You can run this script in a production environment. It will not make any changes to the system. Make sure you provide execute permissions before you run it. Of course!

Hope this helps.

Wednesday, 28 June 2017

Shell Script To Create VMs From Command Line

While I agree PowerCLI API's are the right way to deploy multiple VMs on an ESXi, I had to fallback to bash script for a project that I have been working on lately. The script is pretty simple. It is divided into 6 functions.

Function 1 is for a VMX file template which has variable input obtained from the remaining functions
Function 2 is for creating a VMDK of required size and provision type
Function 3 is for MAC address generation
Function 4 is VM Uid and VC Uid generation
Function 5 is VM Registration
Function 6 is for VM Power On

Let's have a look at this:

While ESXi does not run "bash" I had to go with the #!/bin/sh shebang to define the interpreter.
The VMX file function has a pre created template with certain options that has a variable input which can be modified by the user later while executing the script:

Create_VM ()
{
read -p "Enter the VM name: " VM_name
read -p "Enter the path of the datastore. /vmfs/volumes/<storage-name>/: " datastore_name
cd /vmfs/volumes/$datastore_name
mkdir $VM_name && cd $VM_name && touch $VM_name.vmx
read -p "Enter the Hardware version for the VM: " HW_version
read -p "Enter the Memory required for the VM: " Memory
read -p "Enter the network type, e1000 / VMXNET3: " Net_type
read -p "Enter the VM Port group name: " Port_group
# VMX File Entries
cat << EOF > $VM_name.vmx
.encoding = "UTF-8"
config.version = "8"
virtualHW.version = "$HW_version"
nvram = "$VM_name.nvram"
pciBridge0.present = "TRUE"
svga.present = "TRUE"
pciBridge4.present = "TRUE"
pciBridge4.virtualDev = "pcieRootPort"
pciBridge4.functions = "8"
pciBridge5.present = "TRUE"
pciBridge5.virtualDev = "pcieRootPort"
pciBridge5.functions = "8"
pciBridge6.present = "TRUE"
pciBridge6.virtualDev = "pcieRootPort"
pciBridge6.functions = "8"
pciBridge7.present = "TRUE"
pciBridge7.virtualDev = "pcieRootPort"
pciBridge7.functions = "8"
vmci0.present = "TRUE"
hpet0.present = "TRUE"
memSize = "$Memory"
scsi0.virtualDev = "lsisas1068"
scsi0.present = "TRUE"
ide1:0.startConnected = "FALSE"
ide1:0.deviceType = "cdrom-raw"
ide1:0.clientDevice = "TRUE"
ide1:0.fileName = "emptyBackingString"
ide1:0.present = "TRUE"
floppy0.startConnected = "FALSE"
floppy0.clientDevice = "TRUE"
floppy0.fileName = "vmware-null-remote-floppy"
ethernet0.virtualDev = "$Net_type"
ethernet0.networkName = "$Port_group"
ethernet0.checkMACAddress = "false"
ethernet0.addressType = "static"
ethernet0.Address = "$final_mac"
ethernet0.present = "TRUE"
scsi0:0.deviceType = "scsi-hardDisk"
scsi0:0.fileName = "$VM_name.vmdk"
scsi0:0.present = "TRUE"
displayName = "$VM_name"
guestOS = "windows8srv-64"
disk.EnableUUID = "TRUE"
toolScripts.afterPowerOn = "TRUE"
toolScripts.afterResume = "TRUE"
toolScripts.beforeSuspend = "TRUE"
toolScripts.beforePowerOff = "TRUE"
uuid.bios = "$uuid"
vc.uuid = "$vcid"
ctkEnabled = "TRUE"
scsi0:0.ctkEnabled = "TRUE"
EOF
}

The create VMDK is simple which uses the vmkfstools -C to get the job done.

Create_VMDK ()
{
read -p "Enter disk format. thin / zeroedthick / eagerzeroedthick: " format
read -p "Enter size: " size
vmkfstools -c "$size"G -d $format $VM_name.vmdk
}

The MAC address generation keeps a static MAC by modifying the VMX and the constant VMware defined prefix with a random generated number for the last octet.

MAC_address ()
{
mac=$(awk -v min=1000 -v max=9000 'BEGIN{srand(); print int(min+rand()*(max-min+1))}' | sed -e 's/.\{2\}/&:/g;s/.$//')
final_mac=00:50:56:00:$mac
}

The similar algorithm is applied for VC UUid generation where the post digits are constant only the first octet is changed.

UUID_generate ()
{
uuid_postfix="1a c2 4e fe 1a 8c d2-db 90 02 81 ce d8 31 15"
vcid_postfix="1a c9 91 4b 4a b9 93-79 23 12 1f b2 c5 37 f8"
uuid_prefix=$(awk -v min=10 -v max=99 'BEGIN{srand(); print int(min+rand()*(max-min+1))}')
vcid_prefix=$(awk -v min=10 -v max=99 'BEGIN{srand(); print int(min+rand()*(max-min+1))}')
uuid="$uuid_prefix $uuid_postfix"
vcid="$vcid_prefix $vcid_postfix"
}

The complete source code{} can be accessed here:
https://github.com/happycow92/Lab-Deploy/blob/master/additional-vm-deploy.sh

A while loop is defined if a user wants to deploy multiple VMs.

Well, that's pretty much it.