Automated Testing of Virtual Machine Recoveries using NetWorker

While Avamar has options for automating the test of virtual machine recoveries, it’s not functionality that’s present in NetWorker.

Recently I was contacted by Ugo Bellavance: would I be interested in checking out a script he’d written to automate testing of virtual machine recoveries. Interested? You bet!

At the end of the post I’ve got a link to Ugo’s blog post and Github for the script, but I’ll set your appetite by going through an example use of it in my lab.

As it stands, here’s how it works: you provide the script some details about your vCenter/ESX environment and the virtual machine you want to do a recovery test against. In the spirit of good randomised testing, the script picks a random backup or clone of the virtual machine and runs the recovery for you.

Here’s the details you need to provide the script:

[Sun May 03 15:49:20]
[• ~/nwautomatedrecover-master •]
root@orilla

$ ./nwrestvmrecover -h
nwrestvmrecover [OPTION]… [-x] vmname - Script to trigger a restore via the Networker REST API
where:
-h shows this help text
-a Authentication digest
-b Networker (backup) server name
-c Destination cluster resource
-d Destination datacenter
-e Destination (ESXi) host
-f VM Folder
-j Job name
-n VM name (once recovered, destination)
-p Power on after recover?
-r Reconnect NIC after recover?
-s Destination datastore
-v vCenter Server
-x Must be present to confirm that the recover must be executed

vmname : Name of the Virtual Machine to recover (source)
All the parameters are required by this script (but may not be by the API)

Note: Always test in a lab first

I had to make one minor modification to get the correct results in my lab — I changed the script’s invoked JSON to instead reference the compute resource instead of the cluster compute resource, given I don’t have a cluster in my lab. (For a business user of the script, that’s likely a modification they won’t need to make.) Here’s an example of running the script:

$ ./nwrestvmrecover -a base64EncodedCredentials -b orilla -c domain-s26 -d datacenter-21 -e host-28 -f group-v22 -j Vulcan_RecMay3 -p false -r false -s datastore-32 -v oaktier.turbamentis.int -n vulcanTestMay3 -x vulcan

==================================================
Here are the values of the parameters you entered:

Server = orilla
VM to recover = vulcan
Auth = xxxx
vCenter Server = oaktier.turbamentis.int
Recover Datastore = datastore-32
Cluster Compute Res. = domain-s26
Datacenter = datacenter-21
Host = host-28
VMFolder = group-v22
Destination VMname = vulcanTestMay3
JobName = Vulcan_RecMay3
PowerOn = false
ReconnectNIC = false

Getting the uuid of the VM to recover…

VM uuid = 500f21cd-5865-dc0d-7fe5-9b93fad1a059

Getting the number of available backups…

Number of available backups: 103

Picking a random backup to recover from…

Random Backup = 51

Getting the backup ID of the randomly selected backup…

Backup ID = 292e9504-00000006-4a7a207b-5e7a207b-02b75000-9f778f56

Choosing randomly if we restore from a backup or a clone…

Value of the cloneornot variable: 1. If the value is 0, it will restore from a backup. If the value is 1, it will restore from a clone

Getting the backup instance ID (That's the ID of the clone or backup)…

Recover URL = https://orilla:9090/nwrestapi/v3/global/vmware/vcenters/oaktier.turbamentis.int/protectedvms/500f21cd-5865-dc0d-7fe5-9b93fad1a059/backups/292e9504-00000006-4a7a207b-5e7a207b-02b75000-9f778f56/instances/1585063094/op/recover

Instance is clone? = true

Getting disk list from the backup…

Here's the contents of the JSON that will be sent to the REST API:

==================================================================

{
"recoverMode": "New",
"computeResourceMoref": "domain-s26",
"datacenterMoref": "datacenter-21" ,
"datastoreMoref": "datastore-32",
"vCenterHostname": "oaktier.turbamentis.int",
"hostMoref": "host-28",
"vmFolderMoref": "group-v22",
"vmName": "vulcanTestMay3",
"jobName": "Vulcan_RecMay3",
"powerOn": "false",
"disks": [
{
"name": "Hard disk 1",
"key": "2000",
"datastoreMoref": "datastore-32"
}
],
"reconnectNic": "false"
}

================================================================

Executing the REST call, here is the output:

About to connect() to orilla port 9090 (#0)
Trying 192.168.100.4… connected
Connected to orilla (192.168.100.4) port 9090 (#0)

I’ve trimmed the remainder of the output. Continuing the REST API theme, here’s my webwatch CGI output capturing the recovery in progress:

For the script, you do have to use the VMware internal identifiers for resources — my datastore I wanted to recover to (‘datastore4-ssd’) is actually datastore-32 as far as VMware is concerned. (Handily, Ugo has provided details on how to do this mapping — and you can always also do a one-off recovery using NetWorker, with debug mode level 9, to see the details captured: remember since this is automated recovery testing, you’ll likely be recovering to a specific location all the time, regardless of what virtual machine you choose.)

Ugo has a blog post introducing his utility, and links to his Github project for it here — please check it out!