When something is going wrong in a NetWorker environment, the first thing you need to do is be able to run up some basic tests. If the issue has anything to do with NetWorker clients, you’ll want to be able to initiate a series of network, probe and index based tests. If you’ve got nothing scripted, ‘check-clients’ from IDATA Tools may very well be what you’re looking for.

As a command line tool, ‘check-clients’ can power through a suite of different tests and data gathering activities against your clients, all with very minimum effort on your part. Let’s look at the tests that are currently available:

[root@nox bin]# check-clients -l
Test Name           Test Description
------------------- ------------------------------------------------------
client_ids          Returns client ID for each configured client
empty               Report clients with empty indices
index               Perform nsrck -L3 on each client
index_rebuild       Perform nsrck -L6 on each client
info                Retrieve client information
list_active         List all configured clients in active groups
list_all            List all clients currently configured
performance         Check backup performance via bigasm
ping                Ping each client
probe               Savgroup probe for each client
resolution          Test/confirm name resolution
rpcinfo             Test rpcinfo/portmapper access
used_space          Calculates used space for backups

Now technically, not all of the above are actually tests as such – for instance, the used_space option was one recently requested by a customer to report on all backups currently held by a backup server for a client. Running it on one of my lab machines, the output looks like the following:

[root@nox bin]# check-clients -g all_active -t used_space
============================================================
Running test: used_space (Calculates used space for backups)
============================================================
        Client                         Used Space (GB)
        ----------------------------   --------------------
        archon                                    362.60783
        faero                                       0.00000
        luyten                                      0.00000
        nox                                       544.40887
        ----------------------------   --------------------
                 Total for 4 clients              907.01669
        ----------------------------   --------------------

To me, that’s a combo test/information gathering option; specifically the customer was after this particular test so that they could spot any newly added clients that hadn’t been backing up (i.e., by having a “Used Space” of 0 GB).

Equally, there’s use in periodically running the “client_ids” test – running and keeping the output of this test will give you help in any sticky situation where you suddenly need access to a previous clients’ host ID:

[root@nox bin]# check-clients -a -t client_ids
=======================================================================
Running test: client_ids (Returns client ID for each configured client)
=======================================================================
        aralathan = 65100d33-00000004-464fcacc-464fcacb-00050000-c0a86404
        archon = 3f33ca7b-00000004-43a4837c-43a484d7-00030000-c0a80006
        asgard = 00b151ed-00000004-43a4837b-43a4837a-00010000-c0a80006
        djwmp = 5560bbf6-00000004-4910cd4b-4910cd4a-01961a00-3d2a4f4b
        faero = 76c06b0a-00000004-453e8e44-453e8e43-00310000-c0a86406
        loki = d3f277da-00000004-4857452f-4857452e-00020000-c0a86404
        luyten = 93166424-00000004-4a2f8cde-4a2f8cdd-01041a00-3d2a4f4b
        nimrod = d6454919-00000004-496aaadc-496aaadb-006f1a00-3d2a4f4b
        nox = 85acae6f-00000004-464fbdd1-464fbdd0-00010000-c0a86404
        valhalla = 61d3ca1e-00000004-495525db-4955299a-00051500-98e71c17

Moving on into actual test territory, multiple tests can be teamed up to do a chunk of information gathering in one command. For instance, combining a ping test and a name resolution test against all active clients is as simple as:

[root@nox bin]# check-clients -g all_active -t ping,resolution
=====================================
Running test: ping (Ping each client)
=====================================
	archon  (0 responses, expected 4)
	faero  (0 responses, expected 4)
	luyten  (4 responses)
	nox.pmdg.lab  (4 responses)

=======================================================
Running test: resolution (Test/confirm name resolution)
=======================================================

	archon
		Name: archon (archon.pmdg.lab) (192.168.100.1) 
		Name: archon.pmdg.lab (archon.pmdg.lab) (192.168.100.1) 
		Addr: 192.168.100.1 (archon.pmdg.lab) 

	faero
		Name: faero (faero.pmdg.lab) (192.168.100.10) 
		Name: faero.pmdg.lab (faero.pmdg.lab) (192.168.100.10) 
		Addr: 192.168.100.10 (faero.pmdg.lab) 

	luyten
		Name: luyten (luyten.pmdg.lab) (192.168.100.18) 
		Name: luyten.pmdg.lab (luyten.pmdg.lab) (192.168.100.18) 
		Addr: 192.168.100.18 (luyten.pmdg.lab) 

	nox.pmdg.lab
		Name: nox.pmdg.lab (nox.pmdg.lab) (192.168.100.4) 
		Name: nox (nox.pmdg.lab) (192.168.100.4) 
		Addr: 192.168.100.4 (balrog.pmdg.lab (unknown))

None of this is re-inventing the wheel of course, but being able to just run a single command that cycles through and tests every active client (or even all clients) is particularly useful.

Even performance testing is catered for with check-clients; reaching out to the clients, the utility can run bigasm tests automatically – a great way for easily testing where performance hits are happening on the network. For example, a quick/basic demo of this option is below:

[root@nox bin]# check-clients -c luyten,nox.anywebdb.com -b Staging -S 50 -t performance
===============================================================
Running test: performance (Check backup performance via bigasm)
===============================================================
        luyten (Solaris/UNIX style test)
                Backup 50 MB to Staging
                50 MB took 12 seconds (4.17 MB/s)
        nox.pmdg.lab (Linux/UNIX style test)
                Backup 50 MB to Staging
                50 MB took 3 seconds (16.67 MB/s)

If you are looking around for a test kit option for NetWorker – and want access to a heap of other goodies at the same time – then ‘check-clients’ out of the IDATA Tools suite may very well be what you need.

 

Once upon a time, if you said to someone “do you have a test environment?” there was at least a 70 to 80% chance that the answer would be one of the following:

  • Only some very old systems that we decommissioned from production years ago
  • No, management say it’s too expensive

I’d like to suggest that these days, with virtualisation so easy, there are few reasons why the average site can’t have a reasonably well configured backup and recovery test environment. This would allow the following sorts of tests could be readily conducted:

  • Disaster recovery of hosts and databases
  • Disaster recovery of the backup server
  • Testing new versions of operating systems, databases and applications with the backup software
  • Testing new versions of the backup software

Focusing on the Intel/x86/x86_64 world, we see where this is immediately achievable. Remember, for the average set of tests that you run, speed is not necessarily going to be the issue. Let’s focus on non-speed functionality testing, and think of what would be required to have a test environment that would suit many businesses, regardless of size:

  1. Virtualisation server – obviously VMware ESXi springs to mind here, if cost is a driving factor.
  2. Cheap storage – if performance is not an issue for testing (i.e., you’re after functionality not speed testing), there’s no reason why you can’t use cheap storage. A few 2TB SATA drives in a RAID-5 configuration will give you oodles of space if you need any level of redundancy, or just in a RAID-0 stripe will give you capacity and performance. Optionally present storage via iSCSI if its available.
  3. Tiny footprint – previously test environments were disqualified in a lot of organisations, particularly those at locations where space was at a premium. Allocating room for say, 15 machines to simulate part of the production network took up tangible space – particularly when it was common for test environments to not be built using rackable equipment.

In the 2000′s, much excitement was heralded over the notion of supercomputers at your desk – for example, remember when Orion released a 96-CPU capable system? The notion of that much CPU horsepower under your desk for single tasks may be appealing to some, but let’s look at more practical applications flowing from multi-core/multi-CPU systems – a mini datacentre under your desk. Or in that spare cubicle. Or just in a 3U rack enclosure somewhere within your datacentre itself.

Gone are the days when backup and recovery test environments are cost prohibitive. You’re from a small organisation? Maybe 10-20 production servers at most? Well that simply means your requirements will be smaller and you can probably get away with just VMware Workstation, VMware Fusion, Parallels or VirtualBox running on a suitably powerful desktop machine.

For companies already running virtualised environments, it’s more than likely the case that you can even use a production virtualisation server due for replacement as a host to the test environment, so long as it can still virtualise a subset of the production systems you’d need to test with. During budgetary planning this can make the process even more painless.

This sort of test environment obviously doesn’t suit every single organisation or every single test requirement – however, no single solution ever does. If it does suit your organisation though, it can remove a lot of the traditional objections to dedicated test environments.

© 2012 The NetWorker Blog Suffusion theme by Sayontan Sinha