Jan 262018
 

When NetWorker and Data Domain are working together, some operations can be done as a virtual synthetic full. It sounds like a tautology – virtual synthetic. In this basics post, I want to explain the difference between a synthetic full and a virtual synthetic full, so you can understand why this is actually a very important function in a modernised data protection environment.

The difference between the two operations is actually quite simple, and best explained through comparative diagrams. Let’s look at the process of creating a synthetic full, from the perspective of working with AFTDs (still less challenging than synthetic fulls from tape), and working with Data Domain Boost devices.

Synthethic Full vs Virtual Synthetic Full

On the left, we have the process of creating a synthetic full when backups are stored on a regular AFTD device. I’ve simplified the operation, since it does happen in memory rather than requiring staging, etc. Effectively, the NetWorker server (or storage ndoe) will read the various backups that need to be reconstituted into a new, synthetic full, up into memory, and as chunks of the new backup are constructed, they’re written back down onto the AFTD device as a new saveset.

When a Data Domain is involved though, the server gets a little lazier – instead, it just simply has the Data Domain virtually construct a synthetic full – remember, at the back end on the Data Domain, it’s all deduplicated segments of data along with metadata maps that define what a complete ‘file’ is that was sent to the system. (In the case of NetWorker, by ‘file’ I’m referring to a saveset.) So the Data Domain assembles details of a new full without any data being sent over the network.

The difference is simple, but profound. In a traditional synthetic full, the NetWorker server (or storage node) is doing all the grunt work. It’s reading all the data up into itself, combining it appropriately and writing it back down. If you’ve got a 1TB full backup and 6 incremental backups, it’s having do read all that data – 1TB or more, up from disk storage, process it, and write another ~1TB backup back down to disk. With a virtual synthetic full, the Data Domain is doing all the heavy lifting. It’s being told what it needs to do, but it’s doing the reading and processing, and doing it more efficiently than a traditional data read.

So, there’s actually a big difference between synthetic full and virtual synthetic full, and virtual synthetic full is anything but a tautology.

Basics – Prior Recovery Details

 Basics, NetWorker, Recovery  Comments Off on Basics – Prior Recovery Details
Dec 192017
 

If you need to find out details about what has recently been recovered with a NetWorker server, there’s a few different ways to achieve it.

NMC, of course, offers recovery reports. These are particularly good if you’ve got admins (e.g., security/audit people) who only access NetWorker via NMC – and good as a baseline report for auditing teams. Remember that NMC will retain records for reporting for a user defined period of time, via the Reports | Reports | Data Retention setting:

Reports Data Retention Menu

Data Retention Setting

The actual report you’ll usually want to run for recovery details is the ‘Recover Details’ report:

NMC Recover Details Report

The other way you can go about it is to use the command nsrreccomp, which retrieves details about completed recoveries from the NetWorker jobs database. Now, the jobs database is typically pruned every 72 hours in a default install (you can change the length of time on the jobs database). Getting a list of completed recoveries (successful or otherwise) is as simple as running the command nsrreccomp -L:

[root@orilla tmp]# nsrreccomp -L
name, start time, job id, completion status
recover, Tue Dec 19 10:28:25 2017(1513639705), 838396, succeeded
DNS_Serv_Rec_20171219, Tue Dec 19 10:39:19 2017(1513640359), 838404, failed
DNS_Server_Rec_20171219_2, Tue Dec 19 10:41:48 2017(1513640508), 838410, failed
DNS_Server_Rec_20171219_3, Tue Dec 19 10:43:55 2017(1513640635), 838418, failed
DNS_Server_Rec_20171219, Tue Dec 19 10:47:43 2017(1513640863), 838424, succeeded

You can see in the above list that I made a few recovery attempts that generated failures (deliberately picking a standalone ESX server that didn’t have a vProxy to service it as a recovery target, then twice forgetting to change my recovery destination) so that we could see the list includes successful and failed jobs.

You’ll note the second last field in each line of output is actually the Job ID associated with the recovery. You can actually use this with nsrreccomp to retrieve the full output of an individual job, viz.:

[root@orilla tmp]# nsrreccomp -R 838396
 Recovering 1 file from /tmp/ into /tmp/recovery
 Volumes needed (all on-line):
 Backup.01 at Backup_01
 Total estimated disk space needed for recover is 8 KB.
 Requesting 1 file(s), this may take a while...
 Recover start time: Tue 19 Dec 2017 10:28:25 AEDT
 Requesting 1 recover session(s) from server.
 129290:recover: Successfully established direct file retrieve session for save-set ID '1345653443' with adv_file volume 'Backup.01'.
 ./tostage.txt
 Received 1 file(s) from NSR server `orilla'
 Recover completion time: Tue 19 Dec 2017 10:28:25 AEDT

The show-text option can be used for any recovery performed. For a virtual machine recovery it’ll be quite verbose – a snippet is as follows:

[root@orilla tmp]# nsrreccomp -R 838424
Initiating virtual machine 'krell_20171219' recovery on vCenter caprica.turbamentis.int using vProxy langara.turbamentis.int.
152791:nsrvproxy_recover: vProxy Log Begins ===============================================
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] Logging to '/opt/emc/vproxy/runtime/logs/vrecoverd/RecoverVM-5e9719e5-61bf-4e56-9b68-b4931e2af5b2.log' on host 'langara'.
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] Release: '2.1.0-17_1', Build number: '1', Build date: '2017-06-21T21:02:28Z'
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] Changing log level from INFO to TRACE.
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 INFO: [@(#) Build number: 67] Created RecoverVM session for "krell_20171219", logging to "/opt/emc/vproxy/runtime/logs/vrecoverd/RecoverVM-5e9719e5-61bf-4e56-9b68-b4931e2af5b2.log"...
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] Starting restore of VM "krell_20171219". Logging at level "TRACE" ...
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] RecoverVmSessions supplied by client:
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] {
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] "Config": {
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] "SessionId": "5e9719e5-61bf-4e56-9b68-b4931e2af5b2",
159373:nsrvproxy_recover: vProxy Log: 2017/12/19 10:47:19 NOTICE: [@(#) Build number: 67] "LogTag": "@(#) Build number: 67",

Now, if you’d like more than 72 hours retention in your jobs database, you can set it to a longer period of time (though don’t get too excessive – you’re better off capturing jobs database details periodically and saving to an alternate location than trying to get the NetWorker server to retain a huge jobs database) via the server properties. This can be done using nsradmin or NMC – NMC shown below for reference:

NMC Server Properties

NMC Server Properties JobsDB Retention

There’s not much else to grabbing recovery results out of NetWorker – it’s certainly useful to know of both the NMC and command line options available to you. (Of course, if you want maximum flexibility on recovery reporting, you should definitely check out Data Protection Advisor (DPA), which is available automatically under any of the Data Protection Suite licensing models.)

 

Apr 272017
 

Regardless of whether you’re new to NetWorker or have been using it for a long time, if there’s any change happening within the overall computing environment of your business, there’s one thing you always need to have access to: compatibility guides.

As I’ve mentioned in a few times, including my book, there won’t be anything in your environment that touches more things other than the network itself than your enterprise backup and recovery product. With this in mind, it’s always critical to understand the potential impact of changes to our environment on the backups.

NetWorker, as well as majority of the rest of the Dell EMC Data Protection Suite, no longer has a static software compatibility guide. There’s enough variables that a static software compatibility guide is going to be tedious to search and maintain. So instead of being a document, it’s now a database, and you get to access it and generate custom compatibility information for exactly what you need.

bigStock Compatibility

If you’ve not used the interactive compatibility guide before, you’ll find it at:

http://compatibilityguide.emc.com:8080/CompGuideApp/

My recommendation? Bookmark it. Make sure it’s part of your essential NetWorker toolkit. (And for that matter: Data Domain OS, Boost Plugins, ProtectPoint, Avamar, etc.)

When you first hit the compatibility landing page, you’ll note a panel on the left-hand from where you can choose the product. In this case, when you expand NetWorker, you’ll get a list of versions since the interactive guide was introduced:

Part 01

Once you’ve picked the NetWorker version, the central panel will be updated to reflect the options you can check compatibility against:

Part 02

As you can see, there’s some ‘Instant’ reports for specific quick segments of information; beyond that, there’s the ability to create a specific drill-down report using the ‘Custom Reports’ option. For instance, say you’re thinking of deploying a new NetWorker server on Linux and you want to know what versions of Linux you can deploy on. To do that, under ‘NetWorker Component’ you’d select ‘Server OS’, then get a sub-prompt for broad OS type, then optionally drill down further. In this case:

Part 03

Here I’ve selected Server OS > Linux > All to get information about all compatible versions of Linux for running a NetWorker 9.1 server on. After you’ve made your selections, all you have to do is click ‘Generate Report’ to actually create a compatibility report. The report itself will look something like the following:

Part 04

Any area in the report that’s underlined is a hover prompt: hovering the mouse cursor over will popup the additional clarifying information referenced. Also note the “Print/Save Results” option – if say, as part of a change request, you need to submit documentary evidence, you can generate yourself a PDF that covers exactly what you need.

If you need to generate multiple and different compatibility reports to generate, you may need to click the ‘Reset’ button to blank out all options. (This will avoid a situation where you end up say, trying to find out about what versions of Exchange on Linux are supported!)

As far as the instant reports are concerned – this is about quickly generating information that you want to get straight away – you click on the option in the Instant Reports, and you don’t even need to click ‘Generate Report’. For instance, the NVP option:

Part 05

Part 06That’s really all there is to the interactive compatibility guide – it’s straight forward and it’s a really essential tool in the arsenal of a NetWorker or Dell EMC Data Protection Suite user.

Oh, there’s one more thing – there’s other compatibility guides of course: older NetWorker and Avamar software guides, NetWorker hardware guides, etc. You can get access to the legacy and traditional hardware compatibility guides via the right-hand area on the guide page:

Part 07

There you have it. If you need to check NetWorker or DPS compatibility, make the Interactive Compatibility Guide your first point of call.


Hey, don’t forget my book is available now in paperback and Kindle formats!

Basics – Taking a turn about the filesystem

 Basics, NetWorker  Comments Off on Basics – Taking a turn about the filesystem
Apr 272015
 

“Miss Eliza Bennet, let me persuade you to follow my example, and take a turn about the room. — I assure you it is very refreshing after sitting so long in one attitude.”

Jane Austin: Pride and Prejudice.

The NetWorker savegrp command has a lot of different command line options, but one which falls into that useful-for-debugging category for me has always been the -n option. This allows you to invoke the save commands for a group (or a single client in the group) in walk/don’t do mode.

While filesystems have become considerably more capable at self-repair and resilient towards minor corruption, there was a time in the past where you could encounter an operating system crash as a result of attempting to access a particularly corrupt file or part of the filesystem. Backups, of course, want to walk all the filesystems (unless you direct them otherwise), and so being able to see what NetWorker might do during a backup was helpful to diagnose such issues. (Even if it meant one more crash.)

These days, if a host being backed up by NetWorker via a filesystem agent gets a lot of changes during a day, you might simply be interested in seeing just how many files are going to be backed up.

The command is pretty straight forward:

# savegrp -nv [-c client] groupName

For instance, consider the following execution:

[root@orilla ~]# savegrp -nv -c mondas Servers
90528:savegrp: mondas:All level=incr
7236:savegrp: Group will not limit job parallelism
83643:savegrp: mondas:All started
savefs -s orilla -c mondas -g Servers -p -n -l full -R -v
mondas:/ level=incr, vers=pools, p=4
mondas:/d/01 level=incr, vers=pools, p=4
mondas:/boot level=incr, vers=pools, p=4
mondas:/d/backup level=incr, vers=pools, p=4
90491:savegrp: mondas:All succeeded.
83647:savegrp: Servers mondas:All See the file /nsr/logs/sg/Servers/832077 for command output
83643:savegrp: mondas:/ started
save -s orilla -g Servers -n -LL -f - -m mondas -t 1430050510 -o MODIFIED_ASOF_TIME:timeval=1430050506;RENAMED_DIRECTORIES:index_lookup=on;BACKUPTIME:lookup_range=1429877707:1430050510; -l incr -W 78 -N / /
83643:savegrp: mondas:/d/01 started
save -s orilla -g Servers -n -LL -f - -m mondas -t 1430050508 -o MODIFIED_ASOF_TIME:timeval=1430050506;RENAMED_DIRECTORIES:index_lookup=on;BACKUPTIME:lookup_range=1429877710:1430050508; -l incr -W 78 -N /d/01 /d/01
83643:savegrp: mondas:/boot started
save -s orilla -g Servers -n -LL -f - -m mondas -t 1430050507 -o MODIFIED_ASOF_TIME:timeval=1430050506;RENAMED_DIRECTORIES:index_lookup=on;BACKUPTIME:lookup_range=1429877709:1430050507; -l incr -W 78 -N /boot /boot
83643:savegrp: mondas:/d/backup started
save -s orilla -g Servers -n -LL -f - -m mondas -t 1430050509 -o MODIFIED_ASOF_TIME:timeval=1430050506;RENAMED_DIRECTORIES:index_lookup=on;BACKUPTIME:lookup_range=1429877708:1430050509; -l incr -W 78 -N /d/backup /d/backup
77562:savegrp: job (832078) host: mondas savepoint: / had WARNING indication(s) at completion
90491:savegrp: mondas:/ succeeded.
83647:savegrp: Servers mondas:/ See the file /nsr/logs/sg/Servers/832078 for command output
90491:savegrp: mondas:/boot succeeded.
83647:savegrp: Servers mondas:/boot See the file /nsr/logs/sg/Servers/832080 for command output
90491:savegrp: mondas:/d/01 succeeded.
83647:savegrp: Servers mondas:/d/01 See the file /nsr/logs/sg/Servers/832079 for command output
90491:savegrp: mondas:/d/backup succeeded.
83647:savegrp: Servers mondas:/d/backup See the file /nsr/logs/sg/Servers/832081 for command output
83643:savegrp: mondas:index started
save -s orilla -S -g Servers -n -LL -f - -m orilla -V -t 1429878349 -l 9 -W 78 -N index:147f6a46-00000004-5457fce2-5457fce1-0016b3a0-02efe8cc /nsr/index/mondas
128137:savegrp: Group Servers waiting for 1 jobs (0 awaiting restart) to complete.
90491:savegrp: mondas:index succeeded.
83647:savegrp: Servers mondas:index See the file /nsr/logs/sg/Servers/832082 for command output
* mondas:All savefs mondas: succeeded.
* mondas:/ suppressed 2038 bytes of output.
...snip...

You’ll see there the output reaches a point where NetWorker tells you “suppressed X bytes of output”. That’s a protection mechanism for NetWorker to prevent savegroup completion notifications growing to massive sizes. However, because we’ve used the verbose option, the output is captured – it’s just directed to the appropriate log file for the group. In this case, the output (underlined above) tells me I can check out the file /nsr/logs/sg/Servers/832078 to see the details of the root filesystem backup for the client mondas.

Checking that file, I can see what files would have been backed up:

[root@orilla Servers]# more /nsr/logs/sg/Servers/832078
96311:save: Ignoring Parallel savestreams per saveset setting due to incompatibl
e -n/-E option(s)
75146:save: Saving files modified since Sun Apr 26 22:15:06 2015
/var/log/rpmpkgs
/var/log/secure
/var/log/audit/audit.log
/var/log/audit/
/var/log/lastlog
/var/log/cron
/var/log/wtmp
/var/log/maillog
/var/log/
/var/run/utmp
/var/run/
/var/lock/subsys/
/var/lock/
/var/spool/clientmqueue/qft3QI22Tg020700
/var/spool/clientmqueue/dft3QI22Tg020700
/var/spool/clientmqueue/
/var/spool/cups/tmp/
/var/spool/cups/
/var/spool/anacron/cron.daily
/var/spool/anacron/
/var/spool/
...snip...

This command only works for filesystem backups performed by the core NetWorker agent. It’s not compatible for instance, with a database module or VBA – but regardless, it is the sort of debugging/analysis tool you should be aware of. (Forewarned is forearmed, and forearmed is a lot of arms… Ahem.)

Check out savegrp -n on a client/group when you have time to familiarise yourself with how it works. It’s reasonably straightforward and is a good addition to your NetWorker utility belt.

file walk

Basics – virtual machine names in VBA backups

 Basics, NetWorker, VBA  Comments Off on Basics – virtual machine names in VBA backups
Mar 262015
 

If you’ve been backing up your virtual machines with VBA, you’ve probably hit that moment when you’ve run an mminfo query and seen output looking like the following:

mminfo_vm_backups_01

As you can see, that’s not the most effective way to see virtual machine names – vm:<id> doesn’t allow you to easily match it back to the virtual machine in question.

However, not all is lost. With VBA backups came a couple of new options. The first one is a “VBA backups” style report, using the command:

# mminfo -k

Using mminfo -k you’ll get a very tailored output focused entirely on your VBA backups, and it’ll resemble the following:

mminfo_vm_backups_02

That’s a really good way of seeing a quick listing of all your VBA-based virtual machine backups, but if you’re wanting a way of reconciling in normal mminfo output, you can also make use of a new mminfo report field, vmname. For example:

mminfo_vm_backups_03

(In the above command I could have used name and vmname in order to reconcile vm:<id> entries to virtual machine names, but elected not to for brevity.)

There you have it – a couple of quick and easy ways of quickly seeing details of your virtual machine backups via mminfo.

 

Sep 012014
 

A question I get asked from time to time is “How do I do X in NetWorker?”, and by how, I mean what’s the order of steps, rather than a general description.

Workflow for adding a new client

To me, the configuration steps in NetWorker are often quite minimal compared to the operational and organisational processes that typically should be followed to ensure an appropriately maintained system. Configuring a new client is a perfect example of this, so below is the procedure I normally recommend following:

  1. Determine if there are any databases or applications on the host that require module-based backups.
  2. Determine if there is anything on the host that should be excluded from backup.
  3. Determine any special retention requirements (vs ‘default’ retention requirements used in the business).
  4. Determine if any SLAs require integration between backup and other data protection processes (e.g., with snapshots, replication targets, etc.)
  5. Check OS and application versions against the compatibility guide if they’re not standard/already known versions.
  6. Ensure the backup system has sufficient capacity for bringing the client on-board.
  7. Determine what tests are to be applied to this client to confirm it’s successfully brought on-board.
  8. Determine whether any backup software to be installed will require an OS or application restart – for example:
    • NMM with GLR might require reboots (and if .Net needs to be installed, 2 reboots may be required).
    • Oracle and other databases may require restarting for library linking.
  9. Determine if any firewalls will need to be adjusted to allow for backup traffic.
  10. Confirm forward/reverse lookups between all appropriate hosts – for example:
    • New client and backup server
    • New client and storage node(s)
    • New client and IP backup storage (e.g., Data Domains)
  11. Confirm network connectivity between all appropriate hosts.
  12. File change requests or work plans as appropriate within the organisation, supplying appropriate installation/back-out plans, peripheral configuration activity (e.g., changing firewalls, etc.)
  13. Confirm change approval and schedule.
  14. Install filesystem client.
  15. Install database module (if required).
  16. Configure filesystem backups in NetWorker.
  17. Test filesystem backups in NetWorker and remediate.
  18. Configure database backups.
  19. Test database backups and remediate.
  20. Integrate client instances with appropriate retention policies and schedules.
  21. Confirm successful next-day operation of automated backups.
  22. Add client into any custom reporting (should fold automatically into standard reporting).
  23. Close off change as required.

Depending on your environment, those processes may change a bit – or they may even be less formal, but cutting corners in data protection can easily lead to a mishap, so if you’re looking for a procedure for adding a client, you could do a lot worse than the one above.

Jan 092012
 

Upgrading NetWorker

So a new version of NetWorker has come out, or is coming out, and it’s been decided that you’re going to upgrade, but you want a few tips for making that upgrade as painless as possible. Here’s my 5 rules for upgrading NetWorker:

  1. Read the release notes. If you’re not going to read the release notes, you are better off staying on your current version, no matter what issues you’re having. I can’t stress enough the importance of reading the release notes and having a thorough grasp of:
    • What has changed?
    • What are the known issues with the current release?
    • What were the resolved issues between the current release and the release you’re currently running?
  2. Do a bootstrap and index backup if upgrading between major or minor releases. If going between service packs on the same release, you can skip the index backup so long as your backups have been successful lately, but ensure you still do a bootstrap backup.
  3. Unload all tapes (physical or virtual) in jukeboxes before the upgrade. You’ll see why shortly.
  4. Upgrade in this order:
    • Storage node(s) on the day of the upgrade, before the NetWorker server
    • Server on the day of the upgrade, after the storage node(s)
    • Client(s) later, at suitable times
  5. After the upgrade but before the NetWorker services are restarted on the storage node(s) and server, delete the nsr/tmp directory on those hosts.

Obviously standard caveats, such as following any additional instructions in the release notes or upgrade notes should of course be followed, but sticking to the above rules as well can save a lot of hassle over time. I’ve noticed over the years that a odd, random problems following upgrades can be solved by clearing the nsr/tmp directory on the server and storage nodes. If there’s no tapes in the jukeboxes when the services first start after the upgrade, there’s less futzing for NetWorker to take care of before it’s fully up and running, too.

Basics – Specifying a default Jukebox

 Basics, NetWorker, Scripting  Comments Off on Basics – Specifying a default Jukebox
Aug 052010
 

If you’ve got multiple jukeboxes within a NetWorker environment, but primarily work with one of them, you may find ‘nsrjb’ to be a bit of a pain any time you forget to specify the jukebox name. If you’re not familiar with this, here’s how nsrjb reacts in this situation:

[root@tara ~]# nsrjb
1:	VTL1 	[enabled]
2:	VTL2 	[enabled]
No jukebox selected.
Please select a jukebox to use:? [1] _

(Slight aside: never assume the numbered list is the same; NetWorker doesn’t guarantee the order being the same between executions – in fact, I actually only put in an RFE about this a couple of days ago, as I’m hoping it could at least be alphabetically ordered at all times…)

If you want to avoid the jukebox-prompt from nsrjb, one of the easiest ways is to specify the jukebox name as part of the command – e.g.,

[root@tara ~]# nsrjb -j VTL1

That’s fine of course, but if the vast majority of the time you perform operations on a single jukebox, you can specify a default jukebox as an environment variable (NSR_JUKEBOX) and streamline your processes. For example, on Linux, using the bash, this might look as follows:

[root@tara ~]# export NSR_JUKEBOX=VTL1
[root@tara ~]# nsrjb
Jukebox VTL1: (Ready to accept commands)
slot  volume         pool           barcode   volume id        recyclable
1: 800840L4       ClientTesting  800840L4  3814088325       no
2: 800841L4       ClientTesting  800841L4  3797311146       no
3: 800842L4       ClientTesting  800842L4  3847642669       no
4: 800843L4       ClientTesting  800843L4  3780533937       no
5: 800844L4       ClientTesting  800844L4  3763756765       yes
6: 800845L4       ClientTesting  800845L4  3864419885       yes
<snip>

Being an environment variable, this is something you can choose to set locally – say, on a per storage-node basis, when you have multiple storage nodes. It’s relatively common for instance to have a tape library on one or more storage nodes, so for the appropriate logins (or even at a system level) on each storage node it would be possible to set the local jukebox as the default, thereby streamlining usage of the units.

As an example, here’s a lab storage node with the setting in use:

[root@fawn ~]# export NSR_JUKEBOX="rd=fawn:VTL3"
[root@fawn ~]# nsrjb -s tara
Jukebox rd=fawn:VTL3: (Ready to accept commands)
<snip>

For something that can take you less than 30 seconds to set, the environment variable NSR_JUKEBOX can certainly be a big time saver if you have multiple jukeboxes in your environment and (like me) you’re a command line junkie.

Basics – jbverify

 Basics  Comments Off on Basics – jbverify
Apr 082009
 

Now, some might say that I’m not the smartest card in the deck for what I’m about to note, but sometimes I don’t notice new commands when they appear in NetWorker, particularly if I skimmed through release notes.

I was pleasantly surprised today to find that a new command had slipped in at some point called “jbverify”. I can immediately see it however entering my stable of must-use commands, particularly in a support environment.

To quote the man page, jbverify:

[V]erifies the devices defined in the NetWorker database, making sure that each one of them is configured properly by checking them  for accessibility  and  usability.

This is the sort of diagnostic tool that support people live for, and sites suddenly experiencing strange jukebox issues should think of as a matter of course.

When run on my lab server this afternoon, I got the following badly formatted but still very useful output:

# jbverify
14866:jbverify:
 Jbverify is running on host nox, Linux 2.6.18-128.1.6.el5

14912:jbverify:
 Processing stand-alone devices...

14913:jbverify:
 Processing /d/nsr/02/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/02/_AF_readonly

14913:jbverify:
 Processing /d/nsr/idata/backup/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/idata/backup/_AF_readonly

14913:jbverify:
 Processing /d/nsr/01/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/01/_AF_readonly

14913:jbverify:
 Processing /d/nsr/idata/backup
14915:jbverify:
 Finished processing /d/nsr/idata/backup

14913:jbverify:
 Processing /d/nsr/idata/clone/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/idata/clone/_AF_readonly

14913:jbverify:
 Processing /d/nsr/01
14915:jbverify:
 Finished processing /d/nsr/01

14913:jbverify:
 Processing /d/nsr/03
14915:jbverify:
 Finished processing /d/nsr/03

14913:jbverify:
 Processing /d/nsr/03/_AF_readonly
14915:jbverify:
 Finished processing /d/nsr/03/_AF_readonly

14913:jbverify:
 Processing /d/nsr/02
14915:jbverify:
 Finished processing /d/nsr/02

14913:jbverify:
 Processing /d/nsr/idata/clone
14915:jbverify:
 Finished processing /d/nsr/idata/clone

14917:jbverify:
 Finished processing stand-alone devices.

14918:jbverify:
 Processing jukebox devices...

14920:jbverify:
 Processing jukebox LTO1_LIB:

14733:jbverify:
 Testing drive 1 (/dev/nst0) of JB LTO1_LIB

14927:jbverify:

 Jukebox LTO1_LIB on nox successfully processed.

14929:jbverify:

 Finished processing jukebox devices.

**********************************************************************

         Summary report of jbverify
         ======= ====== == ========

Hostname   Device Handle    Blocksize  Jukebox  Drv No. Status
--------   -------------    ---------  -------  ------- ------
nox        /d/nsr/02/_AF_readonly 131072 N/A    N/A     Pass
nox        /d/nsr/idata/backup/_AF_readonly 131072 N/A N/A Pass
nox        /d/nsr/01/_AF_readonly 131072 N/A    N/A     Pass
nox        /d/nsr/idata/backup 131072  N/A      N/A     Pass
nox        /d/nsr/idata/clone/_AF_readonly 131072 N/A N/A Pass
nox        /d/nsr/01        131072     N/A      N/A     Pass
nox        /d/nsr/03        131072     N/A      N/A     Pass
nox        /d/nsr/03/_AF_readonly 131072 N/A    N/A     Pass
nox        /d/nsr/02        131072     N/A      N/A     Pass
nox        /d/nsr/idata/clone 131072   N/A      N/A     Pass
nox        /dev/nst0        65536      LTO1_LIB 1       Pass

**********************************************************************

If you’ve come from NetBackup, the nature of this program is somewhat reminsicent of the robtest utility. I don’t claim EMC are special for having introduced this tool, but I do applaud that it’s there (and lament that I didn’t notice it sooner).

(One thing to note: after running jbverify, make sure you reset your jukebox.)

Mar 222009
 

I’m not fond of software encryption (or compression, for that matter). Particularly in a 24×7 enterprise environment, clients (i.e., production servers) have better things to be doing than doing on-the-fly software encryption or compression. In these environments, hardware encryption routers should be the product of choice for achieving totally secure backups. Such devices also have advantages in terms of key management – much more flexible, scalable and appropriate for role based data access.

That being said, in smaller environments, or environments where servers are relatively idle overnight, NetWorker’s datazone encryption can be sufficient to achieve a reasonable modicum of backup protection with minimum effort – and most importantly, cost.

To get started using NetWorker datazone encryption, you first need to assign a pass phrase. This is done in the NetWorker server properties (typically accessed within NMC):

datazone-encryptionWith the pass phrase in place, you can then configure directives within NetWorker to make use of AES 256 bit encryption. However! As soon as you turn encryption on, you lose all potential for hardware based compression for your media. Why? Quite simply, compression is about finding patterns in data and reducing all the matching patterns to a single reference point; however, encryption is all about eliminating patterns, making the data appear completely random.

Thus, if you want to still get some measure of compression, you should, when using this method, employ software based compression in your directive as well.

Thus, a base directive might look like the following:

<< / >>
+compressasm: .
+aes: .

This will apply compression first to all files encountered, then once the file has been compressed, it will be encrypted. A side benefit of this is that by compressing first, you reduce the amount of data to be encrypted*.

So long as the datazone pass phrase is stored in the server, encryption will occur, and no password will be required to recover the data. Remember, this style of encryption, using a single pass-phrase, isn’t about being able to restrict whom within the datazone can recover the data, but instead it’s about keeping the data stored on-tape (which is potentially off-site, or otherwise at higher risk of theft), from being recovered.

[Edit, 2009-08-15]

It’s been pointed out to me that you can’t compress + encrypt at the client side. Indeed, I’ve now found the part in the administration guide that explicitly says this. What is extremely disappointing about this is that NetWorker actually doesn’t warn you that it’s not going to compress + encrypt! To me, that’s a security issue.

So, for the examples above, forget about enabling client side compression as well as encryption – you can have one or the other, but not both.


* In the same way that ice-cream that’s 99% fat free, but 87% sugar is a “benefit”.

%d bloggers like this: