Jobquery finally gets the duct tape removed

 Architecture, Databases, NetWorker, Scripting  No Responses »
Jan 082010
 

Some time ago, I posted that EMC had added a jobquery utility to allow probing of the NetWorker jobs database (the one created/maintained by nsrjobd). Unfortunately, at the time, jobquery had been somewhat muzzled – you could give it commands, and it would spit output back to you, but it would never give you any indication that it was waiting for you. No prompting, no nothing. It made working with jobquery somewhat of a hassle.

Thankfully though, as of NetWorker 7.6, the duct tape has been fully removed and jobquery will happily give you that meta-information:

[root@nox ~]# jobquery


NetWorker jobs query utility.


Use the "help" command for help.


jobquery> help


Legal commands are:


          print [query] (set current query)


           show [attrlist]


          types


            all


           quit


           help [command]


              . [query]


              ? [command]


Where:


        query ::= attrlist


        attrlist ::= attribute [; attribute]*


        attribute ::= name [: [value [, value]* ]


jobquery> types


     Known types: job indication, save job, probe job,


                  savegroup job, session info, index save job,


                  savefs job, bootstrap save job, utility job,


                  active job db;

Now sure, in previous versions you could type commands in and get output, but not having the prompt to tell you when jobquery was waiting, or when it was working, or when (as I originally thought) it was hanging on startup, is a fairly critical part in having a useful user interface.

Having a responsive user interface makes jobquery a little nicer to work with. For instance, let’s look at the “save job” type and run a backup. There’s a lot of fields in the “save job” resource type, so I’m going to limit it as follows:

jobquery> show command:; group name:; host:; job state:; level:
jobquery> print type: save job
                     command: \
"save -s nox.anywebdb.com -g \"Staging Servers\" -LL -f - -m nox\
 -t 1262601936 -l incr -q -W 78 -N /d/nsr/01 /d/nsr/01";
                  group name: Staging Servers;
                        host: nox;
                   job state: COMPLETED;
                       level: incr;
…

Then, once the backup starts, if I run (and abbreviate the output, remembering it shows all jobs in the database!), I can see:

                     command: \
savepnpc -s nox.anywebdb.com -g archon -LL -f - -m archon -t 1262818820 -l in\
cr -q -W 78 -N / /;
                  group name: archon;
                        host: archon;
                   job state: ACTIVE;
                       level: incr;

However, the “ACTIVE” state simply means that the job is actively queued, not that it’s actually sending data. If you want to only see jobs that are actively backing up rather than just simply active, you’d look for a job state of “SESSION ACTIVE”:

jobquery> print type: save job; job state: SESSION ACTIVE
                     command: \
savepnpc -s nox.anywebdb.com -g archon -LL -f - -m archon -t 1262818820 -l in\
cr -q -W 78 -N / /;
                  group name: archon;
                        host: archon;
                   job state: SESSION ACTIVE;
                       level: incr;

                     command: \
savepnpc -s nox.anywebdb.com -g archon -LL -f - -m archon -t 1262818814 -l in\
cr -q -W 78 -N /Volumes/Yu /Volumes/Yu;
                  group name: archon;
                        host: archon;
                   job state: SESSION ACTIVE;
                       level: incr;
What does this mean? For a start, it provides a way of checking (outside of NMC) which jobs are currently queued to run, and which jobs are actually running. As is always the case, the better you can monitor regardless of circumstance, the more likely you are to be able to understand your server state. Now that jobquery however at least tells us when it’s waiting for input, I’m looking forward to properly exploring what it can do. I previously said that I anticipated it being a useful tool. Now I know it’s going to be a useful tool, and will do some further digging/testing and do a future posting on using it further to track activities.
 Posted by at 14:25  Tagged with:

Why I’d choose NetWorker over NetBackup Every Time

 Architecture, Backup theory, NetWorker  7 Responses »
Jan 082010
 

I thought it about time that I cited the two key reasons why, if faced with a choice between NetWorker and NetBackup, I would choose NetWorker every time.

As you might expect, given my focus on backup as insurance, both of these reasons are firmly focused on recovery. In fact, so much so that I still don’t really understand why EMC doesn’t go to market with these points time and time and time again and just smack Symantec around until it’s blue in the face and begging for mercy.

Reason 1: NetBackup does not implement backup dependencies

I struggle to call NetBackup an “enterprise” backup product because of this simple fact. Honestly, backup dependencies are critically important when it comes to guaranteeing anything but last-backup recoverability.

What does this mean?

In short, as soon as a backup hits its retention period in NetBackup, it’s toast – it’s a goner.

Irrespective of whether there are any backups of the same filesystem/data set that requires the “outside retention” backup for recovery purposes.

I can’t sum this up any other way: in a backup product, I see this as recklessly irresponsible. It provides a focus on media savings that even the most miserly bean cruncher would admire. Well, until the bean cruncher’s system can’t be recovered from 6 weeks ago to fulfil audit requirements.

Reason 2: True Image Recovery is “optional”

If you’ve grown up in a NetWorker world, where the emphasis has always been, and will always continue to be on recovery, this will, like the reason above, make you soil yourself. Imagine having a full backup plus six incremental backups of a directory, and wanting to recover the filesystem from last night. Now imagine just selecting the full plus the incrementals for recovery and getting back everything generated during that time.

Even the files that had been deleted between backups. I.e., you don’t get back what the filesystem looked like at the time of the backup that you’re recovering from, but what it looked like for every backup that you’re recovering from.

NetWorker, once, in the 5.5.x stream implemented this. It was called a BUG. In NetBackup, it’s a “feature”. In order to enable a correct recovery, you have to turn on “true image recovery”, something that takes extra resources, and is typically advised  that you keep the data just for a small cycle (e.g., 7 days) rather than the complete retention time for the backups.

There’s another word for this: Joke.

On another front…

As recently as December I mentioned that I wished EMC would get their act together and implement inline cloning – one of the few things where I saw that NetBackup had a distinct competitive advantage over NetWorker.

Maybe it was the glow of the cider, but I had an epiphany in Copacabana on a hill watching (probably illegal) fireworks in Avoca and Terrigal on new years eve. Inline cloning is no longer a compelling factor in a backup product. Why? Media streaming speeds have reached a point where companies with serious amounts of data just should not be implementing direct-to-tape backup solutions any more. Inline cloning was developed at a time when you’d want to generate both sets of tapes as quickly as possible, but only companies with very small data sets will find themselves not backing up to some disk unit first (be it say, ADV_FILE, or VTL, in NetWorker), and those companies won’t be constrained on backup/clone windows to a point where they’d need inline cloning anyway.

When not backing up direct-to-tape, there are several factors that mitigate the need to do inline cloning. In organisations with a very strong need for offsiting, there’s replication at a VTL or disk backup unit layer. In organisations that just need a second copy generated “as soon as possible”, doing disk/virtual tape to physical tape cloning following the backup should be fast enough to handle the cloning at appropriate performance levels.

In other words: there’s no need for EMC to implement inline cloning. As a technology, it’s a dead-end from a tape-only time. I feel somewhat silly this didn’t occur to me sooner.

 Posted by at 12:47  Tagged with: , , , , , , , , , ,
© 2012 The NetWorker Blog Suffusion theme by Sayontan Sinha