A week before easter, I was doing some Powerplate exercises, nothing I hadn’t done hundreds of times before, when stepping off, I fell backwards, slammed into some other exercise equipment, and twisted my ankle bad enough to result in a very audible crack. After X-Rays and CT scans, we’ve confirmed my ankle is still definitely there, with all the bones in their correct configurations – this is good news, of course. However, there was definitely ligament damage, and also a couple of days in, because I don’t do things by halves apparently, an infection from the swelling.
Never fear, this blog post isn’t going to be about me somehow drawing a relationship between backups and ankles. (That would be silly, of course. Ankle health is like RAID-1: you want both to be working, and if one doesn’t work, it slows you down. Case closed. Though recent events for me might have demonstrated the case for having hot spares.)
No, I brought up the injury mainly to help explain why I spent a lot of time for almost two weeks doing ‘thought experiments’. Not lab testing, not blogging – thinking. Some might call it mindfulness, others might call it day-dreaming; at the time I called it laying on my back with my foot elevated above my heart and being unable to use a computer in that posture. (Note to self: invent an upside-down desk that I can strap a laptop into to float above my head. Second note to self: padded helmet in case laptop falls out of desk…)
One of the things that I’ve been thinking about is how we should optimally arrange NetWorker configuration in the 9.x world via policy. Yes, I know the NetWorker 9 series has been around for a while, but it’s time to start talking more about how to best arrange your NetWorker 9.x configuration as NetWorker 8 wanders off into the sunset. Following some thought experiments, I want to suggest you’ll get your best policy configurations by treating them like a service catalogue.
There’s nothing particularly out there in what I’ve just said – it’s not really a new idea. It’s the goal of NetWorker policies from a design standpoint to allow a more service-catalogue aligned backup configuration, and that’s something I’ve also talked about since I introduced NetWorker 9.
(Perhaps, if I’d injured myself so well 18 months ago, I might have made this blog post earlier.)
So, let’s think of the four ‘default’ container policies that come with a NetWorker 9 install, waiting to be populated (ignoring the server one, which is there for the NetWorker server itself):
- Platinum
- Gold
- Silver
- Bronze
That just screams service catalogue, right?
Let’s step back for a moment and think about how NetWorker 8.x and lower groups used to work. In terms of getting backups running, the group was the centre of all configuration. You added clients to groups, you assigned a start time to the group, and then a bunch of other properties as well – even more so if you wanted the group to overwrite client configuration options, etc.
That’s a simplified view of course, but the above diagram gives you an idea of the general approach for NetWorker configuration practices in v8 and below. In fact, it’s the sort of configuration practices I’d been following since v4 (though I rarely did groups separated by operating systems, unless customers demanded it).
Overall though, you’d get configurations similar to the above – not shown of course is the logical extrapolation for other backup frequencies, and browse and retention times – such as:
- Monthly Linux Prod Servers
- Monthly Solaris Prod Servers
- Monthly Windows Prod Servers
- Monthly Linux NonProd Servers
- Monthly Solaris NonProd Servers
- Monthly Windows NonProd Servers
- Yearly Linux Prod Servers
- Yearly Solaris Prod Servers
- Yearly Windows Prod Servers
- Yearly Linux NonProd Servers
- Yearly Solaris NonProd Servers
- Yearly Windows NonProd Servers
- and so on.
Say you had the following key breakdowns within your environment:
- Operating systems:
- Windows
- Linux
- Solaris
- Data differentiators:
- Production
- Development
- Test
- Databases:
- Oracle
- SQL Server
- Exchange
- Retention processes:
- Daily backups (either daily incremental or weekly full): 6 weeks
- Monthly full backups: 12 months
- Yearly full backups: 7 years
Even that sort of basic breakdown might result in at least 36 or more different groups (daily/monthly/yearly prod/dev/test linux/windows/solaris/oracle/exchange/sql).
(Yes, development and test systems might not need monthly or yearly backups – but it helps to show how quickly a relatively small requirements list can explode into a lengthy set of groups in a ‘classic’ configuration.)
That’s not just a NetWorker problem, by the way: I’ve seen configurations in NetBackup for instance where there might be 500-1000 clients, and hundreds of policies. It’s easy in a functional view to generate lots of different collections.
So how do NetWorker 9 policies differ? Simply put, they differ by allowing you to apply higher level collection criteria that map to business views of what you’re protecting. Let’s pause here and think about the logical structure for data protection within NetWorker 9 onwards. You have:
- Protection Policies, which contain:
- Workflows, that comprise of:
- A group of systems the workflow runs for and
- One or more actions performed by the workflow.
- Workflows, that comprise of:
The group has changed in terms of functional use in NetWorker 9. In v8 and below, it really was the be-all and end-all of most things related to getting systems backed up (excluding VMware policies – which were, in their own way, a harbinger of where NetWorker was headed). Now it really is just a list – it might be a list of clients, a list of savesets, a list of saveset criteria, or a list of client tags for dynamic client policies.
The workgroup, I’d suggest, is the closest parallel to the old v8 group, but even then it’s considerably more powerful, since it’s designed around higher scalability than groups, and with more functionality (e.g., being able to concurrently clone a backup to two different pools, etc.)
That leaves you the option – and I’d suggest it should be your default – of using the policies to establish more readily identifiable collections based around the importance of the data protection, and its retention. Under v9, you might instead end up with a configuration such as the following:
In this configuration, each additional policy tier beyond Platinum steps down in order of the level of data protection being performed – and ideally, the retention policies that you’d define for that protection. Instead of a purely functional configuration, we can have a logical configuration, one that maps to service ideals – for instance:
- Platinum policies and their workflows:
- Get cloned twice, because they represent mission critical data
- Would be the only workflows that get yearly backup retention as well as daily and monthly – i.e., being kept for 7+ years for compliance purposes
- Gold policies and their workflows:
- Get cloned once, because they’re just ‘normal’ workloads within the environment
- Would get monthly backup retention as well as well as daily – i.e., once-a-month fulls kept for 12 months
- Silver policies and their workflows:
- Would get cloned once, again because they’re just ‘normal’ workloads within the environment
- Would only get daily backups that are retained for say, 5-6 weeks
- Bronze policies and their workflows:
- Don’t get cloned at all*
- Just get daily backups that are retained for say, 2 weeks.
I’ve not called out the retention times in the diagram: workflows would still be expanded out to encompass the different retention times or backup frequencies; however, given the change in how we use groups, we can use the same group (e.g., “Platinum_Fileservers”) for each of the Platinum Daily, Monthly and Yearly Fileservers workflow.
Thet net result is a configuration which is much simpler to see (particularly with the visual representation option), and one which instantly makes more sense. Instead of piles of groups where the only immediate categorisation is the name, you can organise your backup protection more logically based on the criticality of the data to the business, as well as its retention requirements.
How does this affect you and your NetWorker configuration?
- If you’re getting ready to upgrade to NetWorker 9.x:
- This is the prime time to start thinking about how to categorise and group your protection policies by data criticality and retention requirements. Yes, when you upgrade, NetWorker will automatically convert all your groups into workflows, placing them in a single policy. You can, for day-1 operations, run with that, but plan how you’re going to move things around and adjust your configuration to get the most of this new approach.
- If you’ve previously upgraded to NetWorker 9.x:
- Hopefully you’ve made the change into a service-catalogue approach; if not, start planning on how to adjust your configuration to get yourself there. You don’t have to do it all at once, of course. Pick a particular type of workload, make use of the predefined Platinum/Gold/Silver/Bronze policies, and move those upgraded workflows in accordingly so you get a precursor of what an end solution will look like.
- If your first NetWorker install was 9.x:
- Hopefully you’ll have gone down the path of configuring based on retention and criticality, putting workflows into the appropriate policies as you were building out your backup environment. Resist creating too many policies – an extremely large NetWorker environment of 2000 or more clients should be able to exist within just 3 or 4 policies, with the appropriate sub-configuration of workgroups. You might add one or two policies (e.g., an “Adhoc” policy might be OK to have), but don’t** start creating policies along the lines of functional breakdowns, AKA v8 groups.
If you approach policies correctly, you’re going to find yourself with a simpler to maintain configuration, a more logical configuration, and a configuration that lets you build a very close correlation to the service catalogue that IT establishes with the business. Every part of this screams better.
* Yes, I do believe you should clone everything. But I am talking a very low service order – in this case, Bronze might be used intermittently for active test systems where you want to preserve a copy of data during testing.
** I will own up to having screenshots and views on this blog of my lab configuration where I’ve got policies such as say, VMware. I’m fully prepared to say these are lazy policies that come from a lab environment and I’d never advocate them in a production environment.