A very traditional approach to configuring automated backups in NetWorker is to make use of the schedule override feature in NetWorker groups. That is, by defining either a schedule or a level at the group level, the backup level from all clients in the group will be in lock-step. Pictorially, this configuration resembles the following:
We frequently encourage this sort of setup because it takes two items which NetWorker can run disparately – start time, and level, and effectively merges the two – something a lot of other backup products just do as the one configuration item. Perhaps even more importantly, in small to mid size businesses with modest data levels, this makes more sense anyway – it allows you to readily construct “classic” backup scenarios, such as “full on Friday, incrementals the rest of the week”. So from the perspective of level and amount of data backed up, your backup week would look similar to the following:
Now, as I said, this works well for businesses with modest data sizes. However, as the image graphically demonstrates, this creates scenarios where there is a significant disparity between the amount of data backed up on regular days and the amount of data backed up for the fulls. Remembering that it’s the full backups that frequently end up straining backup architectures, companies will often end up revisiting their architecture when the amount of data backed up on the “full day” becomes unmanageable.
For some companies, the full day is chosen for sound business reasons – finance companies for instance may have to do weekly full backups starting close of business Friday, and full monthly backups on the last Friday every month. In these scenarios, where there are important business reasons for keeping full backups on a single day of the week/month, the backup architecture must remain constantly configured to handle the massive spike that full backups create.
However, in other companies where there are no strong business reasons for running all the fulls on the same day, it’s worth remembering that there is an alternate configuration – ironically enough, it’s very much the “default” NetWorker configuration, it’s just one most sites tend not to use. This configuration sees the group control only the start time/collection of clients, and does not have a schedule/level override assigned. Instead, the schedule of each client defines what level backup will be done for that client. This sort of configuration resembles the following:
As you can imagine, this does require a slight change of administrative policies in relation to setting the correct schedule at the client level, and potentially needing additional client instances to handle the daily and monthly backups, but the advantage of this is that you can then start having groups where both incremental and non-incremental backups are done concurrently, spreading out the load of the full backups to create a significantly lower spike in resource requirements. So from the perspective of level and amount of data backed up, your backup week would instead look like the following:
This style of schedule isn’t for everyone – as I said, if you have a strong business need to restrict all full backups to a particular day, it’s very unlikely to work. I’d suggest as well that it may not be a good strategy if you happen to have a high staff turnover, as it does realistically add a little more complexity into the environment. (While your environment should be as simple as possible, that doesn’t always mean “as simple as conceivable”.)
In larger environments though with significantly higher amounts of data requiring backup, this style of configuration can be a real boon. Compare weekly fulls of say, 10TB (effectively tiny) with weekly fulls of say, 500TB, and you can instantly see the attraction of this programme. Instead of having to design a system capable of handling 500TB in 24 hours, you might instead be able to limit your design to a system that at most has to handle 100TB over a 24 hour period (factoring in incrementals + fulls on any given night). That’s not an insignificant difference.
[Edit, 2010-05-11]
What’s this got to do with large groups? It occurred to me overnight that while the title of the post was originally “Large group backups”, I diverged somewhat between the original intent of the post and the actual resulting post.
So, the other area where this can be useful is in situations where you have groups with large numbers of clients. For example, in environments with 500+ clients, where a single group may have hundreds of clients in it, switching to mixed levels in the one group has the same effect as for an entire large environment, but at a single, localised group.