Does your backup administrator have a say in change control?

…and if not, why?

A common mistake made in many companies is the failure to include the backup administrator (or, if there is a team, the team leader for data protection) in the change control approval process.

Typically the sorts of roles involved in change control include:

CIO or other nominated “final say” manager.
Tech writing the change request.
Tech’s manager approving the change request.
Network team.

Obviously there’s exceptions, and many companies will have variances – for instance, in most consulting companies, a sales manager will also get to have a say in change control, since interruptions to sales processes at the wrong time can break a deal.

Too infrequently included in change control is the backup administrator, or the team responsible for backup administration. The common sense approach to data protection would seem to suggest this is lunacy. After all, if a change fails, surely one potential remedy will be to recover from backup?

The error is three-fold:

Implicit assumption that any issue is recoverable from;
Implicit assumption that the backup system is always available;
Implicit assumption that what you need backed up is backed up.

Out of all of those assumptions, perhaps only the last is forgivable. As I point out in my book, and many have pointed out before me, it’s always better to backup a little too much than not quite enough. Thus, in a reasonable environment that has been properly configured, systems should be protected.

The three-fold assumptions error can actually be sumarised more succinctly though – assuming that having a backup system is a blank cheque on data recovery.

Common issues I’ve seen caused by failures to include backup administrators in change control include:

Having major changes timed to occur at the same time as scheduled down-time in the backup environment;
Kicking off full backups of large systems prior to changes without notification to the backup administrators, swamping media availability;
Scheduling changes to occur just prior to the next backup, making possible the maximal amount of data loss within the periodic backup frequency;
Not running fresh, full backups of version-critical database content after upgrades, and thus suffering significant outages later when a cross-version recovery is required;
Not checking version compatibility for applications or operating systems, resulting in “upgrades” that can’t be backed up;
Wasting backup administrators time searching for reasons why failures occurred because change outages ran during the backups.

To be blunt, any of the above scenarios that occur without pre-change signoff are inexcusable and represent a communications flaw within an organisation.

Any change that has potential to impact on or be impacted by the backup system should be subject to approval, or at the least, notification by the backup administrators. The logical consequence of this rule is: any change that has anything to do with IT systems should logically impact on or be impacted by the backup system.

Note that by impact on, I don’t mean just cause a deleterious effect to the backup system, but also more simply, require resources from the backup system (e.g., for the purposes of recovery, or even additional resources for more backups).

All of this falls into establishing policies surrounding the backup system, and I’m not talking what backs up when – but rather, implications that companies must face as a result of having backup systems in place. Helping organisations understand those policies is a major focus of my book.

Share this using...