Training – it’s an uphill battle

 Backup theory  No Responses »
Apr 192012
 

Training is an uphill battle

To be perfectly blunt, staff training in backup and recovery products is somewhat of an uphill battle.

There’s a commonly held belief in many organisations that knowledge and understanding of backup products, even enterprise ones, should be acquired on the job via a review of product manuals and online forums.

Yet data protection is somewhat unique in this assumption – there are few organisations that believe storage administrators should learn how to manage the arrays that critical production data resides on: after all, one mistake and significant data loss can occur. If not data loss, significant production issues – slow downs, outright stalls, reduced failure capabilities, etc.

Backup and recovery systems touch on even more components of an environment than storage – arguably, in terms of IT, they may touch on more items of an environment than even the IP network (after all, they can encompass fibre networking as well). The reach of an enterprise backup system, fully deployed and fully protecting an organisation, is staggering in its breadth.

Trying to manage that using untrained staff is like trying to manage the fleet maintenance for an airline using self-taught mechanics who have excellent access to instruction manuals. Sure, they may muddle through regularly – but how well do they really understand what they’re doing?

10-15 years ago, the real struggle in IT was to get management to recognise the need for backups.

Now, the struggle is to make sure IT and business management understand they don’t really have a backup system until they have trained administrators. After all, if you look at what goes into making a backup system, the technology itself only plays a small part:

Backup system

All those components are actually fairly disparate – and there needs to be a unifying factor. That unifying factor is actually training; knowledge-empowered staff are able to appropriately test, are able to utilise the documentation and the technology to integrate with the processes, are able to liaise on the establishment of SLAs, etc.

Without training, everything comes with a higher risk factor. Sure, with training there still is a risk factor, but training can significantly diminish it.

 

 Posted by at 08:56  Tagged with:

Check in – New Years Resolutions

 Architecture, Backup theory  No Responses »
Jan 312012
 

Resolutions Check-in

In December last year I posted “7 new years backup resolutions for companies”. Since it’s the end of January 2012, I thought I’d check in on those resolutions and suggest where a company should be up to on them, as well as offering some next steps.

  1. Testing – The first resolution related to ensuring backups are tested. By now at least an informal testing plan should be in place if none were before. The next step will be to deal with some of the aspects below so as to allow a group to own the duty of generating an official data protection test plan, and then formalise that plan.
  2. Duplication – There should be documented details of what is and what isn’t duplicated within the backup environment. Are only production systems duplicated? Are only production Tier 1 systems duplicated? The first step towards achieving satisfactory duplication/cloning of backups is to note the current level of protection and expand outwards from that. The next step will be to develop tier guidelines to allow a specification of what type of backup receives what level of duplication. If there are already service tiers in the environment, this can serve as a starting point, slotting existing architecture and capability onto those tiers. Where existing architecture is insufficient, it should be noted and budgets/plans should be developed next to deal with these short-falls.
  3. Documentation – As I mentioned before, the backup environment should be documented. Each team that is involved in the backup process should have assigned at least one individual to write documentation relating to their sections (e.g., Unix system administrators would write Unix backup and recovery guidelines, etc., Windows system administrators would do the same for Windows, and so on). This should actually include 3 people: the writer, the peer reviewer, and the manager or team leader who accepts the documentation as sufficiently complete. The next step after this will be to handover documentation to the backup administrator(s) who will be responsible for collation, contribution of their sections, and periodic re-issuing of the documents for updates.
  4. Training – If staff (specifically administrators and operators) had previously not been trained in backup administration, a training programme should be in the works. The next step, of course, will be to arrange budget for that training.
  5. Implementing a zero error policy – First step in implementing a zero error policy is to build the requisite documents: an issues register, an exceptions register, and an escalations register. Next step will be to adjust the work schedules of the administrators involved to allow for additional time taken to resolve the ‘niggly’ backup problems that have been in the environment for some time as the switchover to a zero error policy is enacted.
  6. Appointing a Data Protection Advocate – The call should have gone out for personnel (particularly backup and/or system administrators) to nominate themselves for the role of DPA within the organisation, or if it is a multi-site organisation, one DPA per site. By now, the organisation should be in a position to decide who becomes the DPA for each site.
  7. Assembling an Information Protection Advisory Council (IPAC) – Getting the IPAC in place is a little more effort because it’s going to involve more groups. However, by now there should be formal recognition of the need for this council, and an informal council membership. The next step will be to have the first formal meeting of the council, where the structure of the group and the roles of the individuals within the group are formalised. Additionally, the IPAC may very well need to make the final decision on who is the DPA for each site, since that DPA will report to them on data protection activities.

It’s worth remembering at this point that while these tasks may seem arduous at first, they’re absolutely essential to a well running backup system that actually meshes with the needs of the business. In essence: the longer they’re put off, the more painful they’ll be.

How are you going?

 Posted by at 06:05  Tagged with: , , , , , , ,

7 New Years Backup Resolutions for Companies

 Backup theory  No Responses »
Dec 272011
 

New years resolutions for backup

I’d like to suggest that companies be prepared to make (and keep!) 7 new years resolutions when it comes to the field of backup and recovery:

  1. We will test our backups: If you don’t have a testing regime in place, you don’t have a backup system at all.
  2. We will duplicate our backups: Your backup system should not be a single point of failure. If you’re not cloning, replicating or duplicating your backups in some form, your backup system could be the straw that breaks the camel’s back when a major issue occurs.
  3. We will document our backups: As for testing, if your backup environment is undocumented, it’s not a system. All you’ve got is a collection of backups, which, if the right people are around at the right time and in the right frame of mind, you could get a recovery from it. If you want a backup system in place, you not only have to test your backups, you also have to keep them well documented.
  4. We will train our administrators and operators: It never ceases to amaze me the number of companies that deploy enterprise backup software and then insist that administrators and operators just learn how to use it themselves. While the concept of backup is actually pretty simple (“hey, you, back it up or you’ll lose it!”), the practicality of it can be a little more complex, particularly given that as an environment grows in size, so does the scope and the complexity of a backup system. If you don’t have some form of training (whether it’s internal, by an existing employed expert, or external), you’re at the edge of the event horizon, peering over into the abyss.
  5. We will implement a zero error policy: Again, there’s no such thing as a backup system when there’s no zero error policy. No ifs, no buts, no maybes. If you don’t rigorously implement a zero error policy, you’re flipping a coin every time you do a recovery, regardless of what backup product you use. (To learn more about a zero error policy, check out the trial podcast I did where that was the topic.)
  6. We will appoint a Data Protection Advocate: There’s a lot of data “out there” within a company, not necessarily under central IT control. Someone needs to be thinking about it. That someone should be the Data Protection Advocate (DPA). This person should be tasked with being the somewhat annoying person who is present at every change control meeting, raising her or his hand and saying “But wait, how will this affect our ability to protect our data?” That person should also be someone who wanders around the office(s) looking under desks for those pesky departmental servers and “test” boxes that are deployed, the extra hard drives attached to research machines, etc. If you have multiple offices, you should have a DPA per office. (The role of the DPA is outlined in this post, “What don’t you backup?“)
  7. We will assemble an Information Protection Advisory Council (IPAC): Sitting at an equal tier to the change control board, and reporting directly to the CTO/CIO/CFO, the IPAC will liaise with the DPA(s) and the business to make sure that everyone is across the contingencies that are in place for data protection, and be the “go-to” point for the business when it comes to putting new functions in place. They should be the group that sees a request for a new system or service and collectively liaises with the business and IT to ensure that the information generated by that system/service is protected. (If you want to know more about an IPAC and its role in the business, check out “But where does the DPA fit in?“)

And there you have it – the new years resolutions for your company. You may be surprised – while there’ll be a little effort getting these in place, once they’re there, you’re going to find backup, recovery, and the entire information protection process a lot easier to manage, and a lot more reliable.

 Posted by at 16:51  Tagged with: , , , , , , , , , , , ,

What’s wrong with vendor training?

 General Technology, General thoughts  1 Response »
Mar 272010
 

A recent discussion on Twitter about the high costs of training from NetApp got me thinking more about vendor training. So I’ll lay my cards on the table here: at the company I work for, IDATA, we do sell our own training. We like to differentiate our training from vendor training as being more focused on getting a chunk of information to the customers in as short a period of time as possible. Why? Because attention is currency, and time is money.

I’ll also reiterate something I’ve said before: I think most certification programmes are bullshit. That’s right, bullshit. If you think I’m being rude, well, personally I think I’m being polite about most certification programmes, so we’ll meet in the middle there.

So with this in mind, what’s wrong with vendor training?

Let’s consider what training should be about. It should be focused on the following goals:

  1. To give customers enough information to avoid misusing a product to the point of having a disaster.
  2. To allow customers to maximise their investment in a product by understanding as much as possible of it.
  3. To reduce as much as possible the amount of times the customer needs to engage with the support arm of the company. This isn’t just important for whomever is supplying the support, but also for the customer!

There’s a fourth goal as well – what I’d call a drag goal though, not a primary goal: good training in a good product should turn a customer into an advocate. But this can’t always be guaranteed. The above three are the essential goals of any training course.

So what’s wrong with a lot of vendor training these days?

Simple: many vendors become greedy with training, lose focus on those above three goals and instead turn it into a revenue stream. Now, I’m not suggesting that every single vendor training course falls into this category, but a lot do – and despite mentioning NetApp in the opening sentence, I’m not picking just on them. Symantec, EMC, NetApp, CommVault, etc., you’re all just as guilty at times as one another.

The biggest sign that someone has started to treat training as a revenue stream in its own right, and lost focus on those core three goals of training is when you start to see padding going into the course. Every course should have exactly the same amount of padding: nil.

Here’s some examples of padding:

  • Spending vast amounts of time trawling over trivial facts. (E.g., a backup training course once that spent three hours talking about the different generations of SCSI.)
  • Labs that go for an hour where the instructor leaves the room or concentrates on email. This usually means it can be done in 15 minutes if the instructor hangs around to help the slower students.
  • Courses that don’t start until say, 9.30 and then everyone’s out the door by 4 or 4.30 in the afternoon. That’s not an early mark, that’s a rip-off.
  • Courses where the content is clearly just the installation and administration manuals converted into powerpoint slides.

When training is treated as a revenue stream, those fundamental goals of training are being occluded – sometimes a little, and sometimes almost completely. Using backup as an example, Symantec, EMC and CommVault all do 5 day administration training courses. To this I say: rubbish, you can do it in 3. Pull out the padding, run the course for the full day (after all, you’re charging for the full day) and keep the labs well timed and you can do the course in 3 days without any loss of information to the customer. In fact, I’d suggest that in most cases a vendor’s 5 day course could be readily shrunk to 3 days and leave customers happier about the experience in almost all instances.

It’s tempting to turn training into a revenue stream, but in doing so, companies lose sight of the core purpose of the training. Training is not about profits – it’s about teaching customers to use what they’ve purchased and making support work efficiently.

 Posted by at 05:57  Tagged with:

Introducing the Support and Services page

 Aside  No Responses »
Dec 172009
 

Regular visitors may note that there’s a new addition to the pages on this blog – one covering Support and Services.

I run this blog in my own time (probably using up a little too much of my own time to be quite truthful) and ask for no payment or reimbursement from my readers – well, other than an occasional pitch for people to buy my book, that is.

My day job however is a consultancy and support role at IDATA Resolutions, and the Support and Services page outlines some of the key things IDATA could do for you, if you happen to be looking for service, support, consulting or training for your environment.

If you’re looking for an independent review of your environment, or considering support options, looking at a new solution or needing some training (whether that’s one-on-one, customised or general), I’d invite you to check out the Support and Services page above to see what IDATA can do for you.

 Posted by at 07:18  Tagged with: , , , , , , , , , ,

What would you like to see in advanced training?

 General thoughts, NetWorker  3 Responses »
Mar 102009
 

Over the years when I’ve been delivering various customised training courses, I’ve had many a customer ask “can we have an advanced training course?”

The time is now arriving – I’m now writing an advanced NetWorker training course. I know from personal experience with documentation and training though that everyone has different opinions of what constitutes “advanced”.

Obviously no course can cover absolutely everything that everyone wants. So, here’s an open question – if you had to give a “top 3 list” of subjects you’d like to see in an advanced training course in order for you to attend, what would they be?

 Posted by at 17:41  Tagged with:
© 2012 The NetWorker Blog Suffusion theme by Sayontan Sinha