Introduction
When I entered the work force, for the first few months I trained as a MIMS consultant, but was then seconded to a system administration team on the other side of the country for 3 months (which became 6 months). Shortly after I returned from that, I joined the BHP IT Unix System Administration team in Newcastle and spent 4 years there. I built a lot of technical knowledge in that group, but what I got most out of that group was an understanding about what makes an excellent system administrator.
I was no means an excellent system administrator when I joined that team – I was wet around the ears, somewhat naïve, and probably too opinionated. The team that I joined taught me what makes an excellent system administrator, but in doing so also gave me an excellent foundation to some of the core requirements to be a good consultant too, and I thought it was time I shared these.
So here are what I’d call the 7 rules for system administrators, distilled from the experience of working with the best system administration team I’ve ever worked with. (While I’m at it – hello to Dave, Scott, Andrew, John, Jason and Russell.)
Knowledge Centric Approach
It wasn’t until after I stopped working with the Newcastle Unix team that I realised (to my horror) there were other ways system administration groups could run. There’s two distinct approaches:
- Knowledge centric approach – everyone knows a little about what everyone else is doing, and while any one person will be an expert on certain things, everyone is capable of getting involved with anything.
- Person centric approach – each system, application or function is administered by one or two people in the group at most, and the ability of the group to maintain those systems without the individuals being around is negligible at best.
My absolute belief is that any system administration team built around a person centric approach has it wrong. They do their users and the business a disservice.
Paranoia
While sometimes some of the people I’ve worked with have taken paranoia and security to extremes I find overboard, paranoia is a trait that should be considered a healthy mental attitude for system administrators. Paranoia in this case means not being overly trusting – having an idea of what processes should be running, requiring empirical evidence that the system is functional, and not making dangerous assumptions.
Testing
If you want to avoid testing, assume it doesn’t work. This is the mentality of a good system administrator. Since assuming everything doesn’t work means you have to assume that everything needs to be fixed, the alternative – having a testing regime and ensuring that changes don’t go into production without appropriate testing seems much easier.
Documentation
Documentation is vital to good systems operation and system administration. That covers the full gamut – system build documentation, procedural documentation, change control, etc. Why? Quite simply, if your systems and processes aren’t documented, then it means that you’re slipping into a person-centric approach to a system administration team.
Being Lazy
A good sysadmin is a lazy sysadmin. Lazy system administration is about automation. If a task that you perform requires you to run three commands, taking the output of each prior command and using it as input to the next command, you should be automating it. Every time you do repetitive, mundane tasks that can be scripted, you’re wasting your own time and company time. In my experience system administrators that religiously avoid scripting repetitive tasks lose up to an hour a day in mundane tasks that could be better spent elsewhere – self training, research, etc. (Of course, every bit of that automation needs good documentation!)
Only make a mistake once
We all make mistakes. Demanding people make no mistakes doesn’t account for how people learn. The trick of course is ensuring that we learn from our mistakes. That means that you should acknowledge that you’ll periodically make mistakes, but be ever determined to not make the same mistake twice.
Ask questions, listen to the answer
I used to say that the only stupid question is the one you don’t ask. This remains partially true, but it could equally be said that a stupid question is one that you ask, but don’t listen to the answer.
All system administrators should be prepared to ask one another questions (again, coming back to the knowledge-centric approach to system administration) – no one person in the team will have the answers to every single situation. But asking the question is only the first part of it. In fact, it’s probably only the first 30% at most.
The larger part – 70% of the effort, is taking the time to listen to the answer and making sure you understand the answer. In many cases that probably means asking some follow-up questions: question TLAs, question terms you don’t understand in the answer, and if the answer itself still doesn’t make sense, ask for more clarification. Sometimes you’ll have it explained to you, and sometimes you may be told that you need to do some research yourself. But don’t pretend to understand the answer when inside you’re just as confused.
In conclusion
While I’ve couched this from the perspective of rules for system administration, the techniques equally apply to just about any IT endeavor – backup administration, application administration, database administration, etc. All of these disciplines and more can follow the above principles and achieve an approach which is more satisfying – to the business, as well as from both a personal and professional perspective to the individual.