Virtualisation.

It’s a fantastic blade to wield through a datacentre. Sweeping and scything, whole racks of equipment are reduced to single servers presenting dozens of hosts. All those driver disks? All those complex and fiddly options for hardware components during OS installation? Brushed aside – all the virtual components are simple and have rock solid drivers. Virtual machine host failing? That’s OK, just push the virtual machines across to another server without the users even noticing.

The improvements virtualisation has made to system efficiency, reliability, etc., in the x86/x86_64 field have been unquestionable.

Yet, like any other sword, it’s double edged.

Virtualisation is about cramming as many systems as is practical within a single bucket.

Backup is something that virtualisation has always handled poorly. And there’s a reason for this – virtualisation is designed for environments where the hosts cooperatively share access to resources. Thin provisioning isn’t just about storage – it’s also about CPU, networking and memory.

Backup isn’t about cooperative sharing of CPU, networking or memory. It’s about needing to get as much data from A to B as possible as quickly as can be done:

The problem with virtualisation backup

Backup at the guest level wants to suck as much data from the virtual network pipes provided by all those machines on the same host at the same time. You want to see the biggest, most powerful virtualisation server your company has ever bought grind to a halt and saturate the network as well? There’s a good chance backing up every guest it runs simultaneously will do the trick just nicely.

When VMware first came up with VCB, it was meant to be the solution. Pull the backup away from the guest, make it part of the hypervisor, and voilà, the problem is solved!

Except it was written by people who believed virtualisation applied only to Windows systems. And thus, it was laughably sad. No, I’m not having a dig at Windows here. But I am having a dig at the notion of homogeneous virtual environments. Sure, they exist, but designing products around them when you’re the virtualisation vendor is … well, I have to say, short sighted.

Perhaps for this reason, or perhaps for less desirable reasons, VCB never really gained the traction VMware likely hoped for, and so something else had to be developed. Something more expansive.

So, VADP was meant to be the big, grand solution to this. And indeed, the VADP API allows more than just Windows systems backups to be performed in such a way that file level recovery from those backups is possible.

What’s the vendor support like though? Haphazard, irregular and inconsistent would probably be the best description. Product X: “Oh, you want to backup a database as well? You need to revert to a guest agent.” Product Y: “Huh? Linux? Guest agent.” Product Z: “Linux? Sure! For any system – well, any that uses ext2 or ext3 filesystems” … you get the picture.

So the problem with VADP is that it’s only a partial solution. In fact, it’s less than half the solution for backing up virtual machines on VMware. It’s maybe 40%. The other 40% is provided by whatever backup product you’re using, and there’s 20% glue.

Between that 40%, 20% and 40%, there’s a lot of scope for things to fall through the cracks.

Where “things” are:

  • Guests using operating systems the backup product doesn’t support VADP with;
  • Guest using filesystems the backup product doesn’t support VADP with;
  • Guests using databases or applications the backup product doesn’t support VADP with.

VADP is the emperor’s new clothes. Everyone is sold on it until the discussions start around what they can’t do with it.

I’m tired of VADP being seen as a silver bullet. That’s the real problem – it doesn’t matter how many hoozits a widget has – if it doesn’t have the hoozit you need, the widget is not fit for your purposes.

I’m not pointing the finger at EMC here. I don’t see a single backup vendor, enterprise or otherwise, providing complete backup solutions under VADP. There’s always something missing.

Until that isn’t the case, you’ll excuse me if I don’t drink the VADP koolaid.

After all, my job is to make sure we can backup everything, not just the easy bits.

 

Earlier in the year, I wrote a post, “Technology is rarely the issue“. In that post, I said:

As techos though, let’s be honest. The technology is rarely the issue. Or to be more accurate, if there’s an issue, technology is the tip of the iceberg – the visible tip. And using the iceberg analogy, you know I mean that technology is rarely going to be the majority of the issue.

Now it’s time for the follow-up.

In that article, I was effectively talking about specific situations – e.g., when someone says to you, “product X is crap; we’ve had it for Y months and it still doesn’t work properly”. While sometimes it will mean that product X is bad, it usually means that the wrong product was purchased, or there wasn’t enough training, or it’s being misused.

In this post, I want to turn from the specific, to the generic, and suggest that it is rarely going to be the case moving forward that technology is the solution. It will be part of the solution, but we are moving out of a situation where a single piece of technology is the entire solution to a problem. In fact I’d suggest that it was rarely the case anyway, but we must be more aware as technology continues to get more powerful – it’s not a magic bullet.

For this reason, the core emerging technology – that we must continue to demand from vendors, and continue to support the development of – is interoperability. This may come from open standards, or it may come from virtualisation, but it has to become core to all future technology.

Why?

We no longer have the luxury of mass swapping and changing of technology. Martin Glasborow, AKA Storagebod, wrote in “Migration is a way of life“:

One of the things which is daunting is the sheer amount of data that we are beginning to ingest and the fact that we are currently looking at a ‘grow forever’ archive; everything we ingest, we will keep forever.

Even though we are less than two years into the project, we are already thinking about the next refresh of technology. And what is really daunting is that with our data growth; once we start refreshing, I suspect that we will never stop.

Not only will we be storing petabytes of new content every year; we will be moving even more old content between technologies every year. We are already looking at moving many hundreds of terabytes into the full production system without impacting operations and with little to no downtime.

While Martin’s organisation is undoubtedly at the “big data” end of town, it reflects a growing problem for many organisations – the shrinking grace period. Previously we had scenarios where capital expenditure periods of say, 3 years worth of equipment purchase, would have short implementation periods, followed by long-term controlled and pre-allocated growth periods, followed by the final preparatory process leading into the next CapEx cycle.

This is increasingly becoming a luxury. As data growth continues, regardless of whether that data is hosted locally or externally, mass migration projects will become a thing of the past. It’s not possible to stop a business long enough to do a migration. They have to run seamlessly and synchronously in the background, transparent to users and the business, and the only way this will happen is via interoperability.

The two methods to achieve this are either compatible APIs/protocols, and virtualisation. In the cloud space for instance, whatever brick level storage is chosen, only a fool would deploy their business storage on just a single cloud provider. So you need two different providers, and you need to be able to interface with the same storage at both providers without every access step being an “If writing to Cloud X, this way, else that way.”

For locally accessible storage, virtualisation is critical – not just at the OS layer, but also at the storage layer. That way, it doesn’t matter whether you’re currently buying vendor X, Y or Z arrays and storage – and which ones are currently active. It should all be transparent to the business.

This is why technology is not the solution. Or rather, specific technology is not the solution. It’s the application of technology, and the interoperability of currently deployed technologies that will be the solution every time.

If you’re not thinking along these lines, you’re still staring into the past.

 

I have to admit, I have great personal reservations towards virtualising backup servers. There’s a simple, fundamental reason for this: the backup server should have as few dependencies as possible in an environment. Therefore to me it seems completely counter-intuitive to make the backup server dependent on an entire virtualisation layer existing before it can be used.

For this reason I also have some niggling concerns with running a backup server as a blade server.

Personally, at this point in time, I would never willingly advocate deploying a NetWorker server as a virtual machine (except in a lab situation) – even when running in director mode.

Let me qualify: I consider ‘director’ mode to be where the NetWorker server acts almost like a dedicated storage node – it only backs up its own index/bootstrap information; with all other backups in the datazone being sent to storage nodes. Hence, as much as possible, all it is doing is ‘directing’ the backups.

But I’m keen to understand your thoughts on the matter.

This survey has now closed.

 

Once upon a time, if you said to someone “do you have a test environment?” there was at least a 70 to 80% chance that the answer would be one of the following:

  • Only some very old systems that we decommissioned from production years ago
  • No, management say it’s too expensive

I’d like to suggest that these days, with virtualisation so easy, there are few reasons why the average site can’t have a reasonably well configured backup and recovery test environment. This would allow the following sorts of tests could be readily conducted:

  • Disaster recovery of hosts and databases
  • Disaster recovery of the backup server
  • Testing new versions of operating systems, databases and applications with the backup software
  • Testing new versions of the backup software

Focusing on the Intel/x86/x86_64 world, we see where this is immediately achievable. Remember, for the average set of tests that you run, speed is not necessarily going to be the issue. Let’s focus on non-speed functionality testing, and think of what would be required to have a test environment that would suit many businesses, regardless of size:

  1. Virtualisation server – obviously VMware ESXi springs to mind here, if cost is a driving factor.
  2. Cheap storage – if performance is not an issue for testing (i.e., you’re after functionality not speed testing), there’s no reason why you can’t use cheap storage. A few 2TB SATA drives in a RAID-5 configuration will give you oodles of space if you need any level of redundancy, or just in a RAID-0 stripe will give you capacity and performance. Optionally present storage via iSCSI if its available.
  3. Tiny footprint – previously test environments were disqualified in a lot of organisations, particularly those at locations where space was at a premium. Allocating room for say, 15 machines to simulate part of the production network took up tangible space – particularly when it was common for test environments to not be built using rackable equipment.

In the 2000′s, much excitement was heralded over the notion of supercomputers at your desk – for example, remember when Orion released a 96-CPU capable system? The notion of that much CPU horsepower under your desk for single tasks may be appealing to some, but let’s look at more practical applications flowing from multi-core/multi-CPU systems – a mini datacentre under your desk. Or in that spare cubicle. Or just in a 3U rack enclosure somewhere within your datacentre itself.

Gone are the days when backup and recovery test environments are cost prohibitive. You’re from a small organisation? Maybe 10-20 production servers at most? Well that simply means your requirements will be smaller and you can probably get away with just VMware Workstation, VMware Fusion, Parallels or VirtualBox running on a suitably powerful desktop machine.

For companies already running virtualised environments, it’s more than likely the case that you can even use a production virtualisation server due for replacement as a host to the test environment, so long as it can still virtualise a subset of the production systems you’d need to test with. During budgetary planning this can make the process even more painless.

This sort of test environment obviously doesn’t suit every single organisation or every single test requirement – however, no single solution ever does. If it does suit your organisation though, it can remove a lot of the traditional objections to dedicated test environments.

 

I’m stepping out of my normal NetWorker zone here to briefly discuss what I think is a fundamental flaw with the current state of thin provisioning.

The notion of thin provisioning has effectively been around for ages, since it’s effectively from the mainframe age, but we started to see it come back into focus a while ago with the notion of “expanding disks” for virtualisation products. Ironically these started initially in the workstation products (VMware Workstation, Parallels Desktop, etc.) before starting to gain popularity at the enterprise virtualisation layer.

Yet thin provisioning doesn’t stop there – it’s also available at the array level, particularly in NAS devices as well. So what happens when you mix guest thin provisioning in a hypervisor with thin provisioning at the array/NAS level providing storage to the hypervisor?

Chaos.

Multiple layers of thin provisioning is potentially a major management headache in systems storage allocation. Why? It makes determining what storage you have available and allocated, when looking at any one layer, practically impossible. vSphere for instance may see that you’ve got 2TB of free space in storage that’s currently unallocated, and your NAS may be telling it there’s 2TB of free space, but it may actually only have 500GB free. Compounding the issue, the individual operating systems leveraging that storage as guests will also each have their own ideas about how much storage is available for use. One system suffering unexpected data growth (e.g., a patch provided by a vendor without warning that it’ll generate thousands of log messages a minute) might cause the entire thin provisioning sand castle to collapse around you.

This leads me to my concern about what’s missing in thin provisioning: a consolidated dashboard. A cross platform, cross vendor dashboard where every product that advertises “thin provisioning” can share information in the storage realm so that you, the storage administrator, can instantly see an exact display of allocated vs available real capacity.

This isn’t something that’s going to appear tomorrow, but I’d suggest that if all the vendors currently running around shouting about “thin provisioning” are really serious about it, they’d come up with a common, published API that can be used by any product to query through the entire storage-access vertical. I regret to say the C-word, but it’s clear there needs to be an inter-vendor Committee to discuss this requirement. That’s right, NetApp and EMC, HDS and HP, VMware and Microsoft (just to name a few) all need to sit at the same table and agree on a common framework that can be leveraged.

Without this, we’ll just keep going down the current rather chaotic and hazardous thin provisioning pathway. It’s like an uncleared minefield – you may manage to stagger through it without being blown up, but the odds are against you.

Surely even the vendors can see the logical imperative to reduce those odds.

Disclaimer: I’m prepared to admit that I’m completely wrong, and that vendors have already tackled this and I missed the announcement. Someone, please prove me wrong.

 

As an employee of an EMC partner, I periodically get access to nifty demos as VMs. Unfortunately these are usually heavily geared towards running within a VMware hosted environment, and rarely if ever port across to Parallels.

While this wasn’t previously an issue having an ESX server in my lab, I’ve slowly become less tolerant of noisy computers and so it’s been less desirable to have on – part of the reason why I went out and bought a Mac Pro. (Honestly, PC server manufacturers just don’t even try to make their systems quiet. How Dull.)

With the recent upgrade to Parallels v5 being a mixed bag (much better performance, Coherence broken for 3+ weeks whenever multiple monitors are attached), on Thursday I decided I’d had enough and felt it was time to start at least trying VMware Fusion. As I only have one VM on my Mac Book Pro, as opposed to 34 on my Mac Pro, I felt that testing Fusion out on my Mac Book Pro to start with would be a good idea.

[Edit 2009-12-08 – Parallels tech support came through, the solution is to decrease the amount of VRAM available to a virtual machine. Having more than 64MB of VRAM assigned in v5 currently prevents Parallels from entering Coherence mode.]

So, what are my thoughts of it so far after a day of running with it?

Advantages over Parallels Desktop:

  • VMware’s Unity feature in v3 isn’t broken (as opposed to Coherence with dual monitors currently being dead).
  • VMware’s Unity feature actually merges Coherence and Crystal without needing to just drop all barriers between the VM and the host.
  • VMware Fusion will happily install ESX as a guest machine.
  • (For the above reason, I suspect, though I’ve not yet had time to test, that I’ll be able to install all the other cool demos I’ve got sitting on a spare drive)
  • VMware’s Unity feature extends across multiple monitors in a way that doesn’t suck. Coherence, when it extends across multiple monitors, extends the Windows Task Bar across multiple monitors in the same position. This means that it can run across the middle of the secondary monitor, depending on how your monitors are layed out. (Maybe Coherence in v5 works better … oops, no, wait, it doesn’t work at all for multiple monitors so I can’t even begin to think that.)

Areas where Parallels kicks Fusion’s Butt:

  • Even under Parallels Desktop v4, Coherence mode was significantly faster than Unity. I’m talking seamless window movement in Coherence, with noticeable ghosting in Unity. It’s distracting and I can live with it, but it’s pretty shoddy.
  • For standard Linux and Windows guests, I’ve imported at least 30 different machines from VMware ESX and VMware Server hosted environments into Parallels Desktop. Not once did I have a problem with “standard” machines. I tried to use VMware’s import utility this morning on both a Windows 2003 guest and a Linux guest and both were completely unusable. The Windows 2003 guest went through a non-stop boot cycle where after 5 seconds or so of booting it would reset. The Linux guest wouldn’t even get past the LILO prompt. Bad VMware, very Bad.
  • When creating pre-allocated disks, Parallels is at least twice as fast as Fusion. Creating a pre-allocated 60GB disk this morning took almost an hour. That’s someone’s idea of a bad joke. Testing creating a few other drives all exhibited similarly terrible performance.
  • Interface (subjective): Parallels Desktop v5 is beautiful – it’s crisp and clean. VMware Fusion’s interface looks like it’s been cobbled together with sticks and duct tape.

Areas where Desktop Virtualisation continues to suck, no matter what product you use:

  • Why do I have to buy a server class virtualisation product to simulate turning the monitor off and putting the keyboard away? That’s not minimising the window, it’s called closing the window, and I should be able to do that regardless of what virtualisation software I’m running.
  • Why does the default for new drives remain splitting them in 2GB chunks? Honestly, I have no sympathy for anyone still running an OS old enough that it can’t (as the virtual machine host) support files bigger than 2GB. At least give me a preference to turn the damn behaviour off.

I’ll be continuing to trial Fusion for the next few weeks before I decide whether I want to transition my Mac Pro from Parallels Desktop to Fusion. The big factor will be whether I think the advantages of running more interesting operating systems (e.g., ESX) within the virtualisation system is worth the potential hassle of having to recreate all my VMs, given how terribly VMware’s Fusion import routine works…

[Edit 2009-12-08 – Parallels tech support came through, the solution is to decrease the amount of VRAM available to a virtual machine. Having more than 64MB of VRAM assigned in v5 currently prevents Parallels from entering Coherence mode.]

 

If you thought in the storage blogosphere that this week had seen the second coming, you’d be right. Well, the second coming of Drobo, that is. With new connectivity options and capacity for more drives, Drobo has had so many reviews this week I’ve struggled to find non-Drobo things to read at times. (That being said, the new versions do look nifty, and with my power bills shortly to cutover to have high on-peak costs and high “not quite on-peak” costs, one or two Drobos may just do the trick as far as reducing the high number of external drives I have at any given point in time.)

Tearing myself away from non-Drobo news, over at Going Virtual, Brian has an excellent overview of NetWorker v7.6 Virtualisation Features. (I’m starting to think that the main reason why I don’t get into VCBs much though is the ongoing limited support for anything other than Windows.)

At The Backup Blog, Scott asks the perennial question, Do You Need Backup? The answer, unsurprisingly is yes – that was a given. What remains depressing is that backup consultants such as Scott and myself still need to answer that question!

StorageZilla has started Parting Shot, a fairly rapid fire mini-blogging area with frequent updates that are often great to read, so it’s worth bookmarking and returning to it frequently.

Over at PenguinPunk, Dan has been having such a hard time with Mozy that it’s making me question my continued use of them – particularly when bandwidth in Australia is often index-locked to the price of gold. [Edit: Make that has convinced me to cancel my use of them, particularly in light of a couple of recent glitches I've had myself with it.]

Palm continues to demonstrate why it’s a dead company walking with the latest mobile backup scare coming from their department. I’d have prepared a blog entry about it, but I don’t like blogging about dead things.

Grumpy Storage asks for comments and feedback on storage LUN sizings and standards. I think a lot of it is governed by the question “how long is a piece of string”, but there are some interesting points regarding procurement and performance that are useful to have stuck in your head next time you go to bind a LUN or plan a SAN.

Finally, the Buzzword Compliance (aka “Yawn”) award goes to this quote from Lauren Whitehouse of the “Enterprise Strategy Group” that got quoted on half a zillion websites covering EMC’s release of Avamar v5:

“Data deduplicated on tape can expire at different rates — CommVault and [IBM] TSM have a pretty good handle on that,” she said. “EMC Avamar positions the feature for very long retention, but as far as a long-term repository, it would seem to be easy for them to implement a cloud connection for EMC Avamar, given their other products like Mozy, rather than the whole dedupe-on-tape thing.”

(That Lauren quote, for the record, came from Search Storage – but it reads the same pretty much anywhere you find it.)

Honestly, Cloud Cloud Cloud. Cloud Cloud Cloud Cloud Cloud. Look, there’s a product! Why doesn’t it have Cloud!? As you can tell, my Cloud Filter is getting a little strained these days.

Don’t even get me started on the crazy assumption that just because a company owns A and B they can merge A and B with the wave of a magic wand. Getting two disparate development teams to merge disparate code in a rush, rather than as a gradual evolution, is usually akin to seeing if you can merge a face and a brick together. Sure, it’ll work, but it won’t be pretty.

 

I’ve been using Parallels Desktop for the Mac for several years now – in fact, when I originally started using it, VMware were only talking about doing a desktop virtualisation product for the Mac.

That’s partly why I stay loyal to Parallels – they supported the platform sooner. The other reason is that after years of hearing tripe from VMware employees about why you couldn’t mix windows from both operating systems, Parallels went ahead and did it with Coherence. (Yes, VMware Fusion’s Unity functionality went there too, around the same time, but the amount of times I heard it wasn’t a feature they were interested in within Workstation, as an example, drove me nuts.)

So when Parallels Desktop v5 for the Mac came out, I jumped on the upgrade bandwagon. It’s given me one major positive – I can now run Solaris 10 AMD 64-bit on my Mac Pro; that was the one remaining OS I absolutely need to periodically run that I was being blocked from doing previously. After about 3 days of installing, reinstalling, downloading more recent versions of Solaris 10 AMD, I even finally managed to get networking operational within the VM too, which meant it was useful.

Other than that, Parallels v5 has been a bit of a disappointment. You see, currently for me, Coherence mode only works if I’ve got a single monitor attached to my machine. The only time that happens is when I’m using my laptop away from my desk, or when I’m traveling – and those are times I’m less likely to run VMs. Since Coherence doesn’t work for me 99% of the time, that means I can’t really try out the Crystal mode – though I’ll admit, I’m unlikely to use it heavily; I dislike the “shared apps” approach offered by Parallels, and the new Crystal view mode seems to rely heavily on this.

It does continue to annoy me that I don’t have the option of virtually turning the monitor off – why can’t I close the console for a running VM? Surely that’s not so huge a thing that it can only be limited to enterprise class virtualisation – aka ESX, VMware Server or Parallels Server. When you have 10+ VMs running at once, even minimised all those consoles start to get annoying.

While overall I’m pretty happy with Parallels performance in terms of memory and CPU, recent support cases have highlighted to me that when it comes to translated IO, Parallels struggles – it seems to peak at about 20MB/s per VM, regardless of the throughput capabilities of the attached device. I first noticed that under v4, and was unhappy to see no change in v5.

If you’re currently a Parallels Desktop for Mac user, and you haven’t yet upgraded, I’d suggest holding off until the next build of v5, rather than jumping into the initial build released. Hopefully by then they’ll at least have Coherence reliably working again.

[Edit, 2009-11-14]

I’d like to take back my comments about Parallels 5 still having mediocre IO performance. I just realised, a short while ago, that one of the VMs I’d been testing with had failed in its VMware Tools update. Now, with both VMs in this particular test config updated to Parallels Tools v5, instead of getting 14-17MB/s transfer speed between them as I’d been getting under Paralles v4, I’m now getting 45-57MB/s. Now that’s a performance improvement.

 

When IT people discuss Mean Time Between Failure (MTBF), the most common focus is on disk drives. We all know the basics for instance – the more drives you put in an array, the lower the cumulative MTBF, etc.

What impact does virtualisation have on MTBF though? Are there any published studies? I suspect not yet.

I’ll be clear from the outset: I like virtualisation.

Just because I like it though doesn’t lead me to question how many sites (particularly smaller ones) implement it, and the risks that they carry of effectively decreased MTBF by putting too many eggs in one basket.

Consider for instance a small business that decides, as part of an infrastructure refresh, to replace their current fileserver, directory server, mail server, database server and internet gateway server with a single VMware ESX server. (We’ll assume of course that they do not virtualise their backup server – something you should never do.)

So, instead of having five primary production servers, each of which has some chance of experiencing a catastrophic failure, we now have one primary production server which can still experience catastrophic failure. I’m not talking at the OS layer here (though that’s still relevant), but at the hardware layer.

Let’s be honest with ourselves – this is IT, and things can go wrong in IT just as they can anywhere else.

Now, in a small business such as the above, it can be argued that the loss of any one server is likely to cause a fair to serious inconvenience, but in each case, other functions are likely to still be carried out while the hardware is being repaired. If people can’t email, they may be able to catch up on some documentation or file related work. If people can’t access the database, they may be able to process things manually while still emailing, etc.

If all five servers go down at once, that’s a significantly more challenging proposition.

Anyone with exposure to virtualisation, high availability/redundancy or data protection should see what is needed here – a second server, shared storage and the ability to have guest systems moved from one virtualisation server to the other. (In smaller companies it may be achieved instead by just having a standby server with storage that can be accessed by the other host if necessary.)

However, it’s clear there’s more to running a virtualised environment than just whacking a big server in and virtualising the hosts that are already in the computer room.

Companies that are now just starting to adopt virtualisation may feel that it’s a mature enough industry that the time is ripe for jumping in – and they’re right. In fact, it’s been mature enough for long enough that virtualisation is practically old hat.

Regardless of the maturity of virtualisation though, it doesn’t change the fact that you’re still at the mercy of hardware failures (or other critical virtualisation-host failures), and you still have to design your systems to provide the appropriate level of protection you can (a) afford and (b) is necessary. When doing cost comparisons, it’s not appropriate to compare say, the cost of replacing 5 servers with another 5 servers vs replacing 5 servers with 1 beefier server – virtualised services should never be about putting all the eggs in just one basket.

Without that consideration, it’s too easy to see MTBF for your computing environment fall through the floor – and blame virtualisation technology instead of the real culprit: the practical implementation.

 

My colleague Brian Norris has been continuing his VMware coverage over at Going Virtual.

Recently he’s been doing a lot of work on securing ESX, integrating ESX into Active Directory, and experimenting with vSphere v4. If you’re interested in VMware and are looking for some tips and coverage from an expert, I’d suggest you keep an eye on his site.

© 2012 The NetWorker Blog Suffusion theme by Sayontan Sinha