Many years ago, a company switched from ArcServe to NetWorker. They did so around the time they made their end of year backups, the ones that they intended to keep ‘forever’ for legal requirements.
Fast-forward several years, and it was requested to recover Lotus Notes backups from those original end of year archives. That’s when the support call came through. You see, those end of year archives were done on a standalone tape drive, not a tape library, and both tapes had, say, ‘YEAR2002’ written on the label. There was a little “1” noted on the first label, and a little “2” noted on the second label. For convenience, we’ll call them the first and second tapes.
When they put the first tape into the library for recovery, their first issue was getting NetWorker to mount the tape, since it didn’t have a barcode. Some non-GUI commands later, the tape was in the drive, but NetWorker wouldn’t keep the tape mounted – every time they tried to mount the tape, NetWorker threw up an error saying that it was expecting tape YEAR2002 with a particular volume ID, not YEAR2002 with a different volume ID that wasn’t in the media database. The second YEAR2002 tape would mount though, but NetWorker couldn’t perform a recovery because all the media wasn’t available.
So, here’s what happened:
- The manual backup was run of a bunch of systems and Lotus Notes.
- A tape was labelled YEAR2002 within NetWorker, and the backup ran until the tape filled up.
- A new tape was put into the tape drive, and since they had no exposure to NetWorker, they labelled that tape as YEAR2002 as well and the backup went on its way.
I’ll qualify here – the Lotus Notes backup was done using the module.
Now here’s the thing – while NetWorker works on the volume ID being unique, it also works on the volume label being unique as well. It won’t support two volumes in the media database at the same time with the same label. It gets pretty strident about that if you try to label one tape with another tapes’ label, but I guess if you’re new to NetWorker it might just seem like there’s a bunch of confirmation boxes you have to click before you can label your next tape.
So the net result was that the backup was written to two pieces of media that couldn’t co-exist in the media database at the same time. Scanning the first necessitates removing the second from the media database, and because this isn’t a filesystem backup, there are limitations that couldn’t be stepped around in recovering from partial savesets.
For a regular filesystem backup as a last resort this still would not be impossible to recover from – using scanner and uasm you can still suck the data off the tape(s) without NetWorker needing both in the media database. Tedious, and not as good as just being able to select data in a recovery program, but it’s better than no recovery at all. But you can’t use scanner and uasm for a non-filesystem recovery
(You also can’t write a new tape label to a fresh tape, then dd the NetWorker data after the label on the other tape onto the newly labelled tape. The volume ID (or some other unique volume identification system) is written into the savestream, and transferring that savestream onto another volume sees NetWorker reject it if you subsequently attempt to scan it.)
Net result? Data that could not be recovered short of sending it off to a specialist forensics data recovery company.
NetWorker’s fault? No. There is after all, only so much that software can do in order to prevent you from shooting yourself in the foot.