{"id":3997,"date":"2012-11-27T17:54:44","date_gmt":"2012-11-27T07:54:44","guid":{"rendered":"http:\/\/nsrd.info\/blog\/?p=3997"},"modified":"2018-12-11T14:29:54","modified_gmt":"2018-12-11T04:29:54","slug":"divisibility-of-eggs","status":"publish","type":"post","link":"https:\/\/nsrd.info\/blog\/2012\/11\/27\/divisibility-of-eggs\/","title":{"rendered":"The divisibility of eggs"},"content":{"rendered":"

The caution about keeping all of ones eggs in the one basket is a fairly common one.<\/p>\n

It’s also a fairly sensible one; after all, eggs are fragile things and putting all of them into a single basket without protection is not necessarily a good thing.<\/p>\n

Yet, there’s an area of backup where many smaller companies easily forget the lesson of eggs-in-baskets, and that area is deduplication.<\/p>\n

The mistake made is assuming there’s no need for replication. After all, no matter what the deduplication system, there’s RAID protection, right? Looking just at EMC, with either Avamar or Data Domain, you can’t deploy the systems without RAID*.<\/p>\n

As we all know, RAID doesn’t protect you from accidental deletion of data \u2013 in mirrored terms, deleting a file from one side of the mirror doesn’t even commit the operation until it’s been completed on the other side of the mirror. It’s the same for all other RAID.<\/p>\n

Yet deduplication is potentially very much like putting all ones eggs in one basket when comparing to conventional storage of backups. Consider the following scenario in a non-deduplication environment:<\/p>\n

\"Backup<\/a><\/p>\n

In this scenario, imagine you’re doing a full backup once a week of 1.1TB, and incrementals all other days, with each incremental averaging around 0.1TB. So at the end of each week you’ll have backed up 1.7TB. However, cumulatively you keep multiple backups over the retention period, so those backups will add up, week after week, until after just 3 weeks you’re storing 5.1TB of backup.<\/p>\n

Now, again keeping the model, imagine a similar scenario but with deduplication involved (and not accounting for any deduplication occurring within<\/em> any individual backup):<\/p>\n

\"Backup<\/a><\/p>\n

 <\/p>\n

Now, again, I’m keeping things really simple and not necessarily corresponding to a real-world model. However, while each week may see 1.7TB backed up, cumulatively, week after week, the amount of data stored by the deduplication system will be much lower; 1.7TB at the end of the first week, 2.3TB at the end of the second, 2.9TB at the end of the third.<\/p>\n

Cumulatively, where do those savings come from? By not storing extra copies of data. Deduplication is about eliminating redundancy.<\/p>\n

On a single system, deduplication is<\/em> putting all your eggs in one basket. If you accidentally delete a backup (and it gets scrubbed in a housekeeping operation), or if the entire unit fails, it’s like dropping the basket. It’s not just one<\/em> backup you lose, but all<\/em> backups that referred to the specific data lost. It’s something that you’ve got to be much more careful about. Don’t treat RAID as a blank cheque.<\/p>\n

The solution?<\/p>\n

It’s trivially simple, and it’s something every vendor and system integrator worth their salt will tell you: when you’re deduplicating, you must<\/em> replicate (or clone, in a worse case scenario), so you’re protected. You’ve got to start storing those twin eggs in another basket.<\/p>\n

Cloning of course is important in non-deduplicated backups, but if you’ve come from a non-deduplicated backup world, you’re used to having at least a patchy safety net involved \u2013 with multiple copies of most data generated, even in an uncloned situation if a recovery from week 2 fails, you might be able to go back to the week 3 backup and recover what you need, or at least enough to save the day.<\/p>\n

The message is simple:<\/p>\n

Deduplication = Replication<\/p>\n

If you’re not replicating or otherwise similarly protecting your deduplication environment, you’re doing it wrong. You’ve put your eggs all in one basket, and forgotten that you can’t unbreak an egg.<\/p>\n

—
\nWell, technically, you could probably sneak in an AVE deployment without RAID, but you’d be getting fairly desperate.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

The caution about keeping all of ones eggs in the one basket is a fairly common one. It’s also a…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[5,19],"tags":[301,820],"class_list":["post-3997","post","type-post","status-publish","format-standard","hentry","category-backup-theory","category-recovery","tag-deduplication","tag-replication"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pKpIN-12t","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/3997","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/comments?post=3997"}],"version-history":[{"count":1,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/3997\/revisions"}],"predecessor-version":[{"id":7473,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/3997\/revisions\/7473"}],"wp:attachment":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/media?parent=3997"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/categories?post=3997"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/tags?post=3997"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}