12 x 5 x 0.035 TB in incremental non-prod backups<\/li>\n<\/ul>\nThat becomes 50TB (full prod) + 10 TB (full non-prod) + 10.5 TB (incr prod) + 2.1 TB (incr non-prod) over any 5 week period, or 72.6 TB instead of 90.75 TB \u2013 a saving of 20%.<\/p>\n
(If you’re still pushing your short-term operational backups to tape, your skin is probably crawling at the above suggestion: “I’ll need more tape drives!” Well, yes you would, because tape is inflexible. So using backup to disk means you can start saving on media, because you don’t need to make sure you have enough tape drives for every potential pool that would be written to at any given time.)<\/p>\n
A 20% saving on operational backups for 15TB of data might not sound like a lot, but now let’s start thinking about long term retention (LTR).<\/p>\n
There’s two particular ways we see long term retention data handled: monthlies kept for the entire LTR period, or keeping monthlies for 12-13 months and just keeping end-of-calendar-year (EoCY) + end-of-financial-year (EoFY) for the LTR period. I’d suggest<\/em> that the knee-jerk<\/em> reaction by many businesses is to keep monthlies for the entire time. That doesn’t necessarily have to be the case though \u2013 and this is the sort of thing that should also be investigated: do you legally<\/em> need to keep all your monthly backups for your LTR, or do you just need to keep those EoCY and EoFY backups for that period? That alone might be a huge saving.<\/p>\nLet’s assume though that you’re keeping those monthly backups for your entire LTR period. We’ll assume you’re also not in engineering, where you need to keep records for the lifetime of the product, or biosciences, where you need to keep records for the lifetime of the patient (and longer), and just stick with the tried-and-trusted 7 year retention period seen almost everywhere.<\/p>\n
For LTR, we also have to consider yearly growth. I’m going to cheat and assume 10% year on year growth, but the growth only kicks in once a year. (In reality for many businesses it’s more like a true compound annual growth, ammortized monthly, which does change things around a bit.)<\/p>\n
So let’s go back to those numbers. We’ve already established what we need for operational backups, but what do we need for LTR?<\/p>\n
If we’re not differentiating between prod and non-prod (and believe me, that’s common for LTR), then our numbers look like this:<\/p>\n
\n- Year 1: 12 x 15 TB<\/li>\n
- Year 2: 12 x 16.5 TB<\/li>\n
- Year 3: 12 x 18.15 TB<\/li>\n
- Year 4: 12 x 19.965 TB<\/li>\n
- Year 5: 12 x 21.9615 TB<\/li>\n
- Year 6: 12 x 24.15765 TB<\/li>\n
- Year 7: 12 x 26.573415 TB<\/li>\n<\/ul>\n
Total? 1,707.69 TB of LTR for a 7 year period. (And even as data ages out, that will still grow as the YoY growth continues.)<\/p>\n
But again, do you need to keep non-prod backups for LTR<\/em>? What if we didn’t \u2013 what would those numbers look like?<\/p>\n\n- Year 1: 12 x 10 TB<\/li>\n
- Year 2: 12 x 11 TB<\/li>\n
- Year 3: 12 x 12.1 TB<\/li>\n
- Year 4: 12 x 13.31 TB<\/li>\n
- Year 5: 12 x 14.641 TB<\/li>\n
- Year 6: 12 x 16.1051 TB<\/li>\n
- Year 7: 12 17.71561 TB<\/li>\n<\/ul>\n
That comes down to just 1,138 TB over 7 years \u2013 a 33% saving in LTR storage.<\/p>\n
We got that saving just by looking at splitting off non-production data from production data for our retention policies. What if we were to do more? Do you really need to keep all<\/em> of your production data for an entire 7-year LTR period? If we’re talking a typical organisation looking at 7 year retention periods, we’re usually only talking about critical systems that face compliance requirements \u2013 maybe some financial databases, one section of a fileserver, and email. What if that was just 1 TB of the production data? (I’d suggest that for many companies, a guesstimate of 10% of production data being the data required \u2013 legally required \u2013 for compliance retention is pretty accurate.)<\/p>\nWell then your LTR data requirements would be just 113.85 TB over 7 years, and that’s a saving of 93%<\/strong><\/span> of LTR storage requirements (pre-deduplication) over a 7 year period for an initial 15 TB of data.<\/p>\nI’m all for backing up a little bit too much than not enough, but once we start looking at LTR, we have to take that adage with a grain of salt. (I’ll suggest that in my experience, it’s something that locks a lot of companies into using tape for LTR.)<\/p>\n
Too much data will kill you,<\/p>\n
Just as sure as none at all<\/p><\/blockquote>\n
That’s<\/em> the lesson we get from Queen for LTR.<\/p>\n…Now if you’ll excuse me, now I’ve talked a bit about Queen, I need to go and listen to their greatest song of all time, March of the Black Queen<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"It’s fair to say I’m a big fan of Queen. They shaped my life \u2013 the only band to have even a…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[3,5],"tags":[240,1350,1348,1349],"class_list":["post-6186","post","type-post","status-publish","format-standard","hentry","category-architecture","category-backup-theory","tag-compliance","tag-elo","tag-long-term-retention","tag-queen"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pKpIN-1BM","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/6186","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/comments?post=6186"}],"version-history":[{"count":9,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/6186\/revisions"}],"predecessor-version":[{"id":7394,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/6186\/revisions\/7394"}],"wp:attachment":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/media?parent=6186"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/categories?post=6186"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/tags?post=6186"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"Glass\"](\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2016\/11\/bigStock-Cup.jpg\")
<\/a><\/p>\n