{"id":2721,"date":"2011-01-14T06:18:08","date_gmt":"2011-01-13T20:18:08","guid":{"rendered":"http:\/\/nsrd.info\/blog\/?p=2721"},"modified":"2018-12-11T18:26:12","modified_gmt":"2018-12-11T08:26:12","slug":"deleting-with-reckless-abaddon","status":"publish","type":"post","link":"https:\/\/nsrd.info\/blog\/2011\/01\/14\/deleting-with-reckless-abaddon\/","title":{"rendered":"Deleting with Reckless Abaddon"},"content":{"rendered":"<p>This is the fifth and final part of our four part series &#8220;Data Lifecycle Management&#8221;. (By slipping in an aside article, I can pay homage to Douglas Adams with that introduction.)<\/p>\n<p>So far in data lifecycle management, I&#8217;ve discussed:<\/p>\n<ul>\n<li><a title=\"Basic data lifecycle\" href=\"https:\/\/nsrd.info\/blog\/2011\/01\/04\/a-basic-data-lifecycle\/\" target=\"_blank\">The basic data lifecycle<\/a>;<\/li>\n<li><a title=\"The importance of being archived (and deleted)\" href=\"https:\/\/nsrd.info\/blog\/2011\/01\/10\/the-importance-of-being-archived-and-deleted\/\" target=\"_blank\">The importance of being archived (and deleted)<\/a>;<\/li>\n<li><a title=\"Sisyphus, the storage king\" href=\"https:\/\/nsrd.info\/blog\/2011\/01\/13\/sisyphus-the-storage-king\/\" target=\"_blank\">The need to delete as well as archive<\/a><\/li>\n<\/ul>\n<p>Now we need to get to our final part \u2013 the need to archive rather than just blindly deleting.<\/p>\n<p>You might think that this and the previous article are at odds with one another, but in actual fact, I want to talk about the recklessness of deliberately using a backup system as a safety net to facilitate data deletion rather than incorporating archive into data lifecycle management.<\/p>\n<p>My first introduction to deleting with reckless abaddon was at a University that instituted filesystem quotas, but due to their interpretation of academic freedom, could not institute mail quotas. Unfortunately one academic got the crafty notion that when his home directory filled, he&#8217;d create zip files of everything in the home directory and email it to himself, then delete the contents and start afresh. Viol\u00e1! Pretty soon the notion got around, and suddenly storage exploded.<\/p>\n<p>Choosing to treat a backup system as a safety net\/blank cheque for data deletion is really quite a devilishly reckless thing to do. It may seem &#8220;smart&#8221; since the backup system is designed to recover lost data, but in reality it&#8217;s just plain dumb. It creates two very different and very vexing problems:<\/p>\n<ul>\n<li>Introduces unnecessary recovery risks<\/li>\n<li>Hides the real storage requirements<\/li>\n<\/ul>\n<p>In the first instance: if it&#8217;s fixed, don&#8217;t break it. Deliberately <em>increasing<\/em> the level of risk in a system is, as I&#8217;ve said from the start, a reckless activity. A single backup glitch and poof! that important data you deleted because you temporarily needed more space is <em>never, ever coming back<\/em>. Here&#8217;s an analogy: running out of space in production storage? Solution? Turn off all the mirroring and now you&#8217;ve got DOUBLE the capacity! That&#8217;s the level of recklessness that I think this process equates to.<\/p>\n<p>The second vexing problem it creates is that it completely hides the real storage requirements for an environment.&nbsp;If your users and\/or administrators are deleting required primary data willy-nilly, you don&#8217;t ever actually have a real indication of how much storage you really need. On any one day you may appear to have plenty of storage, but that could be a mirage &#8211; the heat coming off a bunch of steaming deletes that shouldn&#8217;t have been done. This leads to over-provisioning in a particularly nasty way \u2013 approving new systems or new databases, etc., thinking there&#8217;s plenty of space, when in actual fact, you&#8217;ve maybe run out multiple times.<\/p>\n<p>That is, over time, we can describe storage usage and deletion occurring as follows:<\/p>\n<p><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/deleting-with-reckless-abaddon.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-2730\" title=\"Deleting with reckless abaddon\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/deleting-with-reckless-abaddon.png\" alt=\"Deleting with reckless abaddon\" width=\"538\" height=\"531\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/deleting-with-reckless-abaddon.png 538w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/deleting-with-reckless-abaddon-300x296.png 300w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/deleting-with-reckless-abaddon-303x300.png 303w\" sizes=\"auto, (max-width: 538px) 100vw, 538px\" \/><\/a><\/p>\n<p>This shows very clearly the problem that happens in this scenario \u2013 as multiple deletes are done over time to restore primary capacity, the amount of data that is deleted but known to be required later builds to the point where its not physically possible to have all of it residing on primary storage any longer should it be required. All we do is create a new headache while implementing at best a crude workaround.<\/p>\n<p>In fact, in this new age of thin provisioning, I&#8217;d suggest that the companies where this is practiced rather than true data lifecycle management have <em>a very big nightmare<\/em> ahead of them. Users and administrators who are taught data management on the basis of &#8220;delete when it&#8217;s full&#8221; are going to stomp all over the storage in a thin provisioning environment. Instead of being a smart idea to avoiding archive, in a thin provisioning environment this could very well leave storage administrators in a state of breathless consternation \u2013 and systems falling over left, right and centre.<\/p>\n<p>And so we come to the end of our data lifecycle discussion, at which point it&#8217;s worthwhile revisiting the diagram I used to introduce the lifecycle:<\/p>\n<p><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/Data-Lifecycle.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-2661\" title=\"Data Lifecycle\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/Data-Lifecycle.png\" alt=\"Data Lifecycle\" width=\"439\" height=\"348\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/Data-Lifecycle.png 439w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/Data-Lifecycle-300x237.png 300w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2011\/01\/Data-Lifecycle-378x300.png 378w\" sizes=\"auto, (max-width: 439px) 100vw, 439px\" \/><\/a><\/p>\n<p>Let me know when you&#8217;re all done with it and I&#8217;ll archive \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is the fifth and final part of our four part series &#8220;Data Lifecycle Management&#8221;. (By slipping in an aside&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[3,5,8,12,13,17],"tags":[119,138,274,278,279,305,308],"class_list":["post-2721","post","type-post","status-publish","format-standard","hentry","category-architecture","category-backup-theory","category-data-loss","category-general-technology","category-general-thoughts","category-policies","tag-archive","tag-backup","tag-data-deletion","tag-data-lifecycle","tag-data-lifecycle-management","tag-delete","tag-deletion"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pKpIN-HT","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/2721","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/comments?post=2721"}],"version-history":[{"count":1,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/2721\/revisions"}],"predecessor-version":[{"id":7532,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/2721\/revisions\/7532"}],"wp:attachment":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/media?parent=2721"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/categories?post=2721"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/tags?post=2721"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}