{"id":525,"date":"2009-06-11T21:15:31","date_gmt":"2009-06-11T11:15:31","guid":{"rendered":"http:\/\/nsrd.wordpress.com\/?p=525"},"modified":"2009-06-11T21:15:31","modified_gmt":"2009-06-11T11:15:31","slug":"hsm-implications-for-backup","status":"publish","type":"post","link":"https:\/\/nsrd.info\/blog\/2009\/06\/11\/hsm-implications-for-backup\/","title":{"rendered":"HSM implications for backup"},"content":{"rendered":"<p>If you&#8217;ve been following this blog for a while, you&#8217;ll know that one key ongoing performance issue I refer to is that created by costs associated with <em>walking<\/em> dense filesystems as part of backups.<\/p>\n<p>One area that people sometimes don&#8217;t take into consideration is the implications of backing up filesystems that use HSM &#8211; Hierarchical Storage Management. In a HSM environment, files are migrated from primary to secondary (or even tertiary) storage based on age and access times. In order to make this seamless to the user, a small stub file with the same name is left behind on the filesystem. Therefore if a user attempts to access the file, they trigger a read from HSM storage.<\/p>\n<p>So, in order to free up space (for more storage) on primary disk, big files are migrated, with tiny files being left behind. Over time, more big files are removed, and more tiny files left behind. You may understand where I&#8217;m heading now: this high number of little files can result in performance issues for the backup. Obviously, HSM systems are configured so that they recognise backup agents, and the stub is backed up rather than the original file being pulled back, so we&#8217;re not concerned about say, backing up 4TB for a 1TB filesystem with HSM; instead, our concern is that the cost of walking a big filesystem with an inordinately large number of small files will seriously impede the backup process.<\/p>\n<p>If you&#8217;re planning HSM, think <em>very carefully<\/em> about how you&#8217;re going to backup the resulting filesystem.<\/p>\n<p>(Coming soon: demonstrations of the impact of dense filesystems on backup performance.)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you&#8217;ve been following this blog for a while, you&#8217;ll know that one key ongoing performance issue I refer to&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[5,16],"tags":[442,732],"class_list":["post-525","post","type-post","status-publish","format-standard","hentry","category-backup-theory","category-networker","tag-hsm","tag-performance"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pKpIN-8t","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/525","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/comments?post=525"}],"version-history":[{"count":0,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/525\/revisions"}],"wp:attachment":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/media?parent=525"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/categories?post=525"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/tags?post=525"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}