{"id":5724,"date":"2015-10-26T06:22:28","date_gmt":"2015-10-25T20:22:28","guid":{"rendered":"http:\/\/nsrd.info\/blog\/?p=5724"},"modified":"2018-12-11T12:01:47","modified_gmt":"2018-12-11T02:01:47","slug":"linux-block-based-backups-in-networker-9","status":"publish","type":"post","link":"https:\/\/nsrd.info\/blog\/2015\/10\/26\/linux-block-based-backups-in-networker-9\/","title":{"rendered":"Linux Block Based Backups in NetWorker 9"},"content":{"rendered":"<p>As mentioned <a href=\"https:\/\/nsrd.info\/blog\/2015\/10\/02\/networker-9-the-future-of-backup\/\" target=\"_blank\">in my&nbsp;introductory post about it<\/a>, NetWorker 9 introduces&nbsp;the option to perform Block Based Backups (BBB) for Linux systems. (This was introduced in NetWorker&nbsp;8&nbsp;for Windows, and has actually had its functionality extended for Windows in v9 as well, with the option to now perform BBB for Hyper-V and Exchange&nbsp;systems.)<\/p>\n<p>BBB is a highly efficient mechanism for&nbsp;backing up without worrying about the&nbsp;cost of walking the filesystem. Years ago I showed just how much&nbsp;<a href=\"https:\/\/nsrd.info\/blog\/2009\/06\/17\/in-lab-review-of-the-impact-of-dense-filesystems\/\" target=\"_blank\">filesystem density can have a massive detrimental&nbsp;impact<\/a> on the performance of a backup. While often the backup product is blamed for being &#8220;slow&#8221;, the fault&nbsp;sits&nbsp;completely with operating system and filesystem vendors for&nbsp;having not produced&nbsp;structures that scale sufficiently.<\/p>\n<p>BBB gets us past that problem by side-stepping the filesystem and reading directly from the underlying disk or LUN. Instead of walking files, we just have to traverse the blocks.&nbsp;In cases where filesystems are&nbsp;<em>really<\/em> dense, the cost of&nbsp;walking the filesystem can increase the run-time of the backup by an order of magnitude or more. Taking that out of the picture allows businesses to protect these filesystems&nbsp;much faster than via conventional means.<\/p>\n<p>Since BBB needs to integrate at a reasonably low level within&nbsp;a system&nbsp;structure in order&nbsp;to successfully operate,&nbsp;NetWorker currently supports only the following systems:<\/p>\n<ul>\n<li>CentOS 7<\/li>\n<li>RedHat Enterprise Linux v5 and higher<\/li>\n<li>SLES Linux 11 SP1 and higher<\/li>\n<\/ul>\n<p>In all cases, you need to be running LVM2 or Veritas Volume Manager (VxVM),&nbsp;<em>and<\/em> be using ext3 or ext4 filesystems.<\/p>\n<p>To demonstrate the benefits of BBB in Linux, I&#8217;ve setup a test&nbsp;SLES 11 host and used my&nbsp;<em><a href=\"https:\/\/nsrd.info\/blog\/2015\/05\/01\/files-and-files-and-files\/\" target=\"_blank\">genfs2<\/a><\/em> utility on it to generate a really (nastily) dense filesystem.&nbsp;I actually aborted the utility when I had 1,000,000+ files on the filesystem \u2013 consuming just 11GB of space:<\/p>\n<figure id=\"attachment_5725\" aria-describedby=\"caption-attachment-5725\" style=\"width: 514px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/genfs2-run.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5725\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/genfs2-run.png\" alt=\"genfs2 run\" width=\"514\" height=\"738\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/genfs2-run.png 514w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/genfs2-run-209x300.png 209w\" sizes=\"auto, (max-width: 514px) 100vw, 514px\" \/><\/a><figcaption id=\"caption-attachment-5725\" class=\"wp-caption-text\">genfs2 run<\/figcaption><\/figure>\n<p>I then configured a client resource and policy\/workflow to do a&nbsp;<em>conventional<\/em> backup of the&nbsp;\/testfs filesystem. That&#8217;s without any form of performance&nbsp;enhancement. From NetWorker&#8217;s perspective, this resulted in about 8.5GB of backup,&nbsp;and with 1,178,358 files (and directories) total took 36 minutes and 37 seconds to backup. (That&#8217;s actually not too bad, all things considered \u2013 but my lab environment was pretty much quiesced other than the test components.)<\/p>\n<figure id=\"attachment_5726\" aria-describedby=\"caption-attachment-5726\" style=\"width: 624px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Conventional-Backup.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5726\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Conventional-Backup.png\" alt=\"Conventional Backup Performance\" width=\"624\" height=\"366\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Conventional-Backup.png 624w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Conventional-Backup-300x176.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/a><figcaption id=\"caption-attachment-5726\" class=\"wp-caption-text\">Conventional Backup Performance<\/figcaption><\/figure>\n<p>Next, I switched over to parallel savestreams \u2013 which has become more&nbsp;capable in NetWorker 9 given NetWorker will now dynamically rebalance remaining backups all the way through to the end of the backup. (Previously the split was effectively static, meaning you could have just one or two savestreams left running by themselves&nbsp;after others had completed. I&#8217;ll cover dynamic parallel savestreams in more detail in a later post.)<\/p>\n<p>With&nbsp;dynamic parallel savestreams in play, the backup&nbsp;time dropped by <em>over<\/em> ten minutes \u2013 a total runtime of 23 minutes and 46 seconds:<\/p>\n<figure id=\"attachment_5728\" aria-describedby=\"caption-attachment-5728\" style=\"width: 774px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Dynamic-Parallel-Savestreams.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5728\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Dynamic-Parallel-Savestreams.png\" alt=\"Dynamic Parallel Savestream Runtime\" width=\"774\" height=\"738\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Dynamic-Parallel-Savestreams.png 774w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Dynamic-Parallel-Savestreams-300x286.png 300w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Dynamic-Parallel-Savestreams-24x24.png 24w\" sizes=\"auto, (max-width: 774px) 100vw, 774px\" \/><\/a><figcaption id=\"caption-attachment-5728\" class=\"wp-caption-text\">Dynamic Parallel Savestream Runtime<\/figcaption><\/figure>\n<p>The next test, of course, involves enabling BBB for the backup. So long as you&#8217;ve met the&nbsp;compatibility requirements, this is&nbsp;just a trivial checkbox selection:<\/p>\n<figure id=\"attachment_5730\" aria-describedby=\"caption-attachment-5730\" style=\"width: 790px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Enabling-Block-Based-Backup.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5730\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Enabling-Block-Based-Backup.png\" alt=\"Enabling Block Based Backup\" width=\"790\" height=\"276\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Enabling-Block-Based-Backup.png 790w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Enabling-Block-Based-Backup-300x105.png 300w\" sizes=\"auto, (max-width: 790px) 100vw, 790px\" \/><\/a><figcaption id=\"caption-attachment-5730\" class=\"wp-caption-text\">Enabling Block Based Backup<\/figcaption><\/figure>\n<p>With BBB enabled&nbsp;the workflow executed in just <strong>6 minutes and 48 seconds<\/strong>:<\/p>\n<figure id=\"attachment_5731\" aria-describedby=\"caption-attachment-5731\" style=\"width: 775px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Block-Based-Backup.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5731\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Block-Based-Backup.png\" alt=\"Block Based Backup Performance\" width=\"775\" height=\"651\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Block-Based-Backup.png 775w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Block-Based-Backup-300x252.png 300w\" sizes=\"auto, (max-width: 775px) 100vw, 775px\" \/><\/a><figcaption id=\"caption-attachment-5731\" class=\"wp-caption-text\">Block Based Backup Performance<\/figcaption><\/figure>\n<p>That&#8217;s a&nbsp;<em>substantially<\/em> shorter runtime \u2013 the backups have dropped from over&nbsp;36 minutes for a single savestream to under 7 minutes using&nbsp;BBB and bypassing the filesystem.&nbsp;While Dynamic&nbsp;Parallel Savestreams did make a substantial difference (shaving almost a third from&nbsp;the backup time), BBB&nbsp;was the undisputed&nbsp;winner for maximising&nbsp;backup performance.<\/p>\n<p>One final&nbsp;point \u2013 if you&#8217;re doing BBB to Data Domain, NetWorker now automatically executes a&nbsp;synthetic full (using the Data Domain virtual synthetic full functionality) at the end of every incremental backup&nbsp;BBB you perform:<\/p>\n<figure id=\"attachment_5732\" aria-describedby=\"caption-attachment-5732\" style=\"width: 816px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Synth-Full.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5732\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Synth-Full.png\" alt=\"Automatic virtual synthetic full\" width=\"816\" height=\"661\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Synth-Full.png 816w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/Synth-Full-300x243.png 300w\" sizes=\"auto, (max-width: 816px) 100vw, 816px\" \/><\/a><figcaption id=\"caption-attachment-5732\" class=\"wp-caption-text\">Automatic virtual synthetic full<\/figcaption><\/figure>\n<p>The advantage of this is that recovery from BBB is trivial \u2013 just point your recovery process (either command line, or via NMC) at the date you want to recover from, and you have visibility of the entire&nbsp;filesystem at that time.&nbsp;If you&#8217;re wondering what FLR from BBB looks like on Linux, by the way,&nbsp;it&#8217;s pretty straight forward. Once you identify the saveset (based on date \u2013 remember, it&#8217;ll contain everything), you can just fire up the recovery utility and get:<\/p>\n<figure id=\"attachment_5733\" aria-describedby=\"caption-attachment-5733\" style=\"width: 815px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/BBB-FLR.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5733\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/BBB-FLR.png\" alt=\"BBB FLR\" width=\"815\" height=\"201\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/BBB-FLR.png 815w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/BBB-FLR-300x74.png 300w\" sizes=\"auto, (max-width: 815px) 100vw, 815px\" \/><\/a><figcaption id=\"caption-attachment-5733\" class=\"wp-caption-text\">BBB FLR<\/figcaption><\/figure>\n<p>Logging in using another terminal session, it&#8217;s just a simple case of browsing to the directory indicated above and copying the files\/data you want:<\/p>\n<figure id=\"attachment_5734\" aria-describedby=\"caption-attachment-5734\" style=\"width: 496px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/BBB-FLR-directory-listing.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-5734\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/BBB-FLR-directory-listing.png\" alt=\"BBB FLR directory listing\" width=\"496\" height=\"347\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/BBB-FLR-directory-listing.png 496w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/10\/BBB-FLR-directory-listing-300x210.png 300w\" sizes=\"auto, (max-width: 496px) 100vw, 496px\" \/><\/a><figcaption id=\"caption-attachment-5734\" class=\"wp-caption-text\">BBB FLR directory listing<\/figcaption><\/figure>\n<p>And there you have it. If&nbsp;you&#8217;ve got highly dense Linux filesystems, you might want to give serious thought towards upgrading to NetWorker 9 so you can significantly increase the&nbsp;performance of their backup. NetWorker + Linux + BBB is a winning combination.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>As mentioned in my&nbsp;introductory post about it, NetWorker 9 introduces&nbsp;the option to perform Block Based Backups (BBB) for Linux systems.&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[15,16],"tags":[138,1271,1272,206,376,1273,1248,732,1274,898],"class_list":["post-5724","post","type-post","status-publish","format-standard","hentry","category-linux","category-networker","tag-backup","tag-bbb","tag-block-based-backup","tag-centos","tag-ext3","tag-ext4","tag-linux","tag-performance","tag-rhel","tag-sles"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pKpIN-1uk","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/5724","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/comments?post=5724"}],"version-history":[{"count":5,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/5724\/revisions"}],"predecessor-version":[{"id":7423,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/5724\/revisions\/7423"}],"wp:attachment":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/media?parent=5724"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/categories?post=5724"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/tags?post=5724"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}