{"id":2481,"date":"2010-09-25T06:33:08","date_gmt":"2010-09-24T20:33:08","guid":{"rendered":"http:\/\/nsrd.info\/blog\/?p=2481"},"modified":"2010-09-25T06:33:08","modified_gmt":"2010-09-24T20:33:08","slug":"networker-7-6-sp1-2","status":"publish","type":"post","link":"https:\/\/nsrd.info\/blog\/2010\/09\/25\/networker-7-6-sp1-2\/","title":{"rendered":"NetWorker 7.6 SP1"},"content":{"rendered":"<h3>Introduction<\/h3>\n<p>I was lucky enough to get to work on the beta testing programme for NetWorker 7.6 SP1. While there are a bunch of new features in NW 7.6 SP1 (with the one most discussed by EMC being the Data Domain integration), I want to talk about three new features that I think are quite important, long term, in core functionality within the product for the average Joe.<\/p>\n<p>These are:<\/p>\n<ul>\n<li>Scheduled Cloning<\/li>\n<li>AFTD enhancements<\/li>\n<li>Checkpoint restarts<\/li>\n<\/ul>\n<p>Each of these on their own represents significant benefit to the average NetWorker site, and I&#8217;d like to spend some time discussing the functionality they bring.<\/p>\n<p>[Edit\/Aside: You can find the documentation for NetWorker 7.6 SP1 available in the <a title=\"nsrd.info Documentation\" href=\"https:\/\/nsrd.info\/docs.html\" target=\"_blank\">updated Documentation area on nsrd.info<\/a>]<\/p>\n<h3>Scheduled Cloning<\/h3>\n<p>In some senses, cloning has been the bane of the NetWorker administrator&#8217;s life. Up until NW 7.6 SP1, NetWorker has had two forms of cloning:<\/p>\n<ul>\n<li>Per-group cloning, immediately following completion of backups;<\/li>\n<li>Scripted cloning.<\/li>\n<\/ul>\n<p>A lot of sites use scripted cloning, simply due to the device\/media contention frequently caused in per-group cloning. I know this well; since starting working with NetWorker in 1996, I&#8217;ve written countless numbers of NetWorker cloning scripts, and currently am the primary programmer for IDATA Tools, which includes what I can only describe as a cloning utility on steroids (&#8216;sslocate&#8217;).<\/p>\n<p>Will those scripts and utilities go away with scheduled cloning? Well, I don&#8217;t think they&#8217;re always going to go away \u2013 but I do think that they&#8217;ll be treated more as utilities rather than core code for the average site, since scheduled cloning will be able to achieve much of the cloning requirements for companies.<\/p>\n<p>I had heard that scheduled cloning was on the way long before the 7.6 SP1 beta, thanks mainly to one day getting a cryptic email along the lines of &#8220;if we were to do scheduled cloning, what would you like to see in it&#8230;&#8221; \u2013 so it was pleasing, when it arrived, to see that much of my wish list had made it in there. As a first-round implementation of the process, it&#8217;s fabulous.<\/p>\n<p>So, let&#8217;s look at how we configure scheduled clones. First off, in NMC, you&#8217;ll notice a new menu item in the configuration section:<\/p>\n<p><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_menu.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-2483\" title=\"Scheduled Cloning Resource, in menu\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_menu.png\" alt=\"Scheduled Cloning Resource, in menu\" width=\"157\" height=\"328\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_menu.png 157w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_menu-143x300.png 143w\" sizes=\"auto, (max-width: 157px) 100vw, 157px\" \/><\/a><\/p>\n<p>This will undoubtedly bring joy to the heart of many a NetWorker administrator. If we then choose to create a new scheduled clone resource, we can create a highly refined schedule:<\/p>\n<p><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-2484\" title=\"Scheduled Clone Resource, 1 of 2\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_1.png\" alt=\"Scheduled Clone Resource, 1 of 2\" width=\"785\" height=\"781\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_1.png 785w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_1-150x150.png 150w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_1-300x298.png 300w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_1-301x300.png 301w\" sizes=\"auto, (max-width: 785px) 100vw, 785px\" \/><\/a><\/p>\n<p>Let&#8217;s look at those options first before moving onto the second tab:<\/p>\n<ul>\n<li><strong>Name<\/strong> and <strong>comment<\/strong> is pretty self explanatory \u2013 nothing to see there.<\/li>\n<li><strong>Browse<\/strong> and <strong>retention<\/strong> \u2013 define, for the clone schedule, both the browse and retention time of the savesets that will be cloned.<\/li>\n<li><strong>Start time<\/strong> \u2013 Specify exactly what time the cloning is to start.<\/li>\n<li><strong>Schedule period<\/strong> \u2013 Weekly allows you to specify which days of the week the cloning is to run. Monthly allows you to specify which dates of the month the cloning will run.<\/li>\n<li><strong>Storage node<\/strong> \u2013 Allows you to specify to which storage node the clone will write to. Great for situations where you have say, an off-site storage node and you want the data streamed directly across to it.<\/li>\n<li><strong>Clone pool<\/strong> \u2013 Which pool you want to write the clones to &#8211; fairly expected.<\/li>\n<li><strong>Continue on save set error<\/strong> \u2013 This is a big help. Standard scripting of cloning will fail if one of the savesets selected to clone has an error (regardless of whether that&#8217;s a read error, or it disappears (e.g., is staged off) before it gets cloned, etc.) and you haven&#8217;t used the &#8216;-F&#8217; option. Click this check box and the cloning will at least continue and finish all savesets it can hit in one session.<\/li>\n<li><strong>Limit number of save set clones<\/strong> \u2013 By default this is 1, meaning NetWorker won&#8217;t create more than one copy of the saveset in the target pool. This can be increased to a higher number if you want multiple clones, or it can be set to zero (for unlimited), which I wouldn&#8217;t see many sites having a need for.<\/li>\n<\/ul>\n<p>Once you&#8217;ve got the basics of how often and when the scheduled clone runs, etc., you can move on to selecting <em>what<\/em> you want cloned:<\/p>\n<p><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_1.png\"><\/a><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_2.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-2485\" title=\"Scheduled Clone Resource, 2 of 2\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_2.png\" alt=\"Scheduled Clone Resource, 2 of 2\" width=\"785\" height=\"781\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_2.png 785w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_2-150x150.png 150w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_2-300x298.png 300w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/sch_clone_2-301x300.png 301w\" sizes=\"auto, (max-width: 785px) 100vw, 785px\" \/><\/a><\/p>\n<p>I&#8217;ve only just refreshed my lab server, so you can see that a bit of imagination is required in the above screen shot to flesh out what this may look in a normal site. But, you can choose savesets to clone based on:<\/p>\n<ul>\n<li>Group<\/li>\n<li>Client<\/li>\n<li>Source Pool<\/li>\n<li>Level<\/li>\n<li>Name<\/li>\n<\/ul>\n<p><em>or<\/em><\/p>\n<ul>\n<li>Specific saveset ID\/clone IDs<\/li>\n<\/ul>\n<p>When specifying savesets based on group\/client\/level\/etc., you can also specify how far back NetWorker is to look for savesets to clone. This avoids a situation whereby you might say, enable scheduled cloning and suddenly have media from 3 years ago requested.<\/p>\n<p>You might wonder about the practicality of being able to schedule a clone for specific SSID\/CloneID combos. I can imagine this would be particularly useful if you need to do ad-hoc cloning of a particularly large saveset. E.g., if you&#8217;ve got a saveset that&#8217;s say, 10TB, you might want to configure a schedule that would start specifically cloning this at 1am Saturday morning, with your intent being to then delete the scheduled clone after it&#8217;s finished. In other words, it&#8217;s to replace having to do a scheduled cron or at job just for a single clone.<\/p>\n<p>Once configured, and enabled, scheduled cloning runs like a dream. In fact, it was one of the first things I tackled while doing beta testing, and almost every subsequent day found myself thinking at 3pm &#8220;why is my lab server suddenly cloning? \u2013 ah yes, that&#8217;s why&#8230;&#8221;<\/p>\n<h3>AFTD Enhancements<\/h3>\n<p>There&#8217;s not a huge amount to cover in terms of AFTD enhancements \u2013 they&#8217;re effectively exactly the same enhancements that have been run into NetWorker 7.5 SP3, which I&#8217;ve <a title=\"First Thoughts - 7.5 SP3 AFTD Enhancements\" href=\"https:\/\/nsrd.info\/blog\/2010\/08\/23\/first-thoughts-\u2013%C2%A0networker-7-5-sp3-aftd-enhancements\/\" target=\"_blank\">previously covered here<\/a>. So, that means there&#8217;s a better volume selection criteria for AFTD backups, but we don&#8217;t yet have backups being able to continue from one AFTD device to another. (That&#8217;s still in the pipeline and being actively worked on, so it will come.)<\/p>\n<p>Even this one key change \u2013 the way in which volumes are picked in AFTDs for new backups \u2013 will be a big boon for a lot of NetWorker sites. It will allow administrators to not focus so much on the &#8220;AFTD data shuffle&#8221;, as I like to consider it, and instead focus on higher level administration of the backup environment.<\/p>\n<p>(These changes are effectively &#8220;under the hood&#8221;, so there&#8217;s not much I can show in the way of screen-shots.)<\/p>\n<h3>Checkpoint Restarting<\/h3>\n<p>When I first learned NetBackup, I immediately saw the usefulness of checkpoint restarting, and have been eager to see it appear in NetWorker since that point. I&#8217;m glad to say it&#8217;s appeared in (what I consider to be) a much more usable form. So what is checkpoint restarting? If you&#8217;re not familiar with the term, it&#8217;s where the backup product has regular points at which it can restart from, rather than having to restart an entire backup. Previously NetWorker has only done this at the saveset level, but that&#8217;s not really what the average administrator would think of when &#8216;checkpoints&#8217; are discussed. NetBackup, last I looked at it, does this at periodic intervals &#8211; e.g., every 15 minutes or so.<\/p>\n<p>Like almost everything in NetWorker, we get more than one way to run a checkpoint:<\/p>\n<p><a href=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/checkpoint.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-2486\" title=\"Checkpoint restart options\" src=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/checkpoint.png\" alt=\"Checkpoint restart options\" width=\"854\" height=\"738\" srcset=\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/checkpoint.png 854w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/checkpoint-300x259.png 300w, https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2010\/09\/checkpoint-347x300.png 347w\" sizes=\"auto, (max-width: 854px) 100vw, 854px\" \/><\/a><\/p>\n<p>Within any <em>single client instance<\/em> you can choose to enable checkpoint restarting, with the restart options being:<\/p>\n<ul>\n<li><strong>Directory<\/strong> \u2013 If a backup failure occurs, restart from the last <em>directory<\/em> that NetWorker had started processing.<\/li>\n<li><strong>File<\/strong> \u2013 If a backup failure occurs, restart from the last <em>file<\/em> NetWorker had started processing.<\/li>\n<\/ul>\n<p>Now, the documentation warns that with checkpoint enabled, you&#8217;ll get a slight performance hit on the backup process. However, that performance hit is nothing compared to the performance and potentially media hit you&#8217;d take if you&#8217;re 9.8TB through a 10TB saveset and the client is accidentally rebooted!<\/p>\n<p>Furthermore, in my testing (which admittedly focused on savesets smaller than 10GB), I inevitably found that with either file or directory level checkpointing enabled, the backup actually ran <em>faster<\/em> than the normal backup. So maybe it&#8217;s also based on the hardware you&#8217;re running on, or maybe that performance hit doesn&#8217;t come in until you&#8217;re backing up millions of files, but either way, I&#8217;m not prepared to say it&#8217;s going to be a huge performance hit for anyone yet.<\/p>\n<p>Note what I said earlier though \u2013 this can be configured on a single client <em>instance<\/em>. This lets you configure checkpoint restarting even on the individual client level to suit the data structure. For example, let&#8217;s consider a fileserver that offers both departmental\/home directories, and research areas:<\/p>\n<ul>\n<li>The departmental\/home directories will have thousands and thousands of directories \u2013 have a client instance for this area, set for directory level checkpointing.<\/li>\n<li>The research area might feature files that are hundreds of gigabytes each \u2013 use file level checkpointing here.<\/li>\n<\/ul>\n<p>When I&#8217;d previously done a blog entry wishing for checkpoint restarts (&#8220;<a title=\"Sub-saveset checkpointing would be good\" href=\"https:\/\/nsrd.info\/blog\/2009\/07\/29\/sub-saveset-checkpointing-would-be-good\/\" target=\"_blank\">Sub saveset checkpointing would be good<\/a>&#8220;), I&#8217;d envisaged the checkpointing being done via the continuation savesets \u2013 e.g., &#8220;C:&#8221;, &#8220;&lt;1&gt;C:&#8221;, &#8220;&lt;2&gt;C:&#8221;, etc. It hasn&#8217;t been implemented this way; instead, each time the saveset is restarted, a new saveset is generated of the same level, catering to whatever got backed up during that saveset. On reflection, I&#8217;m not the slightest bit hung up over <em>how<\/em> it&#8217;s been implemented, I&#8217;m just overjoyed to see that it <em>has<\/em> been implemented.<\/p>\n<p>Now you&#8217;re probably wondering \u2013 does the checkpointing work? Does it create any headaches for recoveries? Yes, and no. As in, yes it works, and no, throughout all my testing, I wasn&#8217;t able to create any headaches in the recovery process. I would feel very safe about having checkpoint restarts enabled on production filesystems right now.<\/p>\n<h3>Bonus: Mac OS X 10.6 (&#8220;Snow Leopard&#8221;) Support<\/h3>\n<p>For some time, NetWorker has had some issues supporting Mac OS X 10.6, and it&#8217;s certainly caused some problems for various customers as this operating system continues to get market share in the enterprise. I was particularly pleased during a beta refresh to see an updated Mac OS X client for NetWorker. This works excellently for backup, recovery, installation, uninstallation, etc. I&#8217;d suggest on the basis of the testing I did that any site with OS X should immediately upgrade those clients at least to 7.6 SP1.<\/p>\n<h3>In Summary<\/h3>\n<p>The only major glaring question for me, looking at NetWorker 7.6 SP1 is the obvious: this has so many updates, and so many new features, way more than we&#8217;d see in a normal service pack \u2013 why the heck isn&#8217;t it NetWorker v8?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction I was lucky enough to get to work on the beta testing programme for NetWorker 7.6 SP1. While there&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[16],"tags":[79,80,102,104,210,211,226,228,1249,865],"class_list":["post-2481","post","type-post","status-publish","format-standard","hentry","category-networker","tag-7-6","tag-7-6-sp1","tag-adv_file","tag-aftd","tag-checkpoint","tag-checkpoint-restart","tag-clone","tag-cloning","tag-networker","tag-scheduled-clone"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pKpIN-E1","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/2481","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/comments?post=2481"}],"version-history":[{"count":0,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/2481\/revisions"}],"wp:attachment":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/media?parent=2481"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/categories?post=2481"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/tags?post=2481"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}