NetWorker 9 Converted Policy Workflow Visualisation<\/figcaption><\/figure>\n(By the way, that “Monthly Filesystem” workflow cloning to the “Default Clone” pool was just a lazy error on my part while setting up a test server – not an error.)<\/p>\n
I know lots of people tested some fairly hairy configuration migrations. If I recall correctly the biggest configuration I tested had over 1000 clients defined and around 300 groups, schedules, etc., associated with those clients. And<\/em> I did a whole bunch of shortcuts and tricks in schedules and<\/em> they converted successfully.<\/p>\nThe back-end changes<\/h2>\n
I’ll undoubtedly do some additional blog articles about the NetWorker 9 policy engine, but it’s time to move on to other topics and other changes within NetWorker. I’ll start with some back-end changes to the environment.<\/p>\n
Media database<\/h3>\n
The “WISS” database format has been around for as long as I can recall with NetWorker. It’s served NetWorker well, but it’s also had some limitations. As of version 9, the NetWorker media database format is now SQLite, which gives NetWorker a big boost for performance and parallelisation of media activities. As per the policy engine, this migration happens automatically for you as part of the upgrade process. (Depending on the size of your media database this may take a little while to complete, but the media database is usually fairly small for most organisations.)<\/p>\n
NetWorker Management Console (NMC) Database<\/h3>\n
Previous versions of NetWorker have used the Sybase embedded SQLAnywhere database for NMC. NetWorker version 9 switches the NMC database to PostgreSQL. If you’re wanting to keep your existing NMC database, you’ll need to take some pre-ugprade steps to export the Sybase embedded database content into a format that can be imported into the PostgreSQL database. Be sure to read the upgrade documentation \u2013 but you were going to do that anyway, right?<\/p>\n
License Server<\/h3>\n
Other than the options around traditional vs NetWorker capacity vs DPS capacity, NetWorker licensing has remained mostly the same for the entire 19 years I’ve been dealing with it. There was a Legato License Manager introduced some time ago but it had mainly been pushed as a means of centralising management of traditional licensing across multiple datazones. Since the capacity formats aren’t so bothered on datazone counts, LLM usage has fallen away.<\/p>\n
With a lot of customers deploying multiple EMC products and EMC moving towards transformative enterprise licensing models, a move to a new licensing service that can handle licensing for multiple products makes sense. From a day to day basis, the licensing server won’t really change how you interact with NetWorker, but you’ll want to deal with your sales\/pre-sales team or your integrator (depending on which way you procure NetWorker licenses) in order to prep for the license changes. It’s not a change to functionality of traditional vs capacity licenses, and it doesn’t signal a move away from traditional licenses either, but it is a much needed change.<\/p>\n
Authentication System<\/h3>\n
NetWorker has by and large used OS provided user-authentication for authorisation. That might be localised on a per-system basis or it might leverage Active Directory\/etc. This however left somewhat of a split between authorisation supported by NetWorker Management Console and authorisation supported from the command line. The new authentication system is effectively a single sign-on approach providing integrated authentication between NMC related activities and<\/em> command line activities.<\/p>\nRestricted Data Zones<\/h3>\n
Restricted datazones get a few tweaks with NetWorker 9, too. I’ve had very little direct cause to use RDZs myself, so I’ll let the release notes speak for themselves on this front:<\/p>\n
\n- \n
You can now associate an RDZ resource to an individual resource (for example, to a client, protection policy, protection group, and so on) from the resource itself. As a result, RDZ resources can no longer effect resource associations directly.<\/p><\/blockquote>\n<\/li>\n
- \n
Non-default resources, that are previously associated to the global zone and therefore unusable by an RDZ, are now shared resources that can be used by an RDZ. Although, these resources cannot be modified by restricted administrators.<\/p><\/blockquote>\n<\/li>\n<\/ul>\n
If you’re using RDZs in your environment, be sure to understand the implications of the above changes as part of the upgrade process.<\/p>\n
Scaling<\/h3>\n
With a raft of under-the-hood changes and enhancements, NetWorker servers \u2013 already highly scaleable \u2013 become even more scaleable. If your NetWorker environment has been getting large enough that you’ve considered deploying additional datazones, now is the time to talk to your local EMC teams to see whether you still need to go down that path. (Chances are you don’t.)<\/p>\n
NetWorker Server Platform<\/h3>\n
There are actually very few environments left where the NetWorker server itself runs on what I’d refer to as “classic” Unix systems \u2013 i.e., Solaris, HPUX or AIX. As of NetWorker 9, the NetWorker server processes (and similarly, NMC processes) will now run only on Windows 64-bit or Linux 64-bit systems. This allows a concentration of development, leveraging the substantially (I’d say massively<\/em>) reduced use of these platforms for better development efficiencies. However, NetWorker client support is still extremely healthy and those platforms are also still fully supported as storage nodes.<\/p>\nFrom a migration perspective, this is actually relatively easy to handle. EMC for some time has supported cross platform migration<\/em>, wherein the NetWorker media database, configuration and index (i.e., the NetWorker server) is moved from say, Linux to Windows, Solaris to Linux, Solaris to Windows, etc. If you are one of those sites still using the NetWorker server<\/em> services on Solaris, HPUX or AIX, you can engage cross platform migration services and transfer across to Windows or Linux. To keep things simple (I’ve done this dozens of times myself over the years), consider even keeping the old server around, renaming it and turning it into a storage node so you don’t really have to change any device connectivity. Then, elevate the backup server to a “director only” mode where it’s not actually doing any client backup itself. All up, this sort of transition can be seamlessly achieved in a very short period of time. In short: it may be a small interruption and change to your processes, but having executed it many times myself in the past, I can honestly say it’s a very small change in the grand scheme of things, and very manageable.<\/p>\nIn summary, the options along this front if you’re using a non-Windows\/non-Linux NetWorker server are:<\/p>\n
\n- Do a platform migration of your NetWorker server to Windows or Linux using your current NetWorker version, then<\/em> upgrade to the new version<\/li>\n
- Stand up a new NetWorker datazone on Windows or Linux and retain the existing one for legacy recoveries, migrating clients across<\/li>\n<\/ul>\n
I’m actually a big fan of the former rather than the latter \u2013 I really have done enough platform migrations to know they work well and they allow you to retain everything<\/em> you’ve been doing. (IMHO the only reason to not<\/em> do a platform migration is if you have a very short retention period for all<\/em> of your backups and you want to start with a brand new configuration approach.)<\/p>\n(Cross platform migrations do<\/em> have to be done by an authorised party \u2013 if you’re not sure who near you can do cross platform migrations, reach out to your local EMC team and find out.)<\/p>\nOne more thing: with the additional services now running on a NetWorker server, you could need more RAM\/CPU in your server. Check out the release notes for some details on this front. Environments that have been sized with room for spare likely won’t need to worry about this at all \u2013 but if you’ve got an environment where you’ve got an older piece of hardware running as your NetWorker server, you might need to increase its performance characteristics a little.<\/p>\n
[Clarifying point: I’m only talking about the NetWorker server<\/strong> platform. Traditional Unix systems remain fully supported for storage nodes and clients.]<\/em><\/p>\nCloning<\/h3>\n
NetWorker gets a performance and optimisation boost with cloning. Cloning has previously been a reasonably isolated process compared to regular save or recovery operations. With NetWorker 9, cloning is now a more integrated function, leveraging the in-place recovery technology implemented in NetWorker 8.2 to speed up cloning of synthetic backups.<\/p>\n
This has some advantages relating to parallelising clones and limiting the need for additional nsrmmd processes to handle the cloning operation, and<\/em> introduces scope for exciting changes in future versions of NetWorker, too.<\/p>\nWith continuing advances in how you can configure and manage cloning from within NetWorker policies, manual command line driven cloning is becoming less necessary, but if you do still use it you’ll notice some difference in the output. For instance:<\/p>\n
[root@sirius ~]# mminfo -q \"name=\/usr,savetime>=24 hours ago\" -r ssid<\/strong>\n4278951844\n[root@sirius ~]# nsrclone -b \"Site-A Clone\" -S 4278951844<\/strong>\n140988:nsrclone: launching backend job on host sirius.turbamentis.int\n140990:nsrclone: Backend started: job Id(160004).\n85401:nsrrecopy: Input client or saveset is NULL, information not updated in jobdb\n09\/30\/15 18:48:04.652904 Clone pool size used:4\n09\/30\/15 18:48:04.756405 Init Clone PARAMS: Network constant(73400320) Saveset computation overhead(2000000 microsec) Threshold(600000000 microsec) MIN-Threads(16) MAX-Threads(32)\n09\/30\/15 18:48:04.757495 Adjust Clone param: Total overhead(50541397 microsec) Threshold(12635349 microsec) MIN-threads(1) MAX-Threads(4)\n09\/30\/15 18:48:04.757523 Add New saveset group(0x0x3fe5db0): Group overhead(50541397 microsec) Num ss(1)\n129290:nsrrecopy: Successfully established direct file retrieve session for save-set ID '4278951844' with adv_file volume 'Daily.001'.\n09\/30\/15 18:49:30.765647 nsrrecopy exiting\n140991:nsrclone: Backend exited: job Id(160004).\n [ORIGINAL REQUESTED SAVESETS]\n4278951844;\n [CLONE SUCCESS SAVESETS]\n4278951844\/1443603606;<\/pre>\nNote that while the command line output<\/em> is a little difference, the command line options remain the same so your scripts can continue to work without change there. However, with enhanced support for concurrent cloning operations you’ll likely be able to speed up those scripts … or replace them entirely with new policies.<\/p>\nPerformance tuners win too<\/h3>\n
The performance tuning and optimisation guide has been getting more detailed information over more recent versions, and the one that accompanies NetWorker 9 is no exception. For example, there’s an entire new section on TCP window size and network latency considerations<\/em> that a bunch of examples (and graphs) relating to the impact of latency on backup and cloning operations of varying sizes based on filesystem density. If you’re someone who likes to see what tuning and adjustment options there are in NetWorker, you’ll definitely want to peruse the new Performance Tuning\/Optimisation guide, available with the rest of the reference documentation.<\/p>\n(On that front, NDMP has now been broken out into its own document: the NDMP User Guide<\/em>. Keep an eye on it if you’re working with NAS systems.)<\/p>\nAdditional Features<\/h2>\nBlock Based Backup (BBB) for Linux<\/h3>\n
Several Linux operating systems and filesystems now get the option of performing block based backups. This can significantly speed up the backup of large\/dense filesystems \u2013 even more so than parallel save streams \u2013 by actually bypassing the filesystem entirely. It’s been available in Windows backups for a while now, but it’s hopped over the fence to Linux as well. Like the Windows variant, BBB doesn’t require image level recovery \u2013 you can do file level recovery from block based backups. If you’ve got really dense filesystems (I’m looking at large scale IMAP servers as a classic example), BBB could increase your backup performance by up to an order of magnitude.<\/p>\n
Parallel Save Streams<\/h3>\n
Parallel Save Streams certainly aren’t forgotten about in NetWorker 9. There are now options to go beyond 4 parallel save streams per saveset for PSS enabled clients, and we’ve seen the introduction of automatic stream reclaiming<\/em>, which will dynamically increase the number of active streams for a saveset already running in PSS mode to maximise the utilisation of client parallelism settings. (That’s a mouthful. The short: PSS is more intelligent and more reactive to fluctuations in used parallelism on clients.)<\/p>\nProtectPoint<\/h3>\n
ProtectPoint is a pretty exciting new technology being rolled out by EMC across its storage arrays and integrates with Data Domain for the back-end storage. To understand what ProtectPoint does, consider a situation where you’ve got say, a 100TB Oracle database sitting on a VMAX3 system, and you need to back it up as fast as possible with as little an impact to the actual database server itself as possible. In conventional agent-based backups, it doesn’t matter what tricks and techniques you use to mitigate the amount of data flowing from the Oracle server to the backup environment, the Oracle server still has to read the data from the storage system. ProtectPoint is an application aware and application\/integrated system that allows you to seamlessly have the storage array and the Data Domain handle pretty much the entire backup, with the data transfer going directly from the storage array to the Data Domain. Suddenly that entire-database server read load associated with a conventional backup disappears.<\/p>\n
NetWorker v9 integrates management of ProtectPoint policies in a very similar way to how NetWorker v8.2 introduced highly advanced NAS snapshot service integration into the data protection management. This further grows NetWorker’s capabilities in orchestrating the overall data protection process in your environment.<\/p>\n
(There’s a good overview demo of ProtectPoint over at YouTube<\/a>.)<\/p>\nNVE<\/h3>\n
Some people want to be able to stand up and completely control a NetWorker environment themselves, and others want to be able to deploy an appliance, answer a couple of questions, and have a fully functioning backup environment ready for use. NetWorker Virtual Edition (NVE) addresses the needs and desires of the latter. For service providers or businesses deploying remote office protection solutions, NVE will be a boon \u2013 and it won’t eat into any operating system licensing costs, as the OS (Linux) is bundled with the virtual machine template file.<\/p>\n
Base vs Extended Client Installers<\/h3>\n
For Unix systems, NetWorker now splits out the client package into two separate installers \u2013 the base version and the extended version \u2013 lgtoclnt<\/em> and lgtoxtdclnt<\/em> respectively. You install the base client on clients that need to get fairly standard filesystem backups. It doesn’t include binaries like mminfo, nsrwatch<\/em> or nsradmin <\/em>\u2013 they’re now in the extended package. This allows you to keep regular client installs streamlined \u2013 particularly useful if you’re a service provider or dealing with larger environments.<\/p>\nVBA<\/h2>\n
There’s been a variety of changes made to the Virtual Backup Appliance (introduced in NetWorker 8.1), but the two I want to particularly single out are the two that users have mentioned most to me over the last 18 months or so:<\/p>\n
\n- Flash is no longer required for the File Level Recovery (FLR) web interface<\/li>\n
- There’s a command line interface for FLR<\/li>\n<\/ul>\n
If you’ve been leery about using VBA for either of the above reasons, it’s time to jump on the bandwagon and see just how useful it is. Note that in order to achieve command line FLR you’ll need to install the basic NetWorker client package on the relevant hosts \u2013 but you need to get a binary from somewhere, so that makes sense.<\/p>\n
Module Enhancements<\/h2>\n
Both the NetWorker Module for Microsoft Applications (NMM) and NetWorker Module for Databases and Applications (NMDA) have received a bunch of updates, including (but not limited to):<\/p>\n
\n- NMM:\n
\n- Simpler use of VSS.<\/li>\n
- Block based support for HyperV and Exchange \u2013 yes, and<\/em> Exchange. (This speeds up both types of backups considerably.)<\/li>\n
- Federated backups for SharePoint, allowing non-primary databases to be leveraged for the backup process.<\/li>\n
- I love the configuration checker \u2013 it makes getting NMM up and running with minimum effort so much easier. It’s been further enhanced in NetWorker 9 to grow its usefulness even more.<\/li>\n
- HyperV support for Partial VSS writer \u2013 previously if you had a single VM fail to backup under HyperV the backup group running the process would register as a failure. Now the backups will continue and only the VM that fails to backup will be be declared a failure. This aligns HyperV backups much more closely to traditional filesystem or VMware style backups.<\/li>\n
- Improved support for Federated backups of HyperV SMB 3 clusters.<\/li>\n
- File Level Recovery GUI for HyperV virtual machine backups.<\/li>\n
- Full integration of policy support for NMM.<\/li>\n<\/ul>\n<\/li>\n
- NMDA:\n
\n- Support for DDBoost over Fibre-Channel for AIX.<\/li>\n
- Full integration of policy support for NMDA.<\/li>\n
- Support for log-only backups for Lotus Notes systems.<\/li>\n
- NetWorker Snapshot Manager support for features like ProtectPoint.<\/li>\n
- Various DB2 enhancements\/improvements.<\/li>\n
- Oracle RAC discovery in the NMC configuration wizards.<\/li>\n
- Optional use of a CONFIG_FILE parameter for RMAN scripts so you can put all the NMDA related customisations for RMAN backups into a single file (or small number of files) and keep that file\/those files updated rather than having to make changes to individual RMAN scripts.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
Policies, Redux<\/h2>\n
Before I wrap up: just one more thing<\/em>. With the transition to a policy configuration engine, the nsrpolicy<\/em> command previously introduced in NetWorker 8.1 to support Virtual Machine Protection Policies has been extensively enhanced to be able to handle all aspects of policy creation, configuration adjustment and policy\/workflow execution. This does mean<\/em> that if you’ve previously used nsradmin<\/em> or savegrp<\/em> to handle configuration\/group execution processes, you’ll have to adjust some of your scripts accordingly. (It also means I’ll have to work on a new version of the Turbocharged NetWorker Administration Guide.)<\/p>\nWrapping Up<\/h2>\n
I wasn’t joking at the start when I said NetWorker 9 represents the biggest set of changes I’ve ever seen in my 19 years of using NetWorker. What I will say is that these are necessary<\/em> changes to prepare NetWorker for the rapidly changing datacentre. (Or even the rapidly changing datacenter<\/em> if you’re so minded.)<\/p>\nThis upgrade will<\/em> require very careful review of the release notes and changed functionality, as well as potentially revisiting any automation scripts you’ve done in the past. (But you can<\/em> do it.) If you’ve got a heavily scripted environment, my advice is to run up a test NetWorker 9 server and review your scripts against the changes, first evaluating whether you actually need to continue using those scripts, and then if you do, adjusting them accordingly. EMC has also prepared some video training for NetWorker 9 which I’d advise looking into (and equally I’d suggest leveraging your local EMC partner or EMC resources for the upgrade process).<\/p>\nIt’s also an excellent time to consider revisiting your overall backup configuration and look for optimisations you can achieve based on the new policy engine and the service-catalogue approach. As I’ve been saying to my colleagues, this is the perfect opportunity to introduce policies that align to service catalogues that more precisely define and meet business requirements. If you’re not ready to do it from day zero, that’s OK \u2013 NetWorker will migrate your configuration and you’ll be able to continue to offer your existing backup and recovery services. But if you find the time to re-evaluate your configuration and reset it to a service catalogue approach, you can migrate yourself<\/em> from being the “backup admin” to being the “data protection architect” within your organisation.<\/p>\nThis is a big set of changes in NetWorker, but it’s also very much an exciting and energising set of changes, too.<\/p>\n
As you might expect, this won’t be my only blog post on NetWorker 9 \u2013 it’s equally an energising time for me and I’m looking forward to diving into a variety of topics in more detail and providing some screen casts and videos of changes, upgrades and improvements.<\/p>\n
(And don’t forget to wear your sunglasses: the future’s looking bright.)<\/p>\n","protected":false},"excerpt":{"rendered":"
Introduction When NetWorker 8 was released I said at the time it represented the biggest consolidated set of changes to NetWorker…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[3,1133,11,14,16,17,19,20,1213],"tags":[1249,1269],"class_list":["post-5636","post","type-post","status-publish","format-standard","hentry","category-architecture","category-best-practice","category-features","category-licensing","category-networker","category-policies","category-recovery","category-scripting","category-vba","tag-networker","tag-networker-9"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pKpIN-1sU","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/5636","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/comments?post=5636"}],"version-history":[{"count":25,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/5636\/revisions"}],"predecessor-version":[{"id":7424,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/5636\/revisions\/7424"}],"wp:attachment":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/media?parent=5636"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/categories?post=5636"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/tags?post=5636"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"Into](\"https:\/\/nsrd.info\/blog\/wp-content\/uploads\/2015\/07\/bigStock-Time-Warp.jpg\")
<\/a>