NetWorker 9.2 Capacity Measurement

 Licensing, NetWorker, Scripting  Comments Off on NetWorker 9.2 Capacity Measurement
Aug 032017
 

As I’ve mentioned in the past, there’s a few different licensing models for NetWorker, but capacity licensing (e.g., 100 TB front end backup size) gives considerable flexibility, effectively enabling all product functionality within a single license, thereby allowing NetWorker usage to adapt to suit the changing needs of the business.

Data Analysis

In the past, measuring utilisation has typically required either the use of DPA or asking your DellEMC account team to review the environment and provide a report. NetWorker 9.2 however gives you a new, self-managed option – the ability to run, whenever you want, a capacity measurement report to determine what your utilisation ratio is.

This is done through a new command line tool, nsrcapinfo, which is incredibly simple to run. In fact, running it without any options at all will give the default 60 day report, providing utilisation details for each of the key data types as well as summary. For instance, against my lab server, here’s the output:

<?xml version="1.0" encoding="UTF8" standalone="yes" ?>
<!--
~ Copyright (c) 2017 Dell EMC Corporation. All Rights Reserved.
~
~ This software contains the intellectual property of Dell EMC Corporation or is licensed to
~ Dell EMC Corporation from third parties. Use of this software and the intellectual property
~ contained therein is expressly limited to the terms and conditions of the License
~ Agreement under which it is provided by or on behalf of Dell EMC.
-->
<Capacity_Estimate_Report>
<Time_Stamp>2017-08-02T21:21:18Z</Time_Stamp>
<Clients>13</Clients>
<DB2>0.0000</DB2>
<Informix>0.0000</Informix>
<IQ>0.0000</IQ>
<Lotus>0.0000</Lotus>
<MySQL>0.0000</MySQL>
<Sybase>0.0000</Sybase>
<Oracle>0.0000</Oracle>
<SAP_HANA>0.0000</SAP_HANA>
<SAP_Oracle>0.0000</SAP_Oracle>
<Exchange_NMM8.x>0.0000</Exchange_NMM8.x>
<Exchange_NMM9.x>0.0000</Exchange_NMM9.x>
<Hyper-V>0.0000</Hyper-V>
<SharePoint>0.0000</SharePoint>
<SQL_VDI>0.0000</SQL_VDI>
<SQL_VSS>0.0000</SQL_VSS>
<Meditech>0.0000</Meditech>
<Other_Applications>2678.0691</Other_Applications>
<Unix_Filesystems>599.9214</Unix_Filesystems>
<VMware_Filesystems>360.3535</VMware_Filesystems>
<Windows_Filesystems>27.8482</Windows_Filesystems>
<Total_Largest_Filesystem_Fulls>988.1231</Total_Largest_Filesystem_Fulls>
<Peak_Daily_Applications>2678.0691</Peak_Daily_Applications>
<Capacity_Estimate>3666.1921</Capacity_Estimate>
<Unit_of_Measure_Bytes_per_GiB>1073741824</Unit_of_Measure_Bytes_per_GiB>
<Days_Measured>60</Days_Measured>
</Capacity_Estimate_Report>

That’s in XML by default – and the numbers are in GiB.

If you do fulls on longer cycles than the default of a 60 day measurement window you can extend the data sampling range by using -d nDays in the command (e.g., “nsrcapinfo -d 90” would provide a measurement over a 90 day window). You can also, if you wish for further analysis, generate additional reports (see the command reference guide or, man nsrcapinfo if you’re on Linux for the full details). One of those reports that I think will be quite popular with backup administrators will be the client report. An example of that is below:

[root@orilla ~]# nsrcapinfo -r clients
"Hostname", "Client_Capacity_GiB", "Application_Names" 
"abydos.turbamentis.int", "2.3518", "Unix_Filesystems"
"vulcan", "16.0158", "VMware_Filesystems"
"win01", "80.0785", "VMware_Filesystems"
"picon", "40.0394", "VMware_Filesystems"
"win02", "80.0788", "VMware_Filesystems"
"vega", "64.0625", "VMware_Filesystems"
"test02", "16.0157", "VMware_Filesystems"
"test03", "16.0157", "VMware_Filesystems"
"test01", "16.0157", "VMware_Filesystems"
"krell", "32.0314", "VMware_Filesystems"
"faraway.turbamentis.int", "27.8482", "Windows_Filesystems"
"orilla.turbamentis.int", "1119.5321", "Other_Applications Unix_Filesystems"
"rama.turbamentis.int", "2156.1067", "Other_Applications Unix_Filesystems"

That’s a straight-up simple view of the FETB estimation for each client you’re protecting in your environment.

There you have it – capacity measurement in NetWorker as a native function in version 9.2.

Basics – Using the vSphere Plugin to Add Clients for Backup

 NetWorker, NVP, vProxy  Comments Off on Basics – Using the vSphere Plugin to Add Clients for Backup
Jul 242017
 

It’s a rapidly changing trend – businesses increasingly want the various Subject Matter Experts (SMEs) running applications and essential services to be involved in the data protection process. In fact, in the 2016 Data Protection Index, somewhere in the order of 93% of respondents said this was extremely important to their business.

It makes sense, too. Backup administrators do a great job, but they can’t be expected to know everything about every product deployed and protected within the organisation. The old way of doing things was to force the SMEs to learn how to use the interfaces of the backup tools. That doesn’t work so well. Like the backup administrators having their own sphere of focus, so too do the SMEs – they understandably want to use their tools to do their work.

What’s more, if we do find ourselves in a disaster situation, we don’t want backup administrators to become overloaded and a bottleneck to the recovery process. The more those operations are spread around, the faster the business can recover.

So in the modern data protection environment, we have to work together and enable each other.

Teams working together

In a distributed control model, the goal will be for the NetWorker administrator to define the protection policies needed, based on the requirements of the business. Once those policies are defined, enabled SMEs should be able to use their tools to work with those policies.

One of the best examples of that is for VMware protection in NetWorker. Using the plugins provided directly into the vSphere Web Client, the VMware administrators can attach and detach virtual machines from protection policies that have been established in NetWorker, and initiate backups and recoveries as they need.

In the video demo below, I’ll take you through the process whereby the NetWorker administrator defines a new virtual machine backup policy, then the VMware administrator attaches a virtual machine to that policy and kicks it off. It’s really quite simple, and it shows the power that you get when you enable SMEs to interact with data protection from within the comfort of their own tools and interfaces. (Don’t forget to ensure you switch to 720p/HD in order to see what’s going on within the session.)


Don’t forget – if you find the NetWorker Blog useful, you’ll be sure to enjoy Data Protection: Ensuring Data Availability.

Jul 212017
 

I want to try something different with this post. Rather than the usual post with screen shots and descriptions, I wanted instead to do a demo video showing just how easy it is to do file level recovery (FLR) from NetWorker VMware Image Level Backup thanks to the new NVP or vProxy system in NetWorker 9.

The video below steps you through the entire FLR process for a Linux virtual machine. (If your YouTube settings don’t default to it, be sure to switch the video to High Def (720) or otherwise the text on the console and within NMC may be difficult to read.)

Don’t forget – if you find the information on the NetWorker Blog useful, I’m sure you’ll get good value out of my latest book, Data Protection: Ensuring Data Availability.

NetWorker 9.1.1 gets out the door

 NetWorker  Comments Off on NetWorker 9.1.1 gets out the door
May 022017
 

I had a fairly full-on weekend so I missed this one – NetWorker 9.1.1 is now available.

Being a minor release, this one is focused on general improvements and currency, as opposed to introducing a wealth of new features.

Upgrade

There’s some really useful updates around NMC, such as:

  • Performance/response improvements
  • Option for NMC to retrieve a vProxy support bundle for you
  • NMC now shows whenever the NetWorker server is running in service mode
  • NMC will give you a list of virtual machines backed up and skipped
  • NMC recoveries now highlight the calendar dates that are available to select backups to recover from

Additionally, NDMP and NDMA get some updates as well:

  • Some NDMP application options can now be set in the NetWorker client resource level, rather than having to establish them as an environment variable
  • NMDA for SAP/Oracle and Oracle/RMAN get more compact debug logs
  • NMDA for Sybase can now recover log-tail backups.

Finally, there’s the version currency:

  • NetWorker Server High Availability is now supported on SuSE 12 SP2 with HAE, and RHEL 7.3 in a High Availability Cluster (with Pacemaker).
  • NVP/vProxy supports vSphere 6.0u3
  • Meditech module supports Unity 4.1 and RecoverPoint 5.0.

As always for upgrades, make sure you read the release notes before diving in.


Also, don’t forget my new book is out: Data Protection: Ensuring Data Availability. It’s the perfect resource for any data protection architect.

NetWorker 9.1 FLR Web Interface

 NVP, Recovery, vProxy  Comments Off on NetWorker 9.1 FLR Web Interface
Apr 042017
 

Hey, don’t forget, my new book is available. Jam packed with information about protecting across all types of RPOs and RTOs, as well as helping out on the procedural and governance side of things. Check it out today on Amazon! (Kindle version available, too.)


In my introductory NetWorker 9.1 post, I covered file level recovery (FLR) from VMware image level backup via NMC. I felt at the time that it was worthwhile covering FLR from within NMC as the VMware recovery integration in NMC was new with 9.1. But at the same time, the FLR Web interface for NetWorker has also had a revamp, and I want to quickly run through that now.

First, the most important aspect of FLR from the new NetWorker Virtual Proxy (NVP, aka “vProxy”) is not something you do by browsing to the Proxy itself. In this updated NetWorker architecture, the proxies are very much dumb appliances, completely disposable, with all the management intelligence coming from the NetWorker server itself.

Thus, to start a web based FLR session, you actually point your browser to:

https://nsrServer:9090/flr

The FLR web service now runs on the NetWorker server itself. (In this sense quite similarly to the FLR service for Hyper-V.)

The next major change is you no longer have to use the FLR interface from a system currently getting image based backups. In fact, in the example I’m providing today, I’m doing it from a laptop that isn’t even a member of the NetWorker datazone.

When you get to the service, you’ll be prompted to login:

01 Initial Login

For my test, I wanted to access via the Administration interface, so I switched to ‘Admin’ and logged on as the NetWorker owner:

02 Logging In as Administrator

After you login, you’re prompted to choose the vCenter environment you want to restore from:

03 Select vCenter

Selecting the vCenter server of course lets you then choose the protected virtual machine in that environment to be recovered:

04 Select VM and Backup

(Science fiction fans will perhaps be able to intuit my host naming convention for production systems in my home lab based on the first three virtual machine names.)

Once you’ve selected the virtual machine you want to recover from, you then get to choose the backup you want to recover – you’ll get a list of backups and clones if you’re cloning. In the above example I’ve got no clones of the specific virtual machine that’s been protected. Clicking ‘Next’ after you’ve selected the virtual machine and the specific backup will result in you being prompted to provide access credentials for the virtual machine. This is so that the FLR agent can mount the backup:

05 Provide Credentials for VM

Once you provide the login credentials (and they don’t have to be local – they can be an AD specified login by using the domain\account syntax), the backup will be mounted, then you’ll be prompted to select where you want to recover to:

06 Select Recovery Location

In this case I selected the same host, recovering back to C:\tmp.

Next you obviously need to select the file(s) and folder(s) you want to recover. In this case I just selected a single file:

07 Select Content to Recover

Once you’ve selected the file(s) and folder(s) you want to recover, click the Restore button to start the recovery. You’ll be prompted to confirm:

08 Confirm Recovery

The restore monitor is accessible via the bottom of the FLR interface, basically an upward-pointing arrow-head to expand. This gives you a view of a running, or in this case, a complete restore, since it was only a single file and took very little time to complete:

09 Recovery Success

My advice generally is that if you want to recover thousands or tens of thousands of files, you’re better off using the NMC interface (particularly if the NetWorker server doesn’t have a lot of RAM allocated to it), but for smaller collections of files the FLR web interface is more than acceptable.

And Flash-free, of course.

There you have it, the NetWorker 9.1 VMware FLR interface.


Hey, don’t forget, my new book is available. Jam packed with information about protecting across all types of RPOs and RTOs, as well as helping out on the procedural and governance side of things. Check it out today on Amazon! (Kindle version available, too.)


 

Mar 272017
 

I’d like to take a little while to talk to you about licensing. I know it’s not normally considered an exciting subject (usually at best people think of it as a necessary-evil subject), but I think it’s common to see businesses not take full advantage of the potential data protection licensing available to them from Dell EMC. Put it this way: I think if you take the time to read this post about licensing, you’ll come away with some thoughts on how you might be able to expand a backup system to a full data protection system just thanks to some very handy licensing options available.

When I first started using NetWorker, the only licensing model was what I’d refer to as feature based licensing. If you wanted to do X, you bought a license that specifically enabled NetWorker to do X. The sorts of licenses you would use included:

  • NetWorker Base Enabler – To enable the actual base server itself
  • OS enablers – Called “ClientPack” enablers, these would let you backup operating systems other than the operating system of the NetWorker server itself (ClientPack for Windows, ClientPack for Unix, ClientPack for Linux, etc).
  • Client Count enablers – Increasing the number of clients you can backup
  • Module enablers – Allowing you to say, backup Oracle, or SQL, or Exchange, etc.
  • Autochanger enablers – Allowing you to connect autochangers of a particular slot count (long term NetWorker users will remember short-slotting too…)

That’s a small excerpt of the types of licences you might have deployed. Over time, some licenses got simplified or even removed – the requirement for ClientPack enablers for instance were dropped quite some time ago, and the database licenses were simplified by being condensed into licenses for Microsoft databases (NMM) and licenses for databases and applications (NMDA).

Feature based licensing is, well, confusing. I’d go so far as to suggest it’s anachronistic. As a long-term NetWorker user, I occasionally get asked what a feature based licensing set might look like, or what might be required to achieve X, and even for me, having dealt with feature based licenses for 20 years, it’s not fun.

bigStock Confusion

The problem – and it’s actually a serious one – with feature based licensing is you typically remain locked, for whatever your minimum budget cycle is, into what your backup functionality is. Every new database, set of clients, backup device or special requirement has to be planned well in advance to make sure you have the licenses you need. How often is that really the case? I’m into my 21st year of working with backup and I still regularly hear stories of new systems or projects coming on-line without full consideration of the data protection requirements.

In this modern age of datacentre infrastructure where the absolute requirement is agility, using feature-based licensing is like trying to run on a treadmill that’s submerged waist-deep in golden syrup.

There was, actually, one other type of NetWorker licensing back then – in the ‘old days’, I guess I can say: an Enterprise license. That enabled everything in one go, but required yearly audits to ascertain usage and appropriate maintenance costs, etc. It enabled convenient use but from a price perspective it only suited upper-echelon businesses.

Over time to assist with providing licensing agility, NetWorker got a second license type – capacity licensing. This borrowed the “unlimited features” aspect of enterprise-based licensing, and worked on the basis of what we refer to as FETB – Front End TB. The simple summary of FETB is “if you did a full backup of everything you’re protecting, how big would it be?” (In fact, various white-space components are typically stripped out – a 100 GB virtual machine for instance that’s thickly provisioned but only using 25GB would effectively be considered to contribute just 25 GB to the capacity.)

The beauty of the capacity license scheme is that it doesn’t matter how many copies you generate of your data. (An imaginary BETB (“Back End TB”) license would be unpleasant in the extreme – limiting you to the total stored capacity of your backups.) So that FETB license applies regardless of whether you just keep all your backups for 30 days, or whether you keep all your backups for 7 years. (If you keep all your backups for 7 years, read this.)

A FETB lets you adjust your backup functionality as the business changes around you. Someone deploys Oracle but you’ve only had to backup SQL Server before? Easy, just install NMDA and start backing Oracle up. The business makes the strategic decision to switch from Hyper-V to VMware? No problem – there’s nothing to change from a licensing perspective.

But, as I say in my book, backup and recovery, as a standalone topic is dead. That’s why Dell EMC has licensing around Data Protection Suite. In fact, there’s a few different options to suit different tiers of organisations. If you’ve not heard of Data Protection Suite licensing, you’ve quite possibly been missing out on a wealth of opportunities for your organisation.

Let’s start with the first variant that was introduced, Data Protection Suite for Backup. (In fact, it was originally just Data Protection Suite.) DPS for Backup has been expanded as other products have been released, and now includes:

DPS for Backup

Think about that – from a single wrapper license (DPS for Backup), you get access to 6 products. Remember before when I said the advantage of NetWorker capacity licensing over ‘feature’ licensing was the ability to adapt to changes in the business requirements for backup? This sort of license expands on that ability even more so. You might start today using NetWorker to protect your environment, but in a year’s time your business needs to setup some remote offices that are best served by Avamar. With DPS for Backup, you don’t need to go and buy Avamar licenses, you just deploy Avamar. Equally, the strategic decision might be made to give DBAs full control over their backup processes, so it makes sense to give them access to shared protection storage via Data Domain Boost for Enterprise Applications (DDBEA), instead of needing to be configured for manual backups in NetWorker. The business could decide to start pushing some long term backups from NetWorker out to Cloud object storage – that’s easy, just deploy a CloudBoost virtual machine because you can. You can mix and match your licenses as you need. Just as importantly, you can deploy Data Protection Advisor at the business layer to provide centralised reporting and monitoring across the entire gamut, and you can take advantage of Data Protection Search to easily find content regardless of whether it was NetWorker or Avamar that protected it.

Data Protection Suite for Backup is licensed – like the NetWorker Capacity model – via FETB. So if you license for say, 500 TB, you can slice and dice that however you need between NetWorker, Avamar and DDBEA, and get CloudBoost, DPA and DP-Search rolled in. Suddenly your backup solution is a much broader data protection solution, just thanks to a license model!

If you’re not an existing NetWorker or Avamar site, but you’re looking for some increased efficiencies in your application backups/backup storage, or a reduction in the capacity licensing for another product, you might instead be interested in DPS for Applications:

DPS for Applications

Like DPS for Backup, DPS for Applications is a FETB capacity license. You get to deploy Boost for Enterprise Apps and/or ProtectPoint to suit your requirements, you get Data Protection Advisor to report on your protection status, and you also get the option to deploy Enterprise Copy Data Management (eCDM). That lets you set policies on application protection – e.g., “There must always be 15 copies of this database”. The application administration team can remain in charge of backups, but to assuage business requirements, policies can be established to ensure systems are still adequately protected. And ProtectPoint: whoa, we’re talking serious speed there. Imagine backing up a 10TB or 50TB database, not 20% faster, but 20 times faster. That’s ProtectPoint – Storage Integrated Data Protection.

Let’s say you’re an ultra-virtualised business. There’s few, if any, physical systems left, and you don’t want to think of your data protection licensing in terms of FETB, which might be quite variable – instead, you want to look at a socket based licensing count. If that’s the case, you probably want to look at Data Protection Suite for Virtual Machines:

DPS for Virtual Machines

DPS for Virtual Machines is targeted for the small to medium end of town to meet their data protection requirements in a richly functional way. On a per socket (not per-core) license model, you get to protect your virtual infrastructure (and, if you need to, a few physical servers) with Avamar, using image based and agent-based backups in whatever mix is required. You also get RecoverPoint for Virtual Machines. RecoverPoint gives you DVR-like Continuous Data Protection that’s completely storage independent, since it operates at the hypervisor layer. Via an advanced journalling system, you get to deliver very tight SLAs back to the business with RTOs and RPOs in the seconds or minutes, something that’s almost impossible with just standard backup. (You can literally choose to roll back virtual machines on an IO-by-IO basis. Or spin up testing/DR copies using the same criteria.) You also get DPA and DP-Search, too.

There’s a Data Protection Suite for archive bundle as well if your requirements are purely archiving based. I’m going to skip that for the moment so I can talk about the final licensing bundle that gives you unparalleled flexibility for establishing a full data protection strategy for your business; that’s Data Protection Suite for Enterprise:

DPS for Enterprise

Data Protection Suite for Enterprise returns to the FETB model but it gives you ultimate flexibility. On top of it all you again get Data Protection Advisor and Data Protection Search, but then you get a raft of data protection and archive functionality, all again in a single bundled consumption model: NetWorker, Avamar, DDBEA, CloudBoost, RecoverPoint for Virtual Machines, ProtectPoint, AppSync, eCDM, and all the flavours of SourceOne. In terms of flexibility, you couldn’t ask for more.

It’s easy when we work in backup to think only in terms of the main backup product we’re using, but there’s two things that have become urgently apparent:

  • It’s not longer just about backup – To stay relevant, and to deliver value and results back to the business, we need to be thinking about data protection strategies rather than backup and recovery strategies. (If you want proof of that change from my perspective, think of my first book title vs the second – the first was “Enterprise Systems Backup and Recovery”, the second, “Data Protection”.)
  • We need to be more agile than “next budget cycle” – Saying you can’t do anything to protect a newly emerged or altering workload until you get budget next year to do it is just a recipe for disaster. We need, as data protection professionals, to be able to pick the appropriate tool for each workload and get it operational now, not next month or next year.

Licensing: it may on the outset appear to be a boring topic, but I think it’s actually pretty damn exciting in what a flexible licensing policy like the Data Protection Suite allows you to offer back to your business. I hope you do too, now.


Hey, you’ve made it this far, thanks! I’d love it if you bought my book, too! (In Kindle format as well as paperback.)


 

Jan 132017
 

Introduction

There’s something slightly deceptive about the title for my blog post. Did you spot it?

It’s: vs. It’s a common mistake to think that Cloud Boost and Cloud Tier compete with one another. That’s like suggesting a Winnebago and a hatchback compete with each other. Yes, they both can have one or more people riding in them and they can both be used to get you around, but the actual purpose of each is typically quite different.

It’s the same story when you look at Cloud Boost and Cloud Tier. Of course, both can move data from A to B. But the reason behind each, the purpose for each is quite different. (Does that mean there’s no overlap? Not necessarily. If you need to go on a 500km holiday and sleep in the car, you can do that in a hatchback or a Winnebago, too. You can often get X to do Y even if it wasn’t built with that in mind.)

So let’s examine them, and look at their workflows as well as a few usage examples.

Cloud Boost

First off, let’s consider Cloud Boost. Version 1 was released in 2014, and since then development has continued to the point where CloudBoost now looks like the following:

CloudBoost Workflow

Cloud Boost Workflow

Cloud Boost exists to allow NetWorker (or NetBackup or Avamar) to write deduplicated data out to cloud object storage, regardless of whether that’s on-premises* in something like ECS, or writing out to a public cloud’s object storage system, like Virtustream Storage or Amazon S3. When Cloud Boost was first introduced back in 2014, the Cloud Boost appliance was also a storage node and data had to be cloned from another device to the Cloud Boost storage node, which would push data out to object. Fast forward a couple of years, and with Cloud Boost 2.1 introduced in the second half of 2016, we’re now at the point where there’s a Cloud Boost API sitting in NetWorker clients allowing full distributed data processing, with each client talking directly to the object storage – the Cloud Boost appliance now just facilitates the connection.

In the Cloud Boost model, regardless of whether we’re backing up in a local datacentre and pushing to object, or whether all the systems involved in the backup process are sitting in public cloud, the actual backup data never lands on conventional block storage – after it is deduplicated, compressed and encrypted it lands first and only in object storage.

Cloud Tier

Cloud Tier is new functionality released in the Data Domain product range – it became available with Data Domain OS v6, released in the second half of 2016. The workflow for Cloud Tier looks like the following:

CloudTier Workflow

CloudTier Workflow

Data migration with Cloud Tier is handled as a function of the Data Domain operating system (or controlled by a fully integrated application such as NetWorker or Avamar); the general policy process is that once data has reached a certain age on the Active Tier of the Data Domain, it is migrated to the Cloud Tier without any need for administrator or user involvement.

The key for the differences – and the different use cases – between Cloud Boost and Cloud Tier is in the above sentence: “once data has reached a certain age on the Active Tier”. In this we’re reminded of the primary use case for Cloud Tier – supporting Long Term Retention (LTR) in a highly economical format and bypassing any need for tape within an environment. (Of course, the other easy differentiator is that Cloud Tier is a Data Domain feature – depending on your environment that may form part of the decision process.)

Example use cases

To get a feel for the differences in where you might deploy Cloud Boost or Cloud Tier, I’ve drawn up a few use cases below.

Cloning to Cloud

You currently backup to disk (Data Domain or AFTD) within your environment, and have been cloning to tape. You want to ensure you’ve got a second copy of your data, and you want to keep that data off-site. Instead of using tape, you want to use Cloud object storage.

In this scenario, you might look at replacing your tape library with a Cloud Boost system instead. You’d backup to your local protection storage, then when it’s time to generate your secondary copy, you’d clone to your Cloud Boost device which would push the data (compressed, deduplicated and encrypted) up into object storage. At a high level, that might result in a workflow such as the following:

CloudBoost Clone To Cloud

CloudBoost Clone To Cloud

Backing up to the Cloud

You’re currently backing up locally within your datacentre, but you want to remove all local backup targets.  In this scenario, you might replace your local backup storage with a Cloud Boost appliance, connected to an object store, and backup via Cloud Boost (via client direct), landing data immediately off-premises and into object storage at a cloud provider (public or hosted).

At a high level, the workflow for this resembles the following:

CloudBoost Backup to Cloud

CloudBoost Backup to Cloud

Backing up in Cloud

You’ve got some IaaS systems sitting in the Cloud already. File, web and database servers sitting in say, Amazon, and you need to ensure you can protect the data they’re hosting. You want greater control than say, Amazon snapshots, and since you’re using a NetWorker Capacity license or a DPS capacity license, you know you can just spin up another NetWorker server without an issue – sitting in the cloud itself.

In that case, you’d spin up not only the NetWorker server but a Cloud Boost appliance as well – after all, Amazon love NetWorker + Cloud Boost:

“The availability of Dell EMC NetWorker with CloudBoost on AWS is a particularly exciting announcement for all of the customers who have come to depend on Dell EMC solutions for data protection in their on-premises environments,” said Bill Vass, Vice President, Technology, Amazon Web Services, Inc. “Now these customers can get the same data protection experience on AWS, providing seamless operational backup and recovery, and long-term retention across all of their environments.”

That’ll deliver the NetWorker functionality you’ve come to use on a daily basis, but in the Cloud and writing directly to object storage.

The high level view of the backup workflow here is effectively the same as the original diagram used to introduce Cloud Boost.

Replacing Tape for Long Term Retention

You’ve got a Data Domain in each datacentre; the backups at each site go to the local Data Domain then using Clone Controlled Replication are copied to the other Data Domain as soon as each saveset finishes. You’d like to replace tape for your long term retention, but since you’re protecting a lot of data, you want to push data you rarely need to recover from (say, older than 2 months) out to object storage. When you do need to recover that data, you want to absolutely minimise the amount of data that needs to be retrieved from the Cloud.

This is a definite Cloud Tier solution. Cloud Tier can be used to automatically extend the Data Domain storage, providing a storage tier for long term retention data that’s very cheap and highly reliable. Cloud Tier can be configured to automatically migrate data older than 2 months out to object storage, and the great thing is, it can do it automatically for anything written to the Data Domain. So if you’ve got some databases using DDBoost for Enterprise Apps writing directly, you can setup migration policies for them, too. Best of all, when you do need to recall data from Cloud Tier, Boost for Enterprise Apps and NetWorker can handle that recall process automatically for you, and the Data Domain only ever recalls the delta between deduplicated data already sitting on the active tier and what’s out in the Cloud.

The high level view of the workflow for this use case will resemble the following:

Cloud Tier to LTR NSR+DDBEA

Cloud Tier to LTR for NetWorker and DDBEA

…Actually, you hear there’s an Isilon being purchased and the storage team are thinking about using Cloud Pools to tier really old data out to object storage. Your team and the storage team get to talking and decide that by pooling the protection and storage budget, you get Isilon, Cloud Tier and ECS, providing oodles of cheap object storage on-site at a fraction of the cost of a public cloud, and with none of the egress costs or cloud vendor lock-in.

Wrapping Up

Cloud Tier and Cloud Boost are both able to push data into object storage, but they don’t have exactly the same use cases. There’s good, clear reasons why you would work with one in particular, and hopefully the explanation and examples above has helped to set the scene on their use cases.


* Note, ‘on-premise’ would mean ‘on my argument’. The correct term is ‘on-premises’ 🙂

Dec 232016
 

I know, I know, it’s winter up there in the Northern Hemisphere, but NetWorker 9.1 is landing and given I’m in Australia, that makes NetWorker 9.1 a Summer Fresh release. (In fact, my local pub for the start of summer started doing a pale ale infused with pineapple and jalapeños, and that’s sort of reminding me of NetWorker 9.1: fresh, light and inviting you to put your heels up and rest a while.)

NetWorker 9.1

 

NetWorker 9 was a big – no, a huge – release. It’s a switch to a more service catalogue driven approach to backups, Linux block based filesystem backups, block based application backups, deep snapshot integration and more recently in NetWorker 9.0 SP1, REST API control as well.

NetWorker 9.1 as you’d expect is a smaller jump from 9.0 than we had from 8.2 to 9.0. That being said, it’s introduced some excellent new features:

  • VMAX SmartSnap integration – the ability to backup and restore a VMAX device based on the device WWN, increasing the depth of snapshot support in NetWorker further.
  • Snapshot Alternate Location Rollback – this lets you do a snapshot rollback, but to a different set of devices.
  • Data Domain High Availability integration – Data Domain now supports high-availability on the earlier 9500 platform, in addition to the 9800, 9300 and 6800 systems. And with v9.1, NetWorker fully understands and integrates with DDHA platforms.
  • Cloud Tier Integration – NetWorker gets deep integration into the Cloud Tier functionality introduced in Data Domain OS 6.0. This lets NetWorker cloning policies control the migration of data out to the Cloud Tier, and more seamlessly integrate with the recall process.

Cloud Tier integration is more than just a tick in the box to though. Consider the module space – NetWorker Module for Microsoft Applications, for instance, doesn’t just get the option to recover data from Cloud Tier, but also perform granular recoveries from Cloud Tier – SQL table level recoveries and Exchange granular recoveries as well.


By the way, the NetWorker Usage Survey is still running – don’t forget to fill in how you’re using NetWorker! (And be in the running for a prize.)


I’ve saved the best – and biggest – feature for last, though. This is a doozy. Say goodbye to needing a EBR/VBA for VMware backups. That EBR/VBA functionality is now embedded in the NetWorker server itself, leaving you to just deploy some very lightweight proxies to handle the data transport processes, all controlled by NetWorker.

The current EBR appliance and proxies will continue to work with NetWorker 9.1, but I can’t think of anyone who’d want to upgrade to 9.1 without rapidly transitioning to the new platform. Here are just some of the advantages of the new process:

  • Less virtual infrastructure required – no EBRs
  • Virtual machines stored in raw VMDK file – no additional processing required for the backup, and this will also mean faster instant access processes, too
  • The FLR web GUI now runs on the NetWorker server itself
  • NMC can be used for FLR instead of the web GUI, making it more accessible to the NetWorker administrators if they don’t have access to the virtual machines being protected
  • Proxies support more concurrent virtual machine backups:
    • Maximum 25 concurrent hotadd operations;
    • Maximum 25 concurrent NBD operations
  • Significantly increased File Level Recovery (FLR) counts from VMware Image Level Backups (recommended 20,000 – more on that in a minute)
  • Significantly faster FLR operations.

In fact, I’m going to spend a little bit of time on FLR for this post, and step through the new NMC-based FLR process to give you an overview of the process. This is using the newly deployed NetWorker VMware Protection (NVP) system, with backup to and recovery from Data Domain virtual edition.

Fig 01: Starting a recovery in NMC

Fig 01: Starting a recovery in NMC

You start by telling NMC you want to do a virtual machine recovery and choose the vCenter server that owns the virtual machine(s) you want to recover data from.

Fig 02: Choosing the virtual machine to recover from

Fig 02: Choosing the virtual machine to recover from

There’s various options for choosing the virtual machine to recover data for – you can enter the name directly, search for it, browse the various backups that have been performed, or browse the vCenter server itself.

Fig 03: Virtual Machine selected

Fig 03: Virtual Machine selected

Once you’ve selected a virtual machine for recovery, you can click Next to choose the backup to recover from.

Fig 04: Choosing the backup to recover from

Fig 04: Choosing the backup to recover from

In this case, I only had a single backup under the new NVP system for that virtual machine, so I was able to just click Next to continue the process. At this point you get to choose the type of recovery you want to perform:

Fig 05: Choosing the type of recovery to perform

Fig 05: Choosing the type of recovery to perform

As you can see, there’s a gamut of recovery options for virtual machines within NMC. I’m focusing on the FLR options here so I chose the bottom option and clicked Next.

Fig 06: Choosing backup instance to recover from

Fig 06: Choosing backup instance to recover from

Next you get to choose the backup instance you want to recover from. If the backup has been cloned it may be that there’s topologically a better backup to recover from than the original, and choosing an alternate is as simple as scrolling through a list of clones.

At that point you get to choose where you want to recover to:

Fig 07: Choosing where to recover data to

Fig 07: Choosing where to recover data to

Next, you’ll supply appropriate credentials for the virtual machine to be able to perform the recovery and initiate a mount of the backup into the proxy server:

Fig 08: Supplying virtual machine credentials to mount the backup

Fig 08: Supplying virtual machine credentials to mount the backup

After you’ve supplied the credentials you’ll click “Start Mount” to make the specific backup available for recovery purposes, and after a few seconds that’ll result in log information such as:

Fig 09: Mounted and ready

Fig 09: Mounted and ready

When the mount is done, you’re ready to click Next and start browsing files for recovery.

Fig 10: Choosing files to recover from an image level backup

Fig 10: Choosing files to recover from an image level backup

In this example, I selected a directory with about 7,800 files in it and the marking of files for recovery took just a few seconds to complete. After which, Next to choose where to recover the data to on the selected virtual machine:

Fig 11: Choosing where to recover data to on the virtual machine

Fig 11: Choosing where to recover data to on the virtual machine

In this case I choose to recover to C:\tmp on the virtual machine. Clicking Next allows finalisation of the recovery preparation:

Fig 12: Finalising the recovery configuration

Fig 12: Finalising the recovery configuration

As you would expect with the tightly integrated controls now, FLR is fully visible within the NetWorker environment – even nsrwatch:

Fig 13: FLR in progress shown in nsrwatch

Fig 13: FLR in progress shown in nsrwatch

And finally we have a completed recovery:

Fig 14: Completed recovery

Fig 14: Completed recovery

That’s 7,918 files recovered from an image level backup in 54 seconds:

Fig 15: Recovered content

Fig 15: Recovered content

I wanted to check out the FLR capabilities a little more and decided to risk pushing the system beyond the recommendations. Instead of just recovering a single folder with 7,900 files or thereabouts, I elected to recover the entire E:\ drive on the virtual machine – comprising over 47,000 files. Here’s the results:

Fig 16: Large scale FLR results

Fig 16: Large scale FLR results

The recovered folder:

Fig 17: Recovered Content

Fig 17: Recovered Content

47,198 files, 1,488 folders, 5.01GB of data recovered as an FLR from an image level backup in just 5 minutes and 42 seconds.

If you’re using NetWorker for VMware backups, here’s the version you want to be on.

You can get it from the EMC Support page for NetWorker today.

Nov 302016
 

Folks, it’s that time of the year again! Each year I run a survey to gauge NetWorker usage patterns – how many clients you’ve got, what plugins you’re using, whether you’re using deduplication, and a plethora of other questions. The survey runs from December 1 (ish) through to January 31 the next year. (This year I’m kicking it off on November 30, just because I have time.)

Take the survey!

That gets assembled into a report in February of the following year, reporting trends across the various years the NetWorker survey has been conducted. If you’d like to see what the reports look like, you can view last year’s report here.

You can fill out the survey anonymously if you’d like, but if you submit your email address at the end you’ll be in the running for a copy of my upcoming book, Data Protection: Ensuring Data Availability, due out February 2017. (Last year’s winner hasn’t been forgotten – the book just got delayed.) If you submit your email address, it will not be used for any purpose other than to notify you if you’re the winner.

The survey is closed now. Results will be published in February 2017.

Basics – Understanding NetWorker Architecture

 Architecture, Basics, NetWorker  Comments Off on Basics – Understanding NetWorker Architecture
Nov 142016
 

With the NetWorker 9 architecture now almost 12 months old, I thought it was long past time I do a Basics post covering how the overall revised architecture for data protection with NetWorker functions.

There are two distinct layers of architecture I’ll cover off – Enterprise and Operational. In theory an entire NetWorker environment can be collapsed down to a single host – the NetWorker server, backing up to itself – but in practice we will typically see multiple hosts in an overall NetWorker environment, and as has been demonstrated by the regular NetWorker Usage Surveys, it’s not uncommon nowadays to see two or more NetWorker servers deployed in a business.

Enterprise Layer

The Enterprise Layer consists of the components that technically sit ‘above’ any individual NetWorker install within your environment, and can be depicted simply with the following diagram:

Enterprise Layer

The key services that will typically be run within the Enterprise Layer are the NetWorker License Server, and the NetWorker Management Console Server (NMC Server). While NetWorker has previously had the option of running an independent license server, with NetWorker 9 this has been formalised, and the recommendation is now to run a single license server for all NetWorker environments within your business, unless network or security rules prevent this.

The License server can be used by a single NetWorker server, or if you’ve got multiple NetWorker servers, by each NetWorker server in your environment, allowing all licenses to be registered against a single host, reducing ‘relicensing’ requirements if NetWorker server details change, etc. This is a very light-weight server, and it’s quite common to the license services run concurrently on the same host as the NMC Server.

Like many applications, NetWorker has separated the GUI management from the core software functionality. This has multiple architectural advantages, such as:

  • The GUI and the Server functionality can be developed with more agility
  • The GUI can be used to administer multiple servers
  • The functional load of providing GUI services does not impact the core Server functionality (i.e., providing backup and recovery services).

While you could, if you wanted to, deploy a NMC Server for each NetWorker Server, it’s by no means necessary, and so it’s reasonably common to see a single NMC Server deployed across multiple NetWorker servers. This allows centralised reporting, management and control for backup administrators and operators.

Operational Layer

At the operational layer we have what is defined as a NetWorker datazone. In fact, at the operational layer we can have as many datazones as is required by the business, all subordinate to the unified Enterprise Layer. In simple parlance, a NetWorker datazone is the collection of all hosts within your environment for which a single NetWorker server provides backup and recovery services. A high level view of a NetWorker datazone resembles the following:

NetWorker Datazone (Operational Layer)

The three key types of hosts within a NetWorker datazone are as follows:

  • Server – A host that provides backup and recovery services (with all the associated management functions) for systems within your environment. There will either be (usually) a single NetWorker server in the datazone, or (in less common situations), a clustered pair of hosts acting as an active/passive NetWorker server.
  • Client – Any system that has backup and recovery services managed by a NetWorker Server
  • Storage Node – A host with access to one or more backup devices, either providing device mapping access to clients (I’ll get to that in a moment) or transferring backup/recovery to/from devices on behalf of clients. (A NetWorker server, by the way, can also function as a storage node.) A storage node can either be a full storage node, meaning it can perform those actions previously described for any number of clients, or a dedicated storage node, meaning it provides those services just to itself.

With such a long pedigree, NetWorker (as described above) is capable of running in a classic three-tier architecture – the server managing the overall environment with clients backing up to and recovering from storage nodes. However, NetWorker is equally able to ditch that legacy mode of operation and function without storage nodes thanks to the benefits of distributed deduplication in tightly integrated systems such as Data Domain and CloudBoost and ClientDirect. That being said, NetWorker still supports a broad range of device types ranging from simple tape through to purpose built backup appliances (Data Domain), Cloud targets, VTL and plain disk. (In fact, I remember years ago NetWorker actually supporting VHS as a tape format!)

ClientDirect, which I mentioned previously, is where clients can communicate directly with network accessible devices such as Data Domain deduplication storage. In these cases, both the NetWorker server and any storage node in the environment is removed from the data path – making for a highly efficient and scalable environment when distributed deduplciation is taking place. (For a more in-depth understanding of the architectural implications of Client Direct, I suggest you review this earlier post.)

Within this operational layer, I’ve drawn the devices off to the side for the following reasons:

  • Devices can (and will) provide backup/recovery media to all layers in the NetWorker datazone – server, storage nodes (if deployed) and individual clients
  • Devices that support appropriate multi-tenancy or partitioning can actually be shared between multiple NetWorker datazones. In years gone by you might have deployed a large tape library with two or more NetWorker servers accessing virtualised autochangers from it, and more recently it’s quite easy to have the same Data Domain system for instance being accessed by multiple NetWorker servers if you want to.

Wrapping Up

The NetWorker architecture has definitely grown since I started using it in 1996. Back then each datazone required completely independent licensing and management, using per-OS-type GUI interfaces or CLI, and it was a very flattened architecture – clients and the server only. Since then the architecture has grown to accommodate the changing business landscape. My largest NetWorker datazone in 1996 had approximately 50 clients in it – these days I have customers with well over 2,000 clients in a single datazone, and have colleagues with customers running even larger environments. As the NetWorker Usage Survey has shown, the number of datazones has also been growing as businesses merge, consolidate functions, and take advantage of simplified capacity based licensing schemes.

By necessity then, the architecture available to NetWorker has grown as well. Perhaps the most important architectural lesson for newcomers to NetWorker is understanding the difference between the enterprise layer and the operational layer (the individual datazones).

If you’ve got any questions on any of the above, drop me a line or add a comment and I’ll clarify in a subsequent post.