{"id":10468,"date":"2021-07-19T06:52:29","date_gmt":"2021-07-18T20:52:29","guid":{"rendered":"https:\/\/nsrd.info\/blog\/?p=10468"},"modified":"2021-07-19T06:52:31","modified_gmt":"2021-07-18T20:52:31","slug":"networker-basics-s3-as-an-aftd-target","status":"publish","type":"post","link":"https:\/\/nsrd.info\/blog\/2021\/07\/19\/networker-basics-s3-as-an-aftd-target\/","title":{"rendered":"NetWorker Basics \u2013 S3 as an AFTD Target"},"content":{"rendered":"\n

There are a few traditional ways you can backup to an S3 target via NetWorker. If you’re in the cloud, you can spin up DDVE, and it’ll do all the work for you, writing to object storage. Regardless of whether you’re in-cloud or on-prem, you can spin up a CloudBoost appliance and write directly to object storage as well. Some cloud vendors even present VTL gateways, but of course, virtual tapes can be just as much a headache to manage as physical ones.<\/p>\n\n\n\n

NetWorker doesn’t have an ‘S3’ type device as it stands, but if you’re prepared to work with the Linux S3-FuseFS kernel extension, could you create an Advanced File Type Device (AFTD) writing directly to an object storage bucket?<\/p>\n\n\n\n

It turns out, yes.<\/p>\n\n\n\n

Critical Note 1<\/h3>\n\n\n\n

NOTE: Before I continue, I need to stress: this is not a documented NetWorker configuration. If you were to do something like this in your environment, I’d suggest extensive testing first. I’d also recommend considering one of the officially supported ways — CloudBoost or DDVE, for instance. In short: this falls far more into my geeky approach of “hey, can I do this?” than it does, “hey, should you do this?”<\/p>\n\n\n\n

Critical Note 2<\/h3>\n\n\n\n

In addition to the warning above, you also have to think of S3 limits – AWS S3 for instance has a maximum object size of 5TB<\/a>, and as each saveset in NetWorker is written as an atomic file, that would place an upper limit on what you can write. I’ve also not experimented with S3-FuseFS for writing multi-GB files (AWS says a single PUT can’t exceed 5GB \u2013 at that point you need to do it as a multi-part operation).<\/p>\n\n\n\n

But let’s experiment, anyway<\/h3>\n\n\n\n

So, with those warnings out of the way, let’s get cracking with some fun backup geekery.<\/p>\n\n\n\n

Starting with S3<\/h2>\n\n\n\n

To get this idea tested, I used the following:<\/p>\n\n\n\n