iTWire – Aussie storage growth above average: Gartner

 Tidbit, Tidbits  No Responses »
Mar 172011
 

There’s a report over at iTWire that has two highly pertinent details. (iTWire – Aussie storage growth above average: Gartner.)

The article is about how Australian spending on storage is growing faster than the rest of the world (IMHO that’s just further proof of how helpful the government stimulus package was), and has two particular points of interest.

First:

The big winner was EMC, which saw its revenue from the region grow from $US533.9 million to $US716.0 million. Most other vendors also saw improved revenues…

That doesn’t surprise me. As an employee of an EMC partner, I know EMC have been very strongly pushing in the Australian market over the last 12 months. I fully believe that other vendors have been pushing hard and (for the most part) achieving good results, but EMC has had a really solid story during this spending cycle, and it’s been paying off – time and time again.

What really didn’t surprise me though was the “but” following that above quote:

…but the biggest loser was Oracle. In 2009, Sun had $US134.4 million revenue in 2009. Now part of Oracle, it only recorded $US82.1 million revenue in 2010

Since the Oracle acquisition of Sun, every single one of my customers who had previously been a large Sun customer has either been resolutely turning away from the vendor, or eyeing them with firm displeasure. Why? Oracle’s higher prices for maintenance and product has had a significant impact on the budgetary options available to one of Sun’s biggest previous customer bases – the educational market. (This, for what it’s worth, is why I penned the article last year, “RIP Solaris“.)

While I’m not normally one to put much stock in analyst reports, this one seems to gel with what I’ve been seeing for the past 12 months.

 Posted by Preston de Guise at 08:33  Tagged with: , , ,

RIP Solaris

 Aside, General Technology  9 Responses »
Apr 222010
 

Having observed Oracle’s strategy now for a while since the acquisition of Sun, and many discussions with a quite a few customers, it’s clear that Oracle is gunning for the “entire vertical” model. Quite simply, their primary focus appears now to be selling an entire solution to a company, starting with the low level storage, and extending all the way to the application tier.

There’s nothing wrong with that kind of strategy – so long as it doesn’t alienate companies who want to buy piecemeal.

Oracle however are most definitely alienating the educational institutions. Several institutions I deal with now have policies that require end-of-life Sun kit to be replaced with comparable Linux or Windows solutions, unless an absolute rock-solid business case can be built.

With educational support rapidly eroding, Solaris as a tent-pole Unix platform is well and truly dead.

[Original post below, 22-04-2010]

I was told yesterday that one of the changes Oracle has wrought at Sun is the killing of all educational discount programmes. Apparently while they’re still listed on the Sun websites, they’re unavailable. Another fascinating change is collapsing support programmes from multiple levels of varying cost to one single level.

From a Unix perspective, I grew up on Solaris, and I’ve always seen the Unix world split into two camps. On one side you’ve got HPUX and AIX, dominated by ‘smit’, and the other side was led by Solaris, with Tru64 close behind.

The HPUX and AIX approach to Unix has always been an interesting one. It’s about rigid controls, and it’s appealed to formal environments and procedure-oriented enterprises. I still remember a comment made by a senior BHP IT manager when I still worked in Newcastle. I’m modifying it slightly so that this article doesn’t get mired down in faeces:

In BHP IT Melbourne if you want to go the toilet, you hold a meeting about it. In BHP IT Wollonging if you want to go to the toilet, you consult a huge procedure about doing it. In BHP IT Newcastle, you do it in the corridor while you keep going with your work.

I know, it’s a wee crass, but as much as anything I saw it as a statement about the platforms in use at the time – particularly Newcastle vs Wollongong. You see, the Newcastle Unix team was dominated by Solaris, with a few Tru64 boxes and a couple of HPUX boxes (hell, even a couple of AT&T boxes). The Wollongong team was dominated by AIX.

What it comes down to is that the administrator mentality behind Solaris is all about free thinking. Not in a hippy sort of way, but in a “hey, here’s a Unix. Do whatever the hell you like with it” sort of way. It’s the result of having year after year of students at Universities using Solaris because that was the cheapest and most flexible Unix for the Universities to deploy. In this case by “free thinking” I’m not referring to any OSS ideals, but to the notion that it’s a full Unix that isn’t constrained by what the vendor feels you should do with it.

I’m taking a roundabout way of getting there, but I think the greatest damage Oracle is doing to Solaris is making the entire Sun platform less attractive to educational markets. People tend to stick to the platforms they learn at University – at least for a while – and so the overall Sun educational discount programme has always been a very clever one: hook them while they’re still learning, teach them that they can use the platform for whatever they want, and they’ll keep coming back to it once they’re out in the work force. This becomes a very powerful drag-sales method. Graduates come out of University looking for jobs on the platforms they have experience with. As they become team leaders or middle managers, they continue to advocate those platforms unless there’s a very strong reason to void the emotional attachment they have to a platform. Net result? The discounts to the educational market are recouped through the full prices in the commercial market. (I’d suggest that at a desktop/laptop level, Apple has been working at this now for some time, and it’s starting to build momentum that Microsoft will have trouble halting.)

Oracle clearly don’t understand that drag-sales model as it has applied with Sun. By killing off educational discount programmes for the entire Sun platform and making the Solaris operating system more costly to install and support, they’re eroding the “use it for anything” base market and mind share that has always been so critical to the continuing popularity of Solaris. I’m sure HP and IBM are both very pleased with this new direction that Oracle is taking. Oracle is making the jobs of HP and IBM sales people that much easier.

If Oracle lock in their current strategy and force this change, Solaris as a tent-pole Unix platform is dead. Somehow, I doubt Oracle would even care.

 Posted by Preston de Guise at 07:03  Tagged with: ,

nsrnmo[291]: -l: not found

 Features, NetWorker, Scripting, Support  2 Responses »
Jan 302010
 

Yesterday I experienced one of those weird NetWorker issues that is such an odd combination of factors that I felt it had to be discussed.

Here’s the scenario. A customer was:

  • Previously running NetWorker 7.4.2 on their backup server.
  • Upgraded the server to 7.5.1.
  • Had a bunch of Windows clients and one Unix client.
  • The Unix client was configured for filesystem backups and Oracle backups.
  • All clients were running 7.4.2(ish). The Oracle module was 4.5.
  • Once the upgrade was done, Unix filesystem backups continued to work but the Oracle backups would fail with:
client:RMAN:/path/to/script.rman 1 retry attempted
client:RMAN:/path/to/script.rman off
client:RMAN:/path/to/script.rman /path/to/nsrnmo[291]: -l:  not found
client:RMAN:/path/to/script.rman nsrnmostart returned status of 127
client:RMAN:/path/to/script.rman /path/to/nsrnmo exiting.

My first thought when a colleague asked me to have a look at it was that somehow there was enough of a slight enough incompatibility between 7.5.x and NMO 4.5 that some argument carried over from an earlier version of NMO was causing problems with talking to a 7.5.x server. This wasn’t the case. (Yes, I knew that the two versions are meant to be compatible, and when I’ve installed and used them they have been, but that doesn’t mean you can’t have one single setting somewhere that tickles a coding error across versions.)

I went back and forth with a few other checks with the customer, noting that there were various issues reported in the NMO applogs, but none specific enough to nail the problem. So since everything looked OK I agreed with the customer that a WebEx would probably help us solve the issue faster.

Even though the customer had given me the client resource, I hadn’t found anything wrong with the backup command or the save set name, so out of curiosity I’d asked the customer when we started the WebEx to show me the client details. The saveset looked fine, so we jumped across to the backup command, and that also looked fine. But then, underneath the backup command, there was the “save operations” field, and in that save operations field held:

VSS:*=off

It hadn’t been recently added. It had been there since before the upgrade, and before the upgrade the backups had been working. But as we know, on pre-VSS Windows systems invoking that will cause backup failures, so I asked the customer to remove that entry and start the backup. Neither of us really thought that this would solve the problem, given the filesystem backups were still working, but lo and behold, with that removed the Oracle RMAN backups started properly working.

In retrospect, this of course was definitely the problem, but working it out was a bit more challenging. The reason was that the configuration shouldn’t have worked under a NetWorker 7.4.x server either, but for some reason it did. The 7.4.x NetWorker server was likely not sending through the VSS directive to the Unix client and the Unix Oracle module, but having upgraded to 7.5.x, the new install stopped “filtering the error” and started causing the problem to manifest. Or alternatively, 7.4.x and 7.5.x both send the save operations setting, but just differently enough to be dangerous.

I wouldn’t exactly say this was NetWorker’s fault – those VSS options are only designed for use with Windows 2003 and higher clients, and I’d guess that the VSS:*=off was just applied to every single client on the customer site without considering the 1 x Unix client.

In retrospect, the following line now completely makes sense:

client:RMAN:/path/to/script.rman off

That was our only “hint” as to the cause of the problem in the savegroup completion. It wasn’t enough by a long stretch. Sometimes, and this is the challenging bit – sometimes you can have configuration errors even if you haven’t changed the actual resource configuration. Different versions of NetWorker will react differently to an incorrect configuration – so the upgrade didn’t cause the problem, it just allowed the problem to appear.

 Posted by Preston de Guise at 05:55  Tagged with: , , , ,

Oracle channels and NetWorker parallelism

 NetWorker, Scripting  2 Responses »
Jun 302009
 

If you’re backing up Oracle with the NetWorker module/RMAN, there are an extremely large number of options you can choose from. RMAN, after all, is a complete backup/recovery system in and of itself, and so when you combine RMAN and NetWorker you, well, find yourself swimming in options.

One such option is the allocate channel command within RMAN. If you’ve not seen a basic RMAN script before, I should put one here for your reference:

connect target rman/supersecretpassword@DB10;

run {
 allocate channel t1 type 'SBT_TAPE';
 send 'NSR_ENV=(NSR_SAVESET_EXPIRATION=14 days,
       NSR_SERVER=nox,NSR_DATA_VOLUME_POOL=Daily)';

 backup format '/%d_%p_%t.%s/'
 (database);

 backup format '/%d_%p_%t_al.%s/'
 (archivelog from time 'SYSDATE-2');

 release channel t1;
}

You’ll note that one of the first commands used in the script is the allocate channel command. This effectively tells RMAN to open up a line of communication with NetWorker. Now, you can consider an RMAN channel to be a unit of parallelism in NetWorker parlance. Thus, if you want to backup (larger) databases with higher levels of parallelism, you need to allocate more channels.

In many NetWorker/Oracle scenarios, the NetWorker administrator has very little, if no, control over the construction and the configuration of the RMAN script. (The introduction of v5 of the module may change this.)

As a consequence, there’s often a reduced level of communication between the NetWorker administrator and the Oracle DBA which can result in reduced performance or scheduling conflicts. One particular issue that can occur though is that the Oracle DBA, eager to have the database backed up as quickly as possible, will throw a lot of allocate channel commands in. That little script above may become something such as say:

connect target rman/supersecretpassword@DB10;

run {

 allocate channel t1 type 'SBT_TAPE';
 allocate channel t2 type 'SBT_TAPE';
 allocate channel t3 type 'SBT_TAPE';
 allocate channel t4 type 'SBT_TAPE';
 allocate channel t5 type 'SBT_TAPE';
 allocate channel t6 type 'SBT_TAPE';
 allocate channel t7 type 'SBT_TAPE';
 allocate channel t8 type 'SBT_TAPE';

 send 'NSR_ENV=(NSR_SAVESET_EXPIRATION=14 days,
       NSR_SERVER=nox,NSR_DATA_VOLUME_POOL=Daily)';

 backup filesperset 4
 format '/%d_%p_%t.%s/'
 (database);

 backup format '/%d_%p_%t_al.%s/'
 (archivelog from time 'SYSDATE-2');

 release channel t1;
 release channel t2;
 release channel t3;
 release channel t4;
 release channel t5;
 release channel t6;
 release channel t7;
 release channel t8;
}

However, there’s a catch to lots of channels being allocated – channel allocation has no bearing on or is in any way impacted by NetWorker client parallelism. You see, the NetWorker client instance has a single saveset – the RMAN script name (or equivilant thereof, when using the Wizard in v5). Thus, to NetWorker, any Oracle client instance only has one saveset. Thus, that client parallelism will not affect the number of channels that can be allocated, but instead the number of simultaneous instances of the client that can be initiated.

The net result? Consider a client with parallelism of 4, that has 6 databases to be backed up. This would have 6 client instances, one per database. Assuming they’re all in the same group*, then at any one instance NetWorker will only allow the backup for 4 of those instances to be running. However, each instance, or each Oracle RMAN script, can start as many channels as it wants. If each RMAN script has been “tweaked” to allocate say, 8 channels like the above script example, this would mean that backing up 4 instances simultaneously would potentially see the client trying to send 32 savesets simultaneously to NetWorker.

Thus, if using multiple Oracle channels in RMAN backups with NetWorker, and particularly if backing up multiple Oracle databases simultaneously, it’s very important to have the NetWorker administrator and the DBA responsible for the RMAN scripts to communicate effectively and plan overall levels of parallelism/number of channels to avoid swamping the NetWorker server, swamping the network, or swamping the Oracle server.

–
* There are other considerations for starting multiple Oracle backups on the same machine and at the same time. In other words I’m not necessarily calling this best practice, just using an example.

 Posted by Preston de Guise at 07:25  Tagged with: , , ,

NetWorker 7.5 and Oracle Module 5

 Databases, Features, NetWorker  19 Responses »
May 072009
 

Since I have more than a passing interest in databases, I always try to keep appraised of the Oracle module for NetWorker. It therefore surprised me a few days ago to see that v5 of the module had been released in March. I guess my excuse is that March was an insanely busy month for me between work and travel. (Well, that’s my excuse, and I’m sticking to it.)

So yesterday I downloaded v5 of the module (for Linux), and spun it up. This is a version I really, really like.

Now, here’s a few bullet points before I get to the most impressive feature:

  • No longer supports Oracle 9i or lower; if you want older, unsupported versions of Oracle you have to use an older version of the module.
  • Requires features that exist only in Networker 7.5.x as the underlying client.
  • Must have the NetWorker regular client installed and running in order for the module software to correctly install and activate,
  • Can work with the 7.4.x NetWorker server with the exception that what I’m about to describe below doesn’t work with a 7.4 server.
  • Now has a client configuration wizard that works within NMC and makes Oracle backup configuration a breeze.

Honestly, if you’re about to do a new NetWorker install into a site that has Oracle, skip everything else and install 7.5.1. I.e., this is one of these compelling reasons for 7.5.x.

The Oracle client configuration wizard is integrated into NMC’s wizards. Right-click on a client in the configuration panel, choose “Client Backup Configuration -> New”, and you’re off and running:

Oracle Client Configuration Step 1

Oracle Client Configuration Step 1

Oracle Client Configuration Step 2

Oracle Client Configuration Step 2

Note that you won’t reach this point if you’ve disabled ‘nsrauth’ authentication on the backup server. I had done so on my lab server as a test on Monday, and spent half an hour trying to work out a … rather inexact … error message.

Oracle Client Configuration Step 3

Oracle Client Configuration Step 3

Oracle Client Configuration Step 4

Oracle Client Configuration Step 4

The above step is where things get fun. Note that if you are given these details, you don’t even need to log onto the client to setup an nsrnmo script any longer. This is the start of A Really Good Thing.

Also, I should note, in the above screen shot, because I was using a temporary database installed just for a few tests and I was in a rush, I used the sys account for connecting to the target database. No, you shouldn’t ever do that – create a backup user and use that account, please.

Note that Oracle, and the Oracle Listener, must both be running on the client in order to clear the above step.

After the above, we then start to get into the ‘regular’ client configuration options:

Oracle Client Configuration Step 5

Oracle Client Configuration Step 5

Oracle Client Configuration Step 6

Oracle Client Configuration Step 6

Oracle Client Configuration Step 7

Oracle Client Configuration Step 7

This summary screen shows you what you’re going to get as far as the configuration is concerned – including the RMAN script that has been automatically generated for you:

Oracle Client Configuration Step 8

Oracle Client Configuration Step 8

Confirmation of sweet success:

Oracle Client Configuration Step 9

Oracle Client Configuration Step 9

The finished client in NMC:

Oracle Client Configuration Step 10

Oracle Client Configuration Step 10

Once configured, you’re ready to start backing up straight away. Honestly, it couldn’t be simpler.

As a closing note, I know some other backup products have had Oracle backup wizards for some time, so I’m not claiming EMC is the first with this style of setup, but I do think it’s a great feature to see included now.

 Posted by Preston de Guise at 14:07  Tagged with: ,
© 2012 The NetWorker Blog Suffusion theme by Sayontan Sinha