However, bear in mind those examples in each case were of full backups – the filesystem walk time was always prohibitive, even if every file was being accessed.

No reason why the other backup vendors cant use the same technology to backup NAS.

For the most part, there’s little that backup vendors can do at the filesystem layer in order to resolve these problems. They appear on just about every operating system and filesystem type, regardless of backup product used – it’s reasonably indicative of improvements that need to be made in most operating/filesystems to do with large sequential walks. (Indeed, I use the “linked list vs tree structure” scenario in my book to help explain dense filesystem backup issues.)

While there are some workarounds (e.g., block level backups), these often present their own disadvantages that make them no better (and indeed, depending on requirements, sometimes worse) than the original problem. For example, block level backup renders high speed backup, but at a cost that file level recovery must be done via cache reconstruction, and performing cache reconstruction of files that were even moderately fragmented at the source can be terribly slow.

We’ve run similar tests before, with more concentration on small files, and differing types of directory structures (wide versus deep, for one, as well as number of files in a given directory), and found that the ‘break’ point on Windows is typically between 25 and 50K average file size.

Smaller than that and we saw a truly marked degradation in backup speed–very similar to your results, actually–normally down to 100′s of K per second.

We first ran those tests under Windows NT, and haven’t seen significant changes with newer versions; the issue resolves around kernel time for user-land processes to do file table lookups, from what I remember.

–Dave