{"id":1086,"date":"2009-09-30T05:32:39","date_gmt":"2009-09-29T19:32:39","guid":{"rendered":"http:\/\/nsrd.wordpress.com\/?p=1086"},"modified":"2018-12-12T15:37:48","modified_gmt":"2018-12-12T05:37:48","slug":"client-side-compression-and-saveset-sizes","status":"publish","type":"post","link":"https:\/\/nsrd.info\/blog\/2009\/09\/30\/client-side-compression-and-saveset-sizes\/","title":{"rendered":"Client side compression and saveset sizes"},"content":{"rendered":"

While it turned out to be unrelated, a recent customer question made me think back to the impact of client side compression on the reported saveset size, and for the life of me I couldn’t remember how client side compression affected saveset size<\/em> reporting.<\/p>\n

Of course, it’s relatively simple to test. So I created a 1GB file on my backup server using:<\/p>\n

# dd if=\/dev\/zero bs=1024k count=1024 of=\/root\/test.dat<\/pre>\n
Next, to test, I configured a client entry with a saveset of just ‘\/root\/test.dat’, and set the backup running without any client side compression. The savegroup completion email showed the sort of size you’d expect:<\/p>\n
--- Successful Save Sets ---\n\n* tara.pmdg.lab:Probe savefs tara.pmdg.lab: succeeded.\n tara.pmdg.lab: \/root\/test.dat     level=full,   1048 MB 00:00:13      3 files\n tara.pmdg.lab: index:tara.pmdg.lab level=full,     3 KB 00:00:00      4 files\n tara.pmdg.lab: bootstrap          level=full,     91 KB 00:00:01    177 files<\/pre>\n
The next step was to enable client side compression. Being lazy and not wanting to launch NMC, I created \/root\/.nsr with the following content:<\/p>\n
<< . >>\ncompressasm: test.dat<\/pre>\n
With the backup re-run, I got the conclusive evidence that the saveset size reported is the data written to media<\/em> (or transferred from the client) not the size of the data itself:<\/p>\n
--- Successful Save Sets ---\n\n* tara.pmdg.lab:Probe savefs tara.pmdg.lab: succeeded.\n* tara.pmdg.lab:\/root\/test.dat 66135:save: NSR directive file (\/root\/.nsr) parsed\n* tara.pmdg.lab:\/root\/test.dat 66135:save: NSR directive file (\/root\/.nsr) parsed\n tara.pmdg.lab: \/root\/test.dat     level=full,    124 MB 00:00:07      3 files\n tara.pmdg.lab: index:tara.pmdg.lab level=full,     5 KB 00:00:00      5 files\n tara.pmdg.lab: bootstrap          level=full,    102 KB 00:00:01    186 files<\/pre>\n
So the next question is – is this a good thing?<\/p>\n
The answer is a little fluid. The correct answer I think is that both sizes should be recorded. Clearly for the purposes of backwards compatibility, current sizing values need to continue to report the data written to media. However, logically, there is significant merit in adding another field to the database – e.g., clsize<\/em> that would report the amount of data the client reads<\/em> for the backup. This would save a lot of hassle. (The “totalsize” field is not used for this, by the way.)<\/p>\n
In the meantime, we just have to keep in mind that the size reported by mminfo, the savegroup completion, etc., is the size written to media<\/em> – or if you will the size transferred from the client to the storage node<\/em>.<\/p>\n","protected":false},"excerpt":{"rendered":"
While it turned out to be unrelated, a recent customer question made me think back to the impact of client…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[11,16],"tags":[221,242,854,858,955,1012],"class_list":["post-1086","post","type-post","status-publish","format-standard","hentry","category-features","category-networker","tag-client-compression","tag-compression","tag-savegroup","tag-saveset-size","tag-sumsize","tag-totalsize"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pKpIN-hw","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/1086","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/comments?post=1086"}],"version-history":[{"count":1,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/1086\/revisions"}],"predecessor-version":[{"id":7609,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/posts\/1086\/revisions\/7609"}],"wp:attachment":[{"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/media?parent=1086"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/categories?post=1086"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nsrd.info\/blog\/wp-json\/wp\/v2\/tags?post=1086"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}