4

I installed on Debian Wheezy zfs-fuse file system and enabled compression gzip-9 on one dataset ("storage/backup"). When I check if the compression is enabled on this dataset, it shows YES:

$: zfs get compression storage/backup NAME PROPERTY VALUE SOURCE storage/backup compression gzip-9 local 

However, when I check the compression rate with du -ah or with sfx get compressratio no any compression can be seen.

All files, including well compressible ones (e.g. text files), take up exactly the same disk size as uncompressed ones:

$: zfs get compressratio storage/backup NAME PROPERTY VALUE SOURCE stor/backup compressratio 1.00x - 

Why does this situation occur?

Here some info from zfs get all about the dataset:

compressratio 1.00x - mounted yes - quota none default reservation none default recordsize 128K default mountpoint /storage/backup default sharenfs off default checksum on default compression gzip-9 local atime on default devices on default 
5
  • 2
    ZFS only compress new files only Commented Jul 3, 2014 at 8:39
  • All the files are new on that dataset. There are copied ones and newly created ones - neither ones are compressed. Commented Jul 3, 2014 at 8:52
  • Are the files already compressed? Commented Jul 3, 2014 at 8:55
  • @mtm No. E.g. I created a new file, entered 8000 instances of one character - 'du -ah' shows that this file has 8 Kb size. Commented Jul 3, 2014 at 9:00
  • Also, a newly created file containing 1 000 000 of one character has 980 kb storage size in 'du -ah'. Commented Jul 3, 2014 at 9:21

1 Answer 1

1

It looks like zfs-fuse will update the compressratio data every 30 seconds with limited IO occurring but there is another trigger to the update as background IO or really large files cause the data update to occur sooner.

I've put some test functions up on a gist. They require a clean (no files) file system that will start at 1.00x.

If the scripts pause forever on the first test then your compression counters are never updating and you have an issue with your install.

Running the scripts on a Debian wheezy box:

$ uname -a Linux zfs-fuse 3.2.0-4-686-pae #1 SMP Debian 3.2.54-2 i686 GNU/Linux 

Results in the following:

$ test_compression compress Testing [compress] Testing size [4096] Waited 0 seconds for [compressratio_is_one] 4096 bytes made up of 1*4096 blocks Waited 20 seconds for [compresstario_is_not_one] 1.12x Testing size [16384] Waited 30 seconds for [compressratio_is_one] 16384 bytes made up of 1*16384 blocks Waited 30 seconds for [compresstario_is_not_one] 1.53x Testing size [1048576] Waited 30 seconds for [compressratio_is_one] 1048576 bytes made up of 1*131072 blocks Waited 30 seconds for [compresstario_is_not_one] 31.44x Testing size [33161216] Waited 30 seconds for [compressratio_is_one] 33161216 bytes made up of 255*131072 blocks Waited 0 seconds for [compresstario_is_not_one] 202.31x 

You can reduce this, normally by about half by doing something intensive in the background which probably triggers the counter update.

In the background

$ while true; do touch somefile; rm somefile; done 

Then testing again:

$ test_compression compress Testing [compress] Testing size [4096] Waited 0 seconds for [compressratio_is_one] 4096 bytes made up of 1*4096 blocks Waited 5 seconds for [compresstario_is_not_one] 1.11x Testing size [16384] Waited 17 seconds for [compressratio_is_one] 16384 bytes made up of 1*16384 blocks Waited 17 seconds for [compresstario_is_not_one] 1.50x Testing size [1048576] Waited 16 seconds for [compressratio_is_one] 1048576 bytes made up of 1*131072 blocks Waited 10 seconds for [compresstario_is_not_one] 29.73x Testing size [33161216] Waited 0 seconds for [compressratio_is_one] 33161216 bytes made up of 244*131072 blocks Waited 0 seconds for [compresstario_is_not_one] 201.35x 

Of note, on FreeBSD the update happens ~ every 5 seconds:

$ test_compression giggidy/compress Testing [giggidy/compress] Testing size [4096] Waited 0 seconds for [compressratio_is_one] 4096 bytes made up of 1*4096 blocks Waited 4 seconds for [compresstario_is_not_one] 1.21x Testing size [16384] Waited 5 seconds for [compressratio_is_one] 16384 bytes made up of 1*16384 blocks Waited 5 seconds for [compresstario_is_not_one] 1.91x Testing size [1048576] Waited 5 seconds for [compressratio_is_one] 1048576 bytes made up of 1*131072 blocks Waited 5 seconds for [compresstario_is_not_one] 39.33x Testing size [33161216] Waited 5 seconds for [compressratio_is_one] 33161216 bytes made up of 1*131072 blocks Waited 4 seconds for [compresstario_is_not_one] 114.25x 

I will add a Solaris based example when I can get on a box.

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.