Saturday, July 2, 2011

How to use HFS Compression for on the fly space saving

OSX Leopard 10.5 and Snow Leopard 10.6 bring a new element to Apple's venerable filing system, HFS.

They introduced a near seamless and transparent feature called HFS Compression where file are transparently compressed on disk, and then de-compressed into RAM as they are loaded.

Basically in the days of super fast CPUs and huge RAM its a tradeoff between disk I/O and other system constraints.

You could also say that it helps Apple offer laptops with smaller SSDs than their platter equivalents because regular files take up typically only about 50% of the space.

OSX manages the compression for system files and applications and speeds loading times because the disk I/O CPU tradeoff is favourable.

What about other files? Well you can play too. There are three options:

  1. Clusters which is a nice GUI built on Apple's compression technologies. It offers background compression, pausing and other nice features.

  2. Ditto, the file command line management utility can create HFS Compressed files

  3. afsctool also allows you to manage file compression, and to report on the results.

Here are some afstool tips:

printout afsctool options and switches

$ afsctool > afsctool.txt

compress a directory:

afsctool -c ~/directory

compress a file:

afsctool -c ~/directory/file.txt

report which files are not compressed:

afsctool -c -l ~/directory

confirm the results of compressing:

afsctool -v ~/directory

the command I use with some useful extra switches:

afsctool -c -m 5242880 -s 5 ~/directory

the "-m" sets the size of the largest file to be compressed. Since the uncompressed file must be held in RAM, there is a tradeoff here. Its the size in bytes, so 5242880 bytes = 5MB. 10485760 = 10MB.

The -s" sets the minimum percentage that must be saved by compressing. The stops you compressing files for which there is no benefit eg media files, jpegs, zips and rars.

Here's the result of compressing a directory of old files:

/Users/directory:
Number of HFS+ compressed files: 87
Total number of files: 98
Total number of folders: 10
Total number of items (number of files + number of folders): 108
Folder size (uncompressed; reported size by Mac OS 10.6+ Finder): 34519979 bytes / 34.8 MB (megabytes) / 33.2 MiB (mebibytes)
Folder size (compressed - decmpfs xattr; reported size by Mac OS 10.0-10.5 Finder): 27890264 bytes / 28.1 MB (megabytes) / 26.8 MiB (mebibytes)
Folder size (compressed): 27902203 bytes / 28.1 MB (megabytes) / 26.8 MiB (mebibytes)
Compression savings: 19.2%
Approximate total folder size (files + file overhead + folder overhead): 28242154 bytes / 28.2 MB (megabytes) / 26.9 MiB (mebibytes)

So I'm saving overall about 20% of the space required which isn't bad for a directory that contains a lot of images and PDFs.

Handily, HFS Compression being very low level plays nicely with pretty much everything else except anything to do with developer tools - for some reason it really doesn't like compressing .c source code files.

So you can combine this with Dropbox to save some local space for all those important files that you don't need very often.