Recovering from a compressed ZFS root pool
Opensolaris and ZFS are great, but not without their limitations. A more recent addition is that Solaris can boot from a zpool, but not if you compress it or raidz it. Only single drives or mirrors, and no compression.
A good practice is to setup your root pool for the system, then setup a separate data pool for your files. Since we are using Solaris in this instance as a Virtual Machine host, this model works quite well.
Compression has some advantages. Aside from saving hard drive space, the compressed information means fewer IO writes and reads which can increase performance under the right conditions. Too often though, the warning to NOT compress the root pool is not present when you read the ZFS literature. Don’t compress your root pool!
For those who have done it, you may not realize your mistake until it is too late. If you turn on compression, it doesn’t compress existing files. Compression settings ONLY take effect when a file is written. That also means that turning compression off is not enough if you need to recover your boot drive.
If you have compression turned on and rewrite your boot files, you will end up with an error message on boot indicating file system corruption. Note the following instructions assume root access, add pfexec in front of each command or su – (opensolaris) as appropriate.
To recover, you will need to do the following:
1) Boot from the live Solaris CD
2) Open a terminal
3) Find your ZFS Pools: zpool list
4) Import your ZFS Pool (Only need the root pool, mine is called rpool): zpool import rpool
5) Turn off compression: zfs set compression=off rpool
6) List your mount points (you need the current mount under ROOT, mine is – rpool/ROOT/opensolaris – take note of the mount point, should be / ): zfs list
7) Setup a folder to mount to: mkdir /mnt/root
8 ) Change the mount point on you rpool: zfs set mountpoint=/mnt/root rpool/ROOT/opensolaris
9) Mount your ROOT: zfs mount rpool/ROOT/opensolaris
10) Move to your root pool: cd /mnt/root
11) Copy each system folder. You NEED to copy it so that a NEW copy is made. With compression off, it will write the new copy uncompressed. Moving it isn’t enough. Copy each folder, then move it back to the original name. Here is the command for the boot folder (notice we copy to _tmp, then move it back over the original): cp -R boot boot_tmp; mv boot_tmp boot
Copy each of these system folders in the /mnt/root folder:
– boot
– kernel
– system/objects
– etc
– platform
12) Now you have uncompressed copies of your system folders, unmount the file system: zfs unmount rpool/ROOT/opensolaris
13) Make sure to close the zpool so there is no corruptoin: zpool export rpool
14) Reboot – type the reboot command and remove the live CD, you should now be able to boot into your Solaris install with an uncompressed root pool.
NOTE: If you realize you compressed things before you reboot, you can do this all before you reboot and save yourself the trouble of mounting the live CD. The system/objects folder will not be able to copy some stuff, but you should be able to reboot correctly.
Hope this save some one some time. Use at your own risk.
Ray Pulsipher
