Ever since I joined my new company two years ago, ZFS has been part of my work every day. And every day, I am amazed how great it is. So naturally, I wanted to move my existing Linux Mint 18 installation to boot off of ZFS. Why, you may wonder? Well that’s easy. Because now I can snapshot my root file system, I can roll back if I need to, and I can restore individual files in a heartbeat.
It took a bit of fiddling in the beginning, but once you know how it works, it’s a piece of cake. This short post shows you how to move your existing Linux installation to ZFS on root (preferably Ubuntu 16.04+ based, may work for others).
Content
1. Requirements and Assumptions
For this example, we’re assuming that you’re running an Ubuntu 16.04+ based operating system (Ubuntu, Kubuntu, Xubuntu, Lubuntu, Linux Mint, …), and that you have another partition (to be used for the ZFS pool) with at least the amount of disk space that your current root partition has. If that is not the case, you can follow step 3 to make room.
2. This example: Kubuntu 16.04
In this example, I’ll be using a regular Kubuntu 16.04 installation. This could be your laptop or workstation, with any derivative of Ubuntu installed.
3. Creating the ZFS pool partition (optional)
Optional (if you have no spare partition): As you can see in the screenshot above, lsblk shows us that the machine has no extra partition for a new ZFS pool (yet), so we’ll have to split the root partition to make room for the ZFS pool partition. If you already have a disk/partition that you want to use as a pool, you can skip this step.
To modify your root partition, you need to first boot an Ubuntu/Kubuntu/Mint Live CD/USB, so you can fiddle with the partition layout in gparted or the KDE Partition Manager. I have opted to boot into an Ubuntu Live CD (to make it clear that this is not your live system).
To split your root partition,
- Open gparted or KDE Partition Manager
- Select your root partition, right click and select “Resize”, enter the target size and hit “Apply”
- Then create a new (unformatted) partition in the empty space after the old root partition
- Click Apply (Warning: Back up your system before you do this. There is a chance of data loss!)
It should look something like this (if you’re using gparted):
After you hit “Apply” and after you rebooted into your actual system, your disk layout changed and you have a new partition (here: /dev/sda3). This is the partition we will use to create the ZFS pool:
4. Creating the root ZFS pool, and copying your OS
Now that we have a free partition for our new pool, let’s first install ZFS (if you don’t have it already):
1 |
$ apt install zfs-dkms zfs-initramfs |
Then, identify the newly created partition and create the root pool:
1 2 3 4 5 6 7 |
$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 20G 0 disk ├─sda1 8:1 0 8.8G 0 part / ├─sda2 8:2 0 1K 0 part ├─sda3 8:3 0 9.2G 0 part # <<<<< Our new partition! It'll likely be different for you! └─sda5 8:5 0 2G 0 part [SWAP] |
Now create the root pool and the root dataset you want to use. I usually call my pools tank and put my systems in tank/os/<osname>, but this is really up to you:
1 2 3 4 5 6 |
$ zpool create tank /dev/sda3 # Warning, choose your partition here! ZFS does not forgive typos! $ zfs create tank/os $ zfs create tank/os/kubuntu1604 # Alternatively, use the by-id symlink like so: # $ zpool create tank /dev/disk/by-id/.... |
After you created the new dataset, it is auto-mounted by ZFS at /tank/os/kubuntu1604. You may now start copying your running system to this location:
1 |
$ rsync -a --one-file-system / /tank/os/kubuntu1604/ |
This may take a while, depending on your disk I/O speed and load; and it may even throw some warnings. Usually these warnings can be safely ignored if they are about files in /proc.
5. Make system bootable
Now that we have the dataset of our new root file system (tank/os/kubuntu1604), we need to tell our bootloader (here: grub2) that it should boot into the new system. To do that, we need to chroot into the system and run update-grub and grub-install:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
$ cd /tank/os/kubuntu1604 $ mount --bind /dev dev $ mount --bind /proc proc $ mount --bind /sys sys $ mount --bind /run run $ chroot . # You are now in the new system. Your current "/" points to "/tank/os/kubuntu1604" # All the following commands will apply only to that system. $ update-grub Generating grub configuration file ... Warning: Setting GRUB_TIMEOUT to a non-zero value when GRUB_HIDDEN_TIMEOUT is set is no longer supported. Found linux image: /boot/vmlinuz-4.4.0-31-generic Found initrd image: /boot/initrd.img-4.4.0-31-generic Found memtest86+ image: /os/kubuntu1604@/boot/memtest86+.elf Found memtest86+ image: /os/kubuntu1604@/boot/memtest86+.bin Found Ubuntu 16.04.1 LTS (16.04) on /dev/sda1 done $ grub-install /dev/sda # Note: This should be your root disk (not partition!) Installing for i386-pc platform. Installation finished. No error reported. $ exit $ reboot |
That’s it. During the reboot, if you press ESC, you can see that the grub menu will show the new system as “Ubuntu”, and the old system as “Ubuntu 16.04.1 LTS (16.04) on /dev/sda1”. That means if you’re having trouble with your ZFS on Root based system, you can always go back to your old system:
Once you booted, you can verify that you’re indeed running on ZFS with zfs list or mount:
6. Snapshot your system regularly, accessing snapshots
You can (and should) now snapshot your root file system, and you can even roll back if you really want to. The first thing you should do is set up a cronjob to regularly snapshot your system:
1 2 3 4 |
$ crontab -e # as root! # Then add a line line this: 0 20 * * * zfs snapshot tank/os/kubuntu1604@`date +\%y\%m\%d\%H\%M` |
Once you’ve done that, you can always restore individual files by accessing the ZFS directory in /.zfs/snapshot/<snapshotname>, like this:
1 |
$ cp /.zfs/snapshot/1612221520/etc/hosts /etc/hosts |
Or, if you really messed up, you can also completely roll back to an old snapshot (use with caution, as running programs may expect the disk contents to be different!):
don’t forgot to export the pool before reboot!
What about expanding the zfs partition into the old root partition after the transfer?
The above procedure does not work if “/boot/efi’ is mounted to a unique partition.
grub-install fails with error saying EFI directory not found.
In my case I had to issue the following before “chroot”: (zpool1 used instead of tank)
cd /zpool1/os/ubuntu/boot
mount –bind /boot/efi efi
cd /zpool1/os/ubuntu
I have followed this guide to migrate a clean install of Kubuntu 16.04, and I would like to know if it is safe to delete the old partition and merge the space into my zpool.
[NOTE] I am not worried about trying to go back to the old filesystem type, since if I need to do that, I have a Kubuntu Live Boot USB stick.
You’ll also need to fix permissions on /media/[username] for fusemount (e.g. mounting usb sticks, windows partitions), it shouldn’t be owned by root.
I read a lot of tutorials like this and no one mention how to do such with more than one device for ZFS and using RAID levels of ZFS.
Also most tutorial suposes you have only one disk, what about having more than ten disks, or simply just two (like notebooks with a hdd caddy where the dvd unit was).
ZFS has a big advantage on data redundancy, so having more than one disk is very apreciated. Not only with multi disk, also with multiple copies of data (parameter copies if i did not remember bad).
Also i must give thanks for not supossing the reader must know how ZFS is managed… idid not yet used anytime but i want to… like a good student i want to know as much as possible prior to start doing things.
Side note: I am having a lot of trouble to find a Linux Live CD with ZFS fully working (without the need of Internet connection, aka, no apt-get install, etc) maybe caused by Linux Kernel and ZFS license been incompatible… also i am having problems to find a good how to update a LiveCD (booting from itself into RAM so disk can be ejected) to add full ZFS to it and re-create the ISO… a rescue disk is really important, and better if that disk do not requiere ineternet to be able to mount, fix errors, etc. on a full ZFS (including / on RaidZ ZFS).
root@qaz-box / # update-grub
bash: update-grub: command not found
Thanks for the documentation! I will try it this way when I find myself in a brave moment.
Could you add some information on how to safely expand the dataset to the now obsolete partition?
You could just “zpool add” the other partition. I would recommend you try this on a pool of sparse files first before trying it with the real system.
I’ve read your tutorial with great interest.
Would the procedure be different if you moved an ubuntu-18.04-installation to a zfs partition ?
Is it possible to dual-boot with windows ?
Yours sincerely
Johannes
Good post, thank you. Could you insert a warning, that boot partitions have to be prepared first, before creating zfs pool and datasets on the destinantion disk when starting with whole new disk?
Without proper boot partition(s) grub cannot be installed in the last step. How to partition see here in step 2.: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS
I”ve combined steps from your guide with the official https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS and now I have working Ubuntu 18.04 server on whole new SSD disk, great!
Only the UEFI grub install didn’t work :(
Usually grub doesn’t need a boot partition at all if it’s a legacy BIOS boot with enough room between before the first partition. If you want to be 100% sure that grub will have enough space and that UEFI boot will also work, you can create a EFI partition and a Grub BIOS partition like I did here: https://blog.heckel.xyz/2017/05/28/creating-a-bios-gpt-and-uefi-gpt-grub-bootable-linux-system/
If you want to dual boot windows 10 UEFI and ubuntu UEFI (ZFS), I had to explicitly set my mount point after following Dani Kalmar’s comment and before rebooting. E.g. zfs set mountpoint=/ tank/os/kubuntu1804
Hey, I did all this but when I rebooted it put me in the old ext4 partition instead of into zfs. Any ideas?
Did your “update-grub” see the new system? When you press ESC during boot, do you see the new system in the menu?
Thanks. This came in handy. I had to change a few things, but overall it worked great. In fact, the same general procedure (with quite a few changes) can be used to setup a raspberry pi with raspian to boot from F2FS
The grub-install fails.
root@zfs1:/# grub-install /dev/sda
Installing for i386-pc platform.
grub-install: warning: this GPT partition label contains no BIOS Boot Partition; embedding won’t be possible.
grub-install: error: filesystem `zfs’ doesn’t support blocklists.
root@zfs1:/#
The current boot drive is /dev/sdb running Ubuntu 18.04 server.
The /dev/sda drive is unused.
I have attempted to put down a new gpt partition table and define a BIOS boot partition of 200MiB, but when I create the zpool on /dev/sda it wipes everything out.
Dear Mr. Heckel:
1. I followed the instructions given by you except for the following deviations:
1.1. I applied your procedure to Linux Kubuntu 18.04.
1.2. I created a very large spare partition while doing a new Kubuntu 18.04 installation. This partition was larger than entire space occupied by the new Kubuntu 18.04 installation. I set up ZFS in this partition strictly according to your instructions except that I copied all the partitions (/, /var, /usr, and /home) in the Kubuntu 18.04 (all installed using Btrfs) to the ZFS partition).
2. The transfer to ZFS went well. However, after I restarted the system, I used the default file browser in Kubuntu, namely, Dolphin, to open the folder “SYSTEM” (I called the pool on my computer “SYSTEM” instead of “tank” as you have done) at which point Dolphin reported following error:
2.1. “An error occurred while accessing ‘Home’, the system responded: The requested operation has failed: Error mounting /dev/sdb7 at /media/sayeedfamily/SYSTEM: unknown filesystem type ‘zfs_member'”.
3. Would you happen to know why this error was reported? Should I be concerned about the ZFS installation not working correctly?
4. Your advice and solution, if one is required, will be much appreciated.
5. If you require any clarification, please feel free to contact me.
6. I await your reaponse.
Yours truly,
Kabeer Sayeed
Canada > Ontario