Install FreeBSD 11 on ZFS mirror (GPT or MBR)

Create mirrored ZFS filesystem

by ross at 03:25:00 on October 25, 2016

Create the pool

For 4k-aligned file system you have to kldload zfs and set sysctl vfs.zfs.min_auto_ashift to 12 as shown in the 4k article.

If you followed GPT instructions

# zpool create -m none -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache \
  system mirror /dev/gpt/system0 /dev/gpt/system1
# zfs set mountpoint=/ system

If you followed MBR instructions

# zpool create -m none -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache \
  system mirror /dev/label/system0 /dev/label/system1
# zfs set mountpoint=/ system

Here I have created a pool with the name "system" (it's not a special keyword, I just don't like the traditional "zroot") because that pool will hold the system. The name could be anything you invent.

Create filesystems

You can save yourself some time by using my little script to create traditional filesystem layout in ZFS pool.

In order to use the script bring network up (em0 is the network interface name):

If you have DHCP just do:
# dhclient em0
If you don't:
# ifconfig em0 up 192.168.10.20/24
# route add default 192.168.10.1
# echo "nameserver 192.168.10.1" > /etc/resolv.conf

Run the ZFS create script:

# cd /tmp
# fetch http://daemon-notes.com/downloads/assets/scripts/zfs-create.sh
# sh zfs-create.sh

Bring the network down (needed for installer to operate properly):

# killall dhclient
# ifconfig em0 down

Or create filesystems manually

The script above will do the following:

# zfs create -o compression=on -o setuid=off system/tmp
# chmod 1777 /mnt/tmp
# zfs create system/home
# zfs create system/usr
# cd /mnt/usr
# ln -s ../home home
# zfs create system/usr/local
# zfs create -o compression=on -o setuid=off system/usr/ports
# zfs create -o exec=off -o setuid=off system/usr/ports/distfiles
# zfs create -o exec=off -o setuid=off system/usr/ports/packages
# zfs create system/usr/obj
# zfs create -o compression=on -o exec=off -o setuid=off system/usr/src
# zfs create system/var
# zfs create -o exec=off -o setuid=off system/var/backups
# zfs create -o compression=on -o exec=off -o setuid=off system/var/crash
# zfs create -o exec=off -o setuid=off system/var/db
# zfs create -o exec=on -o compression=on -o setuid=off system/var/db/pkg
# zfs create -o exec=off -o setuid=off system/var/empty
# zfs create -o compression=on -o exec=off -o setuid=off system/var/log
# zfs create -o compression=on -o exec=off -o setuid=off system/var/mail
# zfs create -o exec=off -o setuid=off system/var/run
# zfs create -o compression=on -o setuid=off system/var/tmp
# chmod 1777 /mnt/var/tmp

Final steps

Make system bootable:

# zpool set bootfs=system system
# mkdir -p /mnt/boot/zfs
# cp /var/tmp/zpool.cache /mnt/boot/zfs/zpool.cache

Type exit now and the installation will begin.

 

Comments
Your guide is missing 3 very important steps:

a)You need to specify a mountpoint for your root pool
b)You need to export/import your pool after it has been created
c)You need to unmount ZFS in the end and specify the new mountpoints

Hope this helps you fix your guide!
-- gkontos
Saturday, January 14, 2012, 13:31:36
Thank you for your comment.

There are a lot of howtos exist about ZFS on root. Most of them are about Solaris or older versions of FreeBSD. When working on this guide I wanted to provide minimum commands and had no intention to make it portable.

It will work flawlessly on v9 of FreeBSD and probably won't work on other versions/OSes.

> a)You need to specify a mountpoint for your root pool
It is already set to "/" when the pool is created using the altroot option. I could set it to legacy as others do but "/" works too, so I ommited this step.

> b)You need to export/import your pool after it has been created
As far as I know this is needed to populate zpool.cache but again, the file is already here when the pool is created.

> c)You need to unmount ZFS in the end and specify the new mountpoints
I simply left unmounting to the shutdown process of the installer. If I run "zfs get mountpoint" on installed system I see that system/usr has mountpoint of /usr, system/var
-- ross
Saturday, January 14, 2012, 14:44:55
a) You use "-o altroot=/mnt".
b) If you don't then the zpool.cache doesn't get updated
c) that would show /mnt/system/usr etc.

My points were made for FreeBSD 9 and not any other OS
-- gkontos
Sunday, January 15, 2012, 14:39:09
You haven't tried to install using the guide, have you?
I have 4 servers and my home machines installed using my own guide at the moment.

> a) You use "-o altroot=/mnt".
I do. And because of that _temporary_ root is set to /mnt. This value isn't stored anywhere. At the same time real mount point (when rebooted into installed OS) is already set to "/".

> b) If you don't then the zpool.cache doesn't get updated
But is _is_ updated. The file specified in -o cachefile is created immediately. So no need to export/import.

> c) that would show /mnt/system/usr etc.
Again, because altroot is a temporary root, when you reboot system/usr _is_ /usr and so on.
-- ross
Monday, January 16, 2012, 4:27:39
ross my apologies!

It seems I was stacked with legacy type root mountpoints :)

Yes, it is working fine and I made a complete full cli installation. With your permission I would like to update my guide www.aisecure.net/2011/11/28/root-zfs-freebsd9/ for non legacy ZFS root mountpoints.
-- gkontos
Monday, January 16, 2012, 13:41:26
sure. glad it worked out.
-- ross
Tuesday, January 17, 2012, 3:50:38
Hello!

I run root on ZFS on FreeBSD 9.2 but when I set everything up I set:
zfs set mountpoint=legacy zroot

Which is causing me issues when doing my backups of the root pool with ZFS send/recv.

Is there a way to change the mount point from legacy to /?

Great site!!

-- Sean
Wednesday, December 25, 2013, 17:24:45
When I try:
# zpool create -m none -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache \
system mirror /dev/gpt/system0 /dev/gpt/system1
I get:
invalid vdev specification
use '-f' to override the following errors:
/dev/gpt/disk0 is part of potentially active pool 'zroot'
-- gn00b
Tuesday, July 15, 2014, 14:04:27
You seems to be trying to create a pool ontop of "system0" and "system1" GPT partitions, but zpool complains about completely different "disk0" partition?

Are you sure you specified correct and empty partitions to "zpool create" command?
-- ross
Sunday, July 27, 2014, 3:18:36
I'm a bit confused.
The title is: Install FreeBSD 10 on ZFS mirror (GPT or MBR)
but on the second comment in this section, you state: When working on this guide...It will work flawlessly on v9 of FreeBSD and probably won't work on other versions/OSes.
So is this guide for v9 or v10?
I still need help with my last post. I haven't been able to figure it out yet.
-- gn00b
Thursday, July 24, 2014, 16:58:20
The article was created at the times of FreeBSD 9. It was later updated for version 10 of FreeBSD, when that version was released.
-- ross
Sunday, July 27, 2014, 3:23:09
Regarding partitioning: there is an option 3. Namely - do not partition the drives at all, just use full raw disks for the zpools (via /dev/diskid/* for example) and go with swap on ZFS. And then the part about dd'ing /boot/zfsboot etc. I am running four servers like that, works like a charm and without legacy clutter. Besides, it's way easier to swap failed hard drives like that, no messing about with GPT or MBR and partition alignment.
-- Freultwah
Friday, May 27, 2016, 8:09:59
Hi... Where can I download a fully functional Linux Mint that boots from a ZFS raidz.??? (3 disks)
-- Wilson
Sunday, August 7, 2016, 17:52:42