Host Your Own Services With FreeBSD: Gmirror GPT UFS Data Partition

In another article we installed FreeBSD onto mirrored pair of disks. We intentionally used fairly small but very fast nvme disks so that our FreeBSD setup works as fast as possible. However, we wouldn't be able to fit much services onto such small storage. Good news is we shouldn't. There are many reasons to put data on separate partition, or - even better - on separate disk(s). First and foremost, exhausting free space on / could crash complete system, while filling up /var could disable syslog functionality. On busy servers which need a lot of disk IOPS and bandwidth it is good to ensure that basic OS functionality won't be slowed down by greedy services from disk IOPS and bandwidth point of view. This article builds upon mentioned article and describes how to make anothergmirror volume from two SATA disks, format it as UFS and mount it under /ufsdata partition.

Assuming we added two disks, ada0 and ada1 (SATA SSDs) to our existing mirrored setup with OS on nda0 and nda1 (NVMes):

sysctl kern.disks
kern.disks: ada1 ada0 nda1 nda0

We label disks:

glabel label UFSDATA0 /dev/ada0
glabel label UFSDATA1 /dev/ada1

If building upon article about installing FreeBSD onto mirrored pair of disks, geom_mirror.ko should already be loaded. Verify that kernel module is loaded:

kldstat
Id Refs Address                Size Name
 1   10 0xffffffff80200000  1d345b0 kernel
 2    1 0xffffffff81f36000    21358 geom_mirror.ko <--- THIS ONE
 3    1 0xffffffff82910000     3220 intpm.ko
 4    1 0xffffffff82914000     2178 smbus.ko
 5    1 0xffffffff82917000     2a68 mac_ntpd.ko

Also that appropriate loader.conf entry exists:

cat /boot/loader.conf
geom_mirror_load="YES"

Create mountpoint which will hold our data partition:

mkdir /ufsdata

Create gmirror from geom labels:

gmirror label UFSDATA /dev/label/UFSDATA0 /dev/label/UFSDATA1

create gpt scheme and gpt partition with its own gpt label on top of gmirror:

gpart create -s gpt /dev/mirror/UFSDATA 
gpart add -t freebsd-ufs -a1m -l UFSDATA-ufsdata /dev/mirror/UFSDATA

Format UFS partition:

newfs -U /dev/gpt/UFSDATA-ufsdata

Use vi to add our newly formatted partition to fstab alongside existing mountpoints:

# jailhost.mimar.rs:/etc/fstab
/dev/gpt/OS-swap         none       swap   sw 0 0
/dev/gpt/OS-root         /          ufs    rw 1 1
/dev/gpt/OS-home         /home      ufs    rw 2 2
/dev/gpt/OS-usr          /usr       ufs    rw 2 2
/dev/gpt/OS-usr-local    /usr/local ufs    rw 2 2
/dev/gpt/OS-var          /var       ufs    rw 2 2
/dev/gpt/UFSDATA-ufsdata /ufsdata   ufs    rw 2 2
proc                     /proc      procfs rw 0 0
tmpfs                    /tmp       tmpfs  rw 0 0

Although not strictly neccessary, we will now reboot to make sure settings will apply on next boot.

After reboot, verify state of mirrors, both OS and UFSDATA:

          Name    Status  Components
     mirror/OS  COMPLETE  label/OS0 (ACTIVE)
                          label/OS1 (ACTIVE)
mirror/UFSDATA  COMPLETE  label/UFSDATA0 (ACTIVE)
                          label/UFSDATA1 (ACTIVE)

Also our mountpints made of gpt labels:

df -h
Filesystem                  Size    Used   Avail Capacity  Mounted on
/dev/gpt/OS-root            992M    277M    635M    30%    /
devfs                       1.0K      0B    1.0K     0%    /dev
/dev/gpt/OS-home            9.7G     48K    8.9G     0%    /home
/dev/gpt/OS-usr             1.9G    778M    1.0G    43%    /usr
/dev/gpt/OS-usr-local       7.7G    8.0K    7.1G     0%    /usr/local
/dev/gpt/OS-var             3.9G    362M    3.2G    10%    /var
/dev/gpt/UFSDATA-ufsdata    992G    8.0K    912G     0%    /ufsdata
procfs                      8.0K      0B    8.0K     0%    /proc
tmpfs                        16G    4.0K     16G     0%    /tmp

Here's transcript of terminal session: