Skip to content

document boot options

I'm bad at writing documentation but I think this should be part of the handbook, so I'll write down my verified findings in this issue. Maybe somebody else can take the useful bits and adjust them so that they can be included into the handbook. Until then, I'll treat this issue as a notebook of mine and update it in the future when I find out more stuff and update this issue text accordingly. This might also be helpful for others interested in these things.

While writing this I found a few issues with the tools which I'm collecting here: reform-tools#2 (closed)

Overview

There are three device choices for the boot process:

  1. where u-boot is located
  2. where kernel, dtb and initrd are located
  3. where the rootfs is located

The device from which u-boot is loaded is decided by the setting of the DIP switch on the Nitrogen8M_SOM CPU module. The switch is located here:

source: https://community.mnt.re/t/operating-system-on-nvme-without-sd-card/110/3

In that photo, the switch is set to OFF. Here is a better close-up of the switch indicating the two possible positions:

anim_opt

  • setting that switch to ON means it will load u-boot from SD-Card
  • setting it to OFF will load u-boot from emmc

Independent of whether u-boot is loaded from sd-card or from emmc, if the same u-boot version is present on both, the behaviour should be the same. U-boot will try to load kernel, dtb and initrd from the following devices in this order:

  1. sd-card
  2. emmc
  3. usb-stick

On each of these devices, u-boot will try to find on the first partition (in this order):

  1. /boot/extlinux/extlinux.conf
  2. /extlinux/extlinux.conf
  3. /boot/boot.scr
  4. /boot.scr
  5. /efi/boot/bootaa64.efi

The first match will then decide which kernel, dtb and initrd are loaded. To interact with u-boot, you have to connect to it via serial (UART) over the S1 connector. In most cases you will have to connect RX to TX, TX to RX and GND to GND to your serial adapter. This is not RS232 but you need a USB to UART TTL like those with an FTDI FT232 chip. You can then set a serial terminal program to 115200,8N1 or just run this command on the system that you attached the serial USB adapter to: screen /dev/ttyUSB0 115200. The S1 serial connector is the only way to interact with u-boot because u-boot can neither display its messages to the internal display nor will it receive input from the keyboard. It will though produce output via HDMI.

The initrd then decides from the entries in /etc/fstab on the rootfs, where the rootfs is located, which can be on any medium supported by linux, including an nvme SSD. After changing your /etc/fstab you have to run update-initramfs -u to regenerate your initramfs. Make sure you have /proc, sys and /dev mounted and the boot partition mounted to /boot before running update-initramfs -u.

This means, that it is completely possible to have u-boot on an otherwise empty sd-card, loading kernel, dtb and initrd from the first partition on emmc and have that load the rootfs from an encrypted lvm volume on nvme. This kind of setup would for example be useful if you do not want to remove the heatsink from the cpu module so that you can flip the DIP switch to choose loading u-boot from emmc. It is also useful if you want to experiment with u-boot as flashing it to an sd-card is more convenient than flashing it to emmc. If the sd-card only contains u-boot, then you can remove the sd-card directly after linux was loaded and also have a free sd-card slot again.

The following sections will cover, how to use reform-boot-config, reform-flash-rescue, reform-migrate and reform-setup-encrypted-nvme to change the root partition to boot from, flash the rescue image to emmc, populate emmc or nvme with a working rootfs or setup a luks encrypted nvme, respectively.

All of these scripts are very short (less than 100 lines of code) so if you want to know what's going on, you should read the scripts to get a better understanding of how they work.

Tools

reform-boot-config

This tool writes the selected root and boot partitions into /etc/fstab and then runs update-initramfs -u to store the selection in the initramfs. The first positional argument chooses the root partition:

args root partition
sd /dev/mmcblk1p2
nvme /dev/nvme0n1p1
usb /dev/sda1
emmc /dev/mmcblk0p2

Running the following command will use /dev/reformvg/root as the root partition:

reform-boot-config /dev/reformvg/root

Since u-boot can read the initrd either from sdcard or emmc, the --emmc switch can be used to choose the first partition on emmc as the /boot partition instead of the first partition on the sd-card (the default).

reform-flash-rescue

This command will download the latest sysimage-v3 from the source.mnt.re gitlab CI pipeline and flash it to emmc. Thus, for this command to work an active internet connection is required. It will also update u-boot on emmc with /boot/flash.bin from your currently running system.

reform-migrate

This command will rsync the contents of / of your currently running system to the give target device. Lastly, it will set up the sd-card such that it will load the rootfs from the target device. This command is useful to populate an existing nvme ssd but assumes that partitions are already set up. Partitions can be created using parted like so:

parted /dev/nvme0n1 "mklabel msdos"
parted /dev/nvme0n1 "mkpart primary ext4 4MiB -1"
mkfs.ext4 /dev/nvme0n1p1

reform-setup-encrypted-nvme

This tool take care of setting up your nvme ssd with a lvm and luks based full-disk-encryption setup that is close to what Debian installer would create. The tool first initializes a luks partition across the whole ssd and then sets up a physical volume, a volume group and two logical volumes inside of it: one with 8G of swap and the remaining space for the root file system. The tool then offers the user to also call reform-migrate with the correct arguments or shows the user how to manually edit /etc/initramfs-tools/conf.d/resume, /etc/crypttab and /etc/fstab before calling reform-migrate.

Examples

Download the latest sysimage-v3, extract it and flash it to your SD-Card. Replace XXX with the correct device name of your SD-Card.

wget -O reform-system.img.gz https://source.mnt.re/reform/reform-system-image/-/jobs/artifacts/main/raw/reform2-imx8mq/reform-system.img.gz\?job\=build
gunzip reform-system.img.gz
sudo dd if=reform-system.img of=/dev/mmcblkXXX bs=10M status=progress

If you just want to update u-boot on your sdcard to the latest version you can run the following:

wget -O flash.bin https://source.mnt.re/reform/reform-boundary-uboot/-/jobs/artifacts/master/raw/flash.bin\?job\=build
sudo dd if=flash.bin of=/dev/mmcblkXXX conv=notrunc bs=1k seek=33

If you want to build your own system image or u-boot (for example if you want to customize them) then you can clone the git repositories and build them by running:

apt install mmdebstrap genext2fs e2fsprogs binfmt-support git mount arch-test qemu-user-static parted
git clone https://source.mnt.re/reform/reform-system-image.git
cd ./reform-system-image/reform2-imx8mq
./mkimage.sh

apt install --no-install-recommends gcc-aarch64-linux-gnu build-essential bison flex device-tree-compiler
git clone https://source.mnt.re/reform/reform-boundary-uboot.git
cd ./reform-boundary-uboot
cp mntreform-config .config
env CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm make flash.bin

After putting the sd-card with the sysimage-v3 into your reform and booting, by first pressing the circle key for a second and then pressing 1, you can log into with the user root and no password. You can now have a look around the system, start a graphical user interface like sway, gnome or kde and connect to the internet via wifi or ethernet. After connecting to the internet, you can update your system on emmc to sysimage-v3 by running:

reform-flash-rescue

Your system on emmc is now up-to-date. By turning the DIP switch OFF you could now use your reform with a rootfs on emmc. If you want to have your rootfs on an encrypted nvme ssd instead, boot from the SD—Card (DIP switch ON) and run the following:

reform-setup-encrypted-nvme

Running this script will prompt you about running reform-migrate as well. If you do not want to add custom settings, you might want to answer that prompt with y. Your system is now configured such that it will load u-boot, kernel and initrd from your sd-card and the rootfs from your encrypted nvme ssd. Reboot with your sd-card still in place to make sure everything is working as expected.

After booting into your encrypted system, if you would like to load kernel and initrd from emmc instead of your sd-card, then you can configure it to do so by running the following command:

reform-boot-config --emmc /dev/reformvg/root

You can test this setup, by writing zeros over the partition table on your sd-card and only flashing u-boot onto it:

sudo dd if=/dev/zero of=/dev/mmcblkXXX bs=10M count=1
sudo dd if=flash.bin of=/dev/mmcblkXXX conv=notrunc bs=1k seek=33

Put the sd-card back into the reform and boot. Even though your sd-card now only contains u-boot and no kernel, initrd or rootfs, your system will load the latter two from emmc and the boot your encrypted rootfs from nvme. After booting successfully, you can remove the sd-card from the sd-card slot of your reform and use the sd-card slot normally. If you would also like to avoid having to insert the sd-card with u-boot on it during boot, you can flip the dip switch to OFF. Your system will now load u-boot, kernel and initrd from emmc and your rootfs from enrypted nvme.

Edited by Johannes Schauer Marin Rodrigues