add a new family of wrappers for EFI based images

debefivm-create is based on mmdebstrap-autopkgtest-build-qemu, which is
is co-authored with Johannes Schauer Marin Rodrigues. Also thanks to
Jochen Sprickerhof for suggesting the --rootsize option for use in
Debusine.

1 files changed, 498 insertions, 0 deletions

diff --git a/bin/debefivm-create b/bin/debefivm-create
new file mode 100755
index 0000000..6efc204
--- /dev/null
+++ b/bin/debefivm-create
@@ -0,0 +1,498 @@
+#!/bin/sh
+# Copyright 2023 Johannes Schauer Marin Rodrigues <josch@debian.org>
+# SPDX-FileCopyrightText: 2023-2025 Helmut Grohne <helmut@subdivi.de>
+# SPDX-License-Identifier: MIT
+
+# shellcheck disable=SC2016 # Intentional quoting technique
+
+: <<'POD2MAN'
+=head1 NAME
+
+debefivm-create - Create an EFI-bootable VM image for various Debian releases and architectures
+
+=head1 SYNOPSIS
+
+B<debefivm-create> [B<-h> I<hostname>] [B<-k> F<sshkey>] [B<-o> F<output>] [B<-r> I<release>] [B<-s> <task>] [B<-z> I<size>] [B<--> I<mmdebstrap options>]
+
+=head1 DESCRIPTION
+
+Create an EFI-bootable disk image image booting into Debian.
+This essentially is a thin wrapper around B<mmdebstrap> much like B<debvm-create>.
+The resulting image contains a GPT partition table with an EFI system partition.
+A Unified Kernel Image is constructed from the installed kernel and initrd and placed into said partition.
+As a result, such an image may be booted with a variety of virtual machine emulators such as incus, libvirt or VirtualBox.
+It may also be booted using B<debefivm-run>.
+No user account is created and root can login without specifying a password.
+
+=head1 OPTIONS
+
+=over 8
+
+=item B<-a> I<architecture>, B<--architecture>=I<architecture>
+
+Set the architecture of the virtual machine.
+This option is forwarded to B<mmdebstrap> and multiple architectures may be supplied.
+The first architecture is used to assemble the Unified Kernel Image.
+It defaults to the native architecture of the installation.
+
+=item B<-h> I<hostname>, B<--hostname>=I<hostname>
+
+Set the hostname of the virtual machine.
+By default, the hostname is B<testvm>.
+
+=item B<-k> F<sshkey>, B<--sshkey>=F<sshkey>
+
+Install the given ssh public key file into the virtual machine image for the root user.
+This option also causes the ssh server to be installed.
+By default, no key or server is installed.
+To connect to the vm, pass a port number to B<debefivm-run> with the B<-s> option.
+
+=item B<-o> F<output>, B<--output>=F<output>
+
+Specify the file name of the resulting virtual machine image.
+By default, it is written to F<vm.img>.
+
+=item B<-r> I<release>, B<--release>=I<release>
+
+Use the given Debian release.
+By default, B<unstable> is being used.
+If you pass a complete F<sources.list> that includes release names to B<mmdebstrap>, you may pass an empty string here.
+
+=item B<-s> I<task>, B<--skip>=I<task>
+
+Skip a particular task or feature.
+The option may be specified multiple times or list multiple tasks to be skipped by separating them with a comma.
+By default, no tasks are skipped.
+The following tasks may be skipped.
+
+=over 4
+
+=item B<autologin>
+
+Skips adding a the customize-autologin.sh to B<mmdebstrap> that configures
+automatic root login on a serial console and also parses the C<TERM> kernel
+cmdline and passes it as C<TERM> to B<agetty>.
+
+=item B<kernel>
+
+skips installing a linux kernel image.
+This can be useful to install a kernel without a package.
+If a kernel is installed via B<mmdebstrap> option C<--include>, automtatic kernel installation is automatically skipped.
+
+=item B<packagelists>
+
+reduces the package lists inside the image.
+The B<available> database for B<dpkg> is not created.
+The package lists used by B<apt> are deleted.
+This generally produces a smaller image, but you need to run B<apt update> before installing packages and B<dpkg --set-selections> does not work.
+
+=item B<systemdnetwork>
+
+skips installing B<libnss-resolve> as well as automatic network configuration via B<systemd-networkd>.
+
+=item B<usrmerge>
+
+By default B<debefivm-create> adds a hook to enable merged-/usr without the B<usrmerge> package given a sufficiently recent Debian release.
+Without the hook, dependencies will pull the B<usrmerge> package as needed, which may result in a larger installation.
+
+=back
+
+=item B<--rootsize>=I<size>
+
+This option may be used as an alternative to B<--imagesize> to determine the size of the output image.
+It is also composed of an integer with optional unit.
+Unlike B<--imagesize> it determines the size of the root filesystem partition and the resulting image will be slightly larger than the size given here.
+
+=item B<-z> I<size>, B<--imagesize>=I<size>
+
+Specify the total image size as an integer and optional unit (example: 10K is 10*1024).
+Units are K,M,G,T (powers of 1024).
+The disk image contains an EFI system partition and an ext4 root filesystem.
+The EFI partition has a static size of about 128MB and the remaining space will be allocated to the root filesystem.
+The default is 10 GB.
+
+=item B<--> I<mmdebstrap options>
+
+All options beyond a double dash are passed to B<mmdebstrap> after the suite and target specification.
+This can be used to provide additional hooks for image customization.
+You can also request additional packages to be installed into the image using B<mmdebstrap>'s B<--include> option.
+Any positional arguments passed here will be treated as mirror specifications by B<mmdebstrap>.
+The B<--architecture> option should not be given here as B<debefivm-create> needs to know its value to construct a Unified Kernel Image.
+
+=back
+
+=head1 EXAMPLES
+
+Create an image suitable for use with B<autopkgtest-virt-qemu> B<--efi>.
+
+    debefivm-create ... -- --include=linux-image-generic,libpam-systemd,passwd,python3 --hook-dir=/usr/share/mmdebstrap/hooks/useradd --customize-hook=/usr/share/autopkgtest/setup-commands/setup-testbed
+
+=head1 SEE ALSO
+
+    debefivm-run(1) debefivm-ukify(1) debvm-create(1) mmdebstrap(1)
+
+=cut
+POD2MAN
+
+set -eu
+
+DEBARCH=$(dpkg --print-architecture)
+EFILABEL=efisys
+FAT_SIZE_KIB=$((126*1024))
+IMAGE=vm.img
+SKIP=,
+SSHKEY=
+SUITE=unstable
+ROOTLABEL=rootfs
+TOT_SIZE_KIB=$((10*1024*1024))
+VMNAME=testvm
+
+SHARE_DIR="${0%/*}/../share"
+BIN_DIR="${0%/*}/../bin/"
+
+# Disk layout:
+#  * GPT partition table in the front. Oversize to align.
+GPT_FRONT_SIZE_KIB=1024
+#  * FAT partition containing the EFI boot loader. Size configured.
+FAT_OFFSET_KIB=$GPT_FRONT_SIZE_KIB
+#  * Main ext4 root partition. Compute as remaining size.
+EXT4_OFFSET_KIB=$((FAT_OFFSET_KIB + FAT_SIZE_KIB))
+# EXT4_SIZE_KIB and TOT_SIZE_KIB depend on one another.
+#  * GPT backup table in the back. Oversize to align.
+GPT_BACK_SIZE_KIB=1024
+
+NON_EXT4_SIZE_KIB=$((EXT4_OFFSET_KIB + GPT_BACK_SIZE_KIB))
+EXT4_SIZE_KIB=$((TOT_SIZE_KIB - NON_EXT4_SIZE_KIB))
+
+nth_arg() {
+	shift "$1"
+	printf "%s" "$1"
+}
+
+die() {
+	echo "$*" >&2
+	exit 1
+}
+usage() {
+	die "usage: $0 [-a architecture] [-h hostname] [-k sskey] [-o output] [-r release] [-s task] [-z imagesize] [-- mmdebstrap options]"
+}
+usage_error() {
+	echo "error: $*" 1>&2
+	usage
+}
+
+opt_architecture() {
+	DEBARCH="$1"
+}
+opt_hostname() {
+	VMNAME=$1
+}
+opt_output() {
+	IMAGE=$1
+	if test "${IMAGE#-}" != "$IMAGE"; then
+		IMAGE="./$IMAGE"
+	fi
+}
+opt_release() {
+	SUITE="$1"
+}
+opt_rootsize() {
+	EXT4_SIZE_KIB="$(numfmt --to=none --to-unit=Ki --from=iec "$1")" ||
+		die "failed to parse size '$1'"
+	TOT_SIZE_KIB=$((EXT4_SIZE_KIB + NON_EXT4_SIZE_KIB))
+}
+opt_skip() {
+	SKIP="$SKIP$1,"
+}
+opt_sshkey() {
+	SSHKEY=$1
+}
+opt_imagesize() {
+	TOT_SIZE_KIB="$(numfmt --to=none --to-unit=Ki --from=iec "$1")" ||
+		die "failed to parse size '$1'"
+	EXT4_SIZE_KIB=$((TOT_SIZE_KIB - NON_EXT4_SIZE_KIB))
+}
+
+while getopts :a:h:k:o:r:s:z:-: OPTCHAR; do
+	case "$OPTCHAR" in
+		a)	opt_architecture "$OPTARG"	;;
+		h)	opt_hostname "$OPTARG"		;;
+		k)	opt_sshkey "$OPTARG"		;;
+		o)	opt_output "$OPTARG"		;;
+		r)	opt_release "$OPTARG"		;;
+		s)	opt_skip "$OPTARG"		;;
+		z)	opt_imagesize "$OPTARG"		;;
+		-)
+			case "$OPTARG" in
+				help)
+					usage
+				;;
+				architecture|hostname|imagesize|output|release|rootsize|skip|sshkey)
+					test "$OPTIND" -gt "$#" && usage_error "missing argument for --$OPTARG"
+					"opt_$OPTARG" "$(nth_arg "$OPTIND" "$@")"
+					OPTIND=$((OPTIND+1))
+				;;
+				architecture=*|hostname=*|imagesize=*|output=*|release=*|rootsize=*|size=*|skip=*|sshkey=*)
+					"opt_${OPTARG%%=*}" "${OPTARG#*=}"
+				;;
+				*)
+					usage_error "unrecognized option --$OPTARG"
+				;;
+			esac
+		;;
+		:)
+			usage_error "missing argument for -$OPTARG"
+		;;
+		'?')
+			usage_error "unrecognized option -$OPTARG"
+		;;
+		*)
+			die "internal error while parsing command options, please report a bug"
+		;;
+	esac
+done
+shift "$((OPTIND - 1))"
+
+if test -n "$SSHKEY" && ! test -f "$SSHKEY"; then
+	die "error: ssh keyfile '$SSHKEY' not found"
+fi
+
+check_skip() {
+	case "$SKIP" in
+		*",$1,"*)	return 0 ;;
+		*)		return 1 ;;
+	esac
+}
+
+test "$EXT4_SIZE_KIB" -lt "$((1024*64))" &&
+	die "implausibly small disk size $TOT_SIZE_KIB"
+
+# In a multiarch setting, we cannot tell in advance which architecture will be
+# supplying the Linux kernel image, but that's the architecture we need the EFI
+# stub for. Hence fail early.
+test "${DEBARCH%,*}" = "$DEBARCH" ||
+	die "multiarch images are not yet supported"
+
+case "${DEBARCH%%,*}" in
+	amd64)
+		EFIARCH=x64
+		ROOTGUID=4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709
+	;;
+	arm64)
+		EFIARCH=aa64
+		ROOTGUID=B921B045-1DF0-41C3-AF44-4C6F280D3FAE
+	;;
+	armhf)
+		EFIARCH=arm
+		ROOTGUID=69DAD710-2CE4-4E3C-B16C-21A1D49ABED3
+	;;
+	i386)
+		EFIARCH=ia32
+		ROOTGUID=44479540-F297-41B2-9AF7-D131D5F0458A
+	;;
+	riscv64)
+		EFIARCH=riscv64
+		ROOTGUID=72EC70A6-CF74-40E6-BD49-4BDA08E8F224
+	;;
+	*)
+		die "unsupported architecture: $DEBARCH"
+	;;
+esac
+
+test -e "$IMAGE" &&
+	die "output file $IMAGE already exists"
+
+if command -v ukify >/dev/null; then
+	UKIFY=ukify
+	UKIFY_VERBOSE=
+else
+	UKIFY="${BIN_DIR}debefivm-ukify"
+	UKIFY_VERBOSE=--verbose
+fi
+
+WORKDIR=
+
+cleanup() {
+	test -n "$WORKDIR" && rm -Rf "$WORKDIR"
+}
+cleanup_abort() {
+	cleanup
+	echo aborted >&2
+	exit 2
+}
+trap cleanup EXIT
+trap cleanup_abort HUP INT TERM QUIT
+
+WORKDIR=$(mktemp -d)
+
+truncate --size="${TOT_SIZE_KIB}K" "$IMAGE"
+
+# Stuff most of our options before the user options to allow them to override
+# ours.
+#
+# We skip replacing start-stop-daemon, because policy-rc.d nowadays works well
+# enough and then we have to not restore it before calling mkfs.ext4.
+set -- \
+	--mode=root \
+	--variant=apt \
+	--architecture="$DEBARCH" \
+	--skip=chroot/start-stop-daemon \
+	"$@"
+
+if ! check_skip kernel; then
+	set -- "--customize-hook=$SHARE_DIR/customize-kernel.sh" "$@"
+fi
+
+set -- \
+	"--customize-hook=printf 'LABEL=%s / ext4 defaults 0 0' '$ROOTLABEL' >"'"$1/etc/fstab"' \
+	"--include=?narrow(?exact-name(systemd-boot),?architecture(${DEBARCH%%,*}))" \
+	'--include=systemd-sysv' \
+	"$SUITE" \
+	/dev/null \
+	"$@"
+
+# set up a hostname
+ETC_HOSTS_TEMPLATE='127.0.0.1 localhost\n127.0.1.1 %s\n::1 ip6-localhost ip6-loopback\n'
+set -- \
+	"--customize-hook=echo '$VMNAME' >"'"$1/etc/hostname"' \
+	"--customize-hook=printf '$ETC_HOSTS_TEMPLATE' '$VMNAME' >"'"$1/etc/hosts"' \
+	"$@"
+
+# allow password-less root login
+set -- "--customize-hook=$SHARE_DIR/customize-delete-rootpw.sh" "$@"
+
+if ! check_skip systemdnetwork; then
+	# dhcp on all network interfaces, and add a dns resolver
+	set -- \
+		"--customize-hook=$SHARE_DIR/customize-networkd.sh" \
+		'--include=?not(?virtual)?exact-name(libnss-resolve)' \
+		"--customize-hook=$SHARE_DIR/customize-resolved.sh" \
+		"$@"
+fi
+
+# add ssh key for root
+if test -n "$SSHKEY"; then
+	set -- \
+		--include=openssh-server \
+		'--customize-hook=mkdir -m700 -p "$1/root/.ssh"' \
+		"--customize-hook=upload $SSHKEY /root/.ssh/authorized_keys" \
+		"$@"
+fi
+
+if ! check_skip packagelists; then
+	set -- --skip=cleanup/apt/lists "$@"
+	set -- "--customize-hook=$SHARE_DIR/customize-dpkgavailable.sh" "$@"
+fi
+
+if ! check_skip usrmerge; then
+	# Avoid the usrmerge package
+	set -- --hook-dir=/usr/share/mmdebstrap/hooks/maybe-merged-usr "$@"
+fi
+
+if ! check_skip autologin; then
+	set -- \
+		--include=login \
+		"--customize-hook=$SHARE_DIR/customize-autologin.sh" "$@"
+fi
+
+# By default, Debian mounts the EFI System Partition to /boot/efi. Keep that
+# default.
+BOOT_EFI_MOUNT_TEMPLATE='[Unit]\nDescription=EFI System Partition\n\n[Mount]\nWhat=LABEL=%s\nWhere=/boot/efi\nType=vfat\nOptions=umask=0077,rw,nodev,nosuid,noexec,nosymfollow\n\n[Install]\nWantedBy=local-fs.target\n'
+set -- \
+	'--customize-hook=mkdir "$1/boot/efi"' \
+	"--customize-hook=printf '$BOOT_EFI_MOUNT_TEMPLATE' '$EFILABEL' >"'"$1/etc/systemd/system/boot-efi.mount"' \
+	'--customize-hook=systemctl --root "$1" enable boot-efi.mount' \
+	"$@"
+
+# Pass the remaining hooks last. These collect data from the generated tree and
+# create the resulting filesystem. All user-supplied customizations should have
+# run before these.
+#
+# We download the artifacts that end up being turned into the EFI.
+EXT4_OPTIONS="offset=$((EXT4_OFFSET_KIB * 1024)),assume_storage_prezeroed=1"
+set -- "$@" \
+	"--customize-hook=download vmlinuz '$WORKDIR/kernel'" \
+	"--customize-hook=download initrd.img '$WORKDIR/initrd'" \
+	"--customize-hook=download '/usr/lib/systemd/boot/efi/linux$EFIARCH.efi.stub' '$WORKDIR/stub'" \
+	'--customize-hook=mount --bind "$1" "$1/mnt"' \
+	'--customize-hook=mount -t tmpfs tmpfs "$1/mnt/dev" -o mode=0755' \
+	'--customize-hook=rm -f "$1/usr/sbin/policy-rc.d"' \
+	'--customize-hook=/sbin/mkfs.ext4 -d "$1/mnt" -L '"'$ROOTLABEL' -E '$EXT4_OPTIONS' '$IMAGE' '${EXT4_SIZE_KIB}K'" \
+	'--customize-hook=umount --lazy "$1/mnt"'
+
+set -- mmdebstrap "$@"
+
+# We need to write the resulting $IMAGE from inside the namespace used by
+# mmdebstrap. If we were to have mmdebstrap --mode=unshare set up that
+# namespace, we could end up being unable to write it if a leading directory
+# component in $IMAGE were not not grant execute permission to any of the
+# subuids mapped. This typically is the case when using PrivateTmp=yes. With
+# that setting, there is a 0700 directory owned by the current user that
+# happens to not be mapped by mmdebstrap.
+#
+# Instead, we take matters into our own hands and set up the namespaces outside
+# mmdebstrap asking it to assume root. In addition to what mmdebstrap would do
+# by itself, we also map the current user to 65536.  The precise uid does not
+# matter, but having it mapped allows CAP_DAC_OVERRIDE to be used for writing
+# the image.
+if test "$(id -u)" != 0; then
+	set -- \
+		unshare \
+		--user \
+		--mount \
+		--ipc \
+		--pid \
+		--uts \
+		--fork \
+		--kill-child=TERM \
+		--map-user=65536 \
+		--map-users=auto \
+		--map-group=65536 \
+		--map-groups=auto \
+		--propagation=private \
+		--mount-proc \
+		--setuid=0 \
+		--setgid=0 \
+		"$@"
+fi
+echo "+ $*" >&2
+"$@" || die "mmdebstrap failed"
+
+CMDLINE="root=LABEL=$ROOTLABEL rw"
+case "${DEBARCH%%,*}" in
+	amd64|i386)
+		CMDLINE="$CMDLINE console=ttyS0"
+	;;
+esac
+
+set -- "$UKIFY" \
+	build \
+	$UKIFY_VERBOSE \
+	--efi-arch "$EFIARCH" \
+	--linux "$WORKDIR/kernel" \
+	--initrd "$WORKDIR/initrd" \
+	--cmdline "$CMDLINE" \
+	--stub "$WORKDIR/stub" \
+	--output "$WORKDIR/efiimg"
+echo "+ $*" >&2
+"$@" ||
+	die "failed to generate UKI"
+
+rm -f "$WORKDIR/kernel" "$WORKDIR/initrd" "$WORKDIR/stub"
+
+truncate -s "${FAT_SIZE_KIB}K" "$WORKDIR/fat"
+/sbin/mkfs.fat -n "$EFILABEL" -F 32 --invariant "$WORKDIR/fat"
+mmd -i "$WORKDIR/fat" EFI EFI/BOOT
+mcopy -i "$WORKDIR/fat" "$WORKDIR/efiimg" "::EFI/BOOT/boot$EFIARCH.efi"
+
+rm -f "$WORKDIR/efiimg"
+
+/sbin/sfdisk "$IMAGE" <<EOF
+label: gpt
+unit: sectors
+
+start=${FAT_OFFSET_KIB}KiB, size=${FAT_SIZE_KIB}KiB, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B
+start=$((FAT_OFFSET_KIB + FAT_SIZE_KIB))KiB, type=$ROOTGUID
+EOF
+
+dd if="$WORKDIR/fat" of="$IMAGE" conv=notrunc,sparse bs=1024 "seek=$FAT_OFFSET_KIB" status=none