Fixing grub 2.06 "error: verification requested but nobody cares"

Problem statement

Previously I described my secure boot with fully encrypted filesystem setup on Alpine Linux. Recently I had a chance to run that same script when installing another machine. To my surprise, grub greeted me with:

error: verification requested but nobody cares: (cryptouuid/$PARTITION_UUID)/grub/x86_64-efi/normal.mod

A quick search shows that I’m not alone in this predicament, and that this problem exists since grub 2.06 (but is absent in the 2.04).

In this post I’ll describe my solution to this problem¹ as well as several dead ends along the way.

Spoiler: sed -i 's/SecureBoot/SecureB00t/' grubx64.efi, but do consider the context!

Reminder: current setup

The current architecture of my secureboot + disk encryption (cribbed from the post linked above) is:

and the relevant portion of my grub setup is:

cat - >/mnt/etc/default/grub <<-__EOF__
GRUB_DISTRIBUTOR="Alpine"
GRUB_TIMEOUT=2
GRUB_DISABLE_SUBMENU=y
GRUB_DISABLE_RECOVERY=true
GRUB_PRELOAD_MODULES="luks cryptodisk part_gpt lvm"
GRUB_ENABLE_CRYPTODISK=y
GRUB_DEVICE=nvmetank/ROOT/alpine
GRUB_DISABLE_LINUX_PARTUUID=true
GRUB_DISABLE_LINUX_UUID=true
__EOF__
sed -i 's/^modules=.*/modules=nvme,zfs/' /mnt/etc/update-extlinux.conf
chroot /mnt grub-mkconfig -o /boot/grub/grub.cfg
chroot /mnt grub-install --target=x86_64-efi --efi-directory=/efi
# The grub install partially fails (the call to efibootmgr does) because of raid.
# Because grub tries to give two devices to efibootmgr (who doesn't like that).
# 
# It's fine, tho, we can use the fallback /EFI/boot/bootx64.efi path within ESP.
mkdir /mnt/efi/EFI/boot
mv /mnt/efi/EFI/alpine/grubx64.efi /mnt/efi/EFI/boot/bootx64.efi
rmdir /mnt/efi/EFI/alpine
# Oh, and grub, f-off with grubenv, will ya? It's on RAID anyway.
rm /mnt/boot/grub/grubenv

# --------------------------------------------------------
# Time to setup the glorious Secure boot
mkdir -m 0700 /mnt/boot/secureboot
# Generate the cert
openssl req -new -x509 -newkey rsa:2048 -keyout /mnt/boot/secureboot/sb.key \
    -out /mnt/boot/secureboot/sb.crt -nodes -days 36500 -subj "/CN=Wejn SB CA/"
openssl x509 -in /mnt/boot/secureboot/sb.crt -out /mnt/boot/secureboot/sb.cer -outform DER
# Sign grub. This is, btw, the command you'll be re-running if you update grub
# binary sometime later (along with the "mv" a few lines above, I guess).
sbsign --key /mnt/boot/secureboot/sb.key --cert /mnt/boot/secureboot/sb.crt \
    /mnt/efi/EFI/boot/bootx64.efi
mv /mnt/efi/EFI/boot/bootx64.efi.signed /mnt/efi/EFI/boot/bootx64.efi
# Copy the "sb.cer" so it can be added to bios. I have it as all of: PK, KEK, db.
cp /mnt/boot/secureboot/sb.cer /mnt/efi/sb.cer

Alternatives considered²

I briefly checked the ’Net and several folks mentioned that the error is due to “shim lock” or grub not having all the modules preloaded.

That’s fair; let’s dive in.

Dead end #1: I’ll just inline all modules

At first I thought – fine, let’s embed all the modules, that will teach it:

grub-mkimage -O x86_64-efi -o foo.efi -p /EFI/boot \
  $(cd /boot/grub/x86_64-efi/; ls *.mod|sed 's/\.mod//')
sbsign --key /boot/secureboot/sb.key --cert /boot/secureboot/sb.crt foo.efi
rm foo.efi
mv foo.efi.signed /efi/EFI/boot/bootx64.efi

Which absolutely doesn’t work. Don’t even bother.

This, btw, shows my level of expertise when it comes to grub2.

Dead end #2: I’ll inline the modules properly

Okay, so I’ll inline the modules properly and life will be good:

MODS=$(for i in $(awk '/insmod/ {print $2}' /boot/grub/grub.cfg | sort -u); do
  test -f /boot/grub/x86_64-efi/$i.mod && echo $i; done)
grub-install --target=x86_64-efi --efi-directory=/efi \
  --modules="$MODS normal test search echo linux efifwsetup" \
  --disable-shim-lock
sbsign --key /boot/secureboot/sb.key \
  --cert /boot/secureboot/sb.crt /efi/EFI/alpine/grubx64.efi
mv /efi/EFI/alpine/grubx64.efi.signed /efi/EFI/boot/bootx64.efi
rm -rf /efi/EFI/alpine/
rm /boot/grub/grubenv

So modules are pre-loaded. The final list took a couple of tries, but what’s a few tens of reboots between friends?

Also had to --disable-shim-lock to disable the next error about missing shim protocol.

And now it bombs again, this time it fails to load kernel and ramdisk, because they aren’t verified:

[...]
error: verification requested but nobody cares: [...] vmlinuz-lts
[...]
error loading initrd: you need to load the kernel first.

OK, you’re getting annoying, grub.

Dead end #3: Fine! I’ll sign everything with PGP!

Okay, so I read the Using digital signatures section of the Grub manual (and a fair bit more of the documentation, actually).

Sounds like an overkill (since without decrypting the boot partition you won’t be booting much of anything), but what the hell:

umask 077
# PGP key
mkdir /boot/secureboot/gpg
cat - > /boot/secureboot/gpg/keygen.txt <<'EOF'
%echo Generating a key
Key-Type: RSA
Key-Length: 1024
Subkey-Type: RSA
Subkey-Length: 1024
Name-Real: Grub
Name-Email: grub@example.com
Expire-Date: 0
Passphrase: grub
%commit
%echo done
EOF
gpg --homedir=/boot/secureboot/gpg --gen-key --batch < /boot/secureboot/gpg/keygen.txt
gpg --homedir=/boot/secureboot/gpg --export > /boot/secureboot/gpg/pubkey
# Generate + sign image
grub-install --target=x86_64-efi --efi-directory=/efi --disable-shim-lock \
  --pubkey /boot/secureboot/gpg/pubkey
sbsign --key /boot/secureboot/sb.key --cert /boot/secureboot/sb.crt \
  /efi/EFI/alpine/grubx64.efi
mv /efi/EFI/alpine/grubx64.efi.signed /efi/EFI/boot/bootx64.efi
rm -rf /efi/EFI/alpine/
rm /boot/grub/grubenv
# Sign everything relevant in /boot
cd /boot
for i in $(find -type f -maxdepth 1 \! -name \*.sig) $(find grub -name '*.cfg' -or
    -name '*.lst' -or -name '*.mod'); do
  echo $i ...
  test -f $i.sig && continue
  #rm $i.sig
  echo grub | gpg --pinentry-mode loopback --homedir=/boot/secureboot/gpg --batch \
    --detach-sign --passphrase-fd 0 $i
done
killall gpg-agent

Surely this will work…

Nope.

Dies with error: verification requested but nobody cares again. And I’m getting pissed off.

Maybe I have the keyfile wrong?

Nope, because Secure Boot with GRUB 2 and signed Linux images and initrds uses the same export method.

Maybe I’m missing the set check_signatures=enforce in grub.cfg? Nope, still fails.

Maybe I’m missing verify.mod? But there isn’t a module of that name in Alpine.

And I’ve had enough.

Solution

Having exhausted all reasonable attempts at finding a standard solution (that would play nicely with grub’s silliness), it’s time to bring the big guns.

During my searches regarding the error I found a post detailing a patch someone made to the shim_lock that gave me an idea.

I fetched grub’s source and found the following pearl:

// This is snip from grub-core/kern/efi/sb.c, a GPLv3 code from 
// https://git.savannah.gnu.org/git/grub.git
// [... snip ...]
/*
 * Determine whether we're in secure boot mode.
 *
 * Please keep the logic in sync with the Linux kernel,
 * drivers/firmware/efi/libstub/secureboot.c:efi_get_secureboot().
 */
grub_uint8_t
grub_efi_get_secureboot (void)
{
  // [... snip ...]

  status = grub_efi_get_variable ("SecureBoot", &efi_variable_guid,
                                  &size, (void **) &secboot);

  if (status == GRUB_EFI_NOT_FOUND)
    {
      secureboot = GRUB_EFI_SECUREBOOT_MODE_DISABLED;
      goto out;
    }

But oh baby! Why didn’t you say that earlier?!

So all of this grub silliness is controlled by a single EFI variable (whatever that is exactly) that BIOS supplies when booting?

Well, that’s an easy fix, then:

MODS=$(for i in $(awk '/insmod/ {print $2}' /boot/grub/grub.cfg | sort -u); do
  test -f /boot/grub/x86_64-efi/$i.mod && echo $i; done)
grub-install --target=x86_64-efi --efi-directory=/efi \
  --modules="$MODS normal test search echo linux efifwsetup" \
  --disable-shim-lock
# =--------------------------------------------------------------------------=
# THIS is the fix:
sed -i 's/SecureBoot/SecureB00t/' /efi/EFI/alpine/grubx64.efi
# =--------------------------------------------------------------------------=
sbsign --key /boot/secureboot/sb.key \
  --cert /boot/secureboot/sb.crt /efi/EFI/alpine/grubx64.efi
mv /efi/EFI/alpine/grubx64.efi.signed /efi/EFI/boot/bootx64.efi
rm -rf /efi/EFI/alpine/
rm /boot/grub/grubenv

In other words – since BIOS authenticates my grub binary, and grub binary only loads the rest when it can successfully decrypt the /boot partition, I don’t need this signing silliness. In fact, I don’t need grub to get involved at all.

By binary editing the generated efi binary and replacing SecureBoot with SecureB00t, I effectively disable any “secure boot” half-cooked and severely over-engineered functionality in grub.

Profit!³

Closing words

I have a slightly bad feeling posting this. Because blindly applying thix “fix” without due consideration of the surrounding context might make other people’s boot process insecure.

But mindlessly applying commands found on the ’Net was always fraught with risk, yes?

And anyway, I still think that the secure boot implementation in grub is both needlessly over-engineered⁴ and half-cooked⁵.

And if it saves someone from spending 3+ hours debugging their otherwise perfectly valid setup… just because someone decided to unilaterally break their bootloader in a minor release, then I think it’s more than fair.