Secure boot related issue after dnf upgrade - Rocky Linux 9 optimized for GCP

Rocky Linux Help & Support
1

A VM hosted in Google Cloud Platform (GCP) running ‘Rocky Linux 9 Optimized for GCP’ (rocky-linux-9-optimized-gcp-v20240213) failed to restart following a routine upgrade:

sudo dnf -y upgrade --refresh

After enabling serial ports, the console displayed:

error: ../../grub-core/kern/efi/sb.c:182:bad shim signature.
error: ../../grub-core/kern/efi/sb.c:182:bad shim signature.
error: ../../grub-core/loader/i386/efi/linux.c:258:you need to load the kernel
first.
error: ../../grub-core/loader/i386/efi/linux.c:258:you need to load the kernel
first.

Press any key to continue...

To attempt resolve this, I first disabled Secure Boot for the VM in the Google Cloud Console (Security and access > Shielded VM > Turn on Secure Boot )

The VM successfully restarted and SSH access was restored. I then tried the following to resolve the issue:

sudo dnf reinstall grub2* shim* mokutil kernel* -y 
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

After a reboot, I re-enabling Secure Boot (Security and access > Shielded VM > Turn on Secure Boot ) which resulted in the same boot failure. Currently, the only working solution appears to be permanently disabling Secure Boot on this instance.

I’m sharing this experience to help others who might encounter similar issues. As a Rocky Linux fan, I welcome any community feedback on best practices or alternative solutions.

Expand to see the specific update that triggered this Secure Boot issue... 
sudo dnf history info last

Transaction ID : 28
...
Return-Code    : Success
Releasever     : 9
Command Line   : -y upgrade --refresh
Comment        : 
Packages Altered:
    Upgrade  google-compute-engine-1:20250124.00-g1.el9.noarch @google-compute-engine
    Upgraded google-compute-engine-1:20241205.00-g1.el9.noarch @@System
    Upgrade  google-guest-agent-1:20250204.01-g1.el9.x86_64    @google-compute-engine
    Upgraded google-guest-agent-1:20241209.01-g1.el9.x86_64    @@System
    Upgrade  google-osconfig-agent-1:20250115.01-g1.el9.x86_64 @google-compute-engine
    Upgraded google-osconfig-agent-1:20240926.03-g1.el9.x86_64 @@System
    Upgrade  google-cloud-cli-510.0.0-1.x86_64                 @google-cloud-sdk
    Upgraded google-cloud-cli-506.0.0-1.x86_64                 @@System
    Upgrade  google-cloud-cli-anthoscli-510.0.0-1.x86_64       @google-cloud-sdk
    Upgraded google-cloud-cli-anthoscli-506.0.0-1.x86_64       @@System
    Upgrade  google-cloud-ops-agent-2.54.0-1.el9.x86_64        @google-cloud-ops-agent
    Upgraded google-cloud-ops-agent-2.53.0-1.el9.x86_64        @@System
    Upgrade  postgresql16-16.7-1PGDG.rhel9.x86_64              @pgdg16
    Upgraded postgresql16-16.6-1PGDG.rhel9.x86_64              @@System
    Upgrade  postgresql16-contrib-16.7-1PGDG.rhel9.x86_64      @pgdg16
    Upgraded postgresql16-contrib-16.6-1PGDG.rhel9.x86_64      @@System
    Upgrade  postgresql16-libs-16.7-1PGDG.rhel9.x86_64         @pgdg16
    Upgraded postgresql16-libs-16.6-1PGDG.rhel9.x86_64         @@System
    Upgrade  postgresql16-server-16.7-1PGDG.rhel9.x86_64       @pgdg16
    Upgraded postgresql16-server-16.6-1PGDG.rhel9.x86_64       @@System
    Upgrade  python3-firewall-1.3.4-9.el9_5.noarch             @baseos
    Upgraded python3-firewall-1.3.4-7.el9.noarch               @@System
    Upgrade  firewalld-filesystem-1.3.4-9.el9_5.noarch         @baseos
    Upgraded firewalld-filesystem-1.3.4-7.el9.noarch           @@System
    Upgrade  firewalld-1.3.4-9.el9_5.noarch                    @baseos
    Upgraded firewalld-1.3.4-7.el9.noarch                      @@System
    Upgrade  bzip2-libs-1.0.8-10.el9_5.x86_64                  @baseos
    Upgraded bzip2-libs-1.0.8-8.el9.x86_64                     @@System
    Upgrade  iptables-nft-1.8.10-11.el9_5.x86_64               @baseos
    Upgraded iptables-nft-1.8.10-4.el9_4.x86_64                @@System
    Upgrade  iptables-libs-1.8.10-11.el9_5.x86_64              @baseos
    Upgraded iptables-libs-1.8.10-4.el9_4.x86_64               @@System
    Upgrade  libxml2-2.9.13-6.el9_5.1.x86_64                   @baseos
    Upgraded libxml2-2.9.13-6.el9_4.x86_64                     @@System
    Upgrade  openssl-libs-1:3.2.2-6.el9_5.1.x86_64             @baseos
    Upgraded openssl-libs-1:3.2.2-6.el9_5.x86_64               @@System
    Upgrade  openssl-1:3.2.2-6.el9_5.1.x86_64                  @baseos
    Upgraded openssl-1:3.2.2-6.el9_5.x86_64                    @@System
    Upgrade  tzdata-2025a-1.el9.noarch                         @baseos
    Upgraded tzdata-2024b-2.el9.noarch                         @@System
    Upgrade  os-prober-1.77-12.el9_5.x86_64                    @baseos
    Upgraded os-prober-1.77-10.el9.x86_64                      @@System
    Upgrade  libnfnetlink-1.0.1-23.el9_5.x86_64                @baseos
    Upgraded libnfnetlink-1.0.1-21.el9.x86_64                  @@System
    Upgrade  linux-firmware-whence-20250114-146.3.el9_5.noarch @baseos
    Upgraded linux-firmware-whence-20241121-146.2.el9_5.noarch @@System
    Upgrade  linux-firmware-20250114-146.3.el9_5.noarch        @baseos
    Upgraded linux-firmware-20241121-146.2.el9_5.noarch        @@System
    Upgrade  iwl7260-firmware-1:25.30.13.0-146.3.el9_5.noarch  @baseos
    Upgraded iwl7260-firmware-1:25.30.13.0-146.2.el9_5.noarch  @@System
    Upgrade  iwl3160-firmware-1:25.30.13.0-146.3.el9_5.noarch  @baseos
    Upgraded iwl3160-firmware-1:25.30.13.0-146.2.el9_5.noarch  @@System
    Upgrade  iwl2030-firmware-18.168.6.1-146.3.el9_5.noarch    @baseos
    Upgraded iwl2030-firmware-18.168.6.1-146.2.el9_5.noarch    @@System
    Upgrade  iwl2000-firmware-18.168.6.1-146.3.el9_5.noarch    @baseos
    Upgraded iwl2000-firmware-18.168.6.1-146.2.el9_5.noarch    @@System
    Upgrade  iwl135-firmware-18.168.6.1-146.3.el9_5.noarch     @baseos
    Upgraded iwl135-firmware-18.168.6.1-146.2.el9_5.noarch     @@System
    Upgrade  iwl105-firmware-18.168.6.1-146.3.el9_5.noarch     @baseos
    Upgraded iwl105-firmware-18.168.6.1-146.2.el9_5.noarch     @@System
    Upgrade  ipset-libs-7.11-11.el9_5.x86_64                   @baseos
    Upgraded ipset-libs-7.11-8.el9.x86_64                      @@System
    Upgrade  ipset-7.11-11.el9_5.x86_64                        @baseos
    Upgraded ipset-7.11-8.el9.x86_64                           @@System
    Upgrade  libstdc++-11.5.0-5.el9_5.x86_64                   @baseos
    Upgraded libstdc++-11.5.0-2.el9.x86_64                     @@System
    Upgrade  libquadmath-11.5.0-5.el9_5.x86_64                 @baseos
    Upgraded libquadmath-11.5.0-2.el9.x86_64                   @@System
    Upgrade  libgomp-11.5.0-5.el9_5.x86_64                     @baseos
    Upgraded libgomp-11.5.0-2.el9.x86_64                       @@System
    Upgrade  libgfortran-11.5.0-5.el9_5.x86_64                 @baseos
    Upgraded libgfortran-11.5.0-2.el9.x86_64                   @@System
    Upgrade  libgcc-11.5.0-5.el9_5.x86_64                      @baseos
    Upgraded libgcc-11.5.0-2.el9.x86_64                        @@System
    Upgrade  grub2-tools-minimal-1:2.06-93.el9_5.x86_64        @baseos
    Upgraded grub2-tools-minimal-1:2.06-92.el9.x86_64          @@System
    Upgrade  grub2-tools-extra-1:2.06-93.el9_5.x86_64          @baseos
    Upgraded grub2-tools-extra-1:2.06-92.el9.x86_64            @@System
    Upgrade  grub2-tools-efi-1:2.06-93.el9_5.x86_64            @baseos
    Upgraded grub2-tools-efi-1:2.06-92.el9.x86_64              @@System
    Upgrade  grub2-tools-1:2.06-93.el9_5.x86_64                @baseos
    Upgraded grub2-tools-1:2.06-92.el9.x86_64                  @@System
    Upgrade  grub2-efi-x64-1:2.06-93.el9_5.x86_64              @baseos
    Upgraded grub2-efi-x64-1:2.06-92.el9.x86_64                @@System
    Upgrade  grub2-common-1:2.06-93.el9_5.noarch               @baseos
    Upgraded grub2-common-1:2.06-92.el9.noarch                 @@System
    Upgrade  qemu-guest-agent-17:9.0.0-10.el9_5.2.x86_64       @appstream
    Upgraded qemu-guest-agent-17:9.0.0-10.el9_5.x86_64         @@System
2

UEFI loads shim. Shim loads grub. Grub loads kernel. Kernel loads its modules.
UEFI has certificates that mokutil can inspect/add. (Not sure whether add is possible on GCP.)
Secure Boot requires that certificates can verify each loaded item mentioned above.

All of them do not have to be signed by same key/certificate. For example, ELRepo uses its own key for kernel modules, and so does dkms.

The question is, is it the shim that has unknown signer (or none), like the error message implies, and what can be done about that?

1 Like
3

looking into this.

thanks for the report.

1 Like
4

I can confirm the el9 artifacts were signed with the testing key. I’m looking into why.

2 Likes
5

Yes, I was thinking the same thing, I mean I don’t see shim itself was updated?

1 Like
6

Any updates on this?

I’ve blacklisted kernel-core-5.14.0-503.19.1.el9_5.cloud.1.0.x86_64 for now but this has been causing havoc with our auto-scaling clusters?

7

Apologies.

I’ve just pushed out the correct version of this kernel. If you reinstall kernel-* from SIG/Cloud, (or update from an existing Google image), this should be fixed.

3 Likes
8

dnf reinstall kernel*

Re-enabled Secure Boot

Happy to report the instance came back up. Thank you @neil

1 Like
9

So did this only affect cloud images, doesn’t affect base Rocky?