# Rocky 9.4 AWS ami not booting after kernel update **URL:** \ **Category:** Rocky Linux Help & Support\ **Tags:** rocky-linux-9\ **Created:** [May 15, 2024, 4:32pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091 "2024-05-15T16:32:32Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![jaredc](https://avatars.discourse-cdn.com/v4/letter/j/dec6dc/32.png) [@jaredc](https://forums.rockylinux.org/u/jaredc)\ **Post date:** [May 15, 2024, 4:32pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/1 "2024-05-15T16:32:32Z")
Hello We have noticed that on our AWS environments since the 9.4 ami was published our servers do not boot up after a kernel/dnf update which is part of our provisioning AMI id: ami-02d6d64456f57b99d AMI Name: Rocky-9-EC2-Base-9.4-20240509.0.x86\_64 Region: eu-west-2 Disk type: gp3 Instance type: t3.medium The serial console snippet: ```auto [15.746077] nvme nvme0: pci function 0000:00:04.0 [15.746470] ACPI: \_SB_.LNKD: Enabled at IRQ 11 [15.770394] nvme nvme0: 2/0/0 default/read/poll queues [15.775054] nvme0n1: p1 p2 p3 p4 [143.289253] dracut-initqueue[429]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks: [143.336391] dracut-initqueue[429]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fmapper\x2floop0p4.sh: "if ! grep -q After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service 2>/dev/null; then [143.436561] dracut-initqueue[429]: [-e "/dev/mapper/loop0p4"] [143.465303] dracut-initqueue[429]: fi" [143.482303] dracut-initqueue[429]: Warning: dracut-initqueue: starting timeout scripts [143.845722] dracut-initqueue[429]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks: [143.892666] dracut-initqueue[429]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fmapper\x2floop0p4.sh: "if ! grep -q After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service 2>/dev/null; then [143.992287] dracut-initqueue[429]: [-e "/dev/mapper/loop0p4"] [144.022260] dracut-initqueue[429]: fi" [144.038265] dracut-initqueue[429]: Warning: dracut-initqueue: starting timeout scripts [144.398278] dracut-initqueue[429]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks: [144.444311] dracut-initqueue[429]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fmapper\x2floop0p4.sh: "if ! grep -q After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service 2>/dev/null; then [144.541977] dracut-initqueue[429]: [-e "/dev/mapper/loop0p4"] [209.332749] dracut-initqueue[429]: Warning: dracut-initqueue: starting timeout scripts [209.332777] dracut-initqueue[429]: Warning: Could not boot. Starting Dracut Emergency Shell... Warning: /dev/mapper/loop0p4 does not exist Generating "/run/initramfs/rdsosreport.txt" Entering emergency mode. Exit the shell to continue. Type "journalctl" to view system logs. You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report. dracut:/# ``` I am new to the forum so will try figure out how to attach the whole console log Has anyone experienced anything similar? Regards Jared ---
**Author:** ![jaredc](https://avatars.discourse-cdn.com/v4/letter/j/dec6dc/32.png) [@jaredc](https://forums.rockylinux.org/u/jaredc)\ **Post date:** [May 15, 2024, 4:34pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/2 "2024-05-15T16:34:31Z")
The dnf update: ```auto [root@ip-10-124-0-18 ~]# dnf update Rocky Linux 9 - BaseOS 1.4 MB/s | 2.2 MB 00:01 Rocky Linux 9 - AppStream 3.6 MB/s | 7.9 MB 00:02 Rocky Linux 9 - Extras 15 kB/s | 15 kB 00:00 Dependencies resolved. ===================================================================================================================================================================================================================== Package Architecture Version Repository Size ===================================================================================================================================================================================================================== Installing: kernel x86_64 5.14.0-427.16.1.el9_4 baseos 5.8 M kernel-core x86_64 5.14.0-427.16.1.el9_4 baseos 21 M Upgrading: kernel-tools x86_64 5.14.0-427.16.1.el9_4 baseos 6.0 M kernel-tools-libs x86_64 5.14.0-427.16.1.el9_4 baseos 5.8 M Installing dependencies: kernel-modules x86_64 5.14.0-427.16.1.el9_4 baseos 39 M kernel-modules-core x86_64 5.14.0-427.16.1.el9_4 baseos 33 M Transaction Summary ===================================================================================================================================================================================================================== Install 4 Packages Upgrade 2 Packages Total download size: 111 M Is this ok [y/N]: y Downloading Packages: (1/6): kernel-5.14.0-427.16.1.el9_4.x86_64.rpm 5.8 MB/s | 5.8 MB 00:00 (2/6): kernel-core-5.14.0-427.16.1.el9_4.x86_64.rpm 12 MB/s | 21 MB 00:01 (3/6): kernel-tools-libs-5.14.0-427.16.1.el9_4.x86_64.rpm 16 MB/s | 5.8 MB 00:00 (4/6): kernel-tools-5.14.0-427.16.1.el9_4.x86_64.rpm 16 MB/s | 6.0 MB 00:00 (5/6): kernel-modules-5.14.0-427.16.1.el9_4.x86_64.rpm 14 MB/s | 39 MB 00:02 (6/6): kernel-modules-core-5.14.0-427.16.1.el9_4.x86_64.rpm 15 MB/s | 33 MB 00:02 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 33 MB/s | 111 MB 00:03 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : kernel-modules-core-5.14.0-427.16.1.el9_4.x86_64 1/8 Installing : kernel-core-5.14.0-427.16.1.el9_4.x86_64 2/8 Running scriptlet: kernel-core-5.14.0-427.16.1.el9_4.x86_64 2/8 Installing : kernel-modules-5.14.0-427.16.1.el9_4.x86_64 3/8 Running scriptlet: kernel-modules-5.14.0-427.16.1.el9_4.x86_64 3/8 Upgrading : kernel-tools-libs-5.14.0-427.16.1.el9_4.x86_64 4/8 Running scriptlet: kernel-tools-libs-5.14.0-427.16.1.el9_4.x86_64 4/8 Upgrading : kernel-tools-5.14.0-427.16.1.el9_4.x86_64 5/8 Running scriptlet: kernel-tools-5.14.0-427.16.1.el9_4.x86_64 5/8 Installing : kernel-5.14.0-427.16.1.el9_4.x86_64 6/8 Running scriptlet: kernel-tools-5.14.0-427.13.1.el9_4.x86_64 7/8 Cleanup : kernel-tools-5.14.0-427.13.1.el9_4.x86_64 7/8 Running scriptlet: kernel-tools-5.14.0-427.13.1.el9_4.x86_64 7/8 Cleanup : kernel-tools-libs-5.14.0-427.13.1.el9_4.x86_64 8/8 Running scriptlet: kernel-tools-libs-5.14.0-427.13.1.el9_4.x86_64 8/8 Running scriptlet: kernel-modules-core-5.14.0-427.16.1.el9_4.x86_64 8/8 Running scriptlet: kernel-core-5.14.0-427.16.1.el9_4.x86_64 8/8 Running scriptlet: kernel-modules-5.14.0-427.16.1.el9_4.x86_64 8/8 Running scriptlet: kernel-tools-libs-5.14.0-427.13.1.el9_4.x86_64 8/8 Verifying : kernel-modules-5.14.0-427.16.1.el9_4.x86_64 1/8 Verifying : kernel-core-5.14.0-427.16.1.el9_4.x86_64 2/8 Verifying : kernel-5.14.0-427.16.1.el9_4.x86_64 3/8 Verifying : kernel-modules-core-5.14.0-427.16.1.el9_4.x86_64 4/8 Verifying : kernel-tools-libs-5.14.0-427.16.1.el9_4.x86_64 5/8 Verifying : kernel-tools-libs-5.14.0-427.13.1.el9_4.x86_64 6/8 Verifying : kernel-tools-5.14.0-427.16.1.el9_4.x86_64 7/8 Verifying : kernel-tools-5.14.0-427.13.1.el9_4.x86_64 8/8 Upgraded: kernel-tools-5.14.0-427.16.1.el9_4.x86_64 kernel-tools-libs-5.14.0-427.16.1.el9_4.x86_64 Installed: kernel-5.14.0-427.16.1.el9_4.x86_64 kernel-core-5.14.0-427.16.1.el9_4.x86_64 kernel-modules-5.14.0-427.16.1.el9_4.x86_64 kernel-modules-core-5.14.0-427.16.1.el9_4.x86_64 Complete! ``` ---
**Author:** ![jaredc](https://avatars.discourse-cdn.com/v4/letter/j/dec6dc/32.png) [@jaredc](https://forums.rockylinux.org/u/jaredc)\ **Post date:** [May 15, 2024, 4:49pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/3 "2024-05-15T16:49:37Z")
Full serial console log [https://rpa.st/LZPQ](https://rpa.st/LZPQ) ---
**Author:** ![jaredc](https://avatars.discourse-cdn.com/v4/letter/j/dec6dc/32.png) [@jaredc](https://forums.rockylinux.org/u/jaredc)\ **Post date:** [May 15, 2024, 4:57pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/4 "2024-05-15T16:57:08Z")
tested with gp2 volume also ---
**Author:** ![label](https://avatars.discourse-cdn.com/v4/letter/l/c5a1d2/32.png) [@label](https://forums.rockylinux.org/u/label)\ **Post date:** [May 15, 2024, 7:10pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/5 "2024-05-15T19:10:16Z")
We are in the process of investigating this. We’re not entirely sure why /dev/loop0 is being referenced (it starts at /etc/kernel/cmdline). Before updating an instance, you could change that file to make sure the root matches whatever is in /proc/cmdline at that time, which should technically get around that issue. However, we want to try to make sure you nor anyone else _has_ to do that. I’ll also point out we do provide LVM variants of our cloud images. The LVM variants of our images are not affected by this, if you wanted to try it out or use it. Thank you for pointing this out, we’ll be working on it throughout the day and hopefully have an idea for a fix soon. ---
**Author:** ![label](https://avatars.discourse-cdn.com/v4/letter/l/c5a1d2/32.png) [@label](https://forums.rockylinux.org/u/label)\ **Post date:** [May 16, 2024, 3:28pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/6 "2024-05-16T15:28:36Z")
As an update, @neil and myself have drilled down to the primary issue that was causing this. Due to this, we will be rebuilding and reissuing all of the cloud images for 9.4 on all applicable architectures. This process can take a while, but when we have it all done, we will notify here when they’re good to go. Thank you all for your patience and apologies for this unfortunate (and rather strange) inconvenience. ---
**Author:** ![jaredc](https://avatars.discourse-cdn.com/v4/letter/j/dec6dc/32.png) [@jaredc](https://forums.rockylinux.org/u/jaredc)\ **Post date:** [May 16, 2024, 3:49pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/7 "2024-05-16T15:49:07Z")
Thanks to you and Neil for your prompt investigation into this We have been using the 9.3 ami in the interim ---
**Author:** ![yegolev](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/yegolev/32/4145_2.png) [@yegolev](https://forums.rockylinux.org/u/yegolev)\ **Post date:** [May 16, 2024, 6:05pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/8 "2024-05-16T18:05:51Z")
This is undoubtedly a “me” problem but I’m only seeing 9.3.20231113 in the AWS Marketplace. I might be doing it wrong. edit: Remembered the old “search for 792107900819” so ignore me. Unless that is also sub-optimal. ---
**Author:** ![frank.wettstein](https://avatars.discourse-cdn.com/v4/letter/f/bb73d2/32.png) [@frank.wettstein](https://forums.rockylinux.org/u/frank.wettstein)\ **Post date:** [May 17, 2024, 9:15am UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/9 "2024-05-17T09:15:32Z")
Just to inform that I have the same issue after reboot on a different cloud-provider (openstack-based). ---
**Author:** ![neil](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/neil/32/34_2.png) [@neil](https://forums.rockylinux.org/u/neil)\ **Post date:** [May 17, 2024, 12:17pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/10 "2024-05-17T12:17:39Z")
Hey Frank - Yeah all the non-LVM based images were affected and are being rebuilt. Should be out today! Thanks everyone for your patience and reports. ---
**Author:** ![doseisan\_ons](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/doseisan_ons/32/4304_2.png) [@doseisan\_ons](https://forums.rockylinux.org/u/doseisan_ons)\ **Post date:** [May 20, 2024, 8:36pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/11 "2024-05-20T20:36:13Z")
I was able to update from 9.3 → 9.4 without any problems in my environment. I found that the AWS account ID of the AMI provider of ami-02d6d64456f57b99d is 792107900819. The official RockyLinux image distribution source is > **[AWS Marketplace: Rocky Linux](https://aws.amazon.com/marketplace/seller-profile?id=01538adc-2664-49d5-b926-3381dffce12d)** The AWS accountID of the AMI provider of the 9.3 image booted from here is 679593333241. Is the AWS account ID for the official AMI image variable…? Looking forward to seeing the 9.4 images. ---
**Author:** ![hegyre](https://avatars.discourse-cdn.com/v4/letter/h/8dc957/32.png) [@hegyre](https://forums.rockylinux.org/u/hegyre)\ **Post date:** [May 22, 2024, 5:20pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/12 "2024-05-22T17:20:25Z")
Indeed if you take the 9.3 AMI, then do a yum/dnf update, it will update to 9.4 and reboots without issues. One thing that I noticed is that the 9.4 AMI is not mentionned on [Download - Rocky Linux](https://rockylinux.org/fr/download) It only exists on AWS public images with the Rocky Linux account “792107900819” The “aws-marketplace” account owner “679593333241” doesn’t have the 9.4 AMI, probably because the 9.4 AMI provided by Rocky Linux team is faulty, thus not passing AWS validation. I’m now thinking if I should use aws 679593333241 instead of rocky 792107900819 for my packer and terraform datasources, if that can free me from some troubles in the future. ---
**Author:** ![lost](https://avatars.discourse-cdn.com/v4/letter/l/22d042/32.png) [@lost](https://forums.rockylinux.org/u/lost)\ **Post date:** [May 24, 2024, 10:24am UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/13 "2024-05-24T10:24:28Z")
Just to note (more for search probably, I see you say it affects all non-uefi cloud images) same issue on OpenStack using Rocky-9-GenericCloud-Base-9.4-20240509.0.x86\_64.qcow2 ---
**Author:** ![lost](https://avatars.discourse-cdn.com/v4/letter/l/22d042/32.png) [@lost](https://forums.rockylinux.org/u/lost)\ **Post date:** [May 24, 2024, 10:28am UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/14 "2024-05-24T10:28:25Z")
@neil @label I have just realised your message saying new images should be out “today” is dated 7 days ago - any update please? ---
**Author:** ![label](https://avatars.discourse-cdn.com/v4/letter/l/c5a1d2/32.png) [@label](https://forums.rockylinux.org/u/label)\ **Post date:** [May 27, 2024, 7:15am UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/15 "2024-05-27T07:15:03Z")
Apologies for the delays. We are also dealing with 8.10 work at the same time as this. We have built new images, we just need to upload, test, and publish. Our initial tests with the generic cloud images show that updates should no longer break the images. ---
**Author:** ![lost](https://avatars.discourse-cdn.com/v4/letter/l/22d042/32.png) [@lost](https://forums.rockylinux.org/u/lost)\ **Post date:** [June 5, 2024, 10:58am UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/16 "2024-06-05T10:58:43Z")
@label I see there is now a [https://download.rockylinux.org/pub/rocky/9.4/images/x86\_64/Rocky-9-GenericCloud-Base-9.4-20240523.0.x86\_64.qcow2](https://download.rockylinux.org/pub/rocky/9.4/images/x86_64/Rocky-9-GenericCloud-Base-9.4-20240523.0.x86_64.qcow2) - is this expected to fix this please? ---
**Author:** ![lost](https://avatars.discourse-cdn.com/v4/letter/l/22d042/32.png) [@lost](https://forums.rockylinux.org/u/lost)\ **Post date:** [June 5, 2024, 11:12am UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/17 "2024-06-05T11:12:41Z")
Ok having stumbled on that new image I just tested it; looks like the rebuild didn’t pull in the new packages as it still wants to update the kernel, but after doing a `dnf update *` I can confirm that image does not hangs after a reboot. ---
**Author:** ![MikeDach](https://avatars.discourse-cdn.com/v4/letter/m/b4bc9f/32.png) [@MikeDach](https://forums.rockylinux.org/u/MikeDach)\ **Post date:** [June 7, 2024, 9:47am UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/18 "2024-06-07T09:47:44Z")
I noticed that grub reconfiguration was not triggered after kernel packages update as per usual. Fixed reboot issue on a faulty Rocky-9-EC2-Base-9.4-20240509.0.x86\_64 AMI by: ```sh dnf -y update grub2-mkconfig -o /boot/grub2/grub.cfg shutdown --no-wall --reboot 1 ``` ---
**Author:** ![simo](https://avatars.discourse-cdn.com/v4/letter/s/90db22/32.png) [@simo](https://forums.rockylinux.org/u/simo)\ **Post date:** [June 10, 2024, 9:49pm UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/19 "2024-06-10T21:49:45Z")
The `792107900819` account is missing the latest `aarch64` image . Currently showing `Rocky-9-EC2-Base-9.4-20240509.0.aarch64`. There is `Rocky-9-EC2-Base-9.4-20240523.0.x86_64` for `x86_64` image, which I think resolved this issue. ---
**Author:** ![label](https://avatars.discourse-cdn.com/v4/letter/l/c5a1d2/32.png) [@label](https://forums.rockylinux.org/u/label)\ **Post date:** [June 11, 2024, 2:00am UTC](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091/20 "2024-06-11T02:00:08Z")
We made images over the weekend also. I’ll work with @neil to get them into AWS (and other cloud providers). [Next page](https://forums.rockylinux.org/t/rocky-9-4-aws-ami-not-booting-after-kernel-update/14091.md?page=2)