# After Update 19/05/2023 my SYSTEM IS FAIL! **URL:** \ **Category:** Rocky Linux Help & Support\ **Created:** [May 19, 2023, 1:17pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954 "2023-05-19T13:17:38Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![ide.nurhalim](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/ide.nurhalim/32/3118_2.png) [@ide.nurhalim](https://forums.rockylinux.org/u/ide.nurhalim)\ **Post date:** [May 19, 2023, 1:17pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/1 "2023-05-19T13:17:38Z")
You are in emergency mode. After logging in, type “journalctl -xb” to view system logs, “systemctl reboot” to reboot, “systemctl default” or “exit” to boot into default mode. Cannot open access to console, the root account is locked. See sulogin(8) man page for more details. ---
**Author:** ![stack](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/stack/32/1217_2.png) [@stack](https://forums.rockylinux.org/u/stack)\ **Post date:** [May 19, 2023, 3:04pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/2 "2023-05-19T15:04:55Z")
Greetings @ide.nurhalim , Ooph. I’m sorry to hear that. I’m guessing this is the 9.1 → 9.2 update. We did discover a bug that exists upstream if you are running LVM. If you know you are using LVM, let me know the work-around is easy to do. If you don’t know, or if you don’t have LVM then you will need to investigate the system logs. Since you have a locked root account, you will have to investigate with a live CD/USB. If you need more information about this, I can write up a few things to try. ~Stack~ ---
**Author:** ![Cphusion](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/cphusion/32/3235_2.png) [@Cphusion](https://forums.rockylinux.org/u/Cphusion)\ **Post date:** [May 19, 2023, 8:12pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/3 "2023-05-19T20:12:12Z")
What bug is that with lvm, my systems setups are using lvm and I haven’t noticed any issues? ---
**Author:** ![stack](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/stack/32/1217_2.png) [@stack](https://forums.rockylinux.org/u/stack)\ **Post date:** [May 19, 2023, 8:19pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/4 "2023-05-19T20:19:20Z")
Upstream made a change in how devices are detected. If a device has an extra \_ at the end of it’s IDNAME in /etc/lvm/devices/system.devices , then it won’t boot. Removing the \_ should make it boot again. ---
**Author:** ![ide.nurhalim](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/ide.nurhalim/32/3118_2.png) [@ide.nurhalim](https://forums.rockylinux.org/u/ide.nurhalim)\ **Post date:** [May 19, 2023, 9:43pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/5 "2023-05-19T21:43:52Z")
When the initial installation of my root account I may lock, I forgot about it, if that happens how can I run the command? ---
**Author:** ![stack](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/stack/32/1217_2.png) [@stack](https://forums.rockylinux.org/u/stack)\ **Post date:** [May 19, 2023, 10:04pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/6 "2023-05-19T22:04:55Z")
Greetings @ide.nurhalim If you don’t have a root password, you may need to reset it. Try the Method #3 password reset here: > **[How to Change and Reset Forgotten Root Password in RHEL 9](https://www.tecmint.com/reset-forgotten-root-password-rhel-9/)** > > While installation of RHEL 9, the installer recommends setting a complex password and while making the password complex, the chances of forgetting are high. Also, before you exit out of the system (after `touch /.autorelabel`) check that there isn’t an \_ after the IDNAME `cat /etc/lvm/devices/system.devices` If there is, edit the file and remove it. Hope that helps. ---
**Author:** ![danielp](https://avatars.discourse-cdn.com/v4/letter/d/82dd89/32.png) [@danielp](https://forums.rockylinux.org/u/danielp)\ **Post date:** [May 20, 2023, 1:35am UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/7 "2023-05-20T01:35:11Z")
I am having the same issue. However, I do know my root password, which I entered and saw the journalctl -xb output. With my limited understanding, it seems to point to an issue with dev/mapper which is timing out. I´ve been using many different Linux distros for years as my daily drivers, but I gotta admit my shortcomings as a system admin. I just don’t know what to do about this one right now. I am googling my way out as usual… but can’t find much info right now. I can boot successfully using the previous kernel which is how I am typing this right now. cat /etc/lvm/devices/system.devices show this: ```auto # LVM uses devices listed in this file. # Created by LVM command lvmdevices pid 3908 at Sat Aug 6 19:09:27 2022 VERSION=1.1.3 IDTYPE=sys_wwid IDNAME=eui.6479a72160163232 DEVNAME=/dev/nvme0n1p3 PVID=F8LLtaFytE2tnkBxeLg8X5sQUVP0gYUF PART=3 IDTYPE=sys_wwid IDNAME=eui.6479a74580200134 DEVNAME=/dev/nvme2n1p3 PVID=01n4N27WELBW0edct0uaKNgUubZEIGji PART=3 ``` my fstab looks like this: ```auto # # /etc/fstab # Created by anaconda on Sat Aug 6 19:09:27 2022 # # Accessible filesystems, by reference, are maintained under '/dev/disk/'. # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. # # After editing this file, run 'systemctl daemon-reload' to update systemd # units generated from this file. # /dev/mapper/rl_localhost--live00-root / xfs defaults 0 0 UUID=c62da415-c05b-41c2-ac3e-0d9c2d01da7a /boot xfs defaults 0 0 UUID=444D-85F8 /boot/efi vfat umask=0077,shortname=winnt 0 2 /dev/mapper/rl_localhost--live00-home /home xfs defaults 0 0 /dev/mapper/rl_localhost--live00-swap none swap defaults 0 0 ``` ---
**Author:** ![ide.nurhalim](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/ide.nurhalim/32/3118_2.png) [@ide.nurhalim](https://forums.rockylinux.org/u/ide.nurhalim)\ **Post date:** [May 20, 2023, 1:39am UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/8 "2023-05-20T01:39:43Z")
I have a production server for the hospital, because like this I am really crazy and have not found a solution, so I use a new server and restore the database ---
**Author:** ![danielp](https://avatars.discourse-cdn.com/v4/letter/d/82dd89/32.png) [@danielp](https://forums.rockylinux.org/u/danielp)\ **Post date:** [May 20, 2023, 2:11am UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/9 "2023-05-20T02:11:05Z")
I just fixed my problem. I did not have a leading space or underscore as suggested by other users. However, look here [https://bugzilla.redhat.com/2208039](https://bugzilla.redhat.com/2208039) Try removing the **system.devices** file. I’m not implying this will work for everyone, but it did for me: `rm /etc/lvm/devices/system.devices` Then restart, and it should boot. I honestly don’t know what I was doing, I don’t even know how lvm works to be honest. I was willing to format and do a fresh install… so, I had nothing to lose. Luckily for me, it worked! ---
**Author:** ![ide.nurhalim](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/ide.nurhalim/32/3118_2.png) [@ide.nurhalim](https://forums.rockylinux.org/u/ide.nurhalim)\ **Post date:** [May 20, 2023, 3:37am UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/10 "2023-05-20T03:37:32Z")
thank u, i will try again later This is what I found while running the update: Running scriptlet: kernel-core-5.14.0-284.11.1.el9\_2.x86\_64 705/705 Devices file sys\_wwid t10.ATA\_\_\_\_\_RMX-256GB\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_AA202207040135\_\_\_\_\_\_ PVID hsfbatPjDgode28LSWAOhAV3Vx164XF1 last seen on /dev/sdb6 not found. Devices file sys\_wwid t10.ATA\_\_\_\_\_RMX-256GB\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_AA202207040135\_\_\_\_\_\_ PVID hsfbatPjDgode28LSWAOhAV3Vx164XF1 last seen on /dev/sdb6 not found. Running scriptlet: kernel-modules-5.14.0-284.11.1.el9\_2.x86\_64 705/705 ---
**Author:** ![neil](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/neil/32/34_2.png) [@neil](https://forums.rockylinux.org/u/neil)\ **Post date:** [May 21, 2023, 6:56pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/11 "2023-05-21T18:56:58Z")
Hi all, We’ve added a section to the release notes for 9.2 which speaks about this issue as well as some workarounds. Hopefully this helps! [Current Release 9.2 - Documentation](https://docs.rockylinux.org/release_notes/9_2/) ---
**Author:** ![sstarche](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@sstarche](https://forums.rockylinux.org/u/sstarche)\ **Post date:** [May 26, 2023, 5:41am UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/12 "2023-05-26T05:41:13Z")
Thank you so much for this! Tried so many things before finding this and this is the only thing that worked for me. ---
**Author:** ![ssoto](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/ssoto/32/3395_2.png) [@ssoto](https://forums.rockylinux.org/u/ssoto)\ **Post date:** [July 25, 2023, 7:12pm UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/13 "2023-07-25T19:12:56Z")
@danielp statement was the only one that worked for me. Here is the details i can offer to this situation. Looking at the Rocky teams comment they stated the following > If you are impacted by this bug, then until the bug fix is released by RHEL and Rocky, please review and follow one of the following options **before** upgrading and rebooting your system: > > 1. After upgrading the `lvm2` package, but prior to rebooting, regenerate `/etc/lvm/devices/system.devices` by running `lvmdevices --update`. Confirm that the contents of `/etc/lvm/devices/system.devices` is changed to match the new format before rebooting the system. > 2. Prior to rebooting after upgrading, rename the `/etc/lvm/devices/system.devices` file (e.g., `mv /etc/lvm/devices/system.devices{,.bak}`) and, once the system has rebooted, run `vgimportdevices --all` to regenerate the file in the new format. > > A correctly formatted `system.devices` file for Rocky Linux 9.2 will not have repeated underscores in the device’s IDNAME field. I am doing [packer.io](http://packer.io) builds on AWS and would get the error of the system being locked out which was strange as this was a stable build before. I added a line in the build to get version info after full yum update and I had the following ```auto NAME="Rocky Linux" VERSION="9.2 (Blue Onyx)" ID="rocky" ID_LIKE="rhel centos fedora" VERSION_ID="9.2" PLATFORM_ID="platform:el9" PRETTY_NAME="Rocky Linux 9.2 (Blue Onyx)" ANSI_COLOR="0;32" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:rocky:rocky:9::baseos" HOME_URL="https://rockylinux.org/" BUG_REPORT_URL="https://bugs.rockylinux.org/" SUPPORT_END="2032-05-31" ROCKY_SUPPORT_PRODUCT="Rocky-Linux-9" ROCKY_SUPPORT_PRODUCT_VERSION="9.2" REDHAT_SUPPORT_PRODUCT="Rocky Linux" REDHAT_SUPPORT_PRODUCT_VERSION="9.2" ``` and started with a base AMI of: `ami-0fe64c0692c69d851` (note was using the most recent but due to all the errors went down a version) After the LVM creation portion of the build i had a cat on `system.devices` to see if it existed and if anything was in it. It output the following: ```auto # Created by LVM command vgcreate pid 33528 at Tue Jul 25 14:07:14 2023 VERSION=1.1.2 IDTYPE=sys_wwid IDNAME=nvme.1d0f-766f6c3037313836373630343065626565336134-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 DEVNAME=/dev/nvme1n1 PVID=7O00WcnQ4mNxWHpjXpXkse99rLBnhowr IDTYPE=sys_wwid IDNAME=nvme.1d0f-766f6c3035303838636538663462626538363632-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 DEVNAME=/dev/nvme2n1 PVID=32SH2Xsr0jpfOYvUQy0tmqaSm1e7hvVf ``` Note there are no underscores found and there looks to be no spaces. If i let the build just finish out just like that, it will not boot in amazon. If i perform a `lvmdevices --update` after the LVM creation it does not solve this problem. It is only when I specifically rm `system.devices` that once I spin up the completed build all the lvm mounts actually mount. when i look for `system.devices` i get that it does not exist like expected ```auto [root@ip-10-2-7-27 rocky]# cat /etc/lvm/devices/system.devices cat: /etc/lvm/devices/system.devices: No such file or directory ``` I can restart and completely halt the system and every time it starts back up. After performing `vgimportdevices --all` i get the following: ```auto # Created by LVM command vgimportdevices pid 1652 at Tue Jul 25 14:36:58 2023 VERSION=1.1.1 IDTYPE=sys_wwid IDNAME=nvme.1d0f-766f6c3034666461306538633339626337636366-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 DEVNAME=/dev/nvme1n1 PVID=7O00WcnQ4mNxWHpjXpXkse99rLBnhowr IDTYPE=sys_wwid IDNAME=nvme.1d0f-766f6c3032386164396637623761386664633765-416d617a6f6e20456c617374696320426c6f636b2053746f7265-00000001 DEVNAME=/dev/nvme2n1 PVID=32SH2Xsr0jpfOYvUQy0tmqaSm1e7hvVf ``` Everything works. So after offer that verbose amount of detail i just want to say that it does not seem to be limited to just IDNAME’s with underscores/spaces because as shown above my `system.devices` would like fine like @danielp’s in our build processes but then not start. Also the first option that Rocky Team offered did not work for us. Hope this someone looking for more detail. ---
**Author:** ![mlescieur](https://sea2.discourse-cdn.com/flex020/user_avatar/forums.rockylinux.org/mlescieur/32/251_2.png) [@mlescieur](https://forums.rockylinux.org/u/mlescieur)\ **Post date:** [December 13, 2024, 6:43am UTC](https://forums.rockylinux.org/t/after-update-19-05-2023-my-system-is-fail/9954/14 "2024-12-13T06:43:27Z")
This Workaround worked for me. I got into this issue after a power outage. Thanks! > > Workaround: > > $ rm -f /etc/lvm/devices/system.devices