Replaced Processor, now stuck at emergency mode

Hi Guys !
I was running into issues with my CPU-motherboard; hence I had to RMA the former.

HArdware wise now things are OK, the connected components
i5-11400F was replaced with i5-11400 by INTEL under warranty.
Asus B560-Plus Wifi TUF gaming motherboard - Asus replaced this as well although they did not need to.
Crucial Ballistix 16GB 3600CL16 * 1 Module.
Kingston V300 120 GB SSD, partitioned to use only single boot Rocky Linux.
NVidia 3060Ti with NVidia drivers.

Linux was running very smoothly before the enforced CPU change. Now, although I don’t have any hardware issues, my system boots into emergency mode.

I have made the following changes to the bios, which are related to RAM and Primary GPU

Right now, I am stuck with this screen

In case you are unable to read what it says, all messages are about nouveau.
I have logged in as root and it asks to systemctl default or exit to boot into default mode
After providing “systemctl default” it goes unresponsive for a while, and then outputs the same thing - You are in emergency mode. After logging in, type “journalctl -xb” to view system logs, “systemctl reboot” to reboot, “systemctl default” or “exit” to boot into default mode.

I’ve tried “exit” as well, but it brings me to the same message - YOu are in emergency mode.

Talk about Groundhog Day!

Someone please advice how to retrieve my data in the 120 GB boot SSD - this is important. I can reinstall Rocky Linux, but I would want to avoid this - but the data in that boot drive I can’t live without.

Thanks and regards,

I would first review journalctl -xb or dmesg for any issues that’s forcing you to emergency mode. There’s usually a reason for this to happen. It may be as straightforward as a disk check, but it could be something else as well.

hi nazunalika!
I am not sure what it is to check. Is there a thread/document to check this? I mean, I did try journalctl -xb’, but I could not figure out anything.

OK. I must mention that the replaced motherboard had its bios updated by Asus Folk. My original motherboard did not.

Thanks!

You updated the MB (and it’s BIOS) and the CPU, and you expect the NVidia driver will still work? That seems unlikely to me. Is there any way to blacklist the NVidia driver while in emergency mode? I’d try that first.

In my case I’m using the AMD video driver, which is built-in and therefore less problematic in this type of situation.

Why would discrete graphics card be affected by MB/CPU changes? Well, if you did hardcode config of X11 to look for GPU from specific PCIe-address … maybe.

Nevertheless, it could/should be possible to go “back to basics” in emergency mode. For example, change the default to text mode (although one can choose that in GRUB-menu interactively too). The multi-user.target.

Hi @jlehtone! Thanks for shedding light on this… im sorry, but how do i do this? Multi-user.target is to be replaced in grub text?

@gerry666uk
I only replaced the motherboard with same model, but with updated bios. The CPU is updated, yes but the added igpu was removed from primary in the bios.

It is possible to change the kernel command-line options at grub menu (edit entry). Those changes are not permanent though.
Therefore, it is possible to append systemd.unit=multi-user.target to options.
(rescue.target, emergency.target, and graphical.target are possible targets too.)

When system is running and you have mounted volumes with read access, you can change the default target.
Systemd does load the default.target, unless kernel command-line has some systemd.unit set.
For example: systemctl set-default multi-user.target
That command sets symlink default.target to point to multi-user.target.

@jlehtone,
thanks man… I am sorry I didn’t inform you all that original system had 3 drives apart from this boot drive -before the processor conked off.
After I got the RMA’d processor, I set up the system the way the way the service centre folks did, with only motherboard 24 pin connector, the CPU power, the RAM and the processor. I had carried my boot drive and it had connected to the system. I now realize they needed only one bootable drive to prove the motherboad is operational.

OK, so there were 1 SSD bootable, 2 HDDs and one DVD drive. what do you suggest I do? Connect all of them and try and see if I can log in?

Again, apologies to @nazunalika, @jlehtone @gerry666uk and anyone else who spent precious time trying to figure this out. It was not my intent to waste your time.

regards,
nmp

Connect them all.

The thing is that you probably have filesystems on all those drives (except the DVD) that are set to be mounted during boot. So far, the boot has emergency, because it can’t mount volumes that “do not exist”.

@jlehtone thanks for your advice and approval. I will do this and report my findings tomorrow, I have some urgent work that has come up. Nevertheless, I wonder if the simple addition/removal of data drives is not allowed without modifying the internal record of the linux kernel? i.e.

when modifying/upgrading the CPU/motherboard, the system has to have all the data drives and boot drive as before, and once booted normally, then we can remove one or add another. Is this assessment correct?

Close. The default setup usually creates five volumes:

  • EFI System Partition. This is where boot loader is and is loaded first during boot. If it is missing, then nothing can be loaded
  • /boot is where boot loader reads kernel and initramfs from
  • / the “root” that has system files. The initramfs has drivers and instructions so that kernel can mount the root. After root is mounted, file /etc/fstab can be read to mount additional filesystems
  • swap partition is extension of RAM (but you really don’t want to run out of RAM)
  • /home is not critical for booting of the system, but without it no regular account can log in properly

The /etc/fstab rule for /home says that the filesystem must be checked and mounted for boot to continue.
That rule could be changed; who needs users anyway?

There is an additional twist: LVM. The default install uses LVM. It can have created /boot/efi and /boot as partitions and then created one “PV” partition to each of the three disks. The PV(s) form a “VG”, a volume that can span multiple physical disks. Then LVs (logical volumes) are allocated from VG for swap, /, and /home. They might thus span multiple disks. This is “great”, because you can conveniently have one large volume for /home, but in order to function you do need every PV, every disk that the VG uses.

Yes, when the system runs, then one can relocate data within VG, possibly make a PV free, unallocated,and remove such PV from VG. Then that physical drive is no longer required.

Note though that the default filesystem is XFS. It is not possible to shrink XFS. The only way to make XFS smaller is to remove it, create smaller filesystem, and restore data to it (from backups).

(User) data can be spread into additional filesystems. Those are not needed for boot nor for logging in. They are needed only when you access the data, the files. Such filesystems can be set to mount only when you actually access files in them. One naturally gets error, if disk is missing when mount is attempted.

Hi @jlehtone

I am sorry, i didnt understand this… I tried connecting all the drives, but it js still stuck where it was

Can u suggest something else?

Thanks

Hey
First of all, try to boot into text mode (formerly level3). If It works so you know that your issue come form graphic stuff.
In tjis case, try to execute a startx command. And look what happens.

Text mode is no better than graphical if problem is that filesystems are missing.

You have all drives connected? Can you run (in emergency)?

lsblk
lsblk --fs
findmnt

hi @jlehtone

thanks, i was able to run in emergency mode the commands listed.

here are the outputs.



@jlehtone & all viewing,
I hope things are going great at your end during this festive season.
I have managed to get another old set of HDD, mobo, CPU, gfx card working together in a zorin configuration, for basic usage.

can you now suggest what else should I do?
Best regards,

No sign of LVM. We see that “sdb” is all NTFS, so probably Windows content.
Nothing from “sdc” is mounted, although they are ext4 filesystems.
The EFI System Partition is mounted on /boot/efi, there is mounted /, home, and swap.

There is no separate /boot, but that is not an error.

If you can cat /etc/fstab, then you/we can see what mounts are attempted. If there are are more than what is already mounted, then those are of interest.

Can you run systemctl set-default multi-user.target?
If that succeeds, then the system would not attempt to start GUI on boot.

thanks I will try this… in the meantime, I am having an idea to use the nuclear option.
I have access to the data on the SSD on which Rocky 8.4 was installed via zorin desktop
Perhaps I can just copy my data AND have a fresh install.

Having a backup (something to copy quite recent version of data from) is always a good idea.