Cannot boot after update to 8.8

Rocky Linux General
,
1

After updating to 8.8, my Rocky Linux installed on a Lenovo W541 hangs on boot with a blank screen.
Even if the fans run, I’m not able to switch to console using Ctrl+Alt+F2 and it’s not reachable on the network.
What can I do to troubleshoot and solve this issue?

I’m still able to boot on 8.7 so I can provide info if needed. Below some info related with the kernel installed and currently running on 8.7

Below the kernels installed:

# rpm -q kernel
kernel-4.18.0-425.3.1.el8.x86_64
kernel-4.18.0-425.13.1.el8_7.x86_64
kernel-4.18.0-477.27.1.el8_8.x86_64

This is the kernel I’m still able to boot on 8.7:

# uname -r
4.18.0-425.13.1.el8_7.x86_64
2

Almost wondering if it’s related to your nVidia GPU, with it being a ThinkPad W541…
Did you install the proprietary nvidia driver in any way, or are you still using the nouveau driver for it?

3

I didn’t install the nvidia driver and the nouveau driver is in use:

# lspci -nnk |grep -iA2 vga
00:02.0 VGA compatible controller [0300]: Intel Corporation 4th Gen Core Processor Integrated Graphics Controller [8086:0416] (rev 06)
	Subsystem: Lenovo Device [17aa:221e]
	Kernel driver in use: i915
--
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107GLM [Quadro K1100M] [10de:0ff6] (rev a1)
	Subsystem: Lenovo Device [17aa:221a]
	Kernel driver in use: nouveau

Do you have any suggestion to investigate and solve this issue?
Do you suggest to use the nvidia driver? I’d avoid to use proprietary driver if possible.

4

There are arguments both for and against the proprietary vs. nouveau driver, in both directions. My main concern would be if you had manually installed the proprietary one directly using nvidia’s installer instead of via a trusted repository which would keep it in sync with kernel changes etc.

Hmm… so if the screen goes totally blank, and it’s not pingable on the network… what if you tried to edit the grub boot entry for that newest kernel, at the grub menu, removing the “rhgb” and “quiet” options? That might show you the actual booting text of the system, and maybe even show you what it last loads or processes right before it hangs.

To do this, reboot the laptop and highlight the newest kernel entry at the grub boot menu (usually the newest one is the default, anyway), then hit the “e” key. Use the arrow keys to move down to the line that says something along the lines of "GRUB_CMDLINE_LINUX=“various things in here” . Inside those quotes will be several things; the rhgb and quiet entries likely will be towards the end. Move the cursor to each of these and use the Del key to delete them.

Don’t worry–this is NOT a permanent change, it only takes effect for this one boot, so you’re not in danger of making a mistake which renders it useless. Worst case, if you mistype or delete the wrong thing(s) and it fails to boot – just reboot and try it again. :grinning:

After you have removed both of those, and made sure you haven’t removed the closing double-quote, simply hit CTRL-X to boot with the modified options.

Since the quiet option hides the text of the system startup and the rhgb option tries to make things further pretty by showing a kind of boot progress bar, each of these are less than helpful when you’re trying to determine what might be wrong…

So if you make those changes correctly then you hopefully will be able to see at least some details of the boot process, and maybe even a hint of where it stops / what makes it hang? If you still end up seeing absolutely nothing but a blank screen, immediately, then I’m not quite sure what to suggest next…

Hope this helps at least a little bit. Glad to hear it’s something which only happens with the very latest kernel update, so it’s not completely unusable!

5

First of all thanks for the detailed guidance on how to debug this issue!

I made the test and I saw some messages during the startup, then the screen become blank as before.
I didn’t see any error message, and the last visible message before the blank screen was:
i915 000:00:02.0: vgaarb: deactivate vga console

I made also an additional test removing the rhgb and quiet entries from the grub entry from the previous kernel entry and I see that also in this case the screen become blank after the same message, but then the screen turned on again after about 1 second with an higher resolution showing all the messages and completing the boot successfully.

Is it possible to configure the system to maintain the vga resolution during the whole boot process?
Maybe this way the screen shouldn’t become blank and it would be possible to see the next messages.

6

So vgaarb is the VGA Arbiter, a linux kernel module which tries to find valid screen(s) for the xorg server. From what you’re describing, it very much sounds like it’s either unable to find a valid screen, or maybe even trying to use an invalid one? Weird.

Are you certain that you’re unable to ssh into the laptop from another computer at the point the display goes blank? Have you ever done that before?

I’m really curious if you managed to ssh into the laptop, then try “modprobe nouveau” at the console, and see if anything happened…

7

NVidia wrote:

Quadro Driver Branch Release 418 is the last NVIDIA professional driver to support Quadro Kepler-series notebook GPUs.

That is, the proprietary driver would have to be from legacy serie. NVidia’s repo seems to at least have 418 something for el8.

If you didn’t have a proprietary driver before, then something in el8_8 Nouveau did change.

I would switch the system to boot to the non-GUI (to diagnose).

systemctl set-default multi-user.target

If new kernel boots fine to text mode, then the issue is only with GUI.

One can switch target (“runlevel”) on the run with isolate:

systemctl isolate graphical.target
8

Hi @LinuxGuy1997 ,
I just tried again to ssh into the laptop.
It’s configured with a fixed ip address, but it don’t respond to the ping and also it’s not possible to connect using ssh.

$ ping 192.168.9.88
PING 192.168.9.88 (192.168.9.88): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

$ ssh root@192.168.9.88
ssh: connect to host 192.168.9.88 port 22: Host is down
9

Have you ever been able to ssh into this laptop before? Just checking to make sure you don’t have some firewall rules blocking it or anything… I’d verify your ability to ssh into it while it’s booted to the older kernel first, then try on the new one.

Also, if that still doesn’t work, I’d try what @jlehtone suggested, and see if the new kernel even boots to the text console if the GUI/xorg is disabled…

Good luck!

10

Hi @jlehtone,
yes, booting with the old kernel I’m able to ssh to the laptop.
I also executed the command systemctl set-default multi-user.target from the old kernel, but then starting the new one the behavior is the same (blank screen) and the last message I can read is still vgaarb: deactivate vga console.
Does it means that xorg is not disabled with the new kernel?
Booting the old kernel after disabled xorg, the “graphical” Rocky Linux screen with the running wheel appears and then it switch to the text console but with a resolution higher than vga.
Since the issue occurs when the system switch from vga to an higher resolution, is there a way to force the system to stay in vga mode during the boot to diagnose?

11

Adding the nouveau.modeset=0 kernel parameter to the latest entry in grub, the laptop boot successfully.
My understanding is that the parameter disables the nouveau drivers, in fact running the lshw -c video command it seems that the i915 driver is currently used:

  *-display
       description: VGA compatible controller
       product: 4th Gen Core Processor Integrated Graphics Controller
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       logical name: /dev/fb0
       version: 06
       width: 64 bits
       clock: 33MHz
       capabilities: msi pm vga_controller bus_master cap_list fb
       configuration: depth=32 driver=i915 latency=0 resolution=1920,1080
       resources: irq:32 memory:b3400000-b37fffff memory:80000000-9fffffff ioport:5000(size=64) memory:c0000-dffff

Now my laptop boots also with the last kernel, but I’m not sure to understand the implications of adding the nouveau.modeset=0 parameter. Do you suggest to leave it or try upgrade to the 9.x release to have it fixed?

12

Are there rhgb and/or quiet options in the grub entry? Installer used to add them and I’ve stripped them out so long that I can’t recall whether el8 has them. The rhgb probably paints that wheel.

Yes, the resolution/font does change if/when “better driver” (Nouveau or NVidia) takes over. (NVidia on legacy boot was less noticeable.)

Disables or prevents loading. Have we now narrowed the issue down to Nouveau in the new kernel being not able to do what it should? Specifically the kernel module, rather than X11/Wayland drivers.

“Upgrade” to 9 is a fresh install. I’d get latest (Live or install) images for 8 and 9 on USB and see whether either of them gets graphics up.

13

Thanks @jlehtone,
I’ll try Rocky Linux 9 live image on USB to see if graphic works.