I’d a daily script to check below
rpm -q --last kernel | perl -pe ‘s/^kernel-(\S+).*/$1/’ | head -1
uname -r
the output are
4.18.0-425.3.1.el8.x86_64
4.18.0-372.19.1.el8_6.x86_64
respecitively.
Which means an update and I reboot the VM. Then the kernal panic and cannot boot up. As this is the production VM, to cut the time in goggle around I roll back a snapshot in August (which teach me snapshot should be more frequent!).
The result of above script is the same that is the Kernel didn’t update, right?
Then I
dnf clean all
53 files removed
dnf upgrade
the output is
…
Dependencies resolved.
Nothing to do.
Complete!
Then reboot but the result is the same.
What should I do to solve?
(Lucky is the VM is running and the web is up that give me time to ask for help)
It woudl be helpful if you were able to provide the kernel panic you’re receiving. Without seeing the kernel panic log, we can only make guesses as to what the issue is.
Hi Nazunalika,
Sorry, I’m a newbee especially in problem solving and physically panic when cannot boot up the VM.
As the VM is in the Cloud, according to my poor memory under panic, I just remember to see something like “Started Session c1 of user root.” (actually I’m not sure about this) then nothing under it in the VNC terminal. Sorry, know that this doesn’t help.
And due to the VM kernel panic happen in the VM before I roll back the snapshot, which means now the /var/log/boot.log in the VM is from last snapshot to that can be boot into 4.18.0-372.19.1.el8_6.x86_64 again without the record in panic (suppose other log is the same).
Maybe there is a way to solve the mismatch in kernal version checking?
If there is a way to see what’s on the screen when it panics, you may want to take a screenshot. It’ll be the only way to know that it’s a panic and then the issue can be diagnosed.
As @nazunalika said, you need to provide more information, he asked you to provide a screenshot of the kernel panic. We cannot help diagnose your problem until you provide more information.
Also I don’t understand why you are using that rpm command, the easiest way to update the system is just do:
dnf update
and you’ll get the latest kernel. Maybe you can explain why you are doing some weird command to update the kernel that way? If you are just doing it to reinstall the existing kernel, then just do:
[root@rocky8 ~]# dnf reinstall kernel
Last metadata expiration check: 0:58:32 ago on Fri 18 Nov 2022 07:18:56 AM CET.
Dependencies resolved.
========================================================================================================================
Package Architecture Version Repository Size
========================================================================================================================
Reinstalling:
kernel x86_64 4.18.0-425.3.1.el8 baseos 8.8 M
Transaction Summary
========================================================================================================================
Total download size: 8.8 M
Installed size: 0
Is this ok [y/N]: ^COperation aborted.
[root@rocky8 ~]# dnf reinstall -y kernel
Last metadata expiration check: 0:58:41 ago on Fri 18 Nov 2022 07:18:56 AM CET.
Dependencies resolved.
========================================================================================================================
Package Architecture Version Repository Size
========================================================================================================================
Reinstalling:
kernel x86_64 4.18.0-425.3.1.el8 baseos 8.8 M
Transaction Summary
========================================================================================================================
Total download size: 8.8 M
Installed size: 0
Downloading Packages:
kernel-4.18.0-425.3.1.el8.x86_64.rpm 711 kB/s | 8.8 MB 00:12
-----------------------------------------------------------------------------------------------------------------------------------------
Total 693 kB/s | 8.8 MB 00:13
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Reinstalling : kernel-4.18.0-425.3.1.el8.x86_64 1/2
Cleanup : kernel-4.18.0-425.3.1.el8.x86_64 2/2
Running scriptlet: kernel-4.18.0-425.3.1.el8.x86_64 2/2
Verifying : kernel-4.18.0-425.3.1.el8.x86_64 1/2
Verifying : kernel-4.18.0-425.3.1.el8.x86_64 2/2
Reinstalled:
kernel-4.18.0-425.3.1.el8.x86_64
Complete!
According the output of uname, it seems the VM is still running the 4.18.0.372.16.1.e18_6 kernal which should be removed, right?
When I try to run dnf reinstall again
Last metadata expiration check: 1:31:56 ago on Fri 18 Nov 2022 02:35:54 PM CST.
Dependencies resolved.
================================================================================
Package Architecture Version Repository Size
================================================================================
Reinstalling:
kernel x86_64 4.18.0-425.3.1.el8 baseos 8.8 M
Transaction Summary
================================================================================
Total download size: 8.8 M
Installed size: 0
Downloading Packages:
kernel-4.18.0-425.3.1.el8.x86_64.rpm 25 MB/s | 8.8 MB 00:00
--------------------------------------------------------------------------------
Total 18 MB/s | 8.8 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Reinstalling : kernel-4.18.0-425.3.1.el8.x86_64 1/2
Cleanup : kernel-4.18.0-425.3.1.el8.x86_64 2/2
Running scriptlet: kernel-4.18.0-425.3.1.el8.x86_64 2/2
Verifying : kernel-4.18.0-425.3.1.el8.x86_64 1/2
Verifying : kernel-4.18.0-425.3.1.el8.x86_64 2/2
Reinstalled:
kernel-4.18.0-425.3.1.el8.x86_64
Complete!
It show the 4.18-0-425.3.1 is successfully reinstall and the 4.18.0.-372 didn’t shown.
But after reboot. the cat /proc/version output
Linux version 4.18.0-372.19.1.el8_6.x86_64 (mockbuild@dal1-prod-builder001.bld.equ.rockylinux.org) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)) #1 SMP Tue Aug 2 16:19:42 UTC 2022
which means the 372.19 is still running, right?
Suppose the VM should run the 4.18.0-425.3.1 but the dnf reinstall fail to make the change in grub (as I can’t see the 4.18.0-425.3.1 kernal shown in the grub menu) to boot into it instead just run the 4.18.0-372.19.1, is this understanding correct?
Yes, but now the question is how bootloader selects the kernel version?
Usually it does use the latest installed kernel, but there are ways around that.
What do you get with:
grub2-editenv - unset saved_entry
grub2-editenv list
The unset should remove the ‘saved_entry’ from the config.
Then reboot and check again:
uname -r
grub2-editenv list
If that does not boot with latest kernel, then we will diagnose more.
If the boot saves (adds) entry back there, then we have to see whether the installation of next kernel modifies the ‘saved_entry’.
Can you see the GRUB menu when the VM boots?
The menu has one entry for each installed kernel (and maybe some additional entries).
The menu has timer. When it expires, it should select the first entry on the list.
Pressing any key should stop the timer. Then you can highlight entry (arrow keys)
and boot it (Enter) or edit (e).
What is the first entry on the menu? Is there / where in the list is an entry for the 425?
You don’t have any menuentry for the new kernel. We are not running it, so we can remove it: sudo dnf remove kernel*4.18.0-425.3.1.el8
After that we reinstall the new kernel: sudo dnf uf
That did print text to terminal, didn’t it? (The command does write to stdout by default.)
Run ls -l /etc/grub2*
You should see two “symlink” files. One points to existing file, the other does not (is a “dangling symlink”).
What is the file that exists?
I’d tried “*” but get below (which is I run again just now whcih show the same result)
sudo dnf remove kernel*4.18.0-425.3.1.el8
No match for argument: kernel*4.18.0-425.3.1.el8
No packages marked for removal.
Dependencies resolved.
Nothing to do.
Complete!
This is the reason why I changed the “*” to “-” for a try.
sudo dnf remove kernel\*4.18.0-425.3.1.el8
No match for argument: kernel*4.18.0-425.3.1.el8
No packages marked for removal.
Dependencies resolved.
Nothing to do.
Complete!
