Cpu disabled by guest operating system, Unwanted reboot issue on rocky linux 9.2

Rocky Linux General
1

I have a VM on VmWare ESXI , with below version.
Rocky linux VM rocky-release-9.2-1.4.el9 on kernel-5.14.0-284.11.1.el9_2.x86_64

Have multiple VMs deployed, which does not have issues.
one of the VM goes for a quick/fast reboot with systemd getting killed and all services restarting.

on esxi side below event can be seen. during these unwanted reboot
The CPU has been disabled by the guest operating system. Power off or reset the virtual machine.

any suggestion on the compatible kernel which can be used with rocky-release-9.2-1.4.el9 ?
any other solution or workaround for the intermittent issue ?

2

Rocky Linux 9.2 is not supported. Rocky Linux Release Version Guide - Rocky Linux Wiki

Update your system(s) to the latest release.

With that said, you need to determine where the cause of this issue is. It may not he as cut and dry as you expect.

https://kb.vmware.com/s/article/2000542

3

thanks Louis for update,
I have gone through the documentation form VMware for this.
In our case I have checked the top results just before the reboot, there is no CPU spike I could see.
there are some CPU freeze issues reported on the kernel-5.14.0-284.11.1.el9_2.x86_64, which also looks similar to my case.

Can we use the kernel version 5.14.0-362.8.1 with Rocky 9.2 ? or it can not used ?

4

Kernels of older releases are also not supported. As I said, update your systems to the latest available versions. Doing so will isolate if some or all of your issues have been resolved in the latest available packages.

5

You did not specify the version of ESXi being used, but it sounds a lot like this vmxnet3 bug.

You could try the included workaround and see if it helps…

Good luck!

6

@LinuxGuy1997 thanks , I will check on this and see if can apply the patch on the esxi.
below is the esxi details :
Client version:1.34.4
Client build number:
16668064
ESXi version:
7.0.1
ESXi build number:
16850804

7

Update

There was one suspect, a cron job to clean the page cache which was running below command.
"sync ; echo 3 > /proc/sys/vm/drop_caches "
Coincidently it was getting conflicting with the other cron job which eventually will be running at same time.
For fixing the time of this drop cache cron has be changed to a non conflicting time.

Even after this change, we are seeing an unwanted reboot happening.
below logs can be seen in /var/log/syslog during the reboot, which indicates some SIGTERM, any way we can get to know the root case of it ?

Mar 09 12:00:53  systemd-journald[329]: Received SIGTERM from PID 1 (systemd).
Mar 09 12:00:59  systemd-journald[704]: Received client request to flush runtime journal.
Mar 09 12:01:09  rsyslogd[942]: [origin software="rsyslogd" swVersion="8.2102.0-113.el9_2" x-pid="942" x-info="https://www.rsyslog.com"] start
Mar 09 12:01:09  rsyslogd[942]: imjournal: journal files changed, reloading...  [v8.2102.0-113.el9_2 try https://www.rsyslog.com/e/0 ]
8

Just to check: have you applied dnf up or do you still have old content?

There is no “Rocky 9.2” and “Rocky 9.3”. There is only “Rocky 9” that is currently based on RHEL 9.3 content. Rocky 9 will shift to RHEL 9.4 content soon™ after RHEL 9.4 is released.

9

Still having the old content. As for now, we will have to stick to below once.
Rocky linux VM rocky-release-9.2-1.4.el9 on kernel-5.14.0-284.11.1.el9_2.x86_64

On recent occurrence of issue, I found that there is one more cron which runs every 2 hrs and run below command.

sync ; echo 3 > /proc/sys/vm/drop_caches

this looks to be one of the suspect to me. This cron has been removed now.
As the issue is not consistent (1 out of 10 VMs), not sure if this is actual fix.
Any thoughts on this ?