Ionos VPS with Rocky Linux 9 hang during cli reboot

[shovas]

I’ve run CentOS VPSs for many years with Ionos and reboots have always been blissfully quick and painless but in the last year or so, and notably since moving to Rocky Linux 9, it seems that after a week or more of uptime a cli reboot appears to shutdown fine but does not ever reboot. I have to go into the Ionos web interface and restart the VPS.

But, after doing this once, if I reboot from the cli again immediately, the reboot is quick and fine, as I would expect.

I’m totally confused what’s going on with it.

I see something about ACPI vs PCI reboots for certain hypervisors but I’m hesitant to touch reboot params on servers.

[iwalker]

If with a vps you might have luck installing the upower package:

dnf install -y upower
systemctl enable --now upower

but a lot of this can also depend on the hardware or the way the hypervisor underneath is configured. I’ve got a dedicated server with a certain VPS provider which will reboot with no problems if it hasn’t been up for long, but after a couple of weeks it will hang at shutdown instead of restarting. So if that doesn’t help, there isn’t probably too much you can do.

Incidently, on physical servers I have everything works fine, just my VPS provider is the issue so something with their hardware - and not all of it - some works, some doesn’t. So perhaps BIOS, firmware, etc.

[shovas]

Ya, physical servers are fine for me, too. AI says there’s something about the hypervisor (qemu, kvm, etc.) that’s causing problems.

A local VirtualBox host I run doesn’t have this problem with Rocky Linux, either.

[RianKellyIT]

To be fair, the long-uptime-specific behaviour points to the reboot method that the kernel uses to actually trigger the hardware restart. On KVM/QEMU hypervisors like Ionos uses, the default PCI reboot method sometimes fails to signal the hypervisor correctly after the system has been running for a while, so it shuts down cleanly but the reboot signal never gets through.

Worth trying the ACPI reboot method. Edit /etc/default/grub and add reboot=acpi to the GRUB_CMDLINE_LINUX line:

GRUB_CMDLINE_LINUX="... reboot=acpi"

Then rebuild the GRUB config:

grub2-mkconfig -o /boot/grub2/grub.cfg

Reboot the VPS from the Ionos control panel to apply it (since the normal reboot doesn’t work anyway), then leave it up for a week and see if the problem recurs.

Some KVM hosts also respond better to reboot=kbd or reboot=bios, so if ACPI doesn’t sort it, those are worth trying in sequence.

A second thing to check: the upower suggestion above is a good one for systems where power management events interfere with shutdown. If upower isn’t installed, the kernel may not properly signal a reboot intent vs a power-off to the hypervisor on some setups.

If neither approach works, it’s worth raising with Ionos support and mentioning the reboot method issue specifically. Some VPS providers have hypervisor-side settings that affect how guest reboot signals are handled.

[iwalker]

@shovas

Ya, physical servers are fine for me, too.

Mine’s actually different, I have a physical server at a hosting provider that doesn’t reboot so for this one it’s potentially something with the hardware or firmware. But I have installed upower on it to see if it will make a difference next time I reboot. All my VM’s work fine, it’s just this one particular physical server that’s a problem but it doesn’t belong to me so can’t do much with it. All the ones I do have in my office work fine. I just put it down to crappy hardware from the hosting provider.

@RianKellyIT

A second thing to check: the upower suggestion above is a good one for systems where power management events interfere with shutdown. If upower isn’t installed, the kernel may not properly signal a reboot intent vs a power-off to the hypervisor on some setups.

I’m hoping this will sort out my physical server. For me, every Rocky VM I have reboots fine.

[shovas]

If I understand correctly, modern kernels default to acpi, and mine wasn’t specified in the grub file, so I added reboot=pci. It rebooted fine (which it does shortly after a hanging reboot and a force server restart) so we’ll see in a week, I guess. Thanks for the instructions.

[shovas]

reboot=pci didn’t work. Based on what I’m seeing, it’s a qemu host misconfiguration issue which I won’t be able to fix from my side. I’ll see if submitting a support ticket helps.

Thought there was something of interest here but it also seems to be about the host:

Rocky 9 KVM host - reboot does not suspend VM's, hard shutdown instead

The missing piece is libvirt-guests.service - this is the systemd unit that handles graceful VM shutdown or suspend when the host powers off. Without it configured, the kernel simply kills all processes including qemu instances when shutdown is initiated, which is why the VMs are seeing a “pulled plug” event. First check if the service exists and is enabled: systemctl status libvirt-guests.service If it’s disabled or not running, enable it: systemctl enable --now libvirt-guests.service Then…

It’s still odd that after a reboot, another reboot will be quick, but after a week or more the reboot will hang…

[hs303]

Are you sure it does not boot again? Can you see the output of the console? If you have multiple disks you could have problems similar to those described in this topic:

Rocky Linux 9.5 sometimes boot into emergency mode for no reason

Hi i have a Rocky 9.5 running on Linode that sometimes after installing updates boots into emergency mode. Its a game server thats internet facing and therefore is searching/installing system updates every day at 5 in the morning. A lot of times it does install updates and reboots after that (the script has a check reboot function) and everything is fine but sometimes for no apparent reason it does boot into emergency mode. Most unfortunately it does not store old boot records. Is there a way …

[shovas]

Thanks for the tip. Lots of handy nuggets in that thread but I can’t see anything obviously amiss based on it.

I can’t see anything in my logs about any errors around reboots. From what I can see and experience, reboots shutdown completely fine but the actual reboot never happens. During this time, Ionos’ web interface remote console is frozen so I can’t see what’s happening there. Their console isn’t always frozen, after a forced reboot, the remote console can be used and monitored as expected. But after a few days it stops working.

I didn’t have persistent journals enabled so I’ve enabled that now. I’ll see what I can find next time I have the issue.

I did contact Ionos support and they were adamant it couldn’t possibly be a host side issue because I’m apparently the only one experiencing it…on all three VPSs…which each have a locally hosted VirtualBox guest to mirror them where I don’t have the problem ugh.

[hs303]

The key takeaway would be to check if your root partition is referenced somewhere by device name like /dev/sda1 and not its UUID or by logical volume name. This could be the case in cloud images. If you had two disks they might get assigned a different device name during boot (sdb instead of sda etc.) preventing the root partition from being found.

root@freddy:~# grubby --info=ALL | grep ^root
root="/dev/mapper/rl_localhost--live-root"
root="/dev/mapper/rl_localhost--live-root"
root="/dev/mapper/rl_localhost--live-root"
root="/dev/mapper/rl_localhost--live-root"

root@freddy:~# grep -E '\s/\s' /etc/fstab
UUID=dd3e2113-40c0-49ce-b64d-d5576f156a25 / xfs defaults 0 0

[shovas]

Doesn’t look like that’s the case:

Summary

[root@17885d7 ~]# grep -E ‘\s/\s’ /etc/fstab
UUID=22de856b-8099-4e62-aca2-16fedf13a779 / xfs defaults 0 0
[root@17885d7 ~]# grubby --info=ALL | grep ^root
root=“UUID=22de856b-8099-4e62-aca2-16fedf13a779”
root=“UUID=22de856b-8099-4e62-aca2-16fedf13a779”
root=“UUID=22de856b-8099-4e62-aca2-16fedf13a779”
root=“UUID=22de856b-8099-4e62-aca2-16fedf13a779”
[root@17885d7 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 20G 0 loop /backup
loop1 7:1 0 40G 0 loop /foo/bar
vda 252:0 0 80G 0 disk
├─vda1 252:1 0 99M 0 part /boot/efi
├─vda2 252:2 0 1000M 0 part /boot
├─vda3 252:3 0 4M 0 part
├─vda4 252:4 0 1M 0 part
└─vda5 252:5 0 78.9G 0 part /
[root@17885d7 ~]# lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0
ext4 1.0 0776dafb-0a83-4ef0-9477-20a9d828d452 17.7G 4% /backup
loop1
xfs 8be2d57f-8d3a-444a-a430-02ab7daa6064 39.6G 1% /foo/bar
vda
├─vda1
│ vfat FAT16 42A7-33E4 91.7M 7% /boot/efi
├─vda2
│ xfs boot 78dc1719-34ab-4edd-95dd-5a4336b8467e 532M 46% /boot
├─vda3
│
├─vda4
│
└─vda5
xfs 22de856b-8099-4e62-aca2-16fedf13a779 69.4G 12% /
[root@17885d7 ~]# grep -lIri vda5 /etc/
[root@17885d7 ~]# grep -lIri 22de856b /etc/
/etc/fstab
/etc/kernel/cmdline
/etc/fstab_bak_260316
/etc/fstab_bak_orig
[root@17885d7 ~]# grep -lIri vda5 /var/log
/var/log/anaconda/anaconda.log
/var/log/anaconda/syslog
/var/log/anaconda/storage.log
/var/log/anaconda/lvm.log
/var/log/anaconda/journal.log
/var/log/cloud-init.log
/var/log/messages-20260412
/var/log/messages-20260419
/var/log/messages-20260405
[root@17885d7 ~]# grep -lIri 22de856b /var/log
/var/log/messages-20260412
/var/log/messages-20260419
/var/log/messages-20260405

I had the reboot freeze after 4 days uptime. I wasn’t able to watch the Ionos web remote console because it was already frozen. I’m sure this is the area (host-guest interaction) where the problem lies but I just don’t know enough about it.

After manually restarting the server from the web interface, I rebooted 3 times in a row without issue. Then I rebooted a whole bunch more times. Whole reboot is super quick, 10-15sec. I was able to watch the remote console and saw the whole process from the grub menu onward (maybe from a bios style screen but it went so quick I couldn’t catch it). No obvious issues. I diff’d the boot logs and couldn’t identify any issues with shutting down or starting up.

After a clean reboot the Ionos web interface remote console is usable for a while, probably for a couple of days.

Tried to out wait a frozen reboot on another VPS and it was well over an hour and nothing ever kicked in to notice it was frozen and reboot it. The web interface showed that it thought it was up. It was still responding to pings, too.

[mh_le]

You are not the only one, I experience this too.

[mh_le]

I tried to reboot from the CLI yesterday and the issue seems to have been fixed.

[shovas]

No luck since I last posted, numerous reboots since then, all hanged when left more than a day or two

At some point, I’ll probably try an AlmaLinux VPS with Ionos and see if it does the same thing

Might also try another one of their linux OSes to see if it happens too