I maintained system images that were swapped back and forth between pxeboot/nfsroot and locally installed for many years (pre-RedHat redhat linux through RHEL6) but only recently (Rocky 8.6) tried migrating the procedure to EL8. I’ve run into a problem I’ve not seen before: all commands return file-not-found, including intr’s that shouldn’t be impacted by missing paths or devices.
PXELINUX 6.04 PXE Copyright (C) 1994-2015 H. Peter Anvin et al
No DEFAULT or UI configuration directive found!
boot: ls
Loading ls… failed: No such file or directory
boot: unset root
Loading unset… failed: No such file or directory
I’m pretty sure I’ve just neglected to include packages/files that are no longer bundled or 'depends-ed like they previously were, mainly because this has been the problem for 99% of all migration issues I’ve had since going to EL8 (payloads moving to new package names, etc). For example on the pxeboot side of things it’s normal to have /boot/loader/entries/ show up correctly, /boot/grub2 and /boot/efi/EFI/rocky be empty, and /boot/efi/EFI/BOOT be missing. So when transferring to local storage you’d add grub2-efi, grub2-efi-modules, shim-x64, and the appropriate grub2-efi-{stub}-modules (.noarch) at a minimum. But this still only provides the /boot/efi/EFI/BOOT/ bits and the /boot/efi/EFI/rocky/{grubx64,mmx64,shimx64,shimx64-rocky}.efi, the rest is still missing. It gets to the boot: prompt but can’t load for any commands, even “set”, so clearly I’m still missing major parts of the infrastructure. But copying from a system with the same packages and versions installed locally (plus the usual depmods, mkinitrds, grub2-mkconfigs, etc, to correct for the fact that the source booted pxe and the dest will boot efi), that doesn’t alter behavior. The stock docs aren’t particularly helpful beyond “does the referenced path actually exist” (they do). Any ideas what other content I’m missing?
Background:
The system(s) in question are primarily pxebooted with the old saw of a readonly nfsroot plus read/write nfs fs slices for /etc, /root, /var, and so on, for times when the system images (aggregate content in the server-side nfsroot’s and fs slices’ file trees), change often across many downstream provisioned clients. They also have local block storage for other production times when the system images are relatively static (making performance the greater priority). For reasons I’m not at liberty to disclose in public forum the physical systems’ SMBIOS needs to be set to only allow pxebooting from the first ether device. My go-to solution for several decades has been to just (scripted)…
- pxeboot the target box and log in
- set up the appropriate local block device with disklabel, partitions, and filesystems
- temporarily mount all local logical devices under /mnt and rsync / to /mnt with the usual omissions (/dev, /sys, /proc, …)
- bindmount the usual /proc, /dev, /sys, /run, /proc/pts into the tree under /mnt
- chroot into /mnt, reinstall grub2-efi grub2-efi-modules shim-x64, and grub2-mkconfig into /boot/grub2/grub.cfg and /boot/efi/EFI/rocky/grub.cfg
- exit the chroot, tear down the mounts, shut down the system
- flip the corresponding pxeconfig from pointing to upstream tftp-provided kernel and initramfs to something like
set default=0
set timeout 10
menuentry “Boot from local disk” {
search --file --no-floppy --set=root /EFI/redhat/grubx64.efi
chainloader ($root)/EFI/redhat/grubx64.efi
}
- hit the box’s BMC to power the system back on
Prior to EL8 the only complication was that, because bios firmware boot options were limited to a single pxe device, I couldn’t “localboot 0”, etc. But chaining worked fine, at least through the end of EL6 and maybe as recently as 7.4(?).