Hi,
We have a growing footprint of 700+ (sadly, necessary) baremetal servers currently running CentOS 8.1. Our field installers use Windows as a matter of corporate policy. For new deployments, we’ve distributed USB sticks burned with the dvd image that they can update by plugging into their workstations and running a script that downloads our latest kickstart, grub config, and setup scripts.
When attempting to migrate this procedure to other RH derivatives (Oracle, Alma, Rocky) I couldn’t get the setup to install from the USB stick until someone over in the Rocky chat advised me to dd the dvd image instead of using ISO mode (in Rufus). That works but unfortunately prevents our field staff from updating their boot media (because Windows) and would force us to rebuild our deployment images and re-flash all the USB sticks on every change to the kickstart.
I’ve found a functional but terrible workaround for this for ISO-burned media, whereby we serve the AppStream/BaseOS repos from the boot media over http in the anaconda environment, and point our kickstart repo directives at the local servers. It works great but we’d rather not.
We recognize this (our?) problem isn’t specific to Rocky. It looks like it was caused by a change to anaconda in RH 8.4.
I’m sure we can do better. Can anyone recommend:
- A more appropriate place to post this
- A process that doesn’t require our field installers to download and burn a 9+ GB image every time we make a change to the kickstart; we do not wish to–and often can’t–network install
- A working (Linux) procedure to repackage the Rocky dvd iso; I’ve tried several including from RH but can never get past installation source even with the stock image unmodified beyond extract → copy →
mkisofs → isohybrid → dd
Thanks!
It seems your problem at hand is inability to transfer custom Rocky installation image to usb device.
Why do you create cd/dvd installation image when you need usb installation image? (they are not the same)
There is piece of software called: geteltorito - it can convert cd/dvd bootable image into usb bootable image. However I have no idea what rufus will think about that. (I use geteltorito for bios upgrades - lenovo used to provide cd iso for laptops without cd drive)
Some thoughts on your process/situation:
- “…Windows as a matter of corporate policy” - what’s the point of policy that does not let you do your job?
- 700+ servers installed with USB - well…I am afraid to ask… what are the reasons?
- server definition - is it a real server with remote management (IMM, iDrac…)? With this number of servers I can’t see how they wouldn’t be, but then there would be no need for “field installers” and rufus-ing installation image to usb… more details needed…
- “field installers” (plural) suggests multiple locations… how is the network configured between locations? 700+ servers that is minimum 20+ full rack cabinets… (assuming servers are 1U). more info needed about network configuration and topology…
- (assuming network can handle it) over network, automated install would be the way to go
- why do you think your workaround (local repositories) is terrible? local repository (per company or even per location is the way to improve integrity and stability of installation process (by reducing human factor with updating usb installation image)
- put kickstart files to local server (https), there is no need to put them on installation image at all (alternatively second usb for kickstart file?)
- you seem to be less than enthusiastic about network install - but that is way to standardize and simplify software deployment… you need to get rid of usb installation completely.
- “A process that doesn’t require our field installers to download …” - NO, you need a process that doesn’t require field installers.
- lack of automation / manually handling deployment of such number of servers, sooner or later will have (negative) impact on the perceived quality of work of the IT department. That can lead to dissatisfaction of bussiness/management part of your organization, and from that there is only one step to outsourcing the unmanageable “IT problem”.
You definitely have a uniquely interesting environment to work with.
1 Like
Hello.
It is true we do not recommend rufus because it does not play well with ISO hybrid images. This is how it has been since Enterprise Linux 7. However, if you are going to continue to use rufus, you need to run it in dd mode, which is what rufus allows you to do on Windows. Rufus is supposed to notify you of this when you select a hybrid image. You can also use:
- Fedora LiveUSB Creator
- Win32 Disk Imager
- Rawrite32
- dd for windows
Alternatively, if large USB drives are being used (eg 16gb), E2B can be used instead where the ISO can be dropped directly on the drive. (The caveat is UEFI may not work as intended).
You’re preaching to the choir. =/
As I mentioned, there is not always a network connection at the time of install, so unfortunately network install isn’t an option.
My workaround doesn’t use a network external to the server, it runs an http server in the Anaconda boot environment to serve the repos from the attached media. Doesn’t that seem a bit silly when the repos are already mounted (/run/install/repo)?
Honestly I don’t think this has much to do with Rufus. The process we’re using has worked up to EL8.1 and no longer works on EL8.4. I believe it was broken by a change to Anaconda; specifically, it no longer defaults to cdrom per this change: Don't set up sources in the refresh method of the Source spoke · rhinstaller/anaconda@8452920 · GitHub
Anaconda is not the issue. If you can drop an ISO on a E2B usb or convert it to imgptn and drop it on a E2B usb, it works. If you image a USB using rawrite32 on windows (or the other aforementioned tools) or dd on linux, it works.
Since Enterprise Linux 7, there is special partitioning being used (isohybrid), so most tools that you would use on windows do not work correctly. This includes: unetbootin, multibootusb, universal usb installer, and rufus.
In the case of rufus, the correct options must be selected for an image to work correctly for enterprise linux (whether it’s rhel, centos, or rocky). If it was working before, then the correct settings were being used initially (dd mode). There have been numerous reports of broken images even before EL8 using Rufus.
https://forums.centos.org/viewtopic.php?f=47&t=70594&sid=aa1ac06a6be70acc5291eeb12c5f8fd9&start=30
https://forums.centos.org/viewtopic.php?f=54&t=76708&p=322748&hilit=rufus#p322624
The recommended tooling will continue to be:
- Fedora LiveUSB Creator
- Win32 Disk Imager
- Rawrite32
- dd for windows
I can assure you we’ve been burning our USBs in ISO mode with CentOS 8.1 and did not experience install issues until we started testing migration to EL8.4 CentOS alternatives.
As the maintainers of the install image my team are able to dd the USBs. It’s just not possible for our field staff to pull down updated kickstarts when the drives won’t mount on Windows.
Thanks for suggesting E2B; I’ll have a look at that.