Hello everyone,
We are currently migrating around 270 CentOS 7 VMs to Rocky 9. This is how we usually install a VM:
We assign the path to the kickstart file and the static IP configuration in the kernel boot parameters. The kickstart file itself then specifies that DHCP should be used for the network configuration. This means that the network is set to DHCP during the installation - is that correct so far?
However, since the release of 9.4 we now have the following problem: A DHCP lease is retrieved during setup and the VM can also be reached via the address. As soon as the setup restarts the machine, we no longer get a DHCP lease.
So we asked our admins for the DHCP server and the following was written in the log:
2024-05-15T16:18:48.834085+02:00 dhcp1 kea-dhcp6[1810]: INFO [kea-dhcp6.leases] DHCP6_LEASE_ALLOC duid=[00:04:c5:eb:93:a8:e0:be:94:d6:7e:7b:c3:4e:9e:47:ed:dd], tid=0xb5eb18: lease for address XXXX and iaid=726689589 has been allocated for 345600 seconds
2024-05-15T16:18:48.815544+02:00 dhcp1 kea-dhcp6[1810]: INFO [kea-dhcp6.leases] DHCP6_LEASE_ADVERT duid=[00:04:c5:eb:93:a8:e0:be:94:d6:7e:7b:c3:4e:9e:47:ed:dd], tid=0x5cc4b4: lease for address XXXX and iaid=726689589 will be advertised
2024-05-15T11:34:06.022663+02:00 dhcp1 kea-dhcp6[1810]: INFO [kea-dhcp6.leases] DHCP6_LEASE_ALLOC duid=[00:04:c5:eb:93:a8:e0:be:94:d6:7e:7b:c3:4e:9e:47:ed:dd], tid=0x8c8ccb: lease for address YYYY and iaid=726689589 has been allocated for 345600 seconds
2024-05-15T11:34:06.005890+02:00 dhcp1 kea-dhcp6[1810]: INFO [kea-dhcp6.leases] DHCP6_LEASE_ADVERT duid=[00:04:c5:eb:93:a8:e0:be:94:d6:7e:7b:c3:4e:9e:47:ed:dd], tid=0x41fbdb: lease for address YYYY and iaid=726689589 will be advertised
As you can see, both VMs have sent the same DUID and IAID to the DHCP server. However, as our DHCP server now keeps the leases for 4 days, the fully installed VM naturally does not get a DHCP lease - because the leas was reserved for the DUID and IAID mentioned in the log.
After further searching through the installation logs, we came across the following:
VM1:
13:34:45,024 DEBUG NetworkManager:<debug> [1714138485.0244] device[064173afaf081f48] (eth0): ip:ac6: grace period stopped
13:34:45,024 DEBUG NetworkManager:<debug> [1714138485.0244] device[064173afaf081f48] (eth0): ip:ac6: set state: done (was pending)
13:34:45,024 DEBUG NetworkManager:<debug> [1714138485.0244] device[064173afaf081f48] (eth0): ip:dhcp6: level: set to managed
13:34:45,024 DEBUG NetworkManager:<debug> [1714138485.0244] device[064173afaf081f48] (eth0): ip:dhcp6: set state pending (was none)
13:34:45,024 DEBUG NetworkManager:<debug> [1714138485.0244] device[064173afaf081f48] (eth0): ipv6.dhcp-iaid: using 726689589 (2b:50:67:35) IAID (str: 'ifname', explicit 0)
13:34:45,024 DEBUG NetworkManager:<debug> [1714138485.0244] /etc/machine-id: 2cc35987912349aeb0038f105adc2452
13:34:45,024 DEBUG NetworkManager:<debug> [1714138485.0245] device[064173afaf081f48] (eth0): ipv6.dhcp-duid: generate lease DUID '00:04:c5:eb:93:a8:e0:be:94:d6:7e:7b:c3:4e:9e:47:ed:dd' (prefer lease)
13:34:45,024 INFO NetworkManager:<info> [1714138485.0245] dhcp6 (eth0): activation: beginning transaction (timeout in 45 seconds)
VM2
14:18:48,720 DEBUG NetworkManager:<debug> [1715782728.7206] device[f3a2582de1c38e23] (eth0): ip:ac6: grace period stopped
14:18:48,720 DEBUG NetworkManager:<debug> [1715782728.7206] device[f3a2582de1c38e23] (eth0): ip:ac6: set state: done (was pending)
14:18:48,720 DEBUG NetworkManager:<debug> [1715782728.7206] device[f3a2582de1c38e23] (eth0): ip:dhcp6: level: set to managed
14:18:48,720 DEBUG NetworkManager:<debug> [1715782728.7206] device[f3a2582de1c38e23] (eth0): ip:dhcp6: set state pending (was none)
14:18:48,720 DEBUG NetworkManager:<debug> [1715782728.7206] device[f3a2582de1c38e23] (eth0): ipv6.dhcp-iaid: using 726689589 (2b:50:67:35) IAID (str: 'ifname', explicit 0)
14:18:48,720 DEBUG NetworkManager:<debug> [1715782728.7206] /etc/machine-id: 2cc35987912349aeb0038f105adc2452
14:18:48,720 DEBUG NetworkManager:<debug> [1715782728.7206] device[f3a2582de1c38e23] (eth0): ipv6.dhcp-duid: generate lease DUID '00:04:c5:eb:93:a8:e0:be:94:d6:7e:7b:c3:4e:9e:47:ed:dd' (prefer lease)
14:18:48,720 INFO NetworkManager:<info> [1715782728.7207] dhcp6 (eth0): activation: beginning transaction (timeout in 45 seconds)
VM3
09:34:05,809 DEBUG NetworkManager:<debug> [1715765645.8094] device[1a3dc216a5e830d0] (eth0): ip:ac6: grace period stopped
09:34:05,809 DEBUG NetworkManager:<debug> [1715765645.8095] device[1a3dc216a5e830d0] (eth0): ip:ac6: set state: done (was pending)
09:34:05,809 DEBUG NetworkManager:<debug> [1715765645.8095] device[1a3dc216a5e830d0] (eth0): ip:dhcp6: level: set to managed
09:34:05,809 DEBUG NetworkManager:<debug> [1715765645.8095] device[1a3dc216a5e830d0] (eth0): ip:dhcp6: set state pending (was none)
09:34:05,809 DEBUG NetworkManager:<debug> [1715765645.8095] device[1a3dc216a5e830d0] (eth0): ipv6.dhcp-iaid: using 726689589 (2b:50:67:35) IAID (str: 'ifname', explicit 0)
09:34:05,809 DEBUG NetworkManager:<debug> [1715765645.8095] /etc/machine-id: 2cc35987912349aeb0038f105adc2452
09:34:05,809 DEBUG NetworkManager:<debug> [1715765645.8095] device[1a3dc216a5e830d0] (eth0): ipv6.dhcp-duid: generate lease DUID '00:04:c5:eb:93:a8:e0:be:94:d6:7e:7b:c3:4e:9e:47:ed:dd' (prefer lease)
09:34:05,809 INFO NetworkManager:<info> [1715765645.8095] dhcp6 (eth0): activation: beginning transaction (timeout in 45 seconds)
During setup, each VM is given the same machine ID and therefore the same DUID and then also the same IAID. Since the IP is not released after the setup (apparently the network manager cannot do this at the moment), we cannot get a lease with the correct /etc/machine-id. It looks like a bug to me, but before I open a bug report, I would like to hear your opinion.