Looking for the (as much as possible detailed) guide for configuring KVM on Rocky Linux 9.4

mfaynberg · July 17, 2024, 10:18pm

Hello everyone,
I installed fresh RL 9.4 yesterday and it seems it was successful. Now I am going to install and configure the KVM for the guest system installation. What I did was like follows:

Installed kvm-related packages:

sudo yum install qemu-kvm libvirt virt-install virt-manager
Started and enabled the libvirtd:
sudo systemctl start libvirtd
sudo systemctl enable libvirtd
Tried to install additional packages:
sudo dnf -y instal epel-release
sudo dnf -y instal libguestfs-tools

For these though I receied a failure notification reporting timeout… I don’t think it is important
at the moment, but still - why could it happen?
Created a network bridge:

Create the bridge

sudo ip link add name br0 type bridge

Bring the bridge up

sudo ip link set dev br0 up

Add an interface to the bridge

sudo ip link set dev eth0 master br0

Verify the bridge configuration

ip addr show br0

Restart NetworkManager

sudo systemctl restart NetworkManager

It was all seemingly smooth and successful, but after all these manipulations the libvirtd
service reports inactive (dead) like

LibvirtdStatus889×257 40 KB

I have 2 questions:

Should I have created br0 at all while there is already virbr0?
Why the libvrtd may become inactive?

Thank you!
Mike

JohnG · July 17, 2024, 10:34pm

The very first thing I always do when setting up for VMs is to remove the virbr0 completely. I’ve never had a use case where it makes any sense to keep it. Why it’s even there by default has always baffled me.

# virsh net-destroy default
# virsh net-undefine default
# service libvirtd restart

jlehtone · July 17, 2024, 11:03pm

It creates a distinct subnet for VMs and the host routes between that and external subnet, with NAT.
Analogous to VMs being machines at home LAN and the host being the “home router”.
If you “just” need to run a VM, the “default” is easy setup.

When we create a bridge on the host without the libvirt, we turn the host into network switch that extends the external subnet to the guests. VM on that bridge is “directly” on the external subnet.

Look at the log messages with: sudo journalctl -xe -u libvirtd

That does create a bridge. However, it is only for the runtime. Not stored. Not there after reboot.
Do use nmcli (or other NetworkManager tool) to create a more persistent bridge.

sweh · July 17, 2024, 11:05pm

Modern libvirt is activation based, rather daemon based. So there’s no persistent daemon listening; it’s started on-demand.

You can verify things are working by running something like virsh list or virsh net-list --all

Ritov · July 17, 2024, 11:19pm

Since RL9 the libvirt stack is modularized. So, some more system unit should be activated …

find /usr/lib/systemd/ |grep virt

JohnG · July 17, 2024, 11:24pm

Yep, that’s exactly where I need them to be. My VMs need to be part of the same network as the physical machines. I’ve never had a use case where that hasn’t been the case, either at home or any of the corporate networks I’ve managed over the years.

mfaynberg · July 17, 2024, 11:25pm

JohnG, thank you so much!

mfaynberg · July 17, 2024, 11:42pm

Thank you everyone!
jlehtone:

Thanks for the pointing out at the temporary nature of the ip commands. I used nmcli to do it permanent;
Now the journalctl returns the information about normal successful completion like follows:

JournalCtl809×318 46.2 KB

Sweh:
thank you! It seems likely and therefore, the libvirtd inactive state should not be of any concern then. However, both commands you suggested: virsh list and virsh net-list --all returned ampty lists. Is that normal?

Ritov:
Thanks for the suggestion. I ran the find /usr/lib/systemd/ |grep virt command and it returned a full screen of filenames. What should one do about it, sorry?

I greatly appreciate your help!

Mike

sweh · July 18, 2024, 12:01am

Yes; virsh list shows all the running VMs. If there’s no VM defined then it’ll be empty. virsh net-list --all shows all networks defined. If you deleted the default one (as was mentioned in an earlier post) then that will also be empty.

That you didn’t get any error then it likely means it’s working properly.

mfaynberg · July 18, 2024, 12:20am

Thanks! But now there is the next step… and there is no clear guidelines for it. This is about the bridge setup. I configured it using the following commands

	# Create a bridge connection
	sudo nmcli con add type bridge autoconnect yes ifname br0

	# Add an interface to the bridge
	sudo nmcli con add type bridge-slave ifname eth0 master br0

	# Bring up the bridge connection
	sudo nmcli con up bridge-br0

however one of the sources says I also need to add the full IP configuration by specifying IP address (which BTW?), DNS servers, and gateway IP like:

BR_NAME=“br0”
BR_INT=“ens1f0np0”
SUBNET_IP=“XXX.YYY.ZZZ.WWW/24”
GW=“XXX.YYY.ZZZ.GGG”
DNS1=“8.8.8.8”
DNS2=“8.8.4.4”
sudo nmcli connection add type bridge autoconnect yes con-name ${BR_NAME} ifname ${BR_NAME}
sudo nmcli connection modify ${BR_NAME} ipv4.addresses ${SUBNET_IP} ipv4.method manual
sudo nmcli connection modify ${BR_NAME} ipv4.gateway ${GW}
sudo nmcli connection modify ${BR_NAME} ipv4.dns ${DNS1} +ipv4.dns ${DNS2}
sudo nmcli connection delete ${BR_INT}
sudo nmcli connection add type bridge-slave autoconnect yes con-name ${BR_INT} ifname ${BR_INT} master ${BR_NAME}

Should I?
As of now command ip ad returns like below:

Thank you!

Ritov · July 18, 2024, 9:39am

Some socket activated services should be enabled, here a one liner to do it (as root and then reboot):

for drv in qemu network nodedev nwfilter secret storage interface; do systemctl enable --now virt${drv}d{,-ro,-admin}.socket; done

jlehtone · July 18, 2024, 1:28pm

The networks you have managed. I’m in the same boat; usually manage the network and can thus hand out addresses for the guests. However, I did once use the default (routed+NAT) because I could not ask for separate address for a VM.

I don’t delete the definition of the “default”; I just disable its auto-start. It being autostarted, FirewallD, etc, seem to be convenience features for “simple use”. We could ask from Red Hat why they deem that useful.

How do you access your host? Is it local login, or remotely (via ssh)?
If it is remotely, does your host have just one network interface or many?

If you access remotely and the host has only one IP address, then changing the network config can easily leave you with no connection to the host. That would be akward.

In other words, your “admin connection” dictates how careful you have to be.

I did wrote earlier that the “bridged” setup is like a network switch. Unmanaged switches have no IP address. Guests can talk via the bridge to outside even when the host has no IP address at all.

Naturally, the host preferably does have an IP address, so that it can do things like dnf up. If the host has one interface and the interface is a “port on switch” br0, then the IP configuration must be on the br0, not on the interface.

Don’t you have DHCP server on your (external) network? DHCP is a convenient (and default) way to get the IP config. Setting config manually is rather static.

Compare:

virsh net-list --all
sudo virsh net-list --all

Regular account sees only the things that that account has started.
Root account sees only the things that root has started. (Most?) Libvirt services run as root.

sweh · July 18, 2024, 3:13pm

Hmm, on my RL9 machine as a non-root user I see everything. Maybe because I’m in the libvirt group…

$ id
uid=500(sweh) gid=500(sweh) groups=500(sweh),0(root),10(wheel),18(dialout),494(libvirt)
$ virsh list
 Id   Name          State
-----------------------------
 1    plexserver9   running
 2    pinky9        running

$ virsh net-list --all
 Name      State      Autostart   Persistent
----------------------------------------------
 default   inactive   no          yes

sweh · July 18, 2024, 3:18pm

If it helps, here’s my network setup. It’s a little complicated because I use VLANs (10=main LAN, 11=guest, 12=IoT, 13=test) so I build 4 bridges, one for each VLAN. The naming scheme I use possibly isn’t needed any more, but it matches what it used to look like on C7, so I’m sticking with it.

Only VLAN 10 gets an IP address 'cos that’s how I access the machine.

Now I can put VMs on any (or all) networks.

nmcli connection delete enp1s0.10 enp1s0.11 enp1s0.12 enp1s0.13 br-lan br-guest 
br-iot br-temp  

nmcli connection modify enp1s0 ipv4.method disabled ipv6.method disabled

nmcli connection add type bridge con-name br-lan ifname br-lan bridge.stp no ip4 10.0.0.133/24 gw4 10.0.0.1 ipv4.dns "10.0.0.1 10.0.0.5" ipv4.dns-search "spuddy.org" ipv6.method disabled
nmcli connection add type bridge con-name br-guest ifname br-guest bridge.stp no ipv4.method disabled ipv6.method disabled
nmcli connection add type bridge con-name br-iot ifname br-iot bridge.stp no ipv4.method disabled ipv6.method disabled
nmcli connection add type bridge con-name br-temp ifname br-temp bridge.stp no ipv4.method disabled ipv6.method disabled

nmcli connection add type vlan con-name enp1s0.10 ifname enp1s0.10 vlan.parent enp1s0 vlan.id 10 slave-type bridge master br-lan
nmcli connection add type vlan con-name enp1s0.11 ifname enp1s0.11 vlan.parent enp1s0 vlan.id 11 slave-type bridge master br-guest
nmcli connection add type vlan con-name enp1s0.12 ifname enp1s0.12 vlan.parent enp1s0 vlan.id 12 slave-type bridge master br-iot
nmcli connection add type vlan con-name enp1s0.13 ifname enp1s0.13 vlan.parent enp1s0 vlan.id 13 slave-type bridge master br-temp

I like the old bridge-utils for seeing things, so this is the result

% brctl show
bridge name     bridge id               STP enabled     interfaces
br-guest        8000.b0416f0e52ab       no              enp1s0.11
                                                        v-pinky9-guest
br-iot          8000.b0416f0e52ab       no              enp1s0.12
                                                        v-pinky9-iot
br-lan          8000.b0416f0e52ab       no              enp1s0.10
                                                        v-pinky9
                                                        v-plexserver9
br-temp         8000.b0416f0e52ab       no              enp1s0.13

gerry666uk · July 18, 2024, 4:11pm

The original post title is “Looking for the … guide”.

Here is the guide:

Note the part about starting the services
Chapter 2
2.1. Enabling virtualization on AMD64 and Intel 64

See also
Chapter 17. Configuring virtual machine network connections
see the note about virbr0

mfaynberg · July 18, 2024, 7:00pm

Gentlemen, thank you for your input! It is a lot for me to chew on now

mfaynberg · July 18, 2024, 9:13pm

Guys,

one more question: is there anything I set up incorrectly when created a bridge if
a) the ip addr show br0 returns

and
b) when I try to create a virtual machine it says “no suitable network” not obviously seeing br0 like follows
NoNetwork

What I did while creating the interface was:
sudo nmcli con add type bridge autoconnect yes ifname br0
sudo nmcli con add type bridge-slave ifname eth0 master br0
sudo nmcli con up bridge-br0

Thank you!

jlehtone · July 19, 2024, 8:48am

Probably.

You did not have device named eth0 earlier.

Plain nmcli shows many details, but here I would look at:

nmcli d s
nmcli c s
ip ro

mfaynberg · July 19, 2024, 5:09pm

Thank you! Actually I reconfigured the bridge from scratch… what i see now is

and the response to
nmcli conn show br0 returns the following lists

It seems OK to me but when I start VM Manager and try to start creating a new VM, it says “No active connection to install on”.

What would you recommend doing?

jlehtone · July 19, 2024, 5:24pm

Note: one can usually copy-paste text from terminals (and use code tags) screenshots are inconvenient.

Note2: There is one thing that nmcli can’t do on create connection and that is to bind by MAC-address (the 802-3-ethernet.mac-address); it requires ifname (aka connection.interface-name). That is not important here, where the name of interface is persistent. The interesting part is that the (Anaconda) installer creates connections that bind by MAC, rather than by name.

If you have no IPv6 in usa, then you can disable it:

sudo nmcli c mod ens1f0np0 ipv6.method disabled

Try sudo virt-manager

Looking for the (as much as possible detailed) guide for configuring KVM on Rocky Linux 9.4

Installed kvm-related packages:

Create the bridge

Bring the bridge up

Add an interface to the bridge

Verify the bridge configuration

Restart NetworkManager