Guest networking for KVM guest using bridge

Rocky Linux Tech Help
1

I am trying to use a traditional bridge for networking on a KVM guest. However, it doesn’t appear to be working. I have tried stopping firewalld without any success.

I assigned the guest’s NIC to “Bridge Device” with Device Name “br0”.

nmcli con show
NAME UUID TYPE DEVICE
br0 60e3740a-37bd-4362-bbf8-ba9b85fba3d9 bridge br0
docker0 e73d5ba1-3e70-44c6-9979-cfe72b91e788 bridge docker0
virbr0 3b64f39b-2347-4567-8096-d06487b8568b bridge virbr0
vnet6 d15e34e6-4b20-4c45-a6c6-6c9058dba8a4 tun vnet6
vnet7 29bd038d-b7b6-44cf-b703-c8090d3d0607 tun vnet7
br0-port1 fdbb994d-ca86-46e7-9fee-edb6017b036f ethernet eno1
eno1 1992d906-8b5d-4775-bc81-af4446f92ee4 ethernet –
eno2 45a5a13f-4cb2-4cc6-87d8-090ad3f45841 ethernet –
enp0s20f0u5u2c2 2c0d8177-a16b-46ec-9b32-7f18a1633ec3 ethernet –
ens6f0 5be4c219-653e-409e-95e7-8f23f3d7012f ethernet –
ens6f1 b0df010c-fe47-455b-ba5d-3b0beb7fa051 ethernet –

ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
link/ether 7c:c2:55:88:9e:40 brd ff:ff:ff:ff:ff:ff
altname enp61s0f0
3: ens6f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 50:7c:6f:73:04:80 brd ff:ff:ff:ff:ff:ff
altname enp207s0f0
4: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 7c:c2:55:88:9e:41 brd ff:ff:ff:ff:ff:ff
altname enp61s0f1
5: ens6f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 50:7c:6f:73:04:81 brd ff:ff:ff:ff:ff:ff
altname enp207s0f1
6: enp0s20f0u5u2c2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
link/ether e6:84:82:b5:10:b0 brd ff:ff:ff:ff:ff:ff
7: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 7c:c2:55:88:9e:40 brd ff:ff:ff:ff:ff:ff
inet 192.168.254.215/24 brd 192.168.254.255 scope global dynamic noprefixroute br0
valid_lft 69685sec preferred_lft 69685sec
inet6 fe80::160b:f4b8:181c:b364/64 scope link noprefixroute
valid_lft forever preferred_lft forever
8: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
link/ether 52:54:00:12:5f:08 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
9: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:fd:ba:af:b5 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
16: vnet6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
link/ether fe:54:00:46:cb:e2 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc54:ff:fe46:cbe2/64 scope link
valid_lft forever preferred_lft forever
17: vnet7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
link/ether fe:54:00:85:25:db brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc54:ff:fe85:25db/64 scope link
valid_lft forever preferred_lft forever

Tcpdump shows the outgoing packets:
tcpdump -i br0 port 67
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:06:00.998739 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 52:54:00:46:cb:e2 (oui Unknown), length 300
15:06:02.618349 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 52:54:00:85:25:db (oui Unknown), length 300
15:06:08.012610 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 52:54:00:85:25:db (oui Unknown), length 300

But nothing is showing up at the dhcp server. I also tried manually setting an IP address on the guest and pinging other machines. I could ping the KVM host, but no other machine.

I changed to using macvtap and now it works, but I would prefer to use a traditional bridge, so that host to guest networking will work.

2

I notice you have docker there; I recently built a Debian 12 machine with docker and a bridge and was seeing similar issues; turns out docker created iptable rules that was blocking things. Oddly on a previous machine it had been working properly but I hadn’t tested for a while.

So it’s possible the latest docker engine breaking things. If you can, try disabling docker and rebooting and see if things work.

3

Disabling Docker and rebooting appeared to work.

Is it possible to combine guests using bridges and Docker on a single machine? I have had this work in the past.

I am using the docker.com repo for docker. Perhaps it might work better with the Rocky/RHEL version?

4

So I just installed docker-ce on an RL9 instance.

Looking at nft output I see

        chain FORWARD {
                type filter hook forward priority filter; policy drop;
                counter packets 0 bytes 0 jump DOCKER-USER
                counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-1
                oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
                oifname "docker0" counter packets 0 bytes 0 jump DOCKER
                iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
                iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
        }

There’s no rule that allows forwarding for non-docker interfaces. Hmm.

A quick and nasty fix would be to change the policy to accept so it forwards all packets. I haven’t tested this but this might work

nft chain filter FORWARD '{policy accept;}'

Note that this might expose all containers to your LAN and I have no idea if this will break things.

One easy solution, of course, is to create a VM just for running docker :slight_smile:

Note that RedHat replaced docker with podman: Is the docker package available for Red Hat Enterprise Linux 8 and 9? - Red Hat Customer Portal

EDIT: I just found this page, which goes into a lot more detail! Nftables, Docker, and a default drop policy - dzx.fr

1 Like