I just installed Rocky 8.5. Generally works like a charm!
I’m now facing one problem that I already previously had with CentOS 8. It seems to be a type of incompatibility between libvirtd and nftables. I’m using nftables rather than firewalld. After nftables is enabled, libvirtd no longer manages to bring up the default network. Instead, I get the following error in the system journal:
libvirtd[2311]: internal error: Failed to apply firewall rules /usr/sbin/ip6tables -w --table filter --list-rules: ip6tables v1.8.4 (nf_tables): table `filter' is incompatible, use 'nft' tool.
It would appear that libvirtd is using iptables instead of nftables somehow. Does anybody know how that default behavior can be changed?
Yes, but I want to use nftables directly via custom rules defined in /etc/sysconfig/nftables.conf. I would like to disable firewalld. The part I don’t understand is why disabling firewalld should somehow prevent libvirtd from bringing up the default network. Is that expected? Does libvirtd depend on firewalld managing nftables?
The error message points to “tables”. Netfilter had tables: filter, nat, mangle, …
Each table had some built-in chains: PREROUTING, INPUT, FORWARD, …
The nftables has no pre-defined chains nor tables. You create a chain and optionally bind it to “hook”, like filter.
I’d guess that the nft-version of iptables does not support option “table” as there is no obvious translation.
A question is: does libvirtd talk to firewalld differently than when it calls ip6tables directly?
The alternative guess is in the fact that the firewalld creates the familiar chains, like INPUT, and the libvirtd/iptables accepts “table” if it finds those.
How dynamic rules do you need with the VMs? If they are relatively static, then you could create them with nftables.conf (if you can disable all rule generation in libvirtd).
PS. I have a custom ruleset file in /etc/nftables/ (where there are examples) and merely include it in /etc/sysconfig/nftables.conf
The rules are static. Such a solution (adding the rules to nftables.conf and disable all rule generation in libvirtd) would be optimal! … How do I disable rule generation in libvirtd?
I have once found those libvirtd scripts – might have been CentOS 6 version – to tamper where (in INPUT) it inserts rules. Not the proper approach.
Almost all of my setups use bridged network. That adds no rules.
Websearch hit one: OpenStack Docs: Disable libvirt networking
So, the rules come as libvirtd “creates virtual network”. (I have that “default” disabled; no auto-start.)
Oh, yes, the upstream documentation:
If you find explanations from that: ok, if a solution: great.
The workaround option is naturally to not create virtual network with libvirtd, but on the host before libvirtd starts. (You need unattached bridge and a dnsmasq that listens on it.) Then libvirtd simple attaches VMs to that bridge (like plug machine into switch) and host routes. (This is how my bridged systems are, except that they are bridged to network outside, not routed.)
I did as you suggested and disposed of the virtual network altogether, creating the bridge manually. I like this setup, seems much more stable and less magic going on!
I still think there is some bug in libvirtd because of how it cannot interact with nftables properly, but since I have an alternate setup that I prefer it’s no longer a concern for me.
