Network issues after upgrading from 9.4 to 9.5

Hi everyone,

I’m having issues upgrading my Rocky Linux machines from 9.4 to 9.5.
They are connected straight to the internet on a public IP and firewalled with nftables ().
It runs smooth on 9.4. But when I upgrade to 9.5 there is no remote connection possible to all the services on the machine except SSH.
Then I rebooted with the old kernel. And everything works as it should.
With the new kernel I also disabled nftables but it still has the issues.

Now I disabled in the yum config to upgrade the kernel, but this is far from ideal.

Issue with kernel 5.14.0-503.21.1.el9_5.x86_64 but also another machine later with (strange enough a earlier version: 5.14.0-503.19
works good with kernel 5.14.0-427.42.1.el9_4.x86_64

I would love to keep the kernel also up to date.
Is there a problem in the new kernel? Of something I can check / reconfigure?

Machines run on VMware ESX.

Thanks in advance.

Which services exactly, how did you test, and what error did you get?

Ow I’m sorry.
It was HTTP and HTTPS not working, timeout after a while via the browser.
And also SNMP was not working (via Nagios check: check_snmp).

It does sound like firewall, but you say you disabled it. Do the rules look the same in old vs new?

Did you try http locally, e.g. using curl, does it respond straight away?

Yes the rules are the same.
And indeed, when I checked on the console using wget it worked instanlty.
I could only check on IPv4 at that moment, because I don’t have an ISP around with IPv6. ( I can check but takes more effort)

I use a “table ip filter” for IPv4 and a “table ipv6 firewall” for IPv6 traffic.
This is the firewall script where I for this example replaced the actual IP-adresses with 99.99.99.99 fictive IP-adres;
It’s just a script to allow http/https public and to allow ssh/snmp/ping from a specific address.

#!/usr/sbin/nft -f

flush ruleset

table ip filter {
        chain input {
           type filter hook input priority 0;
           policy drop;

	    # allow connection from loopback
           iifname lo accept;

    # established/related connections
    ct state {established, related} accept;

    # drop invalid connections
    ct state invalid drop;

	# allow connection for ping from specific IP-adresses.
ip saddr 99.99.99.99 icmp type echo-request  accept;  #my office
ip saddr 99.99.99.99 icmp type echo-request  accept;  #my monitoring


	    # allow connection from management for ssh
           ip saddr 99.99.99.99 tcp dport ssh accept;
           ip saddr 99.99.99.99 tcp dport ssh accept;

           ip saddr 99.99.99.99 tcp dport 161 accept; #monitoring via SNMP
           ip saddr 99.99.99.99 udp dport 161 accept; #monitoring via SNMP


	    # allow webserver
	    tcp dport http accept;
	    tcp dport https accept;

           # everything else
           reject with icmp type port-unreachable
 
        }
        chain forward {
                type filter hook forward priority 0;
        }
        chain output {
                type filter hook output priority 0;
        }
}

table ip6 firewall {
  chain incoming {
    type filter hook input priority 0;

    # established/related connections
    ct state established,related accept;

    # invalid connections
    ct state invalid drop;

    # loopback interface
    iifname lo accept;

    # icmp
    icmpv6 type {echo-request,nd-neighbor-solicit,nd-neighbor-advert,nd-router-solicit,nd-router-advert,mld-listener-query,destination-unreachable,packet-too-big,time-exceeded,parameter-problem} accept;

    # allow webserver
    tcp dport http accept;
    tcp dport https accept;




    # open udp ports
#    udp dport {...} accept;

    # drop everything else
    drop;
  }

      chain forward {
                type filter hook forward priority 0;
        }
        chain output {
                type filter hook output priority 0;
        }



}

Can I re-check, when you say you “disabled nftables” in the original post, what exact command did you use?

In addition, where it says “straight to the internet”, do you mean the client (web browser), is making the request over the internet, if so, are you able to test connecting a second machine using a network cable? It seems odd that ssh still works.

Hi gerry666uk, Thanks for you replies.
I’m not sure which command I used to disable. And I have to recreate the situation to be sure.
But I see something else now.
In the yum update I see also updates for

firewalld.noarch                      1.3.4-9.el9_5                      baseos
firewalld-filesystem.noarch           1.3.4-9.el9_5                      baseos
iptables-libs.x86_64                  1.8.10-11.el9_5                    baseos
iptables-nft.x86_64                   1.8.10-11.el9_5                    baseos

So it seems there are multiple firewalls installed. I will uninstall firewalld and iptables and then try it again with the new kernel. Maybe they interfere with eachother.

There ought to be at most two services to choose from: firewalld.service and nftables.service. The former is the default, so you should have disabled it in order to use the latter.
The “iptables” included in el9 is a translator that converts iptables-syntax to nftables syntax, not a service.

You can see the active rules that are in the kernel with:

nft list ruleset
1 Like

Hi jlehtone, thanks for you reply. That makes it more clear with the different firewalls.
The nft list ruleset shows the right rules (those I expected)

After your reply I removed the firewalld and iptables and installed the nftables (again, because it was removed with the yum remove listed below)

First I removed the exclude=kernel* from the /etc/yum.conf (because I placed this lock before)
Then:

yum remove firewalld iptables
yum install nftables
systemctl enable nftables
yum update
reboot

kernel old: 5.14.0-427.42.1.el9_4
kernel new: 5.14.0-503.19.1.el9_5

Works like a charm now!
The issue is solved! :grinning:
Thank you all for reading and gerry666uk and jlehtone for replying to my question. :pray:

For additional info:
One can start and stop a service. e.g.

systemctl start firewalld
systemctl stop firewalld

and one can disable and enable a service. e.g.

systemctl disable firewalld
systemctl enable firewalld

A disabled service should not get started on boot.
However, you can call the start and stop on disabled service.

More importantly, other systems/services can start a disabled service too.
For that is masking:

systemctl mask firewalld

This makes a symlink to /dev/null
If anything tries to start masked service, well systemctl does not do that for /dev/null

Firewall and display manager are examples of services for which there are many alternatives, and where you would mask the unwanted ones (if you have them installed).

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.