I am having a issue where my ssh drops with a connection lost error. After this happens I am unable to reconnect and pings stop working. If I ping the gateway from the server, then everything works again for a period of time. Is there something I have to do with firewalld to reply to arp requests from the gateway? Is there something else that might be causing this? Thanks
Create in your ~/.ssh directory a filed called config with the following:
Host *
ServerAliveInterval 300
ServerAliveCountMax 2
that will send keepalives at a regular interval, just in case it’s due to lack of activity on the ssh session.
The .ssh directory you will find in your home directory, and in here you can create the config file.
You don’t normally need to do anything with firewalld, it’s most likely some other firewall in between your server and computer that is causing the problem.
This server has a direct connection to the internet. no router in between. I can understand a ssh timeout but why does being able to ping the server stop at the same time? I also am unable to re-connect to the server for a period of time afterwards? The timeout would kick me out if idle but I should be able to immediately re-connect right? I should add that I have a private network interface as well that does not experience this issue just the default public interface. I have added the logged dropped packets and I do not see the ip I’m coming from being listed there.
How is this direct connection provided? I have a VPS at a hosting provider with a public IP but there will be a router somewhere that the packets go through.
Are you 100% sure the problem is firewalld? Did you stop firewalld and the problems went away? I’m more inclined to think this is more a network issue than a firewalld issue unless firewalld has been configured incorrectly causing this behaviour. In which case we need to know what config has been applied to this server that is causing this since on a default install this isn’t possible.
Yes true on the providers side, just not one in house. Its a bridged connection directly to the provider.
work (active)
target: default
icmp-block-inversion: no
interfaces:
sources: x.x.x.x
services: dhcpv6-client
ports: 2222/tcp 80/udp 80/tcp
protocols:
forward: yes
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
I have the public facing interface in the drop zone. I haven’t tried without firewalld enabled. The public interface being the only one having the issue.
I’d be tempted to just put the public interface in the public zone, and then just remove from the public interface the services/ports you don’t need. It could well be the problems have been cause because you put it in the drop zone which means it could well be dropping everything before it’s even had a chance to check other zones.
So my issue with the Public zone is ping flooding. The one thing that enabling logging has shown is that the amount of probes is crazy. I hope that this wouldn’t be the problem as pings and ssh do work from time to time meaning that the firewall is failing to do its job if that is how it is suppose to work. I found one site that talked about turning off icmp but it wasn’t real clear if it turned it off for all zones or not.
It is so random on when it comes and goes:
Pinging x.x.x.x with 32 bytes of data:
Request timed out.
Request timed out.
Reply from x.x.x.x: bytes=32 time=19ms TTL=60
Reply from x.x.x.x: bytes=32 time=27ms TTL=60
In which case, if you want all zones to be evaluated, then:
- Remove the interface from the drop zone.
- Set drop zone as the default zone.
That way, you get to evaluate all zones, rather than what you have right now, where the public interface is tied to the drop zone and potentially causing the situation you have.
That would then mean, unless a rule is satisfied, say work zone, then it will just hit the drop zone as it being the default. At least in theory…
I’m gonna test this on a VM now, I’ll report back.
Right well, it seems the interface must be assigned to a zone. So cannot do what I thought was possible.
So, my public interface if you like:
[root@rocky-fwtest ~]# firewall-cmd --get-default-zone
drop
[root@rocky-fwtest ~]# firewall-cmd --info-zone=drop
drop (active)
target: DROP
icmp-block-inversion: no
interfaces: enp1s0
sources:
services:
ports:
protocols:
forward: yes
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
I then have a zone where I want to allow access:
[root@rocky-fwtest ~]# firewall-cmd --info-zone=ext-zone
ext-zone (active)
target: default
icmp-block-inversion: no
interfaces:
sources: 10.10.11.0/24
services: ssh
ports:
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
So this is the IP address that I’m allowing SSH and ping from. I then have another zone for my internal network, so:
[root@rocky-fwtest ~]# firewall-cmd --info-zone=int-zone
int-zone (active)
target: default
icmp-block-inversion: no
interfaces: enp7s0
sources: 10.1.9.0/24
services: ssh
ports:
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
Also, only one default route configured:
[root@rocky-fwtest ~]# ip route
default via 10.1.7.1 dev enp1s0 proto static metric 100
10.1.7.0/24 dev enp1s0 proto kernel scope link src 10.1.7.11 metric 100
10.1.9.0/24 dev enp7s0 proto kernel scope link src 10.1.9.11 metric 101
The internal network doesn’t have an internal route, therefore not confusing things and potentially causing packet drops by going to the wrong default route by mistake.
So far I’m not experiencing any issues with ping/ssh to the server from the IP address allowed in the ext-zone. All of my other networks in the office, different subnets cannot ping the server despite the packets being routed by the main firewall/router. So I’ve managed to more or less simulate your setup.
The only thing I’d ask now is, do you also only have a single default gateway configured, or do you have another one configured for your internal network as well?
Same as you only 1 default route. moving the public interface to public with the default zone to drop does allow ping to work from public.
Any security profiles applied to the server when it was set up? I know CIS profiles can make certain SSH configuration changes that block me as a user when I enter the wrong password on an SSH connection and after 3 attempts get blocked for a certain amount of time. So maybe there is also something else applied about inactive SSH sessions that disconnects them?
About the only other thing I can think of right now. My server that I’ve just installed doesn’t have any security profiles applied. That and I use the .ssh/config file with the ServerAlive entries to send keepalive from my laptop to the SSH server I’m connected to.
none. disabled SElinux. I can be active with the ssh session when it kicks me out.
The only other difference, on my zones I set up, ext-zone and int-zone, I don’t have forward enabled. Yours has forward: yes and mine has forward: no only the drop zone has forward: yes
If the work zone you posted is the one you are using, maybe change forward to no here.
I changed it but no luck
TBH I’m not sure firewalld is the problem here. I can’t replicate it. I’m convinced there is a problem between your server and the gateway especially since when you lose access, you ping the gateway from your server and then it starts working again.
About the only real way to verify is to stop firewalld and see if the problem persists or not. That does mean however all your ports will be open for that period. But if you are able to replicate the problem relatively quickly in that it happens within a few mins, then the risk would be pretty low.
Either that or some power management on the server is dropping the network connection.