Network issue - cannot connect via ssh

Rocky Linux Tech Help
1

I have an issue with network connectivity over Open-VPN tunnel, but let me describe the situation first:
Let’s say that I have two servers in the same LAN: server A (RL9.4) and server B (proxmox latest version). I connect LAN using Open-VPN client from pfSense firewall.

Let’s say that https and ssh services run on both servers and both servers are in the same network… let’s say that no VLANs are configured.

I CAN connect both servers on 443 (https) port but I can connect ONLY server B on 22 port (sshd) from VPN client. I can ping/tracert from VPN client (windows 10) both servers.

The problem is I CANNOT CONNECT server A using ssh directly from Open-VPN client. I can ssh to server B and then ssh to server A - this works fine.

So… I tired disabling temporarily firewall, SElinux on server A but it did not help. I still cannot connect ssh… Actually I can but I got time out. The message is:

C:\Users\bzc0fq>ssh -vvv 192.168.xxx.xxx
OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/config error:2
debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2
debug2: resolve_canonicalize: hostname 192.168.xxx.xxx is address
debug2: ssh_connect_direct
debug1: Connecting to 192.168.xxx.xxx [192.168.xxx.xxx] port 22.
debug1: Connection established.
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_rsa error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_rsa.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_rsa type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_rsa-cert error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_rsa-cert.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_rsa-cert type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_dsa error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_dsa.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_dsa type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_dsa-cert error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_dsa-cert.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_dsa-cert type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_ecdsa error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_ecdsa.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_ecdsa type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_ecdsa-cert error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_ecdsa-cert.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_ecdsa-cert type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_ed25519 error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_ed25519.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_ed25519 type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_ed25519-cert error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_ed25519-cert.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_ed25519-cert type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_xmss error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_xmss.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_xmss type -1
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_xmss-cert error:2
debug3: Failed to open file:C:/Users/bzc0fq/.ssh/id_xmss-cert.pub error:2
debug1: identity file C:\\Users\\bzc0fq/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1
debug3: recv - from CB(2) ERROR:138, io:0000020FC928C1D0
kex_exchange_identification: read: Connection timed out

I did not see anything suspicious in log files on server A.
I do use fail2ban on server A, but I think this is not an issue… I tried disabling it and still not luck :frowning:

Apart from ssh, samba shares are also not accessible from server A so I do not think this is ssh specific issue.

Any idea how to troubleshoot the issue further?

Thanks!

2

I would suspect routing, if nothing talks “directly”, but you have 443/tcp “ok”.

The sshd and PAM access can limit who can connect to sshd, but that does not expain CIFS.

I do usually listen with tcpdump in order to see if and what packets come and go (better limit to port 22 and not from B).

3

I thought it was a matter of ssh version but I have also tested other servers (debian like) and ssh works fine there… it does not work only with two servers with latest RL 6.4.

I have an older backup server that is also backed up - so, I will restore backup from before RL6.4 update and test it… - hope this will put more light on the issue…

4

I have restored backup from end of 2023 (rl9.3) with negative result - issue remained.
I am not sure but this might be a complex issue…

The last thing I have changed was Open-VPN configuration on pfSense firewall… will test this next… somehow…

5

I have restored Open-VPN configuration from end of 2023y and issue still remains.
Only RL servers are affected, ssh on other devices works fine.

Any thoughts on this please?

6

OK… with samba… issue solved… the reason why it did not work was missing ip adress from vpn client in host allow in smb.conf. After adding appropriate IP samba started working fine.

What regards ssh… is it possible that ssh is limited to hosts from one network? My vpn client’s ip address is from different network that LAN is.

7

I have turned sshd debugging on and I got the following output:

Jul 31 01:28:34  systemd[1]: Starting OpenSSH server daemon...
Jul 31 01:28:34  sshd[67058]: debug3: already daemonized
Jul 31 01:28:34  sshd[67058]: debug3: oom_adjust_setup
Jul 31 01:28:34  sshd[67058]: debug1: Set /proc/self/oom_score_adj from 0 to -1000
Jul 31 01:28:34  sshd[67058]: debug2: fd 3 setting O_NONBLOCK
Jul 31 01:28:34  sshd[67058]: debug1: Bind to port 22 on 0.0.0.0.
Jul 31 01:28:34  sshd[67058]: Server listening on 0.0.0.0 port 22.
Jul 31 01:28:34  sshd[67058]: socket: Address family not supported by protocol
Jul 31 01:28:34  systemd[1]: Started OpenSSH server daemon.
Jul 31 01:29:31  sshd[67058]: debug3: fd 4 is not O_NONBLOCK
Jul 31 01:29:31  sshd[67058]: debug1: Forked child 67096.
Jul 31 01:29:31  sshd[67058]: debug3: send_rexec_state: entering fd = 7 config len 3688
Jul 31 01:29:31  sshd[67058]: debug3: ssh_msg_send: type 0
Jul 31 01:29:31  sshd[67096]: debug3: oom_adjust_restore
Jul 31 01:29:31  sshd[67058]: debug3: send_rexec_state: done
Jul 31 01:29:31  sshd[67096]: debug1: Set /proc/self/oom_score_adj to 0
Jul 31 01:29:31  sshd[67096]: debug1: rexec start in 4 out 4 newsock 4 pipe 6 sock 7
Jul 31 01:29:31  sshd[67096]: debug1: inetd sockets after dupping: 4, 4
Jul 31 01:29:31  sshd[67096]: Connection from 192.168.yyy.yyy port 56420 on 192.168.xxx.xxx port 22 rdomain ""
Jul 31 01:29:31  sshd[67096]: debug1: Local version string SSH-2.0-OpenSSH_8.7

As I wrote before connection is established OK, but later I got timed out.
Unfortunately, no further debug information is available.

8

OK… finally I have found what caused the issue…
I have suricata installed on my firewall and found ET INFO Server Responded with Vulnerable OpenSSH Version (CVE-2024-6409) that blocked packages.
Changing action temporarily from drop to alert solved the issue…

1 Like
9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.