Idle system issuing HTTPS requests

If I fire up my Rocky 9.5 laptop and login then connect to my server via ssh and start a tcpdump of all traffic then walk away for a while, the resulting tcpdump trace shows a continual stream of HTTPS requests from the laptop to sites such as fedoraproject, amazonaws and google.
What could cause this unsolicited traffic?
How can I stop it?

GUI with gnome? → extensions.gnome.org version checks

Je ne comprends pas.
What do you mean?

Yes, but what if you don’t connect to this server? I’m trying to clarify that this will happen anyway, with just the laptop?

Yep, connecting to the server via SSH won’t have anything to do with it. This is why people have firewalls to control incoming/outgoing traffic. Configure your filewall to only allow outbound traffic from specific computers and to specific sites. If something doesn’t work, like dnf cannot find updates for your system, then you will have to unblock the laptop so that it can connect to Rocky mirrors as well as EPEL, etc.

When logged into the GUI (if gnome) the desktop environment checks if installed extensions are up to date. More sources of connections could be time synchronisation, package/repo metadata update …

1 Like

I used the server to do the trace as it has scripts to analyse traffic and is the gateway/firewall for all other devices on my net.
Rerunning the trace on the laptop with no other user activity apart from the terminal session running the trace gives the same results.
A steady stream of thousands of HTTPS packets to Amazon, Google etc.
Do Rocky repositories live on these sites?
Why would there be this constant stream all day every day?
(Time sync does not use HTTPS.)

Yes, Rocky mirrors do exist on GCP, I expect there are also some on AWS as well. There are mirrors provided by Rocky, and then the Rocky community as well - a list of which you can find here: Mirrors - Mirror Manager

OK. But even so that doesn’t explain why these streams go on all day every day without any interruption and cause my bandwidth allocation to fill up two-fold.
This has only been an issue for the past six months.

The easiest solution for real time monitoring might be to install nethogs from EPEL repository which shows you the network bandwidth used by each process.
I think this is the easiest method to spot what is going on.
( My freshly installed Rocky 9.5 ( Server with GUI) is completely silent, nothing is send to *:443).

If the streams really do stay up for a while, and are transferring data, it should be possible to open a terminal as root, then do
ss -ntp
But if the connections are just a fraction of a second, and there’s no data transfer, that command won’t work.
To do a more concise test, it would be better to confirm the laptop has a pure clean install of Rocky, and don’t go via the gateway/server, connect it direct to the internet. Make sure no apps are running when testing.

I have tried nethogs several times but get output like:

172.67.74.82:443-203.2.199.101:33256/0/0 0 0.0695312
18.64.50.106:443-203.2.199.101:43518/0/0 0 0.0695312
99.86.212.18:443-203.2.199.101:50918/0/0 0 0.0695312
sshd: scldad@pts/3/1474/1000 0.690234 0.0644531
34.107.221.82:80-203.2.199.101:54474/0/0 0 0.0515625
23.217.103.66:80-203.2.199.101:44978/0/0 0 0.0257812
203.2.199.4:25-178.162.136.160:42760/0/0 0 0.0144531
34.160.144.191:443-203.2.199.101:38930/0/0 0 0

where only the sshd line looks anything like the documentation.
What do the other lines mean?

I think what you see there is forwarded network traffic, that is why you see two IP addresses on a line. (Similar to tcpdump output …)
Locally IP 203.2.199.101 is talking to the outside world and the machine with nethogs is forwarding the traffic.
Interestingly two request use plain HTTP on port 80, so if you monitor outgoing traffic to port 80 on 203.2.199.101 (with tcpdump’s verbose flag or wireshark) you can see the full URL requested ( not only the host address with HTTPS).
The IP 178.162.136.160 is talking to a (local) SMTP server on 203.2.199.4:25
If 203.2.199.101 is a Linux machine you can try to run nethogs on that machine locally.

On a minimal install of Rocky, I don’t see this problem which means whatever else you have installed is causing this.

maybe if I waited a bit, I might see if dnf actually does anything to refresh it’s data. I would have expected those nethog processes to show the PID number, and you would then be able to find out what application you have installed is the cause of it.

That said, 0.06KB is not huge bandwidth amounts but since that was probably only running a few seconds, you would probably need to leave nethogs running for say 1 hour to actually get an idea of how much bandwidth is being used up. Either way, once you find the PID of the process responsible, you should get the answer you are looking for.

If you run as root:
ss -apn
you can see what processes have open connections and work from there.

What does:
Unknown connection: 203.2.199.101:23-2.51.193.187:33585
mean (in nethogs output)?

The host with IP 2.51.193.187 uses port 33585 to connect to IP 203.2.199.101 on port 23.
Port 23 is telnet port, so someone is trying to open a telnet connection.
You see this in nethogs output because the traffic is routed through the host you run nethogs on.

What I meant to ask is the meaning of “unknown connection”.
Another example is:
Unknown connection: 203.2.199.101:7777-4.156.237.39:40773
The telnet bit was coincidental.

I get a steady flow of telnet and ssh connection attempts but they are blocked by my firewall.
I was running nethogs on 203.2.199.101.

In that case someone might try to connect to a port where no server process is listening, here port 7777 on your machine.
As no process is listening on port 7777 this might result in “unknown connection” instead of showing the attached process.

I’m confused that you have to disable telnet and ssh from the outside manually. This is normally blocked by default firewall configurations and you have to manually alter the configuration to allow outside connections to any ports. The fact that an outside IP ( 4.156.237.39 ) can initialize a connection to local port 7777 is not what you expect from a default firewall configuration.
( port 7777 is used by some applications, e.g. it is the default port of game Terraria.)
Only to be sure: is your firewall a dedicated device or firewall software running on 203.2.199.101?

Historically (for the past 30 odd years), all of my workstations have had my server (203.2.199.4) as their gateway/default route and the iptables firewall was just on .4.
As a result of the current issues, I have switched .101 to go directly to the internet router and added a local firewall to control access.
Without the various port blocks, my logs would be full of failed logins and my bandwidth chewed up. Worse still, someone might actually break in.
Last week I took the rather extreme step of totally disconnecting from the internet for 12 hours.
This has dramatically reduced the bad-guy traffic but still leaves me without an explanation.