How to monitor ssh remote commands

Hi All,
I know that ssh connections on the serve are logged in /var/log/secure
However, where can I find a record of single commands that are executed before disconnecting.
Example :
ssh 192.168.1.1 uname -r

1 Like

Nowhere.

The fact that the user got a shell by connecting and authenticating with ssh is incidental to what the user did once logged in and running that shell with the ability to do whatever that user is authorised to do on the host.
Since you are already heading down the rabbit hole which is wanting to know what the users whom you trust (but don’t trust) are doing on your host and are already engineering clever solutions to capture and log things via shell profile and rc configs, don’t bother. By the time you realise that it’s almost impossible to reliably capture what you think is the simple series of commands which would look nice in a log file, you might as well have used the tool for the real job: auditing.

Setting up auditing to capture the things you (should) want isn’t terribly difficult, but is far from trivial, however the real question - aside from the storage and transport considerations* - is who or what is going to analyse that audit data. If the answer is your SIEM and the Security team, then go ahead; else if it’s you one day after something bad happens, then you need to work out in advance how to manage the audit data consistent with the badness**, but if you think that you or some junior sysadmin’s job will now include “Review audit data for… um… things”, then don’t bother.

There are some other approaches to more specific cases which might be better suited to your environment, but this sounds like the ages-old problem of users with a shell on the host. Of course if those users don’t need a (unrestricted) shell on the host, then that’s a different matter.

* [Get it wrong and your host will run out of space and shut down shortly after booting before you can determine why.]
** [e.g. Did the user delete the file versus call in an air strike on the data center through an unsecured API.]

Unfortunately there are still (!) no perfect open source solutions for this. It’s just a hard problem when user has a shell.
pam_tty_audit and keystrokes from audit log is one clumsy solution.
tlog is not much better and has issues with scp and many other problems.

Maybe those options could work for you but they are far from perfect. A bastion host dedicated for session recording would probably work better.

1 Like

I have been using snoopy, can be found on github.

You can also switch from the standard shell (sh, bash, etc) to a logging shell, like sudosh.

joeblow@server1:~$ ssh server2 "/sbin/ifconfig"

root@server2:~# sudosh-replay
...
02/14/2023 12:20:08 -c arg   joeblow       joeblow       /sbin/ifconfig

There is an effective way to do this, using built-in feature and command on Linux:

PROMPT_COMMAND='history -a >(tee -a ~/.bash_history | logger -t "$USER[$$] ($SSH_CONNECTION) bash")'

Bash will read the PROMPT_COMMAND variable every time before it displays the prompt, so we will insert a command for auditing purposes there.