Route traffic via VM guest?

I have Rocky 8.5 running on real hardware, and inside I have a Rocky 8.5 vm guest using kernel virtual machine. I can ping the guest from the host, and the host from the guest, and the guest can also connect to the outside world. The network in use is called the “default” network.

virsh net-list --all
 Name      State    Autostart   Persistent
--------------------------------------------
 default   active   yes         yes

On the guest, I have a VPN client called CheckPoint snx which uses old 32bit libraries and expects to run SETUID, and it’s closed source. I don’t want to install it on the host…

So the main question is:

Is there some way I can connect from the host to the guest, and then from the guest to the corporate VPN, and then receive a response back to the host?

In simple terms, I just want to use TigerVNC on the host (because the guest does not have a GUI).

Hi,

Should work, you just need to set the guest as a gateway:

If you have firewalld enabled you probably need add some direct rules to allow the routing.

Then with the guest connected the VPN, you need to change the default route/gw to the IP of the guest.

You can either do this with nmcli or use route command e.g

sudo route delete default gw ip.of.rou.ter eth0
sudo route add default gw ip.of.gue.st eth0

to revert back either reboot or issue:

sudo systemctl restart NetworkManager

Or if you nmcli, just put the old default route back in.

Thanks Tom.

OK, setting as a gateway is what I had in mind. I checked the first command

sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

and it’s saying ip forwarding is enabled, so I guess that must be the default on Rocky 8.5.
I partially understand the concept of ip forwarding in the context of two physical machines with real NICs, but I don’t know if libvirt will be happy…

Hi,

All I can suggest is try it.

If it doesn’t work, first thing I would try is temporarily disable the firewall on the guest. If it works, then its just a case of working out the firewall rules.

Regards Tom.

You have Rocky on the metal and you have libvirt’s “default” network. The Rocky does route between the “default” subnet and the physical subnet that the metal is wired to. That is, the libvirt must enable net.ipv4.ip_forward for you. The default (without libvirt) is to not route.

The “default” subnet is also hidden with masquerade – with sNAT.

You have a program in VM (the VPN) that connects to some external (VPN) server. For that to work, the host must route traffic from the VM into outside subnet, probably to the router that connects the outside subnet into WLAN/“Internet”. If the host does not have routes to do that, then the VM cannot connect. If the VPN connection now depends on the host to have the outside router as the gateway, as the default route, then pointing default route to the VM will break things.

There are bit different VPN solutions. The VPN probably modifies the routing table (of the VM). It keeps explicit routes for reaching the router (the baremetal host in case of VM), but quite often sets the default route to point into address inside (and at the other end) of the VPN “tunnel”.

That is, before starting VPN the VM is in one subnet, the “default”. After starting VPN it is in two subnets: the “default” and a subnet created by the VPN service.

In “old-school VPN” the VPN tunnel is a subnet that links two physically separated subnets together. For example, company has offices at two towns. The machines in offices see each other. The traffic passes over internet between the towns, but the traffic is encrypted in packets that the VPN-applications pass to each other. A virtual private network.

Another old-school is “roadwarrior”. Single machine connects to office. Only the office-side has subnet. Furthermore, the default gateway of the roadwarrior is via the office.

Now, you want your baremetal host to send traffic to the VM so that the VM sends it through its “VPN subnet” to office. You can’t set the default route of the host to point to VM trivially, but it might be possible. Bigger question is whether the VPN client allow a subnet at your end, or does it limit strictly to the “localhost”, the VM?

Note: the ‘route’, ‘ifconfig’, and ‘netstat’ are tools that got a better alternative: iproute2 package two decades ago.
Alas, the ‘ip route …’, ‘ip address …’, and ‘ss’ are now only for temporary ad hoc and debugging.
One should configure network with the service that by default in Rocky is NetworkManager. That is, with ‘nmcli’, ‘nmtui’, or some GUI gadget.

1 Like

Ah, it’s enabled because I have libvirt running on my host?

This is the part that makes me think it could be difficult (or impossible). The VM has to pass traffic back to the host in order to get to the home router. Potentially a massive feedback loop.

That is probably lesser of the challenges; yes, routing can be interesting but doable.

The bigger issue is that IF the other end of the VPN is set to talk to only one machine at this end of the tunnel, the VM, and not to a subnet (that could have the host in it) AND that VPN config cannot be changed THEN how to pretend that VM talks when host talks?

Yes, it’s possible without routing. You can do with using a combination of ssh and tunneling. I’m going to use a working example to my work office to connect to a host via RDP.

So, on the Rocky VM once the VPN is opened, you will do a command like this:

ssh -R 22222:10.1.2.200:3389 ian@rockyhost

Now let’s break that down. This is a reverse tunnel SSH connection whereby we are going to connect to rockyhost (your host machine), this can be done over the IP - most likely 192.168.122.1 if you are using the standard KVM natted network. So it could be ian@192.168.122.1

Now the middle part of that command. Port 22222 will be open on rockyhost after the ssh connection has been made. The IP in the middle (10.1.2.200), is the IP we want to connect to on the other side of the VPN - so either RDP or VNC or whatever it is you want to connect to. My example shows a Windows Server via RDP.

Then, on your rockyhost, you do something like this:

rdesktop localhost:22222

As you know, port 22222 is the reverse tunnel via SSH to your rockyvm. So it will connect to your rockyvm and redirect everything to the RDP Windows Server which is the destination through the VPN tunnel.

The only downside, is if you have a lot of applications to connect to via the tunnel, then you would need to open reverse tunnels from the VM for all the services you want to connect to, than dealing with routing for example.

2 Likes

For completeness, a screenshot of this in action. The top left console window is my laptop (so rockyhost for you). The bottom left console window you can see an SSH connection to the rocky VM on my machine.

In the first console window you will see I do a telnet connection to show it fails - so that you can see I’m not faking it. Then I run the reverse tunnel command in the rockyVM, which you can then see has connected to my laptop (hostname in the console prompt). You will then see I do telnet again to that same port, and now the connection is open.

I then run the rdesktop command, and the end result is the window on the right showing the remote desktop of the server via the VPN tunnel on the rocky VM.

I was able to get it working; doing it without routing was a good idea.

First I had to get the CheckPoint VPN client working on the guest. I didn’t want to run it SETUID, so I had to use ‘linux capabilities’ to allow various network interactions. I’m now able to run it as standard user, which solves a number of problems with the profile and it means the files it creates are safely in user dirs.

Once connected to the office VPN, a new device and connectoin become available on the guest e.g. with the office name servers. At that point, I can ping machines on the office network (from the guest).

I first tested setting up plain ssh (as opposed to RDP).
on the guest
ssh -R 22222:10.17.0.35:22 gerry@192.168.122.1
10.17.0.35 is the destination machine I want to get to
192.168.122.1 is the local vm host
at this point it bounces me onto the host and I can see the new listener
On the vm host, I can now do
ssh localhost -p 22222
and it connecs me to the destination machine on the office network where I can use terminal commands and then logoff when finished.

The next test was using TigerVNC and GUI on the vm host.
On the guest (still connected to the office vpn) I run
ssh -R 22222:10.20.10.161:5900 gerry@192.168.122.1
10.20.10.161 is the destination machine
5900 is the port for vnc
Now on the vm host I go to “Applicatoins : Internet : TigerVNC” and enter
localhost:22222
and it connects to the destination machine and shows the GDM login screen
I was a bit surprised that port 5900 worked becuase it’s set up with socket activation and adds one to the port number on each connection.

2 Likes

What you can also do is if you need multiple sessions at the same time, you can change the port. So for example, if you wanted to do the ssh session and the VNC at the same time, you can change port 22222 to anything you want. For example, for the ssh session we could change it to 20001 and for the VNC have 20002 - then simultaneously you can ssh localhost to port 20001 and tigervnc to 20002.

In fact you could even add all of it to a script on the VM and run the script after the VPN has connected, to open up all the connections you need.

Yes, I’m hoping to do this with ‘vnc’ connecting to a remote linux host, and ‘rdp’ connecting to a remote Windows host. I tested some other things like using ssh-agent to avoid some prompts and also using switches like -T and -N. With -T I was able to get the session running without it jumping to the vm host terminal screen; you have to press CTRL+D to exit. With -N I wan’t able to exit in a clean way, had to press CTRL+C.

Regarding ssh-agent; on the vm host, it automatically populates the environment variables into the shell after logging in, but on the vm guest it does not. I’m not sure why, could be related to not having GUI on the guest.

1 Like

I know it’s marked as “solved”, but I’m still interested in knowing more about host/guest vm networking. Regarding the two ssh sessions.
From host to guest:
ssh 192.168.122.15
From guest to host:
ssh -T -R 22222:10.20.10.161:5900 gerry@192.168.122.1
at first I was wondering “how it can work on port 22”, but is it the case that the first connection goes from a high numbered port on the host to port 22 on the guest, and then the second connection goes form a high numbered port on the guest to port 22 on the host?
In other words, the second connection doesn’t even use port 22 on the guest?

The other questoin is how can it create the 22222 listener on the host; I’m guessing it connects to the host on port 22 first, and then uses programming code to create a new socket (as a standard user)?

It’s an SSH tunnel. Using the -R parameter opens a reverse tunnel. Your VM connects to your host machine, and creates a tunnel from your host machine to the IP you wanted connect over the VPN. gerry@192.168.122.1 connects to your host machine on port 22. The tunnel is then opened and within the SSH connection.

You can do local tunnels using -L but if you did that on your VM, then you can only use your VM to connect over port 22222 or whatever. You wanted to use your host machine to connect over the VPN tunnel opened on the VM - so a reverse tunnel was needed.

This link you can read for more detail: How an SSH tunnel can bypass firewalls, add encryption to application protocols, and help access services remotely. and this link for examples of what I provided in this post as a solution for you (which includes using local tunnels with the -L parameter): SSH port forwarding/tunneling use cases and concrete examples. Client command, server configuration. Firewall considerations.

Thanks for the additional links, which I had not seen before and even have a diagram.
I’m also still interested in the ip forwarding and routing idea (not tested yet).

I’ve been able to simplify things slightly using -L on the initial connection from the host to the guest (instead of having to use -R on the guest after connecting). This also means I don’t have to “wait” and press CTRL+D.

I don’t know how (or why) it works.

The new way to connect from the host to the guest is
ssh -L 22222:10.20.10.161:5900 vmg01
this does a normal ssh from host to guest and then sets up a listener on the host. At the time of the ssh connect, there’s no ‘10.20.10.161’ ip address; it appears later after starting the vpn client on the guest.
Once that’s done, I then connect with TigerVNC on the host using [::1]:22222
For some reason, it’s using IPv6.
So far this is all working with default firewall rules on both host and guest.

In essence it doesn’t make much difference whether you do it on the host with -L or whether you do it on the VM with -R. Both are tunnels. Why it works is because you open a SSH connection to the VM. Your tunnel is redirecting to port 5900 to the IP you want to connect to.

With mine it was the other way around, so reverse tunnel. You initiated it from the VM to the host. As per the links I provided, it explains it there. I can give a scenario where one is going to be better than the other.

Let’s say you have a desktop/server at work which has all inbound traffic blocked. That means at home, you are not going to be able to connect to it and open a tunnel using -L. However, let’s assume in this instance, the desktop/server at work has full outbound internet access. Which means you could use the desktop/server at work to connect to your home computer/laptop/server. Since you are in control of your home internet connection and firewall, you can configure it to allow access. You cannot configure the office firewall unless you are the administrator of it (or even if you are, work policy might not allow you to configure inbound firewall rules - we’re also going to ignore the fact that most corporations restrict outgoing internet access as well just for the theory of the argument).

In such a situation, the reverse tunnel comes in handy, because you use the desktop/server at work to connect to your home computer with a reverse tunnel. This then means when you are at home, you can use that tunnel to get to VNC on another computer, or Rdesktop or whatever.

When you are in control of both sides of the internet connection, or, as you are in control of the host machine and the VM machine, then whether you use -L or -R achieves the same result. The only real difference being where you initiate the tunnel from - inside the VM as a reverse tunnel or normal tunnel from the host.

Ok, that makes sense. In my case I’m using the Checkpoint vpn client on the vm guest to connect to the office network, so I’m not trying to directly connect a work machine to a home machine, so using -L means I only need to create one SSH tunnel from the host to the guest.