Hi Everyone,
My setup is like this:
OS - Rocky Linux 9 (minimal)
Firewall - nftables
Network Interfaces -
LAN - 172.16.0.10/24
WAN1 - 192.168.3.10/24
WAN2 - 192.168.13.10/24
I want to forward traffic from one set of random IPs in LAN to WAN1 and other set of IPs in LAN to WAN2. In case of any WAN failure I should be able to redirect them to other working WAN. If need for details I will be ready to share.
Any guidance / help is appreciated. Thanks in advance.
-Barani
You do need Policy-based Routing (aka source-based routing).
Red Hat describes it in: Chapter 27. Configuring policy-based routing to define alternative routes | Red Hat Product Documentation
Hi jlehtone
Thank you for the response. It is working. I saw and tried that Red Hat link earlier. However my requirement is that local network is a single subnet (172.16.0.0/24) from which the some of the hosts should use enp1s1 (192.168.13.10) while others use enp1s0 (192.168.3.10) to connect to internet.
man ipv4.routing-rules says:
ipv4.routing-rules
A comma separated list of routing rules for policy routing. The format is based on ip rule add syntax and mostly compatible. One difference is that routing rules in NetworkManager always need a fixed priority.
and the man ip-rule describes various SELECTORS. Probably “prefix” or “fwmark”.
The prefix does not have to be a subnet (e.g. 172.16.0.128/25). It can be a single IP address (e.g. 172.16.0.130).
The fwmark is set by firewall (prerouting), so you could mark traffic from certain MAC even when its IP changes.
How you will take a connection down when its uplink is dead? That I don’t know.
I didn’t go through ip-rule or ipv4.routing-rules. I think I have to study fully. Thank you for pointing me in that direction.
On that linked base, in the “with nmcli” part, the step 3 was:
nmcli connection add type ethernet con-name Internal-Workstations ifname enp8s0 ipv4.method manual ipv4.addresses 10.0.0.1/24 ipv4.routes "10.0.0.0/24 table=5000" ipv4.routing-rules "priority 5 from 10.0.0.0/24 table 5000" connection.zone trusted
which includes:
ipv4.routing-rules "priority 5 from 10.0.0.0/24 table 5000"
This is the typical “if source (sender) on packet is X (here within 10.0.0.0/24), then decide where to deliver the packet based on routes on table Y (here 5000), rather than the main list of routes”
Packets from X follow policy Y.
Lets say that default policy is to forward via WAN1.
You add another routing table “Y”, where the default is via WAN2.
You add rules that direct packets from some machines to use table Y.
Hi @jlehtone
Thank you very much. I got that working now.
Further to my above setup and expanded view as given below
With firewalld policy I could able to access internet based on my above requirement. Here I already have a Samba server (172.16.3.5) in another network (172.16.3.0/24) for which the access is through Router 2. I added a static route for 172.16.0.0/24 via 172.16.0.1 in Router 1. Now from host I could able to ping the Samba server but could not access Samba services.
Any help please.
This a totally different question that does not really involve source-based routing.
Lets say that you have three subnets here:
The Samba has simple routing:
default via 172.16.3.1
172.16.3.0/24 dev A
The Router 2 has one more route:
default via 172.16.0.10
172.16.3.0/24 dev A
172.16.0.0/24 dev B
Every Host in 172.16.0.0/24 should have:
default via 172.16.0.10
172.16.3.0/24 via 172.16.0.1
172.16.0.0/24 dev B
and finally the Router 1 something like:
table main:
default via 192.168.3.x
172.16.3.0/24 via 172.16.0.1
172.16.0.0/24 dev B
192.168.3.0/24 dev C
192.168.13.0/24 dev D
table other:
default via 192.168.13.y
172.16.3.0/24 via 172.16.0.1
172.16.0.0/24 dev B
192.168.3.0/24 dev C
192.168.13.0/24 dev D
If I mention static route in every hosts in 172.16.0.0/24 for 172.16.3.0/24 via 172.16.0.1, they all can access shares from the Samba server.
But my requirement is that:
a) In the Samba Server and Router 2 all that suggested are already in place.
b) All the hosts in 172.16.0.0/24 will have “default via 172.16.0.10”.
c) Router 1 has static route “172.16.3.0/24 via 172.16.0.1”
With this the hosts could able to ping the Samba server successfully but could not access shares. I could see in Router 1 with tcpdump the packets are going to Samba server but there is no return traffic.
Note: This setup is working fine with Debian 12 as Router 1 but not with Rocky Linux 9.
Why?
The 172.16.3.0/24 is behind 172.16.0.1, so why should members of 172.16.0.0/24 have to talk to 172.16.0.10 in order to get to 172.16.0.1?
You want to talk to 172.16.3.5, so you send a packet to 172.16.0.10. The 172.16.0.10 sends the packet to 172.16.0.1. The 172.16.3.1 sends packet to 172.16.3.5. The 172.16.3.5 sends a reply to 172.16.3.1. The 172.16.0.1 forwards the reply to you. You did send to 172.16.0.10, but reply arrives from 172.16.0.1. How do you know that it is a reply to what you did send? How does your firewall know? I bet it does not, so it discards the “bogus reply”.
True that logically & technically.
But I have around 150+ hosts in 172.16.0.0/24 network and it is difficult to configure static route “172.16.3.0/24 via 172.16.0.1” in all of them. Instead if default gateway (172.16.0.10) of hosts in the network can does that, by redirecting, would save time and effort.
My current Router 1 on Debian 12 does that job nicely. It was also working earlier on CentOS 7.
A reason why organizations prefer DHCP. One can dictate at DHCP server what network config clients get, including static routes. That produces more consistent config and allows dynamic changes too. Are there still devices that do not support DHCP (or whose DHCP client does not accept static route config)?
Are you sure that your older routers do not resort to NAT? In the above traffic flow:
… The 172.16.0.10 does sNAT (“from 172.16.0.10”) and sends the packet to 172.16.0.1.
…
The 172.16.0.1 send reply to 172.16.0.10. The 172.16.0.10 restores the original source, and forwards the reply to you.
Regrets for delay in response.
I don’t want to use DHCP server/service for various reasons.
No! I haven’t used any NATing. Currently I am using a Debian 12 as router, with simple nftables rules, which fulfils my need in this regard. I am trying Rocky Linux to fulfil my need for “policy based routing” along with this need. There must be a way to do this without NAT.
I do use ‘dnsmasq’ as DNS, DHCP, and TFTP, but if that is off the table …
I would listen test traffic with tcpdump on Router 1, Router 2, and client in order to see how the packets actually flow and what they have – whether replies reach the client and via what route.
@jlehtone
Thank you very much for patiently responding all queries. With your initial suggestion my main requirement is useful. As far as routing is concerned I must look for and try various options, I understand. Thank you again.