Can Not Route Packets from One Interface to Another

How to forward packets from one network interface via another interface

Managed to find the issue.

Whilst adding the route for the container in the other subnet I haven't correctly specified the gateway. The gateway still points to the host machine in which docker is run (see above figure). So I added the correct routing rule specific to the two end-containers - c1 & c3.

c1 - ip route add 10.23.0.0/24 via 10.12.0.1 dev peervpn12
c3 - ip route add 10.12.0.0/24 via 10.23.0.1 dev peervpn23

In the meantime, had to add the correct FORWARD rules in the c2 container's iptables:

 iptables -A FORWARD -s 10.12.0.2 -i peervpn12 -d 10.23.0.2 -o peervpn23 -j ACCEPT
iptables -A FORWARD -s 10.23.0.2 -i peervpn23 -d 10.12.0.2 -o peervpn12 -j ACCEPT

With this setup I was able to achieve the flow I expected.

Thank You and I don't know why it's down-voted.

Maybe if I know the reason I can correct myself in future :)

UDP connect doesn't route traffic for particular interface

So in my case, the issue was asymmetry of ip rules between the ethernet and wifi interface that cause routing problems. If you run into a similar thing, you can see some rules that are set with:

ip rule show

and/or

iptables --list

Packet forwarding between interfaces

N.B - edited the command (added eth2 as outgoing interface)

I understand from the setup described that the that the wan side on your router is eth2 which has address 10.2.20.3.
Hence I would expect outgoing packets (on eth2) to have the source IP 10.2.20.3 and not 10.2.40.4.
I think this indicates that you got your nat rules reversed or something.

In fact it seems that the only iptables rule that you need is:
iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE

I suggest you remove all other DNAT/SNAT/MASQUERADE rules and try this one.

How to route TCP/IP responses through a different interface?

You could add an additional address to the lo interface on each system and use these new addresses as the TCP connection endpoints. You can then use static routes to direct which path each machine takes to get to the other machine's lo address.

For example:

Machine A:
ip addr add 1.1.1.1/32 dev lo
ip route add 2.2.2.2/32 dev eth0 via <eth0 default gateway>

Machine B:
ip addr add 2.2.2.2/32 dev lo
ip route add 1.1.1.1/32 dev gr0

Then bind to 1.1.1.1 on machine A and connect to 2.2.2.2.

Can't establish connection over second NIC (two hops)

Not an authoritative answer, but my first working attempt (applying what I managed to understand):

sudo ip route add 192.168.1.0/24 via 192.168.222.254 from 192.168.222.200 dev eth1 table 253 
sudo ip rule add from 192.168.222.200 table 253

Update: from and devarguments in the ip route command aren't required (it works perfetly well without them).

...after isuinng first command I couldn't connect yet, but after issuing second one yes.

The logic behind that comes from this text i found in this document:

Linux-2.x can pack routes into several routing tables identified by a number in the range from 1 to 255 or by name from the file /etc/iproute2/rt_tables By default all normal routes are inserted into the main table (ID 254) and the kernel only uses this table when calculating routes.

Actually, one other table always exists, which is invisible but even more important. It is the local table (ID 255). This table consists of routes for local and broadcast addresses. The kernel maintains this table automatically and the administrator usually need not modify it or even look at it.

In fact, I finally ended up using another routing table, identified by its id (253) instead of what I now understand it is just an alias (defined in /etc/iproute2/rt_tables file).

...and checking again that file, I now see that there was an alias ("default") already defined for that routing table (next to the "main" one which is indeed 254 as the text fragment I pasted previously says.

What I don't know yet is which is the logic behind this naming (the "default" for 253 routing table I mean) and if, for any reason, is better to use lower routing tables (1, 2, 3...) like this solution (already mentioned in the question) does.

But, for the sake of simplicity, if we aren't going to build complex routing policies and just want to fix this connectivity issue, I guess it could be a good solution to use something like (not yet tested):

gateway 192.168.222.254 table 253
post-up ip rule add from 192.168.222.200 table 253

I still need to test and check if I need an additional via 192.168.222.254 in the gateway row or if it won't work at all and need to add it with another post-up command instead.

I will update this answer with the results.

Edit 1: Same works with default routes:

sudo ip route add default from 192.168.222.200 via 192.168.222.254 table 253
sudo ip rule add from 192.168.222.200 table 253

Edit 2: First (now fully¹) working approach

After playing for a while with a testing machine, I think that the best solution is to add following rows to the second NIC configuration in /etc/network/interfaces file:

gateway 192.168.222.254 table 1
post-up ip rule add from 192.169.222.200 table 1
pre-down ip rule del from 192.168.222.200 table 1
post-up ip route add 192.188.222.0/24 dev eth1 src 192.168.222.200 table 1

Comments:

  • Adding table 1 to the gateway keyword worked well so additional (less readable) post-up command to add that default route was not necessary.

    • ...in fact, using specific table (other than main) for first NIC together with a similar rule than what we used for our second NIC would be a bad idea because, that that rule will only apply when 192.168.111.200 is going to be used as source address so there will not be any "default default gateway". Leaving first NIC configuration in the main routing table, will make all ("locally generated") outgoing connections to remote LANs will go though our first default gateway by default.
  • First post-up command adds a rule that packets with the source address of that NIC, should be routed using table 1 (otherwise our new default gateway wouldn't be used).

  • pre-down command removes that rule. It is not mandatory but, without it, multiple network service restarts will duplicate this rule every time.

  • I also tried to use dev eth1 instead of from 192.169.222.200 (to avoid having to duplicate network address) but it didn't work. I guess which NIC to use to for "response" packets were "not yet decided".

  • I used table 1 for eth1 (our second NIC) and I could use table 2 for an eventual third one and so on. It wasn't needed to specify any table/rule for first NIC because it comes to the main table (not "default": see below note).

  • Finally(¹) the second post-up command make all things work well because (as I now realize) only (first matching) one routing table is used so the default network route (automatically created when the interface brought up) doesn't apply because it was created in table main.

    • I still don't know if there is a way to force it to be crated directly into table 1.

NOTE: By command sudo ip rule list we can see current routing rules as follows:

0:      from all lookup local 
32765: from 192.168.222.200 lookup 1
32766: from all lookup main
32767: from all lookup default

As I can understand, they are added decreasingly from 32767 to 0 and tried
increasingly until one matches. Last two ones and the "0" were already
defined by default. The former because of the logic I previously cited
from this document but that documents says that rules starts from "1"
so I guess "0" should also be some predefined "default starting point".

Edit 3:

As I said in the Edit 2 (of the question), I found this Linux Advanced Routing & Traffic Control HOWTO that helped me a lot in clarifying things.

Concretely the Routing for multiple uplinks/providers chapter was very useful to me in the task of understanding setups having "network loops" (even in our case we aren't acting as a router to Internet).



Related Topics



Leave a reply



Submit