Question

Hello. Recently I encountered a strange routing issue.

FW ver.: RUT9XX_R_00.06.04.3

Router model Teltonika RUT955 LTE

Firmware version RUT9XX_R_00.06.04.3

Kernel version 3.18.44

Bootloader version 3.0.1

# ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP group default qlen 1000

    link/ether 00:1e:42:19:31:51 brd ff:ff:ff:ff:ff:ff

3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master br-lan state DOWN group default qlen 1000

    link/ether 00:1e:42:19:31:52 brd ff:ff:ff:ff:ff:ff

4: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default

    link/tunnel6 :: brd ::

5: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 32

    link/ether 02:4e:89:11:56:7a brd ff:ff:ff:ff:ff:ff

6: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 32

    link/ether de:36:7c:42:b9:80 brd ff:ff:ff:ff:ff:ff

7: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN group default

    link/gre 0.0.0.0 brd 0.0.0.0

8: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN group default qlen 1000

    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff

9: ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN group default

    link/[823] 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00

11: wwan0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000

    link/ether 5a:55:60:60:b3:39 brd ff:ff:ff:ff:ff:ff

    inet 172.28.0.3/29 brd 172.28.0.7 scope global wwan0

       valid_lft forever preferred_lft forever

    inet6 fe80::5855:60ff:fe60:b339/64 scope link

       valid_lft forever preferred_lft forever

12: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default

    link/ether 00:1e:42:19:31:51 brd ff:ff:ff:ff:ff:ff

    inet 172.21.21.1/24 brd 172.21.21.255 scope global br-lan

       valid_lft forever preferred_lft forever

    inet 172.21.21.254/24 brd 172.21.21.255 scope global secondary br-lan

       valid_lft forever preferred_lft forever

13: tun_c_a: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100

    link/none

    inet 192.168.21.3 peer 192.168.21.1/32 scope global tun_c_a

       valid_lft forever preferred_lft forever

# ip r

default via 172.28.0.4 dev wwan0  proto static  src 172.28.0.3

default via 172.21.21.253 dev br-lan  proto static  metric 10

10.0.0.0/8 via 192.168.21.1 dev tun_c_a

172.16.0.0/12 via 192.168.21.1 dev tun_c_a

172.21.21.0/24 dev br-lan  proto kernel  scope link  src 172.21.21.1

172.28.0.0/29 dev wwan0  proto kernel  scope link  src 172.28.0.3

172.28.0.4 dev wwan0  proto static  scope link  src 172.28.0.3

192.168.0.0/16 via 192.168.21.1 dev tun_c_a

192.168.5.0/24 via 172.21.21.253 dev br-lan  proto static

192.168.21.0/24 via 192.168.21.1 dev tun_c_a

192.168.21.1 dev tun_c_a  proto kernel  scope link  src 192.168.21.3

# traceroute -n 10.10.0.111

traceroute to 10.10.0.111 (10.10.0.111), 30 hops max, 38 byte packets

 1  10.126.2.1  701.872 ms  179.483 ms  179.861 ms

 2  192.168.10.206  249.848 ms  229.637 ms  289.779 ms

 3  192.168.10.205  159.924 ms  239.324 ms  219.833 ms

 4  192.168.22.77  439.599 ms  239.680 ms  249.756 ms

 5  192.168.22.78  249.748 ms  209.680 ms  229.558 ms

 6  10.10.0.111  249.965 ms  *  198.709 ms

# cat openvpn-636C69656E745F61.status

OpenVPN STATISTICS

Updated,Fri Dec 27 10:35:34 2019

TUN/TAP read bytes,0

TUN/TAP write bytes,7676944

TCP/UDP read bytes,17432316

TCP/UDP write bytes,5957790

Auth read bytes,7991444

END

Although a route to 10.0.0.0/8 exists specifically thru vpn connection 192.168.21.1, the kernel uses default route, which is strange. 

I can reboot the system and the correct working will start, but as this is a production environment which should work 24/24 without problem, my question is to solve the issue so it will not happen again.

Thanks.

Answer 1

Hello,

Could you, please try adding static route manually via routers WebUI?

Network > Routing > Static Routes > Add

Best Regards

Answer 2

There are already two static routers added.

There also were an ipsec vpn set up, which were not connecting, just trying. I disabled ipsec vpn and the route came to normal. Strange happening.

Comments

After disabling IPsec instance issue was resolved?

---

Yes, strangely the routing started to go as it should be. The strange thing is that Ipsec could not be working ie making a stable connection as it were set up in case the openvpn server were not working and the line would be switched to a router with ipsec server. In any moment only one of the two vpns (ipsec and openvpn) could make a successful connection.

---

I would not suggest using two different VPN instances for purposes of failover.

Could you configure OpenVPN keep alive function for this purpose? It would try reestablishing the tunnel when it is down.

---

I would not use, but this is the demand of buyer. Openvpn already use keepalive and reconnects. The ipsec is for the case when openvpn server is not working and line (apn) is switched to a cisco asa router, which does not have openvpn.