10879 questions

12963 answers

20221 comments

26143 members

0 votes
180 views 3 comments
by
Hi Guys

Upgraded one of our RUT X11 router configs from 7.0.0 to 7.0.2
everything appeared to go well...
OpenVPN split tunnel established ....
I can see the web admin pages of the x11 from the remote internal LAN...

But now the device users are complaining that they can get to the internet, but cannot
access any service hosted remotely through the VPN tunnel - some protocols get through e.g. you can ping
but you cannot browse, FTP, SMB, anything that normal users do.

Even more interestingly - starting up OpenVPN not on the router, but on the wifi client device direct to the
VPN server's FQDN via the internet (i.e. not using the router's VPN tunnel) is similarly affected ???? The VPN
establishes and routes are pushed from the VPN server directly to the wifi device as well as redirection of the default
gateway - but even in that scenario - you cant browse any content behind the VPN ?.

I started to think at this point it was a VPN server issue, so I dropped the wifi device off of the x11 wifi onto its own 4G
connection and restarted the VPN . It connected , traffic flowing, browsing fine end to end.

I then moved it onto the X11 wifi again, allowed time for dead peer detection and tunnel re-establishment, the tunnel came up, but NOW...
no throughput. This suggests to me that its a port / protocol level issue , maybe in the firewall ??  

So ... diagnostic time

First thing I notice is that to build the split tunnel you have to block the VPN client from accepting the
pushed routes , then add in the router specific static routes . I had done this manually in the config page.

When I looked in the GUI config page ,the additional command order was reversed!! the command to block the
client from accepting pushed routes came after definition of the locally specified routes ??? what the hell ??

So I edited the config file, and restarted the router - GUI was now up , pings went through , browsing still didn't work over the VPN

After several fruitless hours in wireshark , with no obvious root cause , it was time to revert.

Fortunately I took a backup of the config as when going backwards from 7.0.2 to 7.0.0 despite stating keep the config,
the router factory reset completely.

I restored the backup and rebooted the router , everything on 7.0.0 started working again immediately.

So guys - I don't know in total what you did for openvpn between 7.0.0 and 7.0.2, but while you fixed the legacy GUI status
issue you have broken OpenVPN here (yet again!) and I have yet to root cause what triggered it ,other than to say regression
to 7.0.0 cures it instantly.

How many QA hours do you guys put into the lab, testing a build before releasing it please ?

Regards

BB

1 Answer

0 votes
by

Hello,

First of all, thank you for the detailed write up. I have some follow-up questions.

  1. Since you're using split tunnel, how exactly is it configured? Are you using the "vpn-policy-routing" package or are you managing all of this without installing any additional packages on the router?
  2. Regarding the users not being able to reach FTP/SMB server or any other resources behind the other end's VPN tunnel gateway - is there any possibility to check if the issue occurs both ways? As in, if users cannot reach the other side LAN resources and other side LAN resources cannot reach the end users?
  3. Does your OpenVPN server (I'm assuming the RUTX11 end is configured as a client) push out specific routes to the router directly or did you define them in a configuration file/custom fields on the RUTX11 side?
  4. The firmware that you've upgraded to - did you mean to say from 7.0 to 7.1.2?
  5. Have you used this [https://wiki.teltonika-networks.com/view/OpenVPN_traffic_split] guide as a reference at any point when configuring the tunnel/split policies?

Static routes shouldn't be necessary in this case, instead I'd recommend using the vpn-policy-routing package. It doesn't have GUI support but it still works when configuring all the rules via CLI.

Additionally, could you generate a troubleshoot file and send it to me via private message? I'd like to see exactly what is configured on the device.

What's a troubleshoot file and how to generate it?

A Troubleshoot file contains a device's event logs, configuration files and other info useful for diagnostics. It can be downloaded from your device's WebUI, Troubleshoot page:

System → Administration → Troubleshoot

Best regards,

Tomas.

by

Good Morning Tomas

Answering your questions inline:
 

  1. Since you're using split tunnel, how exactly is it configured? Are you using the "vpn-policy-routing" package or are you managing all of this without installing any additional packages on the router?

Not using the policy routing package as far as I know ( I didn't even know there was such a thing ! :-) ) I quite simply, tell the client not to accept static routes or default gateway pushed by the server (route-nopull) , then manually add my own static routes in their place. This is configured in the GUI  / config file . The approach works 100% in 7.0.0. its very simplistic but as they say "If it isnt broken, dont fix it !"

2.Regarding the users not being able to reach FTP/SMB server or any other resources behind the other end's VPN tunnel gateway - is there any possibility to check if the issue occurs both ways? As in, if users cannot reach the other side LAN resources and other side LAN resources cannot reach the end users?

The fixed network admins are able to browse and edit the router configs, but as there is a NAT boundary inside the device,  they cannot see the LAN anyway unless port forwarded. I confess that I forgot to check that, focussing more on "the tunnel is up so why no traffic ?"

3. Does your OpenVPN server (I'm assuming the RUTX11 end is configured as a client) push out specific routes to the router directly or did you define them in a configuration file/custom fields on the RUTX11 side?

Yes, by default the VPN Server pushes static routes and redirects the default gateway. However on the RUT X11 and 955s this forces all traffic through the VPN which users complained was too slow (even after many hours of MSS tuning in iperf) so, as outlined above,  route-nopull , then we define them in the configuration file (the method was described on the openvpn website) 

4. The firmware that you've upgraded to - did you mean to say from 7.0 to 7.1.2?

Mea culpa!  :-) yes I did sorry about that , trying to get everything down - must do better next time!
 

5. Have you used this [https://wiki.teltonika-networks.com/view/OpenVPN_traffic_split] guide as a reference at any point when configuring the tunnel/split policies?

I have not , but I will look through it now - thanks 

Additionally, could you generate a troubleshoot file and send it to me via private message? I'd like to see exactly what is configured on the device.

Very sorry but for operational security reasons I had to get the device running again ASAP - the quickest way to achieve that was to reflash to 7.0.0 and reload. I can try and repeat the exercise when a spare device becomes available, no idea when that will be right now I'm afraid.

Regards

BB

by

Generally speaking, even though the VPN policy routing package does not have a WebUI implementation, we recommend using it in case there is a requirement to route specific clients via either specific VPN tunnels or to make sure only some part of the clients has their traffic routed via the VPN tunnel as the default gateway and keeping the default gateway of some other batch of IPs on the default WAN connection.

More resources and information regarding the package can be found on the following sites: 

https://openwrt.org/docs/guide-user/network/routing/examples/pbr_app

https://docs.openwrt.melmac.net/vpn-policy-routing/

Best regards,

Tomas.

by

I can confirm that an upgrade to firmware 7.1.4 was attempted and appears to have resolved this issue.

[EDIT] No it didn't - I could VPN into the device remotely, which gave the impression that it worked , but
traffic did not flow back from the device either to the internet or the VPN. I have had no choice but to 
revert to 7.0.0 to restore service 



Regards

BB