Question

we are troubleshooting a RUTX11 4G connection. We have connectivity issues for clients(connected to an accesspoint other than the one of the RUTX)  behind the device towards internet. It's a small office space.

When only 1 person comes in and uses it's laptop and phone, all is fine. When 2 to maximum of 4 users are in the office. Connections seem to drop randomly.

We saw that a ICMP to 1.1.1.1 and 8.8.8.8 for example statys very stable, but TCP connections to 80 or 443 or 53 (DNS) start to fail when 4 laptops are using the 4G connection outbound together with maybe 2 of 3 iphones.

The RUTX is using NAT mode, DHCP for the clients.

I'm looking for some kind of possible connection limit to be either present in the RUTX (NAT, TCP sessions, mac tables whatever) or at my 4G provider. Could this be the case? I have no other explaination other than some kind of limit is reached.

Usage bandwith-wise is very limited.

any idea's perhaps?

Comments

we've found the issue. for reference:

Our service provider for the 4G connection, KPN in the Netherlands, confirmed they are limiting the maximum number of connections to 512 when you use the automatic APN selection, or to be more precise: this limit are is place for any APN they broadcast other than the APN called "advancedinternet".

When you need a less restricted (in regards to maximum connections, port usage etc.) and without extra NAT 4G connection, you have to use the APN "advancedinternet".

Recommendations are to have good/better security in place for the connection, thinking of a firewall etc, because the less secure nature of the connection compared to the default 4G APN.

Setting that APN solved our issues.

Answer 1

Hi,

Apologies for the delayed response. As checked in the logs, No NAT limiting specific rule see on the firewall section or logs, System logs show - nf_conntrack version 0.5.0 (4096 buckets, 16384 max) indicating sufficient sessions, If RUT was dropping the sessions there would have been nf_conntrack: table full, dropping packet - Which is not existing in the logs. Thus no such throttling seen on the RUT.

Thanks

Comments

thanks for confirming this! we'll take this further with our provider to see what limit exists there.

Answer 2

Hi,

- Please could you clarify if the LAN clients are connecting to the RUTX11 Wifi for Internet access OR are they connecting to the above
  mentioned access point that in-turn connects to Rutx11 ?

- Rutx11 supports upto 150 simultaneous Wifi connections

- Under the wireless interface configuration try disabling "Disassociate On Low Acknowledgement" feature and monitor for any changes ?
  Also, please check here if any "inactivity" based parameters have been set causing such disconnections.

- What authentication method is in use for clients ? radius or psk.

- On Network >> Interfaces >> MOB1S1A1 >> advanced settings >> "use dns servers advertised by peer" (disable this) and input open dns server ip address 208.67.222.222  208.67.220.220 and "save and apply" to see any difference in behavior.

- On the same Interfaces >> MOB1S1A1 >> advanced settings >> Mobile data limit >> Check if "Enable data connection limit" has been enabled ?
- On the Mobile >> SIM Switch - Any parameters set here ?

- Firewall > traffic rules -- Check here if any specific rules configured for specific subnets / zones.

- If this also involves the clients getting knocked off the WIFI totally then please also check the DHCP scope and lease time under the LAN   interface

- Also from the question I assume there is no other WAN connection accept the single "4G" on the RUT ?

- What is the firmware on Rutx11 ?  

- Lastly, Install "Wireshark" on the laptops and capture the traffic stream post which you can use the ip.adr == <laptop ip address>&&tcp.port eq 443 same for 80, 53 to check and simultaneously capture the troubleshoot file on the RUT.

Comments

WIFI on the RUT is disabled, it's handled by an Ubiquti AP.

i already use other DNS servers.

There is no data connection limit configured whatsoever.

Firewall is default. No rules added/active

4G connection is the only WAN connection on the RUT

Firmware on the RUT is the latest , RUTX_R_00.07.01

Testing method:

i'm testing from a laptop using vmping utility. GitHub - R-Smith/vmPing: Visual Multi Ping. Color-coded ping utility for monitoring multiple hosts.

The utility is pinging continously to the internal devices (accesspoint, controller etc.) on the network without interuption.

Same is the case when pinging 1.1.1.1 and 8.8.8.8 on the internet. No interruption.

However, the utility can also continously connect to a certain tcp port. I do this to port 53 for my dns server and 443/80 to some websites. This connection randomly and frequently drop while the ICMP traffic has no problem to the same hosts on the internet.

I'm not 100% sure this is RUT related or something my provider is causing, but any ruling out some stuff helps during troubleshooting.

Answer 3

Hi,

For testing purposes would it be possible to (isolate the Ubiquiti), enable the RUTX11 WIFI and have the laptops connect to it to monitor the behavior while also capture and share the troubleshoot file and tcpdump logs from RUT as soon as you see the loss of Http/s and DNS traffic and Wireshark from the laptop's experiencing the traffic drops.

Just to double confirm there is no proxy service involved and the ping utility to monitor the hosts and ping the external ip's is being done from within the LAN ? (Also, provide : traceroute to the Lan clients ip from this ping tool and to external 8.8.8.8).

Thanks

Comments

i already isolated the ubiquiti AP by bypassing it using wired connection to the RUT. Same issue for the client connected wired.

I troubleshooted some more.

I can monitor the amount of NAT connections for the RUT by using the cli command "cat /proc/sys/net/netfilter/nf_conntrack_count"

If this counter is getting closer to the value of 600, new sessions start to fail. Arround 300 or below as a connection count, all is fine and stable. We can simulate this by opening alot of sessions by opening alot of browser tabs.

What's the best way to determine if the RUT is causing issues with these connection counts or maybe our service provider is somehow throttling the connection count coming from a single SIM ?

Answer 4

Hi,

During the "NAT" connections simulation during which you see the failed sessions, please can you capture the "troubleshoot" file from the RUT and send it over for analysis to me ?

Thanks

Comments

sure. I've sent it over to you. today at 14:38 we started testing, connectivity was bad arround 14:40, returned stable on 14:43. Session counter was between 500-600 when we had troubles and sessions were dropping, when stable session counter was below 300.

we are also in contact with our provider and they are also looking into it and have already said a session limit arround 600 could be the case on their side.

But still, good to check on both sides for such possible limits.