Community Fibre - throttling / limiting TCP connections?

tomciti · 4 Jun 2022 at 19:47

Hi

I got "Community Fibre" FTTP 150Mbps download, 150Mbps upload package 2 weeks ago and I'm currently testing it. Community Fibre state they throttle or limit connections in any way but it seems to me they do. I got Linksys Velop router and ADTRAN 621i XGS-PON ONT modem from them.

The problem and how I can very easily reproduce it on my end in 1-3 minutes max:

- when torrenting/P2P (using qBitTorrent client with TCP peer connection protocol or TCP and UDP) on desktop client connected to the Linksys Velop router using ethernet cat 5E cable, after 1-3 minutes when torrent reaches over 25-30 connections in total, devices on the network connected to their broadband experience TCP connection time out errors e.g. can't open web pages (or it takes ages for pages to load) on mobile device via WIFI or a torrenting/P2P desktop client connected via LAN cat 5E cable or my Eufy Homebase2 CCTV hub goes down

- the issues go away when torrenting desktop client is connected to a VPN e.g. PrivadoVPN via Wireguard protocol (as then for Community Fibre side it's one connection so there are no connections time out errors on my network)

- I have eliminated the Linksys Velop router - I used old TP-Link Archer C50 router connected to the ADTRAN modem they provide and reproduced the issue on their broadband (so it's 100% not Linksys Velop router fault) using exactly the same test

- I'm not able to reproduce the issue on Virgin M100 broadband (which I still have here) using VirginHub 3 router and the same mobile and desktop clients connected exactly the same way - no issues whatsoever even when torrent / P2P reaches 400 connections in total after 40 minutes of testing

- the conclusion here is either that Community Fibre broadband can't handle many P2P/torrent connections or generally they set a limit on a number of TCP connections customer can establish or maybe the ADTRAN modem is causing issues?

I have contacted their support few days ago but they don't seem to be very helpful so far apart of telling me to reboot the Velop router and ADTRAN modem or clearing the MAC table reservations on their end (probably because I said I reproduced the issue on my own router)

Has anyone experienced similar issues with them?

Caged · 4 Jun 2022 at 20:03

Does it smell like this?

When your Fiber ISP's 'Dumb' Device Screws Your TCP Sessions - Neel Chauhan

UPDATE: This article is inaccurate. The real issue is another feature called Broadcom Packet Flow Cache which puts a hard limit of 16384 TCP sessions on the 716GE-I R2. Other ONTs do not have this issue. When I moved neighborhoods within Seattle, I subscribed to CenturyLink’s fiber-to-the-home...

www.neelc.org

tomciti · 4 Jun 2022 at 23:34

Caged said:
Does it smell like this?

When your Fiber ISP's 'Dumb' Device Screws Your TCP Sessions - Neel Chauhan

UPDATE: This article is inaccurate. The real issue is another feature called Broadcom Packet Flow Cache which puts a hard limit of 16384 TCP sessions on the 716GE-I R2. Other ONTs do not have this issue. When I moved neighborhoods within Seattle, I subscribed to CenturyLink’s fiber-to-the-home...

www.neelc.org

Hm...It does a bit, thanks. Will ask the Community Fibre support to look into it - hard to say whether they will or not, first they will blame me and tell me to restart router and modem 10 times probably.

CaNsA · 14 Jun 2022 at 06:19

Hey man,

I was having a similar issue, albeit with a different ISP.
Try changing the peer connection protocol in the qBitorrent settings from "TCP and uTP" to just "TCP"

Rroff · 14 Jun 2022 at 06:54

It may be ISP configuration/policy but the connection cycling done by torrents can be poorly handled at any point in the chain potentially - it may have a relatively benign explanation.

tomciti · 14 Jun 2022 at 07:47

Thanks. I have heard back from Community Fibre - the said:

"We believe this is causing because you are on our base product which comes with CGNAT and it has session limit of 1000. If on your network the limit reaches to 1000 sessions it includes each and every open session on all the devices which are connected to the network."

They put me on a dedicated dynamic public IP where there is no limit for sessions for testing and all problems went away. The dedicated dynamic public IP comes with higher packages they offer - 500 Mbps and up. I have decided to stay on 150 Mbps and use VPN (NordVPN; tested also PIA but P2P speeds were very slow in my tests).

Rroff · 14 Jun 2022 at 07:50

Ah the old CGNAT - should only really ever be a thing on mobile connections :s

ChrisD. · 14 Jun 2022 at 08:35

Yeah CGNAT should have gone in the bin long ago for home connections.

Rroff · 14 Jun 2022 at 08:41

ChrisD. said:
Yeah CGNAT should have gone in the bin long ago for home connections.

Shame there doesn't seem to be a good way to move everything to IPV6.

Caged · 14 Jun 2022 at 08:41

CGNAT will be around for as long as IPv4 is because it solves a problem (more homes wanting connections than IP addresses available).

What's inexcusable is new ISPs that are forced into using CGNAT not being IPv6 from day one.

Rroff · 14 Jun 2022 at 08:47

Caged said:
CGNAT will be around for as long as IPv4 is because it solves a problem (more homes wanting connections than IP addresses available).

What's inexcusable is new ISPs that are forced into using CGNAT not being IPv6 from day one.

I've had odd experiences with CGNAT - for instance my TP-Link MR6400 CGNAT is barely even an inconvenience while other routers (which don't have problems generally on non-CGNAT connections) even getting Steam to work properly with CGNAT can be an issue.