Load Balancing/Bonding Router

[B1scu1T]

I have a friend who kinda lives a bit off the beaten track, and although they have a VDSL line, he is so far from the cabinet that the speeds are garbagio. To try and supplement his connection, they signed up for a "Community Broadband" system which is (as I understand it) direct microwave connection to a nearby village.

Their plan was to run them side by side for a while and see which one makes the most sense for them, but I plan to head over and help them make sure what their real bottleneck is.

I suggested that one thing they could try is replacing their routers, as they just have the ones provided by the ISPs.
A thought that had crossed my mind, was they could try and get a dual WAN router so they can get the best out of VDSL (probably better latency) and the community broadband link (possibly better link speeds).

Its something I have never looked into in any great depth from a consumer level, but I would usually buy something lower end Cisco for a task like this, but I would ideally like to avoid something that is totally serviceable by non-network type people.

There is also a desire to throw up some higher power wireless AP (big house), so I did look into the possibility of a Ubiquiti USG with a little POE switch. The prices are not prohibitive and I might be able to persuade them to go all in for the "ecosystem" which would make life a lot more straight forward. I have been informed, however, they don't actually support any kind of load balancing, and only a failover. The Edgerouters seem to have more features in this respect, but they lose the easy configuration...

Are there any other options/product lines that I should really look into for them for a slightly-better-than-basic-consumer, or almost-but-not-quite-pro solution?

Suggestions of a single router/firewall/switch/hotspot would also be perfectly fine. Asus devices worthwhile?

Edit: How well do some of the open source OS'es such as PFSense handle this kind of task? I have a PC I could throw together to do some testing before they invest.

 

[BigT]

I ran a tri-WAN connection for similar reaosns before proper VDSL came to my village. My solution was pfSense on a six-NIC AES-NI CPU compatible micro PC - about £250. It was great for routing traffic by various policies so that I made the most out of a wired and two 4G connections. This is/was coupled to dumb switches and Ubiquiti access points for a really nice experience for end users although it required a bit of setting up. While I don't need the three WAN connections anymore, I still use the same solution and wouldn't go back from it.

 

[B1scu1T]

The PC I have to test such a solution is an old Athon AM1 with a dual intel NIC, do you reckon such a basic machine will be suitable?

 

[bremen1874]

This sounds like an overly complicated solution to be inflicting on a friend. Are you going to support it in the long term?

IMO they should pick the best connection and leave it at that. If they're both really bad then there might be some point to load balancing but isn't a best first option.

There are some very nice mesh systems available that would handle the wireless and are designed to be easy to use.

 

[B1scu1T]

They are both really bad, apparently. I may actually even suggest they look at 4G options.

At the moment it's just for testing, they're not completely un-technical and should be able to manage it at a basic level. pfSense probably wouldnt be suitable as a long term solution, if it worked as a concept I would like to move to something else more consumer focused. I think some more common routers from companies like Asus have a dual wan mode which is a fairly simple flip of an option in the browser gui.

I hope to get them set up with each individual system working as best as it can without any clever solution in the middle, then add/replace it with clever solution, but make sure they can unplug and replug if something goes wrong and im not available to help.

Mesh probably wouldn't be necessary, as the house is very well connected with CAT5e, multiple independent hotspots would overall be a better solution in this case IMO.

 

[BigT]

The PC I have to test such a solution is an old Athon AM1 with a dual intel NIC, do you reckon such a basic machine will be suitable?

It'll be fine as long as you're not expecting to run a VPN on it and evaluate before v2.5 comes out, because that will mandate an AES-NI compatible CPU which I assume the Athlon is not. But for general routing duties on the current 2.4.x version it'll be OK.

@bremen1874 makes a good point though. Even switching to a more consumer friendly router, introducing dual WAN of any form is going to see the odd situation where something will need your help because the ISPs aren't going to provide support for such a configuration.

 

[B1scu1T]

It'll be fine as long as you're not expecting to run a VPN on it and evaluate before v2.5 comes out, because that will mandate an AES-NI compatible CPU which I assume the Athlon is not. But for general routing duties on the current 2.4.x version it'll be OK.

Believe it or not, the Kabini chips actually do have hardware AES-NI built in!

@bremen1874 makes a good point though. Even switching to a more consumer friendly router, introducing dual WAN of any form is going to see the odd situation where something will need your help because the ISPs aren't going to provide support for such a configuration.

The point is certainly taken onboard. I suppose my friend and I are equal parts looking to improve his situation, and have a bit of a fun experimentation project.... plus the ISP hasn't been particularly helpful to them as it is, which is why they signed up for the community broadband thing anyway.
Could be that I get there and start looking at their set up and immediately spot something amiss, resulting in the BT line working great.

 

[BigT]

I've just thought, you're going to need 3 NICs. 1 x LAN and 2 x WAN

 

[B1scu1T]

I've just thought, you're going to need 3 NICs. 1 x LAN and 2 x WAN

The motherboard has one built in also... question is whether they're all supported or not.

Dual NIC (Intel 82576): https://www.amazon.co.uk/gp/product/B071R3YS2H/ref=oh_aui_search_asin_title?ie=UTF8&psc=1
Motherboard (Realtek® 8111GR): https://www.asus.com/uk/Motherboards/AM1MA/specifications/

 

[bremen1874]

You're going to need a VDSL modem as well (unless the ISP router can be bridged or you're happy to double NAT).

 

[Avalon]

Believe it or not, the Kabini chips actually do have hardware AES-NI built in!

The point is certainly taken onboard. I suppose my friend and I are equal parts looking to improve his situation, and have a bit of a fun experimentation project.... plus the ISP hasn't been particularly helpful to them as it is, which is why they signed up for the community broadband thing anyway.
Could be that I get there and start looking at their set up and immediately spot something amiss, resulting in the BT line working great.

Even if it lacked AES-NI other (some would argue better or at least not subject to the childish outbursts of someone who should know better) projects exist such as OPN/Mono/IPFire/VyOS/XG exist depending on your preference/priority.

May I suggest (with your friends permission) you run the line details through the BT DSL checker to see what the line is supposed to be capable of? That will tell you if you’re barking up the wrong tree.

 

[B1scu1T]

You're going to need a VDSL modem as well (unless the ISP router can be bridged or you're happy to double NAT).

Thats a good point, I'm kinda relying on there being a "modem mode" for the BT hub, although to enable them to be able to roll back to a simple single router, double NAT might be the better bet for the time being. No clue what the community broadband is going to be in terms of hardware.

Even if it lacked AES-NI other (some would argue better or at least not subject to the childish outbursts of someone who should know better) projects exist such as OPN/Mono/IPFire/VyOS/XG exist depending on your preference/priority.

May I suggest (with your friends permission) you run the line details through the BT DSL checker to see what the line is supposed to be capable of? That will tell you if you’re barking up the wrong tree.

Is there any of those other projects which stand above as clearly better in some way?
Will certainly be checking that once I'm there (been planning to do this for a while but a good time hasnt popped up)

 

[Avalon]

Thats a good point, I'm kinda relying on there being a "modem mode" for the BT hub, although to enable them to be able to roll back to a simple single router, double NAT might be the better bet for the time being. No clue what the community broadband is going to be in terms of hardware.


Is there any of those other projects which stand above as clearly better in some way?
Will certainly be checking that once I'm there (been planning to do this for a while but a good time hasnt popped up)

Run the line details first, its pointless wasting time on anything else till you know what the line is capable of and if it's worth pursuing.

 

[Caged]

What's the Three reception like where they are?

http://www.three.co.uk/Discover/Devices/Huawei/HomeFi?memory=0&colour=Black

 

[B1scu1T]

Run the line details first, its pointless wasting time on anything else till you know what the line is capable of and if it's worth pursuing.

I'd just like to focus my research a little bit first. Might be some time before I go round there due to work and I'd like to be prepared with some ideas. Just wondering if there are any of those OS's I can rule out before I start digging into them.

PFsense for example, is not particularly easy to research with the direction the parent company is taking it.

What's the Three reception like where they are?

http://www.three.co.uk/Discover/Devices/Huawei/HomeFi?memory=0&colour=Black

I know the O2 reception is pretty good there and the EE isnt, not sure about 3 but a device like this is something I have definitely thought about for them.

 

[MTA99]

Have you looked at the Draytek 29xx routers for an "off the shelf" solution?

We use the non-WiFi version of this in the office with unifi APs. We have 4 connections (2 x virgin fibre, 2 x DSL ........ Don't ask!!) with 2 balanced/bonded through a Draytek.

https://www.draytek.co.uk/products/business/vigor-2926

Get the right usb 4g modem and you can add a 3rd (and 4th) connection.

 

[Avalon]

I'd just like to focus my research a little bit first. Might be some time before I go round there due to work and I'd like to be prepared with some ideas. Just wondering if there are any of those OS's I can rule out before I start digging into them.

PFsense for example, is not particularly easy to research with the direction the parent company is taking it.

I know the O2 reception is pretty good there and the EE isnt, not sure about 3 but a device like this is something I have definitely thought about for them.

If the ADSL line is crap (I know people in similar situations where 'best effort' is basically 512k if you're lucky and will loose sync regularly/don't raise faults or that will be withdrawn), then it's a dead end and you can move on, so why not spend two minutes identifying the connectivity options before you waste further time going down a potential dead end. Same with the three network coverage checker, shove the postcode in and you know the variables involved then. If you rule both of those out as viable options then the community broadband is your best chance and i'd suspect community = USO compliant as it'll be a grant subsidised set-up and that will have strings attached. To answer your question pretty much all of those projects can do what you want, but at this stage you have no clue if that's even what you want because you haven't checked.

Although i'm not a massive fan of pfsense's actions, the hate that pfsense directed towards OPN was frankly unbecoming of any project worth using, let alone one that it is trying to present itself as a professional security outfit and one particular senior member of the team's community responses outright suck at times, it's one of the easiest projects to find information/guides on as it's community is that large and uses it for a wide range of purposes, it can be made to do most things, though that isn't always 'best practice'. Load balancing is a very generic term, but the finer points are what can kill it's application for a set-up such as this.

 

[Caged]

As above, you must have an address that you can shove into a checker for FTTC/4G which is going to let you quickly rule out certain options.

Load balancing also isn't bonding, so that might be a consideration that needs to be had. Once you get into bonding though you're talking about deploying equipment somewhere to terminate VPN tunnels on, but without using public cloud as you'll run into a bunch of problems with content region locking.

By focusing on the OS that you're going to run on a load balancing router you've skipped several steps and ultimately the OS you run doesn't matter if the underlying service is a couple of 5Mbps links and you're trying to run an 8Mbps stream over it.

 

[B1scu1T]

So after FINALLY managing to find some time (but still only a few hours)to get round there, both the vDSL and the community broadband actually seemed ok, but they said both would fluctuate a lot so it could well have just been timing. We decided not to investigate anything super advanced like load balancing, traffic rules, failover (...etc) and for the moment and to just get some better wifi reception around the house. I grabbed all my old extenders, routers and AP's and took them round.

The current BT situation
Line check doesnt work with their address for some reason...
We ran a speed test and got speeds between 5-10Mbps down, with 1-2Mbps up... not too bad. The latencies were fluctuating heavily.
I think some of the issues they're having with the BT solution are related to the wiring in the house. BT have brought a huge multicore into the house wired up to 2 krone blocks which are then punched into a couple of (what appear to be) CAT5 cables, which then go into the office upstairs (probably another 70-100ft) where the microfiltered plate is, followed by an overly long RJ11 cable that was coiled up right next to all the power supply for the network cabinet... which the wireless router was placed inside.
The router itself is a typical business hub and seemed to be ok, worked pretty good for me plugged in via Ethernet or with Wifi, but the coverage was poor as you would expect.

I moved the router away from the cabinet and uncoiled the cable in the hope that it would give them a little extra stability/performance. Unfortunately they didnt know their admin password and have lost the card, so I wasn't able to go in and have a look/change at the way their hub is set up with regards to the LAN, DHCP, Wireless channels, and what it's capable of.

The current Community Broadband situation
I didnt see the actual dish for the link, but Im pretty sure this is a microwave dish on the top of the house somewhere. It's connected via a router with a POE port for the dish and then the typical 4 port dual band WIFI.
I couldnt connect to the wifi at all, and they said that it was normal and you had to just keep trying... red flag. I think there is perhaps something wrong with the wifi settings on the router.
When I connected to the ethernet everything worked fine, getting a Class C address and can access the admin homepage... BUT... the documentation doesnt state what the admin and password is for their router, it wasnt the default, and isnt written on the router... I have a suspicion they have deliberately locked out the users... another annoyance and a bit of a roadblock.
Speeds were a little better, more consistent and... I cant remember what the latency was like actually... but it doesnt matter for them at this point, they just want it to work. In hindsight I wish I had though to run some traces to see whats actually going on.

The testing/temporary solution
So armed with some old wireless N routers and a range extender, used the house's existing infrastucture to give them wireless access in areas they didnt previously have it. It's cheap and dirty, but they're going to run it for a week or so to see if there is any point in investing in some hardware. They can pull the cables from one router and plug it into the other, and despite being sub-optimal. For the moment they can keep trying to figure out what works better for them, but I think they have a pretty good idea already.
One thing that was clear from testing is that the walls barely let any signal through, so there wont be any shortcuts, this is going to take a few APs.

What they want and will probably do
Ignoring the built in Wifi, The community broadband seemed to offer them the best stability whilst I was there, but I think they will eventually want to have a backup. The idea for the backup has shifted now though, instead of both the existing lines, one will be cancelled and the other will be a 4g solution on an auto failover, as they do get decent O2 reception in a sparsley populated area, so good speeds also.

How much they want to invest, I'm not sure. I reckon it's going to take 2 powerful APs for the main part of the house, and another 1 or 2 less powerful ones to get them full coverage, they asked me to give them figures and I told them they could put something cheap together that would work, but wouldnt be "smooth" for around ~£100, or they could invest £500-600 and get a good wifi mesh system with Ethernet backhaul.... or really anywhere inbetween depending on how they want to approach it.
We'll see what they come back with in terms of numbers, but I'm expecting it to be closer to the top end rather than the bottom end, purely for coverage.

TL;DR
Neither line they currently have is perfect, but one way or another we're going to have a "fixed" broadband line with a backup 4G, so we need still need a dual WAN router.

The key element is going to be in the Wifi system they select but importantly, if you look at the datarates they're getting on the line, I dont need to be too concerned about super duper 2200AC APs with crazy data rates. Range and power is more important.

I think the kit list should be:
New dual WAN router in the cabinet
4G Bridge that needs to be outside the cabinet (I think something like this, then the supplied dongle plugged in). Infrastructre in the house is good enough that they could literally put this anyway and trunk it to the cabinet for the best results.
Possibly a small fanless switch (I can donate) or POE switch, depending on the selected APs.
Probably start with 3 APs, perhaps 4, but would like a system that can scale with good roaming features and ethernet backhaul. Mesh systems seem to offer this, even if we don't actually use the meshing backhaul itself.

 

[Avalon]

Your friend with an office, comms cabinet, business hub and two connections that now requires failover sounds a lot like a business. Admittedly a really, really badly run business on the IT side, but a business non the less.

In the residential side connectivity stats can be viewed without logging in, as from memory the hardware is near identical on both, i’d hope for similar on the business side. Also if you can’t locate the password, call BT and ask for it (the account holder that is). Speedtest tells you what the peering is like to a set end point with the lowest ping, it doesn’t always follow that tyts the endpoint with the greatest throughput and it’s dependant I’m off network routing/peering in many cases.

Same with the community broadband, ring them, ask for connection details.

Also did you check the postcode with Three? Since your last post they have really started pushing the unlimited 4G service as a credible alternative to fixed line.