10Gbps switch/router query

[safc10g]

Hi All

With a 10Gbps internet connection into a building (FTTP) what kit should it be paired with for optimum performance?

I am assuming the connection comes in, then it should be connected to a 10Gbps switch, this is where I become confused after this..

Where does a router come into play and what kind does it need to be?

• PC will have a 10Gbps network card.
• Switch will be 10Gbps.
• Current wireless routers (ASUS RT-AX92U AX6100 Wi-Fi 6) are only 1Gbps Ethernet capable

Use case requirements would be:

• PC to be able to use the 10Gbps connection
• All other devices on network to need maximum of a gigabit connection, including a NAS.

Without spending an absolute fortune what is a suitable way to go?

So far I have considered:

• Netgear RX9000
  • This has 1 10Gbps port which I assume would connect to a switch, with the switch connected to the ONT?
• S3900-24T4S 24-Port 10/100/1000BASE-T Gigabit L2+ Stackable Managed Ethernet Switch with 4 10Gb SFP+ Uplinks, Fanless
  
  • This would connect to the router above via 10G port and then all of my other kit would connect to the standard gigabit ports.
  • Leaves some spare ports for future proofing
• TP-LINK T1700G-28TQ JetStream 24 Port Gigabit Stackable Smart Switch with 4 10GE SFP+ Slots
  
  • As above, little bit cheaper - but its a name I have at least heard of!
• 10G network cards in a PC to handle routing
  • Connect PC direct to switch and then run something inside a VM (pfsense etc?) to act as a router
  • Would this be better/worse than using the Netgear router above? (Netgear router will cost about 300 quid)

Thats all for now!

 

[safc10g]

The first thing connected to the incoming connection is going to be the router. Any switches and other devices will connect behind it.

Doesn't the connection provider have recommendations? A router capable of handling a connection of that speed is going to be non-trivial. You aren't going to be looking at the sort of domestic devices the likes of Asus produce, even their top-end stuff. That Netgear, for example, does have a SPF+ port so is 10GbE capable, but that's only on the LAN side. On the WAN side, it's still only Gigabit.

They have suggested the XG-7100 - a pfSense device - going to be not short of £1k though!

The SPF+ port on those Netgear routers can be configured to also act as a WAN port, but as there is only one how would I connect this to the ONT and switch?

Ref: https://kb.netgear.com/000060605/Ho...or-WAN-on-my-NETGEAR-Nighthawk-R8900-or-R9000

I think if I want to look at the consumer side of things it is going to be this: https://www.asus.com/Networking/RT-AX89X/

That would allow a connection to the ONT and also to a 10Gb switch. But I cant find that anywhere in the UK.

However...looking at a separate switch and router, the price is going up of course - so maybe I am indeed better off looking for an all in one like that XG-7100 or similar? (Looking at the spec though, surely a PC could handle this with ease, even in a VM? It is only a quad core atom with 8GB ram and a couple of 10Gb network cards...I think?

 

[safc10g]

Have you considered saving an incredible amount of cash and settling for 1Gbps? You want to route and NAT 10Gbps, that isn't cheap.

I have and it is starting to become more and more obvious that I really should stick with 1Gbps! Must be a way though....surely?

 

[safc10g]

So would this work:

1 x Netgear R9000 with the 10Gbps port set as WAN, connected to a 10Gbs switch - which is then connected to the ONT from the provider?

Then I can connect a PC with a 10Gbps card to the switch and anything else (gigabit) to the gigabit ports on the switch?

 

[safc10g]

Is the Internet connection actually 10Gbps? (I wasn't aware FTTP was even an option at 10Gbps yet)

Even if you have a 10Gbps connection, then what is the use case?
i.e.
Does one PC need to be able to utilise the whole connection (in which case that PC and the rest of the network needs to be 10Gbps), or is it a case of e.g. 10PCs all needing 1Gbps at the same time?

Bearing in mind that even with a 10Gbps connection, you probably won't be able to download at 10Gbps

It is, yep. 10Gbps sequential!

Main use case is one machine acting as the master machine, which will have a 10Gbps card in. This will be used for uploading and downloading.

Thought process is:

Netgear R9000 - with the WAN port set to be the 10Gbps port. This will be connected to a 10Gbps switch, then the switch will connect (initially) only to a PC via 10Gbps, everything else will be 1Gbps.

 

[safc10g]

When you jump from 1Gbit to 10Gbit that's a huge jump in load on all network devices especially a router with a 10Gbps internet connection due to firewall rules and other services you may enable on it. Even the XG-7100 doesn't appear to be able to max out a 10Gbit for the firewall, see here:
https://store.netgate.com/XG-7100.aspx

That shows 6.81 Gbps for firewall and with more services/complex traffic its even slower. It can L3 forward well over 10Gbps but that's not what you would be using it for. You really need to ensure that all the devices in the chain can manage 10Gbit with the features you require and that's not cheap.

I would be surprised if the R9000 has the power to firewall 10Gbps (not checked, just going on cost/hardware).

Yeah that was what put me off the 7100, didn’t seem to be able to actually handle 10Gbps. Been reading up and pfSense have another product, TNSR which seems to be better suited to the higher speeds.

Good point on the R9000, probably not!

 

[safc10g]

Is this even possible? It's specifically marketed as a faster option for connecting something like a NAS. Even if it does allow it there's zero chance that router will have the processing power to get anyway near the throughput you'd need.

What's the budget for this? £100s or £1,000s?

It is, according to Netgear anyway. https://kb.netgear.com/000060605/Ho...or-WAN-on-my-NETGEAR-Nighthawk-R8900-or-R9000

Budget I want to keep under £1000 really. The lower the better, but accept lower price could well just equal rubbish!

 

[safc10g]

10Gb through a single connection will cost you more than £1000. If the stuff you're trying to do is multi-threaded then there are less expensive options.

No way that Netgear is capable of routing at 10Gbps.

You're going to need to spend on a router capable of routing at that speed with 2 SPF+ ports, one for WAN and one for connection to your PC which will also need a 10G capable card (hope you've got a fast disk array in it because that'll become a bottleneck otherwise)

It's by no means my area of expertise but even with that healthy £1,000 budget I can only think of building a monster PC and running *Sense on it or a Mikrotik CCR1036-8G-2S+

If you want 10G firewalling with an official Netgate appliance you're looking at the XG-1537 which is twice the budget. No doubt enterprise solutions are available from Cisco etc. for more $$$$

I don't know what you're going to do with all that bandwidth that wouldn't be perfectly adequate with 1Gbps bandwidth (unless you've got hundreds of clients) but good luck to you as I am more than slightly envious sat here on my FTTC connection.

Thanks both, very useful info as ever!

I essentially need (need being a very subjective word of course!) 10Gbps only on one PC. Nothing else on the network will be/need 10Gb.

Does this change anything at all?

 

[safc10g]

No - what is the traffic flow that needs to hit 10Gbps to your one PC? Is it a single connection or is it made up of multiple connections working alongside each other, such as what happens when Steam downloads a game.

Flow would be from things like usenet and also to/from Google Cloud. Anything else I could make do with slower connections.

I suppose I could make i work with the one input split into multiple? Fairly open to different suggestions, preference would be for one big pipe as I say, but wouldn’t be the end of the world should it need to be split to stop things becoming silly expensive.

I’m happy to spend around 3k max on a PC/Server/Workstation and the right hardware to get the best out of this. I was thinking 1k max for this as I had earmarked about 2k for a PC/Workstation kinds a setup.

Could I go all out with the computer and tick all the boxes at once maybe?

 

[safc10g]

If you mean Gdrive, you know if you can saturate that link to Google’s servers you’ll breach the daily upload limit in under a quarter of an hour?

And not that I wish to make any assumptions on your use case, but research Plexguide, go gigabit on your internet and save yourself money overall.

Yep, that’s a fair point. I know plexdrive gets around this by using team drives and rclone (or similar, cant remember the name now). Thats a good call actually, I could run plexguide in a VM locally to handle all of that side. Had completely forgot about that tbh, thanks for the reminder!

I’m really leaning towards just going gigabit. Will be gigabit as it happens for the first few weeks whilst they wait for an upgrade to complete on their backbone - so I will get a good feel for that at home. Only ever had it on a remote server, so never seems real if you get what I mean!

If the gigabit works well (why wouldn’t it?) I could just stick with that, or even add a couple more for pennies (27/month for each additional)

Always jump into stuff head first, it’s a bad habit haha!

 

[safc10g]

Looks like I should knock spend off the PC (as it was overkill anyway it seems!) and concentrate on a proper 10G router/switch like the Mikrotik CCR1036-8G-2S+

Suppose it makes sense, if I scrimp on the connectivity then it’s pointless.

If I just remove the 2080 card out of the PC spec it’s paid for! I didn’t need that sort of card anyway, as I like the idea of gaming on a PC but in reality I never would!!

Actually reading more, not sure that Mikrotik would work, cant see how much firewall throughout it can handle?

Only thing I have found with actual figures that seem to be able to max out the 10Gbps when firewalled is something like DrayTek Vigor 3910. Anything else?

Getting there! You know what is catching my eye though, looking at the spec of each device I find, seems to be a quad core processor with 4/8GB ram and some network cards.

So far the best I can find within a reasonable price, is that Draytek 3910:

Quad Core 1.2Ghz CPU
Unknown RAM (probably 4GB, that seems to be the norm?)
Up to 9Gps throughput
Up to 3Gbps VPN.

What I cant understand is why wouldn’t the money better spent on a PC which has a much more powerful CPU and more RAM, with the network cards. Surely that would give me better performance? (Is it just CPU/RAM that would determine the VPN and NAT performance?)

 

[safc10g]

Because even chucking money at the problem isn't a solution - PFSense may do 10Gbps with the right hardware and configuration but it isn't guaranteed.
TNSR will do it (and up to the 100Gbps range), but the fact it's pricing is "contact a sales advisor" tells me all I need to know.

The Draytek or something similar is probably the easiest route, but I wouldn't bank on getting anywhere near 10Gbps, merely "better than gigabit" in real world scenarios

Yep that was my concern with TNSR, contact a sales advisor does indeed tell you everything about that... ££££££

Am I right in thinking that the firewall “bottleneck” is only applicable for services I chose to have firewalled? I.e could I, for example, have raw download/upload speeds to a particular host by having a rule to disable the firewall for that? In that instance, the “lower” speeds will not concern me as much I guess!

I didn’t mean spend more on a PC btw, I was just floating ideas of more capable hardware at a lower cost. Something I was looking at elsewhere was using a Dell R620, with dual Xeon, 64gb RAM and 10Gb network cards to handle the 10Gbps throughput. That was pricing up at about £400. So this would work out cheaper than the Draytek and arguably better as its a much more capable machine and can be upgraded if ever required. Just an example I found, I am sure there are better/cheaper again?

Hitting my 15 posts per 24 hours limit here, so trying to put more into each post!

 

[safc10g]

What hardware does the ISP provide with a 10Gbps connection? (i.e. don't they supplier a Router with it?)

So far, just the ONT ( i think that’s what’s its called?)

I’ve done some more digging and found another ISP called B4RN who are rolling out 10Gbps and they are providing this kit (£300+ install cost which I guess covers it?) I can’t find any stock of it, so called B4RN to see if they can sell me one, and now much for. Could have found the one at last??

ZYXEL AX7501-B0

Combine that with something like a UniFi Dream Machine Pro? Use the Zyxel as the router/firewall?

Or still not as powerful as the more expensive options and going to hinder speeds?

My head hurts

 

[safc10g]

https://www.zyxel.com/uk/en/product...N-VoIP-IAD-with-SFP--AX7501-B0/specifications

Nope - that only has a single 10Gb (SFP+) port in for fibre connectivity, it only has 4x Gigabit LAN ports + AX wifi (up to 6Gb/s).

These sort of routers are designed for normal home use - i.e. multiple users who will share a 10Gb connection (either via wired connections or more easily now Wifi).


Your use case is outside the "norm" so you are going to struggle with anything that isn't enterprise kit (and consequently price).

Personally I'd question how important it is to be able to download your "linux isos" at 10Gb, rather than just sharing out the 10Gb connection with a "home" grade router like the above, or settling for a likely more appropriate 1Gbps connection, until the hardware market catches up.

Looks like it has an SFP+ fibre for WAN and then 1x10Gbps Ethernet port and 4x1Gbps Ethernet.

So my theory would that will allow me to use that 1 10Gbps Ethernet port to connect to a 10Gbps switch like the UniFi Dream Machine Pro and that would do it.

BUT....as you say, it may not work like that.

I could well just split the 10Gbps into multiple machines/connections, absolutely, whilst I wait for the market to catchup and offer something else. Not in a mad dash to get sorted for day 1.

 

[safc10g]

Apologies yep - it does have a 10Gb copper port as well as the SFP+ (I misread the 10G as 100 in the photo when I looked before).

In theory it should be fine, and if B4RN are rolling them out, then I'd say they've looked at what options are available and it should be decent kit (as from everything I've read B4RN seem to do a great job with their fibre roll out)

I’ve called them and asked how much to sell me one. Strangely the manufacturer suggested this!

Again it depends on your use case, B4rn do know what they are doing but the 10Gbps service is more of a demonstration of what their network can do and I'd be amazed if they have more than a handful of their customers on it, so I wouldn't necessarily take that as an endorsement of being able to push 10Gbps through it all day long.

You're being quite vague with the answers to questions and that's going to limit the amount of advice people can give you. What ISP are you intending to use? Whether they use PPPoE or not is going to be a huge factor in the performance of any router you decide to use. What exactly are you using this 10Gbps connection for? I'm not saying you have to say what files you're moving around but "I'm putting things on Google Drive" and "I am going to start up connections to 20 different Google Drive accounts and use it to chunk copies of movies into" is different.

sorry not trying to be deliberately vague, I’m just thinking as i go!

ISP is YouFibre - a new startup in the area. Ran by the same people as community fibre I believe. I don’t know about PPPoE that’s a good call, I will find out.

Main use will be downloading, from one machine, here is where i would like as much speed as possible. Also will be uploading, via something like rclone, to google drive (team drives within one account) Very similar to what plexguide does really.

Another thing will be keeping 3 x cctv cameras synched up to the cloud - instead of just recording to the HDD inside the NVR. Files are a decent size so speed would come in handy here too.

And finally plex, but not many connections from outside here, so that’s not concerning me too much.

 

[safc10g]

The Unifi Dream Machine Pro won’t route 10GbE at line speed. It’s rated at 3.5Gbps with everything switched on and 8Gbps with everything switched off. You’re looking at one of the top-end Mikrotiks to do this on a budget.

If you can find a UniFi USG-XG-8 that can route 10GbE but it’s EOL hardware now.

Cheers. Stumbled on the UniFi USG-XG-8 previously, but was going for about £2.5k!

So if I go with one of the top end Mikrotik’s - that is going to give me everything i need? If I go with one that has 4 SFP+ ports and a bunch of gigabit ports, that should cover me yep?

Looking at the test results on each product (nice touch that!) they seem to handle the required throughput from about the $995 and up range. Which one stands out for you out of those?

 

[safc10g]

This one seems very good value for money: MIKROTIK CCR1036-12G-4S

Can be found for around £700 - would that give me all I need, or would I still be lacking?!

 

[safc10g]

That one has 4SFP (gigabit ports), you'd need the
CCR1036-8G-2S+

Sorry yes, I messed up on the copy/paste there, I was searching for the wrong one! Thats the one I intended to search for the price on. So that’s about £800ish. I’m happy with that to avoid messing about and having it all running on a PC. Means I dont need any of my other gigabit switches either, this has enough ports to cover all usage for me.

Fantastic! Got there, thanks for your patience (and everyone else!)

 

[safc10g]

I know you're shooting for 10Gbps but honestly I can't see anything in your use case that needs it. Looking at YouFibre's website the costs are £250+VAT (£300) for 10Gbps, vs. £50 for a 1Gbps residential service. That's £3000 per year extra to download the entire contents of a 50GB Blu-ray disc in 45 seconds instead of 7 minutes. It's the difference between downloading 10TB in a day and 100TB. With the savings you can go on a very nice holiday.

You don't need 10Gbps for a single computer, that sort of service is designed for 50 people in a media company. You won't get anywhere near 10Gbps throughput into Google Drive, anyway.

You're massively underestimating what a symmetrical gigabit connection is capable of, especially when you're talking about connecting to Internet destinations rather than your own private WAN.

You are very much correct of course!

I’m slowly coming around to the idea of just sticking with the original plan of 1Gbps. Going to have too much to invest here for no additional benefit, you are all correct!

Off topic, but £3000 also buys a considerable amount of legal content.

It does. Got lots of that on my apple account from over the years. Nice touch getting the free upgrades drop in your library when they finally get around to it too!