Linking 2 PCs over 10G query

Associate
Joined
19 Mar 2009
Posts
1,615
Location
Leicester
Hi Guys,

After some networking advice. I want to link my PC with our home server using a 10G connection to improve transfer speeds. Currently accessing the server as a network drive and both are running windows.

My plan is to buy two 10G NICs and directly connect the two PCs whilst still using the onboard 1G NICs connected to the router for internet and the rest of the network.

I know I will need to assign the 10G NICs IP addresses off the main domain base address to give them their own network but how do I make sure they talk to each other via the 10G connection rather than going via the router over the 1G links?

Many thanks in advance.

Hawker
 
Last edited:
After some networking advice. I want to link my PC with our home server using a 10G connection to improve transfer speeds. Currently accessing the server as a network drive and both are running windows.
Assuming you are transferring files big enough and between storage devices that actually will benefit from a 10Gb connection?

My plan is to buy two 10G NICs and directly connect the two PCs whilst still using the onboard 1G NICs connected to the router for internet and the rest of the network.
Switch would obviously be the better option. 2.5Gb might also be worth a consideration (e.g. if your router already has it etc)

I know I will need to assign the 10G NICs IP addresses off the main domain base address to give them their own network but how do I make sure they talk to each other via the 10G connection rather than going via the router over the 1G links?
If you assign the 10Gb NICs an IP on a completely different range (e.g. if your normal IP range is 192.168.0.x, then you could use 10.0.0.x), then access them via IP address in windows explorer etc.
 
Assuming you are transferring files big enough and between storage devices that actually will benefit from a 10Gb connection?


Switch would obviously be the better option. 2.5Gb might also be worth a consideration (e.g. if your router already has it etc)


If you assign the 10Gb NICs an IP on a completely different range (e.g. if your normal IP range is 192.168.0.x, then you could use 10.0.0.x), then access them via IP address in windows explorer etc.

Thanks for the reply. Yes I am ripping my video library so will make use of the speed.

My longer term plan is to get a 10G switch but I need to get some ethernet cabling run round the house to make proper use of this.

Will that let me map the drive via windows explorer?
 
Last edited:
While it's not answering your question directly... is 10GB needed or would you be happy with something more than 1?

A 2 port PCIe NIC (giving each machine 3 full interfaces) would have windows automagically bond the connections (SMB3.x "just works" when it comes to spotting teaming opportunities, it'll team ethernet and wifi too or whatever else is available as complete, separate circuits between 2 nodes) and give around 2900Gb/sec.

Edit: Actually, as this goes, it should ALSO spot the 10Gb link between the 2 and give 11 total, don't worry about the 1Gb via router link "being seen when moving files" it'll use that AND the 10Gb when doing file transfer. Said with 100% the 3x 1Gb link option being one I've used, all via an 8 port switch with the 1 link from that to router for internet.
 
Last edited:
While it's not answering your question directly... is 10GB needed or would you be happy with something more than 1?

A 2 port PCIe NIC (giving each machine 3 full interfaces) would have windows automagically bond the connections (SMB3.x "just works" when it comes to spotting teaming opportunities, it'll team ethernet and wifi too or whatever else is available as complete, separate circuits between 2 nodes) and give around 2900Gb/sec.

Edit: Actually, as this goes, it should ALSO spot the 10Gb link between the 2 and give 11 total, don't worry about the 1Gb via router link "being seen when moving files" it'll use that AND the 10Gb when doing file transfer. Said with 100% the 3x 1Gb link option being one I've used, all via an 8 port switch with the 1 link from that to router for internet.

Yeah I would rather just future proof the system and go all the way to 10G :)

Thats interesting, would I need to give them static IPs on the same range (i.e. all on 192.168.0.x for example) for it to work this out or would it work on different range i.e. router on 192.168.0.x and these two on 192.0.0.x)
 
Thats interesting, would I need to give them static IPs on the same range (i.e. all on 192.168.0.x for example) for it to work this out or would it work on different range i.e. router on 192.168.0.x and these two on 192.0.0.x)
Historical it never used to, but might have changed with SMB3 etc
 
I know I will need to assign the 10G NICs IP addresses off the main domain base address to give them their own network but how do I make sure they talk to each other via the 10G connection rather than going via the router over the 1G links?

You need to set the routing priority. To do this you give different routes different metrics; TCPIP should choose the connection with the lowest metric . For best speed enable jumbo frames.

However, why not make it easy on yourself and get yourself a switch with two - or more - 10 Gb ports? I have 8 port TP-Link switches.
 
You need to set the routing priority. To do this you give different routes different metrics; TCPIP should choose the connection with the lowest metric . For best speed enable jumbo frames.

However, why not make it easy on yourself and get yourself a switch with two - or more - 10 Gb ports? I have 8 port TP-Link switches.
Thanks Quartz, will jumbo frames affect data transfer to other devices on the network? I.e. a streamer reading from the server with with 1gb connection? I read there could be issues.

I will get a switch at some point but wanted to set this up in the interim. Which TP-link switch do you have and how is the fan noise? Atm if I get one it will need to sit under my desk and I’ve heard they can be loud!
 
Yeah I would rather just future proof the system and go all the way to 10G :)

Thats interesting, would I need to give them static IPs on the same range (i.e. all on 192.168.0.x for example) for it to work this out or would it work on different range i.e. router on 192.168.0.x and these two on 192.0.0.x)

Can all have whatever for IP range/subnet. Mine previously were all still DHCP from router.

Historical it never used to, but might have changed with SMB3 etc

Yeah, never used to. DEFINITELY had this going between my main box and a donkey/download/bit part server machine at home. Was quite pleased (bits of windows that "just work"!) as it was a decent solution for me at the time. Win 7 for my use with it.

Jumbo frames wise... I THINK it's fairly automatic on Gb+ nics but worth a check. It's the "small weird, device specific properties" bits from the devices manager console, should be in there. There's SMB multichannel and link aggregation in very modern windows. I THINK for your purpose, either works but might be worth a look/read/check.
 
Last edited:
Thanks, I guess it will be fine the way I am going to set it up now as the two 10G NICs will only talk with each other so jumbo frames wont interfere with anything else but down the line if i sort out the home network and introduce a switch and its trying to stream to a firestick or something it might be a problem?
 
What drives do you have in the machines? One ssd alone isn’t going to touch 10gb speeds. Take this into consideration as well.
 
Thanks, I guess it will be fine the way I am going to set it up now as the two 10G NICs will only talk with each other so jumbo frames wont interfere with anything else but down the line if i sort out the home network and introduce a switch and its trying to stream to a firestick or something it might be a problem?
Jumbo frames is.... pretty much a standard thing since about 2005. There'll be some weirder/older stuff that'll be a little less uniform in it's use but it's pretty standard in everything (Gigabit Ethernet at VERY least) for a decade or 2 at this point. I'd generally not even consider it a thing to be wroth checking/worrying about unless you notice speeds not where they'd be expected.
In some business/heavy server type settings you get issues around MTU sizes ("how big can a single packet be?"/OS network packet buffers, all related to jumbo frames) but for chucking a card in each of a couple of home machines, I'd not expect to have to touch anything there.


What drives do you have in the machines? One ssd alone isn’t going to touch 10gb speeds. Take this into consideration as well.

If it's NVMe - which... is probably most of us by this point, 1250MB/sec would be the max 10GbE can handle (10 gigaBIT) - most single NVMe drives would be able to saturate that. Even an Intel 660p (PCIe 3 - intended at it's prime as a "slow nvme" drive) does 1800MB/sec (on paper at least).
 
Last edited:
Jumbo frames is.... pretty much a standard thing since about 2005. There'll be some weirder/older stuff that'll be a little less uniform in it's use but it's pretty standard in everything (Gigabit Ethernet at VERY least) for a decade or 2 at this point. I'd generally not even consider it a thing to be wroth checking/worrying about unless you notice speeds not where they'd be expected.
In some business/heavy server type settings you get issues around MTU sizes ("how big can a single packet be?"/OS network packet buffers, all related to jumbo frames) but for chucking a card in each of a couple of home machines, I'd not expect to have to touch anything there.




If it's NVMe - which... is probably most of us by this point, 1250MB/sec would be the max 10GbE can handle (10 gigaBIT) - most single NVMe drives would be able to saturate that. Even an Intel 660p (PCIe 3 - intended at it's prime as a "slow nvme" drive) does 1800MB/sec (on paper at least).

Thanks that really helpful I will play with the jumbo frames and see if it causes any trouble.

...yes running NVME drives so should definitely make use of the increased bandwidth.
 
Thanks Quartz, will jumbo frames affect data transfer to other devices on the network?

No.

Which TP-link switch do you have and how is the fan noise?

I have two TL-ST1008 switches but I replaced the fans with quiet Noctua ones so there is no noise issue.

Just looking elsewhere I see you can get a 5 port switch for under £200 and an 8 port switch for under £250. RJ45, not SFP.
 
Thanks Quartz, will jumbo frames affect data transfer to other devices on the network? I.e. a streamer reading from the server with with 1gb connection? I read there could be issues.

I will get a switch at some point but wanted to set this up in the interim. Which TP-link switch do you have and how is the fan noise? Atm if I get one it will need to sit under my desk and I’ve heard they can be loud!


Jumbo frames relates to the sending MTU of network datagram packets, essentially [header] your data here, more MTU, more data [footer]. 1500 bytes is the original standard (goes up to around 9000 bytes). It CAN help, depending on what the physical layer is (ethernet cable/wifi/microwave) and how reliable/prone to interference it is, the point being the bigger the MTU, the more data you're transmitting compared to the required header/footer wrap around the packet but as packet size increases, corrupted packets need resending so you lose more data per corrupted packet. On a home network down a few metres of cat 5/6 utp/stp - it's not really going to matter. You could make a case for bigger MTU if you REAAAAAALLY wanted to get the last few bytes of extra throughput, as it's unlikely there'll be much packet loss going on but it's pretty academic from a home user perspective.

Slap the cards in, expect well over 1GB (intentional big B)/sec transfers nvme to nvme. Start looking at settings/cable choice/routing away from mains/fans/etc if not.
 
Last edited:
Direct Ethernet connection to second NIC is fine. Use 10.0.0.1 and 10.0.0.2 as IP addresses, 255.255.255.0 as subnet and NO router address. With your requirement, don't need to change any NIC settings.

Do the drive mapping by IP address, not host name.

SSD on SATA 3 should still get 5,000Mb for read but probably less for write. You could do a RAID 1 pair to increase throughput at the risk of losing it all if one drive fails.

Personally have an ex-enterprise RAID 5 card with a SSD cache drive in my server with 6 x 4TB WD Red and getting quicker than SSD throughput.
 
If you need 2x or more 10g switches that are silent/passive (no fan mod needed) and all copper (RJ45), the available ones in the UK with available UK support are:

TP Link SX-105. It's a 5 port 10g unmanaged switch, but there are many reports of needing to RMA these back at some point due to a firmware issue where the ports die on their own and don't recover, pricing is not ideal most of the time though. So may want to avoid if you're not feeling lucky or if you'd rather not need to contact Customer Support to RMA. Been out for a few years and more recent ones are coming out that don't appear to have the same issue as the one reported with this model. ~£240 odd.

TrendNET TEG-S750. This is a 5 port 10g unmanaged switch also, much like the TP Link SX-105 above. And unfortunately, it appears to have come from the same generation of design and hardware, as there are also reports of issues with ports failing and needing to be replaced via RMA. ~£220 odd. Also, support is iffy on this one, as whilst possible to obtain in the UK, it looks like mostly imports and thus might be an issue getting support.

QNAP QSW-2104-2T or the 2T-R2 (more recent) edition. 2x 10g ports and 4x 2.5g ports. Can probably service what you need for the near future. But given the price of this normally, you're probably better off looking at one of the 4x or more 10g port switches, especially if you're thinking of adding more 10g capable devices down the line soon. Also has some reports of it going awry, but having got the 1st edition one here, it's mostly finnikey quality cable connecting devices and temperature issues that's the cause of the issues. Was ~£160 odd but has gone EOL so prices have risen.

QNAP QSW-3205-5T. This is a 5x 10g port, fairly recent release and largely seen as a replacement for the 2014-2T above. The only thing is, it's an unmanaged switch still, so if you need any fancy features to control the network, this one won't be viable. But if you just need a core switch that can provide 10g to all node points, then this one can work. ~£250 odd.

Unifi Flex 10gbe. This is a 4x 10g port and 1x 1gb port, has been out for a bit now but still recent. I believe this is a managed switch so if you need those features this one might be for you. Can be powered via POE or AC adapter. ~£280 odd.

Mikrotik CRS304-4XG-IN. This like the Unifi Flex 10gbe above, is a 4x 10g port and 1x 1gb port switch. It has capabilities for management, but unsure on the level offered (as I haven't gone that far in yet with mine). Has a host of methods to power the switch. ~£160 odd. Funnily, with this pricing, you can actually grab two of these and nest them to get 8 ports of 10g, with 6 ports 10g for your devices or nodes, 1 for your router, and 2 (one on each switch) to connect to each other. You then also have the option to add the two 1gb ports to it as well. So can be quite flexible for use. But requires experience or knowledge of their RouterOS to get the most out of configuring them. The 10g ports come active straight away so no need to configure or anything, just plug and play.

Typically the 8 ports or more requires a fan in their cases (as seen and mentioned by @Quartz regarding their 8 port 10g switches), so have not included them here.
 
He might never need 10GbE but its fun to have. :D
At the point we're at gigabit FTTP, I DO think it's time the cheap home kit moved on a bit.

Doesn't have to be anything more than cheap RJ45 (no SFP/+ excesses need) that'll do more throughput. The 2.5 GbE trend seems a bit of a cop out imo.

Edit: That said, £35/40 for a 2.5GbE switch is totally fair enough I guess.
 
Last edited:
Back
Top Bottom