Help with windows smb file speeds

[qbhatti]

Hi

I have I think a fairly simple setup.

Router: opnsense on forcepoint 1100 with 16gb ram-->Sfp+ dac --> tp link 10g switch

from here I have another sfp+ to my connectx3 card in my main pc (5900x/32gb ram/b550 mobo/1080ti)
and the whole house is connected into this ( 2 more switches, poe, going into this )

I have 1 Netgear wax630 and one wax615(the outdoor one but using it inside as I got a good price)

I never get more then 100mbps on windows file transfers between my PCs and I don't understand why.


the main pc:
-happily downloads at full gigabit internet
-happily pushes Plex untouched streams around the house and remotely.

other pc: laptop
-in the attic, around 6m away from the wax630 which sits in an airing cupboard in the attic; windows says 1200mbps connection.
-can happily download from internet at 800mbps.


I have enabled jumbo frames on the main pc, I have used the mellanox optimiser and tried each of the settings.

I kinda had enough last night and managed to.stumble upon iperf on my router.
it gets max 4.3gbps from server to router.


What can be wrong.

 

[Avalon]

What you have said isn’t an SMB issue, it’s a network issue and slow SMB is merely a symptom of the actual issue.

My out of the box iperf3 numbers doing SFP+ to switch and then SFP+ router or another 10Gb SFP+ server are 7.1Gb/s, that’s 100% vanilla, no jumbo frames or tweaks applied, but non of them run windows. Obviously iperf isn’t using SMB in either case, so investigate PC to router, determine what’s going on between them first as either your tweaks are causing detrimental results or a driver/cable issue. Try a live version of Ubuntu to rule out win/driver issues? I don’t use CX3’s that often and never under Windows (Linux/BSD with x520 and Myricom are my weapons of choice), but those that do rate them and rarely seem to have issues.

Also if your other PC’s are connected via gigabit class connections to your 10Gb PC, then you’ll only ever get gigabit on them, also gigabit = 110MB/s ish on an efficient protocol/transfer (can’t comment on SMB, but they added multiple streams to try and improve it).

 

[qbhatti]

What you have said isn’t an SMB issue, it’s a network issue and slow SMB is merely a symptom of the actual issue.

My out of the box iperf3 numbers doing SFP+ to switch and then SFP+ router or another 10Gb SFP+ server are 7.1Gb/s, that’s 100% vanilla, no jumbo frames or tweaks applied, but non of them run windows. Obviously iperf isn’t using SMB in either case, so investigate PC to router, determine what’s going on between them first as either your tweaks are causing detrimental results or a driver/cable issue. Try a live version of Ubuntu to rule out win/driver issues? I don’t use CX3’s that often and never under Windows (Linux/BSD with x520 and Myricom are my weapons of choice), but those that do rate them and rarely seem to have issues.

Also if your other PC’s are connected via gigabit class connections to your 10Gb PC, then you’ll only ever get gigabit on them, also gigabit = 110MB/s ish on an efficient protocol/transfer (can’t comment on SMB, but they added multiple streams to try and improve it).

Thanks for the response.
I have already ordered a mellanox compatible dac...let's see if it makes a difference.
I'm only really looking to get gigabit anyways but I'm struggling with that

 

[Avalon]

I would have booted a live image and tested before buying anything. No point in swapping random parts till you know where the issue lies, that’s how you waste time/money. Narrow it down, then diagnose.

 

[ChrisD.]

You don't need jumbo frames.

Also please post the correct abbreviation, ie Mbps, MB/s. These details matter.

A network diagram would help.

 

[qbhatti]

You don't need jumbo frames.

Also please post the correct abbreviation, ie Mbps, MB/s. These details matter.

A network diagram would help.

Thank you.
Where do I get a network diagram from?

 

[ChrisD.]

Thank you.
Where do I get a network diagram from?

Draw one yourself.

 

[qbhatti]

Um, okay.

 

[stooeh]

Are you saying you only get ~10MB/sec file transfers over SMB?
I have 2xPCs with 2.5Gbit ports and a large file will transfer at maybe 283MB/sec - if its a folder with many snall files speeds will fluctuate wildly but not 10MB/sec bed

 

[qbhatti]

Are you saying you only get ~10MB/sec file transfers over SMB?
I have 2xPCs with 2.5Gbit ports and a large file will transfer at maybe 283MB/sec - if its a folder with many snall files speeds will fluctuate wildly but not 10MB/sec bed

Yes
That's what I mean.
Both PCs will download from internet at full 1Gigabit so I know their connections are fine and for some reason it looks like smb is limited itself to 100megabit

 

[WJA96]

Normally I’m not much of a ‘grammar nazi’ but the units are really important when we’re taking about file transfer speeds. I genuinely can’t make out if yiu have an issue of not.

What I would say is that the Forcepoint 1100 is not that quick a router with pfSense/OPNSense. It will not route much more than 3-4Gbps even if it is fitted with a 10Gbps network card. And if you start switching on fancy features it will slow down quite fast. OPNSense is not what it was designed to run and it will have some custom SoC for its original software so I’m not surprised that it could be a bottleneck. I doubt it’s the issue here and a very quick google shows lots of people coming to the conclusion that SMB is a very problematic protocol, particularly when it comes to speed. I assume you’ve checked that each SFP+ card is set to run at 10Gbps?

 

[ChrisD.]

Normally I’m not much of a ‘grammar nazi’ but the units are really important when we’re taking about file transfer speeds. I genuinely can’t make out if yiu have an issue of not.

OP won't even draw a diagram, so may as well put a finger in the air and guess what the issue is.

 

[Caged]

Pointless trying to get SMB performing until the link has been proven with iperf or similar. Reading the OP again it looks like one of the clients in this discussion is on Wi-Fi.

 

[Mercutio]

If any part of your link is gigabit, you will get a max of about 100MB/sec transfers in windows.

The one thing you CAN do to improve it is lay on more network paths between machines.
Each gigabit NIC you can add will put another 80-100MB on the file transfer. Windows SMB 3+ (so since vista) automatically uses any links it finds going to/from the same location.

You could have:
1GB motherboard
2x1GB pci card (I used to do this, still got em, not something I care about now).
~600Mbps network

at each end and windows would "magically" give you roughly 400-430MB/sec on file transfers.

While I wouldn't recommend going too crazy, there's USB3.x to gigabit network cables/devices. It'll help a bit if you don't have other, better options.

 

[bledd]

You say laptop and 1200mb connection. What type of connection is this?

Draw a network diagram, list the connection types.

Help us help you.