MTU / Jumbo packet problem...

[faxfan]

In an interested is squeezing as much "line" speed out of my network as possible I attempted to enable Jumbo packets on my network.

Elements involved -

Windows 10, Asus X99 deluxe motherboard with Intel 1218-v NIC
Cisco WS-C3560CX-12PC-S
ESXi 5.5, Broadcom NetXtreme NIC (E1000E driver)
Ubuntu 14.4 (under ESXi) SMB share

Baseline file transfer speed as tested 100MB/s

Steps -

Enabled Jumbo frames on Windows to via the intel interface (latest drivers) to 9088 (preset value)
Enabled MTU on the Cisco switch to 9000
Increased MTU on ESXi switch to 9000
Increased MTU on Ubuntu to 9000
(rebooted all elements)

Tested using MTUroute on windows to Ubuntu which confirmed I 9000 MTU, tested from Ubuntu to Windows ping <host> -M do -s 8972 this also confirmed 9000.

All good and well, right?

So baseline 100MB/s - new speed 85MB/s ?!?
Decreased the MTU on the Windows client to 4088 (preset value) - new speed 106MB/s.

So MTU is kinda working and the problem "appears" to be on windows or a compatibility issue between Windows and Ubuntu.

Anyone played with MTU before?

 

[faxfan]

I'm not disappointed at all, just playing around and seeing what is possible. All native read / write speed are more than fine.

I've just loaded a Windows 2012 server on ESXi and boom! 115MB/s - line speed (ok so 125MB/s is line speed but 115MB/s is probably the highest expectation). So it does like a Ubuntu / Windows client / SMB issue... hmmm

 

[faxfan]

This is all using SMB, pretty much proved it's a Linux / Linux SMB issue -

All elements set to 9000 MTU and proved with basic MTU testing.

SMB file transfer speeds from Windows 10 client to Linux SMB share -
1500 MTU (on Windows) = ~100MB/s
4000 MTU (on Windows) = ~106MB/s
9000 MTU (on Windows) = ~85MB/s

SMB file transfer speeds from Windows 10 client to Windows server SMB share -
1500 MTU (on Windows) = ~100MB/s
4000 MTU (on Windows) = ~106MB/s
9000 MTU (on Windows) = ~115MB/s <=== This is expected speed

Ftp maxes out at around 40MB/s

 

[faxfan]

all should be MB/s as per Windows shows on the file copy. Sustained read / write are well in excess of the copy speeds so that is not the bottleneck. Anyway pretty sure it's a Linux / Linux SMB problem as Windows to Windows gives the expected results.

 

[faxfan]

SMB is a problem full stop. A certain well publicised youtube reviewer lately has had problems with a huge file server that should be doing GB transfers and was only capable of about 300MB due to SMB.

NFS would probably work better.

NFS is my next stop on Linux (given that Windows to Windows is fine).