NAS OS/CIFS (Samba) Performance

Soldato
Joined
26 Apr 2004
Posts
9,348
Location
Milton Keynes
Hi Guys,
Was wondering if anyone has done any 'side by side' testing to find the best performing SAMBA NAS OS/software setup?

I have an i3-2100 with 4GB RAM serving multiple NTFS HDDs round my home network, and now also my unified Steam folder, and I've noticed that SAMBA is definitely bottlenecking performance, more than I'd expected in some cases. (Most games work pretty seemlessly fine, but Borderlands: The PreSequel for example takes over 5 **minutes** to load over LAN rather than on the clients internal HDDs)

I've had no luck at getting NFS to the state it 'just works' on Win10 clients without a lot more configuration than I'm comfortable and ISCSI doesn't seem to fit the profile of what I want to do (multiple clients accessing the same files at the same time from the same partition) and seems a bit awkward to get running. I'm not a Linux/Networking guru so some of it may just have gone over my head!

So far, OpenElec is the easiest, it basically just works, and very easily, it also has good write performance, read speed is decent but not amazing, and was a leftover from when the server actually was a media machine.

NAS4Free 11 with ASIO and large read/write enabled has better read speeds (about 20-30% in some cases), but write speeds take a massive hit. ~100MBps -> 40MBps on sequential for example.

I'd seen that NAS4Free v9 had better SAMBA performance, but it doesn't seem to like one of my NTFS partitions, and also cannot seem to get it to just accept an anonymous CIFS connection from a windows machine or broadcast properly like V11 does.

As it stands, I've gone back to OpenELEC for simplicity, I did look at FreeNAS briefly but note it seems to want more than 4GB RAM.

Anyone found themselves in a similar predicament, tested out the various free NAS OS solutions etc and can share results and save me some time? :)
 
Last edited:
Associate
Joined
10 Jun 2014
Posts
227
If your game loads lots of small files, then it needs to be on a local device. iSCSI would mitigate this to some degree, but not completely. This would also mean it could only be connected to one device due to a lack of a cluster file system for Windows.
 

Deleted member 138126

D

Deleted member 138126

Gigabit Ethernet gives you about 100MBps (Mega Bytes per second), whereas a SATA 3 SSD gives you 500-550 MBps. That 5:1 reduction, plus the added latency if you're opening lots of small files (each request has to round-trip across the network), are unavoidable.

Using OpenElec is almost certainly not the answer. Can you not do Windows?
 
Soldato
OP
Joined
26 Apr 2004
Posts
9,348
Location
Milton Keynes
I am aware of the limitations of network storage, this system is more a media server that also hosts games. As such I am looking to get the best performance I can within what is actually possible. For what it's worth its pulling from mechanical drives not ssd, even if the drive (4TB X300 Toshiba, it'll do up to about 200MBps sequential) is faster than the GbE connection.
When you compare one OS/config which offers writes of 37MBps and then another offers 100MBps it is clear that simply choosing the most effective software package can improve throughput and likely improve latency.

For what it's worth it works surprisingly well most of the time, I'm just trying to optimise as much as possible:)
Trying to avoid windows installs for the NAS system and stick with something that can run off of USB.
 
Last edited:
Associate
Joined
20 Jun 2007
Posts
1,595
Location
Nottingham
parity systems will hit write speeds, some NAS implications don't play nice with windows.

do any of the systems you have looked into support SSD caching? I know unraid will write data to an SSD and then flush it to the array at 3.40am. The cache drive doesn't need to be an SSD it just delays the parity calculation, so gives you full 1Gbit network speeds.

My synology NAS (with raid 1) will transfer files both ways at 100MB/s all day long to any type of client.

Edit: maybe take a look at Xpenology (which is the open source version of the synology OS).
 
Last edited:
Soldato
OP
Joined
26 Apr 2004
Posts
9,348
Location
Milton Keynes
Thanks, will take a look :)

Last night I was checking out OpenMediaVault. Seems like it might be my winner so far.

Aside from Borderlands: The Pre-Sequel, which seems to have an ungodly level of small files (seriously, it's rediculous, 20K-30K files in the folder+, which is probably why it's results over network are so out there in comparison to other sofware I've tried which has performed fine, and not much worse than locally!), I've been using CrystalDiskMark and 100MB test samples, which gives the following stats:

Seq Q32T1: Sequential (Block Size=128KiB) Read/Write with multi Queues & Threads
4K Q32T1: Random 4KiB Read/Write with multi Queues & Threads
Seq: Sequential (Block Size=1MiB) Read/Write with single Thread
4K: Random 4KiB Read Write with single Queue & Thread

OMV is somewhere in between OpenElec and Nas4Free on 4K out the box, but absolutely pastes them on the Sequential and Threaded results. It's about 20% faster than Nas4Free in places, which was already faster than OpenELEC, seeing over 100MBps so with threaded/sequential. It's pushing right up to the absolute level GbE can achieve, which is great and perfect for A) serving media to multiple clients at once seamlessly B) pushing game loads out as quickly as possible.

This evening if I get a chance, I'll upload some pictures so you can see the margin of difference I'm seeing. If I can find a way to raise the 4K performance without too much detriment elsewhere, it'll be the clear winner.

I'd also arguably say in many ways OMV seems easier to configure than Nas4Free, as it does thinks like pickup your HDDs, and once you've mounted, your partitions, automatically, so you can chose what to set up shares for more quickly. Just seems a bit more intuitive.

Nas4Free is a little unnecessarily fiddly if you want to do something simple, as you have to manually mount all your partitions etc. Unnecessary steps for something as simple as a home/small office server.

Only area where I'd say N4F was a little easier, was it at least gives you some very basic text GUI on the server to configure the initial IP/reset server passwords etc on boot, whereas OMV essentially leaves you at a linux CLI, so had to take a quick look at my router to see what IP it had shown up on to connect to the web interface.
 
Last edited:
Associate
Joined
16 Apr 2007
Posts
2,192
My 2p is, openmediavault is the worst OS i've ever used. Nothing is simple. Would never go back to using it. Ever.

Ive been using OMV for a few years but recently started moving everything off that to another VM.
OMV gave me no issues and the plugin system and webui seemed ok but I want to move to a more container (LXD) based system which I have moved all my services to now nearly.
The only thing that OMV does now is serve up the HDDs over NFS/SMB.

What did you not think was simple out of curiosity?
I think ill move the disk sharing to another box at some point but not sure what yet.
 
Soldato
OP
Joined
26 Apr 2004
Posts
9,348
Location
Milton Keynes
I'm testing using the newest 3.0.58 beta version so can't speak for the older versions, but the newest one at least seems fairly user friendly as long as you have some IT experience, don't need to have a huge amount of Linux/Networking experience. Only bugbear is it likes shares to have no space in the name, so have to change the name to say MediaDrive rather than Media Drive as I'd used previously.

OMV v3 reads all the partitions, partition/drive labels, displays logically etc, none of the faffing around I had with N4F.
 
Associate
Joined
7 Aug 2012
Posts
948
What did you not think was simple out of curiosity?
I think ill move the disk sharing to another box at some point but not sure what yet.


The simple task of trying to move some directories from one location another. We had the shares setup OK, however just trying to preform that task was x10000000 more difficult than it should have been. This was using the web-ui.

In the end we decided to just transfer files via the network shares Out of the server > through the router > back to the server.
 
Soldato
OP
Joined
26 Apr 2004
Posts
9,348
Location
Milton Keynes
As promised here are the current stats. If not for the 4K, OMV is clearly in the lead so far.

Nas4Free v11.0.04 rev 3460
NAS4Freev11_zpsxw7np5ox.png

Borderlands load to exit – 5mins 02 seconds

OpenELEC 7
OpenElec701_zpscflvvsu1.png

Borderlands load to exit – 8 mins 36 seconds

OMV 3.0.58
OMV3058_zpspihnwiau.png

Borderlands load to exit – 6 mins 22
 
Associate
Joined
19 Jul 2011
Posts
2,343
You're using a NAS to have one installation of game files served out to multiple locations. I can see the benefit of this (only one installation to keep up-to-date, can move from one box to another without waiting for the second to have an install done), but you're always going to run into the Client - LAN - NAS bottlenecks you're seeing.

Is it worth maintaining locally cached copies of the LAN data on the clients (assuming you have the local disk space), with a robocopy or other syncing script which loads the latest versions of files before launching the game? So you update in one place, and clients take those updates on launch?

My own NAS is NAS4FREE as I found the interface clearer than FreeNAS. But then I run it on virtualised disks inside VMWare ESX, so I'm clearly not bothered by performance :)
 
Soldato
OP
Joined
26 Apr 2004
Posts
9,348
Location
Milton Keynes
Primarily the box is a media server, but I'm all for unification to cut down necessary footprint in the base units. My wife is very clear she wants our next machines to be smaller, so knocking even a few HDDs out and moving to the NAS will save a lot of hassle, and leave more room for the rest of the cooling and GPUs so it's all in tandem :)

I'm always expecting a performance hit, I'm just trying to optimise my OS/settings to get the best balance. OMV is very damn close as you can see above, I just want to see what I can do to tweak to improve the small file performance as much as possible within the hardware/OS/LAN limitations :) (or use the best alternative, as can be seen above, there are pretty significant differences in network performance even across distros all using Samba!).

Best trade off as it were :) Most games seem to work really really well with the setup to be honest, being more optimised for HDD and sequential loads rather than SSD/small loads, just a slight trade off on load times!

I'm only using Borderlands Pre-Sequel for testing as its the worst example I can find out of the literally thousands of games me and the wife have between us; nothing else I've seen acts/loads quite like it, even Skyrim/Fallout etc etc

The likelihood is I will run local copies for the outlier Steam Games like this, with the rest centralised, but just trying to get the best I can before I 'let it be' as it were, and that way I get the optimal physical footprint. Much easier to store the odd game locally, and keep the multi-TBs centrally, with no unnecessary duplication :) Steam's updated multiple game folders options make this much easier than it used to be too.
 
Last edited:
Back
Top Bottom