24 Bay 4U NAS/Home Server Build Advice

Iormangund · 25 May 2017 at 00:44

Heya,
I have decided to upgrade my current nas setup and try and shift my various different nas devices into a single unit. After a lot of searching for cases and solutions I have come up with a few different options and am having a hard time deciding which to go for. I was hoping some more experienced people with server/nas setups could have a look and give some advice. (prices are rough as not everything can be gotten on OCuk)

The case options:
24 bay 4U hotswap case with SAS 6Gb/s backplane (6x SAS ports) @ £300
24 bay 4U hotswap case with SAS 6Gb/s backplane and built in SAS expander (2x SAS ports) @ £700

Mobo, Ram & CPU options:
Supermicro X11SSH-CTF, 2x10Gbe and 8 port SAS onboard @ £450
Xeon E3 1240 v6 @ £270
Samsung 16GB DDR4 ECC Unbuffered (single stick) @ £160

Or I use my currently unused i7-2600k, MSI Z68 and 16GB DDR3 I already have @ Free

The next part I am a bit more unclear about what the best course of action is. The HBA/SAS side.
The supermicro board has 8-port SAS so if I get the case with the SAS expander afaik I would need nothing else (other than cables ofc).

If however I use my old sandybridge stuff, the case without SAS expander or go for something like a Supermicro X10SDV-2C-TLN2F, then I would need either 3x 8-port HBA, or a 24-port HBA.

Btw, the advantage to the case without the SAS expander (aside from cost) is that it is 10cm shorter.

For the HBA cards it seems I am looking at around this:
24-port (broadcom 9305 maybe) HBA @ £620
8-port HBA @200ish

Another big question is bottlenecks, with 24x sata3 drives will using, say, 6Gb/s SAS expander and an 8pport PCIE 2.0 x8 HBA give the same read/write performance as using 3x 8-port PCIE 3.0 x8 HBA and no expander, or a single 24-port PCIE 3.0 x8 HBA?

Is it worth looking at 12Gb/s SAS expanders and HBAs or will that make no difference with mechanical drives?

Ideally I am looking for a little future proofing, hence opting for the mobos with 10gbe if I am going to buy a new one, and I may well buy SAS drives in the future also however atmo all my drives are sata3, mostly nas drives.

In terms of software, that will be decided later but will most likely be one either unraid, freenas or xpenology, maybe esxi instead of baremetal for the last two.

Any help or advice would be greatly appreciated, I am a somewhat experienced system builder but have never dealt with SAS or server grade builds before.

Carbon Copied · 25 May 2017 at 07:38

Personally if it's for home non-critical use I would stick to buying clean server pulled hardware based around a new motherboard - is there a reason you need the E3 v6 as (without knowing your current spec), an E3 v3 should still serve well on say a feature rich Asrock Rack board. 10gbe on the board is all well and good, but unless you have hardware on the other side (switch etc. and traffic volume) thats actually going to be able to use it then it just seems a moot buying point personally (i.e a dual or quad integrated nic that supports teaming might be more than what you need now and down the line).

It might help to provide some more info on the requirements of the nas itself though.

Iormangund · 25 May 2017 at 09:00

Currently I have multiple computers for various tasks, my goal is to get rid of those and have just the home server and main desktop computer. The server will need to be able to deal with VMs of windows, mac and linux, many different docker containers, iscsi target for the desktop computer with well over 1gbit speed (ie for steam, video editing etc), will also be acting as a backup server and provide media (ie plex), plenty more things I will be using it, but that's the main points.
With all that I will be using it for, tbh I think an E3 v6 or xeon-d is probably the bare minimum.

At some point in the future I will probably invest in a 10gbe switch, but the initial plan is just a 10gbe card in the desktop and a direct connection to the home server.

In terms of buying used hardware, that depends somewhat on what configuration of SAS setup to go for, which was really the main thing I was having trouble deciding on.
If I'm going for used stuff, initially I might as well use my old sandybridge stuff, ie 3x 8-port hba or 1x 24-port hba and no expander or 1x 8-port hba and expander.
The choice of new mobo and cpu was largely to have something with sas and 10gbe onboard. (plus I do like having a warranty and reliability of new hardware).

Hope that's helped explain what I am trying to achieve

Iormangund · 25 May 2017 at 17:49

After some thought I came to a realisation, the i7-2600k doesn't have VT-d, so any passthrough to VMs is not going to work, therefore my best course of action is not bothering to use that. I still think the supermicro mobo and xeon E3 are probably the best bet unless I get lucky on ebay, so my main question now is:

Which is better (ignoring cost)?
3x 8-port HBA for 24 ports (no expander)
1x 24-port HBA (no expander)
1x 8-port HBA with case built in SAS expander

The SAS expander is 6Gb/s, will using all 6Gb/s HBA cards or using 12Gb/s HBA cards make any difference with 24 mechanical drives?

Edit:
So, I worked out that unless I am going for 3x 8-port cards, using 6Gb/s for either the cards or expander is going to cause bottlenecks. Finding a mobo with at least 3x pcie 3.0 8x is pretty difficult unless I go for a Xeon E5 or decide not to have a spare pcie slot for upgrades (ie 10gbe) in the future.

I have found a 24 bay case that has 12Gb/s expander built in, it is a bit more expensive however (but only because it comes with a dual redundant psu where the other cases I can just stick an atx psu in).
Think I am going to go for that one plus the X11SSH-CTF, 16gb ram and Xeon E3-1240v6, total should come out just under £2k.
Pretty sure this setup provides plenty future proofing for using much faster SAS ssds, running full 10gbe network and being able to do all the stuff I need the server for.

I have been searching ebay and really to do the same job I am not finding much that is cheap enough to be worth the risk or lack of warranties.

Edit (again, lol):
Changed my mind again. I have ordered a LSI 9210-8i HBA from ebay for about £60 and will use that with my old sandybridge kit and the 24 bay case with 12Gb/s expander built in. Decided that I can make do with the slower speeds from that for now and upgrade to a proper server board with xeon, onboard 10gbe and onboard SAS a little bit later.

At least this way I have future support ready for 12Gb/s setup and wont need to reinvest in new backplanes/case etc, while also giving me time to get used to SAS, better test what I further hardware I need and watch ebay for any xeon stuff that might crop up that fits my needs. Plus of course it saves me spending £2k in one go

Can always sell the HBA card if/when i get a mobo with onboard SAS.

For anyone interested the cost of the case with 12Gb/s expander and a dual redundant 800w PSU is about £1k, unfortunately the only 24 bay case on OCuk is only 6Gb/s and no expander so getting it from another site which I can link if someone wants, if that's even allowed.

Not sure if anyone except Carbon Copied has even read this thread, but even if no one else gives me any advice at least my thought process might help others.