Home Server & Network Setup

[Conanius]

Hi Everyone.

Want to sort out my home server situation and would appreciate some advice.

Currently have:

- Seagate 4 Bay Business NAS with 4x4TB HDDs, RAID10
- Mac Mini 2011 i7 (4 Threads), 16GB RAM, 1x250GB SSD, 1x2TB HDD
- Time Capsule 3TB with AC WiFi

I'm about to go on one of the Cisco Meraki Courses where I'll be treated to some free hardware for my troubles (Firewall, Switch & Access Point), so my plan is to retire my Time Capsule, and review my entire setup.

At the same time, my NAS is woefully slow - at peak I get about 80Mbits from it - and it appears thats down to the cheap CPU within it, so, I want to build a server, and do something like:

- 1x VM for FreeNAS (to replace the existing NAS I have and provide my storage & Time Machine capability for my kit)

- 1x VM for Security Onion - want to have a play with this, looks like it benefits from plenty of threads/cores & RAM - https://security-onion-solutions.github.io/security-onion/ for those that are interested

- 1x VM for my Plex Library and media management(So I can take it off the Mac and just return that for day to day desktop use).

- Ability to spin up other VMs as needed for playing around (Wouldn't mind sorting out a VM to do things like DNS, Proxy & VPN that I can then feed into the Security VM too)... also if I can find time to play around with it, I may go back to using Smoothwall.

So... I need to build something that I can chuck the discs into, and have enough grunt to drive the VMs.

I was thinking about getting something like below for Hardware

- AMD FX-8320 CPU (16MB Cache, 8 Cores, seems to do OK in benchmarks against chips that are more expensive). As I'm looking at something that will be running various VMs, I am guessing more cores/threads the better, so the limitations of AMDs in single mitigated situations is minimised?
- Appropriate MoBo for above
- 32GB RAM
- Small SSD for the Host OS (Recommendations on a suitable host OS hugely appreciated)
- Hardware RAID Card for the 4x 4TB Discs I'll be taking out of the existing NAS. I'd look to run in RAID 5 to get some of the storage back ideally, but would consider other options if suitable. I presume the obvious choice is the HP PERC card that people seem do recommend I get off eBay ?

What are the obvious things I'm not considering that I need to be thinking about?

All help and criticism appreciated !

 

[El Pew]

- Hardware RAID Card for the 4x 4TB Discs I'll be taking out of the existing NAS. I'd look to run in RAID 5 to get some of the storage back ideally, but would consider other options if suitable. I presume the obvious choice is the HP PERC card that people seem do recommend I get off eBay ?

What are the obvious things I'm not considering that I need to be thinking about?

All help and criticism appreciated !

Why do you need a hardware RAID card if you're going to be running FreeNAS? ZFS prefers direct access to the disks so you may as well pass through the motherboard's disk controller to the FreeNAS VM rather than buying an additional component to do the same thing.

Either way, don't try and use hardware RAID5 in FreeNAS.

 

[Conanius]

Thanks for the reply. I did some reading on FreeNAS yesterday and didn't realise it didn't need a RAID controller card. Also looks like it really doesn't like being virtualised, so I may put that in a separate box then...

 

[ASE001]

Thanks for the reply. I did some reading on FreeNAS yesterday and didn't realise it didn't need a RAID controller card. Also looks like it really doesn't like being virtualised, so I may put that in a separate box then...

It does virtualise very well, but you will have to pass-through the hard drives directly to the VM. Only certain motherboards allow this feature, from my experience it is a little challenging to make it work! But once done you should expect similar disk I/O to a dedicated server. SMART works and all the other tools which need direct access to the physical drive. I know a AMD FM2 motherboard with a 'AMD® A88X' Chipset allows hdd pass-through and PCI card pass-through.

I then used ZFS to create a 4*2Tb RAID and 30Gbyte SSD as drive cache. If you are virtualising stay away from deduplication, it is CPU and memory intensive (it killed my server).

see this, but ignore the windows section:
http://fdo-workspace.blogspot.co.uk/2011/10/vmware-esxi-41u1-raw-sata-hdd-pass.html

 

[Frozennova]

Thanks for the reply. I did some reading on FreeNAS yesterday and didn't realise it didn't need a RAID controller card. Also looks like it really doesn't like being virtualised, so I may put that in a separate box then...

Done correctly there is nothing wrong with virtualising FreeNAS/ZFS

You need to ensure the guest OS has full and proper access to the disks

I use a Dell SAS 6/IR raid card flashed with the LSI IT (initiator Target) firmware which removes the RAID functionality of the card and turns it into a "dumb" controller card which is passed through to the FreeNAS guest with hardware passthrough in ESXI

Using AMD based hardware means you are limited to a handful of motherboards which fully support IOMMU (I'm using Aslockton 970 Pro3) the Extreme 3/4 both support this fully as well

In ~3 years I have a ZFS flag one file with a checksum error. Firstly it was a bad drive and secondly ZFS repaired the file

 

[El Pew]

I use a Dell SAS 6/IR raid card flashed with the LSI IT (initiator Target) firmware which removes the RAID functionality of the card and turns it into a "dumb" controller card which is passed through to the FreeNAS guest with hardware passthrough in ESXI

Slight thread hijack, but why do you use a card that powerful? What's the advantage to using one of those in IT mode rather than a cheapo non-RAID SATA controller or something?

 

[Frozennova]

Slight thread hijack, but why do you use a card that powerful? What's the advantage to using one of those in IT mode rather than a cheapo non-RAID SATA controller or something?

They're cheap and readily available (£15-20 on eBay)
They use an LSI chipset meaning they will play nicely with ESXI
They're reliable

However I would actually recommend a HP H700 or IBM M1015, They're more expensive but they overcome the biggest downfalls of the SAS 6/IR as they are 6Gbps controllers and support >2TB disks

 

[khrall]

Also bear in mind that the free meraki kit comes with a 3 year license.