Help me choose a storage file system to use

AmateurExpert · 24 Aug 2017 at 13:04

Frank Jeagar said:
Hmm, are you suggesting reliable consumer-grade RAM or going all the way with ECC?

There are a couple of approaches you can take on this.

One is to do enough testing (e.g. memtest86+ for a few passes) with your set-up to know that the actual RAM you have is not faulty in the first place (at stock settings) and then testing at a range of overclocked settings to see at which point errors are seen, then backing away by the amount of safety margin you want. The limitation is that this does nothing about the risk of a fault developing at some point in the future and not knowing about it (at least, not without tedious repeat testing).

The other is to use ECC (and ensure your host is configured to report and alert hardware errors); you'll trade-off price and potential memory speed (compared to enthusiast-grade overclockable RAM) but ECC exists to address this concern.

AmateurExpert · 24 Aug 2017 at 13:21

rtho782 said:
ZFS doesn't really do tiering, although you can use an L2arc cache etc, this is not the same thing and doesn't bring the same benefits

Yup - L2ARC only directly benefits reads, but still useful if you don't have a lot of RAM to play with. On ZoL (ZFS on Linux) L2ARC isn't persistent yet, but it is (still) being worked on - would be nice to finally get this!

pc-guy · 24 Aug 2017 at 14:35

This is such an interesting thread. Started with question on file system, went down to the hardware choice for virtualization then went off regarding URE and back onto error rates and ECC ram and so on. It is got to be the most divergent discussion ever!

ECC ram for a Nas and pfsense? I don't think so. You don't need it. Also i processors don't support it. You need xeons.

FreeStream · 24 Aug 2017 at 17:46

pc-guy said:
This is such an interesting thread. Started with question on file system, went down to the hardware choice for virtualization then went off regarding URE and back onto error rates and ECC ram and so on. It is got to be the most divergent discussion ever!

ECC ram for a Nas and pfsense? I don't think so. You don't need it. Also i processors don't support it. You need xeons.

I disagree about ECC, system integrity is more important than cost and outright performance.

At the end of the day it's doing a job which it needs to do 24/7 without fault.

You could argue that modern-non ECC is just as reliable, but I'm not going to test that theory for the sake of a few quid.

skyripper · 24 Aug 2017 at 18:06

24/7 without fault? Its not a FTSE-100 business, its a home server, tucked away under the stairs holding backups, photos, movies, music and [possibly] intricate diagrams of hamster cages.

If the OP needed 365/24/7 uptime, with fault tolerance all the way down to the PSU level, multiple switch links plus aggregation, it wouldn't be being built out of hardware left at the back of a drawer, specced up with software by a bunch of randoms on a forum and without a support contract in sight to call on.

If the OP is really that worried about dataloss they should buy a HP or Dell server or a Synology NAS box (other makes are available), and institute a robust regular backup strategy. Or stick their data in the cloud and pay someone to take care of all the security patching, uptime, backups, maintenance windows etc.

Raid / ZFS / Mirroring etc. are all there to allow the server to limp along in the event of a problem. If a lightning strike hits the lamppost outside and kills your box, they won't help one bit.

PS, I am aware this is not a helpful post.

Praetorian · 24 Aug 2017 at 18:59

OpenMediaVault (OMV) is my recommendation - https://www.openmediavault.org/

Runs on Debian, so lightweight, and also has ZFS baked in. I'm running it virtualised with 4gb of RAM running on ESXi and it flies along. Admittedly it's only running 2x HGST 6TB drives but that's all I need.

FreeStream · 24 Aug 2017 at 19:28

skyripper said:
24/7 without fault

Completely OT, but yes, my servers have all had 100% (not including fiddling/upgrade time) uptime in 7+ years (across three setups).

EEC is also about data integrity, not just stability/uptime.

Point is: You get what you pay for and it is obviously personal preference.

skyripper · 24 Aug 2017 at 22:11

skyripper said:
24/7 without fault... Blah blah

I must have had a stick up my ass when I wrote that post!

CaptainCrash · 25 Aug 2017 at 01:01

pc-guy said:
ECC ram for a Nas and pfsense? I don't think so. You don't need it. Also i processors don't support it. You need xeons.

Errr... no.

https://ark.intel.com/Search/FeatureFilter?productType=processors&ECCMemory=true

Frank Jeagar · 4 Sep 2017 at 17:43

Fore some reason I didn't get these updates, which is really annoying because the discussion got great!

Frank Jeagar · 4 Sep 2017 at 17:46

As for RAM, I have absolutely no intention right now of going for ECC. It would involve a large change to the setup which is designed to be a cheap project that provides a network storage system, not a survive the apocalypse server haha

Frank Jeagar · 4 Sep 2017 at 17:49

Praetorian said:
OpenMediaVault (OMV) is my recommendation - https://www.openmediavault.org/

Runs on Debian, so lightweight, and also has ZFS baked in. I'm running it virtualised with 4gb of RAM running on ESXi and it flies along. Admittedly it's only running 2x HGST 6TB drives but that's all I need.

Now this is an interesting suggestion, thanks I'll definitely look into it!

But 4Gb of RAM for 12TB of storage? Are you personally using ZFS or something else?

GWANGY · 5 Sep 2017 at 09:07

Stick with windows , its free-ish , you understand it. As to data backup , maybe just keep two drives in a semi mirror eg two drives in one box, and you set a timed job to copy any new files from one drive to the other. At worst you lose a days stuff, your "backup" disk can be almost instantly brought into use and if you really want a further backup get a sub attached disk. It will cost you in disks, but you will understand exactly whats going on, and what to do in the event of crisis. You could even swap your disks around every six(?) months to even out the wear.
What windows really lacks is a data scrubbing program to read and check for ongoing bit errors , but those are not important in Photos or Video files.
Never underestimate the ease of pulling a disk out and slotting into your/mates/friends machine and it being fully readable.

Praetorian · 5 Sep 2017 at 19:40

Frank Jeagar said:
Now this is an interesting suggestion, thanks I'll definitely look into it!

But 4Gb of RAM for 12TB of storage? Are you personally using ZFS or something else?

I'm personally using ZFS in a mirrored setup for those two disks, so I only ever get 6tb but that's more than enough for my needs. I don't follow the 1gb per TB mantra for ZFS and I never use more than 75% of the available memory that the VM has, even when scrubbing the disks. Neither do I have anything like DEDUP, Encryption, Compression enabled etc.

I moved from a MDADM+JFS setup to ZFS and haven't looked back.

Frank Jeagar · 6 Sep 2017 at 14:37

Praetorian said:
I'm personally using ZFS in a mirrored setup for those two disks, so I only ever get 6tb but that's more than enough for my needs. I don't follow the 1gb per TB mantra for ZFS and I never use more than 75% of the available memory that the VM has, even when scrubbing the disks. Neither do I have anything like DEDUP, Encryption, Compression enabled etc.

I moved from a MDADM+JFS setup to ZFS and haven't looked back.

That's interesting, thanks for the input!