Storage advice please: data/server/NAS?

alexkyriak

alexkyriak

alexkyriak

Associate
Joined
8 Jan 2011
Posts
369
Location
London, UK
Hi all, would love some advice please. I am pondering what sort of storage I need. I have a large amount of personal data (family photos, etc), some work backups, media files, and a very big Steam collection.

I am debating how to currently store all this. My currently solution is an external hard disk enclosure running in RAID1 which has 12tb. This is not networked.

I have some this data spread over several disks on my desktop PC. So, some of the data is ‘back up’ on the external disk, whilst some of the data on the external disk is only held there.

I feel like I need more storage capacity. Although I don’t think I need to spend hundreds of pounds storing Steam libraries on expensive disks, the flip side is I’d like this data to ideally be accessible via a network - media files, family photos, etc.

I’ll also be adding a second PC in the house and ideally I would want to have global access to all of this data.

Putting budgets aside for one sec, what is the best way to structure this and what is the best solution to accommodate this scenario.

My current thinking is to get a 4 or 5 bay network disk enclosure, and take out the two drives from my existing external disk and supplementing it with two more, to create two separate raid 1 disks (utilising the 4 disks). Although have also been intrigued by to so called efficient of RAID5 vs RAID1 in terms of use of disk space (RAID1 being least efficient).

Would love some advice to help determine the best solution! Much appreciated. :)
 
I guess my other question is: are Enterpise class disks not suitable for home use, or are they ok? Have read they're a lot louder than a 'consumer' level disk. But they seem to be a lot cheaper in terms of pound per tb.
 
For sheer convenience a NAS is hard to beat. I've been down the DIY route and had HP Micro servers and, while they are technically good solutions, they seemed to need more maintenance. But be aware that the first thing anything that's not running Windows will do is to format your HDDs so you must have your data backed up elsewhere before moving drives across.

Instead of creating two RAID1s I would add two drives the same size as the existing pair and use RAID 10. You'd need drives all the same size for RAID5 also but I really wouldn't fancy the build time and even more the rebuild time with 12TB drives.
I'd also go for NAS HDDs rather than consumer or enterprise so WD Red Plus or Seagate IronWolf. I currently have six 6TB IronWolfs (Wolves?) in RAID10 which works very well. Another alternative is to shuck the HDDs from WD Elements external drives. What come out are white-label drives with various designations but they all seem to work well in NAS. The disadvantage here is claiming under warranty for a faulty drive could be tricky...

Disclaimer: I have an Asustor 4-bay NAS for sale in this currently locked thread which I can unlock on request....
 
A few answers for you below:

alexkyriak said:
I’ll also be adding a second PC in the house and ideally I would want to have global access to all of this data.
Click to expand...
A NAS is the best option to be able access for more than 1 PC, and also opens up options for accessing your data over the internet when you are away from home.

alexkyriak said:
Hi all, would love some advice please. I am pondering what sort of storage I need. I have a large amount of personal data (family photos, etc), some work backups, media files, and a very big Steam collection.
Click to expand...
alexkyriak said:
I feel like I need more storage capacity. Although I don’t think I need to spend hundreds of pounds storing Steam libraries on expensive disks, the flip side is I’d like this data to ideally be accessible via a network - media files, family photos, etc.
Click to expand...
Likely you need to categorise your data before you can work out the best options in terms of how much storage you need, and how many different types of backups you need.

E.g. Family photos and the like are irreplaceable, but likely don't take up that much room - you should plan to store these in at least 3 different places (e.g. NAS for main access, cloud storage for regular offsite backups, USB drive for less regular cold backups).

Media files tend to take up lots of space, but can be replaced (although it's an inconvenience in terms of time to re-rip/re-organise), therefore you don't necessarily need backups, but some degree of high availablity e.g. RAID5 or Unraid parity protection helps protect these to a degree, but without much penalty in terms of storage capacity.

Finally, stuff like steam collections take up huge amounts of space, but are easily redownloaded (especially given faster internet speeds), so aren't particularly good value in terms of using NAS storage etc.


alexkyriak said:
I am debating how to currently store all this. My currently solution is an external hard disk enclosure running in RAID1 which has 12tb. This is not networked.

I have some this data spread over several disks on my desktop PC. So, some of the data is ‘back up’ on the external disk, whilst some of the data on the external disk is only held there.
Click to expand...
If data is only on the external disk, it isn't in any way protected. As above categorise your data and consider where to store each category.

alexkyriak said:
My current thinking is to get a 4 or 5 bay network disk enclosure, and take out the two drives from my existing external disk and supplementing it with two more, to create two separate raid 1 disks (utilising the 4 disks).
Click to expand...
There is no benefit to creating 2 separate raid1 volumes vs a 4 disk raid10 array. The available space is the same, and the risk of failure is almost identical. The disadvantage is that you end up with 2 separate volumes and so can't necessarily make as effective use of the same space (e.g. you might have a 12TB volume full of Media and have run out of room, but still have 90% free on your "personal" volume.)

alexkyriak said:
Although have also been intrigued by to so called efficient of RAID5 vs RAID1 in terms of use of disk space (RAID1 being least efficient).
Click to expand...

RAID5 is more space efficient than RAID1/10 given the same number of disks but the risk of data loss is greater.
- For a 4 Disk RAID10 array, your available capacity is half the total capacity, whereas a 4 Disk RAID5 is 75% of the total capacity.
- RAID10 can tolerate up to 2 disks failing, as long as they are not both part of the same mirror pair
- RAID5 can tolerate any single disk failing and your data is still available
- however the stress that RAID5 places upon the remaining disks following a failure, and the length of time that a Rebuild takes, can often cause an additional disk to fail before the rebuild is complete, resulting in total data loss.
- a RAID1/10 rebuild is significantly less stressful and takes less time as it's just a simple 1:1 copy of a single disk, resulting in a reduced risk window.
 
Guys, thanks so much for the awesome detailed responses, cheers. A lot to digest, and a lot of confirmation of some of my assumptions also. And more questions also..!

Snapshot said:
You'd need drives all the same size for RAID5 also but I really wouldn't fancy the build time and even more the rebuild time with 12TB drives.
Click to expand...
Apologies for not understanding, but what do you mean when you refer to 'rebuild' time? Assuming I just put the two existing disks I have already configured in RAID1, into a new RAID drive enclosure, does it recognise them as a pair automatically, or does the data need to be 'rebuilt' across the mirrored pair in the new enclosure?


Snapshot said:
Another alternative is to shuck the HDDs from WD Elements external drives.
Click to expand...
I was referring to exactly this when talking about potenitally reusing my existing RAID1 disks - they're currently in a WD Duo enclosure. I was hoping this might be a way to save some money on the overall quantum of storage, but noting the points on this thread re: 2 volumes on two 2-disk RAIDs, vs a 4 disk RAID5. I guess the issue if I went 4 disk RAID5 is that all disk need to be the same?

Snapshot said:
(Wolves?)
Click to expand...
Yes I believe so!

Armageus said:
E.g. Family photos and the like are irreplaceable, but likely don't take up that much room - you should plan to store these in at least 3 different places (e.g. NAS for main access, cloud storage for regular offsite backups, USB drive for less regular cold backups).
Click to expand...

Yes, currently I actually do have these in three locations (at least): on a few backup disks in the safe, on the external RAID1 disk I have, and on the disks in the PC itself. I probably need to sort out cloud storage, but need to look into this: who to go with, whether I already have cloud storage through google, norton, etc. My photos from iphone are also on icloud - but not all of my DSLR shots (obviously). However, I would like to be able to access these from other locations and devices, such as the big tv in the living room, or other networked devices, that sort of thing.


Armageus said:
The disadvantage is that you end up with 2 separate volumes and so can't necessarily make as effective use of the same space (e.g. you might have a 12TB volume full of Media and have run out of room, but still have 90% free on your "personal" volume.)
Click to expand...
This is quite an astute point actually, I know what you mean, I have encountered similar 'problem' to this in the past.


Armageus said:
If data is only on the external disk, it isn't in any way protected. As above categorise your data and consider where to store each category.
Click to expand...
I should have qualified my statement: this meant that only the Media files are held on the external disk, on the basis they're relatively easy to re-rip and not precious like family photos, etc.


Armageus said:
Finally, stuff like steam collections take up huge amounts of space, but are easily redownloaded (especially given faster internet speeds), so aren't particularly good value in terms of using NAS storage etc.
Click to expand...
True, I guess I have an old school mentality of "I must own the game - physically". Spot the contradictions running amok there... Yes they're redownloadable, I guess I shouldn't worry about Steam disappearing overnight in some sort of apolocalypse. Interesting reading online about people's approach to this, I've seen many having to ensure local copies are always retained. But given this is by far the biggest volume of data I'd be storing, I shouldn't let this drive the cost of the overall solution, doesn't make sense.

I also have a bunch of non-critical stuff that I like to keep a backup of at least: small projects, random information, etc. This I can manage easily without the full-on deployment of expensive storage.

In terms of planning for drive failures, I need to understand this a but more, but clearly my setup is not 'mission critical', it's more about maintaining a clear backup of anything valuable in case of any single failure, and being able to access information in a networked way. Discounting the Steam issue for a second. My work files are easily backed-up, I would not be relying on this storage solution to host my work stuff. Just need a reliable backup in case of failure, but it's not mission critical.

So, I was considering something like Asustor NIMBUSTOR 4 AS5304T 4 Bay NAS Enclosure, or Synology DS418 4 Bay Desktop NAS Enclosure. Or perhaps a 5 bay unit like TerraMaster F5-221 NAS 5, depending on how I setup the RAID.
 
alexkyriak said:
Guys, thanks so much for the awesome detailed responses, cheers. A lot to digest, and a lot of confirmation of some of my assumptions also. And more questions also..!

Apologies for not understanding, but what do you mean when you refer to 'rebuild' time? Assuming I just put the two existing disks I have already configured in RAID1, into a new RAID drive enclosure, does it recognise them as a pair automatically, or does the data need to be 'rebuilt' across the mirrored pair in the new enclosure?
Click to expand...
What I meant was what happens when a drive fails and a new one has to be added to the RAID. The time taken and the stress on the remaining drives during the rebuild process would both be high as Armageus says when discussing RAID5.

Remember what I said about the NAS reformatting any drives put into it? You will not retain your data if you move the drives to a different device.
alexkyriak said:
I was referring to exactly this when talking about potenitally reusing my existing RAID1 disks - they're currently in a WD Duo enclosure. I was hoping this might be a way to save some money on the overall quantum of storage, but noting the points on this thread re: 2 volumes on two 2-disk RAIDs, vs a 4 disk RAID5. I guess the issue if I went 4 disk RAID5 is that all disk need to be the same?
Click to expand...
Note that Armageus and I agree that RAID10 is the best way to utilise four large drives.
alexkyriak said:
So, I was considering something like Asustor NIMBUSTOR 4 AS5304T 4 Bay NAS Enclosure, or Synology DS418 4 Bay Desktop NAS Enclosure. Or perhaps a 5 bay unit like TerraMaster F5-221 NAS 5, depending on how I setup the RAID.
Click to expand...
You won't get any arguement from me about the Nimbustor 4 as it's the one I'm selling and that's only because I wanted more bays and a 10GbE port.
 
Can't unlock it so I'll create a new thread later this morning.
More bays means more drives working in parallel so more data going onto the 10Gb/s link. I see up to 4.5Gb/s when transferring large files so the 2.5Gb/s ports on the AS5304T were a limitation for me. Four drives in RAID10 saturate the 2.5Gb/s link so the AS5304T is well balanced.
 
5+ bays also gives you more flexibility in terms of having multiple volumes (e.g. a 2 disk RAID1 mirror for "irreplaceable data", and a 3 Disk RAID5 for "media") - although as mentioned above having separate volumes can bring it's own issues.

It also allows for the effective use of RAID6 (like RAID5 but with Double parity data, 2 drive capacity penalty), whereas with only 4 drives RAID6 is more complicated and no better than RAID10
 
One thing I forgot to mention. I've never actually run a 5400 speed disk ever in any pc I've owned (up until I started using SSDs back in 2016), but for NAS setup for media and data backup, is 5400 a big no-no, or is it tolerable? Cost differentials make it interesting on paper.
 
Sorry about the delay - it was under three months since I locked the thread so I wasn't allowed to create a new one and had to ask a moderator to unlock it. This is now done so https://forums.overclockers.co.uk/threads/asustor-as5304t-nas-with-optional-hdds.18954926/ is open for business again....

5400rpm drives work well in a NAS as there are several of them working in parallel. They're also generally quieter than 7200rpm drives. My AS6706T with six 5400rpm drives is almost silent except for a few seconds when the drives spin up.
 
The ST8000DM004 is intended for desktop use and is not optimised for NAS use. As I said much earlier, you need to look at Seagate IronWolf or WD Red Plus and the white label drives shucked from WD Elements external drives also work well in NAS.
 
Ok. Thanks snapshot. I don’t think I get what ‘optimised for nas’ means. As in, how is a disk of a certain speed and capacity differentiated between desktop and nas use (or any other for that matter). Other than it saying it’s a nas drive in the description, what are the differentiators? Thanks :)
 
The firmware is different between desktop and nas drives, in that desktop drives do their best to keep trying to recover data if an error is encountered, nas drives don't - they report the error immediately so that the Raid controller or software can decide how to deal with it.

Additionally NAS drives are normally slightly better made, in that they have slightly different components to cope with the additional vibration from multiple drives placed close to each other as used in a nas.
 
Armageus said:
The firmware is different between desktop and nas drives, in that desktop drives do their best to keep trying to recover data if an error is encountered, nas drives don't - they report the error immediately so that the Raid controller or software can decide how to deal with it.
Click to expand...
And the real use difference in that is that, if drive tries to re-read data for too long, RAID's control logic can think drive died and kicks it completely out from array.
Again in NAS/RAID friendly drives there's time limit for how long drive tries re-reading data before telling those blocks can't be read.
At that point RAID's control logic simply reads data from other drive, while giving some notification of error in one drive.
 
And a further differentiator is that many desktop drives now use shingled magnetic recording (SMR) rather than conventional magnetic recording (CMR) and SMR is not good in a NAS as it can cause delays. You can search here for threads on the subject if you're interested. Seagate does not use SMR in its NAS drives and WD uses it in some Red but not in Red Plus.
 
Thank you all for the insight. Really useful.

Considering RAID5 as the configuration: I have narrowed it down to two possible options; 5x8tb drives vs 4x10tb drives. The 8tb drive option achieves 32tb and is way cheaper than a 5x10tb solution, which would get 40tb capacity, and is still quite a bit cheaper than the 4x10tb option which would get 30tb.

So more smaller, and cheaper, drives gets a slightly better capacity. But it uses more drives.

The 8tb option is taking advantage of some good deals on that particular big name brand of NAS drive it has to be said.

Am I missing something here in terms of assessment of price to performance and considerations for redundancy, different number of disks, usage, etc?

Thanks :)
 
You must log in or register to reply here.
Back
Top Bottom