4TB backup solution - recommendations on how please!

Absolutely cracking replies! Please don't apologise for posting so much, it's exactly what I need to know, I need to research as much as possible before investing, your posts are fantastic and much appreciated! :D

There has been quite a lot said since my last post so I'll take em one post at a time starting with post 14 from JonJ678.

From what you've said I think the hardware will be fairly simplistic so I will return to that in later posts (think I've said that once), I still need to be absolutely clear on the setting up of it I think. Once I've got that sorted the rest should be easy enough hopefully.

Multiple HDDs and failure rates
Really interesting point about running 9 HDDs from the same make! Say I buy 5 Samsung F1s and 4 Western Digitals...would they all work in exactly the same way without any problems if they perform at slightly different rates?
I'm still a little unsure how it would be setup exactly and that relates to HDDs.

Raid Arrangement
OK, it's a RAID 5 system I'm sorted on...either 4 + 1 disks, or 8 + 1 disks.
Would it be seen by the OS (whichever I pick) as one big disk that I could partition into separate 2TB drives? Or can I pick which drives I make into drives? Just thinking about using HDDs from different manufacturers, if I could select WD as one drive, and Samsung F1s as another drive it might negate any possible issues.

Distinguishing hardware and software
I'm still a little hazy on what's hardware and what's software...I was under the impression (no doubt wrongly!) that adding in a controller card was hardware, and using the motherboard was a software RAID...unless it's a dedicated chip on the motherboard that handles the RAID, obviously it would be hardware then. Either way I would be knackered if it conked out so I think software RAID would probably be the way to go. Speed isn't an issue, getting my data back after a catastrophic failure is! I have absolutely no knowledge of software RAID though...from what you are saying JonJ is a software RAID handled by the OS? So Ubuntu or Freenas would create and manage the RAID, then nothing can fail because except for HDDs failing there is nothing to go wrong that can't be fixed with a reinstall of the software that would recognise the RAID straight away?
(Sorry, you've probably answered that already but I'm being a bit slow with it...a simple yes confirmation will do lol).

OS
Windows/Home Server is out then...from what you have written I think I'm leaning towards Ubuntu.
Is it harder to setup than Freenas then?
Would Ubuntu allow NTFS formatting and show 4TB disks as well?

Look no hands!...or monitor keyboard or mouse
Controlling it remotely sounds absolutely brilliant...having this virtually silent box running in the corner, without a spaghetti junction of wires behind it and no extra space taken up sounds fantastic!!! Is that something that is just done with Freenas or can Ubuntu manage that as well?

Budget
I haven't set one yet (always a bad sign), but of course, spending as little as possible is always preferable...the system itself sounds like it will be cheap but I think I'll be forking out for a case at least and a decent PSU...I was thinking of the Antec 1200 case, I use x5 Antec 900s at the office and they are whisper quiet and keep things chilly...any reason for using the P182 over the Antec 1200?


Reply to Evil-I
A hot spare sounds like a good idea, is that easy to implement?
sldsmkd mentions a cold spare...is that literally an unplugged HDD ready to swap out for minimum downtime?

I really like the idea of a UPS, considering it will be housing all the critical info, doesn't sound cheap but I'll look into it!

Just to confirm I am reading what you and JonJ are saying correctly...you use a 160GB, and JonJ uses a pendrive as your OS yes? And that's a different drive to the RAID...the RAID is separate so you can screw up the OS and it won't affect the RAID?

Ch3m1c4L, thanks for confirming that XP can see greater than 2TB as long as it's not booting off it, that's good to know!

Thank you again one and all for your posts, your information is absolutely invaluable and I'm very grateful that you take the time to post it! :D
 
Last edited:
Hello again :)

A 9 hard drive raid is a lot of data storage. The issue is more that if one drive has an X% chance of failing, 9 of them have a 9X% chance of failing. So a 5 drive system in inherently more stable than a 9 drive system, this is where raid 6 comes in (raid 5 but that can take two drives failing, and costs two drives data capacity). Hot spares are brilliant for avoiding downtime, it's very easy to set up in ubuntu at least. You type --hotspares=1 partway through one of the commands.

Hardware is a card, pretty much. Software is either the motherboard, which is hidden from the os but uses the normal processor and has the flaws of hardware without the benefits. Or operating system based, software called mdadm in the case of linux, which indeed you set up entirely within the operating system. I think its easier in ubuntu, but then Ive never used freenas. Ubuntu at least has had full ntfs support for ages. I have a suspicion that you could format it as ext3 and the windows machines would be none the wiser, anyone able to confirm?

Controlling it remotely is easily done. If you want a graphical interface, software called no-machine, nxclient or similar does it very well. If just to shut down, you can ssh (secure shell) into it from a windows machine if you like, it gives you a working command line exactly as if you were sitting at it. Ubuntu can definitely do this.

Reason for the P182, it's quiet and looks like it'll neatly hold 6 hard drives. No particularly good reason beyond that, I just like the look of it. I thought it was a m-atx case, but it seems I was wrong in this. I intend to use the mini 180 for my next build, but it doesn't really offer any advantages over being little.


OS on a different drive is well worth doing, I've got a normal install of ubuntu on my pen drive because I couldn't fit another hard drive in my case. Works well
 
Multiple HDDs and failure rates
Really interesting point about running 9 HDDs from the same make! Say I buy 5 Samsung F1s and 4 Western Digitals...would they all work in exactly the same way without any problems if they perform at slightly different rates?
I'm still a little unsure how it would be setup exactly and that relates to HDDs.

**Good hardware raid will usually happily mix drives, it will usually use the size of the lowest common denominator. To be honest if you have the choice though, stick to the same make and model if possible.**

Raid Arrangement
OK, it's a RAID 5 system I'm sorted on...either 4 + 1 disks, or 8 + 1 disks.
Would it be seen by the OS (whichever I pick) as one big disk that I could partition into separate 2TB drives? Or can I pick which drives I make into drives? Just thinking about using HDDs from different manufacturers, if I could select WD as one drive, and Samsung F1s as another drive it might negate any possible issues.

**Depending on your RAID hardware you can set up as many seperate volumes at the RAID bios level, For instance, I'd typically set up a a small Raid5 500Gb volume that is bootable then create a second Raid5 volume with the remainder of the space, both these volumes would be spread over the entirety of the drives on the array. Not entirely sure on the over 2Tb limit. In work we just carved it into 2Tb partitions, although I think there are ways of having larger partitions, just not sure how to be honest, have a vague recollection of someone mentioning using iScsi targets or something?.**

Distinguishing hardware and software
I'm still a little hazy on what's hardware and what's software...I was under the impression (no doubt wrongly!) that adding in a controller card was hardware, and using the motherboard was a software RAID...unless it's a dedicated chip on the motherboard that handles the RAID, obviously it would be hardware then. Either way I would be knackered if it conked out so I think software RAID would probably be the way to go. Speed isn't an issue, getting my data back after a catastrophic failure is! I have absolutely no knowledge of software RAID though...from what you are saying JonJ is a software RAID handled by the OS? So Ubuntu or Freenas would create and manage the RAID, then nothing can fail because except for HDDs failing there is nothing to go wrong that can't be fixed with a reinstall of the software that would recognise the RAID straight away?
(Sorry, you've probably answered that already but I'm being a bit slow with it...a simple yes confirmation will do lol).

**Most on board motherboard hardware RAID chipsets are actually what is called Bios assisted software raid. I had loads of problems with these chipsets and linux, as basically linux tended to see the individual drives and not the RAID array you'd set up in the raid bios.

A RAID volume from a proper hardware raid card is just seen by the OS as a device in itself and therefoire is very easy to deal with in whatever OS.

To be honest I still say proper hardware raid is the way to go. Buy Amcc/3Ware, you will always be able to get a replacement card if yours fails, as its one of the more common server raid cards these days and as such replacements are easy to find. **

OS
Windows/Home Server is out then...from what you have written I think I'm leaning towards Ubuntu.
Is it harder to setup than Freenas then?
Would Ubuntu allow NTFS formatting and show 4TB disks as well?

**NTFS not an issue in Linux anymore, but to be honest why use NTFS? Use a linux filesystem such as EXT3, EXT4 or ReiserFS, they perform much better than NTFS and seeing as I gather the arrays just going to be a share on the network, NTFS or EXT4 is irrelevent.**

Look no hands!...or monitor keyboard or mouse
Controlling it remotely sounds absolutely brilliant...having this virtually silent box running in the corner, without a spaghetti junction of wires behind it and no extra space taken up sounds fantastic!!! Is that something that is just done with Freenas or can Ubuntu manage that as well?

**All Linux can if you have the time to sort. I use a mixture of SSH for terminal access and VNC to log in in a visual environment.**

Budget
I haven't set one yet (always a bad sign), but of course, spending as little as possible is always preferable...the system itself sounds like it will be cheap but I think I'll be forking out for a case at least and a decent PSU...I was thinking of the Antec 1200 case, I use x5 Antec 900s at the office and they are whisper quiet and keep things chilly...any reason for using the P182 over the Antec 1200?


Reply to Evil-I
A hot spare sounds like a good idea, is that easy to implement?
sldsmkd mentions a cold spare...is that literally an unplugged HDD ready to swap out for minimum downtime?

**Hot spare easy on proper RAID hardware, just asign the drive as hot spare. Yes cold spare is just having an appropriate drive you can quickly switch out*

I really like the idea of a UPS, considering it will be housing all the critical info, doesn't sound cheap but I'll look into it!

Just to confirm I am reading what you and JonJ are saying correctly...you use a 160GB, and JonJ uses a pendrive as your OS yes? And that's a different drive to the RAID...the RAID is separate so you can screw up the OS and it won't affect the RAID?

**Yup, most important thing in my opinion**

Ch3m1c4L, thanks for confirming that XP can see greater than 2TB as long as it's not booting off it, that's good to know!

Thank you again one and all for your posts, your information is absolutely invaluable and I'm very grateful that you take the time to post it! :D[/QUOTE]
 
Freenas would probably be ideal for you.

My backup box at home -

Freenas ( easy to set up, basically just set up networking and point a web browser to it)
AMD Athlon 64 X2 4850e 2.5GHz (Fairly low power usage )
GA-MA74GM-S2H
Old ide drive to install freenas onto
Western Digital Green drives
Software raid 5 ( Not had any problems with it but would prob go hardware with a proper raid card if it was in a working environment )

Only issue was freenas didnt pick up the onboard lan so had to add in a network card, prob be fixed in future releases.

Freenas manual - http://www.freenas.org/downloads/docs/user-docs/FreeNAS-SUG.pdf

Freenas Tour -

 
If it wasn't for the fact that my home server is used for other things like hosting teamspeak, torrents and a few other things I'd be tempted by freenas. Looks really good!

E-I
 
Thanks again for the replies guys, I really think I'm beginning to piece together the puzzle now! :)

Quick question for Evil-I though...you mention why use NTFS...would Windows be able to see a linux filesystem? How is it better?


I'm still undecided over Freenas or Ubuntu (many thanks Chaos for that link and video!)...I like how simple Freenas sounds but not knowing how much I might want to fiddle or change things at a later date I may find it a bit limiting in time...decisions decisions!

I think I should start looking at hardware now...

What is the best way of housing 10 SATA drives in one PC? Most motherboards come with six SATA slots at most, is it just a case of using add in cards?

This is my thoughts on the build, I would welcome any comments:


Case
Antec 1200 case: £137.99


HDDs
x5 Samsung F1 1TB drives (maybe more if I want an 8TB RAID, not sure about splitting brands) £64.99x5 = £324.95

Motherboard
Gigabyte GA-MA770-UD3 AMD 770 - £60.98


CPU
AMD Athlon 64 X2 Dual Core 7850+ Black Edition 2.80GHz - £49.99


CPU silent fan
Arctic Cooling Freezer 64 Pro PWM CPU Cooler (Socket 754/939/940/AM2) - £17.24


RAM
Corsair XMS2 DHX 2GB (2x1GB) DDR2 PC2-6400C4 TwinX Dual Channel (TWIN2X2048-6400C4DHX) - £24.99


PSU
Tagan PipeRock TG600-BZ 600w Modular £101.99

Additional SATA Ports
4 port controller - £77.04


I have a spare graphics card I can harvest should it be needed on the initial setup.


This is a pic of it all, what do you think?
24w91zc.jpg
 
If using hardware raid, match the drives perfectly. If software, it wont care at all, and a mix of wd and samsung is the better choice. Imagine if you'd built the entire thing on 7200.11 hitachis a few years back, only to have the entire lot die on you. Don't put all your eggs in one basket.

I'm pretty sure you can use ext3 or similar. Windows doesnt write to the array, it requests data from and sends data to the server, so as long as the server can read the array you're good. As for benefits of ext3 over ntfs, I'm not even going to start. The linux one is conclusively better.

Id probably get an matx board, onboard video is a useful thing to have. The psu is a little mental for the purpose, worried about it failing I guess? Not sure modular matters here.

Can get cheaper 4 port sata cards than that, cant name any off the top of my head though.

Next step is to download and burn an ubuntu livecd, then boot from it and have a play. After an hour or so you'll know if you want to use ubuntu or freenas :)

edit: it wont be hard to fit 10 drives in a case, since there are many brackets out there for fitting hard drives into 5.25" bays. My akasa would comfortably hold 11 and a dvd drive were I so inclined
 
Last edited:
if you have trouble finding a chassis that will host ten drives, get a smaller case, a two port eSATA card, and an eight bay DAS SATA drive cage. Hook it up using a pair of eSATA cables, set up disks as if they were in the machine. Google for Edgestore and DAS801 which I think might be the sort of thing you would find interesting.
I prefer OpenFiler for my open sauce file server needs, but both are good programs - test em in VirtualBox or VMWare if you like, they both work in it :-)
 
TheBigCheese said:
Quick question for Evil-I though...you mention why use NTFS...would Windows be able to see a linux filesystem? How is it better?
Click to expand...

If your file server is running linux, you're better off using a linux file system such as EXT3, EXT4 or ReiserFS. Don't ask me to explain why as I'm not sure I understand it myself, but the linux file systems are quicker and more fault tolerant than good old NTFS, something to do with them being a true journaled filesystem or something?

As far as any of your windows machines on the network are concerned they don't care what file system the shared drive is as the linux windows networking component Samba deals with all that.

From personal experience my linux server uses the EXT3 file system, which seems to self defragment, and generally look after it own health very well without any intervention from the user. And as far as my windows machines are concerned they just see a share that they can read from very happily.

E-I
 
JonJ678 said:
If using hardware raid, match the drives perfectly. If software, it wont care at all, and a mix of wd and samsung is the better choice. Imagine if you'd built the entire thing on 7200.11 hitachis a few years back, only to have the entire lot die on you. Don't put all your eggs in one basket.
Click to expand...

I'd stick with all Samsung to be honest. They've proven to be one of the most reliable drives I've ever used. And the likelihood of multiple drives failing at the same time is very slim, so as long as you are running RAID5 with a running hot spare you'll be fine. Or if you're really paranoid run RAID6 with a running hot spare (not sure if you can do this in software raid?).

If you're really paranoind change the normal Samsung for thier enterpise range, or the seagate equivelents, but expect to pay nearly 3 times the price of the standard drives for the extra reliability.

I'd alos check really carefully that the adaptec card is well supported by whatever OS you intend to use.

E-I
 
StevenRaith said:
if you have trouble finding a chassis that will host ten drives, get a smaller case, a two port eSATA card, and an eight bay DAS SATA drive cage. Hook it up using a pair of eSATA cables, set up disks as if they were in the machine. Google for Edgestore and DAS801 which I think might be the sort of thing you would find interesting.
I prefer OpenFiler for my open sauce file server needs, but both are good programs - test em in VirtualBox or VMWare if you like, they both work in it :-)
Click to expand...

I use an old coolermaster stacker case for mine with 2 x 120mm fanned drive cages that each hold 4 drives. theres still room for another 4 drive cage if I need to add more.

It also helps that the case is so big as I got carried away and bunged a dual xeon motherboard and 2 x quad core processors in it. Serious overkill thats probably costing me fortune in electricity running 24/7. Ah well... boys and toys....
 
No price or performance hit for mixing samsung with wd, both brands are very respected. Eggs in one basket again. Hot spares in software raid are no problem at all, could do raid 10 with hot spares if you were so inclined. Absolutely any configuration is possible, you strap some drives together into /dev/md0, others into the next in the series, and can raid the /dev/md0 devices. Linux thinks everything is a file. So raid 30 striped is fair game if you wished.

Good advice on checking that its supported by the operating system, hardware support is something linux doesn't do that well at. In fairness, they're generally given no help from hardware manufacturers and have to write the drivers from scratch. That introduces a delay relative to windows being handed them already written.

Evil-I said:
It also helps that the case is so big as I got carried away and bunged a dual xeon motherboard and 2 x quad core processors in it
Click to expand...
I'm very jealous. I'd very much like a dual socket system
 
I use a Coolermaster stacker case at home as my games machine, I have always found it great but a bit noisy compared to the Antec 900s I have now, can barely hear them working.

Cases
So I think I'll go for the Antec 1200 case, sounds like it will do all I need to do, it only has space for 9 3.5" drives though, but I think I'm right in thinking that I can get an adaptor that will convert a 5.25" bay into a space where I can install several more HDDs, a bit like a 4 in 3 module for the Coolermaster stacker...I can't seem them on OCUKs website though :( I'll have to get some longer SATA cables as well!

RAID type
I think RAID 6 will definitely be the way to go, just reading up on Wiki I came across this when describing it:
According to the Storage Networking Industry Association (SNIA), the definition of RAID 6 is: "Any form of RAID that can continue to execute read and write requests to all of a RAID array's virtual disks in the presence of any two concurrent disk failures
Click to expand...

That sounds pretty good to me, especially considering the amount of disks that will be involved.

I think the 8TB system might be the way to go...so that will require 10 disks in RAID 6 yes?

I have several 1TB Samsung F1 drives already so I'll buy some WD drives to go with them.

And I'll set them up with a Linux filing system...you guys do realise that I am going to have a thousand and one questions about the software when I get it built don't you??? lol


Software question
Can both Ubuntu and Freenas be run from pendrives rather than CDs? If I am filling the system up with HDDs I won't have any room for a CD drive and would like to keep it out of the system if possible, I'm sure flash memory is quicker for one thing!


Compatibility issues
Is there anyway of finding out which extra controller cards will work with Ubuntu/Freenas? I don't want to buy it all and put it together just to hit a hurdle at the last stage.
 
One thing to remember is that with RAID you're not just worried about disk failure, but as storage capacity within an array increases you're also worried about the uncorrectable bit error rates (or chance of a bit-flip happening!)

In a RAID-5 array should one of your disks fail and you replace it you start rebuilding the array by recalculating all of the checksums etc. However if you've had a bit flip (or unreadable bit error) on one of the drives then the rebuild will fail potentially loosing you the entire array. As a RAID-5 array approaches 10tb it's actually "probable" for it to experience this during a rebuild - very scary with all that data!
RAID-6 goes some way to helping with this problem and it's why I'd always suggest it for multiple-tb arrays as you now have two parity sets to compute against.

I'd also suggest that a decent hardware RAID card is important as you can then run background media checks, having the controller constantly the checksums and correcting them if they're wrong. It'll also handle bit errors/drive errors better - marking bad sectors rather then ignoring them for instance.

If you're up for a bit of mucking about then my current preferred solution is actually an OpenSolaris box running ZFS. Entirely software based but I've been very very impressed with the ZFS software raid-5 and raid-6. Actually I've been so impressed that we're currently rolling out Sun storage systems that don't have hardware raid cards - it's entirely based on OpenSolaris and ZFS (Sun Open Storage S7000 system) and it's delivering performance that's better then the high end hardware raid card systems we run.

If you prefer something nice and easy then FreeNAS and OpenFiler are both good easy to use NAS OS's that should give you more features then you could ever want :)
 
Thanks York for the words of caution...awwwwww...makes me wonder whether I should bother at all!

My initial (and niave) view was that if I cobble a RAID together then the data is double backed up and there's no problem...seems like unless I mirror the drives there is no guarantee that if one drive goes down I will be able to get the data back...:(

I'm a bit flummoxed with it all I think.
 
Last edited:
Don't get me wrong - RAID is still very good. You just have to understand all the pitfuls so that you can choose a solution that suits you :)

If you have a decent HW raid card, running RAID-6 and your raid card is doing background checks of your parity data every week then you should be fairly well off.

The other thing you can do to try and improve you chances is to run the higher end SATA drives. Standard SATA drives tend to have a unrecoverable bit error (UBE) every 10^13 bits. A 1tb SATA drive has 10^12 bits, and so 10 of those has 10^13 bits - hence the high likelyhood of getting a UBE on recovery.
However the higherend SATA drives (like the Seagate ES.2 series of drives) have a UBE of 10^14, meaning you'd need 100 sata drives in an array before you'll hopefully get a UBE!

I hope that helps :)
 
I have just built a media server, but it's the same principal.

I'm using a "Gamer / Server PC Case with 250mm Fan" (google) which has 9x 5.25" bays. I've modded mine for silence, an removed the side panel fan, filled the hole with a sheet of metal, filled all the gaps in the side panels and added some carpet tiles inside, covered with cheap PC soundproof material for neatness (still a wip :p) For drives, I'm using some scythe 4x drive in 3x 5.25" bay silencing mounts (don't do much, but easy to mount). The system is installed on a 4gb CF card in IDE adaptor (to save ports and drive space), E2xxx C2D, gigabyte P35 board I removed from my PC when downsizing (8x sata ports), 2gb of ddr2, bottom end nvidia card and an Intel Gigabit Pro NIC running XP SP3. I've set it to sleep after 1hr, but with Wake-on-LAN turned on, so if files are being accessed it turns on/stays on - if you access the drive with the server sleeping, theres a 3 secondish delay, but nothing major. This feeds my HTPC, but I also have it scheduled to run backup to my other pcs and running utorrent with sleep after download complete turned on.

It's pretty quiet, with a mini ninja passive and a passive gpu, drives set to shutdown if not accessed. I've a 120mm Yate-Loon at the rear and another modded to the drive cage pulling air in, keeping the drives cool.
 
Back in the day I would've used PCBackup and 2,777,778 1.44MB floppies - I only bought them for backups, so much durable than the cheap-o 5.25" ones! :p

Sorry for the unhelpful post, I was stunned by the amount of data you've got there! :)
 
York said:
One thing to remember is that with RAID you're not just worried about disk failure, but as storage capacity within an array increases you're also worried about the uncorrectable bit error rates
Click to expand...

Every now and then I read something which makes me very unhappy. I didn't realise hard drives got bits wrong and didn't notice.

What can be done about this, even on a single drive?
 
I've just had a look at the 12 port controller cards which is what I would be needing if I had a 8TB array...they are around £550 for a 3ware card :eek::eek::eek:

Ouch!

I know you can't argue when it comes to securing data...but that is a heck of a whack on top of buying the machine...

1. What's the likelyhood of Ubuntu/Freenas being able to recognise it?
2. Should it conk out after the warranty period, would it really be as simple as buying a new 3wire card with similar specs and plonking it in place?
3. What would happen if in 3 years 3wire has gone bust and the card breaks...would there be any other way of retrieving the data?
 
You must log in or register to reply here.
Back
Top Bottom