4TB backup solution - recommendations on how please!

TheBigCheese

TheBigCheese

TheBigCheese

Soldato
Joined
6 Sep 2005
Posts
3,781
Hi guys

I need to backup and store data from five PCs:
x4 on XP 32bit
x1 on Vista 64 Ultimate

I would like to have 4TB of storage mirrored (8TB of HDDs worth) so the data is protected should one drive fail.

I would really like some recommendations on the best way of achiving this.

I have thought about a NAS system, but I don't know the first thing about them...

Are they just like building a standard PC with many HDDs in them?

XP Can't see more than a 2TB drive, so would I be able to create x2 2TB mirrored arrays?

What software should I use? I don't think I need Windows Home Server, I don't need auto backups etc. but I don't know if the freeware versions work correctly with Windows...I'm in a bit of a muddle! :confused:

Any thoughts would be greatly appreciated! :D
 
It would be more effective to host the BU solution on a PC which can see ALL the space. Use RAID 1+0 on that PC and PULL all the data from other PCs instead of pushing.
 
Buy a small server chasis or build a very cheap PC with a decent RAID card then stick 6 disks or so in RAID 3/5 and then network it. No need to mirror really if running a good RAID setup

- Pea0n
 
I'd go with a raid5 in an itx board setup (for quietness), small chassis that can hold lotsa disks, FreeNAS (it just works), use NTBACKUP (comes with XP, should also come with Vista too) schedule it to dump to a network drive.
 
Thanks very much for the replies guys, it's much appreciated! :D

I have to rule out pulling the data from different machines onto the one machine for a couple of reasons - the other machines may need to access the data, and it would be irritating to navigate to four sets of drives copying data, far easier (for me) going from machine to machine copying data across.

Raid 5 has been mentioned a couple of times, why would you recommend that? That's where the data is mirrored and striped? So you get the speed of striping with the backup of mirroring.
How much space does that give me if I use 3 1TB drives to create a Raid 5 array? Is it 2TB of storage?
Would that mean I could still have the 4TB of storage with only 6 disks?

As you can tell I'm a little confused with what I need to get to accomplish results, I would be very interested to hear the replies! Once this gets decided upon I can move onto the hardware :)
 
I probably would be tempted to build a NAS system with 4 x [2x1TB] (mirrored). You could set permissions so that each desktop has access to a certain area on the drive raids. Say 1 share each per 2TB. That means the data can just be copied over.

What sort of data are you talking about here if you don't mind me asking?
 
System_ofa_Down said:
Isn't there some issue with large amounts of terabyes and raid? Eg if it fails it logically loses data.

must be dreaming :o
Click to expand...

"RAID 0" means you will lose data if a disk dies (which it sounds like you are referring to this), but then RAID 0 isn't technically RAID as it offers no redundancy. Mirroring/RAID 1/3/5 is the entire point of the situation, i.e. if it fails then you DON'T lose the data.

- Pea0n
 
quick and dirty

raid 0
striped across all member drives, no parity, all data is lost if one drive fails 100% of hdd space available for data, increased read write speed

raid 1
mirrored across 2 or more drives, one drive can fail without data loss, 50% or less if using multiple mirrors hdd space available, write speed penalty

raid 5
striped across 3 or more drives with distributed parity, one drive can fail, 66% oh hdd space available for data in a 3 drive array 75% in a 4 drive array, write penalty, read speed oos, read/write penalty when one drive dies

raid 0+1
stripes of mirros, hink of it as a two level raid, level one is raid 0 for incread read/write performance, level 2 is raid 1 for redundany, require 4 or 6 hdds or more, has a read write speed boost and 2 hdds can die if on eiter side of the mirror

I'd go for raid 5 as you will most likely be limited by the network interconnect between Pcs, gigabit only capable of 113MB/s, raid 5 had the highest percent of original drive capacity whilst heving redundancy, get a 3 or 4 member array, partiton it to 2gb partitons so you xp can see them all
 
Netgear Readynas Pro Pioneer [same as the Pro business but without drives and some of the more advanced options] - costs the thick end of a grand but has more options than you can shake a stick at, and comes with NTI Shadow - or if you want, just set up Filezilla server on each machine, and the gui is easy peasy for setting up FTP based backups. And it's bloody fast over gigabit ethernet, you can hit over 100MByte/sec if your network and disk are up to it. That's not a typo, it can saturate a Gbit NIC link. Which is why is supports 802.3ad [IIRc...] NIC teaming for a 2Gbit link.

Single 1Gb link, downloading from 3x750gb drives in FLEXRAID to a 4x15,000rpm RAID0 array on Ubuntu over FTP, jumbo frames enabled, file system journalling turned off - eighty megabytes per second.


I suggest the Pioneer Pro because it has the 6 bays, so plenty of capacity, and it has RAID6 [and a prorietary, autoexpanding RAID called FlexRAID or something], although you can buy it as a 6Tb [raw capacity] box. It can also back itself up to a seperate location, so when 4Tb disks are cheap as chips in a couple of years time, just tell it to FTP/RSync/whatever to your other machine with lots of capacity...Oh, and it also does about a billion other useful things like torrents, snapshots, etc.

No faffing about building a machine, checking hardware compatability [other than for the disks, natch], tuning it, etc. Plug and go.

Had one at work for about six months now, and I'm constantly amazed at how I just can't phase it with anything :cool:

Manufacturers linky:
http://www.netgear.com/Products/Storage/ReadyNASProPE/RNDP600E.aspx
 
Just build up a cheap system with one of those motherboards with graphics, network, RAID on board and 4 1.5TB-2TB drives and set up a couple of Raid1 arrays thats what i'd do. What was posted above is pretty much overkill unless this is for business but i'm guessing from the language used by the OP this for a home solution, which a cheap server would be perfect.
 
Interesting mix of advice here. You don't need to spend a grand on a propriety box, Building your own pc and putting 6 drives in it is the most obvious answer.

Mini Itx was poor advice, as the board can run at most 4 drives, and not one of the cases available hold more than 4. So it just won't work for this. m-atx probably makes sense. I think the antec P182 and similar hold an matx board, 6 hard drives, one optical drive. This is pretty much ideal. Any dual core non-atom system will do excellently at this. I'm running a four disk raid 5 from a 7750be edition at the moment. Actually one of the drives has failed, so I'm running on a degraded raid 5 at the moment.

Raid 1 for 8 discs is madness. Raid 10 on 8 discs would give you 4tb, but you don't need the performance increase it'll supply as the bottleneck is the ethernet link anyway. Raid 5 or 6 are definitely the way to go.

Raid five, buy one more drive than the capacity you need. Raid 6, buy two more. If one drive fails in raid 5, it doesn't matter. If two drives fail in raid 6, it doesn't matter. I'd suggest a five disk raid 5 as being pretty much ideal in terms of probability of failure vs lost capacity. Raid 5 costs less than raid 1, so I don't understand DIABLOs post.


That's about it for hardware. As for operating system, windows is probably capable of doing this and seems to be what you're familiar with. FreeNAS will do rather better, cost nothing and probably works out of the box. Ubuntu offers far more freedom but will take at least a couple of days to get used to. If you go for the linux option, you can do software raid that is far, far easier to recover if things go wrong. You can also do things like add more hard drives to the array if you wish for more space, which can't be done through most hardware controllers. Plus there are people on here who are very familiar with mdadm (the software that normally does raid in linux) so the transition would be a gentle one.

Depends a bit how automated you want it to be. Once set up, ubuntu could be left completely alone and it'll just continue working. Easiest option on the non-server computers is to have a few folders where anything left in them will be backed up. Desktop, my documents etc, plus a misc folder somewhere on the C:\ drive for anything else you'd like to back up. Linux is probably scary the first time you meet it though, so depends if you're prepared to learn about a new system.

Looking forward to finding out what you intend to do. Cheers
 
Thank you one and all for your replies, I appreciate all ideas!

JonJ678 seems to have covered most of the bases on the previous comments so I'll post from that...

Hardware
I think building my own machine is definitely the way to go, so just to check what I would need:
Basically any low-mid range system would be fine for a RAID array?

RAID Question
I think RAID 5 sounds like a good bet, so if I want 4TB or storage I buy 5 1TB drives, is that correct? (and if I wanted to go mad and have 8TB I would buy 9 drives?)

RAID hardware or software
Seems to be a bit of a tricky one this...can you explain what the difference between them is?
I always thought that software RAID was the poor mans way of doing RAID. Several years ago I used to have a RAID 5 system and RAID 0 in the same machine. Both were controlled by the motherboard (built in RAID options) and the RAID 5 was continually falling over, and rebuilding itself took a flipping age and virtually ground the machine to a halt whilst it rebuilt.

My understanding was that a hardware RAID (from a dedicated controller card) was by far the better option, it was faster and more stable...have times changed or have I got that wrong?


Software
Tricky one this, and one that I have a couple of questions about...

FreeNas
I have absolutely no understanding of this, though I think I could learn it. This may be a completely idiotic question...but can Windows PCs see the HDDs setup in a FreeNas system?

Windows
This does sound like a simplier option...yes I do know Windows already...should I just buy a copy of XP/Vista OEM and set it up from there?

Anti Virus
I had thought that if it was on the network and shared then it would be exposed to any viruses etc...I run Kaspersky on my PCs now, if I had a Windows setup then I could install it on there as well...would FreeNas be protected from viruses as it's not a Windows machine (in the same way that Apple was virus free for so long)?

Automation
I don't need it to be automated at all, we work on the files/folders on our local drives then when we have finished, or need to archive anything they can be moved across to the NAS where they are automatically double backed up, that's the idea...about as simple as it gets really. :)

Linux
I hadn't considered that...I have breifly looked at Ubuntu once about a year ago, and it seemed a bit of a pain to get going, and rather unfamiliar (obviously).

Switching it off
It would be going in an office and I would be wanting to switch it off at night to help keep costs down...is that simple enough to do or are they designed/meant to be left on 24/7?

Problems
You mention that Linux is far easier to recover from...what sort of things can go wrong? I thought about the worst was a drive failing, but it could simply be taken out and swapped for another and all the data would be intact...am I being a bit naive there?

Limitations
With using XP machines, am I able / should I be making 2 2TB partitions? It would be easiest for me if all the computers can see the whole NAS and data can be copied into the appropiate place from any machine, security isn't important as everyone using the machines is trusted and competant.

Hardware
Phew...I think that's enough questions for one post...I'll come back to that in a little while!
 
Last edited:
Accidentally wrote lots, apologies

Anything dual core or higher would do it excellently. Dual core atom might do, I was looking at that but was persuaded not to by people on here. 1gb ram, 7750be amd, very budget motherboard here. Running like a charm, even though the operating system is on a usb stick. It'll run bittorrent at the same time as rebuilding an array and streaming a film, so I'm quite happy. Yeah, 8tb would be 9 drives. 9 drives all at once, you're looking at considerable odds of one failing. One idea is to buy 50/50 wd and samsung say, as the odds of both brands failing at a similar time period are less than four identical ones going down all at once.

Hardware vs software is difficult. The former will be marginally quicker as the processing is done on the card, so the operating system doesn't need to. Note that motherboard built in raid doesn't have a processor. It might be more stable, but you'll have to define stable before I can be certain on that one. However, if the card dies, you'll have to replace the card with exactly the same model and pray that it detects your array. If it doesn't, you are sol. Similar theme with motherboard raid, changing from ich8 to ich9 may fail completely. Swapping out with an identical board stands a good chance of working, but even this isn't certain. It isn't just hard drive failures you have to worry about with hardware raid.

With software however, it doesn't matter what it's running on. I could put two of my drives in usb enclosures and it would still run just fine. I routinely run it from an ubuntu livecd with no problems. So while you have to set it up in software, and have to type a few lines when you reinstall the operating system/change motherboard, it doesn't matter if you change hardware. I think mine is on its third motherboard so far. It's more resilient from this perspective.

Freenas/ubuntu/debian is an operating system. Yep, windows will see the raid as network shares. Linux has a subsystem called samba written solely for this purpose. Freenas is a stripped down/streamlined version of linux designed to be very effective as a nas but to do nothing else. It's very popular, but I wouldn't use it as the inflexibility would drive me mad. Ubuntu server is built for the purpose, Id recommend this.

Windows. YMMV. It has a home server version which is presumably capable. Yes it will get viruses, and do all the nasty things windows normally does. Refuse to let you do anything remotely out of the ordinary being my objection. I suspect the windows version of software raid to be a mess, but perhaps its usable. There's probably a reason why everyone using windows seems to recommend raid through the motherboard.

FreeNAS/linux/macintosh all broadly speaking ignore viruses. Their systems are harder to attack, less likely to have exciting information lying around, and do unkind things like use root passwords. Security is still an issue, and a firewall is generally worth setting up. Linux out of the box is likely to run peacefully and calmly indefinitely, in a fashion I've never seen from windows. If the server doesn't need internet access, blocking ports 80 and 25 would make attacking it ludicrously hard, even from one of the windows machines. Thinking about it, the weak point on your network is the windows systems anyway. were you so inclined you could pass internet access through the server, and run a very effective firewall on the server, protecting the windows systems behind it.

Ubuntu is doing rather well these days for user friendliness. Load a jaunty livecd and have a play for a bit, it wont install anything to your computer if you don't explicitly ask it to. It's come a long way from 7.10. Naturally a system built on arch would be quicker and more configurable, but I don't think its what you're looking for.

Given so little automation is desired, freenas is the better choice. Each computer would see a 4tb drive in my computer, which it could write to at will. It's not how I would do it personally, but simple certainly. I'm pretty sure the volume can be any size, the only limit I know of is windows can't boot from a volume greater than 2tb. As you'd be running freenas off a usb stick (if sensible, keeping os separate to data storage is always wise), this won't be an issue. Make sure you format it as ntfs.

Shutting it down would be simple yes. Could do it remotely if you were so inclined, ssh into the server and issue the command 'halt'. Otherwise just walk in and click off. I'd use the former, since then it doesn't need a screen/mouse/keyboard. I don't turn mine off, but then I don't pay electricity bills.

recovery is in terms of hardware failure really, but I accidentally covered that above. Dead motherboard and you're pretty ruined if you were running your raid off it. Adding another drive later on is a bit intense, but at least its possible and there are fairly extensive guides. If you think you'll give ubuntu a shot, I'll find some guides for you and answer any questions you have. If freenas, I'll hand you over to someone else, or if need be ill set a copy running in virtualbox. If windows, I wish you luck.

What budget have you set? I'm fairly sure I'm going to end up moving my current server into a P182, looks more tempting every time I see a photo of one.
 
Hi There,

My 2 pennies worth!

Definately use RAID 5, preferably also running a hot spare. I have a home file server that has 6 x 500 Gb drives, 5 are in the RAID 5 array so I get approximately 2 Tb. The 6th drive is set as a hot spare, so in the event of a drive failure the sixth drive will kick in a replace which ever drive has failed and the array will rebuild automatically with only a slight performance decrease whilst it rebuilds. the failed drive can then be swapped out at your leasure.

So in your case I'd probably reccomend 6 x 1Tb drives in a similar arrangement.

Software or Hardware RAID? Definately go hardware. All the built in, on motherboard solutions I've used have been pretty dubious, seems to work fine until you have a failure then it all goes pear shaped. AMCC/3Ware, LSI or Adaptec are the ones to go for. I use the AMCC/3ware both on my home server and in work on some very large multiply reduntant file servers and they work very well.

If the data's very important to you I'd suggest fitting the RAID card battery backup unit, this will save any data in memory if the event of a power failure, so when power returns it can be written to the disks. Alternatively get a decent UPS on your home server which will automatically and safely shut it down in the event of a power failure.

OS? I run Fedora core 10 linux on my home server, but unless you're reasonably happy with linux stuff I'd probably suggest some flavour of windows instead, just for ease of setup and use. As suggested by others the free linux NAS solutions sound interesting, the great thing about a linux solution is that once you've got it working it will be rock solid and usually much quicker than an equivelent windows setup. Its also much more secure from viruses etc.

By the way, just to be totally anal and because I do have a habit of breaking operating systems by fiddling, I actually have a seperate 160 Gb hard drive, seperate to the main array for my system drive, so no matter what I do with the operating system all my data is safe and seperate.

Good luck!

E-I
 
Evil-I said:
I do have a habit of breaking operating systems by fiddling, I actually have a seperate 160 Gb hard drive, seperate to the main array for my system drive
Click to expand...

Sensible man. I particularly agree with this point, keeping os and data seperate was the best thing Ive done with computers. I think Id be fond of raid cards if I bought a decent one, the difficulty of recovery when the raid card dies is what puts me off.

Hot swap is solid too, I thought that was excessive for me. I really should rma my dead drive but just not getting around to it.

How's fedora? I'm fairly firmly in the debian family at present, considering trying arch or gentoo this summer mainly as a learning exercise.
 
A bit off-topic, but how does RAID 0 compare with RAID 5 in terms of performance? I'll be moving my RAID 0 to a new motherboard (ie wiping it :() later this year and I'm tempted to get a second drive in order to get some redundancy. It'll mainly be used as my normal machine, doing normal stuff (programming, web browsing and the occasional bit of gaming).
 
got about half way and cba to read more, so sorry if its been posted.

I am running a RAID 5 array using my onboard controller with samsung F1 1TB drives. I have 1 drive for boot since no windows will boot off of a partition larger than 2tb, and the other 4 in the raid 5 array giving 2.76TB of total space. 250gb is one NTFS partition, the rest is one large GPT partition. I am running vista 64 btw. My other 3 pcs and my house mates pc + laptop on XP pro SP3 can all see and access the 2.5tb partition perfectly fine....which before i tested i was under the impresion it would nto be able to do but meh.

Write speed is slower than a single drive, but read spead is at around 250mb/sec, which is good enough for me, since i cant fit/afford a decent hardware raid card in my machine.
 
OrphanBoy said:
A bit off-topic, but how does RAID 0 compare with RAID 5 in terms of performance?
Click to expand...

Different game. Raid 0 will outperform raid 5 fairly conclusively. The answer you're looking for is raid 0 plus backup drive, not raid 5. Or get two more drives and go for raid 10, which performs the same as raid 0, gives redundancy, but needs twice as many drives.

rsync is awesome.
 
You must log in or register to reply here.
Back
Top Bottom