Advice on a 1TB RAID 5 setup

AtomicBanana

AtomicBanana

AtomicBanana

Associate
Joined
28 Sep 2003
Posts
1,724
Location
Leamington Spa
Hi there guys, been trying to figure out the most foolproof way of doing this, but I think it's best I ask some advice before I start parting with cash.

Basically, I want to know what hardware/software you guys would reccomend for a budget raid 5 setup. This will mean software raid I assume, and I have no problem with that. The problem I'm having is that I'd like to use SATA drives but I only have 2 sata connectors on the mobo in the server currently (that I *think* are supported) and that isn't enough.

So,

What's the cheapest, linux compatible SATA card going to be that I can get a good few drives going on?

Any pit-falls of linux software RAID 5 I should know about?

Anything else?

Cheers guys, much appreciate any advice I get from you guys :]
 
When I had raid 5 on linux I used a 3ware sata raid card, cant remember the model, might be able to pick one up on auction sites.

BTW AtomicBanana you are not allowed animated gifs as a sig.
 
As nice as a card is, I think the flexibility and redundancy offered by software linux raid / LVM out strips all benefits. (unless you have big budget, and can afford to have spare cards, lying about in case one fails.)

Edit, oops just re-read the post - are you suggesting the SATA card purely as a way to add drives?

Hmmm, I remember reading a while ago about something.. .http://linuxmafia.com/faq/Hardware/sata.html

Has quite a bt of handy info.
 
BTW AtomicBanana you are not allowed animated gifs as a sig.
Click to expand...

Sorry, hadn't noticed. Removed it :]

I've gone under the assumption that hardware RAID is too expensive and that linux software RAID will do fine. So basically I'm looking for decent Sata controllers that work well with linux - I don't have the money to waste on buying somethnig that I can't get to work :(
 
from experiance the cards that i've found to be the easyest and to work with 99% of distros have been 3Ware cards.

Cheers
Deano
 
Yeah, had a look around today, and hardware RAID wise, looks like 3ware is the best option. Although at this time I don't want to flash about that much cash! How much for a decent 4 port sata controller that's pci and works well with linux? I'm having trouble finding ones that I can be sure will work before I buy them . . .
 
0. Use EVMS.. not the standard volume manager. It's free and available from sourceforge
1. Get at least 2.6.11 kernel because the version of EVMS that allows volume expansion requires it.
2. Get the latest version of EVMS from sourceforge.
3. Set it up play with it to get the feel of the partitions, devices etc..
4. when using EVMS GUI functionality you must keep the EVMS GUI process alive (no logging out or closing the window .. best run X on the box or use the curses text gui instead).
6. Label the drives :D with the device id (/dev/hda or hdb etc) do if it fails you know which it is.
7. Use persistence superblocks - that way the raid software knows each drive even if you pull the cables out and plug the drives into different controller ports.
8. Install linux without the raid 5 drives attached to ensure the distro installer doesn't attempt to use the drives "cleverly" for mount points during install. Nothing like finding your boot block is on a raid drive..

9. Once installed be careful of EVMS and Linux updates - ensure you read the interweb before deciding to install. Nothing like finding your 1TB is no more after an update.

When creating the array it will spend time rebuilding the array in the background - as it would if the array was running a rebuild after a drive failure.

Welcome to 24/7 computing and get an APC UPS.. power spikes/failures are the best way to kill a hard drive.

here's mine when it was all being installed/tested: click me! and Loadsa drives!

My hardware:
Barton XP2500 at stock
512MB stick of value ram
NF7-S v2 - 2 port Silicon Image 3112A
PCI 4 port SATA controller card Silicon Image 3114
Gf3ti200
1x 80GB IDE Spinpoint linux boot drive
2x 200GB SATA Maxtor DM9
4x 200GB SATA Maxtor DM10
Antec 550W TruePower
APC 500VA BackUPS

You'll note - the raid volume uses drives over both the controller in the NF7-S and the PCI card. Hardware "RAID" support is not required! All that's required are a set of drives - even IDE and SATA drives can be used in the same RAID5 volume!


Last thing to mention - linux software RAID raids the partitions not the drives so as long as the drives are big enough to contain the partition the geometry of the drive (size/sectors etc) doesn't matter. This means you can use drives such as a 300GB with 200GB drives but your partition will still have to be 200GB. The additional 100GB doesn't go to waste as you could make another 100GB partition and use that - even mirror it with another 100GB partition on a different drive that may not even be part of the RAID5 volume!
EVMS & Linux software raid is both powerful and flexible.
 
Last edited:
you can get a 4 port sata 2 3ware raid card for £210, thats cheap i paid £138 for a 2 port sata card not long after they came out.

understand £200 is expensive but looks as if 3ware cards have come down in price quite abit!

cheers
deano
 
NickK said:
[stuff]
Click to expand...

Wow, that is quite scary, that's pretty much exactly what I had planned, same case, mobo etc etc!

What controller card did you use specifically? What distro? Having a brain dead evening and finding it hard to find a cheap controller card using that chipset >_>

Cheers for all the info .. giving me some more reading to do :]
 
Avoid cheap controllers. I used a Silicone Image chipset card on an old system before I had a sata motherboard. It worked, but the kernel reported lots of errors when I was writing large files. Never had any data corruption, but I was fairly frightened I would get it one day.
 
riven said:
Avoid cheap controllers. I used a Silicone Image chipset card on an old system before I had a sata motherboard. It worked, but the kernel reported lots of errors when I was writing large files. Never had any data corruption, but I was fairly frightened I would get it one day.
Click to expand...

What kernel version was the distro?

I've not noticed any errors with massive files with the setup I've got. The only problem I had is that the versions of Apache and PHP that came with the distro couldn't cope with files over a 32bit size.
 
AtomicBanana said:
Wow, that is quite scary, that's pretty much exactly what I had planned, same case, mobo etc etc!

What controller card did you use specifically? What distro? Having a brain dead evening and finding it hard to find a cheap controller card using that chipset >_>

Cheers for all the info .. giving me some more reading to do :]
Click to expand...

I used SuSE 9.3 IIRC with a completely rebuild kernel from kernel.org.
uname -a gives me 2.6.11.4-21.7 so that's a tad old considering the latest is 2.6.15+ now :D
I'm also using evms-2.5.3 which is probably not the latest either.

I would also recommend getting a good VNC client for desktop access via a window in XP.

The machine is powerful enough to run more than the RAID5!
I have mysql, two versions of apache, samba etc etc.

As far as I know virtually any good distro comes with evms now..
 
NickK said:
What kernel version was the distro?

I've not noticed any errors with massive files with the setup I've got. The only problem I had is that the versions of Apache and PHP that came with the distro couldn't cope with files over a 32bit size.
Click to expand...
I used multiple kernel versions (gentoo user), it was about a year ago, and would have been 2.6 for deffinate. I think I found evidence to it maybe being a drive/controller combination that was a bit dodgy. There are drives that are blacklisted with the silcone image kernel drivers, and IIRC mine was one of them. This was then meant to implement a safer write method. I did actually hack the kernel one time and remove my drive from the blacklist. I had no i'll effects but the errors persisted. I now have the same drive on an nvidia controller and it runs fine. Just one of those things I guess.
 
I'm currently running a software RAID5 array under Ubuntu - works very well.

A couple of things:
- as mentioned in a post above, it's definitely worth considering using EVMS, as it handles everything for you (partitioning, raiding, formatting, etc.). It comes preinstalled and configured under Ubuntu, so unless you're stronly considering another distro I'd suggest going for Ubuntu to save yourself the hassle of compiling/setting up EVMS yourself (not that it's a huge hassle though).
- one big advantage of using software RAID is that you don't need to have all your drives on one controller - I had 4 drives on the onboard ports, and have now just got an extra controller to add more drives to the same array.
- regarding cheap controllers, I've just bought myself an XFX Revo 5-port SATA controller. It's supported straight out of the kernel as of 2.6.13 (Ubuntu Dapper has 2.6.15 so you'd be ok there). I'm trying this out now, so will let you know how that goes. It's only £50 new, and would also do hardware RAID if you needed it to.

Good luck; I'm sure it'll all go fine. I've had 2 drives fail (not at the same time) since the array's been running (which has been 18 months). Each time the array stayed up, I lost no data whatsoever. I just RMA'ed the drive and plugged the replacement back in and got it to resync, et voila!

Note one big feature of EVMS is that it allows you to expand an existing RAID5 array to add an additional drive. This is extremely useful.

Best of luck.
 
Yeah, cheers guys, got enough info to give this a go and start spending the cash. I want to have this and my mythtv box up and running before I'm back to uni ;)
 
The Belgain said:
Note one big feature of EVMS is that it allows you to expand an existing RAID5 array to add an additional drive. This is extremely useful.
Click to expand...

I think my array took 10 hours to expand the 800MB volume by adding a 200GB drive. The more drives/bigger the array - the longer it takes to expand.
Also it's the GUI portion of EVMS that does the expansion (and so has to be kept alive although it will usually be able to restart from where it left off).
 
You must log in or register to reply here.
Back
Top Bottom