Adaptec 3805 Vs. Highpoint 2740

enigmo · 25 Oct 2011 at 16:53

Got a bit of an issue with a home server. It's currently got:
Adaptec 3805 -> Supermicro CSE-M28E2 > Several Toshiba 1TB 2.5" SATA HDs

I've had nothing but problems with this setup, what with the controller card randomly deleting logical drives and demanding a rescan to pick them back up again. To be honest, I'm not sure where the issue is, but I strongly feel it's the controller card and am considering cutting losses and replacing it. I have a Highpoint RocketRAID 2740 already, just unused, and am keeping one card and selling the other. So my basic question is:
Is the Adaptec 3805 performance hugely better than the Highpoint? Can't find comparisons of the two online.
If so, is the performance gain high enough to justify keeping the possibly unstable card and troubleshooting further?
If not, is the 2740 good enough to warrant replacing the Adaptec card and knocking it on, assuming it doesn't display the same problems as the Adaptec?

smargh · 25 Oct 2011 at 21:19

Are the Toshiba drives on the Adaptec certified list? Presumably they're the enterprise variety?

enigmo · 26 Oct 2011 at 09:25

smargh said:
Are the Toshiba drives on the Adaptec certified list? Presumably they're the enterprise variety?

Actually, no. They are not enterprise, they are the bog standard 2.5" SATA Toshiba disks you can get for laptops and last I checked were not on the Adaptec approved list. The Enterprise grade drives and 2.5" SAS drives were just too expensive for a home server. So, probably begging for trouble from the start, but still the controller randomly dropping the logical drives is a pretty major issue and far more than a simple 'incompatibility' I reckon.

enigmo · 26 Oct 2011 at 10:32

Right, swapped out the Adaptec card for the Highpoint and its all the same story again. Must be the Supermicro caddy that's broken. Problem I've had all the way along, though, is finding anyone with experience of using these caddies with stock SATA drives...

dewi · 26 Oct 2011 at 10:58

I think the problem is more likely to be with the drives themselves. The problem with using consumer drives in RAID arrays is that they're likely to drop out of the array when there's a disk error. The larger the number of drives in the array, the more likely this is to occur. See, for example:

http://wdc.custhelp.com/app/answers...-edition-and-raid-(enterprise)-edition-drives

I use Western Digital RE4 enterprise drives with an Adaptec ASR-5404 RAID controller, and I've had no problems at all.

It seems to me you have a number of choices:

1. You could replace the Toshiba drives with enterprise drives.
2. You could switch to RAID 1. I believe that consumer drives work fine in RAID 1.
3. Do you really need RAID at all in a home server? RAID is really designed for use in business environments where you need to hot-swap drives without powering down the server. Would it not be sufficient to back up the server regularly? It goes without saying that you should be doing that anyway. I back up both my office server and my home server onto external USB hard drives. Windows Server 2008 R2 allows me to schedule automated backups; Windows Home Server v1 requires me to initiate the backup manually from the WHS console.

enigmo · 26 Oct 2011 at 14:00

I could swap out the drives, or sell the lot and restart with a more sensible solution, which is looking increasingly tempting

I was looking at the Startech miniSAS to 2x2.5" bays earlier. They seem less of an enterprise solution and so maybe more compatible with home grade drives. Or just drop the whole idea and stick with 3.5" WD RE4s.
As for RAIDing the drives, I was trying to avoid it if possible. The whole reason for getting the setup in the first place was purely for storage density; cramming the most GB into the smallest space, and at the time 8x1TB 2.5" drives in 2x5.25" bays was it, I just needed the SAS backplane and HBA to do it. The drives are currently backed up to a NAS box anyway, so RAIDing of any sort seemed and unnecessary use of expensive space.
Have you any links to articles about consumer grade drives dropping out of arrays? Enterprise grade drives aren't immune to errors, so wouldn't they drop out of arrays too, just less often?

enigmo · 26 Oct 2011 at 14:26

dewi said:
The problem with using consumer drives in RAID arrays is that they're likely to drop out of the array when there's a disk error.

The drives I have are not RAIDed, they're in JBOD.

dewi · 26 Oct 2011 at 15:03

enigmo said:
Have you any links to articles about consumer grade drives dropping out of arrays? Enterprise grade drives aren't immune to errors, so wouldn't they drop out of arrays too, just less often?

The link I posted in my earlier message explains why consumer drives drop out of RAID arrays: http://wdc.custhelp.com/app/answers...-edition-and-raid-(enterprise)-edition-drives

The reason enterprise drives work better in RAID arrays is not because they're more reliable, it's because they have different firmware. Enterprise drives implement TLER, which means that if there's an error, they just report it to the RAID controller so that the RAID controller can sort it out. Consumer drives try and fix the error, which can take several minutes, during which time the RAID controller assumes the drive is dead and drops it out of the array.

enigmo · 26 Oct 2011 at 15:13

dewi said:
The link I posted in my earlier message explains why consumer drives drop out of RAID arrays: http://wdc.custhelp.com/app/answers...-edition-and-raid-(enterprise)-edition-drives

The reason enterprise drives work better in RAID arrays is not because they're more reliable, it's because they have different firmware. Enterprise drives implement TLER, which means that if there's an error, they just report it to the RAID controller so that the RAID controller can sort it out. Consumer drives try and fix the error, which can take several minutes, during which time the RAID controller assumes the drive is dead and drops it out of the array.

So, if I were to edit the CCTL of these Toshiba drives (the equivalent of WDs TLER), then the error and recovery problems might go away, although obviously not a good solution?
Instead of reducing/killing the TLER of the drives then, is there usually any way at all of increasing the wait time of the controller?

dewi · 26 Oct 2011 at 15:21

enigmo said:
The drives I have are not RAIDed, they're in JBOD.

I've never used a JBOD, but I imagine JBOD looks a lot like a RAID 0 array to a RAID controller, with all its attendant drawbacks.

Have you considered using Windows Home Server v1, if you can still get hold of it? Drive Extender would give you a single logical volume without needing to use a RAID controller at all. Unfortunately, Microsoft removed the Drive Extender functionality in Windows Home Server 2011 (they now recommend using a RAID array instead!).

If you're going to build any kind of array with as many as 8 drives, I think you should be looking at enterprise drives (which probably means going down the 3.5" WD RE4 or Seagate Barracuda ES route if you want to keep things reasonably affordable). RAID 5 would give you protection from a single drive failure without wasting too much storage space, and you've already got a good RAID controller.

dewi · 26 Oct 2011 at 15:26

enigmo said:
So, if I were to edit the CCTL of these Toshiba drives (the equivalent of WDs TLER), then the error and recovery problems might go away, although obviously not a good solution?

It looks like it, yes.

enigmo said:
Instead of reducing/killing the TLER of the drives then, is there usually any way at all of increasing the wait time of the controller?

I don't know.

enigmo · 26 Oct 2011 at 15:35

dewi said:
I've never used a JBOD, but I imagine JBOD looks a lot like a RAID 0 array to a RAID controller, with all its attendant drawbacks.

Have you considered using Windows Home Server v1, if you can still get hold of it? Drive Extender would give you a single logical volume without needing to use a RAID controller at all. Unfortunately, Microsoft removed the Drive Extender functionality in Windows Home Server 2011 (they now recommend using a RAID array instead!).

If you're going to build any kind of array with as many as 8 drives, I think you should be looking at enterprise drives (which probably means going down the 3.5" WD RE4 or Seagate Barracuda ES route if you want to keep things reasonably affordable). RAID 5 would give you protection from a single drive failure without wasting too much storage space, and you've already got a good RAID controller.

I'm currently running Server 2008 r2 64bit.
The whole idea was to spend a reasonable amount on a controller card and drive chassis, and then increase the number of drives as and when needed. I'm thinking you might be right; just had a look a decently sized 2.5" SAS drives and I'd have to sell a kidney. Might get rid of the stock 2.5s and chassis and just get a few 3.5" RE4s.

enigmo · 26 Oct 2011 at 15:42

dewi said:
Instead of reducing/killing the TLER of the drives then, is there usually any way at all of increasing the wait time of the controller?

I don't know.

Right... I'll have a look about and see if I can change the wait time for either RAID controller to something more like 60secs, and if I can't, I'll radically re-think my storage setup.
As someone who knows what they're talking about then

given that four consumer 2.5" Toshiba 1TB drives, an Adaptec 3805, and this Supermicro M28E2 are incompatible as a bunch, what would you get rid of/replace for a decent home storage solution? At this stage, I'm quite prepared to repurpose the 1TB drives into my laptop and PS3, and sell the remaining disks and caddy and rebuild using another caddy and 3.5" RE4s. Can you think of anything better?

dewi · 26 Oct 2011 at 17:21

My office server is built into a Fractal Design R3 case, which has 8 x 3.5" HDD trays. The HDDs aren't hot-swappable, though. There may well be another case out there that does offer 8 hot-swappable bays.

http://www.overclockers.co.uk/showproduct.php?prodid=CA-008-FD

RimBlock · 27 Oct 2011 at 07:01

JBOD is exactly as it sounds, Just a Bunch Of Disks. This means the drives are just displayed as individual drives to the OS the same as if you are connecting them to the motherboard controller.

Enigmo.
First question is why are you using a SAS/SATA controller of that calibre but not using any of its advantages. Any SAS HBA 8 port controller will do the same, more or less, as you seem to be using that card for. An IBM Serveraid M1015 or LSI 9211 will do the same for a lot less. Someone is selling a M1015 on the US auction site for around 80 quid plus delivery if they will ship to the UK.

I would shy away from the 2.5" Toshibas if they are giving issues. I was considering the 2.5" WD blacks or even the 1TB blues due to their size and low power requirements but may have to rethink that.

I would also not bother with enterprise drives unless you are really serious about the data you have stored and need zero downtime if a drive fails. There are many people running WD Blacks (3.5") drives in arrays reporting no issues. With the card you already have, you could run 8x2TB WD blacks in raid 6 giving you 12TB storage and 2 drive failure resilience with hot swap and rebuild. Seagates offerings may also be just as good but I have no first hand knowledge.

If you didn't have the Adaptec card I would suggest getting a M1015 card from ebay as a new server pull and running it as a HBA for whichever 8 drives and using Windows Server to manage any spanning to make one big pool. You would obviously need a backup as one drive failure would loose the data and you cannot boot from it so you would need a separate boot drive but ... if your controller fails you can take the drives and put them in any other Windows 2008r2 Server / Win 7 / Win XP / WHS 2011 machine and access the array again (only Win Server / WHS if you do software raid 5 I believe). The point being you are not card bound to be able to access the array.

RB

enigmo · 28 Oct 2011 at 11:32

dewi said:
My office server is built into a Fractal Design R3 case, which has 8 x 3.5" HDD trays. The HDDs aren't hot-swappable, though. There may well be another case out there that does offer 8 hot-swappable bays.

http://www.overclockers.co.uk/showproduct.php?prodid=CA-008-FD

Just saw one of those cases at work a few mins ago as it happens! Very nice. Considering one of their other models to replace my desktop now.

enigmo · 28 Oct 2011 at 11:46

Enigmo.
First question is why are you using a SAS/SATA controller of that calibre but not using any of its advantages.

Basically for GB per bay storage. I have a CM Stacker 810 and having filled the rest of the case with seperate HDs for seperate things (optical drive, mirrored 300GB for boot, mirrored 500GB for the wifes stuff, 4x500GB RE3 for media etc), I found myself with two bays free and need for more storage, so the Supermicro caddy that squeezes 8x2.5" HDs into two 5.25" bays seemed perfect (potentially 8GB in two bays!), but I needed a SAS controller card to run it, so found the 3805 on eBay for a truely bargain price, I think it was like £85. Hence, wildly overspecced HBA, decent caddy, but still failing setup

I would also not bother with enterprise drives unless you are really serious about the data you have stored and need zero downtime if a drive fails. There are many people running WD Blacks (3.5") drives in arrays reporting no issues.

Now that's interesting. I was just looking at 1TB RE4s, but if Blacks will do then that might be the way to go. Would the timeout issue not still be present with them, though, being desktop-class drives?
Could I (in theory) connect four WD Blacks directly to the Adaptec card with no SAS expander or anything in between and use them in a RAID5 array?

if your controller fails you can take the drives and put them in any other Windows 2008r2 Server / Win 7 / Win XP / WHS 2011 machine and access the array again (only Win Server / WHS if you do software raid 5 I believe). The point being you are not card bound to be able to access the array.
RB

This last is also important to me, having been the victim of nVidia RAID death multiple times! My media 4x500GB RAID5 array is software based precisely for easy recovery. Speed is less of an issue as I rarely/never stream more than one HD stream from the server.

enigmo · 28 Oct 2011 at 12:00

Wow... Now those are interesting prices for hard drives... Right. Well. Now I have a rough idea of what's wrong (Dewi: I think you were right, ta) and an idea of what to do about it, I just have to wait for either a lottery win or Thailand to dry out!

(No offence intended)

RimBlock · 29 Oct 2011 at 11:02

enigmo said:
... found the 3805 on eBay for a truely bargain price, I think it was like £85. Hence, wildly overspecced HBA, decent caddy, but still failing setup

Wow, nice card for an amazing price.

enigmo said:
Now that's interesting. I was just looking at 1TB RE4s, but if Blacks will do then that might be the way to go. Would the timeout issue not still be present with them, though, being desktop-class drives?

I have only used them in a striped array (raid 0) so cannot comment from personal experience for raid 5 but if you take a look at various home server forums (servethehome and wegotserved to name but two) it seems most people are not willing to splash out on RE or enterprise class disks just for a home server. Most seem, from what I have read, to be happy with WD Blacks or the same type from another manufacturer. Also have a browse through the unraid forums as well as they can sometimes have some good info.

enigmo said:
Could I (in theory) connect four WD Blacks directly to the Adaptec card with no SAS expander or anything in between and use them in a RAID5 array?

Apart from cables

, yes. Each SFF-8087 port will connect to 4 hard drives with a breakout cable (SFF-8087 -> 4xSATA). EBay is your friend for these as they can be stupidly expensive for home use. A SAS expander is onlt used when you run out of channels (4/port) and want to connect more drives. An expander works sort of like a switch on a network managing traffice to drives connected to it and back to the original controller again. Not needed for you unless you want to go above 8 drives with your card.

RB

dewi · 31 Oct 2011 at 09:29

RimBlock said:
I would also not bother with enterprise drives unless you are really serious about the data you have stored and need zero downtime if a drive fails. There are many people running WD Blacks (3.5") drives in arrays reporting no issues. With the card you already have, you could run 8x2TB WD blacks in raid 6 giving you 12TB storage and 2 drive failure resilience with hot swap and rebuild. Seagates offerings may also be just as good but I have no first hand knowledge.

RimBlock said:
I have only used them in a striped array (raid 0) so cannot comment from personal experience for raid 5 but if you take a look at various home server forums (servethehome and wegotserved to name but two) it seems most people are not willing to splash out on RE or enterprise class disks just for a home server. Most seem, from what I have read, to be happy with WD Blacks or the same type from another manufacturer.

Sorry, but I think that's poor advice. Consumer drives are fine in RAID 0 or in RAID 1. Western Digital claim, "Desktop Class Hard Drives are tested and recommended for use in consumer-type RAID applications (RAID-0 / RAID-1)". You may get away with using them in a RAID 5 or RAID 6 array, or you may not. The more hard drives there are in the array, the greater the chance of running into problems. I think you'd be unwise to build a RAID 5/6 array of 8 consumer drives. The RE in Western Digital RE4 stands for RAID Edition. The clue is in the name. The main difference between a WD Caviar Black and a WD RE4 is not the MTBF, but TLER.

I fail to see what using an enterprise drive (or not) has to do with "zero downtime if a drive fails".

For RAID 0 or RAID 1 (or no RAID at all), I'd use Caviar Black (or Seagate Barracuda). For RAID 5 or RAID 6, I'd use Caviar RE4 (or Seagate Barracuda ES.2). I don't know how the Adaptec card handles JBOD.

I'm not convinced that using consumer drives is the cause of the OP's problems. I wrongly assumed he was using RAID 5 or RAID 6 because he was using an expensive Adaptec controller. £85 for the Adaptec was a bargain!

I still think a good solution for the OP would be to keep the Adaptec card and to use 8 x WD RE4 (or Seagate Barracuda ES.2) in RAID 5 or RAID 6.