RAID problem

RabbiJacob · 14 Dec 2010 at 07:40

Hi guys,

I have a DL380 G5 server, standard raid card, that's gone belly up with a RAID 5 array in it. One drive in the array has recently failed but since replacing it with a new drive the server no longer boots. Windows 2003 attempts to load, the loading screen appears but hangs with a black screen shortly afterwards.

Not a problem I thought, we perform fortnightly backups of this data. Unfortunately, the person in charge of the backups tells me the last full backup was in June which renders the backup useless (we'll be having words about this later)

Luckily I have an identical server and I've put the drives into this server in an attempt to recover the data. However, this server black screens exactly the same as the original. Researching on the net indicates that it could be a problem with the array and breaking the raid could fix it. The trouble is one drive has already failed and hadn't fully recovered as of last night.

I'm running windows setup on the second server to try and run a check disk from the recovery console to see if this can repair any errors on the array. Also, I've managed to boot the original server by breaking the mirror on the system disks and I've installed the latest Proliant support packs and firmware, HP forums suggest that the latest firmware for the raid card could detect and fix potential problems with the array.

Anything else I can try? am I missing anything?

Thanks for your help, my ass is on the line

iaind · 14 Dec 2010 at 08:22

I assume you're booting from the raid5?

You could try a Linux live CD to see if the data is still there. If it is, it's hopefully just a corrupt system file that a repair install of windows will resolve.

If not, it's going to be big ££££££ for professional recovery. 2 weeks backup regime seems silly anyway, let alone if the backups haven't been run since June! Is this critical data in any way?

RabbiJacob · 14 Dec 2010 at 08:35

The raid 5 is purely data, and is high importance. Both servers have a raid 1 mirror for the OS drive.

I've got to the windows recovery console on the server with the raid 5 disk in it, although I can see the drive (as E) running a "dir" just locks up, or appears to hangs the server. It appeared to hang last night when loading windows setup but eventually loaded.

Indeed, 2 weeks is not enough. I've inheritted the current system from the previous administrator. Ironically, I'm plan to completely revamp the entire backup/DR regime in the new year.

RabbiJacob · 14 Dec 2010 at 09:40

RabbiJacob said:
I've got to the windows recovery console on the server with the raid 5 disk in it, although I can see the drive (as E) running a "dir" just locks up, or appears to hangs the server. It appeared to hang last night when loading windows setup but eventually loaded.

Running "dir" from the recovery console reports: "An error occured during directory enumeration"

edscdk · 14 Dec 2010 at 09:50

RabbiJacob said:
Running "dir" from the recovery console reports: "An error occured during directory enumeration"

I suggest you cry, well its not your fault so if its not your data you can probably laugh...

1) is the data worth more than 1 - 3 K? if so go for data recovery. Don't do anything else, if there is 100 grand of data on the drive its not worth trying to save a few grand.

jsut noticed, you say you broke the mirror but initially you said you had a raid 5... do you mean you ahve a mirror OS and a raid 5 data setup?

if so the error on the dir of the boot disk is not an issue..

edscdk · 14 Dec 2010 at 09:51

if I guess your setup right can't you jsut slap on a os to the boot drive and look at the data on the raid?

edscdk · 14 Dec 2010 at 09:54

edscdk said:
if I guess your setup right can't you jsut slap on a os to the boot drive and look at the data on the raid?

ok i read you other posts I see your setup now...

RabbiJacob · 14 Dec 2010 at 09:58

edscdk said:
1) is the data worth more than 1 - 3 K? if so go for data recovery. Don't do anything else, if there is 100 grand of data on the drive its not worth trying to save a few grand.

The data is related to assessments so it's essential. 1-3k isn't an issue, I think we can afford this if we need to send the disks away to attempt recovery. Can you recommend any firms?

The data disks are in our spare identical server, which doesn't have the latest firmware & support pack. I've updated the original server firmware/drivers, I could put the disks back in this server hoping that the update could detect & fix any errors?

jsut noticed, you say you broke the mirror but initially you said you had a raid 5... do you mean you ahve a mirror OS and a raid 5 data setup?

if so the error on the dir of the boot disk is not an issue..

Sorry I should have been clearer, it's been a long night

The server has 2 disks as mirror for the OS and 6 disks in RAID 5 for data. I had to break the mirror to get the OS to boot, but this is now fixed. It's the data that is the problem

edscdk · 14 Dec 2010 at 09:59

if your data is not worth the data recovery fee I would flash the latest firmware to the controller, make sure the raid controller thinks the raid is ok then run a chkdsk...

this WILL trash the setup even more if its anything more than a messed up NTFS volume issue...

edscdk · 14 Dec 2010 at 10:10

RabbiJacob said:
The data is related to assessments so it's essential. 1-3k isn't an issue, I think we can afford this if we need to send the disks away to attempt recovery. Can you recommend any firms?

The data disks are in our spare identical server, which doesn't have the latest firmware & support pack. I've updated the original server firmware/drivers, I could put the disks back in this server hoping that the update could detect & fix any errors?

I would

1) get several quotes from recovery firms
2) tell the boss you can try and recover the data BUT you cannot garantee it will work and it could make the situation worse
3) give the boss the quotes and let him decide

by trying to fix this you can easily make it harder to fix (or impossible)

Nikumba · 14 Dec 2010 at 11:26

RabbiJacob said:
Can you recommend any firms?

We have used a company called Disklabs in the past with very good results, pretty quick and seemed to be the cheapest

Give Paul Allen [[email protected]] a call and he should be able to help you out

Kimbie

RabbiJacob · 14 Dec 2010 at 12:35

edscdk said:
I would

1) get several quotes from recovery firms
2) tell the boss you can try and recover the data BUT you cannot guarantee it will work and it could make the situation worse
3) give the boss the quotes and let him decide

by trying to fix this you can easily make it harder to fix (or impossible)

Thanks for your help, had a meeting this morning and said pretty much the same thing

Nikumba said:
We have used a company called Disklabs in the past with very good results, pretty quick and seemed to be the cheapest

Give Paul Allen [[email protected]] a call and he should be able to help you out

Kimbie

Thanks for that, called him and asked for a quote, nice chatty bloke

Nikumba · 14 Dec 2010 at 12:44

RabbiJacob said:
Thanks for your help, had a meeting this morning and said pretty much the same thing

Thanks for that, called him and asked for a quote, nice chatty bloke

Yeah he is chatty, its nice I find to find a chatty supplier, actually make you feel you are valued by them