Power/voltage issue with my Dell T410??

StevieP · 30 Sep 2017 at 11:05

Hi all, I've been experimenting with unRAID on a 2nd hand Dell T410 that I picked up and last night was the first time I've actually left it running overnight.

I walked into the room this morning and it was still humming away - until I noticed the LCD readout on the front had a message saying "E122E Onboard Regulator Failed - Contact Support" - and then, with no interaction from me, it shut down completely and another message popped up on the display saying "E1000 Failsafe Voltage Error - Contact Support"!?!?!?

Now this is a 2nd hand server and I appreciate that it's possible that there are inherent risks with that - but the weird part is that it was (I assume) fine all night and then chose that exact time to fail as I walked into the room?? (and I literally did nothing but look at the LCD!)

I've rebooted the server and am currently running memtest to see if the RAM could have been the issue - anyone else ever experienced anything like this and have any suggestions??

BaroUK2 · 30 Sep 2017 at 11:21

We did have E1000 Failsafe Volgate error on a server and it was caused by HDD Backplane. Swapping that fixed the issue. Could also be caused by faulty usb devices if you connected one.

Try removing power cable for 10 seconds and reconnect.
By the way since you ve bought this unit did you clear all hardware logs from it?
Also check system logs for more info otherwise if you are comfortable run the server with bare minimum and see if error comes back then you can add other parts one by one...

StevieP · 30 Sep 2017 at 11:34

Thanks for the reply!

My HDD's aren't connected using a backplate - they are connected to the motherboard via SATA and the power connection has been sort of "re-jigged" from a SAS connector to the more common SATA power connection (I appreciate this is a bit unorthodox but someone told me these were backwards compatible??)

Re the hardware logs - short answer is no, mainly because a) I have no idea what they are and b) would have no idea how to check! - any advice on how to check/delete these would be appreciated!

Also, the only USB device connected is the one that has the unRAID OS on it!?!?

BaroUK2 · 30 Sep 2017 at 11:51

Before removing anything lets clear hardware logs to make sure your hardware logs are clear and any new log entry is for current running state. Take a note on recent entries as well.
See following link on clearing hardware logs...Verified answer is the correct solution...
http://en.community.dell.com/support-forums/servers/f/906/t/19470371

As for power from SAS connector it should be fine as long as you manage to fix this error otherwise that could also cause this issue and might need to remove that to test barebone system...

StevieP · 30 Sep 2017 at 14:49

Before I carry on it's probably worth mentioning that the system actually has 2 PSU's - I've only been using one because the guy I bought the system off said it would work fine with just one connected - anyway I've swapped to the other PSU and slot - not sure if this is relevant but I thought it was worth a try!?!

Anyway, I've managed to clear the hardware logs - the last few before I cleared were as follows: (events 118 to 121 - before this the only logs were for "intrusion" - which I think were for when I've had the side off the machine to poke around!)

[/IMG]

Also, I left memtest running earlier and it went through a single pass with no errors!

Where should I go from here??

Thanks for the continued help!

Ricimer · 30 Sep 2017 at 14:51

I have seen this once before, if it is actually the regulator, do *NOT* put any other CPUs in it to test. If it does fail completely, do not move the cpu to anything else.

I didn't get it fixed, as it would probably need board, cpu and psu replaced. I also ended up with another dead test board and two fried CPUs.

StevieP · 30 Sep 2017 at 15:16

The problem with leaving it and hoping is that I'll always be nervous about the system failing and losing all of my data!

neil_g · 30 Sep 2017 at 15:33

funnily enough I had the same issue with a R720 this week. seems to have sorted itself out, possibly poor input volts from the UPS as the input was reading lower on PSU 1 than 2. Dell support took the iDRAC logs and suggested that as the fault had cleared itself to update the firmware on the R720 PSU (yes I know, apparently the PSU has firmware). Not sure if that's an option for you.

StevieP said:
system failing and losing all of my data!

do you not have a backup copy of your data?

BaroUK2 · 30 Sep 2017 at 15:52

You can try draining all power from your system and if that doesnt work we need to run the system barebone to rule out any other controller causing this error.
To drain system remove power cord, press and hold power button for 20 seconds. Plug power cord again and turn on system. Let us know how you get on....

StevieP · 30 Sep 2017 at 15:59

neil_g said:
Not sure if that's an option for you.

do you not have a backup copy of your data?

I wouldn't have a clue where to start regarding firmware updates to be honest and yes I do have a backup of critical data but this server is supposed to be replacing my NAS and if it's falling over after a single night then it's not looking too good for stability!

BaroUK2 said:
You can try draining all power from your system and if that doesnt work we need to run the system barebone to rule out any other controller causing this error.
To drain system remove power cord, press and hold power button for 20 seconds. Plug power cord again and turn on system. Let us know how you get on....

Will give this a go and report back - thanks to you both!

StevieP · 30 Sep 2017 at 16:09

I've done the power drain reset (although I'm not quite sure I understand how holding the power button does anything while there isn't any power going into the system but I'll take your word for it!

)

Is there anything I can/should do to test the stability now other than just leaving it running? I've got unRAID running but without the array being started - I assume this is still testing the system without the risk of any random shutdown corrupting any data? - Maybe leave memtest running for a longer period??

BaroUK2 · 30 Sep 2017 at 16:36

Capacitors hold electric so holding the power button makes sure they are all drained properly : )

You can try prime95 or AIDA64 if you have windows installed, keep an eye out on your temperatures.
Alternatively there are linux distros for stress testing but i find using AIDA64 easier then distros....

neil_g · 1 Oct 2017 at 09:01

A quick Google suggests it could be any number of issues, this error has been triggered in other Dell systems by faulty PCI Risers, or hard drive back planes for example.

As it's a second hand unit and you don't have the luxury of firing iDRAC logs to Dell, you're going to have to do some good old detective work to try and narrow it down.

It could be:

Bad input voltage (AC supply, PSU, Power distribution board, motherboard)
Faulty component (PSU, Power distribution board, motherboard, PCI card(s), SATA/SAS backplane etc etc)

Personally, although not that it matters as you have backups, I'd be pulling any disks from the unit and be 100% sure the unit is operational before trusting it.

Presumably the previous owner had no issues, was it shipped and has anything come loose in transit etc etc.

StevieP · 1 Oct 2017 at 13:14

The previous owner didn't tell me about any similar issues (and to be fair the hardware logs didn't suggest any similar issues dating back to March(ish) if I recall correctly!)

It was a local pickup so nothing too traumatic in terms of transit!

I ran memtest overnight with no problems (or errors) and I've left unRAID running today, albeit without the array running so hopefully no risk to the disks!

I've currently got it running with the other PSU and I've put a 5V fuse in the plug now - just not sure how long to leave it before giving it a go with drives in!?! (I presume there will be more stress on the PSU when I add a couple more drives and can't think of a way to simulate that without physically connecting them!)

neil_g · 1 Oct 2017 at 13:37

as above something like prime will load the machine up.

but as i mention it may not be related to load. wouldnt expect it to be RAM related personally.

also do you mean 5 amp fuse?

StevieP · 1 Oct 2017 at 13:49

Yeah sorry, 5 amp fuse! (doh!)

StevieP · 3 Oct 2017 at 19:17

Quick update on this - server has been up and running for 2 days & 11 hours - seemingly without any issues!

I'm still just running the single 2TB disk in my array but I have installed Plex and messed around with Krusader within unRAID - not sure how "power-hungry" these applications are though!

BaroUK2 · 3 Oct 2017 at 19:25

I think you will be fine, power usage depends on CPU transcoding you will do in Plex. Typical single 1080p transcoding will use roughly 1/3 of your cpu processing power.