Asus X99 WS - Being re-used as TrueNAS Server, power usage

[bikerchris]

Hey all,

My god I've not posted here for ages, hope everyone is well?!?

So the title should explain, I'm kind of interested in anyone else's experience with power usage.

Here's my spec:

OS: TrueNAS
CPU: E5-1650 v4
RAM 64GB ECC Registered
HDD:
1 x NVMe
2 x SSD
8 x HDD's (half are set to spin down when no activity)

I know that the hard drives are going to be using a fair amount of power, but just wondering if these figures sound about right:

124-125 Watts - Idle, no activity
125-130 Watts - Watching a Plex video (no encoding, original quality)
~165 Watts - Watching Plex with forced encoding

I've done what I can to lower the power usage, but it seems most energy savers require Windows to be installed...doh! I tried the EPU...right before I read in the manual that the Win software is required...poo!

Thanks in advance

 

[nox_uk]

The hdd’s will vary - for example ‘greens’ will be slower, but lower power, blacks at the other end. I have an x99 server, with x5820k, it’s juicy enough I only switch it on when I need it. I’ve pretty much decided to replace it with a couple of nucs and a NAS, but I want a few vm’s too. I figured I’m closing in on the time when electricity bill will cover they upgrade!

 

[bikerchris]

Thanks for that @nox_uk , yes one day I may give up on it and try some smaller units. I have a snapshot server that's i3-9100F based, and that isn't juicy at all, though I've yet to power test it.

 

[BongoHunter]

X99 boards typically have pretty good BIOS options, you could try lowering the CPU voltage or underclocking it slightly, is you RAM 2133 or 2400? You might be able to drop the speed without noticing a performance difference.

That power figure you have for idle doesn't appear crazy, as a rough guide a HDD is about 15w IIRC, and the X99 chipset + NVME + CPU is never going to get super low. You could also try disabling any other unused devices on the Mobo such as extra LAN controllers, audio etc maybe? Does the BIOS have a low power profile you could try?

 

[bikerchris]

Thanks for that @BongoHunter , I did have a look around the BIOS and did as much as I could. I think I even tried reducing the active cores, from 6 to 4, but I recall it didn't make much of a difference.

My RAM is 2133, good idea, I could try reducing it manually and I've seen that option within the BIOS

Crikey, does sound that most of the power usage is spinny things, when I rebuild, I'll try TrueNAS with just the OS drive plugged in and check the power usage for comparison.

Good idea about disabling unused devices...especially those silly serial ports! The BIOS does have a power profile, but it seems associated with the overclocking. I've disabled all auto overclocking, I could even disable the turbo mode I guess, in case it's running high at times.

Thanks again

 

[Rroff]

Beyond the scope of a quick post here but making sure the right S states are enabled/setup in the BIOS and tweaking the power plan in the OS will massively reduce idle power use on these chips without disabling cores, etc.

(Though if you aren't using Windows that is harder to accomplish)

You can usually massively undervolt the 1650 series of CPUs as well though it would need some stability testing.

 

[Quartz]

124-125 Watts - Idle, no activity

The price of electricity is about 17p per kw/h so that idle time is costing you about 50p per day. Is that really worth worrying about?

 

[bikerchris]

Thank you @Rroff , I have seen S States mentioned in the BIOS, I just remember last time I fiddled with it, it buggered the machine up. I'll have another look though, thank you!

 

[bikerchris]

The price of electricity is about 17p per kw/h so that idle time is costing you about 50p per day. Is that really worth worrying about?

It's a very fair point, it's not that much in the grand scheme of things, I'm just trying to control regular costs, and £182 a year would be better spent on my alcohol budget.

 

[nox_uk]

It's a very fair point, it's not that much in the grand scheme of things, I'm just trying to control regular costs, and £182 a year would be better spent on my alcohol budget.

Haha, yeah I’m in that boat too, but I think my alcohol budget is a fraction higher… can even consider things like cloud hosted options - but…. They will likely be a bit more, but you do only pay for what you need.

 

[bikerchris]

Welcome to my...boat! Yeah my alcohol budget is higher than that too

I did think about hosted, but I've got a lot of data at rest, as well as a lot of work related files that I like to keep offline (ish). I am using the server for a few other things at least, like wifi controller NextCloud/Syncthing (not made my mind up yet) and it's a Plex server too, so at least I'm getting a few uses out of the expense.

 

[sja360]

That CPU is listed as 140W TDP on the Intel site, i get there are C States etc to bring it to a lower frequency etc, maybe as the poster above said because Truenas cant adjust the cpu power its running a higher power consumption. Best option is to turn it on without the mechanical hdd, to see if that makes a difference.

 

[decto]

It does seem a little high.

I have a couple of X99 boards, one a Gigabyte X99 SLI and the other a Supermicro workstation.
Stripped bare with 10 core E5-2660V3, 16GB ECC Dimm and only a CPU fan they register ~38-40W at the wall on my power meters, almost nothing in it.
Adding more memory to 64GB only adds a few watts.
That is booted to Unraid install and includes a Nvidia GT710 passive as neither board has IPMI

The rest of the power is usually drives and fans, especially those that don't spin down.

I'm at ~110W idle, drives spun down acording to my UPS with 8 drives, 3x GPU (Quadro P1000, P2000) and GTX1650 installed +plex running etc.
Ramps up to ~140W using GPU decode for Plex and more if the VM's are running.

In Unraid there is a power management setting which cuts the idle power, not sure how that's applied in Truenas but the lower power states do need OS support.

 

[bikerchris]

Thanks @sja360 , yes I might try that. I did "un-standby" a few drives and noted the power went up around 5w per drive. If I'm feeling brave I'll unplug them all, though disabling those SATA's in BIOS might be a safer route perhaps?

That's very interesting @decto , thank you. I haven't seen many power options in TrueNAS, you have way more H/W in yours than mine, and yet less power?

I checked my supermicro i3-7100F setup, in fairness that's still running Win10, has 4xHDD's and the most basic of GPU's (GT210 or something?). That uses around 60W idle, with no HDD sleep modes active.

 

[FreeStream]

Seems like a lot although X58/X79/X99 did always have very high idle power usage.

My E3-1230V3, X10SLL-F, 16G, 8x 4TB (although now 4x10/14TB), 2x2.5" + 2xSSDs + Quad NIC + 660W Platinum PSU idled at 35-40W (30-35 without the NIC) when the disks were spun down, or 55-60W otherwise.

 

[bikerchris]

Seems like a lot although X58/X79/X99 did always have very high idle power usage.

My E3-1230V3, X10SLL-F, 16G, 8x 4TB (although now 4x10/14TB), 2x2.5" + 2xSSDs + Quad NIC + 660W Platinum PSU idled at 35-40W (30-35 without the NIC) when the disks were spun down, or 55-60W otherwise.

Cheers for that (sorry for delay, lot of building work), well perhaps it is fairly typical.

Oddly I now find it's far lower, though I have set a pool of 4 disks to standby when not used, so may be that helps.

 

[Rroff]

X79 V2 (Xeons) have fairly decent idle power use if setup right (the Xeons have a ton of clock/voltage headroom out the box). X58, X99 and X79 V4 aren't the most optimal if you want low idle draw.

 

[bikerchris]

X79 V2 (Xeons) have fairly decent idle power use if setup right (the Xeons have a ton of clock/voltage headroom out the box). X58, X99 and X79 V4 aren't the most optimal if you want low idle draw.

Thanks for an uber quick response. I did fiddle with the BIOS to lower the power, but oddly it seemed to break 2 x 8GB RAM sticks...it could have been coincidence of course, they were a few years old. To my knowledge, TrueNAS isn't great at adjusting CPU speed, though it's something I need to investigate further.