I built a franken-server

Associate
Joined
19 Jul 2011
Posts
2,343
Graduated from a microserver to running a Fujitsu server which serves as a host for various things; pfsense, xigmanas, pihole, Plex server and various Linux and windows VMs running on ESXi.
It was old, DDR3, and the Xeon is a lowly 1220, 4 core. And it's not a pleasure to work on when I want to change anything.

So I've been umming and ahhing about buying a more modern (but not new) HP or Dell. Then I stumbled across something a bit niche.


It was a Datto server which is really an ASRock rack, in a 1u pizza style case. Xeon-D2143 CPU, 64gb ram and a smattering of drives, spinning rust, SSD and nvme.
Plus it has two 10GBe, has built in remote KVM and management and enough onboard controllers for twelve drives.
And at 400ish quid I figured I couldn't build one for that.

Downside was it was noisy as **** being designed for a server room. But that's fine cos I had a plan.

The existing server had 4 drives in a raid 0+1 and an SSD, and this server came with 2 more, plus the SSD and nvme drives.
So I needed a case to take a menagerie of drives.



I gutted the Datto, and built it back up in a Fractal Node 804 case.
It's small but boxy, it's got a rudimentary fan controller to keep the noise down. And no RGB. Which is a plus point in my head.

Fired it up ok on the first attempt which was reassuring. TrueNAS found all the hardware and reported all ok.
What it also uncovered was a heat problem from doing the transfer to a new case.
The noisy ass 40mm fans were directed over the CPU which because this is a SOC also contains the drive, network and various other bits normally farmed off elsewhere on the motherboard.
But in the fractal it was hitting 85c pretty quick.

So I looked around for replacement coolers, looked on YouTube etc.
And in the end hotglued a spare 80mm fan straight onto the heatsink.
35c all day now and no discernible noise.

So plan then was to get VMWare up and running, transfer the drives from the old into it and see how it goes.

I had some upgrades on the way, 4x NVMe drives to see how fast an array it will support. The board does bifurcation and I grabbed an Asus card off eBay for cheap.
Voila..

4x 2TB NVMe usable directly by VMware, or presented to VMs using an RDM

And then I may try and find a cheap 10GBe switch, although most server load will be onboard VMs I think. But I love to tinker and this has scratched an itch that gaming simply isn't right now.
 
Last edited:
Associate
OP
Joined
19 Jul 2011
Posts
2,343
I've been watching SSD prices plummet recently, the NVMe drives above came from the WD online store, with a big ol' discount on 2 or more and then I decided to also pounce on a load of SATA 2TB SSDs at eighty notes each as well from elsewhere.

I've successfully passed them thru VMWare to TrueNAS, the NVMe's into a ZFS stripe+mirror just for doing some amateur speedtests and the SATA disks went in a ZFS n+2 array.
Which has given me 4TB of superfast Virtual machine storage and 12TB of fault tolerant storage all running silent and at decent speed.

The other thing I did was swap out the Fractal fans for some PWM fans, because the motherboard couldnt speed control them. So the whole box runs silent apart from the minature power supply.
I'll deal with that in another payday.
 
Associate
OP
Joined
19 Jul 2011
Posts
2,343
Variation of this motherboard by the looks of things:

Nice hardware with a choice of different mezzanine network cards including 40Gb

Almost identical by the look of it. I'm still on 1GBe switches for now, but the speed and connectivity is fine for what I'm playing about with - throwing up VMs and practicing installs and teardowns of ERP software.
 
Associate
Joined
18 Dec 2007
Posts
169
Location
London
I bought the same server a little while ago.
This weekend i'll finally have some time to tinker with it.
Apparently asrocks bios 1.09 works on it, I might give it a try.

It currently lives in a phanteks p400 untill I get the Meshify 2.

I did buy Noctua A6x25 PWM for the cpu, but it wouldn't spin up?
Did you change anything for pwm to work? Or do you run all the fans from the fan controller?
Currenlty I have the Noctua A6x25 flx cabletied to the cpu, after replacing the cooling paste.

Trying to find the cpu cooler from Supermicro is a bit of a pain, it keeps getting delayed to 2-3 months now. I might just cancel it and keep the cabletie solution.

My original plan was to go with a proxmox/truenas setup. With the 8port sata controller passed through, but for some reason proxmox wouldn't enable iommu for pass-through?
Dont know if you had any issues with it?

After trialling unraid, i might go down that road instead.
 
Soldato
Joined
29 Dec 2009
Posts
7,175
I bought the same server a little while ago.
This weekend i'll finally have some time to tinker with it.
Apparently asrocks bios 1.09 works on it, I might give it a try.

It currently lives in a phanteks p400 untill I get the Meshify 2.

I did buy Noctua A6x25 PWM for the cpu, but it wouldn't spin up?
Did you change anything for pwm to work? Or do you run all the fans from the fan controller?
Currenlty I have the Noctua A6x25 flx cabletied to the cpu, after replacing the cooling paste.

Trying to find the cpu cooler from Supermicro is a bit of a pain, it keeps getting delayed to 2-3 months now. I might just cancel it and keep the cabletie solution.

My original plan was to go with a proxmox/truenas setup. With the 8port sata controller passed through, but for some reason proxmox wouldn't enable iommu for pass-through?
Dont know if you had any issues with it?

After trialling unraid, i might go down that road instead.

What Supermicro cooler is it you're after? I can probably help get one quite quickly.
 
Associate
OP
Joined
19 Jul 2011
Posts
2,343
I bought the same server a little while ago.
This weekend i'll finally have some time to tinker with it.
Apparently asrocks bios 1.09 works on it, I might give it a try.
If you got it around the price I did, then its an absolute bargain. I found it by chance following some youtubes about building a NAS.

It currently lives in a phanteks p400 untill I get the Meshify 2.

I did buy Noctua A6x25 PWM for the cpu, but it wouldn't spin up?
Did you change anything for pwm to work? Or do you run all the fans from the fan controller?
Currenlty I have the Noctua A6x25 flx cabletied to the cpu, after replacing the cooling paste.
The original fans from the Node case werent PWM, so they worked but at 100%.
When I switched to PWM fans, they worked fine again at 100%, so I have set the fan speed [duty level?] in the bios manually, as I couldnt work out how to do it thru IPMI or BMC.




Trying to find the cpu cooler from Supermicro is a bit of a pain, it keeps getting delayed to 2-3 months now. I might just cancel it and keep the cabletie solution.

One of the youtubes I saw bought an old cheap cooler [AMD opteron cooler?] from aliexpress which had a u shaped bracket, which seemed to be fitted when pushed into the fins on the heatsink.
And they just fitted a regular fan to that.

I saw decent enough temps with a fan pushed by hand against the heatsink, and ended up just using a gluegun to secure it - the glue blobs can be peeled off easy enough if needed.

Nothing wrong with cableties if you're not entering a beauty contest :)

My original plan was to go with a proxmox/truenas setup. With the 8port sata controller passed through, but for some reason proxmox wouldn't enable iommu for pass-through?
Dont know if you had any issues with it?

I used VMware to passthru the Drives, not the controllers.
I found I needed a separate card to passthru controllers in the past [not least because the motherboard controller was needed by the OS for booting up], but saw no performance difference with passing thru a drive to a VM from the old way of passing thru a controller. I havent used Proxmox, although I may be dabbling in Oracle Virtual Server at some point.
 
Associate
Joined
18 Dec 2007
Posts
169
Location
London
If you got it around the price I did, then its an absolute bargain. I found it by chance following some youtubes about building a NAS.


The original fans from the Node case werent PWM, so they worked but at 100%.
When I switched to PWM fans, they worked fine again at 100%, so I have set the fan speed [duty level?] in the bios manually, as I couldnt work out how to do it thru IPMI or BMC.






One of the youtubes I saw bought an old cheap cooler [AMD opteron cooler?] from aliexpress which had a u shaped bracket, which seemed to be fitted when pushed into the fins on the heatsink.
And they just fitted a regular fan to that.

I saw decent enough temps with a fan pushed by hand against the heatsink, and ended up just using a gluegun to secure it - the glue blobs can be peeled off easy enough if needed.

Nothing wrong with cableties if you're not entering a beauty contest :)



I used VMware to passthru the Drives, not the controllers.
I found I needed a separate card to passthru controllers in the past [not least because the motherboard controller was needed by the OS for booting up], but saw no performance difference with passing thru a drive to a VM from the old way of passing thru a controller. I havent used Proxmox, although I may be dabbling in Oracle Virtual Server at some point.

I had a good play around with it over the weekend.
It seemed that the uid button was stuck on the io shield. Causing a few issues, one of them was hw monitor not loading up in the bios. I wonder if that stopped the pwm from kicking in.
Will have to try it out with a cheap pwm fan at some point. The current 6cm fan keeps it cool enough.

I'll be ordering the node 804 case, placing the stock fans in the hdd area and pwm in the mb compartment. It looks like its a great nas case.

Using ipmi, you might have to block outgoing traffic. I've read that the Datto firmware will call home.
 
Associate
OP
Joined
19 Jul 2011
Posts
2,343
The node 804 is great for lots of 3.5in drives. But only has delivered space for 4 - 2 near the mobo, 2 under the front panel.

Nice tip about the Datto firmware dialling home, I'll have a little look on my router-firewall and see if I want to block it
 
Last edited:
Associate
Joined
18 Dec 2007
Posts
169
Location
London
The node 804 is great for lots of 3.5in drives. But only has delivered space for 4 - 2 near the mobo, 2 under the front panel.

Nice tip about the Datto firmware dialling home, I'll have a little look on my router-firewall and see if I want to block it
I havent plugged in the IPMI yet to have a look around in it.
Only used the ipmi tool in unraid.


Apparently the IPMI is calling home to one of the following domains every 8 seconds or so:
dlt-rly-tracker-1.datto.com
dlt-rly-tracker-2.datto.com
dlt-rly-tracker-3.datto.com
dlt-rly-tracker-4.datto.com
dlt-rly-tracker-5.datto.com
dlt-rly-tracker-6.datto.com
 
Back
Top Bottom