Home server AI stack anyone?

TCabot · 2026-02-27T18:26:05+0000

Hi All,

Skimmed through a few pages of threads and didn't spot any home server AI stack builds. Has anyone here built himself a home server for own AI stack?
I'm interested in assembling a budget rat rod server, so would really appreciate suggestions on minimum spec for minimum budget to keep it viable. The first build would be purely to test it out before I commit some serious spondoolicks to it. Ideally, start small and upgrade as I go.
Will most likely put ubuntu server on it.

From initial reading, GPU is key while I can probably save a few quid on CPU and motherboard?

Any thoughts or suggestions welcome.

Thanks!

Rroff · 2026-02-27T18:29:11+0000

Personally I haven't gone beyond using a Dell QBM1250 with Intel 265 for AI related tasks (as an offload from my main systems) - for any serious local LLM use, etc. you quickly need video cards with a LOT of VRAM.

aaronyuri · 2026-02-27T18:39:48+0000

AMD Strix Halo is king in this space. It is a cracking amount of AI compute for not much money.

TCabot · 2026-02-27T18:54:58+0000

aaronyuri said:
AMD Strix Halo

Ouch, That's way outside my current budget. I was thinking more along the lines of a used RTX 3060 12GB.

This is mainly for the Mrs for AI testing - if it works for her, I'd treat her to something more powerful.

TCabot · 2026-02-27T19:46:33+0000

Anyway, my current budget home PC build is happily running on B550 Ryzen 5500G,.

Am I too boring to consider a build on B450 with Ryzen 5500 or 5600x and RTX 3060 12GB?
What form factor to pick? mini-ITX in metalfish T60 perhaps? Would there be an appropriately-sized server PSU for this?

Thanks.