Home server AI stack anyone?

TCabot · Friday at 18:26

Hi All,

Skimmed through a few pages of threads and didn't spot any home server AI stack builds. Has anyone here built himself a home server for own AI stack?
I'm interested in assembling a budget rat rod server, so would really appreciate suggestions on minimum spec for minimum budget to keep it viable. The first build would be purely to test it out before I commit some serious spondoolicks to it. Ideally, start small and upgrade as I go.
Will most likely put ubuntu server on it.

From initial reading, GPU is key while I can probably save a few quid on CPU and motherboard?

Any thoughts or suggestions welcome.

Thanks!

Rroff · Friday at 18:29

Personally I haven't gone beyond using a Dell QBM1250 with Intel 265 for AI related tasks (as an offload from my main systems) - for any serious local LLM use, etc. you quickly need video cards with a LOT of VRAM.

aaronyuri · Friday at 18:39

AMD Strix Halo is king in this space. It is a cracking amount of AI compute for not much money.

TCabot · Friday at 18:54

aaronyuri said:
AMD Strix Halo

Ouch, That's way outside my current budget. I was thinking more along the lines of a used RTX 3060 12GB.

This is mainly for the Mrs for AI testing - if it works for her, I'd treat her to something more powerful.

TCabot · Friday at 19:46

Anyway, my current budget home PC build is happily running on B550 Ryzen 5500G,.

Am I too boring to consider a build on B450 with Ryzen 5500 or 5600x and RTX 3060 12GB?
What form factor to pick? mini-ITX in metalfish T60 perhaps? Would there be an appropriately-sized server PSU for this?

Thanks.

TCabot · 2026-02-28T10:22:56+0000

I'm a bit shocked by RAM prices. I picked up a pair of new 16gb DDR4 corsairs for about £50 less than 2 years ago. 32gb RAM seems to cost £250 now. Wow. Should have invested my life savings into ram sticks in 2024.

Does anyone have build plans for a budget-friendly time machine? Happy to buy used parts and Chinese knock-offs.

Looks like I'll just re-purpose my current box into an ATX case to run some AI crunching on a new GPU.

Vestas · 2026-02-28T14:35:29+0000

12GB on a 3060 isn't enough for anything other than a tiny model with a minimal context window. That might be OK for your purposes but it'd have to be VERY specific tasks with models suitable for that - you won't get much choice.

Anything under 32GB "VRAM" isn't going to be much use but I suppose YMMV...