Home server AI stack anyone?

TCabot · 27 Feb 2026 at 18:26

Hi All,

Skimmed through a few pages of threads and didn't spot any home server AI stack builds. Has anyone here built himself a home server for own AI stack?
I'm interested in assembling a budget rat rod server, so would really appreciate suggestions on minimum spec for minimum budget to keep it viable. The first build would be purely to test it out before I commit some serious spondoolicks to it. Ideally, start small and upgrade as I go.
Will most likely put ubuntu server on it.

From initial reading, GPU is key while I can probably save a few quid on CPU and motherboard?

Any thoughts or suggestions welcome.

Thanks!

Rroff · 27 Feb 2026 at 18:29

Personally I haven't gone beyond using a Dell QBM1250 with Intel 265 for AI related tasks (as an offload from my main systems) - for any serious local LLM use, etc. you quickly need video cards with a LOT of VRAM.

aaronyuri · 27 Feb 2026 at 18:39

AMD Strix Halo is king in this space. It is a cracking amount of AI compute for not much money.

TCabot · 27 Feb 2026 at 18:54

aaronyuri said:
AMD Strix Halo

Ouch, That's way outside my current budget. I was thinking more along the lines of a used RTX 3060 12GB.

This is mainly for the Mrs for AI testing - if it works for her, I'd treat her to something more powerful.

TCabot · 27 Feb 2026 at 19:46

Anyway, my current budget home PC build is happily running on B550 Ryzen 5500G,.

Am I too boring to consider a build on B450 with Ryzen 5500 or 5600x and RTX 3060 12GB?
What form factor to pick? mini-ITX in metalfish T60 perhaps? Would there be an appropriately-sized server PSU for this?

Thanks.

TCabot · 28 Feb 2026 at 10:22

I'm a bit shocked by RAM prices. I picked up a pair of new 16gb DDR4 corsairs for about £50 less than 2 years ago. 32gb RAM seems to cost £250 now. Wow. Should have invested my life savings into ram sticks in 2024.

Does anyone have build plans for a budget-friendly time machine? Happy to buy used parts and Chinese knock-offs.

Looks like I'll just re-purpose my current box into an ATX case to run some AI crunching on a new GPU.

Vestas · 28 Feb 2026 at 14:35

12GB on a 3060 isn't enough for anything other than a tiny model with a minimal context window. That might be OK for your purposes but it'd have to be VERY specific tasks with models suitable for that - you won't get much choice.

Anything under 32GB "VRAM" isn't going to be much use but I suppose YMMV...

TCabot · 6 Mar 2026 at 17:21

I've settled on a 16gb 5060 card, as I'll need a gpu for a kid's PC soon anyway.

At the moment, it's very specific text-only jobs, so I suspect 16gb will get me started OK.

Any suggestions on what to look out for in 24gb or 32gb segment? There seem to be a few older datacenter cards that need liquid cooling. Never done it before - how hard (& expensive) is it to fit a radiator?

Rroff · 6 Mar 2026 at 17:42

TCabot said:
There seem to be a few older datacenter cards that need liquid cooling. Never done it before - how hard (& expensive) is it to fit a radiator?

Some of these need a little care taken with heatsink/block mounting as they are direct die and too much torque will crack them. Generally not an issue with 2 finger tightening technique but they are intended for use with a torque screwdriver/wrench.