Dusting off an old PC - home server?

[iamgigglz]

My home server is currently a micro-PC running a 9th gen i5. It does brilliantly running Home Assistant, Pihole, all the Arrs and Plex inside Proxmox, but I'm running into trouble with transcoding as the micro-PC can't take a GPU. I'd also like to play around with a local LLM for voice commands.

I've been given an old X79 box with an i7 3820, 24GB of DDR3 and a nostalgia-inducing OCZ 650W power supply.

I have no idea how the IPC performance compares to modern stuff, but I figure a £50 graphics card and £30 for a spicier Xeon CPU would be a drastic upgrade from my current home server - or am I being optimistic?

 

[AMDPower]

This is really hard to answer, it really depends on what you are going to run for the LLM model, ideally to get anything to perform reasonably well you need something with at least 16GB Vram or at the very least 12GB, you might be able to squeeze it into 8GB with quantization but reduced accuracy would not advise as it gives little room for more better performing models. Maybe look for something like rtx 3060 or 4070ti if you want something that performs well, you could get AMD or Intel cards but personally myself prefer to stay on the NVidia ecosystem for AI related stuff but intel and AMD are also an option but their ecosystem is more narrow, make sure your model runs on them before buying. If the model spills form VRAM into system RAM it can slow things down drastically.

you have an option to run the whole model using just cpu & system ram but speed you get has many factors, from DDR ram speed, size and, number of memory lanes ect but is generally several orders slower than running everything in Vram.I would recommend getting a Nvidia GPU and running everything on there, the CPU is less important in this situation but still need decent CPU for other aspect of AI. I would also recommend nvme drives in general apart for long term storage/archives.

if you are big on AI, your next best option would be a mac mini with 16-32GB ram, its definitely another option and performance is quite respectable.