Dusting off an old PC - home server?

[iamgigglz]

My home server is currently a micro-PC running a 9th gen i5. It does brilliantly running Home Assistant, Pihole, all the Arrs and Plex inside Proxmox, but I'm running into trouble with transcoding as the micro-PC can't take a GPU. I'd also like to play around with a local LLM for voice commands.

I've been given an old X79 box with an i7 3820, 24GB of DDR3 and a nostalgia-inducing OCZ 650W power supply.

I have no idea how the IPC performance compares to modern stuff, but I figure a £50 graphics card and £30 for a spicier Xeon CPU would be a drastic upgrade from my current home server - or am I being optimistic?

 

[AMDPower]

This is really hard to answer, it really depends on what you are going to run for the LLM model, ideally to get anything to perform reasonably well you need something with at least 16GB Vram or at the very least 12GB, you might be able to squeeze it into 8GB with quantization but reduced accuracy would not advise as it gives little room for more better performing models. Maybe look for something like rtx 3060 or 4070ti if you want something that performs well, you could get AMD or Intel cards but personally myself prefer to stay on the NVidia ecosystem for AI related stuff but intel and AMD are also an option but their ecosystem is more narrow, make sure your model runs on them before buying. If the model spills form VRAM into system RAM it can slow things down drastically.

you have an option to run the whole model using just cpu & system ram but speed you get has many factors, from DDR ram speed, size and, number of memory lanes ect but is generally several orders slower than running everything in Vram.I would recommend getting a Nvidia GPU and running everything on there, the CPU is less important in this situation but still need decent CPU for other aspect of AI. I would also recommend nvme drives in general apart for long term storage/archives.

if you are big on AI, your next best option would be a mac mini with 16-32GB ram, its definitely another option and performance is quite respectable.

 

[iamgigglz]

This is really hard to answer, it really depends on what you are going to run for the LLM model, ideally to get anything to perform reasonably well you need something with at least 16GB Vram or at the very least 12GB, you might be able to squeeze it into 8GB with quantization but reduced accuracy would not advise as it gives little room for more better performing models.

I was thinking of the Whisper Medium or Turbo models on an 8GB card

I would also recommend nvme drives in general apart for long term storage/archives.

X79 doesn't have nvme but I could also get a PCIe expansion card if I really needed it.

if you are big on AI, your next best option would be a mac mini with 16-32GB ram, its definitely another option and performance is quite respectable.

I'm talking about adding £80 of upgrades to a 10 year old PC that was given to me. A £450 Mac Mini isn't really the kind of step up I'm looking for.

 

[rare]

I’ve got a 3930k you can have for free mate gives you an additional 2 cores? Send me a pm with your address if you want it I will post it out

 

[Armageus]

My home server is currently a micro-PC running a 9th gen i5.

Which one in particular? (as there quite a lot of difference between e.g. a 35w 8400T and a 65w 8500 non-T)

I'm running into trouble with transcoding as the micro-PC can't take a GPU.

The integrated GPU on an 8th gen should be fine for Transcoding - I'm guessing the issue is you haven't got it passed through to the VM/Docker that Plex is running on.

I've been given an old X79 box with an i7 3820, 24GB of DDR3 and a nostalgia-inducing OCZ 650W power supply.

Factor in a new PSU as well, as that's likely well past it.

I have no idea how the IPC performance compares to modern stuff,

Not particularly great - using 95w less, an 8400T has higher single thread performance, and higher multithreaded performance.

Intel i7-3820 vs i5-8400T vs Xeon E5-2667 v2 vs i5-8500 vs i7-3930K [cpubenchmark.net] by PassMark Software

but I figure a £50 graphics card and

A £50 Graphics card isn't going to do much for LLMs - I'd imagine most don't even support the required level of CUDA for current models etc.

£30 for a spicier Xeon CPU

Xeon 2667v2 for £30 is probably the most cost effective Xeon - it gets you onto Ivy Bridge-E, 8 cores, and a 4.0Ghz boost frequency,

would be a drastic upgrade from my current home server - or am I being optimistic?

I think it's the wrong way to go personally - you already have a decent power efficient device - the transcoding issue can likely be solved. That just leaves LLM's which imo given how fast they are evolving, are probably best left in the cloud (and letting someone else burn money on hardware that is obsolete the second they buy it)

 

[iamgigglz]

I’ve got a 3930k you can have for free mate gives you an additional 2 cores? Send me a pm with your address if you want it I will post it out

Absolute legend. Thank you. PM incoming

Which one in particular? (as there quite a lot of difference between e.g. a 35w 8400T and a 65w 8500 non-T)

9500T - 35W 6C6T 3.7GHz

The integrated GPU on an 8th gen should be fine for Transcoding - I'm guessing the issue is you haven't got it passed through to the VM/Docker that Plex is running on.

It's fine but seems slow AF. You might be right about the passthrough - I'll check.

Factor in a new PSU as well, as that's likely well past it.

Looks a lot newer than the rest of the rig but your point stands. A more efficient PSU would probably be smart too.

Not particularly great - using 95w less, an 8400T has higher single thread performance, and higher multithreaded performance.

Yeah - 130W is quite a bit, although my hope is the CPU wouldn't be used much

A £50 Graphics card isn't going to do much for LLMs - I'd imagine most don't even support the required level of CUDA for current models etc.

I really only need a whisper model - either medium or turbo which need <8GB Vram. £50 might be optimistic tbf. Maybe a £100 RTX2060 or 3050. Might stretch to a 12GB 3060Ti to do image recognition later.

I think it's the wrong way to go personally - you already have a decent power efficient device - the transcoding issue can likely be solved. That just leaves LLM's which imo given how fast they are evolving, are probably best left in the cloud (and letting someone else burn money on hardware that is obsolete the second they buy it)

I get your point, but the latency is awful. You want snappy responses for voice recognition.

 

[SoundsGood]

Definitely check your passthough as I've had a 9500T working ok with Jellyfin transcoding (single stream).

 

[Armageus]

9500T - 35W 6C6T 3.7GHz

For some reason I read it as 8th Gen, but a 9500T is still a great chip

Yeah - 130W is quite a bit, although my hope is the CPU wouldn't be used much

It's not just the CPU that burns energy on older/workstation platforms though - the chipset uses an absolute truckload compared to newer desktop platforms. Even with the CPU at idle, I think you'll be looking at a total idle draw of 40-50w from a socket 2011 platform.

I really only need a whisper model - either medium or turbo which need <8GB Vram. £50 might be optimistic tbf. Maybe a £100 RTX2060 or 3050. Might stretch to a 12GB 3060Ti to do image recognition later.

Depending on what mini PC it is, do you have a spare NVME slot? If so there are some M.2 to PCI-E risers available - you'll have to MacGuyver a case and PSU for the graphics card, but just throwing it out there as an option.

I get your point, but the latency is awful. You want snappy responses for voice recognition.

Fair point, although again I don't know how good a cheap/low VRAM GPU fairs in this regard

 

[SoundsGood]

What about getting an ATX motherboard for your 9500T so you can use a proper GPU and probably add more RAM slots, you could use the case & PSU from the X79 setup

 

[AMDPower]

I was thinking of the Whisper Medium or Turbo models on an 8GB card


X79 doesn't have nvme but I could also get a PCIe expansion card if I really needed it.


I'm talking about adding £80 of upgrades to a 10 year old PC that was given to me. A £450 Mac Mini isn't really the kind of step up I'm looking for.

for whisper you can get away with 3060 8gb but personally myself would try and get something with a little more Vram if budget allows it will provide more wiggle room, but depending on what use case obviously , running AI locally and getting meaningful results has become cost prohibitive as time goes on, with nvidia removing Nvilink did not help either, running basic models is still possible but their utility apart from fringe cases can be limited. But its more sensible to pay for cloud based services unfortunately.

 

[iamgigglz]

What about getting an ATX motherboard for your 9500T so you can use a proper GPU and probably add more RAM slots, you could use the case & PSU from the X79 setup

This forum keeps shooting down my dumb ideas with sensible solutions.