AI card

qbhatti · 12 Feb 2025 at 14:48

Hi folks.

I am looking for an AI accelerator to run some alorithms locally - basically comparing photos provided by people to a known database of people to work out with reasonable accuracy their height/weight. We have the code and it runs well, but needs to be faster as we are trying to run it in real-time.
We have an R640 dual CPU server which is currently running a bunch of VMs on proxmox.

Cant work out what best PCIe solution we should use - I have heard of the A100 which is a bit too expensive, and V100 but no idea if better to just run a normal gfx card in the rig.

any ideas?

jigger · 12 Feb 2025 at 15:49

You’re probably going to limited by the size of card, power and cooling the Dell system can deliver.

LostCorpse · 12 Feb 2025 at 19:49

what recourcese are you running it on at the moment? cpu?

qbhatti · 12 Feb 2025 at 21:32

Currently used an old i7 7th gen as a test of principle.
The server has nice big 1500w psu's I think so I'm not worried about that.

Rob_B · 12 Feb 2025 at 21:47

I guess it depends on many things, what are you running? how often? for how many people? etc etc

jigger · 12 Feb 2025 at 22:51

qbhatti said:
Currently used an old i7 7th gen as a test of principle.
The server has nice big 1500w psu's I think so I'm not worried about that.

It’s typically not that simple. Dell should have documentation with supported lists of add in cards.

RSR · 13 Feb 2025 at 01:28

R640 are a 1U pizza box solution, normally when you running GPUs in the Dells they are in a R7X0 or larger chassis. We use R760’s with 3 GPUs in them, which are A40’s. So you may have limited options.

Dell sell a number of options but the A40 or newer model cards may work well for your solution providing the PSUs are spaced for that level of use.

LostCorpse · 13 Feb 2025 at 08:27

if GPU arent an option for some reason possible something like googles coral with an M2 PCI card

alterntive is is to split it of the server and run it on a dedicated box with a GPU.

RSR · 13 Feb 2025 at 09:22

LostCorpse said:
if GPU arent an option for some reason possible something like googles coral with an M2 PCI card

alterntive is is to split it of the server and run it on a dedicated box with a GPU.

The M2 card on Dells, is normally a BOSS card. I'd hazard a guess it doesn't have the space or the slots are taken up with NIC / BOSS cards etc..

Id recommend a R7X0 or larger chassis for dGPU setups, also make sure it has the upgraded PSUs with those as well as they can pull quiet a bit of power.

LostCorpse · 13 Feb 2025 at 12:07

RSR said:
Id recommend a R7X0 or larger chassis for dGPU setups, also make sure it has the upgraded PSUs with those as well as they can pull quiet a bit of power.

not to familiar with them, but it likely will need to order the PCIE riser at the same time even if not ordering the cards.
often dell dont ship the right PCI riser card unless ordering with a GPU in my expirnece or spesifically asking for the riser kit.

RSR · 13 Feb 2025 at 13:43

LostCorpse said:
not to familiar with them, but it likely will need to order the PCIE riser at the same time even if not ordering the cards.
often dell dont ship the right PCI riser card unless ordering with a GPU in my expirnece or spesifically asking for the riser kit.

There are a whole number of changes which take place when you order GPU setup. If you have a Dell AM, I'd advise you to speak to them about your kit as they will be able to talk you through the options.