GPU for AI (considering RTX 3060 12GB)?

[Rikmeister]

I'm interested in getting a GPU for running AI apps locally (mainly text and image). As I understand it I would best consider nVidia cards with as much vram as possible. Given a very limited budget (up to about £250) I could probably run to a second hand RTX 3060 12Gb card - but I'm open to other suggestions.

Is there to choose between different 3060 cards? I mean they will all have the same number of Cuda cores and Tensor cores and things like ray tracing are irrelevant.

NB. I don't play games and the only other intensive GPU activity I might undertake would be some video editing (probably not 4K in the foreseeable future)

Thanks in advance

 

[Rikmeister]

I'd have a look at the A770 too, since articles like this suggest it could be quite good if drivers/apps are optimised to take advantage (Arc has additional hardware like nvidia does).

That's an interesting suggestion. The A770 has a generous 16GB Vram, however it was never widely available in this country. Whereas the price in the US seems to have fallen nearer to $200 here in the UK it is over £300.

 

[Rikmeister]

I would suggest getting the RTX 3060 12GB as well for that budget you're rocking. I believe tests done so far show that the 4060 isn't any better, so you'd need to head to the next band up to get any more performance and capability out of it (more VRAM).

Whilst Intel and even AMD could be jumped onto for AI, they have more hoops to jump through. So if you're just starting off, the Nvidia cards would be an easier starting off point as many more software is compatible with them over the other (edit) GPUs (/edit), which may require work to get going, or in some cases, is near impossible to get going without a re-write of the software (edit) to support them.

:: edit ::
The 3000 series is suggested because of the position it has rooted itself as for AI software; it was the latest widely supported GPU generation for a while. Whilst 2000 series is supported as well, the issue is they lack the VRAM and power available and updates to improvements from what is available from the 3000 series. The 4000 series has largely been seen as a minor side grade to the 3000 series, which means if the silicon doesn't have the performance boost (or enough VRAM) then there's only marginal improvements on the most used AI software. Plus there's the price premium on the 4000 series over the 3000 series. So if you are just starting off, and with that budget range you're looking at, if you're not looking to do much debugging and getting into things, then the Nvidia 3000 series GPUs would be your best bet I say. (Although remember to get as much VRAM as possible).

Really good advice. Thanks.