CUDA programming question

[Cromulent]

I have an Nvidia 1080Ti in my main computer which is currently running Linux. I also have an old 980Ti in a computer that doesn't need such a powerful graphics card. I was thinking of getting a really cheap graphics card to replace the 980Ti in the old computer and then putting the 980Ti in the same computer as the 1080Ti.

Would I be able to use both the 1080Ti and the 980Ti in the same computer? Would that result in better CUDA and AI performance?

 

[AHarvey]

I've not heard of CUDA but it looks interesting, what are you using it for?

I have no idea if CUDA will be able to use 2 different cards in a single machine, it'll be interesting to find out though.

https://developer.nvidia.com/cuda-faq
Yes to multiple GPU's but no mention of mode;

https://stackoverflow.com/questions/36758522/cuda-multiple-gpus-all-gpus-the-same-model
Confirms they can be different.

Not sure on performance though but I guess it depends how you've written the code. One would assume that more is better in this case regardless of them being different.

 

[Cromulent]

I've not heard of CUDA but it looks interesting, what are you using it for?

I want to do some AI stuff. Plus I just switched over to Linux from Windows 10 and one of my pet projects was doing GPU passthrough to a Windows 10 virtual machine and for that you need two GPUs (one for the host and one for the guest) so I thought I might as well use the 980Ti when I wasn't running a virtual machine.

I know you can accelerate some AI tasks using CUDA so I thought I'd get into that field a little bit as I don't know much about it.

I have no idea if CUDA will be able to use 2 different cards in a single machine, it'll be interesting to find out though.

https://developer.nvidia.com/cuda-faq
Yes to multiple GPU's but no mention of mode;

https://stackoverflow.com/questions/36758522/cuda-multiple-gpus-all-gpus-the-same-model
Confirms they can be different.

Not sure on performance though but I guess it depends how you've written the code. One would assume that more is better in this case regardless of them being different.

Thanks for the links. That makes me feel better about doing this. I dread to think what my electricity bill is going to be though .

 

[D.P.]

Yes, you can use different GPUs. You sometimes have to be careful of the software you are using, and to amke life easy you need to run the lowest CUDA version so you might not get features of Pascal enabled.
With TensorFlow it is generally quite easy.

 

[NickK]

I have an Nvidia 1080Ti in my main computer which is currently running Linux. I also have an old 980Ti in a computer that doesn't need such a powerful graphics card. I was thinking of getting a really cheap graphics card to replace the 980Ti in the old computer and then putting the 980Ti in the same computer as the 1080Ti.

Would I be able to use both the 1080Ti and the 980Ti in the same computer? Would that result in better CUDA and AI performance?

I've been doing GPU programming for years - OpenCL in my case and ATI's CloseTotheMetal before that.

The question here is implementation.

Firstly. Data Upload is slow but the killer is data download transfer speed from the GPU into memory.
Having two GPUs on the motherboard may work but if the north bridge is designed for one GPU's access path to the memory bus at one time.. then this will be a bottleneck.
So if you can keep both the data you need and the results in the GPU memory on the card then you're laughing, if you have to transfer data between memory and GPU you're not going to really get top performance .. worse if you have to keep syncing data between GPUs.

How the code and the algorithm is designed makes all the difference. Both in parallel speed but also keeping that parallel model including low dependencies on the main memory or syncing. For example I have code where I upload camera images into a GPU 15 fps, then process using OpenCL and then render the output into an OpenGL texture to render to the screen the final image.

For AI - yes, however the implementation of your AI is something that needs tuning to the architecture of the host for best performance.

I take it this is deep learning or convolutional networks? If so - GPUs will speedup.

You can code to handle load sharing between GPUs yourself but typically the processing is done in parallel then execute the next kernel based on the output of the previous - if you have a slow GPU then it may end up slowing because your CPU code will have to wait for the slower GPU result of that step.