Yes gpu does have a bit of an impact on bigadv processing. Depends on the WU and depends on how beefy the processor is. But basically on a HyperThreading i7 you can count on losing the performance of one thread per GPU. When you get to multiple gpus you start to using apps to set affinity for the cpu and gpu, so you setup the folding client to only use 7 threads and then an affinity manager like winAFC to push the cpu client onto the first 7 cpu threads and limit the gpu client to the 8th cpu thread.
You can also see it the other way around when the gpu's don't have enough cpu allocated to feed them their performance drops as well.