from the graphics card sure but from the cpu I think not
sure there are still less overheads but when your sitting in your os what is your cpu doing? %1 load at best most likely
You're simply underestimating the overheads and inefficiencies PCs have to deal with with regards to performance. This is what mantle is about, and it goes to show the sort of performance increases you can get by having direct hardware access.
The CPUs in the consoles will be fine.