Ironically Deep learning is largely constrained by half-precision FP16 performance (and in some cases even Int8), FP64 would never be used for deep learning. that is why a big performance gain for Vega is enabling double speed half-precision FP performance. Double precision is important in many other applications of GP-GPU/HPC.
I have a feeling that in the future Nvidia will actually separate out a new ASIC for deep-learning that focuses on FP16, int8 and FP32 performance without the FP64 hardware. Strip away more of the graphics functionality, and concentrate on pure efficiency. Google's deep-learning accelerator and Intel's new FPGA offerings are a big threat to nvidia.
I was thinking the same. A more specialised design makes a lot of sense, and NVIDIA certainly have the R&D and money to do that, while having the separate Quadro and GeForce lines without issue.
Will be interesting to see what AMD talk about in regards to RadeonPro come Saturday at the NAB Show; since they're partnering with Dell and HP there. Might shed some light on Vega, even if it's the Pro line.