It think I am going to have to send it back. Just done some more testing in Heaven, 3D Mark 11 benchmark and any game I play its fine.
If I run Valley Benchmark, or 3d Mark 11 Demo or 3d Mark with SkyDiver it artifacts bad or in the latter just crashes and locks up. I get blue/green flashes in the 3d Mark demo and in the Valley I get white flashes on the textures on the hills.
So I googled it and found several examples of the same on Gigabyte 780ti's. I even found one guy who had done a youtube video of the 3d Mark 11 demo with his artifacting and mine had the same issue in the same places his did apart from his flashes were green and mine were blue. He managed to fix it by downclocking the memory to 1698mhz. I tried this but everytime I push the memory slider to the left and hit apply it goes back to 0 again. So I cant downclock the memory. I downclocked the GPU to 915mhz but this made no difference.
I put my R9 290 back in and its fine in everything.
Strange thing is I play Titanfall or Far Cry 3 and its fine.
I have a Gigabyte GTX 780Ti WindForce 3x OC and don't draw too much from benchmark performances, as the overclocks I can set aren't always the same and differ from benchmark to benchmark. The only two sharing the same clocks for benches are Heaven and Valley.
The highest I have benched my card (air-cooled) in those two is 1250/2009, but in 3DMark Skydiver it is 1241/1750. If I did try to move the memory clock from stock it would crash like you've experienced. I get the same in Assetto Corsa. 3DMark 11 is another benchmark that doesn't like it when there are high clocks set on my card unlike those I can attain in Firestrike. In some games, like Wolfenstein and GTR2 I have run the card at 1241/2000 without any noticeable issues.
The flashing in the Valley benchmark from what I have read is generally recognised as an issue with nVidia drivers released after 337.50 affecting the GK110 GPU. Mine has been displaying this glitch on every driver release, including some more unusual betas like the iCafe 337.82 & 337.90 drivers. There will be a fix, eventually.
Also why not give the GURU II OC tool a try?
It's funny but I don't like either MSI Afterburner or EVGA precision tools. I found the Gigabyte tool to be much simpler and do what I need. If you want it to load on start up, just make sure it is set to run as administrator and a shortcut placed in your start-up folder. That way it will load any saved overclock profile and fan profile for you. It will even notify you that it has loaded an overclock and you can click the notification to return to defaults if need be.
Currently on 1241/1750 so I can play Assetto Corsa without lock ups. If it were GTR2 then it would be at 1241/2000.
I've been happy with mine and it does what I need it to.