Caporegime
I noticed today that If it does get to the full 4GB being used in BF4 i would see some spikes that froze me for around 1-2 seconds. (rare though)
Alt-Tab out then back in and it drops the vram to 3.2ish and fine again till the next round.
Seems there is a slight memory leak
Its a BF4/Mantle bug, only dice can fix it.