You're completely looking at windows memory usage graph wrong as well, the graph itself only displays 'physical usage'. Underneath the graph it also lists how much additional ram is being used to cache on top of that, and how much ram is still free.
Most of the additional system ram will be used up fpr caching everything including Vram data and windows leaving next to nothing free. You need far more than 4 Gb to allow this to happen so you don't get lag spikes.
Most of the additional system ram will be used up fpr caching everything including Vram data and windows leaving next to nothing free. You need far more than 4 Gb to allow this to happen so you don't get lag spikes.