If you're gaming on a 50" display, then you'll be to far away from the screen to notice the extra eye candy 4GB of VRAM might afford you anyway.
Defo the opposite IMO. Bigger screens = bigger pixels = bad IQ. You would need to turn up settings like AA to get a similar IQ to a 21 inch with a 50 inch. Smaller screens are much better.