The only person that I see pushing the needing more than 4GB is Kaap and his examples consistently get shown up. Shadow of Mordor and Xcom 2 use way over 4GB of memory at 4k, if you enable the uncompressed textures. If you use the setting below which is identical quality and compressed textures and uses below 4GB and runs great on Fury X.
Games don't generally come with uncompressed textures, it's just a waste of space, bandwidth of downloading, storage space on an SSD and waste in efficiency of processing more data for absolutely no reason.
These uncompressed textures appear completely by magic in Nvidia gameworks games and when Nvidia guys on forums like to run around telling everyone you must have more than 4GB for 4k.
There is absolutely no reason at all to use uncompressed textures. For photo editing, in photoshop with images using several gigabytes of data, absolutely uncompressed raw data. For game textures, none, absolutely none.
If people can't contain themselves and use what is essentially identical but more efficient settings that is really down to them.
Effectively in Shadows of Mordor, high textures are the highest quality and Ultra is just a less efficient version of high textures, nothing more or less. So would Kaap also not recommend newer drivers which are more efficient in games, because he's purposefully telling everyone they need more memory just to use a less efficient setting within a game.