For me, the atmosphere of the game trumps the plot and gameplay. However, to a certain extent both of these contribute heavily to the atmosphere, along with the visuals and the music.
Example: Bioshock felt like a well-defined world - the plot was decent, but I didn't really pay much attention to it until about half way through. What I did pay attention to was the world I was in, the way the music and the gameplay enhanced that sort of creepy atmosphere.
Infinite, on the other hand - the world lost a bit of the brilliance that Bioshock had, but I felt like the gameplay was marginally improved, and the plot massively so - to the point that for me, both games balance one another in terms of how much I enjoyed them.