Are you playing batman?
Mine definitely gets louder when in game, and if I manually close the game from the switcher the fan slows down - expected tbh, there's still a fan in there, it's not a passive system.
The lowest volume on my headphones means I can't hear it, so not an issue. If I was to put it further away or not at head height I'd not be able to hear it either.
I sent mine back to Sony for an exchange. Mine had a gpu fault where screen flickering/glitch occurred on the Home Screen.