Yeah! Not only are there games that aren't on the PS4 and play brilliantly on PS5 (Returnal, Ratchet & Clank: Rift Apart etc.) but you've also got the games that are on both consoles that just play so much better on PS5.
I held off on finishing Ghost of Tsushima on PS4 as I found the combat struggled/wasn't as fluid at 30fps - but on PS5 it's just phenomenal.
YouTube videos don't really do it justice, keep in mind you've always got a level of compression there and issues effecting quality. The PS5 is 3 years old and has sold 40 million units, it's very much fully in it's life right now!