Definitely, some of the covers were brilliant. My favourite was (spoiler towards later part of the game):BTW,the way they took 80s songs and made them sound turn of the century was great!!
The black lady sitting on the stockade podium singing a soulful rendition of CCR's Fortunate Son. See here. I wasn't just one of the best covers, it is one of the best moments in the game for me as it's such as powerful statement, especially from the setting it's being sung in.
Towards the later part of the game you understand that it was Albert Fink using tears to steal music from other universes and re-recording them as his own works for Columbia. I thought it was a nice idea to explain how the music ended up there.