That's true, polaris has dedicated hardware to help with Tessellation.Witcher 3 is fine on Polaris though - I tried it with my same old ancient CPU two weeks ago.
I was getting around 45FPS with an RX470 4GB at those settings with tessellation to the max and Hairworks to the max(GTX1080 is 80ish FPS). An RX480 8GB should be somewhat faster due to the 15% extra shaders,more VRAM and higher speed RAM.
Remember this is not with a very new CPU either.
The Polaris cards definitely seem to handle tessellation and Gameworks effects better than older ones.
Even then Hairworks hits performance so damn hard. I'll never forget how great it felt to get SLI 980Ti so I could max the game at 1440p with Hairworks.
Then came Tomb Raider and Pure Hair, a 4fps hit for better hair was great.