If you're playing ultra graphics settings, knock the shadow level down *1* notch, very little change in appearance but about 1/3-1/2 additional FPS.
WoW is an excessively CPU orientated game due to the age of the coding [the core engine is from approximately 2001-2004], especially at higher resolution or when involved in higher group number instances/areas.
Blizzard have added more and more coding onto the original engine and since WotLK it has been heavily showing it's age; and the game itself is only really optimised for 2 cores, with the 1st being pummelled, even on a 3.4Ghz Q6600, as Cores 3/4 aren't used.
Addons which take up CPU runtime, like QuestHelper (BIG fps loss from this one when active!) just make this worse and worse.
There are also reputedly issues with the games hardware efficiency/scaling, especially with new systems (and both ATI/NV cards), whereby the game is NOT making the most out of various aspects of the system (less than 100% GPU usage for example), the underlying problem here really is Blizz's coding, before the next expansion, they really could do with a large scale overhaul or optimisation focus on the existing code, above and beyond new content, because as it is, the game is performing worse and worse for the majority of the playergroup, except those who have excessively high machine specs, or fiddle around with a lot of graphics settings and resolutions to find the best setting for thier own hardware.
Servers theoretically shouldn't affect game performance either, but people have reported changing to different servers in different locations again also affect the game; which again points back to an issue with the coding of the game (which to be fair is likely very, very, very complicated).