A lens with f2.8 is a trade-off: higher prices and softer images when shot at 2.8, in exchange for the versatility to handle a wider range of shooting conditions. If you're chasing maximum sharpness, a fair rule of thumb is to use somewhere around f8 (some lenses are sharpest slightly above or slightly below that, and it depends on camera body too). And then if you are aiming for f8, it does negate the value of an f2.8 lens to some extent.
Sometimes an even bigger problem for long-distance sharpness can be the atmospheric conditions, no matter what lens you're using. Post-processing helps a little there for some kinds of haze, but heat-distortion can throw all kinds of bulges and ripples into a distant mast that are not simple to correct.
Of the lenses you mentioned, the Canon seems to be well regarded for sharpness, and might give you enough of an improvement to be noticeable over your current lens. A quick search for samples from the others shows they can suffer more from chromatic aberration and corner softness.
As Scurra says, the f2.8 (II) version of the Canon is ridiculously sharp, and I've read a few reviews in the past that say the f4 lens is almost as sharp. The problem with picking a single prime for shooting distant buildings, etc. is that you might not have enough leeway with your vantage point to be able to move around and get the framing you want, so personally I'd go for a zoom (or a selection of primes, of course!).
