It's more likely, in my opinion, that high nitrates are correlated with high BOD (biological oxygen demand) through overfeeding, over stocking or poor maintenance, which causes CO2-induced respiratory stress in fish through the Bohr effect as a result of the nitrifying bacteria stripping large amounts of oxygen from the water in order to process all that ammonia and nitrite. Or even that polluted water reduces the steepness of the diffusion gradient for ammonia between the fish's bloodstream and the water column and ammonia is therefore expelled more slowly, causing damage over time.
I'm in no way disagreeing that having greatly increased nitrates compared to your tap water is probably a bad sign, or that water changes aren't almost always a good thing, but the toxicity of the nitrate ion itself is unlikely to be the major issue. As I said, it's a smoking gun. Seeing as anecdotal evidence is all we have to go on, one of the planted aquarium 'gurus' in the US ran a small experiment where he dosed nitrates to a steady 160ppm for three weeks and saw no detrimental effects on his fish, including (if I remember correctly) a breeding group of Sturisoma. Still not exactly rigorous, but it works as a demonstration.
And as I mentioned previously, getting accurate and repeatable measurements of nitrate at home is nigh on impossible. Or, for that matter, in a lab - I know a guy who works as a water quality scientist with access to thousands of pounds worth of lab equipment and he can't manage meaningful nitrate readings with all that, which is why most water pollution studies now measure BOD if possible (still too difficult for most labs) or fluorescence analysis of tryptophan as a measure for levels of dissolved organic matter.
Which is why I always say everyone should have plants in their tank (especially floaters or emergents with access to atmospheric CO2), because then you can completely forget about the entire issue and be sure that your water quality is probably OK because the plants are actively capturing and exporting nitrogenous compounds, heavy metals and all kinds of other crap.