Harshness is a result of peaks, exaggeration in the response, distortion or a combination of those.
There's no such thing as a pure sounding headphone, amp, dac. When it comes to neutrality, the most recommended equipment for it is always on the warmer side rather than the colder side and I agree that warmth is natural..
When I look for something natural, it must never sound out of place, coherency, tonality, how human voices sound, if I hear harshness then it's a result of one of the above I mention or all of them. Buttery smoothness is something I enjoy because it's euphoric, relaxing, intimate but buttery smooth can also be natural. There's too many headphones, amps that focus on extracting detail at the expensive of peaks in the response causing harshness/roughness and so many enthusiast have it in their head that this is neutral.
If something sounded dead flat, it would bore most to tears I imagine but it's impossible to achieve that. I believe warmth is reality. I guess it's a personal thing so what I believe to be natural may be way off what you believe.