If ambient is too high to keep it solid (i.e. easy to eat): yes. Otherwise, no.
Aside from the obvious textural difference, many of the flavors we perceive in food come from aromatic compounds that are released when food is warm. These compounds become less volatile and less likely to reach our olfactory receptors when food is cold. You can of course get around this by melting it in your mouth over time, but personally I don't usually want to wait. Also, our taste buds are less sensitive at lower temperatures. And I'm sure there's some other chemical reaction in play which would occur slower at lower temperatures.
I do have one super specific exception, though: Nestle extrafine milk chocolate, which I buy in Spain: I just much prefer the taste of it from the fridge. This might be a nostalgia thing from my days as a kid there where you had to refrigerate it.