There might be something in this.
My Jalapenos and Cayennes are already about 4" tall but the Habanero is only about 1 1/2" tall. It has plenty of leaves and looks healthy but it is growing a lot slower than the others. I know they're slow growing, I'm just not sure how slow and if this is normal. It'll be interesting to see if they actually get anywhere. They're not short of heat as there is a Folding rig next to them blasting out heat from an i7 950 and and overclocked GTX 690 running full tilt They're also in a South(ish) facing bay window so they get the maximum of what sun there is.
I think its all down to luck really. My scotch bonnet plants grew very slowly as expected, but they did get hundreds of flowers. Then we got a couple of weeks of hardly any sun at all and all the flowers just dropped.
If your plants flower when we have a good few days of strong sun then they should produce fruit. If not then you have a better chance next year as they will have a head start.