Trust me its in your head and if you aren't working out for more than 1 hour i would leave the bannana too, simple sugars like that work against weight loss, you want to minimize all insulin spikes, at most id have fruit early morn after that you really dont want to be having sugars.
Think about it like this, your body is a layered fuel tank, the top layer is sugars, these burn quickly and are mostly used up within an hour of any exercise (assuming broken non consecutive training like weights). The second layer is glycogen reserves, these start to burn during prolonged activity, you will certainly be using these during a 1hr workout however depending on your diet you may well not deplete these reserves in a typical weights session of around 1hr.
The third and most stubborn layer is fat, this is what you need to burn, and im sorry to say your body doesnt want to. Until you have completely depleted blood sugar and muscle/liver glycogen stores, the body will not start true lipolysis (fat digestion). Thus if you ate a snickers bar before every training session, even with a great, but calorie neutral, diet, you wont lose weight, at least not directly from the exercise in the gym.
The systems involved are more complex than i have described here, much much more is going on, but as a rule of thumb, to lose weight you should deplete your carbs, once depleted you are burning fat, fat, fat. Its efficient and it works like science should. But in reality fatties love their food and thats all there is to it. Its all on you.
