Yea, no consequence at all. Because spending 8 hours a day every weekday for decades being totally miserable has no detrimental effect on your well-being at all as long as you've loads of money to spend when you go home
Obviously everyone's tolerance for frustrating and annoyance at work increases in line with salary but come on, to say that happiness at work is of 'little' consequence is one of the most bizarre things I've read on here for a long time.
I suspect you wouldn't have said it if you were not, as you claim to be, happy at work. Being unhappy at work causes health and wellbeing issues that have the potential to render the salary irrelevant.