I've already apologised for being insensitive, it's clearly a bigger issue for some men than I had anticipated. I personally find it really hard to understand that particular issue about hair loss. Though it has been made clear that for some men this is hugely troubling, something that I went through (I've been bald/balding since my early 20s - in my 40s now) and never really even thought about it in the aspects that others have highlighted here.
One thing that has quite an impact is how you look bald as well - my body tends to maintain lean and I currently wear glasses and shaved head I tend to look like I'm late stages of a terminal disease - completely different story if I ditch the glasses and keep up the effort to maintain bulk (and a bit of stubble around the chin doesn't hurt the look).
(It is actually a night and day difference to the point people don't even recognise me)