I know a lot of muscular men that are aerobically very very fit and flexible. In fact most people at my gym are, even a lot of "meatheads" can be quick and flexible, having big muscles doesn't necessarily mean you become a solid lump. I agree that an expert in a martial art would inflict probably a lot of damage, however the bid dude need only land one punch or grab the guy and it's all over. IT's a pointless debate because the martial arts dude wouldn't get into a fight in the first place, secondly in the real world, fights don't happen like they do on the movies. All the girls I've been with have always preferred a muscular rugby type and that's the type of women I tend to go for at any rate.
Ultimately going to the gym for big weights doesn't mean you're unfit, it doesn't mean you're slow and it doesn't mean you have no flexibility - you can have all of that, and still have bulk. Furthermore having the bulk helps protects you against blows and other damage. It's a fine balance. I'm not built for long distance, but I've ran a 1/2 marathon before and crossed the line in the top half of the race, I'm near on 15st, surely that's not possible?! Secondly, I've had a play fight with a friend who is black belt in taekwondo or karate (I can't remember which), and whilst he did get some good blows in and some good shots in I ultimately over powered him - so was it a lack of skill on his behalf or just the fact that things on paper do not always equate to things that happen in the real world.
I'd get your head out of the sand and start to realise that the world around you isn't like it is in the movies.