It's practically impossible for then to rock up at your base with 5x your unit count.
Even if they are Terran =]
No, usually there is some pretty major mistakes going on if that happens.
Generally (I forget what race you play, but let's assume protoss), the most common mistake I see is early double gas. It might not seem like much, but the impact of putting 3 extra probes on gas early on (like at less than 30 supply) is huge.
If you're doing a particularly gas intensive build (and are necessarily sacrificing early tier 1 units) then fine.
But you can't support 4 gateways with double gas until you're well over 50 supply. The other mistake I see is people will build 4 gateways AND THEN build units. No.
You build a gateway, a cyber core, research warpgates and build zealot, stalker, sentry, zealot, stalker while chrono boosting warpgates. When you stop being able to spend all your money on units, you start adding your 2nd, 3rd and 4th gateways.
It's amazing how just focusing on getting those 4 gates up as efficiently as possible will change how much stuff you get and how quickly.
When I play 2v2s, sometimes with gold and below players, their basic openings are so horribly broken. It's not so much about builds though, but rather just spending their money, and especially spending it efficiently.