Did you use a core drill?
I fitted a tumble dryer vent at the weekend, I didn't use a core drill but drilled loads of holes using a masonry bit and then chiselled it out.
The hole was nowhere near round so spent ages chipping away with the chisel to get the vent pipe to fit.
The only good point was using Bosch Multipurpose Drill bit's
https://www.screwfix.com/p/bosch-straight-shank-multipurpose-drill-bit-14-x-250mm/65876
They go through brick so much easier than standard masonry bits.
Things I've now learnt.
1. Wear hearing protection!!!!!
2. Buy a SDS+ drill (only £60 at Screwfix)
3. Use a core drill for big holes.
4. Stop buying drill bits other than Bosch Multi ones (though can't get them for SDS)
5. Buy a proper club hammer to use with a chisel and not a mini sledge hammer
6. Start drilling the hole in non-hammer action so it creates a mini hole and then switch to hammer action, stops the bit jumping about.