Do you prefer sport bikes or naked? A sports bike will cost more on the insurance.
Have a look at a Yamaha YBR125 and the Honda CB125F/CBF125 both are great bikes.
If I was you, I would sit on both and see what felt right and most comfortable. Should budget around £150 for a helmet and consider the expense on gear as with British weather on a bike you will be exposed to the elements namely rain. Gear is expensive and you'll be looking on spending around £500 for reputable branded gear.
An A2 licence would require training which only an instructor could judge on how much training you would need, talking around £400+ for an A2 licence which consists of MOD 1 and MOD 2 tests.
A decent 125 that hasn't be dropped or slid on with low miles you're talking around £1500 at a guess.
Insurance for me quoted at 23, no driving licence and just CBT with a 2011 YBR125 was £310 annually.
The trouble you will have with an A2 licence is the restrictions, you will have to search far and wide for a bike that is A2 compliant otherwise you will have to restrict the bikes power to a maximum of 47BHP and a power to weight ratio of 0.2kw and the original power cannot be more than double the restricted power.
Take a look here,
LINKY