You can do more than chip away at it. I understand the comfort of having large savings, and using them to fully pay off your mortgage will take a significant chunk of the money; but, if I was in the position of having more than enough to pay my mortgage off, I would be doing that either immediately or over a 2-3 year period.
It will lower your savings but what major life changing events, and the likelihood of it happening, would mean you will be caught short financially when you have a guaranteed roof over your head?
Honestly (and I do mean this genuinely, not taking the mick) - you sound like someone that likes to see the "zeros in your bank balance" more than anything else and you have your priorities a little mixed up IMHO.
You say you have £30 left of your £20k ISA allowance this year so you can save a significant amount per year as that doesnt include money you are putting into your CHASE Savings or Premium Bonds.
Are you able to share your mortgage interest rate and term left?