I haven't found anywhere that refuses £50 notes, businesses can't refuse legal tender for the settlement of a debt like a restaurant meal or taxi fare etc anyway.
I think that you’re definitely right with this, but I remember reading somewhere that unless it’s a debt, (as you said, taxi fare, restaurant bill), then legal tender can be refused.
This has me wondering why, e.g., I can’t imagine buying a £65, £75, or £80 shirt, pulling up the required amount in legal twenties, tens, and fives, and having the clerk say, “Can’t accept that sir, do you have a Debit or Credit Card?”