Its simply becasue they are so few of them about, and they are the most forged.
Thats not true, they are in fact the LEAST forged notes according to the Bank of England.
Counterfeit and Genuine Bank of England Banknotes in 2008 (numbers of notes)
Denomination
Counterfeit banknotes taken out of circulation
(rounded to nearest 1000)
Genuine banknotes in circulation
(averaged for the year rounded to nearest 1mn)
£5
3,000
254,000,000
£10
5,000
619,000,000
£20
675,000
1,314,000,000
£50
3,000
157,000,000
Total
686,000
2,344,000,000
That said, they are a nuisance and you generally only get them if you ask for them in the bank which means its serves you right fro trying to look rich