No, you don't but I'd want a rechargeable battery for definite, it's not 1998 any more
As for the PS5 storage prices, hopefully they'll release a good range of compatible SSD's - the just released Samsung 980 pro is around the same price but there's bound to be slightly slower options around £100 for 1TB which is much much more appealing. With MS you've got one option, take it or leave it.
Hot swapping rechargeable AA's is a pretty good solution.