Surely you just need pockets with zips or put them in a bag.
Till they get ripped, or the zip fails, or you forget to zip your pockets up(which I do often) . If I'm riding with my bum bag, I'll clip them on but at bike parks/when the cars at the bottom of the hill I'll quite often ride without a pack.