If you're not scheduled for the shift in the rota, you don't have to give a reason that you can't do it as its personal .... the manager can't force you to work an unscheduled shift and no one can hold that against you if any disciplinary action comes up.
Now with the flexi-time, you're on grey area here. If you have a signed contract that says you're allowed to have flexi-time, then your butt is covered. Check with HR if anything was put into writing by your old boss and HR confirming the agreement, the more written proof the better.
With no contract or no HR confirmation (eg. goodwill, handshake agreement, etc), you will need to have any correspondence with your old boss in form of letters/emails/etc to prove that it was pre-arranged. Then you will need to sit down with HR and the manager to come to a compromise. Unfortunately, a lot of these non-contractual flexi-time, etc are the discretion of the manager, that means your new boss can still revoke your flexi-time as its not a contractual obligation if flexitime is not in the employment contract/agreement you signed.