I don't think it does happen with a phone/watch, at least with Apple Pay, I've never been forced to use my card and PIN. Very occasionally I decide to use my physical card (for example through a drive through where it's tricky to reach out the window to pay with my watch) and from time to time, it's forced me to authenticate but that's never, ever happened with Apple Pay. I guess the logic there is that the device has to be unlocked to work. If someone stole my phone or the watch off my wrist, they wouldn't be able to use it to pay because it's locked. In that respect, these electronic methods are more secure than a plastic card.