I think the key point you are missing is there has to be an "accident" in order for him to leave the scene or tell the police.
depends what you think constitutes an accident, or rather what the police think.
my situation years back was a car coming in the opposite direction crashed after they passed me (at least 100 yards) in the other direction. Their "accident" was the thing which is important. The fact your vehicle has no damage (and neither did mine, as we never touched) had no bearing. All it took was a witness to say I was there at the time and then drove off I got done.