Damage limitation, I would still phone police on the non emergency number. Who know how many other people that driver has done this to.
He was pretty casual about it - so either has done this a fair bit to other people before or it was a genuine mistake (unlikely but still).