In anger - possibly.
In cold sober thought, no. Why break a home? It's none of my business. I don't meddle in other people's lives, it's not for me. If I was directly affected by this, then my outlook may be different, but as a bystander I think whilst people feel it would be the "right thing" to do, I can't help but feel it would also cause more trouble than good. The affair might be a short lived thing (not that I'm condoning it) and the person may realise the errors of his ways and go back to this family and have a great life.
It's not straight cut for me.