Your argument would stand up if religion wasn't the source of most evil in the world, and in many respects cause people to stop 'living' and death (and if one doesn't subscribe to murder as part of their own personal interpretation of a fairytale, they are at the least an enabler of those who do).
In the most part, religions and religious people don't by their very nature 'live and let live'. It is 'live my way', or I'll either force change upon you or abuse you in some way for not changing.
After centuries of oppression, murder and abuse (which hasn't ended in its entirety - ie the Catholic church), the fact the churches and religions demand 'live and let live' is laughable. Like all bad things in this world, it needs to be fully eradicated.