If it really was only a very small portion of Muslims like you say then we wouldn't see so many atrocities being carried out would we? Because we don't see wall to wall murders etc from small minorities of other religions do we? for example, the minority of Christians who do unspeakable things to others in the name of Christianity, show me the wall to wall coverage of their atrocities please?
Christians bombed a hospital in Afghanistan yesterday. It got quite a bit of press coverage.
In my opinion if i stood outside a church and protested peacefully that there is no god i would probably get some funny looks or verbal abuse, if i did the same outside a mosque i think i would be in for a severe kicking or worse.
Outside a CoE church, you're probably right. Try doing that in Texas though.