I said it was not morally wrong. That's not the same as being morally okay. You've excluded your middle.
To a simple minded person like me, if you say something is not wrong - that implies that it's either unknown or right.
I'm pretty certain that sexual attraction to children is perhaps one of the most morally wrong things anybody could think about, let alone act out. It's certainly not an unknown, as far as I can tell.
If we were to imagine a paedophile who buys one of those super-creepy child-like real dolls and shags it every night they're doing something that I (and I imagine most people) would personally find utterly revolting but I'd argue that as long as they leave actual children alone they're not actually doing anything that we can class as morally wrong.
I think that sort of behaviour is absolutely morally wrong, the fact that it might be a doll simply doesn't matter, it's the mindset of it being "ok" to fantasise over having sex with children, whether it's with a doll, or even child-porn artwork (which is an offence, not sure about dolls), which is wrong.
If people think it's not morally wrong to fantasise of sex with kids, (be it dolls, artwork, whatever) then I think you have a very big problem, because I think, based on the way human beings behave, that it would progress - as it does in many cases, to real instances of child abuse.
We don't live in a perfect world, people are always looking for ways to enhance their experience, someone having sex with a child doll is merely putting off the inevitable, in my opinion.