It should be pronounced Vol-de-Mor with the 'T' silent, it's French for Flight of death.
I think it crept into the first film, which once released they couldn't really go back on so had to leave the pronunciation as it was.
Yeah everyone knows that was how she came up with the name now but tbh who ever read it 'Voldermort' and thought 'ah silent T!' - zero percent probably considering nobody flagged it during filming!