An .iso can be bootable, but needs to be written to disc properly for it to work.
I have heard of a program called TransMac that could convert .dmg to .iso. If your Mac is running you may be able to convert the .dmg to a CD Master. Copy this to the Windows machine, rename to a .iso and then burn.
There may be an easier way as it has been a while since I looked into this!