If they are soot deposits rather than paint scorch marks then yes I would be concerned. If it's soot then it's not burning the gas fully so could be generating carbon monoxide. If soot, I would not be leaving that on over night or if at risk of falling asleep.
I'm not a gas engineer, but also take a look at the flame, it should be burning fully blue. If there's orange in it then it's incomplete combustion and a risk of carbon monoxide.
I would err on the side of caution.