I think your position is a fair one (not ‘the’ correct one) but only framed in your ‘intellectual vacuum / closed system’ for which every statement must be deemed to have secondary connations, or otherwise in light of your perception of the movement which I (admittedly) drew you to.
So I’ll revise my position slightly to say that whether ‘black lives matter’ is racist or not depends on the actual intention and attitudes of the person making the statement. I can effectively prove that saying ‘black lives matter’ isn’t racist in a particular context by stating it myself out loud. I am a deeply flawed human in many ways but I can say with certainty that there is no unintended racism sitting beneath my statement. If you said ‘yes but what about other races’ then I’d say ‘yes, of course, those matter too’.
It is surely better in the absence of confirmatory knowledge of actual racism to simple take the declaration of ‘black lives matter’ as making a narrow statement and perceive those words as having a meaning limited to those words. Namely ‘black lives matter’, rather than ‘only black lives matters’ or ‘black lives matter more than other lives’.
The twist to all of this, as you point out, is that some people may say ‘black lives matter’ and have a dubious ideaology whereby ‘black lives are the only lives that matter or are relevant’. There is a degree of fair criticism of a portion of that actual movement and really that stems from the actual intentions of some people parading that banner, not the words being said.
In summary then, it’s best to take a narrow interpretions of what people say, give them the benefit of the doubt (I.e. do not infer moral dubious-ness) but then allow your opinion to change by maintaining a healthy skepticism further to the observation of their actions.
If, instead, we infer connotations (of racism) to statements that, when given their narrow interpretation or actual intention, are not racist, then we end up in the situation whereby people are too frightened to say what they think because they will be accused of being immoral. I think you’ll agree that’s something we all want to reduce.
Sorry for the waffle