Basically my point is that you can have two people engaging in a conversation who take completely different things away from it and neither one of them is in the wrong despite both parties feeling aggrieved or offended despite none being intentioned.
Ultimately, it comes down to this. I rather not go through life assuming every negative interaction or comment I find a bit left is down to racism or prejudice. Granted I have had my fair share of genuinely racist interactions over the years but I can brush that crap off.
I never want anyone to feel like they can't ask anyone what their persons family history looks like or originated from. Often you'll get really interesting stories. I love when some asks me about my family heritage because it's actually got a few cool stories within in (Due to the whole enslavement thing it's quite hard to really trace our family tree but we do know that our direct ancestors were involved in the Maroon war and we still have family residing in Flagstaff today. It's even the name of our family bakery)
It's just wording is important and if someone says they're from X and you insist on asking the same question that would insist they're not telling the truth, it gets frustrating when you do the same dance a few times.