I have had this conversation with a few people before and my housemate brought it up again last night
Do you think colours look the same to all people? I dont think there is a way of ever finding out the answer.
For example, when I am younger my mum or dad will have pointed at the grass and told me "that grass is green" or I might have seen it in a book. I automatically associate this colour of 'green' with anything that looks similar.
Whats to say that my 'green' might look like what someone else sees as my 'brown'
Maybe in future this could be worked out when we know more about the brain
Knowing stuff about the brain doesn't really help. You can never really know what someone else's subjective experience is like - no matter how much you know about the chemical/electrical processes that go on within the brain.
You can say similar of any sensation. When i eat chocolate, i have a particular sensation that i associate with that act. You also have a sensation that you associate with eating chocolate. But if i were able to experience what you experience, perhaps your experience would seem the same as the sensation i get when i eat cheese, or see the color blue, or scratch an itch etc
What is really interesting about this kind of stuff, is that it's not what these things are actually 'like' that matters - it's the fact we all share the same frame of reference. When we eat chocolate, we all have a sensation we can define as 'eating chocolate'. We can communicate about it even though what each of us actually experiences could be completely different.
If you really want to drive yourself insane on this stuff you should go read some Wittgenstein or something.