Hate crime: a crime, typically one involving violence, that is motivated by prejudice on the basis of race, religion, sexual orientation, or other grounds.
So if a crime has been committed and the motivation of the crime was the victim being ginger, then yes, it should be classed as a hate crime.
However if it's just words and intended as a joke tat the extent of the souless, pale, daywalkers then it's just that a joke.
That's not to say the person being joked about can't get offended, just like the person telling the joke has a right not to care who they offend.