I don't see it as a swear word though when used in that context.
Yet it is regardless. A vulgarity doesn't necessarily mean it is offensive...the word is still vulgar regardless of the context. There are some exceptions to this, but the most commonly used word your are referring to is certainly not one of those.
