I've found the word to be mostly used by people who don't actually want "banter", what they want is to endlessly wind other people up and then when it works and they've gotten their childish little kicks they brush it away by saying "just banter lol".
Don't get me wrong I don't mind "actual" banter, i.e a bit of fun at the expense of a friend who can take it, however I really do dislike it when it's just used as an excuse to be a dick of a troll.
This is how I tend to see it, A bit of light mick taking if you know the person can handle it is fine I tend to do this anyway but The majority just tend to take the **** then get offended if you pull them on it saying its only banter.