Even quite cheap headphones often produce a good quality of sound; I expect you would hear a difference in sound quality. I take it the card is for music mostly? So long as they are good mp3's or better you should hear the difference.
Another advantage people often want with soundcards is EAX, which is a feature to improve the sound in games (most notably it adds reverb/echo to the environments in game).