That thing annoys the hell out of me.
To get rid of it, (assuming you're XP here, not sure if it is in Vista, god help me I'm upgrading today) go to:
Control Panel ->
Regional and Language Options ->
Languages (Tab) ->
Details (Button) ->
Language Bar... (Button) ->
Deselect "Show the Language bar on the desktop".
All done! Alternatively, if you only use the one language and US (or other) keyboard/settings are installed uninstall them, and that should do it.