I have the same issue so I am very interested in the recommendations. I've tried a few times to do this before. There are some programs, including iTunes, that compare ID3 tags and looks for duplicates, but it doesn't help when there are many dupes with bad ID3 tags. I tried to hack together a wee script that would sha1 sum all the files, write them to a big list, and compare the strings looking for matches but it didn't work out so well. I had a ton of duplicates but for each one I had to sit and think about which one belonged and which one didn't. This method didn't catch the inevitable occurrences where you have one rip of the song you made back in the day at 128 kbps, and the time you re-ripped it at 320 VBR. I only really want the higher-quality version. Furthermore sometimes you have the same song twice but in a legitimate way. You might have a live version and a studio version, both of which may have the same ID3 tags but you'd want to keep both. You might have the same song on two different albums, as may be the case if you have an original album in your collection along with a best-of compilation from the same artist.
All-in-all with the tools I've tried there were just too many judgment calls to make a simple rule or two to decide which will stay and which will go, especially if your duplicate songs number in the tens of thousands. It'd take you a week to straighten 'em out in iTunes, for example.
