I find it amazing how many people do not understand how mac apps work yet use them every day. To clarify...
When you download an application it usually comes in a file called a dmg, this is a way of storing the application because mac os x applications are not one file but a folder full of files which you can execute by double clicking, so as you cannot download a folder online only files. The folder is put inside a disk image file called dmg which keeps the application structure, and can easily be compressed in a zip or other type of archive.
When you open the dmg it appears on your desktop as a disk like a cd or hdd would do. You then can copy the application to your applications folder and then eject the disk image (dmg).
Once you complete these steps your set, at this point you can delete the old dmg file.
It is as with MSN messenger possible to just keep it on the desktop mounted forever but this is not ideal.