Deleting files from USB stick/ external HDD

[Doohickey]

I was just searching for something random and came across this

There is one more stupid thing in Finder. Delete your files from USB stick, forget to empty thrash can, go to work and you're screwed - because Finder doesn't move them, but hides them your pen is almost empty, but with 0 space available on it, unless you can go back to your mac to empty that can, it's either format or a lot of swearing.

Which reminded me how annoying this is, and surely there must be some remedy for this? I don't understand why I need to empty the trash if I want to free space, what if I want to keep things in the trash for whatever reason, and still need to use the drive? Anyone have a solution?