Throwing his stuff out will put you totally in the wrong, as would kicking him out thinking about it properly -especially if you go the court route, you need to be able to demonstrate you've been very reasonable to strengthen your case.. I'm sure you would have some kind of duty of care, at least landlords certainly do, and they can't put outside/ dispose of tenants property legally without being very careful..
Thinking about it, small claims (it's not called small claims any more but its basically the same) could be the way to go, even with no contract as you have an implied contract (evidence of him paying rent for some time) and witnesses (your housemates).
These things are judged on balance of probability rather than hard evidence (although it certainly helps) so you probably have a case thinking about it.
It would be quite important to get a forwarding address for him, or a parents address for when you file the claim.
Weather you actually recover any money is a different matter but he may well end up with a judgement against him, which will cause him a lot of bother with references & credit checks etc.
You should formally (recorded letter and email)give him 30 days to pay up before you file a claim, so the court can see you have given him ample opportunity to pay.