I have a '97 Mk3 Golf 1.6 SE and really enjoy it. Took advise from this forum and others and went for the later model with better engine. The early Mk3's suffer with weak locks and very heavy steering. Also, all Mk3's post 96 were galvanised so they wont rust anywhere near as badly as the earlier cars.
Parts are expensive, so be weary of any car you check out. They aren’t cheap to fix. So far, I've only had two problems with mine, one was central locking pump and the other a faulty aerial. The pump was worn and can be costly to replace, but I got one cheaply from my VW club and had it fitted free, as for the aerial, the last owner had put a new one in that was faulty, so hardly the cars fault.
I disagree with Fox, build quality isn't irrelevant. 9 years down the line my golf isn't rattling, nor many of the other Golf's I’ve been in from Mk1 upwards. However, get in my friends old 5-series (1995) or my other friends '98 Fiesta, different story.
They are reliable, well built and comfortable. The performance won't set the world on fire, but it copes fine. They are expensive to buy and some sellers are very unrealistic about prices. On the other hand, when you want to sell, it should go the week you put it up.
*Edit* - Just to add, I don't think its worth getting a Mk3 Golf unless its a 96 - 97 car. As these won't really be in budget, there are better bargains out there.