The 64Bit Version can map huge amounts of physical memory. This will be great in a couple of years, but since most of us are still using 1 or 2gb its not really a concern atm.
Applications written for 64Bit generally run a bit faster. The only problem is nobody is compiling any 64bit code yet! Again, we'll have to wait a couple of years.
Most of the applications you run now will have been compiled in 32bit, and will therefore be emmulated in Vista64. This emmulation is done on the CPU mostly, so you'll see practically no difference on 32bit applications. 16bit apps (DOS programs) won't work at all tho.
Drivers must be signed off by Microsoft, although I believe there's a workaround. Driver support in general will be a bit scetchy for a few months.
Security will be a lot stronger in Vista 64.
If you take the leap now, you'll probably have a few more issues to deal with. Probably best to leave it 6 months to a year, most of us will be aiming to be running it by 2009.