You don't need much for a HTPC, but because you'll want HD playback make sure CPU and GFX card is fast enough. Basically you want quiet components, it is just a normal PC, the only differences are..
Cut down on the number of hard drives/rotation speeds
Passive motherboard/passive graphics
Front end software/OS
Remote Control
Audio & video outputs to amp/display
Quiet PSU
Quiet case fans & CPU fan
Cool running CPU/Mobo/GFX
For HD I would recommend 2Ghz A64 or similar. I use a 3700+ Single Core, 1GB RAM, Asus CSM MATX (Nvidia 6150 IGP), M-Audio Revolution 5.1, Seasonic S12 430W PSU, ATI X300 128MB PCI-E (for TV-Out) Western Digital 2500KS & 3200KS, Pioneer 106S DVD-ROM, Silverstone TJ-08. That plays back 1080P easily. Oh don't forget to scale up to your native rez of your panel with FFDSHOW. Also select the resize if "<" as you only want to scale video that's under the native rez of the panel, and not to downscale any higher than native to native as stuttering will occur.
Building a HTPC is simple, it's the software/codec/no keyboard requirement in use & remote control integration side that's most difficult.
For Software I use Media Centre, FFDSHOW, Theatretek. Yet to get my music collection working as MC media library is rubbish, easier to use a Squeezebox (have one of these too) instead really as you don't need the TV to be on, and can manage your collection how you like with the SB server app. Only £165 which is a bargain really as a VFD is about £50 anyway.
I would also recommend a good learning IR remote, if you have several IR/RF components as it'll remove the clutter and be quicker/ easier to use.