Maybe I am missing something but it depends on how you're going to be streaming this content doesn't it?
If you're watching HD allowing the clients to decode the stream then you should get away with a not-so-special server.
If, however, you're using MythTV or something on the server using "Extender"-style clients then won't your server will be doing the decoding and need to be beefier?
My plan is to let the HTPC and client PC's do the decoding as they are beefy enough as it is. However, what is MythTV?