I like the Dell SFF machines. Decent performance for dirt cheap with a few little upgrades.
Currently running a virtual pinball machine (just for testing parts) on a 755 with Q9650, 8gb ram, Radeon R7 250 1gb GDDR5 version and a 250gb SSD. It's not bad, needs more grunt in the graphics department really. If I was going to build this into a full machine, then I'd strip it out of the case and fit a better PSU and a 4gb RX470 or something along those lines.
Some other dinky machines worth a look:
Alienware Alpha and Alpha R2. Not cheap, but well worth it especially if you can get the AGA external GPU housing with the R2. The first release is quite limited, but still good enough, especially with the i7 option, 8 or 16gb and an SSD. Other half uses R1 as her home machine. Generally silent with her usage. My R2 is currently in bits... but was in use under the TV. Wasn't using the AGA, just the built in GTX960. It was fine.
Lenovo M93p i7 4770 - very dinky. Very expensive.
Now, for XP... Going to sound silly but some thin clients work really well, are available often very cheap and are tiny. WYSE Dx0Q, HP T620 (or the T620 Plus) for example.