The 360 copies an image of the disc to the HDD, so there is no lengthy install like on the PS3, it takes about 5 minutes and eats about 5-6gb of your HDD. But it's well worth it.
IIRC the only game it doesn't improve loading times of is Halo 3, due to the use of the HDD to cache information. Other than that, it's quieter and quicker so no reason not to if you have the HDD space spare.