No offence, but I don't really get why you wouldn't want to install the drivers. This is especially true if you happen to have the disks. If it is something like a 56K modem I usually don't bother to install drivers (assuming the person isn't using it) and either disable it in device manager or remove the card.
Its basically improving system performance, may as well have it!