Try disabling the audio on your graphics card (worth a shot as it could be conflicting).
check DPC latency. if you have big spikes it'll likely be a driver for something causing the issue. try different drivers for mobo chipset/gfx etc until you find the culprit.
for me, i have a focusrite saffire pro 24, and using the latest firewire driver causes cracking/popping, but switching to the legacy one fixes the issue. don't know if you'll have the same thing with usb tho, but worth a look.