I love my Ultimate, I use an old TheArcade by Suzo joystick with it. I have an modern ArcadeR which supports separate fire buttons but it's a bit long-throw for me.
You can get gamepads with secondary fire that's essentially dpad up.
USB controller support is problematic, as each controller will enumerate the controls differently. You really need some form of device ID recognition or button remapping to be usable, and that's a lot of work.
As it is even the USB mouse support isn't perfect. The most period accurate form factor modern mouse - Retrogames 'TheMouse' as supplied with the TheA500, doesn't work for instance.