I had an infestation of mice at my first rented house. It didn't really bother me (I quite liked seeing them scurry around), but droppings in the kitchen meant they had to go. I tried non lethal traps, and caught plenty, but it didn't seem to really impact their numbers.
A few weeks later I got a pair of cats (had been planning to for a long time anyway). That did the trick. For the next couple of weeks I would find mouse corpses in various states, mouse heads, limbs, tails and general wobbly entrails at random places through the house, though my moggies were kind enough to leave most if them by the front door.
So yeah... cat. Your mouse problem will vanish, plus, you get a cat. Which is awesome anyway.