But this doesn't address my point - what if the OP had been somewhere where getting out would not really have been an option, or would potentially have been more unsafe than in a taxi with a driver on a mobile.
I don't think it is particularly dangerous but that is somewhat risky, but the fact is if you are paying for a service you have every right to expect them to provide that service, which is to be driven to your destination in a safe and legal fashion. They failed to provide this service, even after OP tried to ensure they did (shouldn't have even had to do this, but I guess it could have slipped the drivers mind that it was illegal to use his mobile)
At the end of the day, I don't think it is worth losing any sleep over, but I still think OP was in the right (but yes, stupid to allow the taxi driver to know where he lives)