What a surprise that everyone's favourite is the one they owned as a child, the one with fond memories! I doubt you're going to get an non-nostalgic answer...
The PSP is the best ever because of the HUGE leap in power over previous handhelds made "console gaming on the go" a reality. Racing games in particular were well suited to it. It was still "simple and cheap enough" to make games for, that you got loads of interesting games such as Loco Roco but packed the hardware to make AAA style games too like Siphon Filter, Midnight Club, GTfrigginA Vice City Stories. Proper sound. Add to that mp3s, internet browsing (before smart phones), standard (albeit Sony) memory card usage, multiplayer without two people needing a copy of the game, watching full length films. It had a lot going for it, so much more than the previous consoles.
The Vita could have been better but it has too many gimmicks, making it too expensive, no games, stupid memory cards...
The GBM gets an honourable mention because it is genuinely small and light enough to slip into a pocket - the best of the 16 bit handhelds. The battery lasts ages on a single charge and the screen is backlit. It is the most handheld handheld.
If I was going on a long train ride or stopping over at my parents for a night I'll take my PSP. If I was waiting at a bus stop or at the docs or something, GBM. In fact it lives permanently in my work bag...