To answer the OP question of “best retro handheld” I would still have to say Steam deck.
I had a go with the ROG Ally and it feels nice but the interface is no where as near as good as the deck yet.
Stam deck ive managed to play XP and Win9x era games using the Proton experimental compatibility layer, run emulation to PS3 and run PCem at 200Mhz to play 3dfx and AWE32 games from the 9x era.
What makes it for me is the interface, customisation ease of controls, trackpads and Proton.
Being able to play modern 'AAA' titles for me is just a bonus really.
I have these installed plus some older games and many emulators - Wreckfest, BeamNG drive, Hotwheels unleashed, GTAV, Flight Sim 2020, Dirt Rally 2.0, Half life 2, Portal 2, Only Up, Quake 2 RTX, Wreckfest, Isle of man TT ride on the edge 2, Total war rome 2, Trackmania Canyon, Planet coaster, Mortal Kombat Komplete edition. All run well but I still prefer the older games I have installed - Atomic bomberman, Midtown madness, Ballance, Arcade pool a2, Motocross madness 2, Soda pipes, 3d pinball Space Cadet, Virtua Fighter 2 (Sega model 2 version running through Proton tricks) and all Sega model 3 games.