My two biggest gripes:
Encrypted, non transferable save games that are locked to your OS and not to your gamertag! I lost all my progress in Batman AA when I moved to Windows 7.
Undisclosed, invisible limited activations. Many games use the older method of key activation which has hidden undisclosed limits. You won't know you've used up your undisclosed number of installs until GFWL rejects your key. Microsoft will only reset your activations once by giving you a new key. This effectively gives your game a finite limited lifespan before you have to buy a new copy. Newer games lock to your Gamertag which is better, but also prevents you from reselling the game.
Other issues include poor, peer-to-peer laggy multiplayer, horrendously slow interface, an update system that takes forever and has a progress bar that's won the booker prize for fiction. GFWL updates often involve rebooting your PC. The messaging system is worse than text messaging on an ancient Nokia, there's no push-to-talk on the in-game chat... etc. etc.
It's no wonder developers are dropping the system in their droves and going with Steamworks.