For instance
The Kindle has the whole "it's integrated with Amazon" thing going for it, but the downsides (I've actually got access to a Kindle keyboard, kindle 4, and 505 here).
They feel cheap (maybe that's just me, I don't like the plastic feel of it).
The buttons are poorly placed/poor for older users - my mum struggles with the stupid little direction pad on the Kindle 4, and the "ok" button that is in the middle of it (I struggle with it, I can't use it with my thumb which is what you naturally tend to to when you're holding the the kindle like a book due to it's position).
They can't seem to do simple things like a proper author or collection sort - My 3.5 year old 505 handles collections better than a new Kindle, it doesn't move the books around in the collection for example (my mother's Kindle seems to move the book that is currently being read to the top of the collection list, which makes it a little harder to work out what the next book in the series is).
I like the Kindle generally, but there as several very simple and stupid design choices that drive me nuts every time I have to use one - the collections issue for example shouldn't be an issue at all, it's a simple software fix (if they wanted they could offer you the choice of how it behaves), and the direction/ok button is something that I'm amazed wasn't spotted in the prototype stage.
The 505 despite being a much older model just seems to beat the Kindle for UI when it comes to the "library" side of the units, and the navigation controls are better.