The major downside for me with single turbo diesels, is that they don't respond quickly coming on-off-on throttle with the power not coming in until 2000+ RPM..
Never ever had this problem in my car. It will shove you to the back of your seat from a standing start before the turbo even gets spinning.
When cold, there can be a fraction of a second delay when you put your foot down when leaving a junction which is not a charateristic of a petrol car. That, quite frankly, is it.