Not driven SE, but a few people have previously rated SE suspension as being a bit more compliant than M Sport for day-to-day slogs.
Personally, i'd always opt for M Sport (even on my xDrive) due to the leather interior, metallic paint, alloys and a few aesthetic improvements, when suspension doesn't matter.
320d should open up the most potential for options - visibility package, nav (pro or business - both should have phone prep. and USB interface - comm box). Loudspeaker/HK speakers (don't stick with the base!!). Then consider manual or auto...
Saying that however, the 520d base specification is higher than the 320d...