I hope the m3's you're looking at aren't in the 5k range, likely to be absolute sheds! (Assuming e46)
No would be looking at up to about 8.5k max but even that looks like it might not be easy to find a decent manual.
If I went down the z4 route I'd really want the msport seats and aero kit but don't want to spend more than 6k. Hardtop would be an added bonus but they seem rare now. A nice looking one went for 5k last week.