I can't open a case now as the seller has accepted the return, meaning he'll get off with nothing
You can still leave negative feedback.
Feedback is independent of anything else.
Even in a sale where the seller has been a saint and been perfect, you can still leave negative feedback.