You've seen the destruction they have brought Ukraine ? They have no intention of keeping the people around, they'll either crush them under the rubble or hope they move out
Not pushing back will mean they will just keep escalating and at some point that escalation involves nuclear weapons, will people like yourself on the side of "we can't dare push back" still have your heads buried in the sand when the nukes are falling and say "we shouldn't retaliate because they might escalate even more" even though the bar of escalation cannot be raised