The idea that people can't travel freely is surely highly questionable and absurd when you think about it?
Because we are no longer nomadic hunter gatherers and there are all these social things that people expect to be paid for the whole "wander whereever you want" ends up being somewhat of an idealistic dream.
There are many human created systems and rules that go against true freedom, borders are just one of them, people have the natural right to travel where they like and i don't see how one group can tell another they can't.
In a word, tough luck. Until all nations have parity on services, resources and freedoms then things such as borders will exist. As for "natural right" in nature the only rights you have are the ones you can enforce.