I don't see how they're related really they seem entirely separate things. Allowing people the freedom to move between different judicial systems with no checks, and with different economies, seems to have a far larger impact than freedom of speech does. Allowing freedom of movement between 28 different economies is a capitalists wet dream.
Isolate the UK for a moment, why do people hoard around London? Money. There are ways to get money legally and illegally there. The majority of the wealth is there, so people move there from up north, scotland, wales, etc etc. Moving workers out of the towns from wherever they came from. How do we pay for the pensioners and benefit claimants in those towns left behind? We tax people and spread it all around the country. Therefore it's fair that London steals all the workers from various towns because we distribute the wealth out of London with taxation.
Now open it up between countries, and people flock to the countries where there is money, to get that money legally or illegally. In the EU countries have different economies and different social benefits. There are state pensions etc. If Country X pays more money than Country Y with better social benefits for the same work and with a better quality of life, and there is no restriction of you moving from Y to X. You'll go and do your job in X. Well what about the economy of the country you leave behind? What about the public infrastructure of what you leave behind? What about the state pension for all the pensioners you leave behind? They're screwed. The EU would eventually resort to the plan that the nations do at a national level, that is: progressively tax based on wealth and distribute it out. The rich EU countries will be taxed more to pay for the poorer ones. And why not just save money and move all national pension schemes into an EU wide one? Why not put all defence and public spending into an EU wide one? And there's your Federal EU Super State, which you might be ok with the idea of. I personally don't like the idea of it but hey-ho. The EU with it's freedom of movement is acting as one large nation. And it will become one. I think it is perfectly valid to decide you don't want to become part of it. I also want the money I lose in taxation spend as locally as possible.
So that's one problem of freedom of movement. Have you got one for freedom of speech? People get their feelings hurt? Ouch. People hear opinions different to their own? Ouch. People actually don't feel ashamed to say what they really want to say? Ouch. People can say what they want without fear of losing their job because a bunch of unemployed people on twitter heard it and are 'outraged' and there's a BBC article which is literally just full of twitter embeds of unemployed people crying? Sounds good.
They are separate things, they have different valid criticisms, and the cases for removing one is not really transferable to the other. One is a freedom of the mind, the other is a political right. They are completely different things.
.