Returned to who?
The assets they have if they were taken, be it by conquest or whatever, were taken by relatives long ago, from relatives long since past. Who do you decide gets what?
Does the success of the British empire mean that the average person in the UK should also make payments to other people, as the average person has no doubt benefited from it in some form or another. The UK stood at the top of the world politically and financially, with less fortunate and less developed countries propping it up, the average Joe may not be at the top of the UK but they were still at the top of most the world because of it.
The fact is that they do bring in money. I am not proud of the royal family and do not have a drop of British blood in my veins but even i can see that they are a symbol for many. I find it so odd to see people have such a chip on their shoulder about the royals. The money and assets of the royal family is theirs more so than anyone elses.
If it is the distribution of wealth that angers you, then the royals are just an insignificant spec in the system.
The current royals didn't steal this wealth and this wealth was never owned by you. It all boils down to jealousy.