LOL I tell this to people all the time when they ask me why I don't like rugby. I used to watch it when I was younger (South African), but then I realised how silly it is. Scrums are a waste of time, as you said, since the team throwing the ball in gets it back 99% of the time. The other thing is where exactly is the skill? Most of the game is:
1. A guy runs with the ball straight into an opposing player and falls on the floor.
2. a) A teammate of his gets the ball then passes it to another guy who repeats (1).
or b) The opposing team manages to get the ball after he's fallen and they repeat (1).
So it's just men running into each other and falling over most of the time until one of them manages to not fall over and reach the other side.
I watch World Cup Football because at least in football players can show some skill with the ball and tackles/interceptions can happen any time. Also they can kick the ball backwards AND forwards