You didn't get invested in it.
A lot of these shows you can't just drop by and watch 5 mins and expect to see anything that will 'grab' you. I remember making fun of the girls who watched Big Brother back in the day, until I started dating one and we started watching a season from the beginning. It also helps to watch with someone who genuinely lives for these type of shows as they provide a good guide to whats going on. The longer you watch the more it grabs you, you learn the contestants, pick favourites, ones that annoy you etc. then just like that you're invested. I briefly had this with TOWIE, thought it was garbage gutter level TV until I had a flatmate who had it on all the time, couldn't help but slowly learn more and more about the show and characters and then the entire flat was watching it.
It's no different to following a sports team, you have your favourite players, rivalries, rumours, drama surrounding the event, the outside conversation that carries over into work the next day..