Because it's a show based on nothing more than dialogue.
I've not long finished it for the 3rd time as my partner hasn't watched it and you realise how many small conversations or the way certain characters act for a few seconds proves to be vital information later in the series.
Without giving anything away there is a 4 second clip, maybe less, of a character in a certain place. If you missed that scene you would never realise there is this other personality to a character and it makes all the things he say make a lot more sense - plus it was funny.
It's not a show you can watch in the background. It demands and quite frankly deserves all of your attention.
There is no filler in The Wire. Everything is there for a reason and shows why Baltimore is the way Baltimore is.
"It's all in the game, yo"