Is it transcoding all of the media? If so that could be the reason.
Not sure what you mean eg watching Benidorm season 3 356mb file
TVersity can convert video files to a another format on the fly, known as transcoding and more commonly used for xbox 360's. If TVersity us set to transcode all, then it will try and convert all the files streamed regardless of it is required or not, this could be the cause of your slow downs
Try loading the media directly off of the same drive TVersity is installed to.
What does taskmanager show when you're streaming?