How reliable are speed tests?

anty1

anty1

anty1

Associate
Joined
9 Feb 2010
Posts
291
Location
London
Over the last several months I have been having increasing problems with Virgin Media. In the early days they were fantastic no issues, no problems. I would one of the few to have them when they first arrived. Few months ago we upgraded to 50mb and recently had it doubled to 100mb. Nowadays I struggle to watch Youtube videos, watch streams, play games and at extreme peak times even web pages load slow. Virgin had identified that my connection has local upstream utilisation and that the fix was on the 6th Feb. They have now pushed back the date to the 3rd of April - thread

Anyway I believe there might be added problems to my connection. The slow down really comes into action at peak times and during the week end. But my question is how come my connection shows different results on different speeds test. For example the ever popular speedtest.net always seems to shows a good fast connection.


Whereas broadband speed checker always shows low speed.
bbspeedchecker.png


ThinkBB


Lastly I found a HTML5 speed tester.
130215214122_3527.png


Why am I getting such varied results? Finally should I keep getting at Virgin for a faster fix because I can't wait 2 months or just drop them?
 
I don't put too much faith in them. I will use them as a rough indicator to see if something is wrong (mainly when gaming, see if its the server or me) though.

There are just too many variables. How much bandwidth does their webserver have? How many are accessing it at the same time? What's stopping a cheeky ISP prioritising traffic to a speed test? Is it one download or a few in parralel?
 
What's the problem?

Speedtesters are OK for a quick check but they are not accurate enough for their results to be taken too seriously.

For one thing, almost all of them are Flash-based apps, the majority running the Ookla engine under the bonnet. The Test Flow & Methodology section of the Ookla Wiki gives an idea as to why it's not particularly fussed about accuracy.

"The fastest 10% and slowest 30% of the slices are then discarded (see * below for more detail). The remaining slices are averaged together to determine the final result.

* Since we are measuring data transported over HTTP via Flash there is potential protocol overhead, buffering due to the many layers between our application and the raw data transfer and throughput bursting due primarily to CPU usage. This accounts largely for dropping the top 10% and bottom 10% of the samples. We also keep our default test length short for the user experience, and compared to this duration the ramp-up period is fairly significant driving us to eliminate another 20% of the bottom result samples."

tl:dr the resuts are massaged and can be skewed by irrelevent factors such as CPU utilisation, anti-virus software etc

Another factor in speedtest reliability is the location of the server you are testing against, as well as it's own throughput.

The location of the server you test with could result in your traffic being routed through slow networks and servers, skewing your results. Distance is also a factor; testing against a server in Australia will produce very high ping times and low throughput. The higher your broadband package is, the more chance of these things happening.

With throughput, a lot of testers will not be capable of showing results higher than the highest DSL lines (~24mbps afaik). If you are on 100mbps and test against a server that is unable to match that, you won't see your full speed being utilised and the result will be poor.

tl:dr If you are faster than the server you are testing against, the server will show a limited result. This is also affected by distance and routing.

What's the solution?

For accurate tests of your throughput, the best method is to saturate your line with file downloads and monitor the utilisation of your network card.

The quick and dirty way is to go into the Networking tab of Task Manager, observe what your NIC is capable of (ex. 100Mbps or 1Gbps) and work out what percentage of that number your BB speed is. Then when you queue up a series of files, that's the percentage you want to see in Task Manager.

This will give you a basic but accurate look at what throughput your line is capable of in general. However, different services can be limited in certain ways which are more difficult to test for (p2p/bt traffic).
IMPORTANT! All testing should be done on a wired connection, and ideally the device being used for the test should be the only one connected to the line. Safe Mode with Networking is also advised if the performance of the PC itself is a limiting factor in testing.
 
Last edited:
I had over-utilisation issues with VM for about 3 months. It was a right PITA. But I stuck with them and it's been absolutely fantastic since. Was on 50Mb last week, jumped to 100Mb and it's still brilliant.

I think you should consider yourself lucky you have over-utilisation issues, because once they actually fix them, you can bet it won't happen again for a very long time. Their service is absolutely unbeatable.
 
Last edited:
You must log in or register to reply here.
Back
Top Bottom