A question we often get is, “how do you compare with speedtest.net?” We summarize the information here.
A question we often get is, “how do you compare with speedtest.net?” It is a fair question because speedtest.net, and its parent company Ookla are the 800lb gorilla in the speed test category. We learned a lot from an article published by Steve Bauer from MIT a couple years back that compared speed tests, and wanted to summarize a few key points based on this information.
The referenced article tells us that the Ookla speed test is designed to deliver the maximum speed that you are getting. This is a perfectly legitimate result that is important for Internet users to know. It does not report on the all the detail performance they may want to know for all web browsing sessions. It is the reason Netflix makes such hay out of its reports on the ISP Speed Index where their published number, specific to their streaming video, doesn’t come close to Ookla’s results. (On a side note, Netflix’s analysis can be thought of as another resource to understand speed that just happens to use its own content as the testing tool)
Ookla is about quantity: it uses multiple TCP connections in parallel to maximize throughput measurement through brute force. They use multiple TCP sessions in order to calculate the maximum speed and dismiss 30% of lowest samples and 10% of highest, which essentially weeds out errors or congestion issues, dismisses retransmissions, packet loss, out of sequence packets, and pushes out as much traffic as possible no matter what.
The folks over at M-LAB, looked at this and advised that Ookla appears to have 90+ measurement instances located within the networks of Canada’s Access ISPs. While this is a very impressive number it does also mean that, if the user is using their ISP’s test or a nearby server for an Ookla test, then their tests are often conducted as close to the client as possible. Again, this is the best way to understand the top speed you expect to be getting because it gives the, “last mile” performance (the performance within the Access ISP). However, it is not necessarily how the Internet works, nor reflective of the full path performance or integrity.
On the other hand, the CIRA Internet Performance Test (i.e. the NDT test from M-Lab) measures the maximum capacity/throughput of a single TCP connection in the normal world. It measures the quality of a connection, and takes in account the latency (time/distance between source and destination). These are issues that can impact the quality of your Skype call, your IP phone, streaming video, and other bandwidth intensive exchanges between your PC and another server.
When things get really fast, the browser matters to the CIRA Internet Performance Test
This is an important distinction because many (but not all!) multimedia applications use single TCP session for streaming, one TCP connect to stream a movie, to make a phone/video call, to download/upload a picture or movie. In order to measure the quality of the connection, the browser needs to do a lot more work to monitor TCP sessions, calculate duplicate packets, identify out of sequence packets, calculate packet loss and jitter, and to retransmit lost data. Choosing the right browser to do the NDT test is important to get accurate results since not all browsers are equal.
It’s about Canada’s Internet
Don’t forget what it is all about. The CIRA Internet Performance Test aims to help individual Canadians understand their own Internet connection, but above all, its goal is to help us all create a better & faster Internet in Canada. So what do we want you to do? Bookmark both tests, and run them both as often as you can. Take the time to learn a bit about how the Internet works – we think it is actually pretty cool.
Rob brings over 20 years of experience in the technology industry writing, presenting and blogging on subjects as varied as software development tools, silicon reverse engineering, cyber-security and the DNS. An avid product marketer who takes the time to speak to IT professionals with the information and details they need for their jobs.
