Finding the best performing CDN for ChinaA test series with the help of CloudHarmony to find best performing CDN for China
The use of CDNs can increase the loading speed of your website significantly, especially for people accessing your website from places that are literally on the other side of the world.
In terms of China, this is particularly true, as it’s difficult to get a website running on a physical server in Mainland China and CDNs can be a way around that by basically outsourcing the storage of parts of your website to specialized companies. Having said that, choosing the wrong CDN can have to opposite effect in that it might make your website unusable in China. This is because some CDNs perform better than others in the Mainland and some don’t perform at all. Therefore, this first of many more tests to come will try to gather some data that can be of help in deciding which CDN to pick.
What is a CDN
First things first, what is a CDN actually. CDN stands for “Content Delivery Network” and basically consists of data centres around the world with copies of part of your website on them.
A good example of how to illustrate what CDNs are doing is Amazon. Now Amazon is in the business of selling things over the internet. They started out in the US with warehouses in the US, so if you ordered something let’s say in New York, you can be pretty sure to get it within one or two days, even if the only warehouse early Amazon has would be in San Francisco. Now Amazon is growing and becoming a global player. As a result, Amazon is opening up new warehouses in Europe, Asia, Australia, etc. to be able to reduce the time people have to wait for their orders. Therefore, if you order the latest book from Malcolm Gladwell in Germany, the book would be sent to you from Berlin or wherever the closest warehouse would be and not from the US.
CDNs do exactly that just for websites. When you sign up for a CDN, what you sign up to is basically an infrastructure of warehouses that make it possible to deliver the content of your website quicker in that they don’t get all the content from the server your website is running on but rather load most of the content from the closest warehouse – that being the closest data centre of your CDN provider.
The problem of CDNs in China
Sticking to the example of the warehouse above, the problem we’re faced with in terms of China is, that Chinese customs at certain times doesn’t allow deliveries from certain warehouses. So even though the data centre of your CDN provider might be in Hong Kong, people in Shenzhen (so right across the border) might not be able to load the content from that CDN, simply because China doesn’t allow its content in or takes a close look at it which slows delivery down.
This is one of the reasons why Google Fonts or other similar content doesn’t load in China – they’re all on Google’s CDN and all those data packages get stopped at the border. Unfortunately, there’s not much independent data out there to find out which CDNs work well in China and which ones don’t. This first in a series of tests tries to provide one piece of those missing independent data in the hope that many more follow provided by other websites and sources.
The test, the tools and its limitations
The test was run using CloudHarmony, a great platform provided by Gartner, which makes it possible to test latency and download speed among other things for hosting providers, CDNs, cloud services, etc.
CloudHarmony allows you to test 23 CDNs covering many of the most popular ones around the world. Having said that, many of the CDNs that particularly specialize on China like CDNetworks or ChinaCache are not covered. For sites that only target Mainland China, those specialized CDNs might be perfect, but for websites where China is just one of many markets, the CDNs featured in CloudHarmony’s test would probably be more appropriate assuming that you only want one CDN per website (which isn’t always the case).
To see what the test is like, feel free to have a go at it yourself by clicking on the following link: http://cloudharmony.com/speedtest-latency-dns-for-cdn
The test itself was limited to latency mainly because of the reason that this can be tested with very small data packages and is less likely to be influenced by spikes up or down in bandwidth, which would be the case for testing download speed. Now what is latency? It’s kind of similar to a ping which is probably better known and can be seen as the time needed to finish one communication cycle with the server, meaning the time that goes by from sending the query to receiving the answer from the server. A great more detailed explanation of latency and ping can be found here: http://www.speedtest.net/articles/what-is-ping-what-is-latency/
Before we get to the results, another quick word on the test process and its limitations. Apart from the limited number of CDNs, I didn’t have any influence on what CDNs from the various providers were actually used. So it might be that CloudHarmony has access to the highest performing CDN for CDN x but only to the cheapest version for CDN y, which might influence the results. This, unfortunately, is out of my control.
The test itself was conducted in Shenzhen, China, which is very close to Hong Kong. I would assume the results would be kind of similar or not vary too significantly all over China, but don’t have the material to prove this. The test data includes 8 sessions between August 7th 2016 and August 20th 2016 with a maximum of 1 session per day.
Finally, please again let me point out, that the data shown below should be taken and consumed with the necessary amount of care. CDN y doing bad in the test doesn’t mean that CDN y is a bad CDN or does never work in China, it just means that in the given time period in Shenzhen with the given internet connection the test provided by CloudHarmony didn’t work for CDN y at the 8 times the test was run.
After all those remarks, let’s get to the results!
The best performing CDNs by rank
This analysis took the results of each of the 8 tests, sorted the CDNs by latency with the lowest on top, highest on bottom and awarded points, 10 for the first down to 1 for the 10th.
The top 5 performers from this angle were:
- CDNsun (second with SkyparkCDN)
- Akamai CDN
- Liquid Web Storm CDN
The best performing CDNs by speed
The second analysis concentrated more on the latency itself. Notice the red columns in the picture above. Whenever a test failed for a given CDN, I awarded 1000ms for that test and marked it red. The reason for a test failing can be, among many, that the CDN can’t be reached, that the connection timed out or that the internet connection stopped working for a moment. Given those reasons, having one or even two red columns in the results for a given CDN doesn’t mean the CDN didn’t work at all during that time or that the CDN isn’t reliable. In fact, even the best performing CDN Quantil, which usually beat the other CDNs by quite a margin, had one failed test, so I wouldn’t take those occasionally failed tests too serious. For those other CDNs, that failed basically every single test, it’s a different story.
The top 5 performers for this analysis (same base data as above) were:
- Akamai CDN
- Azure CDN from Verizon
I will get to the CDNs that performed best in a series of posts over the next weeks devoting one post to each of them and analysing why they did perform the way they did in the test.
In terms of CDNs that finished further down the line, an interesting thing to notice was, that Alibaba’s CDN didn’t perform all too well. Even though it mastered all tests, the latency was pretty high especially when compared to other CDNs. Rackspace Cloud CDN is another CDN that did disappoint a bit in this test, especially given that Rackspace has a physical presence in Hong Kong and proved to be one of the top performers when testing server providers (we’ll get to that tomorrow).
For other big CDNs like AWS, CloudFlare or MaxCDN it’s probably most important to notice that all of them work and that the latency is acceptable, even though not top. Akamai CDN, one of if not the biggest provider of CDN services in the world, performed very well, never scoring the top position but working well consistently.
As mentioned, more on the top performers over the next days and weeks.
The raw data
Here you can download the screenshot of the 8 test results that were takin as basis for the analysis. Feel free to download the ZIP file and build your own conclusions based on the test result from CloudHarmony.
To again put this test into the right perspective, what is shown above is only one snapshot and by no means representative for all of China or the quality of the tested CDNs. Nevertheless, I hope it can be of help to people searching for what CDN to use and can be used as one of the pieces (emphasis on ONE OF, so please search the internet for more sources) in their decision making process.
Should you have any opinions, information or facts that might be helpful in the discussion of this topic, feel free to get in touch. I would be happy to hear from you!
A quick introduction to CDNsun, one of our top performers in the most recent CDN test
A quick introduction to Akamai CDN, one of our top performers in the most recent CDN test
A quick introduction to SkyparkCDN, one of our top performers in the most recent CDN test