About Corey Quinn
Over the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group
, the author of the weekly Last Week in AWS
newsletter, and the host of two podcasts: Screaming in the Cloud
and, you guessed it, AWS Morning Brief
, which you’re about to listen to.
: Welcome back to Networking In The Cloud, a special 12 week mini feature of the AWS morning brief sponsored by ThousandEyes. This week's topic, The Cloud in China, but first, let's talk a little bit about ThousandEyes. You can think of ThousandEyes as the Google maps of the internet, just like you wouldn't leave San Jose to drive to San Francisco without checking which freeway to take because local references are always going to resonate the best when telling these stories, business rely on ThousandEyes
to see the end to end paths that their applications and services are taking from their servers to their end users, to identify where the slowdowns are, where the pile ups are, and what's causing these issues. They can use ThousandEyes
to figure out what's breaking and ideally notify providers before their customers notice. To learn more, visit thousandeyes.com
. And my thanks to them for their sponsoring of this mini series.
Now, when we're talking about China, I want to start by saying that I'm not here to pass judgment. Here in the United States, we're sort of the Oracle cloud of foreign policy, so Lord knows that my hands aren't clean any. Instead, I want to have a factual discussion about what networking in China looks like in the world of cloud in 2020. To start, China is a huge market. The market for cloud services in China this year is expected to reach just over a hundred billion dollars. So there's a lot of money on the table, there's a lot riding on companies making significant inroads into an extremely lucrative market that is extremely technologically savvy.
Historically, according to multiple Chinese cloud executives who were interviewed for a variety of articles, China's enterprise IT market is probably somewhere between five to seven years behind most Western markets. That means that there's a huge amount of opportunity for companies to be able to make inroads and make an impact on that market before it winds up being dominated, like a lot of the Western markets have been by certain large Seattle-based cloud providers, ahem, ahem.
Now, due to Chinese regulations, in order to run a cloud provider in China, it has to be operated by a Chinese company. That's why Microsoft works with a company called 21Vianet, whereas AWS has two partners, Beijing Sinnet and NWCD. Those local partners in fact own and operate the physical infrastructure that the cloud providers are building in China and become known as the seller of record. Although the US cloud companies of course do, or at least ostensibly retain all the rights to their intellectual property, either trademarks, their copyrights, etc.
That said, if you take a look at any of the large cloud providers, service and region availability tables, there's very clearly a significant lag between when services get released in most regions and when they do inside the mainland China regions. Some of the concern, at least according to people off the record, comes down to concern over intellectual property theft. And in the current political climate where we have basically picked an unprovoked trade war with China, it winds up complicating this somewhat heavily. If for no other reason, then companies are extremely skittish about subjecting what they rightly perceive to be their incredibly valuable intellectual property to the risks of operating inside of mainland China, so on the one hand they don't want to deal with that. On the other, there are over half a billion people in China with smartphones, just shy of 900 million people on the internet in one form or another. So there's an awful lot of money at stake. So companies find themselves rather willing to overlook some things that they otherwise would not want to bother with. Now again, I'm not here to moralize, I just find the idea to be somewhat fascinating.
Most of that stuff you can find out just from reading news articles and various press releases. So let's go a little bit further into how companies are servicing the Chinese market. Not for nothing, but picking on AWS because they are the incumbent in this space, and this is the AWS morning brief. But looking at the map on my wall, they have regions in Tokyo, in Seoul, in Hong Kong, in Singapore and Mumbai. If you squint enough, that sort of forms a periphery around the outside of mainland China. Here in the real world, if it's at all feasible, companies tend to use those regions that are more or less scattered around China, rather than within China if it is even slightly feasible and then provide services to their customers inside of China through those geographically local regions without having to deal with having a physical presence inside of China. You can learn a lot about this by looking at ThousandEyes 2019 Public Cloud Performance Benchmark Report, where they wound up figuring out what's going on with IBM, AWS, Azure and Google Cloud, and of course Alibaba this year, which is interesting and we'll get there in a minute because this is restricted to real clouds.
Oracle cloud is not a real cloud and thus was not invited. Figure out what the different architectural conductivity differences are between these cloud providers. Take a look at the AWS global accelerator and how it pans out and what you can actually expect from real world networks going to other real world networks, and see what it is that makes sense for various use cases. My thanks again to ThousandEyes
for sponsoring this podcast. You can get your own copy of the report at snark.cloud/real clouds
, that's snark.cloud/realclouds
One of those real clouds as mentioned is Alibaba. The reason that I bring them up is that they currently dominate China's cloud market. Alibaba has something on the order of a 43% market share inside of mainland China. Second behind them with 17.4% is 10 Cent. 10 Cent is also growing rapidly. AWS is up there as well, given their significant posture and other places. But then there's a whole smattering of small scale cloud operators that are still vying for a piece of a very large, very lucrative pie.
Now, if you're talking to any of those providers inside of China, then the networking works pretty much like you'd expect it to anywhere else on the planet. The challenge and why this is worth an entire episode is what happens when you try to network outside of China into the rest of the internet. Let's talk a little bit about China's great firewall. This was started roughly in 1998 in order to enforce Chinese law. News, shopping sites, stereo search engines and pornography are all blocked through a wide variety of methods in accordance with Chinese law, that tends to change and ebb and flow. Not everything is blocked all the time and keeping up with it is more than a full time job. Last week and the great firewalls block list, however, would not be nearly as interesting of a newsletter so I don't write that one.
They do this through a variety of different methods. DNS can be black holed to the point where no regular domain name doesn't resolve to anything that works. IP addresses can be routed to absolutely nowhere. When they get a little bit more sophisticated with some other approaches, they can conduct deep packet inspection on traffic that traverses the firewall and determine whether or not a given request should be serviced or not. This isn't just a pass or fail scenario. The process behind this can also add significant latency. Corporate VPNs for example, can die randomly or work in the morning but then fail in the afternoon, and then come back again in the morning and then come back again to working by later that evening.
They can attempt man in the middle attacks, defeat TLS or SSL, depending upon which term you prefer, don't at me. And what's fascinating is that a number of VPN technologies are treated differently. Open VPN is fascinating in that, some key exchanges are not permitted at all and others are permitted but are slowed to a speed of less than 56 kilobits per second. IP SAC also suffers from that dramatic speed reduction as well. So good luck replicating virtually anything over that slow of a link. So if you're trying to replicate data from a region outside of China into China, you have to understand that sometimes, that replication link is going to break, other times it's going to wind up being incredibly slow, and still other times if you want to get around that, you just can't use encryption and just subject all of your valuable corporate data to inspection, not just by the Chinese government, but by anyone who can get a sniff of that traffic between the two end points between which it's speaking.
So that's generally a nonstarter as Verner says in his t-shirts at the re-invent keynote he loves giving, "encrypt everything." I have a spoof t-shirt that says, "encrypt everything unless it's hard." But traversing international borders is one of those times you absolutely want to have things encrypted. It's the only thing that really makes sense.
So what's the takeaway here? What does this mean for you that's actionable if you're needing to do business inside of mainland China? The honest answer is, this is complex enough and there's enough shades of nuance and technical and policy based challenges, that I would strongly recommend consulting with someone who has experienced this before. I'm not that person. I generally try to avoid dealing with complex geopolitical issues when I'm trying to troubleshoot networking issues at the same time. I have nothing to sell you in this context. If you are trying to solve for this problem, do reach out to me on Twitter @quinnypig
, or email me, firstname.lastname@example.org
, and I'll be thrilled to do a little digging for you if I can't come up with another solution by the time this airs.
So in short, the Chinese networking environment is radically different than you're going to find anywhere else on the planet. If you're doing business there, you need to do an awful lot of research, you need to go in prepared and you probably want to have competent legal counsel who understands the intricacies of doing business cross-border in the Chinese market. In short, if this applies to you, good luck because here be dragons.
I'm Corey Quinn. This is the AWS Morning Briefs, Networking In The Cloud podcasting mini series sponsored exclusively by ThousandEyes
. My thanks to them for their generous sponsorship. My thanks to you for listening. And as always, please feel free to leave an excellent review in Apple podcasts, whether or not you've actually enjoyed this episode at all. Thanks.
Announcer: This has been a HumblePod production. Stay humble.