Subscribe
Share
Share
Embed
Matthew Prince, Co-founder and CEO of Cloudflare, has joined Corey this week to talk about cloud costs and the multiplicitous things cloud costs are associated with. Matthew breaks down what exactly Cloudflare is up to, and how they are handling their clients cloud costs needs. Those needs that, according to Matthew, “over time and at scale, bandwidth costs go to zero.”
Matthew and Corey focus on Cloudflare as a company, and the retinue of options they are bringing to their clients. They discuss their mutual concerns with some of AWS costs to include the achilles heel of egress charges. Matthew discusses Cloudflare’s suite of tools they’ve built internally, and have been making available over time. Check out the conversation for more details!
Screaming in the Cloud with Corey Quinn features conversations with domain experts in the world of Cloud Computing. Topics discussed include AWS, GCP, Azure, Oracle Cloud, and the "why" behind how businesses are coming to think about the Cloud.
Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.
Corey: Writing ad copy to fit into a 30 second slot is hard, but if anyone can do it the folks at Quali can. Just like their Torque infrastructure automation platform can deliver complex application environments anytime, anywhere, in just seconds instead of hours, days or weeks. Visit Qtorque.io today and learn how you can spin up application environments in about the same amount of time it took you to listen to this ad.
Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate: is it your application code, users, or the underlying systems? I’ve got five bucks on DNS, personally. Why scroll through endless dashboards, while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other, which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at Honeycomb.io/screaminginthecloud. Observability, it’s more than just hipster monitoring.
Corey: Welcome to Screaming in the Cloud, I’m Corey Quinn. Today, my guest is someone I feel a certain kinship with, if for no other reason than I spend the bulk of my time antagonizing AWS incredibly publicly. And my guest periodically descends into the gutter with me to do the same sort of things. The difference is that I’m a loudmouth with a Twitter account and Matthew Prince is the co-founder and CEO of Cloudflare, which is, of course, publicly traded. Matthew, thank you for deigning to speak with me today. I really appreciate it.
Matthew: Corey, it’s my pleasure, and appreciate you having me on.
Corey: So, I’m mostly being facetious here, but not entirely, in that you have very publicly and repeatedly called out some of the same things I love calling out, which is AWS’s frankly egregious egress pricing. In fact, that was a title of a blog post that you folks put out, and it was so well done I’m ashamed I didn’t come up with it myself years ago. But it’s something that is resonating with a large number of people in very specific circumstances as far as what their company does. Talk to me a little bit about that. Cloudflare is a CDN company and increasingly looking like something beyond that. Where do you stand on this? What got you on this path?
Matthew: I was actually searching through really old emails to find something the other day, and I found a message from all the way back in 2009, so actually even before Michelle and I had come up with a name for Cloudflare. We were really just trying to understand the pricing on public clouds and breaking it all down. How much does the compute cost? How much does storage cost? How much does bandwidth cost?
And we kept running the numbers over and over and over again, and the storage and compute costs actually seemed relatively reasonable and you could understand it, but the economics behind the bandwidth just made no sense. It was clear that as bandwidth usage grew and you got scale that your costs eventually effectively went to zero. And I think it was that insight that led to us starting Cloudflare. And the self-service plans at Cloudflare have always been unlimited bandwidth, and from the beginning, we didn’t charge for bandwidth. People told us at the time we were crazy to not do that, but I think that that realization, that over time and at scale, bandwidth costs do go to zero is really core to who Cloudflare is.
Cloudflare launched a little over 11 years ago now, and as we’ve watched the various public clouds and AWS in particular just really over that same 11 years not only not follow the natural price of bandwidth down, but really hold their costs steady. At some point, we’ve got a lot of mutual customers and it’s a complaint that we hear from our mutual customers all the time, and we decided that we should do something about it. And so that started four years ago, when we launched the Bandwidth Alliance, and worked with almost all the major public clouds with the exception of Amazon, to say that if someone is sending traffic from a public cloud network to Cloudflare’s network, we’re not going to charge them for the bandwidth. It’s going across a piece of fiber optic cable that yeah, there’s some cost to put it in place and maybe there’s some maintenance costs associated with it, but there’s not—
Corey: And the equipment at the end costs money, but it’s not cloud cost; it just cost on a per second, every hour of your lifetime basis. It’s a capital expense that is amortized across a number of years et cetera, et cetera.
Matthew: And it’s a fixed cost. It’s not a variable cost. You put that fiber optic cable and you use a port on a router on each side. There’s cost associated with that, but it’s relatively de minimis. And so we said, “If it’s not costing us anything and it’s not costing a cloud provider anything, why are we charging customers for that?”
And I think it’s an argument that resonated with almost every other provider that was out there. And so Google discounts traffic when it’s sent to us, Microsoft discounts traffic when it’s sent to us, and we just announced that Oracle has joined this discounting their traffic, which was already some of the most cost-effective bandwidth from any cloud provider.
Corey: Oh, yeah. Oracle’s fantastic. As you were announced, I believe today, the fact that they’re joining the Bandwidth Alliance is both fascinating and also, on some level, “Okay. It doesn’t matter as much because their retail starting cost is 10% of Amazon’s.” You have to start pushing an awful lot of traffic relative to what you would do AWS before it starts to show up. It’s great to see.
Matthew: And the fact that they’re taking that down to effectively zero if you’re using us is even better, right? And I think it again just illustrates how Amazon’s really alone in this at being so egregious in how they do that. And it’s, when we’ve done the math to calculate what their markups are, it’s almost 80 times what reasonable assumptions on what their wholesale costs are. And so we really do believe in fighting for our customers and being customer-centric, and this seems like a place where—again, Amazon provides an incredible service and so many things, but the data transfer costs are just completely outrageous. And I’m glad that you’re calling them out on it, and I’m glad we’re calling them out on it and I think increasingly they look isolated and very anti-customer.
Corey: What’s interesting to me is that ingress to AWS at all the large public tier-one cloud providers is free. Which has led, I think, to the assumption—real or not—that bandwidth doesn’t actually cost anything, whereas going outbound, all I can assume is that one day, some Amazon VP was watching a rerun of Meet the Parents and they got to the line where Ben Stiller says, “Oh, you can milk anything with nipples,” and said, “Holy crap. Our customers all have nipples; we can milk them with egress charges.” And here we are. As much as I think the cloud empowers some amazing stuff, the egress charges are very much an Achilles heel to a point where it starts to look like people won’t even
consider public cloud for certain workloads based upon that.
People talk about how Netflix is a great representation of the ideal AWS customers. Yeah, but they don’t stream a single byte to customers from AWS. They have their own CDN called Open Connect that they put all around the internet, specifically for that use case because it would bankrupt them otherwise.
Matthew: If you’re a small customer, bandwidth does cost something because you have to pay someone to do the work of interconnecting with all of the various networks that are out there. If you start to be, though, a large customer—like a Cloudflare, like an AWS, like an Azure—that is sending serious traffic to the internet, then it starts to actually be in the interest of ISPs to directly interconnect with you, and the costs of your bandwidth over time will approach zero. And that’s the just economic reality of how bandwidth pricing works. I think that the confusion, to some extent, comes from all of us having bought our own home internet connection. And I think that the fact that you get more bandwidth up in most internet connections, and you get down, people think that there’s some physics, which is associated with that.
And there are; that turns out just to be the legacy of the cable system that was really designed to send pictures down to your—
Corey: It wasn’t really a listening post. Yeah.
Matthew: Right. And so they have dedicated less capacity for up and again, in-home network connections, that makes a ton of sense, but that’s not how internet connections work globally. In fact, you pay—you get a symmetric connection. And so if they can demonstrate that it’s free to take the traffic in, we can’t figure out any reason that’s not simply about customer lock-in; why you would charge to take data out, but you wouldn’t charge to put it in. Because actually cost more from writing data to a disk, it costs more than reading it from a disk.
And so by all reasonable accounts, if they were actually charging based on what their costs were, they would charge for ingress but they want to charge for egress. But the approach that we’ve taken is to say, “For standard bandwidth, we just aren’t going to charge for it.” And we do charge for if you use our premium routing services, which is something called Argo, but even then it’s relatively cheap compared with what is just standard kind of internet connectivity that’s out there. And as we see more of the clouds like Microsoft and Google and Oracle show that this is a place where they can be much more customer-centric and customer-friendly, over time I’m hopeful that will put pressure on Amazon and they will eliminate their egress fees.
Corey: People also tend to assume that when I talk about this, that I’m somehow complaining about the level of discounting or whatnot, and they yell at me and say, “Oh, well, you should know by now, Corey, that no one at significant scale pays retail pricing.” “Thanks, professor. I appreciate that, but four years ago, or so I sat down with a startup founder who was sketching out the idea for a live video streaming service and said, ‘There’s something wrong with my math because if I built this on AWS—which he knew very well, incidentally—it looks like it would cost me at our scale of where we’re hoping to hit $65,000 a minute.’” And I checked and yep, sure enough, his math was not wrong, so he obviously did not build his proof of concept on top of AWS. And the last time I checked, they had raised several 100 million dollars in a bunch of different funding rounds.
That is a company now that will not be on AWS because it was never an option. I want to talk as well about your announcement of R2, which is just spectacular. It is—please correct me if I get any of this wrong—it’s an object store that lives in your existing distributed-points-of-presence-slash-data-centers-slash-colo-slash-a-bunch-of-computers-in-fancy-warehouse-rooms-with-the-lights-are-always-on-And-it’s-always-cold-and-noisy. And people can store data there—
Matthew: [crosstalk 00:10:23] aisles it’s cold; in the other aisles, it’s hot. But yes.
Corey: Exactly. But it turns out when you lurk around to the hot aisle, that’s not where all the buttons are and the things you’re able to plug into, so it’s freeze or sweat, and there’s never a good answer. But it’s an object store that costs a fair bit less than retail pricing for Amazon S3, or most other object stores out there. Which, okay, great. That’s always good to see competition in the storage space, but specifically, you’re not charging any data transfer costs whatsoever for doing this. First, where did this come from?
Matthew: So, we needed it ourselves. I think all of the great products at Cloudflare start with an internal need. If you look at why do we build our zero-trust solutions? It’s because we said we needed a security solution that was fast and reliable and secure to protect our employees as they were going out and using the internet.
Why did we build Cloudflare Workers? Because we needed a very flexible compute platform where we could build systems ourselves. And that’s not unique to us. I mean, why did Amazon build AWS? They built it because they needed those tools in order to continue to grow and expand as quickly as possible.
And in fact, I think if you look at the products that Google makes that are really great, it ends up being the ones that Google’s employees use themselves. Gmail started as Caribou once upon a time, which was their internal email system. And so we needed an object store and the sometimes belligerent CEO of Cloudflare insisted that our team couldn’t use any of the public cloud object stores. And so we had to build it.
That was the start of it and we’ve been using it internally for products over time. It powers, for example, Cloudflare Images, it powers a lot of our streaming video services, and it works great. And at some point, we said, “Can we take this and make it available to everyone?” The question that you’ve asked on Twitter, and I think a lot of people reasonably ask us, “What’s the catch?”
Corey: Well, in my defense, I think it’s fair. There was an example that I gave of, “Okay, I’m going to go ahead and keep—because it’s new, I don’t trust new object stores. Great. I’m going to do the same experiment twice, keep one the pure AWS story and the other, I’m just going to add Cloudflare R2 to the mix so that I have to transfer out of AWS once.” For a one gigabyte file that gets shared out for a petabyte’s worth of bandwidth, on AWS it costs roughly $52,000 to do that. If I go with the R2 solution, it cost me 13 cents, all of which except for a penny-and-a-half are AWS charges. And that just feels—when you’re looking at that big of a gap, it’s easy to look at that and think, “Okay, someone is trying to swindle me somewhere. And when you can’t spot the sucker, it’s probably me. What’s the catch?”
Matthew: I guess it’s not really a catch; it’s an explanation. We have been able to drive our bandwidth costs down low enough that in that particular use case, we have to store the file, and that, again, that—there’s a hard disk in there and we replicate it to make sure that it’s available so it’s not just one hard disk, but it’s multiple hard disks in various places, but that amortized over time, isn’t that big a cost. And then bandwidth is effectively zero. And so if we can do that, then that’s great.
Maybe a different way of framing the question is like, “Why would we do that?” And I think what we see is that there is an opportunity for customers to be able to use the best of various cloud providers and hook the different parts together. So, people talk about multi-cloud all the time, and for a while, the way that I think people thought about that was you take the exact same workload and you run it in Azure and AWS. That turns out not to be—I mean, maybe some people do that, but it’s super rare and it’s incredibly hard.
Corey: It has been a recurring theme of most things I say where, by default, that is one of the dumbest things I can imagine.
Matthew: Yeah, that isn’t good. But what people do want to do is they want to say, “Listen, there’s some really great services that Amazon provides; we want to use those. And there’s some really great services that Azure provides, and we want to use those. And Google’s got some great machine learning, and so does IBM. And I want to sort of mix and match the various pieces together.”
And the challenge in doing that is the egress fees. If everyone just had a detente and said there's going to be no egress fees for us to be able to hook these various [pits 00:14:48] together, then you would be able to take advantage of a lot of the different technologies and we would actually get stronger applications. And so the vision of what we’re trying to build is how can we be the fabric that can stitch the various cloud providers together so that you can do that. And when we looked at that, and we said, “Okay, what’s the path to getting there?” The big place where there’s the just meatiest cost on egress fees is object stores.
And so if you could have a centralized object store, and you can say then from that object go use whatever the best service is at Amazon, go use whatever the best service is at Google, go use whatever the best service is at Azure, that then allows, I think, actually people to take advantage of the cloud in a way which is what people really should mean when they talk about multi-cloud. Which is, there should be competition on the various features themselves, and you should be able to pick and choose the best of all of the different bits. And I think we as consumers then benefit from that. And so when we’re looking at how we can strategically enable that future, building an object store was a real key part of that, and that’s part of what we’re doing. Now, how do we make money off of that? Well, there’s a little bit off the storage, and again, even [laugh]—
Corey: Well, that is the Amazonian answer there. It’s like, “Your margin is my opportunity,” is a famous Bezos quote, and I figure you’re sitting there saying, “Ah, it would cost $52,000 to do that in Amazon. Ah, we can make a penny-and-a-half.” That’s very Amazonian, you could probably get hired over there with that philosophy.
Matthew: Yeah. And this is a commodity service, just [laugh] storing data. If you look across the history of what Cloudflare has done, in 2014, we made encryption free because it’s absurd to pay for math, right? I mean, it’s just crazy right?
Corey: Or to pay for security as a value-add. No, that should be baked into whatever you’re doing, in an ideal world.
Matthew: Domain registration. Like, it’s writing something down in a ledger. It’s a commodity; of course it should go to whatever the absolute cost is. On the other hand, there are things that we do that aren’t commodities where we are able to better protect people because we see so much traffic, and we’ve built the machine learning models, and we’ve done those things, and so we charge for those things. So commodities, we think over time, go to effectively, whatever their cost is, and then the value is in the actual intelligent services that are on top of it.
But an object store is a commodity and so we should be trying to drive that pricing down. And in the case of bandwidth, it’s effectively free for us. And so if we can be that fabric that connects the different class together, I think that makes sense is a strategy for us and that’s why R2 made a ton of sense for us to build and to launch.
Corey: There seems to be a lack of ability for lots of folks, at least on the internet to imagine a use case other than theirs. I cheated by being a consultant, I get to borrow other people’s use cases at a high degree of turnover. But the question I saw raised was, “Well, how many workloads really do that much egress from static objects that don’t change? Doesn’t sound like there’d be a whole lot of them.” And it’s, “Oh, my sweet summer child. Sure, your app doesn’t do a lot of that, but let me introduce it to my friends who are hosting videos on their website, for example, or large images that get accessed a whole bunch of times; things that are written once and then read forever by the internet.”
Matthew: And we sit in a position where because of the role that Cloudflare plays where we sit in front of a number of these different cloud providers, we could actually look at the use cases and the data, and then build products in order to solve that. And that’s why we started with Workers; that’s why we then built the KV store that was on top of that; we built object-store next. And so you can see as we’re sort of marching through these things, it is very much being informed by the data that we actually see from real customers. And one of the things that I really like about R2 is in exactly the example that you gave where you can keep everything in S3; you can set R2 in front of it and put it in slurp mode, and effectively it just—as those objects get pulled out, it starts storing them there. And so the migration path is super easy; you don’t have to actually change anything about your application and will cut your bills substantially.
And so I think that’s the right thing to enable a multi-cloud world where, again, it’s not you’re running the exact same workload in different places, but you get to take advantage of the really great tack that all of these companies are building and use that. And then the companies will compete on building that tech well. So, it’s not just about how do I get the data in and then kind of underinvest in all of the different services that I provide. It’s how can we make sure that on a service-by-service basis, you actually are having real competition over time. And again, I think that’s the right thing for customers, and absolutely R2 might not be the right thing for every use case that’s out there, but I think that it wi—enabling more competition is going to make the cloud better for everyone.
Corey: Oh, yeah. It’s always fun hearing it from Amazonians. It’s, “You have a service that talks to satellites in orbit. You really think that’s a general-purpose thing that every company out there has to deal with?” No. Well, not yet, anyway.
It also just feels to me like their transfer approach is antithetical to almost every other aspect of how they have built their cloud. Amazonians have told me repeatedly—I believe them—that their network is effectively magic. The fact that you can get near line rate between any two points without melting various [unintelligible 00:20:14], which shows that there was significant thought, work, effort, planning, technology, et cetera, put into the network. And I don’t dispute that. But if I’m trying to build a workload and put it inside of AWS, I can control how it performs tied to budget; I can have a lot of RAM for things that are memory intensive, or I can have a little RAM; I can have great CPU performance or terrible CPU performance.
The challenge with data transfer is it is uniformly great. “I want to get that data over there super quickly.” Yeah, awesome. I’m fine paying a premium for that. But I have this pile of data right here. I want to get it over there, ideally by Tuesday. There’s no good way to do that, even with their Snowball—or Snow Family devices—when you fill them with data and send them into AWS, yeah, that’s great. Then you just pay for the use of the device.
Use them to send data out of AWS, they tack on an additional per-gigabyte fee for getting the data out. You’re training as a lawyer, you went to the same law school that my wife did, the University of Chicago, which, oh, interesting stories down that path. But if we look at this, my argument is that the way to do an end-run around this is to sue Amazon for something, and then demand access to the data you have living in their environment during discovery. Make them give it to you for free, though, they’d probably find a way to charge it there, too. It’s just a complete lack of vision and lack of awareness because it feels like they’re milking a cash cow until it dies.
Matthew: Yeah, they probably would charge for it and you’d also have to pay a lot of lawyers. So, I’m not sure that’s the cost [crosstalk 00:21:44]—
Corey: Its only works above certain volumes, I figure.
Matthew: I do think that if your pricing strategy is designed to lock people in to prevent competition, then that does create other challenges. And there are certainly some University of Chicago law professors out there that have spent their careers arguing why antitrust laws don’t make any sense, but I think that this is definitely one of those areas where you can see very clearly that customers are actually being harmed by the pricing strategy that’s there. And the pricing strategy is not tied in any way to the underlying costs which are associated with that. And so I do think that, especially as you see other providers in the space—like Oracle—taking their bandwidth costs to effectively zero, that’s the sort of thing that I think will have regulators start to scratch their heads. If tomorrow, AWS took egress costs to zero, and as a result, R2 was not as advantaged as it is today against them, you know, I think there are a lot of people who would say, “Oh, they showed Cloudflare.” I would do a happy dance because that’s the best thing [thing they can do 00:22:52] for our customers.
Corey: Our long-term goals, it sounds like, are relatively aligned. People think that I want to see AWS reign ascendant; people also say I want to see them burning and crashing into the sea, and neither one of those are true. What I want is, I want someone in a few years from now to be doing a startup and trying to figure out which cloud provider they should pick, and I want that to be a hard decision. Ideally, if you wind up reducing data transfer fees enough, it doesn’t even have to be only one. There are stories that starts to turn into an actual realistic multi-cloud story that isn’t, at its face, ridiculous. But right now, you have to pick a horse and ride it, for a variety of reasons. And I don’t like that.
Matthew: It’s entirely egress-based. And again, I think that customers are better off if they are able to pick who is the best service at any time. And that is what encourages innovation. And over time, that’s even what’s good for the various cloud providers because it’s what keeps them being valuable and keeps their customers thinking that they’re building something which is magical and that they aren’t trapped in the decision that they made, which is when we talk to a lot of the customers today, they feel that way. And it’s I think part of why something like R2 and something like the Bandwidth Alliance has gotten so much attention because it really touches a nerve on what’s frustrating customers today. And if tomorrow Amazon announced that they were eliminating egress fees and going head-to-head with R2, again, I think that’s a wonderful outcome. And one that I think is unlikely, but I would celebrate it if it happened.
Corey: This episode is sponsored by our friends at Oracle Cloud. Counting the pennies, but still dreaming of deploying apps instead of "Hello, World" demos? Allow me to introduce you to Oracle's Always Free tier. It provides over 20 free services and infrastructure, networking databases, observability, management, and security.
And - let me be clear here - it's actually free. There's no surprise billing until you intentionally and proactively upgrade your account. This means you can provision a virtual machine instance or spin up an autonomous database that manages itself all while gaining the networking load, balancing and storage resources that somehow never quite make it into most free tiers needed to support the application that you want to build.
With Always Free you can do things like run small scale applications, or do proof of concept testing without spending a dime. You know that I always like to put asterisks next to the word free. This is actually free. No asterisk. Start now. Visit https://snark.cloud/oci-free that's https://snark.cloud/oci-free.
Corey: My favorite is people who don’t do research on this stuff. They wind up saying, “Oh, yeah. Cloudflare is saying that bandwidth is a fixed cost. Of course not. They must be losing their shirt on this.”
You are a publicly-traded company. Your gross margins are 76% or 77%, depending upon whether we’re talking about GAAP or non-GAAP. Point being, you are clearly not selling this at a loss and hoping to make it up in volume. That’s what a VC-backed company does. Is something that is real and as accurate.
I want to, on some level, I guess, low-key apologize because I keep viewing Cloudflare through a lens that is increasingly inaccurate, which is as a CDN. But you’ve had Cloudflare Workers for a while, effectively Functions as a Service that run at the edge, which has this magic aura around it, that do various things, which is fascinating to me. You’re launching R2; it feels like you are in some ways aiming at becoming a cloud provider, but instead of taking the traditional approach of building it from the region’s outward, you’re building it from the outward in. Is that a fair characterization?
Matthew: I think that’s right. I think fundamentally what Cloudflare is, is a network. And I remember early on in the pandemic, we did a series of fireside chats with people we thought we could learn from. And so was everyone from Andre Iguodala, the basketball player, to Mark Cuban, the entrepreneur, to we had a [unintelligible 00:25:56] governor and all kinds of things. And we these were just internal on off the record.
And I got to do one with Eric Schmidt, the former CEO of Google. And I said, “You know, Eric, one of the things that we struggle with is describing what is Cloudflare.” And without hesitation, he said, “Oh, that’s easy. You’re the network I plug into and don’t have to worry about anything else.” And I think that’s better than I could say it, myself, and I think that’s what it is that we fundamentally are: we’re the network that fits together.
Now, it turns out that in the process of being that network and enabling that network, we are going to build things like R2, which start to be an object store and starts to sort of step into some of the cloud provider space. And Workers is really just a way of programming that network in order to do that, but it turns out that there are a bunch of workloads that if you move them into the network itself, make sense—not going to be every workload, but a lot of workloads that makes sense there. And again, I think that you can actually be very bullish on all of the big public cloud providers and bullish on Cloudflare at the same time because what we want to do is enable the ability for people to mix and match, and change, and be the fabric that connects all of those things together. And so over time, if Amazon says, “We’re going to drop egress fees,” it may be that R2 isn’t a product that exists—I don’t think they’re going to do that, so I think it’s something that is going to be successful for us and get a lot of new users to us—but fundamentally, I think that where the traditional public clouds think of themselves as the place you put data and you process data, I think we think of ourselves as the place you move data. And that’s somewhat different.
That then translates into it as we’re building out the different pieces, where it does feel like we’re building from the outside in. And it may be that over time, that put versus move distinction becomes narrower and narrower as we build more and more services like R2, and durable objects, and KV, and we’re working on a database, and all those things. And it could be that we converge in a similar place.
Corey: One thing I really appreciate about your vision because it is so atypical these days, is that you aren’t trying to build the multifunction printer of companies. You are not trying to be all things to all people in every scenario. Which is impossible to do, but companies are still trying their level best to do it. You are staking out the bounds of where you were willing to start and where you’re willing to stop, in a variety of different ways. I would be—how do I put it?—surprised if you at some point in the next five years come out with, “And this is our own database that we have built out that directly competes with the following open-source project that we basically have implemented their API and gone down that particular path.” It does not sound like it is in your core wheelhouse at that point. You don’t need—to my understanding—to write your own database engine in order to do what you do.
Matthew: Maybe. I mean, we actually are kind of working on a database because—
Corey: Oh, no, here we go again.
Matthew: [laugh]—and yeah—in a couple of different ways. So, the first way is, we want to make sure that if you’re using Workers, you can connect to whatever database you want to use anywhere in the world. And that’s something that’s coming and we’ll be there. At the same time, the challenge of distributed computing turns out not to be the computing, it turns out to be the data and figuring out how to—CAP theorem is real, right? Consistency, Availability, and Partition tolerance; you can pick any two out of the three, but you can’t get all three.
And so you there’s always going to be some trade-off that’s there. And so we don’t see a lot of good examples. There’s some really cool companies that are working on things in the space, but we don’t see a lot of really good examples of who has built a database that can be run on a distributed workload system, like Cloudflare to it do well. And so our team internally needs that, and so we’re trying to figure out how to build it for ourselves, and I would imagine that after we build it for ourselves—if it works the way we expect it will—that that will then be something that we open up.
Our motivation and the way we think about products is we need to build the tools for our own team. Our team itself is customer zero, and then some of those things are very specific to us, but every once in a while, when there are functions that makes sense for others, then we’ll build them as well. And that does maybe risk being the multifunction printer, but again, I think that because the customer for that starts with ourselves, that’s how we think about it. And if there’s someone else’s making a great tool, we’ll use that. But in this case, we don’t see anyone that’s built a multi-tenant, globally-distributed, ACID-compliant relational database.
Corey: I can’t let it pass on challenge. Sure they have, and you’re running it yourself. DNS: the finest database in the world. You stuff whatever you want to text records, and now you have taken a finely crafted wrench and turned it into a barely acceptable hammer, which is what I love about doing that terrible approach. Yeah, relational is not going to quite work that way. But—
Matthew: Yes. That’s a fancy key-value store, right? So—and we’ve had that for a long time. As we’re trying to build those things up, the good news is that, again, we’ve run data at scale for quite some time and proven that we can do it efficiently and reliably.
Corey: There’s a lot that can be said about building the things you need to deliver your product to customers. And maybe a database is a poor example here, but I don’t see that your motivation in this space is to step into something completely outside your areas of expertise solely because there’s money to be made over there. Well, yeah, fortune passes everywhere. The question is, which are you best positioned to wind up delivering an actual transformative solution to that space, and what parts of it are just rent-seeking where it’s okay, we’re going to go and wherever the money is, we’re chasing that down.
Matthew: Yeah, we’re still a for-profit business, and we’ve been able to grow revenue well, but I think it is that what motivates us and what drives us comes back to our mission, which is how do you help build a better internet? And you can look at every single thing that we’ve done, and we try to be very long-term-oriented. So, for instance, when we in 2014 made encryption free, the number one reason at the time, when people upgraded for the free version of our service, the paid version of our service is they got encryption for that. And so it was super scary to say, “Hey, we’re going to take the biggest feature and give it away for free,” but it was clearly the direction of history and we wanted to be on the right side of history. And we considered it a bug that the internet wasn’t built in an encrypted way from the beginning.
So, of course, that was going to head that direction. And so I think that we and then subsequently Let’s Encrypt, and a bunch of others have said, it’s absurd that you’re charging for math. And again, I think that’s a good example of how we think about products. And we want to continue to disrupt ourselves and take the things that once upon a time were reserved for our customers that spend $10 million-plus with us, and we want to keep pushing those things down because, over time, the real opportunity is if you do right by customers, there will be plenty of ways that you can earn some of their budget. And again, we think that is the long-term winning strategy.
Corey: I would agree with this. You’re not out there making sneakers and selling them because you see people spend a lot of money on that; you’re delivering value for customers. I say this as one of your paying customers. I have zero problem paying you every month like clockwork, and it is the least cloud-like experience because I know exactly what the bill is going to be in advance, which is apparently not how things should be done in this industry, yadda, yadda, yadda. It is a refreshingly delightful experience every time.
The few times I’ve had challenges with the service, it has almost always been a—I’ll call it a documentation gap, where the way it was explained in the formal documentation was not how I conceptualize things, which, again, explaining what these complex things are to folks who are not steeped in certain areas of them is always going to be a challenge. But I cannot think back to a single customer service failure I’ve had with you folks. I can’t look back at any point where you have failed me as a customer, which is a strange thing to say, given how incredibly efficient I am at stumbling over weird bugs.
Matthew: Terrific to have you as a customer. We are hardly perfect and we make mistakes, but one of the things I think that we try to do and one of the core values of Cloudflare is transparency. If I think about, like, the original sins of tech, a lot of it is this bizarre secrecy which pervades the entire industry. When we make mistakes, we talk about them, and we explain them. When there’s an error, we don’t throw up a white page; we put up a page that has our logo on it because we want to own it.
And that sometimes gets blowback because you’re in front of it, but again, I think it’s the right thing to do for customers. And it’s and I think it’s incredibly important. One of the things that’s interesting is you mentioned that you know what your bill is going to be. If you go back and look at the history of hosting on the internet, in the early days of internet hosting, it looks a lot like AWS.
Corey: Oh, 95th percentile transit billing; go for one five minutes segment over and boom, your bill explodes. Oh, I remember those days. Unkindly.
Matthew: And it was super complicated. And then what happened is the hosting world switched from this incredibly complicated billing to much more simplified, predictable, unlimited bandwidth with maybe some asterisks, but largely that was in place. And then it’s strange that Amazon came along and then has brought us back to the more complicated world that’s out there. I would have predicted that that’s a sine wave—
Corey: It has to be. I mean—
Matthew: —and it’s going to go back and forth over time. But I would have predicted that we would be more in the direction of coming back toward simplify, everything included. And again, I think that’s how we’ve priced our things from the beginning. I’m surprised that it has held on as long as it has, but I do think that there’s going to be an opportunity for—and I don’t think Amazon will be the leader here, but I think there will be an opportunity for one of the big clouds.
And again, I think Oracle is probably doing this the best of any of them right now—to say, “How can we go away from that complexity? How can we make bills predictable? How can we not nickel and dime everything, but allow you to actually forecast and budget?” And it just seems like that’s the natural arc of history, and we will head back toward that. And, again, I think we’ve done our part to push that along. And I’m excited that other cloud providers seem to be thinking about that now as well.
Corey: Oh, yeah. What I do with fixing AWS bills is the same thing folks were doing in the 70s and 80s with long-distance bills for companies. We’re definitely hitting that sine wave. I know that if I were at AWS in a leadership role, I would be actively embarrassed that the company that is delivering a better customer experience around financial things is Oracle of all companies, given their history of audits and surprising people and the rest. It is ridiculous to me.
One last topic that I want to cover with you before we call it an episode is, back in college, you had a thesis that you have done an excellent job of effectively eliminating from the internet. And the theme of this, to my understanding, was that the internet is a fad. And I am so aligned with that because I’m someone who has said for years that emerging technologies are fads. I’ve said it about cloud, about virtualization, about containers. And I just skipped Kubernetes. And now I’m all-in on serverless, which means, of course it’s going to fail because I’m always wrong on these things. But tell me about that.
Matthew: When I was seven years old in 1980, my grandmother gave me an Apple ][+ computer for Christmas. And I took to it like a just absolute duck to water and did things that made me very popular in junior high school, like going to computer camp. And my mom used to sign up for continuing education classes at the local university in computer science, and basically sneak me in, and I’d do all the homework and all that. And I remember when I got to college, there was a small group of students that would come around and help other students set their computer up, and I had it all set up and was involved. And so, got pretty deeply involved in the computer science program at college.
And then I remember there was a group of three other students—so they were four of us—and they wanted to start an online digital magazine. And at the time, this was pre-web, or right in the early days of the web; it was sort of nineteen… ninety-three. And we built it originally on old Apple technology called HyperCard. And we used to email out the old HyperCard stacks. And the HyperCard stacks kept getting bigger and bigger and bigger, and we’d send them out to the school so [laugh] that we—so we kept crashing the mail servers.
But the college loved this, so they kept buying bigger and bigger mail servers. But they were—at some point, they said, “This won’t scale. You got to switch technologies.” And they introduced us to two different groups. One was a printer company based out in San Francisco that had this technology called PDF. And I was a really big fan of PDF. I thought PDF was the future, it was definitely going to be how everything got published.
And then the other was this group of dorky graduate students at the University of Illinois that had this thing called a browser, which was super flaky, and crashed all the time, and didn’t work. And so of the four of us, I was the one who voted for PDF and the other three were like, “Actually, I think this HTML thing is going to be a hit.” And we built this. We won an award from Wired—which was only a print magazine at the time—that called us the first online-only weekly publication. And it was such a struggle to get anyone to write for it because browsers sucked and, you know, trying to get students on campus, but no one on campus cared.
We would get these emails from the other side of the world, where I remember really clearly is this—in broken English—email from Japan saying, “I love the magazine. Please keep writing more for the magazine.” And I remember thinking at the time, “Why do I care if someone in Japan is reading this if the girl down the hall who I have a crush on isn’t?” Which is obviously what motivates dorky college students like myself. And at that same time, you saw all of this internet explosion.
I remember the moment when Netscape went public and just blew through all the expectations. And it was right around the time I was getting ready to graduate for college, and I was kind of just burned out on the entire thing. And I thought, “If I can’t even get anyone to write for this dopey magazine and yet we’re winning awards, like, this stuff has to all just be complete garbage.” And so wrote a thesis on—ehh, it was not a very good [laugh] thesis. It’s—but one of the things I said was that largely the internet was a fad, and that if it wasn’t, that it had some real risks because if you enabled everyone to connect with whatever their weird interests and hobbies were, that you would very quickly fall to the lowest common denominator. And predicted some things that haven’t come true. I thought for sure that you would have both a liberal and conservative search engine. And it’s a miracle to this day, I think that doesn’t exist.
Corey: Now, that you said it, of course, it’s going to.
Matthew: Well, I don’t know I’ve… [sigh] we’ll see. But it is pretty amazing that Google has been able to, again, thread that line and stay largely apolitical. I’m surprised there aren’t more national search engines; the fact that it only Russia and China have national search engines and France and Germany don’t is just strange to me. It seems like if you’re controlling the source of truth and how people find it, that seems like something that governments would try and take over. There are some things that in retrospect, look pretty wise, but there were a lot more things that looked really, really stupid. And so I think at some level, I had to build Cloudflare to atone for that stupidity all those years ago.
Corey: There’s something to be said for looking back and saying, “Yeah, I had an opinion, and with the light of new information, I am changing my opinion.” For some reason, in some circles, it feels like that gets interpreted as a sign of weakness, but I couldn’t disagree more, it’s, “Well, I had an opinion based upon what I saw at the time. Turns out, I was wrong, and here we are.” I really wish more people were capable of doing that.
Matthew: It’s one of the things we test for in hiring. And I think the characteristic that describes people who can do that well is really empathy. The understanding that the experiences that you have lead you to have a unique set of insights, but they also create a unique set of blind spots. And it’s rare that you find people that are able to do that. And whenever you do—whenever we do we hire them.
Corey: To that end, as far as hiring and similar topics go, if people want to learn more about how you view things, and how you see the world, and what you’re releasing—maybe even potentially work with you—where can they find you?
Matthew: [laugh]. So, the joke, sometimes, internal at Cloudflare is that Cloudflare is a blogging company that runs this global network just to have something to write about. So, I think we’re unlike most corporate blogs, which are—if our corporate blog were typical, we’d have articles on, like, “Here are the top six reasons you need a fast website,” which would just be, you know, shoot me. But instead, I think we write about the things that are going on online and our unique view into them. And we have a core value of transparency, so we talk about that. So, if you’re interested in Cloudflare, I’d encourage you to—especially if you’re of the sort of geekier variety—to check out blog.cloudflare.com, and I think that’s a good place to learn about us. And I still write for that occasionally.
Corey: You’re one of the only non-AWS corporate blogs that I pay attention to, for that exact reason. It is not, “Oh, yay. More content marketing by folks who just feel the need to hit a quota as opposed to talking about something valuable and interesting.” So, it’s appreciated.
Matthew: The secret to it was we realized at some point that the purpose of the blog wasn’t to attract customers, it was to attract potential employees. And it turns out, if you sort of change that focus, then you talk to people like their peers, and it turns out then that the content that you create is much more authentic. And that turns out to be a great way to attract customers as well.
Corey: I want to thank you for taking so much time out of your day to speak with me. I really appreciate it.
Matthew: Thanks for all you’re doing. And we’re very aligned, and keep fighting the good fight. And someday, again, we’ll eliminate cloud egress fees, and we can share a beer when we do.
Corey: I will absolutely be there for it. Matthew, Prince, CEO, and co-founder of Cloudflare. I’m Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated this podcast, please leave a five-star review on your podcast platform of choice along with a rambling comment explaining that while data packets into a cloud provider are cheap and crappy, the ones being sent to the internet are beautiful, bespoke, unicorn snowflakes, so of course they cost money.
Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.
Announcer: This has been a HumblePod production. Stay humble.