The core of the decision making for me is in every team
you’re looking at what skills do you bring to bear and
what problem that you’re off to go solve for customers?
Do the nuances of these products make it easier to solve?
Welcome to Screaming in the Cloud.
I’m Corey Quinn.
I am joined today, once again by my friend
and yours, Miles Ward, who’s the CTO at SADA.
However, he is, as I think of him, the closest
thing the Google Cloud world has to Corey Quinn.
Now, let’s be clear, not the music and dancing part that is Forrest Brazeal, but
Forrest works at Google Cloud, whereas Miles is a reasonably salty third-party.
Miles, thank you for coming back and
letting me subject you to that introduction.
Corey, I appreciate that introduction.
I am happy to provide substantial salt.
It is easy, as I play brass instruments that produce my spit in high volumes.
It’s the most disgusting part of any possible introduction.
For the folks in the audience, I am surrounded by a
collection of giant sousaphones, tubas, trombones,
baritones, marching baritones, trumpets, and pocket trumpets.
So, Forrest threw down the gauntlet and was like, I can
play a keyboard, and sing, and look cute at the same time.
And so I decided to fail at all three.
We put out a new song just a bit ago that’s, like, us thanking all of
our customers and partners, covering Kool & the Gang “Celebration,”
and I neither look good, [laugh] play piano, or smiling, or [capturing]
any of the notes; I just play the bass part, it’s all I got to do.
Ready to take your AWS cloud security chops to the next level?
Then Sonrai Security’s Virtual Summit, ACCESS ‘24, happening
on Thursday, September 19th, is where you need to be!
ACCESS '24 is a FREE virtual summit, where you’ll get to hear in-depth keynotes
from leading voices at AWS, EY, Global Atlantic Financial Group, and more.
Tune in as they share real-world case studies explaining how they successfully
navigated complex cloud security challenges and stick around to get
insights into benchmark data for cloud identities, human, and non-human.
Register for FREE at sonrai.co/access24 and gain access
to the full days’ worth of value-packed sessions.
No fluff just the 411 on tactics and strategies from cloud
security leaders to help you meet your own cloud security goals!
Once again that’s sonrai.co/access24 to
register for ACCESS ‘24, on Thursday, September
19th.
See you there!
So, one thing that I didn’t get to talk a lot about because it’s not
quite in my universe, for one, and for another, it is during the pre
Invent—pre:Invent, my nonsense thing—run up, which is Google Cloud Next.
Yes.
And my gag a few years ago is that I’m not saying that
Google is more interested in what they’re building and what
they’re shipping, but even their conference is called Next.
Buh dum, hiss.
[laugh]
. So, I didn’t really get to spend a lot of attention on the Google Cloud
releases that came out this year, but given that SADA is in fact the, I
believe, largest Google Cloud partner on the internet, and thus the world—
[unintelligible] new year, three years in a row back, baby.
Fantastic.
I assume someone’s watch got stuck or something.
But good work.
So, you have that bias in the way that I have a bias, which is your
business is focused around Google Cloud the way that mine is focused on
AWS, but neither of us is particularly beholden to that given company.
I mean, you do have the not getting fired as partner, but that’s a bit of
a heavy lift; I don’t think I can mouth off well enough to get you there.
So, we have a position of relative independence.
So, you were tracking Google Next, the same way that I track re:Invent.
Well, not quite the same way I track re:Invent;
there are some significant differences.
What happened at Cloud Next 2021, that the
worst of us should be paying attention to?
Sure.
I presented 10% of the material at the first re:Invent.
There are 55 sessions; I did six.
And so I have been at Cloud events for a really long time
and really excited about Google’s willingness to dive into
demos in a way that I think they have been a little shy about.
Kelsey Hightower is the kind of notable deep exception to that.
Historically, he’s been ready to dive into
the, kind of, heavy hands-on piece but—
Wait, those were demos?
[Thought] was just playing Tetris on stage for the love of it.
[laugh] . No.
And he really codes all that stuff up, him and the whole team.
Oh, absol—I’m sorry.
If I ever grow up, I wish to be Kelsey Hightower.
[laugh] . You and me both.
So, he had kind of led the charge.
We did a couple of fun little demos while I was there, but they’ve
really gotten a lot further into that, and I think are doing a
better job of packaging the benefits to not just developers, but
also operators and data scientists and the broader roles in the
cloud ecosystem from the new features that are being launched.
And I think, different than the in-person events where there’s
10, 20,000, 40,000 people in the audience paying attention,
I think they have to work double-hard to capture attention
and get engineers to tune in to what’s being launched.
But if you squint and look close, there are some, I think, very interesting
trends that sit in the back of some of the very first launches in what I
think are going to be whole veins of launches from Google over the course
of the next several years that we are working really hard to track along
with and make sure we’re extracting maximum value from for our customers.
So, what was it that they announced that is worth paying attention to?
Now, through the cacophony of noise, one announcement that [I want to note]
was tied to Next was the announcement that GME group, I believe, is going
to be putting their futures exchange core trading systems on Google Cloud.
At which point that to me—and I know people are going to yell at me,
and I don’t even slightly care—that is the last nail in the coffin of
the idea that well, Google is going to turn this off in a couple years.
Sorry, no.
That is not a thing that’s going to happen.
Worst case, they might just stop investing it as
aggressively as they are now, but even that would be just
a clown-shoes move that I have a hard time envisioning.
Yeah, you’re talking now over a dozen, over
ten year, over a billion-dollar commitments.
So, you’ve got to just really, really hate your stock price if you’re
going to decide to vaporize that much shareholder value, right?
I mean, we think that, in Google, stock price is a material
fraction of the recognition of the growth trajectory for
cloud, which is now basically just third place behind YouTube.
And I think you can do the curve math, it’s not like it’s going to take long.
Right.
That requires effectively ejecting Thomas Kurian as the head of Google
Cloud and replacing him with the former SVP of Bad Decisions at Yahoo.
[laugh] . Sure.
Google has no shyness about continuing to rotate leadership.
I was there through three heads of Google Cloud, so I
don’t expect that Thomas will be the last although I think
he may well go down in history as having been the best.
The level of rotation to the focuses that I think are most critical,
getting enterprise customers happy, successful, committed, building
macroscale systems, in systems that are critical to the core of the
business on GCP has grown at an incredible rate under his stewardship.
So, I think he’s doing a great job.
He gets a lot of criticism—often from Googlers—when I wind up getting
the real talk from them, which is, “Can you tell me what you really
think?” Their answer is, “No,” I’m like, “Okay, next question.
Can I go out and buy you eight beers and then”— and it’s
like, “Yeah.” And the answer that I get pretty commonly
is that he’s brought too much Oracle into Google.
And okay, that sounds like a bad thing because, you know, Oracle,
but let’s be clear here, but what are you talking about specifically?
And what they say distills down to engineers are no
longer the end-all be-all of everything that Google Cloud.
Engineers don’t get to make sales decisions, or
marketing decisions, or in some cases, product decisions.
And that is not how Google has historically
been run, and they don’t like the change.
I get it, but engineering is not the only hard thing in the world and
it’s not the only business area that builds value, let’s be clear on this.
So, I think that the things that they don’t
like are in fact, what Google absolutely needs.
I think, one, the man is exceptionally intimidating and
intentionally just hyper, hyper attentive to his business.
So, one of my best employees, Brad [Svee] , he worked together with me to lay
out what was the book of our whole department, my team of 86 people there.
What are we about?
What do we do?
And like I wanted this as like a memoriam to teach new hires as got brought in.
So, this is, like, 38 pages of detail about our process,
our hiring method, our promotional approach, all of it.
I showed that to my new boss who had come in at the
time, and he thought some of the pictures looked good.
When we showed it to TK, he read every paragraph.
I watched him highlight the paragraphs as he went through,
and he read it twice as fast as I can read the thing.
I think he does that to everybody’s documents, everywhere.
So, there’s a level of just manual rigor that he’s brought
to the practice that was certainly not there before that.
So, that alone, it can be intimidating for folks, but I think
people that are high performance find that very attractive.
Well, from my perspective, he is clearly head and shoulders above Adam
Selipsky, and Scott Guthrie—the respective heads of AWS and Azure—for one key
reason: He is the only one of those three people who follows me on Twitter.
And—
[laugh]
. —honestly, that is how I evaluate vendors.
That’s the thing.
That’s the only measure, yep.
I’ve worked on for a long time with Selipsky, and I think that it will be
interesting to see whether Adam’s approach to capital allocation—where he
really, I think, thinks of himself as the manager of thousands of startups,
as opposed to a manager of a global business—whether that’s a more efficient
process for creating value for customers, then, where I think TK is
absolutely trying to build a much more unified, much more singular platform.
And a bunch of the launches really speak to that, right?
So, one of the product announcements that I think is critical is
this idea of the global distributed cloud, Google Distributed Cloud.
We started with Kubernetes.
And then you layer on to that, okay, we’ll take
care of Kubernetes for you; we call that Anthos.
We’ll build a bunch of structural controls and features into Anthos
to make it so that you can really deal with stuff in a global way.
Okay, what does that look like further?
How do we get out into edge environments?
Out into diverse hardware?
How do we partner up with everybody to make sure that, kind of like comparing
Apple’s approach to Google’s approach, you have an Android ecosystem of
Kubernetes providers instead of just one place you can buy an outpost.
That’s generally the idea of GDC.
I think that’s a spot where you’re going to watch Google actually
leverage the muscle that it already built in understanding
open-source dynamics and understanding collaboration between
companies as opposed to feeling like it's got to be built here.
We’ve got to sell it here.
It’s got to have our brand on it.
I think that there’s a stupendous and extreme
story that is still unfolding over at Google Cloud.
Now, re:Invent this year, they wound up talking all about how
what they were rolling out was a focus on improving primitives.
And they’re right.
I love their managed database service that
they launched because it didn’t exist.
Yeah Werner’s slide, “It’s primitives, not frameworks.” I was like,
I think customers want solutions, not frameworks or primitives.
[laugh] . What’s your plan?
Yeah.
However, I take a different perspective on all of this, which is that is a
terrific spin on the big headline launches all missed the re:Invent timeline,
and… oops, so now we’re just going to talk about these other things instead.
And that’s great, but then they start talking about industrial IOT, and
mainframe migrations, and the idea of private 5G, and running fleets of robots.
And it’s—
Yeah, that’s a cool product.
Which one?
I’m sorry, they’re all very different things.
Private 5G.
Yeah, if someone someday will explain to me how it differs
from Wavelength, but that’s neither here nor there.
You’re right, they’re all interesting, but none of them are actually
doing the thing that I do, which is build websites, [unintelligible]
looking for web services, it kind of says it in the name.
And it feels like it’s very much broadening into everything, and it’s very
difficult for me to identify—and if I have trouble that I guarantee you
customers do—of, which services are for me and which are very much not?
In some cases, the only answer to that is to check the pricing.
I thought Kendra, their corporate information search thing
was for me, then it’s 7500 bucks a month to get started with
that thing, and that is, “I can hire an internal corporate
librarian to just go and hunt through our Google Drive.” Great.
Yeah.
So, there are—or our Dropbox, or our Slack.
We have, like, five different information repositories, and
this is how corporate nonsense starts, let me assure you.
Yes.
We call that luxury SaaS, you must enjoy your dozens of overlapping
bills for, you know, what Workspace gives you as a single flat rate.
Well, we have [unintelligible] a lot of this stuff, too.
Google Drive is great, but we use Dropbox for holding anything
that touches our customer’s billing information, just because I—to
be clear, I do not distrust Google, but it also seems a little
weird to put the confidential billing information for one of their
competitors on there to thing if a customer were to ask about it.
So, it’s the, like, I don’t believe anyone’s doing anything
nefarious, but let’s go ahead and just make sure, in this case.
Go further man.
Vimeo runs on GCP.
You think YouTube doesn’t want to look at Vimeo stats?
Like they run everything on GCP, so they have
to have arrived at a position of trust somehow.
Oh, I know how it’s called encryption.
You’ve heard of encryption before?
It’s the best.
Oh, yes.
I love these rumors that crop up every now and again that Amazon
is going to start scanning all of its customer content, somehow.
It’s first, do you have any idea how many compute resources
that would take and to if they can actually do that and access
something you’re storing in there, against their attestations to
the contrary, then that’s your story because one of them just makes
them look bad, the other one utterly destroys their entire business.
Yeah.
I think that that’s the one that gets the better clicks.
So no, they’re not doing that.
No, they’re not doing that.
Another product launch that I thought was super interesting
that describes, let’s call it second place—the third place will
be the one where we get off into the technical deep end—but
there’s a whole set of coordinated work they’re calling Cortex.
So, let’s imagine you go to a customer, they say, “I want to understand
what’s happening with my business.” You go, “Great.” So, you use SAP, right?
So, you’re a big corporate shop, and that's your infrastructure of choice.
There are a bunch of different options at that layer.
When you set up SAP, one of the advantages that something like that
has is they have, kind of, pre-built configurations for roughly your
business, but whatever behaviors SAP doesn’t do, right, say, data
warehousing, advanced analytics, regression and projection and stuff
like that, maybe that’s somewhat outside of the core wheelhouse
for SAP, you would expect like, oh okay, I’ll bolt on BigQuery.
I’ll build that stuff over there.
We’ll stream the data between the two.
Yeah, I’m off to the races, but the BigQuery side of the house
doesn’t have this like bitching menu that says, “You’re a
retailer, and so you probably want to see these 75 KPIs, and
you probably want to chew up your SKUs in exactly this way.
And here’s some presets that make it so that this is operable out of the box.”
So, they are doing the three way combination: Consultancies plus
ISVs plus Google products, and doing all the pre-work configuration
to go out to a customer and go I know what you probably just want.
Why don’t I just give you the whole thing
so that it does the stuff that you want?
That I think—if that’s the very first one, this little triangle
between SAP, and Big Query, and a bunch of consultancies like mine, you
have to imagine they go a lot further with that a lot faster, right?
I mean, what does that look like when they do it with Epic, when
they go do it with Go just generally, when they go do it with Apache?
I’ve heard of that software, right?
Like, there’s no reason not to bundle up what the
obvious choices are for a bunch of these combinations.
The idea of moving up the stack and offering full
on solutions, that’s what customers actually want.
“Well, here’s a bunch of things you can do to wind
up wiring together to build a solution,” is, “Cool.
Then I’m going to go hire a company who’s already done that is going to sell it
to me at a significant markup because I just don’t care.” I pay way more to WP
Engine than I would to just run WordPress myself on top of AWS or Google Cloud.
In fact, it is on Google Cloud, but okay.
You and me both, man.
WP Engine is the best.
I—
It’s great because—
You’re welcome.
I designed a bunch of the hosting on the back of that.
Oh, yeah.
But it’s also the—I—well, it costs a little bit more that way.
Yeah, but guess what’s not—guess what’s more expensive than that
bill, is my time spent doing the care and feeding of this stuff.
I like giving money to experts and making it their problem.
Yeah.
I heard it said best, Lego is an incredible business.
I love their product, and you can build almost any toy with it.
And they have not displaced all other plastic toy makers.
Right.
Some kids just want to buy a little car.
[laugh]
. Oh, yeah, you can build anything you want out of Lego bricks, which are
great, which absolutely explains why they are a reference AWS customer.
Yeah, they’re great.
But they didn’t beat all other toy companies worldwide, and eliminate
the rest of that market because they had the better primitive, right?
These other solutions are just as valuable, just
as interesting, tend to have much bigger markets.
Lego is not the largest toy manufacturer in the world.
They are not in the top five of toy manufacturers in the world, right?
Like, so chasing that thread, and getting all the way down
into the spots where I think many of the cloud providers
on their own, internally, had been very uncomfortable.
Like, you got to go all the way to building this stuff that they need
for that division, inside of that company, in that geo, in that industry?
That’s maybe, like, a little too far afield.
I think Google has a natural advantage in its more partner-oriented
approach to create these combinations that lower the cost to
them and to customers to getting out of that solution quick.
So, getting into the weeds of Google Next, I suppose, rather than a whole
bunch of things that don’t seem to apply to anyone except the four or
five companies that really could use it, what things did Google release
that make the lives of people building, you know, web apps better?
This is the one.
So, I’m at Amazon, hanging out as a part of the team that built
up the infrastructure for the Obama campaign in 2012, and there
are a bunch of Googlers there, and we are fighting with databases.
We are fighting so hard, in fact, with RDS that I think we are the only ones
that [Raju] has ever allowed to SSH into our RDS instances to screw with them.
Until now, with the advent of RDS Custom, meaning that you can actually get
in as root; where that hell that lands between RDS and EC2 is ridiculous.
I just know that RDS can now run containers.
Yeah.
I know how many things we did in there that were good for
us, and how many things we did in there that were bad for us.
And I have to imagine, this is not a feature that they
really ought to let everybody have, myself included.
But I will say that what all of the Googlers that I talk to, you know, at
the first blush, were I’m the evil Amazon guy in to, sort of, distract them
and make them build a system that, you know, was very reliable and ended up
winning an election was that they had a better database, and they had Spanner,
and they didn’t understand why this whole thing wasn’t sitting on Spanner.
So, we looked, and I read the white paper, and then I got all drooly,
and I was like, yes, that is a much better database than everybody
else’s database, and I don’t understand why everybody else isn’t on it.
Oh, there’s that one reason, but you’ve heard of it: No
other software works with it, anywhere in the world, right?
It’s utterly proprietary to Google.
Yes, they were kind—
Oh, you want to migrate it off somewhere else, or a fraction of it?
Great.
Step one, redo your data architecture.
Yeah, take all of my software everywhere, rewrite every bit of it.
And, oh all those commercial applications?
Yeah, forget all those, you got, too.
Right?
It was very much where Google was eight years ago.
So, for me, it was immensely meaningful to see the launch at
Next where they described what they are building—and have now
built; we have alpha access to it—a Postgres layer for Spanner.
Is that effectively you have to treat it as
Postgres at all times, or is it multimodal access?
You can get in and tickle it like Spanner,
if you want to tickle it like Spanner.
And in reality, Spanner is ANSI SQL compliant; you’re still writing
SQL, you just don’t have to talk to it like a REST endpoint, or
a GRPC endpoint, or something; you can, you know, have like a—
So, similar to Azure’s Cosmos DB, on some level, except for the part
where you can apparently look at other customers’ data in that thing?
[laugh] . Exactly.
Yeah, you will not have a sweeping discovery of incredible security
violations in the structure Spanner, in that it is the control
system that Google uses to place every ad, and so it does not suck.
You can’t put a trillion-dollar business on
top of a database and not have it be safe.
That’s kind of a thing.
The thing that I find is the most interesting area of tech
right now is there’s been this rise of distributed databases.
Yugabyte—or You-ji-byte—Pla-netScale—or PlanetScale,
depending on how you pronounce these things.
[laugh] . Yeah, why, why is G such an adversarial consonant?
I don’t understand why we’ve all gotten to this place.
Oh, yeah.
But at the same time, it’s—so you take a look at all these—and they
all are speaking Postgres; it is pretty clear that ‘Postgres-squeal’
is the thing that is taking over the world as far as databases go.
If I were building something from scratch that used—
For folks in the back, that’s PostgreSQL, for the
rest of us, it’s okay, it’s going to be, all right.
Same difference.
But yeah, it’s the thing that is eating the world.
Although recently, I’ve got to say, MongoDB is absolutely
stepping up in a bunch of really interesting ways.
I mean, I think the 4.0 release, I’m the guy who wrote the MongoDB on
AWS Best Practices white paper, and I would grab a lot of customer’s and—
They have to change it since then of, step one: Do not
use DocumentDB; if you want to use Mongo, use Mongo.
Yeah, that’s right.
No, there were a lot of customers I was on the phone with where
Mongo had summarily vaporized their data, and I think they have made
huge strides in structural reliability over the course of—you know,
especially this 4.0 launch, but the last couple of years, for sure.
And with all the people they’ve been hiring from AWS, it’s one of those,
“Well, we’ll look at this now who’s losing important things from production?”
[laugh] . Right?
So, maybe there’s only actually five humans who know how to do operations,
and we just sort of keep moving around these different companies.
That’s sort of my assumption on these things.
But Postgres, for those who are not looking to depart
from the relational model, is eating the world.
And—
There’s this, like, basic emotional thing.
My buddy Martin, who set up MySQL, and took it public, and then
promptly got it gobbled up by the Oracle people, like, there
was a bet there that said, hey, there’s going to be a real
open database, and then squish, like, the man came and got it.
And so like, if you’re going to be an independent, open-source
software developer, I think you’re probably not pushing your
pull requests to our friends at Oracle, that seems weird.
So instead, I think Postgres has gobbled up the best minds on that stuff.
And it works.
It’s reliable, it’s consistent, and it’s functional in all these
different, sort of, reapplications and subdivisions, right?
I mean, you have to sort of squint real hard, but down
there in the guts of Redshift, that’s Postgres, right?
Like, there’s Postgres behind all sorts of stuff.
So, as an interface layer, I’m not as interested about how it
manages to be successful at bossing around hardware and getting
people the zeros and ones that they ask for back in a timely manner.
I’m interested in it as a compatibility standard, right?
If I have software that says, “I need to have Postgres
under here and then it all will work,” that creates this
layer of interop that a bunch of other products can use.
So, folks like PlanetScale, and Yugabyte can say, “No, no, no, it’s cool.
We talk Postgres; that’ll make it so your application works right.
You can bring a SQL alchemy and plug it into this,
or whatever your interface layer looks like.”
That’s the spot where, if I can trade what is a fairly limited
global distribution, global transactional management on literally
ridiculously unlimited scalability and zero operations, I can handle
the hard parts of running a database over to somebody else, but I get
my layer, and my software talks to it, I think that’s a huge step.
Backblaze
B2 Cloud Storage helps you scale applications and deliver services globally.
Egress is free up to 3X of data stored, and completely free
between S3-compatible Backblaze and leading CDN and compute
providers like Fastly, Cloudflare, Vultr, and CoreWeave.
Visit backblaze.com to learn more.
Backblaze—cloud storage built better.
I think that there’s a strong movement
toward building out on something like this.
If it works, just because—well, I’m not multiregion today,
but I can easily see a world in which I’d want to be.
So, great.
How do you approach the decision between—once
this comes out of alpha; let’s be clear.
Let’s turn this into something that actually ships, and no, Google that
does not mean slapping a beta label on it for five years is the answer
here; you actually have to stand behind this thing—but once it goes GA—
GA is a good thing.
Yeah.
How do you decide between using that, or PlanetScale?
Or Yugabyte?
Or Cockroach or or SingleStore, right?
I mean, there’s a zillion of them that sit in this market.
I think the core of the decision making for me is in every
team you’re looking at what skills do you bring to bear
and what problem that you’re off to go solve for customers?
Do the nuances of these products make it easier to solve?
So, I think there are some products that the nature of what you’re building
isn’t all that dependent on one part of the application talking to another
one, or an event happening someplace else mattering to an event over here.
But some applications, that’s, like, utterly
critical, like, totally, totally necessary.
So, we worked with a bunch of like Forex exchange trading desks that literally
turn off 12 hours out of the day because they can only keep it consistent
in one geographical location right near the main exchanges in New York.
So, that’s a place where I go, “Would you like to trade all day?”
And they go, “Yes, but I can’t because databases.” So, “Awesome.
Let’s call the folks on the Spanner side.
They can solve that problem.”
I go, “Would you like to trade all day and rewrite all
your software?” And they go, “No.” And I go, “Oh, okay.
What about trade all day, but not rewrite all your software?” There we go.
Now, we’ve got a solution to that kind of problem.
So like, we built this crazy game, like, totally other end of the ecosystem
with the Dragon Ball Z people, hysterical; your like—you literally play
like Rock, Paper, Scissors with your phone, and if you get a rock, I throw a
fireball, and you get a paper, then I throw a punch, and we figure out who wins.
But they can play these games like Europe versus Japan,
thousands of people on each side, real-time, and it works.
So, let’s be clear, I have lobbied a consistent criticism at Google
for a while now, which is the Google Cloud global control plane.
So, you wind up with things like global service outages from time to
time, you wind up with this thing is now broken for everyone everywhere.
And that, for a lot of these use cases, is a problem.
And I said that AWS’s approach to regional isolation is the right way to do it.
And I do stand by that assessment, except for the part where it turns
out there’s a lot of control plane stuff that winds up single tracking
through us-east-1, as we learned in the great us-east-1 outage of 2021.
Yeah, when I see customers move from data center to AWS, what
they expect is a higher count of outages that lasts less time.
That’s the trade off, right?
There’s going to be more weird spurious stuff, and maybe—maybe—if they’re
lucky, that outage will be over there at some other region they’re not using.
I see almost exactly the same promise happening to folks that come from
AWS—and in particular from Azure—over onto GCP, which is, there will be
probably a higher frequency of outages at a per product level, right?
So, like sometimes, like, some weird product takes a screw sideways, where there
is structural interdependence between quite a few products—we actually published
a whole internal structural map of like, you know, it turns out that Cloud SQL
runs on top of GCE not on GKE, so you can expect if GKE goes sideways, Cloud SQL
is probably not going to go sideways; the two aren’t dependent on each other.
You take the status page and Amazon FreeRTOS in a
region is having an outage today or something like that.
You’re like, “Oh, no.
That’s terrible.
First, let me go look up what the hell that is.” And I’m not using it?
Absolutely not.
Great.
As hyperscalers, well, hyperscale, they’re always things that
are broken in different ways, in different locations, and
if you had a truly accurate status page, it would all be red
all the time, or varying shades of red, which is not helpful.
So, I understand the challenge there, but very often, it’s a
partition that is you are not exposed to, or the way that you’ve
architected things, ideally, means it doesn’t really matter.
And that is a good thing.
So, raw outage counts don’t solve that.
I also maintain that if I were to run in a single region of AWS or
even a single AZ, in all likelihood, I will have a significantly
better uptime across the board than I would if I ran it myself.
Because—
Oh, for sure.
—it is—
For sure they’re way better at ops than you are.
Me, right?
Of course.
Right?
Like, ridiculous.
And they got that way, by learning.
Like, I think in 2022, it is unlikely that there’s going to
be an outage in an AWS availability zone by someone tripping
over a power cable, whereas I have actually done that.
So, there’s a—to be clear in a data center,
not an AWS facility; that would not have flown.
So, there is the better idea of of going in that direction.
But the things like Route 53 is control plane single-tracking
through the us-east-1, if you can’t make DNS changes in an outage
scenario, you may as well not have a DR plan, for most use cases.
To be really clear, it was a part of the internal documentation on the AWS
side that we would share with customers to be absolutely explicit with them.
It’s not just that there are mistakes and accidents which we try to limit
to AZs, but no, go further, that we may intentionally cause outages to
AZs if that’s what allows us to keep broader service health higher, right?
They are not just a blast radius because you, oops, pulled the pin on
the grenade; they can actually intentionally step on the off button.
And that’s different than the way Google operates.
They think of each of the AZs, and each of the regions, and the global
system as an always-on, all the time environment, and they do not have
systems where one gets, sort of, sacrificed for the benefit of the
rest, right, or they will intentionally plan to take a system offline.
There is no planned downtime in the SLA, where the SLAs
from my friends at Amazon and Azure are explicit to, if
they choose to, they decide to take it offline, they can.
Now, that’s—I don’t know, I kind of want the contract
that has the other thing where you don’t get that.
I don’t know what the right answer is for a lot of these things.
I think multi-cloud is dumb.
I think that the idea of having this workload that you’re going to
seamlessly deploy to two providers in case of an outage, well guess what?
The orchestration between those two providers is going to cause
you more outages than you would take just sticking on one.
And in most cases, unless you are able to have complete duplication
of not just functionality but capacity between those two,
congratulations, you’ve now just doubled your number of single points
of failure, you made the problem actively worse and more expensive.
Good job.
I wrote an article about this, and I think it’s important to
differentiate between dumb and terrifyingly shockingly expensive, right?
So, I have a bunch of customers who I would characterize
as rich, as like, shockingly rich, as producing
businesses that have 80-plus percent gross margins.
And for them, the costs associated with this stuff are
utterly rational, and they take on that work, and they
are seeing benefits, or they wouldn’t be doing it.
Of course.
So, I think their trajectory in technology—you know, this is a quote from a
Google engineer—it’s just like, “Oh, you want to see what the future looks like?
Hang out with rich people.” I went into houses when
I was a little kid that had whole-home automation.
I couldn’t afford them; my mom was cleaning house there,
but now my house, I can use my phone to turn on the lights.
Like—
You know, unless us-east-1 is having a problem.
Hey, and then no Roomba for you, right?
Like utterly offline.
So—
Roomba has now failed to room.
Conveniently, my lights are Philips Hue,
and that’s on Google, so that baby works.
But it is definitely a spot where the barrier of entry and
the level of complexity required is going down over time.
And it is definitely a horrible choice for 99%
of the companies that are out there right now.
But next year, it’ll be 98.
And the year after that, it’ll probably be 97.
[laugh]
. And if I go inside of Amazon’s data centers, there’s
not one manufacturer of hard drives, there’s a bunch.
So, that got so easy that now, of course you use more than one;
you got to do—that’s just like, sort of, a natural thing, right?
These technologies, it’ll move over time.
We just aren’t there yet for the vast, vast majority of workloads.
I hope that in the future, this stuff becomes easier, but
data transfer fees are going to continue to be a concern—
Just— [makes explosion noise]
— Oh, man—
—like, right in the face.
—especially with the Cambrian explosion of data because the
data science folks have successfully convinced the entire
industry that there’s value in those mode balancer logs in 2012.
Okay, great.
We’re never deleting anything again, but now you’ve got to
replicate all of that stuff because no one has a decent handle
on lifecycle management and won’t for the foreseeable future.
Great, to multiple providers so that you can work on these things?
Like, that is incredibly expensive.
Yeah.
Cool tech, from this announcement at Next that I think is very
applicable, and recognized the level of like, utter technical mastery—and
security mastery to our earlier conversation—that something like
this requires, the product is called BigQuery Omni, what Omni allows
you to do is go into the Google Cloud Console, go to BigQuery, say
I want to do analysis on this data that’s in S3, or in Azure Blob
Storage, Google will spin up an account on your behalf on Amazon
and Azure, and run the compute there for you, bring the result back.
So, just transfer the answers, not the raw data that you just
scanned, and no work on your part, no management, no crapola.
So, there’s like—that’s multi-cloud.
If I’ve got—I can do a join between a bunch of rows that are in
real BigQuery over on GCP side and rows that are over there in S3.
The cross-eyedness of getting something like that to work is mind blowing.
To give this a little more context, just because it gets difficult to reason
about these things, I can either have data that is in a private subnet in AWS
that traverses their horribly priced Managed NAT Gateways, and then goes out to
the internet and sent there once, for the same cost as I could take that same
data and store it in S3 in their standard tier for just shy of six full months.
That’s a little imbalanced, if we’re being direct here.
And then when you add in things like intelligent tiering and archive
access classes, that becomes something that… there’s no contest there.
It’s, if we’re talking about things that are now
approaching exabyte scale, that’s one of those, “Yeah,
do you want us to pay by a credit card?”—get serious.
You can’t at that scale anyway—“Invoice billing, or do we just, like,
drive a dump truck full of gold bricks and drop them off in Seattle?”
Sure.
Same trajectory, on the multi-cloud thing.
So, like a partner of ours, PacketFabric, you know, if you’re a big,
big company, you go out and you call Amazon and you buy 100 gigabit
interconnect on—I think they call theirs Direct Connect, and then you
hook that up to the Google one that’s called Dedicated Interconnect.
And voila, the price goes from twelve cents a gig
down to two cents a gig; everybody’s much happier.
But Jesus, you pay the upfront for that, you got to set
the thing up, it takes days to get deployed, and now
you’re culpable for the whole pipe if you don’t use it up.
Like, there are charges that are static over the course of the month.
So, PacketFabric just buys one of those
and lets you rent a slice of it you need.
And I think they’ve got an incredible product.
We’re working with them on a whole bunch of different projects.
But I also expect—like, there’s no reason the cloud providers
shouldn’t be working hard to vend that kind of solution over time.
If a hundred gigabit is where it is now, what
does it look like when I get to ten gigabit?
When I get to one gigabit?
When I get to half gigabit?
You know, utility price that for us so that we get to rational pricing.
I think there’s a bunch of baked-in business and cost logic that
is a part of the pricing system, where egress is the source of
all of the funding at Amazon for internal networking, right?
I don’t pay anything for the switches that connect
to this machine to that machine, in region.
It’s not like those things are cheap or free; they have to be there.
But the funding for that comes from egress.
So, I think you’re going to end up seeing a different model
where you’ll maybe have different approaches to egress
pricing, but you’ll be paying like an in-system networking fee.
And I think folks will be surprised at how big that fee likely is because of the
cost of the level of networking infrastructure that the providers deploy, right?
I mean, like, I don’t know, if you’ve gone and tried
to buy a 40 port, 40 gig switch anytime recently.
It’s not like they’re those little, you know, blue Netgear ones for 90 bucks.
Exactly.
It becomes this, [sigh] I don’t know, I keep thinking that’s not
the right answer, but part of it also is like, well, you know, for
things that I really need local and don’t want to worry about if
the internet’s melting today, I kind of just want to get, like,
some kind of Raspberry Pi shoved under my desk for some reason.
Yeah.
I think there is a lot where as more and more businesses bet bigger and bigger
slices of the farm on this kind of thing, I think it’s Jassy’s line that
you’re, you know, the fat in the margin in your business is my opportunity.
Like, there’s a whole ecosystem of partners and
competitors that are hunting all of those opportunities.
I think that pressure can only be good for customers.
Miles, thank you for taking the time to speak with me.
If people want to learn more about you, what you’re up
to, your bad opinions, your ridiculous company, et cetera—
[laugh]
. —where can they find you?
Well, it’s really easy to spell: SADA.com, S-A-D-A dot com.
I’m Miles Ward, it’s @milesward on Twitter;
you don’t have to do too hard of a math.
It’s miles@sada.com, if you want to send me an email.
It’s real straightforward.
So, eager to reach out, happy to help.
We’ve got a bunch of engineers that like helping people move from Amazon to GCP.
So, let us know.
Excellent.
And we will, of course, put links to this in
the [show notes] because that’s how we roll.
Yay.
Thanks so much for being so generous with your time, and I look forward
to seeing what comes out next year from these various cloud companies.
Oh, I know some of them already, and they’re good.
Oh, they’re super good.
This is why I don’t do predictions because like, the stuff that I know
about, like, for example, I was I was aware of the Graviton 3 was coming—
Sure.
—and it turns out that if your—guess what’s going to come up
and you don’t name Graviton 3, it’s like, “Are you simple?
Did you not see that one coming?” It’s like—or if I don’t know
it’s coming and I make that guess—which is not the hardest
thing in the world—someone would think I knew and leaked.
There’s no benefit to doing predictions.
No.
It’s very tough, very happy to do predictions in private, for customers.
[laugh]
. Absolutely.
Thanks again for your time.
I appreciate it.
Cheers.
Myles Ward, CTO at SADA.
I’m Cloud Economist Corey Quinn, and this is Screaming in the Cloud.
If you’ve enjoyed this podcast, please leave a five-star review on your
podcast platform of choice, whereas if you’ve hated this podcast, please
leave a five-star review on your podcast platform of choice and be very
angry in your opinion when you write that obnoxious comment, but then
it’s going to get lost because it’s using MySQL instead of Postgres