1
00:00:00,268 --> 00:00:03,468
Matt: We build more network capacity than we need, both from a reliability

2
00:00:03,468 --> 00:00:05,997
perspective so that we have a lot of redundancy and resiliency.

3
00:00:06,278 --> 00:00:09,558
Many devices can fail, many links can fail, we still have sufficient capacity

4
00:00:14,776 --> 00:00:16,265
Corey: Welcome to Screaming in the Cloud.

5
00:00:16,566 --> 00:00:17,586
I'm Corey Quinn.

6
00:00:17,626 --> 00:00:22,736
Somehow, I have managed to breach containment and go talk to people at Amazon.

7
00:00:22,896 --> 00:00:28,136
My guest today is Matt Rehder, who is the VP of Global Networking.

8
00:00:28,466 --> 00:00:29,646
Matt, thank you for joining me.

9
00:00:29,706 --> 00:00:30,925
Matt: Yeah, thanks for having me.

10
00:00:31,485 --> 00:00:35,305
Corey: This episode is sponsored in part by my day job, Duckbill.

11
00:00:35,305 --> 00:00:37,786
Do you have a horrifying AWS bill?

12
00:00:38,065 --> 00:00:39,926
That can mean a lot of things.

13
00:00:40,176 --> 00:00:45,266
Predicting what it's going to be, determining what it should be, negotiating

14
00:00:45,266 --> 00:00:50,675
your next long-term contract with AWS, or just figuring out why it increasingly

15
00:00:50,675 --> 00:00:55,205
resembles a phone number, but nobody seems to quite know why that is.

16
00:00:55,516 --> 00:00:59,065
To learn more, visit duckbillhq.com.

17
00:00:59,385 --> 00:01:03,415
Remember, you can't duck the Duckbill bill, which my

18
00:01:03,415 --> 00:01:07,656
CEO reliably informs me is absolutely not our slogan.

19
00:01:08,036 --> 00:01:09,895
So what is it you do exactly?

20
00:01:09,895 --> 00:01:11,786
Networking, what's the point of it all?

21
00:01:11,786 --> 00:01:14,975
"That feels like something old people care about," said the old person.

22
00:01:15,135 --> 00:01:15,575
Matt: Yeah.

23
00:01:15,606 --> 00:01:17,485
Uh, I mean, it… The simplest answer is

24
00:01:17,485 --> 00:01:19,035
we make all the servers talk to each other.

25
00:01:19,035 --> 00:01:20,366
Um, but, um- Otherwise,

26
00:01:20,366 --> 00:01:21,176
Corey: they're just expensive space heaters.

27
00:01:21,176 --> 00:01:22,785
Matt: Otherwise, they're very expensive space heaters.

28
00:01:22,785 --> 00:01:23,005
Yeah.

29
00:01:23,035 --> 00:01:25,385
So it's, it's really interconnecting all the servers, interconnecting

30
00:01:25,416 --> 00:01:27,945
all the data centers around the world, and then connecting them to the,

31
00:01:27,946 --> 00:01:30,566
all the people around the world on the internet, is, is the gist of it.

32
00:01:30,566 --> 00:01:33,436
Corey: Yes, I… You were kind enough to recently give me a

33
00:01:33,436 --> 00:01:37,165
tour- Mm … of the networking lab, one of the networking labs.

34
00:01:37,166 --> 00:01:38,545
I'm guessing there might be more than one.

35
00:01:38,545 --> 00:01:38,815
We have more

36
00:01:38,815 --> 00:01:39,315
Matt: than one.

37
00:01:39,505 --> 00:01:39,752
Corey: Because we're Amazon.

38
00:01:39,752 --> 00:01:39,929
Matt: Yeah.

39
00:01:39,929 --> 00:01:40,105
Yeah.

40
00:01:40,105 --> 00:01:41,996
We didn't show you the super-secret one, but we can, yeah.

41
00:01:41,996 --> 00:01:42,855
Corey: Oh, exactly.

42
00:01:42,855 --> 00:01:42,955
Yeah.

43
00:01:42,955 --> 00:01:43,796
The, the real secret one.

44
00:01:43,796 --> 00:01:43,925
Yeah.

45
00:01:43,986 --> 00:01:46,006
That's where the cables are longer so we can fit

46
00:01:46,006 --> 00:01:48,016
more data into them before it comes back around.

47
00:01:48,045 --> 00:01:49,465
No, yeah, that's how networking works, right?

48
00:01:49,825 --> 00:01:52,286
My networking background myself- Mm … is such that

49
00:01:52,336 --> 00:01:54,685
it's just enough to know what I don't know, and… but it

50
00:01:54,685 --> 00:01:56,926
does guide me toward asking some of the right questions.

51
00:01:56,935 --> 00:01:56,965
Mm-hmm.

52
00:01:57,215 --> 00:01:58,555
There's a whole different sense of scale

53
00:01:58,556 --> 00:02:00,706
that I don't think most folk appreciate.

54
00:02:00,725 --> 00:02:00,965
Yeah.

55
00:02:01,075 --> 00:02:07,066
Uh, one of the enduring qualities of AWS that has also led to frustration,

56
00:02:07,066 --> 00:02:10,435
but I maintain is the right answer, is your networking is phenomenal.

57
00:02:10,465 --> 00:02:13,755
It i- You have a harsh separation between regions.

58
00:02:13,985 --> 00:02:18,915
You have, knock wood, uh, never yet seen a global networking outage.

59
00:02:18,915 --> 00:02:21,735
There's never been a global rolling outage of things going down.

60
00:02:21,925 --> 00:02:27,315
Error, failures are region-bound in virtually every case I am aware of.

61
00:02:27,575 --> 00:02:30,045
And that was a choice, and it does mean that when you log into an AWS

62
00:02:30,266 --> 00:02:34,095
account, each region makes it look like there are now 31 AWS accounts.

63
00:02:34,135 --> 00:02:36,025
Go hunt and find it down.

64
00:02:36,475 --> 00:02:43,036
But the resilience and durability story are unparalleled by any other company.

65
00:02:43,036 --> 00:02:45,476
I include every other hyperscaler in that list.

66
00:02:45,805 --> 00:02:49,395
Resilience is great, and it's been that way for a long time.

67
00:02:50,195 --> 00:02:55,396
Now it seems that- Systems have gotten good enough across the

68
00:02:55,396 --> 00:02:58,916
board, especially since modern AI systems are not that reliable.

69
00:02:58,916 --> 00:03:02,176
We're not seeing five nines of uptime on anything, but companies

70
00:03:02,176 --> 00:03:05,186
are putting it in their critical path to a point where it feels

71
00:03:05,186 --> 00:03:09,105
like resilience is something companies pay lip service to largely.

72
00:03:09,106 --> 00:03:12,036
It is an afterthought more than a lot of other things.

73
00:03:12,346 --> 00:03:14,455
It just isn't an area of focus.

74
00:03:14,895 --> 00:03:18,026
My sense is that this is something that happens when you've

75
00:03:18,026 --> 00:03:20,565
been too long without a really good, really notable outage.

76
00:03:20,565 --> 00:03:22,505
It's sort of like a self-inflicted, we've been too good.

77
00:03:22,516 --> 00:03:24,216
People forget this stuff can break.

78
00:03:24,664 --> 00:03:25,384
How do you see it?

79
00:03:25,664 --> 00:03:26,994
Matt: Well, first of all, thank you for the kind words.

80
00:03:27,124 --> 00:03:28,853
Uh, we care a lot about resiliency.

81
00:03:29,004 --> 00:03:30,554
That is our number one guiding principle,

82
00:03:30,554 --> 00:03:33,064
especially for our network, is massive reliability.

83
00:03:33,103 --> 00:03:35,874
And it, and obviously we wanna contain outages to within

84
00:03:35,874 --> 00:03:38,464
a region, but we never wanna have them span to a region.

85
00:03:38,793 --> 00:03:41,834
We go to the level of isolation, even with inside of individual data

86
00:03:41,834 --> 00:03:44,914
centers, we're, we're building isolation at every level to get to that.

87
00:03:45,144 --> 00:03:47,734
In, in terms of resiliency, I, I've definitely noticed a trend.

88
00:03:47,734 --> 00:03:50,583
There's, there's increased risk appetite, I would say, from a

89
00:03:50,584 --> 00:03:53,483
lot of customers, who they wanna move faster, uh, and they're

90
00:03:53,483 --> 00:03:57,304
willing to, to trade off some of the multiple nines of availability

91
00:03:57,304 --> 00:04:00,613
that traditionally have been a primary focus for all customers.

92
00:04:00,933 --> 00:04:00,963
Mm-hmm.

93
00:04:01,003 --> 00:04:04,293
Uh, but I think, I think that's a, a smaller number of customers.

94
00:04:04,293 --> 00:04:07,103
I think most customers still care deeply about resiliency.

95
00:04:07,483 --> 00:04:09,324
Uh, I talk to lots of them all the time- Yes … and

96
00:04:09,363 --> 00:04:11,074
they're, they're very focused on resiliency.

97
00:04:11,404 --> 00:04:13,983
They're pushing harder and harder for higher availability, higher

98
00:04:13,983 --> 00:04:16,163
reliability on the network all of the time, which is great.

99
00:04:16,244 --> 00:04:17,763
Uh, that's, that helps us all be better.

100
00:04:18,134 --> 00:04:21,383
Uh, I think for us it's a, it's a balancing act of how do we

101
00:04:21,384 --> 00:04:24,413
build great services that have all the level of resiliency that

102
00:04:24,423 --> 00:04:28,443
some customers want, uh, but don't slow ourselves down with all

103
00:04:28,453 --> 00:04:31,274
the resiliency, uh, in order to move as fast as we possibly can.

104
00:04:31,543 --> 00:04:33,704
So that's the, that's the challenge that we're facing right now.

105
00:04:33,713 --> 00:04:33,933
I- I-

106
00:04:33,933 --> 00:04:35,904
Corey: it's interesting.

107
00:04:35,904 --> 00:04:39,574
The, the resilience demands are much higher in cloud than

108
00:04:39,574 --> 00:04:42,504
they are in the era when most companies ran their own data

109
00:04:42,504 --> 00:04:44,933
centers- Yes … just because of the correlation risk.

110
00:04:44,933 --> 00:04:48,374
Like, great, today my bank is down, tomorrow your grocery store is down.

111
00:04:48,374 --> 00:04:51,493
Well, you're bad at running websites, and so am I, the end.

112
00:04:51,984 --> 00:04:57,693
When AWS has a problem, suddenly lots of companies are suddenly impacted,

113
00:04:57,843 --> 00:05:01,594
even those who have previously done a great job of planning for resilience.

114
00:05:01,594 --> 00:05:05,633
But, oh, no, we have a critical path dependency on a third-party vendor who

115
00:05:05,633 --> 00:05:08,783
did not realize they themselves had that ven- Yep … dependency exposure.

116
00:05:09,053 --> 00:05:10,203
Everything's interconnected.

117
00:05:10,243 --> 00:05:11,883
Some of these are circular.

118
00:05:12,253 --> 00:05:15,363
And the fact that this is not broadly appreciated, understood,

119
00:05:15,363 --> 00:05:18,833
et cetera, by even by many engineers and technologists, again,

120
00:05:18,913 --> 00:05:23,014
is testament to you folks getting this very right very early on.

121
00:05:23,073 --> 00:05:23,253
Yeah.

122
00:05:23,783 --> 00:05:25,843
Which brings us to, I guess, the showpiece thing that

123
00:05:25,853 --> 00:05:29,163
you wanted to show me in your networking lab, which is

124
00:05:29,173 --> 00:05:32,823
the, I guess, the embodiment of the jellyfish paper.

125
00:05:32,904 --> 00:05:33,193
Matt: Mm-hmm.

126
00:05:33,573 --> 00:05:37,104
Corey: Can you explain that in a way that… I, I mean, I could make an

127
00:05:37,104 --> 00:05:40,313
attempt at it- Yeah … but I'm gonna sound way dumber than you will.

128
00:05:40,333 --> 00:05:40,803
Please take it away.

129
00:05:40,803 --> 00:05:43,764
Matt: Yeah, no, a- as you said, uh, like, when AWS was building

130
00:05:43,764 --> 00:05:46,974
the cloud, we, we knew customers were trusting us with their

131
00:05:46,974 --> 00:05:49,553
business, and it's a, it's a tremendous responsibility.

132
00:05:49,883 --> 00:05:52,224
And in order to convince them that this was a good idea,

133
00:05:52,224 --> 00:05:54,783
we knew we had to have nearly perfect availability.

134
00:05:54,783 --> 00:05:57,493
We had to be far superior to what they've been able to do.

135
00:05:57,874 --> 00:05:59,234
And again, that's always been a guiding principle,

136
00:05:59,234 --> 00:06:01,123
especially at our infrastructure and network layers.

137
00:06:01,484 --> 00:06:02,803
Network is underneath everything.

138
00:06:02,803 --> 00:06:05,483
If you have a network issue, it affects many services, many customers.

139
00:06:05,803 --> 00:06:08,643
And we've had a very, very reliable network for many, many years.

140
00:06:08,643 --> 00:06:09,774
We're very proud of our network.

141
00:06:09,803 --> 00:06:12,943
But we saw an opportunity to make it even better, uh, and

142
00:06:12,943 --> 00:06:15,414
particularly in terms of its availability and resiliency.

143
00:06:15,414 --> 00:06:17,064
And so the new network is called Resilient

144
00:06:17,094 --> 00:06:19,872
Network Graph Resilience, or RNG for short.

145
00:06:20,112 --> 00:06:24,332
Uh, and the idea is, is the way networks have been built for really the

146
00:06:24,332 --> 00:06:27,702
past 20 or 30 years, they're hierarchical, they're factory networks for

147
00:06:27,702 --> 00:06:31,642
people who know about networks, and it's really stacked layers of switches.

148
00:06:31,682 --> 00:06:34,032
And so there's a tier one, there's a tier two, there's a tier three.

149
00:06:34,032 --> 00:06:35,631
They're all interconnecting into each other,

150
00:06:35,632 --> 00:06:37,481
and that's how you get big scale networks.

151
00:06:37,481 --> 00:06:37,512
Oh, and

152
00:06:37,512 --> 00:06:39,451
Corey: three, you have aggregation switches that are on, on leaf.

153
00:06:39,451 --> 00:06:39,481
Yep.

154
00:06:39,482 --> 00:06:39,782
You have

155
00:06:40,042 --> 00:06:40,122
Matt: edge switches.

156
00:06:40,122 --> 00:06:41,122
Distribution cores, edges.

157
00:06:41,172 --> 00:06:41,202
Yes.

158
00:06:41,202 --> 00:06:41,222
You

159
00:06:41,222 --> 00:06:41,962
Corey: have a core switch.

160
00:06:42,111 --> 00:06:44,162
You have an out of band switch for the managing- All

161
00:06:44,162 --> 00:06:45,371
the things … the servers that like to break, and

162
00:06:45,591 --> 00:06:45,922
Matt: mm. Yes.

163
00:06:46,052 --> 00:06:46,661
All the things.

164
00:06:46,671 --> 00:06:46,691
Yeah.

165
00:06:46,701 --> 00:06:48,192
And so many layers of switches.

166
00:06:48,201 --> 00:06:50,701
And for a long time, people have asked, "Why do you need all

167
00:06:50,701 --> 00:06:52,871
those layers in your network, Matt? Why, why can't you have a

168
00:06:52,871 --> 00:06:55,802
flatter network?" And, uh, mathematical theory will show you that

169
00:06:55,802 --> 00:06:58,392
as- is actually the most efficient way to build a network, both

170
00:06:58,392 --> 00:07:01,151
from a cost perspective, but also from a reliability perspective.

171
00:07:01,531 --> 00:07:03,962
The problem with building a flat network is all of the

172
00:07:03,972 --> 00:07:07,361
interconnectedness that needs to happen to, to plug everything together.

173
00:07:07,801 --> 00:07:11,391
At minimal scale, you can take a, a switcher that might

174
00:07:11,401 --> 00:07:13,691
have, like, 32 ports on it, and if you have a small number

175
00:07:13,691 --> 00:07:16,350
of switches, you can connect them all together Easy, right?

176
00:07:16,630 --> 00:07:18,370
Uh, if you have many, many thousands of

177
00:07:18,370 --> 00:07:20,080
switches, you can't plug them all together.

178
00:07:20,080 --> 00:07:21,200
You don't have enough ports to.

179
00:07:21,540 --> 00:07:24,710
So you need to effectively randomly interconnect them with some

180
00:07:24,710 --> 00:07:27,470
links, and then you have to figure out how to route through

181
00:07:27,490 --> 00:07:29,640
multiple different switch hops to get to your destination.

182
00:07:29,640 --> 00:07:32,360
There's, there's no structure like there is in a hierarchical

183
00:07:32,360 --> 00:07:35,040
network where everything is very, like, clean and, and orderly.

184
00:07:35,349 --> 00:07:38,669
So Matt has taught us for years this is the best way to build a network.

185
00:07:38,870 --> 00:07:40,639
Many people have tried to build networks like this.

186
00:07:40,679 --> 00:07:42,629
No one, to my knowledge, has ever actually

187
00:07:42,630 --> 00:07:44,500
built a network like this at any sort of scale.

188
00:07:44,770 --> 00:07:46,799
We decided, uh, three or four years ago

189
00:07:46,829 --> 00:07:48,750
that we think we can actually pull this off.

190
00:07:48,869 --> 00:07:50,600
And so we started working down that path and doing

191
00:07:50,600 --> 00:07:52,580
a bunch of research, doing a bunch of simulation.

192
00:07:52,970 --> 00:07:54,719
This is a very novel concept.

193
00:07:54,839 --> 00:07:57,920
Most of the network engineers on my team thought this was a really bad idea.

194
00:07:58,110 --> 00:07:58,959
"Don't do this, Matt.

195
00:07:58,990 --> 00:08:00,999
You know, this is not how we run networks.

196
00:08:00,999 --> 00:08:01,629
It's gonna break.

197
00:08:01,630 --> 00:08:04,460
It's gonna be weird." We, we eventually convinced ourselves through

198
00:08:04,460 --> 00:08:08,319
a lot of simulation that, no, like, this will work, uh, and it

199
00:08:08,319 --> 00:08:10,859
will work more reliably than the network that we have today.

200
00:08:11,319 --> 00:08:14,189
And so we invested, built a- built an engineering team, ag- did

201
00:08:14,189 --> 00:08:16,380
a lot of work, actually started to build these things for real.

202
00:08:16,720 --> 00:08:19,640
Uh, learned a lot of lessons, and that's, that's one of the other keys to

203
00:08:19,640 --> 00:08:22,660
resiliency is learning a lot of lessons and actually taking the learnings

204
00:08:23,010 --> 00:08:25,500
from those lessons and then trying to, like, bake them into your product.

205
00:08:25,500 --> 00:08:27,760
And so in order to build this for real, we had to build this

206
00:08:27,760 --> 00:08:30,290
in smaller scale, we had to test this out, we had to see how

207
00:08:30,290 --> 00:08:32,289
it was going to fail, the problems we were gonna have, and

208
00:08:32,289 --> 00:08:34,869
then iterate quickly into that before we could deploy at scale.

209
00:08:35,289 --> 00:08:37,270
And today we're in production in multiple data centers.

210
00:08:37,279 --> 00:08:40,939
Uh, and this is now the new default network for all core services for AWS.

211
00:08:41,539 --> 00:08:44,159
For any new data center we're building, we're, we're-- basically, we've switched

212
00:08:44,169 --> 00:08:47,550
to this new RNG flat network from the traditional networks we used to build.

213
00:08:47,779 --> 00:08:48,039
Corey: Yeah.

214
00:08:48,150 --> 00:08:52,249
Uh, my take on this is not that i- it's not gonna be possible for you to

215
00:08:52,249 --> 00:08:55,190
do… A real brave take on my part given you already done it and proven it.

216
00:08:55,190 --> 00:08:58,129
Yeah, with the benefit of hindsight, I, I sound real awesome.

217
00:08:58,389 --> 00:09:00,640
No, my question is, why do it at all?

218
00:09:00,650 --> 00:09:00,750
Why

219
00:09:00,750 --> 00:09:01,210
Matt: do it at all?

220
00:09:01,249 --> 00:09:01,829
Corey: Exactly.

221
00:09:01,839 --> 00:09:02,059
Yeah.

222
00:09:02,059 --> 00:09:04,559
Because at some point it's no longer your bottleneck.

223
00:09:04,559 --> 00:09:07,409
Your, your network is already ridiculously reliable.

224
00:09:07,419 --> 00:09:10,229
So at some point, making it even more reliable-

225
00:09:10,239 --> 00:09:12,409
Yeah … seems like that's no longer the bottleneck.

226
00:09:12,439 --> 00:09:14,689
That is no longer the area of focus that-

227
00:09:14,699 --> 00:09:17,369
Yeah … of contention that is customer exposed.

228
00:09:17,689 --> 00:09:21,049
Why spend the investment, research, energy, hardware time-

229
00:09:21,059 --> 00:09:24,249
Yeah … et cetera, on that rather than other areas that are more,

230
00:09:24,569 --> 00:09:27,760
I guess, directly perceived as impactful to customer workloads?

231
00:09:27,879 --> 00:09:28,159
Matt: Yeah.

232
00:09:28,159 --> 00:09:32,289
I, I see any outage in the network as a problem, and while our network

233
00:09:32,290 --> 00:09:35,499
is extremely durable and customer applications are highly reliable, we

234
00:09:35,499 --> 00:09:38,570
still see behind the curtain, and we still see the opportunities and

235
00:09:38,570 --> 00:09:40,939
the risks, and it's just we wanna make this better for our customers.

236
00:09:40,939 --> 00:09:43,750
Like, to me, the network is still in the way in the

237
00:09:43,750 --> 00:09:45,650
sense of you know it's there, you feel it's there.

238
00:09:45,650 --> 00:09:47,589
Sometimes you still have network degradation.

239
00:09:47,980 --> 00:09:50,200
If we can make it even more reliable, it becomes more

240
00:09:50,200 --> 00:09:52,359
invisible, and it enables more customer workloads.

241
00:09:52,359 --> 00:09:54,819
And so it's part of our customer obsession, I guess, at

242
00:09:54,819 --> 00:09:56,719
the end of the day of like, it's just not good enough.

243
00:09:56,959 --> 00:09:59,290
And I think it never will be good enough, really.

244
00:09:59,290 --> 00:10:01,749
It's just a constant, uh, work that we're doing

245
00:10:01,749 --> 00:10:04,009
to continually try and drive improvement in this.

246
00:10:04,389 --> 00:10:06,500
And not all customers will care, but some of them do

247
00:10:06,609 --> 00:10:08,399
and, and therefore, it's worth it for those customers.

248
00:10:08,590 --> 00:10:10,050
Corey: Somewhere between 10 and 20 years ago.

249
00:10:10,060 --> 00:10:10,080
Yeah.

250
00:10:10,180 --> 00:10:15,680
I wound up d- helping with a cluster build-out for more or less a

251
00:10:15,720 --> 00:10:21,240
256-node, uh, HPC cluster w- alike, where they wound up putting a

252
00:10:21,240 --> 00:10:25,040
whole bunch of servers into racks in a data center cage that they had.

253
00:10:25,049 --> 00:10:25,900
Great, awesome.

254
00:10:26,059 --> 00:10:28,520
And the idea was that customers would then pay to basically

255
00:10:28,520 --> 00:10:31,009
load their data into this, do all kinds of number crunching.

256
00:10:31,020 --> 00:10:31,040
Yeah.

257
00:10:31,040 --> 00:10:33,229
And it was a multi-petabyte cluster, which

258
00:10:33,260 --> 00:10:35,720
at that time was reasonably impressive.

259
00:10:35,720 --> 00:10:37,199
Today, you're like, "Ha, that's cute, I have

260
00:10:37,199 --> 00:10:39,989
S3 buckets like that," which, different era.

261
00:10:40,389 --> 00:10:42,809
And a question I had for them during the planning process was,

262
00:10:42,809 --> 00:10:46,150
okay, great, I see the aggregation switches at the top of each rack.

263
00:10:46,180 --> 00:10:49,039
I see the, uh, the core networking structure here.

264
00:10:49,049 --> 00:10:50,230
Great, awesome.

265
00:10:50,529 --> 00:10:52,990
So what… how are these petabytes of data getting here exactly?

266
00:10:52,990 --> 00:10:54,979
"Oh, customers will ship us a pile of disks."

267
00:10:55,059 --> 00:10:55,760
Awesome, great.

268
00:10:56,189 --> 00:10:58,729
"And then we plug them in over there on the bench in the rack. See,

269
00:10:58,729 --> 00:11:01,470
we thought about this. Stop with your impertinent questions." Cool.

270
00:11:01,669 --> 00:11:04,169
Just one more, um, what is the sustained

271
00:11:04,180 --> 00:11:06,649
throughput rate between all of those switches?

272
00:11:06,809 --> 00:11:09,759
And that… and when you saturate, even assuming line rate, how

273
00:11:09,759 --> 00:11:13,900
long does it take to fill all of those servers throughout there?

274
00:11:14,259 --> 00:11:19,400
And the answer distilled down to, "Oh, no." Because it's a narrow pipe problem.

275
00:11:19,420 --> 00:11:20,719
How do you do this?

276
00:11:20,889 --> 00:11:23,019
Uh, it is… and that was not well understood.

277
00:11:23,020 --> 00:11:24,219
These are smart people, I'm not trying to dunk on them.

278
00:11:24,219 --> 00:11:24,549
Oh, yeah.

279
00:11:24,549 --> 00:11:25,249
Yeah.

280
00:11:25,249 --> 00:11:28,050
It, it's the sort of thing that's really obvious the second time.

281
00:11:28,050 --> 00:11:29,649
The first time, you have questions on this.

282
00:11:29,819 --> 00:11:34,020
And even now, companies that were born in AWS and are exploring, "Well, what

283
00:11:34,020 --> 00:11:37,640
if we move this workload to our own data center? Let's go ahead and build

284
00:11:37,640 --> 00:11:44,739
that out," they are misled by a very strange aspect of AWS that I confess

285
00:11:44,740 --> 00:11:47,899
I don't fully understand myself, and I'm hoping you can shed some light.

286
00:11:48,329 --> 00:11:51,338
Uh, in my experience- And you can do this

287
00:11:51,367 --> 00:11:53,198
concurrently with basically everything you imagine.

288
00:11:53,468 --> 00:11:56,377
Pick any two points in your, uh, between, in the same

289
00:11:56,377 --> 00:11:58,527
availability zone, in the same cluster, et cetera.

290
00:11:58,827 --> 00:12:03,798
You can get damn near line rate network transfer between those points.

291
00:12:03,808 --> 00:12:03,938
Mm-hmm.

292
00:12:04,318 --> 00:12:06,168
And then when I have a bunch of things doing that

293
00:12:06,168 --> 00:12:09,617
simultaneously, I don't see a subsequent degradation.

294
00:12:09,627 --> 00:12:11,238
It is still right there- Yeah

295
00:12:11,247 --> 00:12:12,198
at line rate.

296
00:12:12,568 --> 00:12:18,767
And the only answer I have on that is witchcraft, which, m- yeah,

297
00:12:18,767 --> 00:12:20,647
from a certain level of ignorance, everything seems like magic.

298
00:12:20,657 --> 00:12:20,687
Yep.

299
00:12:20,697 --> 00:12:21,137
I get it.

300
00:12:21,377 --> 00:12:22,478
How do you do that?

301
00:12:22,657 --> 00:12:22,907
Matt: Yeah.

302
00:12:23,178 --> 00:12:24,307
We overbuild the network.

303
00:12:24,338 --> 00:12:26,057
I mean, it, the, it is that simple.

304
00:12:26,228 --> 00:12:29,468
We build more network capacity than we need, both from a reliability

305
00:12:29,468 --> 00:12:31,997
perspective so that we have a lot of redundancy and resiliency.

306
00:12:32,277 --> 00:12:34,137
Many devices can fail, many links can fail.

307
00:12:34,137 --> 00:12:36,867
We still have sufficient capacity for all of the customer traffic.

308
00:12:37,017 --> 00:12:40,017
And so it's, it's really is that overbuilding and building as much

309
00:12:40,017 --> 00:12:43,787
capacity as we possibly can to create that illusion of elasticity

310
00:12:43,787 --> 00:12:46,338
or that illusion that basically there is no network there.

311
00:12:46,338 --> 00:12:48,307
These servers are all just directly wired together.

312
00:12:48,587 --> 00:12:50,708
You can send as much data as you want between them all the time.

313
00:12:51,037 --> 00:12:55,257
Um, and that's really been our mission for the last 15 years is if you wanna

314
00:12:55,268 --> 00:12:58,128
do that and you wanna do that at massive scale, you have to do it reliably,

315
00:12:58,128 --> 00:13:01,787
but you also have to have a cost structure so that you can affordably do that.

316
00:13:02,107 --> 00:13:05,727
A lot of the reasons that there are network problems, it comes

317
00:13:05,728 --> 00:13:09,177
down to constraints or lack of capacity, and then people get

318
00:13:09,207 --> 00:13:11,817
creative in that they wanna, like, maximize the usage of the

319
00:13:11,817 --> 00:13:14,787
network, and you get into things like quality of service or how

320
00:13:14,787 --> 00:13:17,638
do I do traffic engineering or how do I move this traffic around?

321
00:13:18,137 --> 00:13:20,736
If you just had more network- You don't actually need

322
00:13:20,736 --> 00:13:24,416
that stuff, and your network runs much, much more reliably

323
00:13:24,746 --> 00:13:24,776
Corey: if I

324
00:13:24,936 --> 00:13:25,226
Matt: have a more-

325
00:13:25,226 --> 00:13:25,996
Corey: Well, how do you even QoS?

326
00:13:25,996 --> 00:13:26,456
Well, I don't know.

327
00:13:26,456 --> 00:13:27,816
If you have the capacity to put it all

328
00:13:27,816 --> 00:13:29,636
through with the same, uh, latency target- You

329
00:13:29,655 --> 00:13:30,926
Matt: don't- … you don't need it … you don't need it.

330
00:13:30,926 --> 00:13:33,685
Yeah, and that, and that's been our mission for many, many

331
00:13:33,686 --> 00:13:35,786
years, is we really try and keep our network very simple.

332
00:13:35,796 --> 00:13:37,626
I mean, that's another secret to our resiliency.

333
00:13:37,956 --> 00:13:40,695
We try not to do creative or fun things in the network.

334
00:13:40,716 --> 00:13:44,586
Keep it as simple as absolutely possible, have a lot of capacity, have a lot

335
00:13:44,605 --> 00:13:47,655
g- more capacity than you actually need so things can fail and you're still

336
00:13:47,656 --> 00:13:52,065
fine, and running a giant network becomes something you can actually achieve.

337
00:13:52,246 --> 00:13:54,096
It- but the, but the key there is it's the cost

338
00:13:54,096 --> 00:13:56,135
structure and the ability to have that much capacity.

339
00:13:56,136 --> 00:13:57,525
Like, how do you actually pull that off?

340
00:13:57,665 --> 00:13:57,826
Mm-hmm.

341
00:13:57,875 --> 00:14:00,875
Uh, for us, it's we started investing in building our own hardware.

342
00:14:01,185 --> 00:14:03,895
And so taking control of our own hardware, we did this about 15 years ago and

343
00:14:03,906 --> 00:14:08,465
started this journey, meant that we could be really prescriptive about just the

344
00:14:08,476 --> 00:14:12,496
things we wanted in the hardware and in the software on our devices, and not

345
00:14:12,496 --> 00:14:15,636
take along everything else that when you buy from a vendor is gonna come along.

346
00:14:15,985 --> 00:14:16,196
It's like- Well,

347
00:14:16,196 --> 00:14:17,785
Corey: you'd have that section of the firmware in case

348
00:14:17,785 --> 00:14:19,756
you wanted to plug it into a Mellanox thing later.

349
00:14:19,756 --> 00:14:19,766
Exactly.

350
00:14:19,766 --> 00:14:21,605
So yeah, we know we're not gonna do that.

351
00:14:21,615 --> 00:14:21,645
Yep.

352
00:14:21,665 --> 00:14:23,605
Why bother, uh, taking up the RAM?

353
00:14:23,625 --> 00:14:24,475
Matt: Delete that code.

354
00:14:24,486 --> 00:14:24,656
Yeah.

355
00:14:24,705 --> 00:14:26,355
It's, it's one less thing that can fail.

356
00:14:26,395 --> 00:14:30,626
Again, uh, few- fewer features, fewer functionality, simple, tailored

357
00:14:30,626 --> 00:14:34,065
to your use case specifically, uh, is, is really the key there.

358
00:14:34,095 --> 00:14:37,765
And so over the last 15 years, we've, again, iteratively matured.

359
00:14:37,766 --> 00:14:39,696
We keep making it better and better and better with every

360
00:14:39,696 --> 00:14:43,665
generation, and you get to this point where today 100% of the

361
00:14:43,665 --> 00:14:46,775
AWS network is built with devices that we, we design ourselves.

362
00:14:46,775 --> 00:14:49,026
And, and the other fun secret about the way we build

363
00:14:49,026 --> 00:14:51,955
our network, we actually use the same switch everywhere.

364
00:14:52,185 --> 00:14:54,605
Uh, so again, most you get these hierarchical networks we

365
00:14:54,605 --> 00:14:57,435
talked about before with core and aggregation and edge and all

366
00:14:57,435 --> 00:15:00,366
these other layers, but also each one of those layers would use

367
00:15:00,366 --> 00:15:03,355
different type of router for, for different fit for purpose.

368
00:15:03,535 --> 00:15:06,245
And there's good advantages and reasons to do that, but then now

369
00:15:06,245 --> 00:15:09,075
you have more complexity, you have more device types to manage.

370
00:15:09,075 --> 00:15:10,585
They all have their own little nuances.

371
00:15:10,925 --> 00:15:14,935
We said, "Well, why can't we just use the basically our top of rack switch, and

372
00:15:14,935 --> 00:15:18,164
why don't we just use it everywhere?" And we've achieved that at this stage.

373
00:15:18,164 --> 00:15:20,404
It took us about 10 years to fully get to the point where

374
00:15:20,404 --> 00:15:22,424
we could do that, but we're bull-headed, and we just

375
00:15:22,424 --> 00:15:24,593
kind of continued to push forward and get to that stage.

376
00:15:24,593 --> 00:15:27,424
But so now, like our entire internet network, our backbone

377
00:15:27,424 --> 00:15:30,444
network, runs on the same top of rack switch that sits

378
00:15:30,444 --> 00:15:32,743
in the top of rack switch and connects to EC2 servers.

379
00:15:32,843 --> 00:15:36,623
Corey: Well, I, I have memories, uh, uh, misspent youth of driving a van with

380
00:15:36,623 --> 00:15:39,413
a core switch in it that cost more than the van we had rented to do this.

381
00:15:39,533 --> 00:15:39,613
Yep, yep.

382
00:15:39,613 --> 00:15:41,963
It's like, "Well, I'm pretty sure we're not insured for

383
00:15:41,963 --> 00:15:44,423
this, but that's the boss's problem," certainly not mine.

384
00:15:44,674 --> 00:15:47,814
It, uh… And we didn't hit anything, so fortunately it was no one's problem.

385
00:15:47,824 --> 00:15:48,324
Yep, yep.

386
00:15:48,324 --> 00:15:48,584
Yeah.

387
00:15:48,584 --> 00:15:51,204
It, it's the, the world has changed.

388
00:15:51,214 --> 00:15:54,613
The, the way you address these things has changed and, and it's, it's wild.

389
00:15:54,633 --> 00:15:56,674
Uh, one caveat, I want- uh, that I know I'm gonna

390
00:15:56,674 --> 00:15:58,703
get comments on this, otherwise if I don't say this.

391
00:15:58,874 --> 00:16:00,383
Like, well, yeah, you talk about economies of

392
00:16:00,433 --> 00:16:03,153
scale, that's why data transfer is so expensive.

393
00:16:03,194 --> 00:16:07,333
I, I get it, but in the context of inside of a VPC, inside

394
00:16:07,334 --> 00:16:10,673
of a subnet, you're getting that full magic line rate between

395
00:16:10,723 --> 00:16:14,614
two endpoints, assume both EC2 instances, that is free.

396
00:16:14,663 --> 00:16:17,793
There is no additional charge metered to customers-

397
00:16:17,803 --> 00:16:20,164
Correct … until it starts crossing other boundaries.

398
00:16:20,174 --> 00:16:20,223
Correct.

399
00:16:20,233 --> 00:16:25,213
So it's, it's not just that you've overbuilt and, uh, made it super awesome.

400
00:16:25,383 --> 00:16:28,843
It, it is not charged explicitly for the most

401
00:16:28,844 --> 00:16:30,863
common use cases where those things matter.

402
00:16:30,883 --> 00:16:31,083
Correct.

403
00:16:31,133 --> 00:16:33,043
And I wanna make sure that nuance is clear.

404
00:16:33,084 --> 00:16:33,263
Correct.

405
00:16:33,263 --> 00:16:36,063
Because otherwise it's, well, yeah, if I was charging X dollars per

406
00:16:36,063 --> 00:16:38,964
gigabyte, I too would invest in making sure you could shove as many

407
00:16:38,973 --> 00:16:41,693
things as possible through it, but that's not what's going on here.

408
00:16:41,743 --> 00:16:41,973
Matt: No.

409
00:16:41,973 --> 00:16:44,894
That's, that's not charged, and as availability zones get larger,

410
00:16:44,963 --> 00:16:47,523
again, there's the magic of elasticity that comes from AWS.

411
00:16:47,523 --> 00:16:50,083
Behind the scenes, those are actual data centers filled with

412
00:16:50,084 --> 00:16:52,664
devices that all have to be interconnected together as we scale

413
00:16:52,664 --> 00:16:55,594
out these availability zones, which means more and more and more

414
00:16:55,594 --> 00:16:58,823
network, and that's also a driver for us to drive cost efficiency.

415
00:16:59,003 --> 00:17:02,134
And so we can keep it free effectively, like we don't want to charge

416
00:17:02,143 --> 00:17:04,304
for that because we want to get the network out of the way and let

417
00:17:04,304 --> 00:17:08,043
customers just move their data at whatever speed they possibly can.

418
00:17:08,194 --> 00:17:08,933
Corey: And that's valuable.

419
00:17:08,973 --> 00:17:11,303
Uh, as long as people can predict that it's not gonna cross those

420
00:17:11,313 --> 00:17:13,483
chargeable boundaries, which is where it's… It's not even that it's too

421
00:17:13,483 --> 00:17:16,793
expensive, it's that I thought it was free and it's not, that scares people.

422
00:17:16,823 --> 00:17:16,943
Yes.

423
00:17:17,093 --> 00:17:20,163
Uh, one other bit of magic in here that I talk to folks,

424
00:17:20,183 --> 00:17:23,833
even folks who have networking backgrounds there, right?

425
00:17:23,833 --> 00:17:25,404
You look at the typical… You, you start

426
00:17:25,404 --> 00:17:27,283
inspecting the traffic that's going over the wire.

427
00:17:27,283 --> 00:17:29,673
You look and you see the physical, uh, the

428
00:17:29,673 --> 00:17:31,494
back address on the physical layer, Layer 2.

429
00:17:31,763 --> 00:17:33,973
You look at the IP logic around Layer 3.

430
00:17:33,973 --> 00:17:37,043
You look at TCP and all the stuff that's being built in as those things happen.

431
00:17:37,054 --> 00:17:37,473
Mm-hmm.

432
00:17:37,483 --> 00:17:41,623
The entire network that you see as a customer is fake.

433
00:17:41,634 --> 00:17:41,664
Yes.

434
00:17:41,673 --> 00:17:46,193
It is an emulated, virtualized imagining of a network

435
00:17:46,194 --> 00:17:50,043
designed to look like networks looked 25 years ago, and it

436
00:17:50,043 --> 00:17:54,063
is entirely living on top of what the reality actually is.

437
00:17:54,103 --> 00:17:56,893
And that is wildly exciting.

438
00:17:57,134 --> 00:17:58,223
Was it always that way?

439
00:17:58,820 --> 00:18:02,260
This episode is sponsored by my own company, Duckbill.

440
00:18:02,550 --> 00:18:04,710
Having trouble with your AWS bill?

441
00:18:04,840 --> 00:18:08,190
Perhaps it's time to renegotiate a contract with them.

442
00:18:08,540 --> 00:18:10,620
Maybe you're just wondering how to predict

443
00:18:10,630 --> 00:18:13,890
what's going on in the wide world of AWS.

444
00:18:13,950 --> 00:18:16,599
Well, that's where Duckbill comes in to help.

445
00:18:16,809 --> 00:18:21,360
Remember, you can't duck the Duckbill bill, which I am reliably

446
00:18:21,360 --> 00:18:25,780
informed by my business partner is absolutely not our motto.

447
00:18:26,000 --> 00:18:29,269
To learn more, visit duckbillhq.com.

448
00:18:30,094 --> 00:18:32,024
Matt: Not in the very, very, very first days.

449
00:18:32,104 --> 00:18:35,754
It, it, when EC2 originally launched, you could see the actual network.

450
00:18:35,774 --> 00:18:38,364
Within about a year, we very quickly realized this was going to

451
00:18:38,364 --> 00:18:41,663
be a bad idea, uh, and that's when we invested in building VPC.

452
00:18:41,663 --> 00:18:41,754
Mm-hmm.

453
00:18:41,754 --> 00:18:46,093
And since, since it's really, I think, since about 2007 or 2008

454
00:18:46,093 --> 00:18:49,204
when we introduced VPC, everything from that point was virtualized,

455
00:18:49,204 --> 00:18:52,044
and the whole idea was make a very simple virtual network for

456
00:18:52,044 --> 00:18:55,284
customers, abstract it and separate it from the physical network.

457
00:18:55,324 --> 00:18:57,383
That way your physical network can change, and you can

458
00:18:57,383 --> 00:18:59,313
do whatever you want on the physical world behind the

459
00:18:59,324 --> 00:19:01,263
scenes, and you're not really disrupting customers.

460
00:19:01,264 --> 00:19:03,834
You're changing the way customers perceive their network to

461
00:19:03,834 --> 00:19:06,714
function, and it's been very powerful for us to have that separation.

462
00:19:06,724 --> 00:19:08,293
Like, I like to tell people I run the real

463
00:19:08,294 --> 00:19:10,533
network at AWS 'cause it's the physical network.

464
00:19:10,533 --> 00:19:11,474
It's the network network.

465
00:19:11,544 --> 00:19:13,154
Corey: I love the fact that it just sounds like you're talking smack.

466
00:19:13,164 --> 00:19:13,673
It's amazing.

467
00:19:13,673 --> 00:19:15,594
Like, well, you know, those fake network things.

468
00:19:15,623 --> 00:19:16,023
I don't know.

469
00:19:16,033 --> 00:19:18,043
Matt: No, and we have, we have amazing teams who build all of the

470
00:19:18,044 --> 00:19:20,983
network services the customers use, uh, and it's super powerful.

471
00:19:20,983 --> 00:19:24,894
Like, I, I work with those teams, but my teams are not tightly coupled, right?

472
00:19:24,943 --> 00:19:28,884
Like, we can build our network relatively separately from the services that are

473
00:19:28,884 --> 00:19:32,194
delivered on top of that network to customers, and that lets us all move faster.

474
00:19:32,983 --> 00:19:37,923
Corey: One thing that I have always wondered about is the reason

475
00:19:37,933 --> 00:19:41,553
that the internet exists is the idea of interoperable standards.

476
00:19:41,564 --> 00:19:41,593
Mm-hmm.

477
00:19:41,604 --> 00:19:42,284
And the one

478
00:19:42,303 --> 00:19:45,173
Like, there were a bunch of early things that came out, like, oh, AppleTalk.

479
00:19:45,483 --> 00:19:46,393
Sure, great.

480
00:19:46,433 --> 00:19:48,403
Uh, IP- ISP, uh, what is it, IPX?

481
00:19:48,403 --> 00:19:50,354
ISP- Yeah … SPX, that.

482
00:19:50,363 --> 00:19:50,653
Yeah.

483
00:19:50,893 --> 00:19:53,133
Things that I don't even remember because that, they did not win.

484
00:19:53,533 --> 00:19:56,083
TCP on top of IP did.

485
00:19:56,193 --> 00:19:56,233
It's

486
00:19:56,244 --> 00:19:56,803
Matt: very durable.

487
00:19:56,844 --> 00:20:00,333
Corey: But if you look at the protocol definitions and see how they are

488
00:20:00,333 --> 00:20:02,674
structured, what they are built for, it's the reason the internet works.

489
00:20:02,683 --> 00:20:07,494
It is designed for a wide variety of network environments with a wide

490
00:20:07,503 --> 00:20:10,533
variety of ever-changing conditions- Yep … in those environments.

491
00:20:11,094 --> 00:20:16,514
Inside of a virtualized network like you have built, a lot of the things that

492
00:20:16,514 --> 00:20:21,123
those services and protocols have been built for historically will not happen.

493
00:20:21,134 --> 00:20:22,763
You can state that with a certainty.

494
00:20:22,763 --> 00:20:26,053
Like, we didn't build the ability for the simulation to ask about the

495
00:20:26,053 --> 00:20:28,363
nature of itself or whatever the, uh- Mm-hmm … the challenge is.

496
00:20:28,893 --> 00:20:32,103
Does that mean that there is a path to a more efficient

497
00:20:32,113 --> 00:20:34,324
protocol for some workloads as long as it doesn't have

498
00:20:34,324 --> 00:20:36,533
to start dealing with the rest of the broader internet?

499
00:20:36,543 --> 00:20:36,713
Yeah.

500
00:20:36,873 --> 00:20:37,723
How do you think about that?

501
00:20:38,174 --> 00:20:38,804
Matt: We already have that.

502
00:20:39,064 --> 00:20:39,534
Um, we're-

503
00:20:39,534 --> 00:20:40,364
Corey: You do under the hood.

504
00:20:40,383 --> 00:20:41,943
I know that much- Yeah … historically.

505
00:20:41,943 --> 00:20:44,274
It was SRD that you were talking about- SRD, yes … a few years back.

506
00:20:44,284 --> 00:20:44,474
Yes.

507
00:20:44,534 --> 00:20:46,374
You announced this at re:Invent, and it was great.

508
00:20:46,374 --> 00:20:50,593
Like, we have this amazing TCP replacement protocol- Yep … that we use at

509
00:20:50,594 --> 00:20:54,564
the time to power, I think it was EFA and a lot of the higher EBS, uh- Yep.

510
00:20:54,623 --> 00:20:56,133
Matt: Everything, everything behind- … resources … EFA

511
00:20:56,133 --> 00:20:59,864
runs on SRD as well as all EBS traffic, uh, within AWS regions.

512
00:20:59,864 --> 00:21:01,183
Yeah, so all the storage traffic.

513
00:21:01,243 --> 00:21:01,523
Corey: Yes.

514
00:21:01,523 --> 00:21:03,204
And, and there was like, "Wow, that's really interesting.

515
00:21:03,204 --> 00:21:06,204
Can I learn more about it?" And the answer was basically no.

516
00:21:06,353 --> 00:21:07,944
And cool, so why are you telling about it?

517
00:21:07,944 --> 00:21:09,443
Like, honestly, it's really neat, and we're proud

518
00:21:09,454 --> 00:21:11,233
of it, and it do- it's how we do these things.

519
00:21:11,636 --> 00:21:14,656
But outside of the context of how we run, what we do, how we

520
00:21:14,656 --> 00:21:16,796
see it- Yeah … it is not something a customer would even find

521
00:21:16,806 --> 00:21:20,556
useful, much less make sense to even go too far down the path.

522
00:21:20,566 --> 00:21:21,096
Yeah, yeah.

523
00:21:21,096 --> 00:21:22,166
I'm talking about the other side of it.

524
00:21:22,216 --> 00:21:25,176
When, once you're in- inside of the virtualized network,

525
00:21:25,186 --> 00:21:28,296
once you have… I have an EC2 instance that's talking to

526
00:21:28,296 --> 00:21:30,556
another, and I wanna send a bunch of this data over- Yeah

527
00:21:30,556 --> 00:21:31,076
very quickly.

528
00:21:31,105 --> 00:21:31,285
Yeah.

529
00:21:31,505 --> 00:21:34,885
Like, as the connection stands up, you start seeing TCP window scaling.

530
00:21:34,886 --> 00:21:34,906
Yeah.

531
00:21:34,916 --> 00:21:36,616
Starts slow, speeds up, et cetera.

532
00:21:36,675 --> 00:21:36,845
Yeah.

533
00:21:36,906 --> 00:21:40,525
If I have certain guarantees around this, I could see, and these are

534
00:21:40,615 --> 00:21:45,286
famous disasters, I could write my own protocol and put traffic over that.

535
00:21:45,316 --> 00:21:48,035
I've been involved with a project once that did it, and I was there- It's hard

536
00:21:48,035 --> 00:21:49,825
monitoring it, I said, "Turn it off." Yeah.

537
00:21:50,125 --> 00:21:51,295
It's really hard.

538
00:21:51,625 --> 00:21:54,426
But I could see you folks putting something like that out for-

539
00:21:54,426 --> 00:21:54,446
Matt: Yeah.

540
00:21:54,446 --> 00:21:55,286
We're gonna… We, we have.

541
00:21:55,296 --> 00:21:59,035
So SRD, you can go to an EC2 instance right now- Mm-hmm … on your, your

542
00:21:59,066 --> 00:22:03,485
ENI or, and you can enable, basically, SRD transport under the covers.

543
00:22:03,825 --> 00:22:06,895
It'll still look like TCP that you're communicating with, but behind the

544
00:22:06,895 --> 00:22:10,945
scenes, we wrap your TCP packets in SRD, which effectively makes them

545
00:22:10,945 --> 00:22:15,505
extremely reliable, lower latency, higher throughput, and send the TCP bits.

546
00:22:15,505 --> 00:22:17,785
And so from a TCP perspective, it just kind of looks like you're

547
00:22:17,786 --> 00:22:20,485
on a magic network with infinite bandwidth and no packet loss.

548
00:22:20,905 --> 00:22:22,206
But you can… But that way, there's no- That one

549
00:22:22,365 --> 00:22:23,395
Corey: snuck completely past me.

550
00:22:23,435 --> 00:22:23,695
Fantastic

551
00:22:23,695 --> 00:22:25,916
… 
Matt: but there's no change for the customer, and that's, that's the big thing.

552
00:22:25,916 --> 00:22:27,485
Like, you can adopt EFA, right?

553
00:22:27,505 --> 00:22:30,545
But if you are adopting EFA, there's a lot of work for you to do

554
00:22:30,545 --> 00:22:33,605
in your application so that you can make it EFA capable, and a lot

555
00:22:33,606 --> 00:22:36,575
of customers just won't do that or can't do that, and we wanted

556
00:22:36,575 --> 00:22:39,146
to, like, how do we make the network better for all of them?

557
00:22:39,566 --> 00:22:41,216
And so right now this is an opt-in feature- Yeah

558
00:22:41,256 --> 00:22:42,376
that you can go turn on.

559
00:22:42,636 --> 00:22:44,646
Again, we're busily working behind the scenes

560
00:22:44,656 --> 00:22:46,086
to be, 'How do we make this the default?

561
00:22:46,096 --> 00:22:49,666
Like, how is this just the way it works for everyone?' And there's a

562
00:22:49,666 --> 00:22:51,945
bunch of technical challenges, but we're, we're chipping away at that.

563
00:22:52,026 --> 00:22:53,206
Corey: Today at least, what are the workloads for

564
00:22:53,206 --> 00:22:55,176
which that makes sense, and which is one of those-

565
00:22:55,205 --> 00:22:56,586
Matt: Almost all workloads, honestly.

566
00:22:56,586 --> 00:22:56,675
Corey: Okay.

567
00:22:56,675 --> 00:22:56,845
Okay.

568
00:22:56,875 --> 00:22:57,795
Is there any exception cases?

569
00:22:57,845 --> 00:22:58,666
Ooh, do not use it

570
00:22:58,666 --> 00:22:58,726
Matt: for

571
00:22:58,726 --> 00:22:59,135
Corey: that one

572
00:22:59,136 --> 00:22:59,186
Matt: thing?

573
00:22:59,236 --> 00:23:00,865
Or- The, the, the exception case is there's,

574
00:23:00,886 --> 00:23:03,935
there is a tax of, uh, maximum packets per second.

575
00:23:04,246 --> 00:23:06,355
Mm. So in order to do that encapsulation on the hardware

576
00:23:06,355 --> 00:23:09,636
layer, the peak PPS you can get is reduced slightly, and that's

577
00:23:09,636 --> 00:23:11,315
the reason we haven't gone and turned it on by default- Okay

578
00:23:11,365 --> 00:23:11,995
for people yet.

579
00:23:12,316 --> 00:23:13,526
Again, we're chipping away and r- and

580
00:23:13,526 --> 00:23:15,515
reducing and kind of removing that bottleneck.

581
00:23:15,546 --> 00:23:17,965
But for the vast majority of workloads, unless you're

582
00:23:18,115 --> 00:23:21,295
very, very sensitive to peak PPS, uh, enabling that

583
00:23:21,296 --> 00:23:24,005
functionality gets you more reliability in your connectivity.

584
00:23:24,055 --> 00:23:25,755
And, and the real way you see this is if you're

585
00:23:25,755 --> 00:23:27,785
looking at, like, tail latency for your application.

586
00:23:28,115 --> 00:23:31,525
You're looking at your P99, P99.9 latency.

587
00:23:31,525 --> 00:23:33,525
That's gon- usually gonna be dominated by some

588
00:23:33,525 --> 00:23:36,165
type of packet loss or some other, uh, other issue.

589
00:23:36,455 --> 00:23:39,825
And if you enable this with SRD based transport, mostly that disappears.

590
00:23:39,855 --> 00:23:41,515
And that's, that's the reason we've done it for EBS.

591
00:23:41,515 --> 00:23:44,855
Tail latency matters a whole lot when you're talking about storage workloads.

592
00:23:44,945 --> 00:23:46,935
And so we were on a mission to be like, 'How do we drive

593
00:23:46,975 --> 00:23:49,016
that tail latency down?' And that's where we came up with

594
00:23:49,016 --> 00:23:52,456
SRD and moved all, uh, all of the EBS traffic over to SRD.

595
00:23:52,695 --> 00:23:53,705
Corey: Okay, I've seen it all now.

596
00:23:53,716 --> 00:23:55,046
All I have to do is talk to one more AWS

597
00:23:55,265 --> 00:23:56,825
customer, and I see things that are new.

598
00:23:57,125 --> 00:24:00,135
Uh, the challenge I always ask, okay, where is this not an appropriate fit

599
00:24:00,136 --> 00:24:03,476
for, is because very often I will talk to folks who hear about these things.

600
00:24:03,476 --> 00:24:06,076
They're like, 'Great, we're gonna go and implement that everywhere.' Great.

601
00:24:06,325 --> 00:24:07,406
What does your workload look like?

602
00:24:07,445 --> 00:24:11,976
'Oh, it calls out to, uh, an AI inference provider, and it waits for the

603
00:24:11,976 --> 00:24:14,685
response, and the response comes back, and we're getting fast speeds.

604
00:24:14,685 --> 00:24:16,985
We're getting almost 200 tokens a second on that.

605
00:24:17,005 --> 00:24:20,020
It's-' I promise the network is not your bottleneck right now.

606
00:24:20,030 --> 00:24:20,050
No.

607
00:24:20,070 --> 00:24:21,000
It's fine.

608
00:24:21,540 --> 00:24:23,100
This is not the place to optimize.

609
00:24:23,110 --> 00:24:23,129
Yeah.

610
00:24:23,139 --> 00:24:25,790
There are, there are better paths forward for you.

611
00:24:25,870 --> 00:24:26,040
Yeah.

612
00:24:26,090 --> 00:24:31,159
I, I, I'm excited for the day where this does become the default,

613
00:24:31,419 --> 00:24:35,279
with the obvious caveat then that I can see a lot of workloads that

614
00:24:35,280 --> 00:24:38,579
people are gonna try deploying somewhere else get disparate results

615
00:24:38,679 --> 00:24:42,459
as a result of this once that becomes a default, and start to wonder.

616
00:24:42,459 --> 00:24:45,270
I mean, sure, the, the blame is going to accrue in directions

617
00:24:45,270 --> 00:24:47,469
that are not aimed at you because you're the good example in this.

618
00:24:47,759 --> 00:24:49,580
How come it's so crappy in our data center?

619
00:24:49,580 --> 00:24:51,659
Probably our terrible networking team.

620
00:24:52,039 --> 00:24:53,089
Which is not true.

621
00:24:53,089 --> 00:24:56,399
It's just this weird- Yeah … almost under- under the hood magic.

622
00:24:56,459 --> 00:24:56,679
Matt: Yeah.

623
00:24:56,679 --> 00:24:59,749
And, and, and really, like something like SRD and our capability

624
00:24:59,749 --> 00:25:02,260
to do that, it's, it's similar to our network investment and

625
00:25:02,399 --> 00:25:04,769
we wanna own our own devices, we wanna own our own software.

626
00:25:05,089 --> 00:25:07,699
And we've done on the, on the server side for years with Nitro.

627
00:25:07,949 --> 00:25:08,899
We have our own hardware.

628
00:25:08,909 --> 00:25:12,770
We, we abstract all of the customer bits with VPCs, which means

629
00:25:12,829 --> 00:25:15,859
we can do interesting thing behind the scenes and we don't have

630
00:25:15,859 --> 00:25:18,830
to change, you know, many, many years long standards like TCP.

631
00:25:19,519 --> 00:25:20,549
You wanna change TCP?

632
00:25:20,919 --> 00:25:21,399
Good luck.

633
00:25:21,569 --> 00:25:23,270
And you wanna try and change everyone's applications.

634
00:25:23,270 --> 00:25:26,699
The way to make meaningful change is to leave TCP alone, but just

635
00:25:26,980 --> 00:25:29,143
basically magically make it better by doing something behind the scenes.

636
00:25:29,143 --> 00:25:29,300
Embrace, extend,

637
00:25:29,300 --> 00:25:29,969
Corey: and yeah.

638
00:25:30,020 --> 00:25:30,230
Matt: Yeah.

639
00:25:30,279 --> 00:25:30,699
Corey: Yeah.

640
00:25:31,099 --> 00:25:33,639
So as you talk to people about what you've been

641
00:25:33,639 --> 00:25:37,259
doing for the last, well, 15 years at least- Yeah

642
00:25:37,289 --> 00:25:40,259
uh, what do you find is the biggest misconception

643
00:25:40,259 --> 00:25:43,039
that customers have about AWS's network?

644
00:25:43,249 --> 00:25:44,039
Matt: About AWS's network?

645
00:25:44,359 --> 00:25:45,620
Most people don't know it exists.

646
00:25:46,550 --> 00:25:47,789
Corey: I agree wholeheartedly.

647
00:25:47,800 --> 00:25:47,820
Yeah.

648
00:25:47,829 --> 00:25:49,100
Yes, yes.

649
00:25:49,100 --> 00:25:49,199
It-

650
00:25:49,199 --> 00:25:49,369
Matt: Yes.

651
00:25:49,529 --> 00:25:52,709
And so it's, it's just the lack of understanding of the sheer scale

652
00:25:52,859 --> 00:25:54,929
and all of the investment and all the work that goes on behind the

653
00:25:54,929 --> 00:25:58,499
scenes to just make all of this function and work for customers.

654
00:25:58,499 --> 00:26:01,779
And, and friends and family and everyone else, they just see it as, well,

655
00:26:01,779 --> 00:26:04,949
there's just, you know, some magic happens to the internet, things happen, so.

656
00:26:05,422 --> 00:26:08,722
Corey: Uh, a while back, uh, in one of his iterations, Peter DeSantis, before

657
00:26:08,722 --> 00:26:14,152
he transitioned to his Amazon role now, was the SVP of Utility Computing.

658
00:26:14,422 --> 00:26:14,482
Mm-hmm.

659
00:26:14,492 --> 00:26:18,912
And I love the term because it, it encapsulated an experience

660
00:26:18,912 --> 00:26:21,082
I think lots of folks have, even if they didn't notice it.

661
00:26:21,362 --> 00:26:25,722
Uh, when something is a utility, uh, like, uh, electricity

662
00:26:25,762 --> 00:26:28,732
or water- Mm-hmm … you don't turn on the switch or turn

663
00:26:28,732 --> 00:26:31,481
on the faucet and then wonder if it's gonna work this time.

664
00:26:31,512 --> 00:26:32,462
It is expected.

665
00:26:32,512 --> 00:26:32,622
Yep.

666
00:26:32,811 --> 00:26:36,152
And we've all gone through that as consumers.

667
00:26:36,332 --> 00:26:41,052
There was a time at some point where when I go to google.com in my web browser

668
00:26:41,071 --> 00:26:44,681
and it doesn't work, in the early days, Google must be having an issue.

669
00:26:44,852 --> 00:26:47,352
And at some point, it switched for all of us, "Oh, my

670
00:26:47,352 --> 00:26:49,611
Wi-Fi must be having problems." Yes, it's my problem.

671
00:26:49,611 --> 00:26:49,621
Yeah.

672
00:26:49,621 --> 00:26:50,362
Exactly.

673
00:26:50,372 --> 00:26:54,121
To the point where on the rare occasion, uh, I can't think of one in the last

674
00:26:54,122 --> 00:26:57,211
10 years, but toward the end of that period when Google would actually be

675
00:26:57,211 --> 00:27:01,091
down and have an issue serving, I didn't realize that was even a possibility.

676
00:27:01,091 --> 00:27:02,151
How could that happen?

677
00:27:02,422 --> 00:27:06,762
It, uh… And I really think the network, adding AWS to the

678
00:27:06,762 --> 00:27:10,282
site, is in so many ways almost a victim of its own success.

679
00:27:10,322 --> 00:27:13,391
Matt: The electricity example's exactly what we have in our head.

680
00:27:13,721 --> 00:27:15,861
Like, we want the network to be like the light switch.

681
00:27:15,861 --> 00:27:18,382
You flip it on and off, you expect it to always work, and

682
00:27:18,382 --> 00:27:20,901
it's extremely rare and surprising when it doesn't work, and

683
00:27:20,901 --> 00:27:23,331
that's, that's been our mission, is make the network invisible.

684
00:27:23,331 --> 00:27:24,131
Get it out of the way.

685
00:27:24,771 --> 00:27:26,691
Corey: A, a question I often get from folks is, "Great,

686
00:27:26,701 --> 00:27:28,641
how, how did you get started?" How did I become me?

687
00:27:28,681 --> 00:27:28,811
Yeah.

688
00:27:28,822 --> 00:27:31,381
And the answer, I started off as a Unix, followed

689
00:27:31,381 --> 00:27:33,232
shortly by Linux- Yeah … systems administrator.

690
00:27:33,541 --> 00:27:37,561
And during the, uh, the global financial crisis in 2008, suddenly

691
00:27:37,561 --> 00:27:40,011
salary freeze, everyone hates their job, but no one's hiring.

692
00:27:40,172 --> 00:27:41,462
What do I do for the next year?

693
00:27:41,492 --> 00:27:43,212
Well, I started learning networking, 'cause that

694
00:27:43,212 --> 00:27:45,551
had always been an area- Yeah … I hand waved over.

695
00:27:45,771 --> 00:27:48,092
And I did dabble as a network engineer after that.

696
00:27:48,092 --> 00:27:50,971
But by and large, it made me a much better systems person.

697
00:27:50,981 --> 00:27:51,001
Yeah.

698
00:27:51,011 --> 00:27:54,091
Because once I understood what was on the wire, I could reason about it.

699
00:27:54,091 --> 00:27:56,601
I could understand- Yeah … what was going on.

700
00:27:57,071 --> 00:27:59,561
The challenge now is that we have complexity on top of

701
00:27:59,561 --> 00:28:02,262
complexity on top of complexity and so on and so forth.

702
00:28:02,422 --> 00:28:04,411
It is dizzyingly high now.

703
00:28:04,661 --> 00:28:11,172
And I worry on some level that networking as a whole is no longer

704
00:28:11,192 --> 00:28:14,872
as central and core to modern engineering understanding of this.

705
00:28:15,252 --> 00:28:19,322
There are publicly traded companies built in the cloud that do very well,

706
00:28:19,552 --> 00:28:23,842
and their internal networking teams, their entire professional expertise

707
00:28:23,852 --> 00:28:26,532
is more or less around configuring things like Transit Gateway- Yeah,

708
00:28:26,732 --> 00:28:28,702
yeah … and Cloud WAN, and these are all abstractions on top of it.

709
00:28:28,822 --> 00:28:32,381
They don't understand that even the old days of MDI-X autosense,

710
00:28:32,531 --> 00:28:34,472
which was a consumer feature then came to us, like, great,

711
00:28:34,472 --> 00:28:37,392
when I plug the ethernet cable in, it's not lighting up.

712
00:28:37,402 --> 00:28:37,772
Why?

713
00:28:37,951 --> 00:28:39,860
Oh, that's a s- that's a rollover cable.

714
00:28:39,860 --> 00:28:40,941
Yeah, you got to cross that over, yeah.

715
00:28:40,941 --> 00:28:41,181
You need a straight through.

716
00:28:41,181 --> 00:28:43,071
It's… Wait, you mean there's different

717
00:28:43,071 --> 00:28:45,282
standards for the, for the wiring standard?

718
00:28:45,282 --> 00:28:47,812
I have a mug that just has the colors of the ethernet B spec.

719
00:28:47,821 --> 00:28:47,992
Mm-hmm.

720
00:28:48,021 --> 00:28:50,352
And every once in a while, someone sees it, they're like, "Yes!"

721
00:28:50,361 --> 00:28:51,702
Like, okay, I found my people.

722
00:28:51,762 --> 00:28:52,232
It's great.

723
00:28:52,551 --> 00:28:54,602
But now I sound like an old man talking about

724
00:28:54,602 --> 00:28:56,552
the Great War- Yeah … once upon a time.

725
00:28:57,071 --> 00:28:59,692
Where does the next generation of all of this come from?

726
00:28:59,852 --> 00:29:02,011
Matt: Yeah, we're, uh… It's something we grapple with.

727
00:29:02,012 --> 00:29:05,141
And, like, again, we've been very successful such that most of our customers

728
00:29:05,141 --> 00:29:08,512
don't have to deal with the vagaries of networking, and you like it.

729
00:29:08,521 --> 00:29:08,541
Yeah.

730
00:29:08,751 --> 00:29:10,101
Most people look at that, and they're like,

731
00:29:10,111 --> 00:29:11,352
"That's terrible. I don't wanna do that." I

732
00:29:11,352 --> 00:29:12,511
Corey: like it, and I'm far enough away from

733
00:29:12,511 --> 00:29:15,151
it now that I can wax nostalgic about this.

734
00:29:15,152 --> 00:29:17,671
If I were on the phone this morning with Cisco TAC because of a

735
00:29:17,671 --> 00:29:20,741
weird routing issue- Man … I would be considerably less charitable.

736
00:29:20,751 --> 00:29:20,781
Yes.

737
00:29:20,781 --> 00:29:22,321
But that's 10 years in my past.

738
00:29:22,332 --> 00:29:23,931
Matt: It… For, for us, it's really grow your own.

739
00:29:23,982 --> 00:29:27,621
Uh, and so, like, I mean, I joined Amazon as a very junior engineer

740
00:29:27,622 --> 00:29:31,771
back in 2008, and my initial job was kind of doing monitoring of the

741
00:29:31,771 --> 00:29:34,691
network and, you know, seeing things breaking and learning to fix it.

742
00:29:34,692 --> 00:29:37,062
And I've stuck around long enough and learned enough

743
00:29:37,101 --> 00:29:39,541
that I managed to grow into this position, uh, and it's

744
00:29:39,552 --> 00:29:41,721
kind of the same path we have with all of our new hires.

745
00:29:41,722 --> 00:29:43,971
We're, we're hiring lots of junior engineers, either

746
00:29:43,972 --> 00:29:46,001
from… We hire lots of people from our data centers,

747
00:29:46,001 --> 00:29:48,501
actually, some amazingly talented people who were there.

748
00:29:48,501 --> 00:29:51,052
I was just talking to someone who was visiting one of our data

749
00:29:51,052 --> 00:29:53,201
centers, and it's someone who was working at a gas station

750
00:29:53,211 --> 00:29:55,742
three years ago, and they were just wowed with what they knew.

751
00:29:55,742 --> 00:29:57,901
And I'm like, "We should go hire that guy for the networking

752
00:29:57,931 --> 00:30:00,091
team," 'cause they actually touch the physical equipment.

753
00:30:00,372 --> 00:30:01,891
They understand how this stuff works, and they,

754
00:30:01,902 --> 00:30:04,041
they often become our best engineers in the future.

755
00:30:04,332 --> 00:30:07,691
Um, but it is a deep investment in us in terms of, like, hiring younger

756
00:30:07,692 --> 00:30:10,981
talent, junior talent, and, like, going through the process and training

757
00:30:10,981 --> 00:30:14,211
and growing them and teaching them how we build networks 'cause there's…

758
00:30:14,262 --> 00:30:15,841
At the end of the day, there's no accelerator for this.

759
00:30:15,851 --> 00:30:18,242
It is increasingly complicated, as you said.

760
00:30:18,242 --> 00:30:21,242
It's much, much more complicated than it was 15 years ago when I was starting.

761
00:30:21,622 --> 00:30:25,332
But you still need to know the same foundational things that I had to know

762
00:30:25,332 --> 00:30:29,811
15 years ago, but you also have to know this increasing mountain of new stuff

763
00:30:29,812 --> 00:30:34,302
over the top, and that only comes from just- Learning the basics, learning the

764
00:30:34,302 --> 00:30:38,832
practical bits, and then slowly adding the knowledge on top with experience

765
00:30:38,962 --> 00:30:39,792
Corey: I mean, 'cause you know that.

766
00:30:39,852 --> 00:30:40,902
Customers don't have to.

767
00:30:41,072 --> 00:30:43,612
I, I wanna highlight just how rare that is.

768
00:30:43,612 --> 00:30:46,392
It, it seems that the idea of developing talent internally-

769
00:30:46,541 --> 00:30:49,531
Yeah … has almost gone out of fashion in the industry.

770
00:30:49,532 --> 00:30:51,972
It's, "Oh, no, this person doesn't have experience. We can't hire them

771
00:30:51,972 --> 00:30:54,382
for the role 'cause it'll take six months to get them up to speed."

772
00:30:54,401 --> 00:30:54,971
Yeah, I get it.

773
00:30:55,381 --> 00:30:56,941
Nine months later, the role's still open.

774
00:30:56,991 --> 00:30:57,081
Yeah.

775
00:30:57,151 --> 00:30:58,421
So what are we really doing here, friends?

776
00:30:58,422 --> 00:30:58,502
Yeah.

777
00:30:58,721 --> 00:31:02,392
The idea of teaching people and, and arming them to do this, 'cause there was

778
00:31:02,392 --> 00:31:07,141
a time where every company that wanted to be on the internet, which let's face

779
00:31:07,142 --> 00:31:09,931
it, is probably most of them- Mm-hmm … needed to have a networking- Yeah

780
00:31:09,942 --> 00:31:11,232
if not person, team.

781
00:31:11,251 --> 00:31:11,531
Team,

782
00:31:11,541 --> 00:31:11,561
Matt: yeah.

783
00:31:11,571 --> 00:31:14,832
Corey: Where it was you need people able to handle this stuff.

784
00:31:14,852 --> 00:31:16,881
It gets really weird really fast.

785
00:31:16,881 --> 00:31:16,941
Yeah.

786
00:31:17,372 --> 00:31:19,291
And now you don't need that.

787
00:31:19,302 --> 00:31:21,931
I mean, I, I went through a parallel evolution running email systems.

788
00:31:21,942 --> 00:31:21,972
Yep.

789
00:31:21,981 --> 00:31:24,551
Now most companies do not run their own mail servers.

790
00:31:24,561 --> 00:31:24,741
Nor

791
00:31:24,741 --> 00:31:24,851
Matt: should

792
00:31:24,852 --> 00:31:24,872
Corey: you.

793
00:31:24,872 --> 00:31:25,452
Thank God.

794
00:31:27,471 --> 00:31:27,551
Yeah.

795
00:31:27,641 --> 00:31:30,012
Yeah, n- and networking is, is still continuing to

796
00:31:30,012 --> 00:31:32,552
evolve, and I'm not naive to th- enough to think that we

797
00:31:32,552 --> 00:31:35,471
are at the end of history, and, like, in our final form.

798
00:31:35,931 --> 00:31:36,721
What's the next step?

799
00:31:36,731 --> 00:31:37,842
What's the next phase?

800
00:31:37,861 --> 00:31:39,132
Where do you see this going?

801
00:31:39,251 --> 00:31:40,131
Matt: Where do I see this going?

802
00:31:40,211 --> 00:31:43,198
Uh, it's- It's really more of the same.

803
00:31:43,198 --> 00:31:47,098
I mean, the, there's so much, there's been an acceleration in investment in

804
00:31:47,098 --> 00:31:50,698
networking because of AI, which is exciting as someone who does networking.

805
00:31:50,707 --> 00:31:52,708
The, there's bandwidth demands are higher, there's

806
00:31:52,708 --> 00:31:54,918
more capacity to build, there's more things to connect.

807
00:31:54,918 --> 00:31:57,588
And so I've seen more energy go into networking in the

808
00:31:57,588 --> 00:32:00,037
last three years than I had in the previous 10 years.

809
00:32:00,318 --> 00:32:03,568
And so there's just accelerated in investments both at the silicon and the

810
00:32:03,568 --> 00:32:06,588
hardware level, but as well as the software, the protocols, everything else.

811
00:32:06,588 --> 00:32:09,837
It, it feels like networking is exciting again i- in a way that

812
00:32:09,837 --> 00:32:12,647
maybe we made it fairly boring by the time you got to 2019.

813
00:32:12,648 --> 00:32:12,978
And that it's just working.

814
00:32:12,978 --> 00:32:13,618
I will say

815
00:32:13,618 --> 00:32:15,547
Corey: when, in my misspent youth, there were times I

816
00:32:15,568 --> 00:32:17,537
wished it had been a lot less exciting- Yes … some nights.

817
00:32:17,537 --> 00:32:17,817
Yes.

818
00:32:17,817 --> 00:32:18,368
But yes.

819
00:32:18,377 --> 00:32:18,777
Matt: Yes.

820
00:32:18,847 --> 00:32:21,458
Um, but yes, uh, and so I, I think it's just gonna be more of that.

821
00:32:21,458 --> 00:32:23,587
Like, we're gonna continue to find… The

822
00:32:23,588 --> 00:32:26,237
world has an insatiable appetite for bandwidth.

823
00:32:26,257 --> 00:32:26,287
Yes.

824
00:32:26,287 --> 00:32:29,087
A- and I think we're only scratching the surface of that.

825
00:32:29,147 --> 00:32:33,427
And if we were able to 100X the internet and network capacity,

826
00:32:33,697 --> 00:32:36,617
it wouldn't take long for applications to find uses for that.

827
00:32:36,867 --> 00:32:39,998
And I think there's still a lot of things that are not done,

828
00:32:39,998 --> 00:32:42,877
or they're done in ineffective ways because of either, uh, not

829
00:32:42,877 --> 00:32:45,957
enough capacity of network and/or, like, costs of networking.

830
00:32:46,088 --> 00:32:47,967
And so the more you reduce that and eliminate that, I

831
00:32:47,977 --> 00:32:50,398
think you just open up new opportunities and innovation.

832
00:32:50,398 --> 00:32:52,557
And so I think it's just gonna be this constant

833
00:32:52,578 --> 00:32:54,572
march for more and more and more and more capacity.

834
00:32:54,572 --> 00:32:54,767
Well,

835
00:32:54,767 --> 00:32:55,937
Corey: we, we've seen that in real time.

836
00:32:55,937 --> 00:32:58,847
The, the thing that drove that home to me more than almost anything else was

837
00:32:58,867 --> 00:33:04,407
the day that GDPR took effect because a lot of US-based websites, uh, suddenly

838
00:33:04,547 --> 00:33:07,427
were not fully compliant, so they turned off all the tracking and all the rest.

839
00:33:07,428 --> 00:33:07,648
Yeah.

840
00:33:07,718 --> 00:33:11,557
And suddenly the internet was blindingly fast.

841
00:33:11,557 --> 00:33:11,597
Super

842
00:33:11,597 --> 00:33:11,807
Matt: fast.

843
00:33:11,957 --> 00:33:13,828
Corey: It was like, wow, you're loading an article, and,

844
00:33:13,828 --> 00:33:16,208
like, the total load was something like- Yeah … 50K.

845
00:33:16,208 --> 00:33:16,227
Yeah.

846
00:33:16,227 --> 00:33:17,448
It was wild to me.

847
00:33:17,638 --> 00:33:19,458
And you look at the same article with all the other stuff turned

848
00:33:19,458 --> 00:33:24,378
on, it's why is loading that 50K article taking 25 megabytes- Yeah

849
00:33:24,378 --> 00:33:25,228
of nonsense?

850
00:33:25,237 --> 00:33:28,544
It-- As soon as you have the capacity, people find ways to fill it.

851
00:33:28,544 --> 00:33:30,017
Or just- I'm not opining on whether it's all useful … or

852
00:33:30,017 --> 00:33:31,898
Matt: just look at streaming video, for example, right?

853
00:33:31,907 --> 00:33:31,927
Yeah.

854
00:33:31,937 --> 00:33:35,834
Like, I mean, you, you can get very good quality with

855
00:33:35,844 --> 00:33:39,194
the, the latest codecs, but it's not the best quality.

856
00:33:39,534 --> 00:33:42,773
Like real broadcast level quality video is, is like an order of

857
00:33:42,774 --> 00:33:45,374
magnitude or two higher bandwidth than what's streamed to you

858
00:33:45,374 --> 00:33:49,084
even if you're getting like the 4K Ultra whatever on your Fire TV.

859
00:33:49,314 --> 00:33:51,344
And so that- the reason is bandwidth.

860
00:33:51,434 --> 00:33:53,234
There's always need for more bandwidth, and

861
00:33:53,234 --> 00:33:54,644
there always will be need for more bandwidth.

862
00:33:54,644 --> 00:33:57,523
And so the more we can do to- Mm … to make it cheaper and more reliable

863
00:33:57,524 --> 00:34:00,133
and just more prevalent, we're only gonna be benefiting our customers.

864
00:34:00,184 --> 00:34:01,924
Corey: No, like you have the, the news anchor trucks

865
00:34:01,924 --> 00:34:03,434
that show up at the scene of something going on.

866
00:34:03,434 --> 00:34:03,507
Yeah.

867
00:34:03,507 --> 00:34:05,043
They have the big satellite dish on the roof.

868
00:34:05,043 --> 00:34:06,824
It's like, do you think that's because they haven't

869
00:34:06,824 --> 00:34:09,243
quite figured out that you can tether to your cell phone?

870
00:34:09,463 --> 00:34:11,483
There are some very dedicated, very

871
00:34:11,483 --> 00:34:13,363
concerning bandwidth requirements around this.

872
00:34:13,364 --> 00:34:13,454
Yes.

873
00:34:13,553 --> 00:34:17,423
And it's one of those areas, networking more so than many, but complexity

874
00:34:17,423 --> 00:34:20,923
passes everywhere, and it just slips below the baseline surface of awareness.

875
00:34:21,003 --> 00:34:23,103
Thank you so much for being as generous with your time as you

876
00:34:23,103 --> 00:34:25,913
are and explaining how some of this, this magic all works.

877
00:34:26,313 --> 00:34:29,383
If someone's watching this or listening to this, depending upon

878
00:34:29,384 --> 00:34:32,874
their point of view, and they wanna learn more about how networks

879
00:34:32,884 --> 00:34:36,263
work and how infrastructure happens, where should they start?

880
00:34:36,263 --> 00:34:39,083
Where should they begin learning about this, this

881
00:34:39,213 --> 00:34:42,034
secret thing that still drives our entire world?

882
00:34:42,283 --> 00:34:42,583
Matt: Yeah.

883
00:34:42,584 --> 00:34:45,283
It, it's, it's again, it's going back to the, the raw material.

884
00:34:45,283 --> 00:34:48,283
And then there's a book written in I think the '70s or '80s.

885
00:34:48,283 --> 00:34:50,844
It's The TCP/IP Illustrated Book by Stevens.

886
00:34:51,114 --> 00:34:52,213
It's where I'd suggest you go.

887
00:34:52,283 --> 00:34:55,303
Like that's where I send anyone who's wants to get into networking and

888
00:34:55,303 --> 00:34:58,213
learn something new 'cause those same fundamentals like you talked about

889
00:34:58,213 --> 00:35:01,508
of all those standards They still matter, and the, the entire world is

890
00:35:01,508 --> 00:35:04,958
built on y- you know, versions of them that have, have evolved over time.

891
00:35:04,958 --> 00:35:07,638
But that fundamental skill set will, will take you a long way, and

892
00:35:07,638 --> 00:35:09,678
you have to have it to really understand how these things work.

893
00:35:09,907 --> 00:35:10,748
Corey: And we'll of course put a link to that.

894
00:35:10,748 --> 00:35:11,758
I love that book myself.

895
00:35:11,758 --> 00:35:11,870
It's a great book.

896
00:35:11,870 --> 00:35:14,418
Though I will say the, some of the bandwidth references

897
00:35:14,418 --> 00:35:16,968
in there of like, "Oh, that's a really fast 10BASE-T

898
00:35:16,968 --> 00:35:19,148
connection there," that might not have aged super well.

899
00:35:19,148 --> 00:35:19,197
Just

900
00:35:19,198 --> 00:35:20,548
Matt: shows you how far we've come.

901
00:35:20,608 --> 00:35:23,077
But yeah, I, I just had someone, uh, actually one of our

902
00:35:23,078 --> 00:35:25,537
data techs this morning was messaging me asking me, "How do I

903
00:35:25,537 --> 00:35:28,177
learn about networking, Matt?" That's exactly what I sent him.

904
00:35:28,678 --> 00:35:28,698
Yeah.

905
00:35:28,698 --> 00:35:29,567
"Go get this on Amazon."

906
00:35:29,737 --> 00:35:31,687
Corey: I have no idea if it's still up, and I'll check it.

907
00:35:31,708 --> 00:35:33,988
If not, my apologies, but there was a great site, WarriorsofTheDotNet.

908
00:35:35,138 --> 00:35:35,908
Mm-hmm.

909
00:35:36,188 --> 00:35:39,637
Was always a video that explained in a like five or six

910
00:35:39,638 --> 00:35:42,668
minute cartoon approach of how packet switching routed

911
00:35:42,668 --> 00:35:45,367
networks work in an extraordinarily accessible way.

912
00:35:45,368 --> 00:35:46,628
I hope that's still out there.

913
00:35:46,628 --> 00:35:46,647
Yeah.

914
00:35:46,647 --> 00:35:48,927
That was always a fantastic primer.

915
00:35:49,317 --> 00:35:51,347
Like, yeah, some of their terminology choices you can

916
00:35:51,348 --> 00:35:53,607
argue with, and like, well, there's a lot more to it there.

917
00:35:53,787 --> 00:35:55,898
Yeah, but you're going from zero to one, and

918
00:35:55,927 --> 00:35:58,897
you're giving people a chance to see what's next.

919
00:35:58,897 --> 00:35:58,977
Yes.

920
00:35:58,987 --> 00:36:00,787
And, huh, that's curious.

921
00:36:00,817 --> 00:36:01,727
I wanna dig deeper.

922
00:36:01,758 --> 00:36:05,168
The thing that actually tipped me over the edge was subnet masks.

923
00:36:05,187 --> 00:36:08,187
Okay, it's this weird number string, and all I know is that when I get it

924
00:36:08,187 --> 00:36:12,047
wrong, some things work and other things don't, and I don't understand it.

925
00:36:12,287 --> 00:36:14,148
Maybe it's time I stop hand-waving over it.

926
00:36:14,287 --> 00:36:14,587
Matt: Yes.

927
00:36:14,647 --> 00:36:17,127
Corey: And for my sins, I learned how networks work.

928
00:36:17,138 --> 00:36:17,158
Yeah.

929
00:36:17,158 --> 00:36:20,177
'Cause I didn't know how networks work, and I didn't believe it was possible.

930
00:36:20,477 --> 00:36:22,328
And now that I know better, I do not believe that

931
00:36:22,328 --> 00:36:24,138
networks work, and I'm astounded that it's possible.

932
00:36:24,248 --> 00:36:26,948
It feels like it works in spite of itself, but clearly it does.

933
00:36:26,977 --> 00:36:26,997
It

934
00:36:27,587 --> 00:36:29,957
Matt: is tremendously complicated, but it's, uh, it gives us a

935
00:36:29,958 --> 00:36:32,198
lot of pleasure to, to make these things simpler for everyone.

936
00:36:32,677 --> 00:36:33,658
Corey: Thank you so much.

937
00:36:33,897 --> 00:36:37,758
Matt Rader, VP of Global Networking at AWS.

938
00:36:38,177 --> 00:36:39,188
I'm Corey Quinn.

939
00:36:39,397 --> 00:36:39,938
Stick around.