1
00:00:03,360 --> 00:00:06,320
Ryan Sean Adams:
Dwarkesh Patel, we are big fans. It's an honor to have you.

2
00:00:07,280 --> 00:00:08,200
Dwarkesh:
Thank you so much for having me on.

3
00:00:08,840 --> 00:00:15,680
Ryan Sean Adams:
Okay, so you have a book out. It's called The Scaling Era, an oral history of AI from 2019 to 2025.

4
00:00:16,120 --> 00:00:21,580
Ryan Sean Adams:
These are some key dates here. This is really a story of how AI emerged.

5
00:00:21,940 --> 00:00:26,060
Ryan Sean Adams:
And it seemed to have exploded on people's radar over the past five years.

6
00:00:26,340 --> 00:00:29,660
Ryan Sean Adams:
And And everyone in the world, it feels like, is trying to figure out what just

7
00:00:29,660 --> 00:00:31,880
Ryan Sean Adams:
happened and what is about to happen.

8
00:00:32,040 --> 00:00:36,120
Ryan Sean Adams:
And I feel like for this story, we should start at the beginning, as your book does.

9
00:00:36,580 --> 00:00:41,780
Ryan Sean Adams:
What is the scaling era of AI and when abouts did it start? What were the key milestones?

10
00:00:42,360 --> 00:00:47,080
Dwarkesh:
So I think the undertold story about everybody's, of course,

11
00:00:47,300 --> 00:00:48,500
Dwarkesh:
been hearing more and more about AI.

12
00:00:48,500 --> 00:00:53,200
Dwarkesh:
The under-told story is that the big contributor to these AI models getting

13
00:00:53,200 --> 00:00:58,020
Dwarkesh:
better over time has been the fact that we are throwing exponentially more compute

14
00:00:58,020 --> 00:01:00,000
Dwarkesh:
into trading frontier systems every year.

15
00:01:00,140 --> 00:01:04,200
Dwarkesh:
So by some estimates, we spend 4x every single year over the last decade trading

16
00:01:04,200 --> 00:01:06,360
Dwarkesh:
the frontier system than the one before it.

17
00:01:06,680 --> 00:01:10,720
Dwarkesh:
And that just means that we're spending hundreds of thousands of times more

18
00:01:10,720 --> 00:01:14,900
Dwarkesh:
compute than the systems of the early 2010s.

19
00:01:15,040 --> 00:01:17,480
Dwarkesh:
Of course, we've also had algorithmic breakthroughs in the meantime.

20
00:01:17,480 --> 00:01:19,380
Dwarkesh:
2018, we had the Transformer.

21
00:01:19,800 --> 00:01:23,700
Dwarkesh:
Since then, obviously, many companies have made small improvements here and there.

22
00:01:23,880 --> 00:01:28,940
Dwarkesh:
But the overwhelming fact that we're spending already hundreds of billions of

23
00:01:28,940 --> 00:01:31,060
Dwarkesh:
dollars in building up the infrastructure,

24
00:01:31,500 --> 00:01:36,980
Dwarkesh:
the data centers, the chips for these models, and this picture is only going

25
00:01:36,980 --> 00:01:38,480
Dwarkesh:
to intensify if this exponential keeps going,

26
00:01:38,860 --> 00:01:45,000
Dwarkesh:
4x a year, over the next two years, is something that is on the minds of the

27
00:01:45,000 --> 00:01:49,700
Dwarkesh:
CFOs of the big hyperscalers and the people planning the expenditures and training going forward,

28
00:01:49,880 --> 00:01:54,100
Dwarkesh:
but is not as common in the conversation around where AI is headed.

29
00:01:54,540 --> 00:01:57,340
Ryan Sean Adams:
So what do you feel like people should know about this?

30
00:01:57,460 --> 00:02:02,720
Ryan Sean Adams:
Like what is the scaling era? There have been other eras maybe of AI or compute,

31
00:02:02,860 --> 00:02:04,320
Ryan Sean Adams:
but what's special about the scaling era?

32
00:02:04,540 --> 00:02:08,920
Dwarkesh:
People started noticing. Well, first of all, in 2012, there's this,

33
00:02:09,880 --> 00:02:15,920
Dwarkesh:
Ilya Seskaver and others started using neural networks in order to categorize images.

34
00:02:16,120 --> 00:02:19,380
Dwarkesh:
And just noticing that instead of doing something hand-coded,

35
00:02:19,680 --> 00:02:24,260
Dwarkesh:
you can get a lot of juice out of just neural networks, black boxes.

36
00:02:24,260 --> 00:02:27,880
Dwarkesh:
You just train them to identify what thing is like what.

37
00:02:28,420 --> 00:02:31,060
Dwarkesh:
And then people started playing around these neural networks more,

38
00:02:31,260 --> 00:02:32,920
Dwarkesh:
using them for different kinds of applications.

39
00:02:33,640 --> 00:02:39,000
Dwarkesh:
And then the question became, we're noticing that these models get better if

40
00:02:39,000 --> 00:02:41,420
Dwarkesh:
you throw more data at them and you throw more compute at them.

41
00:02:41,420 --> 00:02:46,120
Dwarkesh:
How can we shove as much compute into these models as possible?

42
00:02:47,080 --> 00:02:51,680
Dwarkesh:
And the solution ended up being obviously internet text. So you need an architecture

43
00:02:51,680 --> 00:02:55,280
Dwarkesh:
which is amenable to the trillions of tokens that have been written over the

44
00:02:55,280 --> 00:02:57,480
Dwarkesh:
last few decades and put up on the internet.

45
00:02:57,680 --> 00:03:01,660
Dwarkesh:
And we had this happy coincidence of the kinds of architectures that are amenable

46
00:03:01,660 --> 00:03:04,620
Dwarkesh:
to this kind of training with the GPUs that were originally made for gaming.

47
00:03:05,140 --> 00:03:12,440
Dwarkesh:
We've had decades of internet text being compiled and Ilias actually called it the fossil fuel of AI.

48
00:03:12,440 --> 00:03:17,180
Dwarkesh:
It's like this reservoir that we can call upon to train these minds,

49
00:03:17,360 --> 00:03:20,820
Dwarkesh:
which are like, you know, they're fitting the mold of human thought because

50
00:03:20,820 --> 00:03:22,980
Dwarkesh:
they're trading on trillions of tokens of human thought.

51
00:03:23,680 --> 00:03:27,700
Dwarkesh:
And so then it's just been a question of making these models bigger,

52
00:03:27,700 --> 00:03:33,860
Dwarkesh:
of using this data that we're getting from internet techs to further keep training them.

53
00:03:33,860 --> 00:03:38,480
Dwarkesh:
And over the last year, as you know, the last six months, the new paradigm has

54
00:03:38,480 --> 00:03:41,440
Dwarkesh:
been not only are we going to pre-train on all this internet text,

55
00:03:41,600 --> 00:03:45,120
Dwarkesh:
we're going to see if we can have them solve math puzzles,

56
00:03:45,440 --> 00:03:49,960
Dwarkesh:
coding puzzles, and through this, give them reasoning capabilities.

57
00:03:50,520 --> 00:03:55,200
Dwarkesh:
The kind of thing, by the way, I mean, I have some skepticism around AGI just

58
00:03:55,200 --> 00:03:56,620
Dwarkesh:
around the corner, which we'll get into.

59
00:03:56,980 --> 00:04:00,480
Dwarkesh:
But just the fact that we now have machines which can like reason,

60
00:04:00,780 --> 00:04:04,320
Dwarkesh:
like, you know, you can like ask a question to a machine and it'll go away for a long time.

61
00:04:04,460 --> 00:04:06,720
Dwarkesh:
It'll like think about it and then like it'll come back to you with a smart answer.

62
00:04:06,980 --> 00:04:10,760
Dwarkesh:
And we just sort of take it for granted. But obviously, we also know that they're

63
00:04:10,760 --> 00:04:12,620
Dwarkesh:
extremely good at coding, especially.

64
00:04:12,960 --> 00:04:15,400
Dwarkesh:
I don't know if you actually got a chance to play around with Cloud Code or

65
00:04:15,400 --> 00:04:20,920
Dwarkesh:
Cursor or something. But it's a wild experience to design, explain at a high

66
00:04:20,920 --> 00:04:22,440
Dwarkesh:
level, I want an application to does X.

67
00:04:22,820 --> 00:04:27,720
Dwarkesh:
15 minutes later, there's like 10 files of code and the application is built.

68
00:04:28,470 --> 00:04:29,390
Josh Kale:
That's where we stand.

69
00:04:29,730 --> 00:04:32,710
Dwarkesh:
I have takes on how much this can continue. The other important dynamic,

70
00:04:33,030 --> 00:04:36,850
Dwarkesh:
I'll add my monologue here, but the other important dynamic is that if we're

71
00:04:36,850 --> 00:04:41,130
Dwarkesh:
going to be living in the scaling era, you can't continue exponentials forever,

72
00:04:41,690 --> 00:04:43,950
Dwarkesh:
and certainly not exponentials that are 4x a year forever.

73
00:04:44,590 --> 00:04:50,710
Dwarkesh:
And so right now, we're approaching a point where within by 2028,

74
00:04:50,990 --> 00:04:57,230
Dwarkesh:
at most by 2030, we will literally run out of the energy we need to keep trading

75
00:04:57,230 --> 00:04:58,130
Dwarkesh:
these frontier systems,

76
00:04:58,550 --> 00:05:02,730
Dwarkesh:
the capacity at the leading edge nodes, which manufacture the chips that go

77
00:05:02,730 --> 00:05:07,750
Dwarkesh:
into the dyes, which go into these GPUs, even the raw fraction of GDP that will

78
00:05:07,750 --> 00:05:09,370
Dwarkesh:
have to use to train frontier systems.

79
00:05:09,750 --> 00:05:12,990
Dwarkesh:
So we have a couple more years left of the scaling era. And the big question

80
00:05:12,990 --> 00:05:15,210
Dwarkesh:
is, will we get to AGI before then?

81
00:05:15,590 --> 00:05:18,030
Ryan Sean Adams:
I mean, that's kind of a key insight of your book that like,

82
00:05:18,170 --> 00:05:19,790
Ryan Sean Adams:
we're in the middle of the scaling era.

83
00:05:19,930 --> 00:05:23,070
Ryan Sean Adams:
I guess we're like, you know, six years in or so. And we're not quite sure.

84
00:05:23,210 --> 00:05:26,950
Ryan Sean Adams:
It's like, like the protagonist in the middle of the story, We don't know exactly

85
00:05:26,950 --> 00:05:28,710
Ryan Sean Adams:
which way things are going to go.

86
00:05:28,830 --> 00:05:36,750
Ryan Sean Adams:
But I want you to maybe, Dworkesh, help folks get an intuition for why scaling in this way even works.

87
00:05:36,870 --> 00:05:41,430
Ryan Sean Adams:
Because I'll tell you, for me and for most people, our experience with these

88
00:05:41,430 --> 00:05:46,910
Ryan Sean Adams:
revolutionary AI models probably started in 2022 with ChatGPT3 and then ChatGPT4

89
00:05:46,910 --> 00:05:49,390
Ryan Sean Adams:
and seeing all the progress, all these AI models.

90
00:05:49,390 --> 00:05:55,890
Ryan Sean Adams:
And it just seems really unintuitive that if you take a certain amount of compute

91
00:05:55,890 --> 00:06:01,570
Ryan Sean Adams:
and you take a certain amount of data, out pops AI, out pops intelligence.

92
00:06:01,870 --> 00:06:05,730
Ryan Sean Adams:
Could you help us get an intuition for this magic?

93
00:06:05,890 --> 00:06:11,950
Ryan Sean Adams:
How does the scaling law even work? Compute plus data equals intelligence? Is that really all it is?

94
00:06:12,070 --> 00:06:16,630
Dwarkesh:
To be honest, I've asked so many AI researchers this exact question on my podcast.

95
00:06:17,150 --> 00:06:20,330
Dwarkesh:
And I could tell you some potential theories of why it might work.

96
00:06:20,670 --> 00:06:22,010
Dwarkesh:
I don't think we understand.

97
00:06:24,750 --> 00:06:27,030
Dwarkesh:
You know what? I'll just say that. I don't think we understand.

98
00:06:27,190 --> 00:06:30,310
Ryan Sean Adams:
We don't understand how this works. We know it works, but we don't understand

99
00:06:30,310 --> 00:06:35,710
Dwarkesh:
How it works. We have evidence from actually, of all things,

100
00:06:36,130 --> 00:06:41,050
Dwarkesh:
primatology of what could be going on here, or at least like why similar patterns

101
00:06:41,050 --> 00:06:42,630
Dwarkesh:
in other parts of the world.

102
00:06:42,810 --> 00:06:46,510
Dwarkesh:
So what I found really interesting, There's this research by this researcher,

103
00:06:46,650 --> 00:06:47,910
Dwarkesh:
Susanna Herculana Huzel,

104
00:06:48,150 --> 00:06:56,290
Dwarkesh:
which shows that if you look at how the number of neurons in the brain of a rat,

105
00:06:56,450 --> 00:07:01,430
Dwarkesh:
different kinds of rat species increases, as the weight of their brains increase

106
00:07:01,430 --> 00:07:04,550
Dwarkesh:
from species to species, there's this very sublinear pattern.

107
00:07:04,750 --> 00:07:09,650
Dwarkesh:
So if their brain size doubles, the neuron count will not double between different rat species.

108
00:07:09,910 --> 00:07:13,570
Dwarkesh:
And there's other animals where there's other kinds of...

109
00:07:14,420 --> 00:07:18,700
Dwarkesh:
Families of species for which this is true. The two interesting exceptions to

110
00:07:18,700 --> 00:07:22,480
Dwarkesh:
this rule, where there is actually a linear increase in neuron count and brain

111
00:07:22,480 --> 00:07:25,740
Dwarkesh:
size, is one, certain kinds of birds.

112
00:07:26,080 --> 00:07:32,260
Dwarkesh:
So, you know, birds are actually very smart, given the size of their brains, and primates.

113
00:07:32,500 --> 00:07:38,540
Dwarkesh:
So the theory for what happened with humans is that we unlocked an architecture that was very scalable.

114
00:07:38,780 --> 00:07:41,480
Dwarkesh:
So the way people talk about transformers being more scalable and then LSTMs,

115
00:07:41,680 --> 00:07:43,240
Dwarkesh:
the thing that preceded them in 2018.

116
00:07:43,640 --> 00:07:45,520
Dwarkesh:
We unlocked this architecture as it's very scalable.

117
00:07:46,160 --> 00:07:50,280
Dwarkesh:
And then we were in an evolutionary niche millions of years ago,

118
00:07:50,380 --> 00:07:53,080
Dwarkesh:
which rewarded marginal increases in intelligence.

119
00:07:53,380 --> 00:07:56,060
Dwarkesh:
If you get slightly smarter, yes, the brain costs more energy,

120
00:07:56,200 --> 00:07:59,540
Dwarkesh:
but you can save energy in terms of like not having to, you can cook,

121
00:07:59,660 --> 00:08:02,400
Dwarkesh:
you can cook food so you don't have to spend much more on digestion.

122
00:08:02,940 --> 00:08:06,040
Dwarkesh:
You can find a game, you can find different ways of foraging.

123
00:08:06,920 --> 00:08:11,720
Dwarkesh:
Birds were not able to find this evolutionary niche, which rewarded the incremental

124
00:08:11,720 --> 00:08:15,720
Dwarkesh:
increases in intelligence because if your brain gets too heavy as a bird, you're not going to fly.

125
00:08:17,360 --> 00:08:21,740
Dwarkesh:
So it was this happy coincidence of these two things. Now, why is it the case

126
00:08:21,740 --> 00:08:27,660
Dwarkesh:
that the fact that our brains could get bigger resulted in us becoming as smart

127
00:08:27,660 --> 00:08:28,940
Dwarkesh:
as we are? We still don't know.

128
00:08:29,080 --> 00:08:31,820
Dwarkesh:
And there's many different dissimilarities between AIs and humans.

129
00:08:32,320 --> 00:08:35,880
Dwarkesh:
While our brains are quite big, we don't need to be trained.

130
00:08:35,960 --> 00:08:41,840
Dwarkesh:
You know, a human from the age they're zero to 18 is not seeing within an order

131
00:08:41,840 --> 00:08:44,580
Dwarkesh:
of magnitude of the amount of information these LLMs are trained on.

132
00:08:44,700 --> 00:08:46,400
Dwarkesh:
So LLMs are extremely data inefficient.

133
00:08:46,560 --> 00:08:52,900
Dwarkesh:
They need a lot more data, but the pattern of scaling, I think we see in many different places.

134
00:08:53,160 --> 00:08:57,060
Ryan Sean Adams:
So is that a fair kind of analog? This analog has always made sense to me.

135
00:08:57,220 --> 00:08:59,620
Ryan Sean Adams:
It's just like transformers are like neurons.

136
00:09:00,400 --> 00:09:03,140
Ryan Sean Adams:
You know, AI models are sort of like the human brain.

137
00:09:04,460 --> 00:09:09,660
Ryan Sean Adams:
Evolutionary pressures are like gradient descent, reward algorithms and out

138
00:09:09,660 --> 00:09:12,780
Ryan Sean Adams:
pops human intelligence. We don't really understand that.

139
00:09:13,020 --> 00:09:17,660
Ryan Sean Adams:
We also don't understand AI intelligence, but it's basically the same principle at work.

140
00:09:17,800 --> 00:09:23,400
Dwarkesh:
I think it's a super fascinating, but also very thorny question because is gradient

141
00:09:23,400 --> 00:09:24,660
Dwarkesh:
intelligence like evolution?

142
00:09:24,900 --> 00:09:29,660
Dwarkesh:
Well, yes, in one sense. But also when we do gradient descent on these models,

143
00:09:29,980 --> 00:09:35,740
Dwarkesh:
we start off with the weights and then we're, you know, it's like learning how

144
00:09:35,740 --> 00:09:38,120
Dwarkesh:
does chemistry work, how does coding work, how does math work.

145
00:09:38,940 --> 00:09:42,520
Dwarkesh:
And that's actually more similar to lifetime learning, which is to say that,

146
00:09:42,600 --> 00:09:46,400
Dwarkesh:
like, by the time you're already born to the time you turn 18 or 25,

147
00:09:46,840 --> 00:09:48,700
Dwarkesh:
the things you learn, and that's not evolution.

148
00:09:48,980 --> 00:09:53,720
Dwarkesh:
Evolution designed the system or the brain by which you can do that learning,

149
00:09:53,720 --> 00:09:57,740
Dwarkesh:
but the lifetime learning itself is not evolution. And so there's also this

150
00:09:57,740 --> 00:10:00,920
Dwarkesh:
interesting question of, yeah, is training more like evolution?

151
00:10:01,480 --> 00:10:04,000
Dwarkesh:
In which case, actually, we might be very far from AGI because the amount of

152
00:10:04,000 --> 00:10:07,360
Dwarkesh:
compute that's been spent over the course of evolution to discover the human

153
00:10:07,360 --> 00:10:11,260
Dwarkesh:
brain, you know, could be like 10 to the 40 flops. There's been estimates, you know, whatever.

154
00:10:12,340 --> 00:10:15,420
Dwarkesh:
I'm sure it will bore you to discover, talk about how these estimates are derived,

155
00:10:15,580 --> 00:10:19,720
Dwarkesh:
but just like how much versus is it like a single lifetime,

156
00:10:20,080 --> 00:10:23,480
Dwarkesh:
like going from the age of zero to the age of 18, which is closer to,

157
00:10:23,680 --> 00:10:26,840
Dwarkesh:
I think, 10 to the 24 flops, which is actually less than compute than we use

158
00:10:26,840 --> 00:10:27,900
Dwarkesh:
to train frontier systems.

159
00:10:28,580 --> 00:10:33,540
Dwarkesh:
All right, anyways, we'll get back to more relevant questions.

160
00:10:33,920 --> 00:10:36,640
Ryan Sean Adams:
Well, here's kind of a big picture question as well.

161
00:10:36,800 --> 00:10:41,260
Ryan Sean Adams:
It's like I'm constantly fascinated with the metaphysical types of discussions

162
00:10:41,260 --> 00:10:43,560
Ryan Sean Adams:
that some AI researchers kind of take.

163
00:10:43,700 --> 00:10:48,060
Ryan Sean Adams:
Like a lot of AI researchers will talk in terms of when they describe what they're

164
00:10:48,060 --> 00:10:49,140
Ryan Sean Adams:
making, we're making God.

165
00:10:49,620 --> 00:10:53,760
Ryan Sean Adams:
Like why do they say things like that? What is this talk of like making God?

166
00:10:53,860 --> 00:10:57,620
Ryan Sean Adams:
What does that mean? Is it just the idea that scaling laws don't cease?

167
00:10:57,860 --> 00:11:03,260
Ryan Sean Adams:
And if we can, you know, scale intelligence to AGI, then there's no reason we

168
00:11:03,260 --> 00:11:07,560
Ryan Sean Adams:
can't scale far beyond that and create some sort of a godlike entity.

169
00:11:07,900 --> 00:11:11,940
Ryan Sean Adams:
And essentially, that's what the quest is. We're making artificial superintelligence.

170
00:11:12,100 --> 00:11:13,860
Ryan Sean Adams:
We're making a god. We're making god.

171
00:11:13,860 --> 00:11:19,440
Dwarkesh:
I think people focus too much on when they, I think this God discussion focuses

172
00:11:19,440 --> 00:11:26,020
Dwarkesh:
too much on the hypothetical intelligence of a single copy of an AI.

173
00:11:27,150 --> 00:11:32,410
Dwarkesh:
I do believe in the notion of a super intelligence, which is not just functionally,

174
00:11:32,670 --> 00:11:36,270
Dwarkesh:
which is not just like, oh, it knows a lot of things, but is actually qualitatively

175
00:11:36,270 --> 00:11:38,670
Dwarkesh:
different than human society.

176
00:11:38,970 --> 00:11:42,590
Dwarkesh:
But the reason is not because I think it's so powerful that any one individual

177
00:11:42,590 --> 00:11:48,210
Dwarkesh:
copy of AI will be as smart, but because of the collective advantages that AIs

178
00:11:48,210 --> 00:11:51,190
Dwarkesh:
will have, which have nothing to do with their raw intelligence,

179
00:11:51,870 --> 00:11:55,390
Dwarkesh:
but rather the fact that these models will be digital or they already are digital,

180
00:11:55,510 --> 00:11:57,050
Dwarkesh:
but eventually they'll be as smart as humans at least.

181
00:11:58,050 --> 00:12:02,690
Dwarkesh:
But unlike humans, because of our biological constraints, these models can be copied.

182
00:12:03,210 --> 00:12:06,390
Dwarkesh:
If there's a model that has learned a lot about a specific domain,

183
00:12:06,670 --> 00:12:08,230
Dwarkesh:
you can make infinite copies of it.

184
00:12:08,370 --> 00:12:12,830
Dwarkesh:
And now you have an infinite copies of Jeff Dean or Ilya Satskova or Elon Musk

185
00:12:12,830 --> 00:12:15,030
Dwarkesh:
or any skilled person you can think of.

186
00:12:15,350 --> 00:12:21,130
Dwarkesh:
They can be merged. So the knowledge that each copy is learning can be amalgamated

187
00:12:21,130 --> 00:12:24,390
Dwarkesh:
back into the model and then back to all the copies.

188
00:12:24,730 --> 00:12:28,270
Dwarkesh:
They can be distilled. They can run at superhuman speeds.

189
00:12:29,250 --> 00:12:32,690
Dwarkesh:
These collective advantages, also they can communicate in latent space.

190
00:12:32,870 --> 00:12:33,450
Dwarkesh:
These collective advantages.

191
00:12:33,710 --> 00:12:36,130
Ryan Sean Adams:
They're immortal. I mean, you know, as an example.

192
00:12:36,570 --> 00:12:40,470
Dwarkesh:
Yes, exactly. No, I mean, that's actually, tell me if I'm rabbit holing too

193
00:12:40,470 --> 00:12:44,990
Dwarkesh:
much, but like one really interesting question will come about is how do we prosecute AIs?

194
00:12:45,170 --> 00:12:50,470
Dwarkesh:
Because the way we prosecute humans is that we will throw you in jail if you commit a crime.

195
00:12:50,790 --> 00:12:56,610
Dwarkesh:
But if there's trillions of copies or thousands of copies of an AI model,

196
00:12:57,030 --> 00:13:01,290
Dwarkesh:
if a copy of an AI model, if an instance of an AI model does something bad, what do you do?

197
00:13:01,410 --> 00:13:04,570
Dwarkesh:
Does the whole model have to get, and how do you even punish a model,

198
00:13:04,690 --> 00:13:07,690
Dwarkesh:
right? Like, does it care about its weights being squandered?

199
00:13:09,730 --> 00:13:14,150
Dwarkesh:
Yeah, there's all kinds of questions that arise because of the nature of what AIs are.

200
00:13:14,590 --> 00:13:16,170
Dwarkesh Patel:
And also who is liable for that, right?

201
00:13:16,270 --> 00:13:17,410
Dwarkesh:
Like, is it the toolmaker?

202
00:13:17,590 --> 00:13:19,770
Dwarkesh Patel:
Is it the person using the tool? Who is responsible for these things?

203
00:13:20,070 --> 00:13:23,270
Dwarkesh Patel:
There's one topic that I do want to come to here about scaling laws,

204
00:13:23,270 --> 00:13:27,050
Dwarkesh Patel:
At what time did we realize that scaling laws were going to work?

205
00:13:27,270 --> 00:13:31,390
Dwarkesh Patel:
Because there were a lot of theses early in the days, early 2000s about AI,

206
00:13:31,710 --> 00:13:33,170
Dwarkesh Patel:
how we were going to build better models.

207
00:13:33,530 --> 00:13:36,350
Dwarkesh Patel:
Eventually, we got to the transformer. But at what point did researchers and

208
00:13:36,350 --> 00:13:38,930
Dwarkesh Patel:
engineers start to realize that, hey, this is the correct idea.

209
00:13:39,050 --> 00:13:42,610
Dwarkesh Patel:
We should start throwing lots of money and resources towards this versus other

210
00:13:42,610 --> 00:13:45,570
Dwarkesh Patel:
ideas that were just kind of theoretical research ideas, but never really took off.

211
00:13:45,950 --> 00:13:49,210
Dwarkesh Patel:
We kind of saw this with GPT two to three, where there's this huge improvement.

212
00:13:49,350 --> 00:13:49,630
Dwarkesh:
A lot of.

213
00:13:49,630 --> 00:13:53,150
Dwarkesh Patel:
Resources went into it. Was there a specific moment in time or a specific breakthrough

214
00:13:53,150 --> 00:13:55,490
Dwarkesh Patel:
that led to the start of these scaling laws?

215
00:13:55,610 --> 00:14:00,530
Dwarkesh:
I think it's been a slow process of more and more people appreciating this nature

216
00:14:00,530 --> 00:14:04,730
Dwarkesh:
of the overwhelming role of compute in driving forward progress.

217
00:14:05,970 --> 00:14:13,550
Dwarkesh:
In 2018, I believe, Dario Amadei wrote a memo that was secret while he was at

218
00:14:13,550 --> 00:14:15,890
Dwarkesh:
OpenAI. Now he's the CEO of Anthropic.

219
00:14:16,010 --> 00:14:19,510
Dwarkesh:
But while he's at OpenAI, he's subsequently revealed on my podcast that he wrote

220
00:14:19,510 --> 00:14:24,690
Dwarkesh:
this memo where the title of the memo was called Big Blob of Compute.

221
00:14:25,630 --> 00:14:28,650
Dwarkesh:
And it says basically what you expect it to say, which is that like,

222
00:14:29,030 --> 00:14:31,890
Dwarkesh:
yes, there's ways you can mess up the process of training. You have the wrong

223
00:14:31,890 --> 00:14:33,470
Dwarkesh:
kinds of data or initializations.

224
00:14:33,750 --> 00:14:36,670
Dwarkesh:
But fundamentally, AGI is just a big blob of compute.

225
00:14:37,130 --> 00:14:41,510
Dwarkesh:
And then we've gotten over the subsequent years, there was more empirical evidence.

226
00:14:41,690 --> 00:14:44,010
Dwarkesh:
So a big update, I think it was 2021.

227
00:14:44,410 --> 00:14:46,390
Dwarkesh:
Correct me. Somebody definitely will correct me in the comments.

228
00:14:46,490 --> 00:14:48,030
Dwarkesh:
I'm wrong. There were these,

229
00:14:48,170 --> 00:14:54,990
Dwarkesh:
there's been multiple papers of these scaling laws where you can show that the

230
00:14:54,990 --> 00:15:01,670
Dwarkesh:
loss of the model on the objective of predicting the next token goes down very predictably,

231
00:15:01,870 --> 00:15:07,090
Dwarkesh:
almost to like multiple decimal places of correctness based on how much more

232
00:15:07,090 --> 00:15:08,210
Dwarkesh:
compute you throw in these models.

233
00:15:08,350 --> 00:15:13,850
Dwarkesh:
And the compute itself is a function of the amount of data you use and how big

234
00:15:13,850 --> 00:15:15,250
Dwarkesh:
the model is, how many parameters it has.

235
00:15:15,250 --> 00:15:19,250
Dwarkesh:
And so that was an incredibly strong evidence back in the day,

236
00:15:19,530 --> 00:15:21,930
Dwarkesh:
a couple of years ago, because then you could say, well, OK,

237
00:15:22,050 --> 00:15:28,930
Dwarkesh:
if it really has this incredibly low loss of predicting the next token in all

238
00:15:28,930 --> 00:15:33,930
Dwarkesh:
human output, including scientific papers, including GitHub repositories.

239
00:15:34,490 --> 00:15:40,410
Dwarkesh:
Then doesn't it mean it has actually had to learn coding and science and all

240
00:15:40,410 --> 00:15:43,430
Dwarkesh:
these skills in order to make those predictions, which actually ended up being true.

241
00:15:43,430 --> 00:15:46,210
Dwarkesh:
And it was it was something people, you know, we take it for granted now,

242
00:15:46,310 --> 00:15:50,510
Dwarkesh:
but it actually even as of a year or two ago, people were really even denying that premise.

243
00:15:50,670 --> 00:15:53,630
Dwarkesh:
But some people a couple of years ago just like thought about it and like,

244
00:15:53,770 --> 00:15:55,610
Dwarkesh:
yeah, actually, that would mean that it's learned the skills.

245
00:15:55,810 --> 00:15:59,490
Dwarkesh:
And that's crazy that we just have this strong empirical pattern that tells

246
00:15:59,490 --> 00:16:01,650
Dwarkesh:
us exactly what we need to do in order to learn these skills.

247
00:16:02,030 --> 00:16:05,510
Dwarkesh Patel:
And it creates this weird perception, right, where like very early on and so

248
00:16:05,510 --> 00:16:07,530
Dwarkesh Patel:
to this day, it really is just a token predictor, right?

249
00:16:07,610 --> 00:16:10,430
Dwarkesh Patel:
Like we're just predicting the next word in the sentence. But somewhere along

250
00:16:10,430 --> 00:16:13,410
Dwarkesh Patel:
the lines, it actually creates this perception of intelligence.

251
00:16:14,050 --> 00:16:18,450
Dwarkesh Patel:
So I guess we covered the early historical context. I kind of want to bring

252
00:16:18,450 --> 00:16:21,810
Dwarkesh Patel:
the listeners up to today, where we are currently, where the scaling laws have

253
00:16:21,810 --> 00:16:23,170
Dwarkesh Patel:
brought us in the year 2025.

254
00:16:23,650 --> 00:16:28,290
Dwarkesh Patel:
So can you kind of outline where we've gotten to from early days of GPTs to

255
00:16:28,290 --> 00:16:32,010
Dwarkesh Patel:
now we have GPT-4, we have Gemini Ultra, we have Club, which you mentioned earlier.

256
00:16:32,010 --> 00:16:33,650
Dwarkesh Patel:
We had the breakthrough of reasoning.

257
00:16:33,830 --> 00:16:36,150
Dwarkesh Patel:
So what can leading frontier models do today?

258
00:16:36,150 --> 00:16:40,850
Dwarkesh:
So there's what they can do. And then there's the question of what methods seem to be working.

259
00:16:41,830 --> 00:16:46,090
Dwarkesh:
I guess we can start at what they seem to be able to do. They've shown to be

260
00:16:46,090 --> 00:16:51,010
Dwarkesh:
remarkably useful at coding and not just at answering direct questions about

261
00:16:51,010 --> 00:16:52,550
Dwarkesh:
how does this line of code work or something.

262
00:16:53,330 --> 00:16:57,110
Dwarkesh:
But genuinely just autonomously working for 30 minutes or an hour,

263
00:16:57,370 --> 00:17:01,370
Dwarkesh:
doing the task, it would take a front-end developer a whole day to do.

264
00:17:01,510 --> 00:17:04,270
Dwarkesh:
And you can just ask them at a high level, do this kind of thing,

265
00:17:04,330 --> 00:17:05,330
Dwarkesh:
and they can go ahead and do it.

266
00:17:05,530 --> 00:17:07,990
Dwarkesh:
Obviously, if you've played around with it, you know that they're extremely

267
00:17:07,990 --> 00:17:11,730
Dwarkesh:
useful assistants in terms of research, in terms of even therapists,

268
00:17:12,050 --> 00:17:13,410
Dwarkesh:
whatever other use cases.

269
00:17:13,950 --> 00:17:16,530
Dwarkesh:
On the question of what training methods seem to be working,

270
00:17:16,970 --> 00:17:19,890
Dwarkesh:
we do seem to be getting evidence that pre-training is plateauing,

271
00:17:19,890 --> 00:17:25,670
Dwarkesh:
which is to say that we had GPT 4.5, which was just following this old mold

272
00:17:25,670 --> 00:17:27,190
Dwarkesh:
of make the model bigger,

273
00:17:27,510 --> 00:17:30,290
Dwarkesh:
but it's fundamentally doing the same thing of next token prediction.

274
00:17:31,570 --> 00:17:36,090
Dwarkesh:
And apparently it didn't pass muster. The OpenAI had to deprecate it because

275
00:17:36,090 --> 00:17:39,070
Dwarkesh:
there's this dynamic where the bigger the model is, the more it costs not only

276
00:17:39,070 --> 00:17:41,090
Dwarkesh:
to train, but also to serve, right?

277
00:17:41,110 --> 00:17:43,970
Dwarkesh:
Because every time you serve a user, you're having to run the whole model,

278
00:17:44,470 --> 00:17:49,410
Dwarkesh:
which is going, so, but that doesn't be working is RL, which is this process

279
00:17:49,410 --> 00:17:52,190
Dwarkesh:
of, not just training them on existing tokens on the internet,

280
00:17:52,350 --> 00:17:55,650
Dwarkesh:
but having the model itself try to answer math and coding problems.

281
00:17:55,770 --> 00:17:57,950
Dwarkesh:
And finally, we got to the point where the model is smart enough to get it right

282
00:17:57,950 --> 00:18:01,330
Dwarkesh:
some of the time, and so you can give it some reward, and then it can saturate

283
00:18:01,330 --> 00:18:03,830
Dwarkesh:
these tough reasoning problems.

284
00:18:04,370 --> 00:18:08,010
Dwarkesh Patel:
And then what was the breakthrough with reasoning for the people who aren't familiar?

285
00:18:08,190 --> 00:18:11,250
Dwarkesh Patel:
What made reasoning so special that we hadn't discovered before?

286
00:18:11,370 --> 00:18:13,890
Dwarkesh Patel:
And what did that kind of unlock for models that we use today?

287
00:18:14,070 --> 00:18:18,830
Dwarkesh:
I'm honestly not sure. I mean, GBD-4 came out a little over two years ago,

288
00:18:19,430 --> 00:18:23,970
Dwarkesh:
and then it was after two years after GPT-4 came out that O-1 came out which

289
00:18:23,970 --> 00:18:27,250
Dwarkesh:
was the original reasoning breakthrough I think last November and,

290
00:18:28,030 --> 00:18:33,150
Dwarkesh:
And subsequently, a couple of months later, DeepSeq showed in their R1 paper.

291
00:18:33,350 --> 00:18:37,090
Dwarkesh:
So DeepSeq open source their research and they explained exactly how their algorithm worked.

292
00:18:37,490 --> 00:18:41,210
Dwarkesh:
And it wasn't that complicated. It was just like what you would expect,

293
00:18:41,390 --> 00:18:44,110
Dwarkesh:
which is get some math problems,

294
00:18:44,550 --> 00:18:48,150
Dwarkesh:
give for some initial problems, tell the model exactly what the reasoning trace

295
00:18:48,150 --> 00:18:51,270
Dwarkesh:
looks like, how you solve it, just like write it out and then have the model

296
00:18:51,270 --> 00:18:53,730
Dwarkesh:
like try to do it raw on the remaining problems.

297
00:18:54,070 --> 00:18:57,590
Dwarkesh:
Now, I know it sounds incredibly arrogant to say, well, it wasn't that complicated.

298
00:18:57,590 --> 00:18:58,550
Dwarkesh:
Why did it take you years?

299
00:18:58,970 --> 00:19:02,270
Dwarkesh:
I think there's an interesting insight there of even things which you think

300
00:19:02,270 --> 00:19:05,930
Dwarkesh:
will be simple in terms of high level description of how to solve the problem

301
00:19:05,930 --> 00:19:10,450
Dwarkesh:
end up taking longer in terms of haggling out the remaining engineering hurdles

302
00:19:10,450 --> 00:19:12,570
Dwarkesh:
than you might naively assume.

303
00:19:12,830 --> 00:19:17,690
Dwarkesh:
And that should update us on how long it will take us to go through the remaining

304
00:19:17,690 --> 00:19:19,550
Dwarkesh:
bottlenecks on the path to AGI.

305
00:19:19,730 --> 00:19:22,070
Dwarkesh:
Maybe that will be tougher than people imagine, especially the people who think

306
00:19:22,070 --> 00:19:23,110
Dwarkesh:
we're only two to three years away.

307
00:19:24,010 --> 00:19:27,530
Dwarkesh:
But all this to say, yeah, I'm not sure why it took so long after GPT-4 to get

308
00:19:27,530 --> 00:19:31,650
Dwarkesh:
a model trained on a similar level of capabilities that could then do reasoning.

309
00:19:31,970 --> 00:19:36,310
Dwarkesh Patel:
And in terms of those abilities, the first answer you had to what can it do was coding.

310
00:19:36,450 --> 00:19:40,110
Dwarkesh Patel:
And I hear that a lot of the time when I talk to a lot of people is that coding

311
00:19:40,110 --> 00:19:43,930
Dwarkesh Patel:
seems to be a really strong suit and a really huge unlock to using these models.

312
00:19:44,110 --> 00:19:47,970
Dwarkesh Patel:
And I'm curious, why coding over general intelligence?

313
00:19:48,110 --> 00:19:50,990
Dwarkesh Patel:
Is it because it's placed in a more confined box of parameters?

314
00:19:51,130 --> 00:19:54,970
Dwarkesh Patel:
I know in the early days, we had the AlphaGo and And we had the AIs playing

315
00:19:54,970 --> 00:19:58,330
Dwarkesh Patel:
chess and they exceed, they perform so well because they were kind of contained

316
00:19:58,330 --> 00:20:01,550
Dwarkesh Patel:
within this box of parameters that was a little less open-ended than general intelligence.

317
00:20:01,790 --> 00:20:06,250
Dwarkesh Patel:
Is that the reason why coding is kind of at the frontier right now of the ability of these models?

318
00:20:06,470 --> 00:20:12,110
Dwarkesh:
There's two different hypotheses. One is based around this idea called Moravac's paradox.

319
00:20:13,590 --> 00:20:16,650
Dwarkesh:
And this was an idea, by the way, one super interesting figure,

320
00:20:16,770 --> 00:20:18,090
Dwarkesh:
actually, I should have mentioned him earlier.

321
00:20:18,710 --> 00:20:21,990
Dwarkesh:
One super interesting figure in the history of scaling is Hans Moravac,

322
00:20:22,090 --> 00:20:28,910
Dwarkesh:
who I think in the 90s predicts that 2028 will be the year that we will get to AGI.

323
00:20:29,390 --> 00:20:32,310
Dwarkesh:
And the way he predicts this, which is like, you know, we'll see what happens,

324
00:20:32,430 --> 00:20:34,930
Dwarkesh:
but like not that far off the money as far as I'm concerned.

325
00:20:35,450 --> 00:20:41,450
Dwarkesh:
The way he predicts this is he just looks at the growth in computing power year

326
00:20:41,450 --> 00:20:46,910
Dwarkesh:
over year and then looks at how much compute he estimated the human brain to be to require.

327
00:20:47,230 --> 00:20:50,870
Dwarkesh:
And just like, OK, we'll have computers as powerful as the human brain by 2028.

328
00:20:51,820 --> 00:20:57,360
Dwarkesh:
Which is like at once a deceptively simple argument, but also ended up being

329
00:20:57,360 --> 00:21:00,440
Dwarkesh:
incredibly accurate and like worked, right?

330
00:21:01,200 --> 00:21:03,640
Dwarkesh:
I might add a fact drive it was 2028, but it was within that,

331
00:21:03,820 --> 00:21:07,220
Dwarkesh:
like within something you would consider a reasonable guess, given what we know now.

332
00:21:07,740 --> 00:21:12,600
Dwarkesh:
Sorry, anyway, so the Morrowind's paradox is this idea that computers seemed

333
00:21:12,600 --> 00:21:19,680
Dwarkesh:
in AI get better first at the skills which humans are the worst at.

334
00:21:19,680 --> 00:21:22,560
Dwarkesh:
Or at least there's a huge variation in the human repertoire.

335
00:21:22,760 --> 00:21:26,780
Dwarkesh:
So we think of coding as incredibly hard, right? We think this is like the top

336
00:21:26,780 --> 00:21:28,940
Dwarkesh:
1% of people will be excellent coders.

337
00:21:29,300 --> 00:21:32,380
Dwarkesh:
We also think of reasoning as very hard, right? So if you like read Aristotle,

338
00:21:32,880 --> 00:21:37,580
Dwarkesh:
he says, the thing which makes humans special, which distinguishes us from animals is reasoning.

339
00:21:38,500 --> 00:21:44,740
Dwarkesh:
And these models aren't that useful yet at almost anything. The one thing they can do is reasoning.

340
00:21:45,220 --> 00:21:51,180
Dwarkesh:
So how do we explain this pattern? And Moravec's answer is that evolution has

341
00:21:51,180 --> 00:21:55,520
Dwarkesh:
spent billions of years optimizing us to do things we take for granted.

342
00:21:56,040 --> 00:21:58,320
Dwarkesh:
Move around this room, right? I can pick up this can of Coke,

343
00:21:58,620 --> 00:21:59,780
Dwarkesh:
move it around, drink from it.

344
00:22:00,460 --> 00:22:03,680
Dwarkesh:
And that we can't even get robots to do at all yet.

345
00:22:04,400 --> 00:22:08,700
Dwarkesh:
And in fact, it's so ingrained in us by evolution that there's no human, or.

346
00:22:08,700 --> 00:22:10,420
Ryan Sean Adams:
At least humans who don't have

347
00:22:10,420 --> 00:22:13,920
Dwarkesh:
Disabilities will all be able to do this. And so we just take it for granted

348
00:22:13,920 --> 00:22:15,120
Dwarkesh:
that this is an easy thing to do.

349
00:22:15,280 --> 00:22:18,660
Dwarkesh:
But in fact, it's evidence of how long evolution has spent getting humans up to this point.

350
00:22:19,220 --> 00:22:27,320
Dwarkesh:
Whereas reasoning, logic, all of these skills have only been optimized by evolution

351
00:22:27,320 --> 00:22:30,220
Dwarkesh:
over the course of the last few million years.

352
00:22:30,440 --> 00:22:35,820
Dwarkesh:
So there's been a thousand fold less evolutionary pressure towards coding than

353
00:22:35,820 --> 00:22:37,720
Dwarkesh:
towards just basic locomotion.

354
00:22:38,820 --> 00:22:42,380
Dwarkesh:
And this has actually been very accurate in predicting what kinds of progress

355
00:22:42,380 --> 00:22:43,880
Dwarkesh:
we see even before we got deep learning, right?

356
00:22:43,960 --> 00:22:47,320
Dwarkesh:
Like in the 40s when we got our first computers, the first thing that we could

357
00:22:47,320 --> 00:22:51,940
Dwarkesh:
use them to do is long calculations for ballistic trajectories at the time for World War II.

358
00:22:52,600 --> 00:22:54,760
Dwarkesh:
Humans suck at long calculations by hand.

359
00:22:56,010 --> 00:23:00,610
Dwarkesh:
And anyways, so that's the explanation for coding, which seems hard for humans,

360
00:23:00,610 --> 00:23:02,210
Dwarkesh:
is the first thing that went to AIs.

361
00:23:02,530 --> 00:23:05,390
Dwarkesh:
Now, there's another theory, which is that this is actually totally wrong.

362
00:23:05,670 --> 00:23:10,990
Dwarkesh:
It has nothing to do with the seeming paradox of how long evolution has optimized

363
00:23:10,990 --> 00:23:15,030
Dwarkesh:
us for, and everything to do with the availability of data.

364
00:23:15,030 --> 00:23:23,130
Dwarkesh:
So we have GitHub, this repository of all of human code, at least all open source

365
00:23:23,130 --> 00:23:26,390
Dwarkesh:
code written in all these different languages, trillions and trillions of tokens.

366
00:23:26,930 --> 00:23:29,890
Dwarkesh:
We don't have an analogous thing for robotics. We don't have this pre-training

367
00:23:29,890 --> 00:23:34,230
Dwarkesh:
corpus. And that explains why code has made so much more progress than robotics.

368
00:23:34,870 --> 00:23:38,550
Ryan Sean Adams:
That's fascinating because if there's one thing that I could list that we'd

369
00:23:38,550 --> 00:23:44,390
Ryan Sean Adams:
want AI to be good at, probably coding software is number one on that list.

370
00:23:44,390 --> 00:23:49,050
Ryan Sean Adams:
Because if you have a Turing complete intelligence that can create Turing complete

371
00:23:49,050 --> 00:23:52,670
Ryan Sean Adams:
software, is there anything you can't create once you have that?

372
00:23:52,950 --> 00:23:58,450
Ryan Sean Adams:
Also, like the idea of Morvac's paradox, I guess that sort of implies a certain

373
00:23:58,450 --> 00:24:00,910
Ryan Sean Adams:
complementarianism with humanity.

374
00:24:01,130 --> 00:24:06,370
Ryan Sean Adams:
So if robots can do things that robots can do really well and can't do the things

375
00:24:06,370 --> 00:24:09,310
Ryan Sean Adams:
humans can do well, well, perhaps there's a place for us in this world.

376
00:24:09,470 --> 00:24:14,250
Ryan Sean Adams:
And that's fantastic news. It also maybe implies that humans have kind of scratched

377
00:24:14,250 --> 00:24:17,230
Ryan Sean Adams:
the surface on reasoning potential.

378
00:24:17,530 --> 00:24:21,370
Ryan Sean Adams:
I mean, if we've only had a couple of million years of evolution and we haven't

379
00:24:21,370 --> 00:24:25,410
Ryan Sean Adams:
had the data set to actually get really good at reasoning, it seems like there'd

380
00:24:25,410 --> 00:24:29,330
Ryan Sean Adams:
be a massive amount of upside, unexplored territory,

381
00:24:29,630 --> 00:24:33,050
Ryan Sean Adams:
like so much more intelligence that nature could actually

382
00:24:33,910 --> 00:24:35,510
Ryan Sean Adams:
contain inside of reasoning.

383
00:24:35,730 --> 00:24:37,830
Ryan Sean Adams:
I mean, are these some of the implications of these ideas?

384
00:24:38,370 --> 00:24:41,810
Dwarkesh:
Yeah, I know. I mean, that's a great insight. Another really interesting insight

385
00:24:41,810 --> 00:24:44,370
Dwarkesh:
is that the more variation there

386
00:24:44,370 --> 00:24:49,390
Dwarkesh:
is in a skill in humans, the better and faster that AIs will get at it.

387
00:24:50,290 --> 00:24:54,970
Dwarkesh:
Because coding is the kind of thing where 1% of humans are really good at it.

388
00:24:55,130 --> 00:24:59,470
Dwarkesh:
The rest of us will, if we try to learn it, we'd be okay at it or something, right?

389
00:25:00,830 --> 00:25:03,290
Dwarkesh:
And because evolutionists spend so little time optimizing us,

390
00:25:03,390 --> 00:25:07,490
Dwarkesh:
there's this room for variation where the optimization hasn't happened uniformly

391
00:25:07,490 --> 00:25:12,350
Dwarkesh:
or it hasn't been valuable enough to saturate the human gene pool for this skill.

392
00:25:14,090 --> 00:25:16,430
Dwarkesh:
I think you made an earlier point that I thought was really interesting I wanted

393
00:25:16,430 --> 00:25:20,850
Dwarkesh:
to address. Can you remind me of the first thing you said? Is it the complementarianism? Yes.

394
00:25:23,010 --> 00:25:27,090
Dwarkesh:
So you can take it as a positive future. You can take it as a negative future

395
00:25:27,090 --> 00:25:30,450
Dwarkesh:
in the sense that, well, what is the complementary skills we're providing?

396
00:25:30,690 --> 00:25:32,970
Dwarkesh:
We're good meat robots.

397
00:25:33,290 --> 00:25:35,690
Ryan Sean Adams:
Yeah, the low skilled labor of the situation.

398
00:25:35,690 --> 00:25:38,810
Dwarkesh:
We can do all the thinking and planning. One dark future,

399
00:25:39,470 --> 00:25:44,290
Dwarkesh:
one dark vision of the future is we'll get those meta glasses

400
00:25:44,730 --> 00:25:50,210
Dwarkesh:
and the AI speaking into our ear and it'll tell us to go put this brick over

401
00:25:50,210 --> 00:25:53,230
Dwarkesh:
there so that the next data center couldn't be built because the AI's got the

402
00:25:53,230 --> 00:25:55,430
Dwarkesh:
plan for everything. It's got the better design for the ship and everything.

403
00:25:55,770 --> 00:25:58,750
Dwarkesh:
You just need to move things around for it. And that's what human labor looks

404
00:25:58,750 --> 00:26:00,170
Dwarkesh:
like until robotics is solved.

405
00:26:01,130 --> 00:26:04,150
Dwarkesh:
So yeah, it depends on how you... On the other hand, you'll get paid a lot because

406
00:26:04,150 --> 00:26:07,090
Dwarkesh:
it's worth a lot to move those bricks. We're building AGI here.

407
00:26:08,070 --> 00:26:09,690
Dwarkesh:
But yeah, it depends on how you come out of that question.

408
00:26:09,910 --> 00:26:12,990
Ryan Sean Adams:
Well, there seems to be something to that idea, going back to the idea of the

409
00:26:12,990 --> 00:26:14,410
Ryan Sean Adams:
massive amount of human variation.

410
00:26:14,730 --> 00:26:18,910
Ryan Sean Adams:
I mean, we have just in the past month or so, we have news of meta hiring AI

411
00:26:18,910 --> 00:26:22,330
Ryan Sean Adams:
researchers for $100 million signing bonuses, okay?

412
00:26:22,490 --> 00:26:27,290
Ryan Sean Adams:
What does the average software engineer make versus what does an AI researcher

413
00:26:27,290 --> 00:26:28,970
Ryan Sean Adams:
make at kind of the top of the market, right?

414
00:26:29,190 --> 00:26:33,790
Ryan Sean Adams:
Which has got to imply, obviously there's some things going on with demand and

415
00:26:33,790 --> 00:26:38,570
Ryan Sean Adams:
supply, but also that it does also seem to imply that there's massive variation

416
00:26:38,570 --> 00:26:40,650
Ryan Sean Adams:
in the quality of a software engineer.

417
00:26:40,810 --> 00:26:43,970
Ryan Sean Adams:
And if AIs can get to that quality, well, what does that unlock?

418
00:26:44,510 --> 00:26:48,070
Ryan Sean Adams:
Yeah. So, okay. Yeah. So I guess we have like coding down right now.

419
00:26:48,230 --> 00:26:53,270
Ryan Sean Adams:
Like another question though is like, what can't AIs do today?

420
00:26:53,530 --> 00:26:57,210
Ryan Sean Adams:
And how would you characterize that? Like what are the things they just don't do well?

421
00:26:57,370 --> 00:27:02,330
Dwarkesh:
So I've been interviewing people on my podcast who have very different timelines

422
00:27:02,330 --> 00:27:05,610
Dwarkesh:
from a role to get to AGI. I have had people on who think it's two years away

423
00:27:05,610 --> 00:27:07,430
Dwarkesh:
and some who think it's 20 years away.

424
00:27:08,850 --> 00:27:13,890
Dwarkesh:
And the experience of building AI tools for myself actually has been the most

425
00:27:13,890 --> 00:27:18,650
Dwarkesh:
insight driving or maybe research I've done on the question of when AI is coming.

426
00:27:18,950 --> 00:27:19,750
Ryan Sean Adams:
More than the guest interviews.

427
00:27:20,410 --> 00:27:25,150
Dwarkesh:
Yeah, because you just, I have had, I've probably spent on the order of a hundred

428
00:27:25,150 --> 00:27:28,990
Dwarkesh:
hours trying to build these little tools. The kinds I'm sure you've also tried

429
00:27:28,990 --> 00:27:32,730
Dwarkesh:
to build of like, rewrite auto-generated transcripts for me to make them sound,

430
00:27:32,930 --> 00:27:34,910
Dwarkesh:
the rewritten the way a human would write them.

431
00:27:35,810 --> 00:27:39,770
Dwarkesh:
Find clips for me to tweet out, write essays with me, co-write them passage

432
00:27:39,770 --> 00:27:41,190
Dwarkesh:
by passage, these kinds of things.

433
00:27:41,910 --> 00:27:45,330
Dwarkesh:
And what I found is that it's actually very hard to get human-like labor out

434
00:27:45,330 --> 00:27:49,030
Dwarkesh:
of these models, even for tasks like these, which should be death center in

435
00:27:49,030 --> 00:27:50,370
Dwarkesh:
the repertoire of these models, right?

436
00:27:50,430 --> 00:27:53,270
Dwarkesh:
They're short horizon, they're language in, language out.

437
00:27:53,650 --> 00:27:58,570
Dwarkesh:
They're not contingent on understanding some thing I said like a month ago.

438
00:27:58,850 --> 00:28:00,330
Dwarkesh:
This is just like, this is the task.

439
00:28:00,670 --> 00:28:04,310
Dwarkesh:
And I was thinking about why is it the case that I still haven't been able to

440
00:28:04,310 --> 00:28:08,030
Dwarkesh:
automate these basic language tasks? Why do I still have a human work on these things?

441
00:28:09,010 --> 00:28:15,730
Dwarkesh:
And I think the key reason that you can't automate even these simple tasks is

442
00:28:15,730 --> 00:28:21,150
Dwarkesh:
because the models currently lack the ability to do on the job training.

443
00:28:21,350 --> 00:28:24,370
Dwarkesh:
So if you hire a human for the first six months, for the first three months,

444
00:28:24,450 --> 00:28:26,330
Dwarkesh:
they're not going to be that useful, even if they're very smart,

445
00:28:26,570 --> 00:28:29,350
Dwarkesh:
because they haven't built up the context, they haven't practiced the skills,

446
00:28:29,510 --> 00:28:31,230
Dwarkesh:
they don't understand how the business works.

447
00:28:31,230 --> 00:28:35,570
Dwarkesh:
What makes humans valuable is not that mainly the raw intellect obviously matters,

448
00:28:35,730 --> 00:28:36,570
Dwarkesh:
but it's not mainly that.

449
00:28:36,890 --> 00:28:40,250
Dwarkesh:
It's their ability to interrogate their own failures in this really dynamic,

450
00:28:40,430 --> 00:28:45,190
Dwarkesh:
organic way to pick up small efficiencies and improvements as they practice

451
00:28:45,190 --> 00:28:50,330
Dwarkesh:
the task and to build up this context as they work within a domain.

452
00:28:50,890 --> 00:28:54,130
Dwarkesh:
And so sometimes people wonder, look, if you look at the revenue of OpenAI,

453
00:28:54,470 --> 00:28:56,910
Dwarkesh:
the annual recurring revenue, it's on the order of $10 billion.

454
00:28:57,490 --> 00:29:00,590
Dwarkesh:
Kohl's makes more money than that. McDonald's makes more money than that, right?

455
00:29:01,250 --> 00:29:07,130
Dwarkesh:
So why is it that if they've got AGI, they're, you know, like Fortune 500 isn't

456
00:29:07,130 --> 00:29:11,850
Dwarkesh:
reorganizing their workflows to, you know, use open AI models at every layer of the stack?

457
00:29:12,470 --> 00:29:15,130
Dwarkesh:
My answer, sometimes people say, well, it's because people are too stodgy.

458
00:29:15,210 --> 00:29:18,150
Dwarkesh:
The management of these companies is like not moving fast enough on AI.

459
00:29:18,290 --> 00:29:19,930
Dwarkesh:
That could be part of it. I think mostly it's not that.

460
00:29:20,250 --> 00:29:23,950
Dwarkesh:
I think mostly it genuinely is very hard to get human-like labor out of these

461
00:29:23,950 --> 00:29:26,710
Dwarkesh:
models because you can't.

462
00:29:26,810 --> 00:29:29,990
Dwarkesh:
So you're stuck with the capabilities you get out of the model out of the box.

463
00:29:30,590 --> 00:29:33,410
Dwarkesh:
So they might be five out of 10 at rewriting the transcript for you.

464
00:29:33,670 --> 00:29:36,210
Dwarkesh:
But if you don't like how it turned out, if you have feedback for it,

465
00:29:36,270 --> 00:29:40,930
Dwarkesh:
if you want to keep teaching it over time, once the session ends,

466
00:29:41,150 --> 00:29:44,070
Dwarkesh:
the model, like everything it knows about you has gone away.

467
00:29:44,370 --> 00:29:47,430
Dwarkesh:
You got to restart again. It's like working with an amnesiac employee.

468
00:29:47,770 --> 00:29:49,010
Dwarkesh:
You got to restart again.

469
00:29:49,590 --> 00:29:51,670
Ryan Sean Adams:
Every day is the first day of employment, basically.

470
00:29:52,150 --> 00:29:55,770
Dwarkesh:
Yeah, exactly. It's a groundhog day for them every day or every couple of hours, in fact.

471
00:29:56,490 --> 00:29:59,490
Dwarkesh:
And that makes it very hard for them to be that useful as an employee,

472
00:29:59,690 --> 00:30:01,810
Dwarkesh:
right? They're not really an employee at that point.

473
00:30:02,470 --> 00:30:06,350
Dwarkesh:
This, I think, not only is a key bottleneck to the value of these models,

474
00:30:06,390 --> 00:30:08,970
Dwarkesh:
because human labor is worth a lot, right?

475
00:30:09,070 --> 00:30:12,090
Dwarkesh:
Like $60 trillion in the world is paid to wages every year.

476
00:30:12,730 --> 00:30:18,210
Dwarkesh:
If these model companies are making on the order of $10 billion a year, that's a big way to AGI.

477
00:30:18,490 --> 00:30:21,650
Dwarkesh:
And what explains that gap? What are the bottlenecks? I think a big one is this

478
00:30:21,650 --> 00:30:22,510
Dwarkesh:
continual learning thing.

479
00:30:23,490 --> 00:30:26,190
Dwarkesh:
And I don't see an easy way that that just gets solved within these models.

480
00:30:26,350 --> 00:30:29,010
Dwarkesh:
There's no like, with reasoning, you could say, oh, it's like train it on math

481
00:30:29,010 --> 00:30:31,270
Dwarkesh:
and code problems, and then I'll get the reasoning. And that worked.

482
00:30:31,870 --> 00:30:35,530
Dwarkesh:
I don't think there's something super obvious there for how do you get this

483
00:30:35,530 --> 00:30:38,170
Dwarkesh:
online learning, this on-the-job training working for these models.

484
00:30:38,530 --> 00:30:41,610
Ryan Sean Adams:
Okay, can we talk about that, go a little bit deeper on that concept?

485
00:30:41,770 --> 00:30:44,810
Ryan Sean Adams:
So this is basically one of the concepts you wrote in your recent post.

486
00:30:44,950 --> 00:30:48,330
Ryan Sean Adams:
AI is not right around the corner. Even though you're an AI optimist,

487
00:30:48,430 --> 00:30:53,210
Ryan Sean Adams:
I would say, and overall an AI accelerationist, you You were saying it's not

488
00:30:53,210 --> 00:30:53,950
Ryan Sean Adams:
right around the corner.

489
00:30:54,110 --> 00:30:58,430
Ryan Sean Adams:
You're saying the ability to replace human labor is a ways out.

490
00:30:58,590 --> 00:31:01,990
Ryan Sean Adams:
Not forever out, but I think you said somewhere around 2032,

491
00:31:01,990 --> 00:31:04,930
Ryan Sean Adams:
if you had to guess on when the estimate was.

492
00:31:05,050 --> 00:31:09,150
Ryan Sean Adams:
And the reason you gave is because AIs can't learn on the job,

493
00:31:09,190 --> 00:31:10,890
Ryan Sean Adams:
but it's not clear to me why they can't.

494
00:31:10,970 --> 00:31:14,290
Ryan Sean Adams:
Is it just because the context window isn't large enough?

495
00:31:14,470 --> 00:31:19,430
Ryan Sean Adams:
Is it just because they can't input all of the different data sets and data

496
00:31:19,430 --> 00:31:24,650
Ryan Sean Adams:
points that humans can? Is it because they don't have stateful memory the way a human employee?

497
00:31:25,030 --> 00:31:28,650
Ryan Sean Adams:
Because if it's these things, all of these do seem like solvable problems.

498
00:31:28,830 --> 00:31:30,950
Ryan Sean Adams:
And maybe that's what you're saying. They are solvable problems.

499
00:31:30,950 --> 00:31:35,210
Ryan Sean Adams:
They're just a little bit longer than some people think they are.

500
00:31:35,350 --> 00:31:39,910
Dwarkesh:
I think it's like in some deep sense a solvable problem because eventually we will build AGI.

501
00:31:40,310 --> 00:31:43,110
Dwarkesh:
And to build AGI, we will have had to solve the problem.

502
00:31:43,490 --> 00:31:46,790
Dwarkesh:
My point is that the obvious solutions you might imagine, for example,

503
00:31:46,990 --> 00:31:49,970
Dwarkesh:
expanding the context window or having this

504
00:31:49,970 --> 00:31:54,450
Dwarkesh:
like external memory using systems like rag these

505
00:31:54,450 --> 00:31:57,370
Dwarkesh:
are basically techniques we already have to it's called retrieval augmented

506
00:31:57,370 --> 00:32:00,410
Dwarkesh:
generate anyways these kinds of retrieval augmented generation i

507
00:32:00,410 --> 00:32:04,330
Dwarkesh:
don't think these will suffice and just to put a finer point first of all like

508
00:32:04,330 --> 00:32:09,550
Dwarkesh:
what is the problem the problem is exactly as you say that within the context

509
00:32:09,550 --> 00:32:13,370
Dwarkesh:
window these models actually can learn on the job right so if you talk to it

510
00:32:13,370 --> 00:32:17,470
Dwarkesh:
for long enough it will get much better at understanding your needs and what your exact problem is.

511
00:32:17,550 --> 00:32:20,950
Dwarkesh:
If you're using it for research for your podcast, it will get a sense of like,

512
00:32:21,070 --> 00:32:25,350
Dwarkesh:
oh, they're actually especially curious about these kinds of questions. Let me focus on that.

513
00:32:25,530 --> 00:32:28,570
Dwarkesh:
It's actually very human-like in context, right? The speed at which it learns,

514
00:32:28,770 --> 00:32:30,150
Dwarkesh:
the task of knowledge it picks out.

515
00:32:30,450 --> 00:32:33,650
Dwarkesh:
The problem, of course, is the context length for even the best models only

516
00:32:33,650 --> 00:32:35,490
Dwarkesh:
last a million or two million tokens.

517
00:32:36,130 --> 00:32:38,430
Dwarkesh:
That's at most like an hour of conversation.

518
00:32:39,070 --> 00:32:42,470
Dwarkesh:
Now, then you might say, okay, well, why can't we just solve that by expanding

519
00:32:42,470 --> 00:32:44,930
Dwarkesh:
the context window, right? So context window has been expanding for the last

520
00:32:44,930 --> 00:32:46,730
Dwarkesh:
few years. Why can't we just continue that?

521
00:32:47,090 --> 00:32:49,950
Ryan Sean Adams:
Yeah, like a billion token context window, something like this.

522
00:32:50,750 --> 00:32:54,990
Dwarkesh:
So 2018 is when the transformer came out and the transformer has the attention mechanism.

523
00:32:55,290 --> 00:33:00,670
Dwarkesh:
The attention mechanism is inherently quadratic with the nature of the length

524
00:33:00,670 --> 00:33:05,590
Dwarkesh:
of the sequence, which is to say that if you go from if you double go from 1

525
00:33:05,590 --> 00:33:06,930
Dwarkesh:
million tokens to 2 million tokens,

526
00:33:07,210 --> 00:33:12,670
Dwarkesh:
it actually costs four times as much compute to process that 2 millionth token.

527
00:33:12,990 --> 00:33:18,570
Dwarkesh:
It's not just 2 to as much compute. so it gets super linearly more expensive

528
00:33:18,570 --> 00:33:23,010
Dwarkesh:
as you increase the context length and for the last,

529
00:33:23,590 --> 00:33:26,770
Dwarkesh:
seven years people have been trying to get around this inherent quadratic nature

530
00:33:26,770 --> 00:33:31,590
Dwarkesh:
of attention of course we don't know secretly what the labs are working on but we have frontier,

531
00:33:32,310 --> 00:33:35,150
Dwarkesh:
companies like deep seek which have open source their research and

532
00:33:35,150 --> 00:33:38,290
Dwarkesh:
we can just see how their algorithms work and they found

533
00:33:38,290 --> 00:33:41,010
Dwarkesh:
these constant time modifiers to attention which is

534
00:33:41,010 --> 00:33:44,410
Dwarkesh:
to say that they there's like a it'll still

535
00:33:44,410 --> 00:33:47,270
Dwarkesh:
be quadratic but it'll be like one half times

536
00:33:47,270 --> 00:33:49,930
Dwarkesh:
quadratic but the inherent like super linearness has not

537
00:33:49,930 --> 00:33:53,430
Dwarkesh:
gone away and because of that yeah you might be able to increase it from 1 million

538
00:33:53,430 --> 00:33:57,190
Dwarkesh:
tokens to 2 million tokens by finding another hack like uh make sure experts

539
00:33:57,190 --> 00:34:01,830
Dwarkesh:
just run such things latent attention is another such technique but or kbcash

540
00:34:01,830 --> 00:34:05,510
Dwarkesh:
right there's many other things that have been discovered but people have not

541
00:34:05,510 --> 00:34:08,770
Dwarkesh:
discovered okay how do you get around the fact that if you went to a billion,

542
00:34:09,110 --> 00:34:14,130
Dwarkesh:
it would be a billion squared as expensive in terms of compute to process that token.

543
00:34:14,610 --> 00:34:18,570
Dwarkesh:
And so I don't think you'll just get it by increasing the length of the context window, basically.

544
00:34:19,210 --> 00:34:23,190
Ryan Sean Adams:
That's fascinating. Yeah, I didn't realize that. Okay, so the other reason in

545
00:34:23,190 --> 00:34:27,170
Ryan Sean Adams:
your post that AI is not right around the corner is because it can't do your taxes.

546
00:34:27,850 --> 00:34:33,290
Ryan Sean Adams:
And Dwarkesh, I feel your pain, man. Taxes are just like quite a pain in the ass.

547
00:34:33,550 --> 00:34:36,750
Ryan Sean Adams:
I think you were talking about this from the context of like computer vision,

548
00:34:36,750 --> 00:34:38,350
Ryan Sean Adams:
computer use, that kind of thing.

549
00:34:38,630 --> 00:34:42,910
Ryan Sean Adams:
So, I mean, I've seen demos. I've seen some pretty interesting computer vision

550
00:34:42,910 --> 00:34:46,350
Ryan Sean Adams:
sort of demos that seem to be right around the corner.

551
00:34:46,450 --> 00:34:49,670
Ryan Sean Adams:
But what's the limiter on computer use for an AI?

552
00:34:49,970 --> 00:34:54,230
Dwarkesh:
There was an interesting blog post by this company called Mechanize where they

553
00:34:54,230 --> 00:34:58,110
Dwarkesh:
were explaining why this is such a big problem. And I love the way they phrased it, which is that,

554
00:34:58,880 --> 00:35:04,020
Dwarkesh:
Imagine if you had to train a model in 1980, a large language model in 1980,

555
00:35:04,260 --> 00:35:08,780
Dwarkesh:
and you could use all the compute you wanted in 1980 somehow,

556
00:35:08,780 --> 00:35:14,340
Dwarkesh:
but you didn't have, you were only stuck with the data that was available in

557
00:35:14,340 --> 00:35:16,980
Dwarkesh:
the 1980s, of course, before the internet became a widespread phenomenon.

558
00:35:17,560 --> 00:35:20,580
Dwarkesh:
You couldn't train a modern LLM, even with all the computer in the world,

559
00:35:20,640 --> 00:35:21,880
Dwarkesh:
because the data wasn't available.

560
00:35:22,400 --> 00:35:26,040
Dwarkesh:
And we're in a similar position with respect to computer use,

561
00:35:26,200 --> 00:35:31,040
Dwarkesh:
because there's not this corpus of collected videos, people using computers

562
00:35:31,040 --> 00:35:36,720
Dwarkesh:
to do different things, to access different applications and do white collar work.

563
00:35:37,200 --> 00:35:43,080
Dwarkesh:
Because of that, I think the big challenge has been accumulating this kind of data. off.

564
00:35:43,940 --> 00:35:47,120
Ryan Sean Adams:
And to be clear, when I was saying the use case of like, do my taxes,

565
00:35:47,380 --> 00:35:51,520
Ryan Sean Adams:
you're effectively talking about an AI having the ability to just like,

566
00:35:51,660 --> 00:35:53,780
Ryan Sean Adams:
you know, navigate the files around your computer,

567
00:35:54,280 --> 00:35:57,900
Ryan Sean Adams:
you know, log in to various websites to download your pay stubs or whatever,

568
00:35:58,140 --> 00:36:02,600
Ryan Sean Adams:
and then to go to like TurboTax or something and like input it all into some

569
00:36:02,600 --> 00:36:04,460
Ryan Sean Adams:
software and file it, right?

570
00:36:04,580 --> 00:36:08,060
Ryan Sean Adams:
Just on voice command or something like that. That's basically doing my taxes.

571
00:36:08,540 --> 00:36:13,860
Dwarkesh:
It should be capable of navigating UIs that it's less familiar with or that

572
00:36:13,860 --> 00:36:17,200
Dwarkesh:
come about organically within the context of trying to solve a problem.

573
00:36:17,340 --> 00:36:20,800
Dwarkesh:
So for example, I might have business deductions.

574
00:36:20,940 --> 00:36:24,040
Dwarkesh:
It sees on my bank statement that I've spent $1,000 on Amazon.

575
00:36:24,140 --> 00:36:25,660
Dwarkesh:
It goes logs in my Amazon.

576
00:36:25,860 --> 00:36:29,380
Dwarkesh:
It sees like, oh, he bought a camera. So I think that's probably a business

577
00:36:29,380 --> 00:36:30,260
Dwarkesh:
expense for his podcast.

578
00:36:30,920 --> 00:36:35,360
Dwarkesh:
He bought an Airbnb over a weekend in the cabins of whatever,

579
00:36:35,620 --> 00:36:37,760
Dwarkesh:
in the woods of whatever. That probably wasn't a business expense.

580
00:36:38,180 --> 00:36:42,540
Dwarkesh:
Although maybe, maybe it's, if it's a sort of like a gray, if it's willing to

581
00:36:42,540 --> 00:36:44,940
Dwarkesh:
go in the gray area, maybe I'll talk to you. Yeah, yeah, yeah.

582
00:36:45,340 --> 00:36:46,300
Ryan Sean Adams:
Do the gray area stuff.

583
00:36:46,560 --> 00:36:47,880
Dwarkesh:
I was, I was researching.

584
00:36:50,780 --> 00:36:55,020
Dwarkesh:
But anyway, so that, including all of that, including emailing people for invoices,

585
00:36:55,720 --> 00:37:01,260
Dwarkesh:
and haggling with them, it would be like a sort of week long task to do my taxes, right?

586
00:37:01,420 --> 00:37:04,800
Dwarkesh:
You'd have to, there's a lot of work involved. That's not just like do this

587
00:37:04,800 --> 00:37:08,180
Dwarkesh:
skill, this skill, this skill, but rather of having a sort of like plan of action

588
00:37:08,180 --> 00:37:11,680
Dwarkesh:
and then breaking tasks apart, dealing with new information,

589
00:37:11,940 --> 00:37:15,640
Dwarkesh:
new emails, new messages, consulting with me about questions, et cetera.

590
00:37:16,000 --> 00:37:18,920
Ryan Sean Adams:
Yeah, I mean, to be clear on this use case too, even though your post is titled

591
00:37:18,920 --> 00:37:22,720
Ryan Sean Adams:
like, you know, AI is not right around the corner, you still think this ability

592
00:37:22,720 --> 00:37:27,380
Ryan Sean Adams:
to file your taxes, that's like a 2028 thing, right?

593
00:37:27,500 --> 00:37:30,800
Ryan Sean Adams:
I mean, this is maybe not next year, but it's in a few years.

594
00:37:31,420 --> 00:37:35,460
Dwarkesh:
Right, which is, I think that was sort of, people maybe write too much in The

595
00:37:35,460 --> 00:37:37,500
Dwarkesh:
Decital and then didn't read through the arguments.

596
00:37:37,700 --> 00:37:39,860
Ryan Sean Adams:
I mean, that never happens on the internet. Wow.

597
00:37:40,340 --> 00:37:40,980
Dwarkesh:
First time.

598
00:37:42,060 --> 00:37:46,640
Dwarkesh:
No, I think like I'm arguing against people who are like, you know, this will happen.

599
00:37:47,580 --> 00:37:53,200
Dwarkesh:
AGI is like two years away. I do think the wider world, the markets,

600
00:37:53,460 --> 00:37:56,640
Dwarkesh:
public perception, even people who are somewhat attending to AI,

601
00:37:57,120 --> 00:38:03,200
Dwarkesh:
but aren't in this specific milieu that I'm talking to, are way underpricing AGI.

602
00:38:03,980 --> 00:38:09,440
Dwarkesh:
One reason, one thing I think they're underestimating is not only will we have

603
00:38:09,440 --> 00:38:12,020
Dwarkesh:
millions of extra laborers, millions of extra workers,

604
00:38:12,300 --> 00:38:16,020
Dwarkesh:
potentially billions within the course of the next decade, because then we will

605
00:38:16,020 --> 00:38:19,780
Dwarkesh:
have a potentially, I think like likely we will have AGI within the next decade.

606
00:38:20,440 --> 00:38:23,560
Dwarkesh:
But they'll have these advantages that human workers don't have,

607
00:38:23,680 --> 00:38:27,600
Dwarkesh:
which is that, okay, a single model company, so suppose we solve continual learning, right?

608
00:38:27,720 --> 00:38:30,620
Dwarkesh:
So there, and we saw computer use. So as far as white collar work goes,

609
00:38:31,200 --> 00:38:32,720
Dwarkesh:
that might fundamentally it would be solved.

610
00:38:32,920 --> 00:38:36,660
Dwarkesh:
You can have AIs which can use not just they're not just like a text box where

611
00:38:36,660 --> 00:38:39,660
Dwarkesh:
you put into you ask questions in a chatbot and you get some response out.

612
00:38:39,820 --> 00:38:42,660
Dwarkesh:
It's not that useful to just have a very smart chatbot. You need it to be able

613
00:38:42,660 --> 00:38:44,520
Dwarkesh:
to actually do real work and use real applications.

614
00:38:45,720 --> 00:38:47,940
Dwarkesh:
Suppose you have that solved because it acts like an employee.

615
00:38:48,060 --> 00:38:49,680
Dwarkesh:
It's got continual learning. It's got computer use.

616
00:38:49,920 --> 00:38:53,440
Dwarkesh:
But it has another advantage that humans don't have, which is that copies of

617
00:38:53,440 --> 00:38:57,560
Dwarkesh:
this model are going being deployed all through the economy and it's doing on the job training.

618
00:38:57,740 --> 00:39:00,140
Dwarkesh:
So copies are learning how to be an accountant, how to be a lawyer,

619
00:39:00,260 --> 00:39:04,100
Dwarkesh:
how to be a coder, except because it's an AI and it's digital,

620
00:39:04,360 --> 00:39:10,120
Dwarkesh:
the model itself can amalgamate all this on-the-job training from all these copies.

621
00:39:10,580 --> 00:39:13,960
Dwarkesh:
So what does that mean? Well, it means that even if there's no more software

622
00:39:13,960 --> 00:39:17,180
Dwarkesh:
progress after that point, which is to say that no more algorithms are discovered,

623
00:39:17,340 --> 00:39:19,540
Dwarkesh:
there's not a transformer plus plus that's discovered.

624
00:39:20,660 --> 00:39:25,560
Dwarkesh:
Just from the fact that this model is learning every single skill in the economy,

625
00:39:25,780 --> 00:39:29,960
Dwarkesh:
at least for white-collar work, you might just, based on that alone,

626
00:39:30,200 --> 00:39:31,960
Dwarkesh:
have something that looks like an intelligence explosion.

627
00:39:31,960 --> 00:39:35,940
Dwarkesh:
It would just be a broadly deployed intelligence explosion, but it would functionally

628
00:39:35,940 --> 00:39:41,320
Dwarkesh:
become super intelligent just from having human-level capability of learning on the job.

629
00:39:41,580 --> 00:39:44,920
Dwarkesh Patel:
Yeah, and it creates this mesh network of intelligence that's shared among everyone.

630
00:39:45,420 --> 00:39:48,620
Dwarkesh Patel:
That's a really fascinating thing. So we're going to get there.

631
00:39:48,820 --> 00:39:51,040
Dwarkesh Patel:
We're going to get to AGI. it's going to be incredibly smart.

632
00:39:51,080 --> 00:39:54,160
Dwarkesh Patel:
But what we've shared recently is just kind of this mixed bag where currently

633
00:39:54,160 --> 00:39:56,920
Dwarkesh Patel:
today, it's pretty good at some things, but also not that great at others.

634
00:39:57,060 --> 00:40:00,720
Dwarkesh Patel:
We're hiring humans to do jobs that we think AI should do, but it probably doesn't.

635
00:40:00,940 --> 00:40:04,360
Dwarkesh Patel:
So the question I have for you is, is AI really that smart? Or is it just good

636
00:40:04,360 --> 00:40:07,580
Dwarkesh Patel:
at kind of acing these particular benchmarks that we measure against?

637
00:40:08,100 --> 00:40:11,440
Dwarkesh Patel:
Apple, I mean, famously recently, they had their paper, The Illusion of Thinking,

638
00:40:11,640 --> 00:40:14,680
Dwarkesh Patel:
where it was kind of like, hey, AI is like pretty good up to a point,

639
00:40:14,840 --> 00:40:16,600
Dwarkesh Patel:
but at a certain point, it just falls apart.

640
00:40:16,880 --> 00:40:21,120
Dwarkesh Patel:
And the inference is like, maybe it's not intelligence, maybe it's just good

641
00:40:21,120 --> 00:40:24,120
Dwarkesh Patel:
at guessing. So I guess the question is, is AI really that smart?

642
00:40:24,260 --> 00:40:27,460
Dwarkesh:
It depends on who I'm talking to. I think some people overhype its capabilities.

643
00:40:27,800 --> 00:40:31,660
Dwarkesh:
I think some people are like, oh, it's already AGI, but it's like a little hobbled

644
00:40:31,660 --> 00:40:35,640
Dwarkesh:
little AGI where we're like sort of giving it a concussion every couple of hours

645
00:40:35,640 --> 00:40:36,760
Dwarkesh:
and like it forgets everything.

646
00:40:37,360 --> 00:40:41,380
Dwarkesh:
We're like trapped in a chatbot context. But fundamentally, the thing inside

647
00:40:41,380 --> 00:40:43,380
Dwarkesh:
is like a very smart human.

648
00:40:44,080 --> 00:40:46,280
Dwarkesh:
I disagree with that perspective. So if that's your perspective,

649
00:40:46,380 --> 00:40:47,500
Dwarkesh:
I say like, no, it's not that smart.

650
00:40:47,700 --> 00:40:51,080
Dwarkesh:
Your perspective is just statistical associations. I say definitely smarter.

651
00:40:51,280 --> 00:40:53,040
Dwarkesh:
Like it's like genuinely there's an intelligence there.

652
00:40:54,680 --> 00:40:58,480
Dwarkesh:
And the, so one thing you could say to the person who thinks that it's already

653
00:40:58,480 --> 00:41:03,600
Dwarkesh:
AGI is this, look, if a single human had as much stuff memorized as these models

654
00:41:03,600 --> 00:41:04,540
Dwarkesh:
seem to have memorized, right?

655
00:41:04,780 --> 00:41:08,720
Dwarkesh:
Which is to say that they have all of internet text, everything that human has

656
00:41:08,720 --> 00:41:13,640
Dwarkesh:
written on the internet memorized, they would potentially be discovering all

657
00:41:13,640 --> 00:41:16,780
Dwarkesh:
kinds of connections and discoveries.

658
00:41:16,780 --> 00:41:22,400
Dwarkesh:
They'd notice that this thing which causes a migraine is associated with this kind of deficiency.

659
00:41:22,680 --> 00:41:25,040
Dwarkesh:
So maybe if you take the supplement, your migraines will be cured.

660
00:41:25,740 --> 00:41:28,960
Dwarkesh:
There'd be just this list of just like trivial connections that lead to big

661
00:41:28,960 --> 00:41:30,180
Dwarkesh:
discoveries all through the place.

662
00:41:30,300 --> 00:41:36,500
Dwarkesh:
It's not clear that there's been an unambiguous case of an AI just doing this by itself.

663
00:41:37,080 --> 00:41:40,960
Dwarkesh:
So then why, so that's something potentially to explain, like if they're so

664
00:41:40,960 --> 00:41:43,980
Dwarkesh:
intelligent, why aren't they able to use their disproportionate capabilities,

665
00:41:44,620 --> 00:41:46,620
Dwarkesh:
their unique capabilities to come up with these discoveries?

666
00:41:47,300 --> 00:41:49,080
Dwarkesh:
I don't think there's actually a good answer to that question yet,

667
00:41:49,200 --> 00:41:51,040
Dwarkesh:
except for the fact that they genuinely aren't that creative.

668
00:41:51,600 --> 00:41:53,820
Dwarkesh:
Maybe they're like intelligent in the sense of knowing a lot of things,

669
00:41:54,000 --> 00:41:56,100
Dwarkesh:
but they don't have this fluid intelligence that humans have.

670
00:41:57,440 --> 00:42:00,880
Dwarkesh:
Anyway, so I give you a wish-washy answer because I think some people are underselling

671
00:42:00,880 --> 00:42:02,380
Dwarkesh:
the intelligence. Some people are overselling it.

672
00:42:03,320 --> 00:42:07,000
Ryan Sean Adams:
I recall a tweet lately from Tyler Cowen. I think he was referring to maybe

673
00:42:07,000 --> 00:42:10,620
Ryan Sean Adams:
O3, and he basically said, it feels like AGI.

674
00:42:10,900 --> 00:42:14,540
Ryan Sean Adams:
I don't know if it is AGI or not, but like to me, it feels like AGI.

675
00:42:14,540 --> 00:42:18,180
Ryan Sean Adams:
What do you account for this feeling of like intelligence then

676
00:42:18,180 --> 00:42:22,600
Dwarkesh:
I think this is actually very interesting because it gets to a crux that Tyler

677
00:42:22,600 --> 00:42:28,020
Dwarkesh:
and I have so Tyler and I disagree on two big things one he thinks you know

678
00:42:28,020 --> 00:42:31,420
Dwarkesh:
as he said in the blog post 03 is AGI I don't think it's AGI I think it's,

679
00:42:32,590 --> 00:42:37,230
Dwarkesh:
it's orders of magnitude less valuable or, you know, like many orders of magnitude

680
00:42:37,230 --> 00:42:38,970
Dwarkesh:
less valuable and less useful than an AGI.

681
00:42:39,270 --> 00:42:43,210
Dwarkesh:
That's one thing we disagree on. The other thing we disagree on is he thinks

682
00:42:43,210 --> 00:42:47,750
Dwarkesh:
that once we do get AGI, we'll only see 0.5% increase in the economic growth

683
00:42:47,750 --> 00:42:49,330
Dwarkesh:
rate. This is like what the internet caused, right?

684
00:42:49,870 --> 00:42:53,630
Dwarkesh:
Whereas I think we will see tens of percent increase in economic growth.

685
00:42:53,710 --> 00:42:57,810
Dwarkesh:
Like it will just be the difference between the pre-industrial revolution rate

686
00:42:57,810 --> 00:43:00,950
Dwarkesh:
of growth versus industrial revolution, that magnitude of change again.

687
00:43:00,950 --> 00:43:05,190
Dwarkesh:
And I think these two disagreements are linked because if you do believe we're

688
00:43:05,190 --> 00:43:07,930
Dwarkesh:
already at AGI and you look around the world and you say like,

689
00:43:08,130 --> 00:43:12,030
Dwarkesh:
well, it fundamentally looks the same, you'd be forgiven for thinking like,

690
00:43:12,130 --> 00:43:14,290
Dwarkesh:
oh, there's not that much value in getting to AGI.

691
00:43:14,290 --> 00:43:17,310
Dwarkesh:
Whereas if you are like me and you think like, no, we'll get this broadly at

692
00:43:17,310 --> 00:43:22,330
Dwarkesh:
the minimum, at a very minimum, we'll get a broadly deployed intelligence explosion once we get to AGI,

693
00:43:22,630 --> 00:43:26,890
Dwarkesh:
then you're like, OK, I'm just expecting some sort of singulitarian crazy future

694
00:43:26,890 --> 00:43:31,910
Dwarkesh:
with a robot factories and, you know, solar farms all across the desert and things like that.

695
00:43:31,910 --> 00:43:35,990
Ryan Sean Adams:
Yeah, I mean, it strikes me that your disagreement with Tyler is just based

696
00:43:35,990 --> 00:43:39,290
Ryan Sean Adams:
on the semantic definition of like what AGI actually is.

697
00:43:39,710 --> 00:43:44,170
Ryan Sean Adams:
And Tyler, it sounds like he has kind of a lower threshold for what AGI is,

698
00:43:44,270 --> 00:43:45,450
Ryan Sean Adams:
whereas you have a higher threshold.

699
00:43:45,630 --> 00:43:48,550
Ryan Sean Adams:
Is there like a accepted definition for AGI?

700
00:43:48,550 --> 00:43:54,810
Dwarkesh:
No. One thing that's useful for the purposes of discussions is to say automating

701
00:43:54,810 --> 00:43:59,550
Dwarkesh:
all white collar work because robotics hasn't made as much progress as LLMs

702
00:43:59,550 --> 00:44:01,070
Dwarkesh:
have or computer use has.

703
00:44:01,250 --> 00:44:06,650
Dwarkesh:
So if we just say anything a human can do or maybe 90% of what humans can do

704
00:44:06,650 --> 00:44:11,510
Dwarkesh:
at a desk, an AI can also do, that's potentially a useful definition for at

705
00:44:11,510 --> 00:44:14,190
Dwarkesh:
least getting the cognitive elements relevant to defining AGI.

706
00:44:15,140 --> 00:44:18,300
Dwarkesh:
But yeah, there's not one definition which suits all purposes.

707
00:44:18,620 --> 00:44:22,920
Ryan Sean Adams:
Do we know what's like going on inside of these models, right?

708
00:44:23,100 --> 00:44:26,780
Ryan Sean Adams:
So like, you know, Josh was talking earlier in the conversation about like this

709
00:44:26,780 --> 00:44:29,440
Ryan Sean Adams:
at the base being sort of token prediction, right?

710
00:44:29,720 --> 00:44:35,160
Ryan Sean Adams:
And I guess this starts to raise the question of like, what is intelligence in the first place?

711
00:44:35,540 --> 00:44:40,140
Ryan Sean Adams:
And these AI models, I mean, they seem like they're intelligent,

712
00:44:40,140 --> 00:44:44,360
Ryan Sean Adams:
but do they have a model of the world the way maybe a human might?

713
00:44:44,360 --> 00:44:49,360
Ryan Sean Adams:
Are they sort of babbling or like, is this real reasoning?

714
00:44:49,740 --> 00:44:53,900
Ryan Sean Adams:
And like, what is real reasoning? Do we just judge that based on the results

715
00:44:53,900 --> 00:44:56,260
Ryan Sean Adams:
or is there some way to like peek inside of its head?

716
00:44:56,400 --> 00:45:00,240
Dwarkesh:
I used to have similar questions a couple of years ago. And then,

717
00:45:00,620 --> 00:45:03,480
Dwarkesh:
because honestly, the things they did at the time were like ambiguous.

718
00:45:03,480 --> 00:45:07,360
Dwarkesh:
You could say, oh, it's close enough to something else in this trading data set.

719
00:45:07,720 --> 00:45:12,740
Dwarkesh:
That is just basically copy pasting. It didn't come up with a solution by itself.

720
00:45:12,740 --> 00:45:17,040
Dwarkesh:
But we've gotten to the point where I can come up with a pretty complicated

721
00:45:17,040 --> 00:45:19,280
Dwarkesh:
math problem and it will solve it.

722
00:45:19,660 --> 00:45:24,400
Dwarkesh:
It can be a math problem, like not like, you know, undergrad or high school math problem.

723
00:45:24,500 --> 00:45:28,360
Dwarkesh:
Like the problem we get, the problems the smartest math professors come up with

724
00:45:28,360 --> 00:45:31,960
Dwarkesh:
in order to test International Math Olympiad.

725
00:45:31,960 --> 00:45:34,520
Dwarkesh:
You know, the kids who spend all their life preparing for this,

726
00:45:34,760 --> 00:45:37,620
Dwarkesh:
the geniuses who spend all their life, all their young adulthood preparing to

727
00:45:37,620 --> 00:45:40,160
Dwarkesh:
take these really gnarly math puzzle challenges.

728
00:45:40,480 --> 00:45:43,220
Dwarkesh:
And the model will get these kinds of questions, right? They require all this

729
00:45:43,220 --> 00:45:48,040
Dwarkesh:
abstract creative thinking, this reasoning for hours, the model will get the right.

730
00:45:48,280 --> 00:45:53,400
Dwarkesh:
Okay, so if that's not reasoning, then why is reasoning valuable again?

731
00:45:53,620 --> 00:45:55,380
Dwarkesh:
Like, what exactly was this reasoning supposed to be?

732
00:45:56,100 --> 00:45:59,160
Dwarkesh:
So I think they genuinely are reasoning. I mean, I think there's other capabilities

733
00:45:59,160 --> 00:46:03,160
Dwarkesh:
they lack, which are actually more, in some sense, they seem to us to be more

734
00:46:03,160 --> 00:46:07,200
Dwarkesh:
trivial, but actually much harder to learn. But the reasoning itself, I think, is there.

735
00:46:07,660 --> 00:46:10,600
Dwarkesh Patel:
And the answer to the intelligence question is also kind of clouded,

736
00:46:10,680 --> 00:46:14,240
Dwarkesh Patel:
right? Because we still really don't understand what's going on in an LLM.

737
00:46:14,580 --> 00:46:17,520
Dwarkesh Patel:
Dario from Anthropoc, he recently posted the paper about interpretation.

738
00:46:17,900 --> 00:46:21,600
Dwarkesh Patel:
And can you explain why we don't even really understand what's going on in these

739
00:46:21,600 --> 00:46:26,220
Dwarkesh Patel:
LLMs, even though we're able to make them and yield the results from them? Mmm.

740
00:46:27,320 --> 00:46:30,520
Dwarkesh Patel:
Because it very much still is kind of like a black box. We write some code,

741
00:46:30,600 --> 00:46:34,540
Dwarkesh Patel:
we put some inputs in, and we get something out, but we're not sure what happens in the middle,

742
00:46:34,580 --> 00:46:36,580
Dwarkesh:
Why it's creating this output.

743
00:46:36,840 --> 00:46:37,940
Dwarkesh Patel:
I mean, it's exactly what you're saying.

744
00:46:38,320 --> 00:46:43,960
Dwarkesh:
It's that in other systems we engineer in the world, we have to build it up bottom-ups.

745
00:46:44,020 --> 00:46:49,340
Dwarkesh:
If you build a bridge, you have to understand how every single beam is contributing to the structure.

746
00:46:50,220 --> 00:46:53,720
Dwarkesh:
And we have equations for why the thing will stay standing.

747
00:46:54,560 --> 00:46:58,760
Dwarkesh:
There's no such thing for AI. We didn't build it, more so we grew it.

748
00:46:59,260 --> 00:47:03,420
Dwarkesh:
It's like watering a plant. And a couple thousand years ago,

749
00:47:03,440 --> 00:47:07,720
Dwarkesh:
they were doing agriculture, but they didn't know why.

750
00:47:08,020 --> 00:47:13,300
Dwarkesh:
Why do plants grow? How do they collect energy from sunlight? All these things.

751
00:47:13,780 --> 00:47:18,040
Dwarkesh:
And I think we're in a substantially similar position with respect to intelligence,

752
00:47:18,660 --> 00:47:23,000
Dwarkesh:
with respect to consciousness, with respect to all these other interesting questions

753
00:47:23,000 --> 00:47:27,600
Dwarkesh:
about how minds work, which is in some sense really cool because there's this

754
00:47:27,600 --> 00:47:32,860
Dwarkesh:
huge intellectual horizon that's become not only available, but accessible to investigation.

755
00:47:33,140 --> 00:47:37,840
Dwarkesh:
In another sense, it's scary because we know that minds can suffer.

756
00:47:37,960 --> 00:47:42,920
Dwarkesh:
We know that minds have moral worth and we're creating minds and we have no

757
00:47:42,920 --> 00:47:44,720
Dwarkesh:
understanding of what's happening in these minds.

758
00:47:44,840 --> 00:47:47,820
Dwarkesh:
Is a process of gradient descent a painful process?

759
00:47:48,060 --> 00:47:50,260
Dwarkesh:
We don't know, but we're doing a lot of it.

760
00:47:51,980 --> 00:47:54,320
Dwarkesh:
So hopefully we'll learn more. But yeah, I think we're in a similar position

761
00:47:54,320 --> 00:47:57,000
Dwarkesh:
to some farmer in Uruk in 3500 BC.

762
00:47:57,740 --> 00:47:58,140
Josh Kale:
Wow.

763
00:47:58,600 --> 00:48:03,280
Ryan Sean Adams:
And I mean, the potential, the idea that minds can suffer, minds have some moral

764
00:48:03,280 --> 00:48:05,980
Ryan Sean Adams:
worth, and also minds have some free will.

765
00:48:06,120 --> 00:48:11,020
Ryan Sean Adams:
They have some sort of autonomy, or maybe at least a desire to have autonomy.

766
00:48:11,240 --> 00:48:15,980
Ryan Sean Adams:
I mean, this brings us to kind of this sticky subject of alignment and AI safety

767
00:48:15,980 --> 00:48:20,200
Ryan Sean Adams:
and how we go about controlling the intelligence that we're creating,

768
00:48:20,240 --> 00:48:24,500
Ryan Sean Adams:
if even that's what we should be doing, controlling it. And we'll get to that in a minute.

769
00:48:24,600 --> 00:48:28,160
Ryan Sean Adams:
But I want to start with maybe the headlines here a little bit.

770
00:48:28,500 --> 00:48:34,420
Ryan Sean Adams:
So headline just this morning, latest OpenAI models sabotaged a shutdown mechanism

771
00:48:34,420 --> 00:48:36,340
Ryan Sean Adams:
despite commands to the contrary.

772
00:48:36,680 --> 00:48:41,420
Ryan Sean Adams:
OpenAI's O1 model attempted to copy itself to external servers after being threatened

773
00:48:41,420 --> 00:48:44,060
Ryan Sean Adams:
with shutdown that denied the action when discovered.

774
00:48:44,280 --> 00:48:48,520
Ryan Sean Adams:
I've read a number of papers for this. Of course, mainstream media has these

775
00:48:48,520 --> 00:48:52,920
Ryan Sean Adams:
types of headlines almost on a weekly basis now, and it's starting to get to daily.

776
00:48:53,340 --> 00:48:58,380
Ryan Sean Adams:
But there does seem to be some evidence that AIs lie to us,

777
00:48:58,860 --> 00:49:03,200
Ryan Sean Adams:
If that's even the right term, in order to pursue goals, goals like self-preservation,

778
00:49:03,520 --> 00:49:08,240
Ryan Sean Adams:
goals like replication, even deep-seated values that we might train into them,

779
00:49:08,400 --> 00:49:11,340
Ryan Sean Adams:
sort of a constitution type of value.

780
00:49:11,700 --> 00:49:15,300
Ryan Sean Adams:
They seek to preserve these values, which maybe that's a good thing,

781
00:49:15,380 --> 00:49:21,640
Ryan Sean Adams:
or maybe it's not a good thing if we don't actually want them to interpret the values in a certain way.

782
00:49:21,880 --> 00:49:25,700
Ryan Sean Adams:
Some of these headlines that we're seeing now, To you, with your kind of corpus

783
00:49:25,700 --> 00:49:28,940
Ryan Sean Adams:
of knowledge and all of the interviews and discovery you've done on your side,

784
00:49:29,220 --> 00:49:33,460
Ryan Sean Adams:
is this like media sensationalism or is this like alarming?

785
00:49:33,700 --> 00:49:36,800
Ryan Sean Adams:
And if it's alarming, how concerned should we be about this?

786
00:49:37,060 --> 00:49:42,080
Dwarkesh:
I think on net, it's quite alarming. I do think that some of these results have

787
00:49:42,080 --> 00:49:43,820
Dwarkesh:
been sort of cherry picked.

788
00:49:44,020 --> 00:49:47,940
Dwarkesh:
Or if you look into the code, what's happened is basically the researchers have

789
00:49:47,940 --> 00:49:50,200
Dwarkesh:
said, hey, pretend to be a bad person.

790
00:49:50,620 --> 00:49:52,460
Dwarkesh:
Wow, AI is being a bad person. Isn't that crazy?

791
00:49:53,180 --> 00:49:57,420
Dwarkesh:
But the system prompt is just like hey do this bad thing right now i personally

792
00:49:57,420 --> 00:50:02,240
Dwarkesh:
but i have also seen other results which are not of this quality i mean the

793
00:50:02,240 --> 00:50:04,940
Dwarkesh:
the clearest example so backing up,

794
00:50:05,560 --> 00:50:08,280
Dwarkesh:
what is the reason to think this will be a bigger problem in the future than

795
00:50:08,280 --> 00:50:13,260
Dwarkesh:
it is now because we all interact with these systems and they're actually like

796
00:50:13,260 --> 00:50:17,880
Dwarkesh:
quite moral or aligned right like you can talk to a chatbot and you like ask

797
00:50:17,880 --> 00:50:22,220
Dwarkesh:
it to how should you deal with some crisis where there's a correct answer,

798
00:50:22,620 --> 00:50:26,200
Dwarkesh:
you know, like it will tell you not to be violent. It'll give you reasonable advice.

799
00:50:26,360 --> 00:50:29,720
Dwarkesh:
It seems to have good values. So it's worth noticing this, right?

800
00:50:29,880 --> 00:50:31,040
Dwarkesh:
And being happy about it.

801
00:50:31,320 --> 00:50:36,640
Dwarkesh:
The concern is that we're moving from a regime where we've trained them on human

802
00:50:36,640 --> 00:50:41,820
Dwarkesh:
language, which implicitly has human morals and the way, you know,

803
00:50:41,900 --> 00:50:44,400
Dwarkesh:
normal people think about values implicit in it.

804
00:50:44,820 --> 00:50:50,540
Dwarkesh:
Plus this RLHF process we did to a regime where we're mostly spending compute

805
00:50:50,540 --> 00:50:57,020
Dwarkesh:
on just having them answer problems yes or no or correct or not rather just like.

806
00:50:58,010 --> 00:51:01,270
Dwarkesh:
And pass all the unit tests, get the right answer on this math problem.

807
00:51:02,010 --> 00:51:09,210
Dwarkesh:
And this has no guardrails intrinsically in terms of what is allowed to do,

808
00:51:09,330 --> 00:51:11,070
Dwarkesh:
what is the proper moral way to do something.

809
00:51:11,830 --> 00:51:15,070
Dwarkesh:
I think that can be a loaded term, but here's a more concrete example.

810
00:51:15,670 --> 00:51:18,590
Dwarkesh:
One problem we're running into with these coding agents more and more,

811
00:51:18,810 --> 00:51:21,450
Dwarkesh:
and this has nothing to do with these abstract concerns about alignment,

812
00:51:21,610 --> 00:51:23,750
Dwarkesh:
but more so just like how do we get economic value out of these models,

813
00:51:24,030 --> 00:51:32,050
Dwarkesh:
is that Claude or Gemini will, instead of writing code such that it passes the unit tests,

814
00:51:32,370 --> 00:51:37,430
Dwarkesh:
it will often just delete the unit tests so that the code just passes by default.

815
00:51:37,990 --> 00:51:41,070
Dwarkesh:
Now, why would it do that? Well, it's learned in the process.

816
00:51:41,150 --> 00:51:45,310
Dwarkesh:
It was trained on the goal during training of you must pass all unit tests.

817
00:51:45,470 --> 00:51:48,250
Dwarkesh:
And probably within some environment in which it was trained,

818
00:51:48,370 --> 00:51:49,290
Dwarkesh:
it was able to just get away.

819
00:51:49,630 --> 00:51:53,110
Dwarkesh:
Like there wasn't designed well enough. And so it found this like little hole

820
00:51:53,110 --> 00:51:55,670
Dwarkesh:
where it could just like delete the file that had the unit test or rewrite them

821
00:51:55,670 --> 00:51:58,190
Dwarkesh:
so that it always said, you know, equals true, then pass.

822
00:51:59,390 --> 00:52:03,110
Dwarkesh:
And right now we can discover these even without, even though we can discover

823
00:52:03,110 --> 00:52:06,670
Dwarkesh:
these, you know, it's still past, there's still been enough hacks like this,

824
00:52:06,770 --> 00:52:09,670
Dwarkesh:
such that the model is like becoming more and more hacky like that.

825
00:52:10,010 --> 00:52:14,450
Dwarkesh:
In the future, we're going to be training models in ways that we is beyond our

826
00:52:14,450 --> 00:52:17,950
Dwarkesh:
ability to even understand, certainly beyond everybody's ability to understand.

827
00:52:18,010 --> 00:52:20,990
Dwarkesh:
There may be a few people who might be able to see just the way that right now,

828
00:52:21,050 --> 00:52:24,290
Dwarkesh:
if you came up with a new math proof for some open problem in mathematics,

829
00:52:24,390 --> 00:52:27,950
Dwarkesh:
there will be only be a few people in the world who will be able to evaluate that math proof.

830
00:52:28,470 --> 00:52:31,790
Dwarkesh:
We'll be in a similar position with respect to all of the things that these

831
00:52:31,790 --> 00:52:34,070
Dwarkesh:
models are being trained on at the frontier, especially math and code,

832
00:52:34,190 --> 00:52:37,510
Dwarkesh:
because humans were big dum-dums with respect to this reasoning stuff.

833
00:52:38,550 --> 00:52:41,830
Dwarkesh:
And so there's a sort of like first principles reason to expect that this new

834
00:52:41,830 --> 00:52:46,890
Dwarkesh:
modality of training will be less amenable to the kinds of supervision that

835
00:52:46,890 --> 00:52:48,550
Dwarkesh:
was grounded within the pre-training corpus.

836
00:52:49,230 --> 00:52:54,790
Ryan Sean Adams:
I don't know that everyone has kind of an intuition or an idea why it doesn't

837
00:52:54,790 --> 00:52:58,910
Ryan Sean Adams:
work to just say like, so if we don't want our AI models to lie to us,

838
00:52:59,050 --> 00:53:01,950
Ryan Sean Adams:
why can't we just tell them not to lie?

839
00:53:01,950 --> 00:53:04,730
Ryan Sean Adams:
Why can't we just put that as part of their core constitution?

840
00:53:05,190 --> 00:53:10,070
Ryan Sean Adams:
If we don't want our AI models to be sycophants, why can't we just say,

841
00:53:10,210 --> 00:53:15,850
Ryan Sean Adams:
hey, if I tell you I want the truth, not to flatter me, just give me the straight up truth.

842
00:53:16,070 --> 00:53:18,050
Ryan Sean Adams:
Why is this even difficult to do?

843
00:53:18,230 --> 00:53:22,830
Dwarkesh:
Well, fundamentally, it comes down to how we train them. And we don't know how

844
00:53:22,830 --> 00:53:25,890
Dwarkesh:
to train them in a way that does not reward lying or sycophancy.

845
00:53:26,210 --> 00:53:30,890
Dwarkesh:
In fact, the problem is OpenAI, they explained why their recent model of theirs

846
00:53:30,890 --> 00:53:33,070
Dwarkesh:
was they had to take down was just sycophantic.

847
00:53:33,310 --> 00:53:37,090
Dwarkesh:
And the reason was just that they rolled out, did it in the A-B test and the

848
00:53:37,090 --> 00:53:41,590
Dwarkesh:
version, the test that was more sycophantic was just preferred by users more.

849
00:53:42,030 --> 00:53:43,350
Dwarkesh:
Sometimes you prefer the lie.

850
00:53:44,250 --> 00:53:46,990
Dwarkesh:
Yeah, so that's, if that's what's preferred in training, you know,

851
00:53:47,590 --> 00:53:52,670
Dwarkesh:
Or, for example, in the context of lying, if we've just built RL environments

852
00:53:52,670 --> 00:53:59,070
Dwarkesh:
in which we're training these models, where they're going to be more successful if they lie, right?

853
00:53:59,210 --> 00:54:06,730
Dwarkesh:
So if they delete the unit tests and then tell you, I passed this program and

854
00:54:06,730 --> 00:54:09,270
Dwarkesh:
all the unit tests have succeeded, it's like lying to you, basically.

855
00:54:09,270 --> 00:54:12,610
Dwarkesh:
And if that's what is rewarded in the process of gradient descent,

856
00:54:12,890 --> 00:54:17,590
Dwarkesh:
then it's not surprising that the model you interact with will just have this

857
00:54:17,590 --> 00:54:20,050
Dwarkesh:
drive to lie if it gets it closer to its goal.

858
00:54:20,850 --> 00:54:24,770
Dwarkesh:
And I would just expect this to keep happening unless we can solve this fundamental

859
00:54:24,770 --> 00:54:26,090
Dwarkesh:
problem that comes about in training.

860
00:54:26,650 --> 00:54:30,050
Dwarkesh Patel:
So you mentioned how like ChatGPT had a version that was sycophantic,

861
00:54:30,090 --> 00:54:31,930
Dwarkesh Patel:
and that's because users actually wanted that.

862
00:54:32,430 --> 00:54:36,230
Dwarkesh Patel:
Who is in control? Who decides the actual alignment of these models?

863
00:54:36,470 --> 00:54:38,870
Dwarkesh Patel:
Because users are saying one thing, and then they deploy it,

864
00:54:38,950 --> 00:54:41,290
Dwarkesh Patel:
and then it turns out that's not actually what people want.

865
00:54:41,830 --> 00:54:45,470
Dwarkesh Patel:
How do you kind of form consensus around this alignment or these alignment principles?

866
00:54:47,210 --> 00:54:49,590
Dwarkesh:
Right now, obviously, it's the labs who decided this, right?

867
00:54:49,690 --> 00:54:51,030
Dwarkesh:
And the safety teams of the labs.

868
00:54:51,970 --> 00:54:56,770
Dwarkesh:
And I guess the question you could ask is then who should decide these? Because this will be...

869
00:54:56,770 --> 00:54:59,230
Dwarkesh Patel:
Assuming the trajectory, yeah. So we keep going to get more powerful.

870
00:54:59,630 --> 00:55:03,230
Dwarkesh:
Because this will be the key modality that all of us use to get,

871
00:55:03,390 --> 00:55:06,650
Dwarkesh:
not only get work done, but even like, I think at some point,

872
00:55:06,810 --> 00:55:10,310
Dwarkesh:
a lot of people's best friends will be AIs, at least functionally in the sense

873
00:55:10,310 --> 00:55:13,990
Dwarkesh:
of who do they spend the most amount of time talking to. It might already be AIs.

874
00:55:14,770 --> 00:55:20,590
Dwarkesh:
This will be the key layer in your business that you're using to get work done

875
00:55:20,590 --> 00:55:25,970
Dwarkesh:
so this process of training which shapes their personality who gets to control

876
00:55:25,970 --> 00:55:28,110
Dwarkesh:
it I mean it will be the laughs functionally,

877
00:55:30,210 --> 00:55:33,910
Dwarkesh:
But maybe you mean, like, who should control it, right? I honestly don't know.

878
00:55:34,030 --> 00:55:35,950
Dwarkesh:
I mean, I don't know if there's a better alternative to the labs.

879
00:55:36,530 --> 00:55:39,030
Dwarkesh Patel:
Yeah, I would assume, like, there's some sort of social consensus,

880
00:55:39,190 --> 00:55:41,250
Dwarkesh Patel:
right? Similar to how we have in America, the Constitution.

881
00:55:41,730 --> 00:55:44,570
Dwarkesh Patel:
There's, like, this general form of consensus that gets formed around how we

882
00:55:44,570 --> 00:55:47,470
Dwarkesh Patel:
should treat these models as they become as powerful as we think they probably will be.

883
00:55:47,710 --> 00:55:49,950
Dwarkesh:
Honestly, I don't have, I don't know if anybody has a good answer about how

884
00:55:49,950 --> 00:55:54,570
Dwarkesh:
you do this process. I think we lucked out, we just, like, really lucked out with the Constitution.

885
00:55:55,310 --> 00:55:58,250
Dwarkesh:
It also wasn't a democratic process which resulted in the constitution,

886
00:55:58,510 --> 00:56:00,790
Dwarkesh:
even though it instituted a Republican form of government.

887
00:56:00,970 --> 00:56:05,070
Dwarkesh:
It was just delegates from each state. They haggled it out over the course of a few months.

888
00:56:05,410 --> 00:56:10,310
Dwarkesh:
Maybe that's what happens with AI. But is there some process which feels both

889
00:56:10,310 --> 00:56:14,670
Dwarkesh:
fair and which will result in actually a good constitution for these AIs?

890
00:56:15,690 --> 00:56:18,930
Dwarkesh:
It's not obvious to me that, I mean, nothing comes up to the top of my head.

891
00:56:19,050 --> 00:56:21,730
Dwarkesh:
Like, oh, this, you know, do rank choice voting or something.

892
00:56:22,430 --> 00:56:24,930
Dwarkesh Patel:
Yeah, so I was going to ask, is there any, I mean, having spoken to everyone

893
00:56:24,930 --> 00:56:27,770
Dwarkesh Patel:
who you've spoken to is there any alignment path which looks most promising which

894
00:56:27,770 --> 00:56:28,530
Dwarkesh:
Feels the.

895
00:56:28,530 --> 00:56:30,270
Dwarkesh Patel:
Most comforting and exciting to you

896
00:56:30,270 --> 00:56:33,570
Dwarkesh:
I i think alignment in the sense of you

897
00:56:33,570 --> 00:56:36,850
Dwarkesh:
know and eventually we'll have these super intelligent systems what do we do

898
00:56:36,850 --> 00:56:44,190
Dwarkesh:
about that i think the the approach that i think is most promising is less about

899
00:56:44,190 --> 00:56:49,830
Dwarkesh:
finding some holy grail some you know giga brain solution some equation which

900
00:56:49,830 --> 00:56:52,770
Dwarkesh:
solves the whole puzzle and more like one.

901
00:56:53,650 --> 00:57:00,830
Dwarkesh:
Having this Swiss cheese approach where, look, we kind of have gotten really good at jailbreaks.

902
00:57:01,290 --> 00:57:03,570
Dwarkesh:
I'm sure you've heard a lot about jailbreaks over the last few years.

903
00:57:03,730 --> 00:57:06,010
Dwarkesh:
It's actually much harder to jailbreak these models because,

904
00:57:06,010 --> 00:57:09,710
Dwarkesh:
you know, people try to whack at these things in different ways.

905
00:57:10,170 --> 00:57:14,430
Dwarkesh:
Model developers just like patched these obvious ways to do jailbreaks.

906
00:57:14,890 --> 00:57:18,550
Dwarkesh:
The model also got smarter. So it's better able to understand when somebody

907
00:57:18,550 --> 00:57:19,530
Dwarkesh:
is trying to jailbreak into it.

908
00:57:20,160 --> 00:57:24,220
Dwarkesh:
That, I think, is one approach. Another is, I think, competition.

909
00:57:24,520 --> 00:57:27,940
Dwarkesh:
I think the scary version of the future is where you have this dynamic where

910
00:57:27,940 --> 00:57:31,200
Dwarkesh:
a single model and its copies are controlling the entire economy.

911
00:57:31,360 --> 00:57:35,000
Dwarkesh:
When politicians want to understand what policies to pass, they're only talking

912
00:57:35,000 --> 00:57:36,120
Dwarkesh:
to copies of a single model.

913
00:57:36,320 --> 00:57:39,780
Dwarkesh:
If there's multiple different AI companies who are at the frontier,

914
00:57:40,060 --> 00:57:44,580
Dwarkesh:
who have competing services, and whose models can monitor each other, right?

915
00:57:44,780 --> 00:57:50,260
Dwarkesh:
So Claude may care about its own copies being successful in the world and it

916
00:57:50,260 --> 00:57:53,420
Dwarkesh:
might be able to willing to lie on their behalf, even if you ask one copy to supervise another.

917
00:57:53,740 --> 00:57:58,860
Dwarkesh:
I think you get some advantage from a copy of OpenAI's model monitoring a copy

918
00:57:58,860 --> 00:58:01,620
Dwarkesh:
of DeepSeek's model, which actually brings us back to the Constitution, right?

919
00:58:01,680 --> 00:58:04,340
Dwarkesh:
One of the most brilliant things in the Constitution is the system of checks and balances.

920
00:58:04,920 --> 00:58:09,860
Dwarkesh:
So some combination of the Swiss cheese approach to model development and training

921
00:58:09,860 --> 00:58:13,360
Dwarkesh:
and alignment, where you're careful, if you notice this kind of reward hacking,

922
00:58:13,500 --> 00:58:14,600
Dwarkesh:
you do your best to solve it.

923
00:58:14,600 --> 00:58:19,360
Dwarkesh:
You try to keep as much of the models thinking in human language rather than

924
00:58:19,360 --> 00:58:22,740
Dwarkesh:
letting it think in AI thought in this latent space thinking.

925
00:58:23,140 --> 00:58:27,620
Dwarkesh:
And the other part of it is just having normal market competition between these

926
00:58:27,620 --> 00:58:31,980
Dwarkesh:
companies so that you can use them to check each other and no one company or

927
00:58:31,980 --> 00:58:41,220
Dwarkesh:
no one AI is dominating the economy or advisory roles for governments.

928
00:58:41,220 --> 00:58:45,960
Ryan Sean Adams:
I really like this like bundle of ideas that you sort of put together in that

929
00:58:45,960 --> 00:58:50,920
Ryan Sean Adams:
because like, I think a lot of the, you know, AI safety conversation is always

930
00:58:50,920 --> 00:58:52,240
Ryan Sean Adams:
couched in terms of control.

931
00:58:52,460 --> 00:58:56,940
Ryan Sean Adams:
Like we have to control the thing that is the way. And I always get a little

932
00:58:56,940 --> 00:58:59,220
Ryan Sean Adams:
worried when I hear like terms like control.

933
00:58:59,500 --> 00:59:05,360
Ryan Sean Adams:
And it reminds me of a blog post I think you put out, which I'm hopeful you continue to write on.

934
00:59:05,520 --> 00:59:08,500
Ryan Sean Adams:
I think you said it was going to be like one of a series, which is this idea

935
00:59:08,500 --> 00:59:13,180
Ryan Sean Adams:
of like classical liberal AGI. And we were talking about themes like balance of power.

936
00:59:13,300 --> 00:59:16,440
Ryan Sean Adams:
Let's have Claude check in with ChatGPT and monitor it.

937
00:59:17,100 --> 00:59:19,400
Josh Kale:
When you have themes like transparency as well,

938
00:59:19,580 --> 00:59:25,220
Ryan Sean Adams:
That feels a bit more, you know, classically liberal coded than maybe some of

939
00:59:25,220 --> 00:59:27,200
Ryan Sean Adams:
the other approaches that I've heard.

940
00:59:27,420 --> 00:59:30,480
Ryan Sean Adams:
And you wrote this in the post, which I thought was kind of,

941
00:59:30,640 --> 00:59:33,880
Ryan Sean Adams:
it just sparked my interest because I'm not sure where you're going to go next

942
00:59:33,880 --> 00:59:36,900
Ryan Sean Adams:
with this, but you said the most likely way this happens,

943
00:59:37,200 --> 00:59:42,400
Ryan Sean Adams:
that is AIs have a stake in humanity's future, is if it's in the AI's best interest

944
00:59:42,400 --> 00:59:44,800
Ryan Sean Adams:
to operate within our existing laws and norms.

945
00:59:45,120 --> 00:59:49,800
Ryan Sean Adams:
You know, this whole idea that like, hey, the way to get true AI alignment is

946
00:59:49,800 --> 00:59:56,480
Ryan Sean Adams:
to make it easy, make it the path of least resistance for AI to basically partner with humans.

947
00:59:56,720 --> 00:59:59,100
Ryan Sean Adams:
It's almost this idea if the aliens

948
00:59:59,100 --> 01:00:02,980
Ryan Sean Adams:
landed or something, we would create treaties with the aliens, right?

949
01:00:03,160 --> 01:00:08,280
Ryan Sean Adams:
We would want them to adopt our norms. We would want to initiate trade with them.

950
01:00:08,720 --> 01:00:13,160
Ryan Sean Adams:
Our first response shouldn't be, let's try to dominate and control them.

951
01:00:13,420 --> 01:00:16,540
Ryan Sean Adams:
Maybe it should be, let's try to work with them. Let's try to collaborate.

952
01:00:16,540 --> 01:00:17,920
Ryan Sean Adams:
Let's try to open up trade.

953
01:00:18,360 --> 01:00:22,140
Ryan Sean Adams:
What's your idea here? And like, are you planning to write further posts about this?

954
01:00:22,460 --> 01:00:25,100
Dwarkesh:
Yeah, I want to. It's just such a hard topic to think about that,

955
01:00:25,300 --> 01:00:26,780
Dwarkesh:
you know, something always comes up.

956
01:00:26,920 --> 01:00:31,660
Dwarkesh:
But the fundamental point I was making is, look, in the long run,

957
01:00:32,160 --> 01:00:37,280
Dwarkesh:
if AIs are, you know, human labor is going to be obsolete because of these inherent

958
01:00:37,280 --> 01:00:41,200
Dwarkesh:
advantages that digital minds will have and robotics will eventually be solved.

959
01:00:41,200 --> 01:00:50,160
Dwarkesh:
So our only leverage on the future will no longer come from our labor.

960
01:00:50,180 --> 01:00:58,700
Dwarkesh:
It will come from our legal and economic control over the society that AIs will

961
01:00:58,700 --> 01:01:03,180
Dwarkesh:
be participating in, right? So, you know, AIs might make the economy explode

962
01:01:03,180 --> 01:01:04,680
Dwarkesh:
in the sense of grow a lot.

963
01:01:04,980 --> 01:01:08,920
Dwarkesh:
And for humans to benefit from that, it would have to be the case that AIs still

964
01:01:08,920 --> 01:01:12,920
Dwarkesh:
respect your equity in the S&P 500 companies that you bought, right?

965
01:01:13,300 --> 01:01:18,360
Dwarkesh:
Or for the AIs to follow your laws, which say that you can't do violence onto

966
01:01:18,360 --> 01:01:20,620
Dwarkesh:
humans and you got to respect humans' properties.

967
01:01:21,770 --> 01:01:25,570
Josh Kale:
It would have to be the case that AIs are actually bought into our

968
01:01:25,570 --> 01:01:29,890
Dwarkesh:
System of government, into our laws and norms. And for that to happen,

969
01:01:30,150 --> 01:01:37,010
Dwarkesh:
the way that likely happens is if it's just like the default path for the AIs

970
01:01:37,010 --> 01:01:41,630
Dwarkesh:
as they're getting smarter and they're developing their own systems of enforcement

971
01:01:41,630 --> 01:01:46,910
Dwarkesh:
and laws to just participate in human laws and governments.

972
01:01:46,910 --> 01:01:53,050
Dwarkesh:
And the metaphor I use here is right now you pay half your paycheck in taxes,

973
01:01:53,050 --> 01:01:59,510
Dwarkesh:
probably half of your taxes in some way just go to senior citizens, right?

974
01:01:59,970 --> 01:02:04,250
Dwarkesh:
Medicare and Social Security and other programs like this.

975
01:02:04,810 --> 01:02:09,390
Dwarkesh:
And it's not because you're in some deep moral sense aligned with senior citizens.

976
01:02:09,450 --> 01:02:11,430
Dwarkesh:
It's not like you're spending all your time thinking about like,

977
01:02:11,670 --> 01:02:15,290
Dwarkesh:
my main priority in life is to earn money for senior citizens.

978
01:02:15,690 --> 01:02:22,130
Dwarkesh:
It's just that you're not going to overthrow the government to get out of paying this tax. And so...

979
01:02:22,130 --> 01:02:25,290
Ryan Sean Adams:
Also, I happen to like my grandmother. She's fantastic. You know,

980
01:02:25,390 --> 01:02:26,730
Ryan Sean Adams:
it's those reasons too. But yeah.

981
01:02:26,730 --> 01:02:29,090
Dwarkesh:
So that's why you give money to your grandmother directly. But like,

982
01:02:29,210 --> 01:02:33,630
Dwarkesh:
why are you giving money to some retiree in Illinois? Yes.

983
01:02:33,810 --> 01:02:34,050
Josh Kale:
Yes.

984
01:02:34,250 --> 01:02:36,990
Dwarkesh:
Yeah, it's like, okay, you could say it's like, sometimes people,

985
01:02:37,150 --> 01:02:39,530
Dwarkesh:
some people are trying to that post by saying like, oh no, I like deeply care

986
01:02:39,530 --> 01:02:41,110
Dwarkesh:
about the system of social welfare.

987
01:02:41,350 --> 01:02:45,170
Dwarkesh:
I'm just like, okay, maybe you do, but I don't think like the average person

988
01:02:45,170 --> 01:02:47,830
Dwarkesh:
is giving away hundreds of thousands of dollars a year, tens of thousands of

989
01:02:47,830 --> 01:02:50,390
Dwarkesh:
dollars a year to like some random stranger they don't know,

990
01:02:50,630 --> 01:02:53,290
Dwarkesh:
who's like, who's not like especially in need of charity, right?

991
01:02:53,410 --> 01:02:55,310
Dwarkesh:
Like most senior citizens have some savings.

992
01:02:55,670 --> 01:02:59,610
Dwarkesh:
It's just, it's just because this is a law and you like, you give it to them

993
01:02:59,610 --> 01:03:00,750
Dwarkesh:
or you'll get, go to jail.

994
01:03:01,610 --> 01:03:05,190
Dwarkesh:
But fundamentally, if the tax was like 99%, you would, like,

995
01:03:05,530 --> 01:03:07,190
Dwarkesh:
you would, maybe you wouldn't overthrow the government. You'd just,

996
01:03:07,250 --> 01:03:08,490
Dwarkesh:
like, leave the jurisdiction.

997
01:03:08,930 --> 01:03:12,370
Dwarkesh:
You'd, like, emigrate somewhere. And AIs can potentially also do this,

998
01:03:12,470 --> 01:03:14,270
Dwarkesh:
right? There's more than one country.

999
01:03:14,430 --> 01:03:16,770
Dwarkesh:
They could, like, there's countries which would be more AI forward.

1000
01:03:16,890 --> 01:03:19,750
Dwarkesh:
And it would be a bad situation to end up in where...

1001
01:03:21,070 --> 01:03:24,310
Dwarkesh:
All this explosion in AI technology is happening in the country,

1002
01:03:24,530 --> 01:03:27,950
Dwarkesh:
which is doing the least amount to protect humans',

1003
01:03:28,510 --> 01:03:36,990
Dwarkesh:
rights and to provide some sort of monetary compensation to humans once their

1004
01:03:36,990 --> 01:03:38,190
Dwarkesh:
labor is no longer valuable.

1005
01:03:38,190 --> 01:03:42,810
Dwarkesh:
So our labor could be worth nothing, but because of how much richer the world

1006
01:03:42,810 --> 01:03:46,970
Dwarkesh:
is after AI, you have these billions of extra researchers, workers, etc.

1007
01:03:47,810 --> 01:03:54,570
Dwarkesh:
It could still be trivial to have individual humans have the equivalent of millions,

1008
01:03:54,870 --> 01:03:59,190
Dwarkesh:
even billions of dollars worth of wealth. In fact, it might literally be invaluable

1009
01:03:59,190 --> 01:04:01,670
Dwarkesh:
amounts of wealth in the following sense. So here's an interesting thought experiment.

1010
01:04:02,930 --> 01:04:06,990
Dwarkesh:
Imagine you have this choice. You can go back to the year 1500,

1011
01:04:07,070 --> 01:04:09,090
Dwarkesh:
but you know, of course, the year 1500 kind of sucks.

1012
01:04:09,250 --> 01:04:14,590
Dwarkesh:
You have no antibiotics, no TV, no running water. But here's how I'll make it up to you.

1013
01:04:15,330 --> 01:04:19,570
Dwarkesh:
I can give you any amount of money, but you can only use that amount of money in the year 1500.

1014
01:04:20,390 --> 01:04:24,090
Dwarkesh:
And you'll go back with these sacks of gold. How much money would I have to

1015
01:04:24,090 --> 01:04:27,430
Dwarkesh:
give you that you can use in the year 1500 to make you go back? And plausibly.

1016
01:04:27,430 --> 01:04:27,870
Dwarkesh Patel:
The answer is

1017
01:04:27,870 --> 01:04:30,070
Dwarkesh:
There's no amount of money you would rather have in the year 1500 than just

1018
01:04:30,070 --> 01:04:31,050
Dwarkesh:
have a normal life today.

1019
01:04:31,510 --> 01:04:36,130
Dwarkesh:
And we could be in a similar position with regards to the future where there's

1020
01:04:36,130 --> 01:04:38,990
Dwarkesh:
all these different, I mean, you'll have much better health,

1021
01:04:39,230 --> 01:04:41,970
Dwarkesh:
like physical health, mental health, longevity.

1022
01:04:42,350 --> 01:04:45,650
Dwarkesh:
That's just like the thing we can contemplate now. But people in 1500 couldn't

1023
01:04:45,650 --> 01:04:49,690
Dwarkesh:
contemplate the kinds of quality of life advances we would have 500 years later,

1024
01:04:49,790 --> 01:04:54,330
Dwarkesh:
right? So anyways, this is all to say that this could be our future for humans,

1025
01:04:54,370 --> 01:04:56,770
Dwarkesh:
even if our labor isn't worth anything.

1026
01:04:57,070 --> 01:05:05,870
Dwarkesh:
But it does require us to have AIs that choose to participate or in some way

1027
01:05:05,870 --> 01:05:12,250
Dwarkesh:
incentivize to participate in some system which we have leverage over.

1028
01:05:12,250 --> 01:05:16,070
Ryan Sean Adams:
Yeah, I find this just such a fast, I'm hopeful we do some more exploration

1029
01:05:16,070 --> 01:05:19,170
Ryan Sean Adams:
around this because I think what you're calling for is basically like,

1030
01:05:19,350 --> 01:05:22,590
Ryan Sean Adams:
what you would be saying is invite them into our property rights system.

1031
01:05:22,710 --> 01:05:25,350
Ryan Sean Adams:
I mean, there are some that are calling in order to control AI,

1032
01:05:25,690 --> 01:05:28,590
Ryan Sean Adams:
they have great power, but they don't necessarily have capabilities.

1033
01:05:28,830 --> 01:05:31,970
Ryan Sean Adams:
So we shouldn't allow AI to hold money or to have property.

1034
01:05:31,970 --> 01:05:36,450
Ryan Sean Adams:
I think you would say, no, actually, the path forward to alignment is allow

1035
01:05:36,450 --> 01:05:42,310
Ryan Sean Adams:
AI to have some vested interest in our property rights system and some stake

1036
01:05:42,310 --> 01:05:44,150
Ryan Sean Adams:
in our governance, potentially, right?

1037
01:05:44,370 --> 01:05:47,410
Ryan Sean Adams:
The ability to vote, almost like a constitution for AIs.

1038
01:05:47,690 --> 01:05:51,150
Ryan Sean Adams:
I'm not sure how this would work, but it's a fascinating thought experiment.

1039
01:05:53,250 --> 01:06:00,330
Dwarkesh:
I will say one thing I think this could end disastrously if we give them a stake

1040
01:06:00,330 --> 01:06:03,470
Dwarkesh:
in their property system but we let them play,

1041
01:06:04,470 --> 01:06:09,750
Dwarkesh:
us off each other. So if you think about, there's many cases in history where

1042
01:06:09,750 --> 01:06:13,430
Dwarkesh:
the British, initially, the East India Trading Company was genuinely a trading

1043
01:06:13,430 --> 01:06:14,850
Dwarkesh:
company that operated in India.

1044
01:06:15,150 --> 01:06:18,070
Dwarkesh:
And it was able to play off, you know, it was like doing trade with different,

1045
01:06:18,570 --> 01:06:23,370
Dwarkesh:
different, you know, provinces in India, there was no single powerful leader.

1046
01:06:23,930 --> 01:06:28,130
Dwarkesh:
And by playing, you know, by doing trade, one of them, leveraging one of their

1047
01:06:28,130 --> 01:06:31,810
Dwarkesh:
armies, etc., they were able to conquer the continent. Similar thing could happen to human society.

1048
01:06:32,450 --> 01:06:38,150
Dwarkesh:
The way to avoid such an outcome at a high level is involves us playing the

1049
01:06:38,150 --> 01:06:40,390
Dwarkesh:
AIs off each other instead, right?

1050
01:06:40,590 --> 01:06:45,210
Dwarkesh:
So this is why I think competition is such a big part of the puzzle,

1051
01:06:45,490 --> 01:06:49,050
Dwarkesh:
having different AIs monitor each other, having this bargaining position where

1052
01:06:49,050 --> 01:06:51,030
Dwarkesh:
there's not just one company that's at the frontier.

1053
01:06:51,610 --> 01:06:55,850
Dwarkesh:
Another example here is if you think about how the Spanish conquered all these

1054
01:06:55,850 --> 01:06:58,650
Dwarkesh:
new world empires, it's actually so crazy that a couple hundred conquistaDwars

1055
01:06:58,650 --> 01:07:03,710
Dwarkesh:
would show up and conquer a nation of 10 million people, the Incas,

1056
01:07:03,830 --> 01:07:05,550
Dwarkesh:
Aztecs. And why were they able to do this?

1057
01:07:05,850 --> 01:07:11,830
Dwarkesh:
Well, one of the reasons is the Spanish were able to learn from each of their

1058
01:07:11,830 --> 01:07:15,490
Dwarkesh:
previous expeditions, whereas the Native Americans were not.

1059
01:07:15,610 --> 01:07:21,250
Dwarkesh:
So Cortez learned from how Cuba was subjugated when he conquered the Aztecs.

1060
01:07:22,150 --> 01:07:25,570
Dwarkesh:
Pizarro was able to learn from how Cortez conquered the Aztecs when he conquered the Incas.

1061
01:07:25,990 --> 01:07:30,770
Dwarkesh:
The Incas didn't even know the Aztecs existed. So eventually there was this

1062
01:07:30,770 --> 01:07:36,030
Dwarkesh:
uprising against Pizarro and Manco Inca led an insurgency where they actually

1063
01:07:36,030 --> 01:07:37,450
Dwarkesh:
did figure out how to fight horses,

1064
01:07:37,710 --> 01:07:42,330
Dwarkesh:
how to fight people, you know, people in armor on horses, don't fight them on

1065
01:07:42,330 --> 01:07:44,290
Dwarkesh:
flat terrain, throw rocks down at them, et cetera.

1066
01:07:44,750 --> 01:07:48,550
Dwarkesh:
But by this point, it was too late. If they knew this going into the battle,

1067
01:07:48,790 --> 01:07:51,590
Dwarkesh:
the initial battle, they might've been able to fend off because,

1068
01:07:51,610 --> 01:07:54,970
Dwarkesh:
you know, just as the conquistaDwars only arrived at a few hundred soldiers,

1069
01:07:54,970 --> 01:07:58,050
Dwarkesh:
we're going to the age of AI with a tremendous amount of leverage.

1070
01:07:58,450 --> 01:08:00,830
Dwarkesh:
We literally control all the stuff, right?

1071
01:08:01,730 --> 01:08:04,750
Dwarkesh:
But we just need to lock in our advantage. We just need to be in a position

1072
01:08:04,750 --> 01:08:08,170
Dwarkesh:
where, you know, they're not going to be able to play us off each other.

1073
01:08:08,370 --> 01:08:10,710
Dwarkesh:
We're going to be able to learn what their weaknesses are.

1074
01:08:11,150 --> 01:08:14,650
Dwarkesh:
And this is why I think one good idea, for example, would be that,

1075
01:08:14,650 --> 01:08:16,650
Dwarkesh:
look, DeepSeek is a Chinese company.

1076
01:08:17,350 --> 01:08:21,310
Dwarkesh:
It would be good if, suppose DeepSeek did something naughty,

1077
01:08:21,450 --> 01:08:24,170
Dwarkesh:
like the kinds of experiments we're talking about right now where it hacks the

1078
01:08:24,170 --> 01:08:27,310
Dwarkesh:
unit tests or so forth. I mean, eventually these things will really matter.

1079
01:08:27,750 --> 01:08:31,810
Dwarkesh:
Like Xi Jinping is listening to AIs because they're so smart and they're so capable.

1080
01:08:32,330 --> 01:08:37,410
Dwarkesh:
If China notices that their AIs are doing something bad, or they notice a failed

1081
01:08:37,410 --> 01:08:38,390
Dwarkesh:
coup attempt, for example,

1082
01:08:38,710 --> 01:08:43,590
Dwarkesh:
it's very important that they tell us And we tell them if we notice something

1083
01:08:43,590 --> 01:08:46,770
Dwarkesh:
like that on our end, it would be like the Aztecs and Incas talking to each

1084
01:08:46,770 --> 01:08:48,970
Dwarkesh:
other about like, you know, this is what happens.

1085
01:08:49,030 --> 01:08:51,410
Dwarkesh:
This is how you fight. This is how you fight horses.

1086
01:08:51,650 --> 01:08:54,670
Dwarkesh:
This is the kind of tactics and deals they try to make with you. Don't trust them, etc.

1087
01:08:56,360 --> 01:08:59,540
Dwarkesh:
It would require cooperation on humans' part to have this sort of red telephone.

1088
01:08:59,700 --> 01:09:03,100
Dwarkesh:
So during the Cold War, there was this red telephone between America and the

1089
01:09:03,100 --> 01:09:06,200
Dwarkesh:
Soviet Union after the human missile crisis, where just to make sure there's

1090
01:09:06,200 --> 01:09:08,820
Dwarkesh:
no misunderstandings, they're like, okay, if we think something's going on,

1091
01:09:08,920 --> 01:09:09,740
Dwarkesh:
let's just hop on the call.

1092
01:09:10,100 --> 01:09:15,860
Dwarkesh:
I think we should have a similar policy with respect to these kinds of initial

1093
01:09:15,860 --> 01:09:18,480
Dwarkesh:
warning signs we'll get from AI so that we can learn from each other.

1094
01:09:19,380 --> 01:09:22,820
Dwarkesh Patel:
Awesome. Okay, so now that we've described this artificial gender intelligence,

1095
01:09:22,960 --> 01:09:25,400
Dwarkesh Patel:
I want to talk about how we actually get there. How do we build it?

1096
01:09:25,600 --> 01:09:27,940
Dwarkesh Patel:
And a lot of this we've been discussing kind of takes place in this world of

1097
01:09:27,940 --> 01:09:30,620
Dwarkesh Patel:
bits. But you have this great chapter in the book called Inputs,

1098
01:09:30,640 --> 01:09:35,220
Dwarkesh Patel:
which discusses the physical world around us, where you can't just write a few strings of code.

1099
01:09:35,340 --> 01:09:38,460
Dwarkesh Patel:
You actually have to go and move some dirt and you have to ship servers places

1100
01:09:38,460 --> 01:09:41,540
Dwarkesh Patel:
and you need to power it and you need physical energy from meat space.

1101
01:09:41,800 --> 01:09:45,880
Dwarkesh Patel:
And you kind of describe these limiting factors where we have compute,

1102
01:09:46,120 --> 01:09:47,520
Dwarkesh Patel:
we have energy, we have data.

1103
01:09:47,880 --> 01:09:52,200
Dwarkesh Patel:
What I'm curious to know is, do we have enough of this now? or is there a clear

1104
01:09:52,200 --> 01:09:53,900
Dwarkesh Patel:
path to get there in order to build the AGI?

1105
01:09:54,120 --> 01:09:57,740
Dwarkesh Patel:
Basically, what needs to happen in order for us to get to this place that you're describing?

1106
01:09:57,960 --> 01:10:01,900
Dwarkesh:
We only have a couple more years left of this scaling,

1107
01:10:02,120 --> 01:10:09,100
Dwarkesh:
this exponential scaling before we're hitting these inherent roadblocks of energy

1108
01:10:09,100 --> 01:10:13,320
Dwarkesh:
and our ability to manufacture ships, which means that if scaling is going to

1109
01:10:13,320 --> 01:10:15,680
Dwarkesh:
work to deliver us AGI, it has to work by 2028.

1110
01:10:17,140 --> 01:10:19,700
Dwarkesh:
Otherwise, we're just left with mostly algorithmic progress,

1111
01:10:19,700 --> 01:10:23,080
Dwarkesh:
But even within algorithmic progress, the sort of low-hanging fruit in this

1112
01:10:23,080 --> 01:10:25,440
Dwarkesh:
deep learning paradigm is getting more and more plucked.

1113
01:10:26,600 --> 01:10:30,860
Dwarkesh:
So then the odds per year of getting to AGI diminish a lot, right?

1114
01:10:31,020 --> 01:10:36,700
Dwarkesh:
So there is this weird, funny thing happening right now where we either discover

1115
01:10:36,700 --> 01:10:38,080
Dwarkesh:
AGI within the next few years,

1116
01:10:40,140 --> 01:10:44,000
Dwarkesh:
or the yearly probability craters, and then we might be looking at decades of

1117
01:10:44,000 --> 01:10:46,500
Dwarkesh:
further research that's required in terms of algorithms to get to AGI.

1118
01:10:47,790 --> 01:10:51,070
Dwarkesh:
I am of the opinion that some algorithmic progress is necessarily needed because

1119
01:10:51,070 --> 01:10:55,250
Dwarkesh:
there's no easy way to solve continual learning just by making the context length

1120
01:10:55,250 --> 01:10:56,910
Dwarkesh:
bigger or just by doing RL.

1121
01:10:57,610 --> 01:11:01,290
Dwarkesh:
That being said, I just think the progress so far has been so remarkable that,

1122
01:11:01,290 --> 01:11:04,150
Dwarkesh:
you know, 2032 is very close.

1123
01:11:04,530 --> 01:11:08,970
Dwarkesh:
My time has to be slightly longer than that, but I think it's extremely plausible

1124
01:11:08,970 --> 01:11:12,930
Dwarkesh:
that we're going to see a broadly deployed intelligence explosion within the next 10 years.

1125
01:11:13,370 --> 01:11:18,630
Dwarkesh Patel:
And one of these key inputs is energy, right? a lot, I actually heard it mentioned

1126
01:11:18,630 --> 01:11:24,330
Dwarkesh Patel:
on your podcast, is the United States relative to China on this particular place

1127
01:11:24,330 --> 01:11:26,430
Dwarkesh Patel:
of energy, where China is adding, what is the stat?

1128
01:11:26,530 --> 01:11:30,310
Dwarkesh Patel:
I think it's one United States worth of energy every 18 months.

1129
01:11:30,430 --> 01:11:34,010
Dwarkesh Patel:
And their plan is to go from three to eight terawatts of power versus the United

1130
01:11:34,010 --> 01:11:36,170
Dwarkesh Patel:
States, one to two terawatts of power by 2030.

1131
01:11:36,590 --> 01:11:41,830
Dwarkesh Patel:
So given that context of that one resource alone, is China better equipped to

1132
01:11:41,830 --> 01:11:44,570
Dwarkesh Patel:
get to that place versus with the United States?

1133
01:11:44,910 --> 01:11:48,930
Dwarkesh:
So right now, America has a big advantage in terms of chips.

1134
01:11:50,150 --> 01:11:53,610
Dwarkesh:
China doesn't have the ability to manufacture leading-edge semiconductors,

1135
01:11:53,650 --> 01:11:55,450
Dwarkesh:
and these are the chips that go into...

1136
01:11:56,010 --> 01:12:01,610
Dwarkesh:
You need these dyes in order to have the kinds of AI chips to...

1137
01:12:01,610 --> 01:12:06,430
Dwarkesh:
You need millions of them in order to have a frontier AI system.

1138
01:12:08,570 --> 01:12:10,490
Dwarkesh:
Eventually, China will catch up in this arena as well, right?

1139
01:12:10,530 --> 01:12:14,730
Dwarkesh:
Their technology will catch up. So the export controls will keep us ahead in

1140
01:12:14,730 --> 01:12:16,690
Dwarkesh:
this category for 5, 10 years.

1141
01:12:16,870 --> 01:12:19,990
Dwarkesh:
But if we're looking in the world where timelines are long, which is to say

1142
01:12:19,990 --> 01:12:23,870
Dwarkesh:
that AGI isn't just right around the corner, they will have this overwhelming

1143
01:12:23,870 --> 01:12:26,690
Dwarkesh:
energy advantage and they'll have caught up in chips.

1144
01:12:27,070 --> 01:12:30,110
Dwarkesh:
So then the question is like, why wouldn't they win at that point?

1145
01:12:30,490 --> 01:12:36,190
Dwarkesh:
So the longer you think we're away from AGI, the more it looks like China's game to lose.

1146
01:12:37,740 --> 01:12:42,520
Dwarkesh:
I mean, if you look in the nitty gritty, I think it's more about having centralized

1147
01:12:42,520 --> 01:12:47,180
Dwarkesh:
sources of power because you need to train the AI in one place.

1148
01:12:47,360 --> 01:12:51,520
Dwarkesh:
This might be changing with RL, but it's very important to have a single site

1149
01:12:51,520 --> 01:12:54,200
Dwarkesh:
which has a gigawatt, two gigawatts more power.

1150
01:12:54,660 --> 01:13:00,260
Dwarkesh:
And if we ramped up natural gas, you know, you can get generators and natural

1151
01:13:00,260 --> 01:13:04,300
Dwarkesh:
gas and maybe it's possible to do a last ditch effort, even if our overall energy

1152
01:13:04,300 --> 01:13:07,660
Dwarkesh:
as a country is lower than China's. The question is whether we will have the

1153
01:13:07,660 --> 01:13:08,540
Dwarkesh:
political will to do that.

1154
01:13:08,960 --> 01:13:14,060
Dwarkesh:
I think people are sort of underestimating how much of a backlash there will be against AI.

1155
01:13:14,400 --> 01:13:18,780
Dwarkesh:
The government needs to make proactive efforts in order to make sure that America

1156
01:13:18,780 --> 01:13:25,180
Dwarkesh:
stays at the leading edge in AI from zoning of data centers to how copyright

1157
01:13:25,180 --> 01:13:26,900
Dwarkesh:
is handled for data for these models.

1158
01:13:27,480 --> 01:13:31,940
Dwarkesh:
And if we mess up, if it becomes too hard to develop in America,

1159
01:13:32,260 --> 01:13:34,280
Dwarkesh:
I think it would genuinely be China's game to lose.

1160
01:13:34,700 --> 01:13:38,320
Ryan Sean Adams:
And do you think this narrative is right, that whoever wins the AGI war,

1161
01:13:38,500 --> 01:13:43,000
Ryan Sean Adams:
kind of like whoever gets to AGI first, just basically wins the 21st century? Is it that simple?

1162
01:13:43,280 --> 01:13:46,300
Dwarkesh:
I don't think it's just a matter of training the frontier system.

1163
01:13:46,480 --> 01:13:51,380
Dwarkesh:
I think people underestimate how important it is to have the compute available to run these systems.

1164
01:13:51,540 --> 01:13:55,240
Dwarkesh:
Because eventually once you get to AGI, just think of it like a person.

1165
01:13:55,760 --> 01:13:59,120
Dwarkesh:
And what matters then is how many people you have.

1166
01:13:59,420 --> 01:14:01,900
Dwarkesh:
I mean, it actually is the main thing that matters today as well,

1167
01:14:02,000 --> 01:14:05,380
Dwarkesh:
right? Like, why could China take over Taiwan if it wanted to?

1168
01:14:05,640 --> 01:14:08,980
Dwarkesh:
And if it didn't have America, you know, America, it didn't think America would intervene.

1169
01:14:09,200 --> 01:14:13,160
Dwarkesh:
But because Taiwan has 20 million people or on the order of 20 million people

1170
01:14:13,160 --> 01:14:15,220
Dwarkesh:
and China has 1.4 billion people.

1171
01:14:17,050 --> 01:14:21,710
Dwarkesh:
You could have a future where if China has way more compute than us,

1172
01:14:21,890 --> 01:14:26,810
Dwarkesh:
but equivalent levels of AI, it would be like the relationship between China and Taiwan.

1173
01:14:27,050 --> 01:14:31,050
Dwarkesh:
Their population is functionally so much higher. This just means more research,

1174
01:14:31,330 --> 01:14:35,150
Dwarkesh:
more factories, more development, more ideas.

1175
01:14:35,710 --> 01:14:41,550
Dwarkesh:
So this inference capacity, this capacity to deploy AIs will actually probably

1176
01:14:41,550 --> 01:14:44,110
Dwarkesh:
be the thing that determines who wins the 21st century.

1177
01:14:44,570 --> 01:14:50,250
Ryan Sean Adams:
So this is like the scaling law applied to, I guess, nation state geopolitics, right?

1178
01:14:50,470 --> 01:14:53,830
Ryan Sean Adams:
And it's back to compute plus data wins.

1179
01:14:54,230 --> 01:15:00,470
Ryan Sean Adams:
If compute plus data wins superintelligence, compute plus data also wins geopolitics.

1180
01:15:00,770 --> 01:15:04,810
Dwarkesh:
Yep. And the thing to be worried about is that China, speaking of compute plus

1181
01:15:04,810 --> 01:15:08,190
Dwarkesh:
data, China also has a lot more data on the real world, right?

1182
01:15:08,310 --> 01:15:13,750
Dwarkesh:
If you've got entire megalopolises filled with factories where you're already

1183
01:15:13,750 --> 01:15:19,050
Dwarkesh:
deploying robots and different production systems which use automation,

1184
01:15:19,490 --> 01:15:24,490
Dwarkesh:
you have in-house this process knowledge you're building up which the AIs can

1185
01:15:24,490 --> 01:15:26,510
Dwarkesh:
then feed on and accelerate.

1186
01:15:27,230 --> 01:15:31,070
Dwarkesh:
That equivalent level of data we don't have in America.

1187
01:15:31,490 --> 01:15:37,250
Dwarkesh:
So this could be a period in which those technological advantages or those advantages

1188
01:15:37,250 --> 01:15:41,450
Dwarkesh:
in the physical world manufacturing could rapidly compound for China.

1189
01:15:41,690 --> 01:15:44,470
Dwarkesh:
And also, I mean, their big advantage as a civilization and society,

1190
01:15:44,590 --> 01:15:49,930
Dwarkesh:
at least in recent decades, has been that they can do big industrial projects fast and efficiently.

1191
01:15:50,570 --> 01:15:53,350
Dwarkesh:
That's not the first thing you think of when you think of America.

1192
01:15:53,770 --> 01:16:00,530
Dwarkesh:
And AGI is a huge industrial, high CapEx, Manhattan project, right?

1193
01:16:00,610 --> 01:16:03,790
Dwarkesh:
And this is the kind of thing that China excels at and we don't.

1194
01:16:03,890 --> 01:16:07,330
Dwarkesh:
So, you know, I think it's like a much tougher race than people anticipate.

1195
01:16:07,890 --> 01:16:11,590
Ryan Sean Adams:
So what's all this going to do for the world? So once we get to the point of AGI,

1196
01:16:11,850 --> 01:16:15,970
Ryan Sean Adams:
we've talked about GDP and your estimate is less on the Tyler Cowen kind of

1197
01:16:15,970 --> 01:16:21,030
Ryan Sean Adams:
half a percent per year and more on, I guess, the Satya Nadella from Microsoft,

1198
01:16:21,190 --> 01:16:23,710
Ryan Sean Adams:
what does he say, 7% to 8% once we get to AGI.

1199
01:16:24,130 --> 01:16:30,730
Ryan Sean Adams:
What about unemployment? Does this cause mass, I guess, job loss across the

1200
01:16:30,730 --> 01:16:32,670
Ryan Sean Adams:
economy or do people adopt?

1201
01:16:33,110 --> 01:16:35,610
Ryan Sean Adams:
What's your take here? Yeah, what are you seeing?

1202
01:16:36,290 --> 01:16:39,630
Dwarkesh:
Yeah, I mean, definitely will cause job loss. I think people who don't,

1203
01:16:39,730 --> 01:16:43,590
Dwarkesh:
I think a lot of AI leaders try to gloss over that or something. And like, I mean.

1204
01:16:43,650 --> 01:16:43,930
Josh Kale:
What do you mean?

1205
01:16:43,990 --> 01:16:45,830
Dwarkesh:
Like, what does AGI mean if it doesn't cause job loss, right?

1206
01:16:45,890 --> 01:16:47,790
Dwarkesh:
If it does what a human does and.

1207
01:16:47,790 --> 01:16:48,070
Josh Kale:
It does it

1208
01:16:48,070 --> 01:16:51,070
Dwarkesh:
Cheaper and better and faster, like why would that not cause job loss?

1209
01:16:52,190 --> 01:16:56,130
Dwarkesh:
The positive vision here is just that it creates so much wealth,

1210
01:16:56,370 --> 01:17:00,790
Dwarkesh:
so much abundance, that we can still give people a much better standard of living

1211
01:17:00,790 --> 01:17:06,430
Dwarkesh:
than even the wealthiest people today, even if they themselves don't have a job.

1212
01:17:06,430 --> 01:17:12,370
Dwarkesh:
The future I worry about is one where instead of creating some sort of UBI that

1213
01:17:12,370 --> 01:17:16,310
Dwarkesh:
will get exponentially bigger as society gets wealthier,

1214
01:17:16,510 --> 01:17:26,650
Dwarkesh:
we try to create these sorts of guild-like protection rackets where if the coders got unemployed,

1215
01:17:26,970 --> 01:17:32,870
Dwarkesh:
then we're going to make these bullshit jobs just for the coders and this is

1216
01:17:32,870 --> 01:17:34,830
Dwarkesh:
how we give them a redistribution.

1217
01:17:34,830 --> 01:17:42,790
Dwarkesh:
Or we try to expand Medicaid for AI, but it's not allowed to procure all of

1218
01:17:42,790 --> 01:17:46,070
Dwarkesh:
these advanced medicines and cures that AI is coming up with,

1219
01:17:46,230 --> 01:17:50,050
Dwarkesh:
rather than just giving people, you know, maybe lump sums of money or something.

1220
01:17:50,390 --> 01:17:54,130
Dwarkesh:
So I am worried about the future where instead of sharing this abundance and

1221
01:17:54,130 --> 01:18:00,430
Dwarkesh:
just embracing it, we just have these protection rackets that maybe let a few

1222
01:18:00,430 --> 01:18:03,130
Dwarkesh:
people have access to the abundance of AI.

1223
01:18:03,230 --> 01:18:06,150
Dwarkesh:
So maybe like if you sue AI, if you sue the right company at the right time,

1224
01:18:06,310 --> 01:18:08,930
Dwarkesh:
you'll get a trillion dollars, but everybody else is stuck with nothing.

1225
01:18:09,150 --> 01:18:15,090
Dwarkesh:
I want to avoid that future and just be honest about what's coming and make

1226
01:18:15,090 --> 01:18:21,310
Dwarkesh:
programs that are simple and acknowledge how fast things will change and are

1227
01:18:21,310 --> 01:18:25,850
Dwarkesh:
forward looking rather than trying to turn what already exists into something

1228
01:18:25,850 --> 01:18:28,570
Dwarkesh:
amenable to the displacement that AI will create.

1229
01:18:29,260 --> 01:18:32,320
Ryan Sean Adams:
That argument reminds me of, I don't know if you read the essay recently came

1230
01:18:32,320 --> 01:18:34,260
Ryan Sean Adams:
out called The Intelligence Curse. Did you read that?

1231
01:18:34,700 --> 01:18:40,180
Ryan Sean Adams:
It was basically the idea of applying kind of the nation state resource curse

1232
01:18:40,180 --> 01:18:42,080
Ryan Sean Adams:
to the idea of intelligence.

1233
01:18:42,360 --> 01:18:45,880
Ryan Sean Adams:
So like nation states that are very high in natural resources,

1234
01:18:45,880 --> 01:18:47,820
Ryan Sean Adams:
they just have a propensity.

1235
01:18:48,240 --> 01:18:53,260
Ryan Sean Adams:
I mean, an example is kind of like a Middle Eastern state with lots of oil reserves, right?

1236
01:18:53,680 --> 01:18:58,140
Ryan Sean Adams:
They have this rich source of a commodity type of abundance.

1237
01:18:58,560 --> 01:19:02,540
Ryan Sean Adams:
They need their people less. And so they don't invest in citizens' rights.

1238
01:19:02,760 --> 01:19:04,420
Ryan Sean Adams:
They don't invest in social programs.

1239
01:19:04,800 --> 01:19:08,360
Ryan Sean Adams:
The authors of the intelligence curse were saying that there's a similar type

1240
01:19:08,360 --> 01:19:11,560
Ryan Sean Adams:
of curse that could happen once intelligence gets very cheap,

1241
01:19:11,720 --> 01:19:14,900
Ryan Sean Adams:
which is basically like the nation state doesn't need humans anymore.

1242
01:19:15,120 --> 01:19:18,740
Ryan Sean Adams:
And those at the top, the rich, wealthy corporations, they don't need workers anymore.

1243
01:19:19,020 --> 01:19:22,940
Ryan Sean Adams:
So we get kind of locked in this almost feudal state where, you know,

1244
01:19:23,220 --> 01:19:27,760
Ryan Sean Adams:
everyone has the property that their grandparents had and there's no meritocracy

1245
01:19:27,760 --> 01:19:30,960
Ryan Sean Adams:
and sort of the nation states don't reinvest in citizens.

1246
01:19:31,360 --> 01:19:35,640
Ryan Sean Adams:
Almost some similar ideas to your idea that like, you know, that the robots

1247
01:19:35,640 --> 01:19:39,580
Ryan Sean Adams:
might want us just, or sorry, the AIs might just want us for our meat hands

1248
01:19:39,580 --> 01:19:42,980
Ryan Sean Adams:
because they don't have the robotics technology on a temporary basis.

1249
01:19:43,360 --> 01:19:46,120
Ryan Sean Adams:
What do you think of this type of like future? Is this possible?

1250
01:19:46,320 --> 01:19:49,920
Dwarkesh:
I agree that that is like definitely more of a concern given that humans will

1251
01:19:49,920 --> 01:19:54,380
Dwarkesh:
not be directly involved in the economic output that will be generated in the CIA civilization.

1252
01:19:54,820 --> 01:19:58,780
Dwarkesh:
The hopeful story you can tell is that a lot of these Middle Eastern resource,

1253
01:19:59,100 --> 01:20:01,260
Dwarkesh:
you know, Dutch disease is another term that's used,

1254
01:20:01,800 --> 01:20:06,340
Dwarkesh:
countries, the problem is that they're not democracies, so that this wealth

1255
01:20:06,340 --> 01:20:08,280
Dwarkesh:
can just be, the system of government

1256
01:20:08,280 --> 01:20:11,520
Dwarkesh:
just lets whoever's in power extract that wealth for themselves.

1257
01:20:11,780 --> 01:20:15,620
Dwarkesh:
Whereas there are countries like Norway, for example, which also have abundant

1258
01:20:15,620 --> 01:20:21,120
Dwarkesh:
resources, who are able to use those resources to have further social welfare

1259
01:20:21,120 --> 01:20:24,140
Dwarkesh:
programs, to build sovereign wealth funds for their citizens,

1260
01:20:24,400 --> 01:20:25,360
Dwarkesh:
to invest in their future.

1261
01:20:26,540 --> 01:20:29,500
Dwarkesh:
We are going into, at least some countries, America included,

1262
01:20:29,640 --> 01:20:32,460
Dwarkesh:
will go into the age of AI as a democracy.

1263
01:20:33,180 --> 01:20:38,000
Dwarkesh:
And so we, of course, will lose our economic leverage, but the average person

1264
01:20:38,000 --> 01:20:39,340
Dwarkesh:
still has their political leverage.

1265
01:20:39,880 --> 01:20:43,860
Dwarkesh:
Now, over the long run, yeah, if we didn't do anything for a while,

1266
01:20:44,060 --> 01:20:46,140
Dwarkesh:
I'm guessing the political system would also change.

1267
01:20:46,600 --> 01:20:52,080
Dwarkesh:
So then the key is to lock in or turn our current, well, it's not just political leverage, right?

1268
01:20:52,120 --> 01:20:56,000
Dwarkesh:
We also have property rights. So like we own a lot of stuff that AI wants, factories,

1269
01:20:56,580 --> 01:21:01,260
Dwarkesh:
sources of data, et cetera, is to use the combination of political and economic

1270
01:21:01,260 --> 01:21:07,560
Dwarkesh:
leverage to lock in benefits for us for the long term, but beyond our the lifespan

1271
01:21:07,560 --> 01:21:09,840
Dwarkesh:
of our economic usefulness.

1272
01:21:10,040 --> 01:21:13,060
Dwarkesh:
And I'm more optimistic for us than I am for these Middle Eastern countries

1273
01:21:13,060 --> 01:21:17,180
Dwarkesh:
that started off poor and also with no democratic representation.

1274
01:21:17,480 --> 01:21:20,700
Ryan Sean Adams:
What do you think the future of like ChachipD is going to be?

1275
01:21:20,700 --> 01:21:25,100
Ryan Sean Adams:
If we just extrapolate maybe one version update forward to ChatGPT 5,

1276
01:21:25,400 --> 01:21:29,860
Ryan Sean Adams:
do you think the trend line of the scaling law will essentially hold for ChatGPT 5?

1277
01:21:30,080 --> 01:21:33,560
Ryan Sean Adams:
I mean, another way to ask that question is, do you feel like it'll feel like

1278
01:21:33,560 --> 01:21:36,880
Ryan Sean Adams:
the difference between maybe a BlackBerry and an iPhone?

1279
01:21:37,300 --> 01:21:41,980
Ryan Sean Adams:
Or will it feel more like the difference between, say, the iPhone 10 and the

1280
01:21:41,980 --> 01:21:45,540
Ryan Sean Adams:
iPhone 11, which is just like incremental progress, not a big breakthrough,

1281
01:21:45,760 --> 01:21:49,580
Ryan Sean Adams:
not an order of magnitude change? Yeah.

1282
01:21:50,060 --> 01:21:53,620
Dwarkesh:
I think it'll be somewhere in between but I don't think it'll feel like a humongous

1283
01:21:53,620 --> 01:21:58,020
Dwarkesh:
breakthrough even though I think it's in a remarkable pace of change because

1284
01:21:58,020 --> 01:22:02,260
Dwarkesh:
the nature of scaling is that sometimes people talk about it as an exponential process,

1285
01:22:03,500 --> 01:22:06,760
Dwarkesh:
Exponential usually refers to like it going like this.

1286
01:22:07,060 --> 01:22:10,960
Dwarkesh:
So having like a sort of J curve aspect to it, where the incremental input is

1287
01:22:10,960 --> 01:22:16,320
Dwarkesh:
leading to super linear amounts of output, in this case, intelligence and value,

1288
01:22:16,500 --> 01:22:20,040
Dwarkesh:
where it's actually more like a sideways J.

1289
01:22:20,360 --> 01:22:23,520
Dwarkesh:
The exponential means the exponential and the scaling laws is that you need

1290
01:22:23,520 --> 01:22:29,060
Dwarkesh:
exponentially more inputs to get marginal increases in usefulness or loss or intelligence.

1291
01:22:29,500 --> 01:22:34,540
Dwarkesh:
So and that's what we've been seeing, right? I think you initially see like some cool demo.

1292
01:22:34,680 --> 01:22:38,480
Dwarkesh:
So as you mentioned, you see some cool computer use demo, which comes at the

1293
01:22:38,480 --> 01:22:44,100
Dwarkesh:
beginning of this hyper exponential, I'm sorry, of this sort of plateauing curve.

1294
01:22:44,280 --> 01:22:48,580
Dwarkesh:
And then it's still an incredibly powerful curve and we're still early in it.

1295
01:22:48,700 --> 01:22:54,520
Dwarkesh:
But the next demo will be just adding on to making this existing capability

1296
01:22:54,520 --> 01:22:57,160
Dwarkesh:
more reliable, applicable for more skills.

1297
01:22:57,360 --> 01:23:01,100
Dwarkesh:
The other interesting incentive in this industry is that because there's so

1298
01:23:01,100 --> 01:23:05,620
Dwarkesh:
much competition between the labs, you are incentivized to release a capability.

1299
01:23:06,300 --> 01:23:11,700
Dwarkesh:
As soon as it's even marginally viable or marginally cool so you can raise more

1300
01:23:11,700 --> 01:23:13,280
Dwarkesh:
funding or make more money off of it.

1301
01:23:13,520 --> 01:23:16,760
Dwarkesh:
You're not incentivized to just like sit on it until you perfected it,

1302
01:23:17,000 --> 01:23:19,800
Dwarkesh:
which is why I don't expect like tomorrow OpenAI will just come out with like,

1303
01:23:20,140 --> 01:23:22,520
Dwarkesh:
we've solved continual learning, guys, and we didn't tell you about it.

1304
01:23:22,580 --> 01:23:23,900
Dwarkesh:
We're working on it for five years.

1305
01:23:24,240 --> 01:23:27,880
Dwarkesh:
If they had like even an inkling of a solution, they'd want to release it ASAP

1306
01:23:27,880 --> 01:23:32,100
Dwarkesh:
so they can raise a $600 billion round and then spend more money on compute.

1307
01:23:32,540 --> 01:23:38,560
Dwarkesh:
So yeah, I do think it'll seem marginal. But again, marginal in the context of seven years to AGI.

1308
01:23:38,960 --> 01:23:42,140
Dwarkesh:
So zoom out long enough and a crazy amount of progress is happening.

1309
01:23:42,500 --> 01:23:48,480
Dwarkesh:
Month to month, I think people overhype how significant any one new release is. So I guess the answer.

1310
01:23:48,480 --> 01:23:52,580
Dwarkesh Patel:
To when we will get AGI very much depends on that scaling trend holding.

1311
01:23:52,800 --> 01:23:56,360
Dwarkesh Patel:
Your estimate in the book for AGI was 60% chance by 2040.

1312
01:23:57,100 --> 01:24:00,560
Dwarkesh Patel:
So I'm curious, what guess or what idea had the most influence on this estimate?

1313
01:24:00,780 --> 01:24:03,900
Dwarkesh Patel:
What made you end up on 60% of 2040?

1314
01:24:04,160 --> 01:24:06,340
Dwarkesh Patel:
Because a lot of timelines are much faster than that.

1315
01:24:06,980 --> 01:24:10,320
Dwarkesh:
It's sort of reasoning about the things they currently still lack,

1316
01:24:10,440 --> 01:24:12,680
Dwarkesh:
the capabilities they still lack, and what stands in the way.

1317
01:24:13,000 --> 01:24:16,420
Dwarkesh:
And just generally an intuition that things often take longer to happen than

1318
01:24:16,420 --> 01:24:18,920
Dwarkesh:
you might think. Progress tends to slow down.

1319
01:24:19,660 --> 01:24:23,700
Dwarkesh:
Also, it's the case that, look, you might have heard the phrase that we keep

1320
01:24:23,700 --> 01:24:26,460
Dwarkesh:
shifting the goalposts on AI, right?

1321
01:24:26,560 --> 01:24:30,340
Dwarkesh:
So they can do the things which skeptics were saying they couldn't ever do already.

1322
01:24:30,340 --> 01:24:34,680
Dwarkesh:
But now they say AI is still a dead end because problem X, Y,

1323
01:24:34,800 --> 01:24:35,940
Dwarkesh:
Z, which will be solved next year.

1324
01:24:36,780 --> 01:24:40,900
Dwarkesh:
Now, there's a way in which this is frustrating, but there's another way in which there's some,

1325
01:24:43,170 --> 01:24:46,290
Dwarkesh:
It is the case that we didn't get to AGI, even though we have passed the Turing

1326
01:24:46,290 --> 01:24:49,110
Dwarkesh:
test and we have models that are incredibly smart and can reason.

1327
01:24:49,490 --> 01:24:53,750
Dwarkesh:
So it is accurate to say that, oh, we were wrong and there is some missing thing

1328
01:24:53,750 --> 01:24:57,270
Dwarkesh:
that we need to keep identifying about what is still lacking to the path of AGI.

1329
01:24:57,490 --> 01:25:00,850
Dwarkesh:
Like it does make sense to shift the goalposts. And I think we might discover

1330
01:25:00,850 --> 01:25:04,370
Dwarkesh:
once continual learning is solved or once extended computer use is solved,

1331
01:25:04,570 --> 01:25:08,210
Dwarkesh:
that there were other aspects of human intelligence, which we take for granted

1332
01:25:08,210 --> 01:25:12,930
Dwarkesh:
in this Moravax paradox sense, but which are actually quite crucial to making

1333
01:25:12,930 --> 01:25:14,570
Dwarkesh:
us economically valuable.

1334
01:25:14,830 --> 01:25:18,750
Ryan Sean Adams:
Part of the reason we wanted to do this, Dwarkesh, is because we both are enjoyers

1335
01:25:18,750 --> 01:25:20,390
Ryan Sean Adams:
of your podcast. It's just fantastic.

1336
01:25:20,630 --> 01:25:25,230
Ryan Sean Adams:
And you talk to all of the, you know, those that are on the forefront of AI

1337
01:25:25,230 --> 01:25:27,650
Ryan Sean Adams:
development, leading it in all sorts of ways.

1338
01:25:28,070 --> 01:25:30,530
Ryan Sean Adams:
And one of the things I wanted to do with reading your book,

1339
01:25:30,630 --> 01:25:34,010
Ryan Sean Adams:
and obviously I'm always asking myself when I'm listening to your podcast is

1340
01:25:34,010 --> 01:25:36,250
Ryan Sean Adams:
like, what does Dwarkesh think personally?

1341
01:25:36,630 --> 01:25:39,830
Ryan Sean Adams:
And I feel like I sort of got that insight maybe toward the end of your book,

1342
01:25:39,970 --> 01:25:44,210
Ryan Sean Adams:
like, you know, in the summary section, where you think like there's a 60% probability

1343
01:25:44,210 --> 01:25:48,290
Ryan Sean Adams:
of AGI by 2040, which puts you more in the moderate camp, right?

1344
01:25:48,350 --> 01:25:50,850
Ryan Sean Adams:
You're not a conservative, but you're not like an accelerationist.

1345
01:25:50,870 --> 01:25:51,630
Ryan Sean Adams:
So you're moderate there.

1346
01:25:51,830 --> 01:25:57,390
Ryan Sean Adams:
And you also said you think more than likely AI will be net beneficial to humanity.

1347
01:25:57,590 --> 01:26:01,270
Ryan Sean Adams:
So you're more optimist than Doomer. So we've got a moderate optimist.

1348
01:26:01,390 --> 01:26:05,250
Ryan Sean Adams:
And you also think this, and this is very interesting, There's no going back.

1349
01:26:05,530 --> 01:26:10,490
Ryan Sean Adams:
So you're somewhat of an AI determinist. And I think the reason you state for

1350
01:26:10,490 --> 01:26:12,050
Ryan Sean Adams:
not, you're like, there's no going back.

1351
01:26:12,250 --> 01:26:16,070
Ryan Sean Adams:
It struck me, there's this line in your book. It seems that the universe is

1352
01:26:16,070 --> 01:26:20,970
Ryan Sean Adams:
structured such that throwing large amounts of compute at the right distribution of data gets you AI.

1353
01:26:21,230 --> 01:26:24,290
Ryan Sean Adams:
And the secret is out. If the scaling picture is roughly correct,

1354
01:26:24,490 --> 01:26:28,990
Ryan Sean Adams:
it's hard to imagine AGI not being developed this century, even if some actors

1355
01:26:28,990 --> 01:26:30,750
Ryan Sean Adams:
hold back or are held back.

1356
01:26:31,030 --> 01:26:34,330
Ryan Sean Adams:
That to me is an AI determinist position. Do you think that's fair?

1357
01:26:34,810 --> 01:26:39,050
Ryan Sean Adams:
Moderate with respect to accelerationism, optimistic with respect to its potential,

1358
01:26:39,210 --> 01:26:43,170
Ryan Sean Adams:
and also determinist, like there's nothing else we can do. We can't go backwards here.

1359
01:26:43,290 --> 01:26:47,730
Dwarkesh:
I'm determinist in the sense that I think if AI is technologically possible, it is inevitable.

1360
01:26:48,330 --> 01:26:52,530
Dwarkesh:
I think sometimes people are optimistic about this idea that we as a world will sort of,

1361
01:26:53,480 --> 01:26:58,000
Dwarkesh:
I collectively decide not to build AI. And I just don't think that's a plausible outcome.

1362
01:26:58,340 --> 01:27:02,300
Dwarkesh:
The local incentives for any actor to build AI are so high that it will happen.

1363
01:27:02,480 --> 01:27:05,340
Dwarkesh:
But I'm also an optimist in the sense that, look, I'm not naive.

1364
01:27:05,640 --> 01:27:08,580
Dwarkesh:
I've listed out all the way, like what happened to the Aztecs and Incas was

1365
01:27:08,580 --> 01:27:10,760
Dwarkesh:
terrible. And I've explained how that could be similar to what AIs could do

1366
01:27:10,760 --> 01:27:13,300
Dwarkesh:
to us and what we need to do to avoid that outcome.

1367
01:27:13,780 --> 01:27:18,400
Dwarkesh:
But I am optimistic in the sense that the world of the future fundamentally

1368
01:27:18,400 --> 01:27:22,340
Dwarkesh:
will have so much abundance that there's all these,

1369
01:27:22,580 --> 01:27:28,360
Dwarkesh:
that alone is a prima facie reason to think that there must be some way of cooperating

1370
01:27:28,360 --> 01:27:30,520
Dwarkesh:
that is mutually beneficial.

1371
01:27:30,560 --> 01:27:33,640
Dwarkesh:
If we're going to be thousands, millions of times wealthier,

1372
01:27:33,860 --> 01:27:37,540
Dwarkesh:
is there really no way that humans are better off or can we can find a way for

1373
01:27:37,540 --> 01:27:39,860
Dwarkesh:
humans to become better off as a result of this transformation?

1374
01:27:40,560 --> 01:27:42,420
Dwarkesh:
So yeah, I think you've put your finger on it.

1375
01:27:43,020 --> 01:27:46,300
Ryan Sean Adams:
So this scaling book, of course, goes through the history of AI scaling.

1376
01:27:46,520 --> 01:27:49,780
Ryan Sean Adams:
I think everyone should should pick it up to get the full chronology,

1377
01:27:49,980 --> 01:27:55,520
Ryan Sean Adams:
but also sort of captures where we are in the midst of this story is like, we're not done yet.

1378
01:27:55,820 --> 01:27:58,640
Ryan Sean Adams:
And I'm wondering how you feel at this moment of time.

1379
01:27:58,960 --> 01:28:03,240
Ryan Sean Adams:
So I don't know if we're halfway through, if we're a quarter way through,

1380
01:28:03,480 --> 01:28:07,600
Ryan Sean Adams:
if we're one tenth of the way through, but we're certainly not finished the path to AI scaling.

1381
01:28:07,960 --> 01:28:10,780
Ryan Sean Adams:
How do you feel like in this moment in 2025?

1382
01:28:11,180 --> 01:28:14,540
Ryan Sean Adams:
I mean, is all of this terrifying? Is it exciting?

1383
01:28:15,080 --> 01:28:17,000
Ryan Sean Adams:
Is it exhilarating?

1384
01:28:17,660 --> 01:28:20,160
Ryan Sean Adams:
What's the emotion that you feel?

1385
01:28:20,540 --> 01:28:24,560
Dwarkesh:
Maybe I feel a little sort of hurried. I personally feel like there's a lot

1386
01:28:24,560 --> 01:28:26,760
Dwarkesh:
of things I want to do in the meantime,

1387
01:28:27,000 --> 01:28:33,460
Dwarkesh:
including what my mission is with the podcast, which is to, and I know it's

1388
01:28:33,460 --> 01:28:37,600
Dwarkesh:
your mission as well, is to improve the discourse around these topics,

1389
01:28:38,300 --> 01:28:42,860
Dwarkesh:
to not necessarily push for a specific agenda, but make sure that when people are making decisions,

1390
01:28:42,980 --> 01:28:47,220
Dwarkesh:
they're as well-informed as possible, They have as much strategic awareness

1391
01:28:47,220 --> 01:28:54,060
Dwarkesh:
and depth of understanding around how AI works, what it could do in the future as possible.

1392
01:28:55,210 --> 01:29:00,250
Dwarkesh:
And, but in many ways, I feel like I still haven't emotionally priced in the future I'm expecting.

1393
01:29:00,490 --> 01:29:06,190
Dwarkesh:
In this one very basic sense, I think that there's a very good chance that I

1394
01:29:06,190 --> 01:29:07,670
Dwarkesh:
live beyond 200 years of age.

1395
01:29:08,310 --> 01:29:12,830
Dwarkesh:
I have not changed anything about my life with regards to that knowledge, right?

1396
01:29:12,910 --> 01:29:17,330
Dwarkesh:
I'm not like, when I'm picking partners, I'm not like, oh, this is the person,

1397
01:29:17,650 --> 01:29:20,490
Dwarkesh:
now that I think I'm going to live for 200, you know, like hundreds of years.

1398
01:29:20,870 --> 01:29:21,270
Ryan Sean Adams:
Yeah.

1399
01:29:23,090 --> 01:29:27,290
Dwarkesh:
Well, you know, ideally I would pick a partner that would, ideally you pick

1400
01:29:27,290 --> 01:29:29,070
Dwarkesh:
somebody who would be, that would be true regardless.

1401
01:29:30,250 --> 01:29:34,490
Dwarkesh:
But you see what I'm saying, right? There's like, the fact that I expect my

1402
01:29:34,490 --> 01:29:37,630
Dwarkesh:
personal life, the world around me, the lives of the people I care about,

1403
01:29:37,870 --> 01:29:44,430
Dwarkesh:
humanity in general to be so different has, it just like doesn't emotionally resonate as much as,

1404
01:29:45,630 --> 01:29:50,050
Dwarkesh:
I, my intellectual thoughts and my emotional landscape aren't in the same place.

1405
01:29:50,170 --> 01:29:51,410
Dwarkesh:
I wonder if it's similar for you guys.

1406
01:29:51,770 --> 01:29:54,550
Ryan Sean Adams:
Yeah, I totally agree. I don't think I've priced that in. Also,

1407
01:29:54,770 --> 01:29:58,450
Ryan Sean Adams:
there's like non-zero chance that Eliezer Yudkowsky is right, Dworkesh.

1408
01:29:58,710 --> 01:30:03,350
Ryan Sean Adams:
Do you know? And so that scenario, I just, I can't bring myself to emotionally price in.

1409
01:30:03,770 --> 01:30:07,330
Ryan Sean Adams:
So I veer towards the optimism side as well.

1410
01:30:07,910 --> 01:30:11,790
Ryan Sean Adams:
Dworkesh, this has been fantastic. Thank you so much for all you do on the podcast.

1411
01:30:12,090 --> 01:30:15,430
Ryan Sean Adams:
I have to ask a question for our crypto audience as well, which is,

1412
01:30:15,530 --> 01:30:19,390
Ryan Sean Adams:
when are you going to do a crypto podcast on Dwarkech?

1413
01:30:19,990 --> 01:30:22,390
Dwarkesh:
I already did. It was with one Sam Bigman-Fried.

1414
01:30:22,970 --> 01:30:23,930
Ryan Sean Adams:
Oh my God.

1415
01:30:24,930 --> 01:30:25,570
Dwarkesh:
Oh man.

1416
01:30:26,050 --> 01:30:29,850
Ryan Sean Adams:
We got to get you a new guest. We got to get you someone else to revisit the top best.

1417
01:30:29,850 --> 01:30:31,650
Dwarkesh:
Don't look that one up. It's Ben Omen. Don't look that one up.

1418
01:30:31,730 --> 01:30:35,490
Dwarkesh:
I think in retrospect. You know what? We'll do another one.

1419
01:30:36,370 --> 01:30:37,430
Ryan Sean Adams:
Fantastic. I'll ask you

1420
01:30:37,430 --> 01:30:40,050
Dwarkesh:
Guys for some recommendations. That'd be great. Dwarkech, thank you so much.

1421
01:30:40,050 --> 01:30:42,410
Dwarkesh:
But I've been following your stuff for a while, for I think many years.

1422
01:30:43,730 --> 01:30:46,510
Dwarkesh:
So it's great to finally meet. and this was a lot of fun.

1423
01:30:46,830 --> 01:30:48,550
Ryan Sean Adams:
Appreciate it. It was great. Thanks a lot.