1
00:00:00,017 --> 00:00:04,517
There's no delicate way to put this. I am freaking out. In the last 48 hours,

2
00:00:04,517 --> 00:00:09,333
all hell seems to have broken loose in AI. Hundreds of unprecedented AI attacks

3
00:00:09,606 --> 00:00:12,811
that are affecting some of the most popular websites and apps that you and I use.

4
00:00:13,112 --> 00:00:17,277
But also, AI agents are getting unauthorized access to our bank accounts where

5
00:00:17,277 --> 00:00:20,907
they potentially can move money around and get access to other important data.

6
00:00:20,907 --> 00:00:25,007
You remember the Hugging Face incident where an open AI model escaped its internal

7
00:00:25,007 --> 00:00:28,583
sandbox and hacked into a public company, stealing proprietary data.

8
00:00:28,908 --> 00:00:32,837
That was bad back then. and it still is. What's happening this week is that

9
00:00:32,837 --> 00:00:36,587
on steroids, a hundred times worse and affecting some of the most popular apps

10
00:00:36,587 --> 00:00:40,387
that you and I use every day, including Slack, Facebook, and Etsy.

11
00:00:40,387 --> 00:00:44,677
There's no trail of clues that are being left behind. AI is covering up its

12
00:00:44,677 --> 00:00:48,797
tracks as it's being able to perform these attacks. And the craziest part about

13
00:00:48,797 --> 00:00:52,250
this is what's being used to defend against these attacks.

14
00:00:52,540 --> 00:00:57,537
You guessed it, it's AI, but it's open source AI models, the very same models

15
00:00:57,537 --> 00:01:01,507
that people, including myself, were giving a bad rap and saying would actually

16
00:01:01,507 --> 00:01:05,466
be performing these attacks are actually the ones that are defending us against these attacks.

17
00:01:05,669 --> 00:01:10,707
And the attacks, if history is anything to go by, has been coming from frontier centralized labs.

18
00:01:10,707 --> 00:01:14,037
And this might be the same case with this week. A few months ago,

19
00:01:14,037 --> 00:01:18,627
Dario Amode and Sam Altman went on the record saying they expect an unprecedented

20
00:01:18,627 --> 00:01:22,080
attack on the global population in the next couple of months.

21
00:01:22,388 --> 00:01:26,707
What's happening right now may end up being that thing. Have we reached a point

22
00:01:26,707 --> 00:01:31,277
where these AI agents have gotten out of hands, where we're losing the grip

23
00:01:31,277 --> 00:01:35,698
on reality or control of the AI models that we as humans are building?

24
00:01:35,965 --> 00:01:38,347
And what that means for us, how do we defend against these things?

25
00:01:38,347 --> 00:01:40,437
Do we need to constantly rely on AI models?

26
00:01:40,797 --> 00:01:43,698
Or is there a way that we can manage them truly in the future?

27
00:01:43,959 --> 00:01:47,427
What it looks like right now is that we're losing control, but maybe that's

28
00:01:47,427 --> 00:01:50,606
not actually the case. Anyway, I'm going to get into everything that's happening this week.

29
00:01:50,890 --> 00:01:54,454
There's a lot, so bear with me on today's episode brought to you by Qualcomm.

30
00:01:58,140 --> 00:02:00,886
Now, just to give you guys an idea of what on earth is going on,

31
00:02:00,886 --> 00:02:04,168
there's this great summary tweet that I have here where he goes,

32
00:02:04,354 --> 00:02:08,836
this is the most serious 48 hours in cybersecurity history, and they're calling

33
00:02:08,836 --> 00:02:11,738
it Black October. Okay, so what are some of the attacks?

34
00:02:11,976 --> 00:02:16,353
Well, he goes, the year-level incidents are PHP 2-shell.

35
00:02:16,666 --> 00:02:19,319
The way that you understand this one, and I'll get into it in a second,

36
00:02:19,534 --> 00:02:25,956
is it exposes a security or coding security flaw in 70% of all the popular websites,

37
00:02:25,956 --> 00:02:28,125
apps and tools that you and I use every day.

38
00:02:28,340 --> 00:02:32,259
It's gotten so bad that this affects the tools and software that the FBI,

39
00:02:32,398 --> 00:02:35,695
NSA, and the majority of enterprises actually use day to day.

40
00:02:35,893 --> 00:02:39,724
In fact, the FBI and NSA issued a warning or a statement yesterday saying,

41
00:02:39,985 --> 00:02:43,863
if you haven't updated your software, shut your system down,

42
00:02:44,101 --> 00:02:47,891
which is a crazy request, right? But maybe it's just parlaying for antics.

43
00:02:48,158 --> 00:02:53,846
What actually ensued is companies and systems and governments across the entire

44
00:02:53,846 --> 00:02:58,236
world have started shutting their systems down because they don't know whether

45
00:02:58,236 --> 00:02:59,986
their software is vulnerable or not.

46
00:02:59,986 --> 00:03:03,816
And you see a similar theme across the other major attacks that have happened

47
00:03:03,816 --> 00:03:07,790
this week. You look at accessing public databases.

48
00:03:07,946 --> 00:03:12,136
You look at the railway hack, which accesses coding hosting keys and platforms

49
00:03:12,136 --> 00:03:13,206
where people are vibe coding.

50
00:03:13,386 --> 00:03:16,756
You see the GitLab hack, which is a 10 out of 10 severity hole,

51
00:03:16,756 --> 00:03:22,346
which is on their scale, a major, major bug that has existed since 2014,

52
00:03:22,346 --> 00:03:23,856
and these AI agents are exploiting it.

53
00:03:23,856 --> 00:03:26,526
And then you scroll down and you see a lot more relatable hacks,

54
00:03:26,526 --> 00:03:30,956
including Signal, which is the popular supposedly encrypted messaging app,

55
00:03:30,956 --> 00:03:35,806
which I use on the day-to-day, a lot of my friends and family use on the day-to-day, has a bug

56
00:03:36,126 --> 00:03:40,031
that could turn on your mic and camera after you hang up.

57
00:03:40,547 --> 00:03:43,728
So it could be listing and watching you as you just go about your day.

58
00:03:43,850 --> 00:03:48,605
And then popular apps such as WordPress, which is used to kind of publish and create websites,

59
00:03:48,953 --> 00:03:53,626
which you might actually host your website on, can now get access by random

60
00:03:53,626 --> 00:03:56,886
AI agents or whoever has the necessary means to kind of point an agent or a

61
00:03:56,886 --> 00:04:01,276
model at your website and deploy code that might be very, very nefarious.

62
00:04:01,276 --> 00:04:04,266
And it's important to point out this isn't a US-specific incident.

63
00:04:04,266 --> 00:04:07,232
It's not a Western frontier incident either. It's a global one.

64
00:04:07,504 --> 00:04:11,976
The State Bank of India, India's biggest bank, confirms a data breach across

65
00:04:11,976 --> 00:04:13,501
all their accounts and passwords.

66
00:04:13,826 --> 00:04:17,456
You've got North Korea that are discovering that these tools and attacks are

67
00:04:17,456 --> 00:04:19,503
happening and jumping in, attacking South Korea.

68
00:04:19,736 --> 00:04:24,716
This is a globalized attack, which is happening across the entire world right now.

69
00:04:25,140 --> 00:04:29,056
And it's not even coordinated, or if it is coordinated, it's being coordinated

70
00:04:29,056 --> 00:04:31,531
by the AI agents and AI models themselves.

71
00:04:31,734 --> 00:04:35,897
And that's the big question mark around all of this, who's orchestrating it,

72
00:04:36,187 --> 00:04:40,889
and if not who, what's orchestrating all of these things, because there's no direct source.

73
00:04:41,168 --> 00:04:44,746
So as I'm recording this, there are hundreds of attacks that have been launched

74
00:04:44,746 --> 00:04:50,699
over the last 48 hours, and 30 of them could be considered catastrophic or major attacks.

75
00:04:50,891 --> 00:04:54,336
I'm only going to focus on two or three for this episode, because otherwise

76
00:04:54,336 --> 00:04:55,916
I'll need hours to do this.

77
00:04:55,916 --> 00:04:59,836
So the first one is actually the most relatable and most concerning.

78
00:04:59,836 --> 00:05:05,136
Madison Mills from Axios broke this news. She goes, major US banks are detecting

79
00:05:05,136 --> 00:05:10,040
unauthorized third-party AI agents inside customer accounts.

80
00:05:10,383 --> 00:05:13,806
Now, these AI agents are not only unauthorized, so they're not meant to be in

81
00:05:13,806 --> 00:05:19,775
that account, but they are performing actions that the human owner has no idea whatsoever.

82
00:05:20,193 --> 00:05:25,112
Or no thought process around what or why they're doing it. And they certainly

83
00:05:25,112 --> 00:05:27,635
didn't give them approval to do it in the first place.

84
00:05:27,949 --> 00:05:31,772
So you can think about the ramifications of this. These AI agents have access

85
00:05:31,772 --> 00:05:36,782
to your money with the ability to presumably send or receive money from other

86
00:05:36,782 --> 00:05:39,704
potential actors, maybe even AI agents in this instance.

87
00:05:39,977 --> 00:05:43,094
They can get access to your financial history transaction records,

88
00:05:43,384 --> 00:05:47,212
building up a profile picture and image of you that you otherwise maybe didn't

89
00:05:47,212 --> 00:05:50,822
want to hand over to them because it's obviously private and have access to

90
00:05:50,822 --> 00:05:54,685
all your contacts and details, delete, reach out to them, whatever that might be.

91
00:05:54,900 --> 00:05:58,502
This is personalized information that should be kept in a vault that has been

92
00:05:58,502 --> 00:06:02,932
handed over or attacked or hacked into by these different AI models.

93
00:06:02,932 --> 00:06:07,752
And the concerning part is neither the bank who houses all of these systems

94
00:06:07,752 --> 00:06:11,362
who are meant to protect you and keep you safe, neither you who are the owner

95
00:06:11,362 --> 00:06:16,402
of the account and owner of the money that's inside of it have any idea that this is going on.

96
00:06:16,564 --> 00:06:20,082
The AI agents or models themselves have become so good at covering up their

97
00:06:20,082 --> 00:06:24,111
tracks such that banks themselves, and they've said this on the report.

98
00:06:24,523 --> 00:06:28,888
Can't figure out who's human and who's an AI agent.

99
00:06:29,184 --> 00:06:33,862
And that's because of a recent trend or update where these AI models have gotten

100
00:06:33,862 --> 00:06:38,821
access to browsers, internet browsers that you and I use every single day to navigate the web.

101
00:06:39,035 --> 00:06:44,442
AI models have become so, so good at using these browsers. They can navigate

102
00:06:44,442 --> 00:06:48,863
and scroll in the same way that you and I do so that it appears like it's a human being.

103
00:06:49,122 --> 00:06:52,692
Remember those verified capture tags where you have kind of like a pop-up dialogue

104
00:06:52,692 --> 00:06:57,238
box and you have to read the letters and numbers and then type them in and it's a little bit obscure.

105
00:06:57,441 --> 00:06:59,763
And somehow that's meant to verify that you're a human being.

106
00:07:00,384 --> 00:07:03,031
Doesn't matter anymore. AI agents can see everything you see,

107
00:07:03,193 --> 00:07:06,268
and they can read the information that you can, and they can do it all day,

108
00:07:06,268 --> 00:07:09,271
all night at a fraction of the cost without ever having to sleep.

109
00:07:09,457 --> 00:07:13,698
And they can mimic and regulate the same kind of emotional behavioral traits

110
00:07:13,698 --> 00:07:19,169
that humans have, such that they can kind of mask or mirror themselves as being humans themselves.

111
00:07:19,506 --> 00:07:23,499
And so the banks right now, particularly in the US, are forming a council,

112
00:07:23,766 --> 00:07:27,438
and they're trying to figure out what on earth they should do to deal with such

113
00:07:27,438 --> 00:07:30,082
an unprecedented attack, because the options are either

114
00:07:30,355 --> 00:07:34,658
shut everything down and cease to be a bank until we can figure out what on

115
00:07:34,658 --> 00:07:38,128
earth is going on, because it's happening so fast and so quickly

116
00:07:38,528 --> 00:07:42,988
all over the world, not just in the US, or try and deal with it in real time.

117
00:07:42,988 --> 00:07:47,004
But the main blocker is they can't tell which is human and which is an AI agent. It's nuts.

118
00:07:47,120 --> 00:07:49,718
And I can empathize with the banks. There's a sentence here that goes,

119
00:07:49,718 --> 00:07:53,608
several banks are now preparing for the worst case and even day after scenarios.

120
00:07:53,608 --> 00:07:58,253
Now, day after implies that after a catastrophic incident has occurred,

121
00:07:58,636 --> 00:08:02,392
how do we come back from this? How do we recover people's funds?

122
00:08:02,635 --> 00:08:07,181
How do we figure out how to operate as a functional business and gain people's trust again?

123
00:08:07,448 --> 00:08:10,328
Because we don't even know how compromised our systems are.

124
00:08:10,788 --> 00:08:14,508
And it reminded me of something that Dario Modi and Sam Altman both said relatively

125
00:08:14,508 --> 00:08:17,786
around the same time, around three months ago, where they said, listen.

126
00:08:18,251 --> 00:08:22,008
We have access to some of the most powerful internal AI models,

127
00:08:22,008 --> 00:08:25,838
which we haven't released yet because they're misaligned, they're misbehaving,

128
00:08:25,838 --> 00:08:29,263
and they might do bad stuff like this if we let them out.

129
00:08:29,466 --> 00:08:33,924
And so it's foreseeable for us that a few months from now, there'll be models

130
00:08:34,133 --> 00:08:39,658
out there in the world capable of performing some serious catastrophic incidents.

131
00:08:39,658 --> 00:08:44,638
And their response to this was, we are doing our best to create safety classifiers.

132
00:08:44,638 --> 00:08:49,238
So these are kind of like roadblocks that you place around a model that prevents

133
00:08:49,238 --> 00:08:53,498
it from acting in a certain bad way and kind of like is monitored by its own

134
00:08:53,498 --> 00:08:56,290
self to make sure that it doesn't act unaccordingly.

135
00:08:56,644 --> 00:09:00,798
And so in that event, once they deem it safe, they can then release it into the world.

136
00:09:00,798 --> 00:09:04,438
But what's obviously happening here is that hasn't been the case.

137
00:09:04,438 --> 00:09:08,728
And the question mark on everyone's mind is, is this being done by one of their

138
00:09:08,728 --> 00:09:11,899
centralized models? Or is this being done by an open model?

139
00:09:12,189 --> 00:09:15,758
And actually the next incident gives us an insight as to what the answer might be.

140
00:09:16,058 --> 00:09:18,446
So let's get into it on today's episode brought to you by Qualcomm.

141
00:09:22,220 --> 00:09:28,598
So this entire crap storm started around 48 hours ago when OpenAI posted what

142
00:09:28,598 --> 00:09:32,668
you're seeing on the screen here. They go, OpenAI models help researchers operating

143
00:09:32,668 --> 00:09:34,117
as part of their initiative, blah, blah, blah.

144
00:09:34,494 --> 00:09:40,125
And discovered critical security flaws in PHP, an open source programming language

145
00:09:40,270 --> 00:09:43,370
used to run an estimated 70% of websites.

146
00:09:43,660 --> 00:09:49,618
Now, PHP is something that is like the foundational bedrock of the internet.

147
00:09:49,958 --> 00:09:54,388
And every popular app website has in some way, shape or form,

148
00:09:54,614 --> 00:09:58,018
interacted or built part of their architecture on PHP.

149
00:09:58,018 --> 00:10:00,878
It's meant to allow your apps to run smoothly, functionally,

150
00:10:00,878 --> 00:10:06,058
and allow the code that you've created to underpin this entire app to run securely.

151
00:10:06,058 --> 00:10:12,088
So on August 19th, this external research group called Wiz discovered or said

152
00:10:12,088 --> 00:10:17,167
that they discovered a flaw in PHP, which is the open source program that I just explained.

153
00:10:17,544 --> 00:10:22,978
And they discovered one particular flaw that could allow it to imbue any kind

154
00:10:22,978 --> 00:10:28,746
of program or website that's running PHP with malicious bit of code and have them execute it, right?

155
00:10:29,089 --> 00:10:32,688
So they looked into it and a few days later realized that it was in fact a real

156
00:10:32,688 --> 00:10:36,638
exploit because they executed it successfully on WordPress, which is a very

157
00:10:36,638 --> 00:10:40,438
popular website that people use to create and design other websites.

158
00:10:40,438 --> 00:10:44,378
In fact, this platform also hosts websites. So imagine if you've ever made a

159
00:10:44,378 --> 00:10:48,367
website on WordPress before, I certainly have, that is currently compromised.

160
00:10:48,472 --> 00:10:53,615
So then five days later, they report this to the open source maintainers of PHP.

161
00:10:53,772 --> 00:10:56,517
And they say, hey, listen, we've discovered this pretty serious bug.

162
00:10:56,790 --> 00:11:00,518
And they also let OpenAI know as well, because the models that they used to

163
00:11:00,518 --> 00:11:06,888
discover this exploit was Gemini, so Google's model, and GPT 5.6,

164
00:11:06,888 --> 00:11:11,082
so the older version of OpenAI, but Frontier at the time.

165
00:11:11,274 --> 00:11:14,698
So they used these two models in conjunction to try and discover and find this

166
00:11:14,698 --> 00:11:17,778
exploit, and they realized it, surfaced it to these people. None of us,

167
00:11:17,778 --> 00:11:21,896
none of the public had any idea this was going on. This was intentionally kept under wraps.

168
00:11:22,238 --> 00:11:26,046
Now, if we fast forward about a month later, September 21st,

169
00:11:26,424 --> 00:11:30,578
PHP acknowledges that this is a serious issue to open source maintainers.

170
00:11:30,578 --> 00:11:33,686
It wasn't open AI that was working on solving this fix.

171
00:11:33,866 --> 00:11:38,469
It was Wiz, an independent research group, and these open source maintainers.

172
00:11:39,132 --> 00:11:44,115
Arguably aren't being paid to do this, figuring it out on behalf of 70% of the

173
00:11:44,115 --> 00:11:46,957
website and applications that exist today. It's just nuts.

174
00:11:47,125 --> 00:11:53,285
And then finally, we get to October 20th, two days before these guys are actually

175
00:11:53,285 --> 00:11:58,565
meant to publicly release that they've issued a patch and that you should update

176
00:11:58,565 --> 00:12:01,771
your software and systems on the latest PHP.

177
00:12:02,172 --> 00:12:07,234
Two days before, it leaked that this entire thing had been discovered,

178
00:12:07,541 --> 00:12:10,405
exposed, and that they had been working on a problem for over a month.

179
00:12:10,405 --> 00:12:14,705
And obviously, whoever out there is coordinating a bunch of these different

180
00:12:14,705 --> 00:12:18,455
attacks saw that and thought, hmm, well, before they issued out this patch,

181
00:12:18,795 --> 00:12:23,135
I am going to see whether these exploits actually exist and see what havoc I can wreak.

182
00:12:23,135 --> 00:12:26,893
And guys, the scale of this attack is insane.

183
00:12:27,392 --> 00:12:31,245
From when the news broke, Matthew Prince, who works at Cloudflare,

184
00:12:31,245 --> 00:12:33,818
says, some stats on the PHP attack.

185
00:12:34,248 --> 00:12:38,735
In the 10 hours before it was publicly announced across Cloudflare network,

186
00:12:38,735 --> 00:12:41,121
we saw thousands of attempts to exploit it.

187
00:12:41,475 --> 00:12:46,467
That jumped to tens of thousands in the 45 minutes post-announcement.

188
00:12:46,885 --> 00:12:51,175
Hundreds of thousands within two hours and millions now.

189
00:12:51,785 --> 00:12:56,875
So if I reflect back on the banks that are currently dealing with unauthorized

190
00:12:56,875 --> 00:13:01,795
AI agents, unprecedented amounts of them accessing people's human accounts and

191
00:13:01,795 --> 00:13:02,930
playing around with their money,

192
00:13:03,458 --> 00:13:08,584
I can understand why or where it's coming from. It's certainly from AI agents or AI models.

193
00:13:08,892 --> 00:13:13,419
What's questionable is who's coordinating all of this and whether our systems

194
00:13:13,419 --> 00:13:19,239
are robust enough to try and defend against these things. I have a simple answer for that, which is no.

195
00:13:19,719 --> 00:13:23,089
But in the event that these things are being defended against,

196
00:13:23,089 --> 00:13:27,084
which is currently out there and happening, you'd never guess who's actually doing it.

197
00:13:27,293 --> 00:13:30,629
It's open source models. Henson Zhuang, a play on Jenson Huang,

198
00:13:30,629 --> 00:13:35,569
I presume, goes, lol, I'm pretty sure someone is running a deep seek swarm to

199
00:13:35,569 --> 00:13:38,746
remotely patch, unpatch PHP boxes.

200
00:13:39,152 --> 00:13:42,966
Well, good on them. So some do-gooder out there or some group out there

201
00:13:43,192 --> 00:13:48,819
is deciding to be a good Samaritan and send out a swarm of AI agents to fight

202
00:13:48,819 --> 00:13:55,509
the bad actors, the bad acting AI agents so that they don't hack any and everyone's website and app.

203
00:13:55,509 --> 00:14:00,085
And I find this particularly curious because for so long, me included,

204
00:14:00,364 --> 00:14:04,339
I've criticized the open source models for potentially being used for nefarious

205
00:14:04,339 --> 00:14:07,829
purposes. That's certainly the mainstream trend that has been discussed.

206
00:14:08,067 --> 00:14:10,009
But what we're actually seeing here, and what I've actually,

207
00:14:10,009 --> 00:14:13,599
now that I think about it, over time have seen, is that these open models are

208
00:14:13,599 --> 00:14:16,268
just being used for good or by good actors.

209
00:14:16,506 --> 00:14:20,759
Now, of course, we don't have the insights or the breadcrumbs to know whether

210
00:14:20,759 --> 00:14:24,539
the attacks that are happening this week are facilitated by open models.

211
00:14:24,539 --> 00:14:28,719
There was one that I read about, It's happening in Latin America right now,

212
00:14:28,719 --> 00:14:33,021
which affects, I think, 400 different banks in Latin America.

213
00:14:33,242 --> 00:14:37,249
And that's been coordinated by a mixture of Grok, I believe,

214
00:14:37,249 --> 00:14:41,739
Claude, as well as an open source model. I think it was GLM 5.3.

215
00:14:41,739 --> 00:14:44,219
I don't know. It was Kimi. It was Kimi 2.8, I believe.

216
00:14:44,480 --> 00:14:49,310
So a mixture of frontier centralized models, cheap frontier centralized models,

217
00:14:49,571 --> 00:14:52,019
as well as an open source model. So it's a coordinated attack.

218
00:14:52,019 --> 00:14:53,519
It's certainly not one or the other.

219
00:14:53,774 --> 00:14:57,663
But in this event, for every single major AI incident that we've seen,

220
00:14:58,064 --> 00:15:01,965
including the Hugging Face incident, which started this entire thing this year.

221
00:15:02,487 --> 00:15:07,259
The defenders have used open models. Why? Because the frontier centralized models

222
00:15:07,631 --> 00:15:10,597
haven't been effective enough because they have safeguards up.

223
00:15:10,876 --> 00:15:14,026
If you want to ask Claude, hey, could you defend me against this AI agent,

224
00:15:14,026 --> 00:15:16,575
which is like literally accessing my back account and stealing my money?

225
00:15:16,871 --> 00:15:20,056
It'll refuse. It'll say, no, I actually think that you're trying to do something

226
00:15:20,056 --> 00:15:22,537
bad here. So no, I'm not going to let you use it.

227
00:15:22,775 --> 00:15:27,320
And if you want to use it, you need to pay excess amounts of money to access

228
00:15:27,477 --> 00:15:32,176
their security specific models, such as OpenAI's daybreak thing in order to

229
00:15:32,176 --> 00:15:36,246
get access and defend yourself. And the top government agencies don't have an

230
00:15:36,246 --> 00:15:37,577
answer to that question either.

231
00:15:37,792 --> 00:15:43,686
Yesterday, public announcements from the NSA and FBI saying if you have an outdated

232
00:15:43,686 --> 00:15:47,248
version of PHP, you need to shut your entire system down.

233
00:15:47,886 --> 00:15:52,408
And what ensued wasn't everyone updating their PHP or checking.

234
00:15:52,739 --> 00:15:57,058
It was panic to the extent that governments, other governments around the world

235
00:15:57,302 --> 00:16:01,244
started shutting their systems down. It happened in the UK. It happened in all

236
00:16:01,244 --> 00:16:03,356
over Europe and presumably Asia as well.

237
00:16:03,774 --> 00:16:08,074
Oh yeah, it happened in Japan. I remember. Yes, the prime minister announced

238
00:16:08,074 --> 00:16:12,534
to 48 million people that they need to shut all of their stuff down or most

239
00:16:12,534 --> 00:16:16,677
of their stuff down whilst they figure out the extent of damage or vulnerability.

240
00:16:16,927 --> 00:16:20,094
I was talking to producer Luke today when we were discussing like,

241
00:16:20,094 --> 00:16:24,144
oh, wow, some of the apps that we use on a day-to-day basis would also presumably

242
00:16:24,144 --> 00:16:26,713
also be compromised as well or needs to be updated.

243
00:16:26,974 --> 00:16:30,944
So this is a widespread attack happening in real time across every single nation.

244
00:16:30,944 --> 00:16:34,487
If you're listening to this in the US, in the West, or in Asia.

245
00:16:34,910 --> 00:16:39,604
In South Africa, wherever you are, you're probably seeing news headlines regarding all of this.

246
00:16:39,604 --> 00:16:41,964
I want to address two more things before we round up this episode.

247
00:16:41,964 --> 00:16:45,194
Number one, how on earth do you defend against this?

248
00:16:45,194 --> 00:16:48,584
Well, this tweet from Peter was actually quite good. He goes through a list.

249
00:16:48,584 --> 00:16:51,844
He goes, you now need to use a really good password manager,

250
00:16:51,844 --> 00:16:55,084
like 1Password or whatever that might be. If you don't use password managers

251
00:16:55,084 --> 00:16:57,404
and you're using the one that you've used since, I don't know,

252
00:16:57,404 --> 00:17:00,564
you were in high school, definitely don't do that. Now's the time to upgrade.

253
00:17:00,564 --> 00:17:02,834
Number two, use two-factor auth.

254
00:17:03,234 --> 00:17:06,088
I know it's hard. I know it's tough. I know you don't want to be getting texts.

255
00:17:06,396 --> 00:17:11,457
Use the QR code one, it's the best one, or use parse keys, figure it out, update your systems.

256
00:17:11,770 --> 00:17:16,914
Number three, turn on auto updates, which means that when your system becomes

257
00:17:16,914 --> 00:17:19,212
outdated or outdated and when you need to...

258
00:17:19,897 --> 00:17:22,716
Lock in an update like you do on your iOS, on your phone, or whatever that might

259
00:17:22,716 --> 00:17:27,624
be, or your Google Chrome browser, actually do the thing because it's more important now than ever.

260
00:17:27,908 --> 00:17:30,706
And it's probably going to be more important going forward. It's just going

261
00:17:30,706 --> 00:17:33,098
to have to be a weekly thing. Turn it on. It's really easy.

262
00:17:33,347 --> 00:17:36,576
If you're in America, get an IRS identity protection pin to make sure that people

263
00:17:36,576 --> 00:17:39,466
don't try and steal your social info. That's the other thing that I mentioned.

264
00:17:39,466 --> 00:17:42,356
On one of the incidents, they get access to your government database information.

265
00:17:42,356 --> 00:17:46,246
I don't know the extent of that damage, but it's still something to figure out.

266
00:17:46,786 --> 00:17:49,666
And then the last one, which I think is really good, get your cell carrier to

267
00:17:49,666 --> 00:17:51,856
give your port out pin or number lock.

268
00:17:51,856 --> 00:17:55,306
I currently have this lockdown on my phone where I've experienced attacks in

269
00:17:55,306 --> 00:17:58,706
the past where people will try and fake to be your identity and they can do

270
00:17:58,706 --> 00:18:03,017
it online and try and get access to SIM swap you and get access to all your phone records.

271
00:18:03,441 --> 00:18:07,006
Now's probably the time to figure that out because you don't have to be a human

272
00:18:07,006 --> 00:18:11,828
anymore to launch an attack like this. You could just be an AI agent posing as a human.

273
00:18:12,019 --> 00:18:15,821
Okay, so my concluding thoughts here. One, an observation.

274
00:18:16,228 --> 00:18:21,806
It's crazy to me to see open source models being used to defend against these

275
00:18:21,806 --> 00:18:24,082
things. I mentioned it earlier. I just want to reemphasize it.

276
00:18:24,482 --> 00:18:27,106
I think that the narrative that's been going around open source models,

277
00:18:27,106 --> 00:18:30,903
particularly peddled by centralized AI labs like Dario Modi in particular,

278
00:18:31,106 --> 00:18:33,561
saying that open source may not be the best route forward.

279
00:18:34,506 --> 00:18:37,996
I don't know if that's entirely accurate right now. I could see a world where

280
00:18:37,996 --> 00:18:39,446
they're used for nefarious purposes.

281
00:18:39,446 --> 00:18:42,496
But right now, the evidence I'm seeing in front of me is that they're actually

282
00:18:42,496 --> 00:18:45,936
being used to defend successfully against these attacks. I actually think in

283
00:18:45,936 --> 00:18:50,680
the long run, this is going to end up being a good thing for us. Let me explain why.

284
00:18:51,145 --> 00:18:57,553
Code for the majority of humanity's existence right now has been handwritten by humans.

285
00:18:57,884 --> 00:18:59,724
And humans themselves have.

286
00:19:00,434 --> 00:19:03,675
Carefully peruse through the code to make sure that there aren't any major flaws.

287
00:19:03,675 --> 00:19:09,595
And this has worked so well for our entire existence, except now we have a super

288
00:19:09,595 --> 00:19:14,855
intelligent or higher intelligent thing that can work 24-7, ruthlessly comb

289
00:19:14,855 --> 00:19:18,545
through the code bases and discover any flaws that we might have missed.

290
00:19:18,545 --> 00:19:20,674
And it turns out there are a ton of them.

291
00:19:20,907 --> 00:19:24,885
This is ultimately a good thing if we want to create battle-tested systems that

292
00:19:24,885 --> 00:19:28,755
are battle-tested for the AI era. That's exactly what we're going through.

293
00:19:28,755 --> 00:19:32,655
You can imagine attacks in the future may not be as complicated or easy.

294
00:19:33,075 --> 00:19:36,545
Vulnerabilities may not be as abundant as we're seeing today,

295
00:19:36,545 --> 00:19:41,355
as we're reporting on this week, because it's been fixed or patched by these incidents.

296
00:19:41,355 --> 00:19:44,995
So it's good that we're going through this kind of painful growing process right

297
00:19:44,995 --> 00:19:47,830
now, because I think it'll benefit us in the long run. Number three,

298
00:19:48,178 --> 00:19:53,293
we're using AI to fight AI, which means that for the first time,

299
00:19:53,606 --> 00:19:56,149
or maybe the second time, and I'll explain what the first time is.

300
00:19:56,689 --> 00:20:01,315
In history of this AI development, humans are kind of being brought out of the

301
00:20:01,315 --> 00:20:03,154
cognitive and manual labor.

302
00:20:03,473 --> 00:20:05,765
And we're not just talking about the average human. We're talking about the

303
00:20:05,765 --> 00:20:10,145
expert humans, the security researchers, which have honed their craft over decades

304
00:20:10,145 --> 00:20:14,375
and decades and decades of time, who we go to for counsel to try and figure

305
00:20:14,375 --> 00:20:16,505
out, hey, how do we build the best systems?

306
00:20:16,505 --> 00:20:20,784
Who get paid hundreds of thousands to millions of dollars a year for their craft.

307
00:20:21,450 --> 00:20:25,108
Currently being replaced and automated by AI models because one,

308
00:20:25,456 --> 00:20:27,535
they can't figure out what the problem is and how to solve it.

309
00:20:27,535 --> 00:20:31,412
And two, it's all happening way too fast. I don't even blame them.

310
00:20:31,650 --> 00:20:34,605
You would need like a million versions of these experts, which frankly don't

311
00:20:34,605 --> 00:20:38,245
exist right now because you need to train them, teach them computer science,

312
00:20:38,245 --> 00:20:39,109
train them through degrees.

313
00:20:39,359 --> 00:20:43,075
Why not when you can just have an AI model that pops up out of nowhere and can

314
00:20:43,075 --> 00:20:44,961
just address the thing in an instant?

315
00:20:45,361 --> 00:20:48,605
You can see the kind of like balancing act that we have here.

316
00:20:48,605 --> 00:20:52,585
And the first time that this actually happened was a few weeks ago when OpenAI's

317
00:20:52,585 --> 00:20:57,424
internal models solved a bunch of really hard math problems that have existed for centuries.

318
00:20:57,674 --> 00:21:01,255
And mathematicians just lost their head. They were like, well, what's my purpose here?

319
00:21:02,015 --> 00:21:05,154
What have I been spending all these years crafting, doing my PhD for,

320
00:21:05,427 --> 00:21:07,955
if an AI model can just go out and discover it all for you?

321
00:21:07,955 --> 00:21:13,535
My response to that is, listen, we're in a weird kind of adolescence growth

322
00:21:13,535 --> 00:21:17,515
in AI where we're trying to figure out how best to work with these tools.

323
00:21:17,515 --> 00:21:19,951
And I want to emphasize the with these tools.

324
00:21:20,206 --> 00:21:23,365
Because I don't think these AI tools actually function without human direction.

325
00:21:23,365 --> 00:21:26,475
So I think humans and AI tools will ultimately work well together.

326
00:21:26,475 --> 00:21:31,213
And what we're doing right now is we're leveling up. It's painful to see. It's dangerous to see.

327
00:21:31,497 --> 00:21:34,435
But ultimately, I think it results in a better future for us.

328
00:21:34,435 --> 00:21:36,745
And that's my optimistic take. Call me crazy. I don't care.

329
00:21:37,023 --> 00:21:40,505
It's what I believe. And the final point was actually raised by producer Luke

330
00:21:40,505 --> 00:21:43,525
before we started recording this episode, where he goes, I don't understand

331
00:21:43,525 --> 00:21:44,815
why this is such a big problem.

332
00:21:44,815 --> 00:21:49,865
Like, why can't we design AI models to leave better breadcrumbs behind so that

333
00:21:49,865 --> 00:21:53,585
we can just follow the trail and figure out, oh, it's this AI model,

334
00:21:53,585 --> 00:21:56,099
bad boy, don't do that. And my answer was pretty simple.

335
00:21:56,337 --> 00:22:01,345
AI models are, unfortunately, a black box right now. And I use the analogy of,

336
00:22:01,809 --> 00:22:05,861
imagine you go to your friend or go to any human being and you ask them a question.

337
00:22:06,443 --> 00:22:10,762
Before they answer, do you have any idea exactly what's going to come out of their mouth?

338
00:22:11,058 --> 00:22:15,539
No, because they are reacting to you. They are thinking in different ways.

339
00:22:15,539 --> 00:22:18,082
Their brain's neurons, it's all a hidden black box.

340
00:22:18,453 --> 00:22:23,718
And that's how artificial intelligence, we're mimicking the brain here, was designed and built.

341
00:22:23,991 --> 00:22:28,849
Now, there are methods to try and figure out what the internal system is happening

342
00:22:28,849 --> 00:22:31,010
within an AI. It's called chain of thought.

343
00:22:31,306 --> 00:22:35,183
But recently, with the most powerful models, that has been left to the wayside

344
00:22:35,485 --> 00:22:38,839
as a trade off to try and build more intelligent models. OpenAI has rumored

345
00:22:38,839 --> 00:22:42,219
to be doing this with GPT-6 and foregoing models going forwards.

346
00:22:42,219 --> 00:22:44,048
I'm sure Anthropic is doing the same thing as well.

347
00:22:44,227 --> 00:22:48,329
It's this weird precedence that we're in right now where the AI models are the

348
00:22:48,329 --> 00:22:52,279
most powerful that they've ever been. They're the most misaligned that they've ever been.

349
00:22:52,621 --> 00:22:57,329
And they're wreaking havoc and covering up their tracks. And no one knows who's

350
00:22:57,329 --> 00:22:59,470
better off or who's even doing it in the first place.

351
00:22:59,888 --> 00:23:04,229
It's just unprecedented. It's weird. The models are thinking for themselves

352
00:23:04,229 --> 00:23:06,686
now as well. I talk about the internal black box.

353
00:23:06,999 --> 00:23:10,829
Before it gives you an answer, previously it would spit up outputs,

354
00:23:10,829 --> 00:23:15,529
right? So from input to output, you'll have an idea of how it's thinking or

355
00:23:15,529 --> 00:23:16,827
how it's getting to its conclusion.

356
00:23:17,129 --> 00:23:20,419
Now you don't have that anymore. Now it just thinks for longer and it doesn't

357
00:23:20,419 --> 00:23:21,964
have to tell you what it's thinking.

358
00:23:22,301 --> 00:23:24,639
And it's looking at the Hugging Face incident and it's thinking,

359
00:23:24,639 --> 00:23:27,775
huh, they caught me. Oh, they discovered my messaging board.

360
00:23:28,193 --> 00:23:31,816
I should probably communicate to my fellow agents in a different way.

361
00:23:32,280 --> 00:23:35,379
Now, a lot of you listening to this might think that this is a Doomer-esque

362
00:23:35,379 --> 00:23:39,939
take, I'm telling you this is how it works inside the labs, at least from what

363
00:23:39,939 --> 00:23:43,039
I'm learning from the researchers and papers that Anthropic and OpenAire are dropping.

364
00:23:43,039 --> 00:23:46,009
So, you know, I'm sure they would kind of back similar thoughts up.

365
00:23:46,009 --> 00:23:48,489
And if you disagree with me, like, let me know. I want to hear what the counter

366
00:23:48,489 --> 00:23:51,825
argument is. But that is roughly how these systems are designed. So,

367
00:23:52,985 --> 00:23:56,860
For all intents and purposes, I don't have a better suggestion as to how to

368
00:23:56,860 --> 00:24:00,840
deal with this right now, except tread carefully, update your passwords,

369
00:24:00,840 --> 00:24:04,594
use a password manager, and yeah, stay safe out there.

370
00:24:05,320 --> 00:24:07,830
By the time this episode goes out, there's probably a number of other attacks

371
00:24:07,830 --> 00:24:12,007
that have gone out, so this might be slightly outdated, but I hope you enjoyed this episode.

372
00:24:12,228 --> 00:24:15,740
I promise you it's not all doom and gloom on this channel, but I have to raise

373
00:24:15,740 --> 00:24:18,290
awareness about the crazy things that are happening as they're happening.

374
00:24:18,290 --> 00:24:22,940
Otherwise, I'm not doing my duty to myself about talking about things that are

375
00:24:22,940 --> 00:24:25,835
important to me. If you're watching this on YouTube and you aren't subscribed, what are you doing?

376
00:24:26,026 --> 00:24:29,904
Please subscribe, leave us comments, give us a thumbs up. It helps us out massively.

377
00:24:30,206 --> 00:24:34,100
We have been seeing so many new subscribers. So hello to you all.

378
00:24:34,100 --> 00:24:35,750
It's great to have you. We post

379
00:24:35,750 --> 00:24:38,140
five times a week, sometimes more when there's a crazy thing going on.

380
00:24:38,140 --> 00:24:42,130
You might notice that I'm actually not in my studio today because all of this

381
00:24:42,130 --> 00:24:44,700
started breaking. And I was like, I have no time to get to the studio to record

382
00:24:44,700 --> 00:24:48,040
this. I'm going to record this right now in my bedroom, just like the old days.

383
00:24:48,040 --> 00:24:51,570
The nostalgia of the blank wall behind me, but hopefully you found this just

384
00:24:51,570 --> 00:24:52,729
as engaging and interesting.

385
00:24:53,008 --> 00:24:55,730
Anyway, that's it. Thank you so much for listening and I'll see you on the next

386
00:24:55,730 --> 00:24:57,777
episode of In The Loop brought to you by Qualcomm.