1 00:00:00,020 --> 00:00:05,480 Ejaaz: Last week, Amazon's entire platform crashed for six hours. No one could shop, 2 00:00:05,680 --> 00:00:07,420 Ejaaz: buy anything, they couldn't even see prices. 3 00:00:07,700 --> 00:00:12,640 Ejaaz: The reason was because a junior developer had submitted an AI-generated piece 4 00:00:12,640 --> 00:00:18,280 Ejaaz: of code which crashed the entire platform, and it cost them millions and millions of dollars. 5 00:00:18,860 --> 00:00:22,500 Ejaaz: Now, Anthropic, the creators of Claude Code, which is what Amazon was using 6 00:00:22,500 --> 00:00:24,320 Ejaaz: to create AI-generated code, 7 00:00:24,730 --> 00:00:28,130 Ejaaz: also had a similar issue where their entire platform has been suffering from 8 00:00:28,130 --> 00:00:29,730 Ejaaz: outages this entire week. 9 00:00:29,910 --> 00:00:33,290 Ejaaz: They actually also released a new product called Code Review, 10 00:00:33,590 --> 00:00:39,350 Ejaaz: which helps use AI to help fix the code problems that their own model is fixing. 11 00:00:39,450 --> 00:00:42,730 Ejaaz: It's all getting incredibly complex right now. And Amazon tried to hide the 12 00:00:42,730 --> 00:00:46,650 Ejaaz: entire thing from a Financial Times reporter. It's all pretty crazy. 13 00:00:46,650 --> 00:00:51,970 Ejaaz: And it forces us to answer the new narrative, which is AI-generated code isn't going anyway. 14 00:00:52,130 --> 00:00:58,670 Ejaaz: Demand is at an all-time high, but have we hit a wall? Has it become too dangerous to use AI to code? 15 00:00:58,970 --> 00:01:01,710 Josh: Yeah, and there's this interesting phenomenon happening. As these AI coding 16 00:01:01,710 --> 00:01:04,470 Josh: abilities in general become better, where currently they account for, 17 00:01:04,550 --> 00:01:06,610 Josh: what, about 4% of total GitHub commits. 18 00:01:06,750 --> 00:01:11,430 Josh: The expectation is at the end of the year, they will account for 20% of total GitHub commits. 19 00:01:11,570 --> 00:01:16,790 Josh: And there is this increasing reliance on these AI tools, but that creates these 20 00:01:16,790 --> 00:01:21,110 Josh: key choke points and points of failure that have a significant effect. 21 00:01:21,190 --> 00:01:24,330 Josh: I mean, And this Amazon outage was a huge deal that was pushed by a single person. 22 00:01:24,390 --> 00:01:27,190 Josh: And then within Claude Code, they've been having a lot of downtime. 23 00:01:27,450 --> 00:01:30,190 Josh: And the Anthropic, I mean, we were trying to use the programs today and it was 24 00:01:30,190 --> 00:01:33,270 Josh: down. The servers weren't working quite right. So there's a lot of these growing 25 00:01:33,270 --> 00:01:34,110 Josh: pains that are happening. 26 00:01:34,370 --> 00:01:38,230 Josh: And it seems like we're running into more issues faster than people are trying to mitigate it. 27 00:01:38,310 --> 00:01:42,810 Josh: So that's where I suspect this Claude Code checking agent comes in that we're 28 00:01:42,810 --> 00:01:45,150 Josh: going to cover later. But this Amazon story was pretty fascinating. 29 00:01:45,290 --> 00:01:48,130 Josh: Like this cost Amazon billions of dollars. 30 00:01:48,130 --> 00:01:53,110 Ejaaz: Yeah, so let me walk you through the timeline for this, because this actually 31 00:01:53,110 --> 00:01:57,810 Ejaaz: isn't the first outage Amazon's experience because of an AI-generated piece of code. 32 00:01:58,010 --> 00:02:04,230 Ejaaz: So back in early December, actually November of 2025, they took a really hard-line 33 00:02:04,230 --> 00:02:06,970 Ejaaz: approach and a new policy was invoked, which was... 34 00:02:07,340 --> 00:02:13,620 Ejaaz: I want 80% of all Amazon's code generated to be AI generated. 35 00:02:13,740 --> 00:02:17,480 Ejaaz: And this was their goal to be achieved by the end of 2026. 36 00:02:17,780 --> 00:02:22,040 Ejaaz: Now, this flips, completely flips from a company that I think employs like hundreds 37 00:02:22,040 --> 00:02:26,400 Ejaaz: of thousands of engineers and wants them all to kind of like handwrite or hand type the code. 38 00:02:26,520 --> 00:02:29,600 Ejaaz: So this is a pretty aggressive flip. Amazon's been laying off tens of thousands 39 00:02:29,600 --> 00:02:32,760 Ejaaz: of people. So this kind of is in trend with what they wanted to do. 40 00:02:33,520 --> 00:02:37,840 Ejaaz: In the middle of December 2025, they experienced their first outage. 41 00:02:37,920 --> 00:02:39,620 Ejaaz: It was 13 hours back then. 42 00:02:39,760 --> 00:02:42,400 Ejaaz: So this is all adding up, by the way, we're talking about 10s of millions of 43 00:02:42,400 --> 00:02:46,100 Ejaaz: dollars, then in late 2025. So at the end of December, there was another outage. 44 00:02:46,280 --> 00:02:48,500 Ejaaz: And then we have the outages that we're speaking about today. 45 00:02:49,120 --> 00:02:53,960 Ejaaz: And the issue that this flags is, although AI is like a really useful tool to 46 00:02:53,960 --> 00:02:59,340 Ejaaz: generate code, and it finds bugs, it actually might create more problems than you'd expected. 47 00:02:59,540 --> 00:03:03,140 Ejaaz: Because the issue that they're seeing is junior developers that come in that 48 00:03:03,140 --> 00:03:08,080 Ejaaz: don't understand Amazon's code base, just kind of use AI to run like code to 49 00:03:08,080 --> 00:03:12,140 Ejaaz: run autonomously, figure out what they want to create, and then they just submit 50 00:03:12,140 --> 00:03:14,260 Ejaaz: it without actually reviewing and understanding it. 51 00:03:14,360 --> 00:03:18,640 Ejaaz: And if this goes unguarded, it creates and results in issues like this. 52 00:03:18,920 --> 00:03:21,780 Josh: Yeah. And there's a problem that's starting to happen now where agents are creating 53 00:03:21,780 --> 00:03:24,680 Josh: code far faster than humans can keep up and check it. 54 00:03:24,800 --> 00:03:28,120 Josh: So it becomes this impossibility where if you want to move at the velocity that 55 00:03:28,120 --> 00:03:31,920 Josh: AI enables, you are simply unable to keep up with the change log of what's happening. 56 00:03:32,140 --> 00:03:36,340 Josh: So you have to defer some sort of trust level to this AI, to its ability to 57 00:03:36,340 --> 00:03:38,460 Josh: check itself, to run tests, to verify that it actually works. 58 00:03:38,560 --> 00:03:41,540 Josh: And in some cases, it doesn't. I mean, I'm sure some people listening to this 59 00:03:41,540 --> 00:03:42,840 Josh: experienced the outage. 60 00:03:42,940 --> 00:03:46,880 Josh: And it wasn't just an AWS outage. This is Amazon, the actual storefront, 61 00:03:47,020 --> 00:03:48,560 Josh: where my dad actually texted me. 62 00:03:48,660 --> 00:03:51,100 Josh: He was like, am I doing something wrong? Can you place this order for me? 63 00:03:51,160 --> 00:03:52,120 Josh: Because it's not going through. 64 00:03:52,280 --> 00:03:55,640 Josh: And then I went to check myself and the whole service was down. I went on Twitter. 65 00:03:55,800 --> 00:03:58,440 Josh: I saw Casey Neistat was posting a bunch of photos about this. It 66 00:03:58,440 --> 00:04:01,280 Josh: was this big deal because anyone who was trying to order anything from 67 00:04:01,280 --> 00:04:04,080 Josh: amazon the entire storefront was just totally offline so 68 00:04:04,080 --> 00:04:07,080 Josh: aws the web services runs a significant percentage 69 00:04:07,080 --> 00:04:10,880 Josh: of the internet amazon the storefront runs a significant percentage of the e-commerce 70 00:04:10,880 --> 00:04:14,920 Josh: and these things have been going down at an increasing rate due to these tools 71 00:04:14,920 --> 00:04:18,580 Josh: like cloud code so they're moving much faster but they're they're breaking things 72 00:04:18,580 --> 00:04:22,080 Josh: this is like the early zuck facebook mantra it's like move fast and break things 73 00:04:22,080 --> 00:04:24,340 Josh: they're now taking it to the extreme and 74 00:04:24,790 --> 00:04:28,270 Josh: As we become increasingly reliant on these tools, we might see this start to, 75 00:04:28,270 --> 00:04:30,170 Josh: I mean, permeate even further than just Amazon. 76 00:04:30,630 --> 00:04:35,090 Ejaaz: Well, the weirdest part about this is that AI kind of has moved from this assistive 77 00:04:35,090 --> 00:04:39,870 Ejaaz: tool to now being the foundational bedrock for a lot of these different like 78 00:04:39,870 --> 00:04:43,030 Ejaaz: services that you've just mentioned. Like AWS runs the entire internet. 79 00:04:43,190 --> 00:04:46,710 Ejaaz: Every single business that you interact with online probably uses AWS on the 80 00:04:46,710 --> 00:04:49,490 Ejaaz: back end. So if they go down, then your business goes down as well. 81 00:04:49,590 --> 00:04:50,670 Ejaaz: That's where we get all these outages. 82 00:04:51,390 --> 00:04:55,390 Ejaaz: Amazon had some pretty severe repercussions. They have now done a complete 180 83 00:04:55,390 --> 00:04:58,310 Ejaaz: on their policy of generating 80% of their code via AI. 84 00:04:58,630 --> 00:05:02,290 Ejaaz: And they've now said, if you're a junior developer or engineer working at Amazon, 85 00:05:02,510 --> 00:05:06,910 Ejaaz: you now need to get your manager's permission to submit code. 86 00:05:07,190 --> 00:05:12,950 Ejaaz: So this is so tough because you've now gone from trying to automate the entire 87 00:05:12,950 --> 00:05:15,570 Ejaaz: thing to now making it incredibly worse. 88 00:05:15,730 --> 00:05:18,630 Ejaaz: So you're using AI to kind of generate the code, but then you still have to 89 00:05:18,630 --> 00:05:23,610 Ejaaz: rely on a human constraint. and there are fewer managers than they are like junior developers. 90 00:05:23,690 --> 00:05:27,810 Ejaaz: So it all just becomes really bogged up as a pipeline to kind of shipping changes. 91 00:05:27,930 --> 00:05:30,050 Ejaaz: So I think this is going to slow down Amazon massively. 92 00:05:30,250 --> 00:05:33,710 Ejaaz: But if it's for the result of, you know, saving your company tens of millions 93 00:05:33,710 --> 00:05:36,550 Ejaaz: of dollars, fair play. But yeah, it's a pretty hard line approach. 94 00:05:36,730 --> 00:05:39,290 Josh: And this all seems kind of connected to the trends that we've been seeing, 95 00:05:39,410 --> 00:05:41,230 Josh: right? Particularly around jobs and distribution. 96 00:05:41,430 --> 00:05:45,150 Josh: It's like a lot of people are cutting jobs, which means they're increasing the 97 00:05:45,150 --> 00:05:49,090 Josh: reliance on these AI tools. But if the result is that the actual output of these 98 00:05:49,090 --> 00:05:53,830 Josh: ai tools is causing a lot of damage it creates this this tension and 99 00:05:53,950 --> 00:05:57,430 Josh: And it seems like the AI labs themselves, Anthropic, OpenAI, 100 00:05:57,710 --> 00:05:59,790 Josh: they cannot hire talented people fast enough. 101 00:05:59,950 --> 00:06:03,250 Josh: They are looking to hire every engineer under the sun who is competent and capable 102 00:06:03,250 --> 00:06:04,810 Josh: of building great products. 103 00:06:04,870 --> 00:06:09,690 Josh: But a lot of other companies are deferring that workload to these AI labs, 104 00:06:09,690 --> 00:06:13,410 Josh: which is starting to create this interesting dynamic where the job market is 105 00:06:13,410 --> 00:06:16,970 Josh: suffering, but the actual productive output of these companies is not to an extent. 106 00:06:17,170 --> 00:06:19,790 Josh: And that's operating under the assumption that these code tools will get better. 107 00:06:19,870 --> 00:06:23,010 Josh: But if Amazon and a lot of other companies start putting in these thresholds 108 00:06:23,010 --> 00:06:27,130 Josh: that, again, bottleneck and throttle that amount of AI production capability, 109 00:06:27,530 --> 00:06:29,950 Josh: like that probably doesn't help the cause a whole lot. 110 00:06:30,150 --> 00:06:35,890 Ejaaz: There's also the angle that AWS or just data centers in general are becoming 111 00:06:35,890 --> 00:06:41,830 Ejaaz: like a really valuable or national security threat level asset, right? 112 00:06:42,070 --> 00:06:45,930 Ejaaz: So if we look at the current Iran conflict that's being waged between the US 113 00:06:45,930 --> 00:06:53,510 Ejaaz: and Iran, Iran actually targeted strikes specifically to Amazon's AWS centers 114 00:06:53,510 --> 00:06:55,390 Ejaaz: out in the UAE and Bahrain. 115 00:06:55,590 --> 00:07:01,530 Ejaaz: And this caused a bunch of the outages as well. So it's not even just like AI coding level threats. 116 00:07:01,630 --> 00:07:03,950 Ejaaz: This is becoming a geopolitical weapon at this point. 117 00:07:04,090 --> 00:07:08,190 Josh: It makes sense. Every single story we tell, like this Game of Thrones narrative 118 00:07:08,190 --> 00:07:10,290 Josh: that's kind of existed throughout the history of Limitless, 119 00:07:10,450 --> 00:07:14,130 Josh: it's just elevating itself higher and higher to the global stage, 120 00:07:14,250 --> 00:07:17,410 Josh: where now it's impossible to have any sort of conflict or 121 00:07:17,410 --> 00:07:20,130 Josh: large decision without ai being in the middle of it like when you think 122 00:07:20,130 --> 00:07:23,390 Josh: about this conflict what are the bigger things it's this like anthropic 123 00:07:23,390 --> 00:07:26,490 Josh: deal with the pentagon they're now striking the aws data centers it's coming 124 00:07:26,490 --> 00:07:29,910 Josh: for the infrastructure that's building these tools that are so powerful but 125 00:07:29,910 --> 00:07:33,890 Josh: this isn't just happening on the worldwide stage this is happening on the consumer 126 00:07:33,890 --> 00:07:36,990 Josh: and commercial level too you were mentioning something about lovable and how 127 00:07:36,990 --> 00:07:39,950 Josh: much they've grown recently could you just explain lovable for the people who 128 00:07:39,950 --> 00:07:42,510 Josh: aren't familiar and what's what how big they have grown recently it's crazy 129 00:07:43,050 --> 00:07:46,770 Ejaaz: Okay, so typically the people who code, this might shock some of you, 130 00:07:47,110 --> 00:07:48,910 Ejaaz: are the people who have learned to code. 131 00:07:49,030 --> 00:07:52,630 Ejaaz: They're very technically focused work. But there are people like you and I and 132 00:07:52,630 --> 00:07:55,210 Ejaaz: a bunch of our friends that don't necessarily know how to code, 133 00:07:55,330 --> 00:07:56,410 Ejaaz: but still want to do that. 134 00:07:56,530 --> 00:07:58,750 Ejaaz: And that's what resulted in Vibe coding, right? 135 00:07:59,070 --> 00:08:04,730 Ejaaz: Now, Claude Code and products like Cursor are still very oriented for technical folks. 136 00:08:05,070 --> 00:08:08,950 Ejaaz: A platform like Lovable is for people who have zero experience coding, 137 00:08:09,110 --> 00:08:12,590 Ejaaz: but still want to code things. So the Lovable platform is actually a really 138 00:08:12,590 --> 00:08:14,770 Ejaaz: cool platform where you just type whatever you want. 139 00:08:14,870 --> 00:08:17,770 Ejaaz: Like I want to create an app that tracks my fitness reps or whatever that might 140 00:08:17,770 --> 00:08:21,690 Ejaaz: be or create or suggest cooking recipes for my type of cuisine. 141 00:08:21,790 --> 00:08:23,970 Ejaaz: And it does so in a couple of minutes. 142 00:08:24,150 --> 00:08:26,610 Ejaaz: And it's really intuitive. It makes it super easy. 143 00:08:26,810 --> 00:08:30,890 Ejaaz: And they have the additional perk of being able to deploy it live as a website 144 00:08:30,890 --> 00:08:34,210 Ejaaz: or an app that you can publish on the app store super easily. 145 00:08:34,330 --> 00:08:35,790 Ejaaz: So they have this end-to-end experience. 146 00:08:36,230 --> 00:08:40,230 Ejaaz: Now, Lovable, when it started, had like a rocket-type trajectory. 147 00:08:40,390 --> 00:08:44,210 Ejaaz: Within the first 12 months, I think they hit $200 million ARR, 148 00:08:44,290 --> 00:08:47,450 Ejaaz: which is just like an insane growth. All of these AI companies are blowing up. 149 00:08:47,730 --> 00:08:53,390 Ejaaz: Now, in the last month alone, they've added an extra $100 million of AR. 150 00:08:53,570 --> 00:08:55,850 Ejaaz: They were at $300 million. Now they're at $400 million. 151 00:08:56,070 --> 00:08:58,930 Ejaaz: So that's like roughly like just over 30% growth. 152 00:08:59,190 --> 00:09:02,930 Ejaaz: Just insane amount of demand. And that's the point I want to make. 153 00:09:03,190 --> 00:09:07,450 Ejaaz: Like when I look at these Amazon outages and Amazon restricting their developers 154 00:09:07,450 --> 00:09:12,490 Ejaaz: from using Vibe Coding to improve their product, I don't think this is necessarily 155 00:09:12,490 --> 00:09:14,050 Ejaaz: a wave that they can stop. 156 00:09:14,250 --> 00:09:17,710 Ejaaz: So they should stop trying to implement policies that are restricting people 157 00:09:17,710 --> 00:09:22,030 Ejaaz: from using it and instead try to figure out what is a better way to implement 158 00:09:22,030 --> 00:09:24,610 Ejaaz: AI-generated code because this thing's not going away. 159 00:09:24,810 --> 00:09:28,710 Ejaaz: Lovable, Cursor, Claw code are all skyrocketing and we're not going to stop that. 160 00:09:29,040 --> 00:09:32,600 Josh: And I think a lot of the, I mean, aside of Claude Code, right, is Lovable and Cursor. 161 00:09:32,900 --> 00:09:35,920 Josh: They're kind of model agnostic. They'll use whatever model you put. 162 00:09:36,080 --> 00:09:39,800 Josh: So I was looking at Polymarket earlier today because they have markets for which 163 00:09:39,800 --> 00:09:42,740 Josh: one of these models is actually going to be the best in which time. 164 00:09:42,860 --> 00:09:45,660 Josh: And this is interesting because if you looked at these charts towards the middle 165 00:09:45,660 --> 00:09:48,360 Josh: of January, you'd notice that Anthropic was kind of the favorite. 166 00:09:48,540 --> 00:09:50,760 Josh: Anthropic was looking like they were going to have the best model. 167 00:09:51,000 --> 00:09:54,960 Josh: But the reality now is that by March 31st, there's a very clear divide that 168 00:09:54,960 --> 00:09:59,380 Josh: has happened over the last six to eight weeks. in OpenAI having an 85% chance 169 00:09:59,380 --> 00:10:00,400 Josh: of having the best model. 170 00:10:00,680 --> 00:10:04,020 Josh: Does that mean there's going to be something new or is that just the current model? We don't know. 171 00:10:04,540 --> 00:10:09,220 Josh: Currently they're at 5.4, which is fantastic. I think everyone is kind of unanimously 172 00:10:09,220 --> 00:10:10,600 Josh: decided that it's the best for coding. 173 00:10:11,220 --> 00:10:14,940 Josh: But then there's also a fun following market here about Claude going down because 174 00:10:14,940 --> 00:10:18,180 Josh: it appears as if Claude is really just not doing as good as it needs to be. 175 00:10:18,260 --> 00:10:21,100 Josh: It's not as stable. We were trying to use it this morning. It wasn't working. 176 00:10:21,380 --> 00:10:24,360 Josh: They're clearly experiencing growing pains because they have all these new users 177 00:10:24,360 --> 00:10:26,380 Josh: and they're having to throttle reasoning. 178 00:10:26,660 --> 00:10:30,500 Josh: They're having to limit the amount of searches that people are able to use. 179 00:10:30,620 --> 00:10:33,700 Josh: I'm sure the researchers who use these to train the models are not happy. 180 00:10:33,920 --> 00:10:42,600 Josh: And it looks like based on Polymarket, we'll be seeing somewhere between five to nine more outages in 181 00:10:42,600 --> 00:10:44,420 Ejaaz: The month of March. Yeah, basically every single day this month. 182 00:10:44,680 --> 00:10:48,220 Josh: Yeah, like it seems like they're having problems. And this is proof. 183 00:10:48,380 --> 00:10:51,480 Josh: I mean, you can actually bet on the problems that they're having and make some 184 00:10:51,480 --> 00:10:54,020 Josh: money on it, right? This is what, $26,000 of volume. 185 00:10:54,160 --> 00:10:57,160 Josh: So thank you, Polymarket, for sponsoring this segment. And I guess now we can 186 00:10:57,160 --> 00:11:00,120 Josh: get onto the solution, which is what Anthropic is starting to roll out. 187 00:11:00,240 --> 00:11:03,120 Josh: But they're doing it in a way that's a little less than desirable. 188 00:11:03,940 --> 00:11:10,180 Ejaaz: Yeah. So the main instigators of this entire AI coding debacle is Anthropic. 189 00:11:10,380 --> 00:11:11,440 Ejaaz: They created Claude Code. 190 00:11:11,800 --> 00:11:16,840 Ejaaz: Amazon owns 21% of Anthropic, which is just crazy to say out loud. 191 00:11:17,560 --> 00:11:22,520 Ejaaz: And so they use Anthropic's model, Claude Code, to do all their AI code generation. 192 00:11:22,960 --> 00:11:27,100 Ejaaz: But Anthropic decided, hmm, instead of trying to make our AI coding models better, 193 00:11:27,300 --> 00:11:32,260 Ejaaz: let's just release a new product called code review, which will review your AI generated code. 194 00:11:32,360 --> 00:11:36,940 Ejaaz: So you now have Claude generating the AI code and then also reviewing the code 195 00:11:36,940 --> 00:11:38,820 Ejaaz: as a separate product that you pay for. 196 00:11:38,980 --> 00:11:43,920 Ejaaz: And it's all for the beautiful price of, I think it's like 15 to 20 bucks. Where is it? Yeah. 197 00:11:44,120 --> 00:11:48,340 Ejaaz: 15 to 20 bucks per review, which some might say is quite expensive. 198 00:11:48,840 --> 00:11:53,300 Josh: And they scale based on the pull request complexity. So that number can get higher. 199 00:11:53,440 --> 00:11:58,260 Josh: And what I find it funny is they're selling the problem where they're selling 200 00:11:58,260 --> 00:12:00,540 Josh: you a model, you use it to code, it's going to create problems. 201 00:12:00,840 --> 00:12:04,180 Josh: And then they're selling you a separate package for the solution where, 202 00:12:04,360 --> 00:12:06,060 Josh: oh, you have problems in our code. 203 00:12:06,180 --> 00:12:10,560 Josh: Well, here for another $25, $50, whatever it may be, we'll actually do a code 204 00:12:10,560 --> 00:12:12,280 Josh: review and we'll fix the problems that we've created. 205 00:12:12,580 --> 00:12:18,460 Josh: And that creates this uniquely disincentivized incentive where there's no real 206 00:12:18,460 --> 00:12:21,500 Josh: reason for Claude to want to fix problems when they could just sell this package 207 00:12:21,500 --> 00:12:23,900 Josh: on top to remove the problem. 208 00:12:23,900 --> 00:12:26,740 Josh: And this comes in the face of chat gpt 209 00:12:26,740 --> 00:12:30,100 Josh: and open ai doing the polar opposite where they're actually offering these 210 00:12:30,100 --> 00:12:32,980 Josh: for free and here's rohan from open ai if you 211 00:12:32,980 --> 00:12:36,380 Josh: want ai code review but don't want to pay 25 per review check 212 00:12:36,380 --> 00:12:40,680 Josh: out codex review it leverages frontier codex models finds complex issues and 213 00:12:40,680 --> 00:12:46,240 Josh: 100 usage based most runs should cost one dollar or less so here we have another 214 00:12:46,240 --> 00:12:50,780 Josh: head-to-head collision happening where chat gpt open ai is going head-to-head 215 00:12:50,780 --> 00:12:53,880 Josh: with cloud code and And it seems like Cloud Code is the favorite for a lot of people, 216 00:12:53,920 --> 00:12:58,100 Josh: but I've noticed a lot of the more technical people on my timeline who are building 217 00:12:58,100 --> 00:13:02,620 Josh: pretty hardcore things have been really relying on 5.4 and using codecs much more. 218 00:13:03,040 --> 00:13:08,000 Ejaaz: Yeah, so to set the precedence, OpenAI and Anthropic now are neck and neck at 219 00:13:08,000 --> 00:13:09,520 Ejaaz: building the best coding models. 220 00:13:09,900 --> 00:13:14,320 Ejaaz: And Codex 5.4 from OpenAI has taken the lead. A lot of engineers are now saying 221 00:13:14,320 --> 00:13:16,980 Ejaaz: it's way better than Opus 4.6, as you just mentioned. 222 00:13:17,220 --> 00:13:21,000 Ejaaz: In terms of these security code review products that each company offers, 223 00:13:21,520 --> 00:13:23,980 Ejaaz: OpenAI is instinctively cheaper. 224 00:13:24,220 --> 00:13:28,020 Ejaaz: If you have a $20 a month subscription, you now get access to this code review 225 00:13:28,020 --> 00:13:31,860 Ejaaz: tool for no extra cost. So it's automatically a better tool to use. 226 00:13:31,980 --> 00:13:35,740 Ejaaz: Now, in terms of quality, in terms of how good it is at code review, 227 00:13:35,940 --> 00:13:38,880 Ejaaz: we don't know because they haven't made any of their datasets public. 228 00:13:39,220 --> 00:13:41,340 Ejaaz: Anthropic has, and it's pretty damn good. 229 00:13:41,720 --> 00:13:46,100 Ejaaz: Also, if you compare Anthropic's product to traditional SaaS vendors that have 230 00:13:46,100 --> 00:13:49,220 Ejaaz: these kind of AppSec security tools, it is still way cheaper. 231 00:13:49,360 --> 00:13:52,460 Ejaaz: A lot of people were in my comments, actually, because I tweeted about this 232 00:13:52,460 --> 00:13:53,960 Ejaaz: saying it was cheaper. They were like, no, it's not. 233 00:13:54,600 --> 00:13:57,520 Ejaaz: I could use this SaaS tool. And I'm like, yeah, but you're not factoring in 234 00:13:57,520 --> 00:14:01,060 Ejaaz: the engineering hours, they spend like 30 to 60 minutes per bug. 235 00:14:01,180 --> 00:14:02,600 Ejaaz: That adds up to about $100. 236 00:14:02,800 --> 00:14:05,800 Ejaaz: So it's still technically cheaper, but I would rather use OpenAI's tool, 237 00:14:05,920 --> 00:14:06,760 Ejaaz: which is just way better. 238 00:14:07,220 --> 00:14:10,460 Ejaaz: Now, Anthropic itself are facing so many outages for Claude. 239 00:14:10,540 --> 00:14:12,980 Ejaaz: You mentioned earlier that, you know, Claude is out for you this morning. 240 00:14:13,080 --> 00:14:17,520 Ejaaz: It's also out for me this morning. And I think this is because OpenAI basically 241 00:14:17,520 --> 00:14:22,060 Ejaaz: has all the money in compute to subsidize this cost for a tool like this, 242 00:14:22,400 --> 00:14:23,660 Ejaaz: whereas Anthropic is struggling. 243 00:14:23,880 --> 00:14:25,200 Ejaaz: They're adding a million users 244 00:14:25,200 --> 00:14:29,800 Ejaaz: to their general database or to the general user base every single day. 245 00:14:30,000 --> 00:14:32,680 Ejaaz: And they're not able to subsidize people's compute anymore. So they're just 246 00:14:32,680 --> 00:14:36,500 Ejaaz: shutting people's access off to it. And it's kind of frustrating to see, to be honest. 247 00:14:36,840 --> 00:14:39,320 Josh: Yeah, and the way you have to think about this is each one of these companies 248 00:14:39,320 --> 00:14:40,760 Josh: has a limited amount of compute. 249 00:14:40,880 --> 00:14:43,880 Josh: And that compute takes care of everything. It has to serve the customers. 250 00:14:44,000 --> 00:14:46,480 Josh: It has to serve the developers, the researchers. 251 00:14:46,800 --> 00:14:50,280 Josh: And a lot of times when you're developing these new technologies or you're training 252 00:14:50,280 --> 00:14:53,480 Josh: new models, the training run consumes a tremendous amount of GPU power. 253 00:14:53,660 --> 00:14:57,020 Josh: The researchers want a lot of GPUs to run tests and run trials on, 254 00:14:57,120 --> 00:15:02,320 Josh: but you still have to find extra resources to serve all of the users who are querying this model. 255 00:15:02,890 --> 00:15:06,230 Josh: Every single day. And I was recently listening to Dario on, I think, 256 00:15:06,330 --> 00:15:09,830 Josh: the Dwarkesh podcast, where he was talking about how they think about how many 257 00:15:09,830 --> 00:15:11,170 Josh: GPUs to order into the future. 258 00:15:11,310 --> 00:15:14,310 Josh: Because they go to Jensen, they go to NVIDIA, they say, hi, Jensen, 259 00:15:14,370 --> 00:15:17,750 Josh: we would like to place this amount of GPU purchases for this year. 260 00:15:18,070 --> 00:15:21,830 Josh: And what he was mentioning that I found interesting is that they can't over 261 00:15:21,830 --> 00:15:26,510 Josh: order because if they're off by just a small percentage, the incremental cost 262 00:15:26,510 --> 00:15:30,970 Josh: of those GPUs will far outweigh the growth, the revenue that they get from growth. 263 00:15:31,470 --> 00:15:35,670 Josh: And what it sounded like he did is he was just taking the current growth and 264 00:15:35,670 --> 00:15:40,450 Josh: mapping it to the future to hedge themselves against this collapse from debt 265 00:15:40,450 --> 00:15:41,750 Josh: that they would have from over-ordering. 266 00:15:41,990 --> 00:15:45,750 Josh: But the reality is, is that since that episode was recorded and now they have 267 00:15:45,750 --> 00:15:51,450 Josh: gone fully vertical, like that curve has steepened significantly more and they're 268 00:15:51,450 --> 00:15:53,870 Josh: going to have to figure out a way to solve this because this is not an easy 269 00:15:53,870 --> 00:15:55,750 Josh: thing. These orders get placed years in advance. 270 00:15:55,970 --> 00:15:58,630 Josh: They're projecting for a certain curve and they're getting a different one. 271 00:15:58,750 --> 00:16:00,570 Josh: So is this a short-term thing? 272 00:16:00,690 --> 00:16:04,190 Josh: Is this going to be durable? I don't know, but it is noteworthy that Anthropoc 273 00:16:04,190 --> 00:16:05,190 Josh: is having some growing pains. 274 00:16:05,910 --> 00:16:11,190 Ejaaz: Okay, so we have two problems here. We have an amazing AI model that can code, 275 00:16:11,290 --> 00:16:14,650 Ejaaz: and that is inevitably going to be the future. Lovable's demand proves that. 276 00:16:14,850 --> 00:16:17,610 Ejaaz: But then we're also running into the issue where we have all these security 277 00:16:17,610 --> 00:16:20,130 Ejaaz: flaws and we could lose tens to hundreds of millions of dollars. 278 00:16:20,270 --> 00:16:22,770 Ejaaz: Amazon demonstrated that, right? So we're at this conundrum. 279 00:16:22,890 --> 00:16:25,910 Ejaaz: How on earth do we solve this? Do you know what the solution might be, Josh? 280 00:16:27,480 --> 00:16:29,620 Ejaaz: It might be AI. It might be AI itself. 281 00:16:30,060 --> 00:16:30,820 Josh: That checks out. 282 00:16:30,960 --> 00:16:35,000 Ejaaz: You have to use AI to solve both of these individual AI problems, I'm afraid. 283 00:16:35,420 --> 00:16:38,680 Ejaaz: So, Andre Carpathy, the godfather of AI, as I like to call him, 284 00:16:39,100 --> 00:16:43,340 Ejaaz: released this really cool experiment that is just completely blown up, 285 00:16:43,500 --> 00:16:47,320 Ejaaz: which hints at what the future solution might be to solving this problem that 286 00:16:47,320 --> 00:16:50,620 Ejaaz: we just explained on this episode, which is called auto-research. 287 00:16:50,740 --> 00:16:57,280 Ejaaz: Now, the best way to think about this is he created an AI model that acts as an AI researcher. 288 00:16:57,480 --> 00:17:01,380 Ejaaz: But the coolest part about it is it autonomously does research. 289 00:17:01,600 --> 00:17:05,080 Ejaaz: Now, typically, when you use an LLM, you need to prompt it. And then it says, 290 00:17:05,220 --> 00:17:06,720 Ejaaz: I don't really know what this means. 291 00:17:07,000 --> 00:17:10,140 Ejaaz: Can you tell me what it means? And you have to keep working back and forth. 292 00:17:10,340 --> 00:17:15,180 Ejaaz: This thing runs completely on its own overnight, and in some cases, for days at a time. 293 00:17:15,300 --> 00:17:18,400 Ejaaz: He's done this overnight when he slept for like 9 to 12 hours, 294 00:17:18,400 --> 00:17:20,960 Ejaaz: and then he's done it again for two to three days. 295 00:17:21,140 --> 00:17:25,400 Ejaaz: And how the AI model works is he sets an objective. He says, 296 00:17:25,640 --> 00:17:28,540 Ejaaz: I want you to try and improve this part of your model. 297 00:17:28,680 --> 00:17:32,720 Ejaaz: So he's talking to the AI model and saying, you need to improve yourself in this particular way. 298 00:17:33,020 --> 00:17:37,580 Ejaaz: Go away and figure it out. The model then runs an experiment every five minutes. 299 00:17:37,720 --> 00:17:41,020 Ejaaz: If the experiment comes out with an improvement, it caches it in. 300 00:17:41,120 --> 00:17:42,680 Ejaaz: It improves its own model weight. 301 00:17:42,860 --> 00:17:45,400 Ejaaz: It improves its own internal insights, right? 302 00:17:45,840 --> 00:17:47,440 Ejaaz: If it doesn't, it discards it 303 00:17:47,440 --> 00:17:50,880 Ejaaz: and it tries again. And it runs these experiments on and on and on and on. 304 00:17:51,020 --> 00:17:54,580 Ejaaz: In this experiment, I think he ran around 150 experiments overnight. 305 00:17:54,580 --> 00:17:58,040 Ejaaz: And it improved itself pretty marginally, as is demonstrated by this graph. 306 00:17:58,160 --> 00:18:00,620 Ejaaz: The loss aversion just went crashingly down. 307 00:18:00,760 --> 00:18:04,040 Ejaaz: So it suggests that maybe Anthropic and Open Air, instead of having to rely 308 00:18:04,040 --> 00:18:07,700 Ejaaz: on releasing new tools every day, constrained by humans, they could just let 309 00:18:07,700 --> 00:18:10,540 Ejaaz: the AI improve itself and everyone's hunky-dory, maybe. 310 00:18:10,780 --> 00:18:15,980 Josh: This seems like an early form of this takeoff situation in which AIs become 311 00:18:15,980 --> 00:18:20,520 Josh: self-improving, because I haven't really seen this architecture before in such a 312 00:18:20,980 --> 00:18:23,720 Josh: open and easily accessible way i mean this is 313 00:18:23,720 --> 00:18:27,360 Josh: all open source this i mean andre says here it's 630 lines 314 00:18:27,360 --> 00:18:30,340 Josh: of code and traditionally it's on the human to iterate on 315 00:18:30,340 --> 00:18:33,260 Josh: the prompt to improve the prompt to improve the 316 00:18:33,260 --> 00:18:36,200 Josh: training code and to work together with the ai but this actually 317 00:18:36,200 --> 00:18:38,920 Josh: does it completely and entirely on its own it goes 318 00:18:38,920 --> 00:18:42,180 Josh: off it runs its own experiments it ingests those experiments updates 319 00:18:42,180 --> 00:18:45,440 Josh: its values that it has updates its view on the world and then creates a new 320 00:18:45,440 --> 00:18:48,960 Josh: set of experiments and runs this iterative process and what we saw in that chart 321 00:18:48,960 --> 00:18:52,580 Josh: is that there was these incremental improvements that andre just didn't see 322 00:18:52,580 --> 00:18:59,880 Josh: and again it's hard to overstate how much of a powerhouse andre is in terms 323 00:18:59,880 --> 00:19:02,120 Josh: of ai engineering and understanding do 324 00:19:02,120 --> 00:19:04,060 Ejaaz: He has two decades of experience in. 325 00:19:04,060 --> 00:19:06,880 Josh: This and like ai what like wasn't even 326 00:19:06,880 --> 00:19:09,680 Josh: around two decades ago like he was the guy who was building the damn thing 327 00:19:09,680 --> 00:19:12,520 Josh: yeah um so for him to come along and for 328 00:19:12,520 --> 00:19:15,500 Josh: the first time ever experience this novel breakthrough in which an ai 329 00:19:15,500 --> 00:19:18,700 Josh: isn't able to improve his code in ways that he didn't initially see 330 00:19:18,700 --> 00:19:21,500 Josh: it's like when ai played go for the first 331 00:19:21,500 --> 00:19:24,880 Josh: time or when ai is playing chess and it makes these erratic moves you would 332 00:19:24,880 --> 00:19:27,900 Josh: not have known that it was optimal but the model is 333 00:19:27,900 --> 00:19:30,560 Josh: is so precise has so much context and so 334 00:19:30,560 --> 00:19:33,760 Josh: much intelligence that it's actually able to make these optimizations that 335 00:19:33,760 --> 00:19:36,600 Josh: humans otherwise wouldn't have seen and this is one of the earlier instances where 336 00:19:36,600 --> 00:19:39,280 Josh: even andre got humbled and the code that he 337 00:19:39,280 --> 00:19:42,880 Josh: was running was significantly improved overnight by this 338 00:19:42,880 --> 00:19:46,040 Josh: self-recursive thing and the most interesting thing about this is it's 339 00:19:46,040 --> 00:19:49,460 Josh: open source it's available to run on your laptop on 340 00:19:49,460 --> 00:19:52,420 Josh: your pc on a computer right now and i 341 00:19:52,420 --> 00:19:55,160 Josh: have to imagine that ai labs are going to take this and run with 342 00:19:55,160 --> 00:20:00,040 Josh: it at scale where if you're training a model why would you not and and this 343 00:20:00,040 --> 00:20:03,680 Josh: model using this auto research can do better than andre i'm sure it could do 344 00:20:03,680 --> 00:20:08,080 Josh: better than the average mid-tier like prompt engineer at anthropic or chat or 345 00:20:08,080 --> 00:20:13,140 Josh: open ai and actually improve the entirety of the company so this feels like something that is 346 00:20:13,960 --> 00:20:16,880 Josh: starting from the bottom where in terms of it's like openly accessible it's 347 00:20:16,880 --> 00:20:20,320 Josh: open source and then we'll move its way up through everything and become this 348 00:20:20,320 --> 00:20:23,440 Josh: like auto-aggressive self-improving loop it's incredible i want to try it on 349 00:20:23,440 --> 00:20:26,180 Josh: something we need a problem that's hard enough to try this on where you can 350 00:20:26,180 --> 00:20:29,640 Josh: truly let it run for a series of days and then come back with the answer that's 351 00:20:29,640 --> 00:20:32,160 Josh: better than anything you could have ever imagined well maybe research 352 00:20:32,160 --> 00:20:37,300 Ejaaz: On how to make limitless the best and number one podcast in the world maybe i i'd be down to run this. 353 00:20:37,300 --> 00:20:40,860 Josh: That's probably we should try that we should try that we can make some assumptions 354 00:20:40,860 --> 00:20:43,720 Josh: on what it would it would say right It would say, well, we probably need to 355 00:20:43,720 --> 00:20:48,320 Josh: convince all of our listeners to go subscribe to the podcast on your favorite podcast player. 356 00:20:48,460 --> 00:20:52,320 Josh: Give it five stars, share it with all of their friends, and then make sure that 357 00:20:52,320 --> 00:20:55,860 Josh: they comment things they either like or don't like about it to keep the engagement 358 00:20:55,860 --> 00:20:58,040 Josh: rate high. Those things will help out the show. 359 00:20:58,260 --> 00:21:00,440 Ejaaz: That actually sounds pretty great. And if you're listening to this, 360 00:21:00,580 --> 00:21:03,080 Ejaaz: remember, you are no more than organic LLM. 361 00:21:03,220 --> 00:21:06,120 Ejaaz: So if you want to process that prompt that Josh just specified, 362 00:21:06,320 --> 00:21:09,040 Ejaaz: please go do it. It helps us out a massive amount. 363 00:21:09,500 --> 00:21:14,320 Ejaaz: But just to round up and wrap up this episode, it's this weird conundrum. 364 00:21:14,640 --> 00:21:17,560 Ejaaz: AI is either zero or 100 at this point. 365 00:21:17,760 --> 00:21:21,900 Ejaaz: It's either deteriorating your product and causing massive outages, 366 00:21:21,940 --> 00:21:25,420 Ejaaz: or it's doing something like this and helping the world's leading expert of 367 00:21:25,420 --> 00:21:27,400 Ejaaz: AI research actually do a better 368 00:21:27,400 --> 00:21:30,600 Ejaaz: job, which is just, there's no middle ground. It's either zero or 100. 369 00:21:30,820 --> 00:21:33,200 Ejaaz: And it's just super exciting to see where all of this is going. 370 00:21:33,500 --> 00:21:36,380 Ejaaz: I tweeted this out the other day. 371 00:21:36,560 --> 00:21:40,560 Ejaaz: My mind, I don't know about you, Josh, is just in a fog at this moment because 372 00:21:40,560 --> 00:21:42,760 Ejaaz: I can't keep up with everything that's going on. 373 00:21:42,860 --> 00:21:45,340 Ejaaz: Something's switched over the last 30 days where 374 00:21:45,710 --> 00:21:49,090 Ejaaz: The exponential curve has just like racketed up. I don't know whether this is 375 00:21:49,090 --> 00:21:52,350 Ejaaz: because I'm in my own echo chamber or not, but it just feels like we are getting 376 00:21:52,350 --> 00:21:57,290 Ejaaz: new model releases every single week at this point. There's so many new Frontier AI labs. 377 00:21:57,570 --> 00:22:02,190 Ejaaz: Jan LeCun, who is the former head of AI research at Meta, has now started his 378 00:22:02,190 --> 00:22:05,550 Ejaaz: own thing and he's focusing on world models. And I'm like, what is going on here? 379 00:22:05,650 --> 00:22:08,610 Ejaaz: It's just, it's so much, but you're going to hear all about it on Limitless. 380 00:22:08,810 --> 00:22:14,270 Ejaaz: Josh and I are working 24-7 doing this. We're releasing four new episodes every week, up from three. 381 00:22:14,270 --> 00:22:17,270 Ejaaz: The viewership has been crazy the comments on our last episode 382 00:22:17,270 --> 00:22:22,850 Ejaaz: where we talk about uploading a fruit flies brain into a laptop the banger of 383 00:22:22,850 --> 00:22:26,170 Ejaaz: an episode go check that out comments are like four times what we normally get 384 00:22:26,170 --> 00:22:30,250 Ejaaz: we're responding to every single one we are the most dialed show on ai that 385 00:22:30,250 --> 00:22:36,110 Ejaaz: you can potentially watch uh please like subscribe leave us comments give us feedback and josh yeah. 386 00:22:36,110 --> 00:22:38,850 Josh: And important to know like the four episodes is just a 387 00:22:38,850 --> 00:22:41,610 Josh: testament to how fast we're going um and just the need to cover 388 00:22:41,610 --> 00:22:44,490 Josh: more things so again you will not miss 389 00:22:44,490 --> 00:22:47,510 Josh: anything if you watch the show and we will continue to scale to match 390 00:22:47,510 --> 00:22:51,930 Josh: the the necessary demand to like sufficiently cover all of this and that's what's 391 00:22:51,930 --> 00:22:54,330 Josh: happening i mean it's just downstream of the craziness we are very much in this 392 00:22:54,330 --> 00:22:58,710 Josh: singularity event in the vertical part of the curve so enjoy it like i'm just 393 00:22:58,710 --> 00:23:02,130 Josh: trying to take it all in i can take it for what it is get excited about it share 394 00:23:02,130 --> 00:23:05,330 Josh: with everyone so yeah thank you guys all for watching as always and we'll see 395 00:23:05,330 --> 00:23:06,830 Josh: you in the roundup tomorrow