1 00:00:01,820 --> 00:00:06,887 [Ben Bajarin]: Hello everyone, welcome to another episode of The Circuit. I am Ben Beharon. 2 00:00:08,206 --> 00:00:09,749 [Jay Goldberg]: Hello world, I'm Jay Goldberg. 3 00:00:11,256 --> 00:00:14,942 [Ben Bajarin]: We, I had a joke a couple of days ago, Jay and I were talking about where 4 00:00:15,162 --> 00:00:20,070 [Ben Bajarin]: I would introduce myself as Jay Goldberg, just to throw everybody off and see 5 00:00:20,090 --> 00:00:23,616 [Ben Bajarin]: what the response would be. But then we concluded that might not be a good 6 00:00:23,636 --> 00:00:23,857 [Ben Bajarin]: idea. 7 00:00:25,107 --> 00:00:26,838 [Jay Goldberg]: I think you've just accomplished confusing everybody. 8 00:00:27,756 --> 00:00:33,779 [Ben Bajarin]: Yes. All right. Ben Beharon is the one talking now. There you go. All right. 9 00:00:33,819 --> 00:00:38,382 [Ben Bajarin]: So it's been an interesting week on the back of a couple of semiconductor earnings 10 00:00:38,402 --> 00:00:46,648 [Ben Bajarin]: calls, ASML and TSMC. This created a lot of different talking points, questions. 11 00:00:48,249 --> 00:00:51,331 [Ben Bajarin]: Something that we're going to dive into today is something we've talked a 12 00:00:51,391 --> 00:00:57,165 [Ben Bajarin]: little bit about before. I wish we had the wherewithal to go even deeper on 13 00:00:57,185 --> 00:01:02,386 [Ben Bajarin]: this thesis, but it's something that Jay and I are working on as well. Relative 14 00:01:02,446 --> 00:01:09,909 [Ben Bajarin]: to the impact that AI has on a number of different computing segments, not 15 00:01:09,929 --> 00:01:14,710 [Ben Bajarin]: just data center, but edge, we've done some specific, like I said, episodes on 16 00:01:14,750 --> 00:01:21,312 [Ben Bajarin]: this, but Jay recently wrote a thesis. on this at Digits to Dollars, which 17 00:01:21,352 --> 00:01:25,378 [Ben Bajarin]: we can link to in the show notes. So I'm just going to let Jay outline his thesis 18 00:01:25,398 --> 00:01:28,463 [Ben Bajarin]: because I followed some of the people that were questioning him on Twitter 19 00:01:29,104 --> 00:01:34,111 [Ben Bajarin]: and seeing the discussion. So lob the thesis out on us and then let's unpack 20 00:01:34,151 --> 00:01:34,211 [Ben Bajarin]: it. 21 00:01:34,882 --> 00:01:38,415 [Jay Goldberg]: So I approach this topic of AI semis. Obviously, everyone's talking about 22 00:01:38,455 --> 00:01:44,451 [Jay Goldberg]: it a lot. We've done a few episodes on it. And I realized that we sort of addressed 23 00:01:44,491 --> 00:01:47,814 [Jay Goldberg]: different parts of the elephant. And so I wanted to take a step back and sort 24 00:01:47,834 --> 00:01:53,059 [Jay Goldberg]: of look at the whole thing. And I think from a high level strategic perspective, 25 00:01:53,079 --> 00:01:59,184 [Jay Goldberg]: there are really three questions around AI semis today. And the, the first one 26 00:01:59,264 --> 00:02:05,689 [Jay Goldberg]: is will AI be additive to the addressable market, the TAM for semiconductors, 27 00:02:06,130 --> 00:02:10,573 [Jay Goldberg]: or will it just cannibalize CPUs and other stuff and market stays the same? 28 00:02:11,838 --> 00:02:15,741 [Jay Goldberg]: Second question is how will the market for edge inference, or excuse me, 29 00:02:15,761 --> 00:02:20,065 [Jay Goldberg]: for inference in general, shape up? What's that competitive dynamic gonna look 30 00:02:20,085 --> 00:02:26,490 [Jay Goldberg]: like? And the last question is can anybody displace NVIDIA, who is clearly 31 00:02:26,710 --> 00:02:30,734 [Jay Goldberg]: the market leader and the dominant player in this space right now? And I think 32 00:02:31,515 --> 00:02:34,777 [Jay Goldberg]: those are the three sort of fundamental questions that are gonna determine 33 00:02:35,658 --> 00:02:40,334 [Jay Goldberg]: some important trends for the next decade of semiconductor. market and who 34 00:02:40,354 --> 00:02:41,524 [Jay Goldberg]: the winners and losers are going to be. 35 00:02:44,360 --> 00:02:49,344 [Ben Bajarin]: Okay, I'm gonna start actually with your third point as a starting point. 36 00:02:49,424 --> 00:02:52,666 [Ben Bajarin]: So this is something I've been thinking about and I haven't asked you this 37 00:02:52,707 --> 00:02:56,530 [Ben Bajarin]: question but I've been talking to other people. We talked a couple episodes 38 00:02:56,570 --> 00:03:02,034 [Ben Bajarin]: ago when we talked about ARM and the data center about everybody sort of, 39 00:03:02,615 --> 00:03:09,180 [Ben Bajarin]: everybody who's in the know, unanimously agreeing that much of the workload 40 00:03:09,280 --> 00:03:16,362 [Ben Bajarin]: in AI is gonna move from training to inference. And in some cases I've seen folks 41 00:03:16,482 --> 00:03:23,028 [Ben Bajarin]: and investors on the sell side, estimate this to be, you know, 77, 78% of the 42 00:03:23,048 --> 00:03:28,893 [Ben Bajarin]: market today in terms of dollars is training. Um, but that could move to a 40, 43 00:03:28,954 --> 00:03:35,499 [Ben Bajarin]: 60% split 50 over the next five years. So people are assuming that training 44 00:03:35,519 --> 00:03:40,708 [Ben Bajarin]: will still be important. Let's again, let's say it's 50%, 60% great. but a 45 00:03:40,768 --> 00:03:45,391 [Ben Bajarin]: huge portion of that moving dollars moving to inference. So the question is, if 46 00:03:45,471 --> 00:03:49,834 [Ben Bajarin]: we believe that, and then perhaps over the 10 year timeframe, even more moves 47 00:03:49,854 --> 00:03:56,379 [Ben Bajarin]: to inference, maybe that's a 20% training, 70% inference, does it even matter 48 00:03:56,780 --> 00:04:01,363 [Ben Bajarin]: if people compete with Nvidia in training? Because there's so much other 49 00:04:01,423 --> 00:04:06,046 [Ben Bajarin]: opportunity for inference, why would we not be focusing more on that versus 50 00:04:06,467 --> 00:04:07,547 [Ben Bajarin]: trying to compete with Nvidia? 51 00:04:08,866 --> 00:04:12,329 [Jay Goldberg]: So I think there's a few parts to that. A lot of that is around inference, 52 00:04:12,349 --> 00:04:16,732 [Jay Goldberg]: which is kind of my second question. But in terms of Nvidia's position in the market, 53 00:04:16,752 --> 00:04:20,956 [Jay Goldberg]: I think we should probably break down a little bit what dominance means. Today, 54 00:04:20,976 --> 00:04:26,140 [Jay Goldberg]: they are clearly dominant in training, close to 100% of the market. The 55 00:04:26,261 --> 00:04:26,761 [Jay Goldberg]: only real 56 00:04:26,618 --> 00:04:26,765 [Ben Bajarin]: Yep. 57 00:04:26,821 --> 00:04:31,445 [Jay Goldberg]: competition they have there are a few internal things at like Google, who 58 00:04:31,505 --> 00:04:37,754 [Jay Goldberg]: also is using Nvidia for training as well. So that's part of it. then the question 59 00:04:37,814 --> 00:04:43,578 [Jay Goldberg]: is, can they extend their position and training into inference? 60 00:04:44,037 --> 00:04:44,325 [Ben Bajarin]: Mm-hmm. 61 00:04:44,959 --> 00:04:50,483 [Jay Goldberg]: And I think we're at a place now where most people assume that AI equals 62 00:04:50,503 --> 00:04:51,084 [Jay Goldberg]: GPU 63 00:04:51,775 --> 00:04:51,899 [Ben Bajarin]: Mm-hmm. 64 00:04:51,884 --> 00:04:53,325 [Jay Goldberg]: and GPU equals Nvidia. 65 00:04:54,481 --> 00:04:54,811 [Ben Bajarin]: Mm-hmm. 66 00:04:57,774 --> 00:05:04,839 [Jay Goldberg]: We can debate who has a better product and is AMD's MI 300 competitive versus 67 00:05:04,750 --> 00:05:04,954 [Ben Bajarin]: Right. 68 00:05:05,459 --> 00:05:12,064 [Jay Goldberg]: whatever. But I think fundamentally, it comes down to the fact that the big customers 69 00:05:12,104 --> 00:05:16,847 [Jay Goldberg]: in particular are not going to ever settle for a world in which they are 70 00:05:17,167 --> 00:05:22,070 [Jay Goldberg]: completely dependent on Nvidia, especially when Nvidia controls both the silicon 71 00:05:22,390 --> 00:05:26,653 [Jay Goldberg]: and the software layer around CUDA. That's just an unacceptable, untenable 72 00:05:27,510 --> 00:05:32,594 [Jay Goldberg]: position for the customers. And so no matter what, this current status 73 00:05:32,614 --> 00:05:38,999 [Jay Goldberg]: quo has to change. Now, I'm not bearish on Nvidia by any means. They're 74 00:05:39,019 --> 00:05:46,245 [Jay Goldberg]: gonna do fine, but absolutely, they are not going to be able to extend that 75 00:05:46,265 --> 00:05:50,229 [Jay Goldberg]: 100% share into inference. There's no way that's gonna happen for a lot of reasons. 76 00:05:51,810 --> 00:05:57,499 [Jay Goldberg]: Can they get displaced out of training? I think... I think everybody's exploring 77 00:05:57,519 --> 00:05:58,100 [Jay Goldberg]: that right now. And 78 00:05:58,536 --> 00:05:58,879 [Ben Bajarin]: All right. 79 00:05:58,781 --> 00:06:00,804 [Jay Goldberg]: I think it's gonna be tough. I think it's actually gonna be pretty tough. 80 00:06:01,345 --> 00:06:05,031 [Jay Goldberg]: But if you start to look at some of these alternative software frameworks 81 00:06:05,051 --> 00:06:12,086 [Jay Goldberg]: that are coming out, PyTorch and Triton are on everybody's lips. I think 82 00:06:12,106 --> 00:06:14,588 [Jay Goldberg]: there are a lot of people in marketing who just assume, okay, PyTorch is 83 00:06:14,608 --> 00:06:18,190 [Jay Goldberg]: coming and it's going to displace CUDA and suddenly we're going to have a little 84 00:06:18,210 --> 00:06:22,713 [Jay Goldberg]: bit of evening out in market share and you can run PyTorch on an AMD or 85 00:06:22,773 --> 00:06:29,598 [Jay Goldberg]: whatever. And that's going to disrupt Nvidia's hold on training. I actually, 86 00:06:29,718 --> 00:06:31,760 [Jay Goldberg]: I'm not convinced. I'm not convinced. I think 87 00:06:31,793 --> 00:06:31,855 [Ben Bajarin]: I 88 00:06:31,840 --> 00:06:32,000 [Jay Goldberg]: there 89 00:06:31,876 --> 00:06:32,063 [Ben Bajarin]: agree. 90 00:06:32,060 --> 00:06:36,303 [Jay Goldberg]: is, there's a world in which you just run PyTorch on CUDA. Right. Uh, 91 00:06:35,600 --> 00:06:35,905 [Ben Bajarin]: Mm-hmm. 92 00:06:36,823 --> 00:06:41,350 [Jay Goldberg]: they don't necessarily, in some ways they compliment each other. Uh. I think. 93 00:06:43,598 --> 00:06:47,479 [Jay Goldberg]: You have the software on training is still very complex. I know a lot of companies 94 00:06:47,412 --> 00:06:47,533 [Ben Bajarin]: Mm-hmm. 95 00:06:47,519 --> 00:06:52,520 [Jay Goldberg]: have tried to get into it. It's just, it's going to, it's going to be, it's 96 00:06:52,540 --> 00:06:56,161 [Jay Goldberg]: going to be a battle. Um, and I think it's just for a lot of companies, it's 97 00:06:56,181 --> 00:07:00,122 [Jay Goldberg]: just not going to be worth it as, especially because like you said, inference 98 00:07:00,162 --> 00:07:04,683 [Jay Goldberg]: is going to become. The majority, I would say, I would argue the vast majority 99 00:07:04,963 --> 00:07:06,104 [Jay Goldberg]: of spend over 100 00:07:06,096 --> 00:07:06,867 [Ben Bajarin]: Yep. 101 00:07:06,144 --> 00:07:06,944 [Jay Goldberg]: time. And 102 00:07:06,948 --> 00:07:07,110 [Ben Bajarin]: Yeah. 103 00:07:06,964 --> 00:07:12,265 [Jay Goldberg]: so it there's how much effort do you want to spend trying to disrupt training 104 00:07:12,325 --> 00:07:15,147 [Jay Goldberg]: when that's, you know, 10% of your spend five years from now, 105 00:07:15,740 --> 00:07:16,204 [Ben Bajarin]: Exactly. 106 00:07:16,447 --> 00:07:19,569 [Jay Goldberg]: right? You're gonna be much more focused on the inference side and training 107 00:07:19,589 --> 00:07:21,230 [Jay Goldberg]: stuff is pretty complicated. 108 00:07:21,928 --> 00:07:22,173 [Ben Bajarin]: agreed. 109 00:07:22,270 --> 00:07:26,393 [Jay Goldberg]: Unquestionably big companies, Google comes to mind in particular, but others 110 00:07:26,433 --> 00:07:29,995 [Jay Goldberg]: as well, they're gonna find alternatives. They're not gonna use training. 111 00:07:30,715 --> 00:07:34,597 [Jay Goldberg]: They're not gonna do all their training on Nvidia long-term probably. But 112 00:07:34,637 --> 00:07:39,380 [Jay Goldberg]: I think for the enterprise, like why, you know, if you're an enterprise, 113 00:07:39,400 --> 00:07:44,867 [Jay Goldberg]: you're a bank and you wanna run training on your data, Like that's hard 114 00:07:44,907 --> 00:07:48,149 [Jay Goldberg]: enough. You have to find AI people, developers to just do that. And then 115 00:07:48,189 --> 00:07:52,592 [Jay Goldberg]: to go the extra mile and like fight the trend and create new software frameworks 116 00:07:52,612 --> 00:07:57,095 [Jay Goldberg]: and integrations. Some people will do that, but I think a lot of people just 117 00:07:57,115 --> 00:07:59,977 [Jay Goldberg]: find going with the default Nvidia for training is the easiest solution. 118 00:08:00,844 --> 00:08:04,685 [Ben Bajarin]: Yeah. So that was kind of like roundabout where I was getting to, right? Which 119 00:08:04,765 --> 00:08:11,667 [Ben Bajarin]: is, I see the allure in competing with them today and trying to compete with 120 00:08:11,687 --> 00:08:16,209 [Ben Bajarin]: them. There's at least half a dozen companies that we know of. There's probably 121 00:08:16,269 --> 00:08:24,191 [Ben Bajarin]: more attempting this. It's not a trivial problem. PyTorch, you're right, is 122 00:08:24,271 --> 00:08:29,061 [Ben Bajarin]: gaining... significant sort of share in terms of academia and research and 123 00:08:29,081 --> 00:08:33,183 [Ben Bajarin]: people who are using it. But again, that's not the only thing here that's 124 00:08:33,223 --> 00:08:41,046 [Ben Bajarin]: being used. So it really does feel like there's a still strong reason to 125 00:08:41,086 --> 00:08:45,008 [Ben Bajarin]: be bullish on Nvidia for GPU. But what you said is exactly right. One of 126 00:08:45,028 --> 00:08:49,990 [Ben Bajarin]: the big questions is, what do they do with Grace Hopper and how do they continue 127 00:08:50,010 --> 00:08:56,692 [Ben Bajarin]: to move their companion CPU part? Um, because if you, if you unpack Jensen's 128 00:08:56,732 --> 00:09:00,373 [Ben Bajarin]: vision, and this is where I, this is the thing I think is going to be the 129 00:09:00,473 --> 00:09:05,755 [Ben Bajarin]: hardest for most companies to really kind of grasp in their, their strategy 130 00:09:05,775 --> 00:09:11,776 [Ben Bajarin]: to compete with Nvidia is Jensen labels this under accelerated computing. And 131 00:09:11,796 --> 00:09:15,717 [Ben Bajarin]: in my mind, he does not limit accelerated computing to even the GPU or the 132 00:09:15,757 --> 00:09:20,419 [Ben Bajarin]: CPU there's networking parts. And I think there's other elements of AI Asics 133 00:09:20,579 --> 00:09:25,540 [Ben Bajarin]: Nvidia can make. So if they are still. a dominant platform player and a dominant 134 00:09:25,960 --> 00:09:30,601 [Ben Bajarin]: seller than of GPUs who compete, who bring companion parts to diversify these 135 00:09:30,642 --> 00:09:34,963 [Ben Bajarin]: workloads. It's actually again, a really strong story because his focus is 136 00:09:35,023 --> 00:09:39,864 [Ben Bajarin]: just, let's just accelerate computing and any bit of Silicon that can do that. Let's 137 00:09:39,904 --> 00:09:43,385 [Ben Bajarin]: just go make it Nvidia. And I think it's hard because who else is going to 138 00:09:43,405 --> 00:09:49,587 [Ben Bajarin]: have that stack? Very few people, maybe AMD, right? Maybe Intel could have all 139 00:09:49,627 --> 00:09:53,849 [Ben Bajarin]: those pieces, but Jensen's much more focused. And that's the part I think is 140 00:09:53,869 --> 00:09:58,553 [Ben Bajarin]: just gonna be really, really tricky at the end to bite off all those pieces. 141 00:09:58,593 --> 00:10:02,035 [Ben Bajarin]: Whether he's successful in those other areas, I don't know, that's debatable. 142 00:10:02,196 --> 00:10:06,459 [Ben Bajarin]: But I think his vision's very clear and his knowledge of the problem is also 143 00:10:06,479 --> 00:10:06,919 [Ben Bajarin]: very clear. 144 00:10:08,114 --> 00:10:13,315 [Jay Goldberg]: Yeah, so let me break that down. There's a couple of things there. One, today 145 00:10:13,335 --> 00:10:20,137 [Jay Goldberg]: when people ask me, like, why is Nvidia so dominant in AI? The sort of default 146 00:10:20,557 --> 00:10:24,858 [Jay Goldberg]: knee-jerk answer is to say CUDA. CUDA is a software layer that in between the 147 00:10:24,898 --> 00:10:28,859 [Jay Goldberg]: operating system and the chip, and you can get a whole bunch of optimizations 148 00:10:28,959 --> 00:10:34,121 [Jay Goldberg]: to run your systems much better because you have CUDA. And I think the advantage 149 00:10:34,161 --> 00:10:40,479 [Jay Goldberg]: that CUDA has conveyed to Nvidia I think we could reasonably argue that is slowly 150 00:10:40,519 --> 00:10:45,080 [Jay Goldberg]: being diluted by all the things I talked about before. CUDA was really important 151 00:10:45,100 --> 00:10:50,563 [Jay Goldberg]: in the early days of AI for enabling all of this. Its sustainability as 152 00:10:50,603 --> 00:10:54,844 [Jay Goldberg]: a durable competitive advantage is probably peaking. That's how I'll put it. 153 00:10:54,854 --> 00:10:55,518 [Ben Bajarin]: Mm-hmm. 154 00:10:54,944 --> 00:10:59,706 [Jay Goldberg]: It's very strong, but it's probably at its peak and it's going to wane. So then 155 00:11:00,867 --> 00:11:04,148 [Jay Goldberg]: the next question will be, oh, does that mean I can shorten video now? And 156 00:11:04,728 --> 00:11:10,882 [Jay Goldberg]: I think the answer is no. setting aside cyclical factors because they're going 157 00:11:10,902 --> 00:11:13,263 [Jay Goldberg]: to blow up in a few quarters because they always do but like in terms of 158 00:11:13,783 --> 00:11:19,244 [Jay Goldberg]: Secular trend I think Nvidia is still in a really good position and You're 159 00:11:19,264 --> 00:11:23,145 [Jay Goldberg]: right. It's all the reasons you stated they have this whole stack They 160 00:11:23,165 --> 00:11:27,086 [Jay Goldberg]: have they have all the pieces it will become very easy to just buy an AI 161 00:11:27,106 --> 00:11:30,988 [Jay Goldberg]: system from Nvidia if you can afford it You just get everything from them 162 00:11:31,488 --> 00:11:34,789 [Jay Goldberg]: and for a lot of companies that will be very appealing maybe not the hyperscalers 163 00:11:34,909 --> 00:11:38,834 [Jay Goldberg]: but maybe them too, but certainly for the enterprise like you just You buy everything, 164 00:11:38,874 --> 00:11:44,319 [Jay Goldberg]: you buy a rack or two of Nvidia solution. Plus on top of that you have all 165 00:11:44,339 --> 00:11:48,402 [Jay Goldberg]: their software offerings. Their models which are trained, their software frameworks 166 00:11:48,422 --> 00:11:53,506 [Jay Goldberg]: which are trained for specific industries and they have a dozen now. I think 167 00:11:53,526 --> 00:11:57,770 [Jay Goldberg]: that's a really compelling vision and I think that speaks to Jensen, like you 168 00:11:57,810 --> 00:12:01,873 [Jay Goldberg]: said, really understands this and knows where it's going. He's multiple steps 169 00:12:01,913 --> 00:12:07,091 [Jay Goldberg]: ahead. The one pushback I would give on that though is not everybody's going 170 00:12:07,111 --> 00:12:10,557 [Jay Goldberg]: to want that. Because again, it conveys a pretty high degree of lock-in, 171 00:12:11,299 --> 00:12:17,138 [Jay Goldberg]: right? If AI is as important as everybody seems to think it is. there's 172 00:12:17,158 --> 00:12:21,799 [Jay Goldberg]: a risk in being so dependent on somebody for an entire solution. And historically, 173 00:12:22,360 --> 00:12:28,622 [Jay Goldberg]: those kinds of dependencies don't last. That being said, it will appeal 174 00:12:28,642 --> 00:12:32,944 [Jay Goldberg]: to enough people and Nvidia is so far ahead on so many software fronts 175 00:12:33,944 --> 00:12:38,086 [Jay Goldberg]: that I think Nvidia will do just fine, even if CUDA goes away. I think 176 00:12:38,106 --> 00:12:38,506 [Jay Goldberg]: they're in 177 00:12:38,508 --> 00:12:38,629 [Ben Bajarin]: Oh, 178 00:12:38,546 --> 00:12:38,826 [Jay Goldberg]: a really 179 00:12:38,669 --> 00:12:39,133 [Ben Bajarin]: for sure. 180 00:12:38,846 --> 00:12:39,327 [Jay Goldberg]: good position. 181 00:12:40,200 --> 00:12:40,603 [Ben Bajarin]: Totally. 182 00:12:40,467 --> 00:12:43,768 [Jay Goldberg]: There's gonna be a lot of friction about people complaining about locking and 183 00:12:43,788 --> 00:12:46,569 [Jay Goldberg]: all that, but still there's a lot of strong appeal there. 184 00:12:47,272 --> 00:12:52,875 [Ben Bajarin]: I totally agree. I mean, on the merits of just the product quality themselves, 185 00:12:52,915 --> 00:12:57,257 [Ben Bajarin]: right, that they're building, it's hard to do what they're doing when it 186 00:12:57,277 --> 00:13:04,461 [Ben Bajarin]: comes to these specific workloads. So by there alone, right, I think my broader 187 00:13:04,502 --> 00:13:09,144 [Ben Bajarin]: sort of just view is, one, we're probably a couple years away, or sorry, not 188 00:13:09,224 --> 00:13:13,767 [Ben Bajarin]: a couple years away, maybe closer than that, but NVIDIA is much now more quickly 189 00:13:13,807 --> 00:13:19,606 [Ben Bajarin]: going to be... a $100 billion company in revenue sooner than we thought, 190 00:13:19,946 --> 00:13:24,248 [Ben Bajarin]: thanks to this trend. That increases their capex. That increases their ability to 191 00:13:24,268 --> 00:13:29,331 [Ben Bajarin]: become a priority share at TSMC. I think they're number four priority now or 192 00:13:29,371 --> 00:13:34,854 [Ben Bajarin]: so, but increasing in that. It just gives them so much leverage with, again, 193 00:13:35,234 --> 00:13:42,518 [Ben Bajarin]: well-execution, good vision, and a large software stack that it's hard to 194 00:13:42,578 --> 00:13:47,714 [Ben Bajarin]: be displaced where they are today. is part of my view. But that leads me to this 195 00:13:47,754 --> 00:13:50,916 [Ben Bajarin]: sort of broader question you and I have been circling around, which is, 196 00:13:52,818 --> 00:13:58,022 [Ben Bajarin]: there are other areas of growth to the data center that's not GPU. And I think 197 00:13:58,062 --> 00:14:02,666 [Ben Bajarin]: we both believe GPU spend of that is going to grow probably faster than 198 00:14:03,046 --> 00:14:09,011 [Ben Bajarin]: CPU, but both are relevant. But we've asked this question before about how 199 00:14:09,031 --> 00:14:15,132 [Ben Bajarin]: much additional lift to the data center, Tam, is AI going to bring? If it's 200 00:14:15,152 --> 00:14:20,553 [Ben Bajarin]: a big number, like I've seen numbers anywhere from, you know, 30 billion 201 00:14:20,613 --> 00:14:24,715 [Ben Bajarin]: to an extra a hundred billion over the next, you know, 10 years. So if it's 202 00:14:24,735 --> 00:14:30,317 [Ben Bajarin]: a big number, I kind of feel like if I was somebody trying to increase my 203 00:14:31,257 --> 00:14:34,358 [Ben Bajarin]: strategy in the data center, I would want to go after this green field of 204 00:14:34,398 --> 00:14:38,919 [Ben Bajarin]: growth versus go for these areas where people are more entrenched. 205 00:14:44,026 --> 00:14:47,126 [Jay Goldberg]: Yeah, I think, well, I think this gets into sort of the second question, 206 00:14:47,146 --> 00:14:51,428 [Jay Goldberg]: which is inference, right? I think going after training right now, especially 207 00:14:51,448 --> 00:14:54,989 [Jay Goldberg]: for a startup is, um, I don't 208 00:14:53,543 --> 00:14:53,774 [Ben Bajarin]: Yes. 209 00:14:55,009 --> 00:14:57,589 [Jay Goldberg]: want to say suicidal, let's call it challenging, 210 00:14:58,152 --> 00:14:58,727 [Ben Bajarin]: challenging. 211 00:14:58,910 --> 00:15:03,511 [Jay Goldberg]: very challenging, but the, the bigger market will be inference. 212 00:15:04,420 --> 00:15:04,689 [Ben Bajarin]: Mm-hmm. 213 00:15:04,631 --> 00:15:08,612 [Jay Goldberg]: It's going to be a very big market and it just can't be run on GPU entirely. 214 00:15:09,392 --> 00:15:09,653 [Jay Goldberg]: Right. The 215 00:15:09,604 --> 00:15:09,925 [Ben Bajarin]: Yes. 216 00:15:09,693 --> 00:15:11,413 [Jay Goldberg]: economics will not work out even if, 217 00:15:11,431 --> 00:15:11,611 [Ben Bajarin]: Right. 218 00:15:11,673 --> 00:15:16,376 [Jay Goldberg]: you know, especially with supply conditions the way they are today. But there 219 00:15:16,656 --> 00:15:20,039 [Jay Goldberg]: are other factors too, right? A lot of these AI, I mean, AI is just software. 220 00:15:20,719 --> 00:15:24,382 [Jay Goldberg]: And so you're going to run your normal corporate software workload, and you 221 00:15:24,402 --> 00:15:27,965 [Jay Goldberg]: can have some AI functionality in it. There are a lot of times when the software 222 00:15:27,985 --> 00:15:32,168 [Jay Goldberg]: architecture is going to dictate that means running the AI on CPU alongside 223 00:15:32,188 --> 00:15:39,073 [Jay Goldberg]: the other part of the workload. So AI is not all GPU. Some of it's going to 224 00:15:39,093 --> 00:15:42,611 [Jay Goldberg]: be on CPU. A lot of companies are going to build accelerators for this. The 225 00:15:42,631 --> 00:15:46,932 [Jay Goldberg]: hyperscalers I think are, you know, are pretty invested in, in accelerators. 226 00:15:47,413 --> 00:15:50,474 [Jay Goldberg]: That's an important category for a lot of them. And they're going to, they're 227 00:15:50,494 --> 00:15:55,096 [Jay Goldberg]: going to, that's, they're going to run inference. In the cloud on accelerators, 228 00:15:55,116 --> 00:15:57,697 [Jay Goldberg]: because the numbers just work out much better that way. 229 00:15:58,538 --> 00:15:58,681 [Ben Bajarin]: Mm-hmm. 230 00:16:00,378 --> 00:16:03,780 [Jay Goldberg]: I haven't quantified this yet. We've been talking about this a lot. But I 231 00:16:03,820 --> 00:16:12,327 [Jay Goldberg]: think my intuition is that the market for inference for generative AI is going 232 00:16:12,347 --> 00:16:15,089 [Jay Goldberg]: to be the economics are going to be so challenging that the only way it's 233 00:16:15,109 --> 00:16:20,693 [Jay Goldberg]: going to work at is if you can push a lot of that inference onto device, onto 234 00:16:20,713 --> 00:16:26,457 [Jay Goldberg]: the edge. Because there and the key thing there is the consumer, the customer 235 00:16:26,497 --> 00:16:29,668 [Jay Goldberg]: is paying for the CapEx, they're buying a phone, they're buying a PC that has 236 00:16:29,708 --> 00:16:33,193 [Jay Goldberg]: some AI functionality and running it on their device. 237 00:16:32,410 --> 00:16:32,636 [Ben Bajarin]: Mm-hmm. 238 00:16:33,213 --> 00:16:35,375 [Jay Goldberg]: That's going to offload it. Doesn't have to be run in somebody's cloud. 239 00:16:35,576 --> 00:16:39,601 [Jay Goldberg]: And I think that's the only way this really works out given the way that 240 00:16:39,701 --> 00:16:44,867 [Jay Goldberg]: generative AI is taking off. And so, yeah, that's, that's where the opportunity 241 00:16:44,887 --> 00:16:46,509 [Jay Goldberg]: is in those, in those areas around inference. 242 00:16:47,704 --> 00:16:51,968 [Ben Bajarin]: And I'd add another element that we highlighted last week that I still think 243 00:16:52,008 --> 00:16:58,434 [Ben Bajarin]: is just one of the most fascinating things to think about is the other argument 244 00:16:58,454 --> 00:17:02,719 [Ben Bajarin]: that why all of this can't continue to be done in the cloud is scarcity of resource 245 00:17:02,759 --> 00:17:04,068 [Ben Bajarin]: of energy. And 246 00:17:04,075 --> 00:17:04,259 [Jay Goldberg]: Right. 247 00:17:04,108 --> 00:17:09,030 [Ben Bajarin]: so to your point, right, CapEx, it gets offloaded if I'm now using another 248 00:17:09,070 --> 00:17:13,492 [Ben Bajarin]: device, my edge device, my car, my PC, my automotive, my camera that's sitting 249 00:17:13,552 --> 00:17:17,914 [Ben Bajarin]: right on a stoplight, um, it's, it's the one handling that power, right? So 250 00:17:17,954 --> 00:17:23,484 [Ben Bajarin]: I'm offloading power as a part of that as well, because I think you could 251 00:17:23,524 --> 00:17:26,767 [Ben Bajarin]: make the strong argument like we have that we just don't have the grid for 252 00:17:26,827 --> 00:17:30,890 [Ben Bajarin]: all of this to be run, you know, in, in the cloud, especially amongst the top 253 00:17:30,930 --> 00:17:36,193 [Ben Bajarin]: three hyperscalers, let alone people include in, in broad terms, Apple as 254 00:17:36,233 --> 00:17:41,177 [Ben Bajarin]: a top four U S hyperscaler, you know, they, it's, you just can't run all this 255 00:17:41,197 --> 00:17:44,880 [Ben Bajarin]: in the cloud. So I think that's an important reason, which again, goes back 256 00:17:44,920 --> 00:17:49,103 [Ben Bajarin]: to your other point, right? A thesis of, of on device, which we did a whole 257 00:17:49,143 --> 00:17:54,090 [Ben Bajarin]: segment on. And I think now we've seen a handful of demos and a bit of extra 258 00:17:54,110 --> 00:18:00,755 [Ben Bajarin]: conversations on device. It's definitely not there today. But at some point 259 00:18:00,795 --> 00:18:04,718 [Ben Bajarin]: in time, you'll be able to do a whole lot more of this on device, and it 260 00:18:04,738 --> 00:18:11,383 [Ben Bajarin]: will feel not that far off from the things you've experienced in cloud-centric 261 00:18:11,443 --> 00:18:12,183 [Ben Bajarin]: experiences. 262 00:18:14,014 --> 00:18:17,735 [Jay Goldberg]: Yeah. I think, I think that's, that's how this is going to work out. And I 263 00:18:17,775 --> 00:18:24,018 [Jay Goldberg]: think that it's just, it's just too cumbersome and the, the workloads are too 264 00:18:24,058 --> 00:18:28,640 [Jay Goldberg]: big to really be run any other way. It has to be some significant portion 265 00:18:28,741 --> 00:18:30,521 [Jay Goldberg]: of offload from the cloud. 266 00:18:32,220 --> 00:18:40,064 [Ben Bajarin]: So the question here that sort of plagues me is, it feels like it's gonna 267 00:18:40,084 --> 00:18:46,988 [Ben Bajarin]: be very hard to run this hybrid AI environment that a lot of people are talking 268 00:18:47,008 --> 00:18:53,011 [Ben Bajarin]: about, meaning that I provide a service, I'm a cloud provider, and I realize 269 00:18:53,071 --> 00:18:58,635 [Ben Bajarin]: that I've got tons of capabilities here to do what I wanna do in terms of this 270 00:18:58,675 --> 00:19:02,620 [Ben Bajarin]: service, what people are paying for. But yes, I want to offload that to the 271 00:19:02,660 --> 00:19:08,543 [Ben Bajarin]: device, but I need to know how much that device can be offloaded to that device. 272 00:19:08,603 --> 00:19:12,505 [Ben Bajarin]: And that's not going to be an all things equal scenario, right? Devices that 273 00:19:12,525 --> 00:19:16,767 [Ben Bajarin]: are five years old are going to need a whole lot more cloud help than devices 274 00:19:16,807 --> 00:19:21,549 [Ben Bajarin]: that are one year old. But people talk about this hybrid environment. I want 275 00:19:21,569 --> 00:19:25,070 [Ben Bajarin]: to have some of it in the cloud and some of it on device. I feel like that's 276 00:19:25,090 --> 00:19:30,841 [Ben Bajarin]: a really tricky architecture. to talk about because again, it feels like 277 00:19:31,681 --> 00:19:35,704 [Ben Bajarin]: the service needs to know, well, I can't offload that or it will be a terrible 278 00:19:35,744 --> 00:19:39,966 [Ben Bajarin]: experience or it's a capable device, I can't offload 80%. I just don't know 279 00:19:39,986 --> 00:19:43,968 [Ben Bajarin]: how this gets worked out but that's kind of how people talk about this hybrid 280 00:19:44,008 --> 00:19:48,931 [Ben Bajarin]: cloud on device today. Could change in three years, but it feels very tricky, 281 00:19:48,951 --> 00:19:50,371 [Ben Bajarin]: very complex to me to do that. 282 00:19:52,206 --> 00:19:53,516 [Jay Goldberg]: I think you just described Android. 283 00:19:54,864 --> 00:19:58,189 [Ben Bajarin]: Well, that's a whole different issue in my brain, 284 00:19:58,010 --> 00:20:00,052 [Jay Goldberg]: Yeah, 285 00:19:58,349 --> 00:19:59,551 [Ben Bajarin]: but you're 100% right. 286 00:20:00,332 --> 00:20:04,415 [Jay Goldberg]: I won't start my Android rant, but I think when you talk about the edge, we're 287 00:20:04,435 --> 00:20:10,379 [Jay Goldberg]: really talking about three things. We're talking about PCs, iPhone, and Android. 288 00:20:11,600 --> 00:20:15,422 [Jay Goldberg]: Yes, there's cameras, those will come, and then there's automotive someday 289 00:20:15,442 --> 00:20:20,266 [Jay Goldberg]: further out. But for the moment, what we're really concerned about is laptop 290 00:20:20,306 --> 00:20:26,466 [Jay Goldberg]: and phone. And at their event, what? Two months ago, AMD actually... started 291 00:20:26,506 --> 00:20:30,971 [Jay Goldberg]: talking about that, including some, some neural processing blocks in their 292 00:20:30,991 --> 00:20:36,016 [Jay Goldberg]: seat, in their laptop CPUs. Apple's been doing it for a while. Obviously 293 00:20:36,036 --> 00:20:37,457 [Jay Goldberg]: they have some in the phone as well. 294 00:20:38,351 --> 00:20:38,477 [Ben Bajarin]: Mm-hmm. 295 00:20:40,222 --> 00:20:43,904 [Jay Goldberg]: Right. That's, that's really where this edge inference is going to take 296 00:20:43,944 --> 00:20:47,587 [Jay Goldberg]: place is in those kinds of devices. And so what's going to, what's it going to 297 00:20:47,627 --> 00:20:52,050 [Jay Goldberg]: take to accomplish that is, uh, Microsoft has to get windows to the point 298 00:20:52,070 --> 00:20:52,630 [Jay Goldberg]: where it can do it. 299 00:20:53,252 --> 00:20:53,557 [Ben Bajarin]: Mm-hmm. 300 00:20:53,491 --> 00:21:00,496 [Jay Goldberg]: Apple has to, it has to do likewise for both Mac OS and iOS. And I would say, uh, 301 00:21:00,516 --> 00:21:04,618 [Jay Goldberg]: Microsoft is clearly fully invested in it and they're, they love generative AI. 302 00:21:04,658 --> 00:21:05,039 [Jay Goldberg]: And I think 303 00:21:05,305 --> 00:21:05,509 [Ben Bajarin]: Yeah. 304 00:21:05,759 --> 00:21:08,481 [Jay Goldberg]: I have to imagine that there are teams inside Microsoft working pretty heavily 305 00:21:08,521 --> 00:21:12,960 [Jay Goldberg]: to bring that. transform or to support into Windows sooner rather than 306 00:21:12,980 --> 00:21:21,587 [Jay Goldberg]: later. Apple already done it for the Mac. And so once those frameworks get 307 00:21:21,607 --> 00:21:25,090 [Jay Goldberg]: set up, I think it actually can happen pretty quickly. We will have this 308 00:21:25,130 --> 00:21:28,713 [Jay Goldberg]: problem with Android where there are going to be a lot of devices that can't 309 00:21:28,753 --> 00:21:29,654 [Jay Goldberg]: run it for years. 310 00:21:30,210 --> 00:21:30,415 [Ben Bajarin]: Mm-hmm. 311 00:21:30,575 --> 00:21:34,038 [Jay Goldberg]: And I think that's just one more thing that has to add that to the list 312 00:21:34,098 --> 00:21:38,181 [Jay Goldberg]: of problems that Google has with Android. Because when Apple thinks it's, 313 00:21:38,461 --> 00:21:43,779 [Jay Goldberg]: you know, consumers really want generative AI support on the iPhone, they'll 314 00:21:43,799 --> 00:21:47,723 [Jay Goldberg]: launch it, you know, if it's not in this iPhone, it'll be, you know, whenever 315 00:21:48,103 --> 00:21:50,646 [Jay Goldberg]: six months after they think consumers are ready for it, right? 316 00:21:51,417 --> 00:21:52,594 [Ben Bajarin]: Yeah. Well, and 317 00:21:52,628 --> 00:21:52,768 [Jay Goldberg]: So 318 00:21:52,635 --> 00:21:52,757 [Ben Bajarin]: it's. 319 00:21:53,169 --> 00:21:55,691 [Jay Goldberg]: I would be surprised if they don't talk about something along these lines 320 00:21:56,612 --> 00:21:57,673 [Jay Goldberg]: in September with the new iPhone. 321 00:21:57,660 --> 00:22:01,923 [Ben Bajarin]: Sure. So there was a report, I'm sure everybody who listens to this saw 322 00:22:01,963 --> 00:22:07,928 [Ben Bajarin]: that Apple, Mark Gurman wrote it at Bloomberg and just saying that they are 323 00:22:07,968 --> 00:22:17,175 [Ben Bajarin]: working on their own GPT model. It sounds like it's a pretty large model in 324 00:22:17,215 --> 00:22:22,716 [Ben Bajarin]: terms of overall size, like definitely large enough that whatever they're. building 325 00:22:22,736 --> 00:22:26,458 [Ben Bajarin]: data set wise is not going to run on device at that size. I mean, roughly 326 00:22:26,478 --> 00:22:31,621 [Ben Bajarin]: the stuff we've tried today seems to be a successful if it's in the 10 to 15 327 00:22:31,661 --> 00:22:37,364 [Ben Bajarin]: billion parameter range. Anything north of that is crashing devices and running 328 00:22:37,424 --> 00:22:41,867 [Ben Bajarin]: exceptionally slow. But that's today, right? That's not where we'll be in 329 00:22:41,907 --> 00:22:47,570 [Ben Bajarin]: two to three years. But it shows kind of what's possible on device versus the 330 00:22:47,630 --> 00:22:51,812 [Ben Bajarin]: size of cloud. But Apple, 100%. right, is going to want to do this. And arguably, 331 00:22:51,852 --> 00:22:56,835 [Ben Bajarin]: like I said, they are a top four hyperscaler. They can create that cloud to 332 00:22:56,855 --> 00:23:01,757 [Ben Bajarin]: device infrastructure as good as anybody, if they want, um, to handle their, 333 00:23:01,937 --> 00:23:05,359 [Ben Bajarin]: their device fragmentation, which as you point out is not as, not nearly as, 334 00:23:05,599 --> 00:23:06,699 [Ben Bajarin]: as tricky as Google's. 335 00:23:07,370 --> 00:23:13,391 [Jay Goldberg]: Yeah, Apple technically is capable of doing it today. There's no question. If 336 00:23:13,411 --> 00:23:17,392 [Jay Goldberg]: they wanted to get generative AI working on the iPhone, it could happen today. 337 00:23:18,573 --> 00:23:21,413 [Jay Goldberg]: I think the question for Apple is, or the question that Apple is asking is 338 00:23:21,493 --> 00:23:26,875 [Jay Goldberg]: more, what are we going to use this for? Apple doesn't like to add features 339 00:23:26,895 --> 00:23:27,955 [Jay Goldberg]: that consumers don't care about. 340 00:23:28,486 --> 00:23:28,696 [Ben Bajarin]: Right. 341 00:23:29,155 --> 00:23:32,596 [Jay Goldberg]: How much do consumers really care about generative AI? How important is 342 00:23:32,636 --> 00:23:35,997 [Jay Goldberg]: it to the user experience, the human experience, excuse me, 343 00:23:35,416 --> 00:23:37,206 [Ben Bajarin]: Right. 344 00:23:36,057 --> 00:23:40,631 [Jay Goldberg]: on an iPhone? And there are a lot of people who think that Apple is behind 345 00:23:40,691 --> 00:23:45,414 [Jay Goldberg]: in AI as well, because they don't have generative AI today. I think they're, 346 00:23:45,554 --> 00:23:49,376 [Jay Goldberg]: they're trying to figure out as am I, and a lot of people, like what is 347 00:23:49,456 --> 00:23:53,618 [Jay Goldberg]: actually really useful for. And I actually had a Twitter debate with somebody 348 00:23:54,099 --> 00:23:57,560 [Jay Goldberg]: who was saying, you know, AI is going to be really important, everyone's 349 00:23:57,580 --> 00:24:02,743 [Jay Goldberg]: going to want it. And I said, well, how much more would you pay for a phone 350 00:24:03,564 --> 00:24:03,944 [Jay Goldberg]: that does 351 00:24:04,036 --> 00:24:04,258 [Ben Bajarin]: Sure. 352 00:24:04,464 --> 00:24:08,466 [Jay Goldberg]: chat GPT or stable diffusion on the phone? And he says, well, I already 353 00:24:08,486 --> 00:24:11,667 [Jay Goldberg]: paid chat GPT $20 a month. And I'm like, yeah, I understand that. But that's 354 00:24:11,687 --> 00:24:17,130 [Jay Goldberg]: not the question. You like generative AI, you're willing to pay for it. How much 355 00:24:17,170 --> 00:24:21,171 [Jay Goldberg]: though are you willing to pay for it to work on your phone in airplane mode? 356 00:24:22,392 --> 00:24:25,613 [Jay Goldberg]: And I don't think anybody knows the answer to that question yet. 357 00:24:27,916 --> 00:24:31,598 [Ben Bajarin]: So there's another though, part of this that feeds into this that I think is 358 00:24:31,638 --> 00:24:38,482 [Ben Bajarin]: fascinating. And this again speaks back to companies who are prioritizing on device, 359 00:24:38,782 --> 00:24:45,846 [Ben Bajarin]: those who are not be playing both sides of cloud to device in terms of their 360 00:24:45,886 --> 00:24:51,109 [Ben Bajarin]: roadmap, is you exactly rightly point out, I don't think anybody is gonna pay 361 00:24:51,149 --> 00:24:54,411 [Ben Bajarin]: more for these things. I think it's gonna have to just be an evolution of 362 00:24:54,451 --> 00:25:00,118 [Ben Bajarin]: the silicon's... roadmap to increase those features. But if I'm a silicon designer 363 00:25:00,138 --> 00:25:05,203 [Ben Bajarin]: today, so let's just say this is MediaTek, Qualcomm, and Apple, to some degree 364 00:25:05,263 --> 00:25:11,208 [Ben Bajarin]: Intel and AMD on PCs, you do have to sort of make some decision about how 365 00:25:11,228 --> 00:25:16,672 [Ben Bajarin]: much transistor budget you're gonna throw to something like an NPU, because 366 00:25:16,693 --> 00:25:20,776 [Ben Bajarin]: it's a relevant decision, right? If I believe that I need that to compete, I 367 00:25:20,796 --> 00:25:24,819 [Ben Bajarin]: gotta take that from something else, right? I gotta take it from my GPU blocks. 368 00:25:25,124 --> 00:25:29,567 [Ben Bajarin]: my CPU blocks, right, something else, right? So they need to make calls on 369 00:25:29,607 --> 00:25:33,110 [Ben Bajarin]: how important that is by how much diary are they're gonna commit to these 370 00:25:33,150 --> 00:25:37,353 [Ben Bajarin]: things going forward. And that's a fascinating trade-off, right, that I think 371 00:25:37,393 --> 00:25:42,097 [Ben Bajarin]: people are stuck with these next two years, when again, you and I can argue 372 00:25:42,157 --> 00:25:47,081 [Ben Bajarin]: all day, people will 100% be willing to pay for those silicon bits in the 373 00:25:47,121 --> 00:25:51,268 [Ben Bajarin]: data center more. They're not gonna pay for them on device. And so that's 374 00:25:51,308 --> 00:25:55,432 [Ben Bajarin]: just a fascinating dynamic of maybe how fast the capabilities develop at 375 00:25:55,452 --> 00:25:59,255 [Ben Bajarin]: the edge, when again, you've got to make these trade-offs with your transistor 376 00:25:59,275 --> 00:25:59,555 [Ben Bajarin]: budget. 377 00:25:59,946 --> 00:26:04,729 [Jay Goldberg]: Yeah, I think that's a really good way of framing it. And I think that sort 378 00:26:04,769 --> 00:26:07,831 [Jay Goldberg]: of throws light on what I was saying about Apple is, Apple at this point 379 00:26:07,851 --> 00:26:12,153 [Jay Goldberg]: doesn't seem to see the need to throw those blocks, those resources at it. 380 00:26:12,534 --> 00:26:13,634 [Jay Goldberg]: They just don't see it as necessary. 381 00:26:16,716 --> 00:26:18,577 [Jay Goldberg]: And I don't think anyone knows when that will change. 382 00:26:20,772 --> 00:26:24,294 [Ben Bajarin]: Well, we'll see. I mean, I think we'll see with two fundamental things in my 383 00:26:24,334 --> 00:26:28,036 [Ben Bajarin]: opinion, that'll happen this fall, right? Qualcomm is going to unveil their 384 00:26:28,096 --> 00:26:32,659 [Ben Bajarin]: new PC chip, PC chip architecture. My hunch is that's going to be loaded with 385 00:26:32,719 --> 00:26:39,564 [Ben Bajarin]: tops. Um, Apple has historically only bumped their tops up. If I recall two 386 00:26:40,364 --> 00:26:46,548 [Ben Bajarin]: to three, you know, uh, I think they went from 10 to 15. Right. So maybe five 387 00:26:46,568 --> 00:26:50,910 [Ben Bajarin]: or so. So, so what they throw at tops in next generation products will be 388 00:26:50,950 --> 00:26:55,452 [Ben Bajarin]: telling. Right. If it's goes from 15 to 30, that's a pretty big jump. Right. 389 00:26:55,492 --> 00:26:59,533 [Ben Bajarin]: But if it goes from 15 to eight to 18, you know, you'll, you'll again, you'll 390 00:26:59,573 --> 00:27:04,295 [Ben Bajarin]: just sort of see how they're prioritizing. So, um, that's an interesting 391 00:27:04,355 --> 00:27:07,137 [Ben Bajarin]: bit, but, but you're right. I think that's, that's really the call they've 392 00:27:07,157 --> 00:27:12,179 [Ben Bajarin]: got to make is, is how much C block they throw to this stuff and it, and 393 00:27:12,219 --> 00:27:13,919 [Ben Bajarin]: could it be premature, you know? 394 00:27:14,870 --> 00:27:21,251 [Jay Goldberg]: Yeah, Apple, I think, typically tends to be a little bit conservative in jumping 395 00:27:21,271 --> 00:27:26,333 [Jay Goldberg]: on the latest feature, right? The original iPhone was a 2G phone deep into 396 00:27:26,353 --> 00:27:32,274 [Jay Goldberg]: the 3G era. That's the classic example. And they've had AI on their phone 397 00:27:32,534 --> 00:27:35,715 [Jay Goldberg]: longer than anybody, right? They don't call it that. It's a neural engine. 398 00:27:35,815 --> 00:27:41,777 [Jay Goldberg]: And it's there for a very specific purpose to help with image processing. I don't, 399 00:27:42,277 --> 00:27:45,858 [Jay Goldberg]: yeah, it would be interesting to see it to what degree they change it, right? 400 00:27:45,918 --> 00:27:48,805 [Jay Goldberg]: Or do they just sort of keep going at their steady incremental pace? 401 00:27:49,528 --> 00:27:57,173 [Ben Bajarin]: Yeah. So I want to end on just circling back to the question at hand about 402 00:27:57,313 --> 00:28:05,378 [Ben Bajarin]: is there even a way to sort of put a model around the additional lift of dollars 403 00:28:05,779 --> 00:28:11,303 [Ben Bajarin]: that's coming to AI? I think we agree we can't do it on device. But to that 404 00:28:11,343 --> 00:28:15,405 [Ben Bajarin]: point, in a number of earnings calls, as well as some commentary of some of 405 00:28:15,445 --> 00:28:21,073 [Ben Bajarin]: the people who are doing this, it appears that it The answer is others haven't 406 00:28:21,113 --> 00:28:27,919 [Ben Bajarin]: done this work either. That these models actually haven't been built or at least 407 00:28:27,999 --> 00:28:32,463 [Ben Bajarin]: in a way that they're confident they will portray it to customers and or investors 408 00:28:32,803 --> 00:28:38,888 [Ben Bajarin]: that they think these custom AI parts or their accelerator bits or whatever 409 00:28:38,928 --> 00:28:42,452 [Ben Bajarin]: could lead to this much more revenue. Like in terms of, I guess their model 410 00:28:42,472 --> 00:28:48,360 [Ben Bajarin]: guide, it seems like everyone kind of believes like we do. There's something 411 00:28:48,400 --> 00:28:53,043 [Ben Bajarin]: there. There will be some additional tam in the data center, but no one's really 412 00:28:53,484 --> 00:28:56,646 [Ben Bajarin]: done that work yet is basically what I'm saying. So as an answer to the question, 413 00:28:57,226 --> 00:29:01,149 [Ben Bajarin]: I'm floating the no yes, it could be, but no, that hasn't worked been done. It's 414 00:29:01,189 --> 00:29:03,651 [Ben Bajarin]: very vague speaking in vagueness. 415 00:29:04,546 --> 00:29:07,826 [Jay Goldberg]: So I will start by saying, the answering the first part of your question, 416 00:29:07,866 --> 00:29:14,028 [Jay Goldberg]: which is, yes, somebody can build this model. I don't think anybody has 417 00:29:14,048 --> 00:29:20,150 [Jay Goldberg]: yet. You and I have toyed around with this. I think our model is as advanced 418 00:29:20,230 --> 00:29:27,932 [Jay Goldberg]: as pretty much anybody's. And it's an important, important area. I tend to 419 00:29:27,972 --> 00:29:32,293 [Jay Goldberg]: think, and I've been debating this a lot with you and with other people and 420 00:29:32,333 --> 00:29:38,856 [Jay Goldberg]: myself lately. I'm coming down on the side that AI is additive to the semiconductor 421 00:29:38,897 --> 00:29:39,137 [Jay Goldberg]: TAM. 422 00:29:39,820 --> 00:29:40,485 [Ben Bajarin]: Mm-hmm, agree. 423 00:29:41,762 --> 00:29:47,086 [Jay Goldberg]: And we're recording this on Friday, July 21st, TSMC reported last night. 424 00:29:47,547 --> 00:29:54,112 [Jay Goldberg]: And they made some comments about seeing AI servers cannibalizing CPU servers 425 00:29:54,733 --> 00:29:59,437 [Jay Goldberg]: in their, in the data they track. But, but what they're really saying is what, 426 00:29:59,457 --> 00:30:04,321 [Jay Goldberg]: what they, what they really saying is because hyperscaler capex budgets 427 00:30:04,381 --> 00:30:09,045 [Jay Goldberg]: for new data centers are fixed. We haven't gotten into the new budgeting cycle. 428 00:30:09,410 --> 00:30:11,130 [Jay Goldberg]: that's going to accommodate this increase in 429 00:30:11,132 --> 00:30:11,252 [Ben Bajarin]: Mm-hmm. 430 00:30:11,250 --> 00:30:16,452 [Jay Goldberg]: AI needs. So that's a very short time window because they say long-term AI is 431 00:30:16,472 --> 00:30:21,134 [Jay Goldberg]: going to be huge in the data center. I will say anecdotally, from what I 432 00:30:21,154 --> 00:30:26,557 [Jay Goldberg]: can tell, I've heard a lot of the big hyperscalers are accelerating, pulling 433 00:30:26,597 --> 00:30:33,580 [Jay Goldberg]: forward their data center physical plant expansion because they need AI, they 434 00:30:33,600 --> 00:30:36,881 [Jay Goldberg]: need it soon, right? So I was talking to somebody recently who owns a plot 435 00:30:36,921 --> 00:30:40,135 [Jay Goldberg]: of land, it's sort of a... tier two or tier three data center location. 436 00:30:41,156 --> 00:30:41,219 [Ben Bajarin]: Hmm. 437 00:30:41,776 --> 00:30:45,159 [Jay Goldberg]: Critically, it has power, it has electricity and they're in a region where 438 00:30:45,559 --> 00:30:50,442 [Jay Goldberg]: the main tier one location is out of power. So they're on a grid, they have 439 00:30:50,502 --> 00:30:54,085 [Jay Goldberg]: electricity and they just won the lottery. They've been sitting on this piece 440 00:30:54,105 --> 00:30:58,368 [Jay Goldberg]: of land for years and now suddenly they have customers, they have all the hyperscalers, 441 00:30:58,548 --> 00:31:02,571 [Jay Goldberg]: all those usual suspects plus many more knocking on the door saying, hey, 442 00:31:02,611 --> 00:31:07,066 [Jay Goldberg]: can I get in? Let's get going, I need this, right? And... it's, you know, 443 00:31:07,086 --> 00:31:09,307 [Jay Goldberg]: they have to build a plant, they have to build the building. So it's not 444 00:31:09,407 --> 00:31:12,668 [Jay Goldberg]: going to come in, you know, next three months, but 445 00:31:13,049 --> 00:31:13,316 [Ben Bajarin]: Right. 446 00:31:13,929 --> 00:31:17,430 [Jay Goldberg]: it is clearly to me, additive to whatever the hyperscalers are doing, they're 447 00:31:17,550 --> 00:31:21,812 [Jay Goldberg]: adding data centers that they know they're adding to the plan. And so it's 448 00:31:21,832 --> 00:31:24,353 [Jay Goldberg]: not going to happen this quarter, but over the, I would say over the next year, 449 00:31:24,773 --> 00:31:26,153 [Jay Goldberg]: we're going to see this spike. 450 00:31:26,980 --> 00:31:32,164 [Ben Bajarin]: Yeah. Nope. I agree. I'm, I'm aligned with that. I think what questions I've 451 00:31:32,184 --> 00:31:35,507 [Ben Bajarin]: heard, which again, I'm it's fine that nobody knows this, but just in terms 452 00:31:35,527 --> 00:31:41,652 [Ben Bajarin]: of, of people being aware of the questions is really just how, how much 453 00:31:41,772 --> 00:31:46,436 [Ben Bajarin]: additional capex could be thrown to this. Like there's a reasonable amount, 454 00:31:46,596 --> 00:31:50,179 [Ben Bajarin]: again, knowing that we're up against limitations of physical space, we can't 455 00:31:50,199 --> 00:31:53,822 [Ben Bajarin]: get enough wafers to meet that demand. So yes, it will grow, but it's not 456 00:31:53,842 --> 00:31:56,656 [Ben Bajarin]: going to go. It's not going to be a hundred percent. right, growth year over 457 00:31:56,697 --> 00:32:03,043 [Ben Bajarin]: year. So I think the understanding the amount at which it can grow and then where 458 00:32:03,063 --> 00:32:07,207 [Ben Bajarin]: are those pockets that might get spent quickest I think are at least helpful 459 00:32:07,587 --> 00:32:11,911 [Ben Bajarin]: if you're trying to come up with where might these dollars go over the next 460 00:32:12,171 --> 00:32:19,699 [Ben Bajarin]: year or two, knowing where the constraints are. And I think that's a better 461 00:32:19,759 --> 00:32:25,130 [Ben Bajarin]: way to kind of. look at this question about how much, because again, I go 462 00:32:25,151 --> 00:32:28,433 [Ben Bajarin]: back to this, people are throwing these astronomical numbers out over the next 463 00:32:28,473 --> 00:32:32,496 [Ben Bajarin]: five years. And I keep asking, well, do we even have the land and the grid? 464 00:32:32,737 --> 00:32:37,641 [Ben Bajarin]: Can they even build out fast enough to meet that revenue number? But there's 465 00:32:37,681 --> 00:32:41,564 [Ben Bajarin]: some reasonable amount of growth that's going to come from AI to these data 466 00:32:41,584 --> 00:32:45,667 [Ben Bajarin]: centers. And so I think if the CapEx goes up, I don't know, I'm just making 467 00:32:45,687 --> 00:32:49,150 [Ben Bajarin]: this up, but let's just say we landed on it's 5% to 8% a year in terms of 468 00:32:49,170 --> 00:32:54,244 [Ben Bajarin]: your flexible budgets. it's a helpful way to look at where that growth can 469 00:32:54,284 --> 00:32:58,271 [Ben Bajarin]: come from, being, again, additive to a number where they were already spending. 470 00:32:59,710 --> 00:33:05,159 [Jay Goldberg]: Yeah, I agree. I agree. We need a little bit more work to put a precise number 471 00:33:05,200 --> 00:33:09,427 [Jay Goldberg]: on it, but we're getting closer. And 472 00:33:08,701 --> 00:33:08,903 [Ben Bajarin]: Yeah. 473 00:33:09,968 --> 00:33:12,733 [Jay Goldberg]: it's going to be, I think, a meaningful amount for a lot of companies. 474 00:33:13,272 --> 00:33:18,854 [Ben Bajarin]: Yes, no, I agree. Um, I will be at an event next week with one of the top 475 00:33:18,874 --> 00:33:22,876 [Ben Bajarin]: three hyperscalers and, uh, we'll have a chance to talk to many of their customers. 476 00:33:22,916 --> 00:33:27,978 [Ben Bajarin]: So we can, we can talk about that maybe on the next episode or episode after 477 00:33:28,018 --> 00:33:32,039 [Ben Bajarin]: that. When, uh, when I can share more of what I've learned, but this is top 478 00:33:32,079 --> 00:33:37,862 [Ben Bajarin]: of mine. This will be amongst my top questions in, uh, in, in CapEx spend 479 00:33:37,902 --> 00:33:43,961 [Ben Bajarin]: for AI specific stuff, so more on that later then. Um, all right. Well, thanks, 480 00:33:43,981 --> 00:33:48,686 [Ben Bajarin]: uh, everybody for listening. Review our podcast, give us likes, tell your 481 00:33:48,706 --> 00:33:54,111 [Ben Bajarin]: friends, share us on socials, et cetera. Uh, we appreciate everybody listening. 482 00:33:54,626 --> 00:33:55,607 [Jay Goldberg]: Thank you for listening everybody.