(Sam Nadler and Jordan Metzner are back — and this week they’re joined by Amar Goel, CEO of BITO, the AI-powered code review agent transforming how engineering teams ship software.
Jordan kicks things off by unveiling a surprise build: a fully custom BITO Slack Bot that can run PR reviews, generate stats, crack developer jokes, write haikus, and even drop into Biddo Disco Mode. Amar reacts live and pulls back the curtain on how BITO’s deep codebase analysis works — revealing how enterprise teams are merging PRs 10× faster, reducing revert rates, and catching issues that “vibe coding” tools simply miss.
From multi-million-line monorepos to legacy systems held together by duct tape, BITO’s agents are surfacing performance bugs, security vulnerabilities, logic issues, and cross-service breakages before humans ever see them. Amar explains why code review is just the start — and why BITO’s deep code intelligence unlocks a new era of AI developer tooling.
Then the trio shifts into the biggest AI news stories of the week: • OpenAI’s internal CODE RED and the escalating model war • Google Gemini’s rise and the threat of distribution • Amazon’s new AI chips and the GPU economics debate • The global AI arms race — from TPUs to supply chains to trillion-dollar CapEx bets
It’s a lively, candid, highly technical conversation with one of the sharpest minds in AI dev tooling.
(0:00) Jordan demos the BITO Slack Bot — PR reviews, jokes, haikus, & disco (1:02) Welcome + introducing guest Amar Goel, CEO of BITO (1:35) Amar’s background + BITO’s mission to build deep codebase AI agents (2:15) Why Jordan built the Slack integration prototype (3:04) What BITO can do today: reviews, tests, explanations, stats & more (4:18) The PR demo: catching security + maintainability vulnerabilities (5:22) Humor in devtools — BITO Fun, BITO Surprise, & developer haikus (6:44) Amar reacts: how customers want notifications & Slack workflows (7:35) Why existing tools fail on large, messy, real-world codebases (8:52) Deep code understanding explained — ASTs, symbol indexes, repo mapping (10:26) Why “vibe coding” breaks down in enterprise environments (11:31) How BITO integrates into Cursor, Windsurf, Claude Code, JetBrains & VS Code (12:10) The explosion of code volume — and why quality gates now matter (13:00) PRs merging 10× faster with BITO + 55% fewer reverted commits (14:05) Training junior devs through AI feedback + customizable sensitivity modes (15:20) What’s next for BITO (without giving away secrets) (16:12) NEWS #1 — OpenAI declares CODE RED (17:01) Google’s Gemini advantage: distribution, docs, slides & ad model economics (18:33) The coming AI model war — NVIDIA, xAI, Anthropic, Google (19:48) NEWS #2 — Amazon’s new AI chips & the GPU supply chain crunch (21:10) NEWS #3 — Global CapEx, GPU shortages & trillion-dollar questions (22:42) Final thoughts + Amar’s closing remarks (23:30) Wrap-up & teaser for next week’s episode
Built This Week is a weekly podcast where real builders share what they're shipping, the AI tools they're trying, and the tech news that actually matters. Hosted by Sam and Jordan from Ryz Labs, the show offers a raw, inside look at building products in the AI era—no fluff, no performative hype, just honest takes and practical insights from the front lines.
Jordan Metzner:
Yeah. I got okay. So I got one last feature of the Bito bot. So let me just try it. Oh,
Amar Goel:
that's awesome. Alright.
Sam Nadler:
Hey, everyone, and welcome to Built This Week, the podcast where we share what we're building, how we're building it, and what it means for the world of AI and startups. I'm Sam Nadler, cofounder here at Ryz Labs. And each and every week, I'm joined by my friend, business partner, cohost Jordan Metzner. And this week, we have a special guest.
Amar Goel:
Yeah. Great. Thanks for having me, Sam and Jordan. Nice to connect and and fan of the podcast. Yeah.
Amar Goel:
My name's Amar. I'm based here in the Bay Area in, in Menlo Park and, you know, been working on GenAI solutions for developers. We build agents. My company, Bitdo, we build agents for developers really centered around deep code based understanding, and I'm excited to be chatting with you guys today. And, obviously, always, lots going on in the in the world of GenAI, so there's some cool stuff to to discuss.
Jordan Metzner:
Hey. Hey, Amur. Great to see you. So happy to have you on the show. Obviously, it's been a crazy, you know, past ten, fifteen days in the world of AI over the Thanksgiving break, and happy to show you what we built.
Jordan Metzner:
I thought it'd be a fun tool as we connected in our pre call. So I think we can we can jump right into that. And let me show you what I built, and then maybe we can have a little fun. So my understanding was Bitto doesn't have a a formal Slack integration yet or Slack bot yet. So I built a kind of refresh of Slack.
Jordan Metzner:
I built this using Google AI Studio which has been really helpful in kind of like building out these types of tools. So, you know, a very basic prompt about building kind of an AI agent, about building a bot agent for Slack, and it kinda helped me imitate the Slack interface. So, you know, how does the Bitto agent work? So the first thing you can do is do like Bitto help, and Bito help will tell you all the things that Bito can do. So it can review code, it can do tests, it can explain, it can do last stats, fun, help, or surprise, or other things.
Jordan Metzner:
So I thought the easiest one is maybe we can just start with Bito last. So that'll give you an actual review of your last PR. So what it's going to do here is say like, okay, here's my last PR that I did. It implemented dark mode with toggle and here it's showing that failed due to a critical security bug. So it looks like my PR didn't pass, and and here's all the reasoning why that I got from the Bitdo platform.
Jordan Metzner:
Okay. Great. So now that I realized my PR failed, is, you know, not so happy, I figured, you know, maybe I could have a little bit of fun. So bit of fun is, hey, Sam, why did the developer go broke? Because because he used up all his cash.
Jordan Metzner:
Okay. So That's awesome. Alright. So Bito's got jokes.
Amar Goel:
I have used that joke in some cold emails that I have sent out. And it gets a really high
Jordan Metzner:
It hits?
Amar Goel:
It hits? Really, mean, it hasn't resulted in amazing responses, but it gets a really high open rate, you know.
Jordan Metzner:
Okay. Cool. So I I also built Bito Surprise. So, let's see what Bito Surprise is. Oh, cool.
Jordan Metzner:
Surprise. Okay. So it actually wrote me a Bito Haiku. So let's see. Code push to main.
Jordan Metzner:
Fingers crossed the tests all pass. Prod is now updated. Okay. Cool. So we have like a little bit of haiku.
Jordan Metzner:
It looks like, you know, I guess I gotta fix my security vulnerability bug and then I can push to prod. Yeah. But I have one last feature.
Amar Goel:
You had some maintainability bugs in there too, which is kinda critical. So
Jordan Metzner:
Yeah. I got okay. So I got one last feature of the Bido bot. So let me just try. Bito disco.
Amar Goel:
Oh, that's awesome.
Jordan Metzner:
Alright. So Bito can dance and play disco music. And that is my Slack Bito bot. I hope you liked it.
Amar Goel:
Yeah. Thank you. That was that was cool. I think we should roll this out to production right away.
Jordan Metzner:
Well, it's I would say it's definitely not ready. It might need a full pull requests before it's ready for that. But, we love using Slack as an integration channel obviously, and we use it all the time for notifications and, like, callouts when, you know, things don't go right, etcetera. So, you know, I I didn't go too deep into it. And, you know, as I'm not a obviously, like, an expert user in the Bito platform, but I'm sure there's some other kind of Bito calls that you could build in here that could make, you know, the the Slack integration incredibly helpful for other developers.
Amar Goel:
We have had this request for, like, hey. Can you make my code review notifications or just, you know, let me know as a developer when the review's finished running or, you know, highlighting issues or maybe for the reviewer, hey, like, Bito's run. Here's some things that it's found, you know. Now you can dig in kind of thing.
Jordan Metzner:
Okay. Cool. Alright. So on our segue into that, tell us a little bit about Vitto. You know, kind of why'd you build it?
Jordan Metzner:
Where did it you know, tell us a little bit about your history of, like, you know, why why you decided to get into the space. Obviously, you know, AI and coding is just the topic du jour in general. So, you know, how how does Bitdo stand out and, you know, kind of what are its differentiators and and, how are companies using it?
Amar Goel:
For for us at Bitdo, you know, we've been focused on developers for, you know, our whole history. So we're really trying to build, you know, powerful kind of enterprise quality agents for developers. I mean, for us, everything really kinda stems from deep code base understanding. So we think that that is kind of critical to unlock all of GenAI or kind of the next level of GenAI. You know?
Amar Goel:
I think, of course, everyone's using tools like Cursor and ClaudeCode and Codex and Copilot and whatnot. But I think what we've seen is that those tools really excel at vibe coding, and they, everyone talks about, hey. When I'm building something totally new, they work great, you know, almost like kinda one shot. But when I start to get into my existing kinda gnarly code base that's been built for five, ten, twenty years, and I have millions of lines of code and hundreds of repos and different services, all of a sudden, tools start to kind of break down. Right?
Amar Goel:
And so our first agent is really around code review, right, where we kind of deeply go and understand your code base, how it's all connected, the different services you have, all those kinds of things. We build symbol indexes and abstract syntax trees. So in GitHub, GitLab, Bitbucket, we can run this really thorough review. You know? And, you know, what teams tell us over and over again is like, hey.
Amar Goel:
The quality of the review that you're able to provide is just far beyond simple things like telling cursor, you know, Claude code, hey. Review my code. You know? Like, it just tends to find a lot deeper kinda issues around performance, security, scalability, you know, vulnerabilities, all those kinds of things, logic issues, etcetera. But, you know, we really feel like that deep code based understanding has a lot of applicability more broadly, and so we're looking at, you know, ways to enable that even to help teams with finding production issues and, you know, other other things.
Amar Goel:
And can they use it to, like, maybe understand code bases better and things like that. So, you know, that's really kind of what we're what we're focused on, you know, and we're excited, you know, seeing a lot of traction with with larger companies.
Jordan Metzner:
Awesome. So, just so I, like, understand correctly, I you know, I'm a big user. Obviously, like and and some of these other tools you mentioned. Obviously, they're great for reviewing the code you just wrote that minute, you know, because it is in some sense or the code it just wrote that minute. But to your point, for, like, a large mono repo or a large microservice based repo, it's really hard to get any of these tools to be able to index that amount of quantity.
Jordan Metzner:
None of them have the ability to do that. And so what happens for us as developer is you're able to analyze what you wrote or what what the AI just vibe coded but not how well that fits within your entire code base both like stylistically security, etcetera. Right? And so essentially, I guess, you know, these integrations, I guess, of writing these vibe codes with this classic code base end up failing and hence where your tool comes in most value. Is that is that like the best way to put it?
Amar Goel:
Yeah. Yeah. I think the way I would think about it is, like yeah. So we actually have, like, you can run our review in Cursor and Windsurf and, Claude code and and stuff as well. So, you know, we Studio Code, JetBrains.
Amar Goel:
So, like, lots of teams use our agent in their IDE as well to review their code because, you know, what we're able to do is, again, with all the different tools and the agentic approach, like, we're able to just go much deeper. We break up each diff into a different view or understanding, like, maybe you do have like, for example, a simple thing would be, like, you know, you're just reviewing some code and you you you say, like, you know, oh, well, you didn't have error handling. Well, maybe that team doesn't really care about error handling or or the the the error handling is being handled more upstream. Right? So those kinds of things.
Amar Goel:
Or again, because you have that deeper understanding of the whole code base, you know, you're able to say, hey, you've changed these this service here, but now, you know, that's gonna result in a problem in this upstream service that that calls it, you know. So you're able to, like, point out those issues, you know, in a much deeper way. And so I think, yeah, I think if you have, like, a simple kind of, you know, code base or you're just doing a Vibe Coded project, yeah, I don't think Bitdo is, like, probably necessary for you. But I think if you live in, like, an enterprise environment, you know, that really makes a difference. Some something that's, you know, a real quality gate.
Amar Goel:
The other thing I would kinda just mention is we're seeing a real proliferation of these tools. Right? So, you know, as you go into larger companies, like, some people are in cursor, some people are in cloud code, and so, you know, a lot of times people like, oh, well, front end team is using this other thing or, you know, we're using, some back end developers using JetBrains. So then you're kind of in this mode of, like, well, is everyone actually reviewing their code locally? We really need a formal kinda gate check as this code is going to production.
Amar Goel:
And I think the other thing we're seeing is that the amount of code that everyone's producing is going up dramatically. And so if you don't really have, like, a formal kinda gate check, who knows what's kinda getting into production? I mean, we've seen teams tell us things like PRs are getting so big that reviewing code is taking days now. And also when a bug gets into production, the developer a lot of times didn't even write the code. So nobody really knows what's going on anymore.
Amar Goel:
You know? It's everyone's like, I gotta go check and figure this out. It's so different than two, three years ago where it's like what somebody's like, I wrote that code. I deeply understand how it works, and I can start thinking about how this bug might be happening. Now people are like, well, I scanned the bug.
Amar Goel:
Maybe I ran some unit tests, but, like, I need to go, like, learn this code now to understand why there's this production issue.
Sam Nadler:
How do you think and that, obviously, I'm not an expert in this domain, but how do you think about, like, the quality versus speed trade off? Like, in terms of you know, I think Bitdo is working to help teams have higher quality code pushed to production, but does it significantly hurt, like, the speed to which they're moving?
Amar Goel:
It's actually speeding teams up. So we're seeing on average PRs merge in, 10 x the speed. So a PR that used to take fifty hours to merge is now merging in five to seven hours because most enterprise teams still have one or two people who manually go and review a PR. But Bitos agent runs and we provide feedback in, you know, five minutes. So team like Rakuten or Gainsight or Whatfix or whatever.
Amar Goel:
I mean, teams are like, hey, I got my PR feedback in five minutes. I used to have to wait maybe a day or twelve hours to get feedback. So the developer can quickly address those. Then the senior reviewer who's comes in you know, the developer who submits that code and the reviewer are still responsible for the quality of that code. You know?
Amar Goel:
And so but then that senior reviewer who comes in is like, okay. I don't have to deal with all the basic blocking and tackling. I can focus more now on kinda architectural level thinking, high level design. So they also know that the PR review isn't this mind numbing task and going through the code line by line and being responsible. So they come in in a couple hours.
Amar Goel:
They don't have as much feedback to give. So the amount of typically, we're providing, like, 80 75, 80% of the total feedback that a PR receives. And, you know, where customers are seeing bugs go down by, like, 53, 55%. Like, the number of kind of reverted PRs, in the month following is down by 55%. Right?
Amar Goel:
So it is helping speed the team up and do a better job because it doesn't get tired. So, it kind of feels to me like a no brainer, know.
Jordan Metzner:
I guess, as these teams start to get these PRs rejected, they get even better at like their processes and so that also inherently speeds up the team. Right? The flywheel gets smaller and smaller for issues that get blocked into PRs. Right?
Amar Goel:
Yeah. I mean you can also implement custom rules and guidelines. The agent actually looks at what the product historical discussions your team has had. It also learns, like, what things your team is not paying attention to that we're flagging. So we're saying, hey.
Amar Goel:
Your team doesn't care. But, yeah, I mean, I think it's a great we've seen a lot of people say, hey. This helps my junior developers learn. Right? Because the the agent is flagging stuff.
Amar Goel:
So some people want the agent to be noisier and chattier in the sense of, like, highlighting more issues because they're like, hey. These teams can learn from this. Other teams are like, hey. We just want, like, production level issues, you know, that are gonna cause big problems in production, and we call it, like, essential mode. So, you know, every we find every team's a little bit different in terms of what their needs are and what their, you know, desire is, but it is a learning tool as well.
Amar Goel:
Absolutely.
Sam Nadler:
Can your customers set these settings? Can they have it, you know, hypersensitive or, you know, lower sensitive? Amazing.
Amar Goel:
Yeah. Yeah. So you can say, hey. I only want, like, really essential issues. I want more comprehensive thing.
Amar Goel:
I want you to flag these other issues. And then the agent, like I said, is learning from what your team actually does. Lots of teams or let me say team leads or engineering leaders have a different vision of what they want than what the team actually does. And so that's also we're finding that it's helpful for us for the agent to learn like what is the team actually doing? And we've seen some teams say the engineering says, don't look at what my team does.
Amar Goel:
Here's what we need. And so I want the agent to do this every single time. And then other people are like, you should learn from what my agent's doing my team's doing and then adjust.
Jordan Metzner:
That's cool. Yeah. Obviously, it depends on the product, what they're building, what their stack is, I mean, what the team's goals are. I I could totally see I can even see just depending on, like, even the repo or even who the developer is. I mean, maybe for a lower level developer, you would maybe want, like, more restrictions on him and, you know, it's a higher level expectations versus maybe higher level developers and maybe give them kind of more freedom over time.
Jordan Metzner:
Super cool. Super cool. Well, Umer, awesome project. Really, really cool application. And One more question.
Jordan Metzner:
Yeah. Sure. You know, obviously, people can go to to bito.ai. But, how else should they get in contact with you? I mean, go ahead, Sam.
Jordan Metzner:
Sorry to
Sam Nadler:
Yeah. I was just you know, without giving anything away that's top secret, kinda what's next? What's on the horizon, that that you're excited about?
Amar Goel:
Yeah. I mean, we actually have an exciting product launch coming up next week, that, I I won't totally share it all yet, but I think, you know, I talked about deep code based understanding being kind of one of the things that we've really, really focused on. And I think we've been, you know, both thinking how can we help customers get more value out of that, but also people asking us and kind of, you know, saying, hey. How do we, you know, how can we leverage this outside of just code review? So that's, you know, an area that we're that we're working on and, really excited about for now.
Jordan Metzner:
That's awesome. Okay. Cool. So we don't wanna ruin that. Awesome.
Jordan Metzner:
And, cool, awesome product for developers. And let's jump into the news. Let's go ahead, Sam.
Sam Nadler:
Yeah. Okay. So, you know, news story number one is Code at OpenAI. Code Red at OpenAI. OpenAI has or Sam Altman has declared a Code Red internally.
Sam Nadler:
This leaked, I think, within the last forty eight hours, you know, pulling back all other initiatives to focus solely on the ChatGPT product. Obviously, I think it's in response to Google and Gemini's, you know, significant advancements over the past several weeks or months. Would love to hear your your thoughts on on this competitive pressure from Google to to OpenAI.
Amar Goel:
For a long time, I've kind of felt like OpenAI went too broad too fast. Know? They they kind of have gotten into so many different areas, you know, voice models, you know, video models, image models. There's chat TBT, there's coding. And I feel like, you know, they got the consumer business, the API business.
Amar Goel:
Now look, they are the biggest player in the space, 20,000,000,000 in revenue, annualized run rate revenue. So, I mean, I think I could I can understand how they got to this situation, but it felt a little bit like they were going if if this space is gonna be so big, you know, you're gonna need to, like, really be good at a few things. And I think by kind of going after everything, you know, they've left a lot of room around their flanks, you know. So, you know, Anthropic has really kind of excelled in the coding use case. Anthropic is also really good at writing.
Amar Goel:
Now Gemini, you know you know, I saw like Mark Benioff over the weekend said he thought Gemini three was significantly better than ChatGPT, and he switched over to that after years of using ChatGPT. I mean, I personally haven't felt like Gemini three is like so groundbreaking per se. I know on the benchmarks it's done really well, but I do think it's quite a good model. Like, I'm able to I'm using it now whereas before I was just kinda defaulting to ChatGPT and Claude. But now I'm like, oh, Gemini sometimes I like its answers better and whatever.
Amar Goel:
But I still think it's like a really, the product is not nearly as rich, you know, can't make a PDF or, you know, things like that. But they have so much distribution. Right? Like, you know, you they've they've got the button in Gemini, the Gemini button sorry, in in Chrome. They've got AI mode now.
Amar Goel:
Right? And so it's kind of front and center, you know, in your face. And they also have an ad model that allows them to not have to charge consumers. Right? They like can make money from from the ads whereas, you know, ChatGPT, you know, is I mean, a lot of the revenue is coming from their ad model.
Amar Goel:
And I mean, there's no Internet product that has achieved mass scale, like, on the order of, you know, a billion users or more by having a paid model. Know? There's just not enough people that are willing to to pay money. So, you know, I think that, it probably is a bit of a code red. I mean, I don't think it's, like, their their, roadkill or anything like that, but I think they do need to probably focus in a a little bit.
Amar Goel:
And, actually, it's funny that there's a sorry. It's a long comment. But, the strategic review blog kinda mentioned this week that they think OpenAI needs to get an ad model. And I thought it was so interesting that, you know, I think these articles have been saying that they put their ad products on hold, you know, to focus in on ChatGPT and I think go more into the, you know, the making the product better, which is probably a good start, but they're still sticking to the paid model as their primary mechanism.
Jordan Metzner:
Yeah. It's been, I mean, I I think from my perspective, I think, you know, it's been a dangerous few months for them. And I think you mentioned some of the points here, but, you know, it's it's obvious, like, Nano Banana has, like, the best image generation as far as the largest models on the market. Obviously, we're seeing a lot of, like, you know, Salesforce just invested, you know, more in Black Forest, so they're trying to double down in in image generation. But, obviously, Google has got a pretty awesome model for video generation.
Jordan Metzner:
I mean, as of the past few weeks, you know, Veo three point one was probably in in the front spot. Obviously, we've seen some some new models come out. But I think going back to what you're saying about Google and Gemini is, you know, most of the things I wanna do with these LLMs is to create slides or to create a Google Doc or to create a Gmail. Right? And so the distribution is so dangerous.
Jordan Metzner:
I mean, I know they've launched like nana bananas out of slides but, you know, the kind of gamification of Google Slides and connecting Gemini into Google Slides and into Google Docs where, you know, half the time of what I'm doing in ChatTPT is just, like, copying and pasting to go create a Google Doc to, like, put it in there, you know. So I think the opportunity is kind of unlimited both, like, on the consumer enterprise side. And I don't know. I think it I think the real danger is that, like, Google has, like, almost unlimited free cash. And, yeah, gbt, I mean, like, who cares about their 20,000,000,000 in revenue?
Jordan Metzner:
Like, how much of it was profit? The answer is probably none of it. And, therefore, like, they're gonna continue to have to raise capital, whereas Google can just give away their products for free and integrate it into their stack. And, you know, we use Google Suite. I think, like, most startups use Google Suite as, like, their Gmail, you know, email suite.
Jordan Metzner:
And, you know, they can probably lose money for longer on, like, you know, financing or sub financing this thing. And then maybe on top of that, you know, maybe Google's TPUs allows them to run this even cheaper. Right? And so they're just like, I can run this for cheaper and longer than you and, you know, here we go. And I don't know if, like, you know, people really understand, like, what a big war this is gonna be.
Jordan Metzner:
You know, I think I think the founders of Google hate Elon at this point. I think Elon hates them. I think Sam hates Elon. I think, you know, Elon hates Sam. I think, you know, I mean, I think this is a straight up war.
Amar Goel:
I mean, x AI x AI, I have to give them a lot of credit. Like, it's miraculous to me how quickly they built, like, a pretty decent model, you know, with Grok four and Grok fast code one. And, you know, it just I mean, within a year, they kinda showed up in the leaderboards and and a really impressive kind of cost performance ratio. So that is I mean I mean, I thought Elon had a lot on his plate, and somehow he stood up a 100,000 GPUs and just went to work, and it's pretty amazing with team.
Jordan Metzner:
Yeah. I think it just means that we have no idea what the future is gonna hold for who's gonna have the best models in the short term or medium or even long term. Right?
Amar Goel:
Yeah. Totally. Totally.
Jordan Metzner:
Alright. Let's jump into Amazon.
Sam Nadler:
Well, yeah, talking about GPUs. Amazon releases the new AI chip, at, Reinvent, I think, yesterday, was it, or two days ago, the Tranium three, supposedly four x faster and a cheaper way to train big models. You know, I'm a very bullish Nvidia bull, and I think most people are, to be honest. But, you know, what are your thoughts on Amazon's new chip? Jordan, by the way, Amr is a, Amazon alum, so he's got some, usually, some strong feelings about, whatever's going on at Amazon.
Amar Goel:
Got it. Interesting. Yeah. I mean, I think if you take a step back, you know, there's a number of folks now. I mean, Google's also got, you know, the TPUs.
Amar Goel:
And so, I think, like, you know, they also now have gone from saying, hey. We're not gonna just offer this in Google Cloud. We're gonna also offer this to other people to set up in their own data centers. Like, supposedly, they're talking to Meta about a, you know, multibillion dollar kind of deal. You know, AMD is, of course, out there as well.
Amar Goel:
So, you know, more and more people are, you know, trying to trying to peel off a piece of that GPU market. I guess I'm a little surprised it took AWS and Google Cloud, like, so long to, like, say, hey. We'll make these chips available to anyone, which then makes me wonder maybe they're not that great at at the price performance curve, you know, compared to NVIDIA chips. I mean, generally, what I've seen is that heard is, like, from when you're doing training, NVIDIA chips are still, like, blow everybody else out of the water. But when it comes to inference, you know, other people are starting to to catch up, and the inference workloads are growing, you know, very, very dramatically.
Amar Goel:
I mean, the sense I still get is that TPUs are still well ahead of, like, the training and chips from Amazon. But then, of course, you've also got folks like Rock with a q and Cerebras and SambaNova that are trying to do really fast. So so, anyways, I think, like, it feels like you would think in the next year if NVIDIA's dominance or market share was to come down that it would happen in the next year. But so far, they've done a good job of kinda keeping a lock on the on the marketplace.
Jordan Metzner:
Yeah. So I I agree with, like, kinda most of your points there. I would say that probably Amazon, you know, didn't start to sell it because they didn't have any. Right? Like, you know, these chips are three nanometers.
Jordan Metzner:
Right? And so, like, how long does it take to, like, fabricate them and, like, get them in data centers and, like, test them and make sure they even work for their own workloads first and, you know, how much money do they save by offloading, you know, third party workloads to other chips, to their own chips. And so, you know, I would guess that there's probably some supply, you know, plus tariffs. Right? So there's probably some actual supply chain issues that, like, actually cause them to get these things, like, up and running.
Jordan Metzner:
I do think that, like, as we see this logarithmic curve of these models that, you know, I don't think any any of these LLMs have done a good job of, I would say, like, routing, you know, taking basically, like, simple tasks and sending it to, like, the cheapest model possible if it's, like, cheap enough and fast enough. And over time, I know, you know, Amazon has Bedrock which is kind of like this, like, LLM kind of routing platform. Right? So you can start to see over time, like, for certain tasks, you can probably offload those tasks to a cheaper processor, to a cheaper GPU, right, and be able to save a significant amount of money especially if it's things like maybe like ETL pipelines and things like that where, like maybe they become really common tasks like AWS alternatives. My guess is like they, again, they just don't have enough.
Jordan Metzner:
It doesn't seem like anybody has enough. I mean we try to buy GPUs online and it's very they're very hard to come by and they're hard to find. You know, whether Amazon's, like, TPU is gonna be able to compete with, like, NVIDIA's top line chips, it doesn't look like it. But I don't know if that's really their goal in the short term. I mean, maybe they can just offline tasks that are, you know, being run by servers that are very expensive today that can move over.
Jordan Metzner:
Right?
Amar Goel:
I do think there's this, like, interesting kind of paradox a little bit in the market today, which is, you know, you said Amazon doesn't have that many chips. Right? All the producers of these goods, whether they're chips or models, are kinda saying, hey. There's we're, like, run out. Right?
Amar Goel:
The the the the demand is dramatically exceeding the supply. And then we're certainly certainly see that with a lot of model providers when we try to access their models. You know, we use billions of tokens a day, and we're like, oh, they're like, hey. We don't have that much capacity for you. Know?
Amar Goel:
But then when you look at the aggregate revenue that's adding up here, it's not yet that significant. You know? I mean, we're probably talking about $4,050,000,000,000 dollars of of revenue, you know, a year max that's happening in AI, which, you know, okay, significant in terms of the growth curve. But, you know, when you look at, like, the half 1,000,000,000,000 or whatever, 400,000,000,000 that's being spent on infrastructure, that isn't that much revenue. You know?
Amar Goel:
So, again, maybe that's just we're early in the investment curve and and that's gonna adjust over over time here.
Jordan Metzner:
Everything's a wrapper. Right? So well, everything's well, let me just finish that. Right? Like, like, everything's a wrapper.
Jordan Metzner:
Right? And so, like, all you need is one bottleneck inside this, like, wrapper flow to, you know, essentially limit capacity and whether that's coming, like, you know, from TSMC out of Taiwan or, like, Nvidia out of the chips or out of the data center or the power limitations or even just, like, you know, the, you know, the pipeline of Internet to be able to get to these data centers. Right?
Amar Goel:
I mean, it's it's not that dissimilar to, like, maybe in that sense, you know, if you look back to, like, late nineties, early two you know, February you couldn't get servers. That's exactly right. Right? Servers were like this commodity not commodity. I mean, were like this precious good.
Amar Goel:
I actually funny story. My next door neighbor when, when I was growing up was this guy, Ed Zander, who is the COO of Sun Microsystems. You know, we put the .in.com. And, you know, getting servers from them was like getting GPUs, you know, last year. It was like $100,000 a server, and it was just crazy how much you had to spend and, like, they didn't have them.
Amar Goel:
They were like, we're sold out. You know? And so everybody was, like, trying to get those things. And like you said, and then they were like, we can't produce the chips fast enough. So, you know, when you go through these kind of boom cycles, you know, that's it's like when oil fields are, like, going cranking and people are like, I can't forget enough people to drill the holes.
Amar Goel:
I don't have enough drills. It's the same thing.
Sam Nadler:
Well, the IBM CEO says, you know, the math isn't mathing, with this CapEx build out and that, you know, it it just the returns aren't gonna make economic sense. I know we just kinda commented on this, but any follow-up thoughts?
Amar Goel:
Yeah. I mean, I think this is very related to the discussion we were just just kinda having, right, which is that, you know, the the investment that's being discussed or or being committed to, I mean, is really, you know, massive. Right? And, I mean, I think, you know, he says in this article here, 8,000,000,000,000 of CapEx means you need 800,000,000,000 of profit, not revenue, profit just to pay for the interest. Right?
Amar Goel:
So that means you probably need, you know, 1 and a half or $2,000,000,000,000 of revenue, you know, to pay for the pay you know, to get that profit to then, you know, just pay the interest cost to cover. So that's not actually, like, even giving you excess profit that you can put in your in your, you know, log below your expense line. Right? So, I mean, that, you know, seems like we're pretty pretty far from that. And again, you know, this is a super huge investment market and whatever.
Amar Goel:
I mean, I think the interesting thing a little bit is, like, do you need to go down this path? Right? So if you look at Anthropa, I think, the New York Times or the Wall Street Journal, somebody just had a big maybe Forbes. Somebody had a big interview with him, with Dario just a day or two ago. Well, I mean, he was kind of, you know, they're take playing a different game, and he was just kinda, like, poking fun at, or or a jest or whatever at OpenAI being like, yeah.
Amar Goel:
Do you really need to spend all this money? I mean, again, they're spending quite a bit of money too, but it's literally an order of magnitude less. I mean, I think their loss this year is supposed to be $2,000,000,000. I think OpenAI is probably like a 10 or 20. And so they're kinda like, hey.
Amar Goel:
Do you really need to spend, you know, that much money to to train your models? And so it does appear that there are other approaches you could, you know, kinda take. I mean, if you look at what's coming out of the Chinese labs, like DeepSeq and Moonshot and all that, I mean, they're training mean, I they're probably making fun of Anthropic. Right? They're training models at one hundredth the price of those models or at least one tenth.
Amar Goel:
So, yeah, I'm not I I look. I'm not an expert on this. So, you know, and everyone loves to be an armchair quarterback, but, you know, it does seem like we're getting a little bit over our heels here, you know, or skis, I mean.
Sam Nadler:
Amar, thank you for joining. Jordan, thank you so much. It was a great episode. If episode 23, if you have a chance, like and subscribe, and we'll see everyone next week.
