Ejaaz: A bunch of AI researchers from China just released a brand new AI model called
Ejaaz: Kimi K2, which is not only as good as any other top model like Claude,
Ejaaz: but it is also 100% open source, which means it's free to take,
Ejaaz: customize and create into your own brand new AI model.
Ejaaz: This thing is amazing at coding, it beats any other model at creative writing,
Ejaaz: and it also has a pretty insane voice mode.
Ejaaz: Oh, and I should probably mention that it is one trillion parameters in size,
Ejaaz: which makes it one of the biggest and largest models to ever be created.
Ejaaz: Josh, we were winding down on a Friday night and this news broke that this team
Ejaaz: had released this model.
Ejaaz: Absolutely crazy bomb, especially with like OpenAI rumored to release their
Ejaaz: open source model this week.
Ejaaz: You've been jumping into this. What's your take?
Josh: Yeah. So last week we crowned Grok 4 as the new leading private model, closed source model.
Josh: This week we got to give the crown to Kimi K2 we got another crown
Josh: going for the open source team they are winning I mean this is
Josh: better than DeepSeek and DeepSeek R2 this is basically DeepSeek R3
Josh: I would imagine um and if you remember back a couple months DeepSeek really
Josh: flipped the world on its head because of how efficient it was and the algorithmic
Josh: upgrades it made and I think what we see with Kimi K2 is a lot of the same thing
Josh: it's it's these novel breakthroughs that come as a downstream effect of their
Josh: needing to be resourceful
Josh: China, they don't have the mega GPU clusters we have, they don't have all the
Josh: cutting edge hardware, but they do have the software prowess to find these efficiencies.
Josh: I think that's what makes this model so special. And that's what we're going
Josh: to get into here is specifically what they did to make this model so special.
Ejaaz: Yeah, I mean, look at these stats here, Josh, like 1 trillion parameters in total.
Ejaaz: It's 32 billion active mixture of expert models. So what this means is,
Ejaaz: although it's really large in size, typically these AI models can become pretty
Ejaaz: inefficient if it's large in size, it uses this technique called mixture of
Ejaaz: experts, which means that whenever someone queries a model,
Ejaaz: it only uses or activates a number of parameters that are relevant for the query itself.
Ejaaz: So it's more smarter, it's much more efficient, and it doesn't use or consume
Ejaaz: as much energy as you would if you wanted to run it locally at home or whatever
Ejaaz: that might be. It's also super cheap.
Ejaaz: I think I saw somewhere that this was 20% the cost of clawed,
Ejaaz: josh which uh we love that insane uh
Ejaaz: for all the nerds that kind of want to run you know
Ejaaz: really long tasks or you know just set and
Ejaaz: forget the ai to to run on like your coding log or whatever that might mean
Ejaaz: you can now do it at a much more affordable rate at one-fifth the cost uh than
Ejaaz: some of the top models that are out there and it is as good as those models
Ejaaz: so just insane kinds of things josh i know there's a bunch of things that you
Ejaaz: wanted to point out here on benchmarks um And what do you want to get into?
Josh: Yeah, it's really amazing. So they took 15 and a half trillion tokens and they
Josh: condensed those down into a one trillion parameter model.
Josh: And then what's amazing is when you use this model, like she said,
Josh: it uses a thing called mixture of experts.
Josh: So it has, I believe, 384 experts.
Josh: And each expert is good at a specific thing. So let's say in the case you want
Josh: to do a math problem, it will take a 32 billion parameter subset of the one
Josh: trillion total parameters, and it will choose eight of these different
Josh: Experts in a specific thing. So in the case of math, it'll find an expert that
Josh: has the calculator tool.
Josh: It'll find an expert that has a fact, like a fact checking tool or a proof tool
Josh: to make sure that the math is accurate.
Josh: It'll have just a series of tools to help itself. And that's kind of how it
Josh: works so efficiently is instead of using a trillion parameters at once,
Josh: it uses just 32 billion and it uses the eight best specialists out of the 384
Josh: that it has available to it. It's really impressive.
Josh: And what we see here is the benchmarks that we're showing on screen.
Josh: And the benchmarks are really good.
Josh: It's up there in line with just about any other top model, except with the exception
Josh: that this is open source.
Josh: And there was another breakthrough that we had, which was the actual way that
Josh: they handled the training of this.
Josh: And yeah, this is the loss curve. So what you're looking at on screen for the
Josh: people who are listening, it's this really pretty smooth curve that kind of
Josh: starts at the top and it trends down in a very predictable and smooth way.
Josh: And most curves don't look like this. And if they do look like this,
Josh: it's because the company has spent tons and tons of money on error correction
Josh: to make sure this curve is so smooth.
Josh: So basically what you're seeing is the training run of the model.
Josh: And a lot of times what happens is you get these very sharp spikes and it starts
Josh: to defer away from the normal training run.
Josh: And it takes a lot of compute to kind of recalibrate and push that back into the right way.
Josh: What they've managed to do is really make it very smooth.
Josh: And they've done this by increasing these efficiencies. So if you can think
Josh: about it, there's this analogy I was thinking of right before we hit the record button.
Josh: And it's if you were teaching a chef how to cook, right?
Josh: So we have Chef Ejaz here. I am teaching him how to cook. I am an expert chef.
Josh: And instead of telling him every ingredient and every step for every single
Josh: dish, what I tell him is like, hey, if you're making this amazing dinner recipe,
Josh: all you need that matters is this amount of salt applied at this time,
Josh: this amount of heat applied for this length of time, and the other stuff doesn't matter as much.
Josh: So just put in whatever you think is appropriate, but you'll get the same answer.
Josh: And that's what we see with this model is just an increased amount of efficiency by being
Josh: direct by being intentional about the data that they used to train it on,
Josh: the data that they used to fetch in order to give you high quality queries.
Josh: And it's a really novel breakthrough. They call it the MuonClip optimizer,
Josh: which, I mean, it's a Chinese company, maybe it means something special there,
Josh: but it is a new type of optimizer.
Josh: And what you're seeing in this curve is that it's working really well and it's
Josh: working really efficient.
Josh: And that's part of the benefit of having this open source is now we have this
Josh: novel breakthrough and we could take this and we could use this for even more
Josh: breakthroughs even more open source models and and that's part that's been really cool to see
Ejaaz: I i mean this is just um time
Ejaaz: and again from china uh so so amazing from their research team so so like just
Ejaaz: to kind of like um pick up your comment on deep seek at the end of last year
Ejaaz: we were utterly convinced that the only way to create a breakthrough model was
Ejaaz: to spend billions of dollars on compute clusters.
Ejaaz: And so therefore it was a pay-to-play game. And then DeepSeek,
Ejaaz: a team out of China, released their model and completely open-sourced it as well.
Ejaaz: And it was as good as OpenAI's Frontier model, which was the top model at the time.
Ejaaz: And the revelation there was, oh, you don't actually just need to chuck a bunch of compute at this.
Ejaaz: There are different techniques and different methods if you get creative about
Ejaaz: how you design your model and how you run the training cluster,
Ejaaz: the training one, which is basically what you need to do to make your model smart,
Ejaaz: you can run it in different ways that is more efficient, consumes less energy,
Ejaaz: and therefore less amount of money, but is as smart, if not smarter,
Ejaaz: than the frontier models that American AI companies are making.
Ejaaz: And this is just a repeat of that, Josh.
Ejaaz: I mean, look at this curve. For those who are looking at this episode on video.
Ejaaz: It is just so clean yeah it's beautiful
Ejaaz: the craziest part about this is when deep
Ejaaz: seek was released they pioneered something called uh reasoning
Ejaaz: or reinforcement learning uh which are two separate
Ejaaz: techniques that made the model super smart um with less energy and less compute
Ejaaz: spend um with this model they didn't even implement that technique at all so
Ejaaz: theoretically this model can get so much more smarter than it already is um
Ejaaz: and they just kind of leveraged a new method to make it as smart as it already is right now.
Ejaaz: So just such a fascinating kind of like progress in research from China.
Ejaaz: And it just keeps on coming out. It's so impressive.
Josh: Yeah, this is this was the exciting part to me is that we're seeing so many
Josh: algorithms or exponential improvements in so many different categories.
Josh: So this was considered a breakthrough by all means. And this wasn't even the
Josh: same type of breakthrough that DeepSeek had.
Josh: So we get this now compounding effect where we have this new training breakthrough
Josh: and then we have DeepSeek who has the reinforcement learning and that hasn't
Josh: even yet been applied to this new model.
Josh: So we get the exponential growth on one end, the exponential growth on the reasoning end,
Josh: those come together and then you get the exponential growth on the hardware
Josh: stack where the GPUs are getting much faster and there's all of these different
Josh: subsets of AI that are compounding on each other and growing and accelerating
Josh: quicker and quicker and what you get is this unbelievable rate of progress and
Josh: that's what we're seeing. So
Josh: reasoning isn't even here yet and we're going to see it soon because it is open
Josh: source so people can apply their own reasoning on top of it i'm sure the moonshot
Josh: team is going to be doing their own reasoning version of this model and i'm
Josh: sure we're going to be getting even more impressive results soon i see you have
Josh: a post up here um about the testing and overall performance can you please share yeah
Ejaaz: Yeah so um this is a tweet that summarizes really well how this model performs
Ejaaz: in relation to other Frontier models.
Ejaaz: And the popular comparison that's taken for Kimi K2 is against Claude.
Ejaaz: So Claude has a bunch of models out.
Ejaaz: Claude 3.5 is its earlier model, and then Claude 4 is its latest.
Ejaaz: And the general take is that this model is just better than those models,
Ejaaz: which is just insane to say, because for so long, Josh, we've said that Claude
Ejaaz: was the best coding model.
Ejaaz: And indeed it was. And then within the span of, what is it, five days?
Ejaaz: Grok 4 released and it just completely blew Claude 4 out of the water in terms of coding.
Ejaaz: Now Kimi K2, an open source model out of China who doesn't even have access
Ejaaz: to the research and kind of proprietary knowledge that a lot of American AI
Ejaaz: companies have also beat it as well, right?
Ejaaz: So it kind of beats Claude at its own game, but it's also cheaper.
Ejaaz: It's 20% the cost of Claude 3.5, which is just an insane thing to say,
Ejaaz: which means that if you are a developer out there that
Ejaaz: wants to try your hand at kind of like vibe coding
Ejaaz: a bunch of things or actually seriously coding something you
Ejaaz: know that's quite novel but you don't have the hands on deck to do that you
Ejaaz: can now spin up a Kimi K2 AI agent actually multiple of them for a very cost-efficient
Ejaaz: reasonable you know salary you don't have to pay like hundreds of thousands
Ejaaz: of dollars or you know hundreds of millions of dollars which is what Meta is
Ejaaz: doing to kind of buy a bunch of these software engineers,
Ejaaz: you can spend, you know, the equivalent of maybe a Netflix subscription or $500
Ejaaz: to $1,000 a month and spin up your own app. So super, super cool.
Josh: And also one added perk that's there is it's that even if you have a lot of
Josh: GPUs sitting around, you can actually run this model for free.
Josh: So that's the cost if you actually query it from the servers.
Josh: But I'm sure there's going to be companies that have access to XS GPUs.
Josh: They can actually just download the model because it's open source,
Josh: open weights, and they could run it on their own.
Josh: And that brings the cost of compute down to the cost per kilowatt of the energy
Josh: required to run the GPUs.
Josh: So because it's open source, you really start to see these costs decline,
Josh: but the quality doesn't.
Josh: And that's every time we see this, we see a huge productivity unlock in encoding
Josh: output and amount of queries used. It's like, this is freaking awesome.
Ejaaz: Yeah josh i saw something else come up as well so so do you remember when claude
Ejaaz: first released um their frontier model i think it was 3.5 or maybe it was four
Ejaaz: one of their bragging rights was it had a one million uh token context window which.
Josh: Oh yes which was huge
Ejaaz: Yeah which for listeners of the show is huge it's like several uh book novels
Ejaaz: worth um of words or characters you could just bung into one single prompt.
Ejaaz: And the reason why that was such an amazing thing was for a while,
Ejaaz: people struggled to kind of communicate with these AIs because they couldn't set the context.
Ejaaz: There wasn't enough bandwidth within their chat log window for them to say,
Ejaaz: you know, and don't forget this. And then there was this.
Ejaaz: And then, you know, this detail and that detail, there just wasn't enough space.
Ejaaz: And models weren't performing enough to kind of consume all of this in one go.
Ejaaz: And then Claude came out and was like, hey, we have one million context windows.
Ejaaz: Don't worry about it chuck in all the research papers that you want chuck in
Ejaaz: your essay chuck in reference books and we got you um i saw this tweet that
Ejaaz: was uh deleted i think you sent this to me um.
Josh: We got the screenshots we always come with receipts yeah i
Ejaaz: Wonder why they deleted it but uh good catch from you um yeah let's get into this.
Josh: What's your take on it was was first posted i think
Josh: earlier today yeah like an hour ago and then deleted pretty shortly afterwards
Josh: and this is from a woman name crystal crystal works with the moonshot team she
Josh: is part of the team that that released kimmy k2 um and in this post it says
Josh: kimmy isn't just another ai it went viral in china as the first to support
Josh: A 2 million token context window. And then she goes on to say,
Josh: we're an AI lab with just 200 people, which is ministerially small compared
Josh: to a lot of the other labs they're competing with.
Josh: And it was acknowledgement that they had a 2 million token context window.
Josh: And for those who, just a quick refresher on the context window stuff,
Josh: it's imagine you have like a gigantic textbook and you've read it once and you
Josh: close it and you kind of have a fuzzy memory of all the pages.
Josh: The context window allows you to lay all of those out in clear view
Josh: and directly reference every single page so when
Josh: you have two million tokens which is roughly two million words
Josh: of context we're talking about like hundreds and hundreds
Josh: of books and textbooks and knowledge and you could really dump a
Josh: lot of information in this for the ai to readily access and
Josh: that if they release that a two million token
Josh: open source model that's huge
Josh: deal i mean even grok 4 recently i believe
Josh: what did we say it was it was a 256 000 uh token context window something like
Josh: that so grok 4 is one eighth of what they supposedly have accessible right now
Josh: which is a really really big deal um so i'm hoping it was deleted because they
Josh: just don't want to share that not because it's not true i would like to believe
Josh: that it's true because man that'd be pretty epic yeah
Ejaaz: And the people are loving it josh um check out this
Ejaaz: graph from uh open router which basically shows
Ejaaz: uh the split of usage between everyone
Ejaaz: on their platform that are querying different models so for context
Ejaaz: here open router is a website that you can go to
Ejaaz: and you can type up a prompt just like you do at chat gpt and
Ejaaz: you can decide which model your
Ejaaz: prompt goes to or you could let open router decide for you
Ejaaz: and it kind of like divvies up your query so if you have a coding query it's
Ejaaz: probably going to send it to claude or now kimmy k2 or grok4 but if you have
Ejaaz: something that's more like to do with creative writing or something that's like
Ejaaz: a case study it might send it to OpenAI's O3 model, right? So it kind of like decides for you.
Ejaaz: OpenRacha released this graphic, which basically shows that KimiK2 surpassed
Ejaaz: XAI in token market share just a few days after launching, which basically means
Ejaaz: that XAI spent, you know,
Ejaaz: hundreds of billions of dollars training up their Grok4 model,
Ejaaz: which just kind of beat out the competition just last week.
Ejaaz: Then KimiK2 gets released completely open source
Ejaaz: and everyone starts to use that more than
Ejaaz: grok 4 which is just an insane thing to say and
Ejaaz: just shows how rapidly these ai models compete with each other and surpass each
Ejaaz: other um i think part of the reason for this josh is it's open source right
Ejaaz: which means that not only are retail users like myself and yourself using it
Ejaaz: for our daily queries you know uh you know,
Ejaaz: create this recipe for me or whatever, but researchers and builders all over
Ejaaz: the world that have so far been challenged or had this obstacle of pots of money
Ejaaz: basically to start their own AI company now have access to a frontier,
Ejaaz: world-renowned model and can create whatever application, website,
Ejaaz: or product that they want to make.
Ejaaz: So I think that's part of the usage there as well. Do you have any takes on this?
Josh: Yeah, and it's downstream of cost, right? We always see this when a model is
Josh: cheaper and mostly equivalent, the money will always flow to the cheaper model.
Josh: It'll always get more queries. I think it's important to note the different
Josh: use cases of these models. So they're not directly competing head to head on the same benchmarks.
Josh: I think what we see is like when we talk about Claude, it's generally known as the coding model.
Josh: And I don't think like OpenAI's O3 is not really competing directly with Claude
Josh: because it's more of a general intelligence versus a coding specific intelligence.
Josh: K2 is probably closer to a Claude. I would assume where it's really good at
Josh: coding because it uses this mixture of experts.
Josh: And I think that helps it find the tools. It uses this cool new novel thing
Josh: called like multiple tool use.
Josh: So each one of these experts can use a tool simultaneously and they could use
Josh: these tools and work together to get better answers.
Josh: So in the case of coding, this is a home run.
Josh: Like it is very cheap cost per token, very high quality outputs.
Ejaaz: I actually think you can compete with OpenAO3, Josh. Check this out.
Ejaaz: So Rowan, yeah, Rowan Cheng put this out yesterday And he basically goes,
Ejaaz: I think we're at the tipping point for AI-generated writing.
Ejaaz: It's been notoriously bad, but China's Kimi K2, an open-weight model,
Ejaaz: is now topping creative writing benchmarks.
Ejaaz: So just to put that into context, that's like having the top most, I don't know,
Ejaaz: smartest or slightly autistic software engineer, at the top engineering company
Ejaaz: working on AI models, also being the best poet or creative script and directing
Ejaaz: the next best movie or whatever that might be,
Ejaaz: or creating a Harry Potter novel series.
Ejaaz: This model can basically do both. And what it's pointing out here is that compared
Ejaaz: to 03, it tops it. Look at this. Completely beats it.
Josh: Okay, so I take that back. Maybe it is just better at everything.
Josh: Yeah, that's some pretty impressive results.
Ejaaz: I think like what's worth pointing out here is, and I don't know whether any
Ejaaz: of the American AI models do this, Josh, but mixture of experts seems to be clearly a win here.
Ejaaz: The ability to create an incredibly smart model doesn't come without,
Ejaaz: you know, this large storage load that is needed, right? One trillion parameters.
Ejaaz: But then combining it with the ability to be like, Like, hey,
Ejaaz: you don't need to query the entire thing.
Ejaaz: We've got you. We have a smart router, which basically pulls on the best experts,
Ejaaz: as you described earlier, for whatever relevant query you have.
Ejaaz: So if you have a creative writing task or if you have a coding thing,
Ejaaz: we'll send it to two different departments of this model.
Ejaaz: That's a really huge win. Do any other American models use this?
Josh: Well, the first thing that came to my mind when you said that is Grok4,
Josh: which doesn't exactly use this, but uses a similar thing, where instead of using
Josh: a mixture of experts, It uses a mixture of agents.
Josh: So Grok4 Heavy uses a bunch of distributed agents that are basically clones of the large model.
Josh: But that takes up a tremendous amount of compute. And that is the $300 a month plan.
Ejaaz: That's replicating Grok4 though, right? So that's like taking the model and copy pasting it.
Ejaaz: So let's say Grok4 was one trillion parameters just for ease of comparison.
Ejaaz: That's like creating, if there was four agents, that's four trillion parameters,
Ejaaz: right? So it's still pretty costly and inefficient.
Josh: Is that what you're saying no it's the actually the opposite direction of k2
Josh: so what they have used is just and again this is kind of similar to tracking
Josh: sentiment between the united states and china where the united states will throw
Josh: compute at it where china will throw like
Josh: kind of clever resource at it so grok yeah
Josh: when they use their mixture of agents it actually just costs a lot more
Josh: money whereas k2 when they use their mixture of
Josh: experts well it costs a lot less instead of using 4 trillion
Josh: parameters in this case it uses just 32 billion and it
Josh: kind of copies that 32 billion over and over and it's really it's a really
Josh: elegant solution that seems to be
Josh: yielding pretty comparable results so i think as we
Josh: see these efficiency upgrades i'm sure they will
Josh: eventually trickle down into the united states models and when they do that
Josh: is going to be a huge unlock in terms of cost per token in terms of the smaller
Josh: distilled models that we're going to be able to run on our own computers um
Josh: but yeah i don't know of any who are also using it at this scale it might be
Josh: novel just to k2 right now and
Ejaaz: And i think that this is the method that probably scales the best josh like.
Josh: Yeah it makes sense efficiency
Ejaaz: Always wins at the end right and to see um this kind of innovation come pretty
Ejaaz: early on in a technology's life cycle is just super impressive to see,
Ejaaz: Another thing I saw is there's two different versions of this model, I believe.
Ejaaz: There's something called Kimi K2 Base, which is basically the model for researchers
Ejaaz: who want full control for fine-tuning and custom solutions, right?
Ejaaz: So imagine this model as the entire parameter set. So you have access to one
Ejaaz: trillion parameters, all the weight designs and everything.
Ejaaz: And if you're a nerd that wants to nerd out you can
Ejaaz: go crazy you know if you have like your own gpu
Ejaaz: cluster at home or if you happen to have a convenient
Ejaaz: warehouse full of of servers that you weirdly
Ejaaz: have access to you can go crazy with it you can if you
Ejaaz: think about like um the early gaming days of counter-strike and then you could
Ejaaz: like mod it you can basically mod this uh model to your heart's desire and then
Ejaaz: there's a second version called k2 instruct which is for drop-in general purpose
Ejaaz: chat and AI agent experiences.
Ejaaz: So this is kind of like at the consumer level, if you're experimenting with
Ejaaz: these things, or if you want to run an experiment at home on a specific use
Ejaaz: case, you can kind of like take that away and do that for yourself.
Ejaaz: That's how I understand it, Josh. Do you have any takes on this?
Josh: That makes sense. And I think that second version that you're describing is
Josh: what's actually available publicly on their website, right?
Josh: So if you go to Kimmy.com, it has a text box. It looks just like ChatGPT like you're used to.
Josh: And that's where you can run that second tier model which
Josh: um you described as that's the the drop in general purpose
Josh: chat and then yeah for the the hardcore researchers there's
Josh: a github repo and the github repo has all the weights and all the code and
Josh: you can really download it dive in use the full thing i
Josh: was playing around with the kimmy tool and it's it's really cool
Josh: it's fast oh i mean it's lightning fast if you
Josh: go from a reasoning model to an inference model like kimmy
Josh: you get responses like this like when
Josh: i'm using grok 4 or o3 i'm sitting there sometimes for a couple minutes it's
Josh: waiting for an answer this you type it in and it just types back right away
Josh: no time waiting so it's it's kind of refreshing to see that but it's also a
Josh: testament to how impressive it is i'm getting great answers and it's just spitting
Josh: it right out so what happens when they add the reasoning layer on top well it's
Josh: probably going to get pretty freaking good
Ejaaz: So the trend we're seeing, and we saw this last week with Grok4,
Ejaaz: is typically we're expected to wait a while when we send a prompt to a breakthrough
Ejaaz: model because it's thinking, it's trying to basically replicate what we have in our brains up here.
Ejaaz: And now it's just getting much quicker and much smarter and much cheaper.
Ejaaz: So the long story short is these incredibly powerful, I kind of think about
Ejaaz: it as how we went from massive desktop computers to slick cell phones,
Ejaaz: Josh, and then we're going to eventually have chips in our brain.
Ejaaz: AI is just kind of like fast tracking that entire life cycle within like a couple
Ejaaz: of years, which is just insane.
Josh: And these efficiency improvements are really exciting because you can see how
Josh: quickly they're shrinking and allowing eventually for those incredible models
Josh: to just run on our phones.
Josh: So there's totally a world a year from now in which like a
Josh: grok 403 kimmy k2 capable model
Josh: is small enough that it could just run inside of in our
Josh: phone and run on a mobile device or run locally on a laptop
Josh: or you're offline and you kind of have this portable intelligence
Josh: that's available everywhere anytime even if
Josh: you're not connected to the world and that seems really cool
Josh: like we were talking a few episodes ago about apple's um local
Josh: free ai inference running on an iphone
Josh: but how the base models still kind of suck like they don't really do
Josh: anything super interesting they're basically good enough to do what
Josh: you would expect siri to do but can't do and these
Josh: models as we get more and more breakthroughs like this that allow you to
Josh: run much larger parameter counts
Josh: on a much smaller device it's going to start really
Josh: super powering these mobile devices and i can't help but think about the open
Josh: ai hardware device i'm like wow that'd be super cool if you had like oh three
Josh: running locally in the middle of the jungle somewhere with no service and you
Josh: still had access to all of its capabilities like that's probably coming downstream
Josh: of breakthroughs like this where we get really big efficiency unlocks
Ejaaz: I mean, it's not just efficiency, though, right? It's the fact that if you can
Ejaaz: run it locally on your device, it can have access to all your private data without
Ejaaz: exposing all of that to the model providers themselves, right?
Ejaaz: So one of the major concerns of not just AI models, but also with mobile phones is privacy.
Ejaaz: I don't want to share all my kind of like private health, financial,
Ejaaz: and social media data, because then you're just going to have everything on
Ejaaz: me and you're going to use me.
Ejaaz: You're going to use me as a product, right? And that's kind of like been the
Ejaaz: quota for the last decade in tech.
Ejaaz: And so with AI, that's a supercharged version of it. The information gets more
Ejaaz: personal. It's not just your likes.
Ejaaz: It's, you know, where Josh shops every day and, you know, who he's dating and
Ejaaz: all these kinds of things, right?
Ejaaz: And that becomes quite personal and intrusive very quickly.
Ejaaz: So the question then becomes, how can we have the magic of an AI model without it being so obtrusive?
Ejaaz: And that is open source locally run AI or privately run AI. and Kimi K2 is a
Ejaaz: frontier model that can technically run on your local device if you set up the right hardware for it.
Ejaaz: And the way that we're trending, you can basically end up having that on your
Ejaaz: device, which is just a huge unlock.
Ejaaz: And if you can imagine how you use OpenAI 03 right now, Josh,
Ejaaz: right? I know you use it as much as I do.
Ejaaz: The reason why you and I use it so much isn't just because it's so smart,
Ejaaz: but it's because it remembers everything about us.
Ejaaz: But I hate that Sam knows or has access to all that data.
Ejaaz: I hate that if he chooses to switch on personalized ads, which is currently
Ejaaz: the model where most of these tech companies make money right now,
Ejaaz: he can, and I've got nothing to do about it because I don't want to use any
Ejaaz: other model apart from that.
Ejaaz: But if there was a locally run
Ejaaz: model that had access to all the memory and context, I'd use that instead.
Josh: And this is suspicious. I mean, this is a different conversation in total,
Josh: but isn't it interesting how other companies haven't really leaned into memory
Josh: when it's seemingly the most important mode that there is.
Josh: Like Grok4 doesn't have good memory rolled out. Gemini doesn't really have memory.
Josh: There's no, Claude doesn't have memory the way that OpenAI does.
Josh: Yet it's the single biggest reason why we both continue to go back to ChatGPT and OpenAI.
Josh: So that's just been an interesting thing. I mean, Kimmy is open source.
Josh: I wouldn't expect them to lean too much into it. But for these closed source
Josh: models, that's just, it's another interesting just observation.
Josh: Like, hey, the most important thing isn't, doesn't seem to be prioritized by
Josh: other companies just yet.
Ejaaz: Why do you think that is so so my theory um at least from xai or grok force
Ejaaz: perspective is elon's like okay i'm not going to be able to build a better chat
Ejaaz: bot or chat messenger than openai has there's not too many features i can um.
Ejaaz: Set Grok 4 apart, then that O3 doesn't already do, right?
Ejaaz: But where I can beat O3 is at the app layer.
Ejaaz: I can create a better app store than they have because I haven't really created
Ejaaz: one that is sticky enough for users to continually use.
Ejaaz: And I can use that data set to then unlock memory and context at that point, right?
Ejaaz: So I just saw today that they released, they
Ejaaz: being um xai released a new feature for grok 4
Ejaaz: called i think it's uh companions josh um
Ejaaz: and it's basically these yeah these animated um
Ejaaz: avatar like um characters so they basically look like they're from an anime
Ejaaz: show and you know how you can use voice mode in open ai and you can kind of
Ejaaz: like talk to this uh realistic human sounding ai you now have a face and a character
Ejaaz: on grok 4 and it's really entertaining, Josh.
Ejaaz: Like I find myself kind of like engaged in this thing because I'm not just typing words.
Ejaaz: It's not just this binary to and fro with this chat messenger.
Ejaaz: It's this human, this cute, attractive human that I'm just like now speaking to.
Ejaaz: And I think that that's the strategy that a lot of these AI companies,
Ejaaz: if I had to guess, are taking to kind of like seed their user base before they
Ejaaz: unlock memory. I don't know whether you have a take on that.
Josh: Yeah, I have a fun little demo. I actually played around with it this morning
Josh: and I was using it totally unhinged, no filter, very vulgar,
Josh: but like kind of fun. It's like a fun little party trick.
Josh: And yeah, I mean, that was a surprise to me this morning when I saw that rolled
Josh: out. I was like, huh, that doesn't really seem like it makes sense.
Josh: But I think they're just having fun with it.
Ejaaz: Can we for a second talk about the team?
Ejaaz: So we've mentioned just now how they've all come from China and how China's
Ejaaz: like really advancing open source AI models, and they've completely beat out
Ejaaz: the competition in America, Mata's Lama being the obvious one.
Ejaaz: We've got Kwen from Alibaba.
Ejaaz: We've got Deep Seek R1. Now we have Kimi K2. The team is basically...
Ejaaz: The AI Avengers of China, Josh. So these three co-founders all have deep AI
Ejaaz: ML backgrounds that hail from the top American universities,
Ejaaz: such as Carnegie Mellon.
Ejaaz: One of them has a PhD from Carnegie Mellon in machine learning,
Ejaaz: which is basically, for those of you who don't know, is like God-tier degree for AI.
Ejaaz: That means you're desirable and hireable by every other AI company after you graduate.
Ejaaz: But it's not just that. They also have credibility and degrees from the top universities in China.
Ejaaz: Especially this one university called Tsinghua, which seemed to be the top of their field.
Ejaaz: I looked them up on rankings for AI universities globally, and they often come
Ejaaz: in number three or four in the top 10 AI universities. So pretty impressive from there.
Ejaaz: But what I found really interesting, Josh, was one of the co-founders was an
Ejaaz: expert in training AI models on low-cost optimized hardware.
Ejaaz: And the reason why I mentioned this is it's no secret that if you want a top
Ejaaz: frontier AI model, you need to train it on NVIDIA's GPUs.
Ejaaz: You need to train it on NVIDIA's hardware.
Ejaaz: NVIDIA's market cap, I think, at the end of last week, surpassed $4 trillion.
Ejaaz: That's $4 trillion with a T. That is more than the current GDP of the entire British economy.
Josh: Where I hail from. And the largest in the world.
Ejaaz: And there's never been.
Josh: A bigger company
Ejaaz: There's never been a bigger company it it's just
Ejaaz: insane to grab your head around and it's not without
Ejaaz: reason they supply basically or they have a
Ejaaz: grasp or a monopoly on the hardware that
Ejaaz: is needed to train top models now kimmy k2
Ejaaz: comes along casually drops a one trillion parameter model one of the largest
Ejaaz: models ever released um and it's trained on hardware that isn't nvidia's um
Ejaaz: and jensen huang i i need to find this clip josh but But Jensen Huang basically
Ejaaz: was on stage, I think it was at a private conference maybe yesterday,
Ejaaz: but he was quoted as saying 50% of the top AI researchers are Chinese and are from China.
Ejaaz: And what he was implicitly getting at is they're a real threat now.
Ejaaz: I think for the last decade, we've kind of been like, ah, yeah,
Ejaaz: China's just going to copy paste everything that comes out of America's tech sector.
Ejaaz: But when it comes to AI, we've kind of like maintained the same mindset up until
Ejaaz: now where they're really just competing with us.
Ejaaz: And if they have the hardware, they have the ability to research new techniques
Ejaaz: to train these models, like DeepSeek's reinforcement learning and reasoning,
Ejaaz: and then Kimi K2's kind of like efficient training run, which you showed earlier.
Ejaaz: They've come to play, Josh. And I think it's worth highlighting that China has
Ejaaz: a very strong grasp on top AI researchers in the world and models that are coming out of it.
Josh: Where are their $100 million offers? I haven't seen any of those coming through.
Josh: None, dude. The most impressive thing is that they do it without the resources that we have.
Josh: Imagine if they did have access to the clusters of these like H100s that NVIDIA is making.
Josh: I mean, that would be, would they crush us?
Josh: And we kind of have this timeline here where we're kind of running up against
Josh: the edge of energy that we have available to us to train these massive models.
Josh: Whereas China does not have that constraint. They have significantly more energy to power these.
Josh: So in the event, the inevitable event that they do get the chips and they are
Josh: able to train at the scale that we are, I'm not sure we're able to continue
Josh: our rate of acceleration in terms of hardware manufacturing,
Josh: large training as fast as they will.
Josh: And they already have done the hard work on the software efficiency side.
Josh: They've cranked out every single efficiency because they are doing it on constrained hardware.
Josh: So it's going to create this really interesting effect where they're coming
Josh: at it from the like ingenuity software approach we're coming at it from the
Josh: brute force throw a lot of compute added approach and we'll see where both both
Josh: sides end up um but it's clear that china is still behind because they are the
Josh: ones open sourcing the models and we know at this point now if you're open sourcing
Josh: your model you're doing it because you're behind
Ejaaz: Yeah yeah i mean one thing
Ejaaz: that did surprise me josh was that they released a one
Ejaaz: trillion parameter open source model i i didn't
Ejaaz: expect them to catch up that quickly um like one
Ejaaz: trillion is a lot um yeah another thing
Ejaaz: i was thinking about is china has dominated
Ejaaz: hardware for so long now so it wouldn't
Ejaaz: really surprise me if like i don't know a
Ejaaz: couple years from now they're producing better models
Ejaaz: at specific things basically because they have better
Ejaaz: hardware than america than the west um but
Ejaaz: where i think the west will continue to dominate
Ejaaz: is at the application layer and i don't
Ejaaz: know if i was a betting man i would say that most of the money is eventually going
Ejaaz: to be made on the application side of things i think grok
Ejaaz: 4 is starting to um kind of show that
Ejaaz: with all these different kinds of novel features that they're releasing i i
Ejaaz: don't know if you've seen some of the games that are being produced from grok
Ejaaz: 4 josh but it is ultimately insane and i haven't seen any similar examples come
Ejaaz: out of uh asia from any of their ai models even when they have access to american
Ejaaz: models So I still think America dominates at the app layer.
Ejaaz: But Josh, I just came across this tweet, which you reminded me of earlier.
Ejaaz: Tell me about OpenAI's strategy to open source model, because I got this tweet
Ejaaz: pulled up from Sam Altman, which is kind of hilarious.
Josh: Yeah. All right. So this week, if you remember from our episode last week,
Josh: we were excited about talking about OpenAI's new open source model.
Josh: OpenAI, open source model, all checks out. This was going to be the big week.
Josh: They released their new flagship open source. Well, conveniently,
Josh: I think the same day as K2 launched, later in the day, or perhaps the very next morning.
Josh: Sam Altman posted a tweet. He says, Hey, we plan to launch our open weights model next week.
Josh: We are delaying it. We need time to run additional safety tests and review high-risk
Josh: areas. We are not yet sure how long it will take us.
Josh: While we trust the community will build great things with this model,
Josh: once weights are out, they can't be pulled back. This is new for us and we want to get it right.
Josh: Sorry to be the bearer of bad news. We are working super hard.
Josh: So there's a few points of speculation. The first, obviously,
Josh: being, did you just get your ass handed to you and now you are going back to
Josh: reevaluate before you push out a remodel?
Josh: So that's one possible thing where they saw K2. They were like,
Josh: oh, boy, this is pretty sweet.
Josh: This is our first open source model. We probably don't want to be lower than them.
Josh: And there is this second point of speculation, which, Ejaz, you mentioned to
Josh: me a little earlier today, where maybe something went wrong with the training one.
Josh: And it's not quite that they're getting beat up by a Chinese company.
Josh: Is that like they actually made a mistake on their own accord and can you explain
Josh: to me specifically what that might be what the speculation is at least yeah
Ejaaz: Well i'll keep it short i think it was a little racist under
Ejaaz: the hood and i i can't find the tweet but basically
Ejaaz: one of these um ai researchers slash
Ejaaz: product builders on x got access to
Ejaaz: the model supposedly according to him and he tested it
Ejaaz: out uh in the background and he said yeah it's it's
Ejaaz: not really an intelligence thing it's just worse than
Ejaaz: what uh you'd expect from an alignment and uh consumer facing approach it was
Ejaaz: it was ill-mannered it was saying some pretty wild shit kind of the stuff that
Ejaaz: you'd expect coming out of 4chan um and so sam altman decided to delay whilst
Ejaaz: they kind of like figured out why um it was kind of acting out.
Josh: Got it okay so we'll leave
Josh: that speculation where it is there's a there's a funny post
Josh: that i'll actually share with you if you want to throw it up which was actually from elon
Josh: and we'll abbreviate but it was like elon was basically saying um
Josh: it's hard to avoid the the libtard slash
Josh: mecha hitler like approach both of them
Josh: because they're on so polar opposite ends of the spectrum and he said he spent
Josh: several hours trying to solve this problem with the system prompt but there's
Josh: too much garbage coming in at the foundation model level so basically i mean
Josh: what happens with these models is you train them based on all the human knowledge
Josh: that exists right so everything that we've believed all the ideas that we've
Josh: shared it's been fed into these models.
Josh: And what happens is you can try to adjust how they interpret this data through
Josh: the system prompt, which is basically an instruction that every single query
Josh: gets passed through, but at some point is reliant on this swath of human data that is just
Josh: It's too overbearing. And that's kind of what Elon shared.
Josh: And the difference between OpenAI and Grok is that Grok will just ship the crazy
Josh: update. And that's what they did. And they caught a lot of backlash from it.
Josh: But what I find interesting and what I'm sure OpenAI will probably follow is
Josh: this last paragraph where he says, our V7 foundation model should be much better.
Josh: And we're being far more selective about training data rather than just training on the entire internet.
Josh: So what they're planning to do to solve this problem, which is what I assume
Josh: OpenAI probably ran into in the case that the AI training model kind of went
Josh: off the rails and it started saying bad things about lots of people is that
Josh: you kind of have to rebuild the foundation model with new sets of data.
Josh: And in the case of Grok, I know one of the intentions for v7 is actually to
Josh: generate its own database of data based on synthetic data from their models.
Josh: And I'm assuming OpenAO will probably have to do this too if they want to calibrate.
Josh: A lot of times people call that the temperature, which is the like variance
Josh: of aggression in which a model uses.
Josh: And I don't know, I think we're gonna start to see interesting approaches from
Josh: that because as they get smarter, you really don't want them to necessarily
Josh: have these evil traits as the default.
Josh: And it's very hard to get around that when you train them on the data that they've been trained on so far.
Ejaaz: It just goes to show how, I guess, cumbersome it is to train these models,
Ejaaz: Josh. It's such a hard thing.
Josh: Yeah. Yeah.
Ejaaz: It's not something that you can just kind of like jump into the code and tweak a few things.
Ejaaz: Most of the time you don't know what's wrong with the model or where it went
Ejaaz: wrong. I mean, we've talked about this on a previous episode, but
Ejaaz: So essentially, if you build out this model, right, you spend hundreds of millions
Ejaaz: of dollars, and then you feed it a query.
Ejaaz: So you put something in and then you wait to see what it spits out.
Ejaaz: You don't really know what it's going to spit out. You can't predict it.
Ejaaz: It's completely probabilistic. and so if you
Ejaaz: release a model and it starts being a little racist or uh
Ejaaz: you know um kind of crazy uh you
Ejaaz: have to kind of like go back to the drawing board and you have
Ejaaz: to analyze many different sectors of of this model
Ejaaz: like was it the data that was poisoned or was it the way that we trained it
Ejaaz: or maybe it was a particular model weight that we tweaked too much or whatever
Ejaaz: that might be so i i think over time it's going to get a lot easier once we
Ejaaz: understand how these models actually work but my god it must be so expensive
Ejaaz: to just continually rerun and retrain these models.
Josh: Yeah when you think about a coherent cluster of 200
Josh: 000 gpus the amount of energy the amount
Josh: of resources just to to retrain a mistake is is huge so i think i mean the more
Josh: we go into it the deeper we get the more it kind of makes sense paying so much
Josh: money for talent to avoid these mistakes where if you pay a hundred million
Josh: dollars for one employee who will give you a strategic advantage to avoid having
Josh: to do another training run, that will cost you more than $100 million.
Josh: You've already, you're already in the profit. So you kind of start to see the
Josh: scale, the complexity, the difficulties.
Josh: I do not envy the challenges that some of these engineers have to face.
Josh: Although I do envy the- I envy the salary.
Ejaaz: I envy the salary, Josh.
Josh: I envy the salary and I envy the adventure. Like how cool must that be trying
Josh: to build super intelligence for the world as a human for the first time in like
Josh: the history of everything.
Josh: So it's gotta be pretty fun. This is where we're at now with the open source
Josh: models closed source models k2's pretty epic i think that's a home run i think
Josh: we've crowned a new model today um do you have any closing thoughts anything
Josh: you want to add before we wrap up here this is pretty amazing i
Ejaaz: Think i'm most excited uh for the episode that we're probably going to release
Ejaaz: a week from now josh when we've seen what people have built with this open source
Ejaaz: model that's the best part about this by the way just to remind the listener that,
Ejaaz: anyone can take this model right now you if you're listening to this can take
Ejaaz: this model right now run it locally at home and tweak it to your preference
Ejaaz: now yes it's going to be you know you kind of need to know how to tweak model
Ejaaz: weights and stuff but i think we're going to see some really cool applications
Ejaaz: get released over the next week and i'm excited to play around with them personally.
Josh: Yeah if you're listening to this um and you can
Josh: run this model let us know because that means you have quite a solid uh
Josh: rig at your home yeah i'm not sure the average person is
Josh: going to be able to run this but that is the beauty of the open weights is that anybody
Josh: with the capability of running this can do so they
Josh: could tweak it how they like and now they have access to the new
Josh: best open source model in the world which i mean just a
Josh: couple months ago from now would have been the best model in the
Josh: world so it's moving really quickly it's really accessible and
Josh: i'm sure as the weeks go by i mean hopefully we'll get open ai's model open
Josh: source model soon in the next few weeks we'll be able to cover that but until
Josh: then just lots of stuff going on this was uh another great episode so thank
Josh: you everyone for tuning in again for rocking with us We actually plan on making this like 20 minutes,
Josh: but we just kind of kept tailing off into more interesting things.
Josh: There's a lot of interesting stuff to talk about. I mean, there's really,
Josh: you could take this in a lot of places.
Josh: So hopefully this was interesting.
Josh: Go check out Kimmy K2. It's really, really impressive. It's really fast.
Josh: It's really cheap. If you're a developer, give it a try.
Josh: And yeah, that's been another episode. We'll be back again later this week with
Josh: another topic. and just keep on chugging along as the frontier of AMI models continues to head west.
Ejaaz: So also we'd love to hear from you guys. So if you have any suggestions on things
Ejaaz: that you want us to talk more about, or maybe there's like some weird model
Ejaaz: or feature that you just don't understand and maybe we can do a job at explaining it, just message us.
Ejaaz: Our DMs are open or respond to any of our tweets and we'll be happy to oblige.
Josh: Yeah, let us know. If there's anything cool that we're missing,
Josh: send it our way and we'll cover it. That'd be great.
Josh: But yeah, we're all going on the journeys together. We're learning this as we go.
Josh: So hopefully today was interesting. And if you did enjoy it,
Josh: please share with friends, likes, comment, subscribe, all the great things.
Josh: And we will see you on the next episode.
Ejaaz: Thanks for watching. See you guys. See you.