Josh: The unthinkable has just happened open ai
Josh: has released an open source model open ai
Josh: has been closed ai since the time that i knew them
Josh: they have been named themselves open ai they were not
Josh: open source they have finally released an open source model and surprise surprise
Josh: it's actually really great and i think the downstream implications of an open
Josh: source model from a company like this that is this good are really it's a really
Josh: big deal i think this really matters a lot just yesterday they announced the release of GPT-OSS.
Josh: There are two models. There's a 120 billion parameter model and there's a 20
Josh: billion parameter model. We're going to get into benchmarks.
Josh: We're going to get into how good they are.
Josh: But the idea is that OpenAI has actually released an open source model.
Josh: And this can compare to the Chinese models because we recently had DeepSeek and we've had Kimi.
Josh: And those would be very good. But this is the first really solid American-based open source model.
Josh: So Ijaz, I know you've been kind of digging in the weeds about how this works.
Josh: Can you explain us exactly why this is a big deal why this happened what's going on here
Ejaaz: Yeah it's it's pretty huge so so here
Ejaaz: are the hot highlights um as you mentioned there's two
Ejaaz: models that came out the 20 billion parameter model which is actually small
Ejaaz: enough to run on your mobile phone right now and they have a 120 billion parameter
Ejaaz: model which is big but still small enough to run on a high performance laptop
Ejaaz: so if you guys have a macbook out there jump in go for it um it's fully customizable.
Ejaaz: So remember, open source means
Ejaaz: that you can literally have access to the design of the entire model.
Ejaaz: It's like OpenAI giving away their secret recipe to how their frontier models
Ejaaz: work. And you can kind of like recreate it at home.
Ejaaz: This means that you can customize it to any kind of use case that you want,
Ejaaz: give it access to all your personal hard drives, tools, data,
Ejaaz: and it can do wonderful stuff.
Ejaaz: But Josh, here's the amazing part.
Ejaaz: On paper, these models are as good as GPT-4 mini models, which is,
Ejaaz: it's pretty impressive, right?
Ejaaz: In practice and i've been playing around with it for the last few hours they're
Ejaaz: as good in my opinion and actually quicker than
Ejaaz: gpt-03 which is their frontier model and i
Ejaaz: mean this across like everything so
Ejaaz: reasoning um it spits out answers super quickly and i can see its reasoning
Ejaaz: it happens in like a couple of seconds and i'm so used to waiting like 30 seconds
Ejaaz: to a couple minutes on gpt-03 josh so it's pretty impressive and an insane unlock
Ejaaz: on coding it's as good and on creativity as well.
Ejaaz: So I'm my mind's pretty blown at all of this, right? Josh, what do you what do you think?
Josh: Yeah, so here's why it's impressive to me is because a lot of the times I don't
Josh: really care to use the outer bands of what a model is capable of.
Josh: Like I am not doing deep PhD level research. I'm not solving these math Olympiad questions.
Josh: I'm just trying to ask it a few normal questions and get some answers.
Josh: And what these models do is an excellent job at serving that need.
Josh: They're not going to go out and solve the world's hardest problems,
Josh: but neither do I. I don't want to solve those problems.
Josh: I just kind of want the information that I want, whether it be just a normal
Josh: Google type search or whether it be asking it some miscellaneous question about
Josh: some work that I'm doing.
Josh: It's really good at answering that. So I think initial impressions,
Josh: because they did allow you to test it publicly through their website,
Josh: it's just really good at the things that I want.
Josh: So the fact that I can run one of these models on a local device on my iPhone,
Josh: well, it feels like we're reaching this place that AI is starting to become
Josh: really interesting because for so long we've had compute handled fully on the
Josh: cloud and now this is the first time where
Josh: Compute can really happen on your computer. It could happen on your laptop.
Josh: I could download the model and I could actually store the model,
Josh: the 120 billion parameter model on a 56 gigabyte USB drive.
Josh: So you can take the collective knowledge of the world and put it on a tiny little USB drive.
Josh: And granted, it needs a bit of a bigger machine to actually run those parameters,
Josh: but you can install all the weights. It's 56 gigabytes.
Josh: It's this incredibly powerful package. And it probably, I don't know if this
Josh: is true, but it's probably the most condensed knowledge base in the history of humanity.
Josh: They've really managed to take a tremendous amount of tokens,
Josh: smush them into this little parameter set, and then publish it for people to
Josh: use. So for me, I'm really excited.
Josh: I like having my own mini portable models. I am excited to download,
Josh: try it out, run it on my MacBook.
Josh: I'm not sure I could run the 120 billion parameter model, but at least the 20B
Josh: and give it a shot and see how it works.
Ejaaz: You need to get the latest MacBook, Josh. I know, I got to upgrade. We can test that out.
Ejaaz: What I also love about it is it's fully private, right? So you can give it access
Ejaaz: to your personal hard drive, your Apple Notes, whatever you store on your computer, basically.
Ejaaz: And you can basically instruct the model to use those different tools.
Ejaaz: So one review that I keep seeing from a number of people who have been testing
Ejaaz: it so far is that it's incredibly great and intuitive at tool use.
Ejaaz: And the reason why this is such a big deal is a lot of the Frontier models right
Ejaaz: now, when they allow you to give access to different tools, they're kind of clunky.
Ejaaz: The model doesn't actually know when to use a specific tool and when not to.
Ejaaz: But these models are super intuitive, which is great. The privacy thing is also
Ejaaz: a big thing because you kind of
Ejaaz: don't want to be giving all your personal information away to Sam Altman.
Ejaaz: But you want a highly personalized model.
Ejaaz: And I think if I was to condense this entire model release in a single sentence,
Ejaaz: Joss, I think I would say it is the epitome of privacy and personalization in an AI model so far.
Ejaaz: It is that good. it is swift it is cheap and I'm going to replace it completely
Ejaaz: with all my GPT-4.0 queries as you said earlier like,
Ejaaz: Who needs to use the basic models anymore when you have access to this?
Josh: Yeah. So it's funny you say that you're going to swap it because I don't think I'm going to swap it.
Josh: I still am not sure I personally have a use case right now because I love the
Josh: context. I want the memory.
Josh: I like having it all server side where it kind of knows everything about me.
Josh: I guess in the case that I wanted to really make it a more intimate model experience
Josh: where you want to sync it up with like journal entries or your camera roll or
Josh: whatever, whatever interesting like personal things, this would be a really cool use case.
Josh: I think for the people who are curious why this matters to them,
Josh: well, we could talk a little briefly about like the second order effects of
Josh: having open source models as powerful, because what that allows you to do is
Josh: to serve queries from a local machine.
Josh: So if you are using an app or let's say you're an app developer and you're building
Josh: an application and your app is serving millions of requests because it's a GPT wrapper.
Josh: Well, what you could do now is instead of paying API calls to the OpenAI server,
Josh: you can actually just run your own local server, use this model,
Josh: and then serve all that data for the cost of the electricity.
Josh: And that's a really big unlock for the amount of compute that's going to be
Josh: available for not only developers, but for the cost of the users in a lot of these applications.
Josh: So for the applications that aren't doing this crazy moon math and that are
Josh: just kind of serving basic queries all day long, this like really significantly drops the cost.
Josh: It increases the privacy, like you mentioned. And there's a ton of really important
Josh: upsides to open source models that we just haven't seen up until now.
Josh: And I'm very excited to see come forward.
Ejaaz: Well, Josh, the thing with most of these open source models,
Ejaaz: we spoke about actually two major Chinese open source models that were released last week.
Ejaaz: It's not accessible to everyone. Like you and me aren't necessarily going to
Ejaaz: go to Hugging Face, a completely separate website, download these models,
Ejaaz: run the command line interface.
Ejaaz: Most of the listeners on the show doesn't even know what that means.
Ejaaz: I don't even know if I know what that means, right?
Ejaaz: But here you have a lovely created website where you could just kind of log
Ejaaz: on and play around with these open source models. And that's exactly what I've been doing.
Ejaaz: I actually have a few kind of demo queries that I ran yesterday, Josh.
Josh: Yeah, walk us through, let's see.
Ejaaz: Okay, so there's an incredibly complex test, which a lot of these AI models,
Ejaaz: which cost hundreds of billions of dollars to train, can't quite answer.
Ejaaz: And that is how many R's, the letter R's are there in the word strawberry? Most say two.
Josh: The bar's on the floor,
Ejaaz: Huh? Yeah, if we were to go with most models, they say two. They're convinced that they are only two.
Ejaaz: And I ran that test today, rather yesterday, with these open source models,
Ejaaz: and it correctly guessed three, Josh. So we're one for one right now.
Josh: We're on our way.
Ejaaz: But then I was like, okay, we live in New York City. I love this place.
Ejaaz: I'm feeling a little poetic today. Can you write me a sonnet?
Ejaaz: And my goal with this wasn't to test whether it could just write a poem.
Ejaaz: It was to test how quickly it could figure it out.
Ejaaz: And as you see it thought for a couple of seconds on this so it literally spat
Ejaaz: this out in two seconds um and it was structured really well you know it kind
Ejaaz: of flowed would i be you know reciting this out loud to the public no but you
Ejaaz: know i was pretty impressed.
Ejaaz: And then, Josh, I was thinking, you know, what's so unique about open source models?
Ejaaz: You just went through a really good list of why open source models work.
Ejaaz: But I was curious as to why these specific open source models were better than
Ejaaz: other open source models or maybe even other centralized models.
Ejaaz: So I wrote a query. I decided to ask it. I was like, you know,
Ejaaz: tell me some things that you could do that are the larger centralized models.
Ejaaz: And I spat out a really good list. I'm not going to go through all of them,
Ejaaz: but, you know, some of the things that we've highlighted so far, you can fine tune it.
Ejaaz: It's privacy. See, I really like this point that it made, Josh,
Ejaaz: that it just shows that AI is probably getting smarter than us,
Ejaaz: which is you can custom inject your own data into these models.
Ejaaz: Now, without kind of digging deeper into this, when you use a centralized model,
Ejaaz: it's already pre-trained on a bunch of data that companies like Anthropic and
Ejaaz: Google have already fed it.
Ejaaz: And so it's kind of formed its own personality, right?
Ejaaz: So you can't change the model's personality on a centralized model.
Ejaaz: But with an open model you have full reign to do whatever you want and so if
Ejaaz: you were feeling kind of uh adventurous you could use your own data and make
Ejaaz: it super personal and customizable so i thought that was really cool and fun
Ejaaz: demo josh have you been playing around with this.
Josh: Yeah it's um it's it's smart it's fun it's smart i wouldn't say it's anything
Josh: novel the like query results that i get are you know on par with everything
Josh: else i don't notice the difference which is good because it means they're performing
Josh: very well it's not like i feel like i'm getting degraded performance because
Josh: I'm using a smaller model.
Josh: But it's just like it's nothing too different, I would say.
Josh: The differences, I mean, again, all this boils down to the differences of it
Josh: being open source versus being
Ejaaz: Run on the server. Well, let me challenge you that, right? OK,
Ejaaz: so you're saying it's good but nothing novel.
Ejaaz: Would you say it's as good as GPT-4.0,
Ejaaz: minus the memory let's just put memory aside for a second would you use it if
Ejaaz: it had memory capability.
Josh: Actually no probably not um i still wouldn't
Josh: because i love my desktop application too much i
Josh: love my mobile app too much and i like that the conversations are
Josh: shared in the cloud um so i can use them on my phone i could
Josh: start on my laptop and go back and forth so even in
Josh: that case i'm probably still not a user um because
Josh: the convenience factor but there are there are a
Josh: lot of people and a lot of industries that would be and this is actually something probably
Josh: worth surfacing is the new industries that are now able to
Josh: benefit from this because a lot of industries have
Josh: a tough time using these AI models because
Josh: of the data privacy concerns particularly I mean if you think about a
Josh: healthcare industry people who are dealing with patients data it's
Josh: very challenging for them to fork it over to open AI and just trust that they're
Josh: going to keep it safe so what this does is it actually allows companies that
Josh: are in like the healthcare industry the finance industry who's dealing with
Josh: very like high touch personal finance the legal industry who's dealing with
Josh: a lot of legality government and defense a lot of these industries that were
Josh: not previously able to use these popular AI models,
Josh: well, now they have a pretty good model that they could run locally on their machines.
Josh: And that doesn't have any possibility of actually leaking out their customer
Josh: data, leaking out financials or healthcare data or, or like any sort of legal documents.
Josh: And, and that feels like a super powerful unlock. So for them,
Josh: it feels like a no brainer, obviously get the 120 B model running on a local
Josh: machine inside of your office, and you can load it up with all this context.
Josh: And that seems to be who this would be most impacting, right?
Ejaaz: But still to that point, I wonder how many of these companies can be bothered
Ejaaz: to do that themselves and run their own internal kind of like infrastructure.
Ejaaz: I'm thinking about OpenAI, who cracked, I think, $10 billion in annual recurring
Ejaaz: revenue this week, which is like a major milestone.
Ejaaz: And a good chunk of that, I think 33% of that is for enterprise customers.
Ejaaz: And to your point, like these enterprise customers don't wanna be giving open
Ejaaz: AI their entire data. You know, they can be used to train other AI models.
Ejaaz: So their fix or solution right now is they use kind of like private cloud instances,
Ejaaz: that I think are supplied by Microsoft by their Azure cloud service or something like that.
Ejaaz: And I wonder if they chose that,
Ejaaz: One, because there wasn't any open source models available or because they kind
Ejaaz: of just want to offload that to Microsoft to deal with.
Ejaaz: My gut tells me they're going to want to go with the latter,
Ejaaz: which is like, you know, just give it to some kind of cloud provider to deal with themselves.
Ejaaz: And they just trust Microsoft because it's a big brand name.
Ejaaz: But yeah, I don't really know how they'll materialize. I still think,
Ejaaz: and maybe this is because of my experience in crypto, Josh, that the open source
Ejaaz: models are still for like people that are at the fringe that are really experimenting
Ejaaz: with these things. but maybe don't have billions of dollars.
Josh: Yeah, that could be right. It'll be interesting to see how it plays out on all
Josh: scale of businesses because I mean, as a, like I think of a lot of indie devs
Josh: that I follow on Twitter and I see them all the time
Josh: just running local servers and they just, if they had this local model that
Josh: they could run on their machine and it takes the cost per query down from like
Josh: a penny to zero, that's like a big zero to one change.
Josh: So he does this model special because there are also a number of breakthroughs
Josh: that occurred in order to make this possible,
Josh: in order to condense this knowledge to be so tight so here's this
Josh: tweet from the professor talking about the cool tech tweaks in
Josh: this new model and what open ai was able to achieve some of
Josh: these i believe are novel some of these are seen before um if
Josh: you look at point two mixture of experts we're familiar with mixture of experts
Josh: we've seen other companies use that like kimmy and deep
Josh: seek basically instead of one brain doing everything the ai
Josh: has this team of experts that are kind of like mini brains
Josh: and specialize in different tasks it picks the right expert for
Josh: the job and it makes it faster so like instead of
Josh: having the entire 120 million parameter model search for one question maybe
Josh: you just take a couple million of those parameters that are really good at solving
Josh: math problems and they use it and that that's what brings compute down the first
Josh: point is this thing called the sliding window attention so if you imagine an
Josh: ai is like reading a really long book
Josh: It can only focus on a few pages at a time this trick
Josh: kind of lets it slide its focus window along the text so
Josh: when you think of a context window generally it's fixed right where you can see
Josh: a fixed set of data this sliding window
Josh: attention allows you to kind of move that context back and forth a
Josh: little bit so it takes what would have normally been
Josh: a narrow context window and extends it out a little bit to
Josh: the side so you get a little bit more context which is great for a
Josh: smaller model again you really want to consider that all of these are
Josh: are optimized for this microscopic scale that
Josh: can literally run on your phone and then the third point is this
Josh: thing called rope with yarn which sounds like a cat toy but this
Josh: is how the ai keeps track of the order of words so like the position
Josh: of the words in a sentence um so rope
Josh: you could imagine it like like the twisty math way to do
Josh: it and yarn makes it stretch further for really long stuff
Josh: so we have the context window that is
Josh: sliding we have this rope with yarn that allows you
Josh: to just kind of like stretch the words a little bit further and
Josh: then we have attention sinks which is the last one which is
Josh: there's a problem when ai is dealing with these endless chats that
Josh: lets it it kind of sinks in or ignores the boring old
Josh: info so it can pay attention to the new stuff so basically what it
Josh: is is if you're having a long chat with it and it determines hey this stuff
Josh: is kind of boring i don't need to remember it it'll actually just throw it away
Josh: and it'll increase that context window a little bit so again hyper optimizing
Josh: for for the small context window that it has and those are kind of the key four
Josh: breakthroughs that made this special again i'm not sure any of them are particularly novel,
Josh: But when combined together, that's what allows you to get these 04 mini results
Josh: or even 03 results on the larger model on something that can run locally on your laptop.
Josh: So it's a pretty interesting set of breakthroughs. I think a lot of times OpenAI,
Josh: we talk about them because of their feature breakthroughs, not really their
Josh: technical breakthroughs.
Josh: I think a lot of times the technical breakthroughs are reserved for like the
Josh: Kimi models or the DeepSeq models
Josh: where they really kind of break open the barrier of what's possible.
Josh: But I don't want to discredit OpenAI because these are pretty interesting things
Josh: that they've managed to combine together into this like one cohesive,
Josh: tiny little model, and then just gave it away.
Ejaaz: Yeah. I mean, they actually have a history of front-running open source frontier breakthroughs.
Ejaaz: If you remember when DeepSeek got deployed, Josh, one of their primary training
Ejaaz: methods was reinforcement learning, which was pioneered by an open AI researcher,
Ejaaz: which who probably like now works at Meta.
Ejaaz: Yeah, and I was I was I was looking at the feature that you mentioned just not
Ejaaz: the feature, but the breakthrough sliding window attention, and you mentioned
Ejaaz: that it can basically toggle reasoning.
Ejaaz: And I was pleasantly surprised to just notice that on the actual interface of
Ejaaz: the models here, Josh, can you see over here?
Ejaaz: You can toggle between reasoning levels of high, medium and low.
Ejaaz: So depending on what your prompt or query is, if it is kind of like a low level
Ejaaz: query where you're like hey just record this shopping or grocery list you know
Ejaaz: that's probably like a medium or a low query so oh it's pretty cool to to see
Ejaaz: that surface to the user like see it actively being used.
Josh: Yeah, no, super cool. I think I like the fine tuning of it.
Josh: And again, allowing you to kind of choose your intelligence levels,
Josh: because I imagine a lot of average people just don't, a lot of average queries
Josh: just don't need that much compute.
Josh: So if you can toggle it for the low reasoning level and get your answers,
Josh: that that's amazing. Super fast, super cheap.
Ejaaz: Did you see that trending tweet earlier this week, Josh, which basically said
Ejaaz: that the majority of ChatGPT users have never used a different model than ChatGPT 4.0?
Josh: I haven't seen it, but that makes sense.
Ejaaz: Yeah i i feel like the bulk of people i was chatting to
Ejaaz: my sister yesterday and she was kind of
Ejaaz: like using it for some research project at work and the
Ejaaz: screenshot she sent me over was foro and i was like hey you know like
Ejaaz: you could just run this on like a model that's like
Ejaaz: five times better than this right uh we'll come
Ejaaz: up with a much more creative set of ideas so just made me think that
Ejaaz: like i don't know how many people like care that they are like
Ejaaz: these brand new novel models and maybe um you know
Ejaaz: this kind of like basic model is good enough for everyone i don't know
Ejaaz: but um but moving on josh um there
Ejaaz: was a big question that popped into my head as
Ejaaz: soon as these models released which was are they as good
Ejaaz: as the chinese open source models right i wanted
Ejaaz: to get some opinions from people and and the reason
Ejaaz: why this matters i'm just give the listeners some context
Ejaaz: is china has been the number one
Ejaaz: nation to put out the best open source
Ejaaz: models over the last 12 months it started with deep seek
Ejaaz: and then alibaba's quen models got involved
Ejaaz: and then recently we had kimmy k2 and i think
Ejaaz: there was another ai lab out of china which came out so they
Ejaaz: have outside of america the highest density.
Ejaaz: Of the top ai researchers they all come out of this one university
Ejaaz: zinghua i believe they kind of like partially work
Ejaaz: or train in the u.s as well so they've got this like kind of hybrid ai
Ejaaz: mentality of how to build these models and they come up with a lot of these
Ejaaz: frontier breakthroughs um kimmy k2 for context had uh one trillion parameters
Ejaaz: in their model right comparing this to like 120 billion and 20 billion parameters
Ejaaz: models from open air i was curious like does this beat them to the punch some people josh.
Ejaaz: Don't think so okay this guy jason lee
Ejaaz: he asks uh is the gpt oss stronger
Ejaaz: than quen or kimmy or chinese open models and then
Ejaaz: he later kind of quote tweets that tweet and says answer the model is complete
Ejaaz: junk it's a hallucination machine overfit to reasoning benchmarks and has absolutely
Ejaaz: zero recall ability so a few things he's mentioning here is one it hallucinates
Ejaaz: a lot so it kind of makes up jargon terms,
Ejaaz: ideas, or parameters that didn't really exist before.
Ejaaz: Number two, he's saying that OpenAI designed this model purely so that it will
Ejaaz: do well on the exams, which are the benchmarks that rate how these models compare to each other.
Ejaaz: So they're saying that OpenAI optimized the model to kind of like do really
Ejaaz: well at those tests, but actually fail at everything else, which is what people want to use it for.
Ejaaz: And the final point that he makes is that it has zero recall ability,
Ejaaz: which is something you mentioned earlier, Josh, which says it doesn't have memory
Ejaaz: or context so you can have a conversation and then open up another conversation
Ejaaz: and it's completely forgotten about the context that it has for you from that
Ejaaz: initial conversation okay.
Josh: So not not the best not to be unfair to open ai but it feels like they delayed
Josh: this model a good bit of times oh yeah and they wanted it to look good and it
Josh: intuitively makes sense to me that they would be kind of optimizing for benchmarks
Josh: with this one um but nonetheless it's still impressive i'm seeing this big wall
Josh: of text now what is what is this what is this post here
Ejaaz: Well it's this post from uh one of these accounts i follow and they have an
Ejaaz: interesting section here which says comparison to other open weights oh sick.
Josh: Yeah what is this
Ejaaz: So he goes while the larger gpt oss
Ejaaz: 120 billion parameter model does not come
Ejaaz: in above deep seek r1 so he's saying that deep seek r1
Ejaaz: just beats it out the park it is notable that
Ejaaz: it is significantly smaller in both total and active
Ejaaz: parameters than both of those models deep seek
Ejaaz: r1 has 671 billion total parameters and
Ejaaz: 37 billion active parameters and is released natively right which makes it 10x
Ejaaz: larger than gpt's 120 billion parameter models but what he's saying is even
Ejaaz: though gpt's model is smaller and doesn't perform as well as deep seek it's
Ejaaz: still mightily impressive for its size.
Josh: Okay that's cool because that gets back to the point we made earlier in the
Josh: show that this is probably the most densely condensed
Josh: however you want to say it like base of
Josh: knowledge in the world they've used a lot of efficiency gains
Josh: to squeeze the most out of it so in this small model
Josh: it is i guess if we're optimizing maybe we
Josh: can make up a metric here on the show which is like um output per
Josh: per parameter or something like that like based on the total parameter
Josh: count of this model it gives you the best value per
Josh: token and that seems to be where this falls
Josh: in line where it's not going to blow any other open source model out of the
Josh: water but in terms of its size the fact that we can
Josh: take a phone and literally run one of these models on a phone and
Josh: you could go anywhere in the world with no service and have access to these models running
Josh: on a laptop or whatever mobile device that that's super
Josh: powerful and that's not something that is easy to do with the other open source
Josh: models so perhaps that's the advantage that open ai has it's just the density
Josh: of intelligence and the efficiency of these parameters that they've given to
Josh: us versus just being this like home run open source model that is going for the frontier,
Josh: it's just a little bit of a different approach.
Ejaaz: Yeah, we need like a small but mighty ranking on this show, Josh,
Ejaaz: that we can kind of like run every week when these companies release a new model.
Ejaaz: No, but it got me thinking, if we zoomed out of that question,
Ejaaz: right, because we're talking about small models versus large models,
Ejaaz: parameters and how effectively they use versus other models that are bigger.
Ejaaz: What really matters in this, Josh? In my opinion, it's user experience and how
Ejaaz: useful these models are to my daily life, right?
Ejaaz: At the end of the day, I kind of don't really care what size that model is unless
Ejaaz: it's useful for me, right? It could be small, it could be personal, it could be private.
Ejaaz: It depends on, I guess, the use case at the time. And I have a feeling that
Ejaaz: the trend of how technology typically goes, you kind of want a really high-performant
Ejaaz: small model, eventually.
Ejaaz: Right? I try and think about like us using computers for the first time,
Ejaaz: you know, back in our dinosaur age.
Ejaaz: And then, you know, it all being condensed on a tiny metal slab that we now
Ejaaz: use every day. And we can pretty much work from remotely from wherever.
Ejaaz: And I feel like this is where models are going to go. They're going to become
Ejaaz: more private. They're going to become more personal.
Ejaaz: Maybe it'll be a combination of, you know, it running locally on your device
Ejaaz: versus cloud inference and trusting certain providers.
Ejaaz: I don't know how it's going to fall out, but I think Like it's not a zero to
Ejaaz: one. It's not a black or white situation.
Ejaaz: I don't think everyone's just going to go with large centralized models that
Ejaaz: they can inference from the cloud.
Ejaaz: I think it'll be a mixture of both. And how that materializes,
Ejaaz: I don't know, but it's an interesting one to ponder.
Josh: Yeah, I think this is funny. This is going to sound very ironic,
Josh: but Apple was the person that got this most right.
Ejaaz: Sorry, who's Apple again?
Josh: Yeah, right. I mean, it sounds ridiculous to say this. And granted,
Josh: they did not execute on this at all.
Josh: But in theory, I think they nailed the approach initially,
Josh: which was you run local compute where all of
Josh: your stuff is so my iphone is the device i never
Josh: leave without it is everything about me it is all of my messages my
Josh: contacts all the contacts you could ever want from me and then the idea was
Josh: they would give you a local model that is integrated and embedded into that
Josh: operating system and then if there's anything that requires more compute well
Josh: then they'll send the query off into the cloud but most of it will get done
Josh: on your local device because most of it isn't that complicated and i think as
Josh: a user when i ask myself what i want from AI.
Josh: Well, I just want it to be my ultimate assistant. I just want it to be there
Josh: to make my life better. And so much of that is the context.
Josh: And Apple going with that model would have been incredible.
Josh: It would have been so great. It would have had the lightweight model that runs
Josh: locally, it has all the context of your life, and then it offloads to the cloud.
Josh: I still think this model is probably the correct one for optimizing the user
Josh: experience. But unfortunately, Apple just has not done that.
Josh: So it's up for grabs. I mean, again, Sam Altman's been posting a lot this week,
Josh: we do have to tease what's coming because this is probably going to be a huge
Josh: week. There's a high probability we get GPT-5.
Josh: And then they've also been talking about their hardware device a little bit. And they're saying how
Josh: It's like it's genuinely going to change the world. And I believe the reason
Josh: why is because they're taking this Apple approach where they're building the
Josh: operating system, they're gathering the context, and then they're just they're
Josh: able to serve it now locally on device.
Josh: They're able to go to the cloud when they need more compute.
Josh: And it's going to create this really cool, I think, duality of AI where you
Josh: have your your super private local one, and then you have the big brain one,
Josh: the big brother that's off in the cloud that does all the hard computing for you.
Ejaaz: Well, one thing is clear. There are going to be hundreds of models and it's
Ejaaz: going to benefit the user, you and I, for so many multiple...
Ejaaz: It's the big company's problems to figure out how these models work together
Ejaaz: and which ones get queried. I don't care.
Ejaaz: Just give me the good stuff and I'm going to be happy.
Ejaaz: Folks, OpenAI has been cooking. This was the first open source models they've
Ejaaz: released in six years, Josh.
Ejaaz: The last one was 2019 GPT-2, which seems like the stone age and it was only like four years ago.
Ejaaz: Thank you so much for listening. We are pumped to be talking about GPT-5,
Ejaaz: which we hope to be released in maybe 24 hours.
Ejaaz: Hopefully this week, fingers crossed. I don't know, we might be back on this
Ejaaz: camera pretty soon. Stay tuned.
Ejaaz: Please like, subscribe, and watch out for all the updates. We're going to release
Ejaaz: a bunch of clips as well if you want to kind of like get to the juicy bits as well.
Ejaaz: Share this with your friends and give us feedback. If you want to hear about
Ejaaz: different things, things that we haven't covered yet or things that we've spoken
Ejaaz: about, but you want to get more clarity on or guests that you want to join the show, let us know.
Ejaaz: We're going full force on this and we'll see you on the next one.
Josh: Sounds good. See you guys soon. Peace.
Music: Music