OpenAI Just Gave Away Their Secret Formula... For Free?

In this episode, we discuss OpenAI's shift to open-source with the release of a 120 billion and a 20 billion parameter model for local use. Ejaaz and Josh highlight the democratization of AI access, enhanced privacy, and customization opportunities. 

We analyze the competitive landscape against major Chinese models and hint at the anticipated GPT-5 release. Tune in for insights into this transformative moment in AI!

------
🌌 LIMITLESS HQ: LISTEN & FOLLOW HERE ⬇️
https://limitless.bankless.com/
https://x.com/LimitlessFT

------
TIMESTAMPS

0:00 OpenAI's Surprising Release
1:30 The Power of Open Source Models
3:28 Local Computing Revolution
5:33 Privacy and Personalization
6:54 The Impact on Industries
9:32 Testing the New Models
17:43 Competing with Chinese Models
24:06 The Future of AI Technology
26:29 Anticipating GPT-5

------
RESOURCES

Josh: https://x.com/Josh_Kale

Ejaaz: https://x.com/cryptopunk7213

------
Not financial or tax advice. See our investment disclosures here:
https://www.bankless.com/disclosures⁠

Creators and Guests

Host

Josh Kale

What is Limitless Podcast?

Exploring the frontiers of Technology and AI

Josh:
The unthinkable has just happened open ai

Josh:
has released an open source model open ai

Josh:
has been closed ai since the time that i knew them

Josh:
they have been named themselves open ai they were not

Josh:
open source they have finally released an open source model and surprise surprise

Josh:
it's actually really great and i think the downstream implications of an open

Josh:
source model from a company like this that is this good are really it's a really

Josh:
big deal i think this really matters a lot just yesterday they announced the release of GPT-OSS.

Josh:
There are two models. There's a 120 billion parameter model and there's a 20

Josh:
billion parameter model. We're going to get into benchmarks.

Josh:
We're going to get into how good they are.

Josh:
But the idea is that OpenAI has actually released an open source model.

Josh:
And this can compare to the Chinese models because we recently had DeepSeek and we've had Kimi.

Josh:
And those would be very good. But this is the first really solid American-based open source model.

Josh:
So Ijaz, I know you've been kind of digging in the weeds about how this works.

Josh:
Can you explain us exactly why this is a big deal why this happened what's going on here

Ejaaz:
Yeah it's it's pretty huge so so here

Ejaaz:
are the hot highlights um as you mentioned there's two

Ejaaz:
models that came out the 20 billion parameter model which is actually small

Ejaaz:
enough to run on your mobile phone right now and they have a 120 billion parameter

Ejaaz:
model which is big but still small enough to run on a high performance laptop

Ejaaz:
so if you guys have a macbook out there jump in go for it um it's fully customizable.

Ejaaz:
So remember, open source means

Ejaaz:
that you can literally have access to the design of the entire model.

Ejaaz:
It's like OpenAI giving away their secret recipe to how their frontier models

Ejaaz:
work. And you can kind of like recreate it at home.

Ejaaz:
This means that you can customize it to any kind of use case that you want,

Ejaaz:
give it access to all your personal hard drives, tools, data,

Ejaaz:
and it can do wonderful stuff.

Ejaaz:
But Josh, here's the amazing part.

Ejaaz:
On paper, these models are as good as GPT-4 mini models, which is,

Ejaaz:
it's pretty impressive, right?

Ejaaz:
In practice and i've been playing around with it for the last few hours they're

Ejaaz:
as good in my opinion and actually quicker than

Ejaaz:
gpt-03 which is their frontier model and i

Ejaaz:
mean this across like everything so

Ejaaz:
reasoning um it spits out answers super quickly and i can see its reasoning

Ejaaz:
it happens in like a couple of seconds and i'm so used to waiting like 30 seconds

Ejaaz:
to a couple minutes on gpt-03 josh so it's pretty impressive and an insane unlock

Ejaaz:
on coding it's as good and on creativity as well.

Ejaaz:
So I'm my mind's pretty blown at all of this, right? Josh, what do you what do you think?

Josh:
Yeah, so here's why it's impressive to me is because a lot of the times I don't

Josh:
really care to use the outer bands of what a model is capable of.

Josh:
Like I am not doing deep PhD level research. I'm not solving these math Olympiad questions.

Josh:
I'm just trying to ask it a few normal questions and get some answers.

Josh:
And what these models do is an excellent job at serving that need.

Josh:
They're not going to go out and solve the world's hardest problems,

Josh:
but neither do I. I don't want to solve those problems.

Josh:
I just kind of want the information that I want, whether it be just a normal

Josh:
Google type search or whether it be asking it some miscellaneous question about

Josh:
some work that I'm doing.

Josh:
It's really good at answering that. So I think initial impressions,

Josh:
because they did allow you to test it publicly through their website,

Josh:
it's just really good at the things that I want.

Josh:
So the fact that I can run one of these models on a local device on my iPhone,

Josh:
well, it feels like we're reaching this place that AI is starting to become

Josh:
really interesting because for so long we've had compute handled fully on the

Josh:
cloud and now this is the first time where

Josh:
Compute can really happen on your computer. It could happen on your laptop.

Josh:
I could download the model and I could actually store the model,

Josh:
the 120 billion parameter model on a 56 gigabyte USB drive.

Josh:
So you can take the collective knowledge of the world and put it on a tiny little USB drive.

Josh:
And granted, it needs a bit of a bigger machine to actually run those parameters,

Josh:
but you can install all the weights. It's 56 gigabytes.

Josh:
It's this incredibly powerful package. And it probably, I don't know if this

Josh:
is true, but it's probably the most condensed knowledge base in the history of humanity.

Josh:
They've really managed to take a tremendous amount of tokens,

Josh:
smush them into this little parameter set, and then publish it for people to

Josh:
use. So for me, I'm really excited.

Josh:
I like having my own mini portable models. I am excited to download,

Josh:
try it out, run it on my MacBook.

Josh:
I'm not sure I could run the 120 billion parameter model, but at least the 20B

Josh:
and give it a shot and see how it works.

Ejaaz:
You need to get the latest MacBook, Josh. I know, I got to upgrade. We can test that out.

Ejaaz:
What I also love about it is it's fully private, right? So you can give it access

Ejaaz:
to your personal hard drive, your Apple Notes, whatever you store on your computer, basically.

Ejaaz:
And you can basically instruct the model to use those different tools.

Ejaaz:
So one review that I keep seeing from a number of people who have been testing

Ejaaz:
it so far is that it's incredibly great and intuitive at tool use.

Ejaaz:
And the reason why this is such a big deal is a lot of the Frontier models right

Ejaaz:
now, when they allow you to give access to different tools, they're kind of clunky.

Ejaaz:
The model doesn't actually know when to use a specific tool and when not to.

Ejaaz:
But these models are super intuitive, which is great. The privacy thing is also

Ejaaz:
a big thing because you kind of

Ejaaz:
don't want to be giving all your personal information away to Sam Altman.

Ejaaz:
But you want a highly personalized model.

Ejaaz:
And I think if I was to condense this entire model release in a single sentence,

Ejaaz:
Joss, I think I would say it is the epitome of privacy and personalization in an AI model so far.

Ejaaz:
It is that good. it is swift it is cheap and I'm going to replace it completely

Ejaaz:
with all my GPT-4.0 queries as you said earlier like,

Ejaaz:
Who needs to use the basic models anymore when you have access to this?

Josh:
Yeah. So it's funny you say that you're going to swap it because I don't think I'm going to swap it.

Josh:
I still am not sure I personally have a use case right now because I love the

Josh:
context. I want the memory.

Josh:
I like having it all server side where it kind of knows everything about me.

Josh:
I guess in the case that I wanted to really make it a more intimate model experience

Josh:
where you want to sync it up with like journal entries or your camera roll or

Josh:
whatever, whatever interesting like personal things, this would be a really cool use case.

Josh:
I think for the people who are curious why this matters to them,

Josh:
well, we could talk a little briefly about like the second order effects of

Josh:
having open source models as powerful, because what that allows you to do is

Josh:
to serve queries from a local machine.

Josh:
So if you are using an app or let's say you're an app developer and you're building

Josh:
an application and your app is serving millions of requests because it's a GPT wrapper.

Josh:
Well, what you could do now is instead of paying API calls to the OpenAI server,

Josh:
you can actually just run your own local server, use this model,

Josh:
and then serve all that data for the cost of the electricity.

Josh:
And that's a really big unlock for the amount of compute that's going to be

Josh:
available for not only developers, but for the cost of the users in a lot of these applications.

Josh:
So for the applications that aren't doing this crazy moon math and that are

Josh:
just kind of serving basic queries all day long, this like really significantly drops the cost.

Josh:
It increases the privacy, like you mentioned. And there's a ton of really important

Josh:
upsides to open source models that we just haven't seen up until now.

Josh:
And I'm very excited to see come forward.

Ejaaz:
Well, Josh, the thing with most of these open source models,

Ejaaz:
we spoke about actually two major Chinese open source models that were released last week.

Ejaaz:
It's not accessible to everyone. Like you and me aren't necessarily going to

Ejaaz:
go to Hugging Face, a completely separate website, download these models,

Ejaaz:
run the command line interface.

Ejaaz:
Most of the listeners on the show doesn't even know what that means.

Ejaaz:
I don't even know if I know what that means, right?

Ejaaz:
But here you have a lovely created website where you could just kind of log

Ejaaz:
on and play around with these open source models. And that's exactly what I've been doing.

Ejaaz:
I actually have a few kind of demo queries that I ran yesterday, Josh.

Josh:
Yeah, walk us through, let's see.

Ejaaz:
Okay, so there's an incredibly complex test, which a lot of these AI models,

Ejaaz:
which cost hundreds of billions of dollars to train, can't quite answer.

Ejaaz:
And that is how many R's, the letter R's are there in the word strawberry? Most say two.

Josh:
The bar's on the floor,

Ejaaz:
Huh? Yeah, if we were to go with most models, they say two. They're convinced that they are only two.

Ejaaz:
And I ran that test today, rather yesterday, with these open source models,

Ejaaz:
and it correctly guessed three, Josh. So we're one for one right now.

Josh:
We're on our way.

Ejaaz:
But then I was like, okay, we live in New York City. I love this place.

Ejaaz:
I'm feeling a little poetic today. Can you write me a sonnet?

Ejaaz:
And my goal with this wasn't to test whether it could just write a poem.

Ejaaz:
It was to test how quickly it could figure it out.

Ejaaz:
And as you see it thought for a couple of seconds on this so it literally spat

Ejaaz:
this out in two seconds um and it was structured really well you know it kind

Ejaaz:
of flowed would i be you know reciting this out loud to the public no but you

Ejaaz:
know i was pretty impressed.

Ejaaz:
And then, Josh, I was thinking, you know, what's so unique about open source models?

Ejaaz:
You just went through a really good list of why open source models work.

Ejaaz:
But I was curious as to why these specific open source models were better than

Ejaaz:
other open source models or maybe even other centralized models.

Ejaaz:
So I wrote a query. I decided to ask it. I was like, you know,

Ejaaz:
tell me some things that you could do that are the larger centralized models.

Ejaaz:
And I spat out a really good list. I'm not going to go through all of them,

Ejaaz:
but, you know, some of the things that we've highlighted so far, you can fine tune it.

Ejaaz:
It's privacy. See, I really like this point that it made, Josh,

Ejaaz:
that it just shows that AI is probably getting smarter than us,

Ejaaz:
which is you can custom inject your own data into these models.

Ejaaz:
Now, without kind of digging deeper into this, when you use a centralized model,

Ejaaz:
it's already pre-trained on a bunch of data that companies like Anthropic and

Ejaaz:
Google have already fed it.

Ejaaz:
And so it's kind of formed its own personality, right?

Ejaaz:
So you can't change the model's personality on a centralized model.

Ejaaz:
But with an open model you have full reign to do whatever you want and so if

Ejaaz:
you were feeling kind of uh adventurous you could use your own data and make

Ejaaz:
it super personal and customizable so i thought that was really cool and fun

Ejaaz:
demo josh have you been playing around with this.

Josh:
Yeah it's um it's it's smart it's fun it's smart i wouldn't say it's anything

Josh:
novel the like query results that i get are you know on par with everything

Josh:
else i don't notice the difference which is good because it means they're performing

Josh:
very well it's not like i feel like i'm getting degraded performance because

Josh:
I'm using a smaller model.

Josh:
But it's just like it's nothing too different, I would say.

Josh:
The differences, I mean, again, all this boils down to the differences of it

Josh:
being open source versus being

Ejaaz:
Run on the server. Well, let me challenge you that, right? OK,

Ejaaz:
so you're saying it's good but nothing novel.

Ejaaz:
Would you say it's as good as GPT-4.0,

Ejaaz:
minus the memory let's just put memory aside for a second would you use it if

Ejaaz:
it had memory capability.

Josh:
Actually no probably not um i still wouldn't

Josh:
because i love my desktop application too much i

Josh:
love my mobile app too much and i like that the conversations are

Josh:
shared in the cloud um so i can use them on my phone i could

Josh:
start on my laptop and go back and forth so even in

Josh:
that case i'm probably still not a user um because

Josh:
the convenience factor but there are there are a

Josh:
lot of people and a lot of industries that would be and this is actually something probably

Josh:
worth surfacing is the new industries that are now able to

Josh:
benefit from this because a lot of industries have

Josh:
a tough time using these AI models because

Josh:
of the data privacy concerns particularly I mean if you think about a

Josh:
healthcare industry people who are dealing with patients data it's

Josh:
very challenging for them to fork it over to open AI and just trust that they're

Josh:
going to keep it safe so what this does is it actually allows companies that

Josh:
are in like the healthcare industry the finance industry who's dealing with

Josh:
very like high touch personal finance the legal industry who's dealing with

Josh:
a lot of legality government and defense a lot of these industries that were

Josh:
not previously able to use these popular AI models,

Josh:
well, now they have a pretty good model that they could run locally on their machines.

Josh:
And that doesn't have any possibility of actually leaking out their customer

Josh:
data, leaking out financials or healthcare data or, or like any sort of legal documents.

Josh:
And, and that feels like a super powerful unlock. So for them,

Josh:
it feels like a no brainer, obviously get the 120 B model running on a local

Josh:
machine inside of your office, and you can load it up with all this context.

Josh:
And that seems to be who this would be most impacting, right?

Ejaaz:
But still to that point, I wonder how many of these companies can be bothered

Ejaaz:
to do that themselves and run their own internal kind of like infrastructure.

Ejaaz:
I'm thinking about OpenAI, who cracked, I think, $10 billion in annual recurring

Ejaaz:
revenue this week, which is like a major milestone.

Ejaaz:
And a good chunk of that, I think 33% of that is for enterprise customers.

Ejaaz:
And to your point, like these enterprise customers don't wanna be giving open

Ejaaz:
AI their entire data. You know, they can be used to train other AI models.

Ejaaz:
So their fix or solution right now is they use kind of like private cloud instances,

Ejaaz:
that I think are supplied by Microsoft by their Azure cloud service or something like that.

Ejaaz:
And I wonder if they chose that,

Ejaaz:
One, because there wasn't any open source models available or because they kind

Ejaaz:
of just want to offload that to Microsoft to deal with.

Ejaaz:
My gut tells me they're going to want to go with the latter,

Ejaaz:
which is like, you know, just give it to some kind of cloud provider to deal with themselves.

Ejaaz:
And they just trust Microsoft because it's a big brand name.

Ejaaz:
But yeah, I don't really know how they'll materialize. I still think,

Ejaaz:
and maybe this is because of my experience in crypto, Josh, that the open source

Ejaaz:
models are still for like people that are at the fringe that are really experimenting

Ejaaz:
with these things. but maybe don't have billions of dollars.

Josh:
Yeah, that could be right. It'll be interesting to see how it plays out on all

Josh:
scale of businesses because I mean, as a, like I think of a lot of indie devs

Josh:
that I follow on Twitter and I see them all the time

Josh:
just running local servers and they just, if they had this local model that

Josh:
they could run on their machine and it takes the cost per query down from like

Josh:
a penny to zero, that's like a big zero to one change.

Josh:
So he does this model special because there are also a number of breakthroughs

Josh:
that occurred in order to make this possible,

Josh:
in order to condense this knowledge to be so tight so here's this

Josh:
tweet from the professor talking about the cool tech tweaks in

Josh:
this new model and what open ai was able to achieve some of

Josh:
these i believe are novel some of these are seen before um if

Josh:
you look at point two mixture of experts we're familiar with mixture of experts

Josh:
we've seen other companies use that like kimmy and deep

Josh:
seek basically instead of one brain doing everything the ai

Josh:
has this team of experts that are kind of like mini brains

Josh:
and specialize in different tasks it picks the right expert for

Josh:
the job and it makes it faster so like instead of

Josh:
having the entire 120 million parameter model search for one question maybe

Josh:
you just take a couple million of those parameters that are really good at solving

Josh:
math problems and they use it and that that's what brings compute down the first

Josh:
point is this thing called the sliding window attention so if you imagine an

Josh:
ai is like reading a really long book

Josh:
It can only focus on a few pages at a time this trick

Josh:
kind of lets it slide its focus window along the text so

Josh:
when you think of a context window generally it's fixed right where you can see

Josh:
a fixed set of data this sliding window

Josh:
attention allows you to kind of move that context back and forth a

Josh:
little bit so it takes what would have normally been

Josh:
a narrow context window and extends it out a little bit to

Josh:
the side so you get a little bit more context which is great for a

Josh:
smaller model again you really want to consider that all of these are

Josh:
are optimized for this microscopic scale that

Josh:
can literally run on your phone and then the third point is this

Josh:
thing called rope with yarn which sounds like a cat toy but this

Josh:
is how the ai keeps track of the order of words so like the position

Josh:
of the words in a sentence um so rope

Josh:
you could imagine it like like the twisty math way to do

Josh:
it and yarn makes it stretch further for really long stuff

Josh:
so we have the context window that is

Josh:
sliding we have this rope with yarn that allows you

Josh:
to just kind of like stretch the words a little bit further and

Josh:
then we have attention sinks which is the last one which is

Josh:
there's a problem when ai is dealing with these endless chats that

Josh:
lets it it kind of sinks in or ignores the boring old

Josh:
info so it can pay attention to the new stuff so basically what it

Josh:
is is if you're having a long chat with it and it determines hey this stuff

Josh:
is kind of boring i don't need to remember it it'll actually just throw it away

Josh:
and it'll increase that context window a little bit so again hyper optimizing

Josh:
for for the small context window that it has and those are kind of the key four

Josh:
breakthroughs that made this special again i'm not sure any of them are particularly novel,

Josh:
But when combined together, that's what allows you to get these 04 mini results

Josh:
or even 03 results on the larger model on something that can run locally on your laptop.

Josh:
So it's a pretty interesting set of breakthroughs. I think a lot of times OpenAI,

Josh:
we talk about them because of their feature breakthroughs, not really their

Josh:
technical breakthroughs.

Josh:
I think a lot of times the technical breakthroughs are reserved for like the

Josh:
Kimi models or the DeepSeq models

Josh:
where they really kind of break open the barrier of what's possible.

Josh:
But I don't want to discredit OpenAI because these are pretty interesting things

Josh:
that they've managed to combine together into this like one cohesive,

Josh:
tiny little model, and then just gave it away.

Ejaaz:
Yeah. I mean, they actually have a history of front-running open source frontier breakthroughs.

Ejaaz:
If you remember when DeepSeek got deployed, Josh, one of their primary training

Ejaaz:
methods was reinforcement learning, which was pioneered by an open AI researcher,

Ejaaz:
which who probably like now works at Meta.

Ejaaz:
Yeah, and I was I was I was looking at the feature that you mentioned just not

Ejaaz:
the feature, but the breakthrough sliding window attention, and you mentioned

Ejaaz:
that it can basically toggle reasoning.

Ejaaz:
And I was pleasantly surprised to just notice that on the actual interface of

Ejaaz:
the models here, Josh, can you see over here?

Ejaaz:
You can toggle between reasoning levels of high, medium and low.

Ejaaz:
So depending on what your prompt or query is, if it is kind of like a low level

Ejaaz:
query where you're like hey just record this shopping or grocery list you know

Ejaaz:
that's probably like a medium or a low query so oh it's pretty cool to to see

Ejaaz:
that surface to the user like see it actively being used.

Josh:
Yeah, no, super cool. I think I like the fine tuning of it.

Josh:
And again, allowing you to kind of choose your intelligence levels,

Josh:
because I imagine a lot of average people just don't, a lot of average queries

Josh:
just don't need that much compute.

Josh:
So if you can toggle it for the low reasoning level and get your answers,

Josh:
that that's amazing. Super fast, super cheap.

Ejaaz:
Did you see that trending tweet earlier this week, Josh, which basically said

Ejaaz:
that the majority of ChatGPT users have never used a different model than ChatGPT 4.0?

Josh:
I haven't seen it, but that makes sense.

Ejaaz:
Yeah i i feel like the bulk of people i was chatting to

Ejaaz:
my sister yesterday and she was kind of

Ejaaz:
like using it for some research project at work and the

Ejaaz:
screenshot she sent me over was foro and i was like hey you know like

Ejaaz:
you could just run this on like a model that's like

Ejaaz:
five times better than this right uh we'll come

Ejaaz:
up with a much more creative set of ideas so just made me think that

Ejaaz:
like i don't know how many people like care that they are like

Ejaaz:
these brand new novel models and maybe um you know

Ejaaz:
this kind of like basic model is good enough for everyone i don't know

Ejaaz:
but um but moving on josh um there

Ejaaz:
was a big question that popped into my head as

Ejaaz:
soon as these models released which was are they as good

Ejaaz:
as the chinese open source models right i wanted

Ejaaz:
to get some opinions from people and and the reason

Ejaaz:
why this matters i'm just give the listeners some context

Ejaaz:
is china has been the number one

Ejaaz:
nation to put out the best open source

Ejaaz:
models over the last 12 months it started with deep seek

Ejaaz:
and then alibaba's quen models got involved

Ejaaz:
and then recently we had kimmy k2 and i think

Ejaaz:
there was another ai lab out of china which came out so they

Ejaaz:
have outside of america the highest density.

Ejaaz:
Of the top ai researchers they all come out of this one university

Ejaaz:
zinghua i believe they kind of like partially work

Ejaaz:
or train in the u.s as well so they've got this like kind of hybrid ai

Ejaaz:
mentality of how to build these models and they come up with a lot of these

Ejaaz:
frontier breakthroughs um kimmy k2 for context had uh one trillion parameters

Ejaaz:
in their model right comparing this to like 120 billion and 20 billion parameters

Ejaaz:
models from open air i was curious like does this beat them to the punch some people josh.

Ejaaz:
Don't think so okay this guy jason lee

Ejaaz:
he asks uh is the gpt oss stronger

Ejaaz:
than quen or kimmy or chinese open models and then

Ejaaz:
he later kind of quote tweets that tweet and says answer the model is complete

Ejaaz:
junk it's a hallucination machine overfit to reasoning benchmarks and has absolutely

Ejaaz:
zero recall ability so a few things he's mentioning here is one it hallucinates

Ejaaz:
a lot so it kind of makes up jargon terms,

Ejaaz:
ideas, or parameters that didn't really exist before.

Ejaaz:
Number two, he's saying that OpenAI designed this model purely so that it will

Ejaaz:
do well on the exams, which are the benchmarks that rate how these models compare to each other.

Ejaaz:
So they're saying that OpenAI optimized the model to kind of like do really

Ejaaz:
well at those tests, but actually fail at everything else, which is what people want to use it for.

Ejaaz:
And the final point that he makes is that it has zero recall ability,

Ejaaz:
which is something you mentioned earlier, Josh, which says it doesn't have memory

Ejaaz:
or context so you can have a conversation and then open up another conversation

Ejaaz:
and it's completely forgotten about the context that it has for you from that

Ejaaz:
initial conversation okay.

Josh:
So not not the best not to be unfair to open ai but it feels like they delayed

Josh:
this model a good bit of times oh yeah and they wanted it to look good and it

Josh:
intuitively makes sense to me that they would be kind of optimizing for benchmarks

Josh:
with this one um but nonetheless it's still impressive i'm seeing this big wall

Josh:
of text now what is what is this what is this post here

Ejaaz:
Well it's this post from uh one of these accounts i follow and they have an

Ejaaz:
interesting section here which says comparison to other open weights oh sick.

Josh:
Yeah what is this

Ejaaz:
So he goes while the larger gpt oss

Ejaaz:
120 billion parameter model does not come

Ejaaz:
in above deep seek r1 so he's saying that deep seek r1

Ejaaz:
just beats it out the park it is notable that

Ejaaz:
it is significantly smaller in both total and active

Ejaaz:
parameters than both of those models deep seek

Ejaaz:
r1 has 671 billion total parameters and

Ejaaz:
37 billion active parameters and is released natively right which makes it 10x

Ejaaz:
larger than gpt's 120 billion parameter models but what he's saying is even

Ejaaz:
though gpt's model is smaller and doesn't perform as well as deep seek it's

Ejaaz:
still mightily impressive for its size.

Josh:
Okay that's cool because that gets back to the point we made earlier in the

Josh:
show that this is probably the most densely condensed

Josh:
however you want to say it like base of

Josh:
knowledge in the world they've used a lot of efficiency gains

Josh:
to squeeze the most out of it so in this small model

Josh:
it is i guess if we're optimizing maybe we

Josh:
can make up a metric here on the show which is like um output per

Josh:
per parameter or something like that like based on the total parameter

Josh:
count of this model it gives you the best value per

Josh:
token and that seems to be where this falls

Josh:
in line where it's not going to blow any other open source model out of the

Josh:
water but in terms of its size the fact that we can

Josh:
take a phone and literally run one of these models on a phone and

Josh:
you could go anywhere in the world with no service and have access to these models running

Josh:
on a laptop or whatever mobile device that that's super

Josh:
powerful and that's not something that is easy to do with the other open source

Josh:
models so perhaps that's the advantage that open ai has it's just the density

Josh:
of intelligence and the efficiency of these parameters that they've given to

Josh:
us versus just being this like home run open source model that is going for the frontier,

Josh:
it's just a little bit of a different approach.

Ejaaz:
Yeah, we need like a small but mighty ranking on this show, Josh,

Ejaaz:
that we can kind of like run every week when these companies release a new model.

Ejaaz:
No, but it got me thinking, if we zoomed out of that question,

Ejaaz:
right, because we're talking about small models versus large models,

Ejaaz:
parameters and how effectively they use versus other models that are bigger.

Ejaaz:
What really matters in this, Josh? In my opinion, it's user experience and how

Ejaaz:
useful these models are to my daily life, right?

Ejaaz:
At the end of the day, I kind of don't really care what size that model is unless

Ejaaz:
it's useful for me, right? It could be small, it could be personal, it could be private.

Ejaaz:
It depends on, I guess, the use case at the time. And I have a feeling that

Ejaaz:
the trend of how technology typically goes, you kind of want a really high-performant

Ejaaz:
small model, eventually.

Ejaaz:
Right? I try and think about like us using computers for the first time,

Ejaaz:
you know, back in our dinosaur age.

Ejaaz:
And then, you know, it all being condensed on a tiny metal slab that we now

Ejaaz:
use every day. And we can pretty much work from remotely from wherever.

Ejaaz:
And I feel like this is where models are going to go. They're going to become

Ejaaz:
more private. They're going to become more personal.

Ejaaz:
Maybe it'll be a combination of, you know, it running locally on your device

Ejaaz:
versus cloud inference and trusting certain providers.

Ejaaz:
I don't know how it's going to fall out, but I think Like it's not a zero to

Ejaaz:
one. It's not a black or white situation.

Ejaaz:
I don't think everyone's just going to go with large centralized models that

Ejaaz:
they can inference from the cloud.

Ejaaz:
I think it'll be a mixture of both. And how that materializes,

Ejaaz:
I don't know, but it's an interesting one to ponder.

Josh:
Yeah, I think this is funny. This is going to sound very ironic,

Josh:
but Apple was the person that got this most right.

Ejaaz:
Sorry, who's Apple again?

Josh:
Yeah, right. I mean, it sounds ridiculous to say this. And granted,

Josh:
they did not execute on this at all.

Josh:
But in theory, I think they nailed the approach initially,

Josh:
which was you run local compute where all of

Josh:
your stuff is so my iphone is the device i never

Josh:
leave without it is everything about me it is all of my messages my

Josh:
contacts all the contacts you could ever want from me and then the idea was

Josh:
they would give you a local model that is integrated and embedded into that

Josh:
operating system and then if there's anything that requires more compute well

Josh:
then they'll send the query off into the cloud but most of it will get done

Josh:
on your local device because most of it isn't that complicated and i think as

Josh:
a user when i ask myself what i want from AI.

Josh:
Well, I just want it to be my ultimate assistant. I just want it to be there

Josh:
to make my life better. And so much of that is the context.

Josh:
And Apple going with that model would have been incredible.

Josh:
It would have been so great. It would have had the lightweight model that runs

Josh:
locally, it has all the context of your life, and then it offloads to the cloud.

Josh:
I still think this model is probably the correct one for optimizing the user

Josh:
experience. But unfortunately, Apple just has not done that.

Josh:
So it's up for grabs. I mean, again, Sam Altman's been posting a lot this week,

Josh:
we do have to tease what's coming because this is probably going to be a huge

Josh:
week. There's a high probability we get GPT-5.

Josh:
And then they've also been talking about their hardware device a little bit. And they're saying how

Josh:
It's like it's genuinely going to change the world. And I believe the reason

Josh:
why is because they're taking this Apple approach where they're building the

Josh:
operating system, they're gathering the context, and then they're just they're

Josh:
able to serve it now locally on device.

Josh:
They're able to go to the cloud when they need more compute.

Josh:
And it's going to create this really cool, I think, duality of AI where you

Josh:
have your your super private local one, and then you have the big brain one,

Josh:
the big brother that's off in the cloud that does all the hard computing for you.

Ejaaz:
Well, one thing is clear. There are going to be hundreds of models and it's

Ejaaz:
going to benefit the user, you and I, for so many multiple...

Ejaaz:
It's the big company's problems to figure out how these models work together

Ejaaz:
and which ones get queried. I don't care.

Ejaaz:
Just give me the good stuff and I'm going to be happy.

Ejaaz:
Folks, OpenAI has been cooking. This was the first open source models they've

Ejaaz:
released in six years, Josh.

Ejaaz:
The last one was 2019 GPT-2, which seems like the stone age and it was only like four years ago.

Ejaaz:
Thank you so much for listening. We are pumped to be talking about GPT-5,

Ejaaz:
which we hope to be released in maybe 24 hours.

Ejaaz:
Hopefully this week, fingers crossed. I don't know, we might be back on this

Ejaaz:
camera pretty soon. Stay tuned.

Ejaaz:
Please like, subscribe, and watch out for all the updates. We're going to release

Ejaaz:
a bunch of clips as well if you want to kind of like get to the juicy bits as well.

Ejaaz:
Share this with your friends and give us feedback. If you want to hear about

Ejaaz:
different things, things that we haven't covered yet or things that we've spoken

Ejaaz:
about, but you want to get more clarity on or guests that you want to join the show, let us know.

Ejaaz:
We're going full force on this and we'll see you on the next one.

Josh:
Sounds good. See you guys soon. Peace.

Music:
Music