Exploring the frontiers of Technology and AI
Josh:
Two years ago, Tim Cook got on stage at Apple's Worldwide Developer Conference
Josh:
and announced what I believe to be the most amazing set of features in the history
Josh:
of Apple, Apple Intelligence.
Josh:
It turned out to be the most disappointing WWDC of all time.
Josh:
It was a complete and total failure, so much so that they are still currently
Josh:
dealing with lawsuits around false advertising for how big of a flop Apple Intelligence was.
Josh:
They promised us the world, they delivered absolutely nothing.
Josh:
In fact, the iPhone that they built for that year in particular for Apple intelligence
Josh:
is not supported by Apple intelligence.
Josh:
This sounds like a disaster, but just yesterday, Apple had an opportunity to
Josh:
fix this wrong, to write the things that they were not able to do the first time.
Josh:
And I got to say, I'm very impressed. I think this year's WWDC left a significantly
Josh:
different taste in my mouth than the last one, because this year was all about AI.
Josh:
And for the first time ever, we have features that are actually going to work.
Ejaaz:
I remember filming last year's WWDC episode and us just being thoroughly disappointed
Ejaaz:
with just the lack of AI stuff. Like they just didn't pay it any kind of attention
Ejaaz:
and they delayed it yet another year.
Ejaaz:
Up until yesterday, actually, Siri couldn't even tell you which month we were
Ejaaz:
in. And that was ironically one of the demos that they showed yesterday.
Ejaaz:
Anyway, a slew of new AI updates.
Ejaaz:
The two flagship headline updates from Apple is Siri is now Siri AI.
Ejaaz:
Powered kind of by Google's Gemini model and also by Apple's own foundational
Ejaaz:
model, which brings me to the second major headline announcement.
Ejaaz:
We've got four to five brand new Apple foundational models, which can run both
Ejaaz:
locally on your device as well as through their own private compute server.
Ejaaz:
And we're going to get into a lot of the details about how this actually works.
Ejaaz:
But my main takeaway from yesterday's WWDC is that Apple is finally taking AI
Ejaaz:
seriously. And while some of the feature updates that we're going to go through
Ejaaz:
may seem like they're kind of basic.
Ejaaz:
Collectively, they actually make your life way, way better. And you listening
Ejaaz:
to this, if you have any kind of an Apple device, an iPad, an iPhone,
Ejaaz:
a MacBook Pro, whatever it might be, you can use a net new feature,
Ejaaz:
once these features actually roll out by the end of the year,
Ejaaz:
which will change the way that
Ejaaz:
you work. And it's done over or across 3.5 billion active Apple devices.
Josh:
So yesterday, they kind of oriented this presentation around three key pillars.
Josh:
The first was stability. The second was kind of how you can use it with your
Josh:
children and security and privacy.
Josh:
The third was AI. We all want to talk about AI. We want to hear about AI.
Josh:
Let's get into the AI first.
Josh:
It's now Siri AI, like you mentioned. And what we're seeing on screen is unique
Josh:
and novel. It's demo that's in real time that actually works.
Josh:
And this was something that I found uniquely interesting for this presentation
Josh:
in general. Generally, when Apple has these things, they have these very pre-produced
Josh:
demos that look very good, that operate very well. When we saw them initially
Josh:
released two years ago, they looked incredible.
Josh:
The difference now is you'll notice that there's no cuts happening in this demo.
Josh:
They're not doing it for speed. They're not doing it for time.
Josh:
They're doing this for proof that it actually works.
Josh:
So what we're seeing on screen is someone from the Apple team who is actually
Josh:
asking it to search his text messages and uncover a text message about a specific
Josh:
song and then go off and play that song.
Josh:
And for the first time ever, Siri has the context. So what we're seeing now
Josh:
is he's showing it a screen from Instagram and Siri can see the content of your screen.
Josh:
It could understand where that location is and then it could route you there on your maps.
Josh:
And this is something unique and novel only to Apple where it has the full permissions of your phone.
Josh:
It could see what's on your device. It can actually engage with your device.
Josh:
And it does everything that I think we imagined and always hoped it would,
Josh:
which is just have full context of your entire life and be able to turn that
Josh:
into some useful information and useful actions.
Josh:
So for me, this was exciting because this was kind of definitive proof that it actually worked.
Josh:
And what we'll notice as we go through the demos from yesterday is that this
Josh:
is actually available for developers right now. So people have been testing it.
Josh:
They have shown that it worked. And there's a lot of cool features that they actually released.
Ejaaz:
I think the concept that we're describing here is basically like grounded context.
Ejaaz:
We've spoken about personal AI agents a lot on this show. And the reason why
Ejaaz:
we've done that is a lot of people think that the best AI features,
Ejaaz:
the best AI models is only achievable if the model is actually larger,
Ejaaz:
the bigger it gets, the more compute that it has.
Ejaaz:
Whereas in reality, you can have a smaller model that is trained on your own
Ejaaz:
personal data that you don't necessarily want to share with ChatGPT or Claude.
Ejaaz:
And it could do this so in such a private way that you can create a much more
Ejaaz:
personalized AI experience.
Ejaaz:
And so some examples that we're about to run through that is summarized by Marques
Ejaaz:
in this tweet is basically, you know, you can message Siri and say,
Ejaaz:
hey, do you remember that restaurant that my friend was talking about about a month and a half ago?
Ejaaz:
I think it was somewhere in my neighborhood and it pulls it up.
Ejaaz:
Or you could say, hey, could you look at my recent location history and let
Ejaaz:
me know what that place was there, saw that beautiful sunset,
Ejaaz:
it and it'll be able to tell you.
Ejaaz:
And it's these little nuances that you can't really achieve in ChatGPT or any
Ejaaz:
kind of LLM chatbot simply because it's not grounded in any particular context.
Ejaaz:
Now, that all sounds great, but let's walk through some examples specifically. So you mentioned that.
Ejaaz:
Siri Air can react to things that are on the screen. This is a classic case
Ejaaz:
where I'm updating my iOS, right?
Ejaaz:
And I don't want to stare at the iOS screen to see when it's finished updating.
Ejaaz:
So I say, hey, Siri, can you set a timer for when this finishes?
Ejaaz:
And it knows what you're talking about.
Ejaaz:
It sees the amount of time remaining and it sets that specific timer,
Ejaaz:
which I thought was pretty cool. And then we've got another example here where
Ejaaz:
it's like, when was my flight last month? And it pulls up your flight manifestor,
Ejaaz:
goes through your email app or your calendar app or whatever that might be.
Ejaaz:
And it says, I think that's your flight to Budapest. It was a British Airways
Ejaaz:
flight. You left at this time and you arrived at this other time.
Ejaaz:
And then there's this other one, which I kind of cracked up at.
Ejaaz:
Where you get a text from someone and it says, you know, have you heard of this plant called Calathea?
Ejaaz:
And it's basically like a tropical house plant. And Siri summarizes it,
Ejaaz:
basically saying the exact same thing that's in the text.
Ejaaz:
Now, the point I'm making here is that it's not exactly perfect.
Ejaaz:
There's some kinks that we need to work out, but I can already see myself using
Ejaaz:
this and it being hugely valuable, even though it's in this V1 format.
Josh:
It's funny, as I was watching this announcement yesterday, I'm sitting there
Josh:
and I'm like holding these two months in parallel. One is that like,
Josh:
oh my God, this is unbelievable. I can't believe Siri works.
Josh:
The other is, okay, so what you're telling me is all this is just to deliver
Josh:
on the promises that you should have made years ago.
Josh:
So it's like, it is this amazing novel experience, but it's like,
Josh:
okay, yeah, they've been working on this for a very long time, we haven't got it.
Josh:
So while we are enthusiastic, it's like finally, thankfully, this actually works.
Josh:
And I think it's important to really understand how big of a deal this is. I think
Josh:
When a lot of people use AI, the thing that they are missing the most is the context.
Josh:
And we have memory when it comes to Anthropic and OpenAI, and you have like
Josh:
a loose memory based on what you've spoken about, but it doesn't have full access to everything.
Josh:
A lot of people, their whole lives live on their devices.
Josh:
It lives on the data on their phone, on their laptop.
Josh:
Apple is the iCloud account that runs all of that. And having the full contextual
Josh:
awareness to go off and then make actions on your behalf is a huge thing.
Josh:
And at the core of this is Siri AI. is Siri that actually works.
Josh:
I mean, for the last, what, five, six years, I've had Siri disabled.
Josh:
It's been worse than useless. In fact, it just gets in the way whenever it hears
Josh:
its name and it gets summoned.
Josh:
Finally, there's an opportunity to turn it back on and start using it.
Josh:
And I think that's a really big deal for, because for those people who don't
Josh:
want to go and subscribe to a frontier model, they don't want to pay $20, $100, $200 a month.
Josh:
They are just interested in personalized AI. This is a huge opportunity, a huge market for Apple.
Josh:
In addition to the developers, I mean, this is a developer conference.
Josh:
Apple developers now have access to these really nice quantized local models
Josh:
that run on your phone that now have the ability to do edge inference on any
Josh:
application that they want.
Josh:
So if I'm an app developer, I'm going to the app store right now because I'm
Josh:
getting AI inference tokens for free that I could deliver near instantly because
Josh:
it's run locally to my customers on my Apple devices.
Josh:
And that's a huge unlock for anybody who's building for the platform anyone
Josh:
who's using the platform and while it did take them a long time it's here and
Josh:
we always talk about apple has an install base of how many tens of billions
Josh:
of devices that now have edge inference ability
Josh:
on device and that's such a huge unlock like i
Josh:
I think this is very much version one now. This is the starting point of this
Josh:
new paradigm of AI first hardware devices.
Ejaaz:
I think there's also some important context to provide. Like if you're an investor
Ejaaz:
and you've been watching Apple over the last couple of years,
Ejaaz:
you just have one clear statement to say about them, which is they just messed up.
Ejaaz:
They completely left the AI race to all the other players.
Ejaaz:
They didn't train their own foundational model and they didn't burn as much
Ejaaz:
capital, but also they got left behind in this entire race.
Ejaaz:
And then you might be asking, well, how have they been able to produce these
Ejaaz:
AI features that are seemingly quite good when you've got Google,
Ejaaz:
Microsoft, and Amazon spending upwards of $260 billion this year alone on AI CapEx?
Ejaaz:
Well, there's two answers to this. One of them you just kind of lightly touched
Ejaaz:
on, which is Apple probably has the largest moat of consumer data of any singular company, right?
Ejaaz:
They have 3.5 billion devices every day collecting all these kinds of personal
Ejaaz:
grounded contacts, which they can use to train a model.
Ejaaz:
When you look at Chinese open source models, a big question that is being asked
Ejaaz:
right now is, how have they been able to catch up with the American Western
Ejaaz:
models that are spending so much money and the Chinese models aren't doing that?
Ejaaz:
The one answer is data. Duwakesh had a really good paper that he released yesterday
Ejaaz:
where he explains that it's primarily data. If you can get access to really
Ejaaz:
personal data, you can train a pretty good model. So that's one way Apple's been able to do it.
Ejaaz:
But the other thing is the elephant in the room, they signed a massive partnership
Ejaaz:
with Google to get access to their Gemini model specifically.
Ejaaz:
Now, the way that people frame this partnership in their head is,
Ejaaz:
oh, they're just going to plug
Ejaaz:
Google's Gemini model into Apple's Siri and that's the brain behind it.
Ejaaz:
And that's partially true. They do use Google's 1.2 trillion parameter model, but it's also not true.
Ejaaz:
Apple has created a series or a slew of different models called Apple foundational
Ejaaz:
models that are versions of models that they've trained by themselves on their
Ejaaz:
own servers, on their own GPUs.
Ejaaz:
At home versus a kind of distilled effect from Gemini or Google's Gemini specifically.
Ejaaz:
There was actually an agreement which got leaked, I think like two weeks ago,
Ejaaz:
which showed that Apple gets access to Google's Gemini model weights,
Ejaaz:
which is just crazy for some kind of proprietary agreement where they pay them
Ejaaz:
a billion dollars a year.
Ejaaz:
So it's this mixed relationship, this mixed partnership, which has allowed them
Ejaaz:
to kind of achieve this particular type of model that we're seeing on screen today.
Josh:
Yeah, and it has some pretty cool technology, how it works. Instead of forcing
Josh:
the entire model into DRAM, which we've talked about many times on the show,
Josh:
if you haven't, go find the episode.
Josh:
The model is actually stored in flash memory. And because the NAND and DRAM
Josh:
bandwidth is too slow to swap weights back and forth, they actually process
Josh:
it on like a per prompt basis. So it's just much faster.
Josh:
It's much more lightweight. It's this kind of novel architecture that they're using.
Josh:
And I imagine a lot of people are going to slowly start to mimic that.
Josh:
And it's also important to note that these models use a novel architecture.
Josh:
They're using DRAM and SRAM to pass things off to each other in ways that are
Josh:
faster and efficient than you traditionally could have.
Josh:
And I think that's why the models aren't going to be running on the iPhone 16
Josh:
Pro, even though they claim that it was built for Apple intelligence,
Josh:
because it really is pushing the limits of the hardware that they have on device.
Josh:
Now, what are the types of things that you can do with that on device intelligence? They're
Josh:
pretty freaking awesome. One of my favorite demos from the whole event that
Josh:
they snuck in with one sentence, it was like three seconds of the whole event,
Josh:
was the ability to split the bill with Siri.
Josh:
So you could actually take a photo of a bill. Let's say you're at a restaurant
Josh:
and you had a receipt that gets dropped on the table.
Josh:
You take a photo, send it to the group chat. Everyone can be assigned what they bought.
Josh:
They could pay with Apple Pay. You're done. You're on your way.
Josh:
It is so clean. It is so cool.
Josh:
There's a second feature that I love, which actually surprised me because it
Josh:
relies on agents to go ahead and do this, is you can fix your password agentically.
Josh:
So inside of Apple, inside of iOS and macOS, there is a passwords app where
Josh:
it'll safely and securely store your passwords for all the accounts that you have.
Josh:
It'll also go out and see if those passwords have been leaked,
Josh:
if they're being reused, if there is any sort of security compromise that's
Josh:
happened to those passwords.
Josh:
And what this new agentic system will do is within the app, it will go out and
Josh:
reset the passwords for you.
Josh:
So if you have an insecure password, if you have a password that's been reused,
Josh:
it'll go off in the background, it'll go into the browser, it'll log in using
Josh:
your credentials, it'll change the password, and then it'll populate the new
Josh:
one into the app without you ever having to do that again. So the days of having
Josh:
to reset your password as you forgot it,
Josh:
Those are in the past. And I think these types of use cases seem silly.
Josh:
They seem small, but they increase the quality of life so much for the day-to-day user.
Josh:
And this is at the core of what they spent almost a third of the show talking
Josh:
about, which is just stability.
Josh:
Like people just want a stable operating system that works and all the features
Josh:
that are advertised worked. And that's what we have. So these two features in
Josh:
particular were very exciting for me.
Ejaaz:
There's something about giving an AI agent access to my passwords that just
Ejaaz:
doesn't sit well with me right now. But maybe I just need to like use the feature
Ejaaz:
and let it change my life.
Josh:
They've been refining this for how many years now that they couldn't figure
Josh:
it out? They've been workshopping this, man.
Ejaaz:
They had all the time. Their timelines are measured in centuries. That's so funny.
Ejaaz:
Kind of similar on like the idea of like how Apple approaches their kind of
Ejaaz:
like design when it comes to AI.
Ejaaz:
We had to like look at the system prompt itself. So with all of these AI models,
Ejaaz:
with all of these AI agents, including Siri. It's not too different from any of the other AI models.
Ejaaz:
It has a base system prompt, which the company, in this case,
Ejaaz:
Apple feeds it in order to kind of like give it its personality and give it
Ejaaz:
understanding of how it's supposed to act.
Ejaaz:
And I'm not going to go through this entire system prompt, but it's just interesting
Ejaaz:
to see that they use visual richness,
Ejaaz:
like quite a few times in this paragraph where they're basically saying,
Ejaaz:
Siri, like you aren't just a chatbot LLM, you visually understand the importance
Ejaaz:
of design in this person's life.
Ejaaz:
And you must make visually rich choices like you must see what they see.
Ejaaz:
You must think how they think and care for them in a particular way,
Ejaaz:
which honestly, I haven't seen in any other kind of system prompt when I look
Ejaaz:
at my Claude or ChatGPT now.
Ejaaz:
I want to spin back to something which kind of gets into the infrastructure
Ejaaz:
of how all of this works because I actually think Apple's made a breakthrough
Ejaaz:
in AI model architecture, which not a lot of other companies have.
Ejaaz:
And we've said on previous shows that Apple has the perfect architecture and
Ejaaz:
kind of device system to run AI models locally and privately,
Ejaaz:
and that gives them a significant advantage in the world or realm of AI agents specifically.
Ejaaz:
Now, I want to briefly go through their foundational models.
Ejaaz:
It's something called Apple Foundational Models, and there are about four or
Ejaaz:
five of them which they release. And there's a voice one, there is an LLM-based
Ejaaz:
specific thing, there's a transcription agent, and there's quite a few others.
Ejaaz:
You have it on our screen here. We've got AFM Cloud, AFM Cloud Image,
Ejaaz:
and there's a few different features that kind of collectively feed into their Siri AI system.
Ejaaz:
But what interested me the most was their 20 billion parameter model.
Ejaaz:
Now, before you get bored, I just want to explain why this is super important.
Ejaaz:
Typically, to run a 20 billion parameter model on your phone,
Ejaaz:
on any kind of Apple device, it would be nigh on impossible because the chips
Ejaaz:
are just not too bleeding edge and it results in a very slow architecture.
Ejaaz:
Now, what they did was something very smart. You mentioned two memory types
Ejaaz:
earlier on, Josh. You've got the NAND flash memory and you've got random access memory, RAM.
Ejaaz:
What they've done is they engineered both of these memories in such a way that
Ejaaz:
you can access a 20 billion parameter model at lightning speed for super cheap
Ejaaz:
costs. This is not something that any other device architecture has been able to achieve.
Ejaaz:
And the only reason why they were able to do it was because of Apple's supply
Ejaaz:
chain. They have one of the biggest and best silicon moats across the world.
Ejaaz:
And they were able to exercise that. And there's like a three tier routing system.
Ejaaz:
So any kind of simple prompts where it's like, hey, can you check my text messages?
Ejaaz:
Can you like tell me what my friend said last week?
Ejaaz:
That runs completely on device, privately encrypted. It never leaves it.
Ejaaz:
It doesn't even need an internet connection at that point.
Ejaaz:
And then any kind of medium-sized request gets routed to their private cloud
Ejaaz:
server where it's still encrypted, but it kind of like routes to another server
Ejaaz:
that's not on your phone.
Ejaaz:
And then for the heavier stuff, it gets sent to Google Cloud specifically,
Ejaaz:
run on NVIDIA GPUs, again, still encrypted. So privacy is at the core of Apple's
Ejaaz:
foundational architecture.
Ejaaz:
And that might sound like an unsexy thing to announce, but it's actually crucially
Ejaaz:
important if you're going to start sharing a lot of your personal information,
Ejaaz:
your financial information, your grocery list, your personal texts with an AI.
Ejaaz:
It's important that that's kept at a privacy level and Apple's really focused
Ejaaz:
on that, which I thought was really cool.
Josh:
Yeah. In fact, at the end of the presentation, Craig Federighi went on stage
Josh:
or he was on camera saying it's actually not even rolling out to the EU just
Josh:
yet because the EU will not allow this amount of encryption on the services that are being offered.
Josh:
They don't allow for a complete and total privacy. And that's actually what
Josh:
Apple is doing. Same thing with China.
Josh:
They're not releasing it in China right now. And they said they're working with
Josh:
legislators to get it over there because they will not compromise on the privacy, on the security.
Josh:
And I think that's a seriously big deal.
Josh:
And when I think about, I recall the bull case or the bear case that we mentioned
Josh:
probably close to a year ago now, which was the ability to have like on-prem
Josh:
compute on the edge inference, like on your local device in a secure and private
Josh:
way with all of your context.
Josh:
And that has not actually been a real consideration because no company has been
Josh:
able to do it. But starting later this year in fall, when iOS 27 is coming out,
Josh:
when all these new updates come out, it's going to be a real thing where
Josh:
If the average person wants to use AI, all they have to do is hold down the
Josh:
power button on their phone and activate Siri.
Josh:
And that gives them enough AI to get done the things that matter to them,
Josh:
to get done all of the contextual tasks, to have small agentic abilities,
Josh:
like going and change your passwords, but also being able to plan a trip.
Josh:
And all of that now is going to live within the Siri application,
Josh:
which is now a standalone app
Josh:
on your phone, similar to what you're used to with ChatGPT, with Claude,
Josh:
with Gemini, where you start a conversation, you continue a conversation,
Josh:
except you never have to train this AI on memories about you.
Josh:
It has all of the memories.
Josh:
It's totally private. You can trust it with your data. Your data is already
Josh:
there. It's an incredibly compelling offering if you are AI curious and you're
Josh:
not a power user who pays 20, 100, $200 for these premium AI services.
Josh:
And I think for a lot of people, that's gonna be enough.
Josh:
So we're gonna see what the implications of that are as this gets rolled out
Josh:
through the end of the year and it's next year in the sense that is this actually
Josh:
going to make a dent in these subscriptions of the larger AI labs because people,
Josh:
it turns out, they don't actually care too much for frontier intelligence.
Josh:
They don't need to go solve biology problems or chemistry problems.
Josh:
They just want to be able to plan their calendar, schedule their grocery delivery
Josh:
automatically, and make sure it's all managed internally by their AI.
Josh:
I mean, it's something I think a lot of people are going to be very intrigued to want to use.
Ejaaz:
Yeah, if you're looking at this just from like an investor perspective,
Ejaaz:
this could be a real problem if Apple actually scales this up, right?
Ejaaz:
Because it proves that like small models work and you don't need a large model
Ejaaz:
and also the cheaper less frontier models are just sufficient enough for people now um,
Ejaaz:
With the concept of subscriptions versus paper usage, I believe Apple is just
Ejaaz:
letting people use their inference and AI features for free.
Ejaaz:
But there's a daily usage limit or like a paper usage limit that might come
Ejaaz:
into effect after you've hit
Ejaaz:
a particular limit. What those tier packages look like, I have no idea.
Ejaaz:
I think it would be crazy if Apple subsidized 3.5 billion active Apple devices.
Ejaaz:
But I think we're going to start to see some kind of a subscription package
Ejaaz:
where it's focused on giving like 80% of Frontier intelligence for free.
Ejaaz:
And then like, if you want anything after that, you can kind of like pay,
Ejaaz:
you know, on a usage based type thing.
Ejaaz:
Now, I have one qualm with all of this, Josh, which is,
Ejaaz:
if you bought a very expensive iPhone 16 a few years ago, which they marketed
Ejaaz:
and advertised as built from the ground up for AI, you can't run like 75% of,
Ejaaz:
of the AI features that they announced yesterday.
Ejaaz:
You need something of the iPhone 17 or 18 and above because of the chip device
Ejaaz:
architecture that I was mentioning earlier.
Ejaaz:
So this is something that I guess, like, is not the greatest thing.
Ejaaz:
It might be another false advertising issue, but, you know, if you had a two-year-old,
Ejaaz:
device that you bought from Apple and you're thinking like, hey,
Ejaaz:
I wanna run a bunch of these features that are rolling out by the end of the
Ejaaz:
year, you can't actually do that.
Josh:
Yeah, it's a bummer. I mean, Apple messed this up a couple of times over.
Josh:
And also a note on the usage limits.
Josh:
Craig actually, during the presentation yesterday, spoke about them briefly,
Josh:
where if you're using local inference on device, you have unlimited infinite.
Josh:
If you're querying the private cloud server, you get a certain amount.
Josh:
The overflow will be directed through your iCloud account, your iCloud service package.
Josh:
So if you're paying $5 a month for two terabytes of storage or whatever backup,
Josh:
that will be enough to kind of absorb the extra queries that you have.
Josh:
So it's already baked into where you are. There's no separate subscriptions.
Josh:
It's all under one roof in terms of this.
Josh:
Yeah, I mean, it sucks. Last year, everyone was buying the new iPhone 16 built
Josh:
for Apple intelligence.
Josh:
Apple intelligence didn't exist.
Josh:
Now that it does, the phone is not compatible. It sucks. It's a bummer.
Josh:
And it's particularly a bummer because of some of these additional features
Josh:
that I do have to share in the photos app. The photos app is getting a really amazing upgrade.
Josh:
And as I was watching this, I have like the ghost of see if I was past perpetually in my head.
Josh:
And I know that he is sitting here cringing because so much of the art form
Josh:
of photography is the real raw and natural way that it comes out,
Josh:
Not the digitized version.
Josh:
What they've done is they've, they've stepped it up. They've taken it all the
Josh:
way to the furthest degree in three features that they announced cleanup,
Josh:
reframe and extend. These are the three core pillars of your new photo app.
Josh:
If you take photos, if you own an iPhone, you're going to love this.
Josh:
So the cleanup feature is basically what we're seeing on screen here is an original
Josh:
photo of someone holding a card in front of their face.
Josh:
In iOS 26, it lacked the contextual awareness to understand what your face actually
Josh:
looked like. So it turned you into a blob. You kind of had like a fudgy nose
Josh:
with like really mixed up eyes.
Josh:
IOS 27 solves this. It applies real intelligence to these photos.
Josh:
It really makes it look like a big deal. Reframe is the second feature.
Josh:
I think reframe is probably the one that I'm most excited about.
Josh:
Reframe will take a photo that you've shot that is framed pretty poorly if you
Josh:
didn't frame it well if it didn't look so good on the first version what it
Josh:
does is it turns it into a splat we're seeing on screen it turns into a 3d splat which basically
Josh:
adjusts and adds layers to the image so there's real depth it allows you to
Josh:
zoom in pan tilt and move the image and then any parts the images that are lost
Josh:
it will actually generate in the background afterwards so you can very quickly
Josh:
reframe it and then it will generate afterwards in your camera roll and it leaves
Josh:
you with really compelling results where, I mean, what we're seeing on screen
Josh:
is this is a totally different image.
Ejaaz:
I'm looking at the background, Josh. Can you see it? Like I'm looking at this
Ejaaz:
person's face over here and I'm wondering whether it gets distorted a bit in
Ejaaz:
the background, but it is a very cool feature. No, it's cool.
Josh:
Yeah. It's amazing. So that's the second one, reframe. The third is something
Josh:
that's really cool that we've, I mean, granted we've had these for a while.
Josh:
We haven't had reframe, but we've had extend for a while in Photoshop,
Josh:
but extend is now making its way to iOS as well. So if you've taken a photo
Josh:
and you need to fill the edges, if you want to remove something,
Josh:
if you need to kind of zoom out, this is what Xtend will do.
Josh:
It will take an image, it will build artificial borders around it that look
Josh:
real, that are artificially generated, but are meant to be hyper-realistic.
Josh:
And this is all leaning on the visual AI local model that runs on your device.
Josh:
You can see on screen, it just generated an entire coffee shop around a uptight,
Josh:
up-close shot of coffee.
Josh:
So it's really impressive. I think a lot of people are going to use these.
Josh:
A lot of people are going to love these features.
Josh:
The example they used inside of the demo on the actual presentation was just
Josh:
photos of your loved ones if you take poor photos that you don't want to lose
Josh:
that moment you can actually manipulate the moment in such a way that
Josh:
it still maintains the integrity of it but captures it in a way that's much
Josh:
more just delightful to see so this in particular was something i'm really excited
Josh:
to use as someone who loves the phones for the photo features having these tools
Josh:
natively built into photos app two thumbs up
Ejaaz:
Okay, so when I was a kid, one of the things I enjoyed doing,
Ejaaz:
because I was such a cool guy, and I had lots of cool friends to hang out with,
Ejaaz:
was play around in the virtual simulator. So I would like, fly around the world
Ejaaz:
and be able to like, see the world for what it is.
Ejaaz:
It looked very much like how Minecraft looked today. And what Apple released
Ejaaz:
yesterday was something pretty
Ejaaz:
cool, which is that 3d Gaussian splat version for Apple Maps itself.
Josh:
And I mean, so sick,
Ejaaz:
Like, I don't need to do I don't need to say anything. Just take a look at this video.
Ejaaz:
This looks so hyper-realistic the trees don't look like broccolis anymore in fact like
Ejaaz:
if you weren't right if you didn't know this was a gaussian spot you would just
Ejaaz:
assume this was like an aerial shot drone video of like your city of new york
Ejaaz:
or wherever you might be so the purpose of this i guess is to make uh travel
Ejaaz:
or navigation way more interactive
Ejaaz:
and way more intuitive for you like maybe you're like looking for a particular
Ejaaz:
shop front but you don't know what it looks like it can like point out like,
Ejaaz:
hey, this is where you are and you can kind of recognize your surroundings around you.
Ejaaz:
Just a really cool feature and use of Gaussian spots.
Ejaaz:
I know that we've been wanting to make an episode on this for a while at Limitless.
Ejaaz:
And some of the examples that we've shown in the past has been primarily focused
Ejaaz:
on Hollywood studio effects and recreating movies from scratch for a couple
Ejaaz:
hundred grand versus millions of dollars.
Ejaaz:
This is a real practical application that will be live by the end of the year
Ejaaz:
in 3.5 billion Apple active devices, which will be super cool to see.
Josh:
Yeah. And we got to put on our conspiracy hats a little bit here because why
Josh:
is it that we only talk about Gaussian splats, 3D splatting when we mentioned Apple?
Josh:
Why is it that they're the only company doing this? And I think a lot of what
Josh:
happened at WWDC this year was just returning to their roots,
Josh:
patching all their bugs, building and establishing a new foundation in which
Josh:
they could then launch the new hardware architecture on top of.
Josh:
John Ternus, the new CEO, this sadly was tim cook's last wwdc
Josh:
after this john turnus steps in he's a hard work guy what tim cook is leaving
Josh:
him is this really strong foundation to then build this next level of hardware
Josh:
on so when we look at these splats
Josh:
how accurate they are when we look at the photo app how it's so good at understanding
Josh:
three dimensions and 3d splatting a lot of this is the infrastructure and the
Josh:
foundation for the next suite of hardware devices for the glasses that they
Josh:
are going to be working on
Josh:
and you'll note that there was a lot of rumors over the past couple of weeks that
Josh:
apple has fully scrapped to their Vision Pro program.
Josh:
The large goggles that everyone loves, I mean, that I love that were largely
Josh:
considered a flop, but I think were the most impressive piece of hardware ever.
Josh:
They're no longer on the upgrade path. They're not going to continue to iterate on those.
Josh:
They're going to build glasses instead. What are those glasses going to rely
Josh:
on? A lot of things like this. Splats. Why?
Josh:
Because splats are very low bandwidth, high fidelity versions of reality that you can interact with.
Josh:
And when you're trying to run these things locally on device,
Josh:
on something as small as glasses, it's really important to have that architecture in place.
Josh:
And what we're seeing here is early tastes of what these future devices are
Josh:
going to look like deployed as really fun products. So we have Google Maps that
Josh:
has really great splats.
Josh:
Why are they doing that? Because they're improving their splat technology for
Josh:
this next generation of hardware.
Josh:
And I think that's really important to note. Another thing that was really exciting
Josh:
is, or maybe not exciting for some,
Josh:
Is that they didn't actually capture any surface area. A lot of times at WWDC,
Josh:
we make the joke of like, what startup are they going to kill this week?
Josh:
Who are they going to like destroy, knock out of business because they replaced
Josh:
their feature? And the answer today was nobody.
Josh:
There's really no novel new things that they were trying to replace.
Josh:
They were just going back to the basics. They're making everything that should
Josh:
have worked in the past work currently, for example.
Josh:
On my Mac, when I press the search bar and I go to search for something,
Josh:
it takes 10, 15 seconds to populate.
Josh:
If I'm searching my email in the mail app, That search bar sucks,
Josh:
dude. I can't find any emails that I want. It's so bad. So they spent this WWDC
Josh:
really just fixing everything, building the foundation for this next frontier of hardware.
Josh:
And I see on screen, you're teasing a resizing feature that also infers there's
Josh:
another piece of hardware coming separate from these glasses that we could expect pretty soon.
Ejaaz:
Yeah, so for a lot of people seeing this demo on screen right now,
Ejaaz:
they might just kind of like look past and be like, okay, whatever, who cares?
Ejaaz:
What we're seeing on the screen is basically the UI or the user interface being
Ejaaz:
resized in real time, flipping between an iPhone orientation to an iPad orientation
Ejaaz:
to maybe even a full MacBook Pro screen sizing.
Ejaaz:
The reason for this is they're reformatting the code, Xcode specifically,
Ejaaz:
for their rumored new device that is going to be a foldable phone.
Ejaaz:
So you should be able to flip between a regular iPhone-sized screen and an iPad-sized screen.
Ejaaz:
And so Apple is really prepping not just their software, but for their future
Ejaaz:
of hardware, which is going to look very much like the new world of AI or for a new era of AI.
Ejaaz:
And it's going to involve a lot, not just like screen and visual outputs,
Ejaaz:
but it's going to involve a lot of transcriptive stuff.
Ejaaz:
It's going to involve a lot of voice-based AIs, as we've seen with the Apple foundational model.
Ejaaz:
So I think the way, if I had to summarize WWDC of this year,
Ejaaz:
I would say that Apple is laying the foundation for taking AI a lot more seriously going forward.
Ejaaz:
And I think with John Turnus at the helm, hardware is going to be at the forefront
Ejaaz:
of their focus. I think they have the foundational elements.
Ejaaz:
They have the Apple Vision Pro, which was just kind of too expensive and not
Ejaaz:
accessible to enough people.
Ejaaz:
They're rumored to be working on a new set of glasses, which has been derived
Ejaaz:
from the hardware that they built using Apple Vision Pro.
Ejaaz:
They're rebuilding their entire iOS system. They've got Siri as their main flagship
Ejaaz:
AI system, which, by the way, doesn't need to use Google Gemini models in the future.
Ejaaz:
It could technically be an orchestrator. That's how they described it on yesterday's
Ejaaz:
event, that it could be an orchestrator of different models.
Ejaaz:
It could be Apple's own foundational models. It could be ChatGPT in the future.
Ejaaz:
It could be Claude in the future. The point is, if Siri AI is the funnel,
Ejaaz:
then Apple owns the entire distribution.
Ejaaz:
And that alone is the most important or rather bullish investment case for Apple alone.
Ejaaz:
Now, despite this, the announcement yesterday of WWDC sent Apple stock cratering
Ejaaz:
around 6%. So I think that was a classic case of buy the rumor, sell the news.
Ejaaz:
But I do think if you're taking a long-term perspective on Apple,
Ejaaz:
at least for me specifically, I'm incredibly optimistic about where they're
Ejaaz:
going to take this simply because they have one moat.
Ejaaz:
OpenAI and Anthropic don't have, which is just so much user data and intents
Ejaaz:
and the hardware to be able to distribute it on.
Ejaaz:
OpenAI is working on their own AI device. They haven't released it yet.
Ejaaz:
And once they do, it's going to be really hard to scale.
Ejaaz:
Anthropic, for all I know, isn't working on a hardware device yet.
Ejaaz:
Maybe they will in the future.
Ejaaz:
But in order to ramp up and compete with Apple, the only other company that can do that is Google.
Ejaaz:
And they're kind of sequestering their own AI model to Apple to use for a billion
Ejaaz:
dollars a year and giving them access to model weight.
Ejaaz:
So I think if you're looking at Apple as an investment case,
Ejaaz:
if you're looking at Apple and thinking, how will they fare in the future for
Ejaaz:
AI? I think they're going for the consumer AI mode specifically.
Ejaaz:
And I think they're really well positioned to do so.
Josh:
Yeah, I think sentiment for me personally, at least, changed in a meaningful way this time around.
Josh:
There was no flashiness, there was no like selling the magic, which is a bummer.
Josh:
But that's not what this needs to be. This needs to be stability,
Josh:
returning to the roots, building that strong foundation, establishing a place
Josh:
in which they can build their next form factor of hardware on top of and in
Josh:
there was very much a mission success and what a gift this is to john turnus
Josh:
the new ceo from tim cook to hand off a
Josh:
feature complete software stack ready for the next generation of hardware which
Josh:
he specializes in and we just need to take a moment to appreciate tim cook one
Josh:
more time because this is the last event that he's going to be doing um
Josh:
The next hardware event is going to be in september that is going to be for the iPhones.
Josh:
That's when John is going to be starting his journey as CEO of Apple.
Josh:
And Tim did the impossible thing of taking over a company from what a lot of
Josh:
people deem as the most incredible entrepreneur of all time,
Josh:
Steve Jobs. And he did that 15 years ago in 2011.
Josh:
And over that 15-year run, he managed to take Apple on a, I think, 2,200% growth rate,
Josh:
of a stock that was impossibly challenging to scale to follow up the most challenging
Josh:
CEO role of all time. And he's just a superstar. He just did a really great job.
Josh:
He managed to steer Apple away from a lot of the drama that could have got them in trouble.
Josh:
Although like the company, they did lack magic during this time.
Josh:
They didn't have any incredible products or hardware.
Josh:
They did the basics right. And still to this day, 15 years later,
Josh:
I am using exclusively Apple products for everything that I do.
Josh:
And outside of the software that they just fixed, I have no complaints.
Josh:
So it was a pretty incredible run by Tim Cook. Just got to give him some props for what he did.
Josh:
And now I'm just really excited for the next generation of Apple.
Josh:
This very much feels like a fresh slate.
Josh:
Tim Cook handed off a really strong company to John.
Josh:
John is now going to take it forward with hardware. And personally,
Josh:
this is what I'm most excited about. We have OpenAI with their AI First device.
Josh:
Apple has a chance to get in the game. They're going to be doing it.
Josh:
And the next couple of years are going to be awesome. We have the 20th anniversary
Josh:
of the iPhone next year. We have a foldable this year.
Josh:
We have the glasses coming probably next year, 2028.
Josh:
I mean, this is a huge, fully stacked lineup. and Apple's got a shot at this thing now.
Ejaaz:
I think something coming potentially sooner than that will also be like their
Ejaaz:
earbuds that have all the airports that have the cameras and they've got some
Ejaaz:
kind of other like pendant type device. Probably get it this fall.
Ejaaz:
All rumors, but maybe this fall.
Ejaaz:
And so I'm excited to see that go head to head with whatever open AI devices.
Ejaaz:
But that brings us to the end of the episode. That is everything and anything
Ejaaz:
you need to know about WODC.
Ejaaz:
Very AI focused, which is what we like on this show.
Ejaaz:
And thankfully things that we can actually use today. If you're a developer,
Ejaaz:
I don't know if you mentioned this earlier, but you can get access to this for about 100 bucks.
Ejaaz:
It might be a little tricky, too. You need to get that membership, but it is accessible.
Ejaaz:
If not, these features will basically roll out by the end of the year.
Ejaaz:
So lots coming by the end of the year. We've got all these IPOs.
Ejaaz:
We've got all these new AI features, all these new rumored AI devices.
Ejaaz:
It's going to be a big, big rest of the year.
Ejaaz:
But that is it for this episode. Josh, any final thoughts?
Josh:
Yeah, no, if you're a developer, if you have that license to go and download
Josh:
iOS 27, you probably already know this, but don't download this on your personal devices.
Josh:
The beta versions of all of these are notorious battery drainers.
Josh:
They will destroy your phones. They're not meant for public use.
Josh:
They're meant for developer use.
Josh:
But if you do have it and you do install it on a second device,
Josh:
let us know what you think. I'd be so curious to hear the reviews.
Josh:
I've been on X scrolling, seeing all the features. I'm curious also just to know,
Josh:
your thoughts on this. Are we, is Apple back? Are we forgiving Apple or can
Josh:
we not forgive them for the fact that if your phone is one year old,
Josh:
you can't even run the front tier models on your device.
Josh:
I understand both sides. I understand both sides. They're both right,
Josh:
but I want to hear which one you stand on. And yeah, that's,
Josh:
that's the update. You are now fully up to date on WWDC.
Josh:
We have a very exciting week of content coming, so please stick around for that.
Josh:
But without any further ado, I think that wraps up. Thank you guys so much for
Josh:
watching and we will see you in the next one.
Ejaaz:
See you guys.