1
00:00:00,000 --> 00:00:02,823
Hey, welcome to another episode of EmpowerApps.

2
00:00:02,903 --> 00:00:08,148
I'm your host, Leo Dion. I'm joined once
again by the wonderful Daniel Steinberg.

3
00:00:08,349 --> 00:00:10,711
Daniel, it's good to see you again.

4
00:00:11,085 --> 00:00:12,886
It's great to be back. Thanks for having
me.

5
00:00:13,992 --> 00:00:18,285
I'm super excited to talk about our topic
today 'cause I think there's

6
00:00:18,285 --> 00:00:22,098
a lot of great value in it. and I know a
lot of people are taking advantage of it.

7
00:00:22,108 --> 00:00:23,989
but of course we have to do the intro.

8
00:00:24,429 --> 00:00:27,241
So for those who don't know who Dim Sum
Thinking is,

9
00:00:27,241 --> 00:00:29,422
I'll let you go ahead and introduce
yourself.

10
00:00:30,190 --> 00:00:35,454
Wow. so I've been around the Mac for a
long time and I write books and teach

11
00:00:35,515 --> 00:00:38,890
and I contract and I just have a ball with
this community.

12
00:00:40,248 --> 00:00:45,732
Awesome. So a lot has changed since the
last time you were on.

13
00:00:46,253 --> 00:00:49,680
I don't know if you've heard I probably
use this joke too many times now,

14
00:00:49,680 --> 00:00:51,491
but like there's this AI thing now.

15
00:00:52,733 --> 00:00:57,525
where where do you stand when it comes to
AI and where it's useful and where it's

16
00:00:57,705 --> 00:00:58,705
not useful?

17
00:00:59,928 --> 00:01:02,780
So I used to teach classes with Matt
Drance,

18
00:01:02,820 --> 00:01:06,001
and Matt had my favorite description of
new technology.

19
00:01:06,401 --> 00:01:10,523
And it was stage one is new technology
comes around and you go,

20
00:01:11,093 --> 00:01:13,424
my God, I hate this. I'm never going to
use this.

21
00:01:14,544 --> 00:01:16,985
And then stage two, it solves a problem
for you,

22
00:01:16,985 --> 00:01:18,726
and you go, my God, I love this.

23
00:01:18,726 --> 00:01:19,847
I'm going to use it everywhere.

24
00:01:20,687 --> 00:01:22,027
And then you come to your senses,

25
00:01:22,027 --> 00:01:24,568
and stage three is I'll use it where
appropriate.

26
00:01:25,209 --> 00:01:28,270
And I'm trying to be in stage three.

27
00:01:28,995 --> 00:01:33,038
But people who are in stage two think I'm
in stage one and I just hate it.

28
00:01:33,258 --> 00:01:35,220
So I try to use it when it's appropriate.

29
00:01:35,680 --> 00:01:41,245
So for example, recently I wanted to
change a Docker image so I could

30
00:01:41,245 --> 00:01:43,887
use it in Apple's containers for Jekyll.

31
00:01:44,968 --> 00:01:48,830
And so I asked Claude, hey, write me this
thing.

32
00:01:49,671 --> 00:01:52,353
And it wrote me this thing, and then it
wrote me more,

33
00:01:52,393 --> 00:01:54,975
and then it wrote me more, and it was
awful.

34
00:01:56,096 --> 00:01:58,418
And then I realized all I really need

35
00:01:58,658 --> 00:02:03,442
really needed to do was build the gem file
lock for me and then I could do the rest.

36
00:02:04,243 --> 00:02:07,105
So I told this to a friend and he said,

37
00:02:07,526 --> 00:02:11,239
you're getting to the the right point
where don't ask it to do the high level

38
00:02:11,239 --> 00:02:15,392
do everything for me. Figure out that
really specific task you need from

39
00:02:15,392 --> 00:02:19,636
it and ask it to do that. And this way I
didn't have to install Ruby and a bunch

40
00:02:19,636 --> 00:02:21,558
of gems to get the thing that I wanted.

41
00:02:21,758 --> 00:02:25,431
So I guess that's where I am right now
where I'm trying to find out where's that

42
00:02:25,431 --> 00:02:26,431
sweet spot for me.

43
00:02:27,032 --> 00:02:29,864
But I don't want it to write code for me
in general.

44
00:02:30,285 --> 00:02:33,367
And I don't want it to write prose for me
because for me,

45
00:02:33,707 --> 00:02:36,049
that's where I learn. When I struggle with
things,

46
00:02:36,049 --> 00:02:37,430
that's how I learn and I get better.

47
00:02:37,430 --> 00:02:41,973
And I don't have a gig where I have to
generate thousands of lines of code a day.

48
00:02:42,093 --> 00:02:45,665
I'm trying to write things and learn from
and write things that I'm proud of.

49
00:02:45,665 --> 00:02:46,966
Does that make sense?

50
00:02:47,707 --> 00:02:53,019
Mm. Yeah. So would you say you use it
every day for small little things still?

51
00:02:52,104 --> 00:02:53,805
No. No.

52
00:02:53,280 --> 00:02:53,885
Okay.

53
00:02:53,885 --> 00:02:55,646
I use it as little as possible,

54
00:02:55,887 --> 00:02:56,647
but when I

55
00:02:56,218 --> 00:02:56,647
Okay.

56
00:02:56,647 --> 00:03:01,650
need it, I reach for it. So I I have it
turned off in Xcode because I found

57
00:03:02,411 --> 00:03:07,404
A it was getting in my way and before this
year it was suggesting APIs that didn't

58
00:03:07,404 --> 00:03:10,066
even exist. So at least it's not doing
that anymore.

59
00:03:10,841 --> 00:03:14,666
Yeah. Did you ever get that like when I do
watch or Mac development,

60
00:03:14,666 --> 00:03:16,468
it'll just it used to do this,

61
00:03:16,468 --> 00:03:20,533
it doesn't do it anymore, but it'll
hallucinate an API for watch

62
00:03:20,533 --> 00:03:24,808
or Mac because it thinks. It sees it for
iOS and it figures out we'll just change

63
00:03:24,808 --> 00:03:29,294
the next or watch kit prefix at the end
and there you go.

64
00:03:28,884 --> 00:03:29,465
Or I'll type

65
00:03:29,465 --> 00:03:34,344
in a method and it'll try to fill in the
parameters and there's there's no API with

66
00:03:34,344 --> 00:03:35,344
those parameters.

67
00:03:35,458 --> 00:03:40,953
Yeah, yeah, exactly. so now that we've
gotten that out of the way,

68
00:03:41,688 --> 00:03:44,577
how how we're legally obligated to ask
that question.

69
00:03:45,518 --> 00:03:51,243
How has has foundation models changed
anything in your perspective on where it's

70
00:03:51,243 --> 00:03:52,384
useful and where it's not?

71
00:03:53,335 --> 00:03:58,486
Yeah, so what I said was I don't want to
use AI to write and generate code

72
00:03:58,486 --> 00:04:00,267
or writing or things I have to think
about.

73
00:04:00,807 --> 00:04:03,708
But that doesn't mean it's not the right
tool to use,

74
00:04:03,708 --> 00:04:08,309
say, to summarize stuff. I have a friend
who's a speaker who will have

75
00:04:08,325 --> 00:04:11,506
one of the AI guys summarize what his
talks about,

76
00:04:11,506 --> 00:04:14,487
and if their summary doesn't match what he
thinks he's talking about,

77
00:04:14,867 --> 00:04:17,507
he needs to adjust. So it's a great tool
for that.

78
00:04:17,848 --> 00:04:20,458
It's a great tool for going in in and
finding

79
00:04:21,108 --> 00:04:23,089
excuse me, finding a resource.

80
00:04:23,109 --> 00:04:28,500
so, you know, you you're researching
something or something that's Siri AI,

81
00:04:28,500 --> 00:04:31,181
the new Siri I think is so much better
than the old Siri.

82
00:04:31,181 --> 00:04:32,071
So I can ask it,

83
00:04:31,839 --> 00:04:32,632
Agreed.

84
00:04:32,632 --> 00:04:39,344
hey, can you ask Apple Music to suggest an
artist like Miles Davis or in the vein

85
00:04:39,344 --> 00:04:45,827
of Miles Davis? So there are great uses
for it and incorporating that into apps

86
00:04:45,827 --> 00:04:46,827
is is really nice.

87
00:04:48,142 --> 00:04:52,528
So total aside question. Did you try or
have you tried the thing in Apple Music

88
00:04:52,528 --> 00:04:55,853
where you're like, create me a playlist
based on a prompt?

89
00:04:57,357 --> 00:05:01,339
So I'm about to experiment with that
because I wanna update so I have this free

90
00:05:01,339 --> 00:05:04,890
music app, so I'm not advertising anything
because I don't charge for anything,

91
00:05:04,900 --> 00:05:08,662
called Top Tracks, where we build top
forty radio stations out of Apple

92
00:05:08,662 --> 00:05:10,512
playlists. And so I wanna try

93
00:05:10,032 --> 00:05:10,512
Okay.

94
00:05:10,512 --> 00:05:14,834
that so I can say, Hey, can you suggest a
playlist based on the things I tend

95
00:05:14,834 --> 00:05:17,135
to listen to or this this other playlist?

96
00:05:17,861 --> 00:05:21,433
Yeah, w when I tried it a few months ago
was it was not very good.

97
00:05:21,814 --> 00:05:23,473
So hopefully they've improved on that.

98
00:05:23,473 --> 00:05:26,287
I need to try it again. so yeah,

99
00:05:26,287 --> 00:05:33,002
let's let's get into it. Let's start off
by just asking what what should developers

100
00:05:33,042 --> 00:05:37,705
know before getting started, especially
those who've used these like AI tools.

101
00:05:38,205 --> 00:05:42,338
Like what what should they know about how
foundation models work and what they

102
00:05:42,338 --> 00:05:45,340
can get out of it? I think that's right.

103
00:05:44,423 --> 00:05:45,343
So first they should

104
00:05:45,343 --> 00:05:49,505
just they should play. And one of the
really nice things is #Playground

105
00:05:49,505 --> 00:05:54,797
makes that really easy. So #Playground is something that you can have

106
00:05:54,797 --> 00:05:57,859
something that fails inside of it or is
async inside of it.

107
00:05:58,179 --> 00:06:02,781
So it's a great tool for just popping a
page and testing your prompt,

108
00:06:02,781 --> 00:06:05,582
your instructions, and seeing what kind of
things you get back.

109
00:06:06,082 --> 00:06:08,977
Also, I'm not as big on the command line
tools,

110
00:06:08,977 --> 00:06:12,629
but you can use FM from the command line
and experiment that way.

111
00:06:13,027 --> 00:06:15,240
So the first thing is just to get in there
and experiment,

112
00:06:15,561 --> 00:06:18,175
is it meeting the use cases that you want
for your app?

113
00:06:19,287 --> 00:06:23,492
Okay, so th is that just like a regular
playground in Xcode I can play around

114
00:06:23,492 --> 00:06:24,492
with?

115
00:06:25,941 --> 00:06:28,203
It's it's not. it it

116
00:06:27,725 --> 00:06:28,203
Okay.

117
00:06:28,203 --> 00:06:31,848
feels more like the Swift equivalent of
preview.

118
00:06:32,389 --> 00:06:33,700
And so use preview

119
00:06:32,934 --> 00:06:33,700
Okay.

120
00:06:33,700 --> 00:06:36,293
and you experiment with SwiftUI code and
you see it.

121
00:06:36,834 --> 00:06:39,737
So #Playground feels like the the
Swift version of that.

122
00:06:41,710 --> 00:06:46,138
Okay. Okay. I do I just do I but

123
00:06:43,291 --> 00:06:46,352
What's nice is I believe I don't know if
they're there

124
00:06:46,352 --> 00:06:49,804
yet, because certainly Foundation Model
isn't cross-platform,

125
00:06:49,884 --> 00:06:51,535
but I believe #Playground,

126
00:06:51,535 --> 00:06:53,746
they're working towards that being
cross-platform.

127
00:06:53,966 --> 00:06:55,046
And so it's a great

128
00:06:54,639 --> 00:06:55,047
Okay.

129
00:06:55,047 --> 00:06:58,568
experiment. I I liked Xcode Playgrounds.

130
00:06:58,568 --> 00:07:02,770
I never liked the the more broad ones that
you could use on iOS.

131
00:07:02,770 --> 00:07:06,922
I th I thought you know, I wanted more of
a Jupyter notebook type of thing

132
00:07:06,922 --> 00:07:08,143
or or something more

133
00:07:07,604 --> 00:07:08,143
Yep, yep.

134
00:07:08,143 --> 00:07:09,143
full.

135
00:07:09,245 --> 00:07:11,127
#Playground is a nice in between.

136
00:07:11,127 --> 00:07:14,971
You can put it in any dot Swift file and
you can iterate.

137
00:07:15,212 --> 00:07:20,487
It's great for algorithms. It's that with
with unit testing is just a a great tool.

138
00:07:22,021 --> 00:07:27,164
So you'd have to like go into an actual
project and like add a Swift file and then

139
00:07:27,164 --> 00:07:29,530
add the playground in there, essentially.

140
00:07:29,193 --> 00:07:29,554
I don't know

141
00:07:29,554 --> 00:07:31,983
if you can do it in a standalone dot swift
file.

142
00:07:31,983 --> 00:07:33,697
I don't I haven't tried it, but perhap

143
00:07:33,061 --> 00:07:33,697
Okay.

144
00:07:33,697 --> 00:07:34,697
perhaps. I don't know.

145
00:07:35,203 --> 00:07:38,426
Yeah, we'll see. So,

146
00:07:38,807 --> 00:07:42,711
you know, there was a lot of improvements
in OS 27,

147
00:07:43,313 --> 00:07:49,799
over twenty six. what what did that get
you that OS twenty six didn't get?

148
00:07:51,051 --> 00:07:56,185
So our first iteration of foundation
models was in 26.

149
00:07:56,785 --> 00:07:58,967
And I like that you're saying OS 26,

150
00:07:58,967 --> 00:08:00,868
because that's sort of iOS, macOS.

151
00:08:01,068 --> 00:08:02,569
It's it's it's all the OSs.

152
00:08:01,553 --> 00:08:02,553
Right, right.

153
00:08:03,511 --> 00:08:05,732
and there were some problems with it,

154
00:08:06,553 --> 00:08:11,356
but I'm okay with that. I I like that
we're being exposed to it and we're seeing

155
00:08:11,356 --> 00:08:15,879
as it evolves. So some of the things are
in OS 27,

156
00:08:15,879 --> 00:08:17,450
some of the things they introduced were so
that.

157
00:08:18,200 --> 00:08:20,501
The improvements don't have to wait a full
year.

158
00:08:20,902 --> 00:08:23,843
Another thing that I think is a little
confusing to people,

159
00:08:24,184 --> 00:08:27,255
it happened in OS twenty six four,

160
00:08:27,255 --> 00:08:29,827
I want to say, when they updated the
model,

161
00:08:30,347 --> 00:08:33,299
that the model lives on the OS.

162
00:08:33,299 --> 00:08:35,549
So if you went to Xcode something,

163
00:08:36,210 --> 00:08:41,193
but your Mac wasn't updated, you weren't
using compatible models and things failed.

164
00:08:41,393 --> 00:08:45,856
And so something you get you get your head
around is not even in the simulator.

165
00:08:46,050 --> 00:08:48,421
When you're doing the iOS simulator,

166
00:08:48,822 --> 00:08:51,883
it still depends on the version of it in
the Mac.

167
00:08:51,883 --> 00:08:55,245
So that's that's right.

168
00:08:52,806 --> 00:08:55,387
In the host system. Yeah, okay.

169
00:08:55,245 --> 00:08:55,495
So

170
00:08:55,495 --> 00:08:59,227
one thing you have to get used to is is
that that and I don't think that's very

171
00:08:59,227 --> 00:09:01,929
well documented, that that can trip you up
easily.

172
00:09:02,409 --> 00:09:07,572
excuse me. Other things that we got in 27
was a better on-device model.

173
00:09:07,732 --> 00:09:09,553
We got the Private Cloud Compute,

174
00:09:09,873 --> 00:09:13,255
we got some tools that allow you to do
interesting things with images.

175
00:09:13,559 --> 00:09:15,760
We got the whole evaluations framework.

176
00:09:15,980 --> 00:09:20,901
we got some really nice APIs for building
a more dynamic system.

177
00:09:21,081 --> 00:09:24,953
And then we got all the stuff that they're
open sourcing that I would say is separate

178
00:09:24,953 --> 00:09:27,905
from foundation model and not what I want
to talk about today,

179
00:09:28,445 --> 00:09:30,746
mainly because I don't understand it very
well yet.

180
00:09:30,746 --> 00:09:33,097
but we we got an awful lot this year.

181
00:09:33,097 --> 00:09:39,150
It was a it was a big year for releases
and so so what I do is is I tend to find

182
00:09:39,150 --> 00:09:40,150
what attracts me.

183
00:09:40,497 --> 00:09:42,830
I teach it for a year, teach workshops for
a year,

184
00:09:42,991 --> 00:09:45,044
and then I decide is it going to be a
book?

185
00:09:45,044 --> 00:09:47,157
And this will be my next book.

186
00:09:47,497 --> 00:09:51,353
last year was just too early because you
could see the holes that they had to fill.

187
00:09:51,353 --> 00:09:53,446
And I think they filled a lot of them at
twenty seven.

188
00:09:54,289 --> 00:09:59,392
Yeah, I we've been talking over the last
year and I could tell like twenty seven

189
00:09:59,392 --> 00:10:02,774
was a big deal for for foundation models
and yeah,

190
00:10:02,774 --> 00:10:06,416
it was probably the right decision to wait
on the book regarding that.

191
00:10:05,386 --> 00:10:07,915
Mm-hmm. I I still haven't

192
00:10:06,556 --> 00:10:07,749
I don't wanna get into

193
00:10:07,915 --> 00:10:11,564
written my Swift 6 book because I don't
consider Swift 6 complete yet.

194
00:10:12,637 --> 00:10:15,376
I I agree with you. Yes. That's true.

195
00:10:16,901 --> 00:10:17,942
And I s and I and

196
00:10:17,262 --> 00:10:17,942
I don't wanna get

197
00:10:17,942 --> 00:10:21,262
and I actually should stop writing books
because no one buys books anymore.

198
00:10:21,262 --> 00:10:23,939
They just buy tokens. So that's another
thing.

199
00:10:24,672 --> 00:10:25,012
Well,

200
00:10:25,012 --> 00:10:26,353
you should sell tokens then.

201
00:10:26,791 --> 00:10:27,791
That's right.

202
00:10:28,215 --> 00:10:30,036
or sell skills, that's the other thing,

203
00:10:30,036 --> 00:10:36,231
right? So there's like a hierarchy too to
what models are available on what device,

204
00:10:36,231 --> 00:10:40,965
right? So like what's it even if you can
install twenty seven,

205
00:10:41,666 --> 00:10:43,947
that doesn't mean you're getting the best
model.

206
00:10:44,508 --> 00:10:46,709
It depends on what what device you have,

207
00:10:46,790 --> 00:10:49,992
right? It's like correct correct me if I'm
wrong.

208
00:10:49,031 --> 00:10:50,031
So

209
00:10:50,201 --> 00:10:53,633
I I don't know I actually don't know the
answer to that question.

210
00:10:53,633 --> 00:10:58,585
What I do know is before you write any
foundation model into your code,

211
00:10:58,946 --> 00:11:03,307
you should write some sort of a test into
your app to make sure that the user

212
00:11:03,307 --> 00:11:05,208
can run stuff. And so

213
00:11:04,898 --> 00:11:05,208
Okay.

214
00:11:05,208 --> 00:11:07,199
you c you can query, you know,

215
00:11:07,199 --> 00:11:08,870
do you have a device that can run this?

216
00:11:09,310 --> 00:11:11,631
If you do, do you ha have you installed
it?

217
00:11:11,952 --> 00:11:15,403
If you've installed it, has it downloaded
and completely installed yet?

218
00:11:15,691 --> 00:11:17,323
You know, until all those things have
happened.

219
00:11:17,463 --> 00:11:21,208
And then there's the one that you don't
test for but was a problem.

220
00:11:21,429 --> 00:11:26,105
I haven't tested this lately, but if your
system language and your Siri language

221
00:11:26,105 --> 00:11:31,822
differ, it doesn't work. And so you've got
to be careful about that.

222
00:11:32,123 --> 00:11:33,995
Even for a while I was

223
00:11:32,600 --> 00:11:34,215
Who would what's the

224
00:11:34,215 --> 00:11:35,215
use case for that?

225
00:11:36,362 --> 00:11:37,362
Well,

226
00:11:38,773 --> 00:11:43,358
Because I'm who I am, I had my system
language as US English,

227
00:11:43,739 --> 00:11:46,882
but I was hoping when I drove round in my
car that the car would say,

228
00:11:46,882 --> 00:11:49,156
You missed your fucking turn in Irish,

229
00:11:49,156 --> 00:11:51,248
so I had an Irish English instead.

230
00:11:51,509 --> 00:11:55,103
And it turned out no it doesn't swear at
you just 'cause you have it set to Irish.

231
00:11:55,835 --> 00:11:58,334
Mm I won't beep that by the way,

232
00:11:58,334 --> 00:11:59,186
so we're good.

233
00:11:59,046 --> 00:12:00,070
A bo a boy can

234
00:12:00,070 --> 00:12:01,315
d a boy can dream.

235
00:12:02,467 --> 00:12:03,467
Ha ha.

236
00:12:05,699 --> 00:12:09,585
yeah, 'cause I just remember like when
they released it was like it was like this

237
00:12:09,585 --> 00:12:12,831
chart of like you have to have this Well,

238
00:12:13,071 --> 00:12:15,775
when did they first announce the Apple
intelligence stuff?

239
00:12:15,775 --> 00:12:17,298
Was that like three years ago?

240
00:12:17,699 --> 00:12:18,620
Like twenty four?

241
00:12:18,090 --> 00:12:18,620
So they didn't

242
00:12:18,620 --> 00:12:20,341
they didn't give us the stuff for it,

243
00:12:20,441 --> 00:12:25,284
but in the meantime, a lot of the the
devices that wouldn't support it,

244
00:12:25,604 --> 00:12:27,565
you know, we're not using any anyway.

245
00:12:27,906 --> 00:12:30,827
And if you look, like even the iPad mini,

246
00:12:30,827 --> 00:12:35,360
I think the new iPad mini they just
announced comes with stuff that supports

247
00:12:35,360 --> 00:12:39,702
AI better. So they're building this neural
engine stuff and and you know,

248
00:12:39,576 --> 00:12:40,213
Right, right.

249
00:12:39,702 --> 00:12:40,542
more more

250
00:12:40,542 --> 00:12:43,154
stuff into just even the base level
devices.

251
00:12:43,394 --> 00:12:45,595
And so that's clearly the way Apple's
moving.

252
00:12:46,216 --> 00:12:47,070
Also

253
00:12:47,070 --> 00:12:52,818
If you if you updated to this to if you
updated dubdub to the latest version

254
00:12:52,818 --> 00:12:58,736
of the OS, you saw how long it took for
Siri AI to install and index everything.

255
00:12:58,736 --> 00:12:59,877
It took a long

256
00:12:59,234 --> 00:12:59,877
Yes.

257
00:12:59,877 --> 00:13:04,884
time. So it's a real rethinking of how
Spotlight and and all these things work.

258
00:13:06,104 --> 00:13:11,649
I still so I just updated today to 27.2 on my Mac and like I feel like

259
00:13:11,649 --> 00:13:16,514
every time I update Spotlight doesn't find
the apps anymore and it has to like it's

260
00:13:16,714 --> 00:13:18,306
yeah, it's weird.

261
00:13:18,070 --> 00:13:18,491
And I want

262
00:13:18,491 --> 00:13:21,004
to say in my Irish voice, but I won't do
it again.

263
00:13:21,265 --> 00:13:23,098
It's right effing there. You know,

264
00:13:23,098 --> 00:13:24,860
how can you not find it? I c

265
00:13:23,393 --> 00:13:24,784
Right, right, exactly.

266
00:13:24,951 --> 00:13:25,951
I can see it.

267
00:13:26,566 --> 00:13:31,179
Right. Or it's in my history. Like you
know I use Xcode every day.

268
00:13:31,179 --> 00:13:31,530
Like

269
00:13:31,298 --> 00:13:31,990
That's right.

270
00:13:31,990 --> 00:13:34,112
yeah. I wanna go back to the error stuff.

271
00:13:34,112 --> 00:13:36,224
How like how would you even handle that?

272
00:13:36,224 --> 00:13:39,777
Because I've I struggle with that myself
where it's like I have this whole

273
00:13:39,777 --> 00:13:41,959
app built around foundation models.

274
00:13:42,659 --> 00:13:47,824
Like okay, what if they install it on a
device that doesn't have foundation

275
00:13:47,824 --> 00:13:51,006
models? This is like my my virtualization
app Bushel,

276
00:13:51,006 --> 00:13:52,006
right? I

277
00:13:52,199 --> 00:13:56,021
stupidly forgot I didn't know this that
you can target only ARM and

278
00:13:56,021 --> 00:14:00,664
now the app was out there and supported
technically Intel,

279
00:14:00,664 --> 00:14:04,655
but virtualization needs ARM and like I
basically have to say my app is useless

280
00:14:04,655 --> 00:14:10,203
on Intel. Like, what do I do what what
would you suggest a app developer do,

281
00:14:10,203 --> 00:14:13,160
I guess, if they don't based on the
different errors that

282
00:14:11,815 --> 00:14:13,157
So you cer you certainly

283
00:14:13,160 --> 00:14:14,160
they can get

284
00:14:15,150 --> 00:14:17,384
No, no, I was just saying based on the
different errors you get,

285
00:14:17,384 --> 00:14:19,889
like whether it's like the device doesn't
support it,

286
00:14:19,889 --> 00:14:22,685
doesn't have it the download the model
downloaded.

287
00:14:21,395 --> 00:14:22,786
So if it's something that the

288
00:14:22,986 --> 00:14:25,047
if it's something the user can fix,

289
00:14:25,408 --> 00:14:29,971
you should let the user know. If it's if
it's you need to go to to enable this

290
00:14:29,971 --> 00:14:32,533
in your settings, you can let the user
know.

291
00:14:32,893 --> 00:14:35,034
If it's an old device, maybe that too.

292
00:14:35,375 --> 00:14:38,582
Your own le Jeroen Leenarts, who runs
do iOS,

293
00:14:38,742 --> 00:14:39,653
the conference coming

294
00:14:39,277 --> 00:14:39,653
Mm-hmm.

295
00:14:39,653 --> 00:14:43,826
up in Amsterdam, he just recently wrote a
really interesting blog post about this

296
00:14:43,986 --> 00:14:48,569
where he said your app should be local
first so that if foundation models isn't

297
00:14:48,569 --> 00:14:49,534
available.

298
00:14:49,534 --> 00:14:51,615
You should have some things that work
locally.

299
00:14:51,875 --> 00:14:53,535
And so it should default to nice,

300
00:14:53,895 --> 00:14:56,656
nice behavior. And perhaps you want to let
the user know,

301
00:14:56,656 --> 00:14:58,987
but you don't want to let the user know
every time.

302
00:14:58,987 --> 00:15:01,007
Once you've told them and they've decided,

303
00:15:01,007 --> 00:15:03,118
okay, I don't want to install this stuff,

304
00:15:03,358 --> 00:15:06,974
you still should should make some behavior
available to them.

305
00:15:06,974 --> 00:15:09,315
The other thing I'll say is you know,

306
00:15:09,315 --> 00:15:11,055
we now have Private Cloud Compute,

307
00:15:12,096 --> 00:15:14,976
and you're tempted to always use Private
Cloud Compute,

308
00:15:15,536 --> 00:15:17,477
but if someone's not on Wi-Fi.

309
00:15:18,033 --> 00:15:21,774
That's not free. You know, they're using
network traffic to download it.

310
00:15:21,954 --> 00:15:25,735
And if someone even is on Wi-Fi,

311
00:15:25,815 --> 00:15:28,256
there's a quota and they're using up their
quota.

312
00:15:28,876 --> 00:15:32,397
So you should use Private Cloud Compute
judiciously,

313
00:15:32,617 --> 00:15:35,498
not from your point of view. It's pretty
much free to developers,

314
00:15:35,678 --> 00:15:37,418
but to the user's point of view.

315
00:15:37,418 --> 00:15:38,669
And you may want to alert them,

316
00:15:38,669 --> 00:15:43,820
hey, you're running out of your quota or
you're running this over cell.

317
00:15:43,820 --> 00:15:45,641
Again, allow them to dismiss it,

318
00:15:45,881 --> 00:15:47,011
but let them know.

319
00:15:48,549 --> 00:15:50,110
So it was like earlier this week.

320
00:15:50,850 --> 00:15:54,112
I remember that Gui had put out that two
million kit.

321
00:15:54,112 --> 00:15:55,292
Do you know what I'm talking about?

322
00:15:55,653 --> 00:15:56,994
No, I'll have to look.

323
00:15:56,733 --> 00:15:57,083
So he

324
00:15:57,083 --> 00:16:02,116
had so I don't know if you remember Pl
Private Cloud Compute had a limit

325
00:16:02,417 --> 00:16:04,130
Yep. Yep.

326
00:16:02,456 --> 00:16:04,197
of who could use it. It was

327
00:16:04,197 --> 00:16:07,519
like two million. And then Gui Rambo,

328
00:16:07,559 --> 00:16:11,681
who's been on the podcast and is pretty
well known for hacking into stuff.

329
00:16:10,109 --> 00:16:12,855
Well he he's awesome. Yeah, he's awesome.

330
00:16:12,121 --> 00:16:13,722
Yeah, yeah.

331
00:16:13,722 --> 00:16:16,814
he had put out a Swift package to get
around it.

332
00:16:17,836 --> 00:16:22,410
but then I guess like Apple either they
had removed the restrictions,

333
00:16:22,410 --> 00:16:23,701
correct me if I'm wrong if you know,

334
00:16:23,861 --> 00:16:27,855
they'd either remove the restriction or
they like there was a it wasn't necessar

335
00:16:27,965 --> 00:16:31,038
because I went to the URL and I'm like
it's not there and I asked he's like,

336
00:16:31,038 --> 00:16:35,171
Yeah, we don't I archived it because it's
not really useful anymore based

337
00:16:35,171 --> 00:16:39,415
on the way things work. Is that is that
still correct though that like private

338
00:16:39,415 --> 00:16:42,908
cloud compute has certain restrictions as
far as who could use it?

339
00:16:43,461 --> 00:16:48,284
So the the two million restriction is
interesting because that's an odd number.

340
00:16:48,284 --> 00:16:52,647
You have to have a pretty big user base to
to be over two million.

341
00:16:52,847 --> 00:16:58,571
And so what it sounds like to me is if
you're a big company with tons of users,

342
00:16:58,871 --> 00:17:00,813
you should be having your own system.

343
00:17:00,692 --> 00:17:02,150
No one hundred percent.

344
00:17:01,433 --> 00:17:02,654
I don't understand

345
00:17:02,654 --> 00:17:08,147
it though, because Private Cloud Compute
is something that is metered from

346
00:17:08,147 --> 00:17:11,180
the user's point of view, not from the
developer point of view.

347
00:17:11,455 --> 00:17:13,248
So I don't know why it matters.

348
00:17:13,248 --> 00:17:16,891
Like it seems like that should be
something that is available to everybody,

349
00:17:16,891 --> 00:17:19,664
but you know, it's not a technical issue.

350
00:17:20,714 --> 00:17:26,458
Or it's just like and I'm sure this is w
the way they would end up going eventually

351
00:17:26,458 --> 00:17:30,350
is like bill people if you have a app
that's using it a lot,

352
00:17:30,350 --> 00:17:33,241
like turn it into a money making
opportunity.

353
00:17:33,241 --> 00:17:36,852
Apple's really good at that. Like I'm
surprised they didn't go that route where

354
00:17:36,852 --> 00:17:37,533
it's like, hey

355
00:17:36,970 --> 00:17:37,740
But I really

356
00:17:37,740 --> 00:17:43,061
I really like this other solution because
you can put out an app and not realize

357
00:17:43,061 --> 00:17:46,712
how much users you're using it and all of
a sudden you're being billed at a rate

358
00:17:46,712 --> 00:17:50,623
that you don't have income to sustain it
and it and then you close down your app.

359
00:17:50,623 --> 00:17:54,685
And so we saw that with with maps and and
other things that use queries that were

360
00:17:54,685 --> 00:18:00,987
metered or like weather apps. And so the
the metering adds up and sometimes catches

361
00:18:00,987 --> 00:18:02,347
a developer by surprise.

362
00:18:03,331 --> 00:18:06,956
But I I guess what I meant was like you
meter you meter until you reach

363
00:18:06,956 --> 00:18:10,211
a certain you don't meter until you
reach a certain threshold,

364
00:18:10,211 --> 00:18:14,227
right? Like CloudKit kind of. Where it's
like so insignificant for most people,

365
00:18:14,227 --> 00:18:18,854
but if you're like Facebook then you think
about options,

366
00:18:18,854 --> 00:18:19,854
you know what I mean?

367
00:18:19,989 --> 00:18:24,760
And the quota is is odd, from the user
standpoint because I believe it's based

368
00:18:24,760 --> 00:18:26,344
on requests, not on tokens.

369
00:18:26,900 --> 00:18:32,662
Okay. That is weird. Yeah. so wh why why
would you pick

370
00:18:33,604 --> 00:18:37,913
why would you pick Private Cloud Compute
over just local models?

371
00:18:38,589 --> 00:18:40,370
Well, for world knowledge, for example.

372
00:18:40,370 --> 00:18:46,909
So I have an example I I'm using in my
workshops in a talk where I show the image

373
00:18:46,949 --> 00:18:51,043
of Van Gogh's sunflowers. And I ask
locally,

374
00:18:51,043 --> 00:18:55,136
hey, you're an art historian, you're
talking to a group of kids who are seeing

375
00:18:55,136 --> 00:19:00,451
this picture, describe it. And it says
this is a picture of a of a sunflower,

376
00:19:00,451 --> 00:19:03,483
it's heavy use of yellows and thick brush
strokes.

377
00:19:04,063 --> 00:19:07,526
Then you use Private Cloud Compute and you
set the reasoning to deep.

378
00:19:07,952 --> 00:19:12,963
And it said this is Van Gogh's sunflower
painted in this year using this technique.

379
00:19:13,244 --> 00:19:15,714
And here's where it's exit. Like it has
world knowledge,

380
00:19:15,714 --> 00:19:16,905
and all of a sudden you get a much

381
00:19:16,331 --> 00:19:16,945
Mm-hmm.

382
00:19:16,945 --> 00:19:21,466
deeper response. I do the same thing with
an example where we're recommending wine

383
00:19:21,766 --> 00:19:24,947
and recommending it locally with less
reasoning,

384
00:19:25,267 --> 00:19:28,018
you get very superficial. You want a red,

385
00:19:28,018 --> 00:19:31,749
you want a white. When we use deep
reasoning on Private Cloud Compute,

386
00:19:31,869 --> 00:19:34,240
we get specific labels and years.

387
00:19:34,240 --> 00:19:34,941
And so

388
00:19:34,941 --> 00:19:35,438
Okay.

389
00:19:35,438 --> 00:19:38,641
you can get much better, but you've got to
decide as a developer,

390
00:19:39,057 --> 00:19:43,528
do you care? I mean sometimes the context
is such that you won't care if it's

391
00:19:43,528 --> 00:19:44,549
a deeper answer or not.

392
00:19:45,448 --> 00:19:48,230
Okay, okay, got it. Can you should we get

393
00:19:47,879 --> 00:19:48,570
Al also

394
00:19:48,570 --> 00:19:51,303
it takes more time because not only is
there a round trip,

395
00:19:51,303 --> 00:19:51,984
but if you said

396
00:19:51,792 --> 00:19:52,685
Yeah.

397
00:19:52,685 --> 00:19:55,248
not just the network, but if you said
deeper reasoning,

398
00:19:55,528 --> 00:19:57,651
it's taking time. And you've seen

399
00:19:57,214 --> 00:19:57,651
Okay.

400
00:19:57,651 --> 00:20:00,935
this on Siri AI where it it says thinking
and you think,

401
00:20:00,935 --> 00:20:04,151
No, you're not. Y you're off getting a
cappuccino.

402
00:20:05,094 --> 00:20:06,915
Yeah, it's as it's known to do.

403
00:20:07,483 --> 00:20:08,483
Ha ha.

404
00:20:09,478 --> 00:20:15,422
I've actually had the issue where like
I've queried it and then it keeps going

405
00:20:15,422 --> 00:20:19,015
and going and going and I'm like I don't
really care that much that I need

406
00:20:19,015 --> 00:20:23,899
you to like get rid of it and then
Spotlight doesn't work and then like

407
00:20:23,899 --> 00:20:25,931
now I can't even launch an an I hate that.

408
00:20:26,551 --> 00:20:28,453
We'll see if they fix that in 27.2.

409
00:20:26,587 --> 00:20:29,099
So one of the things that got one

410
00:20:29,099 --> 00:20:32,832
of the things that got turned off during
the beta cycle was for the terminal,

411
00:20:32,832 --> 00:20:38,137
you used to be able to use Private Cloud
Compute and they turn that off.

412
00:20:38,137 --> 00:20:40,809
But if you set it and do a query,

413
00:20:41,210 --> 00:20:45,173
it just looks like it's thinking and
thinking and thinking and it's not.

414
00:20:45,313 --> 00:20:46,454
It's not doing anything.

415
00:20:45,978 --> 00:20:46,694
Okay.

416
00:20:46,694 --> 00:20:49,677
You set it back to system and it gives you
your answer like that.

417
00:20:50,619 --> 00:20:53,600
So you're you're talking about like the FM
command.

418
00:20:52,815 --> 00:20:54,296
Mm-hmm. Mm-hmm.

419
00:20:54,183 --> 00:20:54,584
So

420
00:20:54,584 --> 00:20:56,803
FM you still can't use Private Cloud
Compute?

421
00:20:57,466 --> 00:20:59,947
I believe the last time I tested it you
still can't.

422
00:20:59,947 --> 00:21:01,049
I th I th they turned it

423
00:21:00,688 --> 00:21:01,049
Okay.

424
00:21:01,049 --> 00:21:04,012
off in one of the betas and I don't think
they turned it back on.

425
00:21:04,493 --> 00:21:04,753
And

426
00:21:04,529 --> 00:21:04,921
But you still

427
00:21:04,921 --> 00:21:06,426
have #Playground though, right?

428
00:21:07,141 --> 00:21:08,383
You still have #Playground.

429
00:21:08,685 --> 00:21:09,185
So from

430
00:21:08,795 --> 00:21:09,185
Okay.

431
00:21:09,185 --> 00:21:10,809
a developer's point of view, you can do
that.

432
00:21:11,375 --> 00:21:14,889
Okay. I wanna we should probably get into
it,

433
00:21:15,290 --> 00:21:17,792
because to kind of break things down,

434
00:21:17,832 --> 00:21:21,736
but let's talk about like the different
components and foundation models

435
00:21:21,797 --> 00:21:22,818
and how they work.

436
00:21:22,374 --> 00:21:24,065
Sure. I love

437
00:21:24,086 --> 00:21:26,387
the APIs. I think they've done a good job.

438
00:21:26,588 --> 00:21:29,190
And I think this year they made them even
nicer.

439
00:21:30,631 --> 00:21:31,502
And so the

440
00:21:31,098 --> 00:21:31,502
Yeah.

441
00:21:31,502 --> 00:21:34,434
first thing you would do is you would you
would create a session.

442
00:21:35,215 --> 00:21:39,198
And at the very base level, you provide
the session with a prompt.

443
00:21:39,939 --> 00:21:44,122
And then that can fail. That can take a
while.

444
00:21:44,543 --> 00:21:46,715
And then when it responds, you reach
inside,

445
00:21:46,715 --> 00:21:50,468
get the content, and you can see the
answer to the prompt or the question that

446
00:21:50,468 --> 00:21:51,120
you made.

447
00:21:51,120 --> 00:21:52,812
That's the very simplest love.

448
00:21:54,926 --> 00:22:01,115
So the one thing I thought was amazing was
this whole Generable macro macro,

449
00:22:01,115 --> 00:22:06,350
right? where you can like say I want my
output to be in this format,

450
00:22:06,650 --> 00:22:07,650
essentially.

451
00:22:08,081 --> 00:22:13,439
So that's that's much further down the
line because it it actually does three

452
00:22:13,439 --> 00:22:14,490
things really well.

453
00:22:15,321 --> 00:22:16,273
Okay.

454
00:22:16,273 --> 00:22:19,237
we can go there right away or I can sort
of get you there.

455
00:22:18,639 --> 00:22:19,378
Yeah, go there right

456
00:22:19,378 --> 00:22:20,502
away. Yeah, yeah, yeah.

457
00:22:21,733 --> 00:22:24,404
So one of the things is if you'd like,

458
00:22:24,404 --> 00:22:26,735
instead of waiting till it's thought all
the way through,

459
00:22:26,735 --> 00:22:30,676
because you know the way these things work
is it it gets the next few tokens,

460
00:22:30,676 --> 00:22:34,747
the next few tokens, and so it's gonna
wait till it's all done and then give

461
00:22:34,747 --> 00:22:38,448
it to you. Or you can stream the the
response.

462
00:22:38,888 --> 00:22:41,009
And so if you stream the response,

463
00:22:41,669 --> 00:22:46,530
you'll use this macro and it'll give you
the partially generated along the way.

464
00:22:47,571 --> 00:22:48,811
Or you say, look.

465
00:22:49,535 --> 00:22:55,139
I want you to reach inside this string of
unformatted stuff and I want you to turn

466
00:22:55,139 --> 00:22:57,361
it into a Swift type if you can.

467
00:22:57,842 --> 00:23:02,355
And so you can say, Hey, here's what my
Swift type looks like that I'm trying

468
00:23:02,355 --> 00:23:04,827
to get you to match, and it will give you
that.

469
00:23:04,827 --> 00:23:07,029
And now you can use it just as a regular
Swift type.

470
00:23:07,149 --> 00:23:09,481
It's sort of like Codable on on steroids.

471
00:23:10,033 --> 00:23:11,938
Right, yeah, one hundred pr I like that.

472
00:23:11,938 --> 00:23:13,452
It is Codable on steroids.

473
00:23:13,452 --> 00:23:16,635
The third place is really nice is if
you're gonna use a tool.

474
00:23:17,296 --> 00:23:18,917
And so if you're gonna use a tool,

475
00:23:19,738 --> 00:23:26,003
you wanna take that prompt, which is
usually an un specially formatted string,

476
00:23:26,123 --> 00:23:30,537
and you wanna turn that prompt into a type
and give that type to the you

477
00:23:30,537 --> 00:23:32,608
can use the Generable macro on the way in.

478
00:23:32,768 --> 00:23:34,209
And so you can use it on the way in,

479
00:23:34,209 --> 00:23:35,240
you can use it on the way out,

480
00:23:35,240 --> 00:23:36,731
and you can use it for streaming,

481
00:23:37,032 --> 00:23:38,032
and it's

482
00:23:38,080 --> 00:23:41,151
It's really one of the coolest parts and
it makes so much of this nicer.

483
00:23:42,421 --> 00:23:44,303
I guess like so you have the prompt,

484
00:23:44,724 --> 00:23:47,587
the prompt gives you a response.

485
00:23:48,088 --> 00:23:49,630
This is all done in a session,

486
00:23:49,630 --> 00:23:51,983
correct me where I'm wrong. And then

487
00:23:51,262 --> 00:23:52,063
And so I think

488
00:23:52,063 --> 00:23:54,684
of the prompt as belonging to the user.

489
00:23:55,285 --> 00:23:55,905
And so the

490
00:23:55,545 --> 00:23:55,925
Okay.

491
00:23:55,925 --> 00:23:59,107
other thing that we can do is we can set
instructions.

492
00:23:59,628 --> 00:24:02,120
And instructions are written by the
developer.

493
00:24:02,120 --> 00:24:07,263
So in my art example, I said, I gave you a
picture to describe,

494
00:24:07,263 --> 00:24:11,456
but my instructions were, you are this
type of person speaking to this type

495
00:24:11,456 --> 00:24:13,828
of of audience. I want you to do this.

496
00:24:13,828 --> 00:24:17,804
And so the instructions are supposed to
always

497
00:24:18,636 --> 00:24:21,648
trump the the prompt. And so if they're in
conflict,

498
00:24:21,729 --> 00:24:25,772
the instruction should win, and this
should keep the the user from being able

499
00:24:25,772 --> 00:24:28,454
to take you off on a on a really big
tangent.

500
00:24:29,607 --> 00:24:29,922
Okay.

501
00:24:29,635 --> 00:24:29,985
So we

502
00:24:29,985 --> 00:24:33,278
have instructions. Instructions belong to
the session.

503
00:24:34,219 --> 00:24:38,402
Prompt belongs to the response or the
stream that you're trying to generate.

504
00:24:38,762 --> 00:24:42,205
And then tools also belong to the session.

505
00:24:42,925 --> 00:24:46,479
And then you can also have generating
options.

506
00:24:46,725 --> 00:24:51,099
And so you can say, look, I want the reply
to only take this many tokens

507
00:24:51,099 --> 00:24:54,571
at the maximum. I want it to have this
sort of temperature.

508
00:24:54,765 --> 00:24:57,494
I want it to, since everything's
probabilistic,

509
00:24:57,654 --> 00:25:00,096
I want you to treat the probabilities in
this way.

510
00:25:00,276 --> 00:25:04,840
And so you can tune the type of response
you get using generating options as well.

511
00:25:05,260 --> 00:25:09,904
And then something they introduced this
year with these excuse me dynamic profiles

512
00:25:10,185 --> 00:25:12,506
where you can combine these things and
say,

513
00:25:12,586 --> 00:25:15,689
you know, here's what I want you to use in
this situation.

514
00:25:15,911 --> 00:25:18,067
And so you can change that along the way.

515
00:25:18,248 --> 00:25:22,469
So that's sort of a a quick summary of the
things that that the framework allows.

516
00:25:24,597 --> 00:25:25,597
Was that helpful?

517
00:25:26,149 --> 00:25:28,571
It's very helpful actually. And one thing
I want to say,

518
00:25:28,571 --> 00:25:30,022
you said the API is really good.

519
00:25:30,022 --> 00:25:36,297
I also like that they used terms that like
people like all these terms still make

520
00:25:36,297 --> 00:25:40,570
sense if you're using Claude or OpenAI or
anything.

521
00:25:39,946 --> 00:25:41,287
Yeah, they use terms of art.

522
00:25:40,570 --> 00:25:41,570
They

523
00:25:41,961 --> 00:25:43,722
right. Instead of just being like,

524
00:25:44,663 --> 00:25:46,294
you know how they are with their branding,

525
00:25:46,294 --> 00:25:48,616
right? It's like, let's come up with a
nice,

526
00:25:48,616 --> 00:25:52,068
cool, cool term that that will market
well.

527
00:25:52,328 --> 00:25:54,977
no, these are all terms that we use
throughout other AI.

528
00:25:53,054 --> 00:25:55,175
But at the same time, some of the things

529
00:25:55,175 --> 00:26:00,158
that they they did this year, so a l a lot
of what they gave us are structs,

530
00:26:00,639 --> 00:26:04,621
a lot of what they gave us look an awful
lot like SwiftUI now.

531
00:26:04,862 --> 00:26:09,524
So for instance, excuse me, we think of a
prompt as being a string,

532
00:26:10,144 --> 00:26:12,166
but a prompt is actually a type,

533
00:26:12,606 --> 00:26:16,389
and you can put anything then in that is
prompt representable,

534
00:26:16,389 --> 00:26:18,550
which it sounds very familiar to you.

535
00:26:18,790 --> 00:26:21,182
And so things that can be used as prompts.

536
00:26:21,382 --> 00:26:25,875
might include an image. And now I can
build it up the way I would build

537
00:26:25,875 --> 00:26:31,539
up a SwiftUI view. My prompt can have a I
can use a dynamic prompt and build

538
00:26:31,539 --> 00:26:36,213
it up using a body just like I would build
up like a V stack containing different

539
00:26:36,213 --> 00:26:39,945
things. I can now have this this thing
that contains different things

540
00:26:39,945 --> 00:26:42,827
for a prompt do the same thing for
instructions.

541
00:26:43,068 --> 00:26:47,531
So they've really been nice about giving
us APIs that that feel very familiar

542
00:26:47,531 --> 00:26:48,822
to us as Swift developers.

543
00:26:50,038 --> 00:26:53,160
I I think you know this, but I'm obsessed
with result builders.

544
00:26:53,663 --> 00:26:55,872
So like yeah, seeing that it's like

545
00:26:54,804 --> 00:26:56,129
And and so they're you they're

546
00:26:56,129 --> 00:26:57,776
they're using result builders everywhere.

547
00:26:58,295 --> 00:27:00,700
Yeah, I think that's a great use case for
that,

548
00:27:00,700 --> 00:27:01,452
absolutely.

549
00:27:01,078 --> 00:27:01,589
There's there's

550
00:27:01,589 --> 00:27:03,805
a prompt builder, there's an instruction
builder,

551
00:27:03,805 --> 00:27:05,880
they've they've really done a nice job.

552
00:27:05,880 --> 00:27:09,208
I've I've loved result builders since they
were called function builders.

553
00:27:09,208 --> 00:27:10,462
It's just such a nice tool.

554
00:27:11,050 --> 00:27:14,036
Or D SLs, right? Is that is that kind of
the same thing?

555
00:27:14,177 --> 00:27:15,177
Kind of.

556
00:27:15,228 --> 00:27:18,869
It it is. Th the idea is is you're if
you've ever built one,

557
00:27:19,389 --> 00:27:22,260
you're basically taking this thing that
can take multiple inputs,

558
00:27:22,260 --> 00:27:26,311
and in the old days, because no one wanted
to write more than ten,

559
00:27:26,591 --> 00:27:30,872
you only got up to ten. But then we got
some improvements in Swift 6,

560
00:27:31,052 --> 00:27:35,013
could have been Swift five, where we got
generics that could be very variadic

561
00:27:35,054 --> 00:27:38,454
and all of a sudden we could have them
take as many as we want.

562
00:27:38,574 --> 00:27:40,675
Like if you remember SwiftUI in the old
days,

563
00:27:41,015 --> 00:27:43,996
a V stack could contain at most ten things
because

564
00:27:44,137 --> 00:27:46,028
Right. Because the generic

565
00:27:44,755 --> 00:27:45,755
That's all we could do.

566
00:27:46,028 --> 00:27:52,355
was capped at that, right? so I wanna take
a step back again and what let's let's

567
00:27:52,355 --> 00:27:57,417
talk about the prompt itself. So the
prompt is gonna be built based on what

568
00:27:57,417 --> 00:28:01,700
the user is just whatever you want the
user I don't know,

569
00:28:01,909 --> 00:28:02,990
I don't even know how to put it.

570
00:28:02,990 --> 00:28:04,611
But the user is gonna have some input.

571
00:28:04,871 --> 00:28:06,962
That input is gonna be used to create the
prompt,

572
00:28:06,962 --> 00:28:07,637
right?

573
00:28:07,637 --> 00:28:12,505
The user can have input. You could do it
you could control it on your end with

574
00:28:12,505 --> 00:28:14,939
a drop down list or build it up anyway.

575
00:28:15,400 --> 00:28:18,165
But the idea is sure. Sure.

576
00:28:15,559 --> 00:28:17,441
Or even just like images like you said,

577
00:28:18,061 --> 00:28:19,222
or photos, right?

578
00:28:20,583 --> 00:28:22,925
And then you basically create the prompt
based on that,

579
00:28:22,925 --> 00:28:23,626
more or less.

580
00:28:23,340 --> 00:28:24,593
That's right. That's right.

581
00:28:24,647 --> 00:28:28,180
Okay. Okay. I'm getting this now.

582
00:28:28,180 --> 00:28:29,799
And then what would be an example though,

583
00:28:29,799 --> 00:28:35,837
I guess, of a tool for taking that input
and formatting it so it could be used

584
00:28:36,177 --> 00:28:38,419
by the the model?

585
00:28:39,179 --> 00:28:43,440
So the the hello world tool that they
always demonstrate is a weather tool

586
00:28:43,860 --> 00:28:46,041
because you know that's really easy.

587
00:28:46,041 --> 00:28:50,562
another one that I often do is a map tool
because I can query the map APIs.

588
00:28:50,842 --> 00:28:54,283
Well, Apple this year, as part of OS 27,

589
00:28:54,563 --> 00:28:57,224
gave us three tools. two of them came from
Vision.

590
00:28:57,444 --> 00:29:01,405
One is a barcode tool reader, and one is
an OCR reader.

591
00:29:01,626 --> 00:29:06,367
And so I can point I can point foundation
models at an image.

592
00:29:06,803 --> 00:29:08,614
And if it there's not a lot of text there,

593
00:29:08,614 --> 00:29:11,205
it's pretty good at reading it and
figuring out what it is.

594
00:29:11,325 --> 00:29:13,306
If it's very dense or small text,

595
00:29:13,446 --> 00:29:18,818
the OCR reader is awesome. The barcode
reader reads the barcode and then

596
00:29:18,818 --> 00:29:22,269
you use your APIs to query. So what
product is this for?

597
00:29:22,269 --> 00:29:23,780
You have to have a database of that.

598
00:29:23,780 --> 00:29:28,122
It doesn't know that, but it can read the
barcode or the QR code and

599
00:29:28,122 --> 00:29:31,173
and get you that information. And that's
that's pretty cool.

600
00:29:31,173 --> 00:29:35,325
And then the final tool I think they added
this year was a spotlight tool.

601
00:29:35,945 --> 00:29:37,615
And so those three are built in.

602
00:29:37,658 --> 00:29:40,182
The first two you need to import Vision to
use.

603
00:29:40,523 --> 00:29:46,102
And I always love when Apple takes stuff
they develop for one device and they make

604
00:29:46,102 --> 00:29:47,774
it available to other devices.

605
00:29:47,774 --> 00:29:50,569
It's just it's a natural. It's kind of
nice.

606
00:29:51,238 --> 00:29:54,702
Yeah. I and now that you say what tool is,

607
00:29:54,702 --> 00:29:56,484
like now I'm like, okay, now I get it,

608
00:29:56,484 --> 00:30:02,702
because when I use Claude code it'll start
using grep or like awk or like all these

609
00:30:02,702 --> 00:30:06,887
bash commands in order to do what it needs
to do as opposed to like having

610
00:30:06,887 --> 00:30:10,521
the AI just burn tokens trying to look for
text.

611
00:30:11,177 --> 00:30:11,297
And

612
00:30:11,297 --> 00:30:14,629
this is where this ties into if you do App
Intents,

613
00:30:14,629 --> 00:30:20,092
you can expose things to Siri so that it
you you basically can create things that

614
00:30:20,092 --> 00:30:23,094
you use tools for. So what a tool does,

615
00:30:23,254 --> 00:30:25,275
if you implement a tool on your own,

616
00:30:25,855 --> 00:30:30,158
you say, okay, here's the type of object
I'm gonna pass to it.

617
00:30:30,158 --> 00:30:32,299
So you create a Generable struct,

618
00:30:32,719 --> 00:30:34,860
and then you create a function called
call,

619
00:30:35,000 --> 00:30:38,005
and call takes the arguments, and now say
you say,

620
00:30:38,005 --> 00:30:39,005
I wanna.

621
00:30:39,727 --> 00:30:43,489
Enjoy gelato in Bologna, where's a good
place?

622
00:30:44,949 --> 00:30:50,431
So you create a struct that has a city and
the type of thing that you wanna look for.

623
00:30:50,932 --> 00:30:53,253
And now you can make a map query from
that.

624
00:30:53,253 --> 00:30:57,704
So your tool is using what you can do in
map kit and you're sort of the interface

625
00:30:57,704 --> 00:30:58,915
to it with this call function.

626
00:30:59,828 --> 00:31:04,472
It's awesome. Yeah. So it's basically a
way to do something that doesn't involve

627
00:31:05,052 --> 00:31:11,837
thinking but and avoid burning tokens for
no reason and give you access to like

628
00:31:12,458 --> 00:31:14,790
another API or MCP, I would assume,

629
00:31:14,790 --> 00:31:15,790
something like that.

630
00:31:16,037 --> 00:31:20,311
Yeah, that that's I mean, there's a couple
things about the way people use AI,

631
00:31:20,311 --> 00:31:23,354
which which you've sort of brought up a
couple times now.

632
00:31:23,574 --> 00:31:27,697
And the burning tokens is there's
sometimes that instead of AI,

633
00:31:27,758 --> 00:31:30,910
it's just an algorithm. There's no AI

634
00:31:29,814 --> 00:31:30,814
Right. Right.

635
00:31:30,910 --> 00:31:34,644
in it. And it feels like AI and it feels
magic,

636
00:31:34,644 --> 00:31:38,737
but we're using something that's built in
that we can do as an algorithm.

637
00:31:38,737 --> 00:31:42,561
And so it's magic, but it's a different
sort of magic.

638
00:31:43,147 --> 00:31:44,147
Right, exactly.

639
00:31:44,833 --> 00:31:47,524
So so dynamic profiles is really powerful.

640
00:31:47,524 --> 00:31:52,987
So an example I've been using in classes
lately is I want a wine recommendation.

641
00:31:52,987 --> 00:31:55,398
And you've got your busboy, you've got
your waiter,

642
00:31:55,618 --> 00:31:59,420
you've got your sommelier, and that's
three very different levels.

643
00:32:00,041 --> 00:32:02,077
And depending on who you're talking to,

644
00:32:02,077 --> 00:32:03,613
you're gonna get different
recommendations,

645
00:32:03,613 --> 00:32:05,884
but also based on what meal you're having.

646
00:32:05,884 --> 00:32:07,755
So you might ask one of them, you know,

647
00:32:07,755 --> 00:32:10,847
what wine goes with fish or chicken or
beef or whatever.

648
00:32:12,343 --> 00:32:17,016
What a dynamic profile allows you to do is
to take a profile for each one of those

649
00:32:17,016 --> 00:32:22,008
types, put it into this unified thing that
reads and can switch off of them.

650
00:32:22,169 --> 00:32:26,407
And maybe the sommelier uses Private Cloud
Compute if it's available.

651
00:32:26,407 --> 00:32:27,857
If it's not available, it can't,

652
00:32:27,857 --> 00:32:32,600
so we can test for that. maybe the other
one uses a different type

653
00:32:33,040 --> 00:32:34,831
and the reasoning level is different.

654
00:32:34,831 --> 00:32:37,743
So in generation options, maybe one uses

655
00:32:38,069 --> 00:32:41,041
a very literal temperature. So their
temperature is zero.

656
00:32:41,041 --> 00:32:42,561
They'll give you a very literal answer.

657
00:32:42,762 --> 00:32:45,503
And their profile, their their
instructions might be,

658
00:32:45,983 --> 00:32:49,355
hey, you're a bus boy. You don't know
anything about wine.

659
00:32:49,355 --> 00:32:53,087
You don't even drink wine, but you can
give basic red or white answers.

660
00:32:53,928 --> 00:33:00,771
And so the dynamic profile allows the user
of your app to switch what

661
00:33:00,771 --> 00:33:03,513
they're using within the same session.

662
00:33:03,633 --> 00:33:07,055
So it still has access to the context of
what the last person

663
00:33:07,275 --> 00:33:09,077
So you could say you could call

664
00:33:08,454 --> 00:33:09,077
okay.

665
00:33:09,077 --> 00:33:11,319
over the sommelier and say, Hey,

666
00:33:12,079 --> 00:33:14,351
I like where where the waiter was going
with that,

667
00:33:14,351 --> 00:33:17,144
but could you build on that and give me
some specific options?

668
00:33:17,304 --> 00:33:20,807
So it still has that context, even though
you've switched the profile.

669
00:33:21,680 --> 00:33:23,902
Okay, the context part, that's the
important part,

670
00:33:23,902 --> 00:33:28,026
is like the different profiles still have
access to the same context.

671
00:33:28,026 --> 00:33:29,668
Okay, that makes total sense.

672
00:33:28,855 --> 00:33:29,776
And that's another thing that

673
00:33:29,776 --> 00:33:35,002
we got in OS 27 is it's much
better of knowing how much you have left.

674
00:33:35,443 --> 00:33:39,428
And so you know when you need to summarize
or do something to your transcript

675
00:33:39,548 --> 00:33:42,331
and start a new session based on what you
were doing before.

676
00:33:42,652 --> 00:33:45,164
The tools are better. They're there's I'm
sorry,

677
00:33:43,034 --> 00:33:45,642
Do they have do they have like a

678
00:33:45,164 --> 00:33:45,642
there's a new

679
00:33:45,642 --> 00:33:49,083
summarized do they have like a summarized
method you can call?

680
00:33:49,767 --> 00:33:53,830
It it's it's more that you're asking it to
summarize and and and to give

681
00:33:53,830 --> 00:33:54,830
you something.

682
00:33:54,905 --> 00:33:55,905
Okay, okay.

683
00:33:56,111 --> 00:33:58,273
and and because you might want to be
specific.

684
00:33:58,273 --> 00:34:00,214
You might want to say, you know,

685
00:34:00,654 --> 00:34:02,565
so when you when you start the next one,

686
00:34:02,565 --> 00:34:04,046
still base it on the instructions.

687
00:34:04,046 --> 00:34:05,457
The instructions haven't changed,

688
00:34:06,098 --> 00:34:10,821
but only use the last part of the
conversation or only use these

689
00:34:10,821 --> 00:34:12,772
you you can be specific about what you
want.

690
00:34:14,697 --> 00:34:19,233
Can we get into like the transcript and
how or context and how that works and what

691
00:34:19,534 --> 00:34:21,170
what control developers have?

692
00:34:23,043 --> 00:34:24,907
you have access to the transcript.

693
00:34:27,623 --> 00:34:29,915
I don't know in terms of control,

694
00:34:30,154 --> 00:34:34,495
you now have a much better Instruments so
you can see things as as they occur.

695
00:34:34,495 --> 00:34:36,056
if you look at the transcript,

696
00:34:36,096 --> 00:34:41,197
I find it instructive because it includes
the things that it's using and

697
00:34:41,197 --> 00:34:46,292
you can see the size grow. you can now ask
the size of it in terms of you know,

698
00:34:46,292 --> 00:34:47,601
how many tokens have you used.

699
00:34:48,431 --> 00:34:48,715
Okay.

700
00:34:48,483 --> 00:34:48,753
this

701
00:34:48,753 --> 00:34:51,092
was stuff that we were supposed to have
before,

702
00:34:51,092 --> 00:34:53,464
but it now works much more reliably.

703
00:34:54,286 --> 00:34:56,927
W so streaming is a is a big deal.

704
00:34:56,927 --> 00:35:00,440
Like it's funny because you were saying
you listened to Tim's episode that

705
00:35:00,440 --> 00:35:03,973
we did recently. That was a big part of
Vapor 5.

706
00:35:04,744 --> 00:35:09,428
and like server-side stuff. And we kind of
got into a whole tangent about

707
00:35:09,428 --> 00:35:11,710
MCP and how that's a big driver of that.

708
00:35:11,931 --> 00:35:16,875
Where why do you think how is streaming
useful if you're using foundation models?

709
00:35:17,760 --> 00:35:22,711
So users can see the answer coming in in
real time.

710
00:35:22,991 --> 00:35:26,652
And so a user could you could either set
it up so they can interrupt

711
00:35:26,652 --> 00:35:29,373
it and and say, Well, I want to see this.

712
00:35:29,453 --> 00:35:34,935
So you ever use one of these these coding
assistants and it's it's taking a long

713
00:35:34,935 --> 00:35:37,946
time and you think of something else you
should have told it and you want

714
00:35:37,946 --> 00:35:41,597
to add to that. So so that's that's
something you're able to do.

715
00:35:42,158 --> 00:35:43,878
the other thing with streaming is

716
00:35:44,322 --> 00:35:47,464
Just like a website loading, the user
might lose interest.

717
00:35:47,464 --> 00:35:48,825
This isn't what they want to do at all,

718
00:35:48,825 --> 00:35:50,326
and they want to navigate away.

719
00:35:50,446 --> 00:35:54,628
And so streaming doesn't hold them up
until the whole answer is available.

720
00:35:55,609 --> 00:35:57,230
It's a better user experience.

721
00:35:58,796 --> 00:36:03,329
do you you have control about whether it's
becomes streamed or whether it becomes

722
00:36:03,810 --> 00:36:05,201
Yeah. Your your your call

723
00:36:04,413 --> 00:36:05,201
one okay.

724
00:36:05,201 --> 00:36:09,775
your call to the session is you're either
responding to or stream response to.

725
00:36:10,095 --> 00:36:13,698
And if you stream, you're making a call,

726
00:36:13,759 --> 00:36:17,371
which you then consume. The result is an
async sequence,

727
00:36:17,371 --> 00:36:18,202
which again

728
00:36:17,921 --> 00:36:18,810
Yeah, I love that.

729
00:36:18,810 --> 00:36:22,396
I love that they like when they introduced
async sequence,

730
00:36:22,396 --> 00:36:26,329
it looked really limited, but it's showing
up in so many Apple APIs.

731
00:36:26,329 --> 00:36:27,290
It's really nice.

732
00:36:26,986 --> 00:36:27,828
Right,

733
00:36:27,828 --> 00:36:31,055
right. Yeah, 100%. okay,

734
00:36:31,055 --> 00:36:32,457
I remembered what I was gonna ask.

735
00:36:32,538 --> 00:36:35,594
Does the transcript show you what it's
thinking?

736
00:36:37,696 --> 00:36:39,378
Can you Okay.

737
00:36:37,787 --> 00:36:40,363
Don't know. I don't know. I'm sorry.

738
00:36:40,920 --> 00:36:42,661
Yeah, I forgive you. It's fine.

739
00:36:42,822 --> 00:36:46,116
I just don't like like that's you're
talking about being able to see what it's

740
00:36:46,116 --> 00:36:48,508
doing. Like if you can see what it's
thinking,

741
00:36:48,749 --> 00:36:51,942
that might also be kind of helpful to like

742
00:36:51,891 --> 00:36:52,172
Don't

743
00:36:52,172 --> 00:36:53,645
I don't know that would be helpful.

744
00:36:53,646 --> 00:36:54,788
for instance when I'm using

745
00:36:54,384 --> 00:36:54,788
Okay.

746
00:36:54,788 --> 00:36:57,926
a coding assistant, it's nice for it to
say here's what I'm trying.

747
00:36:58,863 --> 00:36:59,953
Right, right. Yeah.

748
00:37:01,054 --> 00:37:01,224
And

749
00:37:01,224 --> 00:37:03,112
and and it's also horrifying, like,

750
00:37:03,354 --> 00:37:05,260
I don't think I gave you access to that.

751
00:37:05,799 --> 00:37:06,129
Yeah,

752
00:37:06,129 --> 00:37:09,771
right. No, don't don't hack into hugging
face,

753
00:37:09,771 --> 00:37:10,222
please.

754
00:37:10,140 --> 00:37:10,356
Ha

755
00:37:10,356 --> 00:37:10,655
ha.

756
00:37:10,655 --> 00:37:15,999
Yeah, so one thing I d I I love about your
presentation you did at iOSDevUK

757
00:37:15,999 --> 00:37:19,602
was the whole stuff about Van Gogh and
like images and stuff.

758
00:37:19,602 --> 00:37:21,704
there's a lot you can do now with images.

759
00:37:22,244 --> 00:37:24,386
and I like that you talked about the whole
tool thing.

760
00:37:25,506 --> 00:37:28,318
what I'll let you go ahead and expand on
that.

761
00:37:28,318 --> 00:37:29,979
Like you so it could be part of the
prompt,

762
00:37:29,979 --> 00:37:31,270
it could be part of the result.

763
00:37:31,631 --> 00:37:32,631
What else?

764
00:37:32,850 --> 00:37:35,311
And so the result is one of the things
that's new.

765
00:37:35,612 --> 00:37:37,612
the way they did images in the past,

766
00:37:38,013 --> 00:37:40,934
now they're you using Image Playground to
do a lot of that.

767
00:37:41,194 --> 00:37:47,257
And so you can either have it look at an
image and then take your prompt

768
00:37:47,257 --> 00:37:50,159
and the prompt in that image, which you
put in as an attachment,

769
00:37:51,020 --> 00:37:52,720
is is what generates the image,

770
00:37:53,261 --> 00:37:56,102
or you could say, Hey, here's the image.

771
00:37:56,420 --> 00:37:58,812
I want you to describe it or extract
something.

772
00:37:58,812 --> 00:38:01,774
And now I want you to create a new image
based on this description,

773
00:38:02,035 --> 00:38:06,328
give very different results. And you can
ask for it like the Image Playground

774
00:38:06,328 --> 00:38:10,771
in different styles. And so they've sort
of gone all in on on Image Playground.

775
00:38:10,771 --> 00:38:15,005
And so on on iOS, you just display a sheet
and say,

776
00:38:15,285 --> 00:38:18,328
Hey, go go go think about this and and
show me the result here.

777
00:38:19,432 --> 00:38:21,200
Is it how is Image Playground?

778
00:38:21,200 --> 00:38:23,601
Is it like pretty good as far as
generating stuff?

779
00:38:25,798 --> 00:38:29,041
Fortunately it's not gonna put anybody out
of business anytime soon,

780
00:38:29,041 --> 00:38:29,411
but

781
00:38:29,117 --> 00:38:30,117
Right, right.

782
00:38:30,432 --> 00:38:33,281
but it's it's fun to play with and you can
say,

783
00:38:33,281 --> 00:38:38,078
you know, create a watercolor of this or a
cartoon image of this and you know,

784
00:38:37,405 --> 00:38:38,405
Okay, okay.

785
00:38:38,408 --> 00:38:41,701
i i for now it's a toy, I think it will
become better.

786
00:38:42,101 --> 00:38:45,364
But I love that they're giving us early
access to things like this.

787
00:38:46,504 --> 00:38:51,372
So w I I guess my c my question was and
then Image Playground that's all local,

788
00:38:51,372 --> 00:38:55,670
right? It doesn't need Private Cloud
Compute if you want to generate an image.

789
00:38:58,010 --> 00:38:59,437
I don't know the answer to that either.

790
00:38:59,467 --> 00:39:00,502
I haven't tried I've

791
00:38:59,526 --> 00:39:00,591
I mean I assume it's

792
00:39:00,612 --> 00:39:03,929
mostly local, but like I'm I'm just
curious.

793
00:39:02,968 --> 00:39:04,159
But I don't know if if

794
00:39:04,159 --> 00:39:05,920
if I've got no network access,

795
00:39:05,920 --> 00:39:08,352
I don't know if it works. I don't I've
never tried it.

796
00:39:08,954 --> 00:39:10,614
Okay. Yeah, okay.

797
00:39:09,733 --> 00:39:10,674
It's the problem,

798
00:39:10,674 --> 00:39:12,744
it's it's in it's kind of fun.

799
00:39:12,744 --> 00:39:15,056
It's the problem that we're always
connected.

800
00:39:15,493 --> 00:39:17,138
Right, that's exactly what I was gonna
say.

801
00:39:16,037 --> 00:39:17,858
in the in the early days

802
00:39:17,440 --> 00:39:17,858
Yeah.

803
00:39:17,858 --> 00:39:19,549
of the web, I lived in Cleveland,

804
00:39:19,549 --> 00:39:24,353
I would go out to O'Reilly and I'd fly
into San Francisco and like nobody there

805
00:39:24,353 --> 00:39:27,455
was using dial-up modems, and so they
didn't understand what it meant to make

806
00:39:27,455 --> 00:39:29,797
a website faster. You know, so

807
00:39:29,285 --> 00:39:30,115
Right.

808
00:39:30,115 --> 00:39:32,452
So you're right, these are things I'll
I'll have to test.

809
00:39:33,565 --> 00:39:34,798
Flip on the I mean what do you do?

810
00:39:34,798 --> 00:39:36,593
Airplane mode, I guess, to test it?

811
00:39:36,593 --> 00:39:38,087
Yeah. Yeah, exactly.

812
00:39:37,603 --> 00:39:38,355
Airplane mode

813
00:39:38,355 --> 00:39:41,232
or you know, like we used to do the
network link conditioner,

814
00:39:41,232 --> 00:39:42,535
which was always a great tool.

815
00:39:41,797 --> 00:39:43,456
right, right, right. Yeah, yeah,

816
00:39:43,456 --> 00:39:44,140
yeah.

817
00:39:44,140 --> 00:39:46,967
You know, what does this look like on on
on edge or something?

818
00:39:47,948 --> 00:39:53,055
speaking of which, I wanna hop back and
talk a little bit more about testing.

819
00:39:54,619 --> 00:39:58,001
so see for testing, the Instruments has
gotten much better,

820
00:39:58,001 --> 00:40:01,322
but I don't think of Instruments as a as a
testing framework.

821
00:40:01,882 --> 00:40:08,135
The problem with testing Apple
Intelligence is it may not give the same

822
00:40:08,135 --> 00:40:11,707
answer every time you ask a prompt with a
set of instructions.

823
00:40:12,027 --> 00:40:15,308
And so you can't test it the way you would
do a Swift test,

824
00:40:15,308 --> 00:40:17,369
which says, I give you this input,

825
00:40:17,369 --> 00:40:22,091
I expect this output. And so you have to
use a different style.

826
00:40:22,571 --> 00:40:26,975
And it actually is a way of also making
your instructions and your prompts better.

827
00:40:27,196 --> 00:40:29,377
It's this new framework called
evaluations.

828
00:40:29,958 --> 00:40:34,913
And again, they've used a term that's
that's common in this in this field,

829
00:40:34,913 --> 00:40:38,557
which is nice. and and they've also used
the term hill climbing,

830
00:40:38,557 --> 00:40:44,003
which is as you see what kind of responses
you're getting to these these prompts

831
00:40:44,003 --> 00:40:46,315
and these instructions, you can

832
00:40:46,869 --> 00:40:49,982
Tailor your instructions to be better and
get better results.

833
00:40:49,982 --> 00:40:51,804
And so that's the hill climbing part.

834
00:40:52,025 --> 00:40:55,929
And so the test is you're testing
something that's probabilistic.

835
00:40:56,050 --> 00:40:58,442
So you're, you're, you can't just say,

836
00:40:58,442 --> 00:41:01,605
well, it passed, I'm done. Test it again.

837
00:41:01,605 --> 00:41:05,440
Test it many times, test it many different
ways because it's going to come

838
00:41:05,440 --> 00:41:08,373
out different. And so your whole strategy
is different.

839
00:41:09,146 --> 00:41:12,480
Yeah, I love that. Would you I I think I
asked you this,

840
00:41:12,480 --> 00:41:14,232
but I'll ask you on the podcast.

841
00:41:14,974 --> 00:41:18,889
You wouldn't run this along with your
typical unit test because

842
00:41:18,889 --> 00:41:20,901
of the probabilistic nature of it,

843
00:41:20,901 --> 00:41:23,685
correct? But you would run it maybe.

844
00:41:22,701 --> 00:41:23,712
I wouldn't also

845
00:41:23,712 --> 00:41:26,013
be also because it takes so much longer,

846
00:41:26,134 --> 00:41:31,433
but but what drives the evaluations are
Swift tests.

847
00:41:31,753 --> 00:41:35,356
And so you write your evaluations which
set everything up and and have

848
00:41:35,356 --> 00:41:40,289
all the structure, and then you have a
Swift test that calls that and then runs

849
00:41:40,289 --> 00:41:45,311
them. And so it's it's to me like if
you've done testing a long time,

850
00:41:45,512 --> 00:41:48,894
the difference between unit tests and sort
of end-to-end tests.

851
00:41:49,278 --> 00:41:52,002
You're not going to run the end-to-end
tests every time,

852
00:41:52,222 --> 00:41:55,326
but you're going to run them nightly or
you're going to run them regularly.

853
00:41:55,547 --> 00:42:00,553
For me, I'm running the these evaluations
as I'm developing the prompts

854
00:42:00,553 --> 00:42:05,079
and the instructions. Once I've got it
once I've got my app in production,

855
00:42:05,120 --> 00:42:07,012
I don't know that I'm running them very
much at all.

856
00:42:07,932 --> 00:42:09,473
Yeah, that makes a lot of sense.

857
00:42:09,693 --> 00:42:12,355
And it's almost like it's almost like UI
tests too,

858
00:42:12,355 --> 00:42:14,197
where it's like you don't want to run it
every time,

859
00:42:14,197 --> 00:42:17,379
just run it every so often to make sure
everything is working.

860
00:42:18,052 --> 00:42:21,675
what what do you mean by made up answers?

861
00:42:24,019 --> 00:42:26,008
You said, What about made up answers?

862
00:42:26,873 --> 00:42:29,022
FM is not designed for world knowledge.

863
00:42:30,421 --> 00:42:34,032
So I have an example before there was
Private Cloud Compute,

864
00:42:34,042 --> 00:42:37,923
where you go into a bookstore and you ask
the friendly bookseller,

865
00:42:38,203 --> 00:42:39,924
Hey, can you suggest three books,

866
00:42:39,924 --> 00:42:45,346
say, in the mystery genre? And because it
doesn't have world world knowledge,

867
00:42:45,586 --> 00:42:48,007
sometimes it would just make up books.

868
00:42:48,308 --> 00:42:49,598
Like in mystery, it would say,

869
00:42:49,598 --> 00:42:53,009
This book is by John Detective or Amy
Murderer,

870
00:42:53,150 --> 00:42:55,310
and you go, That's not a real book.

871
00:42:55,511 --> 00:42:56,433
so

872
00:42:56,433 --> 00:42:59,516
The thing I talked about one thing I
didn't like about AI.

873
00:42:59,516 --> 00:43:03,490
The other thing I don't like about AI is
it never says I don't know.

874
00:43:03,246 --> 00:43:04,277
Don't know. Yeah.

875
00:43:04,171 --> 00:43:04,391
It

876
00:43:04,391 --> 00:43:08,976
never says I don't know. And so it knows
that it doesn't know because it knows

877
00:43:09,217 --> 00:43:11,299
its level of probability of certainty.

878
00:43:11,699 --> 00:43:13,961
And so you I would love to be able to say,

879
00:43:14,142 --> 00:43:16,064
if you're less than this certain,

880
00:43:16,504 --> 00:43:19,127
just sit there. Just tell me you don't
know.

881
00:43:19,820 --> 00:43:21,391
Yeah, yeah, one hundred percent.

882
00:43:21,391 --> 00:43:24,062
There's a vertical and horizontal question
I have.

883
00:43:24,523 --> 00:43:27,645
is this available? Yeah,

884
00:43:25,814 --> 00:43:26,930
Do do I have to turn?

885
00:43:27,645 --> 00:43:28,565
I know. Sound like

886
00:43:28,565 --> 00:43:30,188
a business speak garbage.

887
00:43:31,806 --> 00:43:36,471
Is it available on all platforms and OSs?

888
00:43:36,751 --> 00:43:38,953
What's like what's the range on that?

889
00:43:38,953 --> 00:43:42,217
And then as far as you can plug in
different models.

890
00:43:42,737 --> 00:43:45,260
That's I guess what I meant by horizontal
and vertical.

891
00:43:46,877 --> 00:43:50,909
So the plugging in different models is an
aspect of this that I said

892
00:43:50,909 --> 00:43:53,060
is is not something I've played a lot
with,

893
00:43:53,120 --> 00:43:55,531
and it it's in my list of things,

894
00:43:55,531 --> 00:43:58,102
but it's it's part of foundation models,

895
00:43:58,102 --> 00:44:01,403
but it's also part of the other frameworks
that they introduced this year.

896
00:44:01,583 --> 00:44:05,148
And so and those are are promised to be
open sourced,

897
00:44:05,148 --> 00:44:08,229
which is kind of cool. as as for all
platforms,

898
00:44:08,709 --> 00:44:11,580
it's still Apple platforms. you know,

899
00:44:11,580 --> 00:44:12,691
that the the Apple

900
00:44:13,529 --> 00:44:18,122
Apple has actually been a very good open
source citizen when it comes to Swift

901
00:44:18,162 --> 00:44:20,293
and some of this these technologies,

902
00:44:20,293 --> 00:44:25,346
but their job is still to sell you Macs
and iPhones and watches.

903
00:44:24,948 --> 00:44:25,726
Yeah, yeah, yeah.

904
00:44:25,726 --> 00:44:27,747
I mean that's that's where their business
comes from.

905
00:44:28,067 --> 00:44:33,410
So it could run on the server side on
Apple Silicon.

906
00:44:33,630 --> 00:44:37,172
It can't yet run on the server side on
Linux.

907
00:44:37,873 --> 00:44:41,534
Some of their other APIs beyond foundation
models.

908
00:44:41,972 --> 00:44:44,652
are acknowledging that as much as they've
tried,

909
00:44:44,652 --> 00:44:47,443
a lot of people are still doing ML using
Python.

910
00:44:47,764 --> 00:44:50,264
And you're just not gonna get those guys
to change.

911
00:44:50,444 --> 00:44:54,726
You know, Chris Lattner went over to
Google and demonstrated much better

912
00:44:54,846 --> 00:44:57,886
performance with this stuff using Swift.

913
00:44:58,487 --> 00:44:59,997
And it just didn't sway people.

914
00:44:59,997 --> 00:45:03,648
It's and it's very hard when you've got
things like Jupyter notebooks and you've

915
00:45:03,648 --> 00:45:06,449
got, you know, all all the Python
libraries,

916
00:45:06,569 --> 00:45:09,940
it's very hard to get people to to to move
over.

917
00:45:10,422 --> 00:45:13,324
I think they're doing great work and I
think they're good citizen.

918
00:45:13,984 --> 00:45:18,797
And what you're seeing in some of the new
APIs is you're able to integrate better

919
00:45:18,797 --> 00:45:21,079
with stuff done in Python and you're able

920
00:45:20,730 --> 00:45:21,079
Yeah.

921
00:45:21,079 --> 00:45:27,093
to use other models. So they they've got
this sort of protocols that other

922
00:45:27,093 --> 00:45:29,734
languages and other agents can plug into.

923
00:45:29,935 --> 00:45:31,696
And so you're gonna see more of this,

924
00:45:31,836 --> 00:45:33,867
but that's sort of beyond what I'm playing
with now.

925
00:45:36,141 --> 00:45:38,763
you said it's available on all platforms,

926
00:45:38,863 --> 00:45:42,205
so like this would I could do this on the
watch.

927
00:45:41,180 --> 00:45:42,666
All Apple platforms.

928
00:45:42,825 --> 00:45:44,546
Right. I could do this on the watch,

929
00:45:44,606 --> 00:45:48,209
I could do this on Apple Vision,

930
00:45:48,229 --> 00:45:52,061
I could do this on tvOS, or like that's
what I'm curious about.

931
00:45:51,211 --> 00:45:52,142
So again again,

932
00:45:52,142 --> 00:45:55,965
the only things I've played with myself
are macOS and iOS,

933
00:45:56,325 --> 00:45:58,467
which I guess is iPad OS as well.

934
00:45:58,687 --> 00:46:02,490
but I I'm I'm guessing, I don't know,

935
00:46:02,590 --> 00:46:05,913
but I'm guessing because of what they
announced with the new watch

936
00:46:05,913 --> 00:46:08,915
12 and and series support for that,

937
00:46:08,916 --> 00:46:12,178
that I'm guessing you could use this on
the watch as well,

938
00:46:12,178 --> 00:46:13,178
but I've not done it.

939
00:46:13,645 --> 00:46:18,392
Okay, okay. We'll we'll have to follow up
on that 'cause I'm curious myself

940
00:46:18,392 --> 00:46:21,636
how far you can take it. Without killing
the battery.

941
00:46:19,959 --> 00:46:22,065
Boy, this is like an hour of I don't know,

942
00:46:22,065 --> 00:46:23,065
isn't it?

943
00:46:23,199 --> 00:46:26,765
No no no hey that's what makes you better
than AI.

944
00:46:27,105 --> 00:46:30,199
Cause if I had you on if I had you on as a

945
00:46:29,407 --> 00:46:30,326
I I need a

946
00:46:30,326 --> 00:46:32,131
I need a t shirt that says that.

947
00:46:32,131 --> 00:46:33,133
Better than AI.

948
00:46:32,589 --> 00:46:34,311
Yeah, yeah that's right.

949
00:46:35,113 --> 00:46:38,558
Daniel, anything else you want cover
before I let you close out and plug

950
00:46:38,677 --> 00:46:39,677
yourself?

951
00:46:40,493 --> 00:46:41,792
No, e has this

952
00:46:41,792 --> 00:46:42,869
made any sense at all?

953
00:46:43,571 --> 00:46:45,583
Yeah, it's been amazing. I guess no,

954
00:46:45,583 --> 00:46:51,231
one more question. Where do you where are
you like this is amazing developers need

955
00:46:51,231 --> 00:46:53,734
to use this that they're missing out on?

956
00:46:53,734 --> 00:46:58,911
Where do you see opportunities for this to
be used that folks aren't really thinking

957
00:46:58,911 --> 00:46:59,911
of?

958
00:47:00,541 --> 00:47:04,302
So I think a lot of this,

959
00:47:04,982 --> 00:47:11,204
the answer is future, because I think
we're gonna get very suspicious

960
00:47:11,204 --> 00:47:15,045
of AI that doesn't run on our device that
we can't run privately.

961
00:47:15,185 --> 00:47:20,227
So I think Apple hasn't nailed this this
message yet,

962
00:47:20,227 --> 00:47:21,767
it hasn't really hammered it home,

963
00:47:22,207 --> 00:47:25,048
but the fact that this stuff runs on your
device,

964
00:47:25,748 --> 00:47:28,809
given Private Cloud Compute, given that
you can call it to other things.

965
00:47:29,051 --> 00:47:34,543
you can still do an awful lot of work on
your device and so you can access private

966
00:47:34,803 --> 00:47:37,003
information without exposing it.

967
00:47:37,284 --> 00:47:42,175
Like I get freaked out with some unnamed
systems that it knows an awful lot about

968
00:47:42,175 --> 00:47:47,156
me and it seems to be cavalier about how
it shares it with other aspects

969
00:47:47,156 --> 00:47:53,258
of the business. I think Apple's been a
very good citizen about privacy and that's

970
00:47:53,258 --> 00:47:55,549
why I think Apple intelligence

971
00:47:56,449 --> 00:47:59,170
is something that we should look at and
and really care about.

972
00:47:59,450 --> 00:48:05,063
What it doesn't allow are shops where they
have to do cross platform apps and they

973
00:48:05,063 --> 00:48:08,334
have to do the same thing on different
like Android and iPhone.

974
00:48:08,775 --> 00:48:10,795
That's that's something I can help you
with.

975
00:48:11,136 --> 00:48:14,437
But shame shame on you, you shouldn't be
doing Android anyway.

976
00:48:15,103 --> 00:48:18,506
That's right. That's right. Don't ever do
whatever it is,

977
00:48:18,506 --> 00:48:22,876
like what forty percent of the smartphone
market or sixty percent of whatever it's

978
00:48:22,876 --> 00:48:27,343
at right now. well, it's funny you
mentioned privacy being a concern,

979
00:48:27,343 --> 00:48:31,996
but I do think I do think the chickens are
gonna come home to roost and I think cost

980
00:48:31,996 --> 00:48:34,618
will be a concern as well where local
models will be

981
00:48:36,020 --> 00:48:38,211
I think cost is gonna be a huge concern.

982
00:48:38,211 --> 00:48:43,945
Apple did that demo and everyone laughed
because the amount of Mac power that

983
00:48:43,945 --> 00:48:48,868
it needed to do that on device demo and I
and I totally understand that,

984
00:48:48,968 --> 00:48:53,830
but at some point the cost of tokens for
big companies is gonna be higher than

985
00:48:54,131 --> 00:48:56,472
investing in that hardware and doing it
locally.

986
00:48:56,472 --> 00:49:00,595
I just recently went to a a Java
Enterprise conference and a lot of their

987
00:49:00,595 --> 00:49:02,826
talk was on device work and and

988
00:49:03,148 --> 00:49:05,222
you know, controlling the cost of those
tokens.

989
00:49:05,222 --> 00:49:08,259
And I I think you're right. I think this
is gonna be a big issue.

990
00:49:08,560 --> 00:49:11,045
I don't think we're paying the price of
tokens right now.

991
00:49:11,346 --> 00:49:12,769
I think we're getting discounts

992
00:49:11,784 --> 00:49:12,769
No way.

993
00:49:12,769 --> 00:49:14,262
that that aren't gonna sustain.

994
00:49:14,878 --> 00:49:17,705
Right, one hundred percent. Daniel,

995
00:49:17,826 --> 00:49:19,390
thank you so much for coming on.

996
00:49:19,731 --> 00:49:21,856
what do you got going on as far as books,

997
00:49:21,856 --> 00:49:23,100
workshops, and talks?

998
00:49:23,924 --> 00:49:28,877
Well, thank you for asking. so I'm
finishing my App Intents book and then

999
00:49:28,877 --> 00:49:30,758
I'm starting my Foundation Models book.

1000
00:49:30,758 --> 00:49:33,950
And so if people want to follow along in
tech review,

1001
00:49:33,960 --> 00:49:36,762
just email me and I'm happy to to let you
follow along.

1002
00:49:36,842 --> 00:49:39,463
I'll probably start that in in December
sometime.

1003
00:49:40,504 --> 00:49:43,725
Workshops if I'm invited. I love to give
workshops.

1004
00:49:44,065 --> 00:49:47,637
as things change, I'm looking at different
style books and I'm trying to figure

1005
00:49:47,637 --> 00:49:49,258
out what I want to be when I grow up.

1006
00:49:49,258 --> 00:49:51,429
And so I'm considering writing

1007
00:49:52,017 --> 00:49:56,020
a book on calculus that I've wanted to
write forever and a book on statistics

1008
00:49:56,260 --> 00:49:59,874
and maybe some graphic novels or some that
have more artwork,

1009
00:50:00,074 --> 00:50:01,895
even though I draw like a two year old.

1010
00:50:01,956 --> 00:50:06,369
So I'm having a ball. I'm learning new
things all the time.

1011
00:50:06,369 --> 00:50:07,810
I love talking to people like you.

1012
00:50:07,810 --> 00:50:11,913
I love our community and I love that I get
to keep going to conferences.

1013
00:50:11,913 --> 00:50:13,824
So that's pretty much what I'm doing.

1014
00:50:14,603 --> 00:50:16,454
Daniel, thank you so much for coming on.

1015
00:50:16,554 --> 00:50:21,215
it's been great. definitely check out Dim
Sum Thinking and his material there.

1016
00:50:21,215 --> 00:50:23,316
There's a lot of good stuff. yeah.

1017
00:50:23,536 --> 00:50:25,607
thank you for joining me. if you like this
episode,

1018
00:50:25,607 --> 00:50:28,238
like and subscribe on YouTube or give me a
review,

1019
00:50:28,238 --> 00:50:29,979
please. share it with your friends,

1020
00:50:29,979 --> 00:50:34,081
share it with your family. and also thanks
to all the Patreon supporters as well.

1021
00:50:34,301 --> 00:50:35,391
if you haven't caught

1022
00:50:35,679 --> 00:50:40,003
my newsletter, definitely take it some
time and subscribe to my newsletter.

1023
00:50:40,003 --> 00:50:41,614
Link will be in the show notes below.

1024
00:50:42,095 --> 00:50:44,276
Thank you everybody and I look forward to
talking to you later.

1025
00:50:44,329 --> 00:50:45,329
Bye.