1
00:00:00,050 --> 00:00:02,319
I feel like our bar as an
industry needs to be higher.

2
00:00:02,319 --> 00:00:05,889
And I, I honestly think that web
development is to blame for some of that.

3
00:00:05,929 --> 00:00:09,299
That's my spicy take because it makes
it easy for us to throw up our hands.

4
00:00:09,880 --> 00:00:13,050
And say, I can't go past
that layer of abstraction.

5
00:00:13,240 --> 00:00:15,650
Whereas in the native
world, you have a problem.

6
00:00:15,670 --> 00:00:19,310
You can dig into those, the C binaries.

7
00:00:19,350 --> 00:00:22,980
it might be hard, but you have the
power to go in and change things.

8
00:00:22,980 --> 00:00:25,210
Whereas on the web, it's
like, it just works that way.

9
00:00:25,210 --> 00:00:25,709
It sucks.

10
00:00:26,070 --> 00:00:26,890
Like too bad.

11
00:00:26,970 --> 00:00:28,600
I think we need aim, aim higher.

12
00:00:28,740 --> 00:00:30,870
Welcome to the local-first FM podcast.

13
00:00:31,130 --> 00:00:34,049
I'm your host, Johannes Schickling,
and I'm a web developer, a

14
00:00:34,049 --> 00:00:37,180
startup founder, and love the
craft of software engineering.

15
00:00:37,610 --> 00:00:41,400
For the past few years, I've been on a
journey to build a modern, high quality

16
00:00:41,400 --> 00:00:43,379
music app using web technologies.

17
00:00:43,570 --> 00:00:47,510
And in doing so, I've been falling down
the rabbit hole of local-first software.

18
00:00:48,030 --> 00:00:50,970
This podcast is your invitation
to join me in that journey.

19
00:00:51,570 --> 00:00:55,820
In this episode, I'm speaking to James
Long, the creator of the local-first app.

20
00:00:56,105 --> 00:01:00,335
Actual Budget and the absurd-sql
project, which helped to pave the way

21
00:01:00,335 --> 00:01:02,275
to bring back SQLite to the browser.

22
00:01:02,765 --> 00:01:06,385
In this conversation, we go deep on
his journey, building Actual Budget,

23
00:01:06,625 --> 00:01:10,704
including implementing a syncing
solution from scratch and expanding

24
00:01:10,705 --> 00:01:14,725
from an Electron app to mobile and the
web while reusing most of the code.

25
00:01:15,385 --> 00:01:18,455
Before getting started, also a
big thank you to Expo and Crab

26
00:01:18,455 --> 00:01:20,155
Nebula for supporting this podcast.

27
00:01:20,755 --> 00:01:22,755
And now my interview with James.

28
00:01:24,059 --> 00:01:24,899
Hey, James.

29
00:01:25,009 --> 00:01:25,849
So good to have you.

30
00:01:26,589 --> 00:01:28,132
Hey,  Thanks for having me.

31
00:01:28,702 --> 00:01:32,612
So I've been a long time
fan of your prior work.

32
00:01:32,662 --> 00:01:36,582
I think this has really been like
the first time where I've seen an

33
00:01:36,872 --> 00:01:39,352
Actual local-first app pun intended.

34
00:01:39,372 --> 00:01:43,802
You've been working on the Actual
Budget app in the past, and which

35
00:01:43,802 --> 00:01:48,355
has led to quite a few, technical
innovations, particularly for the web.

36
00:01:48,695 --> 00:01:53,005
So that's what I'm looking forward to
exploring today, but I'm curious what has

37
00:01:53,005 --> 00:01:57,675
led you to exploring local-first and what
has led you to work on Actual Budget.

38
00:01:58,019 --> 00:01:58,369
Sure.

39
00:01:58,399 --> 00:02:01,709
So first of all, I'll just state
that Actual Budget is, uh, I

40
00:02:01,709 --> 00:02:03,109
don't actually work on it anymore.

41
00:02:03,119 --> 00:02:05,029
And I open sourced it about two years ago.

42
00:02:05,029 --> 00:02:08,258
And so the community has taken it
over and done a great job with it too.

43
00:02:08,569 --> 00:02:11,349
Um, but I started it around 2017.

44
00:02:11,358 --> 00:02:16,089
And back then I just really wanted
a local app, just the web back

45
00:02:16,089 --> 00:02:19,509
then was even worse than it is
now, just like development-wise.

46
00:02:19,529 --> 00:02:22,259
And I don't know, I, it
just, It didn't excite me.

47
00:02:22,349 --> 00:02:23,629
It's a personal finance manager.

48
00:02:23,659 --> 00:02:24,999
It felt like it should be local.

49
00:02:24,999 --> 00:02:30,029
It felt like I should be able to throw raw
SQL queries at it and get my data back.

50
00:02:30,189 --> 00:02:32,198
So it just felt like a very good fit.

51
00:02:32,449 --> 00:02:35,359
And using Electron back then was
amazing because I could just like

52
00:02:35,378 --> 00:02:41,179
load up SQLite and load a SQLite
database from a local disk and use

53
00:02:41,179 --> 00:02:43,069
the native SQLite C bindings, right?

54
00:02:43,069 --> 00:02:44,449
It was, it was fantastic.

55
00:02:44,509 --> 00:02:48,229
So I built a basic local app just
because I wanted it to be local

56
00:02:48,259 --> 00:02:50,309
and I just didn't want to deal
with like hosting it somewhere and

57
00:02:50,309 --> 00:02:51,699
I was like, this is just for me.

58
00:02:51,699 --> 00:02:53,799
This is just a fun thing
for me for certain apps.

59
00:02:53,809 --> 00:02:55,089
I've always liked the idea of it.

60
00:02:55,089 --> 00:02:56,698
Just like being super local.

61
00:02:56,918 --> 00:02:58,449
Uh, you, you own everything.

62
00:02:58,458 --> 00:03:02,419
It's not dependent on anything else and
that you just have raw access to the data.

63
00:03:02,858 --> 00:03:05,469
Obviously, at some point, I hit this
problem where it's like, well, shoot, if

64
00:03:05,469 --> 00:03:07,434
I drop my laptop, All my data is gone.

65
00:03:07,474 --> 00:03:07,824
Right.

66
00:03:07,894 --> 00:03:11,814
or like my wife just wants to check
one charge somewhere and she's on

67
00:03:11,814 --> 00:03:13,134
her laptop and she can't do that.

68
00:03:13,464 --> 00:03:16,873
And so, uh, eventually I was like,
well, crap, if I want to make a

69
00:03:16,873 --> 00:03:20,104
business out of this, especially I got
to solve the collaboration problem.

70
00:03:20,234 --> 00:03:22,013
And so that's what led
me down into syncing.

71
00:03:22,014 --> 00:03:24,494
And so I investigated a couple of things
and it worked out really, really well.

72
00:03:24,564 --> 00:03:27,554
And so I went down that path from
there and I just continually.

73
00:03:27,889 --> 00:03:28,709
Invested into it.

74
00:03:28,809 --> 00:03:29,349
And it was fun.

75
00:03:29,479 --> 00:03:32,979
That is super impressive, particularly
given that you've already started

76
00:03:32,979 --> 00:03:38,659
working on that in 2017, just for
reference, the official local-first essay

77
00:03:38,659 --> 00:03:41,519
by Inc&Switch, uh, came out in 2019.

78
00:03:41,648 --> 00:03:46,819
So it looks like you've been on the same
journey, uh, maybe aware, maybe unaware

79
00:03:46,819 --> 00:03:51,569
that other people have been also exploring
the ideas of local-first, and then you've

80
00:03:51,569 --> 00:03:54,459
just arrived on very similar conclusions.

81
00:03:54,809 --> 00:03:59,729
So I'm curious to learn more about
the technical challenges that

82
00:03:59,739 --> 00:04:03,829
you've been facing, really, uh,
innovating on so many fronts there

83
00:04:03,839 --> 00:04:05,959
to make this, this vision a reality.

84
00:04:06,059 --> 00:04:10,049
And I would like to better understand
also one aspect you've been mentioning

85
00:04:10,088 --> 00:04:12,654
that the, uh, App should work local.

86
00:04:12,704 --> 00:04:17,154
Are you referring to an app here as a
desktop app from something like Electron?

87
00:04:17,234 --> 00:04:21,524
Or are you also talking about a
more of like a progressive web

88
00:04:21,524 --> 00:04:23,784
app that runs also in the browser?

89
00:04:23,984 --> 00:04:27,263
So back then at the time, I was mostly
thinking just like a desktop app.

90
00:04:27,263 --> 00:04:28,833
I want to see the icon in the doc.

91
00:04:28,834 --> 00:04:31,134
I want, I wanted it to just be native.

92
00:04:31,214 --> 00:04:32,164
Native-ish.

93
00:04:32,184 --> 00:04:35,364
I know we're kind of faking that
with Electron, but I wanted just

94
00:04:35,384 --> 00:04:38,824
an app that I can click and I
open my finances and I search and

95
00:04:38,824 --> 00:04:40,404
then I do command Q and it's gone.

96
00:04:40,444 --> 00:04:42,284
So back then it was more a desktop app.

97
00:04:42,354 --> 00:04:44,473
And eventually I did port
everything to the web.

98
00:04:44,474 --> 00:04:47,723
And that was a whole nother thing because
it's really hard to compete with the web.

99
00:04:47,903 --> 00:04:51,494
I mean, just not only
distribution, uh, but just like,

100
00:04:51,494 --> 00:04:52,624
that's just where everybody is.

101
00:04:52,714 --> 00:04:56,029
So it's evolved a little bit more
to now mean probably Like when I

102
00:04:56,029 --> 00:04:59,859
say local app, it could totally be
a web app as well, or a mobile app.

103
00:04:59,939 --> 00:05:01,999
Like all of it should just be
something that can work locally.

104
00:05:01,999 --> 00:05:04,969
I think the web still needs to catch
up in a lot of ways to get there.

105
00:05:05,089 --> 00:05:07,408
But at this point, I, when
I say it, I mean like a.

106
00:05:07,819 --> 00:05:09,519
It could be a web app or a desktop app.

107
00:05:09,819 --> 00:05:14,409
So before diving a little bit more into
the nitty gritties of the technological

108
00:05:14,409 --> 00:05:17,899
choices that you've made, you've
already mentioned that you've been

109
00:05:17,909 --> 00:05:20,489
choosing SQLite for the data layer.

110
00:05:20,659 --> 00:05:23,739
So I think the, the web world is.

111
00:05:23,909 --> 00:05:28,749
Sometimes like divided across
front-end and back-end, I think using

112
00:05:28,769 --> 00:05:33,909
a SQL database is much more common
on the, on the back-end and on the

113
00:05:33,909 --> 00:05:39,029
front-end, I think you're more used
to things like Redux, MobX, et cetera.

114
00:05:39,619 --> 00:05:43,229
And given your use case of
an Electron app, this is sort

115
00:05:43,229 --> 00:05:45,569
of like a murky in between.

116
00:05:45,829 --> 00:05:49,309
So I'm very curious to hear
more about your intuition.

117
00:05:49,319 --> 00:05:53,849
What led you to wanting to use
a SQLite database that I guess

118
00:05:53,999 --> 00:05:58,578
in the realm of Electron rather
falls into the front-end realm.

119
00:05:59,028 --> 00:06:01,669
So I'm curious what led
that design decision.

120
00:06:01,919 --> 00:06:05,529
Well, like I said, I mean, for the data
layer itself, it was just so obvious

121
00:06:05,539 --> 00:06:07,449
to me that it's a small set of data.

122
00:06:07,559 --> 00:06:11,418
Some, some people worry about SQLite not
scaling well to like millions of rows.

123
00:06:11,689 --> 00:06:15,208
I think I have at this point, nine years
of my transactions in there and I, I'd

124
00:06:15,208 --> 00:06:18,789
have to count, but it's, I think it's
in the like tens of thousands, uh, aims.

125
00:06:18,798 --> 00:06:20,458
So this is like not a large set of data.

126
00:06:20,469 --> 00:06:22,589
SQLite would just seem like
such the perfect fit for it.

127
00:06:22,824 --> 00:06:27,544
But then once you have SQLite and it's
a local app and then you're writing in

128
00:06:27,544 --> 00:06:32,214
front of components, it seems silly to
have to design this API layer that's

129
00:06:32,214 --> 00:06:37,994
either like a WebSockets messaging layer
or like an HTTP like URL based API.

130
00:06:38,193 --> 00:06:43,344
Like it seems purely you're going
through the motions to have an API

131
00:06:43,554 --> 00:06:47,114
that literally just is intercepted
locally and then runs the data locally.

132
00:06:47,404 --> 00:06:48,934
So once and then like you get that.

133
00:06:49,209 --> 00:06:50,379
Data back as like JSON.

134
00:06:50,389 --> 00:06:53,839
So you, you query something with
like an HTTP or like web socket

135
00:06:53,949 --> 00:06:55,089
API, and then you get the data back.

136
00:06:55,359 --> 00:06:59,899
It's like you, why not just go ahead
and just like write SQL queries, right?

137
00:06:59,999 --> 00:07:00,989
In those components.

138
00:07:01,249 --> 00:07:04,469
It literally makes no difference
because it's intercepted locally.

139
00:07:04,639 --> 00:07:07,764
So then I started kind of,
Exposing SQLite even more and more.

140
00:07:07,894 --> 00:07:11,954
And I ended up coming up with my own
little data, data querying language,

141
00:07:11,954 --> 00:07:15,933
because when you're doing really
quick stuff, it is nice to, it's not

142
00:07:15,933 --> 00:07:19,044
an ORM at all, but it's basically
like a query builder type thing.

143
00:07:19,094 --> 00:07:21,634
It's, it's a pretty common thing
that I think a lot of people do.

144
00:07:21,634 --> 00:07:23,834
I think there's a library called Knex.

145
00:07:23,854 --> 00:07:26,819
K N E X that like helps you
build sort of like that.

146
00:07:26,819 --> 00:07:30,019
But I, I built it because I like
building stuff myself and it was, it

147
00:07:30,019 --> 00:07:32,389
was a pretty interesting thing, but it
does basically let you construct SQL

148
00:07:32,858 --> 00:07:35,599
things and then it hands that to the
backend and it knows how to execute it.

149
00:07:35,608 --> 00:07:37,798
The other thing that it knows how
to do is it knows how to do like

150
00:07:38,058 --> 00:07:40,769
live queries, which you don't get
with like raw SQL queries, unless

151
00:07:40,769 --> 00:07:43,999
you parse the query or something,
but it knows which tables to watch.

152
00:07:44,289 --> 00:07:47,879
And so as other syncing messages come in,
the data will just automatically update.

153
00:07:47,929 --> 00:07:50,599
And so, yeah, it just felt
like a really nice fit.

154
00:07:51,079 --> 00:07:52,389
Since everything is local anyway.

155
00:07:52,659 --> 00:07:54,149
That sounds super compelling.

156
00:07:54,179 --> 00:07:59,538
And I think we've arrived at a few
similar conclusions given that with my

157
00:07:59,539 --> 00:08:04,589
work on Riffle and LiveStore, I've also
built some similar aspects such as like

158
00:08:04,589 --> 00:08:06,699
the reactivity that you've pointed at.

159
00:08:06,879 --> 00:08:10,009
I'm curious, like how you actually
went about implementing that since

160
00:08:10,439 --> 00:08:15,449
for the listeners who might not
be as aware of all the SQLite

161
00:08:15,559 --> 00:08:20,569
internals, SQLite itself doesn't
really help you much with reactivity.

162
00:08:20,579 --> 00:08:24,819
There is a few hook points, but,
uh, you get to roll quite a bit

163
00:08:24,889 --> 00:08:26,608
of the stuff yourself on top.

164
00:08:26,609 --> 00:08:28,258
So I'm curious how you went about that.

165
00:08:28,378 --> 00:08:29,098
Yeah, that part.

166
00:08:29,118 --> 00:08:30,958
Honestly, it's not super innovative.

167
00:08:30,998 --> 00:08:34,518
I actually remember around 2017
when I started doing this kind

168
00:08:34,518 --> 00:08:38,738
of a stuff, I went really deep on
SQLite internals and there's a hook.

169
00:08:38,938 --> 00:08:42,598
I forget what it's called, but
it's like SQLite underscore pre

170
00:08:42,638 --> 00:08:44,618
post update or something like that.

171
00:08:44,618 --> 00:08:47,838
There's a hook that you get that gets
called whenever, whenever an update

172
00:08:47,838 --> 00:08:51,508
happens and you get like the data
before and the data after, but it turned

173
00:08:51,508 --> 00:08:52,998
out to just be a really weird hook.

174
00:08:53,178 --> 00:08:55,858
When you want to do live
reactive stuff, you, you want

175
00:08:55,858 --> 00:08:57,118
it to be somewhat fine grained.

176
00:08:57,129 --> 00:08:59,723
It doesn't have to be super fine
grained, but if, if you want A change

177
00:08:59,723 --> 00:09:03,583
happens, then the, the entire, like
all of the data on the app re renders.

178
00:09:03,853 --> 00:09:06,903
SQL is fast and it's
local, but it's still slow.

179
00:09:06,903 --> 00:09:10,153
If you're going to re render every
single time you change one little thing.

180
00:09:10,183 --> 00:09:13,723
And so you want to make sure that you'd
somewhat scoped to like the table that

181
00:09:13,723 --> 00:09:15,563
changed or something like that, at least.

182
00:09:15,783 --> 00:09:19,523
And that hook, like it
provided back like weird IDs.

183
00:09:19,553 --> 00:09:22,423
Like I could never figure out how
to actually get the data itself

184
00:09:22,423 --> 00:09:24,593
that changed, like this column.

185
00:09:24,808 --> 00:09:28,938
And this row changed from X
to Y it was like, I couldn't

186
00:09:28,948 --> 00:09:30,238
even get like that basic stuff.

187
00:09:30,238 --> 00:09:31,558
And maybe I was just doing it wrong.

188
00:09:31,798 --> 00:09:34,208
There are some hooks in
there that seem promising.

189
00:09:34,308 --> 00:09:37,878
But I think from what other people are
doing, they're using like, like the wall

190
00:09:37,878 --> 00:09:39,378
file to do like really interesting things.

191
00:09:39,378 --> 00:09:41,808
I think that seems like a
more reasonable approach.

192
00:09:41,888 --> 00:09:44,998
But overall, actually, honestly,
it's pretty, it's pretty basic.

193
00:09:45,178 --> 00:09:47,088
All updates have to go through an API.

194
00:09:47,128 --> 00:09:52,418
And so unfortunately, if you open up the
SQLite file locally, you can update it.

195
00:09:52,738 --> 00:09:54,968
But those messages will
not be synced around.

196
00:09:55,268 --> 00:09:59,138
You'll have to like reset the file
across all devices, which it's possible.

197
00:09:59,148 --> 00:10:01,758
And people have done that before, if they
really want to like mess with something.

198
00:10:02,048 --> 00:10:05,278
But, um, it's not a thing where
you can open up the SQL file and

199
00:10:05,318 --> 00:10:06,468
update things directly from there.

200
00:10:06,468 --> 00:10:11,308
You have to call the update function,
which then creates a bunch of like CRDT

201
00:10:11,318 --> 00:10:13,418
stuff, sends it out, and then actually.

202
00:10:13,643 --> 00:10:14,963
I take that back a little bit.

203
00:10:15,153 --> 00:10:19,033
You need to call the update function to
generate the CRDT, like the sync messages.

204
00:10:19,243 --> 00:10:21,913
And they go out to the server
and they get applied locally.

205
00:10:22,123 --> 00:10:24,773
It's the part of the pipeline
that applies the messages,

206
00:10:24,773 --> 00:10:26,743
which does, does the reactivity.

207
00:10:26,863 --> 00:10:28,953
That allows me to kind of
watch for changes, right?

208
00:10:28,953 --> 00:10:33,853
Because All mutations through the system,
even if they're just local mutations, go

209
00:10:33,853 --> 00:10:36,233
through this like CRDT messages system.

210
00:10:36,533 --> 00:10:41,923
And so you can see when column X
on table transactions changes, it's

211
00:10:41,923 --> 00:10:44,923
changing from this value and I have
the before value and after value.

212
00:10:45,173 --> 00:10:49,383
And so when those messages get pumped
through is when it fires off all

213
00:10:49,383 --> 00:10:52,963
of the listeners that are listening
for that small piece of data.

214
00:10:53,123 --> 00:10:54,863
And yeah, that's basically how it works.

215
00:10:55,213 --> 00:10:55,663
Got it.

216
00:10:55,673 --> 00:10:59,123
So you've captured most of that
and probably in, in JavaScript.

217
00:10:59,523 --> 00:11:03,943
And since you're all using, you're
building the library, you're building the

218
00:11:03,953 --> 00:11:06,023
app, you know how to use it correctly.

219
00:11:06,423 --> 00:11:09,953
So you've built a lightweight
query builder on top.

220
00:11:10,233 --> 00:11:12,073
How are you using that query builder then?

221
00:11:12,103 --> 00:11:14,903
Um, have you like wrapped
those in React hooks?

222
00:11:15,513 --> 00:11:20,313
Or how are you wiring up your data
into the UI directly at some point?

223
00:11:20,813 --> 00:11:25,513
Yeah, there's a useLiveQuery() hook,
and then there's a usePageQuery() hook.

224
00:11:25,573 --> 00:11:28,523
The page query one is interesting
because it allows you to like page

225
00:11:28,523 --> 00:11:31,873
in results, and it returns an object
that has like a dot next function.

226
00:11:32,073 --> 00:11:35,823
I mean, it just knows how to automatically
add the limit and, and, and offset.

227
00:11:35,973 --> 00:11:37,553
And it does some other
pretty fancy things too.

228
00:11:37,553 --> 00:11:40,588
And so they're just hooks
that Knows how to rerender the

229
00:11:40,588 --> 00:11:42,588
component when the data changes.

230
00:11:43,298 --> 00:11:43,678
Got it.

231
00:11:44,268 --> 00:11:49,378
I mean, uh, I've gone through a similar
journey of where I used like some

232
00:11:49,388 --> 00:11:52,858
state management libraries and react
front-ends in the past, and then,

233
00:11:53,218 --> 00:11:57,198
uh, actually being able to use and
embrace SQLite for all its benefits and

234
00:11:57,198 --> 00:12:01,258
the front-end is quite magical, like
little tricks that I found to be super

235
00:12:01,288 --> 00:12:03,458
compelling is that I can actually like.

236
00:12:03,513 --> 00:12:08,163
Touch the SQLite file, whether I can
like look at it and see the values being

237
00:12:08,173 --> 00:12:12,713
updated by the app, or also like go to
it and like delete it, reload the app.

238
00:12:13,033 --> 00:12:14,463
And I'm starting from scratch.

239
00:12:14,673 --> 00:12:17,493
Those little things, they're
just super compelling and make

240
00:12:17,493 --> 00:12:19,033
it super fun to work on the app.

241
00:12:19,373 --> 00:12:24,563
Is there some things like that, that
you found that just gave you like a

242
00:12:24,593 --> 00:12:28,903
really nice boost in your development,
like velocity and productivity and fun?

243
00:12:29,023 --> 00:12:29,373
Sure.

244
00:12:29,373 --> 00:12:29,693
Yeah.

245
00:12:29,693 --> 00:12:30,103
I mean, it's.

246
00:12:30,278 --> 00:12:33,998
It's totally true that it kind of
forces it to be like, if you're not

247
00:12:33,998 --> 00:12:37,828
using Postgres and you're using SQLite
instead, and they're all files, it's

248
00:12:37,838 --> 00:12:40,478
so easy to just like move things
around and create like a fresh file.

249
00:12:40,488 --> 00:12:42,858
The, the demo app literally
just copies like a demo.

250
00:12:42,858 --> 00:12:46,328
SQLite file into the
app's like database file.

251
00:12:46,328 --> 00:12:49,008
And if you're using Postgres, that
kind of stuff just gets a lot harder

252
00:12:49,048 --> 00:12:52,148
because it's so like process oriented
and like process based, you can't

253
00:12:52,148 --> 00:12:53,758
just easily do that kind of stuff.

254
00:12:53,818 --> 00:12:56,768
But like, and yeah, being, being
able to dig into things and do

255
00:12:56,828 --> 00:12:59,138
SQLite queries, it felt really fast.

256
00:12:59,408 --> 00:13:01,833
I think you, You know, if you're
developing the app, you have

257
00:13:01,833 --> 00:13:03,063
access to the Postgres database.

258
00:13:03,063 --> 00:13:04,913
So you can do that kind of stuff as well.

259
00:13:04,973 --> 00:13:08,033
But one of the things that was, does
come to mind is when I was building

260
00:13:08,033 --> 00:13:10,613
the Electron app, I had this strategy.

261
00:13:10,613 --> 00:13:12,943
And I think I wrote a post
about this, where there is a,

262
00:13:12,963 --> 00:13:14,143
there's actually two windows.

263
00:13:14,153 --> 00:13:17,193
One window is for the front-end and
one window is actually for the backend.

264
00:13:17,193 --> 00:13:21,003
So the backend does not run
in a, like just a node process

265
00:13:21,033 --> 00:13:23,263
that is, you know, Invisible.

266
00:13:23,363 --> 00:13:27,593
It actually runs in another Electron
window with like node integration set

267
00:13:27,603 --> 00:13:30,013
to true, and it can access node APIs.

268
00:13:30,213 --> 00:13:32,633
What's cool about that is
that I can open the dev tools.

269
00:13:32,713 --> 00:13:35,383
I expose a bunch of stuff
to the top level, and then I

270
00:13:35,383 --> 00:13:37,063
can like query around stuff.

271
00:13:37,063 --> 00:13:38,603
I can query the database directly.

272
00:13:38,693 --> 00:13:41,193
I can get objects back and
get it that really nice Chrome

273
00:13:41,193 --> 00:13:42,543
dev tools, like object viewer.

274
00:13:42,843 --> 00:13:44,073
Um, I can track performance.

275
00:13:44,083 --> 00:13:46,173
So I can click the performance
tab and click start.

276
00:13:46,293 --> 00:13:48,733
Start performance recording,
do a bunch of stuff on the UI.

277
00:13:49,033 --> 00:13:51,173
And then like, I can set,
um, stop the performance.

278
00:13:51,173 --> 00:13:52,503
You can do performance tracing.

279
00:13:52,733 --> 00:13:55,523
So you can actually start tracing
how long the SQL queries took.

280
00:13:55,553 --> 00:13:59,943
So being able to access the backend dev
tools or the Chrome dev tools for the

281
00:13:59,953 --> 00:14:03,313
backend, and like being able to interact,
like directly interact with, with the

282
00:14:03,313 --> 00:14:05,163
database was like super, super great.

283
00:14:05,493 --> 00:14:07,403
Yeah, that, that sounds amazing.

284
00:14:07,543 --> 00:14:12,683
So you've been using SQLite with Electron,
but you've already hinted at before

285
00:14:12,773 --> 00:14:18,173
that at some point you realize, okay, a
single Electron app is not quite enough.

286
00:14:18,353 --> 00:14:19,633
We also carry phones.

287
00:14:19,633 --> 00:14:21,163
We might carry other devices.

288
00:14:21,163 --> 00:14:24,143
Um, What if your, your
single device gets lost?

289
00:14:24,173 --> 00:14:25,893
What about all of your app state?

290
00:14:26,293 --> 00:14:31,103
So I'm curious at which point
you've then found your way to, to

291
00:14:31,103 --> 00:14:35,583
also implementing collaboration or
syncing and how you went about that.

292
00:14:35,853 --> 00:14:40,653
So I, I can't remember when it
was exactly, maybe 2018, 2019, I

293
00:14:40,653 --> 00:14:42,823
started looking into this and I.

294
00:14:43,253 --> 00:14:47,263
Went about it by just hearing things
that my friends were talking about

295
00:14:47,393 --> 00:14:48,293
that were kind of interesting.

296
00:14:48,293 --> 00:14:52,443
And so they were, they were doing like
Raft and some of those protocols in

297
00:14:52,443 --> 00:14:55,453
the backend and kind of just, they
were deep into that kind of area.

298
00:14:55,543 --> 00:14:58,393
And so I heard, honestly,
I almost gave up.

299
00:14:58,443 --> 00:15:01,173
I was like, this just, all of this
stuff seems way, way too complicated.

300
00:15:01,173 --> 00:15:02,443
I do not have time to do this.

301
00:15:02,443 --> 00:15:05,563
But then I ended up just poking
around to see if there was any

302
00:15:05,563 --> 00:15:07,053
possibilities of things I was doing.

303
00:15:07,053 --> 00:15:10,283
I had this like a really initial
implementation, which was super naive.

304
00:15:10,573 --> 00:15:13,793
I, there's a gist that I explained
it somewhere and it was like

305
00:15:13,793 --> 00:15:14,783
really, really easy to pick up.

306
00:15:15,288 --> 00:15:15,828
Pick it apart.

307
00:15:15,828 --> 00:15:18,208
Once I started implementing it and
like running a test against it, like

308
00:15:18,208 --> 00:15:22,148
there was like, I think it was sort of
operational transform based, but it just

309
00:15:22,148 --> 00:15:23,998
like was, it fell apart way too easily.

310
00:15:24,118 --> 00:15:25,788
And I was like, this is really hard stuff.

311
00:15:25,848 --> 00:15:30,048
And when I looked at CRDTs, like I
could never grasp what they were.

312
00:15:30,078 --> 00:15:33,878
Nobody really, I, back then, at least
it was just not a good, it was too much

313
00:15:33,898 --> 00:15:35,938
math, too much, very, very intimidating.

314
00:15:36,068 --> 00:15:37,048
And they're really not that hard.

315
00:15:37,048 --> 00:15:40,148
Like it does not need to be explained
that way, but something clicked at

316
00:15:40,148 --> 00:15:42,018
some point when I finally implemented.

317
00:15:42,433 --> 00:15:46,393
A basic last right winds map that
the thing that really unlocked.

318
00:15:46,393 --> 00:15:47,903
It was hybrid logical clocks.

319
00:15:47,953 --> 00:15:51,413
I can't remember where, like, when
I found that, but when I started

320
00:15:51,413 --> 00:15:54,843
looking into that and reading that
paper, uh, there's a simplicity about

321
00:15:54,853 --> 00:15:56,853
them that I found really compelling.

322
00:15:56,853 --> 00:15:59,903
And it really matched match my
own technical kind of approach

323
00:15:59,903 --> 00:16:01,223
for things, which is like.

324
00:16:01,383 --> 00:16:05,543
make things as simple as possible and,
and, and the least surprising as possible.

325
00:16:05,613 --> 00:16:06,903
And you get a lot of benefits from that.

326
00:16:06,933 --> 00:16:11,213
Now there are, there are drawbacks
to HLCs, but the benefit of them,

327
00:16:11,223 --> 00:16:14,383
especially for this use case
seemed like a really good match.

328
00:16:14,383 --> 00:16:17,043
And so I went off and, you know,
it was one of those things where

329
00:16:17,043 --> 00:16:18,843
like everything kind of came
together in a couple of weeks.

330
00:16:19,303 --> 00:16:22,113
And I started seeing some really
compelling success with that.

331
00:16:22,123 --> 00:16:25,483
And also the ability to unlock
things like undo and Yeah.

332
00:16:25,523 --> 00:16:29,753
Redo, because once you start using the
system that like mutates everything

333
00:16:29,763 --> 00:16:33,013
through these like messages, you can
suddenly start tracking those messages

334
00:16:33,323 --> 00:16:35,003
and you can invert the messages.

335
00:16:35,293 --> 00:16:39,053
So undo literally becomes take this
batch of messages that happened

336
00:16:39,063 --> 00:16:43,093
in the last action and invert them
and then apply those messages.

337
00:16:43,383 --> 00:16:47,643
And suddenly, Actual turned out to
have a really robust undo and redo

338
00:16:47,643 --> 00:16:49,113
system, which I'm super proud of.

339
00:16:49,183 --> 00:16:51,733
And like, work, like literally
everything that you do in Actual,

340
00:16:51,733 --> 00:16:53,433
you can press command Z to undo.

341
00:16:53,633 --> 00:16:57,363
If you import 2000 transactions and
it runs a bunch of rules and mutates

342
00:16:57,383 --> 00:17:03,293
them, press command Z and in like 500
milliseconds, or not 500 100 milliseconds.

343
00:17:03,633 --> 00:17:05,493
Um, everything will go
back to where it was.

344
00:17:05,723 --> 00:17:10,063
So I started seeing signs of
this architecture, which was

345
00:17:10,073 --> 00:17:12,363
like really exciting, and then
I just kind of went from there.

346
00:17:12,993 --> 00:17:18,073
So just as a side questions for, for those
of us in the audience who might not be

347
00:17:18,073 --> 00:17:22,603
familiar with hypological clocks, could
you give a quick explainer, uh, what the

348
00:17:22,603 --> 00:17:24,763
concepts are and what they're used for?

349
00:17:24,978 --> 00:17:25,298
Sure.

350
00:17:25,308 --> 00:17:26,298
So I'll try to be fast.

351
00:17:26,338 --> 00:17:30,478
I think we could probably talk
about this, uh, area of research

352
00:17:30,488 --> 00:17:31,678
probably for the rest of the time.

353
00:17:32,038 --> 00:17:35,848
Hybrid logical clocks is a way to
solve the coordination problem.

354
00:17:35,858 --> 00:17:39,478
So the problem with distributed systems,
which a local-first app is a distributed

355
00:17:39,488 --> 00:17:44,018
system, because you have copies of the
app and copies of the data across multiple

356
00:17:44,018 --> 00:17:48,038
devices, is you need to know when, if you
have something like a last write wins set.

357
00:17:48,038 --> 00:17:50,438
If two people write that
offline, then come back online

358
00:17:50,518 --> 00:17:51,888
to sync up, which one wins?

359
00:17:52,228 --> 00:17:56,318
So you, you, you have to have a clock
for every single mutation in the system.

360
00:17:56,468 --> 00:17:57,448
There's a vector clocks.

361
00:17:57,448 --> 00:18:00,758
There's a more advanced clocks that are
built on top of things like vector clocks.

362
00:18:00,798 --> 00:18:03,788
There's a lot of different approaches for,
to, to solve this kind of a problem, to

363
00:18:03,788 --> 00:18:06,478
say, which one came after the other one.

364
00:18:06,538 --> 00:18:09,948
To take a pretty simplistic approach
here, but it's still super robust.

365
00:18:09,968 --> 00:18:14,058
And the very short summary of it is
it actually, the neat thing about it

366
00:18:14,058 --> 00:18:15,858
is that it serializes into a string.

367
00:18:15,858 --> 00:18:19,903
Um, And so the comparison of if this
message came after or before is you just

368
00:18:19,903 --> 00:18:24,023
compare the string, like it says less than
or greater than the string is always the

369
00:18:24,023 --> 00:18:26,953
same length and so it's lexically ordered.

370
00:18:27,023 --> 00:18:31,113
So you can just say, if I wanted to
inspect my database and get the messages

371
00:18:31,143 --> 00:18:34,913
in order, I would say, select star for
messages and then like order by the.

372
00:18:35,143 --> 00:18:38,733
CRDT, HLC, and then everything
comes back ordered, right?

373
00:18:39,023 --> 00:18:42,363
Whereas like vector clocks and all these
other complicated ones are like complex

374
00:18:42,373 --> 00:18:46,423
object data structures that have to keep
track of the ID or the counts or whatever

375
00:18:46,623 --> 00:18:48,513
of every single device in the whole world.

376
00:18:48,523 --> 00:18:52,363
And you have to like name, name those
devices and, and, and do a lot of things.

377
00:18:52,453 --> 00:18:57,353
HLCs are a thing that Take the
current time, like the clock of the

378
00:18:57,353 --> 00:18:59,083
system, which sounds terribly scary.

379
00:18:59,093 --> 00:19:01,643
If you know anything about this
thing, like involving the clock

380
00:19:01,643 --> 00:19:04,673
of the current local computer, it
sounds awful, but they do it in a

381
00:19:04,673 --> 00:19:06,063
way that is really, really novel.

382
00:19:06,403 --> 00:19:10,053
So the first part of the string
is like the, the UTC, like that

383
00:19:10,053 --> 00:19:13,163
timestamp in the format, that's
like that Z at the end, right?

384
00:19:13,193 --> 00:19:14,583
That like, I forget what it is.

385
00:19:14,703 --> 00:19:18,263
And so it's that, it's that big
timestamp and then it's dash.

386
00:19:18,283 --> 00:19:21,293
And then there's like
a, a padding of bits.

387
00:19:21,743 --> 00:19:23,273
And then there's another like dash.

388
00:19:23,548 --> 00:19:27,288
And then there's the device ID,
I think, at, at, at the very end.

389
00:19:27,628 --> 00:19:31,458
And so the way that you get
deterministic order is that,

390
00:19:31,508 --> 00:19:35,448
that little bucket of bits in the
middle, that's the key there, right?

391
00:19:35,848 --> 00:19:38,518
So that bucket of bits
represents an integer.

392
00:19:38,588 --> 00:19:41,858
When you receive a message,
normally you use your local time.

393
00:19:41,898 --> 00:19:43,178
Well, that's not exactly true.

394
00:19:43,179 --> 00:19:45,388
You actually use your local time.

395
00:19:45,608 --> 00:19:50,558
Or the last highest time that you've
ever seen from the whole system.

396
00:19:50,598 --> 00:19:53,808
So if you are receiving a bunch
of messages, you're reading the

397
00:19:53,818 --> 00:19:54,958
times off of those messages.

398
00:19:54,958 --> 00:19:58,268
And you say, if that time is greater
than my time, so that laptop's

399
00:19:58,288 --> 00:20:01,808
clock is like one minute faster,
then I'm going to have a timestamp.

400
00:20:02,108 --> 00:20:02,878
One minute faster.

401
00:20:02,878 --> 00:20:06,608
I need to, I need to fast forward
my time to match that clock's time.

402
00:20:06,828 --> 00:20:09,308
Cause that was like that person's
clock is later than mine.

403
00:20:09,658 --> 00:20:12,868
But when you start doing that kind of
a stuff, you can't generate another

404
00:20:12,868 --> 00:20:14,598
message with the same exact time, right?

405
00:20:14,628 --> 00:20:16,928
You need to differentiate those
two messages locally somehow.

406
00:20:16,928 --> 00:20:18,088
And so you increment that.

407
00:20:18,248 --> 00:20:22,758
That number that's stored in
those bits in the middle by one.

408
00:20:23,018 --> 00:20:26,248
And so if things are generating with
the same timestamp, well, suddenly that

409
00:20:26,248 --> 00:20:28,068
timestamp is not ordering things anymore.

410
00:20:28,288 --> 00:20:31,278
Now it's the little set of bits in
the middle that are ordering things.

411
00:20:31,318 --> 00:20:34,278
And you can serialize them as like
a hex value, I think, or something

412
00:20:34,278 --> 00:20:37,168
like something that's still like a
string that can be lexically ordered.

413
00:20:37,268 --> 00:20:40,558
And so you're bumping that up, but
the minute that your, your system

414
00:20:40,808 --> 00:20:42,318
moves forward, like a second.

415
00:20:42,518 --> 00:20:46,008
Or, or some, some amount of time, you
can reset that back to zero because

416
00:20:46,018 --> 00:20:50,068
the, the, the minute that your system
meets up with the time that you had

417
00:20:50,148 --> 00:20:53,368
seen from the other system, and then
you can start using your local system

418
00:20:53,368 --> 00:20:57,288
again, then you can start using, then
you reset the counter to zero and you

419
00:20:57,288 --> 00:20:58,788
don't need to use that anymore, right?

420
00:20:58,808 --> 00:21:02,538
So it's, it's really tricky to, this
is really interesting technique where.

421
00:21:02,843 --> 00:21:06,753
You can sort of leverage your
local time and other everybody in

422
00:21:06,753 --> 00:21:09,973
the system can coordinate on this
and you get this really simplistic

423
00:21:09,973 --> 00:21:11,323
approach throughout this whole thing.

424
00:21:11,423 --> 00:21:15,833
Now, the big downside of this is that
you can't have something that's so far

425
00:21:15,833 --> 00:21:19,943
ahead in time that you, your timestamp
locally is just meaningless now and that

426
00:21:19,943 --> 00:21:21,083
you're always incrementing the bits.

427
00:21:21,113 --> 00:21:24,753
You're going to hit a ceiling where
you can't, that integer is too large.

428
00:21:24,978 --> 00:21:28,428
And you can't store it in like the,
you know, whatever the 24 bits it is.

429
00:21:28,598 --> 00:21:29,328
So then you're screwed.

430
00:21:29,368 --> 00:21:30,868
Then that thing is all busted.

431
00:21:31,128 --> 00:21:33,218
So there's this whole like
thing where there's approach.

432
00:21:33,218 --> 00:21:37,458
You can say like all of the
devices in this system need to be

433
00:21:37,458 --> 00:21:41,018
synchronized with at least like
five minutes or like an hour.

434
00:21:41,258 --> 00:21:45,398
And if you try to generate or
send a message or get a message.

435
00:21:45,638 --> 00:21:48,298
That is outside of that
timeframe, you just reject it.

436
00:21:48,398 --> 00:21:53,168
So it's a little bit of a like
brute force simplistic approach.

437
00:21:53,428 --> 00:21:56,848
Uh, but for something like Actual,
it worked out really, really well.

438
00:21:56,958 --> 00:21:59,928
It would not work in a complex
distributed system where there's

439
00:21:59,928 --> 00:22:01,678
like many, many, many clients.

440
00:22:01,908 --> 00:22:05,173
And you know, they're, they're,
it's not as robust for sure.

441
00:22:05,323 --> 00:22:08,763
That's as simple as I can explain it,
hopefully as, as a little bit longer

442
00:22:08,763 --> 00:22:11,143
than I was hoping, but that's, that's,
that's the best way that I can help.

443
00:22:11,333 --> 00:22:13,113
No, this was super insightful.

444
00:22:13,143 --> 00:22:17,753
And I think going deep into those
kinds of topics, I think that's, that's

445
00:22:17,784 --> 00:22:19,903
what the audience is interested in.

446
00:22:19,934 --> 00:22:23,113
So thanks so much for, for
like taking that little detour.

447
00:22:23,153 --> 00:22:25,323
I understand that now much better.

448
00:22:25,413 --> 00:22:31,494
So you've taken this concept and then
went from your local SQLite database

449
00:22:31,889 --> 00:22:36,299
And how did you take Hyperlogical
Clocks with your SQLite database

450
00:22:36,509 --> 00:22:38,349
and now made things collaborative?

451
00:22:38,599 --> 00:22:42,899
So once you have Hyperlogical Clocks,
it becomes pretty easy to sync things

452
00:22:42,899 --> 00:22:47,399
around because I essentially, honestly,
I don't use a super complicated.

453
00:22:47,408 --> 00:22:48,899
I know, is it Martin Kleppmann?

454
00:22:48,899 --> 00:22:52,149
I think works on like a lot
of really, really robust data

455
00:22:52,149 --> 00:22:53,679
structures that work really well.

456
00:22:53,729 --> 00:22:56,269
When you're in this distributed
world, you have to make sure that

457
00:22:56,279 --> 00:22:57,989
things don't end up in a bad state.

458
00:22:57,998 --> 00:22:59,199
So if you want like a.

459
00:22:59,489 --> 00:23:00,319
Tree data structure.

460
00:23:00,319 --> 00:23:03,869
And you want to say, Hey, this, like,
there should never be any orphaned

461
00:23:03,879 --> 00:23:05,639
nodes in this tree tree data structure.

462
00:23:05,639 --> 00:23:08,489
Well, it's really easy to get
that state in a naive thing.

463
00:23:08,489 --> 00:23:12,618
If you say like a last right, when set
that it's like parent child, and then

464
00:23:12,619 --> 00:23:17,219
like one person updates a node and the
other person had deleted that node.

465
00:23:17,298 --> 00:23:19,019
And so the node gets like.

466
00:23:19,279 --> 00:23:22,669
removed from the parents, but
then the message comes in later

467
00:23:22,809 --> 00:23:24,529
that this person edited that node.

468
00:23:24,649 --> 00:23:27,619
So the node gets created again,
but it's an orphaned node.

469
00:23:27,829 --> 00:23:29,218
Like that's a weird place to be.

470
00:23:29,269 --> 00:23:33,018
So like there's work in the CRDT
world, which is fantastic and makes

471
00:23:33,059 --> 00:23:34,649
those kinds of things super robust.

472
00:23:35,179 --> 00:23:36,539
I did not use any of that kind of stuff.

473
00:23:36,589 --> 00:23:39,459
So like those kinds of things, I just kind
of accepted and was like, well, I just

474
00:23:39,459 --> 00:23:41,549
code defensively against like bad data.

475
00:23:41,699 --> 00:23:45,589
And generally speaking in my app,
I didn't have a ton of places where

476
00:23:45,599 --> 00:23:47,289
things needed to be super robust.

477
00:23:47,289 --> 00:23:50,969
It was pretty easy to defend against
bad data, but essentially  like I said,

478
00:23:51,008 --> 00:23:53,463
there's like an update, a crate and a.

479
00:23:53,763 --> 00:23:58,023
Delete function, they all actually
intercept to a single like lower level

480
00:23:58,033 --> 00:24:01,963
update function because instance and
updates are exactly the same thing that

481
00:24:02,093 --> 00:24:04,533
take a, take an object of things to set.

482
00:24:04,663 --> 00:24:06,673
And that object has to contain an ID.

483
00:24:06,923 --> 00:24:09,193
It turns it into CRDT messages.

484
00:24:09,203 --> 00:24:10,853
So every single field set.

485
00:24:10,863 --> 00:24:13,953
So if you said, set the transaction
amount in the transaction date to

486
00:24:13,983 --> 00:24:18,223
X and Y, those would become two
different messages with the same.

487
00:24:18,808 --> 00:24:20,358
object ID, right?

488
00:24:20,368 --> 00:24:22,288
Like they're, they're
targeting the same object.

489
00:24:22,498 --> 00:24:26,498
One is set the field amount to X and
one is set, set the field date to Y.

490
00:24:26,738 --> 00:24:30,658
There's two messages get created and then
this gets sent off to a syncing server.

491
00:24:30,948 --> 00:24:32,588
The syncing server is
really, really stupid.

492
00:24:32,728 --> 00:24:36,328
It just holds the messages and
when it's, when a client asks for

493
00:24:36,328 --> 00:24:38,258
messages, it gives us messages back.

494
00:24:38,498 --> 00:24:41,658
And so it can ask for messages
since a certain point in time,

495
00:24:41,788 --> 00:24:46,658
which again, in our, uh, HLC clock
world, that is our point in time.

496
00:24:46,898 --> 00:24:50,348
So it's literally just a Postgres query
that says select star for messages

497
00:24:50,548 --> 00:24:55,248
where HLC is greater than X and X
is the HLC that you gave it, right?

498
00:24:55,328 --> 00:24:58,118
So, so nice that you can just do
that, like greater than comparison.

499
00:24:58,248 --> 00:25:01,428
And then it, so it gets the messages back
and then it says, Hey, I'm, I'm, I'm,

500
00:25:01,438 --> 00:25:03,518
I'm all synced up and that's essentially.

501
00:25:03,843 --> 00:25:04,353
How it does.

502
00:25:04,353 --> 00:25:06,853
There is like a Merkle tree here in there.

503
00:25:06,853 --> 00:25:08,233
And I'm happy to talk about that.

504
00:25:08,363 --> 00:25:08,623
Yeah.

505
00:25:08,623 --> 00:25:09,513
Let's get into it.

506
00:25:09,603 --> 00:25:11,613
Uh, how Merkle tree is fitting in here.

507
00:25:11,913 --> 00:25:12,283
Sure.

508
00:25:12,313 --> 00:25:15,073
So yeah, the, so there's
a big problem here, right?

509
00:25:15,123 --> 00:25:17,993
Like, how do you know that
you're actually in sync?

510
00:25:18,113 --> 00:25:19,833
How do I know what messages to ask for?

511
00:25:19,923 --> 00:25:24,433
So when I, when I open up Actual and I
hit sync, do I ask for all the messages

512
00:25:24,433 --> 00:25:25,883
in the entire history of the world?

513
00:25:26,023 --> 00:25:28,243
Or like I could ask for.

514
00:25:28,983 --> 00:25:32,573
The messages since I've last opened
the app, like that seems nice, or like

515
00:25:32,613 --> 00:25:34,643
the last time I have synced, right?

516
00:25:34,723 --> 00:25:36,963
And then I would get those
messages and apply them.

517
00:25:37,083 --> 00:25:38,943
But there's like several questions there.

518
00:25:39,053 --> 00:25:43,143
One, what if another client had created
a message and just hadn't synced it

519
00:25:43,143 --> 00:25:47,313
yet, and created the message before
you last synced, and then you closed

520
00:25:47,323 --> 00:25:48,543
the app and like didn't open it.

521
00:25:48,723 --> 00:25:52,663
And then the other app, Came up, came
up and synced and then it like sent that

522
00:25:52,663 --> 00:25:56,453
message into the system and then you
open, you know, the, the, the original

523
00:25:56,453 --> 00:25:59,703
app, you would miss that message because
I've, I've seen since, since I've

524
00:25:59,703 --> 00:26:03,073
last synced, there couldn't possibly
any, couldn't possibly be any more

525
00:26:03,073 --> 00:26:05,063
messages, but that's not true at all.

526
00:26:05,073 --> 00:26:08,983
In this distributed world, you have
to assume everything bad, everything

527
00:26:08,993 --> 00:26:10,513
out of order is going to happen.

528
00:26:10,853 --> 00:26:12,313
You cannot code like that.

529
00:26:12,423 --> 00:26:13,753
The other problem.

530
00:26:14,163 --> 00:26:18,003
Is when I have synced up, like, let's say
I asked for all, all of the messages in

531
00:26:18,003 --> 00:26:24,593
the world and I apply them locally, how
do I know that I'm just actually valid?

532
00:26:24,723 --> 00:26:26,413
Like there could be a bug in my system.

533
00:26:26,533 --> 00:26:29,293
And so like, how do I know that
I have, like, when you, when you

534
00:26:29,293 --> 00:26:32,163
get all of those messages back,
how do I know which ones to apply?

535
00:26:32,403 --> 00:26:35,443
So locally, there's like kind of a
ledger of things that you've applied.

536
00:26:35,473 --> 00:26:38,023
And so you, you, you go through
every single message and you

537
00:26:38,023 --> 00:26:39,353
say, have I applied this message?

538
00:26:39,393 --> 00:26:40,223
I've already applied it.

539
00:26:40,543 --> 00:26:41,183
Don't apply it.

540
00:26:41,513 --> 00:26:46,693
If I have not applied it and it is a valid
message, like if it's a last right, when

541
00:26:46,713 --> 00:26:52,413
set, it'll say, if this has been written
by a message, like later than this one,

542
00:26:52,613 --> 00:26:54,383
then I can just describe this as well.

543
00:26:54,643 --> 00:26:57,543
So there's logic about how
to apply the messages there.

544
00:26:57,943 --> 00:27:01,303
That could be buggy or just like
a network request is so weird.

545
00:27:01,303 --> 00:27:03,463
And there's a state that I
just, I didn't anticipate.

546
00:27:03,553 --> 00:27:05,923
You need look kind of like a, a, another.

547
00:27:06,248 --> 00:27:08,688
piece of data structure, and
that's, that's, that's the Merkle

548
00:27:08,688 --> 00:27:10,958
tree to sort of audit things.

549
00:27:11,168 --> 00:27:13,718
It's, it's, it's a hash of
everything in a system, right?

550
00:27:13,718 --> 00:27:17,998
So if you hash all of the objects, all
of the CRDT messages, you can think of

551
00:27:18,008 --> 00:27:21,318
all the CRDT messages as like leaf nodes.

552
00:27:21,498 --> 00:27:26,018
And then you have like them grouped into
buckets and every node in the tree, all

553
00:27:26,018 --> 00:27:27,828
the way up to the root is a new hash.

554
00:27:28,048 --> 00:27:31,668
And so the hash at the root is a hash
of everything in the entire system.

555
00:27:31,698 --> 00:27:35,758
So you can quickly compare if like,
have I seen all of these CRDT messages?

556
00:27:35,788 --> 00:27:37,658
I just compared the two root hashes.

557
00:27:37,768 --> 00:27:40,418
And then if I, if those hashes are
exactly the same, then I know I've

558
00:27:40,598 --> 00:27:42,348
like, these have been processed.

559
00:27:42,668 --> 00:27:48,528
Where it gets weird is that like,
I use, A base three system, I think

560
00:27:48,608 --> 00:27:52,078
of like, basically every node in
the tree is a zero one or two.

561
00:27:52,588 --> 00:27:57,508
And basically you can construct
a timestamp of basically

562
00:27:57,528 --> 00:27:59,658
how I bucket the messages.

563
00:27:59,718 --> 00:28:01,348
Like I talked about buckets, right?

564
00:28:01,358 --> 00:28:03,478
You have to have like,
what are those buckets?

565
00:28:03,478 --> 00:28:05,638
The buckets for me were
basically time windows.

566
00:28:06,208 --> 00:28:09,938
And so I had time windows of
like, down to like, 10 minutes.

567
00:28:10,638 --> 00:28:15,808
So every single like CRDT messages
all applied within a 10 minute window

568
00:28:15,818 --> 00:28:17,758
would be hashed together into one hash.

569
00:28:17,758 --> 00:28:19,598
And that would be one
leaf node in the tree.

570
00:28:20,128 --> 00:28:22,108
And those would be all
the separate leaf buckets.

571
00:28:22,778 --> 00:28:26,528
So the bucket above it represented
a new, a larger window.

572
00:28:26,613 --> 00:28:27,483
Of time, right?

573
00:28:27,523 --> 00:28:31,323
And it was a, because it was base three,
you could reconstruct this timestamp.

574
00:28:31,513 --> 00:28:34,903
The thing that I wanted to guard
against was these are a lot of messages.

575
00:28:34,903 --> 00:28:36,773
There could be tens of
thousands of messages.

576
00:28:37,033 --> 00:28:41,513
You have to come up with a, a system
that is detailed enough so that you're

577
00:28:41,513 --> 00:28:46,623
not requesting too much, too, too, like
too many messages, but it needs to be.

578
00:28:46,843 --> 00:28:49,913
Coarse enough so that the tree
just doesn't get massive because

579
00:28:49,913 --> 00:28:52,843
this tree is sent across the
network every single time you sync.

580
00:28:52,863 --> 00:28:56,203
It's stored in your database and
updated, like updated as a blob

581
00:28:56,283 --> 00:28:57,593
every single time that you sync.

582
00:28:57,853 --> 00:28:59,593
So it cannot be a huge tree.

583
00:28:59,663 --> 00:29:03,553
And so this base three system encoded
these windows in a way that allowed

584
00:29:03,553 --> 00:29:07,493
me to calculate things down to the
five minutes windows and not be huge.

585
00:29:07,493 --> 00:29:09,963
Like I think it could ever
only get like 10 leaves deep.

586
00:29:10,243 --> 00:29:12,053
And so that solves the depth problem.

587
00:29:12,063 --> 00:29:13,073
It doesn't solve the.

588
00:29:13,238 --> 00:29:14,648
breadth problem of the tree.

589
00:29:14,648 --> 00:29:17,108
So the Merkle tree could still
get really wide if you're using

590
00:29:17,108 --> 00:29:18,138
the app over years and years.

591
00:29:18,188 --> 00:29:21,418
So if you think of this huge tree, we're
ever seeing in windows of time, there's

592
00:29:21,418 --> 00:29:25,098
only ever two paths down 10 nodes deep.

593
00:29:25,478 --> 00:29:28,098
And before that, it just
prunes them all away.

594
00:29:28,098 --> 00:29:28,968
It just deletes them.

595
00:29:29,218 --> 00:29:33,448
What that means is that if I miss a
message, like I think it was about a

596
00:29:33,448 --> 00:29:38,078
nine month mark, that if a message comes
through past nine months previous to that,

597
00:29:38,718 --> 00:29:41,018
I don't know which things to download.

598
00:29:41,108 --> 00:29:44,888
I check that I've received all of the
messages by comparing the hashes, right?

599
00:29:45,008 --> 00:29:47,898
If the hash of the root is wrong,
then I go down and I compare.

600
00:29:48,118 --> 00:29:50,538
That's how I get the window
to retrieve messages by.

601
00:29:50,848 --> 00:29:54,428
So I compare down and I get, I get
the, the node that is different.

602
00:29:54,678 --> 00:29:57,658
And then from that different node,
I can construct a window and says

603
00:29:57,978 --> 00:30:01,008
something came through like, like
a message from like, like three

604
00:30:01,008 --> 00:30:02,698
weeks ago, there's something here.

605
00:30:02,863 --> 00:30:03,493
That's a difference.

606
00:30:03,543 --> 00:30:06,863
I need to, I need to retrieve all of
the messages from three weeks ago, and

607
00:30:06,863 --> 00:30:10,343
I need to reapply them all because like
something changed their Merkle tree hat.

608
00:30:10,343 --> 00:30:14,463
Like the, the server sends the
Merkle tree, it's Merkle tree to me.

609
00:30:14,463 --> 00:30:18,083
And so I can compare the service Merkle
tree to mine and say, Hey, the service

610
00:30:18,103 --> 00:30:20,203
Merkle tree like changed around this time.

611
00:30:20,203 --> 00:30:21,443
So I need to get those messages.

612
00:30:22,063 --> 00:30:25,153
There's a certain point in time, which if
a message, if the Merkle tree like gets.

613
00:30:25,498 --> 00:30:29,588
Change like a long time ago, like
a really old message comes to the

614
00:30:29,588 --> 00:30:31,158
system and just screws everything up.

615
00:30:31,538 --> 00:30:34,278
I actually won't know that because
I've already pruned that tree away.

616
00:30:34,538 --> 00:30:37,818
So you'll either have to redownload
all of the messages, which is

617
00:30:37,818 --> 00:30:40,548
probably not practical, or you just
reject that client and you like

618
00:30:40,548 --> 00:30:41,748
detach it from the whole system.

619
00:30:42,248 --> 00:30:44,678
So there's edge cases there that
you have to sort of think about,

620
00:30:44,908 --> 00:30:47,758
but the really nice thing that the
Merkle tree gives me is that window.

621
00:30:48,123 --> 00:30:52,663
And a validation that I have actually
processed, like if the roots hashes

622
00:30:52,683 --> 00:30:55,813
are the same on the server and the
client, then I know I'm up to date.

623
00:30:55,943 --> 00:31:00,643
So thank you so much for giving
this, uh, quite in depth overview

624
00:31:00,683 --> 00:31:05,083
of how Merkle tree works and more
specifically how you applied them

625
00:31:05,103 --> 00:31:07,613
on the, the Actual syncing scenario.

626
00:31:07,673 --> 00:31:09,143
That was very insightful.

627
00:31:09,153 --> 00:31:13,728
I've now seen Merkle trees applied on
a, on a few different technological

628
00:31:13,798 --> 00:31:19,118
scenarios and the way how you've now
used them with the time buckets, I

629
00:31:19,128 --> 00:31:23,128
think is super elegant where it can,
in the happy path, save a lot of

630
00:31:23,128 --> 00:31:25,648
work, uh, for, for the syncing engine.

631
00:31:25,648 --> 00:31:29,068
And once you've understood the entire
system, actually quite simple to think

632
00:31:29,068 --> 00:31:33,878
about, and I'm sure also when you, maybe
something went wrong at some point and

633
00:31:33,878 --> 00:31:38,378
you at least still had sort of like an
intuitive mental model, how you can debug

634
00:31:38,428 --> 00:31:39,978
this, et cetera, and how you can test it.

635
00:31:40,138 --> 00:31:41,728
So I'm sure you didn't.

636
00:31:41,963 --> 00:31:48,223
just build a syncing system just for one
or two Electron apps, but probably for

637
00:31:48,253 --> 00:31:51,193
a vision to go beyond the Electron app.

638
00:31:51,533 --> 00:31:54,133
So how did Actual evolve

639
00:31:54,163 --> 00:31:54,773
from here?

640
00:31:54,833 --> 00:31:55,173
Yeah.

641
00:31:55,173 --> 00:32:00,323
So once I validated that I was going
to build a syncing engine and that

642
00:32:00,353 --> 00:32:02,243
it worked, I built a mobile app.

643
00:32:02,363 --> 00:32:06,373
Because obviously that was part of the
reason for wanting to do syncing is to

644
00:32:06,373 --> 00:32:07,723
be able to support like a mobile app.

645
00:32:07,783 --> 00:32:11,763
And I sort of naively was like, I'm
not going to build a shoddy mobile app.

646
00:32:11,763 --> 00:32:13,883
I'm going to, I'm going to do
this and I'm going to do it right.

647
00:32:13,893 --> 00:32:15,873
And so I, I, I did use React Native.

648
00:32:15,873 --> 00:32:20,068
I didn't, I wasn't so naive that I was
going to build a separate Android and iOS

649
00:32:20,068 --> 00:32:24,118
at myself, but I did attempt to use React
Native and I built it with React Native.

650
00:32:24,128 --> 00:32:27,998
And then I leveraged a project that
was at the time was having a decent

651
00:32:27,998 --> 00:32:31,588
amount of maintenance and investment
from, I forget which company started

652
00:32:31,588 --> 00:32:33,218
it, but it's a project called Node.

653
00:32:33,228 --> 00:32:34,638
js mobile.

654
00:32:35,028 --> 00:32:38,998
And so they basically took Node and
they compiled it for iOS and Android.

655
00:32:38,998 --> 00:32:42,598
And they basically built this thing
that would load your JavaScript bundle.

656
00:32:42,648 --> 00:32:45,308
And you could write JavaScript
based off of Node APIs.

657
00:32:45,488 --> 00:32:46,018
And it was great.

658
00:32:46,113 --> 00:32:50,093
Cause like HTTP, like all of the
standard node APIs that you would

659
00:32:50,093 --> 00:32:53,563
expect worked there and it fired off
as like a separate process, like it

660
00:32:53,573 --> 00:32:55,623
did it in, in the, in the proper way.

661
00:32:55,703 --> 00:32:57,913
And so  I got a prototype working
and it worked really well.

662
00:32:57,913 --> 00:32:59,893
And so I was able to use
the exact same backend.

663
00:33:00,133 --> 00:33:02,303
And then I built a new
front-end in react native.

664
00:33:02,313 --> 00:33:03,483
That was like mobile specific.

665
00:33:03,493 --> 00:33:06,083
Cause I was like, I really want to
think this through in like a mobile way.

666
00:33:06,083 --> 00:33:10,103
And I'm not going to like shrink
down this transactions table and from

667
00:33:10,103 --> 00:33:13,393
desktop and like, Try to shove in
this complicated table on the mobile.

668
00:33:13,423 --> 00:33:15,313
I'm going to have a proper
mobile app and do it right.

669
00:33:15,363 --> 00:33:19,723
But yeah, having to, to design and
develop two separate UIs for two different

670
00:33:19,723 --> 00:33:21,443
use cases was just, was just awful.

671
00:33:21,443 --> 00:33:24,413
I mean, just time time wise, it was
just a bad decision, but I did, I

672
00:33:24,443 --> 00:33:27,823
did get the mobile app working and I
released it like, and it was a used

673
00:33:27,853 --> 00:33:31,703
thing that most people use, I had
like, I don't know, 900 installs on,

674
00:33:31,713 --> 00:33:35,313
on iOS and, uh, some, a couple of
hundred on Android, I think as well,

675
00:33:35,383 --> 00:33:37,573
people use it and it, it had the same.

676
00:33:37,573 --> 00:33:41,018
So like, React Native
powered the UI part of it.

677
00:33:41,128 --> 00:33:44,638
Um, and then in the backend, the syncing
engine and all the exact same code ran.

678
00:33:44,638 --> 00:33:47,598
And so the syncing, the syncing stuff
would, would run and it would send

679
00:33:47,598 --> 00:33:50,168
messages off to the server and then
it would be synced back to the desktop

680
00:33:50,168 --> 00:33:51,208
and, and it worked pretty well.

681
00:33:51,418 --> 00:33:53,288
That must've been a magical moment.

682
00:33:53,338 --> 00:33:56,998
Once you had your data from the desktop
app show up in the mobile app, you make

683
00:33:56,998 --> 00:34:00,818
changes on the mobile app and things
are appearing on your desktop app.

684
00:34:00,968 --> 00:34:02,138
All that work paying off.

685
00:34:02,413 --> 00:34:08,093
Yeah, yeah, I think I have a, I might have
a tweet, I think that showed them side by

686
00:34:08,093 --> 00:34:11,023
side and it was like, look, you can change
one thing here and it shows up over there.

687
00:34:11,023 --> 00:34:13,763
And it's always a cool,
like a cool demo able thing.

688
00:34:13,913 --> 00:34:20,363
Yeah, I think if as like local-first apps
are luckily becoming more and more normal,

689
00:34:20,383 --> 00:34:22,263
I think we will take it for granted.

690
00:34:22,623 --> 00:34:27,223
But I feel quite nostalgic about like
the, the days where this is not normal.

691
00:34:27,473 --> 00:34:29,393
And like the, the magic is really strong.

692
00:34:29,393 --> 00:34:33,493
And in my opinion, it's still
strong when you see like the, the

693
00:34:33,493 --> 00:34:37,243
real time collaboration of apps, et
cetera, and those things just work.

694
00:34:37,763 --> 00:34:41,438
So you've mentioned that you've
been using what was called Node.

695
00:34:41,438 --> 00:34:42,383
js mobile.

696
00:34:42,443 --> 00:34:45,293
And so that allowed you to
bring most of your code there.

697
00:34:45,483 --> 00:34:50,323
Did even like the, the SQLite bindings
and the SQLite reactivity system, did all

698
00:34:50,323 --> 00:34:52,693
of that just work also on React Native?

699
00:34:52,883 --> 00:34:55,673
It didn't just work, but it
wasn't too hard to get it to work.

700
00:34:55,673 --> 00:34:57,323
It was sort of just like Electron.

701
00:34:57,323 --> 00:34:59,343
I had to figure out how to load in.

702
00:34:59,523 --> 00:35:02,773
I think there was already an
Electron like C library that

703
00:35:02,773 --> 00:35:04,783
allowed me to easily access SQLite.

704
00:35:05,063 --> 00:35:08,463
But for React Native, I actually
built, I think I built my own SQLite.

705
00:35:08,648 --> 00:35:11,998
Bindings for react native.

706
00:35:12,108 --> 00:35:14,408
I'm actually trying to remember,
maybe I, maybe I did it.

707
00:35:14,418 --> 00:35:17,378
Maybe it did just work because
the, no, I think it was weird.

708
00:35:17,538 --> 00:35:18,128
I think the Node.

709
00:35:18,128 --> 00:35:22,148
js mobile allowed you to
compile C dependencies, but

710
00:35:22,148 --> 00:35:23,508
like it didn't fully work.

711
00:35:23,518 --> 00:35:25,628
And so I had to kind
of get things working.

712
00:35:25,628 --> 00:35:28,848
So it didn't just work, but it wasn't
like super hard to get working.

713
00:35:28,928 --> 00:35:31,318
You basically had to compile SQLite for.

714
00:35:31,383 --> 00:35:32,373
IOS and Android, right?

715
00:35:32,373 --> 00:35:35,903
And so you have to like hook in their
build process and get that loaded.

716
00:35:35,903 --> 00:35:36,303
And Node.

717
00:35:36,303 --> 00:35:41,413
js's mobile support for like C
dependencies from Node wasn't straight

718
00:35:41,433 --> 00:35:44,623
out of the box, but honestly it
worked, it worked well enough to where

719
00:35:44,623 --> 00:35:45,943
it wasn't that hard to get working.

720
00:35:45,943 --> 00:35:48,153
So I was, I was pretty impressed
by that, but I did, I did have

721
00:35:48,153 --> 00:35:51,653
to kind of wire up the, the, the
core things that I still needed.

722
00:35:51,813 --> 00:35:52,143
Got it.

723
00:35:52,173 --> 00:35:57,353
But I suppose that's mostly one of work
and the overall goal of like reusing

724
00:35:57,363 --> 00:36:01,233
the same code from your Actual desktop
app in the mobile app to the most

725
00:36:01,233 --> 00:36:03,783
degree aside from the UI that paid off.

726
00:36:03,953 --> 00:36:07,493
Yes, I think that paid off like
that specific investment worked.

727
00:36:07,813 --> 00:36:12,903
The real downside to all of this stuff
about actually like actually having an

728
00:36:12,913 --> 00:36:17,583
app and especially for Mobile is that
the ecosystem just changes and your app

729
00:36:17,583 --> 00:36:19,763
is in this like proprietary app store.

730
00:36:19,763 --> 00:36:23,743
And like the proprietary app store
is like forced you to, to update it.

731
00:36:23,753 --> 00:36:27,993
Like I think Android at one point forced
me to like, they basically deprecated an

732
00:36:27,993 --> 00:36:32,543
API and I could not release a new version
of the app until I stopped using that

733
00:36:32,543 --> 00:36:34,303
old API and started using like a new one.

734
00:36:34,593 --> 00:36:37,413
The new one requires some really
weird thing that like triggered

735
00:36:37,413 --> 00:36:39,583
a fault, like a crash on Node.

736
00:36:39,583 --> 00:36:40,293
js mobile.

737
00:36:40,403 --> 00:36:42,123
So like, that's a super risky.

738
00:36:42,403 --> 00:36:46,243
Place to be in and so it, it paid
off, but like, it was a continuous

739
00:36:46,413 --> 00:36:52,023
investment risk that didn't require
any maintenance really, except when

740
00:36:52,023 --> 00:36:54,093
I was forced to buy the ecosystem.

741
00:36:54,153 --> 00:36:55,653
And so that's the real
win of the web, right?

742
00:36:55,663 --> 00:36:57,903
Like, you're not like, the
web definitely is weird.

743
00:36:58,438 --> 00:37:00,928
sucks in a lot of ways and like
they change things and they

744
00:37:00,928 --> 00:37:02,508
force you into certain things.

745
00:37:02,808 --> 00:37:04,768
But generally speaking,
backwards compatibility is a

746
00:37:04,768 --> 00:37:06,248
huge, huge value on the web.

747
00:37:06,248 --> 00:37:08,578
And it's just not the case
in the mobile ecosystem.

748
00:37:08,578 --> 00:37:11,388
And they have leverage on you
because you have to go through

749
00:37:11,388 --> 00:37:15,238
their app stores, which just, I
just hate, like, I love mobile apps.

750
00:37:15,238 --> 00:37:18,418
I love having like a real
app, but like that Node.

751
00:37:18,598 --> 00:37:21,238
js mobile was like a risk for sure.

752
00:37:21,268 --> 00:37:25,618
Because There's like certificates and how
things are assigned that kept changing.

753
00:37:25,618 --> 00:37:29,168
And like was, I luckily
never got totally stuck.

754
00:37:29,198 --> 00:37:31,718
Andre actually, I think was the
only other person using this.

755
00:37:31,988 --> 00:37:34,438
And so luckily he had
like a PR that fixed it.

756
00:37:34,458 --> 00:37:37,478
Cause he, he was stuck in the same way,
but he actually knows how to change stuff.

757
00:37:37,568 --> 00:37:41,418
And so he had a PR merge like
three days earlier that luckily

758
00:37:41,418 --> 00:37:42,318
I was able to update Node.

759
00:37:42,318 --> 00:37:45,208
js mobile and it finally
worked, but it's scary, right.

760
00:37:45,218 --> 00:37:45,668
To be.

761
00:37:45,953 --> 00:37:49,233
Like I could have just been totally
stuck as far as I know, there's not

762
00:37:49,253 --> 00:37:53,913
a great way to have that set up today
without a similar level of risk to,

763
00:37:53,933 --> 00:37:57,003
to, because you're, you would have to
build a pretty bespoke sort of custom

764
00:37:57,243 --> 00:38:02,633
native app on, on mobile to, to do this
whole, uh, truly local-first type thing.

765
00:38:02,853 --> 00:38:03,163
Got it.

766
00:38:03,193 --> 00:38:03,583
Yeah.

767
00:38:03,633 --> 00:38:08,233
I think, well, as you mentioned, uh,
well, most platforms are evolving over

768
00:38:08,233 --> 00:38:12,793
time, some more aggressively and yeah,
with little tolerance for developers

769
00:38:12,793 --> 00:38:16,893
who don't update the apps, I think the
web is more graceful in that regard.

770
00:38:16,923 --> 00:38:20,023
On mobile, the rock is sometimes
being pulled underneath you.

771
00:38:20,303 --> 00:38:24,943
But on the, seeing the glass half
full, you also get hopefully improved

772
00:38:24,953 --> 00:38:27,523
APIs and the ecosystem is improving.

773
00:38:27,603 --> 00:38:32,873
I've been recently getting a bit closer to
the React native ecosystem and Expo seems

774
00:38:32,873 --> 00:38:35,183
to make a lot of that quite a lot nicer.

775
00:38:35,203 --> 00:38:36,963
So there's an impressive amount of.

776
00:38:37,468 --> 00:38:40,278
Bindings to native APIs that
you, that you might need.

777
00:38:40,638 --> 00:38:44,698
So, and also given that JavaScript
is also evolving in terms of

778
00:38:44,708 --> 00:38:46,308
standards, supported standards.

779
00:38:46,818 --> 00:38:50,168
Uh, I think there's now also the
lines are getting a bit blurrier

780
00:38:50,248 --> 00:38:51,798
of what needs to run a Node.

781
00:38:51,798 --> 00:38:55,088
js or what is just supported
by like other JavaScript

782
00:38:55,138 --> 00:38:56,818
execution environments or mobile.

783
00:38:57,118 --> 00:38:59,878
You have JavaScript
core on iOS on Android.

784
00:38:59,878 --> 00:39:03,918
I don't know what it's called, but
you also have Hermes, the tool by the

785
00:39:03,918 --> 00:39:08,248
JavaScript runtime by Facebook, I think,
which is specifically designed for mobile.

786
00:39:08,558 --> 00:39:12,538
So I think you get more options,
but there's still like sharp edges.

787
00:39:13,188 --> 00:39:17,178
So did you eventually give up
on those mobile apps or did

788
00:39:17,178 --> 00:39:19,128
you, did this lead you to web?

789
00:39:19,418 --> 00:39:21,808
How, how did the journey
continue from here?

790
00:39:22,048 --> 00:39:22,398
Sure.

791
00:39:22,428 --> 00:39:26,288
So I'd never gave up on the mobile
apps until I fully open source Actual

792
00:39:26,308 --> 00:39:29,678
when I kind of gave up on Actual and
entirely as, as a whole business.

793
00:39:29,678 --> 00:39:33,308
And that was when I told the
community, Hey, if you want to,

794
00:39:33,388 --> 00:39:36,428
like, this is the source code, I
included the source code of it.

795
00:39:36,428 --> 00:39:40,498
And so if you want to figure out how to
like build this and purchase a developer

796
00:39:40,498 --> 00:39:41,893
account and get this working, like it's.

797
00:39:42,053 --> 00:39:42,833
Totally up to you.

798
00:39:42,983 --> 00:39:46,133
But in terms of the Actual apps
that were on people's phones at the

799
00:39:46,133 --> 00:39:48,563
time, yeah, there was never going
to be another update of those.

800
00:39:48,783 --> 00:39:50,143
I'm still working on shutting down Actual.

801
00:39:50,143 --> 00:39:53,093
And so those apps still do exist in the
App Store as far as I know, but they

802
00:39:53,093 --> 00:39:54,873
will be removed when I shut down Actual.

803
00:39:54,913 --> 00:39:59,513
So there was another phase where
I had not given up yet, but I did

804
00:39:59,543 --> 00:40:03,868
sort of admit defeat in terms of
Where the investment should go.

805
00:40:03,958 --> 00:40:06,278
And I just was like, I
need to be on the web.

806
00:40:06,298 --> 00:40:08,708
Like at that, there was a
point when Actual just like,

807
00:40:08,708 --> 00:40:09,598
didn't even work on the web.

808
00:40:09,598 --> 00:40:10,568
It just was not a thing.

809
00:40:10,718 --> 00:40:13,508
And so I think I might have
had a demo on the web, but it

810
00:40:13,508 --> 00:40:14,738
just like, didn't support it.

811
00:40:14,738 --> 00:40:18,208
Loaded the loaded in the
entire SQLite file locally.

812
00:40:18,238 --> 00:40:20,058
And then it just like,
didn't persist anything.

813
00:40:20,058 --> 00:40:20,228
Right.

814
00:40:20,228 --> 00:40:24,318
It just like the app ran, but it didn't
actually, it kind of like removed a

815
00:40:24,318 --> 00:40:27,458
lot of the functionality and it was
just like a quick demo, but like the.

816
00:40:27,868 --> 00:40:31,528
Man forcing people to, to, to
download an app and running into

817
00:40:31,528 --> 00:40:32,918
problems with their local device.

818
00:40:32,928 --> 00:40:36,378
Like it just, the, the web is such
a powerful distribution mechanism.

819
00:40:36,378 --> 00:40:40,388
And I think that's, you know, everybody
knows that and it's really hard to, to

820
00:40:40,388 --> 00:40:44,088
fight against to fight against that and
to force people to, to download apps.

821
00:40:44,088 --> 00:40:45,898
I honestly still love apps.

822
00:40:45,898 --> 00:40:48,378
Like once it gets set up,
I think it's really nice.

823
00:40:48,408 --> 00:40:51,038
Like if it's an app that I use almost
every day, or even just a couple,

824
00:40:51,068 --> 00:40:52,368
like a couple times a week to have it.

825
00:40:52,579 --> 00:40:54,938
In my, in my doc and I can like close it.

826
00:40:54,938 --> 00:40:55,568
I can quit it.

827
00:40:55,958 --> 00:40:57,558
It doesn't have to be in my mess of tabs.

828
00:40:57,838 --> 00:41:01,988
I have a billion tabs and I, so
like managing my, an app that I use

829
00:41:01,988 --> 00:41:05,098
frequently in the browser to me, I,
I don't really like that, but at the

830
00:41:05,108 --> 00:41:09,368
other, at the same time, There are
things that I use all the time on the

831
00:41:09,368 --> 00:41:11,368
web that are only in my tabs, actually.

832
00:41:11,558 --> 00:41:14,918
And it's very kind of nice to
like be able to close the tab and

833
00:41:14,918 --> 00:41:17,488
then like quickly open up a tab
and go to the app really fast.

834
00:41:17,488 --> 00:41:20,938
Whereas like the app startup for,
you know, Mac OS tends to be like

835
00:41:20,938 --> 00:41:22,068
at least two or three seconds.

836
00:41:22,178 --> 00:41:23,928
So I don't know, it's, it's weird.

837
00:41:23,938 --> 00:41:26,913
I kind of say that I like local
apps and yet I live in a browser

838
00:41:26,913 --> 00:41:28,338
like 80 percent of my time.

839
00:41:28,338 --> 00:41:29,968
So there's something there.

840
00:41:30,338 --> 00:41:32,748
And as much as web technology
kind of sucks, and it's so

841
00:41:32,748 --> 00:41:36,528
confusing and I don't like it,
the web is a really powerful draw.

842
00:41:36,818 --> 00:41:39,738
And so I eventually was just like, you
know what, I need to figure this out.

843
00:41:39,948 --> 00:41:43,258
So I never like stopped investing
in the Electron and, and Node.

844
00:41:43,258 --> 00:41:45,528
js or the um, mobile Node.

845
00:41:45,528 --> 00:41:46,728
js mobile stuff.

846
00:41:46,808 --> 00:41:48,138
I would still update.

847
00:41:48,298 --> 00:41:50,538
Do a release of those
every couple of weeks.

848
00:41:50,928 --> 00:41:54,458
But the first, I remember the first time
that I got Actual working on, um, on the

849
00:41:54,458 --> 00:41:58,338
web and I used, you know, that's where
absurd-sql came from, where I was able

850
00:41:58,338 --> 00:42:03,658
to, to compile things out at a compiled
SQL light to web assembly and use

851
00:42:03,698 --> 00:42:07,718
techniques such that like you can open
the app in multiple tabs and actually

852
00:42:07,728 --> 00:42:09,258
change the data, actually query the data.

853
00:42:09,258 --> 00:42:12,768
And like, when you change data
on one tab, that when you.

854
00:42:12,868 --> 00:42:15,708
Query that data on another
tab, it actually shows up.

855
00:42:15,748 --> 00:42:17,518
I got all that working, which is
really, really great, which is

856
00:42:17,518 --> 00:42:18,808
something that we can talk about more.

857
00:42:19,038 --> 00:42:22,618
I remember like the first time that
I released that and people were

858
00:42:22,618 --> 00:42:24,238
just like instantly able to use it.

859
00:42:24,528 --> 00:42:28,018
And then when I was able to fix a bug
and I was just like with like, basically,

860
00:42:28,538 --> 00:42:30,878
basically in, in, in our sync command.

861
00:42:31,298 --> 00:42:35,218
Was able to like, get that bug in
people's hands or that, that, that

862
00:42:35,218 --> 00:42:39,828
bug fix, um, out there was just like
insanely addicting that, that, I mean,

863
00:42:39,828 --> 00:42:43,698
that's the power of the web, because also
actually, it's just a local app, right?

864
00:42:44,118 --> 00:42:47,268
That's the other thing that I
realized when I built the web app, it

865
00:42:47,268 --> 00:42:49,408
literally is a set of static files.

866
00:42:49,818 --> 00:42:50,648
There's no.

867
00:42:50,858 --> 00:42:54,698
Like there's a syncing service needed,
but like in terms of the entirety of

868
00:42:54,698 --> 00:42:58,278
the app itself, I don't need to do
any database mutations when I deploy.

869
00:42:58,278 --> 00:43:02,198
There's no, there's no large CI pipeline
that does all of these complicated

870
00:43:02,208 --> 00:43:05,978
things to deploy, to restart services, to
update services and do all of this stuff.

871
00:43:06,248 --> 00:43:10,538
It is literally an HTML file and
like five JavaScript bundles.

872
00:43:11,098 --> 00:43:18,018
I can just rsync those over to my web
server that host static files and users

873
00:43:18,108 --> 00:43:21,558
like load those new static files and then
it queries their local data and all of

874
00:43:21,558 --> 00:43:25,998
their local stuff is now reading that
like fresh stuff, but like, just the

875
00:43:26,198 --> 00:43:29,503
ability to just like sync those files
over and just, you know, Deploy like

876
00:43:29,693 --> 00:43:34,123
a hundred times a day just unlocks an
iterative development speed that just

877
00:43:34,153 --> 00:43:36,763
cannot be matched by mobile development.

878
00:43:36,833 --> 00:43:37,153
Yeah.

879
00:43:37,213 --> 00:43:40,753
Waiting for the iOS app store to
finally approve your app to be

880
00:43:40,763 --> 00:43:43,123
released is not a fun state to be in.

881
00:43:43,203 --> 00:43:47,673
And having the ability to just in
a matter of seconds, release a new

882
00:43:47,683 --> 00:43:49,303
version of the, of the web app.

883
00:43:49,598 --> 00:43:51,088
It's just so liberating.

884
00:43:51,398 --> 00:43:55,858
So you've now went from this transition
of initially building an Electron

885
00:43:55,868 --> 00:43:57,678
app, which was just an Electron app.

886
00:43:57,718 --> 00:44:02,028
And then you went to another
platform, mobile, Android, and iOS,

887
00:44:02,418 --> 00:44:04,268
where you could still bring Node.

888
00:44:04,278 --> 00:44:05,528
js with you.

889
00:44:05,778 --> 00:44:10,898
And so the, the platforms were still
like similar enough and they, you could

890
00:44:10,898 --> 00:44:15,788
leverage the native aspects of those
platforms still to the extent that

891
00:44:15,798 --> 00:44:20,218
you've so far needed, um, uh, But the
web is quite different in that regard.

892
00:44:20,468 --> 00:44:23,688
There is no native C
bindings you can leverage.

893
00:44:24,088 --> 00:44:29,618
And I'm not sure how far Wasm was along at
that point, but there would have been one

894
00:44:29,618 --> 00:44:34,948
path where we just say, okay, I need to
completely rewrite Actual, maybe even give

895
00:44:34,948 --> 00:44:40,558
up on SQLite and just like embrace all the
standard things that people do on the web

896
00:44:40,968 --> 00:44:45,678
or somehow bring the architecture and the
technological choices that you had so far.

897
00:44:45,983 --> 00:44:47,123
Bring them to the web.

898
00:44:47,153 --> 00:44:50,603
And that means like a
pretty intense pioneer path.

899
00:44:50,983 --> 00:44:52,633
And I think you've chosen the letter.

900
00:44:52,833 --> 00:44:54,033
So tell me more about that.

901
00:44:54,263 --> 00:44:54,963
Absolutely.

902
00:44:55,003 --> 00:44:58,853
Yeah, it was really a fun, a fun
experiment because the things

903
00:44:58,853 --> 00:45:00,943
that I was using were not that.

904
00:45:01,048 --> 00:45:04,498
Like novel SQLite was really like
the biggest one that just did not

905
00:45:04,498 --> 00:45:06,148
work on the web, everything else.

906
00:45:06,148 --> 00:45:08,228
You know, like if you need like
a background process, you can

907
00:45:08,228 --> 00:45:09,638
just fire up a, a web worker.

908
00:45:09,638 --> 00:45:11,058
It's, it's not too bad.

909
00:45:11,268 --> 00:45:12,908
I'm trying to think of other things I did.

910
00:45:12,968 --> 00:45:17,838
So I was using the, the old node async
hooks, which is now the async local

911
00:45:17,838 --> 00:45:22,688
storage for some really, really neat
stuff for, for the undo and redo mechanism

912
00:45:22,708 --> 00:45:28,158
for Actual, uh, it actually tracks the
messages that get generated for an entire.

913
00:45:28,333 --> 00:45:29,413
Like API requests.

914
00:45:29,523 --> 00:45:30,853
So I have these like handlers.

915
00:45:31,153 --> 00:45:35,773
And so it sets in local and async
local storage, like a buffer of

916
00:45:35,773 --> 00:45:38,073
messages that is fresh each time.

917
00:45:38,273 --> 00:45:42,743
And so when you send an action to do,
while it's executing that entire, like

918
00:45:43,303 --> 00:45:47,923
transaction create method, it tracks
all of the messages that are done

919
00:45:47,933 --> 00:45:49,443
during, like created during that time.

920
00:45:49,813 --> 00:45:53,373
And then it, and then when that
action is done, it reads that from

921
00:45:53,373 --> 00:45:56,983
that buffer and then it packages
them up as like an, like an undo.

922
00:45:57,273 --> 00:45:58,293
Packet, right?

923
00:45:58,313 --> 00:46:00,673
That, and that then gets
stored onto another queue.

924
00:46:00,943 --> 00:46:04,703
And so if you want to undo that,
that's how it knows like which

925
00:46:04,763 --> 00:46:08,683
messages and through which points
of time it needs to like, it needs

926
00:46:08,683 --> 00:46:10,343
to undo up to a certain time, right?

927
00:46:10,423 --> 00:46:12,543
And so the async local source is
really, really great for that.

928
00:46:12,543 --> 00:46:15,153
Cause I didn't have to thread
through all of this stuff.

929
00:46:15,483 --> 00:46:17,793
The web just doesn't support
those kinds of things at all.

930
00:46:17,893 --> 00:46:20,273
So there are things like
that, that I had to undo.

931
00:46:20,528 --> 00:46:23,968
Luckily, that actually improved things
on mobile as well, because Node.

932
00:46:23,968 --> 00:46:27,208
js mobile was actually, I was
finding some, some bugs with some

933
00:46:27,208 --> 00:46:29,588
of those more advanced Node APIs.

934
00:46:30,088 --> 00:46:33,008
So there are things that I had to
simplify and like kind of undo,

935
00:46:33,288 --> 00:46:36,168
but for the most part, SQLite
was like the main hard problem.

936
00:46:37,018 --> 00:46:41,288
And so yes, this is where I, I
experimented and I played and I

937
00:46:41,408 --> 00:46:44,478
was like, it's I mean, WebAssembly
was pretty mature at this point.

938
00:46:44,478 --> 00:46:46,438
I think this was like 2020 or so.

939
00:46:46,468 --> 00:46:49,278
Um, I could be wrong on that, but it
was, you know, like WebAssembly was

940
00:46:49,298 --> 00:46:50,898
like a pretty good at that point.

941
00:46:50,908 --> 00:46:53,958
I had no problems like trying
to bet on WebAssembly then.

942
00:46:54,068 --> 00:46:55,883
It was not that hard to compile.

943
00:46:56,173 --> 00:46:58,503
Uh, SQLite's WebAssembly
at that time, the SQL.

944
00:46:58,503 --> 00:47:01,773
js was, I think, the big library that
already came with the pre compiled SQLite.

945
00:47:01,793 --> 00:47:04,363
So I got the app running with SQL.

946
00:47:04,364 --> 00:47:06,993
js and without a ton
of work, to be honest.

947
00:47:07,223 --> 00:47:10,143
Then it was like, crap, what
happens when you change the data?

948
00:47:10,183 --> 00:47:15,653
Because SQLite literally just slurps in
the whole database in local memory, right?

949
00:47:15,683 --> 00:47:16,473
So you can change the data.

950
00:47:16,473 --> 00:47:17,743
You can be working with your data.

951
00:47:17,853 --> 00:47:19,563
I mean, it all works totally fine.

952
00:47:20,083 --> 00:47:22,583
But then you refresh the tab
and then all that data is lost.

953
00:47:22,593 --> 00:47:24,313
So obviously that's not,
that's not going to work.

954
00:47:24,353 --> 00:47:28,303
So I came up with all these ideas,
like, well, what if I like stored the

955
00:47:28,313 --> 00:47:31,263
messages persistently in the background?

956
00:47:31,283 --> 00:47:33,723
And then like, when you
refresh the tab, it.

957
00:47:34,243 --> 00:47:36,893
You know, reapply this
messages after loading it up.

958
00:47:37,073 --> 00:47:40,163
But that means that every single time
you open up a new tab, it's loading

959
00:47:40,163 --> 00:47:41,943
in the entire database into memory.

960
00:47:41,963 --> 00:47:45,713
And like, for me, at least that
was like a hard requirement.

961
00:47:46,003 --> 00:47:48,613
Do not load the whole
database into memory.

962
00:47:48,823 --> 00:47:52,763
Even worse than that, do not write
the whole database back into memory.

963
00:47:52,793 --> 00:47:55,563
Cause that was, that was the thing that
some people were doing at this time.

964
00:47:55,793 --> 00:47:57,543
They were like, Oh, just
solve this persistent thing.

965
00:47:57,713 --> 00:47:57,913
Okay.

966
00:47:57,933 --> 00:48:00,853
You change one number from four to
five, and you're going to rewrite

967
00:48:00,853 --> 00:48:02,873
this entire six megabyte database.

968
00:48:03,273 --> 00:48:06,163
Back into memory, it is so inefficient.

969
00:48:06,173 --> 00:48:09,523
And I think that's one of the, my gripes
with the web is that like, we've, we've

970
00:48:09,523 --> 00:48:16,323
lost the plot of software development
where it's, where our bar for, for quality

971
00:48:16,383 --> 00:48:22,098
and, and good engineering practices
is so low and it's, Might be a spicy

972
00:48:22,098 --> 00:48:27,108
take, but like, we just accept the
fact that things have to be this bad.

973
00:48:27,228 --> 00:48:31,898
And like, it's so like acceptable to be
like, well, there's nothing I can do.

974
00:48:31,898 --> 00:48:33,068
It's out of my control.

975
00:48:33,478 --> 00:48:33,808
Right.

976
00:48:33,828 --> 00:48:34,928
So I'm just going to write, it's right.

977
00:48:34,928 --> 00:48:35,898
Six megabytes of memory.

978
00:48:35,898 --> 00:48:39,048
Every single time I change
something that is like so bad for

979
00:48:39,068 --> 00:48:41,128
like your computer's hard drive.

980
00:48:41,129 --> 00:48:42,008
It's so bad for your.

981
00:48:42,353 --> 00:48:45,793
Memory consumption, your, your power
is so bad in all sorts of ways, right?

982
00:48:45,793 --> 00:48:46,813
It's so wasteful.

983
00:48:46,843 --> 00:48:51,013
Man, it just makes the app entirely
slow and prone to all kinds of problems.

984
00:48:51,013 --> 00:48:55,253
Like it's, I do have gripes with
just, I feel like our bar as

985
00:48:55,253 --> 00:48:56,403
an industry needs to be higher.

986
00:48:56,403 --> 00:48:59,973
And I, I honestly think that web
development is to blame for some of that.

987
00:49:00,013 --> 00:49:03,383
That's my spicy take because it makes
it easy for us to throw up our hands.

988
00:49:03,963 --> 00:49:07,133
And say, I can't go past
that layer of abstraction.

989
00:49:07,323 --> 00:49:09,733
Whereas in the native
world, you have a problem.

990
00:49:09,753 --> 00:49:13,393
You can dig into those, the C binaries.

991
00:49:13,433 --> 00:49:17,063
it might be hard, but you have the
power to go in and change things.

992
00:49:17,063 --> 00:49:19,293
Whereas on the web, it's
like, it just works that way.

993
00:49:19,293 --> 00:49:19,793
It sucks.

994
00:49:20,153 --> 00:49:20,973
Like too bad.

995
00:49:21,053 --> 00:49:22,683
I think we need aim, aim higher.

996
00:49:22,773 --> 00:49:25,103
And so that's, that's, that
was my approach, right?

997
00:49:25,133 --> 00:49:26,583
I was like, I'm not, I'm
not going to accept this.

998
00:49:26,583 --> 00:49:27,503
This is unacceptable.

999
00:49:27,543 --> 00:49:29,813
Is there a way I, um,
I can get this to work?

1000
00:49:30,123 --> 00:49:30,883
And maybe there wasn't.

1001
00:49:30,883 --> 00:49:32,743
And then I would have
to just sort of give up.

1002
00:49:32,893 --> 00:49:34,953
But I discovered a kind of
novel thing that ended up

1003
00:49:34,953 --> 00:49:36,073
working actually pretty well.

1004
00:49:36,133 --> 00:49:39,708
And that's how absurd-sql came
about was I, I figured out a way.

1005
00:49:39,808 --> 00:49:43,798
The real trick here was, so don't
read everything from memory, right?

1006
00:49:44,078 --> 00:49:45,798
So how do I not read
everything from memory?

1007
00:49:45,798 --> 00:49:49,658
Well, I have to store it in an existing
database that exists on the web.

1008
00:49:49,688 --> 00:49:52,618
There's no question at the time
that that was IndexedDB, right?

1009
00:49:52,628 --> 00:49:55,508
That was like the only
mature database abstraction.

1010
00:49:55,838 --> 00:49:59,348
Okay, so I can store all this
in IndexedDB, this whole blob.

1011
00:49:59,513 --> 00:50:02,433
But I'm still having to read
this whole thing into memory.

1012
00:50:02,443 --> 00:50:03,673
How do I stop that?

1013
00:50:04,163 --> 00:50:06,833
So I figured out, I looked
into SQLite's internal APIs.

1014
00:50:06,833 --> 00:50:09,543
At this point, I was pretty familiar
with SQLite's internal APIs.

1015
00:50:09,543 --> 00:50:11,123
Honestly, it's a really fun read.

1016
00:50:11,163 --> 00:50:14,283
They're well documented,
really straightforward C code.

1017
00:50:14,303 --> 00:50:15,693
It's really, really fun.

1018
00:50:15,793 --> 00:50:17,673
So basically, SQLite works in blocks.

1019
00:50:17,733 --> 00:50:19,773
So you have like, I
forget what size it is.

1020
00:50:20,318 --> 00:50:21,368
Is it 5k blocks?

1021
00:50:21,558 --> 00:50:24,678
Like you can actually change, change the
block size that SQLite works, works by.

1022
00:50:24,738 --> 00:50:27,538
But like, I think the default is 5k.

1023
00:50:27,538 --> 00:50:30,848
I think like 4k, or 8k,
something like that.

1024
00:50:30,848 --> 00:50:32,758
But like, yeah, that,
that ballpark, I think.

1025
00:50:32,938 --> 00:50:33,168
Yeah.

1026
00:50:33,168 --> 00:50:36,488
So that's, that's the amount that it
reads from your hard disk per chunk.

1027
00:50:36,508 --> 00:50:38,788
Like if it needs to run,
read one little bit.

1028
00:50:39,148 --> 00:50:41,178
It reads at least that amount, right?

1029
00:50:41,188 --> 00:50:43,998
So it like reads a whole block in
and then it does what it needs to.

1030
00:50:44,198 --> 00:50:47,168
And then it writes that block back
in if it, if it needs to write stuff.

1031
00:50:47,278 --> 00:50:51,368
So it does, that's how it like, doesn't
read everything in from all the memory.

1032
00:50:51,418 --> 00:50:54,928
Um, and so I thought, I was like,
okay, well, what if, what if in

1033
00:50:55,268 --> 00:50:56,378
indexedDB we store these blocks?

1034
00:50:56,708 --> 00:51:00,448
Like that is the, like, this needs to
be just basically like a file system.

1035
00:51:00,458 --> 00:51:02,798
Like I need to make this
treat as a file system.

1036
00:51:02,798 --> 00:51:03,748
And so I.

1037
00:51:04,058 --> 00:51:07,248
Started playing with this
and I, I, conceptually, I was

1038
00:51:07,248 --> 00:51:08,908
like, this, this could work.

1039
00:51:08,908 --> 00:51:11,768
Like, I don't see any reason why
this couldn't work because once

1040
00:51:12,038 --> 00:51:14,658
I have that, once you can read
different blocks and you don't have

1041
00:51:14,658 --> 00:51:16,018
to read the whole thing into memory.

1042
00:51:16,258 --> 00:51:18,848
Well, SQLite, it was very
straightforward in SQLite's code

1043
00:51:18,858 --> 00:51:20,178
about how it writes stuff down.

1044
00:51:20,228 --> 00:51:22,578
I was like, well, I can also, I
can also just write the blocks.

1045
00:51:23,198 --> 00:51:23,578
Right?

1046
00:51:23,928 --> 00:51:27,238
Every, I feel like every, every
three or four years I have this,

1047
00:51:27,238 --> 00:51:31,838
like, I see a gap and like, it's,
it's intriguing to me a lot.

1048
00:51:31,878 --> 00:51:34,538
And like, I, I like prettier
was kind of this thing too.

1049
00:51:34,548 --> 00:51:37,078
Like I see this thing that's like, man,
there's this problem that everything

1050
00:51:37,078 --> 00:51:40,318
is having, but like, I see this thing
that works really well over here.

1051
00:51:40,318 --> 00:51:41,338
Like, can we just.

1052
00:51:41,893 --> 00:51:43,673
Take that from over there
and like apply it here.

1053
00:51:43,673 --> 00:51:44,733
Like, like, does that work?

1054
00:51:44,853 --> 00:51:47,603
And you know, a third, a third
of the time, it ends up being an

1055
00:51:47,603 --> 00:51:48,653
interesting thing that does work.

1056
00:51:48,713 --> 00:51:53,533
Essentially all of that to say,
I discovered how to store these

1057
00:51:53,533 --> 00:51:58,553
blocks in, into IndexedDB to read
and write them from, from IndexedDB.

1058
00:51:58,803 --> 00:52:02,313
And I was able to leverage
IndexedDB's transactional

1059
00:52:02,333 --> 00:52:05,063
semantics so that locking worked.

1060
00:52:05,133 --> 00:52:09,018
And that's the real critical thing here
is that like SQLite, heavily relies

1061
00:52:09,048 --> 00:52:13,458
on file system locking API, such that
says, I'm going to lock this block.

1062
00:52:13,478 --> 00:52:14,928
I'm going to lock this piece of data here.

1063
00:52:15,308 --> 00:52:16,818
Do not let anything else write to that.

1064
00:52:16,818 --> 00:52:18,028
That is a stable database.

1065
00:52:18,048 --> 00:52:19,798
Like that is how it does not get corrupt.

1066
00:52:19,888 --> 00:52:22,608
If it, if you break those
semantics, it will get corrupt.

1067
00:52:22,978 --> 00:52:25,368
And so, um, I was able to leverage.

1068
00:52:25,878 --> 00:52:27,538
IndexedDB is like locking semantics.

1069
00:52:27,538 --> 00:52:30,658
And I was like, I can map these
onto these such that like, when I'm

1070
00:52:30,658 --> 00:52:32,868
writing this thing, nothing else
should be able to read from it.

1071
00:52:32,938 --> 00:52:33,678
And it worked.

1072
00:52:33,758 --> 00:52:36,458
So you can like load in
the data in different tabs.

1073
00:52:36,458 --> 00:52:39,988
You can write to that in like one tab
and like see that data in the other tab.

1074
00:52:40,068 --> 00:52:41,108
And it worked out really well.

1075
00:52:41,108 --> 00:52:43,028
And I was like, this would be
a really interesting thing to

1076
00:52:43,028 --> 00:52:43,938
open source and talk about.

1077
00:52:43,958 --> 00:52:44,738
And so.

1078
00:52:44,738 --> 00:52:48,388
It became absurd-sql and it became
like a, a pretty influential thing.

1079
00:52:48,388 --> 00:52:52,578
I think, honestly, this kind of project,
I invest like a month or two of my life

1080
00:52:52,578 --> 00:52:54,128
into, and then like, I have a day job.

1081
00:52:54,178 --> 00:52:55,928
And so I just did not have a lot of time.

1082
00:52:55,948 --> 00:52:59,268
Like I got it working for Actual and it
deployed Actual and it worked on the web.

1083
00:52:59,268 --> 00:53:03,238
And that was amazing, but I just didn't
have the resources or the time to really

1084
00:53:03,238 --> 00:53:07,028
like build it out or like fix really
obscure bugs that people were coming up, I

1085
00:53:07,028 --> 00:53:10,203
sort of like kind of left it as a project
that wasn't, that wasn't Maintained

1086
00:53:10,203 --> 00:53:13,143
super well, and sometimes I feel guilty
about that, but I think there's other

1087
00:53:13,173 --> 00:53:14,623
community members that have taken it on.

1088
00:53:14,623 --> 00:53:17,273
It's, it's more of like an
influential prototype, I think,

1089
00:53:17,553 --> 00:53:19,783
more than like a mantained thing.

1090
00:53:19,928 --> 00:53:24,378
Yeah, I think for me, this is
actually kind of like a, a definition

1091
00:53:24,398 --> 00:53:26,838
of a kind of project category.

1092
00:53:26,868 --> 00:53:30,658
Like not every project needs
to like prettier lived on and

1093
00:53:30,658 --> 00:53:32,148
now it's like super common.

1094
00:53:32,178 --> 00:53:36,778
And like, I think that's one very
successful way how a project can evolve.

1095
00:53:37,078 --> 00:53:41,518
But I think with absurd-sql, you've built
something it's even more than a proof of

1096
00:53:41,518 --> 00:53:44,158
concept because you used it in production.

1097
00:53:44,198 --> 00:53:46,343
I, I use it for a while in production.

1098
00:53:46,353 --> 00:53:50,523
Other people use it for a while in
production, but, uh, like you said, like

1099
00:53:50,533 --> 00:53:52,983
it became a very influential project.

1100
00:53:53,003 --> 00:53:57,813
And I think it showed to the world,
like, Hey, we don't need to settle for

1101
00:53:57,833 --> 00:54:02,893
saving six megabytes or 50 megabytes
for every little change as you do.

1102
00:54:02,903 --> 00:54:07,113
Want to do persistence in the web before
people would just like ride, like.

1103
00:54:07,498 --> 00:54:12,218
80 megabyte JSON files, like all the
time to, to index to be, and then we're

1104
00:54:12,448 --> 00:54:16,158
then wonder why their, the app is slow
and their, their CPU is going crazy.

1105
00:54:16,438 --> 00:54:20,158
So you aiming for higher, I
think that has a massive impact.

1106
00:54:20,188 --> 00:54:24,118
And I think that's now those
ideas are still like leveraged

1107
00:54:24,128 --> 00:54:25,648
now in other projects.

1108
00:54:25,658 --> 00:54:26,903
Whether I think.

1109
00:54:26,903 --> 00:54:32,433
The wa-sqlite, um, project that is,
I think also using some, some similar

1110
00:54:32,433 --> 00:54:37,993
approaches and by now we also have a
different storage mechanism and the web

1111
00:54:37,993 --> 00:54:42,513
that is more and more common, namely
OPFS, original private file system.

1112
00:54:43,353 --> 00:54:47,483
Which also already gives you a file
system representation where you can

1113
00:54:47,493 --> 00:54:49,493
lock individual files, et cetera.

1114
00:54:49,494 --> 00:54:53,223
I think the details are still being
figured out right now, but that's, for

1115
00:54:53,223 --> 00:54:57,743
example, what I'm using for Overtone and
in production, that's already quite nice.

1116
00:54:57,783 --> 00:55:03,233
But I think you've really, you, you went
super early on that and very daring from

1117
00:55:03,243 --> 00:55:07,123
first principles and that's so admirable
and that, that like very inspiring.

1118
00:55:07,233 --> 00:55:07,893
Thank you very much.

1119
00:55:07,893 --> 00:55:08,963
I appreciate that.

1120
00:55:08,973 --> 00:55:10,773
Sometimes I, I don't know.

1121
00:55:10,793 --> 00:55:11,973
I, I don't really track.

1122
00:55:12,403 --> 00:55:16,843
The, like some of the fallout sometimes,
like I, I just had to invest in my job

1123
00:55:16,843 --> 00:55:18,333
a lot for the next like six months.

1124
00:55:18,373 --> 00:55:20,173
And so sometimes I'm, I'm not sure.

1125
00:55:20,173 --> 00:55:22,813
I don't, I don't know how much influence
it has, but it's, it's good to hear

1126
00:55:22,813 --> 00:55:25,483
feedback that work is impactful.

1127
00:55:25,583 --> 00:55:29,123
I, I would love to hear about your
experience with, with it, with OPFS.

1128
00:55:29,133 --> 00:55:31,623
And, and, uh, are you using
the file system native access

1129
00:55:31,733 --> 00:55:33,363
stuff as well and in your app?

1130
00:55:33,603 --> 00:55:34,233
How's that going?

1131
00:55:34,473 --> 00:55:35,473
The latter, not yet.

1132
00:55:35,503 --> 00:55:39,233
I'm also planning to, since like
for Overtone, which is a music

1133
00:55:39,233 --> 00:55:43,353
player, I do want to also support
you bringing your own music that

1134
00:55:43,353 --> 00:55:47,043
you might have like in a download
folder on a music folder somewhere.

1135
00:55:47,103 --> 00:55:50,683
Right now I don't use that
particular browser APIs yet.

1136
00:55:51,113 --> 00:55:54,413
But I'm using OPFS for many purposes.

1137
00:55:54,603 --> 00:55:59,353
I'm using OPFS for persistence
of a SQLite database, where you

1138
00:55:59,353 --> 00:56:01,403
would have used IndexedDB before.

1139
00:56:01,403 --> 00:56:05,213
And I think some people still
use IndexedDB for targeting

1140
00:56:05,263 --> 00:56:06,513
older browsers as well.

1141
00:56:06,573 --> 00:56:11,133
But I'm also using OPFS for like what
you would use a file system for as

1142
00:56:11,133 --> 00:56:13,083
well, which is like storing files.

1143
00:56:13,113 --> 00:56:17,998
So for example, Before displaying
images in the app, I actually don't

1144
00:56:18,008 --> 00:56:22,368
use like just an image tag and
then point to an external URL since

1145
00:56:22,618 --> 00:56:24,538
those URLs might go away, et cetera.

1146
00:56:24,538 --> 00:56:30,088
So I actually download those images, store
them in OPFS and then pre process them.

1147
00:56:30,313 --> 00:56:34,093
Um, on a worker and sent them
over on a, on an off screen

1148
00:56:34,123 --> 00:56:35,243
canvas to the main thread.

1149
00:56:35,703 --> 00:56:40,123
So I'm like using some, some more
like native development practices here

1150
00:56:40,123 --> 00:56:41,573
and trying to bring them to the web.

1151
00:56:41,683 --> 00:56:45,953
Very, very much sharing the same
opinion, like your spicy take from,

1152
00:56:46,133 --> 00:56:49,683
for me, not so spicy take that
we should aim higher in the web.

1153
00:56:49,703 --> 00:56:51,923
And I think we can learn quite a lot from.

1154
00:56:52,253 --> 00:56:54,423
More native development backgrounds.

1155
00:56:54,813 --> 00:57:00,153
And this is, I think this is how we
get, uh, after all pretty fast web apps.

1156
00:57:00,403 --> 00:57:04,793
Figma is another notable example
there where how you get actually a

1157
00:57:04,833 --> 00:57:10,583
really high performance app that feels
nice is by aiming higher and, uh,

1158
00:57:10,593 --> 00:57:14,693
bringing some of those methodologies
from other environments to the web.

1159
00:57:14,953 --> 00:57:15,993
So I think.

1160
00:57:16,168 --> 00:57:19,748
There's still a couple of
interesting aspects in absurd-sql

1161
00:57:19,768 --> 00:57:21,868
that we haven't yet gotten into.

1162
00:57:21,968 --> 00:57:26,528
So you've mentioned that you're
using index to be, uh, with, uh, the

1163
00:57:26,528 --> 00:57:30,428
block, um, the, the block storage
and the index to be transactions,

1164
00:57:30,868 --> 00:57:33,318
but to actually write that you could.

1165
00:57:33,578 --> 00:57:37,048
I guess either do, could you do
that on the main thread since you've

1166
00:57:37,048 --> 00:57:38,718
chosen to do that on a worker?

1167
00:57:38,938 --> 00:57:42,458
So maybe you can talk a little bit
about the, the threading model and

1168
00:57:42,468 --> 00:57:49,338
also how you even went beyond IndexedDB
transactions and using atomics and

1169
00:57:49,338 --> 00:57:54,628
shared array buffers to still slay some
dragons that needed to be slayed here.

1170
00:57:54,798 --> 00:57:59,348
Yeah, there is one little trick that I
discovered and I think I was actually

1171
00:57:59,348 --> 00:58:01,668
a live stream and I think I have a
recording of it, which was kind of fun.

1172
00:58:01,668 --> 00:58:04,758
It's like a fun little blew
my mind at, um, at the time.

1173
00:58:04,848 --> 00:58:07,328
And I think this is a trick
that was independently

1174
00:58:07,328 --> 00:58:08,608
discovered by several people.

1175
00:58:08,618 --> 00:58:12,678
I think there's a, uh, there's another
library that I think from is from some,

1176
00:58:12,698 --> 00:58:15,998
some Google folks about how, like,
like loading in third party things

1177
00:58:15,999 --> 00:58:19,618
on, you know, In like an iframe or
something to kind of like sandbox things

1178
00:58:19,668 --> 00:58:21,478
that that uses this same technique.

1179
00:58:21,498 --> 00:58:23,998
But essentially here's,
here's the problem.

1180
00:58:24,288 --> 00:58:28,548
When you compile SQLite through
WebAssembly, you can intercept

1181
00:58:28,568 --> 00:58:30,238
the API calls that it makes.

1182
00:58:30,248 --> 00:58:33,778
So you can intercept the like read
The read and write command, like

1183
00:58:33,778 --> 00:58:37,148
there's a C API is reading right
for reading and writing files.

1184
00:58:37,338 --> 00:58:41,028
And so you can say, okay, I'm going to
implement the read command for like,

1185
00:58:41,328 --> 00:58:45,078
through WebAssembly, the WebAssembly
compiled version of this SQLite.

1186
00:58:45,118 --> 00:58:47,218
I'm going to implement the
read command and like read some

1187
00:58:47,218 --> 00:58:48,388
data for it and give it back.

1188
00:58:48,658 --> 00:58:52,348
So I can just read from index, from index
db and everything is going to work right.

1189
00:58:52,478 --> 00:58:55,208
The, the problem is, is that
the read command from C is

1190
00:58:55,238 --> 00:58:56,678
completely synchronous, right?

1191
00:58:56,708 --> 00:59:00,478
So the WebAssembly compiled binary
expects that to be synchronous.

1192
00:59:00,528 --> 00:59:03,768
Like you can, it'll call out to
JavaScript and you can implement the read

1193
00:59:03,768 --> 00:59:06,338
command, but you cannot await in there.

1194
00:59:06,348 --> 00:59:09,918
Like you literally have to, in that event
loop, synchronously return some data.

1195
00:59:10,328 --> 00:59:13,418
And so, uh, when I hit that, I was
like, I thought I was dead in the water.

1196
00:59:13,418 --> 00:59:14,318
I was like this, this.

1197
00:59:14,708 --> 00:59:15,428
I just can't do this.

1198
00:59:15,718 --> 00:59:18,098
Maybe, maybe it wasn't that bad
actually, because I think I did

1199
00:59:18,148 --> 00:59:22,428
see, so one option is you can in
WebAssembly, I think it has a way to

1200
00:59:22,538 --> 00:59:24,518
convert synchronous APIs to async.

1201
00:59:24,988 --> 00:59:27,188
I think it's called like
asynchronify or something like that.

1202
00:59:27,228 --> 00:59:28,978
I was terrified of it to be honest.

1203
00:59:29,358 --> 00:59:30,708
And so I thought it just
wasn't going to work.

1204
00:59:30,778 --> 00:59:32,048
I was terrified about the performance.

1205
00:59:32,158 --> 00:59:36,388
I've had experiences in some
codebases that overuse of asynchronous

1206
00:59:36,458 --> 00:59:41,168
APIs really killed performance
because waiting one promise tick.

1207
00:59:41,533 --> 00:59:46,363
In every single layer of the
abstraction is very not noticeable

1208
00:59:46,363 --> 00:59:49,963
at a small scale, but it very
becomes noticeable at a large scale.

1209
00:59:50,113 --> 00:59:54,033
And the really pernicious thing about it
is that it's death by not a thousand cuts.

1210
00:59:54,173 --> 00:59:55,333
It's death by a million cuts.

1211
00:59:55,653 --> 00:59:57,803
Like, it's, you can't
go in there and remove.

1212
00:59:58,013 --> 00:59:59,683
20 by 20 awaits, right?

1213
00:59:59,823 --> 01:00:00,683
And like, it'll be fine.

1214
01:00:00,763 --> 01:00:03,693
The thing that I don't like about
async code is that as you abstract

1215
01:00:03,693 --> 01:00:06,393
things away, so if you take it from
two abstractions to 10 abstract, like

1216
01:00:06,403 --> 01:00:10,523
you split a two async functions into
10 async functions, well, every one

1217
01:00:10,523 --> 01:00:12,163
of them has to await on each other.

1218
01:00:12,443 --> 01:00:14,513
And so it's literally making it slower.

1219
01:00:14,698 --> 01:00:16,618
And that's not  the
case , with synchronous stuff.

1220
01:00:16,618 --> 01:00:17,598
You can make it two functions.

1221
01:00:17,598 --> 01:00:19,358
You can make it 500 functions.

1222
01:00:19,358 --> 01:00:21,608
You can abstract things
the way that you want to.

1223
01:00:21,808 --> 01:00:25,008
The shape of your abstraction
literally impacts performance when

1224
01:00:25,008 --> 01:00:26,568
you're in, in, um, async world.

1225
01:00:26,668 --> 01:00:31,258
So I have experience with code bases that
like, that did things in a really weird

1226
01:00:31,298 --> 01:00:33,508
way, like awaited literally everything.

1227
01:00:33,648 --> 01:00:36,488
And it was, it was causing
all sorts of perf problems.

1228
01:00:36,778 --> 01:00:40,438
And so that, For that reason, from my
experience, I was terrified of it because

1229
01:00:40,438 --> 01:00:44,778
I thought it made every single function
in C in the WebAssembly binary async.

1230
01:00:44,928 --> 01:00:48,528
It turns out that I was probably wrong
and I probably should have done my

1231
01:00:48,528 --> 01:00:52,278
own investigation in like performance
profiling because I got some feedback.

1232
01:00:52,713 --> 01:00:56,973
Later on, like months after I released
absurd-sql, I think from some people

1233
01:00:56,973 --> 01:01:00,463
that I respected pretty well that
were saying, uh, the async stuff is

1234
01:01:00,463 --> 01:01:03,883
actually fine, like it actually has
a very negligible performance impact.

1235
01:01:04,143 --> 01:01:06,823
So there's probably some tricks
that they, they, they do that.

1236
01:01:07,003 --> 01:01:07,723
So it's probably fine.

1237
01:01:07,723 --> 01:01:11,673
And I think that WA SQLite
project that you mentioned, I

1238
01:01:11,673 --> 01:01:13,993
think it actually uses this mode.

1239
01:01:14,273 --> 01:01:17,693
And the good thing about that is
that it doesn't need atomics, which

1240
01:01:17,713 --> 01:01:19,603
means it doesn't require HTTPS.

1241
01:01:19,953 --> 01:01:21,083
So I'm probably.

1242
01:01:21,243 --> 01:01:24,163
Let me jump back a little bit to
explain why atomics are even needed.

1243
01:01:24,333 --> 01:01:26,493
So let's go back to that
synchronous method, right?

1244
01:01:26,683 --> 01:01:30,333
So I need to call into IndexedDB
and get and do an async API.

1245
01:01:30,713 --> 01:01:31,723
How am I going to do that?

1246
01:01:31,753 --> 01:01:37,263
Well, there's a little trick that
you can do by making an async call.

1247
01:01:37,393 --> 01:01:42,403
And I believe that it's Has
to be on a different thread.

1248
01:01:42,493 --> 01:01:44,443
So there's like two
background threads, right?

1249
01:01:44,483 --> 01:01:46,123
There's like the normal backend thread.

1250
01:01:46,133 --> 01:01:48,083
And then there's another separate thread.

1251
01:01:48,303 --> 01:01:51,773
That's like the thread that reads and
writes from index CB like asynchronously.

1252
01:01:51,773 --> 01:01:52,073
Right.

1253
01:01:52,423 --> 01:01:55,963
Between those two threads, you
share a shared array buffer, which

1254
01:01:55,963 --> 01:01:59,548
is this really low level, really
interesting thing  on the web, which

1255
01:01:59,548 --> 01:02:01,138
is shared memory across threads.

1256
01:02:01,538 --> 01:02:05,478
And on top of that, you, uh, there's
APIs you can use called atomics, which

1257
01:02:05,488 --> 01:02:08,478
allow you to interact with this shared
array buffer, and you can actually

1258
01:02:08,528 --> 01:02:12,348
coordinate across the threads and do
some really, really powerful things.

1259
01:02:12,358 --> 01:02:15,468
So one of those things that you can
do is you can write to that shared

1260
01:02:15,498 --> 01:02:16,938
array buffer in a certain way.

1261
01:02:17,078 --> 01:02:19,888
And then in another thread,
you can call atomics.Wait.

1262
01:02:20,428 --> 01:02:24,118
And what atomics.wait does is it literally
synchronously blocks that thread from

1263
01:02:24,118 --> 01:02:26,198
running like it, you call that thread.

1264
01:02:26,218 --> 01:02:29,288
And if it does not wake up,
then it won't wake up ever.

1265
01:02:29,358 --> 01:02:33,288
Like you call atomic weight and you
tell it, wait for this bit to be

1266
01:02:33,288 --> 01:02:34,778
flipped in the shared array buffer.

1267
01:02:34,998 --> 01:02:37,228
And then the other thread can
flip that bit when it's not.

1268
01:02:37,293 --> 01:02:40,123
done, and then the other
thing can continue executing.

1269
01:02:40,423 --> 01:02:44,423
Using that technique, we can read from an
asynchronous thing in the second thread,

1270
01:02:44,523 --> 01:02:48,883
do whatever we need to, and store that
data in some sort of buffer somewhere.

1271
01:02:49,133 --> 01:02:52,173
And then while the first thread is
blocked on that atomic set, wait.

1272
01:02:52,453 --> 01:02:55,133
And then once the, once that
bit is flipped and it continues

1273
01:02:55,133 --> 01:02:58,143
executing, then it can read from that
buffer and actually get the data.

1274
01:02:58,423 --> 01:03:02,008
And so using that technique, Um, I
use that technique in every single API

1275
01:03:02,018 --> 01:03:06,138
in, in Upstairs SQL to turn an async
function into a synchronous function.

1276
01:03:06,428 --> 01:03:07,898
And that's how it can
interface with IndexedDB.

1277
01:03:09,168 --> 01:03:12,678
The downside though, is that to use
atomic set weight, it's one of those

1278
01:03:13,368 --> 01:03:18,628
newer APIs that Chrome and other browsers
force HTTPS a secure context on you.

1279
01:03:18,708 --> 01:03:22,318
And so your app has to be running under
HTTPS, which I've always said, like.

1280
01:03:22,603 --> 01:03:23,103
Who cares?

1281
01:03:23,113 --> 01:03:24,933
Like, of course you're going to
be running under that anyway.

1282
01:03:25,073 --> 01:03:27,473
It's been enough of a,
kind of a pain point.

1283
01:03:27,483 --> 01:03:30,063
Like people are using like weird
reverse proxies or doing their

1284
01:03:30,063 --> 01:03:32,573
own things where suppose, I guess
some people just don't really care

1285
01:03:32,583 --> 01:03:33,853
and they just want to run HTTP.

1286
01:03:33,973 --> 01:03:37,443
And so Actual, like the open source
version of Actual can't run under HTTP.

1287
01:03:37,463 --> 01:03:40,193
And that's like, you always
have to be setting up to me.

1288
01:03:40,193 --> 01:03:42,883
It's, I don't know, I still am a
little bit like, I don't really care.

1289
01:03:42,923 --> 01:03:45,843
Like you can just like set up
HTTPS, but it's enough of a source

1290
01:03:45,843 --> 01:03:48,873
of a pain that I can see benefits
and not, not having to require it.

1291
01:03:49,178 --> 01:03:49,578
Right.

1292
01:03:49,588 --> 01:03:53,828
Yeah, I think this was also related
to the, uh, Spectre exploit at

1293
01:03:53,828 --> 01:03:55,368
some point, uh, vulnerability.

1294
01:03:55,788 --> 01:03:59,988
And I think it's not just that you need
to run on HTTPS by now, but I think you

1295
01:03:59,998 --> 01:04:02,998
also need to have a few HTTP headers set.

1296
01:04:03,058 --> 01:04:07,298
I think like cross origin, uh,
open policy and embedders policy.

1297
01:04:07,358 --> 01:04:10,758
And I think there was also like
a limitation that Safari didn't

1298
01:04:10,758 --> 01:04:12,698
support it well for, for some time.

1299
01:04:12,718 --> 01:04:17,728
So yeah, there it's, Still, browsers
are still growing up, but I think at

1300
01:04:17,728 --> 01:04:21,628
some point this can be assumed that, uh,
this will just work, but on the other

1301
01:04:21,638 --> 01:04:26,108
side, I think those tricks might also,
you mentioned that asyncify approach.

1302
01:04:26,328 --> 01:04:28,428
So that is also another option.

1303
01:04:28,428 --> 01:04:31,778
And I think there's even a new
approach stabilizing right now.

1304
01:04:32,468 --> 01:04:36,178
Where WebAssembly natively
can integrate with Promises.

1305
01:04:36,588 --> 01:04:41,918
So I think that's a, that's a new, um,
development around WASM and browsers.

1306
01:04:42,298 --> 01:04:47,168
So I think there's, you, you certainly
use quite the bleeding edge there.

1307
01:04:47,478 --> 01:04:51,098
And so, yeah, right now it's
getting more stabilized.

1308
01:04:51,548 --> 01:04:55,308
But I do think there is some truth
to what you've mentioned in regards

1309
01:04:55,338 --> 01:04:59,648
to avoiding asynchronous code when
possible, since it's not just like

1310
01:04:59,648 --> 01:05:02,388
a potential performance overhead.

1311
01:05:02,558 --> 01:05:06,808
So, and I think some people go even
as far as saying that going from

1312
01:05:06,848 --> 01:05:09,418
callbacks to async await and promises.

1313
01:05:09,738 --> 01:05:12,008
Was one of the biggest
mistakes in JavaScript.

1314
01:05:12,048 --> 01:05:14,208
I think that can also be
considered a hot take.

1315
01:05:14,478 --> 01:05:17,848
Let's see where we'll end up
in a couple of years on that.

1316
01:05:17,938 --> 01:05:22,208
But, uh, aside from the performance,
I think another common downside

1317
01:05:22,238 --> 01:05:25,938
of asynchronous code is that it
basically introduces distributed

1318
01:05:25,938 --> 01:05:28,288
systems problems into, into your code.

1319
01:05:28,618 --> 01:05:32,528
In this case, where we basically
just wrap an API, I think it's okay.

1320
01:05:32,528 --> 01:05:37,853
Um, But, uh, that's a, that's another
notable difference of like how, what

1321
01:05:37,883 --> 01:05:42,563
we've been exploring with LiveStore
and, and Riffle is by really making

1322
01:05:42,563 --> 01:05:46,573
the, like in a browser context, still
from the main thread, allowing for

1323
01:05:46,573 --> 01:05:52,083
synchronous SQL queries, which return
very fast and therefore like you can just

1324
01:05:52,243 --> 01:05:54,143
write your normal JavaScript as you do.

1325
01:05:54,478 --> 01:05:56,808
comes at a risk of potentially
blocking the main thread.

1326
01:05:56,808 --> 01:06:01,058
That's a, that's a different
challenge, but if you, if you have

1327
01:06:01,138 --> 01:06:04,168
performance under control, it gives
you a much simpler programming model.

1328
01:06:04,448 --> 01:06:05,618
So that's another weak one.

1329
01:06:05,688 --> 01:06:09,568
Yeah, that's, I'm all here
for synchronous SQL queries.

1330
01:06:09,618 --> 01:06:11,238
I think it's crazy.

1331
01:06:11,238 --> 01:06:14,488
I think there's been no libraries
before that integrated with SQLite 3 and

1332
01:06:14,488 --> 01:06:16,418
like made all of the APIs, um, async.

1333
01:06:16,458 --> 01:06:17,138
It was crazy.

1334
01:06:17,138 --> 01:06:20,348
Like it's such a in
memory super close local.

1335
01:06:20,613 --> 01:06:24,153
thing and , you really want at the
low level to, to provide the at

1336
01:06:24,153 --> 01:06:25,233
least option to be synchronous.

1337
01:06:25,473 --> 01:06:28,793
And it, so that I just remembered
actually, it's not just

1338
01:06:28,793 --> 01:06:30,313
wrapping an async function.

1339
01:06:30,513 --> 01:06:32,933
So like it's, it's an internal
implementation detail.

1340
01:06:33,203 --> 01:06:37,903
If you go the asynchronous by route,
which turns C functions into asynchronous

1341
01:06:37,903 --> 01:06:40,993
functions, therefore you can interact
with like asynchronous stuff.

1342
01:06:41,428 --> 01:06:46,618
Believe that that, that forces you
when you call a SQLite by like method,

1343
01:06:47,118 --> 01:06:48,798
that method is an asynchronous method.

1344
01:06:49,198 --> 01:06:51,878
You cannot synchronously execute a SQL.

1345
01:06:51,898 --> 01:06:56,898
That was a big reason why I also really
did not want to use WI SQL because I

1346
01:06:56,898 --> 01:06:58,308
wanted my functions to be synchronous.

1347
01:06:58,598 --> 01:07:01,778
My app depends on many of them being
synchronous and just my workflows.

1348
01:07:01,778 --> 01:07:04,428
And it just, it just greatly
simplifies the entire workflow.

1349
01:07:04,658 --> 01:07:06,548
There's no reason for
me to make this async.

1350
01:07:06,788 --> 01:07:08,278
This is a single client.

1351
01:07:08,798 --> 01:07:11,788
App, there's one request
coming through at a time.

1352
01:07:11,798 --> 01:07:13,768
I can control it entirely, right?

1353
01:07:13,768 --> 01:07:16,738
This is not a web server handling
thousands of requests at a time

1354
01:07:17,378 --> 01:07:21,158
to take on the complexities of
asynchronous code with the performance

1355
01:07:21,208 --> 01:07:23,438
hits was just ridiculous to me.

1356
01:07:23,518 --> 01:07:28,633
And so I, This technique I still think
has has merit, actually, the more I think

1357
01:07:28,643 --> 01:07:33,893
about it, and it's something that I owe
this community like a great deal of, of

1358
01:07:34,193 --> 01:07:36,483
like blog posts and and and research.

1359
01:07:36,483 --> 01:07:39,163
I need to sit down and really
like go through what the latest

1360
01:07:39,163 --> 01:07:40,603
and greatest is and really.

1361
01:07:40,928 --> 01:07:44,188
Vetted and see, see what people
are landing on, because if it's

1362
01:07:44,248 --> 01:07:47,008
still not possible to do that,
I think that's a huge downside.

1363
01:07:47,258 --> 01:07:47,808
I do.

1364
01:07:47,818 --> 01:07:51,058
I did just see that the, the
file system access APIs, I

1365
01:07:51,058 --> 01:07:52,738
believe they finally converted.

1366
01:07:53,018 --> 01:07:56,918
Originally there were some of them
in the, in a worker thread that

1367
01:07:56,958 --> 01:08:00,438
were supposed to be synchronous, but
in the spec, we're actually async.

1368
01:08:00,798 --> 01:08:03,268
And that made me really, really not happy.

1369
01:08:03,298 --> 01:08:05,158
And I filed like a GitHub ticket.

1370
01:08:05,368 --> 01:08:07,108
I think they finally, if I.

1371
01:08:07,163 --> 01:08:08,453
Just checked earlier this morning.

1372
01:08:08,453 --> 01:08:11,393
They finally have on the MDN
page says that they are now

1373
01:08:11,393 --> 01:08:12,263
synchronous, which is great.

1374
01:08:12,323 --> 01:08:14,333
So hopefully we're moving
in the right direction.

1375
01:08:14,653 --> 01:08:17,413
But I think it's, yeah, I think it's
really, I'm all about synchronous 'cause

1376
01:08:17,413 --> 01:08:21,883
it means that you can use them in context
that like are synchronous, like there you

1377
01:08:21,888 --> 01:08:27,313
might be removing a lot of functionality
because once something is async, the thing

1378
01:08:27,313 --> 01:08:28,993
that uses it has to be async as well.

1379
01:08:29,113 --> 01:08:32,088
And a lot of these cases for local-first
apps especially, it just is, there's,

1380
01:08:32,233 --> 01:08:33,763
there's literally no benefit to it.

1381
01:08:33,923 --> 01:08:35,033
, it's a local app.

1382
01:08:35,238 --> 01:08:37,968
There's not a second user that
can come be querying this.

1383
01:08:38,278 --> 01:08:38,628
I don't know.

1384
01:08:38,668 --> 01:08:39,898
I think I get it.

1385
01:08:40,188 --> 01:08:42,578
I think we've, we've over
indexed a little bit on this.

1386
01:08:42,578 --> 01:08:44,988
And I do think that I
forgot about the headers.

1387
01:08:45,128 --> 01:08:48,608
That is because like reverse
proxies can like drop those headers

1388
01:08:48,608 --> 01:08:50,938
and then like a user is like,
why isn't this app even loading?

1389
01:08:50,958 --> 01:08:51,868
So I, I don't know.

1390
01:08:51,878 --> 01:08:52,528
It's a trade off.

1391
01:08:52,558 --> 01:08:55,518
I still probably lean a little
bit towards it's worth it.

1392
01:08:55,568 --> 01:09:00,408
But yeah, I completely agree with, uh,
I think this is the same theme again,

1393
01:09:00,428 --> 01:09:05,848
where in web development, we've kind
of gotten so used to some practices.

1394
01:09:06,048 --> 01:09:09,708
And I think it's one is, uh,
being efficient and performance

1395
01:09:09,708 --> 01:09:13,318
minded, but another is just
like the programming models, how

1396
01:09:13,318 --> 01:09:15,398
they've kind of eroded over time.

1397
01:09:15,808 --> 01:09:18,508
we need to deal with distributed
systems problems where they're

1398
01:09:18,508 --> 01:09:20,128
just completely accidental.

1399
01:09:20,508 --> 01:09:25,208
And , we're just so used to, to like
so many things being asynchronous,

1400
01:09:25,278 --> 01:09:27,068
which doesn't need to be asynchronous.

1401
01:09:27,398 --> 01:09:30,508
We kind of went from callback
hell to async hell in a way.

1402
01:09:30,568 --> 01:09:35,108
And in React we use, useEffect for
so many things where we shouldn't.

1403
01:09:35,438 --> 01:09:38,128
And  is just so wild
that like most of our.

1404
01:09:38,358 --> 01:09:43,088
Data interactions are asynchronous,
like in a way it's almost like if

1405
01:09:43,138 --> 01:09:48,252
react would not just give you a value
right away, but would give you like a

1406
01:09:48,282 --> 01:09:52,912
tuple of like either it's loading or
a value is like we're already halfway

1407
01:09:52,922 --> 01:09:55,542
there just in, in such a bad direction.

1408
01:09:55,872 --> 01:10:00,292
So I, uh, I think this is kind of a
subtle difference for people to understand

1409
01:10:00,312 --> 01:10:04,792
how much synchronous code execution
can simplify your app development.

1410
01:10:05,322 --> 01:10:08,712
But I think I'm, I'm preaching
to, to the choir here.

1411
01:10:09,842 --> 01:10:10,612
Yeah, sure.

1412
01:10:10,822 --> 01:10:11,242
Totally.

1413
01:10:11,242 --> 01:10:12,702
I mean, it helps with debugging.

1414
01:10:12,712 --> 01:10:15,432
Like if you're stepping over code,
whenever you're hitting like async

1415
01:10:15,492 --> 01:10:19,642
code, Chrome tries to do this thing
where it like will step over the await,

1416
01:10:19,642 --> 01:10:21,142
but it only works like half the time.

1417
01:10:21,152 --> 01:10:23,322
Like it's yeah, it's super annoying.

1418
01:10:23,452 --> 01:10:27,352
I do get this needed probably most of
the time, but it makes me sad when I just

1419
01:10:27,382 --> 01:10:29,602
see it like applied without any thought.

1420
01:10:29,897 --> 01:10:35,367
So after fulfilling your goal with
bringing Actual to the web through

1421
01:10:35,367 --> 01:10:39,847
absurd-sql, which I think is like
just to look back, like how much

1422
01:10:39,867 --> 01:10:42,817
pioneering work you've really
done to make this app happen.

1423
01:10:42,827 --> 01:10:45,077
Like you started this journey before.

1424
01:10:45,422 --> 01:10:47,442
The term local-first was there.

1425
01:10:47,472 --> 01:10:51,782
You've built one of the first credible
local-first apps and really invented

1426
01:10:51,912 --> 01:10:56,322
so many things along the way, like all
by yourself with the help from, from

1427
01:10:56,342 --> 01:11:00,212
some of your, your friends, but you,
like you figured out like how to use

1428
01:11:00,252 --> 01:11:05,672
SQLite in like even in a web context in
a reactive way, but then also made it

1429
01:11:05,702 --> 01:11:09,512
collaborative through your sync engine and
ultimately brought all of this to the web.

1430
01:11:09,912 --> 01:11:11,712
That is an impressive journey.

1431
01:11:11,807 --> 01:11:15,487
And I think you've been on the journey,
not just building all of this full

1432
01:11:15,487 --> 01:11:20,247
time, but you actually had like some,
some, some Actual full time job next

1433
01:11:20,247 --> 01:11:24,867
to that, which I think at some point
was just too much, which led you to,

1434
01:11:24,897 --> 01:11:29,307
to at some point, uh, hand over the
project to the broader community.

1435
01:11:29,607 --> 01:11:32,097
Can you share a little bit
more about that transition?

1436
01:11:32,357 --> 01:11:32,677
Sure.

1437
01:11:32,697 --> 01:11:33,297
Absolutely.

1438
01:11:33,297 --> 01:11:37,787
So I, there was about three or four
years when I was doing consulting work.

1439
01:11:37,837 --> 01:11:43,497
And that was during that time was around
when I started it in 2017, around 2019,

1440
01:11:43,517 --> 01:11:46,997
I think was the year that I full time
just didn't do any consulting work.

1441
01:11:46,997 --> 01:11:48,747
And I was like, I'm just going
to give this a try to see if I

1442
01:11:48,747 --> 01:11:49,817
can build this out as a business.

1443
01:11:50,082 --> 01:11:51,562
And it got moderately successful.

1444
01:11:51,562 --> 01:11:55,942
I think I got up to around 800
users, I think at, um, at the height.

1445
01:11:55,982 --> 01:11:59,562
And I was only charging 4 a month, which
again, I had no idea what I was doing.

1446
01:11:59,582 --> 01:12:03,652
Um, so too low, but like, you
know, like if you do the math,

1447
01:12:03,662 --> 01:12:05,112
that's like, that's not terrible.

1448
01:12:05,162 --> 01:12:06,842
A lot of people aren't
even able to get to that.

1449
01:12:07,292 --> 01:12:10,552
Like that much, like, but it wasn't even
close to becoming like a full time thing.

1450
01:12:10,572 --> 01:12:13,162
I honestly look, looking back and
all the experience that I have now

1451
01:12:13,162 --> 01:12:14,912
too, I, I, I could have built it out.

1452
01:12:14,912 --> 01:12:17,962
And especially if like Mint shut
down now, like there's so many

1453
01:12:17,972 --> 01:12:20,112
things that I've missed that I, I
probably could have built it out.

1454
01:12:20,112 --> 01:12:22,442
But, um, building a business
is really hard though.

1455
01:12:22,502 --> 01:12:27,242
And it's, I think I realized that I
also had a hard time finding people

1456
01:12:27,272 --> 01:12:29,382
to kind of like work with me on it.

1457
01:12:29,422 --> 01:12:33,892
Like I kept trying to, people were like
sort of interested in helping and which

1458
01:12:33,892 --> 01:12:37,325
made this kind of weird dynamic where
There is like four times when I tried to

1459
01:12:37,325 --> 01:12:41,995
allow contributors, uh, like without pay,
they just wanted to help, but without,

1460
01:12:42,155 --> 01:12:43,395
without paying them, it's just weird.

1461
01:12:43,425 --> 01:12:45,695
Like, they would do something for
a week and then be gone for two

1462
01:12:45,695 --> 01:12:47,225
weeks and not even like speak to me.

1463
01:12:47,225 --> 01:12:48,355
Like, I can't operate like that.

1464
01:12:48,475 --> 01:12:49,695
I'm operating a business here.

1465
01:12:49,785 --> 01:12:51,325
So I try to like, I try to.

1466
01:12:51,645 --> 01:12:56,275
Think about how I could hire a developer,
but I was so busy with everything else.

1467
01:12:56,275 --> 01:12:58,975
It just felt, I had no
idea how to filter through.

1468
01:12:58,975 --> 01:13:01,415
I don't know, should I use like,
you know, one of those like Upwork

1469
01:13:01,425 --> 01:13:05,195
or other things I had no, this just
seemed like so full of spam and, and

1470
01:13:05,215 --> 01:13:08,585
people who had no idea what they were
doing that it just was overwhelming.

1471
01:13:08,585 --> 01:13:10,055
I never figured out how to hire people.

1472
01:13:10,065 --> 01:13:11,435
And that was, that's my biggest regret.

1473
01:13:11,435 --> 01:13:14,015
Probably it's not figuring out how
to involve people in a way that

1474
01:13:14,015 --> 01:13:15,825
would help share some of the burden.

1475
01:13:15,825 --> 01:13:19,140
And so that, it, Would help it a little
bit, be a little bit more sustainable.

1476
01:13:19,310 --> 01:13:22,280
And I was just, I was ready to work with
people again, work at a company that

1477
01:13:22,280 --> 01:13:25,930
was like, I could grow from grow, how
to like, learn how to lead people and

1478
01:13:25,940 --> 01:13:29,250
learn how to, you know, lead teams and
do like product requirements and kind

1479
01:13:29,250 --> 01:13:30,810
of focus more on just what I'm good at.

1480
01:13:31,150 --> 01:13:35,260
And so ultimately, yeah, I just
wasn't, I wasn't serving Actual well,

1481
01:13:35,260 --> 01:13:37,060
I wasn't serving so that I did that.

1482
01:13:37,060 --> 01:13:38,580
Well, I did do that for a whole year.

1483
01:13:38,710 --> 01:13:42,840
which was, which was good, but
it, it, I wasn't growing enough

1484
01:13:42,850 --> 01:13:43,930
for it to be like a real thing.

1485
01:13:43,930 --> 01:13:48,010
And so I got a little burned out on
trying to make it like a full time thing.

1486
01:13:48,110 --> 01:13:51,120
Um, and so I was talking to a
friend who referred me to Stripe.

1487
01:13:51,120 --> 01:13:52,330
And so I got hired at Stripe.

1488
01:13:52,470 --> 01:13:55,320
I did try to work on it kind of
like in the mornings for like an

1489
01:13:55,330 --> 01:13:57,070
hour or two and then work at Stripe.

1490
01:13:57,080 --> 01:14:00,620
But obviously like that, just when I
got more and more involved in Stripe

1491
01:14:00,620 --> 01:14:04,040
and doing like really interesting things
there that kind of absorbed my time.

1492
01:14:04,040 --> 01:14:07,930
And so two years ago, I just was like,
I'm not serving Actual well, I'm not.

1493
01:14:08,030 --> 01:14:08,970
Spending enough time there.

1494
01:14:08,970 --> 01:14:09,870
I'm not serving stripe.

1495
01:14:09,890 --> 01:14:13,780
Well, cause I'm not spending enough time
or like, it was too tiring to like do

1496
01:14:13,780 --> 01:14:15,790
both and it wasn't serving like my family.

1497
01:14:15,820 --> 01:14:17,300
I wasn't serving my
own personal interests.

1498
01:14:17,300 --> 01:14:20,310
Well, either it's just like too
way, like way too overinvested.

1499
01:14:20,780 --> 01:14:23,370
So I decided to, to
not work on it anymore.

1500
01:14:23,370 --> 01:14:25,940
And so the choices was either sell
it or open source it, or just.

1501
01:14:26,160 --> 01:14:26,730
Shut it down.

1502
01:14:26,810 --> 01:14:28,440
I investigated selling it a little bit.

1503
01:14:28,440 --> 01:14:31,820
And this is like, again, some of the
downside of, of doing such a novel app is

1504
01:14:31,820 --> 01:14:33,450
that, um, I talked to a couple of people.

1505
01:14:33,450 --> 01:14:36,360
It's just so clear that nobody
had not much interest in it.

1506
01:14:36,370 --> 01:14:39,920
One, one, it just wasn't making enough
money, but two, like even starting

1507
01:14:39,920 --> 01:14:43,690
to explain how it worked, that
just, it's not a, you know, People

1508
01:14:43,690 --> 01:14:44,690
aren't interested in buying that.

1509
01:14:44,690 --> 01:14:47,840
Like they're interested in buying
like the super run of the mill apps

1510
01:14:47,850 --> 01:14:51,800
that they, that they can take and
grow from a thousand to 10, 000 users.

1511
01:14:51,930 --> 01:14:53,400
So this app was not that app at all.

1512
01:14:53,430 --> 01:14:56,770
So selling it was not in the cards
and with the money it was making, it

1513
01:14:56,770 --> 01:14:57,780
wasn't going to sell for that much.

1514
01:14:57,780 --> 01:15:00,210
So I decided to open source it and
it was a decent amount of work.

1515
01:15:00,220 --> 01:15:02,210
It was decent amount of work
for me for like the year after

1516
01:15:02,210 --> 01:15:03,410
that too, to help transition.

1517
01:15:03,840 --> 01:15:04,350
Doing it.

1518
01:15:04,350 --> 01:15:06,450
And sometimes I was like,
should I have just shut it down?

1519
01:15:06,620 --> 01:15:09,160
But I'm really glad I did because
now it's running entirely on its own.

1520
01:15:09,170 --> 01:15:12,010
Like I, I'm, I'm not even
on, on the discord anymore,

1521
01:15:12,010 --> 01:15:12,970
which I probably should be.

1522
01:15:13,040 --> 01:15:16,190
Honestly, I was starting to get
involved in it again in January.

1523
01:15:16,190 --> 01:15:18,870
And then like a work thing
happened and totally absorbed

1524
01:15:18,870 --> 01:15:20,060
my time for the last two months.

1525
01:15:20,280 --> 01:15:23,410
So I will, I want to start interacting
with the community again, but there

1526
01:15:23,450 --> 01:15:26,000
there's, they seem to be, they
seem to be running it really well.

1527
01:15:26,080 --> 01:15:27,950
And so it's, it's been great.

1528
01:15:27,960 --> 01:15:28,420
They've.

1529
01:15:28,540 --> 01:15:30,880
They've been taking it and
like fixing a lot of bugs.

1530
01:15:30,880 --> 01:15:35,190
And like it, there's definitely like a
power of, of, of open source at work.

1531
01:15:35,190 --> 01:15:36,470
It's, it's messier.

1532
01:15:36,470 --> 01:15:39,390
I honestly think that there's things,
parts of the app that don't look as good.

1533
01:15:39,470 --> 01:15:44,010
They're, they're not as clean, not as,
not as polished, not as thoughtful, but.

1534
01:15:44,355 --> 01:15:46,915
There's a lot of added functionality,
a lot of stuff that is good and a

1535
01:15:46,915 --> 01:15:48,435
lot of stuff that is improved too.

1536
01:15:48,535 --> 01:15:51,215
Thank you so much for, for
sharing this entire story.

1537
01:15:51,275 --> 01:15:55,725
I mean, I have massive respect to
you of how you've like navigated

1538
01:15:55,765 --> 01:15:59,505
this entire journey and takes a lot
of, I think there's not just on a

1539
01:15:59,515 --> 01:16:01,475
technological level, there's like doubt.

1540
01:16:01,495 --> 01:16:03,145
It's just like, is this possible?

1541
01:16:03,155 --> 01:16:04,145
Does this make sense?

1542
01:16:04,175 --> 01:16:07,515
But you're like building a
product, you're building a company.

1543
01:16:07,755 --> 01:16:11,755
So there's a lot of uncertainty if you're,
if you're not in a full time job where

1544
01:16:11,755 --> 01:16:15,305
someone else Takes care of the things
and you have your responsibilities.

1545
01:16:15,325 --> 01:16:17,235
Here's like, everything is uncertain.

1546
01:16:17,655 --> 01:16:21,645
And so navigating that while
also like having a family and

1547
01:16:21,665 --> 01:16:24,785
other responsibilities, that's,
that's a lot of commitment.

1548
01:16:25,085 --> 01:16:29,455
So I have a lot of respect for
like, Not just the decisions you've

1549
01:16:29,455 --> 01:16:32,768
made, but like also , how thoughtful
you went about those transitions.

1550
01:16:32,778 --> 01:16:37,138
Like how much you, like you said, like
you owe the community, like an update.

1551
01:16:37,138 --> 01:16:39,938
I don't think you owe the community
anything you've given the community

1552
01:16:39,938 --> 01:16:44,718
a lot, but I think all of your
contributions are really well received.

1553
01:16:44,728 --> 01:16:47,408
And, and I think highly
valued by a lot of people.

1554
01:16:47,818 --> 01:16:52,848
And so I think even if Actual now did
not work out as like that, uh, That

1555
01:16:52,858 --> 01:16:55,408
billion dollar startup by itself.

1556
01:16:55,458 --> 01:16:59,488
Uh, I think it will actually influence
a lot of those and who knows, maybe

1557
01:16:59,498 --> 01:17:04,018
you take another stab at the same thing
at another thing in a similar way.

1558
01:17:04,348 --> 01:17:08,668
And I can't wait to hear sort of what
you'll be innovating on at that point.

1559
01:17:08,738 --> 01:17:11,948
You've mentioned that every four
years of like, you're, you're staring

1560
01:17:11,958 --> 01:17:14,798
at a problem where you feel like,
ah, there has to be a better way.

1561
01:17:15,088 --> 01:17:19,558
So can't wait to see which sort of absurd
things you might build in the future.

1562
01:17:20,128 --> 01:17:24,418
So you mentioned that right now
you're at Stripe and I think at Stripe

1563
01:17:24,428 --> 01:17:29,978
you're working on design systems
and like more UI related things.

1564
01:17:30,028 --> 01:17:33,408
So which sort of things are
you, are you working at Stripe?

1565
01:17:33,568 --> 01:17:34,018
Yes.

1566
01:17:34,028 --> 01:17:36,298
So I work on our design system.

1567
01:17:36,328 --> 01:17:37,108
It's called SAIL.

1568
01:17:37,378 --> 01:17:41,558
And so SAIL recently, it used to just be
a single team called the design system

1569
01:17:41,588 --> 01:17:45,068
team, but now there's been changed into
like a larger org, which is really cool.

1570
01:17:45,228 --> 01:17:49,538
Exciting in some ways, it kind of, you
know, messed up our team a little bit.

1571
01:17:49,548 --> 01:17:52,068
Some, some people got moved
around, which was, we had to kind

1572
01:17:52,068 --> 01:17:53,728
of work through that, but we did.

1573
01:17:53,738 --> 01:17:56,178
And like, there's a wholesale
org now, which is great.

1574
01:17:56,278 --> 01:17:58,318
So it's like a, a bigger investment.

1575
01:17:58,388 --> 01:18:00,098
SAIL itself is becoming this thing.

1576
01:18:00,098 --> 01:18:02,458
That's actually more than
just a UI design system.

1577
01:18:02,458 --> 01:18:04,508
We're also starting to integrate other.

1578
01:18:04,518 --> 01:18:08,018
Concerns that you always hit when you
start doing front-end work, which is like,

1579
01:18:08,018 --> 01:18:09,398
how do you internationalize something?

1580
01:18:09,408 --> 01:18:11,408
How do you, how do you do observability?

1581
01:18:11,478 --> 01:18:15,828
How do you do like GraphQL queries or how
do you just fetch data like in general?

1582
01:18:15,828 --> 01:18:18,468
And so SAIL is becoming like
a bigger platform overall.

1583
01:18:18,548 --> 01:18:21,868
I am focusing more on the, on the
UI design system part, but I'm

1584
01:18:21,868 --> 01:18:24,648
also collaborating a lot with the
others as well to sort of integrate

1585
01:18:24,648 --> 01:18:26,078
this into a cohesive platform.

1586
01:18:26,248 --> 01:18:27,578
Internally, we have a, a.

1587
01:18:27,668 --> 01:18:30,678
A whole system for like
variants and tokens.

1588
01:18:30,688 --> 01:18:33,378
And we have a, like a
view and a, and a CSS API.

1589
01:18:33,468 --> 01:18:36,098
And it's something that I don't
think we've talked a ton about.

1590
01:18:36,098 --> 01:18:38,568
I think we really should talk
about it more, um, openly.

1591
01:18:38,878 --> 01:18:40,918
And cause it's, it's, it's pretty neat.

1592
01:18:40,978 --> 01:18:42,008
We try to.

1593
01:18:42,328 --> 01:18:44,258
Use third party libraries when we can.

1594
01:18:44,258 --> 01:18:46,138
We actually leverage a lot of React Aria.

1595
01:18:46,178 --> 01:18:48,298
So a lot of our components
use, use React Aria.

1596
01:18:48,388 --> 01:18:48,738
Yeah.

1597
01:18:48,918 --> 01:18:51,908
We actually use the, also the like
lower level hooks API, which is like in

1598
01:18:51,908 --> 01:18:55,578
some ways really cumbersome to use, but
like, that's kind of intentional, like,

1599
01:18:55,578 --> 01:18:59,148
because it gives you like direct access
to the entire way that the things work.

1600
01:18:59,478 --> 01:19:02,528
They now have a higher level API, like
the React Aria components API, which

1601
01:19:02,528 --> 01:19:03,848
is all like really, really great.

1602
01:19:03,948 --> 01:19:07,168
But we do things with React Aria
that I don't think anybody else does.

1603
01:19:07,168 --> 01:19:10,578
I think that like, there's like
classes that we import and use.

1604
01:19:11,053 --> 01:19:13,433
That, like, we've talked to the
ReactARIA team and they're like,

1605
01:19:13,563 --> 01:19:15,973
nobody else has imported that class
and, like, used that directly.

1606
01:19:16,243 --> 01:19:19,623
There's, like, a list, like, a list
collection class that is, that it uses.

1607
01:19:19,673 --> 01:19:21,933
And we import that and create
our own collection system.

1608
01:19:22,253 --> 01:19:25,543
And, like, cause, like, Stripe
has pretty specific needs, right?

1609
01:19:25,543 --> 01:19:28,343
And so what I love about ReactARIA
is that it's taken this, like,

1610
01:19:28,393 --> 01:19:31,703
unabashed approach to being cumbersome.

1611
01:19:31,704 --> 01:19:33,113
And it's, like, kind of intentional.

1612
01:19:33,123 --> 01:19:36,333
Like, if you look at the docs for some
of the hooks, like, it's a lot of code

1613
01:19:36,343 --> 01:19:43,003
to get a, Menu working, but because it
is that openly exposed, you have like

1614
01:19:43,133 --> 01:19:45,033
total control over how everything works.

1615
01:19:45,163 --> 01:19:47,863
So it's really allowed us to go
in and really, really wire things

1616
01:19:47,863 --> 01:19:48,923
up the way that we want to.

1617
01:19:49,133 --> 01:19:49,963
So yeah, it's, it's great.

1618
01:19:49,963 --> 01:19:51,123
It's a fantastic library.

1619
01:19:51,123 --> 01:19:52,073
It's super, super good.

1620
01:19:52,393 --> 01:19:55,278
Um, lots of good accessibility
that, that you get from that.

1621
01:19:55,428 --> 01:19:57,548
But that's our component system, right?

1622
01:19:57,808 --> 01:20:01,078
We still have a lower level,
like view and CSS APIs.

1623
01:20:01,078 --> 01:20:04,748
And so how you do CSS, how you do tokens,
how you, how you do variants, how do

1624
01:20:04,748 --> 01:20:09,648
you make sure that you, when you, uh,
create a new component, that everything

1625
01:20:09,648 --> 01:20:15,103
is wired through such that like, The, um,
like if you pass styles to it, but you

1626
01:20:15,113 --> 01:20:19,513
also internally want to style the, the
same div, like, like the same top level

1627
01:20:19,513 --> 01:20:23,543
div that you take the styles from your
props and you apply styles to that div.

1628
01:20:23,763 --> 01:20:26,373
But like in that component,
you also want to apply styles.

1629
01:20:27,218 --> 01:20:27,678
Do that div.

1630
01:20:27,708 --> 01:20:29,468
How do you, how do you
combine those styles?

1631
01:20:29,658 --> 01:20:33,878
I guess you could like spread on the
styles that you get from the props, right?

1632
01:20:33,948 --> 01:20:35,928
And then spread on your
own styles as well.

1633
01:20:35,928 --> 01:20:38,458
But then you get into like
precedence problems, like which

1634
01:20:38,468 --> 01:20:39,838
order do you spread those on?

1635
01:20:40,178 --> 01:20:44,418
So we have a whole way of like creating
a component, a whole pattern for

1636
01:20:44,708 --> 01:20:46,238
taking the props that you get and.

1637
01:20:46,508 --> 01:20:50,418
And spreading, and like, spreading
those props onto some internal element.

1638
01:20:50,838 --> 01:20:54,638
And we have a very strict precedence
order for how styles and how

1639
01:20:54,658 --> 01:20:58,528
variants and, and how other things
like that get merged together, um,

1640
01:20:58,528 --> 01:20:59,948
in a way that's a very intuitive.

1641
01:21:00,218 --> 01:21:01,778
Um, and it's all very, very consistent.

1642
01:21:01,788 --> 01:21:03,838
So we have like a, an
API called create view.

1643
01:21:04,068 --> 01:21:07,228
So that's how you create a new component
that interacts with our whole system.

1644
01:21:07,438 --> 01:21:11,058
When you call create view, you get back
an exact same React component, just like

1645
01:21:11,058 --> 01:21:12,528
you would call it forward ref, right?

1646
01:21:12,538 --> 01:21:16,528
Like it's, it's taking a component, but
it's giving it additional capabilities.

1647
01:21:16,538 --> 01:21:18,668
And the, the, the two things, like
there's a couple of things that

1648
01:21:18,668 --> 01:21:20,518
it gets, it gets like a CSS prop.

1649
01:21:20,878 --> 01:21:25,728
So we, we pass CSS by, by saying
button CSS equals, and then an object.

1650
01:21:26,028 --> 01:21:29,268
And in that object, we have a
whole like little system that

1651
01:21:29,268 --> 01:21:30,408
is wired up with our tokens.

1652
01:21:30,409 --> 01:21:33,818
And so you can say margin
is, you know, small.

1653
01:21:34,073 --> 01:21:35,573
And small gets resolved to a token.

1654
01:21:35,583 --> 01:21:39,213
We also do like things where you
can, um, say like small is eight

1655
01:21:39,243 --> 01:21:43,313
pixels and we have an ESLint rule
that automatically knows that for

1656
01:21:43,313 --> 01:21:48,093
the current theme and for the current
system wired up, small is eight pixels.

1657
01:21:48,303 --> 01:21:52,503
So you've hard coded a token here and it
automatically actually fixes that for you.

1658
01:21:52,503 --> 01:21:55,273
So it's changes eight pixels to small.

1659
01:21:55,373 --> 01:21:58,373
So we, we try to let developers build.

1660
01:21:58,568 --> 01:21:59,598
Like not get in their way.

1661
01:21:59,598 --> 01:22:02,598
Basically, we really want them to
sort of, we have the same, like fall

1662
01:22:02,598 --> 01:22:05,948
into the pit of success where you
should go off and build a product

1663
01:22:05,978 --> 01:22:07,128
the way that you want to build it.

1664
01:22:07,408 --> 01:22:11,978
And sale should meet you where you
are there and like, let you build

1665
01:22:11,978 --> 01:22:15,498
components that end up getting wired
up the way that we want them wired up.

1666
01:22:15,808 --> 01:22:18,608
But just because that's the
The way that feels natural.

1667
01:22:18,618 --> 01:22:22,278
Like you shouldn't have to feel like
you have to like go against our system

1668
01:22:22,278 --> 01:22:26,278
and like begrudgingly use it to like
be conformed to Stripe's design system.

1669
01:22:26,278 --> 01:22:26,538
Right.

1670
01:22:26,718 --> 01:22:28,298
We want you to be like happy using it.

1671
01:22:28,498 --> 01:22:32,048
And so a lot of our work goes into
like making the APIs feel good.

1672
01:22:32,198 --> 01:22:32,428
Yeah.

1673
01:22:32,428 --> 01:22:34,558
There, there's a lot of stuff here
that I think we should talk about more.

1674
01:22:34,558 --> 01:22:37,808
Like we have our own variant system
and our whole, like how tokens are

1675
01:22:37,808 --> 01:22:39,188
wired up is really, really interesting.

1676
01:22:39,208 --> 01:22:41,028
Cause one thing about Stripe.

1677
01:22:41,858 --> 01:22:43,878
It's a very, very broad company.

1678
01:22:43,878 --> 01:22:45,218
We have a lot of different products.

1679
01:22:45,218 --> 01:22:46,078
We have the dashboard.

1680
01:22:46,078 --> 01:22:46,958
We have connect.

1681
01:22:47,128 --> 01:22:50,498
We have like people taking
pieces of functionality and

1682
01:22:50,658 --> 01:22:52,078
embedding it in their own website.

1683
01:22:52,098 --> 01:22:54,978
And then they're theming that
content, but it's like exactly the

1684
01:22:54,978 --> 01:22:57,448
same little widget that you would
take from the dashboard, right?

1685
01:22:57,458 --> 01:22:59,588
It's like a payments list that
you would take from the dashboard

1686
01:22:59,588 --> 01:23:01,428
and embed into your own page.

1687
01:23:01,628 --> 01:23:03,318
And then we also have like custom hosted.

1688
01:23:04,053 --> 01:23:06,593
Invoices and a bunch of other
little like third party things.

1689
01:23:06,593 --> 01:23:09,283
Then like checkout and like elements
as a whole, another thing as well.

1690
01:23:09,503 --> 01:23:12,713
We're trying to bring all of that
together into us and to use the same

1691
01:23:12,713 --> 01:23:17,723
design system that can be themed and like
leverage and customized for each surface.

1692
01:23:17,863 --> 01:23:19,823
And so it's really hard problems.

1693
01:23:20,078 --> 01:23:22,708
Like honestly, and it's, it's, it's
a lot of work, but it's really fun.

1694
01:23:22,868 --> 01:23:24,468
That, that sounds fascinating.

1695
01:23:24,518 --> 01:23:29,448
Uh, I feel like I want to learn a lot
more about how to do design systems.

1696
01:23:29,498 --> 01:23:33,408
I should also educate myself more of
like, should I use a design system

1697
01:23:33,418 --> 01:23:35,028
even for a smaller scale products?

1698
01:23:35,038 --> 01:23:37,958
For example, would you now
looking back, would you've used

1699
01:23:37,958 --> 01:23:39,528
a design system for Actual?

1700
01:23:39,808 --> 01:23:43,578
Is this like, rather like, does it
solve an organizational problem or does

1701
01:23:43,578 --> 01:23:47,638
it really help even on a smaller scale
when fewer engineers are involved?

1702
01:23:48,003 --> 01:23:52,473
But I most certainly love the design
principle of like the pit of success

1703
01:23:52,533 --> 01:23:54,353
that can't go wrong with that.

1704
01:23:54,633 --> 01:23:55,403
Yeah, it's great.

1705
01:23:55,413 --> 01:23:58,503
It's, it's been a good, like a
good way to frame things for sure.

1706
01:23:58,703 --> 01:24:03,253
So you're building with React,
uh, Stripe it seems, and you've

1707
01:24:03,263 --> 01:24:05,533
also used React for Actual.

1708
01:24:05,593 --> 01:24:09,633
React has changed quite a bit over
the years, so I'm curious to hear

1709
01:24:09,633 --> 01:24:13,983
whether you have any opinions on
where React has gone over the last

1710
01:24:13,983 --> 01:24:15,213
few years and where it's going.

1711
01:24:15,518 --> 01:24:18,688
I don't have super strong opinions
only because I feel like I

1712
01:24:18,688 --> 01:24:20,318
can't back them up right now.

1713
01:24:20,338 --> 01:24:23,148
I haven't given it the time to
sit down and really like write

1714
01:24:23,148 --> 01:24:27,108
out like a thorough argument for
why I should feel a certain way.

1715
01:24:27,138 --> 01:24:29,638
I think it's been hard for just
in general, like everybody.

1716
01:24:29,698 --> 01:24:32,918
I respect the team in a
lot of ways because I think

1717
01:24:32,918 --> 01:24:34,048
they have a high bar, right?

1718
01:24:34,078 --> 01:24:35,148
Unlike many other teams.

1719
01:24:35,528 --> 01:24:38,178
Developers, generally speaking,
I feel like we don't have a high

1720
01:24:38,178 --> 01:24:40,938
enough bar and I feel like the
React team has a high bar and like,

1721
01:24:41,098 --> 01:24:43,468
ultimately, I do respect them for that.

1722
01:24:43,528 --> 01:24:47,458
I do think there's, there's been
times when it's just like very heavy

1723
01:24:47,458 --> 01:24:50,888
investment and very complicated
things when it feels like there's

1724
01:24:50,928 --> 01:24:52,838
lower hanging fruit, which are like.

1725
01:24:53,433 --> 01:24:54,823
Man, this really sucks to have to do this.

1726
01:24:54,833 --> 01:24:58,233
Like every single time sucks to have
to use forward ref every single time.

1727
01:24:58,233 --> 01:24:59,563
Why can't just ref be a freaking prop?

1728
01:24:59,823 --> 01:25:02,443
And like, I know that there's like
backwards compatibility problems and

1729
01:25:02,443 --> 01:25:05,953
like all sorts of reasons, but it, when
like two years are spent on this really

1730
01:25:05,953 --> 01:25:09,213
complicated thing, and like, we're still
hitting these problems on a day to day

1731
01:25:09,213 --> 01:25:13,097
development thing, sometimes there's a
little dissonance there that can be a

1732
01:25:13,097 --> 01:25:16,703
little bit like, ah, you know, we also
don't pay for react, it's almost like.

1733
01:25:17,113 --> 01:25:18,543
I don't know, like we're
getting it for free.

1734
01:25:18,593 --> 01:25:20,563
It's up to them to ultimately decide.

1735
01:25:20,573 --> 01:25:22,583
We're not, they're not
forcing us to use React.

1736
01:25:22,953 --> 01:25:24,823
There's a little bit
of a lock in for sure.

1737
01:25:24,823 --> 01:25:27,793
Like I don't know how Stripe would
possibly move away from React, but

1738
01:25:27,803 --> 01:25:30,813
ultimately like the members of the
team, I respect really, really well.

1739
01:25:30,813 --> 01:25:33,103
And I'm not going to say
anything like super bad about it.

1740
01:25:33,208 --> 01:25:35,518
I think react server components
is really interesting.

1741
01:25:35,658 --> 01:25:38,428
Again, it's like, sure, like the
local-first stuff might be able to

1742
01:25:38,438 --> 01:25:39,678
be cleaned up a little bit with it.

1743
01:25:39,678 --> 01:25:42,588
Like, there's kind of some interesting
things there where like, maybe you

1744
01:25:42,588 --> 01:25:44,658
could like run components on the server.

1745
01:25:44,658 --> 01:25:47,518
Like, even if it's just like a
backend web worker process, but

1746
01:25:47,518 --> 01:25:49,738
the wins are much less for sure.

1747
01:25:49,788 --> 01:25:50,158
Like.

1748
01:25:50,683 --> 01:25:51,603
Everything's already local.

1749
01:25:51,603 --> 01:25:53,373
I don't care if it goes
through a WebSocket message.

1750
01:25:53,373 --> 01:25:55,303
Like I can, like, you
can embed SQL queries.

1751
01:25:55,393 --> 01:25:57,093
Like we're embedding SQL queries already.

1752
01:25:57,093 --> 01:25:58,723
We don't need React
Server components for it.

1753
01:25:58,753 --> 01:26:01,673
So it's less convincing to
me if you're already doing it

1754
01:26:01,683 --> 01:26:02,903
the way that we're doing it.

1755
01:26:02,943 --> 01:26:04,993
But for Stripe, like React
Server components could be

1756
01:26:04,993 --> 01:26:06,253
potentially pretty compelling.

1757
01:26:06,343 --> 01:26:10,398
But it's still, again, overall,
like, Man, I just want support

1758
01:26:10,398 --> 01:26:11,888
for exit animations, right?

1759
01:26:11,888 --> 01:26:15,358
Like I want the ability to not have
to wrap something in animate presence

1760
01:26:15,408 --> 01:26:19,678
to just freaking get something to
like maintain it in a dom while it

1761
01:26:19,798 --> 01:26:21,378
animates out and then unmount it.

1762
01:26:21,388 --> 01:26:23,048
Like React still doesn't
allow you to do that.

1763
01:26:23,068 --> 01:26:24,958
Like exit animations are
a huge pain in the butt.

1764
01:26:25,018 --> 01:26:25,528
Yeah,

1765
01:26:25,728 --> 01:26:26,788
I, I fully agree.

1766
01:26:26,788 --> 01:26:30,748
And I think the React server components,
what you've mentioned, I think the,

1767
01:26:30,758 --> 01:26:34,608
the way from my current perspective,
how it would most meaningfully help

1768
01:26:34,668 --> 01:26:36,498
in a local-first context as well.

1769
01:26:36,818 --> 01:26:41,578
is on the initial upload experience,
since that is sometimes a bit of

1770
01:26:41,578 --> 01:26:45,668
like the cost that you're paying that
you're, you're kind of with local-first,

1771
01:26:45,688 --> 01:26:47,108
you say like no more spinners.

1772
01:26:47,278 --> 01:26:51,888
Well, sort of, since you have like one
front loaded, typically larger spinner.

1773
01:26:52,188 --> 01:26:55,738
And I think this can be addressed also
with react server components, where

1774
01:26:55,738 --> 01:26:57,618
you get sort of an hybrid approach.

1775
01:26:57,948 --> 01:27:03,208
Where you get much more quickly reactive
initial version of the app that then

1776
01:27:03,258 --> 01:27:05,118
upgrades itself to be local-first.

1777
01:27:05,138 --> 01:27:09,828
So that, that's my take on react
server components, but the other pain

1778
01:27:09,828 --> 01:27:14,088
point that you've pointed at with
like exit animations, et cetera, and

1779
01:27:14,088 --> 01:27:18,188
where this is kind of where we are
now paying the cost for virtual DOM,

1780
01:27:18,198 --> 01:27:20,458
which constantly updates everything.

1781
01:27:20,508 --> 01:27:24,598
I've had actually some, some
interesting results now with just

1782
01:27:24,598 --> 01:27:26,028
using web components for that.

1783
01:27:26,458 --> 01:27:30,298
Where it's a bit funny since like, in
some regards, it feels like going a bit

1784
01:27:30,308 --> 01:27:36,738
back in history where your DOM is more
static in a way, but this way you actually

1785
01:27:36,738 --> 01:27:41,358
also can think a little bit differently
about like, for example, animations.

1786
01:27:41,438 --> 01:27:47,718
So, and this is where I feel also on
the general theme of learning more from

1787
01:27:47,748 --> 01:27:52,708
other native programming environments,
other native platforms, where for

1788
01:27:52,708 --> 01:27:54,858
example, on iOS, when you have like.

1789
01:27:55,088 --> 01:27:58,178
Things like a collection view or
a table view controller where you

1790
01:27:58,178 --> 01:28:02,948
have those cells and they're like,
they don't constantly cycle out.

1791
01:28:03,008 --> 01:28:07,258
They cycle out if you're like, if they
leave your view, but based on like

1792
01:28:07,268 --> 01:28:10,298
entering the view, leaving the view,
this is how we can do animations.

1793
01:28:10,708 --> 01:28:15,373
And with an approach like web
components, you can actually do that by.

1794
01:28:15,523 --> 01:28:20,763
Using the native aspects of the web, like
native to the platform web, much more.

1795
01:28:20,763 --> 01:28:23,943
And I feel we're now fighting
a bit of an uphill battle to

1796
01:28:23,973 --> 01:28:25,593
get those benefits from React.

1797
01:28:25,753 --> 01:28:26,133
Agreed.

1798
01:28:26,173 --> 01:28:26,563
Yeah.

1799
01:28:26,653 --> 01:28:27,443
It's interesting.

1800
01:28:27,493 --> 01:28:29,173
There's a decent escape
hatch there, right?

1801
01:28:29,173 --> 01:28:33,023
Where you can mount a div and then in
an effect, like you can do whatever you

1802
01:28:33,023 --> 01:28:35,683
want there, which is, which is nice,
especially for a company like Stripe,

1803
01:28:35,693 --> 01:28:37,733
because we can sort of like opt out when.

1804
01:28:38,113 --> 01:28:42,143
Needed, but something does like, there are
certain things that are like systematic.

1805
01:28:42,143 --> 01:28:44,823
And, you know, as something, somebody
who works, works on a design system,

1806
01:28:44,823 --> 01:28:47,903
seeing like animations are not something
that you can just like opt out for.

1807
01:28:48,193 --> 01:28:48,523
Right.

1808
01:28:48,563 --> 01:28:51,583
That's something that you apply on
like almost every single, I don't

1809
01:28:51,583 --> 01:28:55,473
know, like a 10th of the elements
on the page, which is a lot.

1810
01:28:55,968 --> 01:28:56,948
have animations.

1811
01:28:56,958 --> 01:28:58,788
You can't just like opt
out when you want to.

1812
01:28:58,928 --> 01:29:00,078
So it's, it's hard.

1813
01:29:00,118 --> 01:29:00,358
Yeah.

1814
01:29:00,368 --> 01:29:02,458
I would love to see
some improvements there.

1815
01:29:02,508 --> 01:29:05,778
Just recently on my blog, I have quick
and dirty type thing, but like I don't

1816
01:29:05,778 --> 01:29:08,368
use re I use react, I use, um, Remix.

1817
01:29:08,368 --> 01:29:10,298
And so I use reactor on the backend,
but I don't use it on the front.

1818
01:29:10,298 --> 01:29:11,918
It just feels so fast and nice.

1819
01:29:11,928 --> 01:29:12,518
It's just a.

1820
01:29:12,748 --> 01:29:16,618
Just don't do it just content, but like I
am starting to add more interactive parts.

1821
01:29:16,638 --> 01:29:19,748
And so I'm sort of playing around with
like, well, okay, what do I do here?

1822
01:29:20,198 --> 01:29:22,248
Do I need to load in the full react?

1823
01:29:22,248 --> 01:29:25,088
Maybe this is where like partial
hydration could be interesting.

1824
01:29:25,148 --> 01:29:26,988
Uh, but like, I just don't, it's my blog.

1825
01:29:26,998 --> 01:29:30,428
I don't need partial hydration on
my frigging blog, but I do want

1826
01:29:30,438 --> 01:29:33,488
to be able to like, um, there's
like, I have like live interactive

1827
01:29:33,488 --> 01:29:35,118
demos and I want to be able to.

1828
01:29:35,373 --> 01:29:36,823
Have a view source button.

1829
01:29:36,913 --> 01:29:40,293
And when you click that, it opens up
this, like, it basically like zooms

1830
01:29:40,293 --> 01:29:44,863
into the interactive demo to where it
shows the demo, like, like the demo is

1831
01:29:44,863 --> 01:29:48,483
still running, but it's running in a
dialogue and the demo is run on the left.

1832
01:29:48,483 --> 01:29:51,473
And then on the right is the code
for the demo and then all sorts

1833
01:29:51,473 --> 01:29:53,073
of other controls for the demo.

1834
01:29:53,083 --> 01:29:57,263
And the way that I implemented that
was I literally transport that real

1835
01:29:57,263 --> 01:30:01,473
Dom node, I take the Dom node instance
itself from the, like the whole

1836
01:30:01,473 --> 01:30:04,033
demo Dom node, and I, I move it.

1837
01:30:04,493 --> 01:30:08,183
I create the dialogue and I put
the code on the right, and I moved

1838
01:30:08,183 --> 01:30:10,283
the DOM node into the dialogue.

1839
01:30:10,513 --> 01:30:12,223
And the demo just keeps
on running just fine.

1840
01:30:12,233 --> 01:30:14,293
Like it, it moves into the dialogue.

1841
01:30:14,353 --> 01:30:16,233
And I did that in like 10 lines of code.

1842
01:30:16,503 --> 01:30:20,213
And there is a certain amount of
like, Oh man, it feels so good to be,

1843
01:30:20,263 --> 01:30:21,863
yeah, I just have access to like this.

1844
01:30:22,323 --> 01:30:26,673
Dangerous, dangerous API APIs, but like
I, to do this in react would have felt

1845
01:30:26,683 --> 01:30:29,953
really backwards, like bending backwards
and doing all sorts of weird things.

1846
01:30:30,213 --> 01:30:32,393
And then you can like, and you
close it and it moves it back and

1847
01:30:32,393 --> 01:30:33,463
it just keeps on running just fine.

1848
01:30:33,493 --> 01:30:33,773
Right.

1849
01:30:33,833 --> 01:30:36,743
And I, I don't know, to, to, to think
about it in the react mental model,

1850
01:30:36,743 --> 01:30:37,553
I would have been like, Oh, okay.

1851
01:30:37,553 --> 01:30:40,123
I need to create like a portal,
create a bunch of components and

1852
01:30:40,123 --> 01:30:44,203
like wire them all, I don't like
just saying like dom node, remove

1853
01:30:44,203 --> 01:30:46,183
node, and then like demo dialogue.

1854
01:30:46,363 --> 01:30:46,863
append.

1855
01:30:47,033 --> 01:30:48,873
Uh, I completely agree.

1856
01:30:48,953 --> 01:30:51,023
I feel like we've been now over the last.

1857
01:30:51,608 --> 01:30:53,598
React has been now around for 10 years.

1858
01:30:53,598 --> 01:30:59,288
I feel like over the last 10 years,
we've been really leaning to use the

1859
01:30:59,288 --> 01:31:01,878
React hammer for like every little thing.

1860
01:31:01,928 --> 01:31:06,478
And I think that brought a whole lot
of benefits to us, but I think we

1861
01:31:06,528 --> 01:31:08,268
don't question it really anymore.

1862
01:31:08,278 --> 01:31:11,338
And like, we, I think this is
like the only way to go about it.

1863
01:31:11,768 --> 01:31:14,878
And the web has since then really evolved.

1864
01:31:14,898 --> 01:31:17,838
Like we've gotten a lot more,
the primitives in the web

1865
01:31:18,138 --> 01:31:20,408
itself have gotten a lot better.

1866
01:31:20,798 --> 01:31:25,028
And so I think web components
have also come quite a long way.

1867
01:31:25,028 --> 01:31:29,548
They haven't seen as much investment
in terms of building a nice ecosystem

1868
01:31:29,548 --> 01:31:34,788
around it, but I think it's the closest
we got to a native feel in the web.

1869
01:31:35,223 --> 01:31:40,253
And, uh, I think sometimes you'd be
surprised how simple things can be if you

1870
01:31:40,253 --> 01:31:42,413
embrace those primitives more directly.

1871
01:31:42,613 --> 01:31:45,893
One little anecdote I want to share
there, a friend of mine, Cheng Lu,

1872
01:31:45,903 --> 01:31:50,933
who's I think also worked on the React
team for a while, he did a couple of

1873
01:31:50,943 --> 01:31:55,073
really fascinating demos and I think he
launched some of them on, on Hacker News.

1874
01:31:55,423 --> 01:32:00,213
Where he, for example, built a photo
gallery and a Hacker News clone as well.

1875
01:32:00,708 --> 01:32:05,778
And so when you try those apps, those are
the smoothest animations and the smoothest

1876
01:32:05,778 --> 01:32:07,888
feel you've ever seen on the web.

1877
01:32:07,948 --> 01:32:10,028
And you kind of feel like,
what is going on here?

1878
01:32:10,028 --> 01:32:14,268
Is this like rendered to web
GPU using Wasm or something?

1879
01:32:14,558 --> 01:32:18,158
And turns out this is all
just normal DOM, normal CSS.

1880
01:32:18,208 --> 01:32:21,828
He's, he's built this like
very simple, elegant system.

1881
01:32:21,828 --> 01:32:26,098
I think the photo gallery is like in
a hundred or 200 lines of JavaScript,

1882
01:32:26,118 --> 01:32:27,828
like no dependencies, nothing.

1883
01:32:28,283 --> 01:32:30,143
And this feels like a total native app.

1884
01:32:30,143 --> 01:32:34,123
So this really reminded me of like
how capable the web really is.

1885
01:32:34,413 --> 01:32:38,903
If we use it directly without
layering too many things in between.

1886
01:32:39,023 --> 01:32:40,923
I have high hopes for, for the web.

1887
01:32:41,083 --> 01:32:43,293
If we set up, set ourselves a high bar.

1888
01:32:43,413 --> 01:32:43,813
Same.

1889
01:32:43,833 --> 01:32:44,093
Yeah.

1890
01:32:44,093 --> 01:32:47,833
And I think React needs to figure out
how to like move with the web without

1891
01:32:48,353 --> 01:32:51,343
breaking backwards compatibility,
find the right trade off there.

1892
01:32:51,423 --> 01:32:54,333
Like, I guess even the like web
component support type stuff in that

1893
01:32:54,333 --> 01:32:55,983
has been like super long in the making.

1894
01:32:56,043 --> 01:32:57,083
So I think that's something that.

1895
01:32:57,303 --> 01:32:58,493
Could be improved, but

1896
01:32:58,603 --> 01:33:02,853
hey, James, you've been very generous
with your time and walking us

1897
01:33:02,853 --> 01:33:06,933
through like your entire journey of
the, the various chapters of Actual

1898
01:33:06,933 --> 01:33:11,263
from Electra and the mobile apps,
the web, absurd sequel, et cetera.

1899
01:33:11,313 --> 01:33:13,793
So thank you so much for your time.

1900
01:33:14,203 --> 01:33:17,543
If there's anything you want to share
with the audience here, any sort

1901
01:33:17,543 --> 01:33:18,883
of shout out, now's a good time.

1902
01:33:19,323 --> 01:33:19,663
Uh, cool.

1903
01:33:19,663 --> 01:33:19,913
Yeah.

1904
01:33:19,913 --> 01:33:20,923
Thank you for having me on.

1905
01:33:20,933 --> 01:33:21,883
This was amazing.

1906
01:33:21,953 --> 01:33:25,613
I mean, honestly, shout out to
Martin Kleppmann, PVH, Peter.

1907
01:33:25,673 --> 01:33:28,433
I found them through
their local-first essay.

1908
01:33:28,433 --> 01:33:33,033
And I, I came and give like a, gave like
a workshop at their research studio and

1909
01:33:33,033 --> 01:33:34,413
like talk to them a bunch since then.

1910
01:33:34,413 --> 01:33:36,723
And they've always been very encouraging
throughout the whole process.

1911
01:33:36,773 --> 01:33:38,953
So super fun to talk to
them about all this stuff.

1912
01:33:38,954 --> 01:33:39,523
Awesome.

1913
01:33:39,523 --> 01:33:42,263
Thank you so much for your time
and coming on the show today.

1914
01:33:42,463 --> 01:33:43,223
Thank you so much.

1915
01:33:43,517 --> 01:33:46,057
Thank you for listening to
the localfirst.fm podcast.

1916
01:33:46,297 --> 01:33:50,027
If you've enjoyed this episode and haven't
done so already, please subscribe and

1917
01:33:50,027 --> 01:33:51,787
leave a review wherever you're listening.

1918
01:33:52,157 --> 01:33:53,587
Please also tell your friends about it.

1919
01:33:53,597 --> 01:33:57,167
If you think they could be interested
in local-first, if you have feedback,

1920
01:33:57,207 --> 01:34:00,947
questions or ideas for the podcast,
please get in touch via hello at

1921
01:34:00,947 --> 01:34:06,727
localfirst.fm or use the feedback form on
our website, special thanks to Expo and

1922
01:34:06,727 --> 01:34:08,777
Crab Nebula for supporting this podcast.

1923
01:34:09,177 --> 01:34:09,937
See you next time.