isn't it really complicated to build web apps?
What if it was just way simpler to build an awesome app that was fast, that was
really responsive, that stored all the data people care about, was easier for
you to develop faster, simpler, less ops maintenance stuff The data ownership
stuff just falls out as a consequence.
Could we leverage the power of databases and the power of local-first
architecture to cut through a lot of the complexity of building
modern web apps and make it simpler.
Welcome to the localfirst.fm podcast.
I'm your host, Johannes Schickling, and I'm a web developer, a
startup founder, And love the craft of software engineering.
For the past few years, I've been on a journey to build a modern, high quality
music app using web technologies.
And in doing so, I've been falling down the rabbit hole of local-first software.
This podcast is your invitation to join me on that journey.
In this episode, I'm speaking to Geoffrey Litt, who's currently
a researcher at Ink and Switch.
Geoffrey has worked on many interesting research projects such
as Riffle, Cambria, and Embark.
In our conversation today, we're talking about the ideas and goals
that motivated Riffle, Which include expressing an app as a query, fast
synchronous reactivity, and unifying UI state and app data in a single system.
Before getting started, also a big thank you to Expo and Crab
Nebula for supporting this podcast.
And now my interview with Geoffrey.
Welcome, Geoffrey, to the show.
So excited to have you.
Thanks.
It's great to be here.
I mean, you and I have been working together now for a few years, But let's
rewind time a little bit and maybe start where you and I have first crossed paths.
This was, I think the end of 2020 and you've been just wrapping
up your first project at Ink and Switch, which was about Cambria.
So would you mind, motivating what the project was about?
Yeah.
Absolutely.
So, I'm a researcher that works on a topic called malleable software, where the idea
is, can we let everybody customize their software tools to meet their own personal
needs rather than everybody changing the way they work to match standardized
software tools that other people develop.
Right?
turns out one of the core problems in this space is wrangling data schemas.
So imagine, you know, you're using your favorite to do app, I'm using
my favorite to do app, and we decide we wanna work on a project together.
What are the options?
We either have to pick the same to do app, or if we wanna each keep using
our own favorite to do app, those apps need to have some way of talking to one
another and sharing a data representation.
Right?
it turns out that this is a pretty gnarly problem because those data
representations our 2 apps wanna use might be different in subtle ways.
You know?
Perhaps your app has subtasks and mine doesn't or something like that.
And so the goal of Cambria was to explore this problem and try to figure
out, could we find a way to enable collaboration across diverse tools,
while also not requiring all of our tools to share exactly the same data
schema and all agree on everything.
And It turns out that this is actually sort of a pervasive problem
across local-first software in a bunch of different contexts.
This problem pops up in many shapes and forms.
Even the simple case of, like, having a centralized application written by
one developer in a local-first context schema migrations And upgrades often
become a lot harder than they are in sort of a server based environment.
So, the Cambria project again was about sort of First, just recognizing that
this problem exists everywhere and that it's really holds back progress
in terms of interoperability and schema management and local-first steps.
And then we sort of prototyped a solution to it, which was you can think of it
sort of as a live data translation system where as different tools are
working on shared data, there's a layer underneath the individual tools that
helps translate data formats between all the various tools in such a way that
each app can feel like it's just working with the data representation it wants.
But, actually, under the hood where we have some sort of translation layer
that's, you know, sort of serving as the glue that lets different
versions of the same app collaborate or lets even different apps entirely
collaborate with one another.
Yeah.
I think that's a brilliant motivation.
I remember, like, having read the essay the first time was super well
illustrated and super well motivated.
It sort of almost provoked me into reimplementing a version
of Cambria myself back then.
This is how much it's it kinda spoke to me since I remember that the
first version of Cambria was an sort of like a DSL expressed in YAML.
I am more someone who's working mostly in TypeScript, so I remember reimplementing
a small version in TypeScript.
So, yeah, I love that problem statement.
And so I think it's also interesting how it fits into this broader
umbrella of malleable software.
and turns out, Looking back over the last few years, this is not
the only project you've done in the realm of malleable software.
So which other problem spaces have you run into while pursuing
this goal of malleable software.
Yeah.
I think it's a fascinating space.
You know?
The way I got into it was originally it was actually I was working at a
start up building normal SaaS software.
And you know, we were doing what SaaS companies do, which is that you
have 30 people in a room somewhere, designing interfaces that thousands
and thousands of people, thousands of miles away are supposed to use.
Right?
We were, in our case, designing education software for schools.
I just remember talking to all these teachers and principals and all these
different schools, and they had very specific requests often as customers do.
And, of course in SaaS, typically, the answer is no.
You have to be disciplined and say, I can't change the wording
of that button just for you.
I'm not gonna add a setting for that.
But I started to find this very frustrating and wanted to
explore what would it feel like if software worked differently.
And, you know, I think there's an interesting connection to local-first
because I remember coming across the original Ink & Switch essay
about local-first when I was starting to think about this stuff.
And I think they said briefly in the essay something about how It
seems like an intriguing foundation if you have data locally and
maybe even code locally available.
That gives you a little bit more access and control over your software than
if You have a cloud SaaS architecture.
And that sentence really sort of sparked a bunch of thoughts in my brain that
have led you know, I think a lot of my work, the way I see it is sort of at
the intersection of how do we use the local-first architecture to enable more
malleable software for end users is a theme that I care a lot So Cambria was
definitely one project in that line.
I've worked on a couple other projects in this space.
one project I worked on early in my grad school research career was a little
browser extension called Wildcard.
And the idea was what if you could access the data underneath any web
page in a spreadsheet uh, but it's not only a read only spreadsheet.
You could even write to the spreadsheet and mod it yourself.
So a very accessible way for normal people to make their own browser
extensions without programming.
I love that.
And I think as a fun fact, this might have actually been the very first
point where we've been put in touch.
Because at that time, I was also working on my little own Chrome
extension, which was also kinda in a direction of malleable software.
This was before I was even aware of the term and, like, that broader field.
I was just motivated by my own frustration that I could that the websites didn't
quite work the way how I intended.
So it just feels like we've been on similar paths there, And what
you've been building with Wildcard was quite a step ahead of what I've
been coming up with at that point.
But yeah,
Well, yeah, I, you know, I remember one thing about that that I think
maybe is a preview of the way we've collaborated since then is that mine
was a more speculative prototype of a interface design, but it wasn't
particularly solidly implemented.
And I remember you had this really, fancy developer experience around
customizing your extensions and stuff that I got inspired by.
And so I think There's a fun interplay there we can get into
later about kind of more speculative research versus engineering
more solid versions of the idea.
Yeah.
So you've been working on on Wildcard.
Which are the results that come out of your your overall research in that field?
Yeah.
So, you know, throughout the years, I've had the chance to work on
A bunch of projects with mostly with really awesome collaborators.
one of the more recent projects which I we actually worked on together, was
the the last project I worked on in grad school, which is called Riffle.
And that was a project that you and I worked on together with, Nicholas Schiefer
and Daniel Jackson, my PhD advisor at MIT.
And sort of the, I guess, the origin story of Riffle, is that Nicholas and
I were having a bunch of conversations that revolved around sort of this
idea of having a personal data store.
So, one way I think about cloud versus local-first is that in
local-first software, one of the ideals is that you own your data.
these valuable thoughts that I'm having and artifacts that I'm producing in the
digital world, they should feel like mine, just like a notebook I can carry around
and, you know, have with me forever.
Right?
And There's this question of great.
If you own your data sure, you get longevity and control, but
what about this ability to take arbitrary tools that I wanna use and
connect them to that personal data.
This is something that's often really hard in SaaS because the data's locked away.
And maybe there's the APIs you want and maybe there's not.
And maybe there's the partnership you want between the 2 tools
you use, but maybe not.
And we were talking about, you know, how would it feel if you had a local-first
data store with all your stuff in it And you could connect whatever app
or tool you wanted to that existing data store that has all your stuff.
Right?
Um, what shape would that data store take?
How would it work?
How would you develop against it?
And one hypothesis we wanted to explore, particularly because of Nicholas's
background, which is in databases, we wanted to sort of ask this question of
what if you had a really powerful database that was designed for this purpose?
Maybe it could be based on relational databases.
And maybe the experience of building an app in that foundation could end up
feeling really different from traditional software development in a bunch of ways.
So that's kind of how we got into the Riffle work.
That that's incredible.
And if I remember correctly, This was not the first time that Nicholas
was also working on on this field.
He didn't just work on on databases, but I think he was at at Apple
previously and was implementing similar systems, I think, related to
to iCloud and how syncing works there.
So yeah.
I mean, those are some pretty big ideals,
That is not a very common thing in the apps that that I'm using.
It's, if, God forbid, something like Notion would shut down.
Like, what what would even happen to to to my data?
So I would love to see that flip of, like, Everything that I'm creatively
producing is by default mine and turn around the relationship.
But Uh, it's not even that you wanted to, from a idealistic perspective,
push for that, but I think you took a step back and say, okay.
What is even the technical foundation for that?
What would a prototype of that look like?
How did the project get off the ground?
Yeah.
So one way that I think about this is exactly the way you put
it about the difference between ideology and technical foundations.
I think that data ownership, one of the reasons we don't have it is because
it's hard to build apps that way.
It's not I think there are often companies that don't really have a
strong desire to build a SaaS data moat.
It just is kind of the default that is promoted by the way
we naturally build stuff.
And because it's so hard to build apps with data ownership,
we don't get that many of them.
And one of the I think most interesting potentials of local-first as an area that
I don't think has been fully realized yet is what if we could make it easier to
build apps this way rather than harder?
Right?
I think a lot of people in this space see that potential for this Radical
simplification of how we build stuff.
I think Peter, who was on the show see, you know, has sort of, Inspired
a lot of my thinking on that.
I think Martin Kleppmann, who was one of the coauthors of the local-first piece,
has given talks about how you sort of have this, uh, many layered cloud architecture.
And in theory, it could be so much simpler if you didn't have
to deal with all that stuff.
But, again, in practice, I think we're still getting there as a community,
and the reality is we're still maybe in the zone where it's harder today.
And so, You know, one of the ways that I thought about the Riffle project and,
you know, Nicholas and I and Daniel thought about it was what if the pitch
to an app developer for something like Riffle didn't start from ideology.
It didn't start from you care about giving your users their data and providing
interoperability, blah blah blah.
The reality is for many people, these are not the most pressing concerns.
What if we started instead from, hey.
Isn't it really complicated to build web apps?
What if it was just way simpler to build an awesome app that was fast, that was
really responsive, that stored all the data people care about, was easier for
you to develop faster, simpler, less ops maintenance stuff The data ownership
stuff just falls out as a consequence.
Right?
So a lot of our Riffle work was focused on, could we leverage the
power of databases and the power of the local-first architecture to cut through
a lot of the complexity of building modern web apps and make it simpler.
And in service of that mission, a few of our starting principles were
can you think of an application as one giant database Query.
What I mean by that is instead of writing a ton of normal code that produces
your UI, could you somehow push a lot of the work into a optimized database
that does a lot of heavy lifting and let it produce UI states for you.
Another thing that couples really well with that is fast synchronous reactivity.
So we have this idea that the data's already there on your device.
Throw away all the async crap that comes with web apps typically.
You know, React suspend network requests and work with the data
that's already there and simplify just talking to it and querying it.
And the last thing, our third principle was Can we take all these different
kinds of state that typically end up in different systems and throw them in one?
So your React UI state and your persistent local device local state
and your synchronized multi user state.
Can you just store them in one system?
And then You know, we like to think of it as maybe there's some setting
checkboxes on each bit of state that tell you should you share it or
whatever, but just have one system that's capable of subsuming all those roles.
And our hypothesis was that if you pulled all this off, it could feel really
awesome to build apps, and you would get Both better apps for end users because
they're fast and they work offline and they have all the local-first goodness.
But also you could get a better developer experience because you've removed a lot
of the layers of of traditional web apps.
So this was a lot.
And spoiler alert, that hypothesis that you've motivated, at least
in my very case paid off big time.
So the developer experience for me as a developer is better.
I hope that the end user experience will be better partially also
because I care a lot about performance and making things nice.
But to crack the chicken egg problem of, like, how do you even build better
apps with this in the first place?
What really was the the motivation and the the catalyst for me was
not all the ideals of local-first.
Sure.
I would love them, but that would not really make the difference for
me, like, start to work on an app and build in one architecture or another.
But the more Things I didn't have to do in the first place, the better.
The more I could really focus on working on the app.
And Just to simplify the entire stack your motivation to say, like, hey.
Actually, why do you even need a server?
Why do you need a back end for the thing?
Ultimately, the app is is working on your client.
How about that being the starting point?
That really spoke to me, and this drew me in as an app developer and
where I believe in in that mission.
So, yeah, I I couldn't agree more, and I would love to dig more into the various
principles that you've mentioned since I think there's quite a bit of, like,
interesting thinking behind each of those.
If I recap it correctly, I think it was simplifying the inner workings of
an app into Uh, basically, database queries, aka, like, complex SQL queries
or however else you wanna express it than the the synchronous uh, uh, graph
of typically how React works, etcetera, coupling that directly together.
And the third part oh, you gotta remind me about the third part again.
But let's let's go through them, like, one by one again and with a bit more detail.
Absolutely.
So, you know, the first principle I talked about is thinking of an
application as a database query.
Right?
And, actually, to throw a little more detail in there,
it's not one database query.
It's actually a graph of database queries with dependencies.
Now that starts to sound kind of fancy, but I think we all understand
this model from using spreadsheets.
The the metaphor, I think, to think about is can You make coding
a rich, complex application as simple as using a spreadsheet.
Spreadsheets dependency tracking of formulas and automatic
Dependency propagation is the bread and butter of spreadsheets.
It's one of the key features that makes them useful.
Right?
When you're using a spreadsheet and you update a cell, The contract
that Excel gives you is typically instantly or at least very quickly,
unless you're in a huge spreadsheet, everything will be up to date.
You're never worried about staleness.
You're never thinking about caching.
That complexity of dependency propagation is pushed down into the system.
Right?
And I would note, I think this is a lesson that we've definitely
learned in UI development.
You could definitely say that reactive UI frameworks like, React JS give
you something like this guarantee.
Right?
They you declare the UI you want and state and DOM are synced automatically.
The problem that I see is that often in our web stacks, this reactivity
is partial to one part of the stack.
So in in a common web application, your React UI, what it's
faithfully representing is not all the state in your system.
It's some view you currently have in this tab of some larger state that is,
you know, maybe primarily on a server.
And that introduces a lot of complexity because all of a sudden, you have
your nice declarative spreadsheety React thing, but then you have At
some point in your stack, you're gonna start hitting data fetching.
You're gonna start hitting sending rights back up to the server.
And, that's just an enormous source of mental model overhead, I think.
On the flip side there have been some interesting research projects which come
out from the other side, And people, for example, have literally tried to
make UIs in things like Google Sheets.
There's a really neat project by Ted Benson and collaborators called Quiltstep.
Literally, they use Google Sheets to power your entire back end and all your your,
you know, UI templating and essentially.
And the problem there is normal spreadsheets aren't really powerful
enough to do that, and they're not really architected the right way.
one way to think about what we're trying to do in Riffle is can you take
the beautifully simple model of just declarative dependency propagation, and
apply that to your entire data stack.
So your mental model as developer should be, I just have all the data locally.
I'm just writing a spreadsheet that turns it into a UI, and the
system will take care of the rest.
Full stack reactivity.
That's sort of principle number one.
That makes that makes total sense.
And I think you've Touched on also, like, the the third principle, which I remember
now is, like, the the unifying the UI states and sort of the the other app data.
And if you really think about it, where do you even draw the line between something
being UI state, something being app data?
There is sort of like a arbitrary line.
Maybe I think de facto in most today's web apps is the app data is, like,
what you fetch over something like React Query as your as your website
boots up or you have it pre hydrated.
And the app state It's more of like the in memory stuff that if you
reload the pages, poof, it's gone.
But, ideally, most of the things should really be treated in a very similar
way when I'm thinking of a native Mac app, for example, like Finder.
If I close a window and I come back, The same items are still selected.
I'm still, like, roughly in the the same position with the the folders that
I've selected and and the hierarchy.
And most web apps, I would actually suffer from that they they have, like
basically, they're suffering from dementia when you and insert something
in a form, and maybe you really need to look up every value and maybe the form
expires, you reload, everything is gone.
That's I think everyone has, like, those terrible user experiences.
And so treating everything as the same data, I think is is a kind
of a bold, but in my experience, like, the the right step.
And then if you can, Therefore, as a conclusion, kind of, like, have all
the data that you need to work with.
Have that locally.
It simplifies everything to such a degree that's really liberating.
So that that's sort of the the third the principle, and I think that's that's a
really bold one that is, I think the most provocative compared to the traditional
way of building web apps today.
Yeah.
Exactly.
You know, just like you said I think it can be a disrespectful user experience
to lose people's valuable input.
Even something like Scroll position or, like you said, intermediate
form state can be thing something that I put a lot of work into and
I don't wanna necessarily lose.
Right?
Now I think the deeper reason we end up with software like this
isn't that people don't care.
It's really that we've made it hard to do.
So, again, it's this technical defaults ruling, the user experience
situation that we talked about where, You know, imagine you're a product
manager and you get a request to please persist to this user input or
share this user input across users.
And the engineer says, oh, man.
That was previously React components data, and now I need to, like, rearchitect
where that data is stored and plummet through a bunch of places and Add a
database column for it, blah blah blah.
It's all this work.
And the goal of having a unified system for your state.
It's not that all states should be treated the same.
Definitely, some state, you know, should be user local, device local, and so on.
There, Yeah.
Some state you want to reset on new sessions by default.
So you can think of that as ephemeral.
But If you store it all in one system and manage it all in one system,
the goal is that it's becomes much easier to move between, these states.
Right?
one example I love is that someone who's working on a collaborative app
told me the state of whether a context menu is open in a spatial canvas app.
They actually realized they needed it to be shared because when I open a
context menu, my cursor is moving around.
If you can't see the context you I'm using, you don't
really know what I'm doing.
And so, I think that's a perfect example of blurring these boundaries
where something that Seems like clearly quote, unquote, local component state
in a traditional framing might actually need to be collaboratively synced
or, you know, maybe even persisted.
So that's what I see as the goal of having this unified state management system.
And in the Riffle work that's basically the the maximalist
view we took is no react state.
Right?
Every single bit of state in the system flows through this one database, and your
UI is just derived from that one place.
And You can always reason about essentially, the pixels on the screen
are a function of the database.
That's it.
Nothing else.
And we found a lot of neat qualities that that design led to along with
a bunch of interesting challenges.
And and I have to say, I was I was not on board with that when we got started.
I was so attached to like, everything is in React, and you have, like, things
inside of React components, etcetera.
And I remember Nicholas, he was, like, on the on the opposite side there.
And so, like, no.
No.
Like, React should do as little as possible and, like, have your
state outside, and that didn't click quite back then for me.
But step by step, I came around to it, and I'm now also, like, on
a maximalist side of, like, most state should live outside of React.
And for example, for Overtone, there's a very simple case where
you for example, for the app for the playback of your player.
Like, you want your playback to happen to happen.
You wanna hear something.
Maybe you captured the the time code of the the current playback progress.
And even if the app is not open, You wanna still hear things.
If you reload the app you might just still want to recover at the
the correct playback position.
So all of that state is kinda headless.
So the state still lives on and, therefore wouldn't quite make sense to
have that be part of React, you say.
Sure.
You could say you have, like, a headless React component, but that's already
feels a bit backwards at that point.
So Once I've started embracing that, it was hugely liberating, and now I can
start to reason about my state outside of the context of React components in
that outside of reactive view state.
So That took me a bit to get around to it, but I'm fully on board
with that principle now as well.
you know, I'll I'll mention, I don't think we're the we're not
the only ones who've of this right.
I think um, the idea of having a reactive state graph separate from
your UI tree is something that other web dev libraries have explored.
MobX, Recoil, arguably, even something like Redux, I think shows some of the
power of having this sort of centralized, outside of your UI tree state management.
So, there's definitely, I think some broad recognition that this
can be a powerful pattern for for some of the reasons you mentioned.
You know, but I'll also mention, like, there are some challenges.
Right?
And, I mean, You've encountered some of these.
I'd be curious to hear your take.
Some of them that I think immediately come to mind for people are, one,
Performance is always an easy one.
Like, typically, we expect button hovers and, you know, selection
changes to happen at 60 120 Hertz.
And we don't expect things like loading thousands of rows of
data from our core data store to necessarily happen at the same speed.
And so that mismatch can be an an an interesting thing to Manage.
Our motto Nicholas's motto to some extent is just make everything fast.
If everything goes at 120 FPS, then don't have a problem.
Right?
Easier said than done, but I think a fun goal to shoot for.
So far, most mostly what we've been talking about is
conceptually a simplification.
Getting on board with that different way of thinking about
your your app on a conceptual basis.
And I think for me, it took a little bit of time to ease into that,
and then, be confronted with the new challenges that are raising.
We we'll we'll go into the challenges in a second since
We've encountered quite a few.
But I think we didn't quite touch too much on the the second principle you've
motivated before, which is, uh, the synchronous reactivity, and, like,
that should be as fast as possible.
We'll we'll go into the performance challenges there in a bit.
But when you've said synchronous reactivity as opposed to
asynchronous reactivity, what what are the why is this a principle.
Like, what is so important?
Can can you motivate the the problem and the status quo and
how Riffle is challenging that?
. Yeah.
Absolutely.
This is a really important principle to me.
I think when we Build web apps that have a server.
We assume a lot of asynchrony at many points.
So, obviously, going on the network to a server is a point of asynchrony.
Even often loading data from you know, like a local disk.
Right?
We think of as an asynchronous operation.
And because of that, UI development typically has a lot of reasoning
about asynchrony, which is a really hard thing to do as a programmer.
For example, you might have a UI where There are loading states.
And everywhere throughout your app, you need to think about, am I still
loading, or do I have the data?
And maybe you have a UI that's composed of different sections, and some of them
might be loaded already and others aren't.
You might have intermediate states to reason about.
Like, After I select something in a sidebar, there's some time
where the main other pane of the app hasn't reacted to that yet.
So It might still be showing the old thing that was there before I selected
the new thing, or it might be showing a loading state while I'm loading the data.
And my experience at least personally has been that We kind of accept
this as just the way it is in UI development, but it's terrible.
It's it's a lot of overhead to reason about.
There are many, many more states your system can be in when you have asynchrony.
Right?
And the the vision for synchronous reactivity is let's make it again
feel more like a spreadsheet.
So the way I want my UI to feel is that when I select something, on the
next tick, on the next frame, every pixel in the UI has fully updated
to reflect that new user input.
And that's obviously good for users because they didn't have to wait
But it's also good for developers because you don't have to reason
about the intermediate states.
You can sort of think transactionally where The state of the world changes,
and, transactionally, we update a bunch of derived views and queries of that state
which end up in the pixels of the UI.
And there was never any intermediate state exposed.
And it's It's sort of a subtle point to get across, but my experience has been
that when you work in a system like this it just Cuts a lot of complexity.
Yeah.
It's not just a simplification in terms of architecture that we collapse
the stack from back end and API, etcetera, and all into the client.
But that's, like, one giant dimension of complexity.
But there is a second, much less talked about vector of complexity,
which is like, how is how are things related to each other?
Can they be synchronously related or can or do they have
to be asynchronously related?
And this is where I think we're getting into the area of distributed systems.
I'm sure we'll talk a lot more in future episodes about broader distributed
systems where they're really needed, where you have You're you have a device.
You're in the US.
I'm in Europe, and so we are distributed.
However, if we are starting to treat a local React app that's
running in single thread And we're coordinating the various React use
states things with React useEffect.
We've kinda accidentally created a distributed system that's It's really hard
to tame and where it is, there can be many unintended bugs and just starts becoming
a lot more complex to to think about.
So this is the other part where I think it really it con where Riffle,
again, liberates so much and simplifies and frees us of the the accidental
distributed systems problem that we have in most modern web apps.
I, you know, I, think to 2 principles that I think about one is don't
make something a distributed system if it really doesn't have to be.
And 2 is if it has to be, someone smarter than me should
do the distributed systems part.
Right.
Really, we should be providing abstractions that allow us to not think
about the hard parts most of the time.
Even, you know, I'm not a CRDT expert, but I know quite a bit about CRDTs and
have done some work on developing CRDTs.
But and so I, you know, I Probably know more about CRTs than a lot of
app devs, but I don't wanna think about distributed systems when
I'm building some product feature.
Right?
And I think That is clearly one of the core challenges in the local-first space
that I think a number of projects and companies are trying to address, which
is what are the Good abstractions you can expose to app developers that allow
them to reason about tricky distributed systems problems in a in a straightforward
way without needing to turn on your sort of distributed systems brain as much.
Yeah.
Exactly.
And I I think the Being confronted with distributed systems problems there is
much more common than most app devs think.
So most of the time when in React, you use useEffect that that hook to, yeah, somehow
tame your local state, Then you already have a distributed systems problem.
Things like React suspense and and so on.
That that's kind of like a a solution to address partially the the distributed
systems nature that you have in React.
So, I remember having when React suspense, and all of those primitives came out.
I was kinda wondering, how does that fit into our Riffle world here?
would Riffle at some point support react suspense, and then at
some point, it clicked for me.
No.
That's, like, addressing a problem that we've already ruled out
before, and we don't even have to think about that complexity at all.
And yeah, this is, like, entirely different approach to addressing
the problems that React, Suspense, etcetera, is addressing in a, I would
say, in a much simpler overall picture.
Yeah.
I think maybe to avoid overclaiming here, I'll say there are some trade offs.
Right?
I think Often, these approaches that involve concurrency are they're optimizing
for either the reality of something like a slow network request or for
sort of worst case performance making sure that you keep some things running
fast even if other things are slow.
And I think there's a different approach you can think about, which is, In a
nutshell, just make everything fast.
And as long as everything's fast, it'll be simple.
If things get slow, you might end up with a worse experience than you
might have in a system that, you know that is designed around concurrency.
For example, maybe in a in a Riffle style approach, if you're trying to
synchronously paint frames and something is slow, you might just end up with a ton
of dropped frames and sort of a frozen UI.
Whereas in a system designed to account for that with better concurrency support,
you might have, you know, a loading spinner and some things remain responsive.
And so I think Part of the challenge is figuring out how to
how to balance those 2 approaches.
And as we've seen in our work, I think sometimes you do need to account for
the reality of sometimes you have to make a network request because, you
know, you're talking to an external service or something, and you do
need ways to model concurrency.
It's just that I think We don't need to, I guess, throw the
baby out with the bathwater.
Like, not everything needs to be concurrent, and some things can be fast.
I think I don't have much game dev experience, but I my understanding is
that a lot of video games, essentially, you know, they paint frames, and
they make sure every frame is fast.
And they don't drop frames because they make sure the frames are fast.
Right?
And I wonder how much of that ethos we can apply to general UI development as well.
So we've been referring to this thing, Riffle, now throughout this conversation.
And clearly, you and I are well aware of what Riffle is, And you've motivated
what Riffle is supposed to be through the principles and sort of what you were
hoping to achieve with it in theory.
But in reality, what is Riffle?
Riffle was a research project.
And so our primary goal was to try to get some of these ideas into people's
heads, but we did need a way to test them out and see, is this even possible?
Is this really a good idea?
And to do that, we built a simple prototype reusing existing
parts as much as possible.
Namely, we we built on React JS as a view templating layer, and we built on
SQLite, which is a popular relational database that can be embedded on
devices and in the browser through Wasm.
And what Riffle is, technically, is a layer that sits between React and SQLite.
And what it does is it lets you specify a bunch of, SQL queries that turn a
SQLite database into a React UI, and it gives you some hooks in your React
UI to do things like query data in a component or make writes to your database.
And you can think of it sort of as a state management framework built on
top of SQLite for React that tries to realize some of these ideals we
just talked about in a concrete way.
Right?
Now, again, this uh, was just sort of a prototype to explore the ideas.
And one challenge we faced early on was how do we really test out if this works
in the kinds of context we care about, which are complex data intensive apps with
pretty large amounts of data relatively speaking and interesting schemas
and strict performance requirements.
To do apps are kinda boring and don't really address any of those
and that's actually how we ended up collaborating with you on this.
So, Maybe do you wanna tell the audience a bit about how you got involved?
Yeah.
Totally.
We've been in touch since You worked on Cambria, and I think we've
been catching up once in a while.
And then at some point I learned about you thinking about this this broader
space and rethinking state management.
With my background of Prisma, I'm obviously no stranger to
thinking about state management.
I've been thinking about how to make it simpler, typically
more in a back end context.
And for me, this was really the the interesting foundation
that you said, like, hey.
Why why do you even want the back end?
You wanna build an app.
You didn't quite say it like this, but this is the way how I how I had sort of
picked it up and framed it in in my head.
And that was a really stimulating idea for me to say, like, actually, yeah,
let's bring the database into the client.
I was using a lot of MobX and Redux and those sort of things in
the past, and they they worked.
They made me productive, but they didn't have the properties that you mentioned
before that state wasn't persistent.
It fundamentally treated state and data differently.
I would fetch my data from from a back end . It was a very Interesting idea for
me to have the data all there locally.
So and I was thinking about building my own music app for for quite a while, and
Riffle seemed like the perfect catalyst for me to really take a stab at it.
And the only way how I could could really make progress on such a
big effort, mostly by myself, was to cut scope as much as possible.
And that's that was a very simple math to say, like, hey.
We can cut the scope from the back end categorically, and I can
focus all of my energy and time on building the best client possible.
And a music client is no nobody talks about, oh, that's the best music back end.
People want the best music app.
And that has led me to become the first design customer, does it first design
partner for Riffle, and that made a a brilliant combination, I think.
Yeah.
I think one alignment and values that has really helped here is that, you're trying
to build a music app for power users.
Right?
And I think power users often care about things like data density, latency.
Um, you know, it's funny.
When you look at apps like Spotify, for example, and we've we've talked about
this many times over the course of this collaboration, Spotify, their desktop
app is quite slow often and also quite network reliant to a surprising degree.
Many page transitions that we've encountered even for data that is already
locally cached, for some reason has a lot of network access going on, and you
often get multi second loading screens.
And I think for serious Music people like DJs you know, some of
your target audience for Overtone.
I think of it a bit the same way that, serious email people like apps like
Superhuman that respect their time and make them really fast at email.
A serious music fan wants an app that feels good and not some, you know, loading
screen filled consumer feeling app.
Right?
And I think that was really aligned with some of the goals that we wanted
to hit with the Riffle approach is, Basically, can you just throw all the
music metadata in a SQLite database use relational queries, which are really
good fit for things like joining together albums and artists and tracks, things
like that, and power, the kind of power user experience that you wanted to build.
Exactly.
And I think this was literally also, like, the starting point for our
first prototypes to do a data model.
I'm kinda employing similar ideas that you use in back end development where
you first lay out your different tables.
In this case, I didn't actually create a user table.
I still don't have a user table since the app just runs locally, but
I did create a tracks table and a albums table and a playlist table.
And we threw that together.
And within the shortest period of time, we had a fully
functional, very crude music app.
And step by step yeah, we we I think we've been looking for reasons why it doesn't
work, and we never found that reason.
And then the months passed, and the collaboration now lasted, I think,
for for almost, like, 2 years.
And you've confirmed the hypothesis that you had with your with your
PhD thesis at MIT for for Riffle.
this made for a brilliant partnership.
Yeah.
I think you know, one of our goals with working with you
was that first of all, yes.
Like you said If it went well, which I think it has gone better
than we expected, then it's more valuable confirmation, right, that,
like, a real serious app works with this than, like, a to do MVC demo.
So I think that was one thing.
But, also, the the primary goal was not that.
The primary goal was uncover what the real challenges are of building
real software in this model.
And maybe you could talk a bit about you know, throughout the project,
you've been a a very useful critic and sort of have pointed out a lot
of the the problems and challenges that come from working in this way.
And I'm curious what your take has been on what the biggest problems
we've had to solve have been.
Totally.
So I as a person, I like pioneering things.
So I have probably a higher risk appetite, and I can deal with, like, a
couple of more paper cuts temporarily than someone else who is just looking
for the the best well trodden path.
So, this this made me a viable partner here as opposed to to someone who
only touches the technology that is already well proven over the years.
And so I was rather looking for an advances on a in a broader scale,
like, on a architectural level and just something that in the long run can help
me reduce, and avoid complexity as much as possible even though temporarily
possibly dealing with enduring a few more paper cuts here and there.
And given that I am both comfortable in the shoes of an app builder as well
as in the shoes of a tools builder.
I think I was uniquely positioned to switch between those those 2 modes
to unblock me etcetera, and to have enough imagination to understand,
oh, this is not a categorical flaw in the way how things are right now.
We can fix that through better tooling or refactoring of the API.
So this I think it was kinda in in waves where there was some waves of
immense productivity, and things have been working much better than expected.
And at times, we've hit some walls of doubting, is this ever gonna work?
Are those principles, the the ones that we've picked Are those the right ones?
Is it a good idea to keep our entire state graph synchronous,
can we make it fast enough?
So we've been hitting those challenges, And the ones that are most memorable
to me was really around performance I can motivate it from my perspective.
I've set it for myself as a challenge.
I want to build a music app That's fundamentally built with
web technology that should still feel as fast as a native app.
So that means If you're now looking at a 60 hertz display or a 120
hertz display, if you scroll, if you interact with the app in some way,
It should work without frame drops.
And given that all state management, everything everything that changes
on your screen, is fundamentally a result an implication of state change.
And that this happens smoothly, it needs to happen in a 120 hertz.
So, that basically gives you per frame, yep, less than 4
milliseconds to do everything, state change, render, react, etcetera.
So performance was always top of mind, and I think that where we've been going
back and forth is it ever gonna work?
Oh, yes.
We can make it work.
And that has kept us busy for for quite a while, and I think we can go more
into depth what that's what that meant.
But that's that's the top challenge that comes to mind besides a few, like, paper
cuts and also figuring out, like, how do we the the starting point, even though
this is a local-first podcast, Riffle Up to this point, it was mostly local only.
And just in a later point, we're making we're introducing the the collaboration
aspect to make it truly local-first.
But yeah, I think performance been the the major challenge that we've been facing.
You know, the way I think about performance in this stack is that the role
of the database is to make things fast.
What a database does is you give it nice declarative queries, And it figures
out how to make them fast because some smart people worked very hard,
and you didn't have to do that work.
Right?
And I think a lot of the challenges we've hit basically boil down to, um,
we don't quite have the right database for this use case or a shape yet.
A lot of UI stuff, I think, boils down to the problem of incremental computation.
Incremental computation is a is a pretty broad, you know, computer science term.
That basically means I have a function and the input changes a little bit.
Can we figure out how the output changed without starting over
from scratch on the fresh input.
Right?
And this is a problem that people have approached from a number of sides.
The sort of programming languages community has approached this problem.
You know, incremental computation is the keyword that can highlight
a lot of the work there.
But the databases community has also thought a lot about this problem.
They Often call it incremental view maintenance.
You know, an example might be I have a big join on thousands of
rows and I add one row to the table.
How does the result change.
Right?
And I think a lot of the reasons we've had performance challenges is that our
technical stack has mostly been based on SQLlite which is a database that is sort
of built for UI, but is not incremental.
We did some explorations with a database called SKDB which is actually a SQL
database designed to be incremental, And that yielded some pretty
interesting and promising results.
We actually published a paper at the UIST conference, which talked
about some of our positive results from working with that technology.
I think, ultimately, you ended up deciding to stick with SQLite for the
production Overtone app because you wanted a sort of a more, you know, mature
database to work with as your foundation.
But the way I see the problem is really that we sort of need the right database
that has the right primitives to make UI development of this sort fast.
And maybe the perfect database for that doesn't quite exist yet.
But I think that's that that's the missing piece and the ultimate, you
know, decade ahead if this really panned out, version of the stack, I
think, would include that database.
Yeah.
Totally.
And I I think it's a matter of, like, now it's early 20 24.
Who knows?
Maybe next year, the the world has already moved closer into the
direction that is a good foundation for for something like Riffle.
The the audience shouldn't have the takeaways like SQLite is slow.
Quite the opposite.
I think this was the insight that SQLite is incredibly fast and remarkably
capable was the foundation to embark on this project in the first place.
Nicholas has been through his prior work at Apple, Done a lot
of work related to SQLite and I think related to to FoundationDB.
And that was one of the key insights that we say, like, Actually, SQLite is
so fast and now can be embedded into web applications through Wasm that it
can be a replacement technology for some for, like, MobX and and Redux.
So before WASM, etcetera, and all of the optimizations that have gone into that.
This wouldn't have been possible.
But now that's possible this allowed for a better foundation for to deal
with complexity since if you build something with MOBX, etcetera, you
still need to wrangle all of your JavaScript objects by yourself.
Like, if you want to have, like, a filter or map or group by, you gotta
implement all of that yourself.
And the app user experience that I want to enable with Overtone
is this will be very common.
Think about it, like, in Notion when you have the tables feature and
you have You configure a few sorts.
You configure some group buys.
You have different views.
Those are all things that get very boilerplatey to implement
with JavaScript, and those are all things databases are super good at.
And this is what got me over the fence to say, like, actually, let's
embrace what the database is good at, and build a bit of, like, the
React nice to have things around it.
And I think the the core of what Riffle is is really like that combination
of a underlying SQLite database and put a reactivity system on top of it.
And reactivity system in that regard is, like, how React Components compose,
and if one thing changes, then the other thing updates to the minimal degree.
That's what Riffle is doing.
And A very simple idea with many implications and remarkably powerful.
Yeah.
You know, I I think you're you're right, and it extends beyond performance too.
I think when you think about the requirements of what a UI needs, I
think that when you remove all the layers that typically sit between a
back end relational database and a UI.
Those layers were there for a reason.
They were solving some problem, and you end up needing to find
ways to solve those problems.
So, often, UIs need tree shaped data to hydrate a tree of a UI.
Right?
Often front end developers don't already know SQL.
And often, actually, SQL, we found can be a mediocre language for
certain parts of UI development.
It often just feels a little bit too low level or something.
Obviously, you know, having worked on Prisma, you, I think, have experience with
some of those problems in the back-end context, but I think some of them become
even more apparent in a front-end context.
Some of the you know, I I don't wanna necessarily necessarily say the ORM
word because I think that's a touchy topic, but I think some of the roles of
associating different kinds of models with each other and things like that,
there's sort of, sometimes feels like there's a need for for that kind of layer.
Right?
I think you know, in our technical prototype stack, what we ended up doing
was stuffing a GraphQL layer in there between SQLite and the UI you know,
because you're a GraphQL expert and that helped alleviate some of these problems.
But, again, I think I see and I think we we see that more as a as a shim.
Right?
Like, sort of, a layer that shouldn't need to be there, but
it's there because the database didn't quite have the right shape.
And I think some of that reflects that maybe no one has ever designed a database
specifically for this role of being embedded so closely to a user interface.
Yeah.
Totally.
I mean, I I think about it this way.
We wanna take one or 2 steps forward, But in order to do that, we also
had to take a step backwards.
And the the step forward that we took is that we get the power of a database,
that we get all of those semantics that a database give us, whether it's group by
and sorting and nested selects, etcetera.
All all of those great things that would really um, freeze up our work
and make some things that we would need to imperatively implement in
JavaScript makes that declarative.
However, the steps that we're taking back is that, In SQLite, it's quite limited.
It doesn't have many data types.
It doesn't even have a Boolean type, etcetera.
So you need to do a lot of translation between the way how SQLite understands
the world and how someone who's, like, a a more sophisticated
TypeScript developer as me.
I'm used to having all of my fancy TypeScript types.
And to create a good translation layer between that embedded SQLite database
and what I want in my JavaScript now we gotta reinvest in that.
And so you've mentioned GraphQL.
That was sort of a technology that I was quite familiar with from prior
work, and I think that's sort of a temporary usage until we we teach SQLite
some of the tricks that TypeScript already knows and until we create a
a better mapping in in between that.
But that's certainly been another, For me, less of a challenge, but it still requires
quite a bit of work to make SQLite feel and a native in a TypeScript setting.
Some people, I think, believe that one of the sort of original sins
of computing, so to speak, is that programming languages and databases
really split into separate fields.
And the idea of, like, the system that you use to wrangle your runtime
in memory state And the system that you use to persist and query your
persisted state are so separate in the way you think about them.
You historically, you have systems like small talk, for which
have much less of a separation.
They're closer to you have objects in your language and you're just saving
those objects and reloading them, And there's much less of a gap there.
Right?
And something I found interesting about some of the more recent work you've
been doing in this space is you know, you've been working on tools that make
it easier to close that gap and let you think about the kinds of in memory
objects that you wanna be thinking about in your program, and turn those into the
things that you wanna be storing in your database and querying and vice versa.
And I think that's again one of the really interesting challenges in
this space is handling that impedance mismatch between a relational database
and, your TypeScript code or whatever.
Do you wanna talk a bit about that area you've been working on?
Yeah.
Totally.
I think this is also, like, a for for you, like, a broader umbrella
of of your work that contextualizes everything you've been doing over the
past few years and will probably still be the foundation for for the the
years to come as Malleable software.
For me a common theme seems to be all around schema management and
kind of fighting that impedance mismatch from different contexts.
And so I've been trying uh, to make that simpler in the context of of Riffle.
And in a way that nicely brings us back to the initial topic that we talked about
with Cambria since it's not just Already hard to deal with that appease mismatch
between your application, types that are expressed, for example, in TypeScript
and the way how a database thinks about it how it stores the the data.
But both can also change over time.
You wanna implement new features, etcetera.
So your database schema might change, Your app types change, so you also get a
deal with with schema change management here, which hopefully, we'll we'll
get to bring some of the goodness of Cambria in into the fold here as well.
Schema migrations is still Sort of a untamed problem, but we're we're
getting ahead of ourselves here.
So, if Someone in the audience thinks about, okay.
Riffle sounds great.
Can I use it?
You've been mentioning that you've been doing this as part of your PhD at MIT.
What is the current state of Riffle?
Yeah.
I get asked this question from time to time, so it's probably
good to provide an update.
So Riffle was a research project fundamentally.
The goal was to explore this idea and see if it was possible or a good idea.
It ended up being part of my PhD thesis, and I wrapped up grad school.
And at this point, sort of the the core Riffle project itself is is,
I would say, it's basically over.
We published a paper at the UIST HCI conference this year that
summarizes some of what we learned.
However the ideas are not, dead.
So, actually, you, Johannes, have been sort of carrying the torch forward
to some of these ideas and building them into a new library that you're
building, that has sort of replaced Riffle as the foundation for your work
on Overtone and so maybe you could talk a bit about your ongoing work there.
Yeah.
Totally.
So for me, what's been the most valuable thing over the course of
the past few years in terms of our collaboration is, a, that collaboration,
that partnership, and really doing a lot of research and exploratory work
together and then, yeah, b, having sort of, like, a prototype implementation
of what that could look like.
But it was really a prototype implementation that could prove
out those different ideas.
But to make actual progress with Overtone, I was running into a lot of
the limitations of the those prototypes where it was working on the happy path.
The not so happy path was unexplored and started causing
a bunch of problems for me.
So this caused me not to throw out the baby with the bathwater and reach back
for something like MobX or or Redux.
But it has led me to yeah, where put on my dev tool builder hat again.
And I started productionizing the ideas of Rifflemore, and I did
that under a new umbrella, which is a library called LiveStore.
It is not yet open source.
I'm hoping to get it open source sometime this year.
Right now.
If someone is curious to give it a try it is possible right now if
you are sponsoring the the project.
That's a topic for another day.
I'm hoping to make development of LiveStore sustainable.
And through GitHub sponsorship is one way how I'm currently planning to do so.
But, yeah, the ideas of Riffle live on and possibly multiple projects in the
future, LiveStore being one of them.
I'm sure your work on Riffle has inspired more folks to implement
similar ideas in the future.
So that's that's what I can contribute right now.
Yeah.
I'm really excited to see where your work with Livestore goes.
You know, I think You have some some cool stuff cooking and places you're trying
live store, and I'm sure you're gonna keep learning a ton more about where
this approach works and new challenges.
Right?
But it's very exciting to me that these ideas are living on in your work.
Yeah.
And so Geoffrey and I are still touching base every so often, and I share my
updates and progress on LiveStore.
And Now step by step I get to make progress on some parts that we haven't
quite got to yet while working on the Riffle research project and
where I get to unlock some progress where Geoffrey thinks, oh, yes.
Finally, we're getting to that point that we haven't gotten
to while working on Riffle.
And I now I'm I have to face all of those bigger, implications that
we've been well aware of, but we just haven't gotten to them yet, whether
it's schema migrations or making the technology actually local-first.
So introducing syncing and collaboration capabilities, and I wanna do so
in a way that is compatible with different kinds of syncing approaches,
whether it might be something like electric SQL or possibly something
like Auto Merge or Yjs in the future.
So there in a way, we've come really far, and in a way, we are just getting started.
So I'm I'm super excited about that.
So we've been talking a lot about SQLite, Riffle, LiveStore, But you've
since moved on to to new projects.
So and I think you're still at Ink and Switch even more so than for I
think you're now full time at Ink and Switch and are already in new projects.
So which sort of problems are you currently looking at?
Yeah.
So since finishing my PhD earlier I guess last year over the summer, I've now
for a few months been full time at the Ink and Switch Research Lab where Peter
Van Hardenberg again is the director.
You know, they're where a lot of the look they they coined the term local-first.
Right?
And so they're what got me into this space in in some sense, and I'm
Really excited to be full time there.
I'm leading our malleable software research track now.
And so that's basically a research track where we explore these ideas
around customizable software.
And we really are interested particularly in the intersection
of the local-first architecture and the malleable software agenda.
And we have a lot of ideas for how those 2 things can be kind of complementary
to each other as I described earlier.
We have some fun experiments brewing.
one thing is that we're increasingly trying to use our own local-first tools.
So one Little project I worked on in my past few months there is
spinning up a writing tool that we use internally at the lab.
We used it to write our last essay that we published and that's based on the auto
merge stack that the lab has developed.
And it's a really nice way to sort of, Again, just get some authentic
experience using the stack and understanding the challenges and
the and the and the good stuff.
Right?
We're then kind of using that stack as a foundation to explore a lot
more speculative stuff going forward.
You Yeah.
Some of the ideas we're excited to explore are the potential for version
control in a local-first environment.
If you have, a layer like auto merge in your stack, which is really designed
around not just local-first data, but also storing, past history of
documents, what can you do with that potential to unlock for end users.
And another theme we're gonna explore, and have started exploring a bit already
is not just data in a local-first way, but code in a local-first way.
So if you put code on people's devices in a way that they can mod
themselves what does that unlock in terms of customizing our software?
This is sort of a fully maximalist local-first approach and it's something
that Peter and the lab have explored in the past but it's, you know, there's a
project called Pushpin, which I encourage people to look at if they haven't
heard of it, which is basically, an exploration of what happens if you have
local-first data plus flexible tools that can op that can operate on that data.
And I'm excited to keep pushing in those directions and see if we can basically
give people more control and empowerment over their computing experience on
top of this local-first data stack.
That is incredible.
I'm very tempted to also somehow contribute more in those various
topics that you've been mentioning.
But
My day has just so many hours, and I think I have still a lot of work ahead
of me with LiveStore and and Overtone.
However, I think there will be many problems that will have positive overlap.
I think we're still not done with the Cambria problem.
Maybe we get to collaborate there in the future.
We've been doing also some really interesting exploratory work around
what does it mean for an app to have version control for for users.
That is something I'm really interested in so I'm I'm sure
we'll we'll get another stab at collaborating on some project together.
But, yeah, Geoffrey, I've been looking forward to this for a long
time and really excited to share what we've been working on over the past
few years with a broader audience.
And thank you so much for for taking the time and sharing all of those
stories and and insights with us.
Thank you.
you
Thanks so much.
It's been fun.
Thank you for listening to the localfirst.fm podcast.
If you've enjoyed this episode and haven't done so already, please subscribe and
leave a review wherever you're listening.
Please also consider telling your friends about it, if you think they
could be interested in local-first.
Thank you again to Expo and Crab Nebula for supporting this podcast.
See you next time.