Nik: Hello, hello, this is Postgre.FM My name is Nik, PostgresAI, and as usual with me, Michael,
pgMustard.
Hi, Michael. Michael C.: Hi, Nik.
Nik: And we have a very, very interesting guest today, Michael Malis,
who created pgrust, which has already, I think, 5,000 stars
on GitHub and a lot of noise around, like a lot of buzz. Hi, Michael, thank you for coming.
Michael M.: Yeah, of course.
Thank you for having me.
Nik: So, of course, I think the 1st question should be how it
all started, why?
Tell us the story, please.
Michael M.: So Jason and I were looking for projects to work
on, and we wanted to do something that we knew super well.
And initially, what we were focused on was reliability and how
can we help people make their websites more reliable.
And we were working with a bunch of people.
And the common pattern that emerged was that A lot of the problems
that caused reliability issues, a lot of them stemmed from their
database and how they were using their database, whether it was
like Postgres or Redis or some other system.
And so we started looking at what are ways that we can solve
this problem.
And we were throwing around a couple of things, and we ended
up figuring out that AI is actually pretty good at rewriting
software at this point.
This is back in April.
And so we're like, hey, why don't we actually try fixing these
problems at the source and actually trying to modify Postgres
and fix a lot of the things that cause people to have different
reliability issues, whether it's connections or it's like certain
queries just take a really long time or like 1 long-running query
can take down your database or vacuums as we're all familiar
with.
And so the idea behind pgrust, the name's a bit misleading because
it's not actually really about Rust.
I think that's actually the least interesting part about it.
It's actually more about how do we leverage AI to re-architect
Postgres and just build a much better database.
Nik: Okay.
And when you say better, what do you mean?
Michael M.: Yeah.
So there's a combination of like different challenges that like
I've just seen people repeatedly have with Postgres and I actually
wrote a blog post called it like the 4 horsemen about a lot of
the challenges that people have.
And some of the most common ones are like, you misconfigure your
connection limit and like all of a sudden like nothing can connect
to your database.
You have like JSON support.
Tons and tons of people use JSON but Postgres doesn't have statistics
for how to actually query JSON.
So when you try to do it, you're going to get really bad query
plans and everything's just going to be really slow.
There's like the wraparound vacuum and you have more than several
billion transactions and like all of a sudden if your vacuum
can't keep up your database falls over and there's just like
all these problems that have been around for such a long time.
And like Postgres is a great product and like it is a really
great system, but the way the Postgres core team approaches things
is they approach it in terms of stability And how do we keep
what we have today?
Like, how do we keep it working?
And how do we make sure we don't break it?
Because like, there's like billions of Postgres instances out
there.
And so number 1 priority is just how do we not break the existing
stuff?
Versus how do we actually fix the things that are not working.
And so pgrust, because it's a new project, we can take a little
bit of a different approach of let's try to actually fix the
things that aren't working.
And maybe some of the existing stuff, it's not going to be quite
as reliable, at least upfront, as Postgres is.
But at least we'll be able to fix some of the long-standing architectural
issues that Postgres has.
Nik: Yeah, it makes sense.
How did you do it?
It's a lot of...
Obviously, it's with Claude, right?
Michael M.: Yeah, it took a couple of attempts to actually figure
out what is the best way to do this.
And what ended up working really well was with Opus, we were
able to take each file of Postgres and basically transpile it
to Rust.
And so if you look at our code and look at the Postgres code
side by side, it actually looks very similar.
Where like the functions are the same, some of the details may
be a little bit different, but overall it's actually really close
to just a straight rewrite of Postgres.
And so we did this across all the files.
There were some things that had to be changed to actually work
in Rust.
1 of the big ones was how memory management is done, because
Rust is very particular about how you allocate memory.
But this approach of going through all the files, rewriting them,
we had a bunch of Claude agents like going at this in parallel.
There were like some conflicts between different files, but we
were able to resolve those and then get pgrust to actually work.
And then from there, we focused on the Postgres test suite, getting
all that to pass.
And we have something that actually looks a lot like Postgres,
but is in Rust.
Michael C.: I have potentially a minor question on the source
code.
Really interesting that you went file by file and that worked
well.
1 of my favorite things in the source code is all the is the
comments and they describe in quite a lot of detail how things
work and I wonder if you've noticed how it rewrote those.
I can imagine the code being somewhat reliable in terms of it
being translated, but I can imagine the comments actually being
harder in some way to be trustworthy.
What have you found on that front?
Michael M.: Yeah, for the comments, like Claude has been like,
Claude is very verbose when it comes to comments.
And like, every file have a whole like big comment about like,
what's in the file that I've actually had to give it feedback,
like I had to give it feedback to comment less.
And I think there's a linter that says files should not have more
than 5% of the line shouldn't be comments.
I haven't paid quite close attention to the actual content of
the comments, but I do know Claude is making a lot of comments
in the new code.
Nik: It does, yeah.
So I remember, now it's version 0.2, right?
Michael M.: Yep.
Nik: Yeah, and I remember version 0.1, I think there was a claim
that all tests pass, but when I wrote some silly stuff, like
obviously not SQL syntax, it accepted it without any errors.
Now it's not so.
In 0.2 it works fine, so it looks like Postgres syntax is implemented
very well.
Was it about some lack of tests in Postgres tests for checking
negative stuff?
I don't know.
Or it just wasn't complete implementation before?
What changed between 0.1 and 0.2?
Michael M.: Yeah, so pgrust 0.1 was largely rewritten by Opus.
And Opus I found to be a fine model, but it's a bit difficult
to work with.
They'll mislead you a bit about how much work is actually completed,
we would ask it to rewrite this file and it would rewrite 0.5
of the file and not all of it.
And so we were working with this underlying unreliable model
and all models are unreliable to various degrees, but with enough
quality controls and layering on top of it, we're able to get
all of Postgres ported, and we're able to get the Postgres regression
suite to pass, but that's actually a really low bar.
The Postgres regression suite on Postgres only has about 0.667
code coverage.
And so you can pass the regression suite, but there's still 0.333
of Postgres that isn't even being tested.
And on top of that, the regression suites are largely more like
functionality tests.
That every feature in Postgres, there is a regression suite for
it that's like, there's 1 for hash joins, there's 1 for lateral
joins, and there's 50,000 of these that basically just go through
every single Postgres feature and make sure it works to some
degree.
But they actually just do a couple tests for each feature and
then move on.
And so the tests are more about just making sure this feature
is there and it exists and works well, but doesn't actually really
like you have a perfect implementation of this feature.
Nik: So you, there is an idea to improve regression test suite,
right?
And this is already, this should go to upstream, maybe, no?
Michael M.: Yeah.
So what we've been doing, the difference between 0.1 and 0.2
is 0.2 was run by Fable, which we have a more reliable model
now that's, is more consistent about, okay, you ask it to port
a file, it'll actually rewrite the file completely.
And so we do end up with something that is just baseline, more
similar to Postgres.
And then before we released 0.2, we started doing a little, a
tiny bit of differential testing and fuzz testing, and I can
talk a lot more about that.
But for 0.3, we're actually like going to have a lot more of
that.
What we've thought of doing is we could add more regression tests,
but that, again, every single regression test is only testing
a small part of the functionality.
And what we really want is not just, hey, I can choose 100 tests
and they'll all pass.
I want to know is this thing actually identical to Postgres?
And some of the things we've been doing, and I was very surprised
by this, that it's even worked.
I would have never thought this.
We found this library called Kani, which can take Rust code and
C code and do formal verification over the code.
Nik: Without any execution?
Michael M.: So it does symbolic execution.
So it'll run your code, but it'll run it in a special way that
it is more just picking out that, hey, this is an if statement
that is checking this condition, as opposed to actually having
variables go through it.
And then I'll be able to convert that into a format that you
can then do formal verification over that has the Rust, I'll
have an expression that represents all the Rust, and I'll have
the same for the C code.
And then we can do across all inputs, are these 2 things gonna
produce the exact same output?
And so Postgres has 3000 different user facing functions, everything
from substring to regular expression matching to there's like
the, like you can calculate the gamma function, there's a function
for that.
And so for about 1000 of these functions, we were actually
able to do actual formal verification that the Rust code and
C code are the same.
But then beyond that, what we've started, like formal verification
doesn't work in all cases.
It's actually narrow.
And so what we've been doing for the rest of the code base has
been, we'll take the Rust code and the C code and put them side
by side, and then have a fuzzer, like, code coverage guided fuzzer,
generate millions of inputs across the 2 and make sure that they
behave exactly the same.
And in the process of doing this, not only did we like find over
100 bugs in like pgrust, but we actually found at this point
a little bit over 20 bugs in Postgres itself.
Nik: I see.
That's interesting.
Have you reported them?
Michael M.: Yeah.
If you look, I think like end of July, like 1st week of August
on the like pgsql or like pgsql bugs mailing list, like you'll
see a lot of the form submissions are actually from me.
Nik: That's cool.
I missed it.
That's cool.
Congrats, actually.
That's good.
Yeah.
Michael M.: None of the bugs.
I found 3 serious bugs, but all of them had been found already
by other people.
I'd been testing on 18.3, 18.4, and in 19 people had found them.
There were a lot of not serious bugs that people had not found.
1 of my favorites is Postgres has a quadtree implementation,
and it will use floating points to represent the positions in
the quadtree.
And there was a bug where, because of floating point rounding
and the arithmetic that Postgres was doing, it was possible for
a point to not be in any of the 4 quadrants.
That the comparison would say, this isn't to the left, this isn't
to the right, and this isn't in the center.
And we're definitely doing like very aggressive.
Nik: This is in the SP-GiST, right?
Michael M.: I think it was in, I don't know exactly.
I want to say it was in GiST, yeah, but basically the amount
of testing we're doing is so thorough that we're actually finding
these extremely hard to find bugs and we're finding quite a few
bugs in Postgres itself.
Nik: This is very important because it means that with AI you
can, you basically have some mechanism which is finding bugs
Michael M.: all the
Nik: time, right?
And it can be maybe like, what could it mean?
Probably there's Postgres Buildfarm for testing, which is used for
all releases.
With this work, it maybe should be extended to involve more AI
and find more bugs at scale, right?
Michael M.: Yeah, there's a couple things I think are really
interesting.
1 is that with Mythos, this whole thing about Mythos can pose
a security risk.
The interesting thing about the models is like, the security,
the way that they pose a security risk isn't that like, the model
looks at your code and can find a bug in the code.
It's actually that the model can build tools that can then find
bugs in your code.
Or like, I've been using Fable and a lot of times it'll fall
back to Opus 5 because it's, oh, you found a memory bug, that
seems like a security issue.
Nik: I think, just From my experience, I also found a couple
of bugs with AI.
A few were security-related.
1 was officially registered and was already patched.
That 1 was just looking at the code.
I didn't use Mythos, even not Fable.
It was like Opus 4.5 back then.
And I just was looking at the code, it was very old code, and
just looking at it, it found SQL injection, which is there already
18 years.
So in some contrib module, which is not used directly, so nobody
cared, but it was actually serious because it's used as an example.
So anyway, it was interesting that just looking at code, it also
can find bugs.
But you're right, building tools is even more powerful because
just looking at the code, you can miss complex relationship between
various code pieces, right?
Michael M.: Yeah.
So with the models, like what you can do is you can basically
point them to be like, hey, I have 100 functions in this code
base, write really thorough tests for all of these different
pieces of code.
And so the way I like to think about it with these models is
you can, if you can get them to do 1 task well, you now have
a repeatable way to do that 1 task 100 times.
And so if you can get a model to be able to test 1 function,
there isn't a reason you can't get it to test 100 or 1000 or
10000 functions.
Nik: Yeah.
How much of AI capacity have you already used?
Is it like just 1 $200 account or no?
I'm just curious, very curious.
Asking for a friend, so to speak.
Michael M.: The 1st version of pgrust, the 0.1, which was done
with Opus, which that spanned 5 different attempts, that cost
about 100 grand.
And then so far, with the new version, like 0.2 and will become
0.3 in total, we've put in probably like another like 300 or
400 grand.
Where the...
We're burning tokens so fast that you can burn through a 5 hour
quota on a $200 a month subscription on the, in the order of
like 15 minutes.
It's because for a while, like what we'll do is like, we'll have
a 40 different Fable instances like running in parallel, each
like working on individual parts of Postgres.
And that just burns through credits super fast.
Nik: Yeah, yeah, that's impressive.
So, yeah.
And it takes also a lot of time, right?
Because like, it's just they are like, you just need to wait
sometimes a lot right what was your the longest run or loop
fully autonomous like days weeks or something
Michael M.: I usually well I've had stuff like run overnight
that worked but I think I try to keep things on the order of,
I try to break things down to like tasks on the order of like,
ideally like an hour, cause any more than that, I find that like
the chance of the model going off and doing its own thing and
like getting.
Nik: Nonsensical.
Yeah.
Michael M.: Yeah.
I found that like models, they work really well.
They enter 2 states.
Like 1 is they like are just like working really well and making
forward progress.
And then other times they'll just start doing their own thing.
They'll just start making like a bunch of random edits to like
code everywhere and not really go anywhere.
And like the more you can keep models on like the productive
side and not on the going in circle side is better.
Michael C.: I'm curious on the, because that's a lot of money
to spend on v0.2.
Are you being public with how you're funding it or like if you
do have investors.
Michael M.: Yeah.
So far it's actually been entirely funded by me where I had a
previous startup, Freshpaint, which did like pretty well.
And I had the chance to sell some of my equity and so far it's
mostly been funded by me.
We're hitting the point where like it's starting to become like
a bit unreasonable and we're starting to look at other ways to
like fund it and can we like be more token efficient and things
like that.
Michael C.: That makes a ton of sense.
So yeah going back to the technical stuff I was going to
the test suite is really interesting to me, and almost as a what
can we learn from a Postgres perspective, as well as how could
pgrust become trustworthy.
So, on testing the functions, I can imagine a huge amount of
the kind of user surface area is going to be covered nicely,
like especially for single user, single query correctness.
But I can't help but feel nervous about like concurrency stuff
and even Postgres and some other databases that have had some
interesting bugs come up when Jepsen have got involved and run
some like interesting tests along the like isolation level side
of things.
Do you, does your testing approach cover that yet?
Or do you, are you going to have needed like a slightly different
approach for some of those things?
How are you thinking about that?
Michael M.: Yeah, we're basically taking what is the modern and
latest approach to testing and trying to incorporate all of that
into how we do things.
And the 1st thing we've been doing is we've started with the
easy part of, okay, here's all the pure functions.
Here's the ones that we can verify.
Here's the ones that we can just fuzz.
And then most recently I've been working on how can we actually
get full coverage of the database where there's a lot of very
stateful things that will only trigger bugs in very rare circumstances.
And if you actually look at the issues people are reporting for
pgrust, it's usually the combination of 2 features of, oh, if
I use a non-default collation with certain string functions,
it'll cause a bug.
And so what we're trying to do now is actually fuzz until we
get to 100% code coverage.
And then for the concurrency stuff, once we have a fuzzer that
can actually properly explore the entire code base, we already
purchased this tool called Antithesis, which does Jepsen-style
fault testing.
It's actually a super cool product.
They run your software inside of a VM and then are able to inject
faults into your running code.
And so that can be everything from the network is unreliable
and your database can't talk to its replica.
Or it can be like, hey, your server crashed and now is your data
corrupted on disk or not?
And then they actually do stuff of they can, they have full control
of the threading.
It's fully deterministic.
And so they can create all these weird out of order sequences
for your threads to try to just throw the craziest stuff at your
software.
And right now we're working on the fuzzer to actually explore
the search space.
And then once we have that, we're going to give that to Antithesis
to then try to come up with all these crazy ways of running the
code together to try to break pgrust.
And so I think that is, we've actually been talking with a lot
of database people through this and basically they're all using
Antithesis for their testing at this point.
Nik: Yeah.
Speaking of testing, I remember I noticed on Hacker News comments
that on the thread that people asked questions about was fsync
on and did you do writes in those tests and so on or is just
only from memory, data fits in buffer pool or something and
nothing touches the disk.
Michael M.: Yeah.
So we have, if you look at the code base, we actually have a
crash simulator in pgrust that mocks out the file system and
then we'll try to just crash the database at different points.
And we've done a good amount of testing.
The thing is that even Postgres does get these things wrong sometimes.
That there was like a fsyncgate almost like a decade ago now,
I think, of like, yeah, fsync can actually fail and Postgres
actually didn't handle that case.
Which be fair to Postgres, that's a case that never ever happens.
Nik: Honestly, I also think to me going to Jepsen tests and multi-node
basically network testing and so on is maybe not the biggest
priority for this maybe.
In my opinion, I recently learned that Postgres own tests are not
checking properly disk outages, for example.
So this is like, what happens if disk just has some problems?
And there are some tools to improve that in Postgres on codebase.
So I'm very curious.
This is purely a vibe-coded thing.
You obviously created a good harness to mock and so on, but still
people reported some segfaults.
It's obvious it's early stage.
Will it reach some point when it's very reliable so it can be
put to production.
So obviously you say yes, but what's your like, why do you think
so?
Why do you think yes?
Because I'm also very AI positive, so to speak.
I think actually in a few months we will be able to revisit with
new models and polish it and so on.
And It gives me so joy to work with AI together, like human plus
AI, we can achieve a lot.
But still, there are so many doubts around.
And people think, for example, it's impossible to achieve a point
where it will be reliable.
You started with reliability, right?
Then you said hackers are too conservative, but there's also
they are too conservative to protect that reliability thing,
right?
Now with vibe code, the thing fully rewritten, won't it take
10 years to achieve a very reliable state?
What makes you think it's possible to achieve it much faster
than 10 years?
Michael M.: So I think for the issues that people are reporting,
I think 1 thing to clarify is that so far for version 0.2, we
had, or actually I think for version 0.2, that was actually before
we had done even like most of the fuzz testing I was talking
about, where like we actually sorted out those issues.
The correctness work has like largely followed version 0.2 and
so far we've only covered like 15 or 20 percent of the codebase
and like there's still a lot more to cover.
And so I'm like I'm totally not surprised that people found issues
in the version we released just because like we hadn't actually
hardened it that much.
And to answer your question of how we can actually get this to
a place where people can trust this, Back to what I said earlier
about, it's not like the models themselves are like reviewing
all the code and like making sure it's correct.
What we're doing is like we're building tools to actually make
it, make sure it's correct.
And we're doing it such a degree that like we're able to do millions
of inputs for individual functions to the point that Postgres,
this 30-year-old battle-tested codebase, we're finding bugs in
it that no 1 had found before, even though people have been using
it all this time.
Nik: You find bugs in the old code, but Maybe you have bugs which
you haven't explored in new code yet.
So this is a really tricky question.
What's your plan to prove it in production?
Just wait for people who will try it in production on replicas,
for example, because I always think This thing can work as a
physical standby.
Is this a way or use some mirroring, for example, PgDog has or
other, maybe some others have, to mirror traffic and see that
it works well?
Or what's your plan to prove with production that it works?
Michael M.: Yeah.
So for proving it like actually in production, our plan is to
do exactly what you said and replicate off of people's existing
databases and be like read-only standby.
That gives people a lot of the benefits of like what we're building
in terms of this like ultra fast columnar workload such that
like people who are trying to do analytics in Postgres and are
struggling to do it, they can stand up pgrust and then move
to analytics over to that.
And the really nice thing about analytics is they tend to be
like semi-production workloads where either like it's internal
usage and okay your database goes down, okay my internal dashboards
are broken for a little bit, it's not the end of the world.
And then over time as like we start to get like battle tested
in these not tier 1 database use cases, that's I think, will
give people more confidence that, hey, this is actually something
that, like, works well.
Nik: So what I like in this idea is that if it's, if pgrust
is powering a physical standby, and then it crashed, for example,
but primary won't be affected, right?
Because, okay, some slot is not used.
And if it's working and you run very long running query on it,
usually it's a bad idea on regular standbys because of hot_standby_feedback
dilemma.
If it's on, it will affect vacuum on the primary.
If it's off, it makes your node basically single user.
It will start lagging.
So both choices are not okay if you want analytics.
But It's only true if those queries last hours.
If it's already, as your blog post suggests, 300 times faster,
then xmin horizon is not blocked by hot_standby_feedback for long.
It's blocked for a short time.
This is very interesting.
This looks promising.
We have very fast standby, which behaves exactly like Postgres.
Like, primary thinks it's like Postgres, okay?
And we have ability to run some aggregates and so on very fast.
Michael M.: That's interesting.
Actually, the thing I find really interesting about pgrust is
there's actually a whole bunch of unexplored design space, new
ways that you can do things.
Where you mentioned, for instance, when you have a replica, due
to the way the WAL replay works, there's all these like complexities
about how did the vacuums work and how does like the replica
like fall behind.
But I think there are options out there.
I haven't explored them thoroughly, but there's options out there.
If we change the file format and the way the WAL replay works,
maybe there's ways to actually get around these problems.
I think this is like a lot of what's really exciting about pgrust
is like, there's a lot of like unexplored design area.
And because it's this like experimental new thing, there's actually
lots of opportunity to explore the design space.
Nik: Right, but that's for sure.
So experimental, you can go very far.
But grounding is when you put it to production.
This is the most interesting practical piece here for me.
Sounds like it's relatively low risk to try it.
There's a way, right?
That's interesting.
So, Yeah, that's cool.
What else?
I saw your blog post, I think today, right?
About JIT just in time.
What attracted your attention?
It was just switched off by default finally, recently.
Was this like, yeah.
Michael M.: Yeah.
As part of the Columnar stuff, we looked at a bunch of the recent
database literature and incorporated a ton of stuff into pgrust.
A lot of it actually coming from the research into Umbra, which
is this very experimental database being developed by this team
in Munich.
And they've been publishing database research for the last decade
or more, and a lot of it comes from them.
But for JIT compilation, and this is a lot of what I talked about
in the post that came out this morning, if you look at all the
databases that have JIT compilation in it, they either use LLVM,
or they'll write C and C++ code and then compile that.
And the big downside of these approaches is that there's actually
a huge amount of latency overhead to compile this code that for
C and C++ or for LLVM, I think it's 50 milliseconds compile time.
And so you're very limited in the cases that you can use it.
And you have this risk of if you're the planner estimates the
query wrong, you now have this extra 50 milliseconds of latency.
But there was this approach to JIT compilation called copy and
I think it's like copy and fix or something along those lines
that was released in someone published a paper on it in 2021
that makes copy and patch.
Yeah, that's it.
Then it's much easier to write.
Then you just have templates of assembly code and then you just
fill in the bits as needed.
You can combine these templates together to get the code you
want.
And so the pgrust JIT compiler actually doesn't use something
like LLVM.
It actually just directly generates the assembly.
And so because of this, the compile times are like 5 microseconds
as opposed to 50 milliseconds for LLVM.
And this gives us a lot more, Like we can JIT compile a lot more
of the query that we actually, for our executor, we JIT compile
large parts of the query, not just expressions, which Postgres
will only JIT compile expressions and tuple deforming, but we
actually do a lot more than that.
And so we get measurable speed ups in many different places.
Michael C.: Do you see any reason Postgres couldn't use a similar
approach?
Michael M.: So 1 of the big, 1 of the challenges I see with Postgres
adopting this approach is I Specifically am targeting only the Graviton
instruction set Whereas pgrust is being built in a very different
world where today most people are running their databases in
the cloud And for instance, there's no Windows support in pgrust
yet.
And so I'm able to, because I know, I'm just assuming that people
are going to run this primarily on AWS and GCP, I'm able to target
1 specific CPU design and 1 specific instruction set and focus
just on that and make that work really well.
And for Postgres, I think it's still possible.
You could have the JIT compiler only work in certain instruction
sets, or you can have different backends for different instruction
sets.
But it's an order of magnitude more work to get that to work
well than what I'm doing.
Nik: And if, even if it's not in cloud, they will run it on Mac
Minis they have now, right?
Should work anyway there, Yeah.
Michael M.: I've seen multiple people with like rooms of like,
they'll have
Nik: a bunch of...
Yeah.
In the corner.
Could be a good Postgres cluster.
Yeah.
So that's interesting.
And you don't use GPT like Codex, no?
Like distributing workloads within?
Michael M.: So in my experience, actually like the, a lot of
the early work we were doing with Codex, like Codex 5.4 and Codex
5.5, the thing that made me switch to Claude is they released
this feature called dynamic workflows, which this is what was
used for the Bun Rust rewrite.
And dynamic workflows are, you can basically, it will write code
that orchestrates a bunch of different agents.
And so you can be like, Hey, I have these 10 files.
Can you rewrite all of them?
And it'll write some JavaScript, they'll then spin up 10 agents,
they'll then rewrite each file and then review it and fix any
issues and then merge it.
And so that level of orchestration made it way easier to use
Claude.
And then in my experience, like Fable just works, it's like way
more reliable and trustworthy.
1 of the actually like biggest challenges I had with the rewrite,
or like 1 of the things that's actually really hard, was the
Postgres regular expression engine.
I don't know what it is about it.
I think it's 10,000 lines of code.
It's a decent amount of code, but not too big.
And I was doing this as a benchmark, and I threw Sol at it on
medium-level thinking, not the highest thinking, but on a decent...
Sol's a pretty good model, And Sol wasn't able to rewrite it
in Rust.
I was like very surprised by that.
I was like fully expecting that given how smart Sol is, like
it should just be able to do that.
Whereas Fable was like a whole level above that, where it was
able to rewrite in Rust and then it was able to do like performance
optimization on it to make it a tiny bit faster than the Postgres
version.
And so like, I think like it probably is possible to use like
Sol, but like I've just found the difference in autonomy between
the 2 models is like not worth the difference.
Michael C.: I think we need to talk about licensing.
I noticed
Nik: you picked
Michael C.: the AGPL.
That's always controversial, but at least open source, right?
What, why pick that 1?
I think I know why, but why?
Michael M.: Yeah, we were looking at the different options we
had for licensing.
And there's like 4 different options.
There's you have MIT or like Postgres.
You have like AGPL or like GPL, where it's like the copyleft.
You have what they call like source available, like you have
SSPL and like BSL, and you have like closed source.
And We knew we wanted to do something open source, and it's now
limited to the permissive or the AGPL.
And the thing was, we wanted something that a third party couldn't
just take and sell themselves to the detriment of the work we're
doing.
Where there's lots of closed source forks of Postgres.
You have AlloyDB, you have Aurora, now Neon is closed source.
And none of that money that those platforms make actually goes
back, or like very, very tiny amounts of it goes back to the
actual Postgres core projects.
And that was just something that like I wanted to avoid with
pgrust.
Nik: But AGPL doesn't protect you from selling.
Anyone can still sell if they don't modify code.
And even if they modify code, they just need to publish it.
That's it.
Michael M.: Yeah.
Yeah.
So at least the history for AGPL is it is like extremely, basically
no 1 has really done that.
But like just the cost of needing to open source your modifications
to it has deterred like Amazon and Google from offering any AGPL
software on their platforms, at least so far.
Nik: Makes sense.
So
Michael M.: that seemed like the right trade-off.
Nik: Good.
I know we're like almost out of time.
It was an interesting discussion.
Thank you for coming, Michael.
I enjoyed it very well.
Good luck with your project.
We will be keeping an eye on it.
Michael M.: Yeah, of course.
Thanks so much.
Nik: Thank you.