Nik: Hello. Hello. This is Postgres.FM. My name is Nik, PostgresAI. As usual, my cohost is Michael pgMustard. Hi, Michael. Michael: Hi, Nik.
Nik: And today, we talk about fundamental topic, and we have returning
guest, Radim, from BoringSQL.
Hello, Radim.
Thank you for coming again.
Radim: Hello, guys. Hello, Nik. Hello, Michael.
Michael: Hey, Radim. Good to have you. Nik: I remember we had very interesting discussion, and you obviously have interesting points of view.
So this topic, you recently published an article, and it was broadly discussed in community on Hacker News Everywhere.
Let's start somewhere.
Like, what caused writing this material? Radim: Okay.
That's a very good question because I'm not sure if you know
it, but I published a series about Postgres internals.
And that wasn't that's on arrival.
I think there is a audience that reads it, but the real feedback
I'm getting is actually during the conferences.
So I think I've decided to write this article during the PG DATA
in Chicago because I'm actually doing the visualizing Postgres internals series. And this is where most of the interesting ideas and interesting
question happens.
And many questions because people on those conferences are Oracle
practitioners now switching to Postgres even if they some of
them won't, some of them are not so keen, and they ask very uncomfortable
questions.
Very uncomfortable in a way why it's built this way.
And then I'm also working with other database systems.
I'm not actually like a Oracle DBA, but I work a lot over the
SQL Server, Informix. I do a lot of things.
So people who I deal with, they ask me the same questions.
Why? Why it happened?
And the final straw was a friend of mine who went to some unrelated
database event and he was just sending me slides how non Postgres
companies are blaming Postgres for MVCC And I was like, there's no response.
So my goal was actually write a piece that will say there is
nobody to blame.
And I hope that message actually got across.
I was actually surprised about the reception.
That's the other thing.
It says the Postgres MVCC is bad, but the goal wasn't to say it's the worst.
Goal will say this is a very difficult topic, that nobody has a universal answer because it all depends on your flow, and those are the examples how can it be done. So actually the question is, because to both of you, you have
experience with Postgres, so how did that article actually how did you read it? That's actually something that I'm curious about. Michael: I can go first.
For me, I really liked the framing.
I think I mostly ignore titles because I know titles often have to get people to read it in the first place. So I didn't feel negatively about the title, MVCC is bad. I was more focused on, oh, this is really interesting.
He's gonna actually compare it to other systems.
And I don't think anyone's done that, at least well or in detail, for a long time. And I found it really interesting to think through how are they
what are they you listed four big kind of architectural decisions
and listed how several database systems had made different trade
offs or chosen different strategies on each of those four axes.
I thought that was the clearest way I've seen explained.
I didn't agree with every one of your conclusions on what's good
and what's bad and what should change, but I did agree with the
premise that there are trade offs.
When it comes to database performance and correctness and things,
different systems have chosen different trade offs even when
their goals are largely seem to be the same. If you think about the goals of Oracle, SQL Server, Postgres,
it's not like they're targeting fundamentally different workloads.
It's not OLAP versus OLTPs.
Largely OLTP systems that have much the same requirements.
So it's I found it really interesting how different some of those
architectural choices were. And I think I knew more about the failure modes in Postgres than I did in all the other systems, so I found it really interesting
learning about some of the others.
Nik: Yeah. So in my case, I think you're touching the topic where people have pain, true pain. And it's like we have had the DBOS recently and also the topic
of implementing queues and Postgres.
It's also, like, painful and but still, these topics, they flame huge discussions because there is real pain people experience.
Right?
And first of all, I liked the depth and visualization piece.
Obviously, you used the AI for it. This is great.
I like it a lot.
And comparing to others, the I think the main thing which is
hanging in the air is that Postgres needs multiple different engines. And this is this is demand for it.
Because this approach that we have right now, the only approach, it has issues in some workloads. And it's inevitable to hit those issues. Right? Radim: You just said it. In some workloads, we can't be comparing apples and oranges. And you I think, actually, you summarized the goal that I had. I wanted only to point out there is no free lunch.
You have to know what you and this is what I'm doing with the
BoringSQL.
You need to know what you're optimizing for.
And, actually, Nik, your queue implementation is one that actually
started the discussion because now I have to admit I was in a I don't mind using Postgres, I work with them, you did actually the turnover. You changed the bloat situations into a something Nik: Just in case it's not it was not me.
I just repackaged Skype's. Okay. Twenty year old implementation.
I just repackaged it so that you can use it everywhere.
It's very old thing.
Just modernized.
That's it.
So it's not my idea.
Michael: I also think calling it a queue is a little bit misleading.
We talked about this a while ago, but I think it Nik: Well, let's not go there. Michael: But I think it's important because I think actual queues
do need updating and things. Nik: This is a I'm ready. Doesn't need.
Yeah.
I'm ready if Brandur is okay and all maintainers of River are
okay.
I'm ready to swap names because this thing is actually a stream
of events.
It's more like a river than a queue.
So let's swap names.
Radim: But I think we just got right into middle of the discussion
because just realizing the difference between queue and stream,
you need to be able to make that distinction because otherwise you will choose the bad design and you will never know what hit you. And from using this analogy at 3AM in the morning, you will get
paged and you don't know what it is.
So for me this was the case, and this is why I'm doing the Postgres
internals, not because I would believe Postgres is the best in
everything, I just believe for a lot of scenarios Postgres is
a good choice and the more you know about the internals and the
trade offs the better design you can do.
And that's why you have to compare the different postmortems
why Postgres didn't work for people And you can evaluate, you
can understand what are the kind of edge cases, why it doesn't
work.
There are other edge cases where different database systems wouldn't work for the same scenarios. I think the queues is a shared topic.
It wasn't an article, but what I actually did as part of the writing, I did the research how different systems work with the queues, and I actually found it's a surprising topic because all of them have a support to make this happen. So this is one of the things I didn't expect.
We see it from Postgres because we are using it, we are interested
to make it better, but I because I don't live in Oracle ecosystem,
but they have commercial product that effectively does the same. Microsoft SQL Server has a optimization for it. So queues is a pattern that actually causes issues, and they all have to work with it in one or another way. Michael: Nice.
On that note, I think you laid it out really nicely in the
article with these four key areas. I wondered if you wanted to lay out kind of what maybe briefly, what is it what's MVCC and why are these four decisions quite important?
Radim: Okay.
So what I'm trying to do during my sessions, I'm trying to make
people aware that as long as you have a one consumer or producer
and that's all you have, life is easy.
Mhmm.
Effectively, you do everything. If you have a short transactions, there is nothing.
I mean, you can pretty much pick any engine you want.
When you introduce concurrency, this is where you start having
issues because concurrency will introduce this element of doubt,
let's call it, because not everything might end up as you're
expecting.
As transaction time will increase, you will be looking at the
different versions of the data, and the longer you have and the
bigger scale you have, the more problems you have.
And, actually, it was something I think you, Michael, said it
when you were reviewing the your perception of the article, and
you mentioned this.
If we would live in a strictly OLTP world, this wouldn't be such
a big issues.
But this is the thing where people are actually mixing OLAP and OLTP, and they introduce the variables that actually affect all this.
So you can have an extremely scalable OLTP system up until somebody
says, want a report.
Sure.
There are ways how to work around this one, but this is not what everybody will do because first, it's the one report. So they will say, okay.
If will use the same database, and suddenly you have a query
that runs for thirty minutes, two hours, every Sunday you will
lock your, let's say, transaction horizon for hours.
And that will cause summers. So for me, the situation is to make people aware that databases
are there to build for thousands of transactions per second.
And in order to support this, there are trade offs. And I wouldn't call, you know, I'm not there to judge the systems.
I think it would be actually silly to judge a four, maybe five decades of research and different database lineage. I think the database lineage also plays the role. So I'm not going to judge it. I'm more like, let's see how different systems work with it.
And especially painful part is when something goes wrong, what is the recovery mode? So that's why I had to do the four charges. I know I was receiving a lot of a lot of lame photos for charges, at least how they are named, because the original versions and
I wrote it somewhere was like Guy Ritchie's style of storytelling. Because yeah. I'm not sure if you are Guy Ritchie fans. I'm not a, like, a storyteller, but I love love his kind of how
how to uncover things.
So for me, the original version actually had four different,
how to put it, not scenes, but, like, setups.
And that was actually describing how it is.
Yes.
Nik: And then you changed it? Radim: Yes. I changed it. So yes.
Nik: Why? Radim: This is all separate discussion because I'm a non English speaker. For me, even though I love to read, I can't write the way how
I would love to write in English.
I spent like a decade, maybe more, getting editors, then learning
how to do certain type of writing, now LLMs they did completely
opposite, they send it in another direction. So if you go through the subtitles, I try to experiment with it, but it's also a very touching factor for some people. If you actually go through the article, you will see how I start learning how to structure, how to keep the idea flowing and
so on.
It's a lot of work for me. And this article hit every single point. I'm actually happy for it.
I'm happy to admit. It's part of my BoringSQL kind of marketing, semi marketing,
but this one was difficult because I think there were some hateful emails that arrived, even, like, saying how can I they don't let the network databases?
Effectively, they thought it's a slop, so that was actually it
was painful at the time, but it served the purpose, So I'm actually
happy for it.
Nik: So those emails, they are against Postgres or against
article or what?
Radim: Against everything. I received hate from MySQL people. I received hate from people that I haven't written a single line
in that article, I received hate that I think there was one comment
somewhere that I can't know more than one or two database systems, which is ironic because I do work with a lot of database system.
So for me this was I knew what I'm doing when I published the
article, but it was the first time I faced this community response
that was very toxic.
Nik: This is true success, first of all. If you receive such emails, it's good.
It's good. But also community, there's no single community in open source. There are, like, there are many bubbles and so on.
So Radim: But I have to say, I did not receive a single negative feedback from Postgres community.
So people most people know me.
So I think with starts, a lot of people were able to that's why I ask you, what was your perception?
Nik: Yeah.
Let me play negative producer role.
I like first of all, I cannot agree with you, like, we shouldn't
judge.
We should judge. Fresh look and I was smiling when you talk about judging because
this literally, I have right now set up with Claude, and I say offload long running coding tasks to Codex to save on Claude capacity, but don't offload judging judging on Claude.
So I think judging is fine.
It's like scientific approach.
You should doubt everything always.
Right?
And check.
And why not why like, why when I realized how Postgres's MVCC,
like, it was many years ago I realized it's like someone designed
the system thinking rollbacks will happen more often. Rollback optimistic and commit to pessimistic approach.
Because you write to new location, not to old one.
It means that if rollback happens, then data will be in the same
position, which is good for, like, nothing moves.
For example, if we used cluster command and our tuples are in
specific order which is very good for performance, we will be
happy if more rollbacks happen on commits because commits move
our tuples unless it's HOT update. Right? And this is weird. I realized that, well, this is weird. Isn't it better to put tuple right there and move old tuple somewhere else?
Maintaining the layout.
Right?
Radim: But I will I will stop you, actually, because this is
us discussing a what are these?
2,026 systems for the last ten years.
We live in a high transactional times.
But this is the reason why when I start thinking about this one. When I first came across databases, they were connected to a
Windows or some sort of fat client, where literally when you open a form, your transaction started and you did the work.
So I think this is the fundamental difference, because this is why I said the lineage and actually time is what makes difference, because right now I do agree with you, we have database systems which have, let's say baseline 5,000 QPS, rollbacks, unless you're doing maintenance or there's an error 2XA.
Happens rarely.
But at the times when I started, it was like it was not CRMs, ERPs, economic systems, and literally people were individual users that are holding transactions for a long time, and when they did something wrong, it came to some sort of bad scenario.
Nik: Okay.
I started with databases twenty five years ago, and it was HDDs,
no SSDs.
Right?
And physical layout of makes a lot of sense, especially if it's
rotating disk, which is sequentially reading something.
If you start moving your with your updates all the time, system
degrades.
It was known long ago.
Ago.
Long ago.
Alright?
But nevertheless, this approach was chosen.
I just think it's a mistake.
Simple mistake.
People just didn't think, didn't compare pros and cons of like,
let's just go this.
And we know Postgres started how it was started.
It was started as a research project.
Right?
And let's just go this way.
And now everyone is suffering.
That's simple position.
Right?
Michael: I agree with you, Nik.
I think and I think Radim agrees as well in terms of right now,
in most systems we look at in a healthy state, it makes tons
of sense to optimize for commits over roll backs.
But I hadn't considered before reading Radim's article the other
side effects of that design, like the MySQL DBAs or Oracle DBAs
that have a runaway update query updating a lot of rows.
Maybe it's already begun for an hour and then they need to cancel
it because it's causing issues.
We've all had to cancel long running queries that are causing
issues.
Right?
And then it takes just as long to cancel or it takes another
hour to cancel.
That is just an issue we don't have in Postgres.
So that's a really interesting difference.
Like, whilst rollbacks might be rare, when they're important,
they can be very important.
So that was a really interesting, like, trade off that I hadn't
thought about before.
Nik: This is exactly what I wanted to discuss.
Yes.
Michael: Okay.
Nik: But still yeah.
Still I wish tuples wouldn't be moved by updates of that,
like, all the time.
There's just a lot of headache all the time.
This is why we talk about partitioning attempt to just to contain
them to smaller places and so on.
Partitioning is important and so on.
But still, yeah, I understand.
And SQL Server also have I I've heard about headaches they
have, which is explained by design there as well.
Yeah.
So, anyway, okay.
I have another thing to criticize your article.
You talk about long running transactions, but somehow you forget
about long running transactions happening on replicas on standbys
and abandoned logical replication slots not used to applications
like catalog xmin horizon or abandoned prepared transactions
and so on.
Right?
Is it just things that need to be polished in your article or
what?
Radim: I wouldn't be writing article in that case.
I'll be writing book about total.
That's the I already have this
Nik: I cannot agree with you because this is exactly what
we see with customers.
They like, the case when you have long running transaction
on the same server, it's like it's a simple the simplest case.
But they hit problems when, like, everyone is playing with logical
replication these days too much.
Right?
We just, like actually, Postgres should have something better
in this area.
Like, it's abandoned slots.
It's headache all the time.
And I'm proposing mentioning them in one paragraph.
Radim: Funnily enough, because I do work on I will continue
with my Postgres internal series.
And I spent a lot of on around the page, what happening on the
physical page.
The next one is the replication, the WAL, how it affects things
like this one.
So you see I'm building the case because I'm not looking at this
one from a I think your position, it might be different because
you are effectively talking about top 1% of the edge cases.
What I'm talking more is the articles, not this one, but the
internal storage internal articles are for people who don't know
it.
Surprisingly, a lot of people even like a staff engineer and
architect level, they usually like that knowledge.
So this is my this is my case for building such a case.
So I can't actually overwhelm people because if I would just
stack everything into a one piece, people wouldn't understand.
I even have problems, and those are not a hypothetical cases,
when I read the feedback of audience during the sessions, you
would be surprised how many people might be using a Postgres
for a decade, and they are not aware, they might not understand
HOT updates.
They might not understand change
Nik: deal with them all the time, with back end engineers
who, like, we always try to find the language to explain difficult
problems in an easier way.
So I cannot agree with you about one percent at all.
And moreover, I think it's like few months ago, we started this
discussion on this very podcast with Michael that it's misleading
to say long running transactions are harmful.
Your actually, your article explains it.
Repeatable read versus read committed.
Read committed shifts xmin horizon.
Right?
If it's like statement level.
While statement lasts, xmin horizon is blocked.
Statement finished, we switched.
So shifted.
Right?
So saying, oh, what happened?
You have a lot of block.
Maybe you have long running transactions.
We heard about these decades, and I'm trying to say, like, we
should just stop saying that.
And we should start we should introduce xmin horizon concept
to everyone.
I know it's, like, maybe not trivial, but saying long running
transaction is the, like, wrong thing completely.
Radim: I do understand what you're saying, but I would still
I would still challenge it on understanding because if we said
about HOT updates, okay, this is Postgres specifics, I'm not
very optimistic about actually people understanding the transaction
isolation.
That's the other thing.
I'm just working with what I have.
What I hear is we can have a discussion, and the humble part
is I wouldn't actually say bad decision back then.
I think decisions that were made, and we're paying tax of it.
We can optimize it by understanding, but we need people who don't
have full understanding to work much better with it.
Because you can design a perfectly working system even if it
if I would accept the premise that Postgres MVCC is really
bad and mistakes were made, you can still work around this one
because I think three of us, we know what are the limits, we
would design it in that way, and this is effectively all the
reviews I'm doing are about the same.
Like, you can't do this one, do it this way.
And then I face the people who have actually they have no interest
in databases and they ask why.
So I think the balance is I'm looking at it, I'm not trying to
change the world like by saying I wouldn't even dare to challenge
like Postgres.
And I'm actually, I have stories from the hacker newsletter from
Postgres how sometimes it's very what's the word?
Humbling to understand all the decisions that were made over
a long period of time.
So I'm actually avoiding that area.
I think this is the one area that I'm trying to avoid because
it might be the same one as other rewrites, you know, AI rewrites.
It only needs to be proven over over time.
I'm not saying Postgres is best, I think Postgres has a reasonable
default and a lot of engineering that went to most of the cases
if you are aware how they work.
So this is effectively my position.
Michael: Nice.
On the charges that you listed, which do you see affecting people
the most, or which do you see people getting confused about the
most?
Radim: I think understanding versioning is there is not a one
charge which would be most dramatic.
I think it starts with versioning.
As soon as you have a multiple version of the same data or similar
data, people get confused over time and that's where the
problem start.
We are right in the middle of MVCC.
Versioning is actually what makes it difficult.
We can discuss whatever it's because people don't get maybe
the full impact of isolation levels.
Mhmm.
But this is something they have to work with.
Specifically to Postgres, and that could be a one thing that
I would love to have improved from the other systems, is indexes.
I actually and I said I won't criticize Postgres.
I don't want this to be like a attack on Postgres.
But I don't like, you know, they are pointing to ctids.
Michael: Mhmm.
Mhmm.
Radim: Because index bloat is if I would say that's 70% of
problems I'm dealing with in performance because index bloat
is there.
And the other article, again, it's always the it's always the
articles that are slightly touching the boundaries.
The vacuum is lie.
That was another one.
And you wouldn't believe how much how much discussions I find
online still months after it was published.
And there are trackers mentioning it and only people realizing
it.
That's why it's slow because they don't even consider there could
be a problem with indexes.
So this is where I believe pointing at the primary key or some
ID which is fixed on the row is actually better designed because
then you don't have to do if you have a over index table, which
is common issue, you don't have to actually do maintenance of
all of them.
Nik: And updates are faster.
Radim: Yeah.
Updates are faster.
Exactly.
Michael: In the spirit of your article, thinking about why don't
we do it, there is at least one cost right every read on that
index, which could be a lot.
Right?
If you think about systems that have a lot of reads for every
write, every single one of them is having to do one more hop.
Right?
Like, going to the logical pointer and then to the physical.
So there is a cost, but that cost is so low, especially nowadays.
It's one hop extra, and it'll always be one hop extra as far
as I understand.
So it's a reasonable cost and it's a fixed cost that we're willing
to pay on every read, but there is an additional cost.
Radim: Yes.
I think that pretty much nails it.
For me, it's just as I said, I don't judge.
If I would have a wish list, this would be a direction.
If we would go there, I would be happy because it would make
a lot of people understanding systems easier and that it would
behave as they expect.
Because as I said, it starts with the versioning.
So now you see how it's connected.
If you pass the storage internals, you can get your head wrapped
around the page and how it's moved.
But when the indexes come, suddenly they don't understand, and
then you have to start understanding one why there is a gap and
so on, and then it goes to the HOT update.
So certain problems can be mitigated, and again, this is my personal
idea.
I actually love the design.
So if there is one point from that article and from the database
systems I ever touch, this is the design I prefer most.
Michael: Mhmm.
That makes a lot of sense.
Nik, if you had one thing, what would you want?
Nik: I would I just want I would want OrioleDB to succeed
so we could choose storage engines and have a lot of good stuff.
This would modernize Postgres a lot.
So I'm reaching for that project a lot.
And I keep and I keep criticizing every piece of Postgres.
This is my nature.
I cannot stop.
So yeah.
Michael: Just to just maybe too much of a tangent, but you brought
up Oriole or OrioleDB Radim, and I wondered if you had thoughts
on the table access method that they that systems like that would
need or a new storage engine would need.
Do we have enough yet in Postgres for them to succeed?
Because I think Oriole still need patches for their implementation.
And I just get the impression it's not ready for an extension
to do it, for sure.
But do we know where we're up to there?
Radim: Okay.
I do agree.
Let's start with this one.
Table access methods, I think it's amazing way.
One of the things that is worth linking to this article is this
Postgres in ten years presentation that was in PGConf Germany,
and it has improvements.
It mentions couple things like zheap, that's the project on its
own.
I think that's now stale anyway.
But table access methods are listed as the number one improvement
that made it.
Based on that, I thought I would be clever, and I thought I would
explore, but I hit the limits quite fast.
It wasn't for anything production.
It was just for experimenting, but I found the limits quite quite
fast.
And if I'm able to do that, I think that actually is the is the
feedback.
Because if a developer of my kind can hit the limits, it's not
going to give us OrioleDB kind of level support we need because
there's much more.
There are things that I'm probably not aware.
I think indexes are generally problem.
I think they are not touched at all.
That was funding I had because I had some crazy idea about indexes
and figure What out what get
Nik: do you mean not touched?
Radim: I think there was some limitation.
I would have to dig because it's a couple months back.
I played with it, I had some issues finding there was no interface
I couldn't find in the table access method that I can use.
Nik: Okay.
Yeah.
There are many improvements, like deduplication in thirteen, fourteen
or we see these things.
There is an ongoing Google Summer of Code projects I'm rooting
for a lot as well, which is trying to implement merge for B-tree.
This would solve many cases with bloat, actually.
Michael: Yeah.
But this is on the this is on the regular.
This is, like, the current implementation of the heap.
Right?
Yeah.
Nik: Many directions things are trying to happen, but it's
there are, of course, obstacles.
And, yeah, it would not eliminate the problem of xmin horizon
being blocked and vacuum cannot delete that tuples, but merge
in B-tree could make B-tree automatically healing after multiple
cycles of vacuuming.
So because right now it can only split page and grow in size
and that's it.
And not necessarily.
Empty if in the middle of it some pages fully empty it can
be removed.
autovacuum will But remove if it's a little if it has little
at least one link to a tuple, this page won't be removed.
That's all.
And for int8, for example, a bigserial or something.
Right?
Or UUIDv7 will always insert on one side.
Like all the pages, if you delete rows, they hang a lot of empty
space.
Yeah.
And they cannot reuse this space cannot be reused.
This is the interesting piece.
So reindex concurrently is inevitable.
Also, that's great that we have Repack concurrently in Postgres
19.
Yeah.
Which is very big gift, think.
And we had the episode about that.
Where in the end of episode I discovered it's going to happen
in '19.
Soon.
Right?
We didn't know.
I was surprised then.
We didn't know it will be brought to core.
We just talked about pg_squeeze.
Right?
So, anyway, OrioleDB is a very big project.
Right?
Radim: I wouldn't say I'm rooting for it.
I think it's exciting project.
I think that's the word that needs to push database development
forward.
And, yes, I'm very excited to see it.
Nik: I dig a little bit of history, how the decision was
made to put newer versions, new tuples to new positions.
It was made originally in eighties.
So there's an article, the design of post the Postgres storage
system from '87 VLDB by Stonebraker and others.
And the decision was like if you maybe know the time Postgres
had time travel capability.
Right?
Radim: That's what I was about to mention.
That logic because And that Yeah.
Nik: Look, follow my hands, so to speak.
So the trick was the idea was let's not have WAL.
Nothing to undo, nothing to redo.
They just keep everything, and then vacuum was born as a
as an archiver.
Right?
And then it stayed like there until in 1999, Vadim proposed and
implemented a change to get rid of table level lock because there
was a table level lock.
So table level lock was removed and the concept of snapshots
of data was introduced.
And a couple of years later, WAL was implemented by Vadim.
So now we have and then the time travel was removed.
This capability was removed.
Now we have a piece of thing which was designed to not to have
of WAL.
Right?
But we fall in snapshots and everything.
This is very interesting evolution.
Right?
And we have all the rights.
And my position is we have all the rights and many people have
all the rights to criticize this state.
But also, it's obvious that it's extremely hard to implement
new approach in the life rapidly evolving system, has releases
every year.
And then also EnterpriseDB attempt zheap was not succeeded
hasn't succeeded, right, in this to implement a new storage.
Yeah.
But anyway, we I think people have all the rights to desire the
choice inside Postgres.
Because, actually, the people who desire the choice, they want
to stay inside Postgres usually.
Right?
They like everything else around.
Right?
Radim: So And I'm actually happy that you mentioned it because
for me this is the lineage, database lineage, where we came from,
how we got to that decision.
That's why I mentioned those systems like different transaction
profiles and everything, because for me those are decisions that
were made over time.
I think I mentioned it I'm not sure, but I'm mentioning it quite
often.
For me, the painful scenario, because my official title when
I'm presenting myself on the conferences, is skeptic turned believer.
I was the guy who destroyed databases because even Postgres,
I mean, Nik said it.
Spinning disk, you need to shut down something, and it writes
and writes.
You hear that loud noise of spinning disk writing something,
or you're hitting that query, killing, doing a rollback, and
it needs to write a lot of data.
And funniest thing is actually the change is a difficult one.
So people from the SQL Server, they actually this is not something
I know, but they actually faced the same problem because they
had a problem with the tempdb, I think it's called.
So it ran out of disk space because there was simply nothing.
Nothing.
So they spent actually, Microsoft did an investment on ADR, if
I'm mentioning it correctly.
So that was that was 2,019, and it's actually a accelerated database
recovery.
And what they did behind the scene is they also introduced something
like a BayQ, because what they needed to do is move where the
data for the log are stored, so it's now on the tablespace,
it's not a different one, so we can go back to a tablespace
discussion behind the different databases.
Even today in Postgres and Oracle, I'm not sure about other systems,
you can work and optimize a lot about tablespaces.
But Microsoft had to do actually a lot of engineering work to
get this one.
So that's why I'm saying I'm disagreeing and agreeing with you
at the same time.
It's extremely difficult to do so.
I can't even imagine how all the core committers find agreement
on doing things like this one.
Postgres is decentralized project and I still don't know how
they do it.
Like, how they find that agreement, that's one thing.
But the point I'm trying to make is, you just said it, we've
been through a series of designs and a lot of difficult choices
we made, but that's also case for the other systems.
So everybody is trying to fight, and if there was one goal of
that article was saying, literally, there's no free lunch.
You have to understand all the moving systems.
I'm now playing with the MLOG on Oracle.
It's also not a free lunch because you move a load to producer,
so you effectively hit the application with that cost.
We can go another article I'm still working on is views, materialized
views, all this.
And there is always so many different different aspects that
needs to be considered, and this is why I'm not trying to judge.
So if you ever read something that I'm judging, I don't mean
judging the AI output, for me, it's always the humbling history.
So that's why I agree and disagree with you at the same time.
So I believe Postgres has a chance to improve.
It will improve.
Projects will improve.
I actually love how Postgres is able to not be ready for one
change year one, and then year plus four, five, you suddenly
have the technology available and still the design decisions
that were made long time ago, they still apply more or less.
So I'm actually trying to find us a balance.
That's why I said I wouldn't ever criticize that decision.
Decisions were made.
And that's the story for me.
Nik: So I agree.
There's no free lunch, but also I want variety of lunches.
I
Radim: do agree.
I would just add because the previous episode I was on this podcast
was RegreSQL and that for me I'm still working on it, by the
way.
Just the learnings I'm getting to understand, sometimes it happens
on the post in the hacker newsletter.
I just the amount of decision and the knowledge that goes through
committers' heads is actually amazing wealth of knowledge that
because I've learned not to act on questioning, but they usually
convince me they know why it was made and I need to respect their
decision.
So this is effectively my background.
Nik: What are your next plans in this direction?
Like you said you said you are working on internals.
You plan to visualize more.
Right?
Try to present internals of Postgres in a new way and so on.
And what's the plan overall in this area?
Radim: Overall plan is expanding the understanding because
I think people still need a simple way how to understand.
I actually love the simplicity kind of.
But for me, this is, like, different approach.
So while I like it, I don't imagine showing this one to a random
attendee of my sessions.
They would just
Nik: Simple already.
Yeah.
It's not simple.
Radim: It's not simple.
That's the thing.
I still try to make things boring and reliable.
AI is playing big role, so I do a lot of work on that harness
level.
So, for example, one thing, just to mention this one, outside
of commercial things, the RegreSQL I'm trying to and I'm actively
testing for now a couple months, a regression harness for Postgres
itself.
It found one bug before it was it was merged.
That's the one of the things.
When you get to commitfest level of reviews.
I'm running, that suite, but I'm usually finding false positives,
so this is where I'm still working in.
I'm still very deep into the statistics ecosystem and how to
enrich it.
I have some new ideas how to make the whole regression testing
faster, but effectively for me, I found out that this is such
a difficult topic, we can discuss, but to get adoption even for
open source tool is actually very difficult.
So I actually took a detour and I'm working on I have an MCP
called DryRun, which finds with other MCPs, which is not the
point.
The full circle is actually to make people get the setup,
they can actually do the regression testing and actually get
LLMs producing code that is judged not by LLMs, but having more
predictable output.
But this is a personal project of mine.
Otherwise, the education still stays, so I'm expanding.
And I'm preparing I think I have now 11 articles in words because
I changed my writing model.
Because, like, when I started, I didn't know how to write for
a blog.
So it was like all energy into a one blog post and then I published
something that didn't have a structure and everything.
Now I have to if I have idea, I do some writing, then I leave
it, I come back to it, and I'm trying to avoid kind of the structure.
So actually, the I actually had to redo some of the parts of
website because now you see, like, there's an internal guide
that actually lists chapters.
It looks like more like a book, and that's why I said, I know
what's the cost of writing a ever expanding articles.
Believe me, I have one article which will go live.
It's not educational one, and it's already read time about forty
five minutes, I can't cut it down.
So this is actually stuff we are talking about, and it's about
four months of work, which so, yeah, what I'm working on.
Nik: Cool.
Cool.
Okay.
Good.
Michael: That sounds amazing.
That sounds exciting.
I can't promise I'm gonna read the 45 one, but I will definitely
try.
But, yeah, it's always fun reading what you put out, and thanks
for fighting the good fight on the education front.
So, yeah, thanks for coming on as well.
Radim: Thanks you for having me again.
Nik: Have a great week.
Bye.
Radim: Okay.
You too.