Nikolay: Hello, hello, this is PostgresFM episode, I don't remember
the number, and today... I'm Nikolay and this is Michael.
Hi, Michael.
Michael: Hello, Nikolay.
I think 85.
Nikolay: Well, honestly, it's already at this point, it
doesn't matter.
Michael: When we get to 100, it will matter.
Nikolay: Yeah, remind me, wake me up.
Yeah, at 100.
So today I'm in the passenger seat and Michael is going to talk
about, I will join of course, but from the passenger seat, about
why the planner behaves not as expected, not choosing index I
have prepared and how to troubleshoot that, right?
Michael: Yeah, yeah, this was my suggestion.
So it's something I see a bunch and I got reminded again yesterday.
I happened to be on LinkedIn of all social networks.
It's surprisingly one of the slightly nicer social networks to
be on these days, which I would have been shocked by a few years
ago.
But I saw a post from Brent Ozar, who I remember from my Microsoft
SQL Server days, and has been recently getting into Postgres
things, which is cool.
I think it would be a great addition to the community.
And he posted what I thought was going to be a bug post, and
he cheekily said, why isn't Postgres using my functional index?
And it turned out it was actually a Stack Exchange question that
he had posted, and he had included links to an Aurora database,
which, not Postgres, but Postgres compatible, which is fun.
And he was genuinely asking why isn't it using my index?
And I'd noticed it was only posted like sometimes on LinkedIn,
you see posts from like 3 weeks ago, but it was only posted about
20, 25 minutes ago or something.
I thought, 'Oh, you know, there's no answers to this.
I, this is something I generally know about.
I've written a whole blog post on.'
So maybe I can help here.
I had a look.
And it just reminded me that this is a remarkably common issue
for beginners for a bunch of reasons, but even for some quite
expert database users, sometimes get confused as to why they've
got an index that Postgres should be using or they think they
do.
And for some reason, Postgres either can't use it, and they're
expecting it to be able to, or doesn't think it will be faster
and is choosing a different plan.
So, yeah, I've seen this a bunch of times.
I feel like there's probably about 10 different reasons it could
be, which is I think slightly, like, people don't realize quite
how many reasons there could be for this.
So I thought it might make an interesting discussion.
Nikolay: Yeah, well, yeah, I remember this struggle.
Now I know how to troubleshoot it, but I remember the struggle
I had many times in the past.
It's not easy sometimes to understand why a planner behaves in
one way, not as you expected.
And sometimes you just, it upsets you a lot and you start thinking,
oh, it's stupid.
I saw also people started blaming Postgres a lot after such events.
They think like, oh, planner is like, I want control.
Also, this is where you probably want to start wanting this hinting
which Postgres lacks, right?
Well, it has it, but it's not like standard.
Michael: Yeah, I think we probably will get to that.
You said you now know how to solve it out of interest.
What's the first thing you'll do?
Nikolay: Well, first thing I'll do, I'll try to understand, okay,
This is the moment, usually I don't pay attention a lot, but
this is the exact moment when I start paying attention to cost
metrics in the explain analyze buffers plan.
So the idea is I try to understand, is it really a huge number
or like it's relatively small or tiny?
And then I think, okay, based on my experience, and here actually
it helps if you have some experience, right?
Based on my experience, if my index was used, what would be this
cost, right?
And then, like, basically I already explained my algorithm.
So, okay, I see some cost.
I wanted to say actual cost, but the word actual we must reserve
for execution, not for planning, right?
But this is the cost you see in the root of this tree of query
execution.
Actually, we don't need to analyze here, right?
We just say, explain and it's enough.
Michael: For the total cost that you're talking about, I guess
you're talking about, they're called startup cost and total cost.
I guess you're talking about the total cost on the root node.
Nikolay: In the root, yeah, yeah.
We just, like, we have the plan, we see our index we hoped would
be used, it's not present there, and we see different index or
maybe sequential scan, and we think, oh, what's happening here?
And so I see the total cost, probably also check the successive
method node where data access is happening, and the cost there
as well.
It's also important.
And then I think, oh, if my index was used, what would be the
cost?
And since I'm experienced, I'm sometimes already understanding
what's happening.
But if I have doubts, I just start playing with these rough knobs.
enable seq scan, enable index scan, enable blah blah, to put
penalty on specific access methods, right?
Or so, to specific steps.
Michael: Yeah, scan types.
Nikolay: Yeah, Right, scan types.
And in this case, very often I start seeing what I expected and
I see, oh, like, this would be the cost.
And I see the difference.
And sometimes it's very small.
So we were very close to choose my plan, but it was not chosen
just because...
And then super often, very often, it's a new system and I ask
people, what random page cost do you have?
Because sequential scan, our index is not used.
Oh, and the random page cost is 4.
Recently, some Amazon guys, they got consultation with me and
this is exactly what happened.
I asked what are the random page costs?
They said 4.
Remember, Crunchy Bridge, after listening to us, changed it to
1.1, and they also published good benchmarks, and it's still
in my to-do to revisit those benchmarks because I had a tendency
to set random page cost to 1, but they said 1.1 is better, actually.
According to our benchmarks, and they had some methodology interesting,
but it's slightly off topic.
So, for those listeners who don't know what random page cost
is, is how the planner thinks about random access to data.
And by default, it expects you using magnetic disks, rotational
disks.
And this is not normal in 2024, definitely.
And I don't know if Aurora and RDS still have 4.
That cluster may be created long ago, but if they still do have
this, This is another question to them.
We had the question about CPU and green color in performance
insights last time.
But today we have a new question.
We need to check.
I haven't created clusters for long.
Yeah.
Michael: Last time I checked, most of them were.
There were a couple of good exceptions like ScaleGrid and now
Crunchy Bridge but there aren't many that have tuned it last
time I checked which was admittedly quite a while ago.
Nikolay: This is super strange and also Postgres community.
It's time to revisit this and we need this for mostly for new
clusters.
You shouldn't think too much about old clusters, they already
have postgresql.conf in place and everything there.
So, change it.
Michael: And I would say, why are we optimizing for the few people
that are still running magnetic disks instead of the vast majority
who are running on SSDs.
It's time to change, I think, for the default.
Nikolay: I might be, again, taking too much time for a passenger,
but let me just, I just feel this that maybe some people don't
understand what we are talking about.
I will be very short.
So there is a seqpage cost and random page cost.
Seqpage cost means sequential, the cost of sequential data access.
And it's 1.
If you change it, you change scale.
I have cases when people change it.
But it's 1, it's our baseline.
It's 1.
And random page cost by default in Postgres is 4.
It expects that random data access is 4 times more expensive
than sequential page access, which is not so if you, for example,
have all data fit in memory, cached, or if you have, if it doesn't
fit, but if you have non-rotational disks, SSD, NVMe SSD, or
something like that, modern disks.
And most of new databases have modern disks, of course.
And Aurora, I think they have very good storage, so obviously
it's not rotational.
I think maybe I'm wrong, actually, right?
Should not be rotational.
It's very slow.
I mean, throughput is terrible, right?
Okay.
So, and why does it matter here?
Because if you have 1 and 4 default settings, 1 sequential and
4 random access, it's 4 times more expensive.
The planner quite often thinks, oh, sequential scan is not that
bad.
I would prefer sequential scan over index access if I need to
fetch a lot of rows, because it seems to be cheaper for me.
Once you shift it to normal 1.1, 1.1 is normal these days.
This is common understanding.
And Crunchy Bridge have good benchmarks.
Let's attach the link.
So in this case, you tell the planner, actually, sequential access,
random access, they're almost the same.
And index scan starts winning.
Yeah.
Michael: Yeah.
Or at least it's only a 10% penalty instead of a 300% penalty,
which is a huge difference once you start returning a bunch of
a lot of rows for a single.
Yeah.
Which happens quite often.
There's quite a lot.
Like, anyway, I'm very happy for you to do a bunch of talking
here.
It's a pretty common topic.
I like that we've gone this deep so quickly, but I do want to
go, I think it's worth going back and saying, this is a great
avenue for like, like you mentioned looking at the costs, but
I think there's an easy, like even for non-experts, you can,
you can use that to your advantage, right?
You can, if you have access, I did, I didn't, when I was looking
into Brent's problem.
It was a read only user, so I couldn't do enable set scan off,
which is the easiest way if you're getting a sequential scan,
flipping it to, well, basically telling the planner that sequential
access is tons more expensive than it really is, and therefore,
if it has any ability to use the index, no matter how expensive
it should be, it will then use it.
If it can't, though, enableSeqScanOff doesn't actually prevent
the planner using seq scans.
It just penalizes it.
It makes it look more expensive.
So if you cannot use the index, which is a bunch of these cases
are Postgres can't use the index for some reason.
Or you've forgotten that you're in an environment that doesn't
have the index.
Like that is a surprisingly common
Nikolay: and valid index.
Michael: Yeah.
Well, Yeah.
I haven't seen that 1 as often.
Like I haven't seen it come up as often.
But quite often people are in the middle of debugging something
and someone's dropped that index or they're on staging and it
doesn't have it when they went on production, they do have it.
So it is worth double checking the index exists, but by using
enable seq scan off, you can quickly check, is it a costing issue?
Now it's trickier with if you're in the case you mentioned where
it's using 1 index and not the 1 you're expecting, you can't
use that.
You can disable different scan types, but you can't tell it not
to use a specific index.
I have seen a trick for that though, which I wasn't familiar
with before, but Haki Benita shared on a blog post.
He does a begin transaction, drop index, the 1 that it was using,
explain the query, and then roll back, which means you can try
and see if that index didn't exist, would it pick my index?
Which is quite a nice trick as well.
So trying to find out, are you in the case where Postgres can't
use the index or you're in the case where it's choosing not to?
The enable parameter is a really nice way of finding that out
quite quickly normally.
So yeah, cool that you start there as well, even if you're eyeballing
the costs.
The one thing I would add on that is, I think explain analyze can
be helpful.
So if you've run that already, or if your query actually runs
and doesn't time out, then the explain analyze part is helpful
because you get the rows returned, and I think when the cost estimates
are off, sometimes it's random page cost, but quite
often it's about the number of rows Postgres expects to be returned
and the number of rows that are actually returned.
So, in fact, maybe you don't need to analyze because if
you know the query, you know the data, you know the query you're
running, if you're expecting only a few rows to be returned and
you see hundreds of thousands in the estimate, you've got a pretty
good clue right there that you're in a case where it's overestimating
the cost of that.
Nikolay: Or underestimating maybe.
But yeah, underestimating wouldn't lead to the index not being used,
but anyway, it can lead to wrong plans.
And yeah, that's a good point.
So, this is the reason number 1, let's say.
First reason is like costs are slightly different.
But this is already talking about the most difficult
case, probably.
There are easier cases.
I just excluded them from my mind because they are easy.
If the cost is off, and indeed, we check the planned rows and actual
rows, if the mismatch is huge, either it's outdated stats, probably
somebody is blocking the autovacuum or something we need to check,
or maybe lack of stats.
Sometimes we don't have stats at all.
It happens.
For example, if you created a functional index but you didn't
run analyze at all yet.
Michael: That was the case yesterday.
Nikolay: In this case, explain analyze buffers also is good.
I agree with you.
But sometimes you have a query which lasts hours.
So, in this case, we need to downgrade to just a regular explain.
Michael: Well, this is one of those, like, I'm obviously a big fan
of explaining those buffers like you, but this is one of those
cases where I don't think you can know, like, maybe you can never
know, but I don't think you can know from a single execution
plan which case you're in, we often need at least 2.
We need the before and the after we've changed one of these parameters.
The first one is not going to tell us that much.
It tells us it's not using the index, but it doesn't tell us
why it's not using the index.
We could get some clues, like row estimates being off or like
the cost number, but they're only they're only clues like we've
only by changing something and running it again that we can see
the difference of when it is using the index.
Nikolay: That's why database branching and experimentation with
iterations matters so much because if you already calculated the
statistics, but then you start having questions, what if I did
something different?
You want to reset and go a different route, different path, right?
In this case, being able to iterate, like reset in a few seconds,
run it again, check this idea, check that idea.
That's why branching and fast cloning matters so much, right?
Exactly.
This is, yeah, because otherwise it's a one-way ticket.
So you already calculated statistics and that's it.
Michael: Yeah, true.
Actually, you can't go back.
You can solve your problem, but you can't go back.
Nikolay: And sometimes you, you, okay, you solve the problem,
but you cannot explain in detail what happened.
But this doesn't build confidence in your team if you cannot
explain, right?
And to explain, you need to go back.
Yeah.
And understand better.
Michael: Yeah.
I like that you're using explain in a different context now.
Nikolay: Okay.
But it's related, right?
It's related.
Michael: It is.
It really is.
And then you can analyze what you're going to do next.
Nikolay: Right.
Analyze what is overused.
Michael: Yeah.
Well, that's exactly what happened yesterday as well.
I didn't well, there's an interesting thing here.
I jumped to conclusions a little bit.
I did spot the it was it was probably an estimation error, but
I thought it might also be a data type casting issue, which is
the kind of thing that we didn't cover that in depth, actually.
We covered it in my
Nikolay: Tom's mismatched query, right?
Let's call it mismatched query, maybe.
Michael: Tim, yeah.
So, basically, if your query can't be answered by the index you
have.
So, like, the typical example of this is a function.
So, for example, if you're running lower on an expression
on a field, but you have only the field index.
And you might think logically that it could use the index.
If, for example, in your...
Date.
Nikolay: Huh?
From timestamp to date, your reduction from timestamp to date,
for example, it's very common.
You convert timestamp to date and expect that an index on timestamp
will work.
It won't.
Right?
Because it's for timestamp, not for your expression.
Michael: So, yeah.
So, in some cases, Postgres handles some of these data type conversions,
like text to varchar, which I wasn't sure about yesterday.
But in some cases, it doesn't.
So, it depends on whether it supports that.
But in general, that's the kind of thing that can prevent the
use of an index that you're expecting to be used, or like the
operator not being supported.
There's other cases where the index type you, so let's say you've
used a B-tree because that's what most of us are using most of
the time.
One example is on a text field using ILIKE.
So like again, a case insensitive search operator is not going
to use.
The B-tree index only supports certain operators.
And if you're using an operator that it doesn't support, like
a greater than on a hash index, for example, or anything other
than equality on a hash index, it won't use the index, kind of
obviously.
So I think there's a few cases that are quite simple for why
it can't use the index and then there's a bunch that are a bit
more complicated along the lines you were talking about which
is what I've just classified as Postgres doesn't think it will
be faster.
So yeah, we've said expensive and cheap a few times, but those
costs, while they're in an arbitrary unit, the idea is for them
to estimate how fast the query will be, or how slow it will be.
The higher the cost, the slower it would be.
That's the idea.
It's not the only measure that Postgres could have tried to optimize
for.
It could have tried to optimize for IO.
And that's correlated.
But it's not the same thing.
It optimizes for speed.
Which is, yeah, interesting and kind of leads us to what could
have gone wrong.
So I actually think we didn't cover a whole case, though, like
a whole simple case of it doesn't think it would be faster, and
It's correct.
That's a case that catches people out a bunch.
Like you're trying to force it to use an index, but it's doing
the faster route already.
Nikolay: For example, if you read 95% of your table, just a sequential
scan might be much faster on the fly filtering out those 5% that
are not needed, than walking on the tree, the B-tree, right? It's
it will be probably...
Even if you balance random page cost and index scan cost,
it still might be better to use a sequential scan in reality.
So,
Michael: Yeah, for sure.
And because of how sequential scans work, because of how efficient
they are, and the fact they don't need to read data from multiple
places.
And if your data fits really tightly on like very few pages it
could be a lot faster and not even not even just at percentages
as high as 95% to be honest. Like for for small tables, for some with
like not very wide rows, I've seen it be as low as 30% or so,
that's still faster as a sequential scan.
So I suspect you've come up with a contrived example of it being
even lower.
Nikolay: Right, so in this case, I have various, like simple,
very basic questions.
And I think we should, like, in many cases, we just should start
from this question when we consider the performance of a query.
We should think how much underlying data do we have, and next,
very high level, how many rows return?
Then it can be like, we can unfold this and say, okay, if it's
just one row, was it an aggregate?
So we need to analyze a lot of rows for real, or just we return...
Last week I had a case, people complained about very slow, like
very bad behavior and they used explain with buffers, they saw
like some gigabytes of data and so on.
And they knew, ORM or GraphQL was involved, I don't remember,
but they knew that they need only like not a lot of rows.
And then we just saw that, like, oh, it's bad, it's bad, it's
bad.
And the question, how many rows are we actually returning?
And they also like, we created all the indexes.
The indexes are here, but somehow it's still sequential scans.
But how many rows?
And we thought like, oh, actually it was like 10,000 rows or
so, maybe 100,000 or 25,000 rows.
I don't know, a lot of rows returned.
Do you really need it or did you just forget the limit?
So the limit is applied on the client side maybe, right?
It's terrible.
And then I said immediately, like, I'm a big fan of using, Again,
maybe off-topic, but I'm a big fan of using, in larger projects,
not in tiny projects when you care about everything in your pet.
So queries for me, like you know this concept, pets versus cattle,
right?
Michael: Yeah.
Nikolay: Yeah, so for virtual machines or for real machines,
for big fleet of infrastructure.
In the case of workload, if the workload is complex, I'm a big fan
of dealing with queries like with cattle as well.
So when I saw this, it was a new client.
When I saw this, that they had this problem, returning too many
rows unexpectedly, I immediately said, stop here.
Like, it's just 1 example of the problem.
And let's return to a high-level, top-down analysis using pg_stat_statements
because it has rows, rows metric, and let's see the
whole picture, how many other queries behave similarly and in
turn too many rows.
But this is exactly when it happens, an index is not used.
Because you request for too many rows here, right?
And selectivity is not good.
It's very weak.
Michael: Yeah, and Postgres is going to do exactly what you asked
it to do.
If you want all of the rows, it's not going to say, let me give
you the first 25 and see if that's enough.
It's going to say, nope, here you go.
Here's all of them.
Nikolay: ROMAN BATURINIKIS Unless it's a special case, which
is slightly uncertain, right?
It's something new for us.
Michael: Right, good point.
I haven't actually considered the new index types.
Nikolay: Yeah.
Anyway.
No more certainty in the SQL world.
Michael: Yeah.
But yeah, there's another case as well.
A big one is if you're selecting a high proportion of the table,
chances are it's actually faster to do a sequential scan.
Possibly a different index might serve your query better.
I haven't seen that one as often.
But another case I've seen really often is small tables.
So even if you're only selecting one row in a table that's like
fewer than 100, which might not be that common in production,
I've seen quite a few like reference tables though that are not
very many rows, or more commonly developer databases on people's
local machines where they just have a tiny bit of data for like
a new feature.
Even if you have the perfect index, Postgres won't choose to
use it when you don't have much data just because it's so...
Because all the data is on one page and it can just simply look
it up very, very quickly, very, very easily.
So small tables are the other exception that I see more often
because of dev boxes.
Nikolay: You know exactly how easy it can be impatient here trying
to interrupt you, right?
This is my favorite topic.
Probably I will not add anything here.
And listeners who follow us for long already know what I would
say here, right?
Because you should always try to deal with full-size databases.
Michael: Yeah, or at least like a
Nikolay: bigger,
Michael: large datasets.
If you're going to have to, yeah.
But it trips people up.
Nikolay: Unless you're a big fan of the hypothetical approach, hyper-pg
and indexed partitioning.
So it also would work, but in a limited way.
Michael: Would it though?
Because if you don't have any data, I don't think...
Nikolay: Ah, no, sorry.
It was only discussed, not implemented.
And there was another project, which like, let's export statistics
from production, import it to a lower environment, and pretend
we have a lot of data.
Michael: That's a cool idea.
Nikolay: Yes, it's a cool idea.
Maybe already...
Hyper-pg has been developed, so definitely partitioning was covered,
but about statistics maybe not, but there was another project
developed in Japan, I guess, and it's not super popular, but
the idea is cool, I think it's really good, but I expect some limitations
of this idea obviously if you can afford testing on full-size
databases this is the best way yeah you just need to make it
cheap and then here I stop yeah
Michael: but yeah this is a so this is the more interesting case.
So right, like once you're past those two of it being, you know,
actually faster, if it's wrong, if Postgres is not estimating
correctly, That's when we get into the more interesting parts
of, like, you mentioned stale statistics or not even having statistics
that are relevant or needed.
So analyze is your friend here on the tables in question.
We've also got a couple of other tools.
We can increase the sample that Postgres will do, if you've got
an oddly distributed-
Nikolay: Default statistics target.
Yeah, some people-
Michael: Not even-
Nikolay: For one column.
Michael: Yeah, exactly.
You can do it globally, but per column...
Nikolay: But 100, default 100 is also quite a lot.
It might not be enough if you have high insert rate and constantly
working on the edge of new data and the distribution of values
in new data is very different compared to the archived data.
So there are things there, definitely.
Michael: That's one of the defaults I'm happiest with, actually.
I think it's a pretty good sweet spot because increasing it,
if you increase it globally, you increase the time for analyze
to run globally.
And I think that has knock-on effects for things like your downtime
for doing certain types of major upgrades and things.
So I
Nikolay: can see...
Major upgrades
Michael: should be
Nikolay: zero downtime.
And if you've jumped from 100 to 1,000, yes, it will probably
increase analyze time maybe two or three times, but not 10 times.
It's not linear.
Michael: Sure, but I haven't seen it cause huge problems
globally.
But yeah, if you've got a column that's like not a distribution,
well in a skewed distribution, increasing it for that column
can really help.
And then the final one I've got on my list was the multi-columns.
Yeah.
Nikolay: Which can be only four columns inside one table yeah it cannot
be unfortunately for two tables, two different columns in two tables.
It would be interesting to have maybe as well. Sometimes one table
fully depends on another, right?
Michael: Yeah, I
Nikolay: don't know.
We
Michael: do have a whole episode on hints, but I think this is
where the hints discussion really comes into play.
It's like, what are your options when we're out of tools within
Postgres to give the planner as much information as we can about
the distribution and the stats.
Nikolay: Right.
Bugs also happen.
So bugs might happen.
I mean, planner bugs or some not developed things, not yet developed
things, some easy stuff which is not yet there, Postgres planner
is not super powerful.
It's quite powerful, but compared to SQL server, for example,
the code base is much smaller and the amount of, the number of
engineering hours invested into SQL Server Planner are much bigger
than into Postgres.
So of course it's evolving, but still some things might be not
yet developed and sometimes people see quite simple things.
To me, sometimes you think, oh, it's so obvious why it's not
here.
And as an example, it's very unrelated, but it still poses.
Last week, statement_timeout was committed by Alexander Korotkov.
Many thanks to him.
And statement_timeout was my idea.
And when I came to this idea, I was thinking, am I stupid?
Like, am I missing something?
Like whole world still didn't raise this.
I searched, I don't see discussion.
Why statement_timeout is not present in PostgreSQL still, like
so many dozens of years of development.
And then a few folks validated it's a good idea actually, and
then it got support.
So if you spend time with Postgres enough, with the Query Planner, you
might indeed see cases which are not yet well-developed.
Or just simply bugs.
Sometimes just bugs happen as well.
Michael: Yeah.
I've spoken to several customers that have long histories with
DB2, Oracle, and they're surprised by some things, but they also
talk about some issues with, like, especially on the Oracle side,
about kind of patchwork solutions.
So sometimes more code in these optimizers is not necessarily
better because it's like patched fix on patch fix on patch fix
and it leads to quite like nasty weird behaviors.
So I do admire the elegance of the simplicity, but equally, I
think you're right people I know that that know these things
well, or have used SQL Server or DB2 in the past, definitely
say nice things about those planners.
Nikolay: Well, my opinion is also based not on my experience.
My last experience outside Postgres was very long ago.
I mean, real experience when you do develop something for long.
It was more than 15 years ago.
So I'm just translating other folks' opinions, but they seem
reasonable, these opinions.
And they are experienced guys who actually came to Postgres and
say, you know, like, let's just admit the planner can be much,
much better.
We need to continue developing things, improving things.
And in this case, let's advertise a recent blog post from David
Rowley.
So I've noticed many items, it was his contribution.
Michael: Yeah, but also many from other people too.
Nikolay: Yeah, so like the post is what's new in the planner
in PostgreSQL 16?
Let's unfold it and so on.
It's great.
It's also in my to-do list to inspect deeper and maybe do some
tests.
So it's a good post.
Michael: And the thing I love most about it was inspiring people
to upgrade.
**Postgres** 16 came out a few months ago now.
We've had some minor, at least 1, I think 2 now minor patches.
Nikolay: February, I have no idea.
Michael: I already know at least 1 company that has upgraded
as a direct result of that blog post coming out, just to see
if some of their queries improved as a result.
Cool.
Nikolay: Cool.
Michael: Good.
So yeah, thank you to David.
Nikolay: This is important to share, by the way, because other
people think like, maybe it's still too early and so on.
And you know, like I recently was approached and just the question,
do you know anything bad about **Postgres** 16?
I said no, actually, it's not, it's not **Postgres** 9.3.
No, 9.3 was terrible.
I remember.
Michael: I think I've got a new answer for you that I think is
more on brand.
I think you should say yes, but they're also true in 15, 14,
13, 12, 11 and 10.
Nikolay: Okay.
Yeah.
Michael: Anyway, I know what you mean.
Did you mean like you don't have anything bad in 16 that's new
in 16?
Nikolay: Nobody bumped into some problems like complaining that
it's bad.
There are a few companies upgraded, I mean, maybe not huge installations,
but still.
It has a good reputation so far.
And of course, the risks are still there, and tomorrow we might
see some problems, but these risks already go down, down, and
down over time, right?
And I mean, it's important to share, so we upgraded, everything
is good, in our case.
It doesn't mean in any case it will be good, but it builds up
the level of confidence over time, and more people will be upgrading.
Nice one.
Okay, good.
Did
Michael: you have anything else you wanted to add?
Nikolay: I always have a lot of things, but I think we're out
of time.
Let's not consume a lot of time.
Of course, it's good if you run more or walk your dog more, listening
to us longer, but still, there are other things to do.
So thank you so much and thank you for support.
We still like, I didn't follow really well, but we see comments
on YouTube, for example, it's good.
Please share your experience as well and other people also see
it and we can highlight some comments not only about Red Bull,
right?
Or beer.
Michael: Free advert.
I'm teasing.
Yeah, I think over a thousand people have listened to the last
few episodes, which is pretty cool.
Yeah.
So thank you all.
Nikolay: Good.
Okay.
Bye bye.