1
0:0:0,06 --> 0:0:2,5
Michael: Hello and welcome to Postgres.FM,
a weekly show about

2
0:0:2,5 --> 0:0:3,58
all things PostgreSQL.

3
0:0:3,58 --> 0:0:6,2599998
I'm Michael, founder of pgMustard
and I'm joined as always by Nik,

4
0:0:6,2599998 --> 0:0:7,24
founder of PostgresAI.

5
0:0:7,24 --> 0:0:7,8599997
Hey Nik.

6
0:0:8,24 --> 0:0:8,94
Nikolay: Hi Michael.

7
0:0:9,5199995 --> 0:0:12,86
Michael: And we have a special
guest, David Ventimiglia, Solutions

8
0:0:12,9 --> 0:0:16,56
Architect at Supabase and the
creator of pg_flight_recorder,

9
0:0:16,56 --> 0:0:17,94
which we're talking about today.

10
0:0:18,08 --> 0:0:18,98
Welcome, David.

11
0:0:19,4 --> 0:0:20,380001
David: Thank you so much.

12
0:0:20,380001 --> 0:0:23,04
Good morning, and I guess good
afternoon and good evening.

13
0:0:23,42 --> 0:0:24,64
It's nice to meet you.

14
0:0:24,84 --> 0:0:27,14
Michael: Yeah, I think we've got
all 3 bases covered.

15
0:0:27,34 --> 0:0:30,32
Where would you like to start,
perhaps, with why another tool

16
0:0:30,32 --> 0:0:31,12
in this area?

17
0:0:31,12 --> 0:0:32,28
What's the origin story?

18
0:0:32,28 --> 0:0:33,18
What's the motivation?

19
0:0:34,08 --> 0:0:37,62
David: The motivation, so as a
solutions architect at Supabase,

20
0:0:37,9 --> 0:0:45,42
my job evidently is to help our
customers and often they come

21
0:0:45,42 --> 0:0:48,06
to us and they say, I've got this
problem.

22
0:0:48,94 --> 0:0:52,46
My database is slow, or this query
is behaving weirdly.

23
0:0:52,64 --> 0:0:53,7
Please help us.

24
0:0:54,58 --> 0:0:58,52
And then we try to bring whatever
tools we have to bear on the

25
0:0:58,52 --> 0:0:59,02
subject.

26
0:0:59,54 --> 0:1:3,28
And to be honest, often we're starting
out with no idea what

27
0:1:3,28 --> 0:1:4,2
the problem is.

28
0:1:4,2 --> 0:1:8,54
And our beautiful customers cannot
be relied upon to, you know,

29
0:1:8,6 --> 0:1:10,46
relay all the information perfectly.

30
0:1:11,12 --> 0:1:13,72
And I just needed more.

31
0:1:15,06 --> 0:1:17,08
I was telling Nik this at 1 point,
you know, some people say

32
0:1:17,08 --> 0:1:17,72
less is more.

33
0:1:17,72 --> 0:1:18,84
I say more is more.

34
0:1:18,84 --> 0:1:21,66
I needed more data and I just wasn't
getting it.

35
0:1:21,66 --> 0:1:24,84
And I did the usual thing that
people would do these days.

36
0:1:25,24 --> 0:1:30,7
Version 0 of this, maybe 4 or 5
months back, it was a rush job

37
0:1:30,76 --> 0:1:32,58
with our good friend Claude Code.

38
0:1:32,64 --> 0:1:35,64
And I just cobbled something together
to get the data that I

39
0:1:35,64 --> 0:1:37,22
needed for a particular customer.

40
0:1:38,32 --> 0:1:41,94
And we got the data that we needed
and we were able to get through

41
0:1:41,94 --> 0:1:44,88
that particular instance rather
swimmingly.

42
0:1:45,06 --> 0:1:48,74
And I was pleased by the outcome
and I thought, you know, let's

43
0:1:48,74 --> 0:1:51,34
try to turn this something into
something a little better and

44
0:1:51,34 --> 0:1:53,24
a little something more real.

45
0:1:53,68 --> 0:1:57,979996
As a sidebar, I think we're all,
most of us are familiar with

46
0:1:57,979996 --> 0:2:4,3
the excellent pg_wait_sampling,
which is a excellent extension,

47
0:2:4,7 --> 0:2:9,52
but sadly is not available on all
managed Postgres services.

48
0:2:10,2 --> 0:2:11,18
Even Supabase...

49
0:2:11,66 --> 0:2:12,48
Nikolay: Cloud SQL

50
0:2:12,5 --> 0:2:13,74
David: is the only 1.

51
0:2:13,74 --> 0:2:19,4
Cloud SQL, but even Supabase,
on whose staff is Alexander Korotkov,

52
0:2:19,48 --> 0:2:23,6
who wrote it, but we don't have
that extension on the platform.

53
0:2:24,32 --> 0:2:28,38
And there's, you know, there's
some resistance within managed

54
0:2:28,38 --> 0:2:30,64
platforms to get new extensions
added.

55
0:2:30,64 --> 0:2:33,88
I mean, there's a rich and vibrant
ecosystem for extensions and

56
0:2:33,88 --> 0:2:38,94
that's 1 of the strengths of PostgreSQL,
but a weakness is getting

57
0:2:38,94 --> 0:2:41,38
those into all of the places where
you needed it.

58
0:2:41,38 --> 0:2:47,06
So I needed to, I needed something
that was a poor person's substitute

59
0:2:47,3 --> 0:2:48,38
for pg_wait_sampling.

60
0:2:48,38 --> 0:2:51,34
That's really how it started out
was just a worse version of

61
0:2:51,34 --> 0:2:53,98
pg_wait_sampling written in SQL
in PL/pgSQL.

62
0:2:55,44 --> 0:2:57,04
And then you know how these things
go.

63
0:2:57,04 --> 0:2:58,82
It just took off from there.

64
0:2:59,54 --> 0:2:59,82
Nikolay: Yeah.

65
0:2:59,82 --> 0:3:2,24
There's a lot of unpack here actually.

66
0:3:2,64 --> 0:3:3,14
Yeah.

67
0:3:3,2 --> 0:3:3,7
Yeah.

68
0:3:4,02 --> 0:3:7,58
And, and I agree with you starting
from the end of your intro.

69
0:3:8,04 --> 0:3:11,52
I keep saying extensions are not
extending us anymore, because

70
0:3:11,52 --> 0:3:14,82
in reality of managed Postgres,
we are limited by only the set

71
0:3:14,82 --> 0:3:17,18
of extensions which are present.

72
0:3:17,3 --> 0:3:21,74
There is pg_tle, and we should discuss
that separately, which

73
0:3:21,74 --> 0:3:23,18
is, I think, a great idea.

74
0:3:24,48 --> 0:3:25,6
Actually, you told me.

75
0:3:25,6 --> 0:3:27,32
It's a lot of happening also.

76
0:3:27,94 --> 0:3:29,52
Let me tell my story.

77
0:3:29,66 --> 0:3:36,14
I think Every experienced Postgres
DBA at least once wrote this

78
0:3:36,14 --> 0:3:41,4
snapshot tool for pg_stat_statements
and other pg_stat views because

79
0:3:41,4 --> 0:3:43,16
it's only cumulative statistics.

80
0:3:43,62 --> 0:3:45,32
Some numbers are growing, that's
it.

81
0:3:45,58 --> 0:3:47,98
We need persistent storage.

82
0:3:48,28 --> 0:3:51,6
Usually for bigger clusters we
have monitoring.

83
0:3:53,0 --> 0:3:57,9
But I remember I was working with
a really big company called

84
0:3:57,9 --> 0:3:58,4
Chewy.

85
0:3:59,78 --> 0:4:2,74
And I remember they had great monitoring
already.

86
0:4:3,26 --> 0:4:7,92
Some clusters were on RDS already,
so they also have some performance

87
0:4:8,0 --> 0:4:8,5
insights.

88
0:4:9,06 --> 0:4:15,08
Still I wrote my snapshot tooling
because I didn't fully trust

89
0:4:15,08 --> 0:4:16,46002
monitoring things.

90
0:4:16,72 --> 0:4:19,76
I also wanted to verify, and some
details were missing because

91
0:4:19,76 --> 0:4:21,98
they didn't capture all metrics.

92
0:4:21,98 --> 0:4:24,18
I needed specific metrics and so
on.

93
0:4:24,24 --> 0:4:28,58
So meanwhile, there are some projects
which implement this idea,

94
0:4:28,58 --> 0:4:30,76
but they are extensions, so they
are Not available.

95
0:4:30,88 --> 0:4:32,92
pg_profile, I think, there is such
a thing, right?

96
0:4:32,92 --> 0:4:33,58
pg_profile.

97
0:4:33,84 --> 0:4:38,56
So that's why many, many DBAs,
maybe not all, but many DBAs at

98
0:4:38,56 --> 0:4:41,02
least once wrote some snapshotting
tool.

99
0:4:41,68 --> 0:4:44,88
And I remember our checkup tool
was also snapshotting.

100
0:4:45,2 --> 0:4:49,36
First versions of checkup tool,
it was a shell script.

101
0:4:49,66 --> 0:4:50,94
So we snapshot it.

102
0:4:51,04 --> 0:4:53,54
You have 2 snapshots, now you need
diff.

103
0:4:54,34 --> 0:4:58,04
And since we were in Bash, we didn't
want to do diff, so we sent

104
0:4:58,04 --> 0:5:1,3
these snapshots back to do diff
on the observed Postgres.

105
0:5:2,02 --> 0:5:7,28
So I saw a lot of solutions which
try to make this data persistent

106
0:5:7,54 --> 0:5:11,76
and have snapshots and then show
like everything without the

107
0:5:11,76 --> 0:5:15,04
need to set up full-fledged monitoring.

108
0:5:15,28 --> 0:5:18,98
And even if it exists still sometimes
we need additional lightweight

109
0:5:19,08 --> 0:5:19,58
solution.

110
0:5:20,04 --> 0:5:23,4
This is 1 thing to warm up us why
it's needed.

111
0:5:23,74 --> 0:5:28,22
Another thing is that I guess Supabase
has a lot of clusters.

112
0:5:29,18 --> 0:5:30,46
Some of them are,

113
0:5:30,66 --> 0:5:31,6
David: we have a few.

114
0:5:31,6 --> 0:5:31,98
Nikolay: Yeah.

115
0:5:31,98 --> 0:5:32,44
Yeah.

116
0:5:32,44 --> 0:5:33,66
Like millions, right?

117
0:5:33,96 --> 0:5:36,64
It's quite unique, interesting
story.

118
0:5:36,94 --> 0:5:40,02
And many of them are small and
you cannot justify full-fledged

119
0:5:40,08 --> 0:5:41,0
monitoring, right?

120
0:5:41,0 --> 0:5:42,48
David: And getting smaller, yeah.

121
0:5:42,7 --> 0:5:46,22
Nikolay: Getting smaller, yeah,
because people just experiment

122
0:5:46,22 --> 0:5:47,08
so much, right?

123
0:5:47,08 --> 0:5:48,62
And they need so many databases.

124
0:5:49,2 --> 0:5:52,24
David: Yeah, we are now, I mean,
we have a skewed distribution,

125
0:5:52,28 --> 0:5:52,86
of course.

126
0:5:52,86 --> 0:5:57,76
We have a few giant customers,
but we have lots of medium-sized

127
0:5:57,9 --> 0:6:1,8
customers and many more small ones
and millions of tiny ones.

128
0:6:1,8 --> 0:6:7,28
And now with the AI builders, we
are shoveling millions of nano

129
0:6:7,28 --> 0:6:9,44
instances into the AI builder furnaces.

130
0:6:9,52 --> 0:6:11,18
Like where are these databases
going?

131
0:6:11,28 --> 0:6:14,36
What's the long-term prognosis
for these tiny databases?

132
0:6:14,64 --> 0:6:15,72
Who knows, who cares?

133
0:6:15,86 --> 0:6:17,68
That's a completely different animal.

134
0:6:17,68 --> 0:6:24,44
But yes, it's a very, it's a vibrant
ecological niche that we've

135
0:6:24,44 --> 0:6:25,44
developed here.

136
0:6:25,68 --> 0:6:26,18
Michael: Yeah.

137
0:6:26,28 --> 0:6:28,38
On that note, who's this tool for?

138
0:6:28,38 --> 0:6:31,68
Like of that distribution, are
there some where it's not appropriate

139
0:6:31,68 --> 0:6:34,86
for them and some where it's ideal
or like where does it fit?

140
0:6:35,42 --> 0:6:37,1
David: Yeah, that's a great question
Michael.

141
0:6:37,2 --> 0:6:43,34
Again, and as Nik indicated, this
is, you know, all these things

142
0:6:43,84 --> 0:6:46,44
what time is a flat circle, all
these things will happen before

143
0:6:46,44 --> 0:6:47,5
and will happen again.

144
0:6:47,64 --> 0:6:50,32
You know, versions of this have
been written before and versions

145
0:6:50,32 --> 0:6:52,0
of this will be written in the
future.

146
0:6:52,12 --> 0:6:54,72
There's nothing really that profound
about this, but this is

147
0:6:54,72 --> 0:6:58,38
the tool that I needed right now
at this time for the reasons

148
0:6:58,38 --> 0:6:59,62
that Nik just described.

149
0:7:0,1 --> 0:7:4,46
Among which is, you know, we have,
I, who, who this tool would

150
0:7:4,46 --> 0:7:10,16
be for, I think would be startups,
SMBs, builders, sort of the

151
0:7:10,16 --> 0:7:12,84
canonical Supabase customer.

152
0:7:13,4 --> 0:7:17,92
Those who are starting out, you
know, building a business, building

153
0:7:17,92 --> 0:7:21,94
a backend, building a project,
they need PostgreSQL and off they

154
0:7:21,94 --> 0:7:22,44
go.

155
0:7:22,64 --> 0:7:27,38
You know, we at Supabase, we,
we don't, I don't think it's any

156
0:7:27,38 --> 0:7:28,06
big surprise.

157
0:7:28,1 --> 0:7:31,72
We don't really have that many
migrations onto Supabase.

158
0:7:31,84 --> 0:7:34,36
I mean, we would love to have more
than anybody who's willing

159
0:7:34,36 --> 0:7:36,72
to bring giant workloads over to
Supabase.

160
0:7:37,12 --> 0:7:37,98
Come on over.

161
0:7:38,14 --> 0:7:43,66
But we know that databases are
infamously sticky tools anyway.

162
0:7:43,66 --> 0:7:45,6
People don't really migrate that
often.

163
0:7:46,76 --> 0:7:50,28
And they're probably less likely
to migrate giant workloads over

164
0:7:50,28 --> 0:7:53,92
from Oracle or Microsoft SQL Server
to Supabase, although we

165
0:7:53,92 --> 0:7:55,32
are entertaining that option.

166
0:7:55,4 --> 0:8:0,44
But if we did do those things,
those folks would probably come

167
0:8:0,44 --> 0:8:5,2
over with DBAs, database experience,
database expertise.

168
0:8:6,56 --> 0:8:9,96
So our sort of customer portfolio
doesn't really reflect that.

169
0:8:9,96 --> 0:8:14,7
What we have are, even our largest
customers, I would say, tend

170
0:8:14,7 --> 0:8:19,64
to be, I mean, they may have 3
or 4 or 5 years of experience

171
0:8:19,64 --> 0:8:25,28
with PostgreSQL now, by dint of
hard effort, but they all started

172
0:8:25,28 --> 0:8:25,94
out small.

173
0:8:26,0 --> 0:8:28,74
Every 1 of our large customers
was a little acorn that grew into

174
0:8:28,74 --> 0:8:29,7
a giant oak.

175
0:8:30,06 --> 0:8:34,74
And we try to make Supabase easy,
and we do.

176
0:8:34,94 --> 0:8:36,72
It's certainly easy to get into.

177
0:8:37,24 --> 0:8:41,32
I've used this analogy too many
times, but it's like the car

178
0:8:41,32 --> 0:8:41,82
dealership.

179
0:8:41,88 --> 0:8:45,2
You can drive it off the lot in
5 minutes, but actually Operating

180
0:8:45,2 --> 0:8:47,54
it, especially at scale is something
different.

181
0:8:48,92 --> 0:8:52,0
And we, Nik knows this, Michael,
you know this, we would all

182
0:8:52,0 --> 0:8:55,22
benefit from more and better automation
and it's coming.

183
0:8:55,4 --> 0:8:59,18
It's coming from within the community
and it's coming from Supabase.

184
0:9:0,18 --> 0:9:4,2
We will be able to help these customers
more seamlessly and operate

185
0:9:4,2 --> 0:9:5,64
their databases in the future.

186
0:9:5,66 --> 0:9:8,68
But right now what we need is tooling
to help customers as they

187
0:9:8,68 --> 0:9:9,94
grow and as they scale.

188
0:9:9,94 --> 0:9:13,1
So in a nutshell, like who's this
for?

189
0:9:13,1 --> 0:9:16,94
People who are not DBAs, not database
experts.

190
0:9:18,08 --> 0:9:19,46
They just want to run a business.

191
0:9:19,64 --> 0:9:23,88
They want to grow that business
and they want some tools to help

192
0:9:23,88 --> 0:9:24,14
them.

193
0:9:24,14 --> 0:9:24,9
That's it.

194
0:9:24,96 --> 0:9:27,9
Nikolay: Some of them start with
some small, very small database

195
0:9:27,9 --> 0:9:32,94
instance paying 25 or some very
low number of bucks, right?

196
0:9:33,34 --> 0:9:38,98
And it's hard to justify paying
right away 150 or $400 or $500

197
0:9:39,4 --> 0:9:41,32
for monitoring a full-fledged solution.

198
0:9:41,32 --> 0:9:43,52
And then you need to spend time
there and so on.

199
0:9:43,52 --> 0:9:45,22
It cannot be justified easily.

200
0:9:45,48 --> 0:9:48,62
And also I wanted to mention, it's
quite elastic.

201
0:9:49,54 --> 0:9:55,06
So if you just inject this tool
inside your Postgres database,

202
0:9:55,12 --> 0:9:58,38
it starts collecting inside, like
self-observed.

203
0:9:59,28 --> 0:10:3,04
And you pay a little bit for those
megabytes per day, I don't

204
0:10:3,04 --> 0:10:3,54
know.

205
0:10:4,74 --> 0:10:9,18
Since I helped with storage to
rewrite it, it's quite efficient.

206
0:10:9,58 --> 0:10:14,44
I again used this approach for
PgQue, rotation of partitions and

207
0:10:14,44 --> 0:10:16,9
truncates, So it's very efficient
and so on.

208
0:10:16,96 --> 0:10:19,54
And I'm just saying it's like a
little bit, you pay a little

209
0:10:19,54 --> 0:10:21,42
bit and it's self observed, right?

210
0:10:21,98 --> 0:10:25,76
And when I was thinking what are
pros and cons, self-observed

211
0:10:26,1 --> 0:10:30,32
versus externally observed, ideally
we need to have both actually,

212
0:10:30,6 --> 0:10:35,5
Because you cannot install agent
right to RDS or super-based

213
0:10:35,58 --> 0:10:36,08
machine.

214
0:10:36,18 --> 0:10:40,9
So if you observe it outside with
external monitoring tool, if

215
0:10:40,9 --> 0:10:44,18
something bad happens, maybe you
don't have connectivity, right?

216
0:10:44,18 --> 0:10:48,66
While this thing sitting inside,
it still keeps observing, right?

217
0:10:48,74 --> 0:10:49,5
David: That's right.

218
0:10:49,64 --> 0:10:52,84
Nikolay: At the same time, if everything
is down, you don't see,

219
0:10:52,9 --> 0:10:54,86
you cannot reach the data, right?

220
0:10:54,86 --> 0:10:57,26
So like external tools also have
benefits.

221
0:10:57,34 --> 0:10:59,76
They have both pros and cons if
you think about it.

222
0:10:59,76 --> 0:11:0,42
It's interesting.

223
0:11:0,78 --> 0:11:4,04
So in my realization, even bigger
clusters should have maybe

224
0:11:4,04 --> 0:11:7,32
a small black box or flight recorder,
right?

225
0:11:7,44 --> 0:11:12,04
While we have a full-fledged solution
outside, they're both remote

226
0:11:12,04 --> 0:11:14,42
telemetry and something internal,
right?

227
0:11:14,48 --> 0:11:15,3
David: Yeah, that's right.

228
0:11:15,3 --> 0:11:17,8
And I think I landed on the name
pg_flight_recorder.

229
0:11:17,8 --> 0:11:20,92
And then at some point, I think
it had some reservations because

230
0:11:20,92 --> 0:11:25,84
I thought, sure, but in the event
of a crash, then maybe the

231
0:11:25,84 --> 0:11:28,54
data aren't available and it's
not really that useful.

232
0:11:28,62 --> 0:11:33,26
But then, I mean, if an actual
airplane crashes, then that airplane

233
0:11:33,26 --> 0:11:35,1
also is not really useful either.

234
0:11:36,42 --> 0:11:37,56
That airplane is dead.

235
0:11:37,56 --> 0:11:38,48
No 1

236
0:11:38,48 --> 0:11:39,36
Nikolay: will be using it.

237
0:11:39,86 --> 0:11:43,24
Just a side note, I just learned
David has a PhD in astrophysics,

238
0:11:43,5 --> 0:11:46,78
so this name is not a random thing,
I guess, right?

239
0:11:47,78 --> 0:11:49,78
David: And a master's in aerospace
engineering.

240
0:11:49,9 --> 0:11:52,44
But at every turn, I was trying
to do something else.

241
0:11:52,44 --> 0:11:54,34
And I was trying to get away from
computers.

242
0:11:54,34 --> 0:11:57,6
And it just, I just kept getting
sucked back in.

243
0:11:57,66 --> 0:12:1,4
But I grew up in the 70s when It
seemed like airplanes were crashing

244
0:12:1,4 --> 0:12:3,66
all the time when they weren't
being hijacked.

245
0:12:3,74 --> 0:12:5,94
Mercifully, that doesn't really
seem to happen all that often.

246
0:12:5,94 --> 0:12:9,16
But I think, I'm not a pilot, but
it's my understanding that

247
0:12:9,16 --> 0:12:12,82
actual, you know, flight recorders
are useful for far beyond

248
0:12:12,9 --> 0:12:13,82
crash investigation.

249
0:12:13,98 --> 0:12:19,02
They're useful for optimization,
for troubleshooting, like, in-flight

250
0:12:19,06 --> 0:12:19,56
incidences.

251
0:12:20,08 --> 0:12:23,1
And so I think the nature of this
is hopefully a little bit more

252
0:12:23,1 --> 0:12:23,6
like that.

253
0:12:23,6 --> 0:12:25,92
Nik, you would know better than
I would, but I have the feeling

254
0:12:25,92 --> 0:12:30,58
that in reality, databases don't
really actually crash all that

255
0:12:30,58 --> 0:12:30,92
often.

256
0:12:30,92 --> 0:12:35,36
What they do is they exhibit behavior,
and we want to be able

257
0:12:35,36 --> 0:12:36,46
to investigate that behavior.

258
0:12:36,46 --> 0:12:39,12
And that's what this helps us do.

259
0:12:40,08 --> 0:12:43,34
Briefly about the tool itself,
again, all it really does is it

260
0:12:43,34 --> 0:12:46,3
takes snapshots of wait events.

261
0:12:46,5 --> 0:12:47,54
That's how it started.

262
0:12:47,58 --> 0:12:48,48
It's like pg_ash.

263
0:12:49,5 --> 0:12:51,42
Nikolay: We developed it in parallel,
actually.

264
0:12:51,42 --> 0:12:51,9
Yeah.

265
0:12:51,9 --> 0:12:56,3
So when I told David that there
should be something small which

266
0:12:56,4 --> 0:12:59,16
self-observes, David just sent
me the link.

267
0:12:59,6 --> 0:13:0,74
It's already done.

268
0:13:1,78 --> 0:13:4,9
It was interesting that we had
parallel courses of development

269
0:13:4,9 --> 0:13:6,92
of pg_ash and pg_flight_recorder.

270
0:13:7,2 --> 0:13:8,76
They're very similar in this case.

271
0:13:8,76 --> 0:13:13,18
David: Yeah, very similar and like
that it captures active session

272
0:13:13,18 --> 0:13:17,12
history, wait events, nature of
course of Vacuum and Idle Hands

273
0:13:17,12 --> 0:13:18,16
of the Devil's Workshop.

274
0:13:18,16 --> 0:13:20,58
You know, with the tools available,
I couldn't resist the urge

275
0:13:20,58 --> 0:13:22,2
to just keep pouring more into
it.

276
0:13:22,2 --> 0:13:26,0
So it records lock activity and
checkpointer activity and background

277
0:13:26,0 --> 0:13:29,94
activity and IO stats and statement
stats and...

278
0:13:30,48 --> 0:13:32,0
Nikolay: Config changes, right?

279
0:13:32,46 --> 0:13:33,8
David: Config changes as well.

280
0:13:33,8 --> 0:13:37,94
And hopefully the conjecture is
that there's some value in, if

281
0:13:37,94 --> 0:13:42,04
not capturing everything, having
an opinionated and curated set

282
0:13:42,04 --> 0:13:46,52
of many things that are captured
simultaneously in a correlated

283
0:13:46,62 --> 0:13:54,4
fashion so that maybe you experience
a checkpoint storm and then

284
0:13:54,4 --> 0:13:57,8
you notice that there has been
a config change recently and you're

285
0:13:57,8 --> 0:13:59,06
able to bring these things together.

286
0:13:59,06 --> 0:14:0,12
So that's the idea.

287
0:14:0,3 --> 0:14:3,76
As Nik indicated, it was when
we talked about this, version

288
0:14:3,76 --> 0:14:4,86
0 was done.

289
0:14:5,14 --> 0:14:6,42
Again, it's a pretty simple tool.

290
0:14:6,42 --> 0:14:11,88
I had a few guiding principles,
1 of which was sort of the Hippocratic

291
0:14:11,98 --> 0:14:13,94
oath, try to do no harm.

292
0:14:14,06 --> 0:14:19,04
I put a lot of effort into making
this safe to run.

293
0:14:19,76 --> 0:14:21,58
Nikolay: Statement timeouts and
so on, right?

294
0:14:21,58 --> 0:14:25,3
David: Yeah, statement timeouts,
circuit breakers, graceful degradation

295
0:14:25,76 --> 0:14:30,02
of some of the components, dozens,
too many configuration settings,

296
0:14:30,02 --> 0:14:33,04
but then configuration profiles
that that capture those to make

297
0:14:33,04 --> 0:14:34,06
it easy to use.

298
0:14:34,22 --> 0:14:37,12
I think I got 3 quarters of the
way there or maybe 50% of the

299
0:14:37,12 --> 0:14:41,02
way there, but there were still
some improvements to be made,

300
0:14:41,1 --> 0:14:46,1
among which the storage engine,
which we can thank Nik for rewriting

301
0:14:46,94 --> 0:14:51,82
using PgQ or essentially the engine
that is part of PgQ.

302
0:14:51,82 --> 0:14:52,78
Correct, Nik?

303
0:14:52,8 --> 0:14:56,04
Nikolay: Yeah, it's like partitions,
rotation, I think daily

304
0:14:56,04 --> 0:14:56,54
partitions.

305
0:14:56,74 --> 0:15:1,46
There is also a roll up for all
data to have it less precise,

306
0:15:1,46 --> 0:15:2,56
not raw but aggregated.

307
0:15:3,24 --> 0:15:4,78
Everything is already implemented.

308
0:15:4,92 --> 0:15:8,12
And I remember I was brainstorming
with Claude Code, like what

309
0:15:8,12 --> 0:15:11,0
kind of storage we should choose
because I think originally you

310
0:15:11,0 --> 0:15:12,66
used a lot of JSON, right?

311
0:15:12,66 --> 0:15:14,94995
It's quite bloated in my opinion
sometimes.

312
0:15:14,94995 --> 0:15:20,68
David: Well, It wasn't JSON, but
I had originally, I was using

313
0:15:20,68 --> 0:15:28,44
skip locked and unlogged tables in
a vain attempt to mitigate dead

314
0:15:28,44 --> 0:15:29,56
tools and bloat.

315
0:15:31,6 --> 0:15:34,2
But If you're not diligent, then
they're still there.

316
0:15:34,2 --> 0:15:36,26
And so that's why you had to rewrite
the engine.

317
0:15:36,98 --> 0:15:39,66
Nikolay: Also, like, data format
is interesting.

318
0:15:39,72 --> 0:15:44,24
And I had multiple ideas, and I
have some, like, brainstorm document

319
0:15:44,24 --> 0:15:46,56
where, like, thinking what to choose.

320
0:15:46,56 --> 0:15:50,02
And some ideas were compressing
data quite a lot, but it was

321
0:15:50,02 --> 0:15:53,36
hard to deal with because it was
basically encoded so much that

322
0:15:53,36 --> 0:15:57,18
it's inconvenient, so I did a trade-off
choice that it should

323
0:15:57,18 --> 0:16:1,52
be human-readable even in raw form,
although I did apply some

324
0:16:2,1 --> 0:16:6,3
tricks from pg_ash as well, like timestamps
are relative to, I think,

325
0:16:6,3 --> 0:16:8,14
I don't know, 2020 or something.

326
0:16:8,42 --> 0:16:11,52
Like Unix timestamp by shifted,
so we have capacity until the

327
0:16:11,52 --> 0:16:12,34
end of century.

328
0:16:12,56 --> 0:16:16,44
And you have as few bytes wasted
as possible, like very compact

329
0:16:16,44 --> 0:16:16,92
way.

330
0:16:16,92 --> 0:16:19,08
And plus this PgQ style rotation.

331
0:16:20,14 --> 0:16:24,86
And also, worth mentioning, there
is a soft requirement, pg_cron.

332
0:16:25,08 --> 0:16:27,66
It's not a requirement, but it's
very recommended, because this

333
0:16:27,66 --> 0:16:29,56
is how it's ticking as well, right?

334
0:16:29,92 --> 0:16:30,6
David: That's correct.

335
0:16:30,6 --> 0:16:33,1
So again, it's a simple tool.

336
0:16:33,16 --> 0:16:37,68
It's 2 sort of packages, 2 simple
install scripts, 2 schemas,

337
0:16:38,0 --> 0:16:40,36
1 of which is required, 1 of which
is optional.

338
0:16:40,44 --> 0:16:43,94
The part that's required, it's
the data model, the tables and

339
0:16:43,94 --> 0:16:47,14
the views and the functions to
record those data.

340
0:16:47,26 --> 0:16:51,88
And then the other optional piece
is a set of functions for analyzing

341
0:16:51,96 --> 0:16:52,58
those data.

342
0:16:52,58 --> 0:16:56,58
But again, they could be analyzed
in raw form in whatever way

343
0:16:56,58 --> 0:16:57,16
you like.

344
0:16:57,16 --> 0:17:2,28
But as Nik indicated, somebody
has got to generate the ticks.

345
0:17:2,32 --> 0:17:5,88
Somebody has got to force the samples
and that could be pg_cron,

346
0:17:6,18 --> 0:17:9,68
it could be something like an outside
scheduler, somebody's got

347
0:17:9,68 --> 0:17:10,24
to do it.

348
0:17:10,24 --> 0:17:12,22
The sort of default way is with
pg_cron.

349
0:17:12,6 --> 0:17:14,94
Nikolay: And pg_cron is available
everywhere, right?

350
0:17:15,18 --> 0:17:19,12
David: Yeah, pg_cron sort of snuck
in before the sort of iron

351
0:17:19,12 --> 0:17:22,0
curtain started to drop on extensions,
maybe.

352
0:17:22,12 --> 0:17:24,62
So it's in a lot of places at least,
you know.

353
0:17:24,62 --> 0:17:26,88
Michael: I, I, yeah, maybe that's
true.

354
0:17:26,88 --> 0:17:31,64
I got the impression it solved
such a useful problem and was

355
0:17:31,64 --> 0:17:36,46
from such a reputable author that
I think people trusted it and

356
0:17:36,46 --> 0:17:38,94
also thought it's simple enough
that we can maintain.

357
0:17:38,94 --> 0:17:42,04
I understand why managed service
providers don't just offer any

358
0:17:42,04 --> 0:17:44,8
extension but if you think about
how much work it would be to

359
0:17:44,8 --> 0:17:49,32
maintain pg_cron if the author ditched
it or you know actually

360
0:17:49,34 --> 0:17:51,36
it's not huge and it's

361
0:17:51,66 --> 0:17:51,82
Nikolay: so useful.

362
0:17:51,82 --> 0:17:53,4
And it should be in core.

363
0:17:54,24 --> 0:17:55,365
David: There's all of it.

364
0:17:55,365 --> 0:17:56,76
That is true.

365
0:17:56,76 --> 0:17:59,18
Nikolay: I wish pg_cron was in core.

366
0:17:59,34 --> 0:18:2,86
And we would have, for example,
automated new partition creation

367
0:18:3,2 --> 0:18:5,78
out of the box without any extensions.

368
0:18:6,4 --> 0:18:6,9
Magic.

369
0:18:7,26 --> 0:18:9,06
So simple thing, actually, right?

370
0:18:10,84 --> 0:18:13,86
Michael: 1 thing about pg_cron from
a database perspective is it

371
0:18:13,86 --> 0:18:18,36
is, I think, once per second is
the lowest you can schedule.

372
0:18:19,02 --> 0:18:22,94
So what do you use, David, when
you're using this with people?

373
0:18:23,1 --> 0:18:28,16
Do you use pg_cron with a one-second
tick, or do you suggest

374
0:18:28,78 --> 0:18:29,66
something else?

375
0:18:30,86 --> 0:18:33,42
David: So far, I have used it just
with pg_cron.

376
0:18:34,54 --> 0:18:39,86
The resolution that has been sufficient
so far, you know, because

377
0:18:39,86 --> 0:18:43,94
with the customers that I've worked
with, the resolution before

378
0:18:43,98 --> 0:18:47,7
has been, I guess, infinite as
in they didn't have this at all.

379
0:18:47,7 --> 0:18:50,26
So it's just worth having the data.

380
0:18:50,54 --> 0:18:53,64
And a finer resolution, I haven't
encountered a demand for that

381
0:18:53,64 --> 0:18:56,42
or a need for that yet, although
certainly plausible.

382
0:18:56,52 --> 0:18:59,34
But again, yeah, that has worked
well so far.

383
0:18:59,34 --> 0:19:1,5
And Again, this is just a simple
tool.

384
0:19:1,76 --> 0:19:7,44
The idea, the objective anyway,
is sort of a set it and forget

385
0:19:7,44 --> 0:19:7,94
it.

386
0:19:8,1 --> 0:19:12,14
Install the tool, then forget that
you've installed the tool.

387
0:19:12,98 --> 0:19:17,9
But because it's safe to run, by
virtue of Nik's hard effort,

388
0:19:17,9 --> 0:19:19,12
it is safe to run.

389
0:19:19,12 --> 0:19:20,78
So just forget that it's there.

390
0:19:20,9 --> 0:19:23,56
And then you have an incident,
and then you think, oh wait, I

391
0:19:23,56 --> 0:19:24,78
have pg_flight_recorder.

392
0:19:24,96 --> 0:19:26,08
Let's find out what happened.

393
0:19:26,08 --> 0:19:29,82
Nikolay: And just point your eye
to the data, maybe a dump of

394
0:19:29,82 --> 0:19:32,04
that data or something, and that's
it.

395
0:19:32,04 --> 0:19:35,28
And the second package you mentioned,
it has interesting functions,

396
0:19:35,4 --> 0:19:37,44
like what happened at or something,
right?

397
0:19:37,44 --> 0:19:39,98
It's based on function names.

398
0:19:40,44 --> 0:19:43,74
I see already your thought, oh,
AI should guess, right?

399
0:19:43,74 --> 0:19:46,82
You designed it for, So it's self-explanatory,
right?

400
0:19:46,82 --> 0:19:47,7
So it's great.

401
0:19:48,42 --> 0:19:50,04
But I also wanted to...

402
0:19:50,06 --> 0:19:51,72
About pg_cron a little bit.

403
0:19:51,78 --> 0:19:56,18
Version 1.5 is, I remember, the
lowest resolution, once per second,

404
0:19:56,4 --> 0:20:1,56
and I use it for pg_ash, but I guess
you use it by default at much

405
0:20:1,56 --> 0:20:4,3
less frequency, especially for
Azure data, right?

406
0:20:4,3 --> 0:20:8,54
Maybe once per 30 seconds or 60,
but it's tunable, right?

407
0:20:8,86 --> 0:20:9,86
David: Everything is tunable.

408
0:20:9,96 --> 0:20:13,44
You know, default sample collection
is, I think I have it set

409
0:20:13,44 --> 0:20:18,44
to once per second, but There are
aggregates that are taken at

410
0:20:18,44 --> 0:20:19,46
a coarser resolution.

411
0:20:19,9 --> 0:20:22,28
There are roll-ups that are taken
at a coarser resolution.

412
0:20:22,46 --> 0:20:25,36
Data are archived at a coarser
resolution.

413
0:20:25,4 --> 0:20:30,8
Then there's retention for the
core tables, as I think that my

414
0:20:30,8 --> 0:20:34,4
default is 7 days for the aggregates
of the 7 days.

415
0:20:34,4 --> 0:20:38,54
And then for, for snapshots, I
think it's by default 30 days,

416
0:20:38,8 --> 0:20:40,14
but all of these are configurable.

417
0:20:40,38 --> 0:20:43,02
So there, there are a few different
cadences that are happening.

418
0:20:43,38 --> 0:20:48,0
And there is, again, as you indicated
on the analyze side, there

419
0:20:48,98 --> 0:20:53,4
is a wall of functions, appropriately
named, meant to be understandable

420
0:20:53,52 --> 0:20:57,78
by humans and by AI alike, so that
they can use these functions

421
0:20:57,78 --> 0:20:59,0
to analyze the data.

422
0:20:59,34 --> 0:21:2,6
But then, of course, it's always
available to be analyzed in

423
0:21:2,6 --> 0:21:3,9
raw form as well.

424
0:21:4,2 --> 0:21:8,0
Nikolay: Yeah, I can share some
interesting story from PgQ about

425
0:21:8,0 --> 0:21:8,96
function names.

426
0:21:9,64 --> 0:21:14,44
When I was developing recently
client libraries for PgQue, multiple

427
0:21:14,44 --> 0:21:17,42
times Claude Code made a mistake
because there is a function

428
0:21:17,72 --> 0:21:22,1
force tick, but it's not ticking,
it's just shifting this pointer.

429
0:21:22,96 --> 0:21:26,18
And then you need to run ticker
in a separate transaction.

430
0:21:26,52 --> 0:21:29,82
And Claude couldn't get it because
it's confusing name actually.

431
0:21:30,3 --> 0:21:33,06
And it made mistake multiple times
developing this.

432
0:21:33,34 --> 0:21:37,54
And this, I had huge flashback
to 15 years, 15 plus years ago

433
0:21:37,54 --> 0:21:40,44
when I made the same mistake manually
without AI, because it

434
0:21:40,44 --> 0:21:42,66
was so confusing to me 15 years
ago.

435
0:21:42,72 --> 0:21:46,66
So I just renamed that function
to force next tick.

436
0:21:46,88 --> 0:21:49,4
So you immediately understand this
is about next tick.

437
0:21:49,4 --> 0:21:51,9
You're not doing, you're just preparing
this job.

438
0:21:51,9 --> 0:21:55,66
And looking at your functions,
what happened at, incident timeline,

439
0:21:55,92 --> 0:21:57,54
I'm just thinking, this is self-explanatory.

440
0:21:58,26 --> 0:22:2,14
Maybe long, but everyone will understand
what it is for.

441
0:22:2,3 --> 0:22:4,38
So it's worth making a lot of.

442
0:22:4,38 --> 0:22:4,88
David: Yeah.

443
0:22:5,14 --> 0:22:5,28
Yeah.

444
0:22:5,28 --> 0:22:6,04
That's right.

445
0:22:6,22 --> 0:22:9,96
And if there are too many functions,
then, again, we can use

446
0:22:9,96 --> 0:22:12,94
AI to paw our way through and figure
out which ones to use.

447
0:22:13,08 --> 0:22:14,02
Just a final thought.

448
0:22:14,16 --> 0:22:18,76
It's meant for a few things, not
just incident response, but

449
0:22:18,76 --> 0:22:22,26
also capacity planning, blast radius,
evaluation.

450
0:22:23,26 --> 0:22:29,02
You know, again, where I intend
to go with this is just getting

451
0:22:29,02 --> 0:22:30,58
back to Supabase briefly.

452
0:22:31,18 --> 0:22:34,54
I have lots of customers that I
have to, I should say I'm blessed

453
0:22:34,54 --> 0:22:38,38
with helping, but so many of them
I want to get to early.

454
0:22:38,42 --> 0:22:42,24
It's, you know, again we talked
about this, but all of these

455
0:22:42,24 --> 0:22:43,48
databases are small.

456
0:22:43,58 --> 0:22:45,14
They start out small anyway.

457
0:22:45,36 --> 0:22:51,18
I would say from a certain point
of view, from the point of view

458
0:22:51,18 --> 0:22:55,84
of scale, many of them are sort
of doing things wrong, but that's

459
0:22:55,84 --> 0:22:57,44
okay because they're small.

460
0:22:57,44 --> 0:23:0,32
You can do, with a small database,
you can do everything wrong

461
0:23:0,32 --> 0:23:2,34
and it's fine, no problems.

462
0:23:2,56 --> 0:23:4,96
But it's when you start to scale
that you need to think about

463
0:23:4,96 --> 0:23:5,46
this.

464
0:23:5,66 --> 0:23:6,74
So it was like exercise.

465
0:23:6,76 --> 0:23:9,62
It's something you have to get
into the habit of doing it early

466
0:23:9,62 --> 0:23:13,04
and often, even though you don't
want to, and maybe you would

467
0:23:13,04 --> 0:23:16,44
benefit from a personal trainer
and some encouragement to get

468
0:23:16,44 --> 0:23:19,2
you on the path early so that it
pays dividends.

469
0:23:20,28 --> 0:23:24,14
When you're old, like I am, and
when you're a big database, like

470
0:23:24,14 --> 0:23:25,78
some of these eventually will become.

471
0:23:26,28 --> 0:23:29,48
Nikolay: I wanted also to mention,
by default, it's consuming

472
0:23:29,48 --> 0:23:33,06
up to a couple of gigabytes for
those 7 days, right?

473
0:23:33,12 --> 0:23:35,7
But again, it's tunable if you,
or less.

474
0:23:36,04 --> 0:23:36,88
David: Yeah, it's tunable.

475
0:23:36,88 --> 0:23:38,04
It's a few gigabytes.

476
0:23:38,22 --> 0:23:40,12
Nik, I think you and I, we benchmarked
this.

477
0:23:40,12 --> 0:23:43,22
I think we, with the new storage
engine, I think we estimate

478
0:23:43,32 --> 0:23:47,9
maybe for like under, on the happy
path, maybe I think around

479
0:23:47,9 --> 0:23:50,14
like 20 gigabytes for the month.

480
0:23:50,6 --> 0:23:51,24
Nikolay: It depends.

481
0:23:51,9 --> 0:23:57,94
David: There's some, 1 of my goals
is to sort of draw more attention

482
0:23:58,08 --> 0:24:2,6
to this so that I could get feedback
and improve it.

483
0:24:2,6 --> 0:24:5,86
And there is some low-hanging fruit
to be plucked insofar as

484
0:24:5,86 --> 0:24:7,28
data retention for path-wise.

485
0:24:7,28 --> 0:24:12,42
Nikolay: I think it depends on
how many queries you have in Pagesaw

486
0:24:12,44 --> 0:24:14,28
statements by default, up to 5,
000.

487
0:24:14,68 --> 0:24:18,0
And Also, you collect data about
indexes and tables, so how many

488
0:24:18,0 --> 0:24:19,74
tables and indices you have.

489
0:24:20,22 --> 0:24:25,68
I think you have limits there,
but still it depends a lot on

490
0:24:25,68 --> 0:24:27,26
the cardinality of these things.

491
0:24:28,28 --> 0:24:32,48
About use cases, I used it recently
for benchmarking PgQue.

492
0:24:34,06 --> 0:24:35,06
For me it's so natural.

493
0:24:35,06 --> 0:24:38,1
I had multiple already projects
like this and I just, okay.

494
0:24:38,1 --> 0:24:42,34
I injected both pg_ash and pg_flight_recorder
because pg_ash has

495
0:24:42,34 --> 0:24:45,48
more frequency and more details
about ash data.

496
0:24:45,54 --> 0:24:47,72
pg_flight_recorder brings a lot of
stuff, right?

497
0:24:47,72 --> 0:24:51,84
So I just injected it into some
synthetic database as provisioned

498
0:24:52,54 --> 0:24:53,86
with pg_cron configured.

499
0:24:53,94 --> 0:24:57,32
And I just asked AI, of course,
to do it, so just inject it.

500
0:24:57,32 --> 0:24:59,68
And then don't forget to dump after
each run.

501
0:25:0,32 --> 0:25:1,36
And then visualize it.

502
0:25:1,36 --> 0:25:3,74
That's it, Only 3 sentences.

503
0:25:4,46 --> 0:25:5,24
David: Yeah, exactly.

504
0:25:5,38 --> 0:25:8,5
Nikolay: And this is how I created
a beautiful looking.

505
0:25:9,0 --> 0:25:13,14
I actually asked to animate benchmarks
because it's great to

506
0:25:13,14 --> 0:25:15,26
look how lines go.

507
0:25:15,42 --> 0:25:18,8
And this is what brought PgQue good
attention because this data

508
0:25:18,8 --> 0:25:21,3
is like easy to understand Right.

509
0:25:21,3 --> 0:25:23,98
So for example, how much WAL was
generated, right?

510
0:25:24,64 --> 0:25:25,38
Lot of stuff.

511
0:25:25,38 --> 0:25:29,0
What was the behavior of check
pointer or autovacuum and so on?

512
0:25:29,76 --> 0:25:32,04
Michael: That was gonna mean my
next question actually on the

513
0:25:32,04 --> 0:25:32,78
wall front.

514
0:25:32,8 --> 0:25:37,44
You mentioned a few minutes ago
about how you originally went

515
0:25:37,44 --> 0:25:38,64
with unlogged tables.

516
0:25:39,06 --> 0:25:42,16
Does that mean these are now logged
and there is

517
0:25:42,16 --> 0:25:42,94
Nikolay: WAL generated?

518
0:25:43,38 --> 0:25:46,82
Yeah, it was my decision to say
that, first of all, important

519
0:25:46,82 --> 0:25:50,86
limitation of all those tools which
are ticking on pg_cron and

520
0:25:50,86 --> 0:25:54,22
write something and it's only PL/pgSQL,
it's primary only.

521
0:25:55,52 --> 0:25:59,7
But we live in this strange situation
for me, old DBA, when a

522
0:25:59,7 --> 0:26:1,54
lot of clusters are single node.

523
0:26:1,72 --> 0:26:6,66
I have even cases clients are coming
like 10 plus like 15 terabytes

524
0:26:6,76 --> 0:26:8,5
on single node and they are fine.

525
0:26:9,14 --> 0:26:12,0
Cloud like resources became quite
relevant.

526
0:26:12,32 --> 0:26:16,56
For some actually I think it's
okay Because backups matter more

527
0:26:16,56 --> 0:26:20,26
than HA because they are fine to
be done, but not to pay for

528
0:26:20,26 --> 0:26:21,9
additional couple of nodes and
so on.

529
0:26:21,9 --> 0:26:24,212
So anyway, this is primary only
because we cannot write to- Can

530
0:26:24,212 --> 0:26:24,88
Michael: I add something?

531
0:26:25,08 --> 0:26:25,38
Yeah.

532
0:26:25,38 --> 0:26:26,4
Can I add something to that?

533
0:26:26,4 --> 0:26:29,84
Because you say single node, but
I think that's slightly simplistic

534
0:26:30,06 --> 0:26:34,54
because a lot of ones I see, they're
HA, but the replicas are

535
0:26:34,54 --> 0:26:36,96
not read replicas, they're like failover
replicas.

536
0:26:36,98 --> 0:26:38,86
Nikolay: Shadow standby node.

537
0:26:38,86 --> 0:26:41,66
Michael: I wouldn't call that a
one-node cluster, but it's still

538
0:26:41,66 --> 0:26:42,34
you only need to

539
0:26:42,34 --> 0:26:42,63
Nikolay: monitor the primary.

540
0:26:42,63 --> 0:26:44,28
Yeah, you're right actually, you're
right.

541
0:26:44,34 --> 0:26:46,72
But in this case you are not interested
because you don't have

542
0:26:46,72 --> 0:26:49,96
any workload on that standby, hidden
standby, right?

543
0:26:49,96 --> 0:26:51,04
Michael: Exactly, exactly.

544
0:26:51,5 --> 0:26:54,6
Nikolay: So you are interested
on the primary and we see so many

545
0:26:54,6 --> 0:26:58,76
projects reaching dozens of terabytes
already, which single node.

546
0:26:59,2 --> 0:27:2,72
You inject it, we need to understand,
okay, it's self-recording,

547
0:27:3,26 --> 0:27:4,86
so it's going to produce some writes.

548
0:27:4,86 --> 0:27:8,24
If it's unlogged table, if it crashed,
it's gone.

549
0:27:8,56 --> 0:27:9,9
That's the key idea.

550
0:27:10,02 --> 0:27:11,12
We cannot afford...

551
0:27:11,12 --> 0:27:14,2
David: At least, where the data
lands initially, those data would

552
0:27:14,2 --> 0:27:15,26
be gone, right?

553
0:27:15,28 --> 0:27:16,82
When they were unlogged tables.

554
0:27:17,04 --> 0:27:19,16
Nikolay: You need to snapshot,
you need external means.

555
0:27:19,16 --> 0:27:24,86
So to understand the incident after
crash, we should use regular

556
0:27:24,86 --> 0:27:25,36
tables.

557
0:27:25,44 --> 0:27:29,28
And when we've redesigned storage,
it's not so super expensive.

558
0:27:29,28 --> 0:27:33,74
Of course, there is some WAL to
be written and some data storage

559
0:27:33,74 --> 0:27:34,28
to be paid.

560
0:27:34,28 --> 0:27:38,2
And of course, a little bit of
shared buffers occupied by our

561
0:27:38,2 --> 0:27:38,7
data.

562
0:27:38,76 --> 0:27:41,5
It goes to, if you have replicas,
it goes to replicas.

563
0:27:42,04 --> 0:27:45,6
Maybe it's not a bad thing because
if it's the health of primary,

564
0:27:46,4 --> 0:27:47,76
we can pay this price.

565
0:27:47,9 --> 0:27:49,08
Michael: What are we roughly talking
about?

566
0:27:49,08 --> 0:27:52,08
You mentioned a few gigabytes up
to maybe 20 gigabyte, like that

567
0:27:52,08 --> 0:27:53,16
kind of amount for storage.

568
0:27:53,16 --> 0:27:56,0
What are we talking about in terms
of WAL generation by default,

569
0:27:56,0 --> 0:27:57,9
just to give people a rough idea?

570
0:27:57,94 --> 0:27:58,88
Nikolay: I don't remember.

571
0:28:0,42 --> 0:28:4,54
I thought about baby clusters like
1 gigabyte once, like free

572
0:28:4,54 --> 0:28:6,18
tier is up to 1 gigabyte, right?

573
0:28:6,18 --> 0:28:9,76
So I thought they should afford
this maybe with a little bit

574
0:28:9,76 --> 0:28:14,36
tuned to less frequency or something
retention wise But it's

575
0:28:14,54 --> 0:28:16,7
Michael: a storage now or WAL
3

576
0:28:16,7 --> 0:28:17,52
Nikolay: both both.

577
0:28:17,52 --> 0:28:18,02
Michael: Okay,

578
0:28:18,16 --> 0:28:19,5
Nikolay: They are connected actually.

579
0:28:19,64 --> 0:28:20,64
If you need to write-

580
0:28:20,64 --> 0:28:21,98
Michael: You mean on super base?

581
0:28:22,8 --> 0:28:24,1
Nikolay: Anywhere, any Postgres.

582
0:28:24,1 --> 0:28:27,68
If you need to write 100 megabytes
to storage, you will produce

583
0:28:27,74 --> 0:28:32,38
like very roughly, you will produce
kind of close to 100 megabytes

584
0:28:32,42 --> 0:28:34,3
to WAL because this is the same
data.

585
0:28:34,4 --> 0:28:37,16
Yes, in different form, but it's
the same data, right?

586
0:28:37,72 --> 0:28:41,32
If you need to write 100 times
more, expect 100 times more of

587
0:28:41,32 --> 0:28:41,82
wall.

588
0:28:42,18 --> 0:28:44,44
David: Yeah, order of magnitude,
it would be about the same.

589
0:28:44,48 --> 0:28:45,78
Nikolay: Yeah, very roughly.

590
0:28:46,22 --> 0:28:49,84
Of course, like full page writes,
all the compression, but it's

591
0:28:49,84 --> 0:28:50,58
very different.

592
0:28:50,82 --> 0:28:53,32
Michael: I also think of them very
differently because with WAL

593
0:28:53,32 --> 0:28:56,42
I think of it as like megabytes
per second, always.

594
0:28:56,42 --> 0:29:0,04
It's always like a, with a time
component, if that makes sense.

595
0:29:0,04 --> 0:29:4,5
So it's like a, it's a constant
amount that we're generating,

596
0:29:5,28 --> 0:29:9,16
of course over a month or whatever
it is, that's a few gigabytes.

597
0:29:9,48 --> 0:29:12,66
But I guess that doesn't actually
add up to very much per second

598
0:29:12,66 --> 0:29:13,86
in terms of, yeah.

599
0:29:13,86 --> 0:29:16,74
Nikolay: I think we should expect
something like 100 to a few

600
0:29:16,74 --> 0:29:21,5
hundreds megabytes per day with
a lot of queries and indexes

601
0:29:21,5 --> 0:29:22,36
and so on.

602
0:29:22,36 --> 0:29:22,72
Yeah.

603
0:29:22,72 --> 0:29:26,68
David: I mean, if you like ballpark
math, if it were 30 gigabytes

604
0:29:26,88 --> 0:29:30,78
of data per month, that would be
roughly, I guess, by the power

605
0:29:30,78 --> 0:29:33,84
of arithmetic, maybe a gigabyte
per day.

606
0:29:34,06 --> 0:29:35,18
Nikolay: It's very stable.

607
0:29:36,04 --> 0:29:39,92
David: And by 3600, you could figure
out megabytes per second

608
0:29:39,92 --> 0:29:41,4
of WAL generation.

609
0:29:41,4 --> 0:29:42,8
Nikolay: Kilobytes maybe already,
right?

610
0:29:42,8 --> 0:29:46,24
And it's very stable because it
depends only on this cardinality.

611
0:29:46,56 --> 0:29:49,84
And if you have some spikes of
workload, it doesn't affect the

612
0:29:49,84 --> 0:29:54,9
amount of data, these snapshots,
right to WAL and data directory,

613
0:29:54,9 --> 0:29:55,4
right?

614
0:29:55,46 --> 0:29:56,34
David: It's a baseline.

615
0:29:56,76 --> 0:30:2,74
So just, you know, you're paying
maybe $25 to Supabase, maybe

616
0:30:2,74 --> 0:30:7,04
pay $26 and just pay for a little
bit more storage.

617
0:30:7,66 --> 0:30:11,88
Nikolay: Not hundreds more as you
would pay if you install full-fledged

618
0:30:11,96 --> 0:30:12,46
monitoring.

619
0:30:13,5 --> 0:30:19,2
David: And if it's only, if it's
to first order for the primary

620
0:30:19,22 --> 0:30:19,68
node.

621
0:30:19,68 --> 0:30:24,1
Again, all of these small projects
are starting out with only

622
0:30:24,1 --> 0:30:25,12
1 node anyway.

623
0:30:25,12 --> 0:30:29,38
I mean, life is complicated and
these people are just trying

624
0:30:29,38 --> 0:30:32,5
to get like a business off the
ground and a job done.

625
0:30:32,74 --> 0:30:37,66
They're not thinking about multiple
nodes, especially early on,

626
0:30:38,86 --> 0:30:43,18
but they, you, I mean, we 3 know
that they will need data to

627
0:30:43,18 --> 0:30:44,54
guide them on their journey.

628
0:30:44,54 --> 0:30:46,0
So this is just part of that.

629
0:30:46,0 --> 0:30:48,12
I mean, Michael, you work in the
observability space.

630
0:30:48,12 --> 0:30:50,52
We sort of dilate this out to a
wider view.

631
0:30:50,86 --> 0:30:55,52
This is just another entry in the
observability space, like maybe

632
0:30:55,52 --> 0:30:59,52
a new generation 0.5 of observability
tools for PostgreSQL.

633
0:30:59,86 --> 0:31:2,66
But I mean, and there's pg_ash and
there's...

634
0:31:2,78 --> 0:31:4,9
Nikolay: So it's not only about
observability.

635
0:31:5,28 --> 0:31:9,14
I see, actually, the word new kind
of breed you used, I think,

636
0:31:9,14 --> 0:31:10,3
in our discussions.

637
0:31:11,38 --> 0:31:13,54
So I have pg_ash, you have pg_flight_recorder.

638
0:31:13,98 --> 0:31:18,46
I also trying to revive PgQ in
this very format, pg_cron and PL/pgSQL

639
0:31:18,78 --> 0:31:19,2
only.

640
0:31:19,2 --> 0:31:19,64
That's it.

641
0:31:19,64 --> 0:31:21,98
So it can be installed anywhere
and just tick.

642
0:31:22,2 --> 0:31:25,76
I also have Lindex, which is not
yet released, which is rebuilding

643
0:31:25,76 --> 0:31:27,08
indexes on pg_cron.

644
0:31:27,18 --> 0:31:27,9
That's it.

645
0:31:28,0 --> 0:31:31,04
You can inject PL/pgSQL and tick
on pg_cron.

646
0:31:31,16 --> 0:31:31,9
That's it.

647
0:31:32,2 --> 0:31:33,26
It's super simple.

648
0:31:33,68 --> 0:31:38,36
I already think about tool for
automated partition creation without

649
0:31:38,8 --> 0:31:39,58
heavy tools.

650
0:31:40,2 --> 0:31:43,16
I don't know, it should be easy
to use.

651
0:31:43,52 --> 0:31:45,14
But then you mentioned pg_tle.

652
0:31:46,16 --> 0:31:46,66
David: Yes.

653
0:31:46,86 --> 0:31:51,14
Nikolay: So can you maybe elaborate
a little bit why pg_tle?

654
0:31:52,04 --> 0:31:56,36
Why not just single SQL file or
PL/pgSQL file?

655
0:31:56,36 --> 0:31:57,04
David: It's both.

656
0:31:57,04 --> 0:32:1,66
So, for those who don't know, TLE
is trusted language extensions,

657
0:32:2,02 --> 0:32:3,96
which I have my own view on that.

658
0:32:3,96 --> 0:32:9,8
I regard it as just a little bit
of extra housekeeping that's

659
0:32:9,8 --> 0:32:14,36
associated with just a simple SQL
install file, but they are,

660
0:32:14,44 --> 0:32:20,08
they, you know, TLE sort of dress
up SQL and PL/pgSQL code as

661
0:32:20,08 --> 0:32:23,44
if they were a sort of kind of
managed extension, but they can

662
0:32:23,44 --> 0:32:25,74
be installed without super user
privileges.

663
0:32:27,5 --> 0:32:32,68
pg_flight_recorder comprises both
just simple install scripts.

664
0:32:32,74 --> 0:32:38,12
You can use psql to install it,
but it also is available as a

665
0:32:38,12 --> 0:32:41,6
trusted language extension that
can be installed through I think

666
0:32:41,6 --> 0:32:45,68
dbdev Because that's how some people
want to be able to install.

667
0:32:46,26 --> 0:32:47,78
Nikolay: Just to track like metadata.

668
0:32:47,78 --> 0:32:50,74
David: Yeah, just so you can do,
it makes housekeeping a little

669
0:32:50,74 --> 0:32:51,34
bit easier.

670
0:32:51,34 --> 0:32:54,48
You can slide pg_flight_recorder
in with an install and if you

671
0:32:54,48 --> 0:32:58,8
don't like it, you can uninstall
it in a very managed fashion.

672
0:32:59,06 --> 0:33:0,18
So that's all that's meant there.

673
0:33:0,18 --> 0:33:4,82
But yeah, it's very TLE, it's a
very lightweight way to have

674
0:33:4,82 --> 0:33:9,44
managed extensions and Flight Recorder
offers that as well.

675
0:33:9,62 --> 0:33:13,52
Nikolay: Yeah, I actually wish
PjCrone and TLE, they're both

676
0:33:13,52 --> 0:33:15,04
inside Postgres itself.

677
0:33:15,46 --> 0:33:18,16
And we would say something like
create package or something.

678
0:33:18,16 --> 0:33:18,68
I don't know.

679
0:33:18,68 --> 0:33:21,5
And it's just a bunch of SQL and
Pell, just go code or maybe

680
0:33:21,5 --> 0:33:24,96
Pell Python if you want like anything
and it just can be installed

681
0:33:24,96 --> 0:33:29,34
anywhere with versioning and so
on with like tracking of CVS.

682
0:33:29,34 --> 0:33:32,26
I don't know, like if there are
any and so on, who knows.

683
0:33:32,54 --> 0:33:32,78
Yeah.

684
0:33:32,78 --> 0:33:37,86
But just like extensions don't
feel like a part of extensibility

685
0:33:38,0 --> 0:33:39,34
of Postgres to me anymore.

686
0:33:39,34 --> 0:33:41,98
This is my honest like feeling
lately.

687
0:33:42,52 --> 0:33:47,54
David: Well, it also, what worries
me is who is testing, I mean,

688
0:33:47,54 --> 0:33:51,0
with, with major version upgrades,
let alone minor versions,

689
0:33:51,3 --> 0:33:53,14
who's testing all of these extensions?

690
0:33:53,76 --> 0:33:57,32
Nikolay: But this question is also
applicable to any regular

691
0:33:57,34 --> 0:33:58,28
backend code.

692
0:33:58,4 --> 0:34:3,22
You use some libraries, You just
import them to your code, somehow

693
0:34:3,22 --> 0:34:4,64
include, and that's it.

694
0:34:4,64 --> 0:34:7,32
And versions also matter there,
and it's on your shoulders.

695
0:34:7,44 --> 0:34:8,76
It should be on your shoulders.

696
0:34:9,18 --> 0:34:12,26
This idea that we're not providing
some extensions because we

697
0:34:12,26 --> 0:34:15,78
will need to maintain, give it
to shoulders of people.

698
0:34:16,68 --> 0:34:18,3
This is a different part of the
thing.

699
0:34:18,82 --> 0:34:23,94
David: This is intention with products
which telegraph or advertise

700
0:34:24,4 --> 0:34:27,48
that were easy to use and you don't
need to worry, we will handle

701
0:34:27,48 --> 0:34:28,34
it for you.

702
0:34:28,42 --> 0:34:28,92
Nikolay: Yeah.

703
0:34:29,16 --> 0:34:33,08
But I think It's great to have
flexibility and if people can

704
0:34:33,08 --> 0:34:35,78
use various, like, choose and...

705
0:34:35,8 --> 0:34:40,18
But they need to be responsible
for upgrades and part of maintaining

706
0:34:40,28 --> 0:34:44,9
as they are already for libraries
and Go language, any language,

707
0:34:44,9 --> 0:34:45,38
right?

708
0:34:45,38 --> 0:34:47,52
TypeScript and so on.

709
0:34:47,58 --> 0:34:48,8
So there's something here.

710
0:34:48,8 --> 0:34:52,4
And I think it's AWS guys who created
pg_tle, right?

711
0:34:52,44 --> 0:34:55,9
So definitely this project was
created with realization that

712
0:34:55,9 --> 0:34:59,68
something is limiting people here
and let's bring something here.

713
0:34:59,68 --> 0:35:0,74
That's a great idea.

714
0:35:1,02 --> 0:35:1,84
And I wish- I mean,

715
0:35:1,84 --> 0:35:4,66
David: Michael, do you experience
that at all with your customers?

716
0:35:5,34 --> 0:35:6,33
Their challenges?

717
0:35:6,33 --> 0:35:9,34
I mean I know you work in a slightly
different space but you

718
0:35:9,34 --> 0:35:13,08
certainly you must encounter this
as well like tensions with

719
0:35:13,08 --> 0:35:15,42
extensions, with managed database
providers.

720
0:35:16,7 --> 0:35:20,58
Michael: Yeah so I get the impression
so I don't speak to people

721
0:35:20,58 --> 0:35:23,42
all the time about this kind of
thing, but I get the impression

722
0:35:23,62 --> 0:35:29,06
that people are looking for a little
bit of advice almost from

723
0:35:29,06 --> 0:35:32,04
their managed service provider
on which extensions they should

724
0:35:32,04 --> 0:35:35,28
trust, which are like the best
at what they do.

725
0:35:35,28 --> 0:35:38,36
You know, often there's a choice
of 2 or 3 and they kind of want

726
0:35:38,36 --> 0:35:41,04
their managed service providers
to pick 1 and say, you know,

727
0:35:41,04 --> 0:35:45,04
this is the 1 we suggest or this
is the 1 we support and I feel

728
0:35:45,04 --> 0:35:48,14
like there's a little bit of that
going on as well so it isn't

729
0:35:48,16 --> 0:35:51,46
just yeah I think it is a little
bit of I don't know if it's

730
0:35:51,46 --> 0:35:54,28
like king making or something but
like people saying this is

731
0:35:54,28 --> 0:35:56,82
the 1 everyone using when people
come to Postgres for example

732
0:35:56,82 --> 0:35:59,24
for the first time they're like
which backup tool should I use

733
0:35:59,24 --> 0:36:1,48
which monitoring tool what's everyone
else using And there's

734
0:36:1,48 --> 0:36:5,46
no kind of like official, there's
no, there's barely any kind

735
0:36:5,46 --> 0:36:6,82
of extension management systems.

736
0:36:6,82 --> 0:36:9,8
There's been about 3 or 4 kind
of created and I think there is

737
0:36:9,8 --> 0:36:11,02
still, is it PGXN?

738
0:36:11,28 --> 0:36:12,88
That's probably the most used.

739
0:36:12,88 --> 0:36:15,4
But it doesn't have like reviews
or it doesn't have, Like it

740
0:36:15,4 --> 0:36:17,56
doesn't have a lot of things people
are looking for in terms

741
0:36:17,56 --> 0:36:20,06
of which ones are actually used
which ones people do that people

742
0:36:20,06 --> 0:36:23,1
actually like which ones have got
a good track record when it

743
0:36:23,1 --> 0:36:29,76
comes to major versions or low
Maybe no CVEs or very few CVEs,

744
0:36:29,76 --> 0:36:32,52
you know that kind of thing So
I think trust is a big part of

745
0:36:32,52 --> 0:36:35,4
it, and also people want a shortcut
as to which ones of these

746
0:36:35,4 --> 0:36:36,44
should I be using.

747
0:36:36,94 --> 0:36:37,9
Nikolay: I actually agree.

748
0:36:38,6 --> 0:36:42,66
Supabase's, this database.dev,
it's another attempt to have this

749
0:36:42,66 --> 0:36:44,18
register of extensions, right?

750
0:36:44,18 --> 0:36:48,34
David: It is yet another attempt,
which seems to be somewhat

751
0:36:48,34 --> 0:36:51,38
honored in the breach but it's
yet another attempt but Michael

752
0:36:51,38 --> 0:36:55,34
I take your point definitely that
there is a need and I would

753
0:36:55,34 --> 0:36:59,54
say a growing need for if not king
making at least someone to

754
0:36:59,54 --> 0:37:3,6
offer guidance and to bless these
You would know as well as I

755
0:37:3,6 --> 0:37:9,52
would, the sort of persona for
database operators definitely

756
0:37:9,52 --> 0:37:10,52
does seem to be changing.

757
0:37:10,52 --> 0:37:15,42
I mean, there once was a time when
databases were an arena for

758
0:37:15,42 --> 0:37:21,04
people to sort of develop and then
project expertise, which is

759
0:37:21,04 --> 0:37:21,82
certainly true.

760
0:37:21,82 --> 0:37:27,02
But more and more, I encounter
people, customers, super-based

761
0:37:27,1 --> 0:37:28,78
users, who are very candid.

762
0:37:28,78 --> 0:37:32,6
They will say to me or to us, I
don't know what I'm doing.

763
0:37:32,6 --> 0:37:33,62
I'm not a DBA.

764
0:37:34,06 --> 0:37:36,04
Some of them will say, I'm not
even a tech.

765
0:37:36,04 --> 0:37:36,98
I'm not even technical.

766
0:37:36,98 --> 0:37:37,72
I'm a founder.

767
0:37:37,72 --> 0:37:39,74
And I just vibe coded my way into
this.

768
0:37:40,16 --> 0:37:43,04
And, you know, there's less of
an urge now than there was in

769
0:37:43,04 --> 0:37:48,12
the past to sort of burnish your
credentials as a database expert.

770
0:37:48,12 --> 0:37:51,88
People are very happily, they're
very candid, and they will say,

771
0:37:51,88 --> 0:37:56,98
I am not a database expert at all,
so please, can you help us?

772
0:37:56,98 --> 0:37:57,94
Can you offer guidance?

773
0:37:57,94 --> 0:38:0,86
If you tell me what extension to
install, I'll install it.

774
0:38:1,02 --> 0:38:7,46
So there's growing need for those
kinds of tools and for greater

775
0:38:7,64 --> 0:38:8,9
and better automation.

776
0:38:11,24 --> 0:38:14,28
That can be a topic for another
session.

777
0:38:14,44 --> 0:38:15,9
That's where my mind is.

778
0:38:17,47 --> 0:38:22,06
Nikolay: And I agree actually with
this authority, what's good,

779
0:38:22,06 --> 0:38:22,3801
what's reliable.

780
0:38:22,3801 --> 0:38:26,08
Sometimes I have cases where we
have huge Postgres, self-managed

781
0:38:26,28 --> 0:38:30,52
Postgres clusters, and when I'm
saying we should add some extension,

782
0:38:30,52 --> 0:38:34,86
I'm saying it's available on this
managed platform, so it's reliable,

783
0:38:34,94 --> 0:38:36,54
you know, let's add it.

784
0:38:36,54 --> 0:38:40,22
It helps me to speed things up
so yeah

785
0:38:41,76 --> 0:38:44,2
Michael: I had a couple of last
things I wanted to make sure

786
0:38:44,2 --> 0:38:47,76
like or to get it'd be great to
get your thoughts on 1 is is

787
0:38:47,76 --> 0:38:50,98
there any like Is there anything
that we haven't talked about

788
0:38:50,98 --> 0:38:52,96
that you have 1 of your favorite
features of the talk?

789
0:38:52,96 --> 0:38:56,04
I've seen there's quite a few in
there and then also is there

790
0:38:56,04 --> 0:38:58,16
anything missing that you really
want to add?

791
0:38:58,92 --> 0:39:4,94
David: There are In reverse order
things that I want to add I

792
0:39:4,94 --> 0:39:10,76
again I really want I would like
to fortify this against observer

793
0:39:10,76 --> 0:39:13,4
effect, against deleterious consequences.

794
0:39:13,98 --> 0:39:17,94
I already know that there are some
important fixes to be made

795
0:39:18,24 --> 0:39:20,44
and I'm committed to doing that.

796
0:39:20,6 --> 0:39:24,72
I'm hoping, my tender hope is that
people will use this to some

797
0:39:24,72 --> 0:39:27,16
degree again, so that I can get
feedback.

798
0:39:27,18 --> 0:39:30,6
If there are, if it needs to be
strengthened, I will strengthen

799
0:39:30,6 --> 0:39:30,8
it.

800
0:39:30,8 --> 0:39:33,1
I will pour effort into it to make
sure that happens.

801
0:39:33,28 --> 0:39:35,98
So I wouldn't say there are features
that I want to add.

802
0:39:35,98 --> 0:39:39,1
It's probably already bloated in
terms of features anyway.

803
0:39:39,14 --> 0:39:41,82
So maybe I'll just stop in terms
of adding features.

804
0:39:42,34 --> 0:39:47,3
In terms of favorite features,
I mean, Capacity planning, Nik,

805
0:39:47,3 --> 0:39:51,86
you know, is another worthwhile
endeavor besides incident management.

806
0:39:52,12 --> 0:39:55,22
And it's something that is sorely
needed for super-based customers.

807
0:39:55,62 --> 0:40:0,98
There are functions within Flight
Recorder to help you project

808
0:40:1,36 --> 0:40:2,66
your capacity needs.

809
0:40:2,92 --> 0:40:5,68
Like everything else, those functions
will be strengthened and

810
0:40:5,68 --> 0:40:8,7
improved, but I really would like
to exercise those.

811
0:40:8,86 --> 0:40:13,82
I'd rather this tool is used to
forestall problems rather than

812
0:40:13,94 --> 0:40:15,94
to investigate ones.

813
0:40:15,94 --> 0:40:17,72
Let's just not have problems at
all.

814
0:40:18,0 --> 0:40:19,34
Nikolay: You touched several things.

815
0:40:19,34 --> 0:40:21,78
I wish we had a separate episode
on each.

816
0:40:21,86 --> 0:40:25,52
And actually, I think how we would
approach incident response,

817
0:40:25,52 --> 0:40:29,32
RCA, with this tool, particularly
step-by-step, it's possible.

818
0:40:30,06 --> 0:40:31,74
And also capacity planning, I agree.

819
0:40:32,28 --> 0:40:36,1
And thing related to observer effect,
1 thing everyone who is

820
0:40:36,1 --> 0:40:41,36
using this new breed of tools ticking
on pg_cron should remember

821
0:40:41,44 --> 0:40:45,04
that pg_cron records logs, right?

822
0:40:45,4 --> 0:40:46,92
And you need to clean them up.

823
0:40:46,92 --> 0:40:50,74
And I think we need to team up
and bring some pull requests to

824
0:40:50,74 --> 0:40:54,44
make this configurable and make
it Unix way, Linux way, when

825
0:40:54,44 --> 0:40:57,84
everything is cool, don't say anything
and don't log anything

826
0:40:57,84 --> 0:40:59,44
because it works, right?

827
0:40:59,68 --> 0:41:1,26
Like some levels, right?

828
0:41:1,26 --> 0:41:4,4
Like warning error level for each
job in pg_cron.

829
0:41:5,34 --> 0:41:8,54
And another thing is this verbosity.

830
0:41:9,52 --> 0:41:13,6
pg_cron also depends on this frequency
and it can produce a lot

831
0:41:13,78 --> 0:41:16,74
of bloated logs right in Postgres
as well.

832
0:41:17,5 --> 0:41:23,0
Especially if we go to sub-second
frequency, which is just implemented

833
0:41:23,0 --> 0:41:28,38
for PgQue, just running a single
stored procedure, ticking 10

834
0:41:28,38 --> 0:41:30,04
times within 1 second.

835
0:41:30,44 --> 0:41:33,16
Yeah, I don't know if it's needed
for pg_flight_recorder.

836
0:41:33,68 --> 0:41:34,78
Maybe not at this point.

837
0:41:34,78 --> 0:41:36,5
It's too much precision, right?

838
0:41:37,08 --> 0:41:37,94
Yeah, anyway.

839
0:41:38,36 --> 0:41:39,14
David: Anyway, yes.

840
0:41:39,14 --> 0:41:42,14
So there is a soft dependency on
pg_cron.

841
0:41:42,34 --> 0:41:45,74
pg_cron is also maybe a little overly
chatty.

842
0:41:46,42 --> 0:41:49,28
My kitchen wall clock just silently
ticks away.

843
0:41:49,28 --> 0:41:53,22
It doesn't generate a daily journal
of the fact that it ticked.

844
0:41:53,36 --> 0:41:54,94
That's what I would like.

845
0:41:55,32 --> 0:41:56,68
This is what we have to live with.

846
0:41:56,68 --> 0:41:57,84
So yeah.

847
0:41:58,04 --> 0:41:59,36
Nikolay: Let's create pull requests.

848
0:41:59,54 --> 0:42:2,5
I can create or you can create
and just support each other.

849
0:42:2,5 --> 0:42:4,92
Maybe pg_cron maintainers will
agree that it's...

850
0:42:4,92 --> 0:42:6,14
Actually, I opened the issue.

851
0:42:6,14 --> 0:42:9,8
I didn't see feedback from them,
but it's definitely an issue.

852
0:42:9,8 --> 0:42:12,22
It was an issue before we started
creating these tools.

853
0:42:12,34 --> 0:42:16,52
I have another places where pg_cron
was chatty in logs.

854
0:42:17,02 --> 0:42:17,8
Yeah, anyway.

855
0:42:19,62 --> 0:42:22,58
And final thing for me, a good
place to start is benchmarking.

856
0:42:22,78 --> 0:42:26,8
If you just do benchmarks, just
inject these tools, pg_flight_recorder,

857
0:42:28,26 --> 0:42:32,8
and ask your AI to visualize the
result of that data, like before

858
0:42:32,8 --> 0:42:35,98
destroying instance or something,
destroying database, just dump

859
0:42:35,98 --> 0:42:37,74
that data and visualize it.

860
0:42:37,8 --> 0:42:39,36
It's so easy these days.

861
0:42:39,84 --> 0:42:40,24
David: It is.

862
0:42:40,24 --> 0:42:40,46
Yeah.

863
0:42:40,46 --> 0:42:42,9
The nature of these tools is changing
and this makes it super

864
0:42:42,9 --> 0:42:43,18
easy.

865
0:42:43,18 --> 0:42:44,54
That's what I learned as well.

866
0:42:44,62 --> 0:42:45,48
But yeah, That's it.

867
0:42:45,48 --> 0:42:46,44
No, it's not very profound.

868
0:42:46,44 --> 0:42:49,66
It's not very complicated, but
we need more and better automation

869
0:42:49,84 --> 0:42:51,6
in the community.

870
0:42:51,9 --> 0:42:53,5
This is just 1 small contribution.

871
0:42:54,14 --> 0:42:58,44
Many more to come, I think from
you 2 and hopefully me and others

872
0:42:58,44 --> 0:42:59,1
in the community.

873
0:42:59,1 --> 0:43:0,54
So looking forward to that.

874
0:43:1,22 --> 0:43:1,92
Michael: Nice 1, David.

875
0:43:1,92 --> 0:43:4,14
And just to check, what's the license
here?

876
0:43:4,94 --> 0:43:7,0
David: It's as generous as I could
make it.

877
0:43:7,0 --> 0:43:7,5
Michael: Nice.

878
0:43:8,6 --> 0:43:9,74
So, open and permissive.

879
0:43:9,88 --> 0:43:11,26
David: Open, yeah, exactly.

880
0:43:11,46 --> 0:43:12,42
Michael: Supabase style.

881
0:43:12,9 --> 0:43:14,18
David: Supabase style, it is.

882
0:43:14,18 --> 0:43:18,64
I work at Supabase, but this belongs
to, if anybody, to the

883
0:43:18,64 --> 0:43:19,14
community.

884
0:43:19,4 --> 0:43:19,9
Michael: Wonderful.

885
0:43:19,94 --> 0:43:21,14
Lovely to meet you.

886
0:43:21,18 --> 0:43:21,9
You as well.

887
0:43:21,9 --> 0:43:22,62
David: Very kind.

888
0:43:23,04 --> 0:43:23,66
I appreciate it.

889
0:43:23,66 --> 0:43:23,86
Thank

890
0:43:23,86 --> 0:43:24,36
Nikolay: you.