1
00:00:00,049 --> 00:00:04,009
Michael: Hello and welcome to Postgres fm,
a weekly show about all things Postgres qr.

2
00:00:04,219 --> 00:00:09,110
I am Michael, founder of PG Mustard, and this is
my co-host Nicolai, founder of Postgres Ai Heyi.

3
00:00:09,110 --> 00:00:10,160
What are we talking about today?

4
00:00:10,767 --> 00:00:11,727
Nikolay: Hi Michael.

5
00:00:11,787 --> 00:00:13,407
I have no idea how to call it.

6
00:00:13,468 --> 00:00:14,529
Let's you do it.

7
00:00:14,958 --> 00:00:18,704
Michael: Well, let's start with calling it database version control.

8
00:00:18,854 --> 00:00:21,369
So source control scheme.

9
00:00:22,899 --> 00:00:23,169
Yeah.

10
00:00:23,739 --> 00:00:25,179
Nikolay: database scheme version control.

11
00:00:25,659 --> 00:00:25,989
I don.

12
00:00:26,349 --> 00:00:26,799
Michael: Yeah.

13
00:00:27,099 --> 00:00:32,979
Nikolay: If, if you say just database version control,
someone can think that it's also about data changes

14
00:00:32,979 --> 00:00:35,859
and like time travel for data and it's very different.

15
00:00:36,579 --> 00:00:38,949
We, we mostly talk about schema, but.

16
00:00:39,690 --> 00:00:47,531
Talking about schema, we should not forget about a big data
warnings when we, you need to adjust your data if you need to

17
00:00:47,566 --> 00:00:50,896
make some schema change, which requires data changes, right?

18
00:00:51,436 --> 00:00:54,302
For example, data type change of a, of a column.

19
00:00:54,302 --> 00:00:54,752
So, yeah.

20
00:00:55,056 --> 00:00:55,686
Michael: absolutely.

21
00:00:55,882 --> 00:01:01,160
Well, I think probably in the application world, this is a well covered topic.

22
00:01:01,194 --> 00:01:05,750
there's been various generations of tools that have improved.

23
00:01:05,990 --> 00:01:06,230
Yeah.

24
00:01:06,400 --> 00:01:08,450
Nikolay: I've, I've recently read articles about it.

25
00:01:08,570 --> 00:01:16,350
There are, there are three generations of code version control
or code change management, or how to source code management.

26
00:01:16,710 --> 00:01:20,360
Thirst was very naive and lock based.

27
00:01:20,915 --> 00:01:24,440
Second was CVS Aversion and others per force.

28
00:01:24,590 --> 00:01:27,560
It was better, but it was very centralized.

29
00:01:28,070 --> 00:01:28,850
It was better in

30
00:01:29,100 --> 00:01:32,940
Michael: I remember Subversion team, Foundation server.

31
00:01:33,240 --> 00:01:33,450
Yeah.

32
00:01:33,620 --> 00:01:34,100
Nikolay: Right.

33
00:01:34,160 --> 00:01:34,550
Right.

34
00:01:35,030 --> 00:01:41,215
And the third generation is GI and Mercury, which are also tier, centralized

35
00:01:41,945 --> 00:01:42,435
Michael: Yeah.

36
00:01:42,435 --> 00:01:42,995
And I think,

37
00:01:43,015 --> 00:01:45,195
Nikolay: and like lock free, decentralized.

38
00:01:45,729 --> 00:01:49,339
Michael: I don't know anybody using
mercurial anymore, so it seems like get one

39
00:01:49,387 --> 00:01:50,527
Nikolay: Several years ago.

40
00:01:50,527 --> 00:01:54,997
I know for sure Facebook used it, but I'm not sure about now.

41
00:01:54,997 --> 00:01:55,387
Yes.

42
00:01:56,192 --> 00:01:59,612
Michael: So, yeah, so on the application
side, it feels like this is pretty standard.

43
00:01:59,612 --> 00:02:04,172
It's very rare that I meet anybody, not
source, controlling their application code.

44
00:02:04,442 --> 00:02:07,792
But I do still meet people who don't have the same.

45
00:02:07,886 --> 00:02:09,626
Systems for their database.

46
00:02:09,926 --> 00:02:18,006
And it's super interesting cause this is a problem I was working on back in
20, even as early as 2010, I was working at a company called Redgate that

47
00:02:18,246 --> 00:02:22,436
we're bringing out a  source control product for a database for SQL Server.

48
00:02:22,766 --> 00:02:30,266
So it's super interesting that still, what, 12 years later, still
pretty common for me to come across teams that don't have it.

49
00:02:30,386 --> 00:02:31,496
And even the ones that

50
00:02:31,836 --> 00:02:32,326
Nikolay: Yeah.

51
00:02:32,436 --> 00:02:33,186
For database you mean?

52
00:02:33,186 --> 00:02:33,516
Right?

53
00:02:33,711 --> 00:02:34,191
Michael: For date.

54
00:02:34,191 --> 00:02:34,731
Yeah, exactly.

55
00:02:34,731 --> 00:02:35,001
For da.

56
00:02:35,361 --> 00:02:35,811
Yes.

57
00:02:35,886 --> 00:02:36,096
Nikolay: Mm-hmm.

58
00:02:37,031 --> 00:02:43,335
Michael: And even the ones that do, don't normally have it hooked
up to they're using it for kind of communicating changes between

59
00:02:43,335 --> 00:02:47,263
developers, but not always for deployments or continuous integration

60
00:02:47,318 --> 00:02:49,628
Nikolay: Yeah, let's, let's discuss the goals.

61
00:02:49,628 --> 00:02:52,508
Let's start for with regular code.

62
00:02:52,508 --> 00:02:54,968
Which goals such tool like gi.

63
00:02:55,643 --> 00:02:57,353
Is aiming to solve.

64
00:02:57,743 --> 00:03:05,393
For example, if you are just a sole developer, it still makes
sense for you to, to track changes and to have history, right?

65
00:03:05,453 --> 00:03:09,653
Like, so you can time travel basically in terms of source code.

66
00:03:09,664 --> 00:03:15,941
But do you need branches and do you need multiple versions of your software?

67
00:03:16,596 --> 00:03:18,151
Michael: Well, this is super.

68
00:03:18,211 --> 00:03:19,801
Do you mean for databases or do you mean for

69
00:03:20,041 --> 00:03:21,841
Nikolay: For, for, for regular applications.

70
00:03:21,931 --> 00:03:25,066
Simple like single page application.

71
00:03:25,066 --> 00:03:25,486
I dunno.

72
00:03:25,546 --> 00:03:26,446
Something simple,

73
00:03:27,196 --> 00:03:30,756
Michael: If you're on your own, I think
branches have minimal additional benefit.

74
00:03:30,756 --> 00:03:36,876
I know some people love them for kind of exploring different
paths if they want to try something experimental and then

75
00:03:36,876 --> 00:03:43,356
quickly come back and fix a bug for a customer, that's, it's
like nice to not have been developing on the main branch.

76
00:03:43,386 --> 00:03:45,436
So I can see benefits to that.

77
00:03:45,436 --> 00:03:52,378
And also I think it teaches good discipline if you, when
you do have collaborators, but I don't see massive benefits.

78
00:03:52,378 --> 00:03:52,918
How about you?

79
00:03:53,375 --> 00:03:53,955
Nikolay: Well, yeah.

80
00:03:53,955 --> 00:03:54,890
This is interesting.

81
00:03:54,895 --> 00:03:58,087
So branches can be used in various ways.

82
00:03:58,087 --> 00:04:06,660
And if, if I'm alone and I have solid developer, I still have some benefit
of them because I can, exactly like you said, I can experiment with

83
00:04:06,660 --> 00:04:10,398
some path of development and then abandon it if, if I consider it wrong.

84
00:04:10,668 --> 00:04:15,438
And with branches, it's good that they are very quickly to create them.

85
00:04:15,648 --> 00:04:17,388
It, it costs nothing very quickly.

86
00:04:17,478 --> 00:04:17,778
Boom.

87
00:04:17,778 --> 00:04:19,120
And I have a branch.

88
00:04:19,420 --> 00:04:26,779
I can experiment and I can uh,  Multiple paths of
development, and then I choose which one to take.

89
00:04:26,779 --> 00:04:28,883
Maybe I will combine multiple ones.

90
00:04:29,423 --> 00:04:32,470
This gives flexibility and freedom, right?

91
00:04:32,860 --> 00:04:37,000
But also branches sometimes are used not as for free.

92
00:04:37,120 --> 00:04:39,485
It's like for feature development or back fixes.

93
00:04:39,785 --> 00:04:44,455
They are used to track different versions of software and Postgres.

94
00:04:44,963 --> 00:04:48,263
Doesn't post this project doesn't use feature branches, right?

95
00:04:48,712 --> 00:04:54,452
It uses major version branches like stable release number 15.

96
00:04:54,842 --> 00:04:57,062
We have a branch and all.

97
00:04:57,108 --> 00:05:06,421
Bug fixes are happening inside this branch, so you have
multiple lines of your tree of development and it, I can easily

98
00:05:06,421 --> 00:05:09,931
imagine if I'm so developer, I develop some popular software.

99
00:05:09,931 --> 00:05:12,151
I still can benefit from branches.

100
00:05:12,151 --> 00:05:15,571
Like in this particular case, I maintain some old version.

101
00:05:15,571 --> 00:05:17,611
I already re refactored that a lot.

102
00:05:17,611 --> 00:05:20,821
I already have new version, but I still maintain the old one.

103
00:05:21,396 --> 00:05:22,656
to back fix it sometimes.

104
00:05:22,926 --> 00:05:24,906
So branches are needed for this as well.

105
00:05:25,596 --> 00:05:29,476
But maybe we went too far from databases already, right?

106
00:05:29,836 --> 00:05:30,346
Maybe not.

107
00:05:30,346 --> 00:05:30,796
Maybe yes.

108
00:05:31,185 --> 00:05:36,997
Michael: I think this is all useful, but I would say that
this is where the next major benefit that I was gonna mention

109
00:05:36,997 --> 00:05:43,927
is obviously in teams being able to work in isolation and
not have to worry about stepping on each other's toes.

110
00:05:44,137 --> 00:05:47,347
And that's where it gets super relevant for databases.

111
00:05:47,587 --> 00:05:53,658
So when I was at Redgate, we actually supported two
different models of source controlling your database.

112
00:05:53,663 --> 00:06:00,838
One we called the shared model because most teams at the
time and probably still now tended to have, even if they had

113
00:06:01,298 --> 00:06:04,578
databases locally for developers, they would tend to have one.

114
00:06:05,133 --> 00:06:09,642
Big database, so I know this is where your product comes in key.

115
00:06:10,182 --> 00:06:17,412
But they would have one large database where you could do
proper testing, proper checking on decent data volumes, and

116
00:06:17,417 --> 00:06:19,752
they had to be really careful of stepping on each other's toes.

117
00:06:19,752 --> 00:06:27,858
For example, if you, you and somebody else happened to be working on the same
stool procedure or function at the same time, you get the case where the.

118
00:06:28,298 --> 00:06:29,858
Last one to change, it wins.

119
00:06:29,888 --> 00:06:32,373
And that's a, that's a real problem in terms of testing.

120
00:06:32,373 --> 00:06:37,473
If you don't understand why, why your feature's not
working and you then you realize somebody else changed it.

121
00:06:37,473 --> 00:06:42,516
So we did support that model and we would, we
actually ended up implementing it via locks.

122
00:06:42,726 --> 00:06:48,059
So, or triggers, actually we called it locks, but you could
lock an object so other people couldn't change it, for example.

123
00:06:48,389 --> 00:06:53,154
So, There was like some super interesting
features around that, but that was a stop gap.

124
00:06:53,274 --> 00:06:59,083
I, I saw it as a stop gap because the better
version was we called it the dedicated model.

125
00:06:59,383 --> 00:07:06,943
So where everybody had their own dedicated version of the database
and they could make changes in isolation and then commit those back

126
00:07:06,943 --> 00:07:16,668
to the repository for, with, or the source code and, and maybe push
those to a central system, like deploy those maybe to an environment.

127
00:07:17,178 --> 00:07:18,258
Nikolay: but okay.

128
00:07:18,294 --> 00:07:25,938
But back to gi as being third generation of source
code management systems there's also some small things.

129
00:07:26,358 --> 00:07:28,198
Which delivers some value.

130
00:07:28,198 --> 00:07:32,338
For example, you can easily check diff before you push.

131
00:07:32,428 --> 00:07:33,628
So you already committed.

132
00:07:34,048 --> 00:07:43,839
You can review your own changes and with additional um,
additions in, in form of GitHub or GitLab, you can also

133
00:07:43,869 --> 00:07:47,728
have very, very powerful process of like a review process.

134
00:07:47,765 --> 00:07:54,318
Pool requests, merge requests, and so mul there are
multiple levels of quality control, I would say.

135
00:07:54,318 --> 00:07:54,648
Right.

136
00:07:54,648 --> 00:07:55,938
And this, this is great.

137
00:07:55,943 --> 00:08:06,063
And like also also forking, So if you have open source
project, it's easy to just fully clone it for it and develop.

138
00:08:06,063 --> 00:08:08,977
And if you see some value push back changes.

139
00:08:09,425 --> 00:08:15,826
Propose suggest in form of pull request or merger request,
suggest these changes to the up like upstream project.

140
00:08:16,276 --> 00:08:20,447
But  benefits are not like in the same area.

141
00:08:20,447 --> 00:08:27,689
There are like different in the different areas, but all of
them allow to build very powerful and very good workflow.

142
00:08:27,959 --> 00:08:29,619
And there are not many workflows.

143
00:08:29,994 --> 00:08:34,532
Not one which are proven to be successful in various teams.

144
00:08:34,948 --> 00:08:42,618
But back to databases, we know that currently I would say we
have like also some generation of schema change management tools.

145
00:08:42,701 --> 00:08:46,164
Some people's name them, I believe it's came from Ruby.

146
00:08:46,294 --> 00:08:46,794
I'm not sure.

147
00:08:47,484 --> 00:08:48,984
Database migration tools.

148
00:08:49,694 --> 00:08:50,054
Right?

149
00:08:50,504 --> 00:08:58,458
Because Ruben Rail's active record has part of it called database
migrations, and it's not migrating from Oracle or to pores.

150
00:08:58,463 --> 00:09:00,520
It's about chemo changes basically,

151
00:09:01,360 --> 00:09:05,350
Michael: Yeah, migrating from one version of
your schema to another version of your schema,

152
00:09:05,833 --> 00:09:06,205
and it.

153
00:09:06,205 --> 00:09:07,166
this is really important.

154
00:09:07,166 --> 00:09:15,026
This comes back to your diff point from earlier in code
in applications we can d and it's, it's simple text files

155
00:09:15,026 --> 00:09:18,116
and we can deploy it by overriding one with the other.

156
00:09:18,236 --> 00:09:21,176
And that's fine because there's no data, there's no, Yeah.

157
00:09:21,476 --> 00:09:29,216
Whereas in databases we need, like if we are going to, we can't
just replace if we, if we rename a column for example, we can't

158
00:09:29,216 --> 00:09:32,276
just delete that column and, and add a new one with the new name.

159
00:09:32,276 --> 00:09:32,396
We

160
00:09:32,476 --> 00:09:32,826
Nikolay: in.

161
00:09:32,826 --> 00:09:34,426
If you roll those data, yes.

162
00:09:34,571 --> 00:09:35,201
Michael: Yeah, of course.

163
00:09:35,501 --> 00:09:41,801
So that's, that's a very simple example, but there are loads of
these examples, and I think this is where at Redgate we had a huge

164
00:09:41,801 --> 00:09:46,451
advantage because we had a schema comparison tool that did these DIFs.

165
00:09:46,481 --> 00:09:48,641
It was, that was the starting point,

166
00:09:48,686 --> 00:09:53,426
Nikolay: this schema comparison hap That doesn't happen without context.

167
00:09:53,431 --> 00:09:55,926
Like what happens with this example?

168
00:09:56,586 --> 00:10:02,076
Michael: This is where it gets really complicated, and
I think it's mostly unviable in Postgres at the moment.

169
00:10:02,076 --> 00:10:04,972
I don't think anybody's doing this in Postgres at the moment.

170
00:10:05,008 --> 00:10:07,408
You need to have a live database.

171
00:10:08,053 --> 00:10:09,343
Source code files.

172
00:10:09,343 --> 00:10:11,623
So these, these files are create statements.

173
00:10:11,623 --> 00:10:15,572
They create statement per object and you need to be able to do comparisons.

174
00:10:15,572 --> 00:10:23,696
But of course if you're comparing two static files, how do you know if it's a
rename or dropping one column and creating another one through static files?

175
00:10:23,696 --> 00:10:24,386
You can't tell.

176
00:10:24,386 --> 00:10:24,626
So

177
00:10:24,866 --> 00:10:25,556
you need to.

178
00:10:26,261 --> 00:10:30,521
You need to add migrations or, or what,
what rails cause migrations at some level.

179
00:10:30,521 --> 00:10:38,697
So they ended up kind of backing into that and ended up also acquiring
a, a tool called Flyaway DB because that was very migration based.

180
00:10:38,967 --> 00:10:40,077
So I think just.

181
00:10:40,587 --> 00:10:44,700
Just quickly, I wanted to say, I think there
are two two d completely different models here.

182
00:10:44,700 --> 00:10:51,631
One of which is kind of nice in principle, but I don't think works in
practice, at least not yet, which is these static create statements.

183
00:10:51,811 --> 00:10:58,652
And the other one, which is a, a single dump of your schema
from a point in time and then migrations between versions

184
00:10:58,652 --> 00:11:04,509
thereafter, which is what all of the frameworks like Rails
and Jango are, are doing, and the other tools do as well.

185
00:11:05,139 --> 00:11:05,679
Nikolay: Right.

186
00:11:05,679 --> 00:11:13,155
And if you have some live database, even if it's not
production you can distinguish drop create versus alter column.

187
00:11:13,195 --> 00:11:15,234
If you check catalog system catalogs,

188
00:11:15,274 --> 00:11:16,234
Michael: Like object IDs.

189
00:11:16,234 --> 00:11:16,714
Yeah, exactly.

190
00:11:17,313 --> 00:11:28,494
Nikolay: Not, Yeah, object or, or, or in pos It's P attribute has
at nu, so like numerical or, or number where column is, and you

191
00:11:28,494 --> 00:11:36,864
can say, Oh, this is the same column, like, just rename so you
can distinguish this case or versus it's a completely new column.

192
00:11:37,814 --> 00:11:38,054
Interesting.

193
00:11:38,059 --> 00:11:40,693
So you mentioned Flyway and now I know Redgate.

194
00:11:41,488 --> 00:11:44,369
Few years ago purchased this project.

195
00:11:44,849 --> 00:11:46,627
There are other projects like Ski.

196
00:11:46,627 --> 00:11:53,797
I like ski because it's like very, quite powerful, although
it's written in Pearl, so it's hard to contribute for me.

197
00:11:54,247 --> 00:11:57,515
It's very powerful and works with many, many databases as well.

198
00:11:57,905 --> 00:12:00,645
Also, there is very popular, like   Liquid base, right?

199
00:12:00,645 --> 00:12:01,335
Liquid base is

200
00:12:01,335 --> 00:12:02,581
very jungle.

201
00:12:02,581 --> 00:12:04,471
Also like a frame being framework.

202
00:12:04,471 --> 00:12:14,036
Jungle has that device migration capabilities similar to Rails, but
all of them, correct me if I'm wrong, all of them leave on top of gi.

203
00:12:14,366 --> 00:12:16,586
They rely on GI for version control.

204
00:12:16,646 --> 00:12:16,946
Right.

205
00:12:17,507 --> 00:12:17,837
Okay.

206
00:12:18,437 --> 00:12:19,757
So what's their value?

207
00:12:20,045 --> 00:12:22,235
Why do we, why do we need them at all?

208
00:12:23,150 --> 00:12:24,860
Michael: Is it to do with deployments?

209
00:12:24,920 --> 00:12:26,090
Is it to do with

210
00:12:26,151 --> 00:12:26,634
Nikolay: Okay.

211
00:12:27,359 --> 00:12:32,229
Michael: actually applying the changes, picking them
up and, and tying them to, to code changes as well.

212
00:12:32,229 --> 00:12:37,899
For example, if I have database change that needs to go
alongside an application change, that's my understanding.

213
00:12:38,679 --> 00:12:39,099
Nikolay: Yeah.

214
00:12:39,249 --> 00:12:51,022
If for example, if we talk about, well of obviously framework in Rails and,
and John Go, they are also, they support both frameworks are also ORMs.

215
00:12:51,262 --> 00:12:55,391
So migration parts of those rms also support like upper.

216
00:12:56,096 --> 00:12:58,496
Like higher language, not sequel.

217
00:12:58,946 --> 00:12:59,186
Right.

218
00:12:59,186 --> 00:13:05,507
So because that's all I am, so we can change chemo also
using our native language, avoiding trying to avoid sequel.

219
00:13:05,777 --> 00:13:09,287
But for quite advanced changes, you still need sequel.

220
00:13:09,377 --> 00:13:10,877
For example, if you want trigger.

221
00:13:11,927 --> 00:13:15,797
Probably you need to write some PL purchase
scale code and some SQL code and so on.

222
00:13:16,487 --> 00:13:21,300
If you need some advanced ct, you also
need or define some view with advanced ct.

223
00:13:21,300 --> 00:13:22,450
You need, you need some sequel.

224
00:13:23,080 --> 00:13:29,363
But generally they try to Of to also
support native language like Ruby or Python.

225
00:13:29,963 --> 00:13:36,713
And also, for example, in Rails, I see that people start,
I observe it in multiple projects on the GI lab as well.

226
00:13:36,713 --> 00:13:39,263
They start with not dumping schema.

227
00:13:39,293 --> 00:13:44,573
So they, we support steps and also dumping schema for additional control.

228
00:13:44,873 --> 00:13:46,613
This is our like, current baseline.

229
00:13:46,613 --> 00:13:47,303
And we shifted.

230
00:13:47,303 --> 00:13:50,089
Shifted, but usually projects start with.

231
00:13:50,180 --> 00:13:52,520
Schema, rb, I think in Rails.

232
00:13:52,580 --> 00:14:01,670
And then they switched to Tructure sql, which is a scale regular dump,
and it's easier for Postgres guys to compare it and deal with it.

233
00:14:01,716 --> 00:14:07,356
So yeah, I remember GitLab did it when they decided to stop supporting my.

234
00:14:08,046 --> 00:14:15,226
Before before they supported both SQL and my uh, pogs
and it was not possible to have a structure skill,

235
00:14:15,226 --> 00:14:19,640
which is, which should be like single database only.

236
00:14:20,900 --> 00:14:25,866
But my question is to this like systems, okay,
You provide some, like usually like forward.

237
00:14:26,781 --> 00:14:27,741
And backward.

238
00:14:28,161 --> 00:14:30,591
You can define forward and backward steps.

239
00:14:31,191 --> 00:14:33,651
Sometimes you can define also testing step.

240
00:14:34,219 --> 00:14:35,779
they allow you to deploy.

241
00:14:36,289 --> 00:14:40,909
They allow you sometimes like to have some dry deploy, not doing anything but.

242
00:14:41,189 --> 00:14:43,859
Checking correctness of your changes.

243
00:14:44,339 --> 00:14:48,359
But why don't you care about database data?

244
00:14:48,759 --> 00:14:49,439
Database is data.

245
00:14:49,829 --> 00:14:52,312
It's not tiny, unlike source code.

246
00:14:52,642 --> 00:14:58,402
Sometimes we have millions of code, of course, but
usually it's relatively small compared to database.

247
00:14:58,972 --> 00:15:00,382
But database is usually big.

248
00:15:00,712 --> 00:15:08,694
And if you need to change something without blocking,
not blocking others, you need to have some DML as well.

249
00:15:08,694 --> 00:15:12,204
Not only ddl, and this DML should be split to batches.

250
00:15:12,954 --> 00:15:17,454
Why don't these all tools supported at all?

251
00:15:18,309 --> 00:15:26,680
Michael: I think it's difficult and I think if we look, if we talked
about those three stages of application progress or application,

252
00:15:26,685 --> 00:15:29,620
source control, progress, I think we are in database land.

253
00:15:29,625 --> 00:15:31,390
We're kind of in step one.

254
00:15:31,395 --> 00:15:31,690
Still.

255
00:15:31,690 --> 00:15:34,720
I think the tools are still quite immature for

256
00:15:35,095 --> 00:15:36,205
Nikolay: Generation one.

257
00:15:36,210 --> 00:15:37,285
You, you, you, like.

258
00:15:37,285 --> 00:15:37,435
We

259
00:15:37,503 --> 00:15:39,093
Michael: Maybe, yeah.

260
00:15:39,142 --> 00:15:40,538
Maybe that's a bit harsh.

261
00:15:40,538 --> 00:15:45,108
Maybe, maybe we are like just sneaking into number
two with some of these tools becoming better.

262
00:15:45,111 --> 00:15:52,266
But it's, if, if you're not on rails or Jango, I think it's
actually quite difficult to, to get this stuff up and running.

263
00:15:52,266 --> 00:15:53,256
So, two.

264
00:15:53,300 --> 00:16:01,191
Well one, one issue, for example, even before you worry about
online changes with big data, but you know, actually worrying about

265
00:16:01,191 --> 00:16:05,091
zero downtime deployments even before then, What about merging?

266
00:16:05,091 --> 00:16:10,168
What about if I make some changes on a branch and add a
migration and you make some changes on a branch and add

267
00:16:10,168 --> 00:16:13,138
a migration, but we want to deploy those together like.

268
00:16:13,863 --> 00:16:14,223
If they

269
00:16:14,398 --> 00:16:15,078
Nikolay: Conflict resolution.

270
00:16:15,083 --> 00:16:15,618
It's called

271
00:16:16,118 --> 00:16:16,278
conflict

272
00:16:16,533 --> 00:16:16,803
Michael: Yeah.

273
00:16:17,245 --> 00:16:17,699
so exactly.

274
00:16:17,979 --> 00:16:19,219
Conflict resolution is difficult.

275
00:16:19,719 --> 00:16:19,959
Nikolay: Yeah.

276
00:16:19,959 --> 00:16:21,097
So two, questions.

277
00:16:21,097 --> 00:16:26,943
We erase zero downtime migrations, for
example, Actually basic diff capability.

278
00:16:27,431 --> 00:16:30,431
If we rely only on gi it'll be very weak.

279
00:16:30,731 --> 00:16:35,371
We, for schema, we need diff and not as pja and others.

280
00:16:36,194 --> 00:16:38,407
very naive or negro or others.

281
00:16:38,407 --> 00:16:44,211
So they have very naive diff like for example,
they say create index not mentioning concurrently.

282
00:16:44,451 --> 00:16:49,543
So if we have substantially beacon and substantially loaded database, we will.

283
00:16:49,878 --> 00:16:52,838
Block them and nobody, no.

284
00:16:52,898 --> 00:16:55,895
Only tiny projects can afford using such approach.

285
00:16:56,075 --> 00:17:00,975
So create index should be concurrently alter table changing column data type.

286
00:17:01,035 --> 00:17:03,045
Also, this is harder.

287
00:17:03,105 --> 00:17:06,385
This should be in batches and so diff.

288
00:17:07,030 --> 00:17:14,238
You, you say conflict resolution, I also say deployment
accord in accordance with this zero down time approach.

289
00:17:14,568 --> 00:17:20,238
So you, we need batch management background
like workers, sometimes lasting days.

290
00:17:21,009 --> 00:17:26,949
if you need, for example, to change data, type in
a colon in the table, which has a billion of rows.

291
00:17:27,834 --> 00:17:32,664
You need to, to be very patient and wait
one or two days and split it to batches.

292
00:17:32,964 --> 00:17:37,235
Batch size should be properly adjusted so they don't care about it.

293
00:17:37,235 --> 00:17:41,555
Why Hard problems, or I, I'm, I'm mistaken and it's not needed at all.

294
00:17:42,416 --> 00:17:45,066
Michael: I think it's a hard problem personally, I think

295
00:17:45,246 --> 00:17:46,326
Nikolay: Three problems already.

296
00:17:46,326 --> 00:17:46,686
Already.

297
00:17:46,686 --> 00:17:46,896
Three

298
00:17:46,896 --> 00:17:49,146
problems, ,right?

299
00:17:49,728 --> 00:17:54,206
Michael: And I've actually been, so I've
got a friend working in this area, kind of.

300
00:17:54,823 --> 00:18:01,433
Somebody I used to work with got called Neil Anderson working on a tool
called Postgres Compare, and I've been trying to encourage him because,

301
00:18:01,493 --> 00:18:09,348
because as you say, the diff tools in most the diff tools that have been
added to most IDs and the open source ones aren't super intelligent.

302
00:18:09,348 --> 00:18:09,768
They're good.

303
00:18:09,768 --> 00:18:14,589
I'm sure a lot of people have put time into them,
but I think he's put a lot more time in the charges.

304
00:18:14,619 --> 00:18:17,649
It's a commercial tool and he's been able to put a lot more time into it.

305
00:18:18,039 --> 00:18:18,759
So it's, it's

306
00:18:18,909 --> 00:18:21,449
Nikolay: It doesn't support zero downtime migrations.

307
00:18:21,639 --> 00:18:22,749
Michael: I'm not sure.

308
00:18:22,809 --> 00:18:23,889
It's mostly for

309
00:18:23,889 --> 00:18:30,990
live schema to, Yeah, it's mostly for like live schema
to schema comparisons and I don't know if there, like

310
00:18:31,190 --> 00:18:32,550
Nikolay: scheme of the without data.

311
00:18:33,030 --> 00:18:33,590
Right.

312
00:18:33,643 --> 00:18:35,203
Michael: No, he does do data as well, actually.

313
00:18:35,247 --> 00:18:36,196
It's in, in the same

314
00:18:36,286 --> 00:18:37,076
Nikolay: Oh, okay.

315
00:18:37,306 --> 00:18:37,796
Okay.

316
00:18:38,031 --> 00:18:38,331
Michael: Yeah.

317
00:18:39,021 --> 00:18:40,977
Nikolay: but it's kind of different features, right?

318
00:18:41,137 --> 00:18:41,457
So

319
00:18:41,457 --> 00:18:47,187
Michael: yeah, I think data, like I was gonna mention data here
actually, cuz I think data can sometimes be an important part of

320
00:18:47,187 --> 00:18:50,667
your schema, which again, some of these tools forget and neglect.

321
00:18:50,727 --> 00:18:56,577
If you've got a lookup table, for example, of
what the, the famous one is country codes or Yeah.

322
00:18:56,577 --> 00:18:59,817
Ex anything where you, you need that

323
00:18:59,922 --> 00:19:01,445
Nikolay: People put it to migrations.

324
00:19:01,445 --> 00:19:02,045
I see it.

325
00:19:02,435 --> 00:19:06,035
Although quite often in Flyway, for example, just in short.

326
00:19:06,965 --> 00:19:07,175
Michael: Yeah.

327
00:19:07,205 --> 00:19:13,554
Nikolay: And sometimes some teams consider this as a
bad practice and try to avoid, but some teams do it.

328
00:19:14,574 --> 00:19:18,774
Michael: I don't know what the alternative is
though, if you're gonna, If that's the migration.

329
00:19:19,601 --> 00:19:20,201
Nikolay: good question.

330
00:19:20,201 --> 00:19:21,011
I don't know either.

331
00:19:21,221 --> 00:19:21,611
So.

332
00:19:22,106 --> 00:19:26,906
Michael: So we, we had to worry about that because
we were having create statements in version control.

333
00:19:26,936 --> 00:19:34,196
So if, if you've, if you are trying to do the alternative, which
is have a create statement per object and manage DIFs, you do need

334
00:19:34,196 --> 00:19:38,036
to c you have to choose which tables need to also go in data wise.

335
00:19:38,036 --> 00:19:41,966
So it's, it is a com I think it's a hard problem and complicated.

336
00:19:41,984 --> 00:19:44,396
So I do have.

337
00:19:44,768 --> 00:19:47,858
empathy with the people making these products and making these services?

338
00:19:47,858 --> 00:19:55,613
I do think databases are fundamentally harder to do this for than
applications, but I do also think there's a lot we can do to make them better.

339
00:19:55,742 --> 00:20:00,962
the create index concurrently as a perfect example of quite
a simple change that probably should be in most of them.

340
00:20:01,910 --> 00:20:02,510
Nikolay: Yeah.

341
00:20:02,510 --> 00:20:03,980
Also, you know, you know what?

342
00:20:03,980 --> 00:20:13,510
I think if  somebody developed open source tool, which supports zero
downtime, diff, I think it would solve the problem of merging and conflict

343
00:20:13,510 --> 00:20:17,938
is adoption because we would see this deep and this is our emerging process.

344
00:20:17,938 --> 00:20:19,018
We just.

345
00:20:19,474 --> 00:20:20,074
Maybe not.

346
00:20:20,127 --> 00:20:24,927
actually If we merge to, for example, main
branch, it's our deployment to production.

347
00:20:25,248 --> 00:20:26,118
What do you think?

348
00:20:26,695 --> 00:20:31,045
Michael: Yeah, I think mergings trickier, but we
might be able to lean on, get for some of that.

349
00:20:31,045 --> 00:20:39,304
So for example, if we are changing the same function, , the function's gonna
look very similar between our two, but if we're ordering a table that's,

350
00:20:39,634 --> 00:20:42,724
maybe we can apply one after the other, but I'm not sure when order matters.

351
00:20:43,294 --> 00:20:44,324
It's a tricky one.

352
00:20:44,361 --> 00:20:46,701
I haven't thought about this for a few years, to be honest.

353
00:20:47,706 --> 00:20:50,796
Nikolay: if like, if you consider a functions triggers as a part of.

354
00:20:51,306 --> 00:20:57,346
Schema, and I think we should this also, we probably, yeah, we
probably already, in previous episodes, we already mentioned the

355
00:20:57,346 --> 00:21:03,199
problem of include in all these tools, current generation 1.5, right?

356
00:21:03,259 --> 00:21:13,129
Tools, if we include different file and keep functional budget there
for convenience, like for example, each function, it, it's a non.

357
00:21:14,014 --> 00:21:16,034
Easier to find it, navigate and so on.

358
00:21:16,034 --> 00:21:20,375
Do we still want to support long sequences of deployments?

359
00:21:20,885 --> 00:21:26,195
For example, we have several installations of
our product and one is already very up to date.

360
00:21:26,465 --> 00:21:29,495
And another one is letting few months of changes.

361
00:21:30,009 --> 00:21:37,717
And the problem is when you include you need the current
version of file, but when you already function changed many.

362
00:21:38,662 --> 00:21:47,122
And you try to replay old migration file it, it tries to
include the very fresh, very new function body and it's,

363
00:21:47,127 --> 00:21:50,152
it's not what it is expected like couple of months ago.

364
00:21:50,152 --> 00:21:50,452
Right.

365
00:21:51,142 --> 00:21:51,852
And you have a problem.

366
00:21:52,348 --> 00:21:59,893
. So you need to somehow fetch, you can use GI history and
fetch older version, which corresponds to this migration file.

367
00:22:00,466 --> 00:22:02,716
But why Nobody cares about this.

368
00:22:02,716 --> 00:22:04,216
I needed to implement it myself.

369
00:22:04,216 --> 00:22:09,851
And like I thought about how to contribute it
to ski and it was like, Quite hard actually.

370
00:22:09,851 --> 00:22:13,811
I raised the topic about it and my linguist and I didn't see support.

371
00:22:13,811 --> 00:22:14,741
Things like it.

372
00:22:14,741 --> 00:22:17,951
It looked to me like I'm alone with this problem.

373
00:22:18,071 --> 00:22:19,991
Like nobody cares, right?

374
00:22:19,991 --> 00:22:21,221
Maybe I'm really alone.

375
00:22:21,371 --> 00:22:24,369
How people do this or from your practice is it the problem?

376
00:22:24,374 --> 00:22:27,549
Or people just don't need to replay along sequences of change.

377
00:22:28,401 --> 00:22:31,851
Michael: I've seen more people kind of like try migrating forwards.

378
00:22:31,911 --> 00:22:36,771
I d I actually see quite a few people not even
worrying about backwards, like moving backwards.

379
00:22:37,041 --> 00:22:39,666
And it's it's an interesting, potentially.

380
00:22:39,871 --> 00:22:47,161
Nikolay: back backwards, it's about also if you have multiple,
you have multiple sit ups and one of them is legging quite

381
00:22:47,581 --> 00:22:55,054
much, they need to replay, say 10 database migrations
and they fetched the very fresh version of our code.

382
00:22:55,354 --> 00:22:59,794
But this code already has a much newer function file.

383
00:23:00,664 --> 00:23:06,025
And then when they try to replay some old
migration, which tries to include this file.

384
00:23:06,265 --> 00:23:10,616
It ex it expected very different function or view, for example.

385
00:23:10,796 --> 00:23:14,636
So problem with includes including different files.

386
00:23:14,816 --> 00:23:15,506
It's a problem.

387
00:23:15,776 --> 00:23:21,480
You always need to snapshot whole function value
to migration, which like blows your code,  right?

388
00:23:21,868 --> 00:23:24,480
But you cannot keep a function in a separate file.

389
00:23:24,690 --> 00:23:26,160
This is the problem with all these.

390
00:23:27,300 --> 00:23:33,486
Michael: Yeah, I'm, If we only have migrations from
a, from a baseline, I don't think this is a problem.

391
00:23:34,251 --> 00:23:44,316
Nikolay: Problem with starts when you try to use, including,
include some file like with ski, it's very ski feeds files to P sql.

392
00:23:44,316 --> 00:23:45,486
So you can use backslash.

393
00:23:45,486 --> 00:23:45,756
I

394
00:23:46,946 --> 00:23:47,616
Michael: Oh, okay.

395
00:23:47,616 --> 00:23:47,976
Yeah.

396
00:23:47,976 --> 00:23:48,376
Okay.

397
00:23:48,846 --> 00:23:49,656
Nikolay: this is a problem.

398
00:23:50,736 --> 00:23:57,546
And if you use besla shy, and this is like two month old
migration, this file already changed and you, you need to re.

399
00:23:58,562 --> 00:24:00,242
And, uh, this is the problem.

400
00:24:00,272 --> 00:24:00,362
Mm-hmm.

401
00:24:00,973 --> 00:24:07,633
. 
Otherwise you need to, every time to, to put a whole
function body to migration and bloods, everything is.

402
00:24:08,098 --> 00:24:08,818
Very huge.

403
00:24:09,058 --> 00:24:16,168
But I mean the, this, if you, if you put every time you say create
a replace function or drop function and create it again and put

404
00:24:16,173 --> 00:24:25,738
everything and in reward as well, like whole function, again, it blows
it so much that I, I'm starting to ask why, like, do we have GI still?

405
00:24:25,768 --> 00:24:28,018
No, Nobody cares about, We have good here.

406
00:24:28,318 --> 00:24:34,258
This is why it was created for Right . Why
we copy paste functions, function versions.

407
00:24:34,348 --> 00:24:36,118
Where is version control at all?

408
00:24:36,581 --> 00:24:46,241
Michael: this is what I liked about the single create files per Object,
but it has such big limitations right now that I don't think it's viable.

409
00:24:46,241 --> 00:24:49,811
So I think we have to accept the putting the full object back in.

410
00:24:50,111 --> 00:24:51,071
Maybe you could argue

411
00:24:51,101 --> 00:24:52,781
Nikolay: But how I, I don't understand.

412
00:24:52,781 --> 00:24:56,688
If you keep object in separate file, how you replace sequences of changes.

413
00:24:56,688 --> 00:24:58,769
So like, You still have versions, right?

414
00:24:59,159 --> 00:25:03,569
Like our first version, then like maybe timestamp, you take day or something.

415
00:25:03,569 --> 00:25:04,079
Timestamp.

416
00:25:04,379 --> 00:25:05,329
This is new version.

417
00:25:05,329 --> 00:25:06,249
New version, new version.

418
00:25:06,389 --> 00:25:09,689
If you keep object in separate file, you need to include it.

419
00:25:10,454 --> 00:25:16,611
And if you try to replay very old migration with
very new state of code, you'll have problem.

420
00:25:17,616 --> 00:25:22,266
Michael: I think you need to have a database aware diff functionality.

421
00:25:22,296 --> 00:25:26,396
And it also probably needs another folder of overrides.

422
00:25:26,401 --> 00:25:35,924
So if you have, for example um, easy migrations, like adding, if we are
always adding, adding columns, adding indexes, adding tables, easy stuff.

423
00:25:35,977 --> 00:25:37,867
We don't need to have any special cases.

424
00:25:37,867 --> 00:25:39,581
We can just The diffs are easy.

425
00:25:40,031 --> 00:25:45,309
If we do something more tricky, let's say from I know
GIT doesn't work like this, but let's say from version

426
00:25:45,309 --> 00:25:49,119
15 to version 16, we need a more complex migration.

427
00:25:49,269 --> 00:25:51,819
We can say, we can put in a folder somewhere.

428
00:25:52,286 --> 00:26:00,086
for going between 15 and 16, which includes going from
14 to 18 or any other way past that boundary, we can say.

429
00:26:00,631 --> 00:26:06,217
Don't use what you plan to use, use this
uh, override instead, but it gets comp.

430
00:26:06,277 --> 00:26:09,937
That's kind of like a hybrid between the
two models we've been discussing and I,

431
00:26:10,187 --> 00:26:14,063
Nikolay: Well, there is a big difference between like objects with data.

432
00:26:14,603 --> 00:26:15,653
And very lightweight.

433
00:26:15,653 --> 00:26:16,073
Objects.

434
00:26:16,073 --> 00:26:17,753
Objects with data is is tables.

435
00:26:17,753 --> 00:26:18,533
It's called tables.

436
00:26:18,533 --> 00:26:18,923
That's it.

437
00:26:18,953 --> 00:26:20,333
Only tables have data.

438
00:26:20,747 --> 00:26:24,587
If we forget about marginalized use, which
is derivative for, for from tables, right?

439
00:26:25,007 --> 00:26:27,047
View views, triggers, functions.

440
00:26:27,047 --> 00:26:29,810
They are light that don't have data, so we always

441
00:26:29,885 --> 00:26:30,125
Michael: you

442
00:26:30,530 --> 00:26:31,940
Nikolay: them, throw them,

443
00:26:31,940 --> 00:26:33,500
away and electrify.

444
00:26:34,475 --> 00:26:35,195
Well, yeah, Yeah.

445
00:26:35,195 --> 00:26:38,507
But in sense it's like part of it's, it's like part of tables

446
00:26:38,937 --> 00:26:39,347
probably, right?

447
00:26:39,347 --> 00:26:41,747
Because they cannot live without tables.

448
00:26:41,747 --> 00:26:47,236
It's like, okay, it's addition to tables, add
onto tables, but functions, triggers and views.

449
00:26:47,326 --> 00:26:47,926
Regular views.

450
00:26:47,926 --> 00:26:51,616
Maybe we trust use as well, because they
like have only derivative data as well.

451
00:26:52,246 --> 00:26:56,326
You, you always can redefine them and you can keep them in files.

452
00:26:56,356 --> 00:26:56,836
That's why

453
00:26:57,676 --> 00:26:59,356
Michael: Wow, that's a really interesting point.

454
00:26:59,457 --> 00:27:02,127
I haven't seen a system that those two.

455
00:27:02,288 --> 00:27:03,758
Nikolay: I built It a couple of times,

456
00:27:03,837 --> 00:27:04,317
Michael: Okay.

457
00:27:04,317 --> 00:27:04,707
Cool.

458
00:27:04,977 --> 00:27:06,327
So you had one system for

459
00:27:06,627 --> 00:27:15,441
Nikolay: Yeah, remember we discussed the approach when functions are
kept in separate ches and you can keep old version of this is smart

460
00:27:15,441 --> 00:27:23,798
and good, and I think it's possible to build without copy based seal,
like to rely on geek for version control and have like version one

461
00:27:23,798 --> 00:27:32,198
schema functions version one, functions version two, and being able
to even keep them several versions at the same time and different

462
00:27:32,528 --> 00:27:35,695
application notes can address different versions of functions.

463
00:27:35,695 --> 00:27:41,847
This is very smart and this is exactly when we
probably need to functions in separate files as well.

464
00:27:42,267 --> 00:27:42,567
Right.

465
00:27:42,777 --> 00:27:50,955
And I, I should mention that this idea was like came
to mind of Valentine kri long, long ago from Solano and

466
00:27:50,955 --> 00:27:53,925
then propagated to Russia, to ATO and others and so on.

467
00:27:54,195 --> 00:27:55,095
So it's great idea.

468
00:27:55,364 --> 00:27:56,234
Still relevant.

469
00:27:56,234 --> 00:28:01,232
I think I'm not sure about triggers because triggers,
they consist of trigger function plus trigger itself.

470
00:28:01,232 --> 00:28:04,772
So trigger itself, it's a bit easier that it also requires.

471
00:28:04,889 --> 00:28:05,339
Things.

472
00:28:05,489 --> 00:28:08,969
Oh, by the way, big question number four or five probably.

473
00:28:09,389 --> 00:28:12,569
Why do all these guys don't care about it?

474
00:28:12,629 --> 00:28:16,427
Like about simple problem of looking attempts?

475
00:28:16,967 --> 00:28:26,827
If you try to alter table at com, you always must have
small, locked time out and not be ready to try otherwise.

476
00:28:27,514 --> 00:28:31,804
Well, you, you don't notice the problem if
you have not big load, not high lot, right?

477
00:28:32,314 --> 00:28:33,064
Not high lot.

478
00:28:33,424 --> 00:28:42,768
But if you have noticeable load like thousand gps, sooner or later
you will bump into this problem very badly as many did already.

479
00:28:43,128 --> 00:28:44,898
So you need be graceful.

480
00:28:45,463 --> 00:28:53,849
Otherwise you try to alter somebody looks and
you cannot acquire a look and you have huge.

481
00:28:54,749 --> 00:28:59,089
Line, including selects after you waiting, so you block everyone.

482
00:28:59,789 --> 00:29:01,199
They don't care about it.

483
00:29:01,503 --> 00:29:01,863
Why

484
00:29:02,433 --> 00:29:02,883
Michael: Well,

485
00:29:03,483 --> 00:29:04,203
Nikolay: Because it's generation

486
00:29:04,293 --> 00:29:06,753
1.5, generation 1.5.

487
00:29:06,753 --> 00:29:07,083
That's why.

488
00:29:07,113 --> 00:29:10,833
Michael: It's a really good point, but that
also lives in a slightly different place, right?

489
00:29:10,833 --> 00:29:13,293
That lives in the, in the config file.

490
00:29:13,323 --> 00:29:18,573
And that's, you know, not necessarily well, is
that in version control, ho, like hopefully.

491
00:29:18,573 --> 00:29:21,303
But it's not in the same version control for most people.

492
00:29:21,308 --> 00:29:25,083
So it's deployed in a different way, updating in a different way.

493
00:29:25,143 --> 00:29:26,823
So it's very, very interesting.

494
00:29:26,913 --> 00:29:27,873
Yeah, really good point.

495
00:29:27,873 --> 00:29:30,453
Maybe, maybe that should be in there as well.

496
00:29:31,078 --> 00:29:33,560
Nikolay: I hope they will develop in the right directions.

497
00:29:33,560 --> 00:29:35,120
So people need these directions.

498
00:29:35,120 --> 00:29:40,102
People need zero downtime, migrations,
conflict resolution graceful deployments.

499
00:29:40,132 --> 00:29:40,432
Right?

500
00:29:40,522 --> 00:29:42,532
And by the way, I want to advertise.

501
00:29:42,592 --> 00:29:48,729
Uh, GitLab has open source and background migrations
with, it's called Migration helper helpers do B file.

502
00:29:49,029 --> 00:29:50,469
So it's open source.

503
00:29:50,469 --> 00:29:51,399
Anyone can check.

504
00:29:51,399 --> 00:30:00,673
And if you are Ruby user developer, you can benefit from
learning their experiences, an enormous volume of experiences

505
00:30:00,678 --> 00:30:03,737
already coded there to support zero downtime aggressions.

506
00:30:04,157 --> 00:30:10,413
Also, actually GitLab has many installations and some of them
may leg and need to, This is exactly the problem I described.

507
00:30:11,118 --> 00:30:15,378
I'm not sure if they keep some objects in separate files, probably not.

508
00:30:15,468 --> 00:30:21,358
But they have this problem to deployments
should be without issues on many, many setup.

509
00:30:21,358 --> 00:30:22,438
So it's, it's,

510
00:30:22,438 --> 00:30:23,398
interesting to check.

511
00:30:23,669 --> 00:30:31,197
I hope that one day they will open source it separately as a
separate jam or like library to be great for Ruby developers.

512
00:30:32,022 --> 00:30:32,922
Michael: That'd be really cool.

513
00:30:32,952 --> 00:30:36,554
there's a really good blog post, about zero
downtime migrations from the team at Braintree.

514
00:30:36,584 --> 00:30:37,544
That's excellent.

515
00:30:37,544 --> 00:30:38,984
That I think I'll include as well.

516
00:30:39,550 --> 00:30:43,300
Nikolay: Don't forget to include my lifetime
out post about this red rice problem.

517
00:30:43,345 --> 00:30:44,035
Michael: Will do.

518
00:30:44,102 --> 00:30:48,806
And yeah, I would love to hear from anybody listening to
this that's screaming at us and saying that you've got this

519
00:30:48,806 --> 00:30:51,386
solved or you've got a really good process on your side.

520
00:30:51,386 --> 00:30:53,012
It would be great to hear how people

521
00:30:53,096 --> 00:30:55,646
Nikolay: Yeah, maybe we are wrong like this.

522
00:30:56,126 --> 00:31:02,066
This episode is full of criticism, but it's
opportunity to improve for our tooling.

523
00:31:02,066 --> 00:31:06,909
So maybe someone can say there is some
very magic tool which solves all problems.

524
00:31:06,914 --> 00:31:07,179
Right.

525
00:31:07,576 --> 00:31:08,356
Michael: Yeah, let us know.

526
00:31:08,632 --> 00:31:09,202
Wonderful.

527
00:31:09,550 --> 00:31:09,580
Nikolay: Okay.

528
00:31:10,124 --> 00:31:10,904
Michael: Well, thanks Nicola.

529
00:31:10,994 --> 00:31:12,464
Thanks everybody for listening.

530
00:31:12,673 --> 00:31:18,583
appreciate everybody that's got in touch recently with feedback
and yeah, looking forward to speaking to you next week.

531
00:31:18,938 --> 00:31:19,498
Nikolay: Great.

532
00:31:19,503 --> 00:31:20,008
Thank you.

533
00:31:20,145 --> 00:31:20,535
Bye bye.