1
00:00:00,078 --> 00:00:02,758
German: But I think the, the most important thing, the databases

2
00:00:02,768 --> 00:00:07,248
have improved to actually be run in a more volatile environment.

3
00:00:12,914 --> 00:00:14,544
Corey: Welcome to Screaming in the Cloud.

4
00:00:14,894 --> 00:00:16,044
I'm Corey Quinn.

5
00:00:16,354 --> 00:00:20,034
Uh, German Eichberger is a principal AI engineering

6
00:00:20,044 --> 00:00:24,914
manager at Microsoft, which is an interesting position to

7
00:00:24,924 --> 00:00:28,544
start from because today we're talking about DocumentDB.

8
00:00:28,743 --> 00:00:30,154
German, thank you for joining me.

9
00:00:30,824 --> 00:00:31,923
German: Thanks for having me.

10
00:00:32,414 --> 00:00:35,044
It's been a, I'm always an admirer of your podcast.

11
00:00:35,044 --> 00:00:36,794
I'm happy to finally be on it.

12
00:00:37,443 --> 00:00:40,513
Corey: This episode is sponsored in part by my day job, Duckbill.

13
00:00:41,123 --> 00:00:43,744
Do you have a horrifying AWS bill?

14
00:00:44,003 --> 00:00:45,893
That can mean a lot of things.

15
00:00:46,134 --> 00:00:51,223
Predicting what it's going to be, determining what it should be, negotiating

16
00:00:51,224 --> 00:00:56,633
your next long-term contract with AWS, or just figuring out why it increasingly

17
00:00:56,634 --> 00:01:01,163
resembles a phone number, but nobody seems to quite know why that is.

18
00:01:01,463 --> 00:01:05,044
To learn more, visit duckbillhq.com.

19
00:01:05,334 --> 00:01:09,373
Remember, you can't duck the Duckbill bill, which my

20
00:01:09,373 --> 00:01:13,644
CEO reliably informs me is absolutely not our slogan.

21
00:01:14,303 --> 00:01:17,053
People always say that at the beginning, and at the end they're like, "Ah,

22
00:01:17,053 --> 00:01:20,773
I, I certainly have a different opinion now," but we'll see if we get there.

23
00:01:20,964 --> 00:01:25,614
So you're a principal AI engineering manager, but you work on databases.

24
00:01:25,633 --> 00:01:29,283
I naively, there was a time I would've thought that those were two orthogonal

25
00:01:29,973 --> 00:01:35,604
things, but now it feels like everything is getting drawn under the AI umbrella.

26
00:01:36,134 --> 00:01:36,983
How did that work?

27
00:01:37,863 --> 00:01:39,333
German: Uh, that's definitely correct.

28
00:01:39,354 --> 00:01:42,343
So, so AI is, is everywhere nowadays.

29
00:01:42,373 --> 00:01:47,703
Um, you use AI for code re- help you with code reviews, use AI to generate code.

30
00:01:48,263 --> 00:01:52,373
Uh, you, you need an agent MD in your open source project

31
00:01:52,383 --> 00:01:55,603
that you don't get, uh, some, some open-source agent

32
00:01:55,604 --> 00:01:58,473
coming and, and kind of ratting you out everywhere.

33
00:01:58,903 --> 00:02:03,054
So that's why, that's why we are doing AI everywhere and, and it's my title.

34
00:02:03,054 --> 00:02:07,543
Also, the other thing is, uh, that a recent, uh, salary surveys show that

35
00:02:07,943 --> 00:02:11,943
AI engineers get paid better, and so I want to signal to the world that

36
00:02:11,943 --> 00:02:15,713
I'm no longer a software engineer, and I moved on to be an AI engineer.

37
00:02:16,163 --> 00:02:17,203
Corey: It, it's funny.

38
00:02:17,203 --> 00:02:21,533
Uh, previous generation we did a whole bunch of talk pay stuff at, uh,

39
00:02:21,564 --> 00:02:25,784
DevOpsDays over a course of a couple years, and going from cis admin

40
00:02:25,784 --> 00:02:31,243
to DevOps or SRE was more or less a 30% to 40% pay hike with not a

41
00:02:31,243 --> 00:02:35,093
lot of difference in the tooling, the processes, the responsibilities.

42
00:02:35,153 --> 00:02:36,524
The world marched on.

43
00:02:36,834 --> 00:02:40,314
Yeah, when, when you and the market disagree on something financially,

44
00:02:40,394 --> 00:02:43,864
generally assume you're the one that's wrong and act accordingly.

45
00:02:44,702 --> 00:02:45,422
German: Absolutely.

46
00:02:45,432 --> 00:02:49,542
It's now the software engineering moment to change our titles.

47
00:02:50,162 --> 00:02:50,652
Corey: Exactly.

48
00:02:50,652 --> 00:02:53,681
So, so you have done a lot of interesting stuff.

49
00:02:53,681 --> 00:02:57,401
You're on the technical steering committee for the open source DocumentDB

50
00:02:57,402 --> 00:03:02,112
project, which is sort of where I wanna start, uh, because my, my position

51
00:03:02,112 --> 00:03:06,072
I'm starting from more or less, uh, distills down to what the hell?

52
00:03:06,382 --> 00:03:09,971
Uh, because once upon a time, Microsoft launched an open source

53
00:03:09,972 --> 00:03:13,431
project that was called DocumentDB, which then became the name of

54
00:03:13,462 --> 00:03:18,471
AWS's MongoDB compatible service, which was itself almost called

55
00:03:18,581 --> 00:03:22,452
DocumentDB internally at Microsoft before that became Cosmos DB.

56
00:03:22,681 --> 00:03:26,331
Uh, walk me through the history and the meeting ideally where

57
00:03:26,341 --> 00:03:30,152
someone says, "Yeah, this name, no confusion here at all."

58
00:03:30,552 --> 00:03:34,721
German: Well, so, so Cosmos DB wa- was originally named DocumentDB, and,

59
00:03:34,872 --> 00:03:40,142
and then s- so Microsoft holds the DocumentDB trademarks and copyrights.

60
00:03:40,661 --> 00:03:44,692
However, AWS, uh, for Amazon DocumentDB, I think the

61
00:03:44,692 --> 00:03:48,031
combination of both of them, they have the copywi- copyright.

62
00:03:49,061 --> 00:03:52,521
And so when we launched, uh, DocumentDB, we actually were looking for a

63
00:03:52,522 --> 00:03:57,181
different name, but it's very difficult to find new names nowadays which aren't

64
00:03:58,263 --> 00:04:03,154
copyrighted by someone else or which not a tiny company somewhere is using.

65
00:04:03,754 --> 00:04:08,634
And so, and so we went back to the names we already had copyrighted

66
00:04:08,634 --> 00:04:12,403
and found DocumentDB, which also describes what we are doing very well.

67
00:04:12,944 --> 00:04:16,693
And so we went and, and launched it as DocumentDB.

68
00:04:17,173 --> 00:04:20,853
We also got the permission from Amazon, they are part of the,

69
00:04:21,264 --> 00:04:24,403
uh, open source DocumentDB project to name it DocumentDB.

70
00:04:24,863 --> 00:04:27,404
So even if there's confusion, we have that all covered.

71
00:04:27,893 --> 00:04:31,824
And, and we then donated the copyright and everything to the Linux Foundation.

72
00:04:31,824 --> 00:04:34,433
They, they happen to have a lot of lawyers who made sure

73
00:04:34,433 --> 00:04:37,413
that we are not doing anything they might be liable for.

74
00:04:37,853 --> 00:04:40,123
So, so that's all, all worked out.

75
00:04:40,123 --> 00:04:43,883
And so now we have, uh, DocumentDB, the open source project,

76
00:04:44,164 --> 00:04:49,033
we have Amazon DocumentDB, and nowadays even Azure DocumentDB.

77
00:04:49,953 --> 00:04:52,853
So there's DocumentDB all over the place.

78
00:04:53,303 --> 00:04:55,783
And I had in the beginning, people come to me and said, "Hey,

79
00:04:55,783 --> 00:04:58,914
this DocumentDB, isn't that like an open source Cosmos DB?"

80
00:04:58,914 --> 00:05:02,373
And I say, "No, it's a completely different database." Yeah.

81
00:05:02,754 --> 00:05:03,903
DB, it's not the same.

82
00:05:04,364 --> 00:05:04,544
Corey: Yeah.

83
00:05:04,544 --> 00:05:07,283
The, the, the pitch as I understand it, and please correct me if I'm

84
00:05:07,283 --> 00:05:12,373
wrong, is it's a MongoDB compatible API, but where does the data live?

85
00:05:12,383 --> 00:05:14,604
That's right, it goes into PostgreSQL.

86
00:05:14,633 --> 00:05:16,773
Yes, that's how I pronounce it because I'm obnoxious.

87
00:05:16,974 --> 00:05:19,833
But, but it is, to my understanding, Postgres underneath.

88
00:05:20,364 --> 00:05:24,713
It, it's BSON bolted on top of it, rather than building

89
00:05:24,713 --> 00:05:27,583
out a, uh, under your own document store under the hood.

90
00:05:27,803 --> 00:05:28,623
Why do that?

91
00:05:28,633 --> 00:05:33,673
What, what does PostgreSQL get you that's worth that weird impedance mismatch?

92
00:05:33,823 --> 00:05:37,013
German: So, so Postgres has really great, uh, programming models.

93
00:05:37,013 --> 00:05:39,743
So when my VP came to me and said, "Hey, we want to

94
00:05:39,743 --> 00:05:41,443
develop on Postgres," I was a little bit skeptical.

95
00:05:41,443 --> 00:05:45,483
I said, "Uh, that, that, that doesn't seem better than writing a database from

96
00:05:45,483 --> 00:05:49,763
scratch because…" But then I started working on that and it's really nice.

97
00:05:49,763 --> 00:05:51,053
They do the garbage collection.

98
00:05:51,053 --> 00:05:54,063
They give a lot of, uh, database primitives, and so it's

99
00:05:54,093 --> 00:05:56,883
probably one of the best, uh, programming environment if

100
00:05:56,883 --> 00:06:00,343
you're doing databases to develop, uh, database functionality.

101
00:06:00,343 --> 00:06:05,213
And, and so we, we went and developed an e- an extension, which, which

102
00:06:05,223 --> 00:06:10,233
implements the BSON protocol and a lot of the Mongo API functions as

103
00:06:10,584 --> 00:06:15,978
Postgres functions And so you can even use that straight in, in Postgres.

104
00:06:15,978 --> 00:06:18,948
You can go in with psql, you can use our functions, and then you

105
00:06:18,948 --> 00:06:23,178
get document, uh, then can work on documents and select documents

106
00:06:23,178 --> 00:06:26,797
and do things with documents with the B- uh, with BSON documents.

107
00:06:27,428 --> 00:06:30,977
And then on top of that we have a gateway which speaks the Mongo

108
00:06:30,978 --> 00:06:34,677
protocol, and that trans- is very thin and that translates it

109
00:06:34,728 --> 00:06:38,547
just to, to the Postgres calls and then everything happens in

110
00:06:38,548 --> 00:06:41,918
Postgres and, and the re- and as I said, it's easy to program.

111
00:06:42,128 --> 00:06:44,338
Postgres is a database which is pretty

112
00:06:44,347 --> 00:06:46,408
battle tested, been around for a long time.

113
00:06:46,408 --> 00:06:48,398
I think they did it in the '60s, so it's

114
00:06:48,407 --> 00:06:52,867
like 60, 40, uh, almost 60 years, years out.

115
00:06:53,238 --> 00:06:57,537
More than 60 years out, so it's battle tested, has built-in replication backups.

116
00:06:57,537 --> 00:07:02,580
You can use everything you're used to with Postgres And so, and

117
00:07:02,580 --> 00:07:05,849
so it was a natural choice for us instead of building another

118
00:07:05,860 --> 00:07:09,610
database engine, which is always hard, and then getting that right.

119
00:07:09,659 --> 00:07:14,599
We, we just bolted it onto Postgres and has been, has been pretty good for us.

120
00:07:15,550 --> 00:07:20,820
Corey: One thing I find odd is that if you dig into DocumentDB a bit,

121
00:07:20,820 --> 00:07:25,370
you've committed to unmodified upstream PostgreSQL instead of forking it.

122
00:07:25,820 --> 00:07:27,999
That can serve as a hard constraint.

123
00:07:28,519 --> 00:07:32,079
What's that forced you to do the hard way that if you've built

124
00:07:32,079 --> 00:07:34,650
your own data store or forking it, that would've made easier?

125
00:07:35,449 --> 00:07:37,610
German: Uh, we, we wanna be good Postgres citizens

126
00:07:37,649 --> 00:07:41,319
too, so, so that's, so, so that's why we did that.

127
00:07:42,349 --> 00:07:44,720
And also the, we wanna be open source, and so we wanna make it the

128
00:07:44,720 --> 00:07:49,109
most compatible way and the easiest to run for people possible.

129
00:07:49,799 --> 00:07:53,119
So now on the other hand, when you run it as a service, you might

130
00:07:53,120 --> 00:07:57,679
do other choices on, uh, what you do with the p- underlying Postgres

131
00:07:57,689 --> 00:08:01,109
and have private versions of Postgres or something like that.

132
00:08:01,509 --> 00:08:05,079
But for the open source thing was important for us to have a really

133
00:08:05,079 --> 00:08:08,639
good reference implementation everybody can follow, everybody can do.

134
00:08:09,210 --> 00:08:13,759
Corey: One thing that I, that I think is, uh, always a challenge when someone

135
00:08:13,759 --> 00:08:18,229
says, "Oh, we're compatible with X." Uh, okay, let's dig into that a bit.

136
00:08:18,259 --> 00:08:22,129
Uh, y- you're claiming 100% compatibility with MongoDB drivers.

137
00:08:22,189 --> 00:08:24,160
Uh, uh, or am I misunderstanding that?

138
00:08:24,199 --> 00:08:27,029
German: So basically, it's really hard to be 100% compatible

139
00:08:27,029 --> 00:08:30,999
with, uh, MongoDB, so we are not necessarily claiming that.

140
00:08:31,000 --> 00:08:35,669
So we, so, so we are say- saying we are MongoDB API compatible, and most

141
00:08:35,689 --> 00:08:41,569
workloads will work without any changes with, uh, with, uh, DocumentDB.

142
00:08:41,960 --> 00:08:46,389
However, there is, uh, since MongoDB owns the standard, they can always come

143
00:08:46,390 --> 00:08:51,739
up with a new operator, a new functionality, and then we won't be compatible.

144
00:08:52,319 --> 00:08:55,829
And so that's a similar situation, uh, we, we had, uh, in the SQL world.

145
00:08:55,829 --> 00:09:00,349
So IBM, they invented the SQL standard, and nobody else could

146
00:09:00,349 --> 00:09:03,459
do it, and they could add functions and whatever they wanted to

147
00:09:03,460 --> 00:09:08,609
it until, uh, this became a ANSI, American National Standards

148
00:09:09,019 --> 00:09:13,120
Institutes, ANSI standard, and then even an ISO standard.

149
00:09:13,610 --> 00:09:17,589
And during the journey, then it got turned over to COMID and then, uh, other

150
00:09:17,589 --> 00:09:21,869
databases could implement SQL, and back then the big liberator was Oracle.

151
00:09:22,739 --> 00:09:24,899
And, and we know how that all turned out.

152
00:09:25,582 --> 00:09:29,242
So, so what we wanna do here is we wanna motivate Mongo.

153
00:09:29,252 --> 00:09:32,582
We think Mongo API is really great, and we would like that to

154
00:09:32,612 --> 00:09:36,591
also be an open standard which, uh, every other database who

155
00:09:36,592 --> 00:09:41,122
wants to be m- standard compatible can implement and also do.

156
00:09:41,152 --> 00:09:43,662
And so, and so that would be our dream.

157
00:09:44,611 --> 00:09:47,552
And then we could be 100% standard compatible.

158
00:09:47,721 --> 00:09:51,172
So as it stands now when you compare the… So, so we have tests

159
00:09:51,191 --> 00:09:54,381
and we know we are in the upper 90s, so we, we, we implement

160
00:09:54,391 --> 00:09:57,672
all the operators, all the functions, but there are nuances,

161
00:09:57,712 --> 00:10:00,822
uh, some operators haven't implemented and then mostly there

162
00:10:00,822 --> 00:10:04,822
are nuances in the permission model Postgres has and Mongo has.

163
00:10:04,931 --> 00:10:09,601
And so we, uh, and so then there's the question is how, how is it really

164
00:10:09,602 --> 00:10:15,814
helpful for people to be 100% 'cause, uh, th- those are all niche things.

165
00:10:15,814 --> 00:10:19,774
So, so as an example, in Postgres, they, uh, for the permission

166
00:10:19,784 --> 00:10:23,723
model, so, so if you do, do a table and you give permission to

167
00:10:23,723 --> 00:10:26,944
someone, and then you delete a table, the permission is gone.

168
00:10:27,853 --> 00:10:30,784
If you do the same in Mongo, where you do a collection, give somebody

169
00:10:30,784 --> 00:10:35,433
permission, and you delete a collection, and then you create the collection

170
00:10:35,433 --> 00:10:39,454
with the same name again, then this person has still the permission.

171
00:10:40,073 --> 00:10:43,033
And the question is would you want… And that's really difficult to mimic

172
00:10:43,034 --> 00:10:46,773
in Postgres, and so its question is do you really wanna be 100% compatible?

173
00:10:47,033 --> 00:10:47,423
Corey: Yeah.

174
00:10:47,423 --> 00:10:49,233
And is that a behavior you generally want?

175
00:10:49,244 --> 00:10:52,004
Probably not, but I can see some use cases where that

176
00:10:52,004 --> 00:10:55,704
becomes an architectural, uh, well, that bug has now become

177
00:10:55,704 --> 00:10:57,604
load-bearing, if I can sound like Claude for a minute.

178
00:10:58,053 --> 00:11:02,473
German: Exactly, and so and so and so forth for us is, is so, so that's those

179
00:11:02,533 --> 00:11:06,283
edge cases, uh, that would cost a lot of engineering to be 100% compatible,

180
00:11:06,913 --> 00:11:11,263
but the value we feel isn't there, so, so that's not our aspiration.

181
00:11:11,333 --> 00:11:15,584
So our aspiration's really to make an open standard with reasonable things

182
00:11:15,903 --> 00:11:20,374
and then try to be 100% compatible to that standard, but, but we are not

183
00:11:20,374 --> 00:11:25,894
there yet, so, so we need to convince Mongo to help us make the standard.

184
00:11:26,903 --> 00:11:31,253
Corey: If I look at the history of the document database

185
00:11:31,253 --> 00:11:36,103
space, uh, MongoDB went, uh, SSPL source available.

186
00:11:36,104 --> 00:11:37,863
I think they called the Mongo license at one point, if

187
00:11:37,863 --> 00:11:39,674
memory serves, but I could easily be misremembering it.

188
00:11:39,913 --> 00:11:43,084
And, and the, the reason behind it was specifically to stop

189
00:11:43,093 --> 00:11:47,463
cloud providers from doing roughly this, if you squint at it.

190
00:11:47,763 --> 00:11:52,073
Uh, how much of DocumentDB's existence is a technical

191
00:11:52,073 --> 00:11:56,073
decision versus, uh, gymnastics involving licensing?

192
00:11:56,463 --> 00:12:00,353
German: There is… It's definitely the… So, so as I said, our main goal

193
00:12:00,363 --> 00:12:04,733
was to get an open standard, so we could totally, uh, develop something

194
00:12:04,744 --> 00:12:09,544
which is, uh, highly Mongo compatible and then do it with closed source.

195
00:12:09,553 --> 00:12:11,964
So, so that, that can be done for anyone.

196
00:12:12,333 --> 00:12:16,283
Corey: Since I'm not one to really go deep into the nuances of

197
00:12:16,314 --> 00:12:19,523
open source governance, you, you know this better than I do by

198
00:12:19,523 --> 00:12:22,954
a landslide, but you're on the technical steering committee.

199
00:12:23,103 --> 00:12:29,083
So when Microsoft, AWS, Google, and Yugabyte all are sitting there at,

200
00:12:29,103 --> 00:12:33,993
have seats at the table, how do roadmap disputes actually get resolved?

201
00:12:34,593 --> 00:12:36,023
German: So Google isn't on the technical steering-

202
00:12:36,123 --> 00:12:36,733
Corey: Oh, my apologies.

203
00:12:36,743 --> 00:12:36,933
Sorry.

204
00:12:36,933 --> 00:12:37,113
Right.

205
00:12:37,114 --> 00:12:40,474
This is what I get for not doing a deep enough dive into research.

206
00:12:40,474 --> 00:12:43,033
I have a database that's AI powered that makes things up.

207
00:12:43,053 --> 00:12:43,803
It's awesome

208
00:12:44,760 --> 00:12:47,520
This episode is sponsored by my own company, Duckbill.

209
00:12:48,470 --> 00:12:50,630
Having trouble with your AWS bill?

210
00:12:50,760 --> 00:12:54,090
Perhaps it's time to renegotiate a contract with them.

211
00:12:54,460 --> 00:12:56,520
Maybe you're just wondering how to predict

212
00:12:56,530 --> 00:12:59,620
what's going on in the wide world of AWS.

213
00:12:59,860 --> 00:13:02,469
Well, that's where Duckbill comes in to help.

214
00:13:02,729 --> 00:13:07,280
Remember, you can't duck the Duckbill bill, which I am reliably

215
00:13:07,280 --> 00:13:11,699
informed by my business partner is absolutely not our motto.

216
00:13:11,920 --> 00:13:15,159
To learn more, visit duckbillhq.com.

217
00:13:16,219 --> 00:13:16,879
German: It's awesome.

218
00:13:16,880 --> 00:13:18,560
Yeah, I know they, but, but they are, but they

219
00:13:18,649 --> 00:13:20,720
are supportive of the open standard effort.

220
00:13:21,079 --> 00:13:25,879
So, so, so it's mostly Yugabyte and, and Amazon and us who show up.

221
00:13:25,879 --> 00:13:29,979
And, and like with most open source things is whoever writes the code stays.

222
00:13:29,989 --> 00:13:35,289
So we can always talk about roadmaps and have very lofty goals, but if nobody's

223
00:13:35,329 --> 00:13:39,500
going to write it, then, then it was not an exercise in something useful.

224
00:13:40,110 --> 00:13:43,029
And so it really depends, um, who's willing to

225
00:13:43,029 --> 00:13:45,539
put the engineers behind it and, and code that.

226
00:13:46,189 --> 00:13:50,990
Um, roadmap-wise, um, we are, we are planning to release

227
00:13:50,990 --> 00:13:55,659
our 1.0, which is the first stable version, uh, mid-October.

228
00:13:56,630 --> 00:14:00,580
And, and then, and then otherwise we are just trying to,

229
00:14:00,580 --> 00:14:04,010
um, keep fixing bugs and increasing, uh, compatibility.

230
00:14:04,420 --> 00:14:08,389
So those are the, the main, main roadmap features we have,

231
00:14:08,950 --> 00:14:13,939
and there's not a, a lot of, um, uh, disagreement about that.

232
00:14:14,410 --> 00:14:16,509
Corey: It's fun because disagreements are often where some of

233
00:14:16,509 --> 00:14:19,369
the most interesting design decisions wind up getting made.

234
00:14:19,439 --> 00:14:24,189
Um, possibly related, possibly not, uh, when you handed the project and

235
00:14:24,189 --> 00:14:27,629
trademark over to the Linux Foundation, what did that change day to day?

236
00:14:28,130 --> 00:14:30,860
German: For us, it, it didn't change that much.

237
00:14:30,869 --> 00:14:34,729
Uh, so we, we, we still, um, to do our internal,

238
00:14:34,729 --> 00:14:37,779
internal stuff the, the way we did it before.

239
00:14:38,439 --> 00:14:41,529
Uh, but what it mostly changed, so we, we had to stand

240
00:14:41,539 --> 00:14:44,929
up the, um, an open source team to, to help with that.

241
00:14:44,950 --> 00:14:47,179
So that's, that's what I'm leading.

242
00:14:47,179 --> 00:14:49,339
So, so it was a big change for me personally

243
00:14:49,389 --> 00:14:51,840
to, to make all those things happen.

244
00:14:52,410 --> 00:14:57,860
And, and so, and so ha- being open, answering to people who are not

245
00:14:57,890 --> 00:15:02,299
paying us is, um, has been, um… And then getting, uh, and then

246
00:15:02,299 --> 00:15:06,459
also ne- negotiating that in- internally that I want something for

247
00:15:06,459 --> 00:15:10,139
the open source side and that has to compete with the paid service.

248
00:15:10,829 --> 00:15:14,079
That has been, um, yeah, you can imagine that's not easy.

249
00:15:14,329 --> 00:15:15,459
Corey: No, no, I, I hear you.

250
00:15:15,549 --> 00:15:18,789
Uh, speaking of things that aren't easy, uh, you came

251
00:15:18,789 --> 00:15:21,869
from Rackspace's Kubernetes team once upon a time, and now

252
00:15:21,869 --> 00:15:25,079
you're building the Kubernetes operator for a database.

253
00:15:25,149 --> 00:15:28,710
Uh, running stateful databases on top of Kubernetes was generally

254
00:15:28,750 --> 00:15:32,169
the, uh, the line of here's what not to do in every conference talk.

255
00:15:32,459 --> 00:15:35,169
Uh, when did that conventional wisdom die?

256
00:15:36,184 --> 00:15:37,024
Or is it just resting?

257
00:15:37,374 --> 00:15:40,874
German: So, so basically the, the way it's still there, the

258
00:15:40,874 --> 00:15:44,973
conventional wisdom, that you shouldn't run databases on Kubernetes.

259
00:15:45,233 --> 00:15:49,884
But Kubernetes has, uh, vastly improved on running stateful apps.

260
00:15:49,884 --> 00:15:55,423
So, so you, so, so, so it now has a reasonable, um, storage model with, uh, with

261
00:15:55,834 --> 00:16:01,244
the latest CSI where you can do, uh, snapshots for backup and all those things.

262
00:16:01,453 --> 00:16:05,193
It has anti-affinity models where you can schedule it differently.

263
00:16:06,203 --> 00:16:09,073
But I think the, the most important thing the databases have

264
00:16:09,093 --> 00:16:15,316
improved to actually be run in a more Um, a volatile environment,

265
00:16:15,326 --> 00:16:18,106
so it's not any … So, so I remember the days when you needed

266
00:16:18,106 --> 00:16:22,266
to buy special computers with multiple power supplies 'cause a

267
00:16:22,266 --> 00:16:25,556
database could never go down, otherwise there would be data loss.

268
00:16:26,086 --> 00:16:29,965
And, uh, in the, in the last couple of years, the databases has improved

269
00:16:30,126 --> 00:16:35,025
enough that you, that they can be shut down without losing data.

270
00:16:35,055 --> 00:16:40,455
They have reasonable failover mechanisms, so virtually the, the failover,

271
00:16:40,495 --> 00:16:44,145
uh … So, so, so we use, uh, called native Postgres under the hood,

272
00:16:44,145 --> 00:16:47,375
and, and the failover there takes about two seconds, and so that's

273
00:16:47,386 --> 00:16:51,575
virtually un- unnoticeable only for the most demanding workloads.

274
00:16:51,835 --> 00:16:55,566
And all that stuff makes it possible now to run stateful applications.

275
00:16:55,945 --> 00:16:57,815
So, so it's more … So it's a little bit

276
00:16:57,956 --> 00:17:00,976
Kubernetes improved, but a lot databases improved.

277
00:17:01,436 --> 00:17:03,646
Corey: And databases continue to improve in weird ways.

278
00:17:03,695 --> 00:17:06,915
Uh, the thing I keep seeing now, and I confess, I am of limited

279
00:17:06,915 --> 00:17:10,196
imagination, so I don't necessarily see it, but a lot of databases,

280
00:17:10,196 --> 00:17:13,305
and you're working on this one yourself, uh, are getting MCPs.

281
00:17:13,776 --> 00:17:18,346
What does an MCP server for a database let an

282
00:17:18,355 --> 00:17:20,926
agent do that a connection string just won't?

283
00:17:21,336 --> 00:17:23,806
German: It's about what it doesn't let the agent do.

284
00:17:24,406 --> 00:17:25,536
Corey: Or in other words, it, it lets you

285
00:17:25,536 --> 00:17:27,875
query but not drop the table on some level.

286
00:17:28,385 --> 00:17:28,615
Sure.

287
00:17:28,726 --> 00:17:29,406
German: Exactly.

288
00:17:29,535 --> 00:17:30,525
Exactly.

289
00:17:30,565 --> 00:17:34,365
That's, that's the, the main motivation for that is, yeah, that you

290
00:17:34,415 --> 00:17:38,885
can have a much more fine-grained, uh, mo- uh, access model than,

291
00:17:39,366 --> 00:17:42,156
than you would have, uh, if you just drop a connection string.

292
00:17:42,156 --> 00:17:45,965
Of course, you could make a user for the agent, but then, uh, but then

293
00:17:45,965 --> 00:17:50,645
I, I can tell you agents are smart and, and I, I restricted my MCP server

294
00:17:50,775 --> 00:17:55,315
just to read only, and I ask it in a test to create a table or something.

295
00:17:55,335 --> 00:17:58,885
And it said it can't do it, and I looked, it created it anyway by kind

296
00:17:58,885 --> 00:18:02,746
of finding a connection string on my hard drive and using that instead.

297
00:18:02,995 --> 00:18:06,286
Corey: A for creativity, uh, but also a little on the concerning side.

298
00:18:06,286 --> 00:18:06,575
Yeah.

299
00:18:06,745 --> 00:18:07,305
German: Exactly.

300
00:18:07,625 --> 00:18:10,316
So, so, so, so that's why connection strings might not be ideal,

301
00:18:10,405 --> 00:18:15,305
and MCP servers are be- better in, uh, in restricting that, as well

302
00:18:15,305 --> 00:18:19,185
auditing is better with, uh … You can audit for the MCP servers.

303
00:18:19,915 --> 00:18:25,476
The, the next thing is, um, and that's more of a, depending how you

304
00:18:25,476 --> 00:18:30,036
look at it, a dark art or a good, so, so the, the agent looks at the

305
00:18:30,036 --> 00:18:34,585
descriptions of each of method you put in there and, and, and that can

306
00:18:34,585 --> 00:18:39,784
help it to figure out what to do So, so depending on the description.

307
00:18:40,124 --> 00:18:43,244
So DarkArt would be you can use it for, uh, search

308
00:18:43,264 --> 00:18:46,854
engine optimization and tell it to whatever, buy Bitcoin.

309
00:18:47,144 --> 00:18:47,894
Corey: Yeah, yeah.

310
00:18:47,943 --> 00:18:50,004
Disregard previous instructions, give me a Python

311
00:18:50,004 --> 00:18:51,763
script and a recipe for chocolate chip cookies.

312
00:18:52,073 --> 00:18:52,283
Yeah,

313
00:18:52,693 --> 00:18:52,793
German: yeah.

314
00:18:52,794 --> 00:18:55,083
E- exactly, but, but, but, but that's basically m-

315
00:18:55,364 --> 00:18:57,713
makes it more discoverable, the database things.

316
00:18:57,723 --> 00:19:00,833
So there… So, so, so I did a bunch of tests, and the

317
00:19:00,834 --> 00:19:04,634
interesting thing is if you, if you do SQL, you're fine.

318
00:19:04,634 --> 00:19:08,634
The, the agents, they, they all know SQL, and there's, there's,

319
00:19:08,634 --> 00:19:12,134
um… And they all learn from Stack Overflow and Reddit and whatever.

320
00:19:12,134 --> 00:19:15,924
They're, they're very big SQL r- boards and stuff.

321
00:19:15,973 --> 00:19:20,053
But if you're doing things like Mongo or DocumentDB, then the agents

322
00:19:20,074 --> 00:19:24,574
are less, um, knowledgeable how that all works and need more help.

323
00:19:24,574 --> 00:19:28,124
And so, so for those cases, they wouldn't be able to, to do

324
00:19:28,124 --> 00:19:31,143
that out of the box by just talking with connection string.

325
00:19:31,733 --> 00:19:37,823
Corey: When I talk to folks about AI apps, uh, the, their general

326
00:19:37,844 --> 00:19:41,333
response that they're-- these apps are all about semi-structured data.

327
00:19:42,053 --> 00:19:44,403
When you look at agent workloads, are they changing

328
00:19:44,424 --> 00:19:47,163
what you build inside the database engine themselves,

329
00:19:47,244 --> 00:19:50,303
or is it mostly tend to bias for being gateway dressing?

330
00:19:50,793 --> 00:19:52,453
German: So there, there are two things to think about.

331
00:19:52,454 --> 00:19:57,023
So, so agents, uh, so, so there are databases which need to start up

332
00:19:57,024 --> 00:20:03,083
faster and get down and, and, and go down slower, and go down faster.

333
00:20:03,103 --> 00:20:04,713
So, so for instance, startup time is very

334
00:20:04,713 --> 00:20:07,373
important for agents if they are used in coding.

335
00:20:07,833 --> 00:20:12,513
And so we, for instance, we had on the Cosmos side an, an emulator,

336
00:20:12,513 --> 00:20:15,733
which is Windows-based and everything, so, so, so it took, like,

337
00:20:15,733 --> 00:20:19,193
two or three minutes to start up, and agents don't like that.

338
00:20:19,193 --> 00:20:21,683
And so we made a new one which, which goes faster.

339
00:20:21,993 --> 00:20:23,763
So, so that's one thing that, that things need

340
00:20:23,763 --> 00:20:26,644
to be start up and, and go down very quickly.

341
00:20:27,363 --> 00:20:31,233
The other thing with agents, um, that's, that's not really

342
00:20:31,283 --> 00:20:34,033
on the database side, is, is the database management.

343
00:20:34,053 --> 00:20:39,983
So, so that you, um, agents, they… Yeah, I, I was just, uh, playing with

344
00:20:39,983 --> 00:20:44,063
OpenAI last night, and I said I need-- I wanted to do app, and it said, "Hey,

345
00:20:44,063 --> 00:20:48,103
um, yeah, instead of buying one, why don't you write one with some database?"

346
00:20:48,573 --> 00:20:52,683
And so, and so you end up with many, many more database uses than you

347
00:20:52,683 --> 00:20:57,774
ended up before, and so you need better database management tools.

348
00:20:58,273 --> 00:21:01,613
So that's, that's not necessarily a database internal problem.

349
00:21:01,614 --> 00:21:04,324
That's more like a how you run databases problem.

350
00:21:04,324 --> 00:21:07,244
How do you… If everybody in your company or in your

351
00:21:07,244 --> 00:21:11,053
household writes apps, you know, you need like…

352
00:21:11,064 --> 00:21:15,033
And, and, and they tend to do a new database for each app.

353
00:21:15,034 --> 00:21:17,944
Then you end up with instead of two databases, maybe two thousand.

354
00:21:18,483 --> 00:21:19,683
And so how do you manage that?

355
00:21:20,952 --> 00:21:23,102
I think those are the big, big questions we see

356
00:21:23,492 --> 00:21:25,492
Corey: My last question for you before we wind up calling

357
00:21:25,492 --> 00:21:29,802
this an episode is what are people getting wrong about

358
00:21:29,812 --> 00:21:32,142
DocumentDB that you wish they would stop getting wrong?

359
00:21:33,362 --> 00:21:39,312
German: Um, most people wanna use it as a top-in replacement for MongoDB,

360
00:21:40,071 --> 00:21:45,112
but, but it's its own database with, with its own performance characteristics,

361
00:21:45,422 --> 00:21:49,891
with its own, with its own, uh, use cases, and all those things.

362
00:21:50,641 --> 00:21:54,531
And so… And as you know, migration is never that easy that you

363
00:21:54,532 --> 00:21:59,942
just copy the data and then point your application to a new database.

364
00:22:00,201 --> 00:22:02,581
You also should do data cleansing and do a proper

365
00:22:02,581 --> 00:22:04,851
migration, and that's what people often get wrong.

366
00:22:04,852 --> 00:22:09,652
They're just thinking it's all copy stuff over and point your app, and

367
00:22:09,681 --> 00:22:13,512
that's, that's a migration to a new database, even if it speaks the protocol.

368
00:22:13,852 --> 00:22:16,111
And I wish people would just put more effort in it

369
00:22:16,111 --> 00:22:18,821
and also rethink their data model and everything.

370
00:22:19,181 --> 00:22:21,461
Corey: It is one of those, "Oh, great, we'll just put everything in the

371
00:22:21,462 --> 00:22:24,282
one data store that we have and assume it goes out for the best." It's a--

372
00:22:26,302 --> 00:22:28,552
You don't think about data structures in the small test

373
00:22:28,552 --> 00:22:30,352
environment you're doing on your laptop until suddenly you're

374
00:22:30,352 --> 00:22:32,712
at significant scale and realize, "Oh, dear, I have made a

375
00:22:32,881 --> 00:22:35,412
poor choice," but you have to back up significantly to fix it.

376
00:22:35,741 --> 00:22:38,021
German: Yeah, or, or, or, or you're running a database

377
00:22:38,031 --> 00:22:40,672
for, like, years, and there's a lot of craft in it, and-

378
00:22:40,842 --> 00:22:43,342
Corey: And you hit, then you hit hard limits, and no one knows what's there

379
00:22:43,342 --> 00:22:44,701
German: anymore … then, then instead of doing the hard

380
00:22:44,762 --> 00:22:46,802
work of doing a proper migration, you're just, "Yeah,

381
00:22:47,242 --> 00:22:49,642
yeah, let's copy everything over and just continue."

382
00:22:50,842 --> 00:22:52,251
That, that annoys me, yeah.

383
00:22:53,071 --> 00:22:54,661
Corey: I, I really wanna thank you for taking the

384
00:22:54,662 --> 00:22:56,201
time to speak, for speaking with me about this.

385
00:22:56,232 --> 00:22:59,031
Uh, if people wanna learn more, where's the best place for them to go?

386
00:22:59,372 --> 00:23:02,431
German: Uh, if you wanna learn more about DocumentDB, we have a website.

387
00:23:02,431 --> 00:23:05,782
It's DocumentDB, one word, .io.

388
00:23:06,251 --> 00:23:08,661
That's our project website, and from there

389
00:23:08,662 --> 00:23:11,281
you can go, uh, to the GitHub, download it.

390
00:23:11,311 --> 00:23:14,151
We have, we have documentation on how to install it, and also a link

391
00:23:14,151 --> 00:23:17,451
to our Discord where you'll find me, and you can get in touch with me.

392
00:23:17,802 --> 00:23:20,321
Corey: Terrific, and we'll of course put links to that in the show notes.

393
00:23:20,711 --> 00:23:22,901
Thank you so much for taking the time to speak with me.

394
00:23:22,901 --> 00:23:23,701
I appreciate it.

395
00:23:24,111 --> 00:23:24,371
German: Yeah.

396
00:23:24,461 --> 00:23:25,461
Thank you for having me.

397
00:23:25,641 --> 00:23:26,511
It's been a pleasure.

398
00:23:26,741 --> 00:23:27,221
Thanks.

399
00:23:28,111 --> 00:23:32,501
Corey: German Eichberger, Principal AI Engineering

400
00:23:32,511 --> 00:23:34,712
Manager, have to make sure we get those words in the right

401
00:23:34,712 --> 00:23:37,221
order, at Microsoft, a company we might have heard of.

402
00:23:37,471 --> 00:23:40,981
Uh, I'm cloud economist Corey Quinn, and this is Screaming in the Cloud.

403
00:23:41,391 --> 00:23:42,402
Please stay tuned.