1 00:00:00,060 --> 00:00:02,900 Michael: Hello and welcome to PostgresFM, a weekly show about 2 00:00:02,900 --> 00:00:03,920 all things PostgreSQL. 3 00:00:04,080 --> 00:00:06,060 I am Michael, founder of pgMustard. 4 00:00:06,060 --> 00:00:08,160 This is Nikolay, founder of Postgres.AI. 5 00:00:08,160 --> 00:00:09,440 Hey Nikolay, how are you doing? 6 00:00:10,080 --> 00:00:11,820 Nikolay: Hi Michael, I'm doing great. 7 00:00:11,840 --> 00:00:12,660 How are you? 8 00:00:12,840 --> 00:00:13,940 Michael: I'm good also. 9 00:00:14,240 --> 00:00:17,600 So this week, well I was in charge of choosing, but I've actually 10 00:00:17,600 --> 00:00:19,340 picked something that you suggested. 11 00:00:19,340 --> 00:00:21,500 I was looking through all the listener suggestions, all of the 12 00:00:21,500 --> 00:00:23,220 ideas we've had in the past. 13 00:00:23,480 --> 00:00:25,020 And this was one of yours, right? 14 00:00:25,440 --> 00:00:26,400 Nikolay: I have no idea. 15 00:00:26,400 --> 00:00:27,340 I already forgot. 16 00:00:27,500 --> 00:00:28,520 It was my idea? 17 00:00:28,520 --> 00:00:29,020 Okay. 18 00:00:29,140 --> 00:00:31,920 Michael: So you suggested a while back that we talk about super 19 00:00:31,920 --> 00:00:36,480 user and especially super user in the new normal, the new context 20 00:00:36,580 --> 00:00:42,240 of cloud providers or managed services and whether we, well, 21 00:00:42,380 --> 00:00:45,560 the fact that we normally don't have superuser access anymore 22 00:00:45,580 --> 00:00:48,340 in those cloud environments, whether we should, that kind of 23 00:00:48,340 --> 00:00:48,780 thing. 24 00:00:48,780 --> 00:00:51,960 So a little bit of a refresher on what a superuser is, what it 25 00:00:51,960 --> 00:00:56,200 can do and, and maybe why we don't have it or why we should have 26 00:00:56,200 --> 00:00:56,700 it. 27 00:00:57,180 --> 00:00:57,680 Nikolay: Right. 28 00:00:58,020 --> 00:00:58,520 Right. 29 00:00:58,840 --> 00:00:59,340 Yeah. 30 00:00:59,760 --> 00:01:00,580 Good questions. 31 00:01:01,700 --> 00:01:01,960 Good. 32 00:01:01,960 --> 00:01:02,320 Good. 33 00:01:02,320 --> 00:01:03,720 What is a superuser, right? 34 00:01:03,720 --> 00:01:08,500 First of all, it's just bypassing all privilege, privilege checks, 35 00:01:08,600 --> 00:01:08,880 right? 36 00:01:08,880 --> 00:01:09,900 This is the idea. 37 00:01:10,640 --> 00:01:12,380 Michael: Yeah, I looked it up in the documentation. 38 00:01:13,260 --> 00:01:17,140 Superuser bypasses all permission checks except the right to 39 00:01:17,140 --> 00:01:17,940 log in. 40 00:01:17,980 --> 00:01:20,860 It's a dangerous privilege, should not be used carelessly, it's 41 00:01:20,860 --> 00:01:23,800 best to do most of your work as a role that is not a superuser 42 00:01:24,180 --> 00:01:26,360 and then talks about how to create one. 43 00:01:26,780 --> 00:01:30,560 Nikolay: So you can create a superuser with no login flag, right? 44 00:01:30,780 --> 00:01:31,280 Or... 45 00:01:32,380 --> 00:01:33,360 Michael: I guess so. 46 00:01:33,640 --> 00:01:34,140 Yeah. 47 00:01:36,860 --> 00:01:37,360 Nikolay: Interesting. 48 00:01:37,700 --> 00:01:44,080 Actually, I have a white spot in my knowledge here, maybe. 49 00:01:44,480 --> 00:01:44,980 Honestly. 50 00:01:46,020 --> 00:01:51,040 But yeah, superuser is what people sometimes use, not thinking 51 00:01:51,040 --> 00:01:52,480 about permissions at all. 52 00:01:52,800 --> 00:01:57,160 Even launching their services and web applications using super 53 00:01:57,160 --> 00:01:58,300 user database. 54 00:01:58,860 --> 00:02:00,420 It's very, very bad practice. 55 00:02:01,160 --> 00:02:02,500 Did you do that ever? 56 00:02:03,220 --> 00:02:05,260 Michael: Yeah, it's the default, right? 57 00:02:05,380 --> 00:02:07,540 It's the default, well, you don't know better. 58 00:02:08,400 --> 00:02:12,440 Nikolay: Actually, yes, and it means defaults here again are 59 00:02:12,440 --> 00:02:17,940 not perfect because they don't encourage you to create a non-superuser 60 00:02:18,520 --> 00:02:21,340 database or user that you will be using. 61 00:02:21,900 --> 00:02:30,240 Also, I can remember some companies, small and big ones, doesn't 62 00:02:30,240 --> 00:02:36,000 matter, which give a single superuser or not. 63 00:02:36,380 --> 00:02:40,240 So it's different kinds of scenes, different scenes. 64 00:02:40,240 --> 00:02:46,360 One scene is, let's use one single superuser for all people who have 65 00:02:46,680 --> 00:02:53,040 DBA access, SRE access, like admin access, and share it. 66 00:02:53,680 --> 00:02:58,000 Or let's give everyone a super user, but named one. 67 00:02:58,040 --> 00:03:00,860 Maybe the first thing, like, it's different, like, so different 68 00:03:00,860 --> 00:03:03,840 scenes, like, use super user by default when you work with 69 00:03:03,840 --> 00:03:06,240 database, checking something. 70 00:03:07,120 --> 00:03:10,360 And a different scene is like, let's share an account, let's share 71 00:03:10,360 --> 00:03:11,400 database role. 72 00:03:12,720 --> 00:03:14,540 Both are not good things, right? 73 00:03:14,640 --> 00:03:19,900 But at least if you separate roles and create multiple superusers, 74 00:03:20,540 --> 00:03:22,320 it's already slightly better. 75 00:03:22,360 --> 00:03:23,220 Not slightly, actually. 76 00:03:23,220 --> 00:03:27,040 It's significantly better because you can see who is doing what, 77 00:03:27,040 --> 00:03:28,020 at least, right? 78 00:03:28,380 --> 00:03:29,440 Distinguish people. 79 00:03:30,040 --> 00:03:34,840 But in general, yeah, Postgres, how it's organized, it provokes 80 00:03:34,840 --> 00:03:38,520 you to use superuser for everything by default, and you need 81 00:03:38,520 --> 00:03:40,820 to make efforts to go out of it. 82 00:03:41,280 --> 00:03:45,000 Most people at least realize this and at least stop using superuser 83 00:03:45,060 --> 00:03:46,600 for application work. 84 00:03:47,220 --> 00:03:48,740 This is number one thing to do. 85 00:03:48,740 --> 00:03:51,960 Okay, you are going to use Superuser for yourself because you 86 00:03:51,960 --> 00:03:55,220 have all the rights, you are maybe the owner of everything, right? 87 00:03:55,520 --> 00:03:58,180 You own this database, so okay, you have Superuser. 88 00:03:58,180 --> 00:04:01,720 It's a separate question, should you always log in as Superuser, 89 00:04:02,580 --> 00:04:02,980 right? 90 00:04:02,980 --> 00:04:06,660 Maybe you should log in as a normal user, a regular one with limited 91 00:04:06,660 --> 00:04:10,740 permissions and only if needed, use superuser access. 92 00:04:11,400 --> 00:04:17,660 But at least all application code must not use superuser. 93 00:04:17,720 --> 00:04:19,300 We use superuser, right? 94 00:04:19,540 --> 00:04:20,420 This is obvious. 95 00:04:20,460 --> 00:04:21,300 Michael: Well, and why? 96 00:04:21,340 --> 00:04:22,900 Like, it's the danger, right? 97 00:04:22,900 --> 00:04:23,900 Nikolay: Well, security. 98 00:04:24,280 --> 00:04:26,180 My favorite topic, security, right? 99 00:04:26,180 --> 00:04:26,900 I'm joking. 100 00:04:28,840 --> 00:04:31,180 Michael: Well, security is a really good reason, but I think 101 00:04:31,180 --> 00:04:35,020 also, like, the danger of being able to, like, drop things, being 102 00:04:35,020 --> 00:04:39,460 able to destroy data, it's not just a security issue, right? 103 00:04:39,620 --> 00:04:43,220 Somebody could steal everything, but they could also just destroy 104 00:04:43,220 --> 00:04:44,840 everything and not steal anything. 105 00:04:44,980 --> 00:04:45,480 Nikolay: Right. 106 00:04:45,720 --> 00:04:47,860 Well, it's also a kind of type of security. 107 00:04:48,080 --> 00:04:54,680 Well, it might be reliability or something, but it's still insecure 108 00:04:55,360 --> 00:04:56,340 to give everyone... 109 00:04:58,080 --> 00:05:01,460 It's not about somebody outside of the company stealing data, but 110 00:05:01,460 --> 00:05:04,740 even inside the company somebody made a mistake. 111 00:05:04,760 --> 00:05:09,880 It's also about, it means that the work is not secure, right? 112 00:05:10,680 --> 00:05:17,060 But I wish auditors dug into this hole deeper. 113 00:05:17,540 --> 00:05:20,640 Like many companies already reported, I mean Postgres companies 114 00:05:20,640 --> 00:05:28,040 reported, they have SOC2, and some companies go to IPO, and they 115 00:05:28,660 --> 00:05:33,840 have a lot of auditing activities from external auditors, right? 116 00:05:34,200 --> 00:05:40,240 And I know companies who are very well-known, they have a bunch 117 00:05:40,240 --> 00:05:43,620 of questions and some of these questions sometimes are related to 118 00:05:43,620 --> 00:05:47,480 Postgres, but when you look at them, being Postgres experts, they 119 00:05:47,480 --> 00:05:49,020 look funny, usually. 120 00:05:49,780 --> 00:05:54,920 So yeah, I think it would be good to create some standard or 121 00:05:54,920 --> 00:05:55,420 something. 122 00:05:55,440 --> 00:05:57,180 Actually, there is some standard, right? 123 00:05:57,740 --> 00:06:03,480 Crunchy Data they shared a big PDF a few years ago, which is 124 00:06:03,480 --> 00:06:08,160 aimed to make Postgres setup more secure and it was for, I think, 125 00:06:08,160 --> 00:06:10,120 for army or something like that development. 126 00:06:10,120 --> 00:06:12,940 Michael: Yeah, the US military I think collaborated with them on 127 00:06:12,940 --> 00:06:13,380 it. 128 00:06:13,380 --> 00:06:17,260 Nikolay: Right, this is a good thing. And even better that they 129 00:06:17,260 --> 00:06:21,540 shared it and this became public so you can use it to grab some 130 00:06:21,540 --> 00:06:24,580 things, and obviously a lot of things are related to permissions 131 00:06:24,620 --> 00:06:27,940 and what kind of database user you use, right? 132 00:06:28,180 --> 00:06:33,620 But if your company right now already got SOC 2 or is doing this 133 00:06:33,620 --> 00:06:41,020 or IPO or something, I would not rely on external auditors to 134 00:06:41,180 --> 00:06:43,120 say, okay, we are good here. 135 00:06:43,620 --> 00:06:44,640 They suck there. 136 00:06:45,060 --> 00:06:47,660 Their questions don't cover this topic almost. 137 00:06:48,000 --> 00:06:51,420 Or at least, what I saw so far over the last 5 years. 138 00:06:51,600 --> 00:06:53,800 I didn't see good questions, honestly. 139 00:06:54,240 --> 00:06:58,340 Like, some of them were kind of good, but this kind of topic, 140 00:06:58,440 --> 00:06:59,960 like, do you use super user? 141 00:06:59,960 --> 00:07:03,480 Do you distinguish users, I mean, humans? 142 00:07:04,080 --> 00:07:05,020 And so on and so forth. 143 00:07:05,020 --> 00:07:08,940 Like, what is your model for these privileges and so on? 144 00:07:09,320 --> 00:07:10,120 Quite weak. 145 00:07:10,120 --> 00:07:11,920 I'm not an expert, as usual, I say. 146 00:07:11,920 --> 00:07:13,780 I'm not an expert in security at all. 147 00:07:13,780 --> 00:07:18,140 This is not, like, this is maybe one of the least favorite topics 148 00:07:18,420 --> 00:07:19,140 in databases. 149 00:07:19,940 --> 00:07:21,780 But it's a super important topic, right? 150 00:07:21,780 --> 00:07:22,920 So, yeah. 151 00:07:23,660 --> 00:07:26,360 Okay, so when should we use superuser? 152 00:07:26,920 --> 00:07:29,120 Michael: That was where I was thinking we were gonna go. 153 00:07:29,120 --> 00:07:33,780 So You mentioned a while back, I think maybe on Twitter, maybe 154 00:07:33,780 --> 00:07:37,960 just to me, I can't remember, that there's a bunch of times when 155 00:07:37,960 --> 00:07:41,000 you're using it, when you're maybe with a client that's using 156 00:07:41,000 --> 00:07:43,820 a managed service provider, that it's frustrating to you that 157 00:07:43,820 --> 00:07:45,700 you don't have superuser access. 158 00:07:45,900 --> 00:07:48,760 So I was interested, like, when are those times, like, What are 159 00:07:48,760 --> 00:07:53,240 you trying to do that you can't do without it or that's difficult 160 00:07:53,240 --> 00:07:54,440 to do without it? 161 00:07:55,200 --> 00:07:57,980 Nikolay: Yeah, there are things that only a superuser can do, 162 00:07:57,980 --> 00:07:58,480 obviously. 163 00:07:59,640 --> 00:08:02,080 For example, COPY FROM PROGRAM is a dangerous thing to do because 164 00:08:02,080 --> 00:08:09,440 you can basically execute any shellcode under the Postgres OS user, 165 00:08:09,480 --> 00:08:10,660 Linux user, right? 166 00:08:12,160 --> 00:08:15,840 And some things like I don't remember exactly But there are certain 167 00:08:15,840 --> 00:08:18,780 types of things where you need superuser, definitely. 168 00:08:20,020 --> 00:08:21,420 Michael: So I found a list. 169 00:08:21,420 --> 00:08:23,680 I was looking at all the different cloud providers and whether 170 00:08:23,680 --> 00:08:25,460 they do or don't provide it. 171 00:08:25,460 --> 00:08:29,220 And there's a really good list in the Supabase docs of what 172 00:08:29,220 --> 00:08:32,540 is unsupported in their highest privileged role. 173 00:08:32,540 --> 00:08:35,060 So they don't, let me just read it quickly. 174 00:08:35,060 --> 00:08:38,220 Supabase provides a default post control to all instances deployed. 175 00:08:38,360 --> 00:08:41,820 Superuser access is not given as it allows destructive operations 176 00:08:42,380 --> 00:08:43,820 to be performed on the database. 177 00:08:44,020 --> 00:08:47,240 And so those unsupported operations are. 178 00:08:48,000 --> 00:08:49,400 Nikolay: Destructive, Okay. 179 00:08:49,740 --> 00:08:52,090 Michael: Yeah, it was an interesting choice of words. 180 00:08:53,380 --> 00:08:58,020 CREATE ROLE WITH REPLICATION, CREATE SUBSCRIPTION, CREATE EVENT 181 00:08:58,020 --> 00:09:01,760 TRIGGER, COPY FROM PROGRAM, as you mentioned, and of course, 182 00:09:01,760 --> 00:09:06,260 ALTER USER WITH SUPERUSER, so you can't make other users superusers. 183 00:09:06,740 --> 00:09:09,880 Nikolay: Well, these are destructive actions. 184 00:09:10,560 --> 00:09:13,040 If they are destructive, let's remove them from PostgreSQL. 185 00:09:17,380 --> 00:09:20,140 This is judgment, like, let me judge, right? 186 00:09:20,140 --> 00:09:23,030 I'm the owner of this database, or who is the owner of the database? 187 00:09:25,240 --> 00:09:28,010 Let me drop my position, it's very simple. 188 00:09:29,140 --> 00:09:29,880 Two things. 189 00:09:29,920 --> 00:09:32,070 First thing, for managed providers, right? 190 00:09:33,000 --> 00:09:35,360 Managed PostgreSQL providers. 191 00:09:35,740 --> 00:09:40,580 If you don't run in a container, question why not? 192 00:09:41,000 --> 00:09:41,420 Right? 193 00:09:41,420 --> 00:09:42,380 Run in a container. 194 00:09:42,720 --> 00:09:45,720 Or actually, or in a separate VM. 195 00:09:45,720 --> 00:09:48,840 You probably run it in a separate VM or in a container at least, 196 00:09:48,840 --> 00:09:49,020 right? 197 00:09:49,020 --> 00:09:50,080 So, it's... 198 00:09:50,080 --> 00:09:51,640 Michael: So, you mean like is it isolated? 199 00:09:51,820 --> 00:09:55,920 So, if somebody breaks out and, or yeah, if somebody else is 200 00:09:55,920 --> 00:09:59,220 being destructive, it doesn't affect you, if, you know, a different 201 00:09:59,220 --> 00:09:59,720 customer. 202 00:09:59,900 --> 00:10:00,700 That, yeah. 203 00:10:01,620 --> 00:10:04,400 Nikolay: A Firecracker microVM or something, a lot of things, 204 00:10:04,600 --> 00:10:05,720 but at least container. 205 00:10:06,400 --> 00:10:07,700 So it's already isolated. 206 00:10:08,240 --> 00:10:12,800 If it's isolated, second point, give me superuser because I'm 207 00:10:12,800 --> 00:10:14,760 the owner of this database, that's it. 208 00:10:15,620 --> 00:10:22,160 And they usually say, most of them say, it's for your safety, 209 00:10:22,280 --> 00:10:22,780 right? 210 00:10:22,920 --> 00:10:24,440 But it's bullshit and a lie. 211 00:10:24,440 --> 00:10:25,420 It's a lie. 212 00:10:25,640 --> 00:10:31,180 Because I know for sure that inside a big provider, very big provider, 213 00:10:31,560 --> 00:10:35,140 teams, this topic pops up from time to time. 214 00:10:36,040 --> 00:10:38,340 And technical people usually say, let's do it. 215 00:10:38,480 --> 00:10:42,220 There are no big reasons to say, like, we protect these users. 216 00:10:42,840 --> 00:10:49,360 Like, when AWS gives you an EC2 instance, it's a virtual machine 217 00:10:49,360 --> 00:10:52,240 with Linux, They give you root, right? 218 00:10:53,100 --> 00:10:54,220 Michael: Oh, I don't know. 219 00:10:54,440 --> 00:10:55,640 Nikolay: They give you root, of course. 220 00:10:55,640 --> 00:10:56,900 Well, I have root. 221 00:10:57,840 --> 00:11:01,860 Then people say, okay, but for Postgres, we have a lot of automation 222 00:11:02,320 --> 00:11:03,780 and you can break it. 223 00:11:03,960 --> 00:11:06,420 Well, if I break it, I break it. 224 00:11:06,420 --> 00:11:10,460 So if I execute copy from program as I did with Crunchy Bridge 225 00:11:10,760 --> 00:11:15,600 and I move pg_wal directory, well this is destructive action. 226 00:11:15,920 --> 00:11:22,540 I copy from program to table and I just MV PgWal2 PgWal2 and 227 00:11:22,540 --> 00:11:25,460 I've got a panic, I mean database got panic. 228 00:11:27,340 --> 00:11:32,480 I've got a big joy observing that I can destroy myself, I mean 229 00:11:32,480 --> 00:11:36,360 my database because this is how I think, okay, I own it at least 230 00:11:36,360 --> 00:11:40,820 like, okay Not directly, but I own it You know, there is a philosophical 231 00:11:41,980 --> 00:11:46,960 There is a philosophical very good statement You can truly own 232 00:11:46,960 --> 00:11:48,500 only what you can destroy. 233 00:11:49,160 --> 00:11:49,660 Ooh. 234 00:11:50,380 --> 00:11:50,880 Right? 235 00:11:51,580 --> 00:11:51,880 Right? 236 00:11:51,880 --> 00:11:53,800 Michael: I've not heard it, but it makes sense. 237 00:11:54,280 --> 00:11:56,780 Nikolay: It makes sense in anything, right? 238 00:11:57,260 --> 00:11:59,860 For example, if you cannot destroy your own company, if you're 239 00:11:59,860 --> 00:12:03,340 a startup guy, founder, you do not own it at all. 240 00:12:03,340 --> 00:12:05,460 Maybe investors own it, right? 241 00:12:05,740 --> 00:12:07,240 And you should realize it. 242 00:12:07,740 --> 00:12:08,680 Who can destroy it? 243 00:12:08,680 --> 00:12:12,720 This is like ownership without the ability to destroy it. 244 00:12:12,720 --> 00:12:15,660 So here we come to destructive actions, but let me judge it. 245 00:12:15,660 --> 00:12:17,980 Let me judge it and let me feel it, right? 246 00:12:18,540 --> 00:12:24,220 So, I don't accept any reasons like that, we protect you, you 247 00:12:24,220 --> 00:12:29,780 can destroy it, our support will be fed up with questions, I 248 00:12:29,780 --> 00:12:30,640 destroyed something. 249 00:12:30,940 --> 00:12:34,240 Well, if you destroy something, you recover from backups, that's 250 00:12:34,240 --> 00:12:34,540 it. 251 00:12:34,540 --> 00:12:40,440 But they also say, okay, if we have a lot of automation, and 252 00:12:40,440 --> 00:12:43,920 if we allow you to copy from programs, superuser, for example, 253 00:12:44,320 --> 00:12:49,940 then you will be able as a user to see our automation pieces, 254 00:12:49,940 --> 00:12:50,440 right? 255 00:12:50,740 --> 00:12:52,100 Reverse engineer it. 256 00:12:53,000 --> 00:12:54,640 This is already a real reason. 257 00:12:55,360 --> 00:12:56,720 This is the real reason. 258 00:12:56,820 --> 00:13:00,260 They don't tell you as a first reason, but this is the number 259 00:13:00,260 --> 00:13:01,780 1 reason, honestly. 260 00:13:02,360 --> 00:13:06,540 They don't want you to see the automation from inside. 261 00:13:09,520 --> 00:13:10,940 Crunchy Bridge is great here. 262 00:13:10,940 --> 00:13:12,440 They don't care. 263 00:13:12,980 --> 00:13:15,140 I'm not sure if their product is open source. 264 00:13:15,140 --> 00:13:18,340 There are doubts on it, because they stopped publishing images, 265 00:13:18,340 --> 00:13:19,460 as I know, and so on. 266 00:13:19,460 --> 00:13:21,560 But here, they give you superuser. 267 00:13:21,680 --> 00:13:25,220 You can go copy from program and explore, you know, like directory 268 00:13:25,840 --> 00:13:30,240 layout and so on everything, find which programs are there, probably 269 00:13:30,240 --> 00:13:31,360 try to execute them. 270 00:13:31,360 --> 00:13:34,460 Like it's your world, you really own it. 271 00:13:34,460 --> 00:13:37,660 They give you this, they charge you extra. 272 00:13:38,940 --> 00:13:42,740 You charge me extra and you protect me from myself? 273 00:13:44,060 --> 00:13:44,940 I don't know. 274 00:13:47,680 --> 00:13:51,020 There will be time when people start realizing it, and I hope 275 00:13:51,020 --> 00:13:52,760 auditors will also realize it. 276 00:13:52,760 --> 00:13:54,160 Who owns this database? 277 00:13:54,840 --> 00:13:55,340 Michael: Yeah. 278 00:13:55,680 --> 00:13:59,400 Of all the ones I checked, Crunchybridge were the only ones that 279 00:13:59,440 --> 00:14:01,360 supplied full superuser. 280 00:14:01,780 --> 00:14:04,740 A lot of the other ones create kind of a pseudo role just below 281 00:14:04,740 --> 00:14:06,500 that with like super base. 282 00:14:06,500 --> 00:14:08,900 I think couldn't find a list of things that were removed from 283 00:14:08,900 --> 00:14:09,400 others. 284 00:14:09,840 --> 00:14:12,380 But Crunchy Bridge does definitely deserve credit for that. 285 00:14:12,380 --> 00:14:15,280 And especially if you consider they were the ones that are publishing 286 00:14:15,280 --> 00:14:16,560 the security guide. 287 00:14:17,180 --> 00:14:20,800 I feel like that's a really good argument for it's possible it's 288 00:14:20,800 --> 00:14:24,140 just maybe people don't want to do it I would say they've got 289 00:14:24,140 --> 00:14:27,940 another possible advantage or at least it may be like some other 290 00:14:27,940 --> 00:14:32,960 reasons I I am not as convinced as you as the support reasons 291 00:14:32,960 --> 00:14:37,120 not a good 1 because even to investigate was it the user that 292 00:14:37,120 --> 00:14:41,000 messed up or was it us that messed up is actually quite difficult 293 00:14:41,000 --> 00:14:41,980 sometimes in support. 294 00:14:41,980 --> 00:14:43,620 I don't know if you've ever had that. 295 00:14:43,620 --> 00:14:46,900 Nikolay: Do you use RDS or Cloud SQL or something? 296 00:14:47,160 --> 00:14:49,780 Do you know how calls with support work? 297 00:14:49,840 --> 00:14:53,900 Do you see how the calls are usually organized? 298 00:14:54,900 --> 00:14:56,660 Michael: I haven't ever used their support. 299 00:14:57,540 --> 00:14:59,780 Nikolay: I mean, any of managed services support. 300 00:15:00,180 --> 00:15:05,380 If you just try something, they won't walk you step by step what 301 00:15:05,380 --> 00:15:07,760 Happened. 302 00:15:07,760 --> 00:15:08,840 They will offer you to recover from backup, blah, blah, blah, 303 00:15:08,840 --> 00:15:09,520 like that. 304 00:15:11,160 --> 00:15:12,840 Of course, there are logs, right? 305 00:15:14,020 --> 00:15:14,440 Michael: So... 306 00:15:14,440 --> 00:15:18,380 And possibly ironically, I have actually seen people have great 307 00:15:18,380 --> 00:15:20,860 experience with Crunchy Data support. 308 00:15:20,860 --> 00:15:24,720 This is not sponsored by Crunchy Data, I promise, but there are 309 00:15:24,720 --> 00:15:26,820 providers out there giving really good support. 310 00:15:27,440 --> 00:15:31,560 And I could imagine an argument for, especially if you're a provider 311 00:15:31,560 --> 00:15:33,920 that provides a free tier, for example. 312 00:15:34,120 --> 00:15:36,660 The ones that charge more than it would cost you to host your 313 00:15:36,660 --> 00:15:41,180 own, I can see the argument for maybe they should provide super 314 00:15:41,180 --> 00:15:42,020 user access. 315 00:15:42,240 --> 00:15:43,600 If it's a free service... 316 00:15:43,660 --> 00:15:44,440 Oh, go on. 317 00:15:45,120 --> 00:15:50,420 Nikolay: So yesterday I had 1 million rows in the table, but 318 00:15:50,420 --> 00:15:53,240 today it's only like minus 10 rows. 319 00:15:53,540 --> 00:15:55,460 Where are those 10 rows, right? 320 00:15:55,840 --> 00:15:57,040 Should I go to support? 321 00:15:57,040 --> 00:16:00,360 Because maybe it's a bug, maybe it's a bug of their automation, 322 00:16:00,360 --> 00:16:03,060 maybe it's a bug of Postgres itself, rows disappeared. 323 00:16:03,960 --> 00:16:06,380 Michael: If you've ever run a product, you're going to get some 324 00:16:06,380 --> 00:16:08,940 customers that come with support questions that it turns out 325 00:16:08,940 --> 00:16:10,640 it's nothing to do with your product. 326 00:16:11,480 --> 00:16:13,300 It happens to all of us, right? 327 00:16:13,860 --> 00:16:15,860 Nikolay: Or maybe I know I executed a delete. 328 00:16:16,720 --> 00:16:20,140 Or maybe I know my application could have executed a delete. 329 00:16:21,020 --> 00:16:22,400 This is some kind of problem. 330 00:16:23,860 --> 00:16:25,120 It's the same question. 331 00:16:25,120 --> 00:16:26,260 The same type of question. 332 00:16:26,260 --> 00:16:28,520 That's why I say then they say we're protective. 333 00:16:28,520 --> 00:16:29,240 It's bullshit. 334 00:16:31,220 --> 00:16:34,700 Michael: On that note, maybe to move on slightly, I did actually 335 00:16:34,700 --> 00:16:39,520 notice in the Crunchy docs that for the superuser role, they 336 00:16:39,520 --> 00:16:44,700 have pgAudit on by default to log what it's doing. 337 00:16:44,700 --> 00:16:46,180 Nikolay: Yeah. This is what I suspected. 338 00:16:46,640 --> 00:16:48,140 Michael: That's interesting, right? 339 00:16:48,680 --> 00:16:48,940 Nikolay: Yeah. 340 00:16:48,940 --> 00:16:49,440 Well, yes. 341 00:16:49,440 --> 00:16:54,300 If you enable it for all users, so your logs will be flooded 342 00:16:54,360 --> 00:16:58,740 with a lot of data, and like it will become a performance 343 00:16:58,740 --> 00:16:59,940 bottleneck very quickly. 344 00:17:00,090 --> 00:17:05,020 So it makes sense to enable it for superusers and capture 345 00:17:05,020 --> 00:17:06,420 everything that's happening. 346 00:17:08,000 --> 00:17:10,440 Michael: And of course, I suspect you could change that if you've 347 00:17:10,440 --> 00:17:11,680 got superuser access. 348 00:17:11,820 --> 00:17:13,000 Nikolay: You can change that. 349 00:17:13,440 --> 00:17:16,200 Michael: But I think the idea is to help people help themselves 350 00:17:16,200 --> 00:17:22,280 and also help you support them if it may be alarming if it looks 351 00:17:22,280 --> 00:17:24,520 like they are using it as their application user or 352 00:17:24,520 --> 00:17:25,580 Nikolay: something like that. 353 00:17:25,680 --> 00:17:27,600 People are not stupid in general. 354 00:17:27,740 --> 00:17:30,940 There are stupid people, but there is only a minority of them. 355 00:17:30,940 --> 00:17:34,540 And if you say we protect you, it means like you don't trust 356 00:17:34,600 --> 00:17:35,440 your own customers. 357 00:17:35,440 --> 00:17:36,060 It's bullshit. 358 00:17:36,060 --> 00:17:37,500 That's why I say it's bullshit. 359 00:17:37,960 --> 00:17:40,640 So this is a good thing to have good defaults. 360 00:17:40,640 --> 00:17:44,240 For example, OK, superusers and PGAudit is enabled. 361 00:17:44,500 --> 00:17:48,980 If someone disables it, this record that got disabled goes to 362 00:17:48,980 --> 00:17:52,860 logs, so we have footprints, right, of this action. 363 00:17:53,420 --> 00:17:55,340 But in general, people can... 364 00:17:56,400 --> 00:18:00,980 At least somehow customers I deal with, I see opposite. 365 00:18:01,080 --> 00:18:03,840 I sometimes think, oh, I need to explain to you this, but they're 366 00:18:03,840 --> 00:18:04,600 quite smart. 367 00:18:04,600 --> 00:18:07,000 They're like, okay, we already got this. 368 00:18:07,540 --> 00:18:10,300 So they can understand what's happening. 369 00:18:10,760 --> 00:18:14,340 My point is that they don't give this to you to protect them, 370 00:18:14,340 --> 00:18:15,040 not you. 371 00:18:15,700 --> 00:18:16,960 They want to be protected. 372 00:18:16,980 --> 00:18:19,780 They want to share what they got. 373 00:18:21,040 --> 00:18:24,260 And this is like a business decision, it's not a technical decision. 374 00:18:24,260 --> 00:18:29,140 So they don't want to share automation and how exactly they adjusted 375 00:18:29,140 --> 00:18:30,840 Postgres and so on and so forth. 376 00:18:30,840 --> 00:18:35,600 But I hope people will start realizing this and cases like Country 377 00:18:35,600 --> 00:18:40,360 Bridge will be more common and people who are truly open-source 378 00:18:40,380 --> 00:18:45,560 believers and lovers, they will shift to a more open approach and 379 00:18:45,720 --> 00:18:49,780 a more trustful approach, like trust your customers, they can decide 380 00:18:49,900 --> 00:18:52,540 if it's destructive or not, right? 381 00:18:53,480 --> 00:19:00,600 And just keep everything open, share your automation, and give 382 00:19:01,500 --> 00:19:03,580 ownership and access to your customers. 383 00:19:03,900 --> 00:19:10,360 In this case, the premium is usually quite significant over the infrastructure 384 00:19:10,640 --> 00:19:11,140 costs. 385 00:19:11,760 --> 00:19:14,320 It would be reasonable to pay, right? 386 00:19:14,800 --> 00:19:18,360 Now there is an imbalance in this world, as I see. 387 00:19:18,480 --> 00:19:21,580 And I hope with time we also realize it and start asking 388 00:19:21,580 --> 00:19:24,380 questions like who is the actual owner of this database? 389 00:19:26,400 --> 00:19:28,080 Michael: Yeah, interesting, good point. 390 00:19:28,120 --> 00:19:32,900 But what do you think about free, like, I completely take your 391 00:19:32,900 --> 00:19:36,020 point on providers that are charging a premium over what it would 392 00:19:36,020 --> 00:19:38,040 cost you to run the service. 393 00:19:38,440 --> 00:19:42,980 But what about the ones that are offering you a free tier, like 394 00:19:42,980 --> 00:19:49,660 Neon or Supabase, or even, there are some newer ones as well 395 00:19:50,140 --> 00:19:51,300 Nikolay: Well, how is it different 396 00:19:52,360 --> 00:19:56,420 Michael: Well, I actually don't know; maybe it's a premium 397 00:19:56,420 --> 00:20:00,800 feature, maybe like, if it ends up in more support 398 00:20:00,800 --> 00:20:03,960 personally, even if you don't agree that it 399 00:20:03,960 --> 00:20:06,360 should end in more support, I think it would overall 400 00:20:06,760 --> 00:20:10,560 Nikolay: I don't think it will be a big part of the whole picture. 401 00:20:10,560 --> 00:20:14,060 won't be a big part I mean this kind of questions like I destroyed 402 00:20:14,060 --> 00:20:17,360 my database and I don't of course if it got destroyed and you 403 00:20:17,360 --> 00:20:20,280 didn't do anything, this is a good question. 404 00:20:20,280 --> 00:20:25,940 But if you have a pgAudit setup, you can close and support 405 00:20:26,120 --> 00:20:30,180 easily can point to small how-to, how to understand what's happening, 406 00:20:30,180 --> 00:20:31,100 and that's it. 407 00:20:31,260 --> 00:20:36,360 And support usually doesn't look inside your database, right? 408 00:20:37,120 --> 00:20:38,340 This is your area. 409 00:20:38,420 --> 00:20:42,620 Like RDS support, for example, you need to have a high level 410 00:20:42,620 --> 00:20:43,840 of support and convincing. 411 00:20:44,760 --> 00:20:48,120 They usually check only the VM and the underlying things and 412 00:20:48,120 --> 00:20:49,700 the infrastructure things, right? 413 00:20:49,700 --> 00:20:51,220 What's happening inside your database? 414 00:20:51,220 --> 00:20:53,760 Who deleted rows or who moved, for example, a pg_wal? 415 00:20:54,020 --> 00:20:56,820 I still say this is the same level of things. 416 00:20:57,340 --> 00:20:58,480 It's your area. 417 00:21:00,300 --> 00:21:00,860 Michael: Yeah, true. 418 00:21:00,860 --> 00:21:02,940 I mean, there's, I think even... 419 00:21:03,280 --> 00:21:05,520 Nikolay: I know many people won't agree with me, actually. 420 00:21:06,220 --> 00:21:09,600 Michael: I actually don't know if they would anymore. 421 00:21:09,600 --> 00:21:13,680 I think there's an increasing education around data processing, 422 00:21:13,980 --> 00:21:18,620 especially with all that the privacy laws in the EU and in California, 423 00:21:20,860 --> 00:21:25,080 around whose data is it and who's processing the data, who owns 424 00:21:25,080 --> 00:21:25,740 the data. 425 00:21:26,060 --> 00:21:26,560 Nikolay: Right. 426 00:21:27,260 --> 00:21:33,060 Yeah, in true spirit of some of those laws, like if this is my 427 00:21:33,060 --> 00:21:36,980 data, I should own it and be able to destroy everything, not 428 00:21:36,980 --> 00:21:39,400 only at logical level, like delete. 429 00:21:39,520 --> 00:21:43,500 Maybe I want to destroy pg_wal right now myself, you know, I 430 00:21:43,500 --> 00:21:45,580 don't know, like this is true ownership. 431 00:21:45,920 --> 00:21:50,360 And see how it works inside, like I need to feel it, this is 432 00:21:50,360 --> 00:21:51,000 true ownership. 433 00:21:51,140 --> 00:21:52,940 In my opinion, inspect it. 434 00:21:53,520 --> 00:21:56,180 Michael: I can see the argument a little bit for things like 435 00:21:56,180 --> 00:21:59,400 replication or you know some of the let's say some of the cloud 436 00:21:59,400 --> 00:22:02,740 providers they said they have for example really easy single 437 00:22:02,780 --> 00:22:06,660 checkbox high availability or something and then you go and 438 00:22:06,660 --> 00:22:11,380 destroy replication or you mess something up that means that's 439 00:22:11,380 --> 00:22:12,480 not working anymore. 440 00:22:13,420 --> 00:22:16,080 Nikolay: Usually such providers don't use Postgres replication. 441 00:22:16,080 --> 00:22:19,060 This checkbox usually is not based on Postgres replication. 442 00:22:19,120 --> 00:22:26,320 It's usually underlying block storage device replication, synchronous 443 00:22:26,320 --> 00:22:26,820 usually. 444 00:22:27,800 --> 00:22:30,420 So, this is what I know about RDS and Cloud SQL. 445 00:22:30,420 --> 00:22:33,140 So, usually this is lower level. 446 00:22:34,180 --> 00:22:34,560 Michael: Cool. 447 00:22:34,560 --> 00:22:35,280 Makes sense. 448 00:22:36,660 --> 00:22:38,380 Maybe you can't even mess that up. 449 00:22:38,480 --> 00:22:38,980 Great. 450 00:22:39,160 --> 00:22:43,820 Nikolay: So anyway, I know we live in the world when people say 451 00:22:43,820 --> 00:22:44,320 like... 452 00:22:45,060 --> 00:22:50,980 Honestly, I also think most of developers look at what RDS did 453 00:22:50,980 --> 00:22:55,820 or CloudSQL did or Microsoft guys did, and they just copy this 454 00:22:56,200 --> 00:22:58,520 approach, not thinking deeper. 455 00:22:59,500 --> 00:23:03,500 Crunchy is, as you mentioned, they are security experts, obviously, 456 00:23:03,600 --> 00:23:05,640 and they are brave. 457 00:23:05,740 --> 00:23:07,960 So kudos, actually. 458 00:23:07,960 --> 00:23:10,080 I already told this a couple of times on Twitter. 459 00:23:10,080 --> 00:23:10,940 This is great. 460 00:23:11,720 --> 00:23:17,760 But others just copy what others, like, say, smaller or new providers, 461 00:23:17,780 --> 00:23:21,300 they just copy decisions from bigger providers and copy their 462 00:23:21,300 --> 00:23:24,300 arguments like this is to protect users, this is to protect customers, 463 00:23:24,440 --> 00:23:28,220 this is for your own safety, let me decide what is destructive. 464 00:23:29,640 --> 00:23:32,900 Michael: Yeah, well and I know quite a few managed service providers 465 00:23:32,900 --> 00:23:35,440 listen and people that work at them it would be great to hear 466 00:23:35,440 --> 00:23:39,960 from you if there's something we've missed or if there's a If 467 00:23:39,960 --> 00:23:42,720 there's a way of explaining this that would be would be better 468 00:23:42,720 --> 00:23:43,580 let us know. 469 00:23:44,060 --> 00:23:47,440 Nikolay: Yeah, actually I can criticize this easily because I 470 00:23:47,440 --> 00:23:50,420 don't develop a managed service, managed Postgres service, right? 471 00:23:50,420 --> 00:23:50,920 Yeah. 472 00:23:51,180 --> 00:23:54,860 Because in this case I would need to be more careful because 473 00:23:54,960 --> 00:23:56,220 I would have my situation. 474 00:23:56,280 --> 00:24:00,560 But honestly, several times I saw this, like how to implement, 475 00:24:01,080 --> 00:24:04,640 how to protect, what we should protect, if you provide supervision, 476 00:24:04,640 --> 00:24:06,340 what kind of dangers exist. 477 00:24:07,200 --> 00:24:10,960 And the list is obviously, this is copied from programs, foreign 478 00:24:10,960 --> 00:24:15,040 data wrappers probably, and so on, like, dangerous parts. 479 00:24:16,160 --> 00:24:23,560 So if you think about it, you can implement a good model, and 480 00:24:23,560 --> 00:24:29,180 I think, of course, maybe Postgres could provide some additional 481 00:24:29,180 --> 00:24:37,580 tools to restrict certain areas, but if it's my database, I should 482 00:24:37,580 --> 00:24:41,120 decide what to enable, what to disable, and at which point for 483 00:24:41,120 --> 00:24:41,880 whom, right? 484 00:24:42,260 --> 00:24:44,700 Michael: Yeah I don't know if Postgres can do anything about 485 00:24:44,700 --> 00:24:49,540 this because ultimately these are features that we need, like 486 00:24:49,540 --> 00:24:51,360 someone needs to be able to do them. 487 00:24:51,500 --> 00:24:53,740 Well, maybe if no one needs to be able to do them, it shouldn't 488 00:24:53,740 --> 00:24:55,220 be in Postgres at all. 489 00:24:55,320 --> 00:24:59,340 But if someone wants to be able to do them, it seems silly to 490 00:24:59,340 --> 00:25:02,120 me not like to disable it because we want to be able to host 491 00:25:02,120 --> 00:25:02,940 it in clouds? 492 00:25:04,160 --> 00:25:06,040 Nikolay: Yeah, well, I want to disable it sometimes. 493 00:25:06,040 --> 00:25:09,920 For example, in Database Lab, we have a job bot, right? 494 00:25:09,920 --> 00:25:13,320 And we don't want to job bot to like we want as much freedom 495 00:25:13,320 --> 00:25:16,980 as possible for end users to execute any SQL. 496 00:25:17,300 --> 00:25:21,020 But if any SQL is copied from a program, it can be dangerous 497 00:25:21,020 --> 00:25:27,440 because all foreign data are hyper because maybe users who are 498 00:25:27,440 --> 00:25:31,400 end users, they are still inside the same team, but maybe admins 499 00:25:31,400 --> 00:25:33,480 doesn't want them to execute it. 500 00:25:33,480 --> 00:25:36,600 And we just want to protect here. 501 00:25:36,880 --> 00:25:41,140 Some users have full access, admin decides, right? 502 00:25:41,580 --> 00:25:47,700 But some users who work only at this level, this SQL experimentation 503 00:25:48,060 --> 00:25:49,540 level, they are restricted. 504 00:25:49,540 --> 00:25:53,800 And at some point we removed all possibilities of copy of program 505 00:25:53,800 --> 00:25:55,460 from the developers and so on and so on. 506 00:25:55,460 --> 00:25:59,060 So they cannot do harm even if they are inside the same team. 507 00:25:59,060 --> 00:26:02,900 But the admin decides which permissions to provide, right? 508 00:26:03,520 --> 00:26:06,680 So in the end of the day, you own this database and you decide 509 00:26:06,680 --> 00:26:07,580 what to do. 510 00:26:08,000 --> 00:26:09,840 When I say you, I mean admins, right? 511 00:26:09,840 --> 00:26:14,280 Because inside, bigger customer there might be some additional 512 00:26:14,680 --> 00:26:15,640 users, right? 513 00:26:15,660 --> 00:26:21,260 And the good tool can provide the ability to control those permissions. 514 00:26:21,760 --> 00:26:23,240 Michael: Yeah, and that's what we're saying, right? 515 00:26:23,240 --> 00:26:26,040 Like, we're not saying RDS should provide superuser to every 516 00:26:26,040 --> 00:26:31,100 single person who has access to the RDS dashboard, but to like 517 00:26:31,100 --> 00:26:31,960 an admin. 518 00:26:31,980 --> 00:26:33,560 That's what we're asking for, right? 519 00:26:33,560 --> 00:26:36,760 1 person or a small group of users. 520 00:26:37,720 --> 00:26:38,760 Nikolay: Right, right. 521 00:26:39,960 --> 00:26:42,220 Well, of course there is a chain reaction. 522 00:26:42,340 --> 00:26:46,220 If, for example, RDS, for example, Okay, we provide superusers, 523 00:26:46,360 --> 00:26:52,200 but then what to do with access to WAL files, backups, physical 524 00:26:52,200 --> 00:26:55,900 backups, physical replication connection, and so on and so on. 525 00:26:55,900 --> 00:26:57,480 There is a chain reaction here, right? 526 00:26:57,700 --> 00:27:01,560 And they restrict you here, partially because it's kind of vendor 527 00:27:01,560 --> 00:27:02,480 locking, obviously. 528 00:27:05,220 --> 00:27:08,300 So they also restrict a number of things, like for example, recovery 529 00:27:08,300 --> 00:27:14,360 target LSN, so you cannot perform 0 downtime upgrades with our 530 00:27:14,360 --> 00:27:17,140 recipe, which involves recovery target LSN. 531 00:27:17,220 --> 00:27:23,640 You can only do with slot advancement, we call it Instacart approach, 532 00:27:23,720 --> 00:27:28,520 right, this article, Instacart approach, which the opinions is 533 00:27:28,520 --> 00:27:29,280 more risky. 534 00:27:31,980 --> 00:27:35,640 Right, so recovery target LSN, you need to do it, like you need 535 00:27:35,640 --> 00:27:36,200 to provide it. 536 00:27:36,200 --> 00:27:40,440 So, I mean, if you provide superuser, this is a Pandora's box 537 00:27:40,440 --> 00:27:42,880 in terms of decisions what to provide to your users. 538 00:27:44,160 --> 00:27:50,020 Michael: Well, it's a Pandora's box, but it's making all of those 539 00:27:50,020 --> 00:27:52,620 decisions at once and saying we let you do anything. 540 00:27:53,420 --> 00:27:54,500 Nikolay: Yeah, that's cool. 541 00:27:54,640 --> 00:27:56,920 You are the owner, you decide what to do. 542 00:27:56,920 --> 00:27:59,340 We provide you automation, we charge for it. 543 00:28:00,600 --> 00:28:01,440 That's it. 544 00:28:01,460 --> 00:28:05,240 Now we live in a different world, we provide you some automation, 545 00:28:05,380 --> 00:28:12,500 we hide a lot of capabilities Postgres has from you, we restrict 546 00:28:12,500 --> 00:28:15,280 you and charge for it. 547 00:28:15,940 --> 00:28:16,980 It's not fair. 548 00:28:17,660 --> 00:28:20,940 But this is the most popular approach right now. 549 00:28:22,580 --> 00:28:25,360 We had discussions about managed services, right? 550 00:28:25,440 --> 00:28:27,220 Michael: Yeah, second episode actually. 551 00:28:27,260 --> 00:28:29,000 Maybe it's a little bit of a loop. 552 00:28:29,760 --> 00:28:30,920 Nikolay: Right, right. 553 00:28:31,520 --> 00:28:35,800 I feel the loop indeed, but a different angle completely. 554 00:28:36,420 --> 00:28:40,320 We had the previous episode also echoing the first one. 555 00:28:40,320 --> 00:28:40,820 Interesting. 556 00:28:41,200 --> 00:28:44,340 So we know the next topic should echo the third one. 557 00:28:44,340 --> 00:28:45,160 Let's do it. 558 00:28:46,020 --> 00:28:46,240 Okay. 559 00:28:46,240 --> 00:28:47,000 Anyway, okay. 560 00:28:47,000 --> 00:28:47,740 That's it. 561 00:28:47,980 --> 00:28:49,060 Michael: Podcast wraparound. 562 00:28:49,860 --> 00:28:50,360 Nikolay: Yeah. 563 00:28:51,560 --> 00:28:52,060 Right. 564 00:28:52,280 --> 00:28:55,360 So maybe this discussion was not too technical, right? 565 00:28:56,300 --> 00:28:58,940 But you can go read the documentation if you want technical. 566 00:28:59,820 --> 00:29:02,460 Michael: Yeah, and there were good talks on roles and security 567 00:29:03,520 --> 00:29:04,840 and just covering the basics. 568 00:29:04,840 --> 00:29:06,680 There's loads of documentation on this kind of thing. 569 00:29:06,680 --> 00:29:07,980 We can include some links. 570 00:29:08,040 --> 00:29:08,760 Nikolay: Right, yeah. 571 00:29:08,760 --> 00:29:10,620 We had some philosophical discussion. 572 00:29:11,600 --> 00:29:13,000 Do you own your database? 573 00:29:13,900 --> 00:29:16,080 RDS users, do you own your databases? 574 00:29:17,040 --> 00:29:18,060 Who owns it? 575 00:29:18,600 --> 00:29:22,960 Maybe people don't care about it until some auditors decide that 576 00:29:22,960 --> 00:29:24,060 it's not truly ownership. 577 00:29:24,060 --> 00:29:27,900 But probably, how AWS is organized, probably they will never 578 00:29:28,260 --> 00:29:29,280 raise this question. 579 00:29:30,180 --> 00:29:31,260 Everyone is happy. 580 00:29:31,920 --> 00:29:33,000 So I don't know. 581 00:29:34,080 --> 00:29:35,960 Michael: And at least there's choice out there now. 582 00:29:36,580 --> 00:29:39,480 There's at least one provider that does, and you can self-host. 583 00:29:40,080 --> 00:29:43,680 You can even use the cloud and manage it yourself. 584 00:29:45,660 --> 00:29:49,080 Nikolay: I also say I want to own my bloat, right? 585 00:29:49,080 --> 00:29:54,380 Because AWS doesn't allow you to take your physical level. 586 00:29:54,380 --> 00:29:55,760 Michael: Logical replication. 587 00:29:55,760 --> 00:30:00,260 Nikolay: Yeah, logical, you lose the bloat your data files have. 588 00:30:00,560 --> 00:30:03,400 And you cannot stop thinking about bloat because it's Postgres. 589 00:30:03,540 --> 00:30:06,020 Bloat is the center of architecture. 590 00:30:07,200 --> 00:30:10,780 So you need to understand how this works if you want good performance. 591 00:30:11,540 --> 00:30:12,600 So, yeah, interesting. 592 00:30:13,220 --> 00:30:17,820 So you work at an abstraction only, some abstraction level. 593 00:30:18,660 --> 00:30:20,260 And you cannot copy files. 594 00:30:21,060 --> 00:30:22,780 Okay, maybe enough. 595 00:30:23,760 --> 00:30:24,720 Michael: Thanks so much, Nikolay. 596 00:30:24,720 --> 00:30:25,380 Thanks, everyone. 597 00:30:25,380 --> 00:30:26,360 And see you next week. 598 00:30:26,360 --> 00:30:27,040 Nikolay: Thank you.