1 00:00:00,049 --> 00:00:04,049 Michael: Hello, and welcome to Postgres FM a weekly show about all things PostgreSQL. 2 00:00:04,369 --> 00:00:08,959 I am Michael founder of pgMustard, and this is my cohost Nikolay founder of Postgres AI. 3 00:00:09,349 --> 00:00:10,997 Hey Nikolay what are we talking about today? 4 00:00:11,567 --> 00:00:12,587 Nikolay: Hi, Michael. 5 00:00:12,587 --> 00:00:18,243 I, I guess today we, again, are going to react to feedback or requests. 6 00:00:18,243 --> 00:00:25,916 We saw on Reddit good request asking us to discuss how to become DBA if you are currently in different area. 7 00:00:25,916 --> 00:00:28,692 And we want to shift to this field. 8 00:00:28,818 --> 00:00:30,018 So let's, let's discuss. 9 00:00:30,747 --> 00:00:31,797 Michael: Yeah, absolutely. 10 00:00:31,802 --> 00:00:32,967 It was really nicely worded. 11 00:00:32,967 --> 00:00:33,387 Wasn't it? 12 00:00:33,387 --> 00:00:34,562 It was around both. 13 00:00:34,862 --> 00:00:41,042 What are the expectations maybe for a junior DBA and also what would their responsibilities typically be? 14 00:00:41,462 --> 00:00:45,272 And I think this is gonna vary quite a lot, depending on the type of organization. 15 00:00:45,272 --> 00:00:47,078 And we can go into that for sure. 16 00:00:47,433 --> 00:00:51,483 But equally there doesn't seem to be a ton of good materials out there on this. 17 00:00:51,483 --> 00:00:53,538 So I'm looking forward to hearing your thoughts on it. 18 00:00:53,538 --> 00:00:53,925 So yeah. 19 00:00:53,925 --> 00:00:59,471 Should we start with, should we start with what maybe the, we use an acronyms or I guess, but yeah. 20 00:00:59,471 --> 00:01:00,551 What do we mean by DBA? 21 00:01:00,671 --> 00:01:06,561 Nikolay: database administrator, I guess over last decade this term changed its its meaning a lot. 22 00:01:06,731 --> 00:01:07,061 Right? 23 00:01:07,061 --> 00:01:09,759 And it's many, many factors here. 24 00:01:10,419 --> 00:01:14,139 Originally DBA was like sysadmin style person. 25 00:01:14,604 --> 00:01:23,787 Who should be invisible if work has done well, it should be like, nobody knows DBA is there because it's like people learn about, 26 00:01:23,877 --> 00:01:27,537 they exist only in the case of some incidents, troubles and so on. 27 00:01:27,761 --> 00:01:38,101 But in cloud error some of work almost disappeared because if you use managed version of SSUS as cloud SQL and GCP or RDS on AWS. 28 00:01:38,641 --> 00:01:42,968 Of course you don't need to be an expert in backups and replication at. 29 00:01:43,448 --> 00:01:49,189 Moreover yesterday remember last day on Twitter, we discussed once again, that dumps are not backups. 30 00:01:49,189 --> 00:01:56,269 And I, I have belief that RRGs plays a big role, other managed positive services. 31 00:01:56,599 --> 00:02:02,862 They play big role in teaching bad things because people don't realize what the cops are. 32 00:02:03,252 --> 00:02:14,191 I mean, Junior people, junior engineers and they also want at the same time to sometimes to have backups in hand, but you cannot take backup from RG. 33 00:02:14,251 --> 00:02:17,811 So you still use dump, but probably this is different topic, right? 34 00:02:18,291 --> 00:02:20,751 Dump sources, normal backups. 35 00:02:20,827 --> 00:02:25,012 But anyway, like some, some functions of DBA vanished. 36 00:02:25,019 --> 00:02:28,620 Because they were replaced by managed service. 37 00:02:28,832 --> 00:02:34,664 Others appear like knowledge about how to manage cloud resources and, and so on. 38 00:02:34,790 --> 00:02:43,093 Somehow reacting to this, some organizations started to introduce new terminology, like DBRE also reacting to SRE 39 00:02:43,185 --> 00:02:43,515 Michael: Yeah. 40 00:02:43,695 --> 00:02:50,175 So we are talking about site, site, reliability, engineers, database, reliability engineers, which is a term, the, the latter 41 00:02:50,175 --> 00:02:53,085 of those is a term I've only started hearing more recently. 42 00:02:53,385 --> 00:03:00,734 But I think that that factors into where these tasks are done in an organization and not necessarily what title they have. 43 00:03:00,734 --> 00:03:01,334 So in, in the. 44 00:03:02,609 --> 00:03:10,079 Well, actually, even nowadays in a small organization, there's a really good chance you don't have any person with DBA in their name. 45 00:03:10,079 --> 00:03:12,918 You might not even have an SRE by title. 46 00:03:12,923 --> 00:03:19,218 If you've only got a team of four, probably the founder handles it, or one of the engineers takes the lead on that. 47 00:03:19,218 --> 00:03:25,608 So I guess if we start with tasks, you've already mentioned backups, but I think there's a, there's like a few more that used to 48 00:03:25,608 --> 00:03:29,538 typically live within the DBA role and still exist, but maybe get. 49 00:03:30,553 --> 00:03:33,433 In different roles these days, but we've got in. 50 00:03:33,643 --> 00:03:38,238 When I was at Redgate, we always had different tools kind of dedicated to different roles. 51 00:03:38,268 --> 00:03:41,791 And we always thought of our monitoring tool as predominantly a DBA tool. 52 00:03:41,841 --> 00:03:46,551 We had backup tools and they were predominantly a DBA tool and we had a few others as well. 53 00:03:47,521 --> 00:03:47,971 Yeah. 54 00:03:47,971 --> 00:03:48,451 Well, yeah. 55 00:03:48,451 --> 00:03:54,294 So rep replication and major upgrades, I'd also consider, were typically part of the DBA role. 56 00:03:54,299 --> 00:04:00,026 Maybe, maybe not so much anymore, but again with managed services handling some of that, but monitoring, I know we've 57 00:04:00,026 --> 00:04:02,448 already discussed it in depth that just recently, but that 58 00:04:02,560 --> 00:04:08,658 Nikolay: Yeah, we, we, should say that those who just joined us a week ago, we discussed Pogo monitoring 59 00:04:08,898 --> 00:04:11,958 and, and I think it's, it was quite successful episode. 60 00:04:11,958 --> 00:04:13,418 So please check it out. 61 00:04:13,457 --> 00:04:23,485 But hundred percent like monitoring is still on the plate and it's very relevant skill that GBA should have, but you are missing big chunk 62 00:04:23,485 --> 00:04:28,422 of, or maybe just not mention it yet, tuning performance, tuning and 63 00:04:28,752 --> 00:04:28,962 Michael: Yeah. 64 00:04:29,442 --> 00:04:31,332 Nikolay: analysis query optimization. 65 00:04:31,465 --> 00:04:31,765 Right. 66 00:04:31,972 --> 00:04:32,452 Michael: Yes. 67 00:04:32,452 --> 00:04:40,372 So sometimes that, so once an organization is, is large enough, sometimes that gets put into a different team than the, than the engineering team. 68 00:04:40,432 --> 00:04:44,812 And that, that is often called the database team or the DBA team or some, something like that. 69 00:04:44,872 --> 00:04:47,482 But I've also seen that work taken on. 70 00:04:47,962 --> 00:04:49,702 Hundred percent by a development team. 71 00:04:49,702 --> 00:04:52,102 So I have seen, I have seen some of that. 72 00:04:52,102 --> 00:04:54,435 I guess, I guess they're doing DBA tasks at that point. 73 00:04:54,435 --> 00:04:55,295 Is that what you're saying? 74 00:04:55,485 --> 00:05:03,868 Nikolay: Well, when, when we discuss these things, we, we see that some things already went to manage services if you use them. 75 00:05:04,228 --> 00:05:13,205 But others, actually, all of them, when I just name them, I immediately distinguish two areas, infrastructure DBA and application DBA. 76 00:05:13,260 --> 00:05:15,525 I very, I like it a lot. 77 00:05:15,525 --> 00:05:17,219 I like this classification. 78 00:05:17,309 --> 00:05:18,029 It's very good. 79 00:05:18,119 --> 00:05:20,119 Michael: I've never seen it in a job advert. 80 00:05:20,149 --> 00:05:25,699 the first time I ever heard the phrase application DBA was reading a Haki Benita blog post. 81 00:05:25,759 --> 00:05:28,339 I don't know if that's where yeah, so either. 82 00:05:28,369 --> 00:05:29,954 Yeah, well kudos to him. 83 00:05:29,954 --> 00:05:30,374 Great blogs. 84 00:05:30,614 --> 00:05:33,224 If you haven't checked them out, I'll link them up in the show notes. 85 00:05:33,291 --> 00:05:35,481 But yeah, that feels like a really good distinction. 86 00:05:35,481 --> 00:05:36,981 And maybe what we're saying is. 87 00:05:37,240 --> 00:05:45,926 In the world of having managed services, there is less need for some of those infrastructure DBA tasks, but just as much need for the application DBA tasks. 88 00:05:45,976 --> 00:05:46,336 Nikolay: Right. 89 00:05:46,576 --> 00:05:47,056 Exactly. 90 00:05:47,236 --> 00:05:57,940 And I, I guess if we talk about junior DBA and how to start, the first question is which database system and which of these two areas you think 91 00:05:57,973 --> 00:06:07,001 most about like infrastructure area — people teams still manage Postgres themselves a lot, like RDS is not like a hundred percent of cases. 92 00:06:07,001 --> 00:06:07,241 Right? 93 00:06:07,571 --> 00:06:10,001 So infrastructure, DBA or application DBA. 94 00:06:10,188 --> 00:06:13,848 This is the main question to start because different skill sets are needed. 95 00:06:14,143 --> 00:06:14,953 Michael: Yeah, that's a good point. 96 00:06:14,958 --> 00:06:15,763 So you're thinking of 97 00:06:15,978 --> 00:06:17,928 Nikolay: There is overlapping of course, right? 98 00:06:17,933 --> 00:06:18,508 This is overlap. 99 00:06:18,833 --> 00:06:19,253 Michael: Yeah. 100 00:06:19,423 --> 00:06:22,513 Well, and at smaller organizations you might need to do both. 101 00:06:22,633 --> 00:06:28,608 Even in that kind of junior type role I've I've definitely seen some people fall into the role. 102 00:06:28,608 --> 00:06:33,760 Like they've one of our early customers described themselves as a, a pretend DBA. 103 00:06:34,090 --> 00:06:37,450 They were, you know, they were just the person in the, they didn't even step forward. 104 00:06:37,488 --> 00:06:42,924 For the role, when it was asked, they just were the one person that didn't step back when they were picking the person that was gonna. 105 00:06:43,219 --> 00:06:44,239 You know, look after it. 106 00:06:44,419 --> 00:06:50,495 But I guess if we talk about larger organizations, it's a bit easier because you can have these specialist teams or areas. 107 00:06:50,495 --> 00:06:58,892 And maybe if you can find a role in one of those where you can learn from the other application DBAs or the other infrastructure DBAs, then you've 108 00:06:58,892 --> 00:07:02,984 got a smaller set of responsibilities that you, you need to learn about. 109 00:07:03,044 --> 00:07:07,664 And you can maybe get skilled at before either learning the others or specializing deeper. 110 00:07:07,869 --> 00:07:08,169 Nikolay: Right. 111 00:07:08,259 --> 00:07:13,165 Well, in, in larger organizations, I think the need in application DBS is usually bigger than an 112 00:07:13,375 --> 00:07:16,699 infrastructure and also speak, thinking about structure. 113 00:07:16,699 --> 00:07:19,891 Infrastructure team probably is like some big department. 114 00:07:19,891 --> 00:07:28,931 And if, if they exist and they can work only, or if manage services or they can manage Postgres themselves, but it's some kind of monolithic entity. 115 00:07:28,931 --> 00:07:36,111 But if we think about engineering in bigger organizations, usually it's split to if, if they follow microservices, 116 00:07:36,111 --> 00:07:38,751 for example, it's a lot of teams, smaller teams. 117 00:07:38,889 --> 00:07:46,060 Even if there is no microservices approach, still, there are many teams and I strongly believe that it's good. 118 00:07:46,060 --> 00:07:52,128 When organization management found a way to have growing knowledge of databases inside each team. 119 00:07:52,518 --> 00:08:01,198 So there are probably few people who are experts, but inside each team, it's good to have junior DBAs and like midlevel DBAs application DBAs. 120 00:08:01,218 --> 00:08:04,848 They don't need to understand details about backups and replication. 121 00:08:05,248 --> 00:08:13,697 Monitoring, probably yes, some part of it, which related to optimization queries and some other basics like transaction looking and so on. 122 00:08:14,027 --> 00:08:18,976 But, I saw people who can optimize SQL, but they don't understand backups at all. 123 00:08:19,006 --> 00:08:19,786 And vice versa. 124 00:08:19,786 --> 00:08:24,615 I saw very good infrastructure DBAs who even cannot write complex sequel at all. 125 00:08:24,855 --> 00:08:25,995 Like only very basic. 126 00:08:26,475 --> 00:08:29,145 So it's like, it's there is split here. 127 00:08:29,475 --> 00:08:39,629 And I think application DBA is, is like overall if account numbers, I, I believe industry needs more application DBAs and than infrastructure DBAs in. 128 00:08:40,257 --> 00:08:41,307 Michael: Yeah, that makes sense. 129 00:08:41,397 --> 00:08:45,464 And it makes sense even if we are not talking solely about job titles. 130 00:08:45,494 --> 00:08:51,626 So a lot of those SREs D Bres a lot of the developers that have to take on DBA responsibilities. 131 00:08:52,551 --> 00:09:01,491 The majority of their, their DBA type work is probably still on the application side, worrying about correct indexing or looking at 132 00:09:01,558 --> 00:09:04,948 some dodgy queries that ORMs are spitting out that kind of thing. 133 00:09:05,068 --> 00:09:10,618 So, and maybe, maybe advising team members on, on pull requests, if they have people in their team that don't know as much 134 00:09:10,618 --> 00:09:17,007 about databases, maybe giving them feedback or explaining to them a better way of doing something in the database. 135 00:09:17,290 --> 00:09:17,560 Nikolay: Right. 136 00:09:17,560 --> 00:09:23,023 So let's start probably with how to be a junior infrastructure to. 137 00:09:23,798 --> 00:09:24,158 Right. 138 00:09:24,638 --> 00:09:29,128 And then we will discuss how to be junior application DBA. 139 00:09:29,148 --> 00:09:38,640 What's where to start, what to do, like what knowledge should be definitely in skill set or knowledge set and how to develop skills further. 140 00:09:38,640 --> 00:09:45,510 If you start, any DBA, but if you start my first recommendation is just to read documentation. 141 00:09:45,693 --> 00:09:48,663 If you print it, it's 3000 pages if I'm not mistaken, 142 00:09:48,813 --> 00:09:49,203 Michael: Really? 143 00:09:49,548 --> 00:09:49,788 Nikolay: Yeah. 144 00:09:49,793 --> 00:09:51,078 I re I remember funny story. 145 00:09:51,083 --> 00:09:58,698 I remember going, going to the office of CTO of very large organization who like, who was my, became my customer. 146 00:09:59,088 --> 00:10:03,673 And his secretary said of, so Postgres AI and she. 147 00:10:04,543 --> 00:10:07,153 Oh, I just printed 3000 pages. 148 00:10:07,513 --> 00:10:08,893 This is, this is you. 149 00:10:09,103 --> 00:10:09,763 I don't know. 150 00:10:09,763 --> 00:10:13,003 I just use it and suggest people how to use it better. 151 00:10:13,003 --> 00:10:13,363 That's it. 152 00:10:13,400 --> 00:10:17,758 so that's how I, I know 3000 pages at a four of letter size uh, format. 153 00:10:18,038 --> 00:10:19,898 It's very well structured, of course. 154 00:10:19,988 --> 00:10:23,198 And I suggest reading it from the very beginning and just read it. 155 00:10:23,198 --> 00:10:29,292 And of course skip some non-relevant parts, but very beginning and a lot of interesting stuff. 156 00:10:29,349 --> 00:10:30,939 So the documentation is great. 157 00:10:30,991 --> 00:10:34,001 It it's missing a lot of things like use cases, but it's great. 158 00:10:34,337 --> 00:10:36,377 Michael: It's actually split quite well for this topic. 159 00:10:36,377 --> 00:10:38,935 I'm just looking at the table of contents right now. 160 00:10:38,995 --> 00:10:42,075 And there's obviously a really good introduction at the beginning. 161 00:10:42,375 --> 00:10:48,404 And then it does a whole load on sequel language which I'd probably consider a bit more on the application 162 00:10:48,554 --> 00:10:48,854 Nikolay: Yeah. 163 00:10:48,854 --> 00:10:54,284 As I've said, don't know, SQL almost like, so , it's possible. 164 00:10:54,441 --> 00:10:57,921 Michael: But then the next whole big section is on server administration. 165 00:10:57,921 --> 00:11:03,404 So that feels like a really good place to start for, for the infrastructure DBA side of things 166 00:11:03,811 --> 00:11:04,441 Nikolay: Exactly. 167 00:11:04,531 --> 00:11:05,011 Yeah. 168 00:11:05,274 --> 00:11:09,724 Michael: and slight aside, but the documentations famously good in Postgres. 169 00:11:09,745 --> 00:11:16,357 So I know, I know not all systems have good documentation, but it is . I found it really approachable when I was first getting to know Postgres. 170 00:11:16,387 --> 00:11:22,991 And the other thing that was found surprisingly accessible to, to me when I was new to Postgres was the source code as well. 171 00:11:22,991 --> 00:11:26,471 The not in the source code itself, but the comments in the source code. 172 00:11:26,471 --> 00:11:35,441 So if you're ever in interested in what, like why something's a certain way or why a certain default is a certain number, that kind of thing often, there's 173 00:11:35,446 --> 00:11:38,481 a, there's a really good comment in the source code that will tell you why. 174 00:11:38,781 --> 00:11:40,191 So don't be scared of that. 175 00:11:40,291 --> 00:11:41,281 Nikolay: Definitely a hundred. 176 00:11:41,286 --> 00:11:41,881 Absolutely. 177 00:11:41,911 --> 00:11:51,128 And I, like, I always say the same and but it's already kind of deeper than junior maybe, but also like if you go to source code 178 00:11:51,158 --> 00:11:54,874 uh, you shouldn't be scared about C and just read the comments. 179 00:11:54,874 --> 00:11:59,314 Sometimes comments are in the readme file in some directory, or sometimes they are in line. 180 00:11:59,614 --> 00:12:06,798 And it's not, it's not just, if you want to go deeper, sometimes important concepts are missing in documentation happens. 181 00:12:06,803 --> 00:12:10,376 For example heap only twos, tops, hip only tops. 182 00:12:10,381 --> 00:12:10,592 Right? 183 00:12:10,592 --> 00:12:17,234 Currently hackers, I saw the discussion hackers and it's about to be added to documentation on the reason 184 00:12:17,264 --> 00:12:20,534 like, or, or just recently added and it'll be in S 15. 185 00:12:20,534 --> 00:12:21,644 So it's still missing. 186 00:12:21,649 --> 00:12:25,720 And I always used link to source code to explain what it is. 187 00:12:25,776 --> 00:12:29,188 So source code it augments documentation. 188 00:12:29,262 --> 00:12:34,440 Michael: Yeah, by the way I looked it up and both two pools and tops are very much acceptable. 189 00:12:34,440 --> 00:12:35,969 So it's, it's good news. 190 00:12:36,046 --> 00:12:36,826 Nikolay: Problem solved. 191 00:12:36,886 --> 00:12:37,636 We discussed it 192 00:12:37,636 --> 00:12:41,596 like three weeks ago or two, right. 193 00:12:41,986 --> 00:12:46,578 But don't expect good how-tos in documentation or in on soft score. 194 00:12:46,638 --> 00:12:48,108 This is what is missing there. 195 00:12:48,108 --> 00:12:55,623 And for that you need books like video courses, maybe, or articles, blog posts that are many, many very good materials around. 196 00:12:55,903 --> 00:12:58,653 planet postgresql org will show some of them. 197 00:12:58,796 --> 00:13:06,329 Howtos usually are there, unfortunately the PSGs documentation doesn't have, howtos almost like step by step and so, 198 00:13:06,646 --> 00:13:08,326 Michael: Otherwise, it would be 10,000 pages. 199 00:13:08,768 --> 00:13:13,758 But yeah, the other thing I'd say is it, I think it's acceptable and okay. 200 00:13:13,758 --> 00:13:21,048 As a junior DBA to only know one way of doing something, so make sure, you know, like the basics and how, how the 201 00:13:21,048 --> 00:13:24,318 default is in Postgres and a way of solving a problem. 202 00:13:24,708 --> 00:13:26,508 And maybe like, don't worry. 203 00:13:27,453 --> 00:13:28,923 There's 20 different backup tools. 204 00:13:28,923 --> 00:13:34,692 Make sure you make sure, you know, and can use one of the better ones or you understand the basics of, of one of them. 205 00:13:34,692 --> 00:13:39,217 I wouldn't worry too much about there's there's so much depth in each of these topic areas that you could go into. 206 00:13:39,222 --> 00:13:45,581 I'd recommend more getting a breadth of the tasks and being able to solve each one a little bit instead of, or how would you, how do you 207 00:13:45,666 --> 00:13:47,586 Nikolay: well, it depends on personality. 208 00:13:47,586 --> 00:13:50,853 Some people have like, they read documentation. 209 00:13:50,853 --> 00:13:51,453 It's like a. 210 00:13:51,574 --> 00:13:55,389 A book which educates you explains the concept. 211 00:13:55,389 --> 00:14:04,769 So shows, shows some examples and so on, but some people just need the like modern time we live in a stack overflow approach is 212 00:14:04,774 --> 00:14:08,970 very popular and sometimes people just need step by step recipes. 213 00:14:09,060 --> 00:14:13,080 And this is probably bad because if you skip reading books, it's bad. 214 00:14:13,140 --> 00:14:18,165 But if you already read the book and just need a few examples how to do something and step by. 215 00:14:19,035 --> 00:14:19,665 It's a good thing. 216 00:14:19,995 --> 00:14:26,743 I, I'm not saying that the documentation does have examples, but it's structured in a way to explain things to teach, not 217 00:14:26,743 --> 00:14:30,913 to show how to perform basic things or more advanced things. 218 00:14:31,393 --> 00:14:40,028 So topics that infrastructure DBA should know of course MVCC and I can recommend materials from Bruce Momjian of course. 219 00:14:40,051 --> 00:14:40,321 Right. 220 00:14:40,351 --> 00:14:41,881 We can, we will provide some links. 221 00:14:41,911 --> 00:14:51,501 Bruce always has perfect presentations, especially explaining basics also backups replication monitoring transaction processing. 222 00:14:51,648 --> 00:14:52,089 Right? 223 00:14:52,114 --> 00:14:52,514 What else? 224 00:14:52,894 --> 00:14:55,594 Michael: Upgrades, minor upgrades, major upgrades, 225 00:14:56,044 --> 00:14:56,714 Nikolay: Upgrades? 226 00:14:56,714 --> 00:14:58,724 great, extensions. 227 00:14:58,944 --> 00:15:03,265 Michael: maybe awareness of operating system and how the operating system and the database 228 00:15:03,818 --> 00:15:10,748 Nikolay: Oh, well, in general, a good DBA infrastructure DBA should be good expert in Linux, definitely. 229 00:15:10,808 --> 00:15:12,970 And file systems and clouds. 230 00:15:12,970 --> 00:15:16,518 If it's a cloud environment most of cases currently are cloud environments. 231 00:15:16,518 --> 00:15:21,380 So of course these areas should be covered a certain depth and actually also. 232 00:15:21,467 --> 00:15:26,963 We discussed before our episode that sometimes people manage multiple database systems. 233 00:15:27,113 --> 00:15:32,870 I feel very rare, like successful stories like that it's quite, it's quite rare. 234 00:15:32,960 --> 00:15:35,570 Database system is very, very advanced system. 235 00:15:35,631 --> 00:15:36,951 It's very deep advanced. 236 00:15:36,951 --> 00:15:37,984 And I saw ideas. 237 00:15:37,984 --> 00:15:40,354 Let's like we have a bunch of experts in Oracle. 238 00:15:40,354 --> 00:15:43,114 For example, let, let teach them all in Pogo. 239 00:15:43,234 --> 00:15:44,804 Our organization has them both. 240 00:15:45,044 --> 00:15:47,501 So they will be experts in both. 241 00:15:47,801 --> 00:15:50,101 It's very difficult to follow both areas. 242 00:15:50,131 --> 00:15:52,042 I don't have time to catch up. 243 00:15:52,242 --> 00:15:59,141 For example, when I'm America, I wake up a lot of European guys posted so many great materials about polyus. 244 00:15:59,411 --> 00:16:01,779 I want to read them all, but I don't have time. 245 00:16:01,779 --> 00:16:02,049 Right. 246 00:16:02,439 --> 00:16:08,726 And , if you put Oracle on or SQL server, you just don't have time to to develop yourself. 247 00:16:08,936 --> 00:16:12,909 So my point is that you should be good expert in just one system. 248 00:16:12,969 --> 00:16:18,483 Of course, it's possible to be PLO it's possible, but you won't be very deep eventually. 249 00:16:18,824 --> 00:16:23,034 Michael: And, and I guess the important point for juniors is that it's totally okay to only know. 250 00:16:24,289 --> 00:16:25,474 Postgres is a really good choice. 251 00:16:25,744 --> 00:16:25,984 Nikolay: right. 252 00:16:25,984 --> 00:16:26,374 Yeah. 253 00:16:26,379 --> 00:16:26,414 Yeah. 254 00:16:26,534 --> 00:16:26,894 I agree. 255 00:16:26,941 --> 00:16:32,361 Michael: but it shouldn't worry you, if you're considering a junior role and you only have experience in one database management system, 256 00:16:32,361 --> 00:16:35,601 it's okay to learn one there will be good jobs out there for you. 257 00:16:36,051 --> 00:16:37,131 Nikolay: right, right. 258 00:16:37,431 --> 00:16:41,819 But you should be not expert, but you should know some languages addition. 259 00:16:42,072 --> 00:16:43,722 because sometimes you need to code something. 260 00:16:43,722 --> 00:16:53,214 And also if your colleagues use some languages, you will need to understand how some libraries or, ORMs various middleware things work. 261 00:16:53,304 --> 00:16:53,723 Right. 262 00:16:53,742 --> 00:16:55,962 Michael: this feels like a really good time to switch over. 263 00:16:56,052 --> 00:16:58,572 I feel like we've talked about infrastructure a little bit, so we let yeah. 264 00:16:58,572 --> 00:16:59,982 Let's switch over to application 265 00:17:00,026 --> 00:17:04,207 Nikolay: Before they do that, just let let's let's mention quite good books. 266 00:17:04,255 --> 00:17:05,575 Both available online. 267 00:17:05,636 --> 00:17:12,449 If you go, if you want to go deeper of course you need to understand internals because internals it's how things work from inside. 268 00:17:12,509 --> 00:17:19,439 If, if you understand them, you can explain many things that, that that's which impossible explain if you don't go inside. 269 00:17:19,559 --> 00:17:23,677 And the first is internals online from Suzuki. 270 00:17:23,737 --> 00:17:24,097 Right? 271 00:17:24,097 --> 00:17:25,027 I, I don't sorry. 272 00:17:25,027 --> 00:17:25,707 I don't remember. 273 00:17:26,062 --> 00:17:26,482 Michael: We'll get it. 274 00:17:26,482 --> 00:17:27,832 I'll get it in the, the link in the 275 00:17:28,032 --> 00:17:33,462 Nikolay: Suzuki was Def sorry about name, but second one recently published from Egor Rogov. 276 00:17:33,651 --> 00:17:35,306 And it was translated from Russian. 277 00:17:35,443 --> 00:17:36,313 It's very good book. 278 00:17:36,433 --> 00:17:40,846 It's missing many parts, of course, but parts it has covered very well. 279 00:17:41,356 --> 00:17:44,806 For example, indexes, how indexes work, various types of indexes. 280 00:17:44,979 --> 00:17:46,348 So I, I recommend both 281 00:17:46,378 --> 00:17:48,751 Michael: yeah, you were right by the way, Hironobu Suzuki. 282 00:17:48,801 --> 00:17:49,131 Nikolay: Sorry. 283 00:17:49,131 --> 00:17:51,411 I forgot the first name, right. 284 00:17:51,442 --> 00:18:01,655 These two books will give you very, very far horizon for development to be more advanced DBA, actually infrastructure first, but also application. 285 00:18:01,715 --> 00:18:06,629 Cause if you want to understand how various parts working, you need to go into internals. 286 00:18:06,647 --> 00:18:07,937 If you, if you're an application, 287 00:18:08,171 --> 00:18:09,161 Michael: Yeah, a hundred percent. 288 00:18:09,161 --> 00:18:17,741 Both of them are excellent and go quite deep, but they're also written in a simple, like they're written simply and it's, I found it able to 289 00:18:17,741 --> 00:18:20,951 follow topics that I didn't know about at all and learn about them. 290 00:18:20,951 --> 00:18:22,661 So I think they're both excellent written. 291 00:18:22,736 --> 00:18:23,036 Nikolay: Yeah. 292 00:18:23,066 --> 00:18:23,306 Yeah. 293 00:18:23,306 --> 00:18:29,546 It looked like they are very different, but, but I enjoy reading them both and I do it all the time. 294 00:18:29,546 --> 00:18:31,256 I return to specific topics. 295 00:18:31,256 --> 00:18:38,896 I use them as a reference to understand specific, like to, to recall something or to understand something which I didn't understand before and so on. 296 00:18:38,926 --> 00:18:42,390 Like all the time, this is more advanced DBA. 297 00:18:42,390 --> 00:18:46,290 Two must have resources this time, in addition to the commutation and source code. 298 00:18:46,290 --> 00:18:47,550 Of course, of course. 299 00:18:48,074 --> 00:18:48,344 Oh. 300 00:18:48,344 --> 00:18:54,749 And finally for simple things, how, how to I criticize the commutation in terms of how to, there are many 301 00:18:54,754 --> 00:18:57,877 block posts but of course let's mention one of them. 302 00:18:57,937 --> 00:19:02,783 This is cyber tech and their posts are always brief, problem oriented. 303 00:19:02,813 --> 00:19:04,553 And I enjoy reading them as well. 304 00:19:04,613 --> 00:19:07,163 They like, let's take this problem and attack it. 305 00:19:07,163 --> 00:19:10,371 And also they explain things, but they show how to do something. 306 00:19:10,399 --> 00:19:10,849 Right, 307 00:19:10,873 --> 00:19:11,575 Michael: Yeah, true. 308 00:19:11,625 --> 00:19:17,159 I like them as well, and I, I think they include some comics as well at the start, especially I think it's Laurenz. 309 00:19:17,179 --> 00:19:17,599 He does. 310 00:19:17,657 --> 00:19:20,447 But the other thing I was gonna say. 311 00:19:21,127 --> 00:19:26,047 There's a blog aggregation site called planet Postgres that not everybody seems to be aware of. 312 00:19:26,047 --> 00:19:27,537 And that's a great resource. 313 00:19:27,537 --> 00:19:34,364 The Cybertec blog gets syndicated to it as do maybe 20, 30, 40 others that get released regularly. 314 00:19:34,664 --> 00:19:41,338 So that's a great place to, if you want to start seeing what people are writing about at the moment, but of course, if you 315 00:19:41,338 --> 00:19:43,887 want to know a bit more about a topic, you can just search it. 316 00:19:43,887 --> 00:19:47,067 And often one of those good blog post will be surfaced right near the top. 317 00:19:47,203 --> 00:19:47,473 Nikolay: Yep. 318 00:19:47,590 --> 00:19:47,950 Okay. 319 00:19:48,340 --> 00:19:50,072 Application DBA where to. 320 00:19:50,296 --> 00:19:57,257 Michael: Right, so what would be expected, let's say, if you've started as a developer or data analyst or something like that. 321 00:19:57,647 --> 00:20:01,217 And you've been taking on more and more database type tasks. 322 00:20:01,217 --> 00:20:05,437 You've, you've become maybe a little bit of the expert in your team when it comes to database things. 323 00:20:05,527 --> 00:20:11,407 But if you look at what a DBA is expected to do, maybe that's quite intimidating and you feel like you've got a lot to learn. 324 00:20:11,647 --> 00:20:12,727 Where should you start? 325 00:20:12,727 --> 00:20:14,887 Or, or could I apply for a junior DBA? 326 00:20:15,257 --> 00:20:21,299 Application DBA role today and then learn those things as I, as I work or, or what should I be doing in 327 00:20:21,622 --> 00:20:26,542 Nikolay: I think the number one step is to learn SQL and learn modern SQL. 328 00:20:26,932 --> 00:20:35,919 It can be not very advanced, but it should be modern, not to be like from SQL 92 standard and good, good place to learn it. 329 00:20:35,919 --> 00:20:39,109 For example, it's modern sql.com from Markus Winand. 330 00:20:39,564 --> 00:20:39,864 Right. 331 00:20:40,584 --> 00:20:49,937 So I also Use the Index Luke these two websites are very like, they are great very great places to, of course, basics can be learned from 332 00:20:49,937 --> 00:20:59,077 anywhere, like from, poss documentation, from books, courses it's, it's not a big deal, but, these two resources will give you some 333 00:20:59,077 --> 00:21:04,002 good momentum towards modern SQL and a little bit more advanced SQL. 334 00:21:04,002 --> 00:21:06,198 And you understand how much you can do. 335 00:21:06,558 --> 00:21:09,818 And of course, application DBA should know SQL well. 336 00:21:10,130 --> 00:21:15,350 Michael: Yeah, and I think that's a really good point, even if you flick through modern SQL and just make sure you don't have any blind 337 00:21:15,350 --> 00:21:21,260 spots, maybe you've already got really advanced on SQL through your, through your development role or, or different role you have, but 338 00:21:21,260 --> 00:21:24,350 maybe you, you just haven't had a need for window functions yet. 339 00:21:24,380 --> 00:21:28,770 Or so there's like one realize like one or two topics that you haven't had to learn yet. 340 00:21:28,770 --> 00:21:35,332 It's just not been necessary, but brushing up on note or learning about those and then maybe trying it out a little bit, that could be really helpful. 341 00:21:35,396 --> 00:21:38,426 I was gonna mention one more book which is The Art of PostgreSQL. 342 00:21:38,446 --> 00:21:40,813 I think that would be re a really good, for an application. 343 00:21:41,258 --> 00:21:47,386 Nikolay: And since we started to talk about application DBA, you already mentioned Haki Benita and has beautiful posts. 344 00:21:47,386 --> 00:21:48,024 Of course. 345 00:21:48,024 --> 00:21:52,783 And some of posts are structured in a way like items 1, 2, 3, 4, and so on. 346 00:21:52,788 --> 00:21:53,383 It's very good. 347 00:21:53,563 --> 00:21:56,863 Like I found them very, very, they also. 348 00:21:57,163 --> 00:22:00,901 We'll educate you and let you know something new all the time. 349 00:22:01,431 --> 00:22:04,195 And of course like let's, let's do some buzzwords. 350 00:22:04,225 --> 00:22:10,010 You mentioned window functions, CTE, so WITH, recursive CTE, lateral, what else? 351 00:22:10,063 --> 00:22:12,350 If you are, there are things which are standard. 352 00:22:12,380 --> 00:22:13,550 These things are standard. 353 00:22:13,747 --> 00:22:20,767 , but there are things which POG has probably for many, many years, which also worked learning, for example, how to work with 354 00:22:20,799 --> 00:22:29,090 arrays, race other all the time, or, or we mentioned in some of previous episodes row type or record type row type, basically. 355 00:22:29,450 --> 00:22:33,350 So how to wrap and unwrap these things. 356 00:22:33,429 --> 00:22:37,428 Many, many functions posts has regular expressions so on. 357 00:22:37,527 --> 00:22:39,247 Michael: jsonb, like there's so 358 00:22:39,289 --> 00:22:44,479 Nikolay: oh, it's language inside language because Jason B is already in stand. 359 00:22:44,562 --> 00:22:48,642 Not Jason B standard Ja Jason support is in modern SQL standard. 360 00:22:49,002 --> 00:22:54,822 So it's like language inside language or they are interconnected, of course. 361 00:22:54,822 --> 00:22:57,382 And of course this is huge area worth learning 362 00:22:57,492 --> 00:22:57,702 Michael: But. 363 00:22:57,743 --> 00:23:03,833 Even to get the basics, you know, even to, to have a look at where in the documentation, can I look up which functions 364 00:23:03,833 --> 00:23:06,623 there are, or, you know, what, what could it be used for? 365 00:23:06,623 --> 00:23:08,483 I think there was, you mentioned cyber tech already. 366 00:23:08,483 --> 00:23:12,203 I think they did a really good blog post on when is it appropriate when. 367 00:23:12,643 --> 00:23:14,173 Why do we have JB support? 368 00:23:14,263 --> 00:23:16,093 When is it a perfect use case? 369 00:23:16,093 --> 00:23:17,353 When is it a terrible use case? 370 00:23:17,353 --> 00:23:24,703 That's the kind of thing an application DBA should be helping their team with in terms of if they, if they decide to just have an ID column 371 00:23:24,703 --> 00:23:27,613 and then a JB column, they want to throw the entire schemer in there. 372 00:23:27,853 --> 00:23:32,353 Maybe it's the application DBA that should be pushing back and saying, that's not the best ideal, you know, here's. 373 00:23:32,833 --> 00:23:38,347 Or maybe if they, they do the opposite and they want a super wide table with loads of columns that are gonna be mostly nulls. 374 00:23:38,352 --> 00:23:46,157 Maybe that could be, you know, knowing when these things are appropriate, feels like the, the job of the application, DBA, even a junior one, maybe. 375 00:23:46,398 --> 00:23:47,808 Nikolay: Yeah, this is good point. 376 00:23:47,928 --> 00:23:53,688 So to start to, to do something, I always use Google and it's so good that recently it was fixed and we 377 00:23:53,688 --> 00:23:57,266 see the latest major version in search result page. 378 00:23:57,716 --> 00:24:04,231 But since you say, like how to choose something here, we already start to discuss very important skill to make experiments. 379 00:24:05,225 --> 00:24:12,769 like, it's so important to learn from very beginning instead of asking how to do something or, or why, why it's not working. 380 00:24:13,039 --> 00:24:13,549 It's good. 381 00:24:13,655 --> 00:24:20,850 If people ask, not like, show me how to do like, but show me how to reproduce something and compare and choose. 382 00:24:21,140 --> 00:24:31,805 Instead of asking what is better json or pure relational approach normalized to BNF boy called normal form B BNS, BCN F right. 383 00:24:32,195 --> 00:24:35,075 Or like third normal form at least. 384 00:24:35,075 --> 00:24:38,165 And so instead of that, you just, you, you need to. 385 00:24:39,125 --> 00:24:42,995 People help me to conduct proper experimenters. 386 00:24:42,995 --> 00:24:53,417 So I, I will be able in future to compare myself and, you know, my approach, like for, for infrastructure DBA, we need multi-session experiments. 387 00:24:53,496 --> 00:25:04,854 We should be alone on machine, but for application debates, it's totally fine to to share one machine among multiple people and focus on IO operations. 388 00:25:04,854 --> 00:25:11,148 Mostly not on timing when you optimize queries and you can conduct experiments and see results. 389 00:25:11,238 --> 00:25:14,308 Even if you have one connection, you don't need to pgbench. 390 00:25:14,368 --> 00:25:16,498 You don't need benchmarking approaches at all. 391 00:25:16,768 --> 00:25:23,463 You just need to design two schemas to write queries, to compare plans, to compare how many you operations in both. 392 00:25:23,673 --> 00:25:25,503 And this is basic recipe. 393 00:25:25,630 --> 00:25:27,943 people always use and I'm always using. 394 00:25:28,060 --> 00:25:29,020 Michael: That's a really good point. 395 00:25:29,020 --> 00:25:33,400 Actually, I would include a lot of performance stuff in application DBA role. 396 00:25:33,460 --> 00:25:40,120 So in terms of knowing maybe you want to learn, even if it's a junior, maybe you want to learn of course, about Bre indexes, but maybe 397 00:25:40,120 --> 00:25:47,428 you also want to learn about the two or three most common types and when they're appropriate as well as performance optimization work in 398 00:25:47,428 --> 00:25:50,887 general, being able to help people when and why their query might be. 399 00:25:51,827 --> 00:25:52,127 Nikolay: Right. 400 00:25:52,127 --> 00:25:59,741 Well, since I work in this area in the area of scaling the process of how teams approach SQL optimization, I 401 00:25:59,801 --> 00:26:03,491 see like how you can grow and you can grow your team. 402 00:26:03,561 --> 00:26:11,131 I, I think everyone should be expert in SQL performance, but I see that 90% of people don't care, they just, they 403 00:26:11,131 --> 00:26:15,031 just want to ship their features and to be good enough. 404 00:26:15,191 --> 00:26:15,501 Right. 405 00:26:15,881 --> 00:26:24,303 So not, not to fail at, at release time, but since like, if you, if application DBAs of course should be performance experts. 406 00:26:24,913 --> 00:26:31,961 Not huge experts, but some kind of experts and they should care about performance and they should protect the 407 00:26:31,961 --> 00:26:35,056 application from poor performance, they should care about it. 408 00:26:35,056 --> 00:26:42,431 And it's good if you like, know how to conduct experiments and if it's also good, if you start using experiments 409 00:26:42,491 --> 00:26:45,589 in some kind of workflow, Constantly in the process. 410 00:26:46,069 --> 00:26:50,929 And it's good when also you ask others to conduct experiments. 411 00:26:50,929 --> 00:26:57,957 If even if they cannot understand all the details, but you have some workflow in your team and artifacts are collected. 412 00:26:57,962 --> 00:27:04,823 So you can be in application DBA, you can analyze them and suggest if it's good or not good, or how to optimize. 413 00:27:04,895 --> 00:27:05,135 Right. 414 00:27:05,195 --> 00:27:07,115 And there are many, many resources here. 415 00:27:07,115 --> 00:27:16,581 I think we like out of time to cover all of them, but of course explain tools are great examples here: explain.depesz, explain.dalibo, pgMustard of course. 416 00:27:16,609 --> 00:27:22,408 And these tools will of course you should have them and use them to see to read plans. 417 00:27:22,931 --> 00:27:23,651 Michael: Yeah, exactly. 418 00:27:23,651 --> 00:27:26,885 Find one you like and get good at using it to solve problems. 419 00:27:26,935 --> 00:27:33,717 Actually I remembered one last thing on the infrastructure side that I forgot that what we, I don't think we mentioned was the maintenance side of things. 420 00:27:33,867 --> 00:27:35,327 Maybe that's both of these roles? 421 00:27:35,343 --> 00:27:35,883 Nikolay: Maintenance. 422 00:27:35,888 --> 00:27:38,566 You mean fighting with bloat, rebuilding indexes or, 423 00:27:38,738 --> 00:27:39,968 Michael: and rebuilding indexes. 424 00:27:39,968 --> 00:27:40,178 Yeah. 425 00:27:40,178 --> 00:27:43,541 So it's, I guess, is that, are they the only maintenance tasks, 426 00:27:43,954 --> 00:27:46,894 Nikolay: Well minor grades, my well major grades. 427 00:27:46,894 --> 00:27:49,114 It's already huge task still. 428 00:27:49,197 --> 00:27:57,841 But minor grids can be considered as a, as maintenance task tasks or some switchovers when we move to some newer hardware, for example, and so on. 429 00:27:58,318 --> 00:27:58,498 Michael: Yeah. 430 00:27:59,168 --> 00:27:59,498 Awesome. 431 00:28:00,018 --> 00:28:00,618 Was there 432 00:28:00,648 --> 00:28:02,868 Nikolay: We can, we can add, add, and a lot. 433 00:28:03,078 --> 00:28:08,691 I, I feel we didn't cover well application DBA because we like performance topic is huge. 434 00:28:09,081 --> 00:28:17,271 For example, I mentioned regular expressions, but if you use them blindly, when you grow to a billion rows, you won't be able to use them because 435 00:28:17,334 --> 00:28:21,008 of lack of some index support, then you will find postgres trigrams. 436 00:28:21,028 --> 00:28:25,628 Then you will find some ways to improve this area, but it's huge. 437 00:28:25,628 --> 00:28:27,818 Like you need to, deal with many extensions. 438 00:28:27,891 --> 00:28:30,831 You need to understand partitioning and so on and so on. 439 00:28:30,831 --> 00:28:30,991 And so. 440 00:28:31,258 --> 00:28:36,885 Michael: Well, but for a junior, let's say for a junior person coming from a development background type thing, I think 441 00:28:36,885 --> 00:28:39,795 they'll have a lot of those things, the basics of them already. 442 00:28:39,795 --> 00:28:47,014 So it might be a wider topic if you were to go from, let's say straight out of school or straight out of college into that role, there would be a lot 443 00:28:47,014 --> 00:28:54,214 to learn, but if you're coming from a different profession, I think there a lot of those skills, you might already have some experience with them, 444 00:28:54,424 --> 00:28:58,390 and maybe it's an easier role to transition into from a different one. 445 00:28:58,660 --> 00:28:59,290 And then the, the. 446 00:28:59,625 --> 00:29:06,055 Other point I wanted to make around that before we finish was that the role could be quite different at different stages of company. 447 00:29:06,055 --> 00:29:11,829 And depending on how you learn best, it might be that you'd, you'd be much better off in one of these than the others, but it's definitely worth 448 00:29:11,834 --> 00:29:19,628 considering, you know, if, if you love being chucked in deep end and you love dealing with problems by facing them head on maybe a startup where 449 00:29:19,628 --> 00:29:26,395 they're scaling very quickly is a great place to learn and maybe getting a developer role and putting your hand up when there are database problems. 450 00:29:26,695 --> 00:29:34,185 That's a great way of learning if that's where you thrive, but if that is terrifying to you and I, I would totally empathize 451 00:29:34,455 --> 00:29:40,080 then maybe you're gonna be better off joining a much bigger company, or much more like maybe steady stable company. 452 00:29:40,085 --> 00:29:41,370 That's uh, smaller. 453 00:29:41,860 --> 00:29:43,990 Where you can learn from somebody who's already been doing it. 454 00:29:43,995 --> 00:29:49,270 And you you've got time to experiment and you've got time to learn these skills when it's not an emergency 455 00:29:49,450 --> 00:29:51,760 and maybe have some help when there is an emergency. 456 00:29:51,760 --> 00:29:55,180 And there's a few of you to be able to jump on the problem together, that kind of thing. 457 00:29:55,283 --> 00:30:02,855 The other third category that I hadn't really thought much about, but is really common in the Postgres world is the consultancy space. 458 00:30:03,095 --> 00:30:07,585 There's lot of people hire DBAs or, or outsource some of this work. 459 00:30:07,585 --> 00:30:11,414 So there are lots of consultancies that are hiring for DBA roles. 460 00:30:11,414 --> 00:30:13,904 And I, I imagine that'd be a great place. 461 00:30:13,904 --> 00:30:14,204 Maybe. 462 00:30:14,414 --> 00:30:21,727 I'm not sure they hire many juniors, but maybe once you've got those basic skills , or maybe some of them do hire juniors and you can, those, 463 00:30:21,727 --> 00:30:25,387 those are the places where you can learn from real, like deep experts. 464 00:30:25,392 --> 00:30:26,857 And that might be where you'd thrive as well. 465 00:30:27,269 --> 00:30:27,599 Nikolay: Yeah. 466 00:30:27,599 --> 00:30:29,969 And they're experts in specific topics. 467 00:30:29,969 --> 00:30:30,689 Of course. 468 00:30:30,779 --> 00:30:31,469 So I agree. 469 00:30:31,469 --> 00:30:35,803 And it's, it's a good idea to, to Depending on the company size. 470 00:30:35,863 --> 00:30:43,750 It it's a good idea to hire consultants for short term or maybe for long term, if you need to grow expertise internally, constantly, 471 00:30:43,783 --> 00:30:46,954 of course consultants will help them to grow all the time. 472 00:30:46,989 --> 00:30:48,579 It's a good, good approach. 473 00:30:48,582 --> 00:30:51,560 But maybe last recommendation from, for me. 474 00:30:51,568 --> 00:31:00,299 Don't like we, we discussed this split between infrastructure and application DBA, but if you chose one side, don't underestimate the other side. 475 00:31:00,659 --> 00:31:05,949 For example, if you came from data analytics area, To application DBA. 476 00:31:05,979 --> 00:31:13,533 And don't, if you don't care about infrastructure at all, you don't, you just, you just want to learn sequel to understand, 477 00:31:13,593 --> 00:31:17,204 explain to understand some indexes of course, some extensions. 478 00:31:17,234 --> 00:31:17,624 Good. 479 00:31:18,044 --> 00:31:23,777 But if you you never read about MVCC about how replication works. 480 00:31:24,287 --> 00:31:31,307 You'll be very, very surprised when you see that each time you update not changing anything, new version, new tap is 481 00:31:31,307 --> 00:31:40,291 created, or for example, long run query on standby note somehow affects the primary node and the performance degrades. 482 00:31:40,591 --> 00:31:48,184 This will be a big surprise to you and only understanding internals and the, the other side of this big area, which, which we call 483 00:31:48,184 --> 00:31:53,043 DBA will help you to know what to do next, how to deal with it. 484 00:31:53,241 --> 00:31:55,088 Michael: Yeah, absolutely wonderful. 485 00:31:55,178 --> 00:31:56,228 Well, thanks so much, Nicola. 486 00:31:56,233 --> 00:31:58,688 I, I think that's a good, hopefully that's a good overview. 487 00:31:58,718 --> 00:32:04,391 Thanks again for the request to cover this, I will, I'll make sure to post this up as a reply. 488 00:32:04,528 --> 00:32:06,028 Nikolay: With all the links. 489 00:32:06,366 --> 00:32:07,296 Michael: Yeah, exactly. 490 00:32:07,393 --> 00:32:11,403 So yeah, I feel like we are giving people who, long reading lists, but it is worth it. 491 00:32:11,503 --> 00:32:11,863 So, yeah. 492 00:32:12,133 --> 00:32:13,453 Thanks everyone for joining us again. 493 00:32:13,480 --> 00:32:15,330 keep the requests and suggestions coming. 494 00:32:15,333 --> 00:32:19,168 Nikolay: Subscribe, like, and share in your social networks and groups. 495 00:32:19,193 --> 00:32:21,593 Michael: Yeah, we really appreciate everybody that's done that so far. 496 00:32:21,670 --> 00:32:22,450 It helps us a lot. 497 00:32:22,450 --> 00:32:25,120 It helps us get more and better suggestions as well. 498 00:32:25,180 --> 00:32:26,080 So we appreciate it. 499 00:32:26,604 --> 00:32:27,464 Nikolay: Feedback drives us. 500 00:32:27,494 --> 00:32:28,454 Thank you so much. 501 00:32:28,540 --> 00:32:28,960 Michael: Yeah. 502 00:32:29,470 --> 00:32:29,830 Take care. 503 00:32:30,838 --> 00:32:31,058 Nikolay: Bye.