1 00:00:00,025 --> 00:00:03,731 Michael: Hello and welcome to Postgres FM, a weekly show about all things PostgreSQL. 2 00:00:04,031 --> 00:00:08,111 I'm Michael, founder of pgMustard, and this is my cohost Nikolay, founder of Postgres AI. 3 00:00:08,111 --> 00:00:09,581 Hey, Nikolay, what are we talking about today? 4 00:00:10,098 --> 00:00:10,878 Nikolay: Hi Michael. 5 00:00:10,878 --> 00:00:13,068 This is episode number 15, right? 6 00:00:13,816 --> 00:00:21,209 Michael: It is, and because we are geniuses are, we plan this really far ahead and we are we gonna be talking about Postgres 15? 7 00:00:21,469 --> 00:00:25,188 Nikolay: Because this week it's released . I think it's already released yesterday. 8 00:00:25,188 --> 00:00:25,398 Right. 9 00:00:25,840 --> 00:00:28,510 Michael: Yeah, exactly the way we, we record beforehand. 10 00:00:28,510 --> 00:00:32,740 So fingers cross the release went well and , if so, it came out yesterday. 11 00:00:32,978 --> 00:00:34,778 So thanks to everybody involved in that. 12 00:00:34,838 --> 00:00:37,486 And sorry in advance if, if anything went wrong. 13 00:00:38,068 --> 00:00:38,628 Nikolay: Right. 14 00:00:38,976 --> 00:00:46,092 and also I should congratulate you and myself because we didn't skip any weeks 15 weeks in a row. 15 00:00:46,092 --> 00:00:46,842 It's a big. 16 00:00:47,457 --> 00:00:48,147 Big achievement. 17 00:00:48,177 --> 00:00:48,447 Right. 18 00:00:48,447 --> 00:00:50,911 And Thank you all, for great feedback. 19 00:00:50,941 --> 00:00:51,301 Again. 20 00:00:51,301 --> 00:00:53,138 We received a very good feedback. 21 00:00:53,138 --> 00:00:53,618 Thank you. 22 00:00:54,171 --> 00:00:56,980 And requests also, we listen to requests. 23 00:00:57,160 --> 00:00:59,556 Requests exceed our capabilities. 24 00:00:59,756 --> 00:01:00,386 Definitely. 25 00:01:00,416 --> 00:01:02,186 But we will try to catch up. 26 00:01:02,566 --> 00:01:03,346 Please continue. 27 00:01:03,491 --> 00:01:04,541 Michael: Yeah, absolutely. 28 00:01:04,798 --> 00:01:08,425 I think it would've been easy to skip a week if we weren't getting lots of nice comments. 29 00:01:08,425 --> 00:01:09,295 So thank you everybody. 30 00:01:10,092 --> 00:01:10,662 Nikolay: Okay. 31 00:01:10,662 --> 00:01:11,412 August 15. 32 00:01:11,445 --> 00:01:12,765 What's your favorite feature? 33 00:01:13,278 --> 00:01:13,878 Michael: straight to it. 34 00:01:13,988 --> 00:01:22,022 you know, I'm a performance fan in general, but I have a few reasons for picking the performance improvements to sorts specifically. 35 00:01:22,132 --> 00:01:23,672 Nikolay: Uh, there are multiple ones. 36 00:01:23,926 --> 00:01:31,195 Michael: yeah, there are lots, and I, I know they are separate features, but when you consider them together as a group, I think 37 00:01:31,195 --> 00:01:39,724 they're so powerful, mostly because, Anybody who upgrades will benefit from them without having to change anything on their side. 38 00:01:40,129 --> 00:01:40,519 Nikolay: Right? 39 00:01:40,524 --> 00:01:41,719 And everyone does it. 40 00:01:41,769 --> 00:01:43,839 Any project has order by, right? 41 00:01:43,985 --> 00:01:44,705 Michael: exactly, 42 00:01:44,969 --> 00:01:45,659 Nikolay: I think so. 43 00:01:45,719 --> 00:01:47,939 99 point 99% have it 44 00:01:48,464 --> 00:01:48,944 Michael: Even. 45 00:01:49,004 --> 00:01:49,694 Yeah, exactly. 46 00:01:49,694 --> 00:01:52,094 And I think it's used in other cases as well, right? 47 00:01:52,094 --> 00:01:57,962 Like I see query plans with sorts in them that don't, you know, they might be other related to other operations as well. 48 00:01:58,487 --> 00:01:58,757 Nikolay: Right. 49 00:01:58,757 --> 00:02:04,607 The question is how much improvement can be not noticeable, but like, I, I don't know. 50 00:02:04,667 --> 00:02:11,951 I haven't seen details, haven't tried myself, but I, I do see many mentioning of whether by small improvements here and there, 51 00:02:11,951 --> 00:02:16,119 like with this indexes and other things I, I noticed it as well. 52 00:02:16,179 --> 00:02:18,559 Yeah, so Order buy was, 53 00:02:19,539 --> 00:02:25,747 Michael: Yeah, and there's a really good blog post by David Rowley, or rarely, I'm not sure, Sorry that I've definitely got that wrong. 54 00:02:25,807 --> 00:02:26,677 At least once. 55 00:02:26,752 --> 00:02:34,221 On the Microsoft blog that I can include obviously benchmarks are tricky, but has some benchmarks on each of them, and there's some decent wins in there. 56 00:02:34,329 --> 00:02:34,637 Nikolay: Oh. 57 00:02:34,768 --> 00:02:39,238 So a whole blog post only about uh, sorting improvements. 58 00:02:39,442 --> 00:02:40,693 Michael: in Postgres 15 yep. 59 00:02:41,352 --> 00:02:48,284 Nikolay: By the way, I don't like the word sorting, and this is official in SQL and p scale. 60 00:02:48,284 --> 00:02:51,654 Release notes use it as well, but sorting sometimes like. 61 00:02:51,979 --> 00:02:53,959 Normal people, not engineers. 62 00:02:53,959 --> 00:02:57,743 They, think about it like, okay, this goes here, this goes there, Right? 63 00:02:57,833 --> 00:03:00,563 Not, not the changing order, you know, this problem, right? 64 00:03:01,103 --> 00:03:02,243 But Ordering. 65 00:03:02,243 --> 00:03:05,681 is much better in, my head than sorting. 66 00:03:05,686 --> 00:03:09,161 But sorting, this is what we have in source code and everywhere and so on. 67 00:03:09,761 --> 00:03:14,021 So yeah, one of the features there, it's a bunch, bunch of improvements, right? 68 00:03:14,021 --> 00:03:21,700 And one of improvements I would like to notice, is Improvement of performance, of sorting or ordering when work MA has exceeded. 69 00:03:21,970 --> 00:03:23,590 This is interesting, right? 70 00:03:24,250 --> 00:03:30,595 If I'm not mistaken from Peter, it was his name was mentioned there, but I would like to test this one. 71 00:03:30,945 --> 00:03:36,975 I'm not sure how much it was improved, but it's definitely sounds interesting because workman sometimes is not enough. 72 00:03:37,245 --> 00:03:39,528 Like we have files and so on, and this is. 73 00:03:40,421 --> 00:03:45,447 Michael: Yeah, there were multiple, there was, yeah, there were improvements to on disc sorts, so that's what we're talking about here. 74 00:03:45,687 --> 00:03:51,957 There were improve improvements to in memory sorts, and there were improvements to the amount of memory 75 00:03:51,957 --> 00:03:55,167 needed for sorting, especially certain data types. 76 00:03:55,167 --> 00:03:59,397 So, Really common data types have had specific optimizations put in for them. 77 00:03:59,817 --> 00:04:07,512 And that's important because it means some sorts that previous, even if you don't change your work, me setting some sorts that previously 78 00:04:07,517 --> 00:04:10,632 would've spilled to disk will now be able to happen in memory. 79 00:04:10,812 --> 00:04:14,461 So that be an extra performance boost around that threshold. 80 00:04:14,791 --> 00:04:19,684 So yeah, so many improvements that hopefully in combination will will help people without them noticing. 81 00:04:19,747 --> 00:04:26,137 Not necessarily without noticing, but hopefully upgrading, like always, hopefully upgrading will give you a performance boost right out of the gate. 82 00:04:26,544 --> 00:04:26,874 Nikolay: Yeah. 83 00:04:26,934 --> 00:04:28,314 Worth testing definitely. 84 00:04:28,314 --> 00:04:28,854 And checking. 85 00:04:28,859 --> 00:04:29,364 Interesting, 86 00:04:29,654 --> 00:04:30,284 Michael: how about you then? 87 00:04:30,284 --> 00:04:32,082 What's your, your favorite feature? 88 00:04:32,095 --> 00:04:33,115 Nikolay: Very small one. 89 00:04:33,415 --> 00:04:38,297 A small feature prob by the way, also, Peter , as I remember, was involved there. 90 00:04:38,297 --> 00:04:40,004 It was collation control. 91 00:04:40,051 --> 00:04:46,505 Like Postgres will I don't know details obviously, but I know the problem very well when we upgrade operational. 92 00:04:47,018 --> 00:04:53,515 glibc gse silent, we talked about it in previous episodes might happen, right? 93 00:04:53,695 --> 00:05:03,325 And usually it's, it's usually happens if you upgrade, for example, from Pogo from Ubuntu 1804 to 2204, for example. 94 00:05:03,475 --> 00:05:08,005 And the question is, is a dangerous sub upgrade of glibc or it's not? 95 00:05:09,025 --> 00:05:10,221 And more often. 96 00:05:10,390 --> 00:05:13,703 Then I would like to, to have it, It's, it's quite dangerous. 97 00:05:14,021 --> 00:05:18,821 So we, you can have some indexes corrupted silently and nobody will tell you. 98 00:05:19,301 --> 00:05:22,301 So this is like a field of minds. 99 00:05:22,348 --> 00:05:24,860 you can step into it and uh, after great. 100 00:05:24,865 --> 00:05:26,420 Also you don't see any problems. 101 00:05:26,420 --> 00:05:29,630 But after a couple of days, your users started to complain. 102 00:05:29,630 --> 00:05:33,879 Some queries don't work as expected, and this is obvious sign of corruption. 103 00:05:33,884 --> 00:05:34,989 So you should test it. 104 00:05:35,081 --> 00:05:35,831 We can check. 105 00:05:36,128 --> 00:05:38,048 so now in Postgres 15, the tool. 106 00:05:38,610 --> 00:05:44,338 That actual version it's not what database expect, and it's controlled on database level, as I know. 107 00:05:44,608 --> 00:05:49,343 So , it's good at least to have immediate error or message. 108 00:05:49,794 --> 00:05:50,093 Michael: yeah. 109 00:05:50,213 --> 00:05:51,353 So is that a log message? 110 00:05:51,353 --> 00:05:52,403 How does it report it? 111 00:05:52,763 --> 00:05:53,873 Nikolay: I don't know actually. 112 00:05:54,053 --> 00:06:00,263 So , I just, I was, I had just saw that this problem was addressed, at least somehow, in my opinion, it should be solved. 113 00:06:00,268 --> 00:06:08,282 Like posi should care and posi should know which glibc version was used when table was created at the base was created, and now it's there. 114 00:06:08,312 --> 00:06:10,742 So it knows and it complains about difference. 115 00:06:11,222 --> 00:06:12,152 How it complains. 116 00:06:12,152 --> 00:06:13,442 I have no idea, unfortunately. 117 00:06:13,442 --> 00:06:13,772 Sorry. 118 00:06:14,582 --> 00:06:24,140 We, we will see unfortunately, In systems I deal with, we will see only in like three or so years, because like, it'll take time to upgrade big 119 00:06:24,140 --> 00:06:28,502 systems, but smaller systems, it's good that it'll be there very fast. 120 00:06:29,132 --> 00:06:33,997 And it's quite common when you, for example, copy operation. 121 00:06:33,997 --> 00:06:35,437 System upgrade is one of cases. 122 00:06:35,437 --> 00:06:40,177 So you can also, for example, take your PPG data and bring to different. 123 00:06:41,352 --> 00:06:45,012 and without noticing that that GIPSY version has changed. 124 00:06:45,012 --> 00:06:52,569 Or for example, you run pogin containers, p data was created using one Gipsy version, but in container, you, 125 00:06:52,569 --> 00:06:55,269 you have different gypsy version and also have problems. 126 00:06:55,269 --> 00:06:58,541 So finally, since POGS 15, we will have visibility. 127 00:06:58,712 --> 00:06:59,652 To this issue. 128 00:07:00,136 --> 00:07:01,756 And this is very important thing, I think. 129 00:07:01,786 --> 00:07:10,517 Like it's, it's like, it feels quite small, but it's so painful to not to have it great at posts 15, finally has it also merch. 130 00:07:10,817 --> 00:07:11,387 Of course. 131 00:07:11,417 --> 00:07:12,742 This is big, right? 132 00:07:13,222 --> 00:07:15,582 I, I have checked the history of merch. 133 00:07:15,654 --> 00:07:20,845 Can you guess when the talks about it started in pos, in the POS project. 134 00:07:21,610 --> 00:07:29,063 Michael: I can cheat because I saw a talk by Simon Riggs at Progress London, and I think he was involved from quite early on. 135 00:07:29,063 --> 00:07:37,753 But yeah, so it is a long, even though, even though I was told a few months ago, I'm, I'm still probably gonna under guess let's say six years ago, 136 00:07:38,683 --> 00:07:39,453 Nikolay: 2005. 137 00:07:40,813 --> 00:07:42,253 Michael: so yeah, 16 years ago. 138 00:07:42,793 --> 00:07:43,093 Nikolay: Yeah. 139 00:07:43,135 --> 00:07:44,095 16 or 17. 140 00:07:44,365 --> 00:07:44,875 Oh, roughly. 141 00:07:45,565 --> 00:07:45,955 Right. 142 00:07:45,955 --> 00:07:50,873 And it was reverted in 2018 in Pog, August 11 yeah, it it had issues. 143 00:07:51,203 --> 00:07:55,463 So Simon Rick committed it, and then he needed to revert it, unfortunately. 144 00:07:55,463 --> 00:07:59,593 And this was a big, like, disappointment moment for Postgres 11. 145 00:08:00,308 --> 00:08:01,718 I remember it quite well. 146 00:08:02,072 --> 00:08:05,067 so we lived with upsert but merch is much more power. 147 00:08:05,647 --> 00:08:07,177 it has like conditions. 148 00:08:07,182 --> 00:08:10,777 It has, it has ability to delete instead of just a update or insert. 149 00:08:10,927 --> 00:08:15,276 So it's like, and it's also Standard com compliant which is very important. 150 00:08:15,546 --> 00:08:24,546 And it also Oracle SQL Server Digital, like big digital, like, okay, two big databases, Oracle and SQL Server. 151 00:08:24,632 --> 00:08:25,532 They support it. 152 00:08:25,532 --> 00:08:36,027 So if you migrate from there, it's like one of very common points of pain , when you need to rewrite your queries now it'll be much more convenient 153 00:08:36,807 --> 00:08:37,760 Michael: Yeah, exactly. 154 00:08:37,760 --> 00:08:38,840 Less work to do. 155 00:08:38,840 --> 00:08:46,557 A big, like, everything we can do to make those migrations from things like Oracle, less work overall mean the balance of. 156 00:08:46,853 --> 00:08:49,344 Nikolay: benefit from kind from the project, right? 157 00:08:49,494 --> 00:08:53,065 Michael: Exactly the cost, benefit ratio keeps going in our favor. 158 00:08:53,073 --> 00:08:55,773 So yeah, thanks to everyone who's worked on that for so long. 159 00:08:56,045 --> 00:08:56,195 Nikolay: Yeah. 160 00:08:56,295 --> 00:08:58,625 I, I was surprised at it, like suddenly Okay. 161 00:08:58,625 --> 00:09:00,035 It, it's committed again. 162 00:09:00,665 --> 00:09:03,290 This time of obviously quality is, great. 163 00:09:03,290 --> 00:09:07,535 So is going to stay and it was a big surprise so many years, right? 164 00:09:07,575 --> 00:09:10,628 It's probably, the longest feature in development. 165 00:09:11,060 --> 00:09:11,620 Merge 166 00:09:11,955 --> 00:09:12,615 Michael: so yes. 167 00:09:12,675 --> 00:09:18,575 And then the, the other thing you mentioned there was SQL Standard complaints, and we've, we talked even quite recently, I think a couple of episodes 168 00:09:18,575 --> 00:09:22,145 ago on why we like how important that is for people choosing Postgres. 169 00:09:22,145 --> 00:09:23,405 So everything you 170 00:09:23,425 --> 00:09:30,672 Nikolay: mention the other standard compliant feature that probably is repeating the path of merge 171 00:09:30,762 --> 00:09:34,492 because it was reverted after first be or second be. 172 00:09:34,492 --> 00:09:35,452 I don't remember exactly. 173 00:09:36,402 --> 00:09:36,912 Right. 174 00:09:37,002 --> 00:09:39,642 SQL J part of SQL Standard. 175 00:09:39,731 --> 00:09:44,096 It was reverted both from 15 already after beta, right. 176 00:09:44,192 --> 00:09:50,567 and from 16 development branch, it was also reverted for some time until it polished. 177 00:09:50,576 --> 00:09:54,967 And it was probably the biggest disappointment of the 15 release. 178 00:09:55,941 --> 00:09:56,871 Michael: Yeah, absolutely. 179 00:09:56,876 --> 00:10:04,132 But when I read the, a lot of the things we've praised progress for recently are things like how how high the quality bar is 180 00:10:04,132 --> 00:10:06,952 and how strict the release process is and things like that. 181 00:10:06,952 --> 00:10:07,762 And it seemed like. 182 00:10:07,910 --> 00:10:15,460 There were really good reasons for not committing it, and that overall we are probably better off with it coming back at a later date in a better state 183 00:10:15,866 --> 00:10:16,256 Nikolay: Right. 184 00:10:16,382 --> 00:10:22,859 Some, some other feature was reverted, but I don't remember which one with smaller feature, but also was reverted in this release. 185 00:10:23,599 --> 00:10:27,899 So already after first better couple of revert actions happened. 186 00:10:28,324 --> 00:10:30,124 So interesting observation. 187 00:10:30,477 --> 00:10:32,767 Michael: it's good to see things happening in the beta phase though. 188 00:10:32,787 --> 00:10:38,040 It means people are trying it, people are, you know, looking at each other's patches and just making sure this 189 00:10:38,040 --> 00:10:40,710 is being held to the, the same standard across the board. 190 00:10:40,710 --> 00:10:47,609 I, I really appreciate it as somebody who mostly relies on Postgres is reliability and performance of course, but mostly reliability. 191 00:10:47,609 --> 00:10:48,809 I think a lot of the community. 192 00:10:49,454 --> 00:10:54,857 Is here because it just works and features that, you know, features that go in. 193 00:10:54,857 --> 00:10:56,867 And then you can end up with weird things. 194 00:10:56,867 --> 00:11:03,888 Like if you look at the data types j I know it's not, related, but data type Jason and then data type type Jason b we, we are forever 195 00:11:03,888 --> 00:11:08,305 having to tell people about Jason B because of Jason being done first. 196 00:11:08,352 --> 00:11:10,119 And I, I'm wondering like if, if. 197 00:11:10,779 --> 00:11:15,669 If that's the kind of thing that maybe wouldn't have happened in the current, current way of releasing things, but I'm not sure. 198 00:11:15,957 --> 00:11:19,317 Nikolay: Well, between them, there are a couple of years of development, so. 199 00:11:19,857 --> 00:11:21,087 Know, Right? 200 00:11:22,300 --> 00:11:22,463 Yeah. 201 00:11:23,183 --> 00:11:24,953 So what's next? 202 00:11:25,343 --> 00:11:26,663 What's the topic? 203 00:11:27,653 --> 00:11:28,343 to discuss? 204 00:11:29,033 --> 00:11:34,703 Michael: Well, we have a couple of like, there are a couple of other things listed in the top line features I'd be interested in your opinion on. 205 00:11:34,703 --> 00:11:39,948 There's some improvements to logical, some logical replication improvements and some. 206 00:11:40,038 --> 00:11:41,058 Nikolay: bunch of improvements. 207 00:11:41,063 --> 00:11:41,358 Right. 208 00:11:42,378 --> 00:11:46,015 Michael: That seems to be something that's getting better and better each major release. 209 00:11:46,100 --> 00:11:50,159 It's not something I use myself, so I haven't read them in detail, 210 00:11:50,459 --> 00:11:51,408 Nikolay: funny effect. 211 00:11:51,416 --> 00:11:55,676 I'm, I'm using it, but not intensively on big production systems 212 00:11:55,681 --> 00:11:58,336 yet because of issue with it. 213 00:11:58,336 --> 00:12:05,763 And I see observing improvements during last couple of years, like much more active improvements compared to several previous years. 214 00:12:06,091 --> 00:12:11,180 I'm excited about it because it feels like soon we will have much better logical replication. 215 00:12:11,180 --> 00:12:12,020 Much, much better. 216 00:12:12,080 --> 00:12:20,248 So, big systems, for example, those who generate more than one terabyte of wall data per day, or having like dozens of thousands of 217 00:12:20,248 --> 00:12:29,827 TPS size like terabytes or dozens of terabytes, the maintenance of logical replication will be not such painful as it it is right now. 218 00:12:30,150 --> 00:12:33,004 And I'm not going to describe all of the features. 219 00:12:33,244 --> 00:12:35,584 It's a bunch of good improvements and features. 220 00:12:35,944 --> 00:12:42,270 I will mention only couple of them, for example, you now, you, you'll be able to skip some actions from the stream of change. 221 00:12:42,777 --> 00:12:51,452 And because if somehow on recipient side, on, subscriber side for example, conflict occurs, unique key violation or lack 222 00:12:51,452 --> 00:12:57,526 of something because of foreign key violation or something, usually, it means that's it for your logical duplication. 223 00:12:57,526 --> 00:13:03,233 You need to start from scratch or, to fix somehow to, to get rid of unique key or something. 224 00:13:03,233 --> 00:13:07,463 It's, it's not good, but right now there, there will be ability to skip. 225 00:13:07,500 --> 00:13:15,962 Some record in this stream and you can continue and understand why this happened and fix it later but, the big goal number one, is to 226 00:13:15,962 --> 00:13:22,712 continue because if you have a lot of changes, you need to continue applying changes and losing just one changes is less problematic than. 227 00:13:22,770 --> 00:13:25,440 Being stuck and not apply changes at all. 228 00:13:25,920 --> 00:13:31,756 So this is a quite interesting feature and some other features are also related to performance and. 229 00:13:32,381 --> 00:13:37,077 and I would like to mention Thatit Cap, who participated in many of these improvements. 230 00:13:37,138 --> 00:13:41,666 He gave a talk in our Positive TV Open Talk series a few months ago. 231 00:13:41,666 --> 00:13:43,256 So go to Positive tv. 232 00:13:43,256 --> 00:13:46,016 It's a YouTube channel or you're already here, right? 233 00:13:46,016 --> 00:13:52,882 If you watch us with our faces, not only on podcast version so, and just listen to that talk. 234 00:13:53,092 --> 00:14:01,202 It was like from firsthand a lot of insight, ideas, thoughts, and observations, How both about Positive 15 and future versions as well. 235 00:14:01,532 --> 00:14:06,654 So this, this is better to listen from there instead of just listening to us here. 236 00:14:07,344 --> 00:14:07,884 Okay. 237 00:14:07,945 --> 00:14:09,385 Let's sit with logical. 238 00:14:09,385 --> 00:14:10,045 What's next? 239 00:14:10,050 --> 00:14:10,915 What do you think? 240 00:14:11,605 --> 00:14:13,135 Michael: We have a couple of other ones. 241 00:14:13,135 --> 00:14:15,107 There's more compression options. 242 00:14:15,117 --> 00:14:15,807 For example, 243 00:14:15,977 --> 00:14:16,377 Nikolay: Yeah. 244 00:14:16,467 --> 00:14:16,707 Michael: backup. 245 00:14:16,707 --> 00:14:17,187 Yeah. 246 00:14:17,497 --> 00:14:23,197 Nikolay: Yeah, PG Facebook is a way to create a, I call it thick clone, like regular clone. 247 00:14:23,197 --> 00:14:30,427 Like you copy, if you have terabyte, you will copy this terabyte to different place on the same disc or different disc, maybe different server. 248 00:14:30,937 --> 00:14:36,637 And of course, compression is good to have because we, we have a lot of CPU power in many cases. 249 00:14:37,207 --> 00:14:39,597 But this can network maybe. 250 00:14:39,641 --> 00:14:41,681 Worse bottleneck than cpu. 251 00:14:41,711 --> 00:14:47,722 So compressing everything and sending less is can be beneficial in terms of time, right? 252 00:14:48,022 --> 00:14:52,546 So I easily can see how we can win in many places in our daily operations. 253 00:14:52,546 --> 00:14:53,626 So, DBA operations. 254 00:14:54,016 --> 00:14:56,672 So this, I'm glad this appeared in post. 255 00:14:57,352 --> 00:14:58,012 Michael: Yeah, it's really cool. 256 00:14:58,072 --> 00:15:02,542 I think we've seen a few compression related features in the last couple of versions, so it feels like there's probably 257 00:15:02,542 --> 00:15:04,942 a few people pushing those, so thank you to them as well. 258 00:15:05,697 --> 00:15:05,997 Nikolay: Right. 259 00:15:05,997 --> 00:15:08,916 And also wall compression can be controlled. 260 00:15:08,921 --> 00:15:11,698 I mean, in wall there is no such thing as wall compression. 261 00:15:11,703 --> 00:15:18,458 There is you, if you enable wall compression, you basically enable, enable not everything, but only for full page, right. 262 00:15:18,920 --> 00:15:21,556 The wall is recorded in two types. 263 00:15:21,687 --> 00:15:28,535 if you change some row in a table, it's recorded as a change, but if full page rights are enabled and they're enabled by default 264 00:15:28,540 --> 00:15:30,855 and should be enabled to avoid corruption in many cases. 265 00:15:30,902 --> 00:15:35,886 First change after checkpoint is recorded as like full page eight kilobytes KB bytes. 266 00:15:36,178 --> 00:15:42,237 And if compression is not, You spend the eight KPIs, if it's enabled, you spend. 267 00:15:42,855 --> 00:15:46,005 And I recommend everyone to enable this whole compression. 268 00:15:46,055 --> 00:15:53,669 However, I saw in Twitter, somebody complained CPU usage, some, some queries degraded after enabling compression in my 269 00:15:53,699 --> 00:16:00,291 own experience dealing with large systems, heavily loaded, all compression was always beneficial and only benefits. 270 00:16:00,291 --> 00:16:00,621 We are. 271 00:16:01,362 --> 00:16:02,352 Significant benefit. 272 00:16:02,352 --> 00:16:05,467 So we, we write much less in wall. 273 00:16:05,827 --> 00:16:10,773 So now, as I remember, positive 15 will allow you to control the compression type. 274 00:16:10,795 --> 00:16:13,795 because usually it's like, it was quite lightweight compression. 275 00:16:13,800 --> 00:16:17,905 Maybe you want to compress more heavily so now you can tune it. 276 00:16:17,965 --> 00:16:19,255 Additionally, it's interesting. 277 00:16:19,341 --> 00:16:19,551 Yeah. 278 00:16:19,581 --> 00:16:20,929 So this, this is what means. 279 00:16:20,929 --> 00:16:26,307 Wall compression only full page race first, change in the page after checkpoint. 280 00:16:26,667 --> 00:16:33,387 Subsequent changes until next checkpoint will be recorded individually only the data, what was changed. 281 00:16:33,771 --> 00:16:34,761 Michael: Okay, I'm with you now. 282 00:16:34,821 --> 00:16:35,341 That makes sense. 283 00:16:35,856 --> 00:16:36,576 Nikolay: Right, Right. 284 00:16:36,606 --> 00:16:40,356 So like more fine tuning for for in hands of DBAs. 285 00:16:40,956 --> 00:16:41,436 This is good, 286 00:16:41,933 --> 00:16:42,263 Michael: Yeah. 287 00:16:42,563 --> 00:16:44,813 And the last one made the. 288 00:16:45,428 --> 00:16:47,978 Kind of a major features list, at least in the release. 289 00:16:48,218 --> 00:16:51,937 The draft release notes was Jason format for Lux, 290 00:16:52,922 --> 00:16:58,602 Nikolay: Oh, before, before we go there yeah I also, have you noticed this is what is interesting item. 291 00:16:58,662 --> 00:17:00,042 I'm looking at it right now. 292 00:17:00,132 --> 00:17:05,202 I had support for writing wall using direct IO on macOS from Thomas Munro. 293 00:17:05,496 --> 00:17:10,240 This is interesting, usually POS has no anything with direct io. 294 00:17:10,600 --> 00:17:13,427 Some systems have my SQL Oracle, they allowed. 295 00:17:13,997 --> 00:17:16,217 So now only on my course, it's. 296 00:17:16,817 --> 00:17:21,583 with additional conditions like Max VCE zero and, and world level is minimal. 297 00:17:21,583 --> 00:17:27,163 So this is, this looks like of experimental thing and only on Makos, which is funny, right? 298 00:17:27,163 --> 00:17:27,253 Who? 299 00:17:27,633 --> 00:17:28,873 Who runs production? 300 00:17:28,903 --> 00:17:29,833 Pogs on Makos. 301 00:17:31,648 --> 00:17:39,428 Michael: No, but then again, I guess sometimes people do be, Yeah, I guess sometimes people do look at performance things On their local machine. 302 00:17:39,668 --> 00:17:43,358 And that's an interesting case, maybe a problem. 303 00:17:43,358 --> 00:17:50,507 But the, the other thing that I've heard of is people wanting to test the, the processes, the MAC processes as, you 304 00:17:50,512 --> 00:17:54,631 know, potentially interesting to run Postgres workloads on. 305 00:17:55,261 --> 00:17:55,561 Yeah. 306 00:17:55,561 --> 00:18:03,556 Not in production yet, but if they, you know, the M one, M two processes, if they're really good, then I wonder like what kind of performance we could. 307 00:18:04,075 --> 00:18:04,555 Database 308 00:18:04,816 --> 00:18:10,649 Nikolay: Yeah, for, Well, it's interesting, like I, I would definitely spend some time benchmarking it and just 309 00:18:10,653 --> 00:18:12,999 to understand what kind of benefits we can have here. 310 00:18:12,999 --> 00:18:15,582 But my opinion, like this is experimental. 311 00:18:15,582 --> 00:18:16,362 Some small move. 312 00:18:16,782 --> 00:18:20,352 I didn't see discussions unfortunately in hackers about it. 313 00:18:20,352 --> 00:18:21,126 But I think. 314 00:18:21,781 --> 00:18:24,721 Something will happen in future, my gut tells me, right? 315 00:18:24,721 --> 00:18:30,607 Because not only on my course, on Linux as well in this area, and this is, this is interesting. 316 00:18:30,884 --> 00:18:33,988 So yeah, this is it about wall. 317 00:18:34,708 --> 00:18:40,308 Some, some improvements based back up also happened, some more control and so on. 318 00:18:40,308 --> 00:18:41,665 Like this is also good. 319 00:18:42,132 --> 00:18:45,286 And exclusive Bcca mold is killed. 320 00:18:45,346 --> 00:18:47,316 So no more exclusive mold. 321 00:18:47,473 --> 00:18:49,963 nobody was using it for several years already. 322 00:18:50,133 --> 00:18:56,239 it was default and I remember some confusion, but now just we forget some cleanup work happened here. 323 00:18:56,489 --> 00:18:57,949 I think that's it about backups. 324 00:18:58,099 --> 00:19:00,544 Let's talk about some develop developer stuff. 325 00:19:00,544 --> 00:19:01,214 What else? 326 00:19:01,490 --> 00:19:04,221 , we discussed, merge, We discussed revert at sql. 327 00:19:04,221 --> 00:19:09,236 Jason, we discussed some sorting or ordering optimizations with es. 328 00:19:09,303 --> 00:19:12,692 There is some work continued related to duplicates. 329 00:19:13,149 --> 00:19:21,189 Remember dation and the improvements in B3 in post 13 and I think in 12 and 14, definitely. 330 00:19:21,240 --> 00:19:26,938 from Peter Gagan and others, and now more cases are supported, right? 331 00:19:27,238 --> 00:19:31,365 So tables with toast, their indexes also have improvements. 332 00:19:31,965 --> 00:19:38,505 This will affect as as you, Yeah, as you said, this will affect everyone if you have a quite big table. 333 00:19:38,565 --> 00:19:44,518 So if table is toasted, meaning that you have records, rough, very roughly more than two kilobytes. 334 00:19:44,848 --> 00:19:45,058 So 335 00:19:45,198 --> 00:19:45,688 Michael: Yeah. 336 00:19:45,748 --> 00:19:49,363 Nikolay: example, Jason text, or okay. 337 00:19:50,233 --> 00:19:50,623 Right. 338 00:19:50,873 --> 00:19:55,943 Well, there are, sometimes we have small J values, but, but often we have quite large ones. 339 00:19:56,153 --> 00:20:03,629 And in this case, index, if this table is indexed with B three Now this benefits, previous releases, introduc. 340 00:20:03,676 --> 00:20:08,669 In terms of index size and how degradation happens when you update it. 341 00:20:08,697 --> 00:20:08,976 So. 342 00:20:09,023 --> 00:20:12,031 Below growth will slow down and so on. 343 00:20:12,058 --> 00:20:12,898 and this is good. 344 00:20:12,988 --> 00:20:16,743 So like, probably it's finalizing the work in this area. 345 00:20:16,743 --> 00:20:18,033 Maybe, I'm not sure. 346 00:20:18,213 --> 00:20:19,713 Maybe there is something else. 347 00:20:19,799 --> 00:20:22,122 But it's feels like what was done before. 348 00:20:22,122 --> 00:20:26,203 Now we have full coverage of case of cases also. 349 00:20:26,203 --> 00:20:28,873 Now remember this, like in unique case we can. 350 00:20:29,657 --> 00:20:30,827 all nus are the. 351 00:20:31,630 --> 00:20:42,109 Which is not what books teach us now should be dis like now equals now no nows are dis distinguished usually because now is unknown. 352 00:20:42,109 --> 00:20:47,065 So we, if we compare nows, usually the result was, no, they are not the same. 353 00:20:47,455 --> 00:20:50,215 But now the is ability to say they are all the same. 354 00:20:50,215 --> 00:20:54,775 And we have only one single knowledge in our universe in terms of unique keys. 355 00:20:54,865 --> 00:20:55,443 And Yeah. 356 00:20:55,443 --> 00:20:56,893 In some cases it's useful. 357 00:20:57,320 --> 00:21:01,083 Michael: There's a blog post by Ryan Lambert on RustProof Labs about that. 358 00:21:01,127 --> 00:21:02,597 I'd forgotten that was in 15. 359 00:21:02,987 --> 00:21:11,046 Just on the development front, I've just been looking at a few of Peter's other commits and I'd forgotten that hash_mem_multiplier has been increased. 360 00:21:11,061 --> 00:21:13,341 So this was, I think, introduced in 14. 361 00:21:13,551 --> 00:21:16,551 So anybody running 14 might be interested in this as well. 362 00:21:16,818 --> 00:21:25,752 It's a multiplier to work_mem that can let you raise the amount of memory available for hashes but without raising it for sorts. 363 00:21:25,752 --> 00:21:35,077 So you could say I want there to be 16 megabytes available for workmen, but I want there to be 32 or 64 available for uh, hashes. 364 00:21:35,227 --> 00:21:37,897 So you could set multiplier to two or four. 365 00:21:37,927 --> 00:21:38,557 Nikolay: Interesting. 366 00:21:38,557 --> 00:21:42,526 How to make this decision like how can, we can make the decision with numbers. 367 00:21:42,586 --> 00:21:45,436 We need some proper analysis before we do it. 368 00:21:45,436 --> 00:21:45,766 Right. 369 00:21:45,976 --> 00:21:46,756 This is interesting. 370 00:21:47,171 --> 00:21:49,541 Michael: It's super interesting that the, the default's being changed. 371 00:21:49,541 --> 00:21:55,031 I think that's a really, that's something that Postgres generally shys away from doing, and I'm really impressed that 372 00:21:55,886 --> 00:21:57,926 Nikolay: You know, my opinion about defaults, right? 373 00:21:58,291 --> 00:21:59,401 Michael: no, I don't think I do. 374 00:21:59,656 --> 00:22:04,066 Nikolay: my opinion, a lot of defaults are absolutely outdated. 375 00:22:04,156 --> 00:22:05,056 It should be changed. 376 00:22:05,429 --> 00:22:09,422 We should care about modern service or SSDs and so on. 377 00:22:09,842 --> 00:22:12,242 And in this context, I will pull us back. 378 00:22:12,276 --> 00:22:15,629 to operational side and log Check Point is on now. 379 00:22:16,105 --> 00:22:17,065 Log check points. 380 00:22:17,125 --> 00:22:19,252 It was off by default and it. 381 00:22:19,614 --> 00:22:23,958 Terrible state because checkpoint data you always wanted. 382 00:22:24,588 --> 00:22:24,918 Right? 383 00:22:25,098 --> 00:22:32,348 And this discussion was like, discussion was like, we don't want to generate a lot of logs because, you know, like sometimes we have small 384 00:22:32,353 --> 00:22:36,658 machine with small disk and we don't want to fill it with logging. 385 00:22:36,888 --> 00:22:41,939 But chip point data is so useful to understand what's happening for DBAs. 386 00:22:41,985 --> 00:22:42,405 it should be. 387 00:22:42,970 --> 00:22:44,140 And this is small win. 388 00:22:44,410 --> 00:22:45,460 I think this is good. 389 00:22:45,910 --> 00:22:52,865 I'm in the camp of, let's make defaults much more modern, up to date and on for checkpoints, definitely a win. 390 00:22:53,315 --> 00:23:02,273 The other one log mean auto vacuum duration was also changed to 10 minutes, and I think it's only partial when I would change it to. 391 00:23:02,284 --> 00:23:05,524 One second, or I, I personally use sometimes zero. 392 00:23:05,884 --> 00:23:09,544 Of course, s a lot of logs, but also useful for analysis. 393 00:23:10,174 --> 00:23:16,950 And even if uh, do Icom took half of second, you have interesting data to analyze you. 394 00:23:16,950 --> 00:23:18,840 You see a lot of interesting stuff. 395 00:23:18,870 --> 00:23:23,303 But of course in some, have a lot of the systems that will produce a lot of logging volumes. 396 00:23:23,394 --> 00:23:28,782 Michael: Yeah, the, the line in the release notes for this is fascinating and I think Shows that maybe you have a slight 397 00:23:28,782 --> 00:23:31,602 difference of ideology with whoever's making these decisions. 398 00:23:31,602 --> 00:23:37,782 Cause it, it says this will cause even an idle server to generate some log output, which might cause problems on 399 00:23:37,782 --> 00:23:40,302 resource constrainted servers without log file rotation. 400 00:23:40,602 --> 00:23:49,266 So the question is, why, why are we optimizing for idle servers without log file rotation versus a lot of people running those things. 401 00:23:49,761 --> 00:23:55,532 Nikolay: Yeah, this is, I think this note is be, is about logic points because we have Check point time out and they happen. 402 00:23:55,792 --> 00:23:58,875 By default, five minutes or also not good default as well. 403 00:23:58,875 --> 00:23:59,835 But anyway. 404 00:24:00,075 --> 00:24:09,435 And I think it's, they just want to avoid the situation that when you install pores, go the, the usual I've been in about pogo from like wide audience. 405 00:24:09,506 --> 00:24:12,963 from long, long ago, it's hard to start. 406 00:24:12,968 --> 00:24:16,413 But once you install it, it's just working and you simply forget about it. 407 00:24:16,413 --> 00:24:16,743 Right. 408 00:24:16,913 --> 00:24:21,883 And Of course, it's, it makes sense to think about it's not good if you installed it and it stopped, 409 00:24:22,005 --> 00:24:25,125 working after a year or two suddenly because of logs. 410 00:24:25,155 --> 00:24:26,715 It may happen, of course, yes. 411 00:24:27,255 --> 00:24:30,498 But log rotation is, it should be enabled, but find a fact. 412 00:24:30,498 --> 00:24:35,122 A couple of days ago we had a system where log rotation, quite important system in our. 413 00:24:35,204 --> 00:24:40,864 Infrastructure where ation was disabled and we had the zero disc space, free disc space. 414 00:24:40,864 --> 00:24:44,278 So it happens, but still log check point on. 415 00:24:44,308 --> 00:24:50,968 Michael: One thing on that note is that I guess the world is changing quite a lot and, and a lot of people, a lot of the time these defaults 416 00:24:50,968 --> 00:24:58,558 don't matter as much because more and more, like, I think I saw a survey not that long ago that suggested it might even be close to 50% 417 00:24:58,563 --> 00:25:02,786 Now of instances are running on managed services on care provider. 418 00:25:02,841 --> 00:25:04,511 Nikolay: their own set of defaults. 419 00:25:04,781 --> 00:25:05,831 Michael: And they can change that. 420 00:25:05,831 --> 00:25:10,681 They can set the defaults for you and they can do these things like logging like log rotation for you. 421 00:25:10,801 --> 00:25:11,551 Exactly. 422 00:25:11,851 --> 00:25:13,501 So I think some of these things, 423 00:25:13,711 --> 00:25:15,601 Nikolay: but there this problem is solved already. 424 00:25:15,601 --> 00:25:22,291 I like, I think, I'm not sure, but I, I'm almost sure that on RRGs luxury points are wrong by default for log. 425 00:25:22,321 --> 00:25:25,559 Maybe I'm wrong, maybe I'm wrong, but it should be solve, This is 426 00:25:25,559 --> 00:25:26,639 very important formation. 427 00:25:26,662 --> 00:25:26,872 Michael: Yeah. 428 00:25:27,012 --> 00:25:30,143 while we're on this topic, I think the other one that makes sense. 429 00:25:30,148 --> 00:25:36,156 You said for modern systems, the first one that I thought of was random page cost, and that's still being. 430 00:25:37,046 --> 00:25:39,656 Nikolay: random patch cost should be very close to sec patch cost. 431 00:25:39,661 --> 00:25:41,186 If you use ssd, definitely. 432 00:25:41,186 --> 00:25:46,946 Or if your database is below your ram, also different like then. 433 00:25:47,006 --> 00:25:47,276 Yeah. 434 00:25:47,306 --> 00:25:53,006 So because it means that if, if, if you are in fully cashed state, it means that sequential random, doesn't matter. 435 00:25:53,396 --> 00:25:54,326 Like similar. 436 00:25:54,386 --> 00:25:57,776 So they should, the cost should, should be close or equal. 437 00:25:58,496 --> 00:26:00,112 But speak back to logging. 438 00:26:00,472 --> 00:26:07,455 Many things can be, should be changed, reconsidered for example uh, log, um, log logging. 439 00:26:08,010 --> 00:26:12,870 You also should have it log bits off by default or, and many things. 440 00:26:13,020 --> 00:26:20,297 Some, some people for example enable logging of connection disconnections as well, but it can spam your logs, definitely. 441 00:26:21,046 --> 00:26:21,436 so. 442 00:26:21,586 --> 00:26:27,046 Michael: But yeah, based on, based on this philosophy, an idle server wouldn't generate loads of logs for either of those. 443 00:26:27,046 --> 00:26:31,246 So I could see, I could see people being more open to that than maybe some of the other ones. 444 00:26:32,086 --> 00:26:33,046 Anyway, it seems like pro 445 00:26:33,256 --> 00:26:33,796 Nikolay: Depends. 446 00:26:34,966 --> 00:26:35,476 Michael: even if 447 00:26:35,506 --> 00:26:36,046 Nikolay: yes. 448 00:26:36,166 --> 00:26:36,796 Michael: we wanted. 449 00:26:36,796 --> 00:26:38,296 It's so much progress. 450 00:26:38,446 --> 00:26:43,241 Nikolay: I hope this the consideration of defaults will continue in the right direction, 451 00:26:44,021 --> 00:26:44,291 Michael: Yeah. 452 00:26:44,441 --> 00:26:47,111 Nikolay: but 10 minutes for auto Icom is not enough. 453 00:26:47,681 --> 00:26:47,891 What? 454 00:26:47,951 --> 00:26:48,561 10 minutes. 455 00:26:49,781 --> 00:26:50,111 Okay. 456 00:26:50,711 --> 00:26:56,081 Michael: The other thing I'm looking forward to this year is seeing how fast each of the cloud providers release new versions. 457 00:26:56,086 --> 00:26:58,444 I think we've seen some some of the newer ones. 458 00:26:58,540 --> 00:27:00,460 Some of them were very fast last year. 459 00:27:01,122 --> 00:27:01,992 Yeah, exactly. 460 00:27:01,992 --> 00:27:06,612 So I think Crunchy Bridge were pretty much the same day, if not a day or two after the release. 461 00:27:06,612 --> 00:27:09,252 Last year they had a version of 14 available. 462 00:27:09,402 --> 00:27:09,762 Nikolay: Mm-hmm. 463 00:27:09,787 --> 00:27:15,114 Michael: Microsoft, one of their services was, I think rds still take a little bit of time but 464 00:27:15,254 --> 00:27:15,614 Nikolay: it's a, 465 00:27:15,824 --> 00:27:16,394 Michael: relatively 466 00:27:16,784 --> 00:27:25,669 Nikolay: I, I saw some science of improvement in terms of speed, but yeah, also Google Interesting Google Cloud which started to improve in terms of. 467 00:27:26,071 --> 00:27:33,436 Pogs suddenly because a few years ago, I, I was, I was saying don't use Google Cloud there are managed pogs right now. 468 00:27:33,586 --> 00:27:34,396 Things are changing. 469 00:27:34,396 --> 00:27:35,506 They have interesting things. 470 00:27:35,986 --> 00:27:38,656 Again, we had, we had the guest on pogs tv. 471 00:27:39,568 --> 00:27:39,918 Ha. 472 00:27:39,923 --> 00:27:40,322 Crossing. 473 00:27:40,343 --> 00:27:44,059 former Skype pogo architect, and it was great. 474 00:27:44,059 --> 00:27:44,689 Talk about 475 00:27:45,702 --> 00:27:46,662 Michael: Yeah, it really was. 476 00:27:47,142 --> 00:27:47,352 Yeah. 477 00:27:47,352 --> 00:27:47,832 And we're, 478 00:27:48,121 --> 00:27:53,512 I'm guilty of being a Google Cloud Postgres care user as well, and they, they are doing some really cool things. 479 00:27:53,532 --> 00:27:56,082 They did release 14 quite quickly as well, 480 00:27:56,122 --> 00:27:56,732 Nikolay: Mm-hmm. 481 00:27:57,102 --> 00:27:58,152 .so things are improving. 482 00:27:58,152 --> 00:27:58,752 I think it's competi. 483 00:27:59,719 --> 00:27:59,929 Michael: Yeah, 484 00:27:59,929 --> 00:28:00,709 absolutely. 485 00:28:01,188 --> 00:28:02,988 And people wanting it, which is good news, 486 00:28:03,411 --> 00:28:05,061 people, users asking for it. 487 00:28:05,634 --> 00:28:14,737 So yeah, encourage everybody to, to check out 15 as and when they can and to keep upgrading, especially I guess anybody on version 10 or 488 00:28:14,742 --> 00:28:19,207 before should definitely be thinking about upgrading as soon as possible. 489 00:28:19,237 --> 00:28:19,927 Cause that will stop 490 00:28:19,972 --> 00:28:23,122 Nikolay: Oh, 10 is 10 is 10 is, That's it. 491 00:28:23,417 --> 00:28:24,077 For 11. 492 00:28:24,077 --> 00:28:25,097 One year left. 493 00:28:25,296 --> 00:28:25,656 Michael: Yeah. 494 00:28:25,776 --> 00:28:30,751 And even, regardless of security patches and things, just the number of performance improvements you 495 00:28:30,751 --> 00:28:33,661 could get by upgrading is is worth checking out them. 496 00:28:34,231 --> 00:28:34,921 Yeah, exactly. 497 00:28:34,921 --> 00:28:37,081 As well as all of these features we've been talking about. 498 00:28:37,347 --> 00:28:37,857 Nikolay: Great. 499 00:28:38,127 --> 00:28:42,437 Some partitioning improvements we haven't mentioned some other things like for, from data. 500 00:28:43,357 --> 00:28:43,847 Okay. 501 00:28:44,057 --> 00:28:47,447 A lot of, a lot of more improvements are there, so it's good. 502 00:28:47,447 --> 00:28:47,507 It. 503 00:28:48,399 --> 00:28:49,539 Michael: So many hundreds. 504 00:28:49,569 --> 00:28:53,529 In fact, actually, another website that I will link up is a good one 505 00:28:53,574 --> 00:28:54,564 Nikolay: upgrade d 506 00:28:54,999 --> 00:28:55,659 Michael: Yes. 507 00:28:55,854 --> 00:28:56,214 Nikolay: com? 508 00:28:56,514 --> 00:28:57,144 Why upgrade? 509 00:28:57,174 --> 00:28:57,414 Yeah. 510 00:28:57,834 --> 00:29:01,699 I, I always use it to show people what they are missing. 511 00:29:02,109 --> 00:29:04,269 Michael: Yeah, it's really good at showing security patches. 512 00:29:04,269 --> 00:29:04,989 You're missing in 513 00:29:05,714 --> 00:29:06,069 isn't it? 514 00:29:06,069 --> 00:29:06,489 And 515 00:29:06,924 --> 00:29:07,044 Nikolay: red. 516 00:29:07,114 --> 00:29:07,514 Red. 517 00:29:07,719 --> 00:29:11,564 Michael: yeah, and additionally, it's, it's got a nice search feature. 518 00:29:11,569 --> 00:29:17,924 So if you're wondering about any changes in the last few major versions to a feature you really care about, you can search for that 519 00:29:17,960 --> 00:29:21,260 feature by name and see all of the commit messages related to that. 520 00:29:21,260 --> 00:29:21,801 Nikolay: Mm-hmm. 521 00:29:21,881 --> 00:29:22,241 . Mm-hmm. 522 00:29:22,316 --> 00:29:22,736 Michael: It's cool. 523 00:29:22,936 --> 00:29:23,326 Nikolay: Great. 524 00:29:23,420 --> 00:29:23,885 Michael: Wonderful. 525 00:29:23,945 --> 00:29:25,036 Any last things? 526 00:29:25,112 --> 00:29:28,592 Nikolay: Yeah, usually reminder to subscribe, like, and provide more feedback. 527 00:29:28,592 --> 00:29:29,102 We like it. 528 00:29:29,102 --> 00:29:29,522 Thank you. 529 00:29:29,527 --> 00:29:29,972 Thank you 530 00:29:30,330 --> 00:29:31,170 Michael: Yeah, thanks. 531 00:29:31,175 --> 00:29:33,538 It keeps us going and we really appreciate it. 532 00:29:33,808 --> 00:29:34,318 So yeah. 533 00:29:34,468 --> 00:29:37,138 Have a good one, people, and see you next week. 534 00:29:37,530 --> 00:29:37,830 Nikolay: Bye