[CLAIRE] Welcome to Talking Postgres. In this podcast, we explore the human side of Postgres, databases
and open source, targeted at the Postgres and application developer communities. Thank you to
the team at Microsoft for sponsoring this community conversation about Postgres. I am the host,
Claire Giordano, and today's topic is helping Rails developers learn Postgres. And our guest
is Andrew Atkinson. So I'm super excited to have Andrew on the show. He is a software developer,
a graduate of University of Minnesota, has worked with Ruby on Rails and Postgres at companies like
Groupon, Microsoft, and a few startups too. He started his own consultancy called Refined Pages
and recently wrote a book that I am holding in my hands right now. The title is High Performance
Postgres for Rails, and you can find out more about it at andyatkinson.com/pgrailsbook,
all one word. So now Andrew gets to add published author to his list of accomplishments. Welcome, Andy.
[ANDREW] Thanks, Claire. Excited to be here. Yeah, thanks for that great intro too.
[CLAIRE] Well, it was easy to put it together. You've done a lot. All right, so let's dive in. Some of our
topics are typically like how I got started as a developer, how I got started in Postgres. But
because of the focus of your book, you and I agreed to really focus this episode on Rails
developers and helping Rails developers learn Postgres. But the book is really,
I don't know, my inspiration for inviting you on the show. I was so excited to see it and to hear
that you were writing it and the reviews have been really, really good. But what I want to do is go
back, back in time and figure out what inspired you. Why did you write a book about this topic?
[ANDREW] Yeah, thanks. Yeah, it's a complicated answer, I guess, because it's a lot of different factors.
But I'd say that I guess one of the big opportunities was I felt like with the
success of a lot of Ruby on Rails apps that are using Postgres as their primary application
database, Ruby on Rails famously or infamously has a built-in object relational mapper that works
great for all sorts of queries, you know, maybe 80% of the kind of queries that an application
might write. But there's an additional set of queries that are going to be challenging to do
with an ORM and developers likely would benefit from knowing how to write, how to identify those
opportunities, whether they're more performance sensitive areas of the application or they're
analytical style queries or that sort of thing. And so developers, I think, have this opportunity
to use SQL directly and write queries. So that's kind of one thing. And I felt like there's not a
lot of, there's kind of a great educational opportunity within the Rails developer world to
learn to better leverage SQL. So that kind of takes us to like any SQL database though, I mean,
whether you're using Postgres or something else. But then what I found in working with Postgres for
lots of years with different Rails apps was it's a really powerful platform that's both
wide and deep in terms of its capabilities. And there's just all sorts of things that are
not even really visible or possibly on the radar of a typical Ruby on Rails developer.
And I kind of define that by myself, like prior to me learning these characteristics,
but also having worked with lots of teams and probably hundreds of different Rails developers
across teams and companies. So I felt like there are all these capabilities, there's opportunities
to use SQL, but then to sort of like focus on one particular database that is so broad and deep and
also extensible, it kind of got me fired up the opportunity to help developers learn to read,
learn to write SQL, learn to read query execution plans, how to put indexes in place to help their
performance, and then all sorts of additional capabilities we can get into. So I'd say that
was one of the main drivers was I felt like there was a real need and opportunity. And
and then another thing I guess was I just felt like I had a great perspective, like I identified
with the learner possibly more than the Postgres expert, where they might want to know those things
and they might know there are things that Postgres offers that they're not familiar with,
they might not have opportunities to kind of practice using them and develop familiarity and
competency with those capabilities. And so the book is a way to do that too. So what we've done
in the book is we have a lot of, ideally there are examples for every section, but some sections even
go beyond that with exercises. So readers can learn what these things are and really put them
to use hands-on with their local development environment and a local Postgres instance,
so they can kind of learn to use them and then bring those skills into their jobs.
And I guess last thing I'll say on that is I had a great opportunity to use Postgres at a higher
scale where I think you start to, you know, it's common to see more challenges at a higher scale.
And by that I mean, you know, more data. So we're talking into the hundreds of millions of rows or
billions of rows and then higher query volume. We're talking like maybe greater than a thousand
queries per second or transactions per second. At that volume, I think what happens is Postgres
just becomes a little bit more demanding in terms of what's required by the operator to
use it well, like operationally. And a lot of folks I think aspire to have those skills as well,
to know how to use Postgres at higher scale. So I felt like that was the, I had some of that
experience from a past employer and I felt like I could bring that into the book as well. So I could
help show people how to, you know, both learn about new capabilities they might not be familiar
with, but also feel comfortable expanding as their application platform grows and becomes
more successful and their demands, the demands on their database increase.
[CLAIRE] So you talked about the opportunity to help Rails developers better understand
SQL, better understand the underlying database and that you identified with these learners,
right? These Rails developers, and then your experience with these data-intensive,
scalable applications or applications that require massive scalability. But I'm fishing for like,
what inspired you? Because there's probably hundreds, if not thousands of developers in
the world who recognize that opportunity or maybe identified with the learner,
but they didn't go write a book, right? Not everybody will then, like you can see an
opportunity and observe it, but that doesn't mean you're going to do anything about it.
So I guess I'm fishing for, were you standing by the barbecue with your wife and kids and
talking to friends? And then somebody said, well, you should write a book about that. Or,
was there a moment? Why? I mean, does everybody in your family write books or
did you know this from the age of seven? Why did you do this?
[ANDREW] Yeah. I don't really tend to think about it too much, I guess. I just,
but I would say that I can answer some of those latter questions first.
[CLAIRE] Was there a barbecue involved?
[ANDREW] There was no barbecue. I'd say there was one main person who I've credited in the book as well,
Jonathan Genick, for kind of making this possibility even known to me. Because to be
honest with you, I never really thought about writing a book prior to this. And I want to
just lead into that for a moment. But I'd say that for me, no one in my family that I'm aware of
has authored any books. And yeah, I don't really know. I've since come to learn authors and learned
a lot about technical books and through this process, as you might imagine. But I was pretty
naive. I didn't really know much about it and inexperienced with books in general and
publishers and that sort of thing prior to this experience. I guess what I have always done though
is I've always written a blog. And even going back to the 90s, I was thinking about,
as far as my developer origin story, it's not really application code, but I was writing HTML,
you know, creating HTML websites in the early days. And there was something, I guess, really
fundamental maybe about like putting things out into the internet and connecting with people.
That way that's appealed to me even back to those days. And so I remember creating like a
personal web page sometime in the 90s with like HTML for dummies or something like that.
But then I'd say like it took on a little more structure in the 2000s, like writing a tech blog.
And no one really read it. It was really just for me to like, but I kind of have learned about
myself since then that the way I attack topics I want to learn about is I write about them.
So it helps me clarify, you know, kind of clarify what the takeaways are from
something I'm learning about, put it into some sort of structure and that kind of thing.
So I think that the blogging was one of the ways that I caught the attention of
an acquisitions editor at a different publisher that I ended up not working with initially,
or eventually, but initially they reached out to me. So the blogging was part of it. And then I
think the other thing was I presented at Postgres Conference New York City in 2021,
which was my first ever Postgres event of any kind, really. And I took it.
[CLAIRE] OK, so that was just after COVID.
[ANDREW] Yeah, just after COVID.
[CLAIRE] December 2021, if I remember correctly. And that was year one of PGConf New York City,
which is now in its fourth year or something like that. They've already announced the dates for next
year. In fact, that's the one you're talking about, right?
[ANDREW] That's the one. Yeah. Yeah. I had a great time and I recommend that conference to anyone that's
interested in it. I've been back once since I wasn't able to go this year, but
I've been back as an attendee as well, for the record. But that particular time was
I had been using Postgres a ton at work and organized what we had done into initially
an internal presentation. It's a slide deck of 20 or 30 slides to give to management and other
engineering leaders and that sort of thing. Here's a bunch of things we did with Postgres primarily.
And the reason that we did that was we were having a lot of scale and reliability challenges. And
I can get into that more, but we were on, we were kind of what I was describing before,
where we were using Postgres as our primary application database for our Rails backend.
And we had gotten to a point where kind of normal day-to-day things were becoming slow or
problematic, or there were more errors and it just became more demanding of the operator.
And then we'd also vertically scaled the database server to what at the time,
the highest instance available. And so there was a lot of, there was some amount of
panic because we knew that re-architecting or even just some of the performance work that we
did end up doing was going to be demanding, but was going to be necessary for us to continue to
serve the traffic that we were serving at that time. So that was really, I mentioned the blogging.
So that experience, that hands-on experience that I had at the company, I think was another
critical part of me feeling confident enough to try to attack a book like this and feeling like
I had somewhat of a story to tell. And the book itself isn't that story exactly, but it's,
all of the exercises have a connection to my real world experience with whether it's, you know,
kind of the basics of understanding the most costly queries and then optimizing the indexes,
whether it means adding or removing them around those demanding queries, or whether it's using
something like table partitioning and when to use that and what the benefits are. They all relate to
like a real work experience. So I gave that presentation internally and I was kind of,
at the time I was really into the idea of, or at the time I was thinking a lot about,
I'd been working as a software developer for a decade plus at that point. And I was thinking
about kind of voting with my time about how I could work on things that had a little bit
bigger impact even beyond the team or the company I was working for. And I was getting, that was
part of what attracted me to Postgres I would say is, it's, you know, it's been around for decades,
it's used by loads and loads of companies. I felt really good about the quality of the
project, the way it's run and the governance of the project. And I felt like for myself,
it was a great investment of my energy and skills. Like it, I kept being interested in
learning more and more. I bought a Postgres books myself and was found that I enjoyed learning it.
And it was a bit of a surprise. I had the opportunity to apply a lot
of what I was learning, which for me made it more engaging. But I was learning, I was enjoying
learning about it. So I had this idea, what if I apply, what if I submit a proposal to
a Postgres conference? And it was PGConf NYC because it lined up timing wise with
when I was looking to do that. It was in the fall and it was after a bunch of work we had done.
And yeah, I was very fortunate to get that talk accepted. And then I was just kind of prepared
like, you know, the way that I deal with anxiety or worry around preparing for a conference
talk is to just excessively prepare. So I just practiced it a ton and tried to send
it around and get some early feedback and give some early versions and things like that.
[CLAIRE] Well, that's a smart tactic. I mean, anybody up on stage that you
see that makes it look effortless and you know, that it's easy to be envious of, they probably
practiced it a ton to get there.
[ANDREW] Yeah. That's a great point. And I've,
I've developed that mindset too, although sometimes it's not obvious because it's,
I guess it's tempting as a human to just think, Oh, that person, they're just so skilled.
Like I don't have those skills or something. And there is of course, differences in presenter skill,
but there's a lot to be said for preparation. And I know that I try to acknowledge
when I see a great presentation that I remind myself like this person prepared a lot, you know,
it's not just some natural skill that they have granted. They may still have skill, et cetera.
I'm trying not to diminish skill, but but they, they often have prepared a ton and, you know,
that takes a lot of work and it's probably coming from a place of them being proud of their content
and wanting to engage with people and that kind of thing. So, yeah, I was, I've been fortunate to,
you know, another side benefit of that is trying to circulate a presentation like that around and
get feedback is it's a great way to network a bit within the community and meet more people that
might attend the conference or might be interested in attending a future one that's similar or that
sort of thing. So I really appreciated people that helped me out on the preparation side.
So I'll just wrap, I was just going to wrap up that. I realize I've been going on
a lot about that, to wrap up the, if you'd like, I could tie that into the kind of the single moment
thing. I realized I've never really addressed that yet.
[CLAIRE] Let's go. Okay.
[ANDREW] So the PGConf NYC talk
went well. Shortly after that I was, I got an email from a different book publisher and they
said, "Hey, I see you blog about Postgres. I see that you just presented at PGConf NYC. Have you
ever thought about writing a book about Postgres? We're trying to publish more books in that area.
It's a growing, you know, the popularity of Postgres is growing. Our catalog is limited.
We want more books. And that was kind of like a "huh, I never really thought about it
before." And I know people say it's very demanding in terms
of time and it's not particularly lucrative. And I kind of just started researching and
all these things were swirling around, but going back to everything I said, hopefully that
whole thing was, people can sort of connect the dots. It felt like, I started
to think about it and I just kept getting more and more excited about it myself. You know, I felt like
I can bring in my experience. I can identify with the reader. I like to write. I like the
challenge of this whole thing. Like it's one of the things I learned in writing and I'll just stop
there is it like gave me permission in a sense to just read every word of a particular documentation
page that I wouldn't really do otherwise at my full-time job. I'd be trying to just sort of like
find a solution quickly, fix the issue and move on where this, I felt like I, that would
miss the mark totally with a book. Like I need to be very comprehensive and I owe it to the reader
to not miss things, you know? So I was, I kind of liked that challenge in a way, you know, like a
kind of raised the pressure a bit, but like in a positive way, I think that was productive.
I still wanted to synthesize information and provide something useful to the reader, but make
sure that I turned over every stone, so to speak.
[CLAIRE] You know, this is a random segue I'm going
to take you on, but you're making me think about this book called Swimming to Antarctica. Have you
by any chance read it? No. Heard of it? Okay. I should look up what the name of the author is,
but basically there's a woman named Lynne Cox who ended up swimming, like swimming the Bering
Strait, swimming into Antarctica, swimming, being the youngest person to swim the English Channel.
I think when she was like 15 years old or something like that. So she accomplished a
lot of amazing things swimming in these cold water, open water situations. But the way it
all started is she was in a pool, right? On a swim team, like with all her siblings and family in
Southern California or something like that. And there was a storm and everybody else was inside
watching like film of other swimmers because, you know, it was a pretty serious training regime.
And she didn't notice, she was still in the pool, even though there was like this big storm
and the water's kind of sloshing all over the place. And this parent had pulled into the parking
lot, saw her swimming in this storm and said to her later, wow, you know, you should
swim the English Channel someday. And they planted the seed of the idea. So then she went to her
uncle who picked her up that day and was like, Hey, yeah, so-and-so thinks I should swim the
English Channel. "Oh, honey, yeah, you could probably do that." And she told her parents
and they're like, "Oh, yeah, you could probably do that." But it's really fascinating how
sometimes somebody just plants the seed of an idea. And then it takes on a life of its own.
And it sounds like that's kind of sort of what happened with you in this book.
It wasn't the plan. It wasn't the expectation. You didn't even have it in your vision of your future
in your career. And then someone made that suggestion. And now you're a published author.
[ANDREW] Yeah, thank you. Yeah, that makes sense to me. I mean, I think as a parent, I think about that for
my kids to like trying to think about, you know, notice what their interests are and where their
talents might be and just suggest ideas like that. I haven't suggested swimming the English Channel
yet. We haven't been to England, but maybe maybe someday. But maybe for us, it'd be like swimming
across the Mississippi River or something. I live near there. But yeah, it's it's a great point to
try to. And I'd like to think the book can maybe reach some people, even if I never have any direct
interaction with a reader in that way. You know, like they might pick up the book because they're
like, we use Postgres at work and there's some things that are broken and nobody on my team
knows about this. And like, this is something I've seen, I'd say, is, you know, most, a lot of
Rails teams typically don't have database administrators, or I would say even often
people that are experienced with, you know, operating a database, I guess, like beyond
the ORM. And the other thing is, of course, like out in the market, there's lots of different types
of databases. There's NoSQL databases like MongoDB or something. There's, I think there's
sometimes developers have a mindset of, you know, having like database agnosticism and not
leveraging features. And so there's a little bit of kind of like trying to influence
changing that mindset a bit in the book. But yeah, I mean, hopefully, I think early in the book,
you know, there is more of an inspirational emphasis, you know, trying to show like Postgres
is gaining in popularity. This is a practical investment of your time for your career.
Here's why we think why, and that kind of thing. But yeah, hopefully,
there's kind of like someone, something roughly similar to that idea of planting a seed, you know,
that would be great to hear a story like that. I've had a couple readers tell me like, Oh, hey,
I learned something and I put it into use. And it's, you know, that the change I made is in
production now. And that's, that's awesome to hear.
[CLAIRE] So there is a, in the beginning of your book,
you have a couple of quotes, validations, if you will, from people who presumably read early copies
of the book before it went to publication. And there's one from Elizabeth Christensen that says,
"I love that this book lives in two worlds, Rails and Postgres, which are tightly coupled together
when running a production application, but are typically separated in terms of knowledge
communities." So and I think that is kind of what makes your book special. I don't know of any other
Postgres books that are targeted single mindedly at Rails developers, or Rails books that are
targeted only on Postgres. Maybe there are, and I'm just only aware of yours. But, but it made me
wonder, like, is this book for Rails developers? Or is it for Postgres users more generally?
[ANDREW] I'd like to think it's for both. It's primarily written for Rails developers, although
if you look at the, if you breeze through the table of contents, you'll see that
a lot of the chapters somewhat stand on their own as Postgres chapters. The content about Postgres,
it's just that I almost think, you know, we unfortunately we didn't have time to re-title
the book, but I think a better title might have been High Performance Postgres. And then something
like with Ruby on Rails or using Ruby on Rails.
[CLAIRE] And then you could do another one for Python and Django?
[ANDREW] Well, let's, let's talk about that offline.
[CLAIRE] I didn't mean to plant a seed or anything.
[ANDREW] No, I mean, I actually am sort of interested in that. I mean, I've, I recently worked with a
team where we use Django and Python and, you know, it's a great framework. There's a lot
of similarities to Ruby on Rails. Claire, I know you know this because I, what I'm about to say,
because you were involved in the conference as well. But earlier this year, I presented at
the POSETTE conference, the online Postgres conference. And there were, there was also
another presenter who I think has a somewhat similar background to me where a lot of development,
software development in their past. And then, but they work with Python and Django and,
you know, use Postgres. And so I've also, another great technical reviewer and industry
kind of colleague and friend Haki Benita was pointing out to me a lot of similarities.
Like Haki's worked with Django a lot more. I've, I haven't worked with it a ton,
but did use it on a project recently. And Haki was pointing out a lot of similarities. And
I think the frameworks, they sort of influence each other, you know, and there there's other
similar kind of monolithic model view controller oriented frameworks like Laravel and PHP.
For example, it'd be another language framework combo. And I'd love for the book to be considered
by folks in those communities as well. With the idea being like, they'd need to do in a,
you know, they'd be working with a Rails app throughout the book with Postgres, but there'd
be a lot that could be portable to their preferred or their, you know, primary language and framework
stack. So yeah, that for me though, people have asked "oh, why Rails specifically?" And for
me, like one of the main reasons was just the authenticity I could bring with
lots of years and lots of companies using it and seeing some similar, you know,
ideally focusing the book around realistic challenges, you know, something that I might
not get if I was sort of a visitor to the framework and language and didn't have as much
production experience with it, where I might be kind of speculating about some of the issues.
With Rails, I felt like I could, for the application focused portions, I could be very
authentic, like I said, tie things to my specific experience at past companies.
And then the other thing is both Ruby and Postgres are extensible. And so Postgres
of course has hundreds of extensions. I know you also know about that, Claire, as you've presented
on it. And so we bring in about 10, I think there's 10 to 20 Postgres extensions that are
used throughout the book with particular use cases. But then on the Ruby side as well, we've got 10
to 20 Ruby gems that are filling in little gaps here and there or adding functionality beyond what
Rails provides that I think are, you know, genuinely useful. And I wanted people to know
about them and so they could determine if they should include them in their stacks.
[CLAIRE] Well, one thing that's really cool about the book, and I'm not an education expert, and I don't claim
to be, but I've been around the block a few times. And what I've certainly observed is people learn
differently. And that is a fact. And some people learn by doing, right? Experientially, like they
have to have their fingers on the keyboard. They can't just learn by reading a book or having
someone tell them something, because it might go in one ear and out the other. They have to do.
And other people can learn really well by just reading something and their brain processes it,
digests it, and retains it, right? And they kind of just capture it all. And what is really cool
to me about your book is it's useful for both those types of learners. So it's chock full of
not just code samples, but exercises of things you can do, right? You start off giving people
a ride-sharing app and say, "Okay, all the exercises in this book are going to be based on this,
you know, small ride-sharing app." And then, but at the same time, if someone's just going to learn
about the concepts by reading, it's got all that explanation as well. And so I think it's really
cool. I'm not saying it reaches all kinds of learners. Obviously, people who just learn
through like video and watching something, they're not going to benefit from a book as much.
But you definitely are benefiting at least two classes of people.
[ANDREW] Thanks. Yeah. I think, besides video as well, I think that due to my own
lack of skills in the illustration department, and not just like drawing cool things, but I mean
technical illustrations, I'd say like one thing that if we ever do a future version of the book,
I think would be a great improvement would be to add more [diagrams], technical diagrams, you know,
which helps learners as well. And I certainly appreciate them when I read things. I just am
really bad at it. And it just takes a lot. It just takes a lot of time. I don't think I'm bad at it.
It takes a lot of time. And writing a book itself is was for me anyways, was so time intensive.
I just felt like sometimes I just had to I think of certain diagram ideas, but I just
couldn't execute on them with the time constraints, I'd say. So there's not a lot of diagrams in the
book. And I think it would benefit from those. But I think that a lot of times,
if you know, if people are just starting to become familiar with the topic and orient themselves to
it, I think a diagram is helpful. What I found is once I get a mental model of a particular concept,
like let's just say table partitioning, I basically know how it works. Then that's where I think,
you know, on one extreme, I guess if you if you know how it works, and you just kind of want to
know the edges of what's possible, or, you know, that's where I think the Postgres documentation,
it's so excellent that I also couldn't, this book is not intended to be a documentation book.
Because the Postgres documentation is excellent. The Ruby on Rails docs are also great. And they
also have, I'd say a little bit more tutorial style content than the Postgres docs do. There
is some tutorial style content in Postgres, but I'd say the docs are mostly for kind of like API
reference, you know, like what is the, what are the capabilities of this? What are the possible
values? What should I expect as the return and the behavior? So I felt like the book needed to
fit kind of between those two, in a sense, you know, it's not meant to be exhaustive documentation
of every scenario. And it's trying to, to some extent, fill gaps with like, people that are just
starting to become familiar with the topic, or they have some baseline familiarity, but they
want to get some more hands on experience. And then they kind of want to see it in action
in a realistic app scenario, not just a database on its own, but used with an app and
thinking about a user's use case and thinking about even things like growing the popularity
of a platform. And we didn't really get into like making money with the rideshare app, but,
you know, thinking about things like how would we scale if it was a real rideshare app, we'd be
wanting to hire more drivers to fulfill more rides for riders. And we'd be expanding to new cities
and things like that. And then tying that back to data, you know, it's like, okay, well, we're going
to be capturing a lot of data about a lot of rides and a lot of cities. And how would we handle that?
[CLAIRE] So I have to ask you, was it hard to write this book?
That's question number one. And then question number two is start to finish. How long did it
take calendar time? Right? Obviously, I'm assuming you didn't spend every second of every day working
on it. But first, was it hard?
[ANDREW] Yeah, it was. I identify more with the word like demanding
and time intensive. Like I never could have done it wit hout making it a priority in my life.
Like finished it, because it's demanding. I mean, to write, you know, to write a lot,
like each chapter is at 15 to 25 pages, 30 pages, maybe. And there's 15 chapters. And then there's
some reference content as well. Each chapter should really, you know, be cohesive and should be
sequenced appropriately and stuff. There's just lots of kind of technical challenges
in producing a narrative style book that are just going to take a lot of time, even if you're an
experienced author, which I'm not like, and you're efficient at the process. Because I think as a
first time author, too, like I was just, I'll just try that. That worked well, that did not work well,
I'll just try that. It just sort of bumbling into it, basically, as far as like, I'm just going to
write a bunch. I'm going to write things out of order. That's one thing I did, for example,
that turned out to be terrible in terms of efficiency later, because I had to put everything
together into a cohesive narrative that flowed, you know, so you didn't want to talk about a topic
that hadn't been introduced yet. You couldn't talk about something in chapter two that doesn't come
up until chapter 10. So I wrote chapters kind of out of order, where my energy was at the time,
you know, so like if I was using, if I'd been working on some, I just added a GIN index for
something at work, I'd be like, I got to make sure I've got GIN covered well and write about some
scenarios. And so reassembling all that later was really demanding to be, you know, and so like,
if I were to write a book again, I might not do that. And I've gotten some advice from others,
like, oh yeah, don't write out of order, for example. So it was demanding in that sense. And
then for me, I was working full time while I wrote the book. So I had to meet my obligations for my
full time job, you know, which was, you know, roughly 40 hours a week. And in order to have
time for the book, that meant I was usually writing in the evenings. And I happened to be
kind of a night owl, naturally. So it wasn't a big deal, except it was, you know, it can be
disruptive to sleep, because I also want to try to get good sleep. But I'd sometimes be writing from,
you know, eight at night, like we put our kids in to bed, and then I'd be in the office writing
until midnight, or sometimes beyond. And so I'd be trying to cram in as much as I could in the week,
sometimes on the weekend. So given those time constraints, for me, the whole thing start to
finish was a bit over two years. It was about 20, yeah, it was kind of the spring of 2022,
until this summer, the summer of 2024, until everything was all said and done.
Some of that too, was the timeline the publisher is on, which is outside of my control,
the publisher is producing multiple books at once. So they, when certain things need to happen that
are kind of like gate scenarios that need to be gone through, like there needs to be a comprehensive
three chapter technical review, I might get put into a queue with other authors and their books.
And so I would sometimes get a time slot, and they'd be like, okay, well, your your reviews
happening in a month from now. And if I didn't have work to do, that would just be somewhat of
a delay for me, because, you know, I wanted to get it done as soon as possible, of course. But
so yeah, some of that was a little bit on the publisher side, I would say. But yeah,
two years overall is, in talking with other authors is not like a crazy amount of time,
like in the sometimes I've heard of people doing it in, you know, six months to a year timeline
that probably have written books before, but I think two years for a first time author working
full time, it's probably what I would recommend to someone considering it is, can you make a two
year commitment in your life to something?
[CLAIRE] Sure. Well, and I don't expect, well, I could be wrong,
but I don't expect a lot of listeners are thinking about writing books. It's just not something most
of us developer types tackle. But there might be a few, and who knows, maybe your story will
inspire them. The rest of us, especially the Rails developers who are listening will hopefully just
benefit from the learnings and the lessons and kind of all the takeaways of in your book.
But I'm just, I suppose, I'm just really curious about the process that you went through. Because
you have produced this thing that is going to benefit people. And I just want to know how it
happened. So I have one more question about the process before we we dive into things that are
probably more relevant to our listeners, which is did you ever just want to quit? Like even for
a moment or a day or a weekend when you're like, that's it, I'm not going to do this anymore. I'm
not going to write this book. Did that ever happen?
[ANDREW] You know, honestly, it didn't for me.
I think that I, for me, what happened more was like an impatience of really just wanting to
get it out, but also recognizing like it needed to be high quality. And I was to some extent staking
my professional reputation on this too. Like if the book was poorly edited or, you know,
content was poor, inaccurate, that wouldn't be, that would be bad for my career, right? Like I
want to work with Postgres at companies and I want companies to, you know, understand that I
can do a good job. And, but yeah, I guess like it, I think like one of the main practical
challenges, to be honest with you, was just making the time available in the family and
that kind of thing. And I'd say that like, it's, it is really hard for anyone that is considering
writing a book, like, you know, it's challenging to work out with your family, you know,
making sure that you can meet your obligations. And I would say I've made a lot of, I would
made some personal sacrifices too. Like I, some of them were not healthy, like, you know,
worse eating and exercise habits and socializing habits, just simply from time constraints in the
day, you know, like I'm working full time, I've got kids and to make the time available.
So I think, I think like I got, you know, and I think the Pragmatic Programmers structure
probably helped me avoid that scenario. I'm sure that they really don't want authors to reach a
point where they're demotivated to the point where they, it would be best to stop, you know,
it would be like a disappointment for lots of folks. And I think that they do a good job of
using a structured process of approaching the book, starting from the beginning with
really like a lot of emphasis on an outlining process that they call story maps,
and helping you form what they call the hero's journey, which in this case, the reader is the
hero, and they're going on like a journey of learning some technical skills. They do a good
job, I think of kind of keeping the author engaged in the learners goals and motivation throughout
the process to help avoid, I think, I think it genuinely does help avoid that problem of like,
oh man, this is so demanding. Like I'm, you know, not able to meet my commitments.
Maybe I should just stop. So I think that got me through the hump, you know, like we had
within the first, I don't remember if there was a simple calculation, like one chapter a month
or something. I mean, that might be a nice cadence to get through at least a rough draft of a chapter.
But like, maybe six months in, like, I just felt like there was enough of a kernel of the book
that I was, it was re-energizing myself, you know, I'm like, this is, this is going to be useful.
I kind of got more excited about it myself. And then it was like a virtuous cycle of
making me want to continue. So beyond that point, I would say, it was more just about trying to
manage my work life, manage my home life, and just try to get it done. Yeah.
[CLAIRE] And you got there.
[Yeah.] Did you, did you throw a big party?
[ANDREW] I was thinking about it. I never really, what's weird is that, no, I didn't.
[CLAIRE] Okay, well, you should, you should throw a barbecue.
[ANDREW] Yeah. I mean, like my immediate family, of course knows a lot about it and
they're excited for me. And people professionally that are friends that they're excited for me,
you know, they know it's like a challenging endeavor. And, but then it's funny because
like my, the actual people I associate with on a day to day basis, like my neighbors and like,
I work remotely too. So I mostly just like see my neighbors and other parents at the bus stop.
They're like, Oh, you wrote a book. Cool. And that's, that's about the extent of the
conversation, you know, if I'm being honest. So it's, it would be a little bit like,
who would I invite? You know what I mean?
[CLAIRE] I know. It's amazing that we have
people in our lives who just don't understand our work. I was on a walk with a friend last night
and she was like, Oh, how was Seattle last week? And you and I were both in Seattle together at
the PASS Data Summit. And, you know, I gave a talk in the afternoon on Wednesday, but I also
gave a five minute demo about Azure Database for PostgreSQL in the morning. And it was in front of
like 1700 people. So it was kind of a big deal. Like my face was larger than life on these massive
screens. And in fact, they, you know, even had hair and makeup on everybody involved in the
keynote and, you know, before that. And I, you know, I was telling my friend about it,
but it was a really big deal, but I don't think she got it. You know, it's like, "Oh, 1700 people,
that sounds like a lot." The situation doesn't come across with the
words. I don't know.
[ANDREW] Right. And it's not, and it's not to say I don't want to,
none of my neighbors would be listening to this anyway, but like the, I mean, it's not that I
don't appreciate their sincerity and their sincere, like people are, they, they know just
generally writing a book is going to be time consuming and stuff. So they're supportive
and that kind of thing. But I think it's kind of the, it's mainly the divide between
the technology world for people that aren't in the technology world, you know, they're not,
they just don't have like mental models for like what software development is and what databases
are and stuff. So it's, you know, I think for my neighbors that are not in the
it or software world, you know, I wouldn't expect them to know anything about it and stuff.
So it's totally fine to just have like a kind of like a, or I mean, I still appreciate
that they would recognize that and be excited about it in terms of like having a party though.
It felt, yeah, I mean, maybe just like [you should throw a book party. You gotta do it.]
Yeah, I mean, I definitely, my wife is my biggest supporter and I think like
we've, she's been really excited for me to finish it up. And, you know, like
we just have like a little mini celebrations maybe if we're on like a vacation with our family or
something like that. One more quick thing to say though, I guess is I would say like with,
for just the way my brain works, it just, it makes sense to be like,
maybe with the Rails community, as far as the most kind of the audience where the book would be
the most relevant for maybe in terms of like celebrating it or whatever. And I did want to
briefly mention that I had the chance to go to Rails World, in September in Toronto and,
had a great experience there where I was able to, I gave out a bunch of copies of the book
and had a little mini book signing event and the Rails World organizers helped
organize that for me, even though I wasn't like a sponsor or anything, which was great.
And another author was there as well, Vladimir Dementyev. And so we got to meet some
Rails developers and some readers, which included people that had read the book, but also a lot of
people that had not heard of it or anything like that, which was great. Cause then I got to kind
of do a little mini pitch on it and that kind of thing. So in a sense, like that was, that was a
bit of a celebration for me. I was kind of like, it was the first bigger Rails
event I would say I was at since the book was fully done and, you know, on online to be purchased and
whatnot. So that was a, for me, that was like a celebration.
[CLAIRE] Yeah. That sounds like a good
celebration. Are there, are there Rails, Ruby on Rails, user groups, the way there are a Postgres
user groups and all other sorts of kinds of user groups. I wonder if that's another way to
not just spread the word, but also have all these little parties.
[ANDREW] Yeah, for sure. Yeah. I think that that little, the little parties concept is, is probably more kind of where I'm
trying to go with this, I guess. And the meetup groups, there used to be one here where I live in
Minneapolis, that is kind of has kind of unfortunately died. I think it's still sort of
exists online, but I don't think that they're active in meeting up anymore. But what I have
seen a lot of is regional conferences that might be, that might attract on the realm of a few
hundred people. And I did get a chance to go to a couple of this year as well. I went out to
Las Vegas for Sin City Ruby back in the Spring and presented there. And a lot of those folks are
using Rails in their job and some of them are using Postgres and gave a presentation on
query performance and both inside and outside of Active Record and that kind of thing, which is
the ORM. And so that was great. And, you know, recently I know of like a Rocky Mountain Ruby
out in the, I think that was in Boulder, Colorado. And then currently RubyConf Chicago is happening.
And I just saw this morning, there's like 600 people there, which is pretty cool,
including first time attendees and folks newer to the Ruby language and community.
So there's definitely a lot of, kind of regional events and smaller events.
[CLAIRE] And yeah, I just dropped a link into the chat of all these Ruby conferences that are happening
worldwide all over the place. In fact, there's something happening in Indianapolis today,
a monthly meetup. So it's, yeah, I guess there is a lot you've got, you've the world is
your oyster there, Andrew, a lot of places you can go make sure people benefit from this book.
But you said something a minute ago, I just have to disagree with, and you were talking about,
and I may get your words wrong, but the divide between the technology and the non-technology
worlds. And, I actually think when it comes to like empathizing with a friend or a neighbor
or somebody, I think it's just hard to understand anybody else's worlds. Like a friend of mine is a
neurosurgeon. He might have a really bad day at the hospital and I would never understand what
it's like to be, you know, in a surgery for 13 hours trying to save somebody. I just have
zero understanding of what his day to day is like. And, the same is true for almost any other
profession. So the world I understand is tech, right. And right now, Postgres, and,
I think we can try and we can try to have empathy, but sometimes if you've never been in the room,
it's just your imagination, right?
[ANDREW] Right. Yeah. My, my wife is a physician and not in
tech and what she does with her patients, I can't really
understand. I don't have the training background and it's not really in my
interest area and stuff. So I usually, I'm trying to ask her, you know, what her
individual experiences were and, you know, as kind of like a API in a sense, like to understand
how it impacts her and whether I can, kind of my role in the conversation,
like being supportive of what her experiences are and stuff like that. So, I mean,
I'll have some neighbors and non-tech folks like ask about what's it like to work with a publisher
and, you know, things that like would be more like touch points for non-tech people or non-tech book
authors or that kind of thing. On the other hand too, another thing that's been cool is I have
met a number of tech book authors and it's been really cool to learn what their experiences
are. I would ask some of the same questions, what's it like to work with that publisher?
What's the process like? What's the promotional experience like? What, you know, what are they,
how do they kind of help you out with like getting the word out about the book, things like that.
[CLAIRE] Now there's something you said earlier in this call that I think is relevant to so many
listeners, which is you talked about how long before the idea for writing a book was even in
your mind, right? You were a blogger, right? You would share your learnings or your insights online
and put it out there. And I think what's so cool, like as you were talking about that,
as you were talking about how when you write something down it, well, you didn't use these
words exactly, but like I felt like you were talking about how it strengthened your understanding,
right? It reminded me of this quote by David McCullough that I absolutely love, which is
"Writing is thinking. To write well is to think clearly. That's why it's so hard."
But I guess I'm wired the same way. Like for me, if I want to understand something, or even if I
want to really remember something, I've got to write it. I've got to explore it with my fingers
on the keyboard. And I think that, well, it's a gift actually. For those people who are
drawn to writing, it does strengthen their analytical thought process, which I think is
pretty cool. And I suppose coding. Coding is just another kind of writing, right? Being a developer,
yeah, you're writing in a particular syntax, but it's still writing.
[ANDREW] Yeah, I do think that they're pretty similar. I mean,
it probably wouldn't surprise you that when it comes to software development, I'm pretty
interested in the design, maybe more design up front than some developers might be. Because I
want to understand what are the requirements and the limitations, what do we want to have at the
end of this thing, and then work backwards from there on some of the build details. What's the
data model going to look like? What are the models? What are the API endpoints and that sort of thing.
And yeah, I do think writing is, I do feel privileged to, like I tend to try to look at
it as a strength that there's, it is true in my career experience anyway that a lot of developers,
most developers I don't think write at least publicly, based on my individual experience
anyways, just asking folks, finding out. Or they're really good at hiding their blogs on
the internet or something like that. But most people don't write publicly anyways. They may
write privately, and that's great. And then a lot of folks too, I think with, especially with the
COVID pandemic and a lot of folks switching to working remotely, there was maybe a degree of
more forced writing in the workplace setting. Being able to share things in written form,
like whether it's on an internal Notion page or something like that with your teammates.
It became to some extent like a replacement for some meetings and that kind of thing that
might not have happened otherwise. So I think there was some more writing. And
I've actually worked remotely for a really long time, more than a decade at lots of different
companies. So I've tried to leverage writing in my jobs as well, like where there are opportunities to
document behavior or even just simple things like taking meeting notes in a meeting.
And yeah, I think it keeps, writing always kind of keeps me more engaged.
It kind of forces some structure. And so in a way, writing the book did feel like an
extension of that. Like writing a blog post, but in a lot greater detail and breadth.
But of course, it also had its own unique challenges too with
some of the things specific to a book that would be beyond what, I mean, a blog post,
I generally don't have a reader for. I'm just sort of writing things to my interest level and
publishing it. But with a book, I really liked the challenge involved with
having readers, early readers of the book. In the beginning of the book, we have an
acknowledgment section. We had like more than 10 people, I think almost close to 20.
[CLAIRE] There are some awesome people that you acknowledge as your reviewers of this book.
[ANDREW] Yeah, I agree.
[CLAIRE] I mean, people I admire.
[ANDREW] Yeah, me too. I mean, it was a real privilege. Really, honestly, everyone, every single person
that is mentioned, I am in debt to. I really appreciate it. Because generally, they're
doing it, they're volunteering their time. And they get, all they really get is they get
a mention and they do get a copy of the book. But that in no way compensates for the care
level that they put into it and that sort of thing. And really, the success of the book
needs to be shared with them too. And I hope all of, I try to kind of ping people periodically.
But I really do genuinely care a lot about, I'm deeply appreciative of the work that
the reviewers put in. Because they were the early readers and they have expertise areas and they
provided feedback. This doesn't really make sense or this could be better, etc., whatever.
And it's really critical, I think, to making a book that is intended for people to read and to
work through those issues. It's really similar in a way to releasing the first version of software
and just having different bugs. Because you didn't have different types of users using it
and different ways and different clients and things like that. And you're probably not going
to get all the bugs right away. So you got to kind of validate the new software with real usage
experience and fix some issues, smooth out some issues. And I would say writing was like that too.
So those early readers were critical and I really appreciate all of their work.
[CLAIRE] Well, here's something else. I don't know if you're going to agree or disagree with this, but
you were talking about how awesome the Postgres open source documentation is.
And I dropped a link into the show notes, into the Discord for that. And I completely agree.
It is awesome. But when I think about when the vast majority of users of Postgres
are probably consuming Postgres either through a managed service provider, right, in the cloud,
and they're probably an application developer. So they're not necessarily an expert in Postgres,
right? They might be looking at the managed service providers documentation first and foremost,
and they might be using Postgres, you know, as a Ruby developer through an ORM.
And so I think that's another thing that's so cool about this book, right? Is it's going to
benefit a lot of users who might not be going and looking directly at the Postgres documentation
every day, right? Who have a couple, they're a little bit further up the stack, if that makes
sense. Do you agree or disagree?
[ANDREW] I totally agree. I mean, I don't know. I mean, at least in my world,
everyone uses Postgres that way. It's, while it might be like a critical, I mean, it is a critical
part of their deployment. It's, you know, besides it's kind of a server software, the client
application is like a Rails app in this case, it could be Django or something else. But, and so
they're mostly interacting with Postgres via the ORM layer, then their model layer. And there's
this whole, you know, there's this whole concept of the object relational divide, where you have
your kind of object, at least in an object oriented programming language like Ruby or Python,
you have objects and that's what you know, as a developer, you know, you have your, like in the
case of rideshare, we have trips and trip requests and drivers and trip positions and things like
that. And then you have your relational database, which has of course, tables and rows and indexes
and joins and there's like Elizabeth Christensen said, there's kind of like two
different worlds a bit, and they have kind of unique non-overlapping language or non-overlapping
learning resources. And the interesting part to me at a higher level of scale too, is that
it's really that interaction that is the, becomes more critical, you know, like using,
you know, ultimately the database is providing the applications,
satisfying the applications query workload, whether it's write and read operations,
reading and writing data. And a lot of performance comes down to efficiency and
achieving higher amounts of scale through greater efficiency. And the way that I've found to drive
efficiency is to have a real deep understanding of the query workload execution and where we can
make changes to improve efficiency. So, you know, an easy example is just introducing an index
where it would be beneficial for a query that is not currently supported by an index,
but that sort of idea, we have an application that's kind of what all
the developers know. That's what our users of the application interact with. We got Postgres
behind the scenes here, but it actually can be a huge driver of the ability for the platform to
scale, to be reliable. And of course, it's storing the super valuable asset of the data for
the platform. And that needs to be done like safely. We don't want, of course, any data
loss. We want to have referential integrity between our relations and that kind of thing.
So I think, going back to something you said earlier too, Claire,
I felt like I had purchased some database books, but they generally are written with
almost like there is, well, the ones that I've read anyways, they're more focused on the
day-to-day of a database administrator performing some critical functions, but probably in a self
hosted Postgres context, making sure that data is backed up, making sure that data can
be restored if needed, things like that, that are critical, that in a lot of cloud hosting contexts
are kind of part of the package. So in that sense, a lot of app developers aren't necessarily
doing those kinds of critical things that you would need to do in a self-hosted Postgres
context. And their concerns are more about, in my view anyways, are more about efficiency in
their use, limiting resource consumption to achieve greater concurrency and greater scale.
And then capabilities, what can I do in my database that might simplify my
operations where I don't need this big swath of code or this second database that is totally
different and performs this narrow function? Can I just do that in Postgres instead?
That sort of thing. So that's where I felt, yeah, I felt there were opportunities out
in the broader development community to bring that Postgres knowledge, but not necessarily
to a DBA type of reader, but to a developer that has those challenges.
[CLAIRE] So I'm going to pivot because we have so much more ground to cover, but there's a story I
really want to explore, and then we're going to come back to your book. But in your path to get
to where you are now as this developer, you're a consultant too, right? So you do consulting for
companies who need help with either their application, their backend, their Postgres.
That's right, isn't it? Is that a good description or you have a better description?
Yeah, that's right. Yeah. Mostly helping them with their use of Postgres and understanding,
observing, and improving it. All right. But here's my question,
and it has to do with cheese. When I had David Rowley on the show, and I'll drop a link to his
episode in the chat, he's a Postgres major contributor and committer, and we explored
how he got started as a developer and in Postgres. And I found, and I did not know this
before we started the podcast episode, that his journey into Postgres began at a cheese factory.
He was working at a cheese factory. And then later I was talking to Melanie Plageman,
who was the next month's guest. We had her on the show in September, and Melanie
also works at Microsoft, is a Postgres contributor, major contributor and committer.
And in her case, we explore her path to becoming a Postgres committer. And I learned that she
has a story having to do with a cheese factory as well that touched on her journey. So, okay,
am I going to get a three for three? Did you ever work at a cheese factory?
[ANDREW] I did work at a cheese factory.
[CLAIRE] See, that is so weird.
[ANDREW] It's so weird. I was, yeah, I think we sort of pre-baked this topic. But I almost like fell
out of my chair when I was listening to both of those episodes. I've had the great pleasure of
meeting David when I worked at a Microsoft company, and we were a small team within Microsoft,
and David was offering Postgres office hours. And it was, it also is a real, it's, I'm glad that,
I also wanted to bring up this cheese story so I could mention David and Melanie, which I'll do in
a second. Yeah, David offered, you know, very generously these office hours internally to help
folks use Postgres. And it was a great opportunity that I was really appreciative to have and asked
all the questions I had about our usage. We talked a lot about vacuum and, you know, some of the
operational challenges we were facing on the team, which was, it was great that David offered that,
and was working on Postgres at the time, and enjoyed, I enjoyed the episode as well. I just
listened to it recently. And then Melanie, I met at a more recent version of Postgres conference.
I think it was the last year one, the New York city one.
[CLAIRE] PGConf New York City. Yeah, I think
last year she was on the talk selection team for that event as well.
[ANDREW] Oh, okay. Yeah. I remember I
had the chance to meet Melanie and I remember we talked a bit about, she had worked on the
pg_stat_io system view. And that was something I wanted to better understand as like one of the
more noteworthy additions to Postgres 16. I wrote a blog post about that. And then it was cool.
Cause I was like, I got to ask her more about it in person. And for anyone listening to this,
that has never been to a Postgres event, I always try to tell people, you know,
like a lot of open source software, it's so cool to me that you can see these pillars
of the community, these prolific developers like David and Melanie. And then often you can just go
to these events and just meet them. You know, you can just say, hi, I'm Andrew. And, you know,
thanks for all your work on Postgres and have a conversation. And so that was great. Yeah. And so
it was really funny when I was listening to both of those episodes as fans of those people and fans
of the podcasts and the topics to hear that they worked at cheese factories, because I, myself,
in my college days, my story is not as cool. Like I did know software development, but I had a job
when I was in college at a cheese factory here in Minnesota, that is like a producer or they're
kind of like a packager or something like they'd get, they'd get these massive, like multi-ton
blocks of cheese in and they'd slice and dice it into smaller quantities and
then kind of ship it around the country. And, yeah, I just worked on the factory floor and
it was a good job to have as a college student. And, you know, it was a funny connection.
[CLAIRE] What's your favorite kind of cheese?
[ANDREW] Oh, I don't know. I always kind of liked
Swiss cheese. I kind of like the savory flavor. So like Swiss cheese or Havarti or I mean, really,
really, I honestly, I love cheese. So any kind of cheese I'm up for trying,
but I'd say I like some of the things with savory or sharp flavors. [Got it.]
Although I will, I will say really briefly that after working for an entire Summer at a
cheese factory, I didn't want anything to do with cheese for quite a while. Like you just,
you just see it all the time and smell it and it just gets a little overwhelming. And then
I took a hiatus from cheese, but I'm back.
[CLAIRE] I had the opposite hiatus once I went to Japan,
to Tokyo on a business trip. And I had the most amazing sushi of my life in Tokyo. It was
incredible. Incredible. I live in California, so we have pretty great sushi restaurants and pretty
great sushi and obviously easy access to seafood being near the coast, I would think. But
Tokyo was so much better. And so when I came back to the United States, I couldn't eat sushi
locally for like six months. I just was like, no, this is not good enough. I can't, I couldn't
partake. And eventually my desire for sushi won out and I went back to eating it. Yeah.
[ANDREW] Sushi is great. I mean, it makes sense to me. It's all kind of relative. Like even just in
Seattle for me last week, I got airport sushi. Like after I landed, I was hungry for lunch.
And even just at the airport, I felt like I noticed a quality improvement level compared
with what I would get in Minnesota, even though here we have a lot of sushi restaurants and
they do fly in, you know, my understanding is they fly fish in daily and whatnot. But,
I just felt, okay, I'm closer to the ocean. Maybe it was psychosomatic, but I felt like the
quality was higher.
[CLAIRE] So I want to go back to the book and I want to talk about whether people have
favorite parts. But before we do that, I realized that I have a discount code for ebooks for any
listeners who want to check out your book and I haven't shared it yet. So I need to share
it. So it's a 35% off discount code on the ebook and the discount code is ha ha ha
TalkingPostgres all one word. So should be easy to remember if you're listening to this talking
Postgres podcast. But the link that you need to go to is aka.ms/talkingpostgres-andyatki-35off
So I'll put the link in the show notes as well for the episode.
And yeah, that'll get you a discount, off of the ebook. There, now that that's done,
I am curious whether you have a favorite part of the book?
[ANDREW] I would say that after writing the chapter on indexes, in the book, the book has one
chapter dedicated to indexes, not like the chapter, like in air quotes, but like the chapter in the
book that's about indexes. I would say that I felt like this is probably if, if readers, I felt,
I guess I liked it the best because I felt like if Rails devs pick up the book and read one
chapter, they'd probably get the most from that chapter in terms of things they could really use
in their day-to-day jobs. Like at any, I would say arguably at any scale, or any usage of Postgres,
because some of the things, part of that is I think, that or maybe data constraints too,
but I'll focus on indexes. But I think part of it is, I tried to write things in the book
that would be intended to be used at high scale. Of course, like it's in the title of the book,
you know, billions of rows. And like I said before, and thousands of concurrent queries,
I would call that higher scale with like a single instance Postgres.
But there's also like a lot of folks work on apps that will never reach that level of scale. It's,
it's not because the app's not successful. It's just used in a,
maybe a business context. And, you know, it's extremely valuable to the users,
but it doesn't have thousands of concurrent queries and it doesn't have billions of rows,
for example. And so I wanted to make sure there was content that would, in the book,
that would make the book valuable for those readers as well. And I think, you know,
indexes and really, really understanding the benefits to both read queries and write queries,
it feels just really fundamental. I guess it also kind of implies reading query execution
plans and knowing how to interpret that info. But yeah, I'd say, if a developer can do that,
then they're, they're pretty well on their way to working well with any SQL database,
you know, really reading query execution plans, and optimizing their indexes for their
real query workload. I think that would be my answer.
[CLAIRE] Cool. I love it. The other thing I wanted to ask you is, especially since I know, like you
gave a talk at the PASS Data Summit last week, you've talked about PGConf NYC.
So you have been to Postgres events. Is there a frequently asked question that you get from
Rails developers about Postgres?
[ANDREW] I think what's interesting about databases, well, one thing would be making schema changes safely.
I think that the Rails framework, it doesn't actually have a concept of schema changes and by,
because the word schema is overloaded here, just to briefly mention the, what I'm referring to,
it's changing the structure of your database, whether it's adding a table or a column
or an index or a constraint or something like that, like a structural change, not data and
not the database schema object, but like the structure. There's a lot of things, you know,
because we expect our database to be running continuously. I always have found the kind of
engineering challenge of how do we basically change this thing at the same time it's,
it's serving data. It's like writing and reading data to other connections,
[it's like filling a plane with fuel while it's in flight] right. It's kind of weird,
like in a physical structure, you wouldn't be expecting that necessarily that it can.
Adapt, you know, modify its own design while still serving requests.
So I always found that to be really cool. And so that's the idea of concurrent structure
modifications, concurrent in the sense of other concurrent clients that aren't,
don't care anything about the structure. You know, they're maybe just running a query.
They do maybe in the sense of the reading data, but they have expectations about
the availability of Postgres and the fields that they're accessing and whatnot. So I'd say,
you know, I think it's a little mystical to some developers, what those both, what things
work and do work because Postgres generally supports, I mean, I think in the book we covered
almost every scenario I could think of as far as an online, you know, while Postgres is not
being taken down or stopped in any way, like an online schema change or structure change.
But like to achieve that sometimes there's some funny hoops to jump through. And I think it's a
little, like I said, I think it's a little mystical. So I just want to quickly shout out,
there's a great in the Ruby world, there's a great Ruby gem. There's several, but the one I like and
wrote about in the book is called strong migrations. So the idea here with migration, again,
as an overloaded term is those incremental schema changes. So not a major data,
not like a heterogeneous database migration, but just an incremental change.
So Rails uses that term migrations for those kinds of things. Django does as well. And so if
we're adding an index, you know, what's the difference between doing that concurrently and not,
and what are the impacts to which operations might be blocked if we didn't perform that
concurrently. So I'd say, yeah, I mean, at a high level, that category of thing,
there's a whole chapter, I believe, if I remember correctly now, it's been a little while,
dedicated to this topic as well, but making safe schema changes, that would be one category.
And I think also like locking and blocking a little bit related to that, but
it also is a little mystical and it's a pretty complex area. So I think there's a lot of appetite
amongst developers for that material, even if it's a little bit, almost evergreen,
it's like, you know, which operations can I run that are going to block other operations and why,
and how do I get visibility into that? And I'd say we have, I think the book has pretty good
coverage of what kinds of queries will block other queries and what kinds of queries acquire which
types of locks. But I would say also that might be another place to get started with this book,
but also then go into the documentation or other maybe Postgres deeper books that focus on locking,
because it's a very complex topic and there's other books with whole chapters
dedicated to that topic. But yeah, I'd say that's another practical thing in the real world where
we've got queries running that are updating a bunch of data and this update should
be five milliseconds, but it's taking 30 seconds, what is going on, that kind of thing.
[CLAIRE] You're reminding me that one of the most popular blog posts ever on the Citus blog was about
locking in Postgres. Blocking. Oops. Sorry, I just got disconnected. You probably can't even hear me.
[ANDREW] Oh, I can hear you. Yep.
[CLAIRE] So I just dropped a link into the chat about it. It just, it performed incredibly well. People were
hungry for that kind of information about locks in Postgres.
[ANDREW] Yeah, because ultimately it's related to
the concurrency you can achieve and it's related to user experience and those kinds of things are
very visible. And consulting clients I've worked with too, it's a common theme is,
how do I get better visibility into which operations are, you know, for my slow operations,
which of them are slow because they, you know, not because the query itself, but because it was
waiting to acquire a lock. And then Postgres has its own, it's kind of this like
spider web of conversation too, where you get into lock types, lockable resources,
transaction isolation levels. So there's just a lot of content there. And so,
yeah, I think it's a good area because it touches everyone from, I'd say, from those two
kind of communities we discussed where it's like developers to database administrators.
And the solutions usually in my experience aren't, it's not like there's a silver
bullet solution, you know, it's usually about trying to just get better understanding into
your operations and then kind of mitigating some of the undesirable behavior,
so reducing lock length, what's driving the length of a lock, like how do we reduce that? And
that's probably going to be specific to your application. But yeah, I think that's another
common area that folks want to learn more about.
[CLAIRE] So because your expertise is in, as what was the
phrase Elizabeth Christensen used, like you're spanning two different worlds, right? The Postgres
world and the Rails world. I'm really curious when people hire you as a consultant, and I don't know
if you tend to get hired more for three month gigs or six months or a full year or how that works,
but do they hire you for your Postgres expertise or your Rails expertise or something else?
[ANDREW] I think it's usually because of my Postgres expertise that is a skill they don't have on
their team or like it might be a skill they have, but the people involved are not dedicated to
that function. But I think it's usually because it's like beyond the ORM and like the higher
usage, the higher kind of scale usage I was talking about, that's where there's usually
from the clients I've had, there's a gap on their team, you know, it's kind of like
I read these blog posts and I kind of know what's going on here, but how would you go about debugging
this problem or what are the kinds of metrics we should be looking at, or what can we do about
this? How do I actually fix it? And then if it's a team that uses Rails, like
and our contract is a lot longer basically, I mean I'm happy to jump into the code base and start to
make changes. And I have done that with some clients, especially earlier on. I would say I
just kind of started doing my software developer thing, so to speak. But I also have found too
with clients, they want the ownership, both the knowledge and the actions that are taken
based on that to be with their team, which is tricky as a consultant because it
it sort of shifts my role to be more of a coach or a guide in some cases, which is
okay. It's just that it's, which I really enjoy doing and actually it's very fulfilling to
bring, you know, for someone to come to you with a problem that's like, I don't know what's going
on, and then for me to feel like I immediately know what's going on and to bring that
information to them, that's very fulfilling. It feels validating of putting in a lot of work
to try to understand some things. However, it also can mean that in terms of a commercial
relationship it might be shorter, you know, versus if I'm doing work day to day. But it
makes sense to me like with the companies I've worked for as well, they want their developers
to gain these skills, they want them to solve the issues. So yeah, I think I've sort of shifted
my, I think it's just sort of a little bit of the trade-off of the kind of specialized consulting
that I usually aim for, which is this kind of specific Postgres guidance within the usage of a
web application. It's just going to be probably a little bit shorter or, you know, scoped in maybe
different ways. But what I've driven towards more recently is more of an assessment model where
I'm going to likely bring in some Postgres, well I mean not likely, I will bring
in some Postgres recommendations beyond what would be within the purview of the app, you know. So
sizing the instance, what kind of cost efficiency are we actually getting,
architecturally what kind of usage can we expect from this architecture, before we might
re-architect or restructure queries or even consider alternative architectures, parameter
tuning and kind of more like system-wide index tuning and things like that. So kind of like a,
you know, a DBA lite function as well for a team that doesn't have someone like that on their team as well.
[CLAIRE] You've talked a bit about scalability and sizing the database and the
resources and things that you need in order to support the application. And so anybody who's a
regular listener on this podcast probably knows my origin story, that I got involved in Postgres by
joining a small startup called Citus Data. And the Citus open source extension of course transforms
Postgres into a distributed database. So I just have to ask, there's no chapter on Citus in this
book. What's up with that?
[ANDREW] Yeah, I did actually, my, one of my criteria was I wanted all of the
content to be open source content. And I felt that spoke to the, a lot of like a lot of why
folks like Postgres, it's arguably the world's most popular open source database as
opposed to a commercial one. And Citus is an open source.
[CLAIRE] "The world's most advanced open source relational database"
There you go. Official tagline.
[ANDREW] Thank you. And so Citus meets that criteria as an open
source extension. And so no concerns there. I mean there are some other extensions that are
in heavy use too that folks wondered why there was no coverage of like PostGIS for example, or
the Timescale extension for time-series oriented data. Honestly, the main answer was just,
I didn't have real authentic production experience to speak to that I felt would be strong
enough to include in the book. Also it kind of was on the bubble and it was cut in terms of scope
because of how long the book was getting, it came in at 15 chapters. I think initially
we were more in the 10 to 12 range and we were in the 300 pages and it ended up being over 400 pages.
And it just takes a lot, the bigger the book, the more production time it takes
because there's, for example, a layout person that goes through every page and makes
sure that code doesn't spill between pages and things like that. And so at a certain point I
did have to cut it off and even the, I even generally...
[CLAIRE] Okay, okay, okay. You're sounding
defensive now. Okay. So in short, it didn't make the cut, but more importantly, you didn't have the
authentic experience. And that is, I think, part of what makes your book so valuable. Like all the
rest of the stuff that you talk about is based on your learnings, the work that you've
done, right, as a developer, running on Postgres. [Yeah. It is.] Ok, that's fair.
[ANDREW] I just want to add one more quick one too, I also really identified with, I kind of
got, I'm not the only person to say this by any means, but like the idea of "just use Postgres",
where Postgres has a lot of capabilities that folks might not necessarily think of
first, they might think of other specialized databases for. I also think of, I also like to
advocate for just using single instance Postgres for as long as you can because distributed systems
are more complex. And if your team has a really good mental model of what the capabilities
are with single instance Postgres, including replication and using replica instances and
you know, both types of replication, which both are covered in the book, for example.
If they have a good mental model of that, you can lean on that for a long time,
you know, and today's instances that you can purchase or, you know, you can, if your business
is successful, you can afford very high powered instances with very good performance, which I do
think it means that it lessens the need for something like Citus a bit, or even for like
application level sharding, which we also discuss in the book. And so I do really like the idea of
advocating for trying to keep things simple as well. And, you know, so one of the feedbacks
I got from one of the readers was even talking about table partitioning, you know,
make sure that there's lots of kind of warnings and things up up front so that people aren't just
kind of diving into this without understanding what the operational new challenges are that
they'll have by way of using table partitioning, for example, and I really appreciated that
feedback, because we could frame the introduction of that to be more around like, if you're
experiencing these problems, then this is a solution, not just like you should dive into
this because it's got these capabilities that you might need someday. It's like, no, wait until you
really have that issue. Because then you'll get the benefits right away. And you'll know
that it's worth the toil that you'll be taking on.
[CLAIRE] I'm looking, I got a little distracted.
I'm looking at the blog post that I guess you published about how readers get their copies of High
Performance Postgres for Rails. And I'm not going to tell you that I'll wait for the second edition,
where you'll be adding a chapter on Citus, ha ha ha, hint hint hint, in order to send you a picture,
I'm going to make sure to get you a photo of myself with this new book. Which, as you said,
is more than 400 pages. So that's why I didn't bring it up to the PASS Summit to have you
autograph it last week. It was too heavy to fit in my luggage. I couldn't, and it is, it's dense.
It is really heavy. So yeah. But for any of you who want an ebook, just take a look at the show
notes. And there's a link there along with a 35% off discount code you can use to get your own copy
digitally. [Awesome]. And I think I've we've covered a lot of ground in this episode. I do not have
anything else that I absolutely have to ask you. Is there anything else that you absolutely have
to share that I neglected to ask you?
[ANDREW] I would say, I guess, I think working in open source technologies, I guess, just to close on like a
people-centric kind of point, if folks are interested in these technologies,
like there are the, there are the books and the courses and ways to learn, you know,
documentation and usage and stuff. And I think there's a lot to be said though, too, for
these in-person events and going and meeting the community. And it, it just feels kind of neat.
It's almost like they're part of your team in a sense, you know, I know that there are these
Postgres folks that attend events I can interact with and then the same on the Rails side too.
Like, you know, like at Rails World, there was the creator of Rails and the creator of the Ruby
language who flew in from Japan, which was amazing. And so I think I love that about open source that
you can connect the people with the things and kind of build your industry participation,
like in person as well, besides, you know, your use at home and at work and that kind of thing.
[CLAIRE] I couldn't agree more. What was really cool this year at PGConf.EU, which just,
just happened a couple of weeks ago in Athens, Greece. PGConf.EU obviously is in Europe and
it changes city and country every year. So it's in a different location. I mean,
they do eventually repeat cities, but not for a decade, right. And it was so cool to
walk in the room and I was just chit chatting with people and someone from the registration desk
just came and handed me my badge. It wasn't one of those impersonal events where you have to
show ID, et cetera, et cetera. I really felt I belonged. Now that didn't happen overnight, right.
I've been involved in this community since 2017. But, it's heartwarming.
And I do love in person, but I will also, you're giving me the opportunity to mention
POSETTE: An Event for Postgres, which is an annual virtual conference with 40-ish talks every year that
are about a half hour long each that my company and my team actually helps—well, we don't just
help—we do organize it. And you were a speaker last year, which I really, really appreciated you
submitting to the CFP, the call for proposals. And literally within days, the CFP is going to
open for next year's POSETTE: An Event for Postgres. So hopefully users of Postgres open source or
Azure Database for PostgreSQL or developers, contributors, et cetera, will all be, you know,
submitting their proposals again. PGConf.dev's CFP is also open right now. FOSDEM PGDay's
CFP is also open. There's a bunch of Postgres speaking opportunities. And sometimes people think,
oh, only the Postgres experts get to speak or only the Postgres contributors are speakers.
But in fact, people love user stories. They love customer stories. So if you use Postgres,
yeah, submit, make a proposal, share your story. Yeah. It could be awesome.
[ANDREW] I wanted to just +1 that too. I talk with developers that have never, you know,
going all the way back to what you're talking about with planting a seed. I think for anyone
listening to this, that is any, in any kind of influential position or, you know, just on a team
of developers, if you all use Postgres and you felt you had some hard earned win, yeah,
consider presenting it at your, maybe just on your team to start if you want, if that's
more comfortable. [Or local user group, maybe.] Local user group or regional conference, or, you
know, if you really want to go for PGConf NYC, PGDay Chicago in the United States is
another one that I'm been involved with. Claire and I were on the CFP, the talk selection committee,
for that as well earlier this year. But yeah, I just wanted to +1 the message that
people want to hear about how it's used both Postgres developers, you know, like they,
they might be very internals focused and they want to know, the Postgres users are
kind of the customers in a sense, they're the using the platform, but also there can be
kind of peer level learning with, you know, oh, this developer over at this other team,
they're doing this feature and I hadn't even considered that or whatever, that kind of thing.
So I think it's a good message to get out that case study style presentations, there's definitely
a lot of appetite for that. And don't diminish your usage experiences as, you know, possibly
being interesting and useful to others.
[CLAIRE] I am a +1. We are in wild agreement with each other.
But at least we had a few disagreements too. So it's all good, right? [It's all good.]
Okay, let's wrap this up. I want to thank you so much, Andrew Atkinson for joining us here on
Talking Postgres. The next episode, episode 22 will be recorded live on Wednesday, December 4th
at 10am PST. And our guest next month in December will be Affan Dar. And the topic will be leading
engineering for Postgres on Azure. Affan is fascinating, and I'm really excited to talk
to him on that episode. So mark your calendar if you want to participate live on the parallel text
chat at aka.ms/TalkingPostgres-Ep22-cal. You can always get to past episodes and get links to
subscribe to the podcast on all the platforms at TalkingPostgres.com. And transcripts are included
on those episode pages at TalkingPostgres.com as well. And before we leave, if you enjoy this
podcast, please tell your friends. Most of us in the developer world do not like being advertised
to or marketed to, but we love word of mouth. So if you've enjoyed the podcast, tell them on social,
tell them in person, in DMs, however you want to tell them. The hashtag is #talkingpostgres,
and word of mouth is one of the best ways to help people discover a podcast. And a big thank you to
everybody who joined the live recording and participated in the text chat on Discord. Thank you.