Subscribe
Share
Share
Embed
Honored two years in a row as a top tech podcast by The Webby Awards, CRAFTED. is show about great products and the people who make them. Featuring incredible founders, innovators, and makers that reveal how they've built game-changing products — and how you can, too.
What trade-offs did they make? What experiments did they run? And what was the moment when they knew they were on to something BIG?
Hosted by Dan Blumberg, an entrepreneur, product leader, and public radio host with chops as both a technologist and as a public radio host. Dan has founded startups and led product releases and growth initiatives at LinkedIn, The New York Times, and as a consultant to big banks and startups. Before getting into tech, Dan produced and guest hosted WNYC's Morning Edition, the most listened to show on the country's largest NPR station.
Listen to CRAFTED. to find out what it *really* takes to build great products and companies.
[00:00:00] Matt Lerner: So if you take any great startup and look backwards, you'll see that 90 percent of their growth came from like 10 percent of the stuff that they tried. So how do you find that 10 percent as quickly as possible?
[00:00:11] Dan Blumberg: That's Matt Lerner, and he's advised hundreds of startups on how to grow. Now, the CEO of SYSTM has written a book called Growth Levers.
[00:00:19] Where he shares his approach. This episode of crafted is full of actionable advice on how you can grow your products and companies. Matt will tell us about the mindset shift founders need to make from thinking about their products to thinking about their customers needs.
[00:00:33] Matt Lerner: So we teach this all the time.
[00:00:35] And Dan, it's hard. This is the hardest thing we do. We'll talk about jobs to be done style interviewing and why it's such a powerful approach, but also why at first Matt was put off by some of the overly academic language that often goes with jobs. But the thing I couldn't argue with was when somebody did a good jobs interview and I would read that transcript, I would know everything I need to know to sell the product.
[00:00:56] So I couldn't argue with the outcome. And we'll talk about how you can get new [00:01:00] customers to that aha moment as quickly as possible. So they stick with your product. Plus, there's lots of real talk about founders and the mistakes they make. I don't think most companies know where their rate limiting step is.
[00:01:12] If we focus on any other part of the business that isn't the bottleneck, we're wasting our time. But if we can find that bottleneck and focus on it, we make the entire business grow faster.
[00:01:25] Dan Blumberg: Welcome to Crafted a show about great products and the people who make them. I'm Dan Blumberg. I'm a product and growth leader and on crafted. I'm here to bring you stories of founders, makers, and innovators that reveal how they've built game changing products and how you can Too Crafted is produced by modern product minds where I advise companies on product discovery, growth, and experimentation.
[00:01:46] Learn more and sign up for the Crafted newsletter@modernproductminds.com. Crafted is sponsored by Artium, a next generation software development consultancy that combines elite human craftsmanship and artificial intelligence. [00:02:00] See how Artium can help you build your future at artium. ai.
[00:02:10] So growth levers, Matt, tell me, how'd you choose the name for this book? I'm assuming it was you jumping out of the bathtub like Archimedes when you came up with the idea for that name.
[00:02:20] Matt Lerner: Yeah, Archimedes and I have so much in common. So if you take any great startup and look backwards, you'll see that 90 percent of their growth came from like 10 percent of the stuff that they tried.
[00:02:30] That was certainly true when I was at PayPal, but I've seen it with many other startups. And I think most people sort of intuitively get that there's one big thing they need to find. And, and the word growth levers, you know, seems to conjure that image in people's minds. But the whole point of the book is if you start with that assumption that 90 percent of your growth is going to come from 10 percent of the stuff you try, then how do you find that 10 percent as quickly as possible?
[00:02:54] And the book's kind of a step by step guide for how to do that. And what was it that prompted you to write the book? You [00:03:00] know, after about 15 or 20 years of building and running growth teams and marketing teams in the Valley, I became a VC and I started to see a pattern. I started to see so many brilliant, talented entrepreneurs taking so much money from hopeful investors and squandering it, making the same basic repeatable mistakes over and over again.
[00:03:18] And so the first step to that was just to stop being a VC and start my business system to just work with startups and help them Skip the obvious mistakes and, you know, accelerate the path to finding growth levers. But after I'd done it 200 times, like I had a pretty tight, kind of knew what the steps were.
[00:03:36] And I've, I've always loved writing and there's so many companies out there who can't afford the program. We can't accept them because we can only work with a certain number of companies and everyone wants my advice. So I just thought, you know, if I can put it in this book, then I can really magnify the impact of these things that I've learned that I think are pretty powerful.
[00:03:53] I'd love to hear more about some of those mistakes that you kept seeing over and over again. And maybe you could put it in the context of in the book, [00:04:00] you talk about three types of people, the under thinkers, the over thinkers, and the delegators. And I'm interested to hear more of how you identify those three groups.
[00:04:08] Yeah. So one of my editors is a philosopher and I don't mean he's like a philosophical guy. I mean, there's an actual philosophy professor at a university or he was. So we got pretty far through the book and he said, Matt, basically what you're saying in this book is companies should. interview their customers and try to understand, you know, why a customer would buy their products, measure your, your company's ability to attract and acquire and delight customers and find the areas of highest impact and focus there.
[00:04:35] And then run lots of experiments and see what works. And he goes, Matt, don't take this the wrong way, but isn't this kind of obvious? Isn't this what all companies do? And I said, you know, Bill, you'd be surprised. This is not what all companies do. So he said, well, what do they do? And that was a great question.
[00:04:53] And the first thing that came to mind were these overthinkers. You see these teams and it usually starts to be honest with [00:05:00] the personality of the founder. And, you know, in a startup, you don't know the right answers. So you ask a lot of people and you talk to experts and you have debates and you pull data and you come up with strategies and you come up with theories and, and those are all good things to do in a company.
[00:05:14] But if that's all you do, it comes at the expense of execution. And, you know, in startups, in any business. No plan ever survives first contact with the customers. And so if you're not actually trying something and putting things out there, you're not going to fail. You're not going to learn. You're not going to succeed.
[00:05:30] And then the next pattern of failure I saw is kind of the opposite of that. I call them under thinkers. So there's this thing, which is very good in startups, which is like a bias for action. And these are people who, you know, quickly build and launch things based on conversations with customers, a sense of the market, and they're very quick to roll things out.
[00:05:48] The problem is that The set of potential solutions is so vast that you can't get this by brute force. You can't just, you know, if you do, you're lucky. And so they end up and each time you ship something else, the product [00:06:00] gets more complicated. It's harder to maintain. It's harder to use. It's harder to explain to potential customers, the documentation, everything gets harder and slows you down at a time where more than anything you need speed.
[00:06:11] And so those are kind of the overthinkers. And the last one. You get someone who's very senior and they come in and they're very humble and they know that they're not an expert in anything, but they can hire great people. So they go out and they hire a great head of marketing and a great head of sales and a great head of, you know, compliance and a great head of engineering and a great head of product and tell all these people to go do your best.
[00:06:33] And so what you end up getting is these people each come in with like their best practices and whatever worked in their last job. And, and they all come to you with proposals that first of all, they're each pulling in slightly different directions. Cause these are functional silos. These are not a company.
[00:06:47] And second, you can't fund even a fraction of these things. And most startups aren't world class at everything. They're like world class at one, one and a half things and everything else. They just sort of do it well enough until they actually can get some traction. [00:07:00] So it's higher end delegate. It's like the third pattern of failure.
[00:07:02] And. And then I was sort of thinking like, what are these patterns of failure all have in common? You know, a startup is in a very low information environment. So first and foremost, you've just got to figure everything out. Who exactly is our customer? Where are we going to find them? What is the first email we're ever going to send to them?
[00:07:18] Going to say, what does the product need to show them the moment they log into it? What kind of people should we hire as engineers? Like there's a million things you've got to figure out. And so if the main thing your organization needs to do, like an early stage startup is learn, go hire a bunch of people and ask them questions is not a quick way to learn.
[00:07:34] So that's sort of the opposite of that. This way of learning quickly is what I've tried to explain in my book.
[00:07:40] Dan Blumberg: Yeah. I mean, the line about the overthinkers, you know, it makes me think of the Mike Tyson quote, right? Everyone has a plan until they get punched in the face. Exactly. So you just gotta, you gotta try stuff.
[00:07:47] You gotta put it out there because you really never know. And that's why the pace of learning is so important. Um, but I want to ask this, this question. Super naive question. I have my own take on, on the answer for why here, but why is the pace of learning so important? Because I imagine you [00:08:00] meet some people who you were advising and it's not actually obvious to them why the pace of learning is such a key metric.
[00:08:05] So I'd love if you could expand on that.
[00:08:07] Matt Lerner: The simple, straightforward answer is in a startup, you've got. You know, depending on your runway and your burn rate, maybe one to two years, and you're going to have to do something that can scale. Like the best startups will scale from zero to IPO, you know, like nine figures, annual recurring revenue, public company.
[00:08:24] In seven years. So if you're going to do that, you got to learn quickly. So like, here's a thought experiment I do with people. Like, you know, who knows, you know, Airbnb took something like five years to find product market fit. You know, Slack started as a games company. I don't know how many wrong turns your startup needs to make before it's, you know, gonna find its magic bullet and be successful, but let's pretend we knew your startup needed to make.
[00:08:49] Exactly. 1, 000 mistakes. And the thing you try thing number thousand and one was going to catapult you to success, right? Yeah. It's kind of a silly idea, but if you thought, [00:09:00] if you knew that was true, how would you run your company? You would test as many things as quickly as you possibly could, right? That would be your only objective.
[00:09:09] So obviously we don't know that it's a thousand. We don't can't put a discrete integer on it, but what does that change? But it's probably more than one. It's, it's definitely more than one or you're not even listening to this podcast. Yeah. So, so what does that change? If you don't know the number, you still want to get through as many of these learning cycles as possible.
[00:09:28] To boost your chance of finding some big levers before you run out of money.
[00:09:32] Dan Blumberg: Totally. And I've already used the boxing analogy, but here's the baseball analogies, like it's multiple times at bat, rapid iteration leads to multiple times at bat. And I know Eric Reese has written a lot about that. And a lot of people use that analogy.
[00:09:44] I wonder if you could share some examples of who you think are some of the, the shining beacons on the Hill of, of running lots and lots of experiments.
[00:09:51] Matt Lerner: So in terms of companies who do this a lot, I can kind of tell by looking at their websites. So like a lot of times we like to look at [00:10:00] pairs of websites.
[00:10:00] So I'm just going to name and shame here, but there's two companies who make landing page optimization software. And that's obviously an area that demands a lot of experimentation. You know, their headline on their website says, we are the number one landing page building software. That's nice. But the other one says, convert more leads.
[00:10:19] Build high converting landing pages with no coding required. And I know if I wake up in the morning and I need to optimize a landing page, I want to convert more leads and I want to build high converting pages with no coding required. So I can see they've been testing language and they know exactly the words that are in their customers heads.
[00:10:36] So that company is Unbounce. You can go in the Facebook ad library and see You know, for any company that you're interested in, if you think their messaging is spot on, go see all the stuff they're testing, go see how many different variants they're testing. And you can really get a sense for who's testing a lot.
[00:10:52] Dan Blumberg: I love your notion in the book of a locksmith moment. And I think the unbounce example is a great example of that, but can you explain what you mean by a [00:11:00] locksmith moment?
[00:11:01] Matt Lerner: I got this idea early on in my time at PayPal. I was living in an apartment complex In San Francisco, and there was like a main gate that opened to a courtyard, uh, where all the apartments were and everyone had a key to that main gate.
[00:11:13] And I came home one day and there was a sticker on the gate and it said 24 hour emergency locksmith, and it had a toll free phone number. And I thought, this is a really smart locksmith. You know, she could have bought ads on podcasts or on the bus or in the yellow pages. I wouldn't have remembered any of that stuff.
[00:11:31] Cause The place I'm going to be when I suddenly need a locksmith more than anything else is like right here locked out of my apartment complex because somebody squirted glue in my locker. A key broke off or something like that. So she figured out exactly where I was going to be. At the moment, I realized I needed her product more than anything else.
[00:11:47] And then I would get in my car and I'd go to work and I realized we at PayPal, it was early on and we were doing partnerships with all the hosting companies and all the e commerce platforms to make sure that PayPal was pre integrated in all those products as [00:12:00] a payment method. So that, you know, cause nobody wakes up in the morning and thinks, Payments time.
[00:12:04] I want to start a business. Let's get payments. Right. They think, yeah, is the domain available? Okay. Can we get a website? Can we get a shopping cart? Can we get products? So we knew already we were kind of step nine in people's journey. So we went and partnered with like step six, seven, and eight. And that was like our locksmith moment.
[00:12:20] And the more I thought about that. See it all around me at the supermarket. Soy milk is in their refrigerator aisle, even though it doesn't need to be refrigerated because when I'm looking for something, you know, a dairy alternative to put in my coffee in the morning, I'm looking in the refrigerator section.
[00:12:35] So there's examples of this all around. And when we work with companies, we have them study their customer journey to figure out what are their possible locksmith moments where they can turn up at exactly the right time.
[00:12:45] Dan Blumberg: Yeah, there are many versions of jobs to be done style interviews. And you write that, you know, at first you found jobs to be done, like overly ceremonial and Nopa Don, who is your colleague.
[00:12:54] And I used to work with, uh, back when I was at city ventures, he helped me to understand jobs to be done and really what you're [00:13:00] looking to learn. I'm curious if you could share more about your initial take on jobs, what, what it was and how you then came around and why you now recommend it. And then the way that you recommend it to other people.
[00:13:11] Matt Lerner: Yeah. Yeah, I think I have to thank Nopadan for this one too. I always liked the idea. I found it seductive when I first read it in Clay Christensen's book, you know, and we all read the milkshake stories and things like that. Yeah. But trying to do it myself, you know, Bob Mesta has been very generous with Nopadan and I, and he spent hours with us working through stuff.
[00:13:29] And, but I always struggled to do good interviews and get my head around it. But the thing I couldn't argue with was when somebody did a good jobs interview and I would read that transcript, I would know everything I need to know to sell the product. I would get so many great ideas out of that. So I couldn't argue with the outcome.
[00:13:45] And then, yeah, you know, you'll see in the book, I explained it all, but I sort of push aside a lot of those terms and a lot of that jargon, other than kind of introducing it and giving some homage to Bob and clay. And just, I try to hide all that and just make it very [00:14:00] practical, kind of nuts and bolts, how to do the interview, because I do think that mindset, that mental shift is, is incredibly powerful.
[00:14:07] Yeah, well, let's, let's get into it. So what is a good jobs interview? What are you listening for when you do that style of interview? So basically it's this, this mindset shift. And Bob talks about this of supply side and demand side thinking. So our heads are full of, you know, if you're founded a startup, you're going to, you've got an idea.
[00:14:25] You tell your friends, they say, it's great. You go out, you talk about your idea. You pitch investors. They say you're a genius. They give you money, and then you've got to go sell your product. And it's natural to keep talking about your product. But of course, and this sounds so cliche, but customers don't care about your product.
[00:14:40] They care about, you know, selfishly about what is going to enable them to do. And it's very hard for people to step outside of thinking about the product and imagine themselves in the customer's view. What's this going to enable them to do? So in the book, I use the example of like a meditation app. And so if you, you know, invented a great meditation app and you [00:15:00] wanted to get customers, you'd thought my customers want.
[00:15:03] NSDR meditations, nons sleep, deep rest. And you know, they want a good narrator voice and they want timers and chance and a sleep timer and they want Sam Harris the best meditator ever to narrate whatever they got. All these things they think their customers want. Yeah. Nobody, no customer wakes up in the morning and thinks, I wonder if I can find Sam Harris meditations.
[00:15:25] Right? Like, well, a few people do, but everyone in the world needs to meditate and most people who need to meditate. are thinking, I need to sleep better. I need to calm down and focus. I, you know, I need to be able to be present with my children after work. So I don't ruin my marriage. You know, like these are, these are the things that normal people think about.
[00:15:45] They don't think about chants and mantras and binaural beats. Right. And so, so to just kind of be able to jump over that wall and say, okay, a person who needs to meditate, what do they really want? They want focus. They want it. you know, sleep, they want to relieve stress and anxiety. Um, [00:16:00] so the goal of a good jobs interview is to get you to be able to understand the world from that perspective.
[00:16:04] Dan Blumberg: So how do you do it?
[00:16:06] Matt Lerner: So we teach this all the time. And Dan, it's hard. This is the hardest thing we do. And if I could figure out a way to get like chat GPT to do this for my clients, I would. But the first thing is like people sit down for an interview and the first thing you want, you want to know why they buy your, bought your product and how they found it.
[00:16:23] So you're going to say, how did you hear about us? And why did you buy our product? And those are just the worst questions. Cause how did you hear about us? Tells you like the last mile and a 10 mile journey, right? Like, how did you hear about our meditation app? Well, I, you know, my friend Dan told me about it.
[00:16:38] It's like, okay, so you just woke up this morning, decided you need to meditate and ask Dan. You've been stressed. You've been sleeping badly for four years. So what we'll do is you have to talk, start with the product. So just explain to me in your own words, what did you buy? But then you want to go to the outcome.
[00:16:54] Okay. And what's that going to enable you to do? Well, I've had trouble sleeping and I'm hoping it's going to help me [00:17:00] sleep better so that I can perform better at work and be more patient with my kids. Okay. Now we're focused on the outcome. And now I want to go back and retrace the whole customer journey.
[00:17:10] So do you remember when you first started thinking about, you know, sleep issues? I was like, Oh yeah, you know, I was having these problems and this and that. And a friend of mine said that, you know, he started seeing a sleep therapist and all that went away. And so I was like, Oh, maybe it's a sleep thing.
[00:17:24] So what'd you do next? Well, I tried a sleep therapist and I didn't work as blah, blah, blah. So you start with the outcome and then you go back to the beginning of the journey. And then you just follow that journey and ask the customer like, okay, and then what did you do? And then what did you do? And then they're gonna, like, I was just literally my last call.
[00:17:42] I was on This guy had bought a mountain bike. They're going to say something ambiguous. So this guy was like, why did you pick this brand of mountain bike? And he's like, the company has a great ethos. What does that mean? Yeah. And like, why is that important to you? What's an ethos? What does that explain in your own words?
[00:17:56] What is their ethos, right? So they're going to say weird things, just [00:18:00] unpack those things. And basically I want to go through the whole story, but what I'm listening for are four things. Well, Six things actually. One of them is like, what's his outcome? What are they trying to achieve? Two is what are the bad things?
[00:18:12] Well, you know, not sleeping. What was that like? Oh, you know, my kids, blah, blah, blah, this at work problems, too much coffee, you know, vicious circle, whatever you want, the bad stuff. And then there's things that stop people from adopting your product, which are going to be anxieties and inertia. Then the other two things are going to be those locksmith moments I talked about and alternative solutions.
[00:18:33] And now as a marketer, I'm going to basically, I'm going to get a list, six lists of all these things, but the action sort of like in a gambling sense is going to be in one of these buckets where I'm going to want to focus first. So, you know, it might be. Nobody even thinks there is a solution out there.
[00:18:50] And so like, you're just going to play up the pain first, or maybe, you know, it's like weight loss. It's, it's a super saturated category and everyone knows there's lots of solutions, but [00:19:00] they're really cynical about them. So you're going to start with the anxiety. So you got to figure out like low hanging fruit, where are the most customers getting stuck?
[00:19:07] So like in that photo book example, it was anxiety. Everyone knows what a photo book is. You don't need to convince them. It would be nice to have a photo book, but they're scared to death. They're going to have to install some horrible software and like spend a good day and a half making this bloody photo book.
[00:19:22] So it was all about understanding and overcoming that anxiety. So that's sort of how I would do an interview and then how I would apply it to a startup. Yeah. My favorite, and you just alluded to it, one of my favorite things that I found so revelatory when I learned the approach of jobs is the quote unquote forces of progress, the things that are pushing you to a new solution and things that are pulling you away, like the anxieties that you just mentioned and just understanding what is that struggling moment as jobs folks call it.
[00:19:47] And cause it really focuses in on what will it take for someone to switch from what they're doing today to what you think they might want to do tomorrow. Yeah. Super valuable. Okay, so can you give advice on what do you do after the [00:20:00] interview? So, the thing about this is there's a lot of value in there, and part of it is finding these nuggets, but part of it is shifting your team's mindset.
[00:20:07] And the way they're going to do that is by starting to read these transcripts, and hear the customers words, or listen to those recordings. So what we'll do is, We'll ask the team to distribute their jobs interviews recordings to the team and then each person should listen to at least three interviews or read at least three transcripts and go through them and pull out quotes that fit into these buckets.
[00:20:30] The four forces of progress, struggling moments and alternative solutions and just like create a document with all those quotes excerpted. Obviously. LLMs could do this faster and easier, but then people wouldn't be reading these words. And part of it is we need to catalyze this mindset shift. So then you'll go through and you'll sort of consolidate that into, and I've got a template in the book, but a customer journey canvas that sort of shows You know, here's where it starts and here's the push.
[00:20:56] And here's the alternative solutions and blah, blah, blah. And the desired outcome. [00:21:00] And until like a single sheet that really documents the customer journey for one of your outcomes, tell me about growth sprints. What is a growth sprint and how do you organize one for success? So the last piece of this, once you've done your jobs, interviews, and you've kind of mapped your growth model and found leverage in your business, You're going to have a lot of ideas, more than you could ever test.
[00:21:24] So growth sprints are basically one to two weeks sprints where you're doing an experiment to test an idea that you have. And the idea here, and the way it's different from product sprints is they're very quick, often don't involve engineering work. You're basically just going to say, we have this idea, what's the riskiest assumption and what's the quickest way we can test it.
[00:21:43] And unlike a product sprint and a growth sprint, the majority of your tests fail. Like if the majority of the features you built and launched failed, that would be a horrible company. Uh, actually I think I worked there once, but, um, but with growth experiments, you're, you should be bold. If everything works, you're, you're not being [00:22:00] bold enough.
[00:22:00] So we'll just have teams come up, look at your backlog. What are the things that the biggest potential impacts. What are the riskiest assumptions and what are the quickest ways we can test those? You'd make predictions, you write a hypothesis. Everybody makes predictions to spread the learning and eliminate hindsight bias to, to force the thinking.
[00:22:16] You run the experiment, you share the results with the team. And then most people are surprised because most of them were wrong. And then you think about, well, why did we get that weird, unexpected result? You do follow ups, you look at your data, you figure out your next experiments and everyone gets a little smarter.
[00:22:32] So even though each of these experiments tends to fail, The net result is even the failures make you smarter. Um, and that gets you closer to figuring out, you know, the thing that's really going to work. What do you need to be ready for a growth sprint? It sounds awesome. Like, yeah, let's go do a growth sprint, but what is the pre work that has to go into it and what's the right team and the right setup so that it's actually successful.
[00:22:53] So I'll give you the short version and the long version. The short version is if you've got an idea and you feel good about it, [00:23:00] just do it. So the minimum viable growth sprint is write a hypothesis. And then go test it. And if it's a big, hard thing, find an easy way to test it. So, um, you know, if you don't have a lot of traffic to your website, you've got two headlines, you like them both, but you don't have enough traffic.
[00:23:16] It would take six months to find out which one's the best headline. Okay. Put those two headlines into ads and show those ads to lookalikes of visitors to your website. Pay a little money and you can get that answer in a week. Or, um, we had a team that had an idea. It was like a B2B learning app. So it was like teaching skilled tradespeople how to do, you know, whatever electrical line repair things or something.
[00:23:38] And they were thinking about building notifications, reminders, um, in their apps to remind people to do their lessons. But it was a very early stage startup and they only had one coder. It was a technical co founder and he had a bunch of other stuff. So should we do this or not? So they just took all their recent signups.
[00:23:53] They downloaded the list of the people and their phone numbers. They divided them into a test and a control group and the co [00:24:00] founders just spent the weekend sending WhatsApp messages to the test group saying, Hey, don't forget to log into the app and do your exercises. I'm going to go ahead and paste it in a little link to the app.
[00:24:07] People who got the WhatsApp messages were four and a half times more likely to do their lessons that weekend than people who didn't get it. And they were much more likely to still be using the app after two weeks. So very suddenly, do we build notifications or not became a very easy decision. So the minimum is I'd say, just like write a hypothesis and go try it and then discuss the learnings.
[00:24:31] But to really. Get the full value to increase your chance of success. I'd say do some jobs to be done interviews because a lot of your best experiment ideas are going to come from unexpected things you hear from your customers and then understand your business well enough to kind of know where the bottleneck is.
[00:24:47] You know, if lots of people are installing your app and none of them are using it after one week, you don't need to be buying more traffic. You don't need to be building a referral program, right? So kind of have a sense of where to focus in your business for [00:25:00] maximum impact.
[00:25:01] Dan Blumberg: Yeah. So let's, let's talk about retention, which, and I can't believe I'm still saying this, but is still so easily overlooked.
[00:25:07] It's like, it's so important that people use your app, not just once, but a second time. And I remember when I was a young product manager at the New York times and I was on the mobile app and I saw for the first time, a survival chart of, you know, the number of people use the app on day one, and then it just falls off a cliff onto day two.
[00:25:22] And it falls a little further on a day three, and then steadies. And, and so we ran lots of experiments on like, how do we get people to use the app twice? Because if it hits a plateau at a higher level, we'll just have more people over time and they'll subscribe and all these good things will happen.
[00:25:35] Yeah. Um, and that's not uncommon with mobile apps. A lot of people, especially back in the early days of mobile apps, you know, you just downloaded something, you try it and whatever. And, and I'd love to hear your thoughts on retention. And in particular, if you've come across, you know, founders and others who are thinking too much about buying more traffic and not enough about what happens when they actually get here onto our property and, and you know, what do they do?
[00:25:56] How do we get them to come back again?
[00:25:58] Matt Lerner: And again, you know, I think [00:26:00] that What, what's that question they ask? Like, what's something you believe that, you know, most other people wouldn't agree with or something. I think when most startups, certainly on mobile apps, most retention problems are actually activation problems.
[00:26:13] So the first time someone installs your app and opens it up and tries it, they're not active, they're curious. They're checking it out. Is this any good? And if they don't open it a second time, it's not because they forgot and you need to send them a push notification or a reminder email is because they didn't think it was very good.
[00:26:30] They got stuck. They didn't ever appreciate the value of it. So what you find in retention curves is, you know, you'll find some number like 11 uses in the first 14 days or something. It's like people who pass that usage threshold tend to have very good retention and that's not because there's something magical about the number 11.
[00:26:50] It's that people who. If you drop a couple of times and like it and get the value, we'll use it 11 times in the first two weeks. And so the goal is [00:27:00] understand like, what does that out of box experience have to be like, what, what are they trying to achieve, you know, jobs to be done? What anxieties do they have?
[00:27:07] What questions do they have? And, you know, address those immediately when they use the app, give them the aha moment, and then they'll remember to come back and do it again. Yeah. So it's, if someone like uses your app twice, it never comes back. That's not a retention issue. Uh, just like if you go on a first date and the person never calls you again, you know, that's not a divorce, right?
[00:27:27] That's an activation issue.
[00:27:29] Dan Blumberg: Hey, I know Facebook, I forget the numbers exactly, but they had like a famous growth insight early on that. If you connect with, I think it was 10 friends in your first seven days, your attention would go up like 10 X. And when I was in the New York times, I was looking for something similar that if people could only do.
[00:27:44] X in the first couple days with the app that their retention would double. And so I looked at all this data. Do they go to the sports section? Do they watch a video? Do they log in on a Saturday, et cetera, and found this simple insight, but very actionable, which is that retention would double. [00:28:00] If people would read just one article, and if they read a second article, their retention would double again.
[00:28:05] And because, you know, you can scan headlines and never actually open an article. And so once we had this insight, we could run all these experiments, some with notifications, but most were around headline writing or different presentation, the mobile app, and, you know, lots and lots of things we could do to just help people get to that first article.
[00:28:20] Matt Lerner: Or Netflix, early on when it was DVDs by mail, they figured out that if they could get people to find three titles that they would add to their queue in the first 90 seconds, they had much higher conversion rate to sign up. Yeah. So you just got to figure out how does the user, what's the value of the New York Times?
[00:28:37] You find a good article, right? Yeah. And how do you get them there quickly?
[00:28:41] Dan Blumberg: I want to go back now to what got you into tech in the first place. I haven't asked this question of a guest in a while, but I had this kind of weird framing and I'd love to try it and see how it goes. And I think you're game. So I'm going to give you the title of a blog post that you have written or that you will soon write.
[00:28:55] And I'd like you to write that blog post for me right here. Okay. Are you game? Let's [00:29:00] try it. All right. So here's the title of the blog post that you are writing right now. The title is everything I needed to know about helping startups I learned in an oil refinery.
[00:29:10] Matt Lerner: I know where you got that. So I had a summer job in an oil refinery.
[00:29:15] It's hot, sweaty, dirty work, by the way. So the first thing I learned is I want to finish and graduate college. So I don't have to spend the rest of my life working in an oil refinery. The second thing I learned there that was super useful was. It didn't matter how smart you were. Like I was in a chemistry lab and there were people with PhDs and they were geniuses, but you know what, if they didn't do the work, no one liked them and their lives were miserable and they were going to get fired.
[00:29:39] So the second thing I learned, which is very different than what they tell you in school is it doesn't matter how smart you are. It matters. You know, if you do the work, if you get along well with your coworkers, but the thing that I come back to so much in my work with startups, and we kind of alluded to this earlier is the idea of a rate limiting step.
[00:29:55] So a refinery. It's a very simple business model. This was actually a used oil [00:30:00] recycling facility. So a company, you know, has some lubricating oil or cooking oil or something and it's dirty. They don't need it. You can't just dump that in the river because that's illegal and horrible. So they would pay us money per gallon to take the waste oil.
[00:30:14] We would separate out the dirt and the water and clean it up. And we would sell it as fuel oil to the steel industry typically. So we got paid to take oil and we got paid to sell the product. So it was great. We got paid at both ends. The only way to screw up this business was if we stopped refining oil because While all of our revenue was delineated by gallons, all of our costs were delineated by hours.
[00:30:37] So salaries of employees, leases on trucks, machinery, and equipment. So our profit was just a function of gallons per hour. And my job at any given time, somewhere in that refinery, there was a bottleneck. And my job was to walk around. With a clipboard and read dials and flow rates and temperatures and find the bottleneck and then tell the shift supervisor and then his or her job would be to go and find the person who's in [00:31:00] charge of that bottleneck and tell them to open it up to keep things moving quickly.
[00:31:03] And I realized that the tech businesses are the same way. Like we talked about, if, if no one's using your product a third time, you don't need to be buying traffic. And it's, it's such a simple insight and I don't think most companies know where their rate limiting step is. And I think if you just start to map things out and say, Hey.
[00:31:20] If we focus on any other part of the business that isn't the bottleneck, we're wasting our time. But if we can find that bottleneck and focus on it, we make the entire business grow faster. So figuring that out, if you don't know where it is, like job one is just go figure out where is your rate limiting stuff.
[00:31:36] Dan Blumberg: Awesome. And I'm especially interested in your take on how writing a book, like literally physically printing a book is not a terribly agile process. It's much more of a big bang launch than you might do digitally.
[00:31:47] Matt Lerner: Yeah, I know. I tell everybody like to experiment one week growth sprints. And then I, you know, it took me two years to write this book and most books don't ever have any positive impact, you know, for their authors.
[00:31:58] Most books don't sell [00:32:00] very many copies. So this is kind of the opposite of lean and the opposite of everything I teach. And the only reason I did it. Um, it's because I very much deeply wanted to do it. I, you know, I love writing. I love thinking about this stuff. And, you know, my, my business, this isn't a startup.
[00:32:16] I'm not trying to make anyone a gazillion dollars. I started this business to explore ideas that I'm really excited about. And for me, like putting those ideas into this book is very much My goal and my purpose and something I love doing and when I got the book back from the editor and read it I absolutely loved it.
[00:32:33] And I thought you know, if nobody else ever buys a copy This is a success because i've written the book that that I want to write. So not a best practice not agile Not anything i'd recommend You do unless you really can't stop thinking about doing it
[00:32:48] Dan Blumberg: Yeah, that's awesome. I'm, so glad you did. I'm, so glad it's working out matt.
[00:32:51] Matt Lerner: Thank you so much my pleasure dan This is a great chat.
[00:32:58] Dan Blumberg: That's Matt Lerner and his awesome [00:33:00] book, nice and short, lots of pictures, his growth levers. I'm Dan Blumberg and Crafted is produced by Modern Product Minds, where I advise companies on product discovery, growth, and experimentation. Learn more and sign up for the Crafted newsletter at ModernProductMinds.
[00:33:13] com. Crafted is sponsored by Artium, a next generation software development consultancy that combines elite human craftsmanship and artificial intelligence. See how Artium can help you build your future at artium. ai. And please, share Crafted with a friend, and if they're skeptical, just ask them. What are they trying to achieve?
[00:33:34] Matt Lerner: You know, jobs to be done. What anxieties do they have? What questions do they have?