Subscribe
Share
Share
Embed
Honored two years in a row as a top tech podcast by The Webby Awards, CRAFTED. is a show about great products and the people who make them. Featuring incredible founders, innovators, and makers that reveal how they've built game-changing products — and how you can, too.
What trade-offs did they make? What experiments did they run? And what was the moment when they knew they were on to something BIG?
Hosted by Dan Blumberg, an entrepreneur, product leader, and public radio host with chops as both a technologist and as a public radio host. Dan has founded startups and led product releases and growth initiatives at LinkedIn, The New York Times, and as a consultant to big banks and startups. Before getting into tech, Dan produced and guest hosted WNYC's Morning Edition, the most listened to show on the country's largest NPR station.
Listen to CRAFTED. to find out what it *really* takes to build great products and companies.
[00:00:00] Gabriella Rosen Kellerman: There's a lot of fun assessments out there: Things like, which Disney Princess are you? Scientifically, That's about up there with a horoscope. But everything we do is evidence-based, which means that if we wanna assess whether you feel a sense of belonging in your organization, we're not just gonna ask, do you feel like you belong?
[00:00:18] Dan Blumberg: That's Dr. Gabriela Rosen Kellerman. She's the Chief Product and Innovation Officer at BetterUp. A multi-billion dollar startup that helps people and companies level up. Through a combination of human coaching and technology, BetterUp is able to offer the kind of executive coaching that used to be reserved for just executives.
[00:00:37] This is part two of my conversation with Gabriela in part one, which you can find in the CRAFTED. show archive. We spoke about her book Tomorrowmind, which describes the key skills we all need to navigate today's “whitewater world of work” where we always have to be nimble and ready to pivot to new careers and opportunities.
[00:00:53] On this episode of CRAFTED., we dig into how BetterUp ships new products, proves they work, and what's next.
[00:01:00] Gabriella Rosen Kellerman : We have a ton of experiments happening with Gen AI way more obviously than what's in production, and there's so many different ways that it. Could possibly help our coaches, help the people getting the coaching, help the partners who are funding the coaching so we feel like kids in a candy store.
[00:01:21] Dan Blumberg: Welcome to CRAFTED., a show about great products and the people who make them. I'm Dan Blumberg. I'm a product and growth leader, and on CRAFTED., I'm here to bring you stories of founders, makers, and innovators that reveal how they build game changing products and how you can too. CRAFTED. is produced in partnership with Docker, which helps developers build, share, run, and verify applications anywhere without environment confirmation or management.
[00:01:47] Docker's suite of development tools, services, and automations accelerate the delivery of secure applications. Learn more at Docker.com. And CRAFTED. is produced by Modern Product Minds, where I advise companies on product discovery, growth, and experimentation. Learn more and sign up for the CRAFTED. newsletter at modernproductminds.com.
[00:02:08] So how does BetterUp help organizations become more resilient, more creative, et cetera? I'd love if you could describe the experience of, of, of how BetterUp works, you know, both with hr, with the employees who are being coached and what the system is.
[00:02:22] Gabriella Rosen Kellerman : Yeah. So we help companies first get clarity on the job to be done.
[00:02:27] So let's say it's a merger of two large companies. Okay, well, they might say something like, we wanna build one integrated culture. What does that mean? What are the two cultures that are coming together? What do you actually want it to look like on the other side? Surely you don't want everyone across the business to be best friends with everyone else.
[00:02:44] 'cause that's not what's happening today. But there might be some core critical teams that we need to accelerate the relationships there. We need to accelerate the ways they come together. We need to give them greater skills on difficult conversations, et cetera, et cetera. So some of it is upfront just to understand the problem with the organizational leaders.
[00:03:03] Once we decide we're gonna apply coaching to a particular challenge, you go and you do a short assessment. And at the end of the assessment you get a readout that's really like, here's your strengths in this, these areas and here are areas of opportunity that based on what you have said, you wanna work on your goals.
[00:03:19] These are things that might be helpful to you. You match with a coach and then you schedule meetings with that individual for, uh, weekly, biweekly, and you start to work with them on those challenges. And then we. Reassess to help you see how you're growing, to help give you a sense of pride and self-efficacy and, and change and confidence.
[00:03:40] And also to help the coach really understand what's working and, and where they need to tweak the approach Now. With the data that we have at the collective level, we're able to give that aggregate vision back to the organization and that then refines the approach and refines the understanding of both the challenges and the job to be done.
[00:04:02] So in that example of a merger, it might be that one of the two companies that's being merged in is in a really tough head space and is stressed and a lot of people are. Thinking that they're not sure this is the right fit for them anymore. Right now, the job to be done might more urgently be the wellbeing of those individuals and supporting them through a transition.
[00:04:25] As a precursor to working on building trust with their new peers. Probably what's already organically happening is those individuals are working on that with a coach, and the system is, is designed to immediately respond to what people need. But it's really helpful to be able to use some of that baseline data to help organizational leaders deeply understand and empathize with the reality and the lived experience of the folks who are, who are now trying to make those changes.
[00:04:55] Dan Blumberg: Once you begin coaching and working with this person, how does the experience differ working with a BetterUp coach who's assisted with technology versus working with a coach that you might find anywhere else?
[00:05:05] Gabriella Rosen Kellerman : Yeah. The word coach is so broadly used and really anyone could call themself a coach. Um, that may be fine, just descriptively.
[00:05:14] That's true in the world, but it means it's very hard to know if someone that you're out there working with is the real deal or not. So. We do a lot to select the coaches, train the coaches to provide them with evidence-based tools and frameworks so that when you're working for them, this is, you know it.
[00:05:33] It's a high caliber of individual with a lot of experience, and they're going to be working with you in evidence. Based ways. Uh, so that's ultimately, it's not a waste of your time and, and so it doesn't do harm now once you're in the experience with them. So we have the data that we're using from the assessment that the coach is given, that then informs recommendations that are provided to them about just as we were talking about before, let's say that.
[00:06:03] You know, my goal is to work on my, my wellbeing and resilience and based on my scores, I'm low in X and I'm high and y. So the coach is going to be surface that and be able to go into this conversation knowing, okay, here's the strengths to help them be aware of, here's the areas of opportunity. There's ways that we will surface to the coach exercises and resources that will be most effective for me based on, you know, this.
[00:06:32] Massive population of folks. We've helped with this with different profiles and backgrounds. We lean into agency for the coach alongside the recommendation. So it's assisting the coach, but it's up to the coach to sort of veto and decide, no. This other thing that I, I think is gonna be more impactful, but we try to be assistive in, in, um, in ways that make it a lot easier for them.
[00:06:57] We've also begun some cool experimentation with smart messaging, both ai, gen, AI based and not gen AI based with sort of templates so that the messages that we send each other in between sessions can feel more. Pointed and can actually be really in service of the continued growth. And we've found that our coaches are really loving that and finding it so valuable and it leads to greater engagement with the people getting the coaching.
[00:07:27] Dan Blumberg: I'd love to understand, as the chief product Officer, how you build new products at BetterUp. And I know this is probably especially important to you as, as a, as a clinical researcher, how you measure the, the efficacy of, of, of new products. Could you take us through a a, a recent launch?
[00:07:41] Gabriella Rosen Kellerman : Sure. Um, one thing I will say about the R&D organization at BetterUp, which is very broad, is because we have a human in the loop.
[00:07:50] Um, because we're an evidence-based company, and because we also lean so heavily into things like gen ai, there's a wide array of expertise involved. It's not your typical trio of just the product manager, the designer, and the engineer. Um, there's a lot more cooks in the kitchen. And that's exciting for me creatively.
[00:08:11] And I think for those of us, you know, who work there and, and bring different layers of expertise to the picture, but it also can be challenging. Um, and it's challenging to keep the pace appropriate in, in an environment where you have so many different voices contributing.
[00:08:25] Dan Blumberg: Who are the other parts of that typical trio of product design and engineering?
[00:08:30] Gabriella Rosen Kellerman : Really depends. Um, ideally we have someone involved if it, if it's a part of the product that's going to directly, uh, involve the coaching relationship. We have someone who's an expert in coaching science, so not just behavioral coaching, but uh, these folks have degrees in behavioral science and often are coaches as well.
[00:08:47] And so we're, they're really layering in that. Both subject matter expertise and sort of lived experience in that role. We have learning experience designers, so this, these are experts with backgrounds and things like, uh, you know, education, but also in things like interactive content design. And so that is different than your, your, let's say, typical interactive designer who.
[00:09:11] Is less looking about information you're gonna retain long-term behavioral changes you're making is more about that moment of interaction. Assessment science is one function that's really important. There's a lot of fun assessments out there. Things like, which Disney Princess are you? And we can take them and we can read.
[00:09:29] You know, what comes after scientifically that's about up there with a horoscope. And, you know, it's, it's fun and it may lead to reflection and it may lead us to see ourselves differently and there's nothing wrong with that. But everything we do is evidence-based, which means that if we wanna assess.
[00:09:47] Whether you feel a sense of belonging in your organization, we're not just gonna ask, do you feel like you belong? We're gonna ask validated items of questions that we have psychometricians who've worked on. Does that actually measure what we think it measures? And then does it hang with all these other things that it should hang with?
[00:10:07] So that team. Works on the validation of those items. They also are working on ways to innovate on assessment because it's somewhat old fashioned today to just ask you a question, ask you for self-report. There's all science around validating your self-report answer, but I. We could also look at the words you're using to answer the question.
[00:10:28] We could look at the interactions you're having, the ways you talk about the people on your team, et cetera. So really trying to accelerate the assessment science, which feeds into everything because it's all about what's working and what's not working. We learn that largely based on the assessment that we do.
[00:10:45] So we have to be really thoughtful about whose input we need, when about when it's critical input. Often those teams of subject matter experts are working on sort of vertical areas that cross the business that will then inform future cycles of product development. So it may be, again, like a new assessment approach or a new toolkit around assessments and reporting data.
[00:11:08] That then will be used by tons of internal customers as they think about new products.
[00:11:15] Dan Blumberg: Not that long ago you launched a new product for group coaching and I'm wondering if you could walk us through a bit of how you launched it. You rolled it out, you measured it. Uh, I know it was particularly, uh, challenging 'cause it was during the pandemic when you rolled that out.
[00:11:28] Gabriella Rosen Kellerman : Yeah, so this is a great example of.
[00:11:32] Layers of mattering in an organization. So in COD, there was such a longing for togetherness. There was such a loneliness, and we had this overwhelming desire to just help and add value. It was the perfect moment, the perfect catalyst. To get something up and running around group coaching, and so we were able to go back into the archives of all of those projects that had happened over the years and really try to learn from those and, and harvest those and bring those to life.
[00:12:04] In this initial launch, we wanted to act quickly, so. Urgency was important and as a design constraint, we uh, therefore had to find a customer with a near term launch that was open to working on that with us. So we're a B two B2C company. This was launched for Chevron, actually is one of our, um, amazing customers and had a need in that moment.
[00:12:29] To really help bring together people from different parts of the company as they had shifted into remote work. And there was a sense of that isolation and loneliness. And there were, uh, a number of decisions that we had to make around simple things like how many sessions should you have, should it be a one-time group?
[00:12:49] And every time you go into group coaching, you're just with whoever's available at the time. Um, sort of like going to a, a workout group at a gym. You don't know who's gonna be there, you can kind of drop in or, or not. Should it be more like a class where you sign up and it's the same group that's together, you can't get in, what's it starts and you're committing to staying there?
[00:13:10] And in that moment, what we felt was that more than anything else, it was really about developing relationships with others at work. And so it was important to design around repeated, lengthy interactions with peers that turned out to be incredibly effective. And that was. That was key to the success.
[00:13:29] Some of the early learnings there was too much content in the first. It felt too much like a class, so we dialed that way back, made it a lot more about the interactions between the people and the group coming together. So from there, the fast followers were really dictated by through this lens of what's gonna connect people most within the group.
[00:13:49] So messaging therefore became a higher priority. It was one of these cases where working in a mission driven company does feel very different because we were rising to our mission as a moment of urgency, but it turned into a very commercially successful part of the product and, uh, continues to be something that is very valuable and that we bake into different aspects of the portfolio of products depending on the use case.
[00:14:17] Dan Blumberg: That's awesome. You mentioned you're using generative AI in a variety of different contexts, and I'm interested to know, are you using it? Is it in full scale production right now? Are you more running lots of different experiments and prototypes? I'm curious how you've approached that technology and implemented it if you're doing it at scale.
[00:14:33] I.
[00:14:34] Gabriella Rosen Kellerman : So AI in general has been always part of what, what we have used, and it's what determines the coach matching algorithm and what you know determines the recommendations on the resources that coaches are offering. Some of the ways that we're using Gen AI in the product today is a generative AI explanation.
[00:14:53] When you get to see the coaches that we are recommending to you. So the way it works is you go through the assessment, then we recommend to you a certain number of coaches. You choose one, but they've all been surfaced by an algorithm for reasons. The algorithm knows. So we actually use Gen AI to explain to you now for each of those individuals, why did this person get surfaced for you?
[00:15:17] So I think it's been a way to help people more quickly get to the heart of what one coach can provide them versus another, and, and build trust and confidence in the platform. We have a ton of experiments happening though with Gen AI way more obviously than what's in production, and there's so many different ways that it could possibly help our coaches, help the people getting the coaching, help the partners who are funding the coaching.
[00:15:46] So we feel like kids in a candy store. We have some incredibly talented folks working on it. The sentiment about Gen ai. Has so much to do with whether we'll adopt it or not. It has so much to do, predictively with how impactful it could be on our development.
[00:16:03] Dan Blumberg: Yeah. What are the hurdles that it has to clear to go from the many, many experiments the kids in the candy store playing to, you know, okay, this is in full scale production of, obviously cost is one factor, but I'm, I'm, I'm interested in what, what are some of the things that you and the, the team, the Trio Plus look at?
[00:16:21] You know, when you decide, okay, this experiment is, is, is go time, like this one, this one launches.
[00:16:26] Gabriella Rosen Kellerman : I think with Gen AI, because it's so new, there's a unusually high threshold in terms of data privacy and ethics and, and those components that, uh, we have to be able to clear with enormous confidence. So that's one piece that, you know, when you, when you launch a redesign with a new color palette, that's not gonna be a factor, right?
[00:16:49] Um, and it really, really is with gen ai, especially when you're dealing with people's very deepest hopes and dreams Yeah. Um, about their future. Beyond that, though, as I mentioned with coaching circles, we are very opportunistic and looking for business needs where Jenny, I can solve problems quickly for us, and so we're more likely to launch something sooner if it's aligned into.
[00:17:11] A product challenge related to an urgent business need, and that's been fantastic. Recently as a, a brand new tool in the toolkit, we are now challenging every single product squad to be thinking about how to bring Gen AI into what they do. We have experts. We have teams that really. Devote and focus on it, but we want it to be much more of, kind of in the soil and, and just in the environment for us as a, as another set of tools that we could offer on any given product line.
[00:17:42] Dan Blumberg: You joined BetterUp seven years ago when there were just 20 people and the company now employs 800. There are 4,000 coaches. How have you scaled BetterUp, and, and I'd love to hear about some of the, the growing pains and also how you as a company at BetterUp have enabled the qualities that you also help other companies foster.
[00:18:00] Gabriella Rosen Kellerman : Uh, such a good question. Um, how have we scaled better up? I mean, I think the way that most high growth startups have scaled, which is, uh, bumpy. You do it, hold on for dear life, and you hold it, you do it as, as well as you can. Um, we had a lot of people hired during Covid, uh, and so a lot of companies did that, but for us, proportionally, it was such a huge proportion of the company that we have had to really lean into.
[00:18:27] Getting together for, uh, offsites in person. We've done experiments where actually your, your onboarding was almost like a retreat type of experience with other new people onboarding, but also with existing employees that you felt some sense of connection. But I think that that was probably the biggest challenge is how quickly the proportion of new people who hadn't met other people grew to be such an overwhelming part of the experience of.
[00:18:55] BetterUp was people had never met each other in person. And for a company that does what we do, that's challenging. So intentionally overcoming that and we, we devoted a fair amount of funding to trying to get people together. I think the other thing that's been interesting about scaling BetterUp from the internal employee perspective is that because of this sort of symphony of players on the r and d side, figuring out.
[00:19:19] Do we scale those proportionally to each other? Where can we slow down the scale of one function? Because it's actually learning how to service other functions at different proportionality. So you need, you know, say one assessment scientist for 10 product squads, or are you always gonna need a different, a different ratio than that?
[00:19:41] And there's no recipe, there's no playbook for that. So figuring that out has been, I think. Challenging and also quite fun.
[00:19:49] Dan Blumberg: Yeah. What, what's next for you? What's next for BetterUp?
[00:19:54] Gabriella Rosen Kellerman : So for BetterUp, uh, I would say on the product innovation side, we are on a multi-year journey to going from the product I described to you, which is a, uh, our signature product and will always be our signature product of this high touch one-on-one coaching to products that are more highly scalable, meaning.
[00:20:13] Lower price point still has human coaching involved, but more leaning into gen AI and to the technical possibilities for digital development. And so couple of products already out in the market, many more in development. This is going, what's gonna let us, you know, continue on this journey of democratizing access to coaching at a a price point that more and more and more people around the world will be able to afford.
[00:20:38] Dan Blumberg: I wanna circle back to the whitewater world of work. Having to constantly be ready for rapids and whirlpools, you know, can be exhausting, can also be a lot of fun if you have agency and you know, you're just being swept down river. What is something that makes you optimistic, uh, about today's world of work?
[00:20:53] Gabriella Rosen Kellerman : I. I love this idea that compared to the industrial era, compared to the agricultural era, which were really about greater segmentation and automation of work, that we are now in a place where you have to be creative to do your work well. And that was so much a part of being a hunter gatherer, is this dynamic relationship to the environment and being resourceful and.
[00:21:18] Having all kinds of skills depending on the particular landscape you are in and that level of dynamism and generalism and creativity. I think it's very rehumanizing for us and I think that it, if we can lean into that and see these as opportunities to broaden our skills to grow and try new things. I think that we all do need to learn how to tolerate this nausea, how to tolerate the, the fear that comes with sudden drops in the white water, and to trust in our own skillset and to trust that the landscape around is gonna change again and things are gonna fall in a different way.
[00:21:58] And we'll be there and we'll be ready and, you know, there'll be another amazing swirl of momentum ahead. Um, yeah, that excites me. I think about the fact for me personally, that the role I'm in. Didn't exist 20 years ago, you know, when I was leaving college, um, and planning my future. So, and I love the role that I'm in, and so what else might lie ahead that I cannot imagine that will be exhilarating for all of us.
[00:22:29] Dan Blumberg: Awesome. Well that, that sounds exciting. I look forward to all of that. Thank you so much, Gabriela.
[00:22:34] Gabriella Rosen Kellerman : Yeah, great to be with you Dan.
[00:22:39] Dan Blumberg: That's Dr. Gabriela Rosen Kellerman and this is CRAFTED. Sign up for the CRAFTED. newsletter and check out the show archive, including our first episode with Gabriela on the Tomorrowmind Skills. We all need to thrive in today's whitewater world of work. You can find that at modernproductminds.com. CRAFTED. is produced in partnership with Docker, which helps developers build, share, run and verify applications anywhere without environment confirmation or management.
[00:23:07] Docker’s suite of development, tools, services, and automations accelerate the delivery of secure applications. Learn more@docker.com.
Special thanks to Artium where I launched CRAFTED. Artium is a next generation software development consultancy that combines elite human craftsmanship and artificial intelligence.
[00:23:27] See how Artium can help you build your future artium.ai. And please share this episode with a friend because if you do, studies show, you'll be happier and more successful.
[00:23:38] Gabriella Rosen Kellerman : Everything we do is evidence-based.