Subscribe
Share
Share
Embed
Testing your ideas against reality can be challenging. Not everything will go as planned. It’s about keeping an open mind, having a clear hypothesis and running multiple tests to see if you have enough directional evidence to keep going.
This is the How I Tested That Podcast, where David J Bland connects with entrepreneurs and innovators who had the courage to test their ideas with real people, in the market, with sometimes surprising results.
Join us as we explore the ups and downs of experimentation… together.
David J Bland (0:2.222)
Welcome to the podcast, Pat.
Pat Copeland (0:4.376)
Thank you. Thanks for having me, David.
David J Bland (0:6.944)
I am so excited of all my guests I've had so far. I don't know if I've been this excited to have somebody on because you have such an extensive experience in testing and innovation and leading companies through that, that I am just excited to pick your brain and have you share some stories with us of what went well and what didn't go well. If you could just give a little bit of our listeners a background who don't know about you, of sort just a little snippet of your career and what you're up to at the
present time.
Pat Copeland (0:37.612)
Well, first thing, thank you for having me. I'm flattered. And, you know, I think of myself as kind of a entrepreneurial thinker that, that tries to do projects that have a high beta associated with them. And I've kind of factored my career, even if I was working in large companies like Microsoft, where I started or Google or Amazon, trying to factor some of that, my energy towards things that
hadn't been solved yet. So I'm not the kind of person that has worked on, know, been a database person their whole career or somebody that is really interested in one narrow part of computer science. I've generally been broad and really focused on trying to figure out how to innovate and make things valuable to the customer and spend a lot of energy thinking about uh iterating my way.
to better and better customer focused products. And I think that that would probably be the thing that I'm most focused on. um Let me pause there.
David J Bland (1:49.424)
Yeah, I mean, I think it's fascinating that you take this almost like customer first approach of, know, let's focus on, I think you called these high beta, maybe you can clarify for our listeners what you mean by high beta, because I don't know if they're all familiar with that framing.
Pat Copeland (2:6.488)
Well, you I started my career kind of running into the burning house and putting out the fire, you know, and being the person that could come into a project that was, you know, not really on track and fixing it, fixing the team, fixing the product strategy, you know, making things work. And so I think that that put me into a mindset of being okay with risky things to do. uh You know, and a lot of times I was working on a
on very successful in working in very successful companies that had, you know, they're basically like a battleship and I'm on, you know, deck number 13 of the battleship. And, you know, my project is not going to make the battleship sink or succeed, but, know, potentially if I'm really successful, it could be something that's notable. uh So, you know, given that safety net, I decided to kind of do things that were higher risk. And I would say, but I mean by beta are, you know, I.
have taken on projects where it was okay to fail. And there was a lot of upside if there were success. so, you know, many, many projects are kind of like this if you seek them out. And certain people like to do them and certain people really don't. uh I would say that my career was kind of, uh you know, had its ups and downs where I didn't really have a plan, but I kind of pursued larger and larger opportunities where I could take on
you know, interesting challenges that were technical or, you know, trying to solve for a customer problem. Um, and, and I think that that mindset drove me to things that were with higher risk and then with higher risk, required better focus on iteration and trying to build confidence or not build confidence based on, you know, testing the context of where you are to step forward and, and move forward with a better and better product idea.
So that's what I mean by high beta is just the high risk factor uh projects that maybe aren't set up to succeed necessarily where there's no safety net. And it requires an iterative thinking process, sometimes nonlinear in order to get to that successful product.
David J Bland (4:21.188)
I like that framing. I'm always fascinated by why people get into this line of work and how they perceive risk and how they deal with large amounts of risk. And I have to say for me, it's almost like personal, some personal challenges I have is I'm risk averse and I have a hard time dealing with lot of uncertainty. And I feel as if it's like personal growth for me to be able to coach people through it using like a lightweight system, you know, or a way of thinking.
And it sounds as if, you know, initially in your career, you were able to navigate and maybe save some projects that were doomed to fail, or I'm sure we weren't able to save all of them, but really be able to course correct some things. And I do a lot of work in Silicon Valley and I've noticed when I go into big companies sometimes, they're like, well, we need entrepreneurial people. So we have to look outside the work and hire, you know, hire external people. And I'm always, I'm always hesitant at first, you know.
uh And usually my response is, why don't we look at some of the people that have been entrepreneurial inside your company, even though maybe that's not the title you gave them, but they were able to walk into situations like you walked into and use their creative thinking skills and manage risk to help things move forward. I'm curious what your thoughts are on that or what's your take on sort of external versus internal and how do you look for people that are kind of like you?
Pat Copeland (5:44.590)
I think it's a really good point. know, entrepreneurial thinking exists everywhere. I actually wrote a article about this for IEEE with a guy named Alberto Savoya back at Google. And I forget exactly what the title of the article was, but it was something like entrepreneurial product thinking. And it asserted exactly what you talked about. And I think Google did a pretty good job of this, actually, of
figuring out how to unlock everybody's potential to be an owner and think outside the box. And so there was this idea that anything was possible there, at least when I was there, it was the kind of company where there was kind of limitless opportunity. we, you know, we were in the position of, you know, the Brinks truck would, would back up to the building and we'd be like, where do we stack all these gold bricks? We just don't have enough room for it.
And so there was a lot of room for, you know, kind of doing side projects and innovation. And uh there was, you know, there was a lot of entrepreneurial thinking. There was actually a database that had hundreds of thousands of ideas that employees had, you know, and it just kind of shows you the wastefulness also of large organizations, you know, the ability for a large organization to harness some of that stuff is, I think it's a lossy conversion.
partially because their eyeballs are so big, you know, like at the Google level, you would need to create a billion dollar business for them to really care. And if you created a $10 million business, they don't even want to do anything. And that externally, that might have been, you know, an incredible success if you created a hundred million dollar company. I mean, you are among the 0.01 percentile, you know, of successful entrepreneurs, but it...
at Google, that would have been something they wouldn't even have, they wouldn't have bent over to pick up that, you know, a hundred million off the ground. So, you know, I think that a lot of it is the context that you're in, you know, either encourages or discourages entrepreneurial, um, you know, success. And some of that is by, you know, the level of investment and the willingness for a company to try new things. And so that I do agree with your premise that.
Pat Copeland (8:6.262)
entrepreneurial thinking is everywhere and that you should look at people that are good at it in the context that you're in. Because a lot of times I think entrepreneurs, like if you look at them in isolation, you say like, wow, they were really successful, but like, what was the context that made them successful? And somebody external, they had a lot of freedom potentially. You know, obviously they had investors that were telling them, you know, kind of things to do and hustling them a little bit, but.
In general, like a CEO outside would have a tremendous amount of freedom to hire who they wanted, to run their process like they wanted. When you get into a company like Google, like there's all kinds of infrastructure around you because of their scale that they require you to align with. And so I think some entrepreneurs, excuse me, will come into a place like Google and be like.
Why are you clipping my wings? Why are you doing this to me? I can't succeed in this nightmarish Byzantine situation. And to the people inside the company, you're like, what are you talking about? This is like a paradise. So I think that context is really important and the type of risk taking is also very different. Like I was trying to say earlier about the difference between the way that you would evaluate whether to invest or not invest. So.
You know, when I think about working at a big company, I think that how do I unlock the people that I have is my primary option. And, you know, trying to replicate somebody's ability to do entrepreneurial stuff in a different context, I think is a, you know, kind of a lossy conversion. There's potential there for sure. And some people have been, you know, serial entrepreneurs and they have a track record. And I think that that's one way that you look at this is like, who is good at getting things done?
within a particular context and let's double down on those people and let's listen to them because they not only do they have experience, you know, being able to do entrepreneurial things and it's, you know, like you're pointing out, it's not just about the idea, you know, like I think entrepreneurialism is also a lot, maybe 99 % execution of that idea and working your way through whatever situation you're in. So I would say entrepreneurial,
Pat Copeland (10:24.418)
you know, people are ones where you can see that they've been successful, but the context is really important. And, you know, how did that context help them or hinder their ability to, you know, deliver that innovation is something that I would really look at.
David J Bland (10:41.136)
Yeah, I think a couple of things one I wasn't aware that you wrote a paper with Alberto so I'll have to look that up I reached out to him to quote him in my book and uh I'm a fan of his work just in general, the pre-typing and everything and so doesn't surprise me I guess but I learned something from that answer so I'm gonna go check that out and to with regards to people it was a big learning curve for me because my thing is let's assemble sort of like a small cross-functional team around idea
and let's iterate to find out is this something we should pivot, persevere or sunset. Like really adopting from like lean startup and my early involvement in that movement and design thinking and agile and all this stuff mashed together. And what I learned was kind of insightful because I thought at first, well, of course these people are gonna stay with this idea and grow it and scale it if it's successful and we have traction. And what I realized instead was there were people who would opt out right before the scaling sort of inflection point and say, look,
I don't want to scale this. That's not something I'm passionate about. Do you have any other ideas that I can work on? And so I think naively, I thought, well, we're going to have these stable cross-functional teams that are just going to scale with the idea. And instead, I ended up realizing that we have this kind of split from a personality perspective, where some people, yeah, they would stay and they believed in the idea and they wanted to scale it. And others would just opt out and say, OK, what do you have next for me to test?
Pat Copeland (12:5.770)
Absolutely. I personally call that trading breadth for depth. And if you're successful, you're gonna start to scale that business or that team a little bit. You're gonna have larger administrative overhead. You're gonna have more complexity because you're gonna have to deal with more customers maybe, or more situations, more of these cross-functional issues that start to emerge as you scale.
And you really have to decide like, what do I care about? You know, and I think that there are people who love the idea itself and are willing to stick with an idea. And some people really love the idea of going from zero to one or one to a hundred. You know, they're, they're the growers and. know, um, I, I personally have done, you know, made that trade off many times where I've taken, you know, uh,
teams that were in the hundreds, hundreds of people team and I've given them away completely in order to start from scratch on something that I believed in, that I thought was uh interesting to work on, maybe an area that I was curious about and taking that risk of just doing something fresh. So I definitely feel like there've been times in my career where I've done exactly that uh thing of moving on once the thing kind of takes flight.
Um, and there's other times where I stuck with it longer, but I, I can, I can definitely understand the mindset of wanting to move on because there's definitely a dopamine and kind of uh a, excitement around doing something new that is completely different than, the refinement exercise of taking it to the next level. And, it really does require like a dedication to.
you know, different types of mindsets, know, ex people that do op, you know, that operate teams for long periods of time. I really admire them in a lot of ways because it's something that I can't really do. I'm not really geared up to be, you know, and I, it sounds weird because I worked at places like Microsoft and Google for literally a decade each. But during that time, I felt like I went through three or four different companies, even within there. And I moved around quite a bit seeking.
Pat Copeland (14:22.818)
you know, uh, the kind of like oxygen, you know, in a cave, you know, you're, you're, you're kind of trapped in this cave system. And it's like, where do I find oxygen so I can continue to survive? And there, there are periods of time where you'll, you know, kind of like, uh, consume all of the air in one area. And you literally need to go to some, some other place in order to refresh yourself. And being willing to do that is, is like this risk taking thing.
that maybe it's slightly dysfunction, but you kind of need it in order to feel like that you're making progress. So I think it does really require different types of mentality. And I've done both so I can understand both. I don't know if there's a DNA that is entrepreneurial because you can switch it to last longer in a job. It's like have patience. But there is an impatience that exists among most entrepreneurs or innovators that is a little different.
And you can see it in the way that they're kind of always pushing and kind of uh trying to drive their idea further. The grind of an innovator is slightly different than the mentality of an operator or a maintainer.
David J Bland (15:39.504)
And I can agree with that. think shifting gears a little bit here. So how we got connected was on LinkedIn. And uh for those of you that don't think LinkedIn is valuable, I think this is maybe a counterpoint to it. I wrote an article on Amazon Dash Button, and I wrote this article a few years ago. And I was just fascinated by the overall like idea of testing something that's more on Amazon. It was different. It was away from screen purchasing, all of that. And you commented on it and
I almost missed your comment because the downside of LinkedIn is I don't get updates on comments or I get hundreds of comments on something and I just everything gets lost. And I just happened to go back and check that post and I was like, who's this guy? And then I realized like you actually worked on uh the dash early stage dash at Amazon. And so that was my initial reason to reach out. And then I found out so much more about you. So I'm just thrilled to have you here.
But maybe we can dive into that a little bit, whatever you can share of where did that idea come from? How were you involved with it? Maybe some of the tests. I'm just happy to dive into that a bit if you're willing to share with our audience.
Pat Copeland (16:53.262)
Absolutely. Yeah. And well, and thanks for reaching out to like, you know, LinkedIn. I'm glad I'm glad that we serendipitously met that way. um Amazon is actually a really interesting kind of entrepreneurial environment and they really encourage this. They encourage all employees to write up new ideas and it's a very kind of document centric culture where you kind of write down your ideas and then those go through a filtration process and you kind of manage up and
you end up kind of arguing for a thing to do. And they call it a PRFAQ. uh It's a way of, when you think of a press release, so you start off one of these press releases like, the New York Times today said that Prime Surprise Suites was launched and.
you and you kind of write it from a future perspective. And so you're in the position of writing it retrospectively after the thing was successful. And it kind of puts you in that mentality of like, well, why was it successful? What did we do? Like who were our customers? What did our customers do when they got this product? Like, why did it, why did it make a, you know, an impact? And then the, the FAQ part is where you, kind of ask yourself a bunch of Socratic questions that, you know, poke at yourself like,
Like, what are you going to do about, you know, chocolates that melt in the mail or what are you going to do about like always surprising people with this product? And then you try to answer those. And so that's kind of how Amazon's innovation machine works. And it really works for them, but you kind of have to take their whole process in your arms. If you want to work there. I mean, it really is a process of getting innovation to go, but once it's committed to Amazon will
grind its way through and support you with resources and like incredible internal technology. So I felt like it was one of these places where if you could work that machine of Amazon to do these things, it was amazing. So the way that Prime Surprise Suites came around, which is like, it was the, not only the dash button, but the very first dash button. And so,
Pat Copeland (19:7.404)
The dash button was a thing that AWS came up with, which was like an internet of things device where you could push it and it would just do, it would execute a Lambda function in the cloud. So you could basically make it do whatever you wanted. And so, you know, I bought one when it first came out and I wasn't even working at Amazon at the time. And I just had it send me an email, you know, when I, when I pushed it and it was funny because I was like rummaging through a door, a drawer the other day.
and I pushed it and it sent me an email like, you know, six years later, like a ghost, you know? So they work and they're, you know, they basically work by connecting to your wifi and they send some event and Amazon picks up that event over the internet, makes a Lambda call and then does whatever you told it to do in the cloud. So it's a cool thing. Now the question is, why do you do with that button? You know?
What is it good for? Like, why did we create this thing? And so they were kind of struggling in AWS with an application. Should we make these things that you kind of, know, could, could it be a doorbell? Could you take a picture in uh a camera type situation in a store? Like, what would this button do? And so Bezos came up with this idea. And so,
Like Bezos is uh Jeff Bezos, obviously CEO of Amazon for people that don't that live in a cave. He is, you know, very entrepreneurial and risk taker himself. And so one of the things that he likes to do is he calls it like putzing around in the morning. And so he just kind of like has time in his morning where he wakes up and he just kind of takes a shower or whatever, has breakfast like, you know, and just does whatever he wants to do during that time, apparently.
He came up with this idea of, hey, let's come up with some kind of cool consumer based product that when I click the dash button, you you could get a box of treats from it. And so the idea would be you would buy this thing, this for, you know, it was like for 4.99, it was like a $5 thing. And we would give you the dash button. And when you pressed it, would place an order for these boutique sweets.
Pat Copeland (21:28.130)
you know, these, they, and they, why would they be boutique sweets? Well, we could send you a candy bar, but that would be kind of lame, you know? So could we think of something that would be a lot more exciting and it would be like a surprise box of like really nice boutique treats. So that was the idea. And, um, and so he kind of handed that idea to our team and then we were responsible for, you know, developing that into, well, what would it really be? You know, so you, you, you look at these things that if they come down from top down,
And you, kind of interpret them. You look at them and you say like, well, would this really work? You know, mean, Bezos, this is like a, maybe a three hour thought that he had and like, are we going to spend our lives like building this thing? And this team is dedicating their careers to make this happen. It's like, do we really believe in it? And how would we make it work? And so there is a level of innovation there, even though the idea is, you know, interesting, was like, how would you implement it? What would the operations look like? Where would you source these treats from?
How would you send them to the customer? Like how often would you accept a dash button being clicked? Could we turn this into a thing where you could buy it for grandma and then you would pay for grandma getting treats every month? Or could you give it as a gift? These were the types of things that we had to work through. And so it really became kind of a, where could we take this idea further? And given that kind of the starting point, it gave us kind of a blank slate.
of being able to do a lot of different ideas. And there were no other dash buttons in progress. And so we were also kind of pioneering the idea of, well, what do we do with these dash buttons? And can we further the idea of what dash buttons are for? So I'll pause there for a second. I said a lot of things, but that was why it started.
David J Bland (23:19.652)
Oh, I could listen to this all day. ah So a couple things. One, I'm learning from your stories anyway, because you know, I do all my research and everything, but I'm literally talking to somebody who's been there. So I've, I've framed this almost. So in my work, and some of my work shoes that Amazon paired with this sort of press release working backwards format, which is I like what I learned from dSchool, which was like desirable, viable, feasible as far as risk goes. And it almost sounds like, yeah, it was
Feasible as far as we think we can have a button granted you have to work out the operations and obviously limit, you know people just pressing it all the time and all that but it was more of a Sort of like desirability risk with who? And what's the value prop to them and what would they use it for and the gains they would get from it? And then maybe even a little bit of viability risk, which we haven't touched on yet, which is the cost incurred
You mentioned a price point for getting the button out to them and everything. But it almost feels like, you know, so many of these ideas that I hear at other companies kind of go to die when they trickle down because the team really have a hard time sort of framing that risk and working their way through it. And I don't know if that's exact framing you used, but it sounds like you were thinking through the right kinds of questions to ask as far as.
how would we implement this in a small way to get some traction?
Pat Copeland (24:47.926)
Yeah. And the other thing to think about is why we would do such a project because, we were talking about before of this is going to be a rounding error, you know, in the grand scheme of things. I mean, compared to, you know, Nike shoes or something like this type of a project is going to make pennies and why would Amazon do it? Well, the idea was, could we test the habitual behavior of a button being pushed by a consumer and, and
could we also test the idea of closing the loop with that button being something that causes an order to be processed and a transaction to be done? so Bezos' idea was, as a first step, could we create a project where the end value that's produced by the consumer is such a cool thing that there's no friction in wanting that thing? Take away the friction of like,
Everybody likes treats, right? Everybody likes cookies or whatever. And so there's no friction in like figuring out what category is of interest to people. And there's no worry about like, you know, the, the, the desire. So the desire of the consumer, you know, was hypothetically there. And that the thing that we were trying to validate was would people push the button a second, a third, a fourth time. And with that habitual behavior, like why would you think about pressing the button? Let's say.
let's say several weeks have gone by and you've, and it's like, Hey, I want some treats. You know, would you run to the button and push it, or would you go to the store instead? So the idea was if you could prove that, then you might be able to do other things with the button. Like for example, replenishment of, you know, a consumable like detergent or something, or, you know, buy paper towels or, you know, so those were the ideas. It's like, this was a precursor idea.
to try to gauge customer intent to purchase, not only intent, but actual purchase. you know, and if, like, if you're a fan of Alberto, like, you know, he'll say like, people, you know, one of his good quotes is like, people are more than willing to open their mouths, but they're not very willing to open their wallets. And, you know, that's absolutely true in innovation. So like, when you do these kind of surveys of consumers and you're, you're trying to build a product,
Pat Copeland (27:11.404)
And you go and you talk to people. And even if you go to talk to big enterprises and customers, they'll say, yes, yes, I'm very interested in whatever you're doing. And, and yeah, we would be, you know, uh, really curious to participate in it. And then it's like, okay, well, this is how much the investment is going to be. Oh yeah. We're not that interested. Um, and so the idea was, could you really test the consumer, uh, full cycle of purchase and have skin in the game, um, where they would do this? And so, you know,
The idea was there, it was really interesting, and we weren't expecting it to be a commercial success in a sense. What we were doing is trying to uh solve a concept, a hypothesis, that uh customers would use these buttons to buy things that they were interested in. And then from there, maybe we could inspire other teams to do other things. And I also think, if I give the project some credit,
and go beyond, you know, what it was. mean, it was like a treat box. It led to some things like, if you think about how Alexa does prompting for reordering or, uh you know, how that happens on the site. I think some of those suggestions and those nudges are, they figured out how to make them a lot more effective. And the button,
kind of requires you to make, and this is something they talk a lot about Amazon, is how do you decrease the cognitive load of a problem? uh so cognitive load meaning, okay, I need to do some task, and the cognitive load, if I have to think about, am I gonna be home when that delivery is made, and where's my credit card? And I have to put that credit card in again, and oh, you know.
Like I have to go through and do this form to put in my address every time that I order something. So Amazon is like, okay, let's just make it friction free, you know, and let's reduce all cognitive load to the smallest thing possible. So that is something that they really focus on. So if you notice on Amazon, like the ordering process is just very friction free. You you can pretty much click a button and buy something now. And that was the concept. So how do we reduce cognitive load of
Pat Copeland (29:33.120)
intent to purchase to, I received a box two days later. And, you know, it's an interesting concept. And I think that they've continued to evolve and think about how to do that in a, you know, much better ways than using a dash button, but circling back to the dash button, the cognitive load of, I want a treat to where is that dash button? Where did I put it? To, I'm going to click it. And then thinking like,
Did it actually order something? I'm not sure. uh What did that button just do? I don't know. I pressed it and maybe some magic is happening somewhere, but I don't really know. uh Like that cognitive load was too high in general. And so like the product was kind of set up in a way that was never gonna be successful. And I think that was one of the primary discoveries is, you know, the habitual usage wasn't there.
And I could get into like other things like the operational problems that existed were tremendous as well. Like there were melted chocolates, like, and you think like everybody loves treats. Everybody would love a box like this. There would be no problem with people getting a, you know, prime surprise sweets in their uh mailbox. But you think about like the things that you, do you like coconut? You know, do you like coffee flavored chocolate? Do you?
you know, are you, you have a nut allergy? Do you, are you gluten free? Do you have a peanut butter, you know, aversion? So all of a sudden, like this world of like, oh, everybody loves sweets too. No, no, no, that's not true. Everybody loves some kind of sweet. And like that diversity, like you start to have to narrow down stuff that's going to work for everybody. And that becomes a very, very small subset of stuff that you can send through the mail that will not be destroyed through the mail.
you know, and will be generally loved by people. So that was the other side of it. Like just the target of it was difficult to be successful with a broad enough audience. So we ended up kind of having like a very niche, you know, kind of uh a peculiar group that were that loved the program. I mean, they would like blog post about it and they would have unboxing ceremonies and, you know, all this stuff. But it didn't really have mass appeal.
Pat Copeland (31:54.286)
uh because of the cognitive load on the one hand, and secondly, the logistics issues of being able to fill a box with things that a broad population would appreciate.
David J Bland (32:7.457)
That resonates with me. There's so much amazing things to unpack there. So with the sweet thing, uh I wouldn't have guessed either until I was at a startup weekend. think we were at somewhere in Silicon Valley and Gagan who uh he was like helping Udemy and now he's doing Maven, but he had this food kind of startup in the Bay area called Sprig. And uh they offered this little treat like a truffle as an extra thing. And they did it for free. They were just an extra thing.
And the backlash they got from it, from their customers was just unprecedented. They couldn't have predicted how many people were complaining about this extra treat. Did I pay for this treat? I didn't want this treat. I'm gluten free. Like all these different things. And so when you were talking about that, I remember talking to him and he was so surprised too of it looked like a quick win, something obviously people would want. And then you get down into the details and it's not as simple. But the other thing I was thinking about when you were talking,
It feels as if, and maybe you can help me understand this, you're really, in a way, we're testing away from screen purchases. And in a way, even though maybe this, although that button does have a cult following, to this day, I see people writing ways to hack that button and still have it do stuff in different ways. But in a way, it almost feels like even though it didn't have a monetary impact.
as its own thing, you influenced Alexa strategy and so many other things inside Amazon just by proving this out with evidence and just kind of being iterative about it.
Pat Copeland (33:44.748)
Yeah, I mean, I think that that's definitely true. And I think that that's why Amazon's model works is that you create these artifacts and these case studies in a sense that have sometimes beginnings and ends. And then other teams will riff off of that. So once Prime Surprise Suite's kind of had its uh ups and downs, like other teams looked at that and said like, oh, well, we're going to create branded buttons.
And so one of the focuses was on all of these, know, um, consumer focused brands like Tide had a button and, uh, Bounty had a button. And then you would have like a series of buttons that you would stick on your wall next to your dryer, maybe in your, your utility room, and you would press it when you need it. But again, the problem with that, you know, out of screen, you know, out of device purchasing was the cognitive load of like knowing that
Uh, when I run out of paper towels, I go to the button and not to what I've been habitualized to do, which is go to my device, which is what's, what's the barrier to entry between those two? It's like getting up and walking to, know, into another room and clicking a button seems really simple, but like the device in my hand is like so much more powerful than that button. And I know it gives me affirmation after I press it, that something happened. they're.
there is like a cognitive load gap there that is somewhat uncrossable that, uh with Alexa, I think that it offers me suggestions for things that I need to replenish. And then I have to actively engage and say, yes, I want that thing. So maybe if there was a return value on the device, it would have been slightly different. But it's hard to say. I don't know if that off screen button
is, you know, it just, I would just say like, would, I would do my own uh editorial and say, I don't know if that was successful in any of the contexts that Amazon tried, but you you never know. You never know. mean, there may be another way of doing it in the future that does something that, that is instantaneous and has high value and is measurable, you know, and, and people love reinforcement. You know, they love to see that a task was completed.
Pat Copeland (36:5.742)
You know, and when it's kind of that open ended, I'm not sure. did the batteries run out already or did it actually place the order or when's the order coming or what did I pay for that order? Did bounty change its pricing or, know, and when's it going to come? Um, like all of that stuff was a question mark with the button that, you know, the device was able to do. So maybe there's a application that would be killer for it, but we were unable to come up with it. But, but the thing, my, I guess my point is.
Other teams did riff off of it. Other teams did look for, you know, off screen, um, applications to shopping. And there, think that some of them have been more successful than others, especially things like replenishment, I think have been really, really good on Alexa, where if they can give a very, very targeted suggestion, you know, and the ones that I've been getting lately are things like, Hey, you need to replenish this or.
some price drop has occurred for something that you were interested in. Do you want to buy that? And those are interesting nudges that sometimes I'm willing to take. But yeah, it's interesting. think that, you know, customers have a very, very keen interest in getting feedback. And like, think shopping is also kind of a, you know, it's like a, it's a dopamine producer when you go, it's like we're,
know, genetically predisposed to go and forage for things. And when you make a discovery, it's like, you feel good about yourself. You you did something positive, you know, you made a progress on something and the dash button was like halfway to that, you know, but it wasn't a full closure.
David J Bland (37:47.352)
Yeah, I could see that. I just, can't say enough about the impact of something like this, even if it's not from a viability point of view, you move the needle in some meaningful way with that specific initiative, just the ripple effects of how it impacts other teams. And I think that's why we want to socialize this kind of work inside companies. You quite often I'm liking a lot of our work to almost like scientific method where we're, hey, we have a hypothesis, we're to go test that. But I think socializing
what has happened, the wins and the failures. And then so many teams I coach, that first thing might not be successful, but if they learn this way of working, they're going to apply it to something that is going to be a big success eventually. And I think I often take that approach too of I'm playing the long game here of I'm really investing in people and their critical thinking skills and how they iterate through things. And um I'm finding that if leaders can create a culture of experimentation or an environment where that can occur.
then that's really special. I think sometimes our iteration, and I wrote a LinkedIn rant about this a few weeks ago, it feels like the thinking's removed from the iteration and the iteration is just about delivery. But I think your story here really emphasizes that, no, we need the critical thinking skills, especially in these high uncertainty, high beta initiatives.
Pat Copeland (39:6.370)
I totally agree. I mean, thanks for bringing that back to the team because, know, speaking of the team, you know, how would I describe them? You know, I actually described the guy who led this project uh on my team. I don't know if I should name him not, but Dirk, I called him a swashbuckling pirate, you know, in his review. And that's how I kind of envisioned him is he's the kind of guy who
is willing to take on mission impossible and to try and iterate, you know, at all costs. And he went on to do other things after this project. And so did a lot of the team. actually repurposed this team to do another entrepreneurial project around doing uh unsolicited samples. And so, and the key being unsolicited. like a lot of times you get a sample at a store, you actually pursue that sample. Like, you know that that sample
is being provided to you and you go and you elect in, well, could we be proactive with those at Amazon and say like, hey, based on your shopping, you we think that you would like this. And so we actually created a whole program and we sent out millions and millions of samples, you know, things like shampoos and toothpaste and, you know, all kinds of things. But, and now he works at Nike and he's also working on a bunch of entrepreneurial programs there. So.
The idea being that I think that if you create that environment and you allow people to explore that, that they will continue, and maybe it's a selection bias, but they'll go on to do multiple other things that will be interesting for either the company or industry. And so I completely agree with your point.
David J Bland (40:51.748)
Yeah, I've had that with some of the people I've coached where they've either left to join startups or other companies and they bring that thinking along with them and try to help create that culture wherever they are. um I could talk Amazon Dash button all day, but you've done other things too. And what are you up to now? I mean, that was a little while ago. Like, what are you doing this year? What gets you excited? Like, are maybe some things you're trying to test?
you know, maybe even your career or what you're up to now, if you could share them with us.
Pat Copeland (41:24.664)
Yeah, for sure. um Well, let me go back 30 years and just say, I got my master's degree at USC in machine learning. And I remember having a, you know, I was a pre-med and I had a conversation with my dad around this and he said, you know, I'm really disappointed that you don't want to be a doctor. He was a doctor and he said, you know, and I'm worried about you making money in computer science. And, you know, you need to be able to support yourself.
David J Bland (41:29.402)
Okay.
Pat Copeland (41:54.284)
The fact that I was like really focused on machine learning, think, you know, concerned him quite a bit. And that sounds absurd today, but so many things have happened in order to make machine learning very, very valuable and important these days that, you know, maybe I don't blame him too much, you know, in hindsight, but yeah, we, we had the original ideas around how to make machines do, um, you know, make interesting decisions based on neural networks.
30 years ago, but it did take massive breakthroughs in distributed computing and processing power and GPUs and hardware in order to get us to today. So I'm really happy that over my career, I've been able to kind of dive in and do machine learning projects where it had application and interest. And to be blunt, think that in the future, I don't think that there is an app that won't have some kind of reasoning or machine learning as one of its core values that it brings.
TikTok with good recommendations or Amazon with its shopping recommendations. You know, I just think machine learning is going to be required in order to survive in any kind of relationship with consumers in the future. So I'm super excited about that. And I just joined a company called Maloco that is a company that does, uh that's building a lot of the advertising products that I built at Amazon. And so
What we're doing is we're taking those concepts of things that uh only very large companies have been able to invest in where they have data scientists and applied scientists and lots of access to uh resources to train these models. And we've been giving them to everybody other than Amazon and Walmart. So could one of our company partners, our really great partners is Wayfair and how do we help them?
uh be able to do some of these connections with customers and be able to have highly relevant experiences using machine learning. And so we are a machine learning company that applies uh specialized models that are trained to companies like Wayfair in order to create that same relevancy that requires a tremendous amount of investment. uh it's a green field, because you look at it and
Pat Copeland (44:20.490)
most companies don't know how to do the depth of uh applied science and machine learning that we have. And so it's really kind of still a specialized skill, just like it was back 30 years ago. But now it's more about how to apply it to an application. And so it's a really interesting time. It's not just about the science, but we're actually working on a three-way win between the...
the commerce site, so somebody like Wayfair, uh the advertisers who are trying to reach the customers and the shoppers themselves, like having very highly relevant experiences. So that three-way win is something that we built uh at Amazon and I'm replicating that again for everybody else. uh you know, and in doing so, what we're doing is we're enabling it at, you know, commerce, which is like on, it has very thin margins.
to be able to be successful and to create competition so that commerce can thrive? Why is only Amazon dominant in retail online? Is that fair or Walmart? Can we create more companies that can survive, thrive, and be able to have great relationships with their customers across the board using the same technology? And the answer is absolutely yes. So it's an exciting time because
people are realizing how valuable this tech is and we're able to apply it now in new ways. So that's what's getting me super excited right now.
David J Bland (45:56.464)
Yeah, I can see why. mean, so much of it's timing. And I think about, know, startups I joined in the dot com era and we just quite frankly didn't have the infrastructure to pull off some of the things we were trying to do or customer behavior wasn't there, you know, the idea of Airbnb and Uber and Lyft. I mean, back then or late nineties, that would have been scary for people. But that's fascinating that you're taking everything you've learned. I guess.
Whatever you're comfortable sharing, is there like one big assumption that you're looking to test this year or is something that keeps you up at night with all of this? you feel like, need some evidence around this part of what I'm doing.
Pat Copeland (46:34.990)
Well, I would say that, you know, how do we scale, you know, a solution like this where we, you know, we perfected it within Amazon in a vertical situation, you know, and one of the kind of inside baseball things I'll say is that, you know, there was a lot of conflict.
in Amazon between the teams that were working on organic search, meaning when you type in a query and it uses NLP to match keywords to products, like that team felt like they were defending the customer and they were very much in charge of like creating a really good customer experience. And for many, many years, decades, their job was to, you know, take those magical words and map them to some cat, some catalog that had a product in it and to give the user what they wanted.
So we call that spearfishing. You're giving the customer what they want. like me, I like to really quickly get what I want and get out of a store generally. I'm not like somebody that hangs around. So that spearfishing creates efficient shopping. It's really important. And then when you come in with these machine learning techniques, and especially when you're looking at introducing advertisers, well,
Why is advertising interesting and why would it create relevance with a shopper? Well, what you're doing is you're saying, hey, advertiser, you know who buys your products. You know who buys your products better than we do because you're built your product for those customers. So how do we unlock that insight in a way that helps, you know, create higher relevancy in that same page? And so no longer is it just about NLP and about keyword matching.
It's also, and the customer's giving us their intent. It's also about trying to match the right brand and the right advertiser to that customer. And if you do that well, well, you just unlock this incredible flywheel effect. uh Incredible flywheel effect. And so the question is, as other companies, as we democratize this ability to go do this, they're all on their own kind of maturity curve in terms of
Pat Copeland (48:48.846)
uh how they think about that problem and how they approach it. And, you know, there's a distribution. Some are learning how to do that for the first time. And some are like, you know, very far along and are thinking about how to do it. And how do we introduce it in a way where, you know, we have limited resources, you know, on our end, but how do we do it in a way where we can scale and help as many customers as we can, you know, without getting caught in
you know, everybody is a snowflake and everybody we have to, um, you know, so it's like creating a generic platform that requires some tailoring, but, um, doesn't require us to be all hands on deck with every single customer. I think scaling is a really big thing that we need to, to work on. it's, and it's, you know, the technology can scale, but it's more about, you know, helping the customers through their life cycle.
and adopt this technology, that's really where the scaling gets challenged is how do you apply it, you know, within a specific context. And yeah, I'm thinking about that quite a bit right now.
David J Bland (49:58.768)
Yeah, I can see why. I I personally have similar experiences at startups where we couldn't scale if we customized every single thing on our platform and we had to have a common platform. There's some growing pains with that of what are the commonalities and will they have accept trade-offs and all that. But it sounds super exciting. It's almost like the future of targeted advertising in a way for companies beyond Walmart and Amazon. So that's super interesting. I wasn't aware that that's what you're working on. So thank you for sharing that.
I do want to be mindful of our time too, because I could chat with you easily for another hour. So maybe if you're up for it, we could have you back uh in the future. But uh today, we covered everything from, wow, from USC from 30 years ago in machine learning to uh your time at Amazon working on this Dash button, which thank you so much for providing insights there. It was fascinating to me and our listeners. uh If somebody is just interested in what you're up to or
they need your help as far as targeting with advertising and everything, how would they reach out to you? What's the best way for them to get in touch with you?
Pat Copeland (51:1.420)
You know, I think probably the best way is like we were talking about, we connected on LinkedIn. mean, I would be more than happy to hear from, you know, a new set of customers or somebody that's thinking about this area, or maybe even people that are interested in joining Maloko, you know, like uh reach out to me on LinkedIn. And I'm pretty active there and I'll respond. So uh yeah, thanks for that invitation to, you know, solicit interest.
David J Bland (51:24.388)
Yeah, so what we'll do on the detail page is we'll also have links to Maloko and to Pat's LinkedIn profile. So you all could click there and get through to. I am so excited. Thank you so much for hanging out with me. Again, I think we easily could do another episode about other things you've tested over the years. But just uh thanks for being so open and honest with us and sharing your thoughts with us today.
Pat Copeland (51:45.048)
Well, and David, thank you. And I really appreciate, know, we met serendipitously, but I really appreciated this conversation and I really appreciate you diving deep and your interest in entrepreneurial thinking and how to test them. So thank you.