Subscribe
Share
Share
Embed
In this episode, Lauren Galanter from Elsevier, Emily DiLeo from The Current, and Daniel Gottlieb from Microsoft’s Developer Division tackle this great ResearchOps question: "In a democratised research environment, how can ResearchOps support the needs of a centralised UXR team while catering to an "army" of researchers?" Around halfway in, Ned Dwyer, the co-founder and CEO of Great Question, shares his point of view on the topic. Hosted by Kate Towsey, Chief Cha Cha and author of Research That Scales: The Research Operations Handbook.
Creators & Guests
What is It’s a Great ResearchOps Question!?
The Cha Cha Club is a members' club for full-time ResearchOps professionals. Every year, we partner with amazing companies to help Club members share their expertise and advance the field of ResearchOps. This time, we partnered with Great Question, the all-in-one UX research platform, to produce a limited-edition, six-part podcast series that tackles the great questions facing ResearchOps professionals today. Every episode features a panel of Club members discussing a great ResearchOps question like, "What are the challenges and opportunities of AI for ResearchOps?" or "How can I convince leadership to establish the first ResearchOps position in the company?" A new episode will be published every two weeks until all six episodes are live.
This is It's a Great Research Ops Question, a 6 part pod cast series produced by the ChaCha Club, a member's club for research ops professionals. In each episode, a panel of club members will tackle a great question about research ops, like what does AI mean for research, or how do you build a compelling business case for research ops? This series is sponsored by Great Question, the all in one UX research platform, and I'm your host, Kate Tauzy, and the founder of the ChaCha Club and the author of Research That Scales, The Research Operations Handbook.
Emily DiLeo:This is episode 2. In the studio, we have Emily. Hi. I'm Emily DiLeo, and I run a knowledge management consulting business called The Current for UXR Research.
Kate Towsey:And we have Lauren.
Lauren Galanter:Hi. I'm Lauren Galanter. I'm a research operations program manager at Elsevier.
Kate Towsey:And last but not least, we have Daniel.
Daniel Gottlieb:Hi. My name is Daniel Gottlieb, and I'm the head of research operations for Microsoft's developer division.
Kate Towsey:All guests views are their own and not that of their employer. We are doing episode 2, and we have a very, very meaty question to answer. It is, in a democratized research environment, how can ResearchUp support the needs of a centralized UXR team while catering to an army of people who do research? One of the things that I learned working in a big company was that, it can be easy to assume that there are words that are, like, everybody understands what they mean. Like, even research.
Kate Towsey:Like, you would be like, of course, everybody knows what research means or insights. But when you start digging into what everybody thinks that words means, you realize that actually there is not a lot of agreement across an organization around what these fundamental words mean. And so, I thought although in my little world I think everybody the word PWDR is used so much now or the acronym PWDR, which stands for people who do research, which is actually a word that I came up with, the word and the acronym, sometime in 2019, I think. To speak about the cohort of people who do research but aren't full time researchers. That's my definition.
Kate Towsey:But I'm really curious to start out with a bit of a sense of, like, what are each of your definitions around what this cohort of people are or what the word PWDR means. Lauren, do you wanna kinda kick us off?
Lauren Galanter:Sure. Yeah. Just to say first, I, completely agree about, like, getting all the words straight and finding the words you use. So, here for this. You know, internally, I only really use p w p w d r.
Lauren Galanter:I guess, like, when I'm talking with my research director or the with my other re reops colleague, like, we don't really use it sort of as a team facing word because I'm not sure, yeah, if you're not in the reops niche space, people are necessarily familiar with it. And we usually just talk about, you know, researchers or designers who do research, because our remit is just within our UX team. But to me, I mean, I pretty much agree with your definition. And I think it's a good acronym because it's very plain language and literal. It's just the people who do any kind of research.
Lauren Galanter:And I guess then the question is how what counts or or or how research is defined in any given organization.
Kate Towsey:I've actually heard, someone a couple of years later call it powder.
Lauren Galanter:I've said it that way too. Yeah, I've said like powders.
Kate Towsey:Yeah. And I thought that is so cool. It's very cool.
Lauren Galanter:Can be a little tongue a little tongue twisty.
Kate Towsey:It is. So so so if you want to, you can you can jump in with powder. Emily, did did you have something to add to that?
Emily DiLeo:Sure. So I'm probably newer to UX than everybody here. Just coming in about three and a half years ago, and I think to me, coming in as a kind of an outsider and also as an anthropologist, I just thought it was a bit curious how, yeah, how so many folks at a company would talk to customers. To me, I think that anyone who talks to customers is like a researcher, like researcher in quotes, because they will have thoughts about and observations about their interactions with customers, whether they're assisting them, whether they're selling to them, and they're gonna have, you know, patterns that come up over and over and how they talk to people. And I think that that is, definitely valuable.
Daniel Gottlieb:Yeah. I I I definitely agree with what you're saying there, Emily, as far as what we define as people who do research. In our organization, we believe that everyone is responsible for learning from the customer and the customer experience. So the degree to what they're doing, a complete research study on their own, or they're involved in a research study that a UX researcher is running. Totally varies.
Daniel Gottlieb:But from that definition, we really consider anybody eligible to be, somebody who does research or PWDR. So our definition is pretty broad, but we don't necessarily use use that term, in our day to day. We just kind of expect that everybody is open to the possibility of learning from customers and everyone's empowered to be able to do that.
Kate Towsey:I, when I worked at Atlassian, we used to call it we talked about AWDRs, Atlassian Studio Research. So maybe for each of your companies, you've got your own your own little acronym in there. It really forces you then to look at the word research and the spectrum of what people might call research. What I found is that if you're not defining the outlines of what research is, as research operations, suddenly you need to look after all customer interactions everywhere. Have you run into these problems in your working environment?
Lauren Galanter:So, yeah, I can start in on that. So it's kind of a yes and no answer. So for research ops in my my organization, in my the part of the organization that I support, It's the 1st dedicated research ops function in the company that we know of. It's a very large enterprise company. And so being one person and and now luckily 2 people, we only have the the resources and and our remit is only for, our our single UX department, because we can't support everyone who does research research, in the company, whether there are other UX teams in debt with dedicated research UX researchers or product managers, sales, etcetera, like other people having those customer interactions, which you could debate, you know, what is research versus other types of customer interaction, I suppose.
Lauren Galanter:And so we, out of necessity, have had to say, research at least our group for research ops, we support our team really primarily. Yes. We've bumped up against needs, requests from other functional teams, other teams, and we try to help and collaborate where we can and knowledge share. But it's it it just is up to a certain limit because of our our resources. We're not at a point where we're supporting the whole company, which, you know, is a healthy constraint because, yes, otherwise, you would have to be, you know, supporting so many different things that would really open the floodgates.
Lauren Galanter:But it's a bit of a, I suppose, a a catch 22, right, of, you get you can get those additional resources when you show that you are supporting beyond what you have.
Daniel Gottlieb:And I and I have to add in there too, though. Like, what a great problem to have if you're feeling the problem is that too many people are coming to you for support and too many people wanna be involved in research. Right? If you have too many product teams that are looking for you to help them be able to do more research, yeah, it's a problem if you can't do it, but how wonderful that everybody wants to be learning from your customers at that point. Obviously, to be successful there, then you need to be able to put some, you know, limitations on, like, look.
Daniel Gottlieb:I can't always do this. Otherwise, you're gonna not be able to do x, y, or z. And so finding customers is a pretty good example of where we draw a line. If I can find customers for our research team, our UX team, but I don't have the bandwidth or really the resources to find customers for all of the, product teams who are looking to do research as well. One of the ways that we solve this in terms of the scale and be able to support everybody there when everyone's coming to research operations is if I can empower my UX research team to be able to support the needs of the product teams that are coming to them, I mean, that is really, like that is what a force multiplier can do.
Daniel Gottlieb:Like, I can multiply the efforts of our UX research team, and then they can all multiply the efforts of the product team. And we can really see our impact
Lauren Galanter:using the researchers as a force multiplier for ops. Definitely agree there. That's something that I have been starting to do more this year and starting to find ways to do that more. In fact, I have a a little workshop about this tomorrow with some researchers on the team, to figure out how we can allow them, kind of like you were describing, Daniel, to go into the different specific product areas that they are embedded in, and help kind of do that through basic React support and field certain basic questions because they're in that context embedded with the product designers, product managers, etcetera. And that way, that's not all coming centrally to ReOps as kind of that first line of defense, for for questions and, things like that.
Lauren Galanter:So I definitely agree there. That can, like, augment more structured research.
Emily DiLeo:Yeah. And I'd also this is Emily. I'd like to say that, you know, in in some companies that I've been in, one in particular was a kind of a curious, setup that they had the UX research team, but they were not actually the ones who were doing the research on the ground per se with their customers. That was the local market team who knew these customers the best because they knew them in person. And I can imagine that can happen in other places too.
Emily DiLeo:And I think what had happened over time is this customer knowledge started to become proprietary. Right? Certain teams would say, well, I know the customer, you know, because I talk to them all the time. So I just think this is, like, that moment is super crucial for research ops or someone to come in and say, let's talk about that. Let's talk about what anecdotal evidence is.
Emily DiLeo:Let's talk about why your, you know, conversation with this really angry customer is emotionally affecting you to believe that that is the opinion of many customers. Right? How those things work for you. If you're customer support, for example. Because I can think of, you know, a million conversations like that.
Emily DiLeo:That would just that would just be really helpful to understand, you know, what information you're gathering and what does it mean.
Lauren Galanter:Also, Emily, agree with what you're saying about the the different ways of knowing. Right? Because a customer interaction, that some that will lead people it will bias people towards thinking, well, this is how, you know, this is a really big customer need or a big pain point. That is a valid way of knowing something, but it's it is not research per se and, like, kind of ex and kind of, educate people on those differences. But but without invalidating the things that support people, salespeople, etcetera, know and learn about the customers.
Lauren Galanter:So it's kind of like interweaving different ways of, like, knowing and insights about your customers that can, like Yeah. Augment, like, more structured research.
Kate Towsey:So you've got people trying to understand what is this research with a capital R, what is all the other knowledge that we're gathering, it's all valuable, how do we codify it, how do we bring it together and how do we understand what we actually know. But are there things that you are doing that are sort of on the more, sort of planning and practical, not that that's not practical, the sort of background and, like, recruiting and incentives and, like, all that kind of jazz that you're doing to help PWDs. What sort of things do you have going on?
Daniel Gottlieb:I will say that a lot of that stuff that you're describing, like helping with gratuity, helping with recruiting, kind of those real, nitty gritty logistics, I'm not doing. And our our research operations team is not doing for people who do research. That kind of has become a little bit of a bridge too far. But we like I said before, we're helping and supporting the UX researchers and all of that and giving them the tools and and resources. And what I can do is I can communicate and educate our product teams on how they can work with their UX researcher.
Daniel Gottlieb:So their UX researcher is gonna be the one to help them out to do that. And it's gonna vary a lot to the degree that they are gonna be helping with those nitty gritty things and helping with recruiting customers or or some of the logistics. But a lot of our research and findings that we have between the researchers and the product teams, they're learning together. And, you know, you talk about research with a big r, and other kind of learnings. I don't find that they're necessarily happening in these, like, silos.
Daniel Gottlieb:They're happening often altogether. So by supporting the UX research team, I say that in effect, I'm supporting then those product teams as well. So sometimes it might be that the product team is saying, Hey, I have this question that I need help answering, and, they'll work with the researcher. Together, they'll recruit. Together, they'll come up with their hypotheses.
Daniel Gottlieb:Together, they'll run those studies and come up with insights. Sometimes it's just, hey. I need some coaching. Can you help me out, and can you help me find some customers? Maybe yes, maybe no.
Daniel Gottlieb:And sometimes they have their own ways of doing that. And sometimes it's more of a I have this question. Do you have the bandwidth to to run most of it? And I'll watch you do it. I will say, though, however, I feel like I didn't quite answer your question, Kate, and I kinda skirted around because the the truth is if you're talking about some of the biggest blockers for supporting those, you know, for us, literally thousands of potential people who do research, That's it, is that we don't have an ops person for them who's helping them recruit, helping them with gratuity, and helping, with some of those logistics.
Daniel Gottlieb:So they're they're scrappy. They know, the value of doing it, and we've done a good job of motivating them to want to learn from customers and wanna do that research. So they're finding ways, but I do wish I had a better way to make it easier. And that's probably one of our bigger blockers.
Kate Towsey:So, your what I find interesting about that is that you're engaging with PWDR powder. I'm gonna stick with that one. By proxy of your researchers. And so you're giving your researchers all the capabilities they need and they become the proxy to the powder. Is that right?
Daniel Gottlieb:Yeah. And that's that's that's absolutely correct. And I I don't mean to say that I'm keeping myself separate from them. Like, I will run workshops, with, the with powder. I'll run workshops.
Daniel Gottlieb:I'll give, like, learning theories where we're talking about the different types of research and why it's beneficial and why it's something that how it's gonna help you in the different, types of research you can do. But a lot of it at the end is, and so I want you to go talk to your UX researcher, and here's your list of the people you can talk to. I've even done activities where I'm like, during orientation, where I pause my lecture and say, alright. Here's a list of your UX researchers. I'm gonna give everyone 5 minutes.
Daniel Gottlieb:Go send them a message. Introduce yourself because they're gonna be the person who's gonna help you get your job done, and I wanna help you make that connection. So I do see a lot of what I do in in, the matchmaking. Right? And making sure that they understand that resource.
Kate Towsey:I I have a feeling Lauren's got something to to add here. But, I'm I'm really curious about that. Have you found that, because the researchers who are that go between are part of their work is not just doing research, but it's also engaging with other people who want to do research. How are they handling are they very comfortable with that part of their role?
Daniel Gottlieb:Absolutely. That is really a defining characteristic of our research team, is that their end goal is the same end goal as the product teams. Right? At the end of the day, they are trying to learn from the customers so they can make right product decisions, build products that help our customers meet our business goals. And so if and the UX research team, their job is to help the product teams understand that, whether it's they are largely doing the research themselves or they're helping the product team do it.
Daniel Gottlieb:As long as they can make that connection and learn from the customers and help that make better products, everybody wins. Now I'm not saying it's always necessarily easy and especially when you have new people on the team that they see as a little different from what they they're used to. And but I'm the researcher. I'm supposed to be the one sharing my showing my expertise. We really quickly, get them into this culture.
Daniel Gottlieb:We, refer to our researchers often as coaches. You're talking about the right the right term, you know, learners. We refer to them as coaches because they are coaching the product teams as well, And we immediately get them into, I don't need to go into too much detail of this now, but we have these workshops where we have new to to our division product team members go through, and they are solving business goals, talking to customers during these workshops. And our UX researchers join as coaches who help them learn from the customers, help them run the interview. And they immediately see the value of that.
Daniel Gottlieb:They gain this pride of being a coach. They'd see, how helpful it can be and how they can drive more impact by helping the product team members, do research. So we really quickly bring them on board to, like, hey. Coaching is awesome. Helping our product team is awesome, and you are being effective in spreading your research further by helping them do it.
Daniel Gottlieb:Yeah.
Lauren Galanter:That sounds like a a really great program. And it's interesting, how different organizations are because I think from a research ops perspective, in the, you know, where I am, you know, you we're really focused operationally on enabling our powders who are the researchers. I kinda can put them in that category and the UX designers, sometimes UX managers as well, to do research using a full suite of self-service tools and systems that Research Ops manages. And so everyone is using the same standard research process, as well as a platform that we migrated to to, you know, automate a lot of that work, you know, as far as recruiting, scheduling, incentives, etcetera. And we just kinda keep it on the rails, provide the governance, the training, the onboarding for how to do all of that operationally and how it dovetails with the best practices for, like, our craft of research, which the researchers are are more that's, well, that's really a collaboration with them as far as defining.
Lauren Galanter:We have, like, a library best practices for different research methods and templates and things like that. That's a bit more of a, collaboration with researchers as a sneeze on that. Although, since I transitioned to this role, from being a researcher, I'm able to, have a unique perspective on that too. So, yeah, that's that's really, you know, our our focus is, I think focusing on a lot of those things to get our team set up for for for success doing research or at least making it easier to do kind of all those front end things, you know, recruiting and incentives and things like that. You know, I think in the future, we would look more towards, sort of more downstream towards analysis and, insights repositories and things like that.
Emily DiLeo:Yeah. This is interesting for me to kind of hear these perspectives. Right? Because it sounds like, Daniel, what you have at your organization is this pretty well established, value of UX research and trust kind of in embedded in the culture, the work culture where you are. And that's certainly not the case everywhere.
Emily DiLeo:That takes, you know, a lot of work and things to establish. I've heard maybe more often what Lauren is describing as, you know, kind of trying to scramble, you know, to get these resources to cover everybody who wants to do research. And I think what what I found helpful in these kind of situations is for ops, if you're in the situation where you have folks that are pretty siloed and they're just not sure about UX research, they're not sure about, like, you know, I'm doing research, but, like, I don't need a coach, is to kind of develop, like, these partnerships, like, more reciprocal relationships between your UX researchers and other folks. Like, I hear a lot about kind of evangelizing research and talking about research, but, like, you know, do we really know what marketing does? Do we really know, you know, do we know what, you know, designers are doing every day?
Emily DiLeo:Do we have we sat in and, like, watched them? You know, we're researchers. Right? So this is kind of what we do, and I feel like this is maybe another opportunity to bring that skill set internal, you know, to us and just kinda be like, let's understand how these folks work and where their pain points are and why they might not be reaching out to us.
Kate Towsey:Emily, I love that you bring up, because so often, it can be missed that operations is as much about the tools, the platforms, the money, and all that sort of stuff to to get research done as it is about setting up relationships. And relationships are kinds of systems too. And so it's going in there and helping people empathize not just with customers, but with each other and how designers do their job and see the world and how marketing do their job and see the world. I think that's really interesting. Is that something that anyone's like Daniel, it sounds like you you did your culture sort of manifest itself spontaneously like that?
Kate Towsey:Or surely not. Have you do special things in
Daniel Gottlieb:there? Yeah. A lot of the way that we got here was about identifying how people work and then meeting them where they are. Right? And saying, okay.
Daniel Gottlieb:This is how you are already getting your, customer insights. Okay. How can we build on top of that? This is how you're already sharing your customer insights. Great.
Daniel Gottlieb:We're not gonna change anything. We're gonna help it make it easier for you and reduce the friction. But it's not about forcing you into our system. So to do that inherently, yes, you need to have a good understanding of how do the product teams work, how do the developers work, How do the designers work? And once you get a sense of that, meeting them like I said, meeting them where they are and building off on top of that.
Emily DiLeo:Yes. 100%. I will say that just because when I, you know, done a lot of this knowledge management work where I'm going in and I'm helping folks stand up a repository and understanding, the first thing I have to understand is how does knowledge circulate now or not as the case maybe. Who talks to who? Like, how do these folks work?
Emily DiLeo:Who you want to, you know, use and utilize this research knowledge? And it is. It's it can be a ton of relationship building, and ops is in such a strategic position to do so. And when I've been on ops teams and, like, I'm gonna go to design ops team stand ups. I'm gonna try and go to, you know, marketing stand ups.
Emily DiLeo:Like, whoever is adjacent in the work that I'm doing because, yeah, I've just not experienced anything but positive results when you, you know, make these connections.
Kate Towsey:Let's take a super short break to hear from Ned Dwyer, the cofounder and CEO of Great Question. He'll share his thoughts about the topic and will rejoin the panel straight after.
Ned Dwyer:I think the first thing that I would start with is acknowledging that balancing the needs of your UXR team and those of people who do research is really hard. And you need support if you're gonna solve it. You're trying to balance the need of power users who are highly experienced and trained, sometimes with advanced degrees and running research all the time with very specific needs with people who do it less frequently, they're less experienced, they require a lot more guardrails, more training if you're gonna make them self sufficient. It's not easy. The first thing I'd say is pick your battles.
Ned Dwyer:Define what initiatives you're gonna focus on and make it a public commitment. Also define what you're not gonna work on. What methods you're gonna be democratizing? How many folks you're gonna be supporting? What tools you're gonna procure?
Ned Dwyer:Creating a roadmap allows you to go and build trust with the stakeholders and alignment, and then stiff arming any new requirements that might come along because you can't do it all. You can't continue to stack on new requirements. It needs to be 1 in 1 out. The second thing I would do is go and compartmentalize your time during the week. So how much time are you actually allocating to each initiative, and go mark that up in your calendar, color code it.
Ned Dwyer:So then you can go back and reflect on it and say, am I allocating my time based on the strategic needs and the priorities of the organization or not? That's your biggest resource is gonna be your time. And the final thing is you need to create leverage. You can't do everything. And so that could come from getting more hands on deck, either hiring more research ops folks, or recruiting other people around the organization to support you.
Ned Dwyer:It could come from building better processes, documentation or self-service training. But then not to speak my own book too much, but I think the biggest leverage you're gonna have is getting the right tool stack. How can we build guardrails and systems in place to allow people to do self-service support with the UXR's or people who do research without allowing people to blow, their foot off. They need to have guardrails to protect themselves, protects your participants, and protects the business. Ultimately, I don't think it's possible to support an army where you're the bottleneck managing everything from recruitment, tool provisioning, approvals, training. You need help, go out there and get it.
Kate Towsey:I wanna get back to the question of scale because we haven't really spoken about numbers yet. There are a couple of interesting things happening at the moment, where it feels like there is I'm seeing messaging also on LinkedIn, and I wholeheartedly agree with it that although researchers, in organizations have the numbers have become less over the last few years for reasons that we all know, The interest and the investment in research has not. That is certainly stable if not growing. And so have you found in your space that there are more people who are doing research or is the same?
Kate Towsey:And what sorts of numbers are you looking at in your space? I'll kick it off when I worked at Atlassian there were around about 500 people who were in roles that would invariably do research like product design, content design, design, visual design. And eventually we were looking after about 365. It's quite a specific number because we did lots of reporting on it. What sorts of numbers do you have in your space?
Kate Towsey:Lauren, do you wanna kick us off?
Lauren Galanter:Yeah. So, in terms of the group that that, my reaps team officially supports, it's our larger it's our UX team, which in total is 70 some people. So it's probably about 60 or so UX designers and I think 12 or 13, dedicated researchers. And then and then, of course, there's the 2 of us in research ops. So that's who we that's who how we support just because of our organization structure.
Lauren Galanter:Now there's other UX teams with their own research teams, and there are tons of product managers, marketing managers. There's a whole separate market research organization. So there's probably 100, if not into the 1,000 of people doing some kind of customer research out there. But, it like I mentioned earlier, it's, getting to a point where we can either officially support, let alone sort of meaningfully engage with with all of that. You know, it's it's, an aspirational goal, and we're, you know, something that we wanna figure out longer term so that we can have more cohesive research across the organization, especially in terms of, customer contact.
Lauren Galanter:So that's not, redundant, etcetera. As our research director likes to say, it's it's less about, you know, trying not to step on each other's toes, but learning how can we dance with each other.
Daniel Gottlieb:So yeah.
Kate Towsey:Is there anyone else back? Yeah. Go, Daniel.
Daniel Gottlieb:Yeah. So for for us, we have, our UX research team is 23 individuals. 21 of them are UX researchers and just 2, myself included, represent our research ops team. And then for our organization as a whole, we have over a 1000, product managers and over 6,000 or so developers. And like I said, any one of them is empowered to learn from the customers.
Daniel Gottlieb:I would say it's much more expected for the product managers to the point that if they're coming out with a new concept idea or change, the question they're gonna get is what customer evidence do you have here? What were your hypotheses? And how did you come to this conclusion? So I do have a expectation, but I cannot confidently say how many of those people are gonna be doing some form of research. But that's a pretty good sense of the scale that, to a degree, whatever level of research we're calling it, they are learning from customers.
Kate Towsey:In other words, humongous. Yes.
Daniel Gottlieb:The official number is humongous.
Kate Towsey:Humongous. Emily, did you wanna add anything?
Emily DiLeo:I think just just some, general thoughts about this. You know, I came into UX during kind of the heyday. Right? It was, like, 2021, and there were lots of jobs and lots of researchers. And, you know, when things dropped off really dramatically, I kept thinking to myself, this is not about the value of UX research because this line of inquiry is very human.
Emily DiLeo:Like, humans naturally want to understand other people and their motivations and their you know, what things mean to them. And I just thought to myself, somebody is doing this research or somebody has been doing it all along. We just didn't know it. It just wasn't, you know, the light wasn't shined there. And now, yes, it's kind of, yeah, this idea of democratization.
Emily DiLeo:What is research really? It's a it it it's a good moment. Like Lauren said, this is a good moment for something, really transformative, hopefully, to happen. I couldn't tell you what that would be, but, it's an interesting moment.
Kate Towsey:It reminds me of, like, the printing press or chat GPT. Mhmm. Yeah. That's sort of more of the most recent sort of shift in things with the iPhone when that came out. Let's let's switch tack a bit onto, knowledge management.
Kate Towsey:Emily, I know this is this is, like, this is your work. This is really where you do, what, 99% of your work, if not 99.9 Yes. Percent of your work. Yes. So how does research democratization affect knowledge management?
Kate Towsey:Like repositories and capturing tacit knowledge. You might even wanna define tacit
Emily DiLeo:Yes.
Kate Towsey:For for the audience.
Emily DiLeo:Yes. Absolutely. So that's a good place to start actually. So tacit knowledge being this kind of knowledge that kinda lives in our heads, things that we kind of understand and know, but never really document anywhere. And I think there's a whole lot of this in UX research just, you know, in the sense of, you know, when I talk to folks about research repositories.
Emily DiLeo:So here's the research report. Okay. There's your insights. Where's where's everything else? Where's everything else that you learned from this study that's not in the report?
Emily DiLeo:And where do you capture that? Where do you put that? That's always one of my questions. And, again, this is gonna be even more important for those folks who are, you know, not formally trained in, like, reflexivity or this kind of ethnography, you know, where you kinda take notes and you think about what the person said and so forth. Right?
Emily DiLeo:So it's just gonna be really important to kind of have conversations where you cap you capture this knowledge or, you know, you could be, you know, even regular team meetings, which actually that's where a lot of test and knowledge comes to the to the forefront. Team meetings, Slack, like, where how is this being, you know, kept track of? And that's just a general that's just a general concern, right, about repositories and knowledge management. But it just multiplies when you're talking about how many people could be capturing this kind of knowledge. And I think it just, again, I think it really depends to set up something, you know, for a UX team plus all these other folks who are doing research, I would really be curious to know.
Emily DiLeo:I'm sure people are doing it already. They probably have, you know, devised ways of keeping track of all this. Maybe they compile, you know, all these these things in a report. If if Lauren, if you or Daniel have have, experiences you'd like to let me know about, that'd be great. But it's just it's a huge to me, it you know, it's interesting because it's a huge challenge and a huge opportunity, and it kind of kind of I mean, certainly affects the tool stack as well.
Emily DiLeo:Right? Because you've had all these repository tools that were, I think, first kind of created thinking of UX teams. And then there was this movement into, like, more people doing research and more stakeholders. And wait a minute, the repositories for the stakeholders. So now the tools cater to a wider audience.
Emily DiLeo:Right? And so now what does this mean for tools and for tooling that when you're gonna have, like, tons of users entering things and, like, need some kind of governance. So it's just a lot. There's a lot to consider.
Daniel Gottlieb:I'd love to jump in because you said something in there really interesting to me, which was about how the tools were built for UX researchers. And a lot of, like, repositories are built that way. And even if we say, hey, this is opened up. This is available to anybody. You don't have to be a UX researcher.
Daniel Gottlieb:If it's built in that way, it can be really intimidating and, really discouraging for people who don't have that formal training. Right? Especially if this is is something that requires, hey, to do this, you need to first put your introduction. You need to put in your hypotheses here. And this is how they all look.
Daniel Gottlieb:And this is a formal report. It's like, woah, I'm just a developer. And I I'm really proud. I talked to 5 customers. I learned something.
Daniel Gottlieb:I want to share it. And now I have to do this formal report, and I'm up against these PhD researchers. No way am I gonna do that. Right? And it's just really, like, you you turn people off.
Daniel Gottlieb:So I think it's really critical when you're doing something for people who aren't formal researchers. You find something that's easy entry. It's not intimidating. And they're celebrated for whatever they do. Right?
Daniel Gottlieb:You're not saying it has to be at this level or else you can't share it Or you're not critiquing, no. No. No. You should have done this. You're celebrating.
Daniel Gottlieb:Wow. You talked to those 5 people. How awesome is that? Let's all learn from this. So, you know, I'll mention in in our experience, you know, I talked about how we tried to meet people where they were.
Daniel Gottlieb:And and, Emily, you talked about how so much of this is in, like, Slack, and that's just where they're sharing. So, we when we started having more product team members doing research, we were looking where are they sharing it, and they're sharing it in email. Like, they're just emailing each other. This is what this is what I learned. And we said, alright.
Daniel Gottlieb:We'll we'll take advantage of that. We'll build an email alias and just share it to this email alias instead. You don't have to change anything else. It doesn't have to be a formal report. Just share it here.
Daniel Gottlieb:And that started to grow and gain traction. One of the biggest things, and I'm really grateful that we had this, our our leadership got on there, and they started replying. And they would reply to messages and say, oh, what a great learning. How how'd you figure that out? Suddenly, now you're seeing your colleague is getting recognition from leadership because they're study.
Daniel Gottlieb:I wanna do that. I wanna do the same thing. I'm gonna write something. You're also getting these opportunities to learn how to write because you're seeing you're seeing all these different reports come through and you're getting ideas. And so it naturally kind of snowballed like that.
Daniel Gottlieb:I think we have at this point over 800 people or so are on this email listserv. Now the problem with it, of course, you're talking about, like, the formality. It's not a great repository. You can't search back to old stuff. We met people where they were, but it it then started to have some of these negative side effects.
Daniel Gottlieb:What what we ended up doing is we worked with our internal team at Microsoft that has an internal research repository they built. And we worked with them that they would take the emails and auto upload them with a couple clicks and approvals into their repository. It didn't match the same style as the other reports, but that was okay. They knew they were a little bit different. But that way, they were saved for longevity.
Daniel Gottlieb:And the key part about that is it required no extra work from the product team members. We said, hey, great news. It's going to the repository. Your change, hit okay on this button when it's asked if you wanna do it. That's it.
Daniel Gottlieb:Because we knew we had something working. And you tweak it a little bit, add a little bit more friction, and it's
Emily DiLeo:gone. Perfect. I love that. That is that is absolutely a really good example of kind of, like, developing a knowledge management strategy, meeting people where they are, which is also something I say a lot too. And I I mean, I I get that I feel like a lot of people want all the customer research in one place.
Emily DiLeo:Like, I have heard that a lot from kind of especially executive leadership, you know, probably not at a place like Microsoft, but it, like, smells like they're like, we have CX. We have UX. We have all these folks, and I have to go to all the like, product managers. I have to go to all these different places to look for customer research. Can't it just be in one place?
Emily DiLeo:You know? And I think that that also could be done. I mean, I hear what you're saying about the intimidation part of it, but I also I don't know. Maybe there's an opportunity to elevate these folks, not maybe everybody, but, like, some folks to say, like, yeah. You know what?
Emily DiLeo:You're really observant. You're really good at these customer calls. And if you want to create, like, a monthly summary report of what you have observed, here you go. Here's a template, you know, upload it to the repository.
Kate Towsey:This reminds me of, I, I just wrote a book. And one of the chapters is called long live research knowledge. Years ago I'd written a blog post, I think it was in 2013 or something, about vertical campfires and about there being a space where people can create knowledge through walls, standing around walls. It was sort of still the era where we mostly worked in offices and actually saw people for real. And, people would be doing analysis and synthesis as a team, and so you'd have the service manager there and the researcher and everybody else in the team analyzing the research that they observed the day before and so they were creating knowledge together.
Kate Towsey:And so for me it was like a vertical campfire because we gather around campfires to tell stories which help us learn. And so I wonder I often feel there's not a lack of a distinction when we think about people who do research, the broader community of learners out there, and knowledge creators too, where you have these campfire spaces that are more relaxed, bohemian, you're creating stories, you're then then they're it's not all perfect. All different sorts of things are there, and they dissolve quite quickly. They they don't tend to be, like, perfectly tagged and structured taxonomies and all these things. And then you have a library and this the real best stuff that comes out of all that campfire talk.
Kate Towsey:They get codified just like we have books. We've had stories around campfires for centuries, decades, probably the whole of human history. And the best ones, or at least some of them, but we'll just call them the best ones. We're codified in books. And, I feel like if we were to take that sort of dual approach to knowledge, we'd have a much better time with it because now suddenly a tool doesn't have to be a campfire and a library.
Kate Towsey:It can be a brilliant campfire, and then we get a really great library
Emily DiLeo:in place, and we treat knowledge differently. That's that's that's awesome. That's incredible. I love that. Because I just, you know, sitting in on, you know, team meetings and, like, team only readouts and things.
Emily DiLeo:I'm like, this is this is where the knowledge is being shared. This is where the learning is happening. No. Not not necessarily in your, you know, slide deck readout to the stakeholders. Right?
Emily DiLeo:It's like these internal chats. I actually worked with 1 manager research team manager who would have, like, a a team only kind of 15 minute stand up where she would kinda ask the same questions every time and record it. And, I mean, how brilliant is that? Where it's just team only information. Right?
Emily DiLeo:All this stuff is important. Like, what why did you do a 3 you know, 180 in the middle of the project? Well, so and so said blah blah blah. Well, that's not gonna appear anywhere. Like, that's really important information to know.
Emily DiLeo:So, anyway, that's just kind of, like, things that come out, right, in, like, kind of daily conversation that are just yeah. That's where the ideas come.
Kate Towsey:How do you maintain the quality bar? We've actually ended up speaking quite a lot about being quite flexible in quality and not very dogmatic about what good research is versus not good research or Royal Research with a capital R versus any other types of learning. But, what what types of things are you doing to maintain a quality bar of research being done by people who aren't trained full time professional researchers?
Lauren Galanter:Yeah. So I can, start us off here with this. So, yeah, we we we focus a lot on this in our team, and one of the main way ways that we do it, we have a program that I called study buddies, you know, because you want it, to be friendly. I think at first, it was called, like, research reviews, and that was a little turned people off a little. And so, again, this is when within the context of a of a UX team with designers and researchers.
Lauren Galanter:So the designers would sign up and get really any aspect of research reviewed or for feedback. But the like, one of the key points is it worked both ways. Researchers are encouraged to get feedback on their work too. Right? Everyone benefits from, you know, kind of getting that peer review, so to speak.
Lauren Galanter:And it can be on anything from your study plan, to, like, something very unformed even before that where you're just trying to muddle your way through, like, you know, how to approach a research problem or doing some doing some longer term research planning. It could be a moderation guide. It could be a survey, a screener, etcetera. And it's baked into our research processes that we have people follow. Again, people on our team who are doing more of that capital r research.
Lauren Galanter:There's sort of these checkpoints baked in where we, you know, strongly suggest, nothing's really mandatory, I suppose, but we strongly suggest people get, for instance, their study plan, get feedback on that from one of the dedicated researchers within their area. And so each of these study buddy groups are aligned with they're led by researchers who are in working in the same product area as the designers. So, so it creates these little cohorts or different groups. And so that way, people can just sign up for a slot during one of their study buddy sessions and get that feedback, without having to spend, you know, it's usually I think it's usually a half hour session. They don't have to spend the first 15 minutes just getting oriented to, okay, wait, what is your product?
Lauren Galanter:Who are the users? You know, they're they already, you know, have that baseline understanding, and they can get to diving into, the, you know, the the the research itself. So that's one of the the main ways that that we do it. And then kind of, I also think about sort of a sort of an op the the ops quality, if you will. So, like, our team of 2, like, we sort of, certainly, we can look out, and give feedback on some of those research craft things, but we really try to create a line so that people aren't confused between re ops and researchers.
Lauren Galanter:So for for us, what we look out for from an ops perspective is, we just kinda do this, like, a a top line check-in the platform that we use for all our studies. We're just checking for things like, are people are using compliant incentive rates, appropriate number of participants, are they using the right templates, things like that. And we just reach out to them, click on Slack or something, you know, if, if they're not and kinda, you know, get those things, buttoned up. So that's really, yeah, the the main way that that we do that within our within our UX team.
Daniel Gottlieb:I I'll just jump in, Lauren. I do really appreciate the way you're describing that the UX researchers are working with people, and they're kind of, helping them along with it. Like, as I've been kind of, evangelizing myself here, I I love that as a way that the UX researchers can drive impact. To to to address the question to, to me, it's it's all about education and creating a foundation of education for the people who are doing research and, whether that means giving strong education at the beginning when they come in to the to the to the division so they understand what research is, giving continual sources where they can learn about different research and having templates and guidance. All that said, I would argue that for research operations, it's I don't see it as my role to maintain a quality bar.
Daniel Gottlieb:Like, I'm I I don't want to oversee any of that. Hopefully, if you are if you're lucky enough to be in a culture where they value the research and you have leadership value research, you're gonna see people who do, you know, great research. They're gonna rise to the top. Right? They're gonna get recognition for that, and people are gonna see that and wanna do more of it.
Daniel Gottlieb:And people who are doing, I don't wanna say phoning it in, but maybe not really taking the care that they should and just doing it to say that checkbox, I talked to a customer. They're gonna get recognized for doing that, and they're going to kinda be forced by the momentum of the culture to learn to do better. I I do feel like having too many, you know, I don't wanna be a gatekeeper. And having too many checkpoints that I would have to do can go back to that diminishing behavior. It's like, oh, man, I'm never gonna pass this.
Daniel Gottlieb:I don't wanna go I don't wanna give this a shot, and I wanna be encouraging wherever I can. But whenever people want that help, yes, man, give them that help. Let them partner with that researcher. Let them watch the study and help them rise raise up. And that's awesome, Lauren, if you're getting people to or like, I wanna know that I'm reaching this bar.
Daniel Gottlieb:Like, that's great that people want to understand that.
Kate Towsey:Let's, let's wrap up with, one final question, which is one final great question, which is, if you could give someone one one top tip for democratizing research from an ops perspective, what would it be? Let's let's go around. Emily, you start.
Emily DiLeo:Oh, gosh. Well, my top tip is gonna be super obvious, and that would be, yes. Set up plan your knowledge management strategy before, like, hopefully, before this all happens and just be really, curious about these things and meeting people where they are and being able to pivot and create new, like, mini strategies as you go?
Lauren Galanter:Yeah. I think it's hard to pick just one thing. Right? But, I think what's most top of mind for me, especially because of recent conversations I've had with some other researchers and REopt's people is, you know, please just, you know, I would encourage people to really implement, like, a self-service DIY research program, especially for recruiting mainly, where you have really strong reops, right, to to con create those guardrails, the governance, kinda maintain all of that and train people on it. But, just because I see not a whole lot and I'd be curious what what the rest of you see, but I I do see if, a fair number of re ops people describe themselves or or or say that they they are doing recruiting and finding customers for for, like, individual studies and things like that or or, like, managing incentives, like, like, directly.
Lauren Galanter:And then there's we just don't need to do that, with a lot of the systems that are available now. And so I would really focus your efforts on that and just attack on to that. Like, really just take an ops first approach if you can. Like, instead of going and hiring a bunch of researchers, like, put that ops foundation in place with the other people already doing research, signers, PMs, what have you. Right?
Lauren Galanter:And and kind of get that, get that, grounded first.
Daniel Gottlieb:Yeah. I I actually I love that Emily and Lauren, all 3 of us are gonna have different answers, and I think both of what you said is really important too. So I'm I'm excited to talk about something else. And for me, it's all about education and learning opportunities. I think it's really critical to educate your people who do research on those research fundamentals.
Daniel Gottlieb:Right? They need to really understand what is UX research. You know, Emily, going back to what you were talking about, a misunderstanding of what that really means and what it means to learn from a customer, and why is it beneficial, and how do you do it, and and probably most importantly, what are appropriate questions that can be answered with research? Like, what is something that you actually can learn, and how do you shape those initial questions and hypotheses? Because I think if you really can nail that, both the how you do it and the why you do it, then the people who do research will be motivated to learn more so they can do it and do it even better.
Daniel Gottlieb:So to me, that, like, if you're talking about the fundamental thing, I wanna make sure everyone has that baseline understanding. Why do I do research? How do I form my question? And from there, you know, Lauren, like you're saying, if you have some self-service tools, they can go and start doing that. Emily talking about if they have their, repositories, they can then go share that.
Daniel Gottlieb:So I think all three of those things together can really set up a a division for success.
Kate Towsey:Well, that was a brilliant summary. Thanks. Thank you for that. And we'll call that a wrap. A big thanks to our guests for sharing their time and expertise and to our sponsor, Great Question.
Kate Towsey:Learn more about Great Question at great question dotc0forward/chacha. That's c h a c h a. In 2 weeks, a different crew of research ops professionals will tackle another great question. So make sure to subscribe and tune in. This podcast is a limited edition series produced by the ChaCha Club.
Kate Towsey:We're a member's club for Research Ops professionals. You can find out more at chacha . club.