Subscribe
Share
Share
Embed
Calabrio Shorts is a fun-sized podcast that covers all sorts of topics around the contact center industry. No topic is off-limits as we cover frequently asked questions, industry trends and definitions, and yes, we will have fun doing it.
How to Create Good QM Evaluation Questions
===
[00:00:00] Dave Hoekstra: Hello everyone and welcome to another episode of Calabrio Shorts. We have a great topic today that we are really excited to get into because it is something that comes up quite a bit. You know, organizations spend a lot of time working on their quality processes and making sure that things go well, but a lot of times before they even get started, one of their primary questions is really how do we create good evaluation questions, right?
[00:00:25] Dave Hoekstra: How do we make sure. We're being objective, but also subjective when we need to, and making sure we really hit those. And so, you know, instead of just listening to me ramble all day, our special guest today for Calabrio Shorts is Therese Fruth and Therese is a Enablement Program Manager here at Calabrio.
[00:00:43] Dave Hoekstra: And Therese, I know you've spent a lot of your career kind of working in the quality monitoring and QM contact center space. So let me, I'm gonna start with a question, but before we do I'd really love to hear what's your background? How did you get into this magical adventure we call contact centers?
[00:01:01] Therese Fruth: Well, Dave like most people, I spent days just dreaming about what my life would be like as a Contact Center Manager. Right? Because that is you too dream. Yes, it is the true dream. It, it like most actually, seriously, it did land in my life very surprisingly. I came from more of a finance background and I worked for a school district here in Minnesota and I was doing finance and there came an opportunity to create a twenty four seven, three sixty five call center.
[00:01:29] Therese Fruth: And I'm like, what the heck is that? Like . So, but I was like I'm down for of games. So I jumped in and literally, Center from the ground up. I had to do, you name it. I had to do it. And then of course, quality became my passion. Learning how to really motivate my agents and get them on board and help our parents with their students.
[00:01:50] Therese Fruth: I mean, when you're talking about. A huge portion of what we did was transportation related. And when you're talking with parents who are frantic cuz their little child is two hours late those are some crucial phone calls. And being able to coach agents on how to handle those properly and quickly, but still get to the, you know, where they need to get to us.
[00:02:07] Therese Fruth: Just a great experience.
[00:02:10] Dave Hoekstra: that's fantastic. And it's good to know that there are people looking out for our kids out there, right. And making sure that, you know, the school bus drops off every single day in front of my house, and I hear it every day. And this army and gaggle of kids come out. And it's always fun to, to kind of see them.
[00:02:25] Dave Hoekstra: It's funny how much energy they have getting on the bus and then getting off the bus. It's a very different story.
[00:02:31] Therese Fruth: but, Unless
they have bunch sugar on the way home.
[00:02:33] Dave Hoekstra: Right. Which happens after Halloween for sure. Here, so one of the things that we talk about when we get into quality, and one of the questions that we hear from a lot of people when they're first really kind of starting to get their journey going is, you know, they're tasked with creating an evaluation form.
[00:02:50] Dave Hoekstra: Right. And a lot of people are to the point where they could probably create a pretty basic form, but the, they want to do it. Right. And one of the ways to do it right is to really take a look at those questions and make sure that they are appropriate. Now, my question to you is what makes an appropriate question, right?
[00:03:08] Dave Hoekstra: What is, what are some of the keys to really creating that, that good evaluation question?
[00:03:13] Therese Fruth: Absolutely. There are three main things you really need to focus on when you're creating your evaluation forms. And one is, do they relate to your customer? Right. So you wanna make sure the questions you're asking are actually getting to what you're trying to search to see if a customer was happy.
[00:03:28] Therese Fruth: So if you're asking questions that don't get to that, that you know, aren't going to measure whether the customer was happy or not, then that was, that's one miss. You also wanna make sure that you are relating to the organizational success. So what's important to my organization? What are the key metrics that they want me to report on?
[00:03:48] Therese Fruth: You have to hit all of those elements so that you can report to them on whether maybe there is a certain measure that needs to be taken from your agents. Maybe there is a compliance piece that's important. They have to do certain things. And then lastly is, do they relate to the agent success?
[00:04:04] Therese Fruth: And this is my favorite because we're not about an I gotcha. Evaluations and quality is not to penalize an agent. It is to raise them up and grow and help them learn. And so really you want to be including questions that are going to help them grow within their roles.
[00:04:24] Dave Hoekstra: Yeah, so I'm hearing a theme here, right?
[00:04:26] Dave Hoekstra: The theme is really making sure that the questions match what the organizational goals are. And I think that's great because I think we forget about that a lot. We forget about the goal of the organization, right? The mission statement of the entire organization is to, you know, deliver this top notch
[00:04:44] Dave Hoekstra: customer experience, effort. And then we get into the questions their questions like, you know, if, did you say the customer's name 37 times? In the call . And that doesn't necessarily lead to the best customer experience. Right. And I really like that. And the other theme and I'll reiterate this back when
[00:05:03] Dave Hoekstra: at least when I first started, and probably when you were getting going in your in your QM journey, there was a lot of gotcha questions. We designed questions that looked to really single out the agents and make sure that they were bad. We went Aha. You fell into our trap kind of a thing.
[00:05:21] Therese Fruth: Yeah. Yeah. It was, you know, a lot of, I, I remember back when I first was creating 'em I went about it the wrong way. I mean, I'm gonna say I and I think most of us do when we first create our one, right? I took all of the things that my agents were doing incorrectly and built those into my questions cause I'm gonna correct all this stuff.
[00:05:39] Therese Fruth: Right? Right. And it. Silly things like, you know, to your point, did it say it so many times, but it was more, did they use the greeting exactly the way I wanted 'em to greet the customer? Did they use their name? Did they say this? Did they do it in this order? Did they do? And if they didn't, I was gonna mark 'em off on that.
[00:05:54] Therese Fruth: And I was like, finally I learned. I'm like, where am I getting with this? Right? Like, it is not all about did they do this right? Were they, you know, were they polite? Were, was there good empathy, was there, those were things that were more important than if they actually said, you know, good morning, thank you for calling Minneapolis Public Schools.
[00:06:12] Therese Fruth: This is Therese, how may I help you? I mean, really? Did that matter?
[00:06:17] Dave Hoekstra: No. And a lot of times you have to stop and think about it as a customer. Do I really care if your greeting is different than the next person? No, probably not, right? The rules of what we have to say on a call usually only fall into a legal spot, right?
[00:06:32] Dave Hoekstra: , you have to read a safe harbor statement or you have to read a compliance statement or something like that. And other than that it's about making sure the customer's experience. Matching to the organizational goals. And that's, I think that's a really great thing is the verbatim greeting could be one of the most hotly contested QM questions of all time.
[00:06:51] Dave Hoekstra: Right. Or the closing, ugh. Don't even get me started on repeating the closing. Exactly. This, is there anything else I can help you with, Dave? Yeah, . Well, you know what's funny? Still, it still throws me off when I call a contact center and the person asks that question, which is great, right? Is there anything else?
[00:07:06] Dave Hoekstra: And they go, okay, bye. And I'm like, oh, you are gonna get so knocked off on your QM. And then I remember that's just the old school way of doing things and not necessarily the new school way, which is great. We're evolving as a QM team. And that's really what I love to hear is that we're spending less time.
[00:07:24] Dave Hoekstra: Focusing on the agent doing something wrong, and more time focusing on the customer, getting the experience they need, which is a really, really fantastic piece. Right. So let's take it from the situation of somebody who's just getting started and creating a form. What kind of questions would you kind of start with?
[00:07:40] Dave Hoekstra: And I know it's sometimes they're very industry specific, so, but from a general standpoint, gimme an example of what a good question might look like and then maybe what a bad question might look like.
[00:07:51] Therese Fruth: Sure. A good question is gonna be very specific and direct. And it's usually, I mean, when, whenever possible it's gonna have a yes or no, they either did this or they didn't do this, it's going to be easier for your evaluators to know if they met the standards or didn't.
[00:08:05] Therese Fruth: And it's also gonna be easier for the agent to know, you know, what is expected of them. And it's gonna ask one specific thing. So what I see a lot is like double barreling where they'll ask a question, but really they wanna know the answer to three or four. So, like, for example, a really bad one would be did the agent use the right greeting and did they say, can I help you?
[00:08:26] Therese Fruth: And did they, you know, and so it's got all these things on top of each other, and then you're like well, yes, but they did it in this order, but they didn't do that. Or, you know, it makes it really difficult to figure out what the heck, how, you know, how did
[00:08:39] Dave Hoekstra: the agent say the, did the agent say the greeting correctly?
[00:08:42] Dave Hoekstra: And did they have a smile in their voice? Right. Yeah. That's a double barrel
[00:08:45] Therese Fruth: question. It is a double barrel question, but that is a very good question. The, because one thing that is in terms of the smile, if I just asked, did the agent sound like they were smiling? Cuz you can hear a smile through a question when someone says it.
[00:09:01] Therese Fruth: So I mean, I would really argue someone who says that they can sound excited and happy with a frown on their face just as much as they can with a smile on their face. So it's not a silly question.
[00:09:13] Dave Hoekstra: Right. Okay. So from a generic customer service standpoint, , what are some more good questions?
[00:09:20] Therese Fruth: Yeah, so you're gonna wanna make sure that you're touching on some of those soft skills, right? So some professionalism skills or some things that gain a customer's trust. So for example, like instead of just saying a bad one would be did the customer, did the agent use empathy? Well, what the heck does that mean, right?
[00:09:38] Therese Fruth: So a better way to say, did the agent use empathy? Would be. Did the agent listen intently and did they ask follow up questions or did they, you know, which is a little bit double barreling on there, but it is. You wanna make sure, cuz one without the other doesn't show that they actually had that empathy.
[00:09:56] Therese Fruth: So in some cases there is a double one,
[00:09:59] Dave Hoekstra: right? Because you know, you could be sitting there listening to a call and the agent could show empathy. And do nothing about it. Right. And so that's a good example of what's the goal, right? I can definitely say to you Therese, I'm so sorry to hear that you went through that.
[00:10:14] Dave Hoekstra: Okay. Any other questions? Right? And it's like, no. You could have just, you could have just asked the next question. It would've been, you know, that, that's a good example of those types of things. If I said instead of asking this, ask this, right? Instead of saying, was the agent nice? Say, did the agent show the appropriate amount of empathy that matched the tone of the call?
[00:10:39] Therese Fruth: Did the agent show the right level of empathy? Kind of sticking on that empathy thing, right? Well, what is the. Level of empathy. Instead of saying that, we say, we could say, did the agent convey compassion by responding with courtesy and empathy to the callers request? Right? So it's more specific about how they responded.
[00:11:01] Therese Fruth: Were they courteous? Were they this, were they, you know, versus just the right level. The right level is too subjective. It's just okay, you don't know. You know what that level is. But one way to kind of get around that too, and if you have to have questions like that is to create a standards quality document.
[00:11:19] Therese Fruth: I highly suggest that more than like, I mean, that is both within your evaluators and your agents who see that, right? So they know that when you do have some of those subjective things. Where you are asking something, where it isn't a yes or no, and there is a level of it, you actually provide examples of what that looks like.
[00:11:38] Therese Fruth: So you provide a good answer would look like this. A great answer would look like this. A, you know, a bad answer would look like this. So it takes that questions out
[00:11:48] Dave Hoekstra: of it. That's a, that's such a great idea. The standards quality document. It's actually kind of like the, you know, the guidelines to how we answered that question.
[00:11:57] Dave Hoekstra: And, you know, we, we did an episode on calibrations, and calibrations are a really key, important part of a good quality program. But this standards document, right? So does that kind of include maybe going down the form and really hitting each question and kind of just giving some guidance and background to each.
[00:12:16] Therese Fruth: Yeah, it not only gives the guidance for that, but it really aligns to make sure like we talked about earlier, that those, that your form actually aligns back up to the organization's success that you're touching on all of those KPIs that are important to the organization, and you can show why those are important within that.
[00:12:33] Therese Fruth: So besides just the question, you can say, you know, we're measuring empathy. Well, why is empathy important? In that document, you actually list why it's important to your organization. And then you show examples of what that's like.
[00:12:46] Dave Hoekstra: That's so, it breaks it down. That's really great. And you know, so for those of you who might be listening, I think that's a really great place to start.
[00:12:53] Dave Hoekstra: When you are looking at your questions to make sure that they're appropriate. Because if they're not, you're gonna spend four and a half pages explaining to people, and that's probably a really good measuring stick as to whether or not it's a good evaluation question or not.
[00:13:06] Therese Fruth: Yeah. It also, not only does it do that, it does instill confidence in the whole process.
[00:13:11] Therese Fruth: Right. So I'll take you back to like, when I first started doing the question, of course, you gotta remember the, my call center was brand new. The people who took on these roles had other roles within our organization. They also too fell into this. Oh my gosh, now I'm in a call center and what the heck does this mean?
[00:13:26] Therese Fruth: Right? And they were being evaluated on all this stuff and told all the things they were doing wrong and all the things that they needed to work on and they didn't understand. And so as soon as I created the quality document that said, here is what we're looking for in a call. This is what we're trying to find and this is why we're trying to find it.
[00:13:45] Therese Fruth: It was like the light bulb went on. It was like, oh, you're not just telling me that I, you know, pardon me, but, you know, suck as a person. Cause that's what they felt like. I was like coming at them personally. Yes. That's what they felt like, you know, they were being attacked and as soon as they. Stood the why and what was going on.
[00:14:05] Therese Fruth: It was like a whole different, it was just a whole different picture. So,
[00:14:09] Dave Hoekstra: and I'll give you an anecdote from my background. I worked at a call center where instead of creating the quality standards document, what they did was, They tried to create the robotic perfect call agent by having upwards of 75 elements on a form, and it create, I mean, granted, a lot of people got 100 s on their calls, but every call took 22 minutes.
[00:14:36] Dave Hoekstra: And we, even we, this is actually one of my favorite stories. We had one agent. Take the form, go, had it laminated and would sit there with a grease pencil during her calls and check off each form to make sure she managed everything. One she took about seven calls a day, but she made 100 s and we said, okay, there's something wrong here.
[00:14:55] Dave Hoekstra: We gotta reevaluate this. It was a, it was one of those things that, like you said, early on, in our careers we, nobody told us this was a bad idea. And that's what we're trying to do here is maybe give you some of the bad ideas mixed in with the good. Now one thing we haven't touched on really is how important the questions are when it comes to reporting and trending.
[00:15:15] Dave Hoekstra: Right. And making sure what are some things about making sure that those reports show up the way we want to? You and I just talked about how fun reports are to run. Let's talk about QA reports and why the questions are important.
[00:15:28] Therese Fruth: Yeah. You know, I'll kind of go back to what I said before too.
[00:15:31] Therese Fruth: The more specific you can get, it's going to help your reporting. And of course the less questions you can put in the actual survey is gonna help too. Right? So I really recommend keeping your Forms to less tens, kind of the magic number. Less than 10 sections and less than 10 questions per 10 sections or per each section.
[00:15:54] Therese Fruth: Now, typically we like to see maybe a form be around maybe 30 questions at the most. Would be a, you know, a better form size. Because to your point, when you're running a report, it is way too hard to focus on what needs to be fixed or where do we need to focus our time, or where do we need to, you know, Things that matter most.
[00:16:15] Therese Fruth: So you really need to make sure that you're just honing in on those key objectives that are important and making them as specific as possible so that when you do run those reports they're much easier to be able to track you know, the actions or behaviors that you're trying to capture,
[00:16:31] Dave Hoekstra: right?
[00:16:32] Dave Hoekstra: Because if you don't have to explain. Why the last eight months of a trend is happening, that means you have a good question, right? It's as well if there's lots of its and ifs and maybes and that kind of thing. The other really important thing about building these questions off correctly at the beginning is because you kind of really can't change a mid mid trend.
[00:16:54] Therese Fruth: It will really throw off your trend reports. If you are, you know, we recommend, I mean, obviously you're going to wanna revise your evaluation forms like, say once a year to make sure that they are keeping up. But you don't really wanna do it too much more than that because yes, to your point, if they completely change, if you add in five more questions that weren't there the month before, now you're not comparing apples to apples.
[00:17:17] Therese Fruth: You're now comparing, you know, two different things. Apples
[00:17:21] Dave Hoekstra: to trucks or Correct. You know? Yeah, correct.
[00:17:23] Therese Fruth: But it is important. And I know you said you were talking about some calibration sessions before, and that's really where you can take the time to tweak some of them. And it's okay to drop some off if you find out that they're not working as well.
[00:17:35] Therese Fruth: You still want, you know, I mean, if it's all garbage , we know that if garbage is in, garbage is coming out. So it is good to tweak 'em, and especially if. Tried a new form for the first time and it's not working, you know, don't feel like, oh my gosh, I have to be stuck with this form for the next year because, you know, I heard Dave and Teresa say that you can't switch 'em around too much when you're first starting.
[00:17:57] Therese Fruth: There is gonna be some tweaking and it's, that's inevitable. But once you have a solid report of where you wanna be, try to keep it as close to that as possible.
[00:18:05] Dave Hoekstra: Yeah. Yeah. We're not saying don't change the form or don't change the question. We're saying if something doesn't work, you need to change.
[00:18:12] Dave Hoekstra: Right. And even if that just does, but you also have to understand the consequences. If you change the form, it has a pretty decent chance of kind of invalidating your history and it's hard to see growth or decline over a time period when every month and a half you keep tweaking the question or changing the meaning, that kind of thing.
[00:18:31] Dave Hoekstra: So we just, we're just saying be careful, that's all.
[00:18:33] Therese Fruth: Yeah. It's even more important when you're, if you're using Predictive Because if you're using Predictive and you keep tweaking it, it's going to have a harder time understanding where you're going .
[00:18:43] Dave Hoekstra: So, absolutely. And we'll talk about Predictive in a So, you know, I think this has been so great and it's super helpful and I think a lot of people are really gonna benefit from the discussion that we had here today. As kind of is tradition with our Calabrio Shorts that we have a guest. I like to give the guest the final word here. So what's the final word on creating good evaluation questions from Therese?
[00:19:06] Therese Fruth: You know, to be honest, I would really say that it's really thinking about what is behind your question. It's kind of a little plan words there a little bit, but really make sure that you're asking the what, and the why. Those are two major things, right? So what are you trying to measure? What are you trying to look at?
[00:19:24] Therese Fruth: What are you trying to improve? What are you trying to, whatever it happens to be and why? If your why is not benefiting the organization or benefiting the agent or benefiting your customer, don't put it in there. You wanna make sure that there's a really good reason why. The question is in there.
[00:19:42] Dave Hoekstra: So that's great. That's, I think that's a great kind of final stamp on the discussion here. So, I want to first thank you there for being a part of this, and this is super informative and helpful for all our Calabrio customers out there, and it's been great. I wanted to mention, you mentioned Predictive.
[00:19:58] Dave Hoekstra: If you want more information on predictive evaluations, Go to Calabrio.com. There's a lot of great stuff there. But as always, if you want to discuss things like this with any of us, go to Calabrio.com let us know. We're happy, we love to have discussions like this. We have tons of people like Therese within the organization that love to talk about these kind of things.
[00:20:16] Dave Hoekstra: So if you have questions that you feel like we can help with, that's what we're here for and that's why we do this podcast. So Therese, thank you so much for joining us. I really appreciate the time. For those of you that are listening, we always appreciate you spending a few minutes of your day with us.
[00:20:30] Dave Hoekstra: As always, if you have any great ideas about what you'd like to hear next on the Calabrio Shorts podcast, let us know. Send in that suggestion email. We'll be glad to to take it under advisement. So Therese thank you so much. Thanks for having me, Dave. Absolutely. And for those of you out there, have a great rest of your day.
[00:20:46] Dave Hoekstra: Thank you so much. From Calabrio Shorts. For more information, go to Calabrio.com and let us know. We'll be glad to take care of you. As for that, my name is Dave Hoekstra thanks everybody. Have a great rest of your day.