1 00:00:05,640 --> 00:00:09,090 Voiceover: Welcome to Testing, Testing 1-2-3 a podcast 2 00:00:09,110 --> 00:00:13,420 brought to you by TestGenius. 3 00:00:13,933 --> 00:00:14,843 Jenny Arnez: Welcome everybody. 4 00:00:14,973 --> 00:00:17,663 Welcome to Testing, Testing 1-2-3. 5 00:00:17,843 --> 00:00:21,283 It's a podcast produced by TestGenius and Biddle Consulting Group. 6 00:00:21,803 --> 00:00:23,073 And I'm Jenny Arnez. 7 00:00:23,093 --> 00:00:25,803 I'm your host and I'm joined today with Mike Callen. 8 00:00:25,813 --> 00:00:29,223 He's our co-host, but also the president of TestGenius. 9 00:00:29,323 --> 00:00:30,353 Mike, you want to say hi? 10 00:00:31,093 --> 00:00:31,583 Mike Callen: Good morning. 11 00:00:32,663 --> 00:00:33,273 Jenny Arnez: Good morning. 12 00:00:33,273 --> 00:00:33,483 Yes. 13 00:00:33,493 --> 00:00:34,353 Or good afternoon. 14 00:00:34,703 --> 00:00:35,213 Good evening. 15 00:00:35,243 --> 00:00:38,203 Depending on when you're depending on wherever 16 00:00:38,203 --> 00:00:39,703 Mike Callen: you are on this round globe. 17 00:00:40,158 --> 00:00:40,868 Jenny Arnez: That's right. 18 00:00:40,908 --> 00:00:42,968 And we're sure glad you joined us today. 19 00:00:43,508 --> 00:00:47,618 So for today's episode, we're going to do things a little bit differently. 20 00:00:47,648 --> 00:00:50,848 We don't have an additional guest joining us today. 21 00:00:51,418 --> 00:00:55,948 What we're going to do is look at a specific question that 22 00:00:55,988 --> 00:00:57,348 a client recently asked Mike. 23 00:00:57,543 --> 00:01:02,493 It is all about some best practices that they could incorporate 24 00:01:02,533 --> 00:01:04,953 in their hiring practices. 25 00:01:05,603 --> 00:01:10,813 And it led to an interesting conversation about, validity and adverse impact. 26 00:01:11,253 --> 00:01:17,193 And so today what I'm going to do is present Mike with a scenario very similar 27 00:01:17,223 --> 00:01:19,833 to what our client offered to him earlier. 28 00:01:20,813 --> 00:01:22,933 And just have a conversation around that. 29 00:01:23,503 --> 00:01:24,403 Does that sound good, Mike? 30 00:01:24,983 --> 00:01:26,093 Mike Callen: Yeah, absolutely. 31 00:01:26,883 --> 00:01:31,593 Jenny Arnez: Alright, so let me present the scenario. 32 00:01:32,133 --> 00:01:39,683 Let's just say Acme County has reached out, and they want to hire, or they 33 00:01:39,683 --> 00:01:42,323 are hiring for Dispatcher I positions. 34 00:01:42,933 --> 00:01:44,123 Now here's some background. 35 00:01:44,873 --> 00:01:48,343 Each department in the county determines their own requirements 36 00:01:48,353 --> 00:01:50,113 for the Dispatcher I position. 37 00:01:51,143 --> 00:01:55,913 All departments are using a pre employment skills test. 38 00:01:57,083 --> 00:02:02,513 And after a phone interview, selected candidates take the pre employment test. 39 00:02:03,303 --> 00:02:06,373 It's only the highest scoring candidates that are selected 40 00:02:06,373 --> 00:02:08,893 for an in person interview. 41 00:02:10,083 --> 00:02:13,703 Acme County has reached out to us and they're concerned that their hiring 42 00:02:13,703 --> 00:02:19,393 practices, including the pre employment skills tests, are resulting in adverse 43 00:02:19,393 --> 00:02:21,813 impact towards protected groups. 44 00:02:22,793 --> 00:02:25,113 Mike, how would you help a client like this? 45 00:02:26,993 --> 00:02:30,143 Mike Callen: Yeah, there's a lot to unpack there. 46 00:02:30,203 --> 00:02:35,998 I, if maybe if we step down through, Each of those bars on this 47 00:02:36,068 --> 00:02:39,348 image right here, we can talk a little about that about each one. 48 00:02:39,938 --> 00:02:46,688 So you have a county that has different departments, all of whom have a job title 49 00:02:46,938 --> 00:02:51,698 called Dispatcher I and then they're determining their own requirements 50 00:02:51,698 --> 00:02:54,518 for these, each of these positions. 51 00:02:54,528 --> 00:02:59,468 Presumably they're slightly different from location to location and so one 52 00:02:59,468 --> 00:03:04,048 of the things that you always want to be concerned with if you're an employer 53 00:03:04,348 --> 00:03:10,968 is the optics in terms of how do things look and feel to the applicant. 54 00:03:11,643 --> 00:03:17,163 And in the case of a county or a private organization, often the applicants 55 00:03:17,163 --> 00:03:19,543 are constituents or customers. 56 00:03:19,773 --> 00:03:23,703 And you want to be a good citizen in your environment and you want to be 57 00:03:23,773 --> 00:03:29,403 doing things that feel good and fair and righteous in terms of their eyes so that 58 00:03:29,403 --> 00:03:31,663 you don't, sully your own reputation. 59 00:03:31,663 --> 00:03:37,633 And if the Dispatcher I position is different in each of those 60 00:03:38,028 --> 00:03:42,218 organizations, it might be better to title them slightly differently. 61 00:03:42,558 --> 00:03:46,118 For instance, it could be Dispatcher One Fire, Dispatcher One Police, 62 00:03:46,358 --> 00:03:49,808 Dispatcher One EMS, or however they're dividing things up. 63 00:03:50,098 --> 00:03:55,018 And that way you get a little bit of disparity from one title to the next. 64 00:03:55,318 --> 00:04:01,693 And I think the reason that I mentioned that is that you have job applicants 65 00:04:02,533 --> 00:04:04,063 who are going to talk to one another. 66 00:04:04,063 --> 00:04:08,443 And if you apply for a position over here and you're tested or you're minimally 67 00:04:08,443 --> 00:04:12,803 tested and you apply for another one over here and you're maximally 68 00:04:12,803 --> 00:04:15,803 tested, they really wear you out. 69 00:04:16,123 --> 00:04:20,003 That's going to feel weird if they're all titled the same thing. 70 00:04:20,493 --> 00:04:27,638 The same job title within different departments could vary and it 71 00:04:27,638 --> 00:04:31,018 could be justifiable to have different recruitment processes. 72 00:04:31,228 --> 00:04:33,688 It probably is justifiable to have different recruitment 73 00:04:33,758 --> 00:04:35,858 processes for each position. 74 00:04:36,128 --> 00:04:40,408 But again, in terms of optics, it might be something to differentiate 75 00:04:40,408 --> 00:04:44,078 between the Dispatcher I position at each of those locations. 76 00:04:44,088 --> 00:04:47,048 So that's definitely one thing to look out for. 77 00:04:47,398 --> 00:04:49,058 Another thing to look out for, 78 00:04:49,318 --> 00:04:52,438 and this comes up very often in these kinds of conversations, 79 00:04:52,918 --> 00:04:55,138 is within the recruitment. 80 00:04:55,208 --> 00:05:00,478 Let's say your recruitment process is different in each of those three or four 81 00:05:00,528 --> 00:05:06,888 departments but your recruitment process within that one particular recruitment at 82 00:05:06,888 --> 00:05:09,158 one department, it's got to be the same. 83 00:05:09,188 --> 00:05:13,618 You can't do different things for different applicants 84 00:05:13,668 --> 00:05:15,258 within the same recruitment. 85 00:05:15,288 --> 00:05:17,098 So you want to be really consistent in that regard. 86 00:05:19,038 --> 00:05:23,508 Jenny Arnez: so this might be a bit of a tangent, but you mentioned 87 00:05:23,508 --> 00:05:28,588 about departments having perhaps changing the job title where 88 00:05:28,588 --> 00:05:30,808 they don't all say Dispatcher I. 89 00:05:31,508 --> 00:05:36,968 I'm wondering how job analysis might relate to that we've had previous 90 00:05:36,968 --> 00:05:41,118 clients or previous guests who've talked about job analysis, just wondering if 91 00:05:41,118 --> 00:05:42,428 you might be able to touch on that. 92 00:05:42,858 --> 00:05:43,488 Just briefly. 93 00:05:43,488 --> 00:05:43,958 Yeah. 94 00:05:44,343 --> 00:05:45,093 Mike Callen: Absolutely. 95 00:05:45,243 --> 00:05:49,693 So a selection plan, which is any part of the recruitment process from beginning 96 00:05:50,053 --> 00:05:53,683 to what should be the job posting all the way through onboarding somebody 97 00:05:54,523 --> 00:05:57,233 should be based upon a job analysis. 98 00:05:57,263 --> 00:06:03,463 This is best HR practices and a job analysis is a really deep dive into, 99 00:06:03,463 --> 00:06:07,353 what the what are the tasks and duties that are performed on the job? 100 00:06:07,683 --> 00:06:09,543 What would be the minimum qualifications? 101 00:06:09,543 --> 00:06:11,973 What would be the desirable qualifications? 102 00:06:12,163 --> 00:06:17,003 What would be the work schedule, the work environment, what personal characteristics 103 00:06:17,003 --> 00:06:21,143 are required in order to perform the job. 104 00:06:21,143 --> 00:06:21,803 So it's really 105 00:06:22,113 --> 00:06:28,043 all of the technical aspects that have to do with any particular position 106 00:06:28,303 --> 00:06:33,803 would be fleshed out in a job analysis and job analyses are really time 107 00:06:33,803 --> 00:06:37,283 consuming, challenging tasks to perform. 108 00:06:37,593 --> 00:06:42,213 And while everybody's supposed to have current job analyses, they very 109 00:06:42,213 --> 00:06:47,233 often don't, and it becomes one of those eating the elephant situations. 110 00:06:47,433 --> 00:06:54,083 How do I do, job analyses when I have 500 different positions within my county or 111 00:06:54,093 --> 00:06:56,363 my organization or company or whatever? 112 00:06:56,713 --> 00:07:01,948 And so the answer for that is, you basically just do it one bite at a time. 113 00:07:02,188 --> 00:07:07,498 It's best in my recommendation, it's best to start with the positions that 114 00:07:07,498 --> 00:07:14,158 are most often recruited for, or the positions that are most often hired 115 00:07:14,488 --> 00:07:17,448 and then reallocated within the organization. 116 00:07:17,448 --> 00:07:22,298 So you have these entry level positions that people come into and then rather 117 00:07:22,298 --> 00:07:26,788 than posting a job externally, they posted internally and then they move 118 00:07:26,798 --> 00:07:28,898 people around within their organization. 119 00:07:29,088 --> 00:07:31,188 So those would be the ones to start with. 120 00:07:31,488 --> 00:07:38,038 And then you keep going, doing the next most high volume positions and until 121 00:07:38,038 --> 00:07:39,838 you get all the way through the process. 122 00:07:39,838 --> 00:07:42,258 And then it's painting the Golden Gate Bridge. 123 00:07:42,258 --> 00:07:46,648 Once you get done, they're really old in three years or when the 124 00:07:46,708 --> 00:07:48,618 position fundamentally changes. 125 00:07:48,618 --> 00:07:51,308 And so you really need to go back and do them again. 126 00:07:51,558 --> 00:07:55,788 But you can use the existing job analyses for the redo, which can 127 00:07:55,788 --> 00:07:57,678 make the process a lot easier. 128 00:07:57,828 --> 00:08:04,058 So job analysis is super important in this particular practice and some 129 00:08:04,058 --> 00:08:08,378 things that come from the job analysis are the job description and the job 130 00:08:08,378 --> 00:08:13,108 description would be a formal paragraph or several paragraphs that talk about 131 00:08:13,108 --> 00:08:15,628 what this person does in this position. 132 00:08:15,858 --> 00:08:20,718 And then from the job description, would be distilled, typically a job posting, 133 00:08:20,718 --> 00:08:26,058 which would be the text that's put into an ad or a posting online that talks 134 00:08:26,058 --> 00:08:30,738 about the job title that's gonna be opening up, some basic requirements, 135 00:08:30,738 --> 00:08:35,758 minimal requirements, some desirable qualifications, and then that little 136 00:08:35,818 --> 00:08:37,888 summary of the job description. 137 00:08:38,308 --> 00:08:41,218 So those, they're a very important part of the selection process. 138 00:08:41,723 --> 00:08:42,533 Jenny Arnez: Okay great. 139 00:08:42,553 --> 00:08:46,463 And actually, I think what I'll do is in our show notes, 140 00:08:46,933 --> 00:08:49,573 I'll post a link to IO predict. 141 00:08:49,603 --> 00:08:53,123 They're one of our partners who actually does job analysis. 142 00:08:53,673 --> 00:08:54,693 And Dr. 143 00:08:54,693 --> 00:08:57,643 Clinton Kelly has been on our show before and talked about that. 144 00:08:58,903 --> 00:09:01,073 Because I think definitely that would be very helpful. 145 00:09:02,083 --> 00:09:04,563 Definitely relates to this scenario here. 146 00:09:05,113 --> 00:09:09,783 As we consider this process that Acme County has for hiring Dispatcher I, 147 00:09:10,693 --> 00:09:13,783 where are the tests in this process? 148 00:09:13,873 --> 00:09:15,103 What are the tests? 149 00:09:15,673 --> 00:09:18,223 Mike Callen: Yeah, that's, it's a really important point 150 00:09:18,293 --> 00:09:19,913 that you're making here, Jenny. 151 00:09:20,263 --> 00:09:25,653 When we're approached by clients or prospects and they talk about, testing 152 00:09:25,653 --> 00:09:30,983 and validation and adverse impact, the focus is always on the literal test. 153 00:09:31,688 --> 00:09:37,048 And the fact of the matter is that there's many other steps of the recruitment. 154 00:09:37,048 --> 00:09:38,228 There are also tests. 155 00:09:38,498 --> 00:09:43,118 I was just talking to Dan Biddle, our CEO, a few minutes ago, and he was 156 00:09:43,118 --> 00:09:48,148 telling me a story about he did some consulting for the Department of Justice. 157 00:09:48,608 --> 00:09:54,898 And he was at a conference where there were about 500 HR practitioners and 158 00:09:54,898 --> 00:10:03,158 the head of the DOJ asked all of the attendees to raise their hands if they 159 00:10:03,158 --> 00:10:06,358 agreed that an interview was a test. 160 00:10:06,923 --> 00:10:09,733 And it was half or less that raised their hand. 161 00:10:10,283 --> 00:10:13,933 And the fact of the matter is an interview is a test. 162 00:10:14,283 --> 00:10:17,153 There are a variety of things called PPTs. 163 00:10:17,193 --> 00:10:22,043 Those are practices, procedures, and tests that occur throughout the 164 00:10:22,043 --> 00:10:28,458 recruitment process in which decision making factors are considered in 165 00:10:28,458 --> 00:10:31,058 this employment selection process. 166 00:10:31,398 --> 00:10:34,338 An interview absolutely positively is one. 167 00:10:34,678 --> 00:10:38,648 People sit down, they talk to another human being or a team of people and 168 00:10:38,648 --> 00:10:43,743 they're rated on their responses and some people are moved forward and 169 00:10:43,743 --> 00:10:45,143 some people aren't moved forward. 170 00:10:45,143 --> 00:10:50,283 And that is part of the selection process and therefore an interview is a test. 171 00:10:50,583 --> 00:10:55,243 But there's other things that are PPTs that also act like tests. 172 00:10:55,473 --> 00:10:59,143 For instance, we talked earlier when you brought up the job analysis, one of the 173 00:10:59,143 --> 00:11:04,673 derivatives of the job analysis is the job posting and the job posting talks about, 174 00:11:04,883 --> 00:11:07,513 hey, here's what you perform on the job. 175 00:11:07,803 --> 00:11:12,923 And if the job posting and its minimum requirements are not valid, 176 00:11:12,963 --> 00:11:17,503 that is, they're not matched exactly to that particular position, then 177 00:11:17,533 --> 00:11:22,543 anyone who's reading the job posting may actually deselect themselves 178 00:11:22,613 --> 00:11:26,773 from the process based upon the information that they've been provided. 179 00:11:27,233 --> 00:11:31,503 And if the information that's provided is incorrect, then you 180 00:11:31,503 --> 00:11:36,803 could have just created adverse impact in the very first step of your 181 00:11:36,803 --> 00:11:41,833 recruitment process by not making an accurate job posting and especially 182 00:11:41,893 --> 00:11:44,613 accurate minimum qualifications. 183 00:11:45,203 --> 00:11:46,523 So that's a test too. 184 00:11:46,763 --> 00:11:52,013 And every little intermediate step along the way that either allows 185 00:11:52,013 --> 00:11:54,603 certain people to continue or 186 00:11:54,923 --> 00:12:00,563 doesn't allow people to continue or rank orders people or weights their scores. 187 00:12:00,583 --> 00:12:06,053 Any of those aspects of the recruitment process that make those kinds of 188 00:12:06,053 --> 00:12:12,013 decisions are also tests and should be scrutinized in the same way that 189 00:12:12,483 --> 00:12:15,683 people and we scrutinize our own tests. 190 00:12:17,043 --> 00:12:17,393 Jenny Arnez: Okay. 191 00:12:17,393 --> 00:12:21,313 So if I'm understanding you correctly, what you're saying is like in our scenario 192 00:12:21,333 --> 00:12:24,333 here, Acme County, they're concerned about 193 00:12:24,573 --> 00:12:29,688 there being adverse impact and certainly they need to look at making sure that 194 00:12:30,098 --> 00:12:33,858 they validated the actual pre employment test, but they actually need to look at 195 00:12:33,858 --> 00:12:39,258 their whole process from job description, job posting, interviewing process, 196 00:12:39,258 --> 00:12:43,578 that phone interview that happens that determines who moves on to the next step 197 00:12:43,578 --> 00:12:44,883 Mike Callen: background checks. 198 00:12:44,933 --> 00:12:49,158 Yeah, that's absolutely correct what you're saying there. 199 00:12:49,458 --> 00:12:57,248 And to that end, people will ask us for, data in terms of pass rates for, certain 200 00:12:57,618 --> 00:13:00,008 protected groups, EEO protected groups. 201 00:13:00,298 --> 00:13:03,928 And we don't have that data because we're not, getting that 202 00:13:04,288 --> 00:13:07,458 EEO data from the applicants. 203 00:13:07,498 --> 00:13:11,338 The employers who are going through the recruitment, they're 204 00:13:11,338 --> 00:13:12,448 getting that information. 205 00:13:12,448 --> 00:13:17,628 They're collecting that information up front and they have a piece of software. 206 00:13:17,913 --> 00:13:21,573 Typically known as an applicant tracking system- an ATS. 207 00:13:21,833 --> 00:13:27,223 And the ATS will actually do analysis at each of these steps to let them 208 00:13:27,223 --> 00:13:32,073 know where they may potentially have problems so they can dig into it and 209 00:13:32,073 --> 00:13:35,863 they can see whether or not there's some adjustments that need to be made. 210 00:13:36,103 --> 00:13:38,183 Now every time 211 00:13:38,423 --> 00:13:43,263 there's adverse impact against a protected group, it doesn't necessarily mean that 212 00:13:43,263 --> 00:13:44,913 you have to do something different. 213 00:13:45,253 --> 00:13:51,103 There are instances where adverse impact occurs and that adverse impact 214 00:13:51,103 --> 00:13:56,113 is simply a part of the process, the part of living in a real world. 215 00:13:56,353 --> 00:13:58,593 And we can talk a little bit later about that as well. 216 00:13:58,983 --> 00:14:01,808 But you always want to stop and you want to look and you want to ask ," Okay. 217 00:14:02,078 --> 00:14:03,548 Is there adverse impact? 218 00:14:03,558 --> 00:14:06,048 Why is that adverse impact occurring? 219 00:14:06,898 --> 00:14:10,448 What could we do differently to avoid that adverse impact?" 220 00:14:10,478 --> 00:14:12,998 Those kinds of things are questions that you ask. 221 00:14:13,378 --> 00:14:17,588 And let me throw out an example because it's a big thing to wrap your head around. 222 00:14:17,818 --> 00:14:19,878 There's a lot of things we're talking about right here. 223 00:14:20,768 --> 00:14:26,838 But the classic example is in a job that requires physical ability. 224 00:14:27,188 --> 00:14:35,288 Let's say, for instance, there's a person who has to load concrete bags onto a 225 00:14:35,288 --> 00:14:39,748 pallet so that it can get shrink wrapped and loaded into a truck and shipped away. 226 00:14:40,008 --> 00:14:43,368 So there's a conveyor belt that brings out these, 227 00:14:43,578 --> 00:14:44,358 these bags. 228 00:14:44,418 --> 00:14:49,428 And so what you might do in terms of a physical ability test to hire 229 00:14:49,428 --> 00:14:53,148 somebody into that position is you might have a conveyor belt. 230 00:14:53,168 --> 00:14:56,108 You might have this person grabbing the bags. 231 00:14:56,238 --> 00:14:57,998 You tell them how you want them stacked. 232 00:14:58,188 --> 00:15:01,338 And you see how many bags they can stack in 5 minutes. 233 00:15:01,338 --> 00:15:06,128 Or can they stack the number of bags that come out in a 5 minute period at 234 00:15:06,128 --> 00:15:07,578 the same rate that would come out later 235 00:15:07,748 --> 00:15:10,788 on the conveyor belt on the job itself. 236 00:15:11,218 --> 00:15:18,158 Now, let's say, for instance, that this particular test is using 60 pound bags. 237 00:15:18,658 --> 00:15:23,438 And so HR calls over to the plant and says, "Hey, we're getting ready 238 00:15:23,438 --> 00:15:25,358 to do this physical ability test. 239 00:15:25,598 --> 00:15:26,923 I want to make sure that. 240 00:15:27,193 --> 00:15:30,313 We're still using 60 pound bags in the plant." 241 00:15:30,783 --> 00:15:33,903 And the supervisor says, "Oh no, I'm so glad you called. 242 00:15:34,203 --> 00:15:37,633 Remember when the risk analysis people said we're going to have 243 00:15:37,683 --> 00:15:42,643 far fewer incidents of injury if we use 40 pound bags instead. 244 00:15:42,643 --> 00:15:44,983 So we switched over to 40 pound bags. 245 00:15:45,283 --> 00:15:49,853 It's we just dodged a bullet because we would have been testing at a level that is 246 00:15:49,893 --> 00:15:52,813 much higher than is required on the job." 247 00:15:53,163 --> 00:15:54,053 And that 248 00:15:54,353 --> 00:15:59,593 a physical ability test is going to almost always result 249 00:15:59,593 --> 00:16:01,453 in adverse impact against women. 250 00:16:01,963 --> 00:16:07,193 Generally speaking, men are stronger than women, and so there will be 251 00:16:07,193 --> 00:16:10,953 a disparity in between the passing rates for men and the passing 252 00:16:10,953 --> 00:16:15,713 rates for women for a physical job, especially one that's really demanding. 253 00:16:16,163 --> 00:16:20,603 And so, there's baked in adverse impact that comes with that kind of position, 254 00:16:21,163 --> 00:16:27,873 but by measuring using the wrong size bag of concrete, you're actually 255 00:16:27,873 --> 00:16:32,953 creating additional adverse impact that otherwise would not have occurred. 256 00:16:33,243 --> 00:16:37,553 And that is an instance that is impossible to defend. 257 00:16:37,793 --> 00:16:39,363 You cannot defend that. 258 00:16:39,473 --> 00:16:45,153 You can defend the adverse impact that occurs between natural differences in 259 00:16:45,153 --> 00:16:51,463 physical strength between men and women, but you cannot defend against the adverse 260 00:16:51,463 --> 00:16:57,053 impact that occurs because you ask them to do more than the job requires. 261 00:16:57,933 --> 00:16:58,413 Jenny Arnez: Okay. 262 00:16:58,503 --> 00:16:59,593 Yeah, that makes sense. 263 00:17:00,003 --> 00:17:04,803 So what you're saying then is that there are times, provided that your 264 00:17:04,803 --> 00:17:09,903 test is valid, that what you require, what your test is requiring of the 265 00:17:09,913 --> 00:17:14,163 candidate is actually a reflection of what they'll be required to do on the job. 266 00:17:14,713 --> 00:17:19,403 When that is in sync, there are times where an adverse impact will occur. 267 00:17:20,483 --> 00:17:21,463 Mike Callen: Yeah, absolutely. 268 00:17:21,513 --> 00:17:23,823 There are going to be times in which it occurs. 269 00:17:23,823 --> 00:17:25,583 It doesn't mean that you don't look for it. 270 00:17:25,583 --> 00:17:28,513 You still look for it because you want to evaluate the process. 271 00:17:28,723 --> 00:17:31,273 If you can measure the same thing in a different way, and it 272 00:17:31,293 --> 00:17:34,613 doesn't have adverse impact, then it's a better way to measure it. 273 00:17:34,623 --> 00:17:35,763 You go that direction. 274 00:17:36,413 --> 00:17:39,313 In our tests, we use something called work sample tests. 275 00:17:39,473 --> 00:17:43,423 So we're actually like the concrete bags. 276 00:17:43,483 --> 00:17:45,503 We're actually simulating the job. 277 00:17:45,723 --> 00:17:51,663 It's very hard to find a testing process that's more valid or 278 00:17:51,683 --> 00:17:56,003 equally valid to actually simulating the job than simulating the job. 279 00:17:56,403 --> 00:18:02,403 And as an employer, if I'm hiring people to load pallets, to shrink 280 00:18:02,403 --> 00:18:07,173 wrap that material and get it onto a truck because I'm selling concrete, I 281 00:18:07,173 --> 00:18:12,063 have the ability to make sure that the pallet loader can load pallets, right? 282 00:18:12,113 --> 00:18:13,963 That's just a fundamental thing. 283 00:18:14,153 --> 00:18:18,813 So whether or not there's adverse impact- it isn't important. 284 00:18:18,853 --> 00:18:23,203 What matters is that the selection process, the test in this case, 285 00:18:23,443 --> 00:18:25,993 is it valid and consistent with business necessity? 286 00:18:26,213 --> 00:18:30,703 If it is, then you are justified in having some adverse impact 287 00:18:30,743 --> 00:18:32,423 that enters into the process. 288 00:18:32,938 --> 00:18:35,058 Jenny Arnez: So you said business necessity. 289 00:18:35,058 --> 00:18:37,928 That sounds like that might be an actual term. 290 00:18:37,948 --> 00:18:42,928 Is there a set of key laws or guidelines that govern pre employment testing? 291 00:18:43,428 --> 00:18:45,198 Mike Callen: And guidelines is the key word. 292 00:18:45,208 --> 00:18:49,988 There's a huge document which can be found at uniformguidelines.com 293 00:18:50,358 --> 00:18:54,188 which is the Uniform Guidelines on Employee Selection Procedures. 294 00:18:54,518 --> 00:18:57,448 And it's a document that was put together, I believe, in the 70's. 295 00:18:57,468 --> 00:19:01,308 It's been revised a couple times and now has a whole host of questions and 296 00:19:01,308 --> 00:19:03,268 answers that comes with it as well. 297 00:19:03,508 --> 00:19:07,118 But it was put together by a really large committee 298 00:19:07,343 --> 00:19:11,893 of practitioners who operate in this space, and it covers things like 299 00:19:12,093 --> 00:19:16,213 job analysis, job descriptions, test validation, also sorts of different 300 00:19:16,213 --> 00:19:23,543 kinds, and it really sets forth the practices and principles that are the 301 00:19:23,543 --> 00:19:26,183 best practices for doing these things. 302 00:19:26,213 --> 00:19:30,103 Now, I don't know that it was ever codified into law. 303 00:19:30,428 --> 00:19:33,288 But the courts have relied on it again and again. 304 00:19:33,298 --> 00:19:38,988 And the DOJ, Department of Justice, the OFCCP, the EEOC, all these different 305 00:19:38,998 --> 00:19:44,108 governing bodies, including other state bodies, have relied upon it as the 306 00:19:44,278 --> 00:19:46,668 standards for doing things correctly. 307 00:19:46,908 --> 00:19:54,248 And so it comes up very often in legal trials and adjudications like 308 00:19:54,248 --> 00:19:58,068 by labor boards and those kinds of things that comes up very often. 309 00:19:58,278 --> 00:20:01,048 So it's a very important document. 310 00:20:01,128 --> 00:20:03,538 Another thing that's really important is the ADA, the 311 00:20:03,758 --> 00:20:05,248 Americans with Disabilities Act. 312 00:20:05,638 --> 00:20:08,388 And we won't really get into that here, because it gets into 313 00:20:08,398 --> 00:20:11,808 a whole other area of essential functions and those kinds of things. 314 00:20:12,188 --> 00:20:17,458 But, yeah, in terms of, dealing with this issue, the Uniform Guidelines is really 315 00:20:17,888 --> 00:20:23,628 probably the main body of literature by which everything should be measured 316 00:20:23,628 --> 00:20:25,288 in terms of doing things correctly. 317 00:20:25,288 --> 00:20:26,668 And that's what we do as well. 318 00:20:27,278 --> 00:20:33,198 In fact, in our software and the TestGenius software, CritiCall and OPAC, 319 00:20:33,218 --> 00:20:40,448 we use a validation wizard, which is based upon section 14c and 320 00:20:40,448 --> 00:20:42,588 15c of the Uniform Guidelines. 321 00:20:43,048 --> 00:20:47,788 Section 14c is the section that deals with content validation. 322 00:20:48,028 --> 00:20:52,248 It's the steps that you need to go through in order to properly content 323 00:20:52,308 --> 00:20:56,218 validate a test, and that's how you determine whether or not it's valid and 324 00:20:56,218 --> 00:20:58,538 consistent with the business necessity. 325 00:20:59,698 --> 00:21:00,468 As well, 326 00:21:00,688 --> 00:21:04,068 15c is the reporting section for the same. 327 00:21:04,068 --> 00:21:07,988 So it talks about what are the things that need to be outputted in a report 328 00:21:08,278 --> 00:21:11,748 that will properly document that this validation has taken place. 329 00:21:12,018 --> 00:21:15,788 And so in our validation wizard, we're running people through a process. 330 00:21:15,798 --> 00:21:18,558 In this case, we would run subject matter experts, 331 00:21:18,788 --> 00:21:22,923 who are performing well on the job for a year or longer or supervisors 332 00:21:22,923 --> 00:21:26,263 of those people are trainers of those people who are abundantly familiar 333 00:21:26,263 --> 00:21:29,973 with the position, we're going to run them through each of the tests that 334 00:21:29,993 --> 00:21:32,033 employer would purport to measure. 335 00:21:32,323 --> 00:21:36,103 And then, at the end of each test will be a survey that will ask them 336 00:21:36,113 --> 00:21:40,203 questions about the job relatedness of that test and that the very 337 00:21:40,223 --> 00:21:41,803 end of each test that would say, 338 00:21:42,473 --> 00:21:48,263 "Jenny Arnez, your score on this test was 96 but given your position and your 339 00:21:48,263 --> 00:21:52,633 knowledge of this job what, in your opinion, would be a minimum qualifying 340 00:21:52,633 --> 00:21:56,083 score for somebody who's entering into this position the first time?" 341 00:21:56,513 --> 00:22:01,893 So you would say, "Hey, I've been doing this for 20 years, and I'm an uber expert. 342 00:22:01,993 --> 00:22:07,993 So 96, it's really good for me but because I directly just felt the difficulty 343 00:22:07,993 --> 00:22:12,833 of that test and I'm knowledgeable about how well skilled enabled someone 344 00:22:12,833 --> 00:22:16,483 should be coming into this position, I would say that if somebody scored 345 00:22:16,523 --> 00:22:20,568 83 or better on this test, then I would consider them to be eligible 346 00:22:20,728 --> 00:22:23,698 to move on from the applicant to the candidate role." 347 00:22:24,178 --> 00:22:29,948 And so the wizard will take in all of the subject matter experts 348 00:22:29,998 --> 00:22:34,368 opinions, including yours and the suggestions for the cutoff scores. 349 00:22:34,628 --> 00:22:38,238 And it will create a validation report that says of this test 350 00:22:38,238 --> 00:22:42,933 battery, let's say it had six tests in there, for this test battery 351 00:22:43,213 --> 00:22:47,033 you should only use these five tests because one of them was determined 352 00:22:47,033 --> 00:22:49,083 to be not valid for this position. 353 00:22:49,093 --> 00:22:54,143 Maybe it was more difficult than the position like the 60 pound bag of concrete 354 00:22:54,153 --> 00:22:56,093 instead of the 40 pound bag of concrete. 355 00:22:56,593 --> 00:22:58,373 And so for that reason, it gets thrown out. 356 00:22:58,413 --> 00:23:02,968 As well, here's four different defensible job related cutoff 357 00:23:02,978 --> 00:23:04,438 scores that you could use. 358 00:23:04,788 --> 00:23:09,808 Your average of your suggested cutoff scores, and then that minus one, two, 359 00:23:09,868 --> 00:23:14,608 and three standard error of measurement, which is a statistical step downward 360 00:23:14,878 --> 00:23:20,848 that one might use to create different significant layers below the high cutoff. 361 00:23:21,138 --> 00:23:25,123 If you don't want to use, let's say 86 was the first one, minus 362 00:23:25,123 --> 00:23:27,493 one SEM might bring you down to 81. 363 00:23:27,793 --> 00:23:30,073 So let's say we didn't have enough people to interview. 364 00:23:30,073 --> 00:23:32,443 We wanna step down to the next appropriate layer. 365 00:23:32,623 --> 00:23:37,753 We might use the cutoff minus one SEM, which would be 81, and that gives us five 366 00:23:37,753 --> 00:23:43,413 people to interview or maybe minus, minus two SEM and it brings this down to 76. 367 00:23:43,413 --> 00:23:47,653 And that might be 10 people to move on to the next step, that, that sort of thing. 368 00:23:48,448 --> 00:23:51,318 Jenny Arnez: Okay, so what you're describing is the validation process, 369 00:23:51,318 --> 00:23:54,178 and in particular, the validation process if someone were to use 370 00:23:54,418 --> 00:23:57,288 TestGenius as their pre employment test. 371 00:23:57,788 --> 00:23:59,688 Mike Callen: Yeah, or if they did it all on their own. 372 00:23:59,708 --> 00:24:03,238 You can do this process manually, and this process is outlined 373 00:24:03,238 --> 00:24:04,548 in the Uniform Guidelines. 374 00:24:04,818 --> 00:24:09,928 It's just that what we've done is we've said, "We value this process so 375 00:24:09,928 --> 00:24:15,158 much that we want to make it as easy as possible for you, our clients. 376 00:24:15,458 --> 00:24:19,788 And so we're creating a wizard that literally just walks them through and 377 00:24:19,788 --> 00:24:23,868 that formats the report and then creates the report and sends it off to people. 378 00:24:25,118 --> 00:24:25,608 Jenny Arnez: Okay. 379 00:24:26,148 --> 00:24:29,218 So let's real briefly, just a high level 380 00:24:29,633 --> 00:24:35,123 outline of the validation processes using the test selection wizard, right? 381 00:24:35,213 --> 00:24:36,423 Mike Callen: we didn't talk about that. 382 00:24:36,423 --> 00:24:39,163 But yeah, you want to start off with the test selection wizard. 383 00:24:39,483 --> 00:24:43,673 Let's show the content validation slide because that's helpful in understanding 384 00:24:43,883 --> 00:24:45,993 the test selection wizard process. 385 00:24:46,173 --> 00:24:46,863 That's okay. 386 00:24:47,373 --> 00:24:52,333 Content validity is like building a bridge across the river. 387 00:24:52,703 --> 00:24:55,103 On one side, you have the test. 388 00:24:55,523 --> 00:24:57,043 So go ahead and go to the next slide 389 00:24:59,203 --> 00:25:01,493 and then on the other side, go ahead and do the next one. 390 00:25:01,823 --> 00:25:05,433 You have the job and it's represented by these duties that 391 00:25:05,463 --> 00:25:11,283 are performed and the bridge is supported in the middle by this peer. 392 00:25:11,503 --> 00:25:12,243 And that is the KSAs. 393 00:25:12,243 --> 00:25:16,893 KSA stands for knowledge, skills and abilities. 394 00:25:17,163 --> 00:25:21,638 So if you look at this process the test requires certain 395 00:25:21,648 --> 00:25:23,168 knowledge, skills and abilities. 396 00:25:23,348 --> 00:25:26,788 The job duties require certain knowledge, skills and abilities. 397 00:25:27,028 --> 00:25:30,858 And so what we're doing is we're connecting the test on one side of 398 00:25:30,858 --> 00:25:34,958 the river to the job on the other side of the river, and the linkage is 399 00:25:34,958 --> 00:25:36,608 the knowledge, skills and abilities. 400 00:25:36,928 --> 00:25:40,038 If we look at the test selection wizard, I don't think we have an image 401 00:25:40,048 --> 00:25:44,428 for it, but basically what we're doing on the test selection wizard is we're 402 00:25:44,428 --> 00:25:50,313 listing duties that might be performed on the job, and so a person is going 403 00:25:50,313 --> 00:25:54,233 to go through, and they're going to check off duties that are performed by 404 00:25:54,493 --> 00:25:57,033 the position that's being recruited. 405 00:25:57,423 --> 00:26:00,923 And as they go through, and they check off those duties that have 406 00:26:00,923 --> 00:26:04,583 linkages to KSA's- knowledge, skills and abilities- which are linked to 407 00:26:04,583 --> 00:26:07,233 specific tests within our system. 408 00:26:07,603 --> 00:26:12,693 And so, you check off 10 duties, and that might create a list of eight 409 00:26:12,693 --> 00:26:18,873 or nine tests that we may suggest that they use for selection purposes 410 00:26:18,873 --> 00:26:20,533 for this particular position. 411 00:26:21,053 --> 00:26:23,553 And so that's the test selection wizard. 412 00:26:24,213 --> 00:26:27,903 The next step that one would go to in order to determine whether or 413 00:26:27,903 --> 00:26:32,803 not the test battery in this case, a battery of individual tests, 414 00:26:33,103 --> 00:26:38,033 is valid and consistent with business necessity is to go through the next 415 00:26:38,033 --> 00:26:39,903 part, which is the validation wizard. 416 00:26:40,373 --> 00:26:41,503 And in that case, 417 00:26:42,003 --> 00:26:46,653 we recommend 7 to 10 subject matter experts, and again, those will be the 418 00:26:46,713 --> 00:26:48,543 incumbents that I described earlier. 419 00:26:48,733 --> 00:26:52,043 People who have been successfully performing on the job one year 420 00:26:52,043 --> 00:26:56,303 or longer or supervisors at that position or trainers for that 421 00:26:56,303 --> 00:26:59,573 position, but people that are abundantly familiar with that job. 422 00:26:59,763 --> 00:27:04,023 And you want to go through and you want to cherry pick really high performers, 423 00:27:04,063 --> 00:27:08,413 but you also want to get a demographic mix that matches that 424 00:27:08,413 --> 00:27:12,203 of your existing workforce and particularly of your recruitment area. 425 00:27:12,623 --> 00:27:17,333 But the idea here is that these people are going to go through those tests. 426 00:27:17,483 --> 00:27:21,083 They're going to take those tests with that wizard turned on, and the wizard 427 00:27:21,083 --> 00:27:24,983 is going to ask them survey questions, cut off score suggestions, and it's 428 00:27:24,983 --> 00:27:26,903 going to result in a validation report 429 00:27:27,163 --> 00:27:31,103 that might be 30 or more pages long that's going to have a table at the end. 430 00:27:31,103 --> 00:27:35,723 It says, here's the test you validated and each of these marked yes, are 431 00:27:35,723 --> 00:27:40,323 ones that you should use on your particular recruitment or could use 432 00:27:40,483 --> 00:27:45,133 in your recruitment as well here's the defensible job related cutoff scores. 433 00:27:45,213 --> 00:27:49,323 So that's those two wizards, how they work, step one, step two. 434 00:27:49,513 --> 00:27:51,833 And then step three is your recruitment itself. 435 00:27:52,263 --> 00:27:58,728 So you take the validated test and you actually employ it for use for 436 00:27:58,728 --> 00:28:01,348 selection with your job applicants. 437 00:28:02,348 --> 00:28:02,698 Jenny Arnez: Okay. 438 00:28:02,698 --> 00:28:04,658 So let's say you have your valid test. 439 00:28:04,668 --> 00:28:06,418 You have candidates taking it. 440 00:28:07,108 --> 00:28:11,618 So do you just assume that the person who scores the highest on the test, 441 00:28:11,618 --> 00:28:13,038 that's your number one applicant? 442 00:28:13,338 --> 00:28:15,718 What's your recommendation there on how to use the scores? 443 00:28:16,148 --> 00:28:17,738 Mike Callen: That's a really great question. 444 00:28:17,838 --> 00:28:24,653 We didn't talk about this, but one of the one of the survey questions that's 445 00:28:24,703 --> 00:28:28,373 asked at the end of each test when you're running your subject matter 446 00:28:28,393 --> 00:28:34,643 experts or your incumbent employees through there is whether or not the 447 00:28:34,873 --> 00:28:37,543 test is performance differentiating. 448 00:28:37,653 --> 00:28:39,043 I'm going to give you an example. 449 00:28:39,053 --> 00:28:43,313 Let's say that you're hiring somebody who has to type. 450 00:28:43,843 --> 00:28:45,063 Dispatcher is a good one. 451 00:28:45,083 --> 00:28:49,333 They're doing short bursts of texts, but they need to be able to type 452 00:28:49,373 --> 00:28:53,303 relatively quickly and they need to be able to type accurately. 453 00:28:53,708 --> 00:28:59,648 So typing with speed and accuracy is an important attribute for that job. 454 00:29:00,248 --> 00:29:06,128 The fastest speed that they may need to type might be 35 words a minute, right? 455 00:29:06,718 --> 00:29:10,878 So the question becomes, does that mean that somebody who types 70 456 00:29:10,908 --> 00:29:15,958 words per minute is twice as good as somebody who types 35 words per minute? 457 00:29:16,328 --> 00:29:19,393 And in this case, the answer is no. 458 00:29:19,893 --> 00:29:24,543 You really only need to do what you need to do and going faster than you 459 00:29:24,543 --> 00:29:28,023 need to go isn't necessarily a benefit. 460 00:29:28,083 --> 00:29:32,463 And so that would be a good example of a test, the results of which are 461 00:29:32,463 --> 00:29:34,143 not performance differentiating. 462 00:29:34,543 --> 00:29:39,013 And so what we would do is we would look at everybody from 35 463 00:29:39,043 --> 00:29:45,393 words per minute and above as being equally suitable for the job. 464 00:29:45,393 --> 00:29:48,083 So there wouldn't be any rank ordering at all. 465 00:29:48,563 --> 00:29:50,173 And that is really 466 00:29:50,943 --> 00:29:55,113 just one of the main reasons why setting the cutoffs is so important. 467 00:29:55,473 --> 00:29:59,713 If you have a defensible job related cutoff score then what it 468 00:29:59,723 --> 00:30:04,143 does is, let's say you just chose the person who could type 70 words 469 00:30:04,173 --> 00:30:05,533 per minute and you hired them. 470 00:30:05,923 --> 00:30:10,763 You've ignored all these other attributes of a human being that may 471 00:30:10,763 --> 00:30:12,543 or may not make them a good employee. 472 00:30:12,848 --> 00:30:16,858 Sure, they can type, but that is not the entirety of any job. 473 00:30:17,138 --> 00:30:23,318 And so by choosing too high, you may have ignored applicants who were equally 474 00:30:23,428 --> 00:30:28,428 qualified in terms of keyboarding, but had so many other things that 475 00:30:28,428 --> 00:30:31,798 they could apply to this work area. 476 00:30:31,798 --> 00:30:35,758 And so that's a really important aspect to keep in mind. 477 00:30:36,208 --> 00:30:40,328 Now there are some that people will go through and say, "Yes, this is 478 00:30:40,548 --> 00:30:45,458 performance differentiating because, people who do scope score higher on 479 00:30:45,458 --> 00:30:50,478 whatever knowledge, skill or ability it is, they may be the best and everybody 480 00:30:50,478 --> 00:30:52,858 who's less than them is less good." 481 00:30:53,678 --> 00:30:57,028 But again, that's just one of the many 482 00:30:57,443 --> 00:30:59,773 reasons why you make a selection decision. 483 00:30:59,773 --> 00:31:02,503 And so again, cutoff scores are really important. 484 00:31:04,313 --> 00:31:08,103 Jenny Arnez: And the validation process helps you to set those cutoff scores. 485 00:31:08,793 --> 00:31:10,223 Mike Callen: It helps you to set those cutoff scores. 486 00:31:10,223 --> 00:31:13,793 And it also will report back to you whether or not something 487 00:31:13,793 --> 00:31:15,333 is performance differentiating. 488 00:31:15,643 --> 00:31:19,133 Which lets you know that once you get to that cutoff score, everybody from 489 00:31:19,133 --> 00:31:23,353 there and above should be considered equally skilled or able for that 490 00:31:23,433 --> 00:31:25,233 particular performance dimension. 491 00:31:25,713 --> 00:31:28,823 And we'll be right back after a word from our sponsor. 492 00:31:30,623 --> 00:31:32,763 Ready to revolutionize your HR strategies? 493 00:31:33,153 --> 00:31:34,443 Head over to TestGenius. 494 00:31:34,473 --> 00:31:37,223 com to discover our latest tools and solutions designed to 495 00:31:37,223 --> 00:31:38,713 streamline your hiring processes. 496 00:31:39,933 --> 00:31:45,503 So I'm going to flip back real quickly add our scenario back on here. 497 00:31:45,513 --> 00:31:50,203 Just so we can summarize where we've what we've touched on Acme County. 498 00:31:50,523 --> 00:31:52,363 They're concerned about adverse impact. 499 00:31:52,433 --> 00:31:56,063 So the first step really is they need to remember that the testing 500 00:31:56,063 --> 00:32:01,163 process or testing happens throughout the whole selection process, 501 00:32:01,813 --> 00:32:04,103 throughout their whole selection plan. 502 00:32:04,243 --> 00:32:07,713 And so they have to pull back the lens, not just focus on the pre 503 00:32:07,713 --> 00:32:12,933 employment skills test, but also to look at their job posting, their 504 00:32:12,943 --> 00:32:14,703 interview practices, and so on. 505 00:32:15,283 --> 00:32:17,013 All the other tests as well. 506 00:32:17,063 --> 00:32:18,173 Jenny Arnez: All the other tests. 507 00:32:19,088 --> 00:32:24,348 And then focusing in on what our specialty is here at test, with TestGenius 508 00:32:24,358 --> 00:32:26,898 is the pre employment skills test. 509 00:32:27,988 --> 00:32:32,808 And then, so then they need to validate that using their subject matter experts, 510 00:32:32,818 --> 00:32:37,568 using the tools that, that they have within TestGenius, assuming that they're 511 00:32:37,568 --> 00:32:42,638 a TestGenius client, using the test selection wizard, the validation wizard, 512 00:32:43,278 --> 00:32:46,268 which then helps them set accurate 513 00:32:49,083 --> 00:32:51,393 - is that the term, accurate cutoff scores? 514 00:32:51,783 --> 00:32:55,013 Mike Callen: Defensible and job related is the important aspect of it 515 00:32:55,043 --> 00:32:56,983 and let me in interrupt one second. 516 00:32:57,423 --> 00:33:00,298 This is our recommended best practice. 517 00:33:00,398 --> 00:33:05,528 Not every client goes through the validation process for every 518 00:33:05,828 --> 00:33:08,058 administration of TestGenius. 519 00:33:08,078 --> 00:33:08,898 We know that. 520 00:33:09,398 --> 00:33:15,848 It's our strong recommendation that they do it because of two things. 521 00:33:15,888 --> 00:33:18,338 We think about the recruitment process. 522 00:33:18,338 --> 00:33:21,728 Think of about it as a coin with two sides. 523 00:33:22,008 --> 00:33:26,273 Okay, on one side, you have defensibility. 524 00:33:26,323 --> 00:33:32,353 So in the instance where somebody questions the hiring practice, is 525 00:33:32,353 --> 00:33:34,063 that hiring practice defensible? 526 00:33:34,223 --> 00:33:37,933 Or if you go to your applicant tracking system and you find that your physical 527 00:33:37,943 --> 00:33:40,623 ability test had adverse impact. 528 00:33:40,713 --> 00:33:43,343 Is that defensible in that particular case? 529 00:33:43,433 --> 00:33:47,973 So being able to answer some sort of challenge to the recruitment 530 00:33:47,973 --> 00:33:49,303 process is really important. 531 00:33:49,643 --> 00:33:53,063 But when you flip that coin over and you look at the other side 532 00:33:53,073 --> 00:33:57,243 from the utility side, or sorry, the defensibility side, utility. 533 00:33:57,573 --> 00:34:04,913 And utility talks about how does this test select people who are going to go on 534 00:34:04,933 --> 00:34:07,733 to be really good employees on the job? 535 00:34:08,103 --> 00:34:11,363 And the beautiful thing about this, and this is the beauty that comes 536 00:34:11,583 --> 00:34:16,773 with the brilliance of the Uniform Guidelines, is because people who were 537 00:34:16,783 --> 00:34:23,718 really experienced in the practices that can both have and result in a 538 00:34:23,718 --> 00:34:29,298 safe recruitment and hire people who do really well on the job are contributing 539 00:34:29,518 --> 00:34:32,118 to this framework that we follow here. 540 00:34:32,118 --> 00:34:36,798 And so that's why it's really important to go through and do the required work 541 00:34:37,098 --> 00:34:42,538 or to do the recommended work in order to be able to end up with a testing 542 00:34:42,538 --> 00:34:46,858 process that's valid and consistent with business necessity and cut off scores 543 00:34:46,868 --> 00:34:48,618 that are defensible and job related. 544 00:34:48,708 --> 00:34:50,708 So that's as far as we can say about that. 545 00:34:50,708 --> 00:34:56,208 If somebody decides that they're not going to go through the validation, that's 546 00:34:56,208 --> 00:35:02,928 their choice, but they've taken on that burden themselves in case the hiring 547 00:35:02,928 --> 00:35:05,388 practice is challenged, for instance. 548 00:35:05,758 --> 00:35:10,498 If it's challenged, then you know they don't have a leg to stand on in 549 00:35:10,498 --> 00:35:16,228 terms of the defensibility, if they go retroactively and do the validation 550 00:35:16,228 --> 00:35:20,938 process and the validation report shows that the test isn't valid and consistent 551 00:35:20,938 --> 00:35:25,798 with business necessity or that they use cut off scores that were too high. 552 00:35:25,998 --> 00:35:30,008 So people are taking that risk upon themselves by skipping 553 00:35:30,008 --> 00:35:31,338 that part of the process. 554 00:35:33,288 --> 00:35:36,288 Jenny Arnez: And how often does the validation process need to happen? 555 00:35:36,288 --> 00:35:37,618 Do we do this every year? 556 00:35:37,678 --> 00:35:38,308 What is that? 557 00:35:38,428 --> 00:35:39,448 How do you determine that? 558 00:35:39,948 --> 00:35:45,288 Mike Callen: So the validation process is good to do, generally 559 00:35:45,288 --> 00:35:47,628 speaking, at about a three year clip. 560 00:35:48,148 --> 00:35:51,678 And that would be the same pace that you would want to try 561 00:35:51,678 --> 00:35:53,708 to review your job analysis. 562 00:35:53,708 --> 00:35:58,748 Now, the difference, or sorry, the the caveat for both of those scenarios would 563 00:35:58,758 --> 00:36:01,208 be unless the job fundamentally changes. 564 00:36:01,578 --> 00:36:05,838 There's been a just a ton of technology that's come into the 565 00:36:05,838 --> 00:36:09,578 dispatch world, for instance, which is to follow our scenario here. 566 00:36:10,218 --> 00:36:12,828 People can text videos and images. 567 00:36:12,998 --> 00:36:16,953 Let's say somebody gives you a license plate number and you type that into 568 00:36:16,953 --> 00:36:21,823 the computer aided dispatch system and they also send you a photograph 569 00:36:21,843 --> 00:36:26,533 of a scene and the photograph has the license plate on there and you can 570 00:36:26,533 --> 00:36:31,963 clearly see that the person reported the wrong license plate number. 571 00:36:32,093 --> 00:36:35,373 So you have to be able to discern the difference between the two. 572 00:36:35,593 --> 00:36:40,543 You have to be able to decide which of those has the right data and then you have 573 00:36:40,543 --> 00:36:42,383 to be able to update it into the system. 574 00:36:42,603 --> 00:36:46,823 Because if you're talking about putting out a BOLO looking for somebody 575 00:36:46,823 --> 00:36:49,893 with this license plate and you use the wrong license plate number, 576 00:36:50,073 --> 00:36:51,423 you know that's really CritiCall. 577 00:36:51,443 --> 00:36:55,563 So that would be a good example of a fundamental change that happened at a 578 00:36:55,563 --> 00:36:59,843 dispatch center that might require you to go back and look at the versions 579 00:36:59,843 --> 00:37:01,293 of these tests that you're using. 580 00:37:01,593 --> 00:37:05,663 Maybe before you were using text only versions, now you want to use versions 581 00:37:05,673 --> 00:37:11,003 that have photographs, images and questions about those images in there. 582 00:37:11,003 --> 00:37:14,123 Jenny Arnez: I think we've done a good job or you've done a great job 583 00:37:14,123 --> 00:37:18,783 on helping Acme County here take a look at their hiring practices and 584 00:37:18,783 --> 00:37:21,123 some action steps that they can take. 585 00:37:22,303 --> 00:37:25,523 And anything else that you want to add to today's conversation? 586 00:37:26,023 --> 00:37:29,943 Mike Callen: We talked about content validity and content validity is 587 00:37:29,943 --> 00:37:33,123 when you're building that bridge from the test to the KSAs, knowledge, 588 00:37:33,123 --> 00:37:35,243 skills, and abilities over to 589 00:37:35,763 --> 00:37:41,223 the job on the other side, and there's another kind of validation, which we 590 00:37:41,223 --> 00:37:44,003 don't do with our wizard, but can be done. 591 00:37:44,003 --> 00:37:45,233 And it's very much valuable. 592 00:37:45,233 --> 00:37:47,293 And that's called criterion validity. 593 00:37:47,793 --> 00:37:53,523 And criterion validity is when you have some sort of aspect of job performance 594 00:37:53,873 --> 00:37:58,633 on the axis over on the left hand side, and then you plot that against test 595 00:37:58,633 --> 00:38:00,873 scores along the bottom access there. 596 00:38:01,183 --> 00:38:05,783 So each of these data points would be some sort of measured 597 00:38:05,783 --> 00:38:07,293 metric of job performance. 598 00:38:07,293 --> 00:38:14,773 Maybe it's supervisor ratings or could be feedback from surveys or, like 599 00:38:14,773 --> 00:38:19,233 tardiness or, all sorts of different things, but each of those data points 600 00:38:19,233 --> 00:38:24,423 would be the intersection between how that particular aspect of job performance 601 00:38:24,753 --> 00:38:28,213 was and what the test score was. 602 00:38:28,223 --> 00:38:31,393 So down in the bottom left, you can see there's two points down there. 603 00:38:31,513 --> 00:38:36,023 So that's people who had very rare, relatively low job performance, and they 604 00:38:36,023 --> 00:38:38,773 also scored relatively low on the test. 605 00:38:39,113 --> 00:38:42,043 And then over at the top you have one blue dot. 606 00:38:42,083 --> 00:38:47,123 That's the hot, the highest scorer who also does very well on the job. 607 00:38:47,163 --> 00:38:52,573 And that black line is a trend line that shows, how did those work together. 608 00:38:52,603 --> 00:38:55,953 So in this case, it's what you would hope for from a test. 609 00:38:55,963 --> 00:38:58,073 The lower scorers have lower job performance. 610 00:38:58,363 --> 00:39:00,563 The higher scorers have higher job performance. 611 00:39:00,783 --> 00:39:06,083 So this kind of study would be taken with existing incumbent employees. 612 00:39:06,443 --> 00:39:08,153 And that would be one way to do it. 613 00:39:08,163 --> 00:39:13,098 Or you could test it against job applicants and then wait A 614 00:39:13,098 --> 00:39:16,268 year and measure how well they were doing on the job and then 615 00:39:16,268 --> 00:39:17,858 go back and correlate it later. 616 00:39:18,178 --> 00:39:20,708 That's a hard thing to do because people don't like to do all that 617 00:39:20,708 --> 00:39:22,468 work and wait a year for the results. 618 00:39:22,638 --> 00:39:26,518 So it's usually done called a concurrent study that's done with incumbent 619 00:39:26,518 --> 00:39:31,688 employees, in which case, you can get the results at that particular point in time. 620 00:39:31,978 --> 00:39:36,418 But what you end up knowing is that people who score higher on the job are going to 621 00:39:36,418 --> 00:39:39,648 have a greater degree of job performance. 622 00:39:39,648 --> 00:39:43,628 And so you can be very confident finding what your cutoff score is 623 00:39:43,848 --> 00:39:47,528 and then interviewing those that are north of the cutoff score. 624 00:39:47,798 --> 00:39:51,508 And you can feel very confident that those people are going to be good performers 625 00:39:51,508 --> 00:39:56,458 on the job, at least in terms of this particular aspect of job performance. 626 00:39:56,808 --> 00:40:00,998 Now with the content validated skill and ability testing, you 627 00:40:00,998 --> 00:40:03,048 don't necessarily find that. 628 00:40:03,438 --> 00:40:07,738 Again, to go back to the person who types 35 words a minute versus the 629 00:40:07,738 --> 00:40:12,418 person who types 70 words a minute, that 70 scorer isn't not going to 630 00:40:12,428 --> 00:40:16,768 be twice the performer that the 35 per minute scorer is going to be. 631 00:40:17,018 --> 00:40:22,078 In many cases, what we're looking for is, do you reach or exceed a threshold? 632 00:40:22,248 --> 00:40:23,928 And that's really all that we want to know. 633 00:40:23,928 --> 00:40:27,313 And that's where you're, content validation approach comes into play. 634 00:40:27,623 --> 00:40:31,593 But in many instances, you can go on and do a criterion validation study 635 00:40:31,873 --> 00:40:33,643 that can be very helpful as well. 636 00:40:34,063 --> 00:40:37,073 Personality tests are always criterion validated. 637 00:40:37,083 --> 00:40:41,303 So basically you're looking at what are the personal aspects of a human 638 00:40:41,303 --> 00:40:46,173 being that would contribute to high performers on the job and what are the 639 00:40:46,203 --> 00:40:50,743 personal aspects of, human beings that contribute to lower performers on the job. 640 00:40:51,023 --> 00:40:54,163 And so when you're testing for personality, you're going to look 641 00:40:54,163 --> 00:40:58,943 for those aspects of personality that match with the higher performers. 642 00:40:58,943 --> 00:41:01,793 And so criterion validation is the only way to go. 643 00:41:03,133 --> 00:41:08,073 Jenny Arnez: So our tests in TestGenius for HR and then also for 644 00:41:08,223 --> 00:41:14,273 our dispatcher test, CritiCall we have the content validation tools but 645 00:41:14,273 --> 00:41:17,273 they are also criterion validated. 646 00:41:17,753 --> 00:41:18,013 Yes. 647 00:41:18,738 --> 00:41:18,998 Yeah. 648 00:41:18,998 --> 00:41:20,318 Can you speak to that a little bit? 649 00:41:20,318 --> 00:41:20,898 Explain. 650 00:41:21,088 --> 00:41:24,618 Mike Callen: Yeah it's always nice to be able to do a criterion validation study. 651 00:41:24,618 --> 00:41:28,138 And one of the beautiful things on the dispatch side is that dispatchers, 652 00:41:28,138 --> 00:41:31,238 call takers, telecommunicators, their jobs are very similar. 653 00:41:31,258 --> 00:41:36,018 There's a universality that exists between little tiny communities 654 00:41:36,018 --> 00:41:38,228 and large federal organizations. 655 00:41:38,358 --> 00:41:44,408 We we re able to do a criterion validation study with several different agencies 656 00:41:44,408 --> 00:41:48,668 who are contributing data to it and then come up with a chart like this one right 657 00:41:48,668 --> 00:41:51,258 here for a variety of different tests. 658 00:41:51,508 --> 00:41:53,698 And then back to the Uniform Guidelines 659 00:41:53,708 --> 00:41:58,628 once again, the Uniform Guidelines in Section 7b talks about how you 660 00:41:58,628 --> 00:42:01,788 can transport criterion validity. 661 00:42:02,118 --> 00:42:07,168 And so what that means is that for criterion validation, they will allow 662 00:42:07,168 --> 00:42:12,928 you to transport a validation study from one organization to your organization 663 00:42:13,218 --> 00:42:18,428 as long as you can establish that the the job performance dimensions 664 00:42:18,428 --> 00:42:24,158 or domains that were measured in that original study are very similar to those 665 00:42:24,188 --> 00:42:26,498 that exist within your organization. 666 00:42:26,833 --> 00:42:29,193 And so it's a very simple survey. 667 00:42:29,193 --> 00:42:31,693 You get in the room with 12 people, you go through those job 668 00:42:31,693 --> 00:42:35,343 performance areas, and then they rate it on a scale of one to five. 669 00:42:35,353 --> 00:42:37,293 One means it doesn't match my environment. 670 00:42:37,483 --> 00:42:40,313 Five means it's exactly like my environment, and then 671 00:42:40,533 --> 00:42:41,823 all the numbers in between. 672 00:42:42,063 --> 00:42:43,093 And so if you get 673 00:42:43,143 --> 00:42:49,353 12 people to agree that, it's 52 and a half or better, roughly three, a rating 674 00:42:49,353 --> 00:42:53,983 of three or better than you're going to be able to transport that validation 675 00:42:53,983 --> 00:42:56,493 study over to your own environment. 676 00:42:56,803 --> 00:43:01,173 And then you can just attach your transport ability sheets to that 677 00:43:01,173 --> 00:43:04,533 validation study, and you can literally claim it as your own. 678 00:43:04,543 --> 00:43:06,213 It's a golden ticket kind of thing. 679 00:43:06,663 --> 00:43:10,133 And I'm really glad that you asked about this because this hits on a 680 00:43:10,153 --> 00:43:11,993 point that we didn't talk about it all. 681 00:43:13,063 --> 00:43:17,353 One of the main reasons that why we have a validation wizard for content validity 682 00:43:17,613 --> 00:43:23,608 is that you cannot transport content validity from one organization to another. 683 00:43:23,608 --> 00:43:24,408 You can't do it. 684 00:43:25,098 --> 00:43:32,258 Employers are responsible for establishing the validity of specific tests within 685 00:43:32,448 --> 00:43:35,078 specific job titles at their organization. 686 00:43:35,618 --> 00:43:37,138 And that's the difference. 687 00:43:37,138 --> 00:43:42,438 That's why we can't do a universal criterion validation study, because 688 00:43:42,448 --> 00:43:45,578 even if you go with administrative assistant, what does that mean? 689 00:43:45,828 --> 00:43:49,008 If you go to a hundred different organizations likely going 690 00:43:49,008 --> 00:43:52,508 to be 100 different jobs with 100 different requirements. 691 00:43:52,858 --> 00:43:58,678 And so you can't really do that for that particular role. 692 00:43:59,428 --> 00:44:00,758 But that's okay. 693 00:44:00,958 --> 00:44:04,458 Because what we've done is we've given you the tools that allow you to 694 00:44:04,458 --> 00:44:06,688 do it in a matter of a couple hours. 695 00:44:06,908 --> 00:44:11,858 And now you have a validation report that says, for Acme County for the position of 696 00:44:11,858 --> 00:44:15,893 Dispatcher I, we've given you to here's our validation report, and it goes page 697 00:44:15,893 --> 00:44:20,953 after page, and it gives them, really a blessing to go through and use that test. 698 00:44:21,413 --> 00:44:24,743 You go to another county, and they want to go ahead and start testing, 699 00:44:24,743 --> 00:44:27,853 they might ask, hey, ACME, send me over the list of tests you're using. 700 00:44:28,253 --> 00:44:31,513 ACME might share those tests with them, but it doesn't mean that they're 701 00:44:31,523 --> 00:44:36,093 valid and consistent with business necessity, nor that the cutoff scores 702 00:44:36,433 --> 00:44:38,803 are job related and defensible. 703 00:44:39,093 --> 00:44:44,043 So that's why you go through the process because, go back to the coin, it's 704 00:44:44,113 --> 00:44:48,083 defensible on one side and has a high degree of utility on the other side. 705 00:44:49,533 --> 00:44:50,023 Jenny Arnez: Excellent. 706 00:44:50,033 --> 00:44:51,193 That was very helpful. 707 00:44:51,783 --> 00:44:54,723 Actually I have the transportability sheet. 708 00:44:54,733 --> 00:44:55,693 Do we want to show that? 709 00:44:56,283 --> 00:44:56,663 Sure. 710 00:44:57,733 --> 00:44:58,483 Mike Callen: Absolutely. 711 00:44:58,918 --> 00:45:00,418 It's a pretty Excel spreadsheet. 712 00:45:00,418 --> 00:45:03,468 It's always fun to show an Excel file with colors in it. 713 00:45:03,838 --> 00:45:07,888 So yeah, basically, we have this for the CritiCall test. 714 00:45:08,248 --> 00:45:11,778 As well we have it for the CritiCall personality test. 715 00:45:11,778 --> 00:45:15,638 And so on the left, you have the work behaviors and orders are too 716 00:45:15,638 --> 00:45:17,798 small for me to be able to see. 717 00:45:18,348 --> 00:45:21,258 But basically, it's going through the different dimensions. 718 00:45:21,533 --> 00:45:25,633 That were measured in the original validation study, and at the top, you 719 00:45:25,633 --> 00:45:30,363 have SME1, SME2, SME as subject matter expert through SME12, and then on the 720 00:45:30,363 --> 00:45:34,723 far right is the rating scale, where 1 is that it doesn't match, and 5 is 721 00:45:34,723 --> 00:45:40,203 that it does, and so basically, with the help of a leader, you go through 722 00:45:40,203 --> 00:45:44,843 and you just number the people and you say, number one, five, number two, five, 723 00:45:44,843 --> 00:45:47,163 number three, four, number four, five. 724 00:45:47,403 --> 00:45:48,983 And you just put the numbers in there. 725 00:45:49,213 --> 00:45:52,433 And then when you get done, it does a calculation down at the bottom. 726 00:45:52,433 --> 00:45:56,403 And I don't know if you have that here, if this is just an image, but it will 727 00:45:56,413 --> 00:46:01,183 just validate that all 12 have filled it in and that the numbers are high enough. 728 00:46:01,183 --> 00:46:02,223 Yeah, there it is down there. 729 00:46:02,443 --> 00:46:06,283 So it's all red now down in the bottom where the orange box is. 730 00:46:06,563 --> 00:46:10,743 When you fill this out, it'll turn to green, and then if it's valid, then 731 00:46:10,743 --> 00:46:15,393 that little red box at the very bottom will turn to green, and that's it. 732 00:46:15,393 --> 00:46:19,563 You print that out, attach it to the validation report again, and you can now 733 00:46:19,583 --> 00:46:22,123 claim that validation report as your own. 734 00:46:22,838 --> 00:46:26,978 On the content validation side, you get your own validation report. 735 00:46:27,258 --> 00:46:32,118 So every validation that you do using the wizard is a custom validation 736 00:46:32,228 --> 00:46:37,078 for content validity for a specific position at a specific location 737 00:46:37,328 --> 00:46:38,818 for use by those people alone. 738 00:46:39,578 --> 00:46:40,708 Jenny Arnez: Oh, that's really cool. 739 00:46:41,158 --> 00:46:45,888 And we will for our clients, if you're listening to this and would like to have a 740 00:46:45,888 --> 00:46:50,758 copy of that, just let reach out to us and we'll make sure that we send that to you. 741 00:46:50,758 --> 00:46:51,753 Mike Callen: Absolutely. 742 00:46:52,623 --> 00:46:55,123 Jenny Arnez: Yeah, so Mike, we've talked about content validity. 743 00:46:55,123 --> 00:46:56,793 We talked about criterion validity. 744 00:46:56,793 --> 00:47:00,293 I've also heard you mentioned face validity, right? 745 00:47:00,403 --> 00:47:01,503 Mike Callen: Face validity. 746 00:47:02,083 --> 00:47:06,053 I'd like to use a physics example here which some people hate physics, 747 00:47:06,053 --> 00:47:10,393 but you have that something called centripetal centrifugal force. 748 00:47:10,693 --> 00:47:14,333 You're in the car and the car turns left and the and you slam over into 749 00:47:14,333 --> 00:47:18,023 the right hand side of the car, and that's called centrifugal force. 750 00:47:18,253 --> 00:47:20,543 And it's really what's known as a pseudo force. 751 00:47:20,753 --> 00:47:21,913 It's not a real force. 752 00:47:21,913 --> 00:47:25,983 It's just that the car turned from underneath you and your 753 00:47:25,983 --> 00:47:27,393 momentum kept going forward. 754 00:47:27,393 --> 00:47:31,163 And so that right hand door ran into you, but it feels like a force. 755 00:47:31,503 --> 00:47:34,063 And face validity is similar to that. 756 00:47:34,433 --> 00:47:36,683 Face validity talks about it. 757 00:47:36,873 --> 00:47:41,373 How does the test feel to the applicant? 758 00:47:41,543 --> 00:47:43,953 Does it feel like a good experience? 759 00:47:44,253 --> 00:47:47,883 And that's another reason why work sample tests are very important. 760 00:47:48,183 --> 00:47:52,963 If you use work sample tests, you're presenting information to people, asking 761 00:47:52,963 --> 00:47:57,043 them to do things that are similar to things that would be done on the job. 762 00:47:57,363 --> 00:48:02,253 But we're doing it in a way where we don't require in prior job knowledge. 763 00:48:02,303 --> 00:48:06,553 So we're asking them generic things, not things that they would need to know 764 00:48:06,553 --> 00:48:08,593 about this particular organization. 765 00:48:08,863 --> 00:48:12,653 Like, how do you use Excel in Acme County, for instance? 766 00:48:12,863 --> 00:48:16,853 That's something that would be trained and we call it WT and LBO- would train 767 00:48:17,203 --> 00:48:19,043 or learned in a brief orientation. 768 00:48:19,273 --> 00:48:21,443 And so you don't want to test on those aspects. 769 00:48:21,493 --> 00:48:26,743 But we test things in a generic format that we can expect anybody with experience 770 00:48:26,743 --> 00:48:31,283 with that particular Excel, for instance, in this case, to be able to know. 771 00:48:31,343 --> 00:48:34,133 And then we translate that to how well they did. 772 00:48:34,133 --> 00:48:37,413 And then then it's validated for the particular job. 773 00:48:37,463 --> 00:48:39,913 So that's how we do that for that part. 774 00:48:40,243 --> 00:48:43,103 And actually we talked about face validity. 775 00:48:43,153 --> 00:48:48,063 So if a test feels like a job and I fail the test, I get a feeling that I probably 776 00:48:48,063 --> 00:48:51,983 wouldn't have done a good job on that job, and I don't feel really bad about that. 777 00:48:52,403 --> 00:48:58,683 Our personality test for dispatchers was created with dispatch types of items, and 778 00:48:58,683 --> 00:49:03,643 so it feels very much like it's asking you questions that if you were a dispatcher. 779 00:49:03,963 --> 00:49:11,473 A lot of standardized personality tests have questions like, do you enjoy long, 780 00:49:11,583 --> 00:49:14,393 quiet strolls on a beach by yourself? 781 00:49:14,953 --> 00:49:17,473 Do you enjoy being in a loud movie theater? 782 00:49:17,753 --> 00:49:21,193 And so it, those things are not very face valid. 783 00:49:21,203 --> 00:49:25,473 They don't make you feel like you just asked me something that 784 00:49:25,483 --> 00:49:27,853 has any applicability to the job. 785 00:49:28,303 --> 00:49:33,093 And so face validity really is not important in terms of a test being valid, 786 00:49:33,423 --> 00:49:36,563 but it goes back to the optics thing that we talked about at the begin with 787 00:49:37,073 --> 00:49:42,303 to begin with, we now in the world realm of social media, everybody's got a voice. 788 00:49:42,603 --> 00:49:46,953 And if I go and I test for a job at Acme company, and it's asking me 789 00:49:46,953 --> 00:49:51,173 about whether I prefer hot dogs or hamburgers, I'm going to talk about that. 790 00:49:51,423 --> 00:49:54,883 It's oh, when you take that test, it's going to be stupid, now all 791 00:49:54,883 --> 00:49:58,353 of a sudden you're branded as, the people with the stupid test. 792 00:49:58,573 --> 00:50:02,633 And so those are the reasons why you want to know about face validity. 793 00:50:02,683 --> 00:50:07,843 And that's another thing that you can really capture, not directly 794 00:50:07,843 --> 00:50:09,093 with our validation wizard. 795 00:50:09,283 --> 00:50:13,283 But you've got a group of people and it's asking them questions like, 796 00:50:13,283 --> 00:50:16,183 is the, here's the knowledge, skill or ability that's being measured. 797 00:50:16,193 --> 00:50:19,493 Does it feel similar enough to the way that the knowledge, skill or ability 798 00:50:19,493 --> 00:50:21,393 is in use on this particular job? 799 00:50:21,523 --> 00:50:25,743 So you do end up getting, that litmus test as to whether or not 800 00:50:25,773 --> 00:50:28,233 there was a face validity or not. 801 00:50:28,273 --> 00:50:31,743 And that can be very helpful just in terms of your reputation. 802 00:50:32,388 --> 00:50:33,278 Jenny Arnez: Oh, for sure. 803 00:50:33,708 --> 00:50:38,128 I've taken our CritiCall test and just the stress that I felt and 804 00:50:38,128 --> 00:50:42,538 then just not having the opportunity to finish one job before the next 805 00:50:42,538 --> 00:50:44,438 call came in was really stressful. 806 00:50:44,658 --> 00:50:44,978 Yeah. 807 00:50:46,163 --> 00:50:48,363 Mike Callen: And it's really differentiating. 808 00:50:48,363 --> 00:50:52,723 It's the thing that says, I started taking that test in 1999 before 809 00:50:52,733 --> 00:50:56,033 it was released and I'm not a multitasker either, and I hate it. 810 00:50:56,353 --> 00:50:59,943 So when I'm talking to people in the dispatch community, it's all about, 811 00:50:59,973 --> 00:51:04,013 Hey, look, I could never do your job and I've never done their job, but 812 00:51:04,013 --> 00:51:07,833 I've done the test and I hate the test, and I don't score well on it and 813 00:51:08,083 --> 00:51:10,033 so it's very effective in terms of it. 814 00:51:10,303 --> 00:51:13,963 In fact, you know what happens with the CritiCall test is people like me who are 815 00:51:13,963 --> 00:51:18,753 not multitaskers will typically get up and walk away from the test rather than 816 00:51:18,763 --> 00:51:23,253 getting hired and get up and getting up and walking away from the job after. 817 00:51:23,433 --> 00:51:26,313 Somebody spent 60 or $70, 000 to train them. 818 00:51:26,603 --> 00:51:31,903 So it's a very useful attribute of the testing process to make people 819 00:51:31,903 --> 00:51:36,963 like you and me uncomfortable and let the real multitaskers get the job. 820 00:51:36,963 --> 00:51:39,343 Jenny Arnez: And we have other tests too, right? 821 00:51:39,343 --> 00:51:45,173 We have a situational judgment tests that actually also is a face valid too. 822 00:51:45,173 --> 00:51:46,148 And. 823 00:51:47,268 --> 00:51:48,818 I found the same experience there. 824 00:51:48,978 --> 00:51:52,468 And I think one of the tests had to do with technical support or 825 00:51:52,498 --> 00:51:53,948 customer service or something. 826 00:51:53,948 --> 00:52:00,548 So all that's to say that TestGenius, and I'm not trying to make a sales statement 827 00:52:00,558 --> 00:52:04,898 here, it's just something I'm really proud of about our software, is that our 828 00:52:04,988 --> 00:52:10,368 tests are face valid You can have the opportunity to make it content valid. 829 00:52:10,838 --> 00:52:15,818 And then also we have tests that have been criterion validated as well. 830 00:52:15,858 --> 00:52:16,288 Mike Callen: Yes. 831 00:52:16,548 --> 00:52:16,878 Yeah. 832 00:52:17,218 --> 00:52:21,768 And those video situational tests that you referred to are phenomenal tests. 833 00:52:22,128 --> 00:52:23,108 They're really great. 834 00:52:23,168 --> 00:52:27,268 And they really hit on performance to mention attributes 835 00:52:27,268 --> 00:52:28,698 that the other tests don't 836 00:52:29,003 --> 00:52:29,653 don't hit on. 837 00:52:30,083 --> 00:52:36,393 Not only does it ask you how you would respond in certain situations, but it 838 00:52:36,453 --> 00:52:41,763 asks you how will that person feel about your response, given your response. 839 00:52:42,863 --> 00:52:46,443 And so there's that it's not an EQ test. 840 00:52:46,453 --> 00:52:52,113 It's not a, um, an EQ test per se that is emotional quotient 841 00:52:52,373 --> 00:52:54,223 which is a different aspect. 842 00:52:54,253 --> 00:52:59,313 But what it does is it requires that somebody has a degree of EQ to be able 843 00:52:59,313 --> 00:53:03,823 to empathize to the person that they would be talking to so that they can 844 00:53:03,853 --> 00:53:05,583 understand how that they were feeling. 845 00:53:05,853 --> 00:53:11,133 And in many jobs, it's a really important aspect, particularly if it's front facing 846 00:53:11,133 --> 00:53:14,653 or, interacting with members of the public is just a really important thing. 847 00:53:14,953 --> 00:53:19,253 And that, and that brings up did we bring up the image with the the test? 848 00:53:19,493 --> 00:53:21,603 The person the testing, the, 849 00:53:21,723 --> 00:53:22,263 Jenny Arnez: oh, we didn't, no. 850 00:53:23,013 --> 00:53:24,953 Mike Callen: Yeah let's show that for a second, because that's 851 00:53:24,953 --> 00:53:26,273 a, that's an important aspect. 852 00:53:26,273 --> 00:53:27,383 We can close on that. 853 00:53:28,808 --> 00:53:31,858 So basically, when you're testing, that would be your job 854 00:53:31,858 --> 00:53:33,538 applicant over to the left there. 855 00:53:33,808 --> 00:53:38,798 And so as you add tests to your test battery or things that you're 856 00:53:38,798 --> 00:53:43,428 talking about in an interview, you're going to get little tiny slivers 857 00:53:43,428 --> 00:53:45,158 of job performance from each one. 858 00:53:45,528 --> 00:53:50,318 And so by having a multifaceted testing strategy, you actually 859 00:53:50,318 --> 00:53:54,418 continue on through this battery, and it might not all be TestGenius. 860 00:53:54,438 --> 00:53:55,988 It might be personality testing. 861 00:53:56,218 --> 00:53:59,488 It might be the Logi-Serve situational judgment testing. 862 00:53:59,688 --> 00:54:01,038 It might be the interview. 863 00:54:01,218 --> 00:54:05,098 It might be all sorts of other aspects, training and experience. 864 00:54:05,288 --> 00:54:09,508 But each one of these things is going to contribute to the picture 865 00:54:09,728 --> 00:54:12,128 of how job ready this person is. 866 00:54:12,378 --> 00:54:15,908 And as you continue on going through this recruitment process, you're going 867 00:54:15,908 --> 00:54:17,968 to learn more and more about them. 868 00:54:18,503 --> 00:54:21,883 And if you pause for just a second right here, what you want to really 869 00:54:21,883 --> 00:54:26,893 remember about this, is that you want to get as many different pieces 870 00:54:26,893 --> 00:54:32,643 of the pie as possible, but it's equally important to make sure that 871 00:54:32,643 --> 00:54:35,183 you're measuring it in the right way. 872 00:54:35,193 --> 00:54:36,973 For instance, audio comprehension. 873 00:54:37,113 --> 00:54:40,383 You could call something audio comprehension, but it's not, if it's 874 00:54:40,383 --> 00:54:44,283 not valid and consistent with business necessity, it's not going to give you 875 00:54:44,483 --> 00:54:46,863 the results that you would expect. 876 00:54:47,093 --> 00:54:52,553 So that's just to something, something to consider and you 877 00:54:52,553 --> 00:54:53,603 can go ahead and fill it out. 878 00:54:53,613 --> 00:54:56,163 You're never going to, you're never going to know everything that you 879 00:54:56,163 --> 00:54:59,933 need to know about a person, but you get to a point where, you've 880 00:54:59,943 --> 00:55:01,463 measured all that you can measure. 881 00:55:01,463 --> 00:55:04,773 And what it does is it helps to mitigate the risk. 882 00:55:05,023 --> 00:55:10,523 I know 80 percent of what I can know about this person from having spent time 883 00:55:10,523 --> 00:55:17,163 with them, talking to them reading their, job application, their experiences, 884 00:55:17,243 --> 00:55:19,413 testing them, looking at those results. 885 00:55:19,593 --> 00:55:25,603 So my, my risk is very low in terms of what I don't know about them. 886 00:55:25,933 --> 00:55:30,843 And, so you can make a hiring decision with a greater degree of confidence when 887 00:55:30,843 --> 00:55:36,583 you have a testing strategy that fits very well with the particular situation. 888 00:55:37,098 --> 00:55:37,998 Jenny Arnez: Yeah, I love this. 889 00:55:38,008 --> 00:55:40,908 This and you've already mentioned this, but this list, it looks like 890 00:55:40,908 --> 00:55:46,368 a dispatcher position, but it could easily be administrative role where you 891 00:55:46,368 --> 00:55:48,688 might have Microsoft Excel on there or 892 00:55:48,708 --> 00:55:49,028 Mike Callen: right. 893 00:55:49,088 --> 00:55:50,028 We're all sorts of 894 00:55:50,028 --> 00:55:54,188 things, telephone answering they're just, spoken ability, English 895 00:55:54,188 --> 00:55:58,138 abilities, foreign language ability, we can measure all of these things. 896 00:55:58,418 --> 00:56:01,803 But it goes back to your test selection wizard. 897 00:56:02,033 --> 00:56:04,283 You're going to go through and you're going to choose the duties that are 898 00:56:04,283 --> 00:56:07,923 appropriate for that job, which are linked to tests that are appropriate for that 899 00:56:07,923 --> 00:56:10,253 job that you then go on and validate. 900 00:56:10,283 --> 00:56:15,643 And then you administer and you have a really good, holistic recruitment 901 00:56:15,643 --> 00:56:18,033 strategy if you follow this process here. 902 00:56:18,573 --> 00:56:18,853 Jenny Arnez: Yeah. 903 00:56:18,853 --> 00:56:20,543 And that's really something to feel good about. 904 00:56:20,543 --> 00:56:25,958 You're creating a process that hires the candidate who's most likely to 905 00:56:25,958 --> 00:56:30,968 be successful on the job, your job, which leads to their fulfillment and 906 00:56:31,018 --> 00:56:33,918 you as an employer having a good fit. 907 00:56:34,468 --> 00:56:36,078 And it's fair and defensible. 908 00:56:36,148 --> 00:56:37,768 That's something to be proud of. 909 00:56:38,558 --> 00:56:40,408 Mike Callen: Yeah, it's fair and defensible and going to 910 00:56:40,408 --> 00:56:41,868 end up with the right people. 911 00:56:42,168 --> 00:56:45,698 One of the things that I don't think people put enough emphasis 912 00:56:45,698 --> 00:56:48,248 on, it's not just about the person. 913 00:56:48,248 --> 00:56:52,308 It's not just about the person that you hire or the person that you don't hire. 914 00:56:52,638 --> 00:56:55,478 It's also about your existing employees. 915 00:56:55,698 --> 00:56:59,618 They're the people who are carrying the burden when you had the job openings. 916 00:56:59,838 --> 00:57:04,758 And so you want to reward these people by bringing somebody into the organization. 917 00:57:05,108 --> 00:57:08,428 That is going to hit the ground running that has the requisite skills and 918 00:57:08,448 --> 00:57:12,378 abilities necessary that can be trained up quickly, and then it's going to 919 00:57:12,548 --> 00:57:18,068 fit with that organization and fit is something that maybe, maybe only a human 920 00:57:18,078 --> 00:57:20,798 being in an interview can determine. 921 00:57:20,808 --> 00:57:27,588 So you want to make sure whoever you're interviewing is somebody who 922 00:57:27,608 --> 00:57:32,098 has the requisite skills and abilities already, because if you find that fit. 923 00:57:32,823 --> 00:57:35,853 And you fall in love with that person and they can't do the job. 924 00:57:36,093 --> 00:57:39,443 You have done a huge disservice to your existing employees. 925 00:57:39,663 --> 00:57:43,903 And they will get so depressed at having gone through that process, waited 926 00:57:43,903 --> 00:57:47,343 for somebody, got them all trained up only to find they can't do the job. 927 00:57:47,343 --> 00:57:49,263 And so the order is very important. 928 00:57:49,533 --> 00:57:52,933 In fact, if you go back to that first Acme slide that you have there. 929 00:57:55,108 --> 00:57:57,888 A lot of people will do this after a phone interview and that third 930 00:57:57,888 --> 00:58:01,608 brown box selected candidates take the preemployment test. 931 00:58:01,938 --> 00:58:05,658 Our recommendation is, ask people on the application. 932 00:58:05,678 --> 00:58:06,918 Everything that you can ask. 933 00:58:06,918 --> 00:58:07,998 Are you willing to work nights? 934 00:58:07,998 --> 00:58:08,658 Are you willing to work? 935 00:58:08,678 --> 00:58:09,608 Evenings, holidays. 936 00:58:10,008 --> 00:58:11,638 Whatever is appropriate for that job. 937 00:58:11,668 --> 00:58:13,508 Ask them on the application. 938 00:58:13,768 --> 00:58:19,018 Then take the people who are qualified applicants and then give them testing. 939 00:58:19,028 --> 00:58:23,118 Don't ever even talk to them because you risk falling in love with the human 940 00:58:23,118 --> 00:58:25,418 being who is not able to do the job. 941 00:58:25,798 --> 00:58:28,388 And in fact, you're using human resources. 942 00:58:28,388 --> 00:58:32,458 And I'm saying that term generically human, the resources of human beings. 943 00:58:32,458 --> 00:58:36,078 You're using them to talk to people, who might not even 944 00:58:36,078 --> 00:58:37,648 take the pre employment test. 945 00:58:37,908 --> 00:58:45,028 So why not save that valuable human time with only the most highly qualified 946 00:58:45,408 --> 00:58:50,268 applicant or candidate pool so that the things that, that software cannot 947 00:58:50,278 --> 00:58:55,688 discern, that a human being can discern get sorted out in the interview. 948 00:58:55,948 --> 00:59:02,473 And that will result in, the, a really great hire with really great benefits. 949 00:59:03,273 --> 00:59:06,713 Jenny Arnez: Yeah, that's, I'm so glad you brought it back to this, Mike. 950 00:59:06,723 --> 00:59:07,593 Thank you for that. 951 00:59:08,143 --> 00:59:08,403 Mike Callen: Many 952 00:59:08,403 --> 00:59:09,103 people do this. 953 00:59:09,103 --> 00:59:12,693 They'll come up to us as shows or on, on the phone and mention it. 954 00:59:12,703 --> 00:59:16,253 And, we have to be careful because, it might be, maybe 955 00:59:16,253 --> 00:59:18,423 that's their civil service laws. 956 00:59:18,553 --> 00:59:21,423 They have to do it that way, so we were a little soft handed 957 00:59:21,423 --> 00:59:23,093 in terms of telling people. 958 00:59:23,703 --> 00:59:27,323 You know what it is that they can do or should do, but the recommendation 959 00:59:27,323 --> 00:59:32,703 is definitely leave the humans out as long as possible and only employ 960 00:59:33,033 --> 00:59:37,753 the human beings time when you know you're working with a really 961 00:59:37,753 --> 00:59:40,143 highly qualified list of candidates. 962 00:59:40,663 --> 00:59:43,923 Applicants apply, candidates are the people that you move forward. 963 00:59:43,933 --> 00:59:47,493 And that's a differentiation, but that's where you're, where you humans 964 00:59:47,493 --> 00:59:51,383 should jump in and start measuring the things that only humans can measure. 965 00:59:51,503 --> 00:59:54,403 Jenny Arnez: I think we're reaching an ending point today. 966 00:59:55,093 --> 00:59:59,043 And Mike, I always learn so much when I talk to you, I've worked with you 967 00:59:59,113 --> 01:00:05,623 and known you for a long time and now you've added another layer on for me. 968 01:00:05,623 --> 01:00:06,533 So I appreciate it. 969 01:00:07,053 --> 01:00:08,663 Mike Callen: I appreciate all your questions. 970 01:00:08,663 --> 01:00:09,843 Your questions are great. 971 01:00:09,853 --> 01:00:14,453 And they really, cause you to start thinking about the process and 972 01:00:14,453 --> 01:00:16,063 think about other aspects of it. 973 01:00:16,063 --> 01:00:20,818 So hopefully the discussions that we've had today will be ones that 974 01:00:20,818 --> 01:00:26,508 will result in, positive step forwards for people who listen to the podcast. 975 01:00:26,538 --> 01:00:27,298 And yeah, it's great. 976 01:00:27,748 --> 01:00:27,998 Really. 977 01:00:27,998 --> 01:00:28,698 Jenny Arnez: I hope so. 978 01:00:29,028 --> 01:00:31,638 Make sure friends to check out the show notes. 979 01:00:31,638 --> 01:00:35,918 We'll have a links to the Uniform Guidelines in there and other information 980 01:00:35,918 --> 01:00:37,728 that we hope that you'll find useful. 981 01:00:38,118 --> 01:00:40,048 So thanks again for joining us today. 982 01:00:42,276 --> 01:00:45,006 Voiceover: Thanks for tuning in to Testing Testing 1 2 3. 983 01:00:45,356 --> 01:00:47,946 Brought to you by TestGenius and Biddle Consulting Group. 984 01:00:48,746 --> 01:00:51,726 Visit our website at testgenius.com for more information.