WEBVTT Kind: captions Language: en 00:00:00.000 --> 00:00:00.698 Understanding Personality  Testing with Dr. Clinton Kelly  00:00:00.698 --> 00:00:05.120 Welcome to Testing, Testing  1, 2, 3, a podcast brought to you by TestGenius.  00:00:05.120 --> 00:00:14.120 Jenny: Welcome everybody. My name is Jenny Arnez.  I'm from TestGenius, and we're so glad you tuned   00:00:14.120 --> 00:00:20.560 in today, or perhaps you're listening. And  today we have Mike Callen with us. He's the   00:00:20.560 --> 00:00:26.640 VP of Products from TestGenius. We also have  Dr. Clinton Kelly from IoPredict. Clinton,   00:00:26.640 --> 00:00:31.560 you want to tell us a little bit about yourself? Clinton: Yeah. Thanks for having me. I'd be happy   00:00:31.560 --> 00:00:36.040 to tell you a little bit about me. I have a  background in industrial and organizational   00:00:36.040 --> 00:00:41.680 psychology. And so for those who aren't  aware of that, I do not work with the   00:00:41.680 --> 00:00:45.760 depressed people at work. That's the question  I most often get from those who don't do this.   00:00:45.760 --> 00:00:50.240 I think I'm a counselor for people. Clinton: In the workforce, but no,   00:00:50.240 --> 00:00:55.640 I actually help people help companies decide  who to hire and who to promote and within   00:00:55.640 --> 00:01:00.200 organizations. So industrial organizational  psychologists, we focus on improving   00:01:00.200 --> 00:01:04.720 organizations, making things more  efficient. I specifically focus on   00:01:04.720 --> 00:01:09.840 hiring within organizations. And that's where  I've spent the almost last 20 years of my   00:01:09.840 --> 00:01:16.200 career doing is making tests for organizations. Clinton: Whether those be personality tests,   00:01:16.200 --> 00:01:20.800 cognitive ability tests, multiple physical  ability, interview questions. And so that's   00:01:20.800 --> 00:01:24.600 what I do is for all sorts of different types  of jobs and different types of organizations,   00:01:24.600 --> 00:01:30.800 I help them make tests to decide who to hire. Jenny: Mike, why don't you say a couple of words,   00:01:30.800 --> 00:01:34.520 let people know who you are as well. Mike: Sure. , I'm Mike Callen. I'm the VP of   00:01:34.520 --> 00:01:40.920 Products at Biddle Consulting Group. Our product  is TestGenius. It's a suite, a hiring suite,   00:01:40.920 --> 00:01:46.640 which is a testing platform. And it contains a  series of off the shelf office skills testing,   00:01:46.640 --> 00:01:53.820 as well as a program called CritiCall, which  is for police, fire, EMS, 911, and utility.  00:01:53.820 --> 00:01:58.880 Mike: Dispatchers, call takers, those  kinds of products. And then we have an   00:01:58.880 --> 00:02:05.440 item banking testing platform system,  which IoPredict Clinton's company actually uses   00:02:05.440 --> 00:02:12.720 for some test administration. And I would add  that Clinton and his partner, Jason, at IoPredict,   00:02:12.720 --> 00:02:18.440 worked with us for several years here at Biddle  Consulting Group and continue to work with us   00:02:18.440 --> 00:02:24.560 here at Biddle Consulting Group, yet they have  gone on and launched their own company. And we're   00:02:24.560 --> 00:02:29.680 collaborating quite a bit as we continue forward  in our own direction. Great collaborations.  00:02:29.680 --> 00:02:38.760 Jenny: Yeah, for sure. So today we're going  to focus on personality testing. And so I   00:02:38.760 --> 00:02:44.480 have to tell you when I think of personality  testing, because I'm not trained in industrial   00:02:44.480 --> 00:02:50.320 organizational psychology, I'm new to this field. Jenny: Honestly, it wasn't until I joined Biddle   00:02:50.320 --> 00:02:54.480 Consulting Group that I even knew  that I was a job like yours existed,   00:02:54.480 --> 00:03:00.520 so I'm new when I think of personality testing.  If I were to go on to Google and   00:03:01.240 --> 00:03:07.040 in fact, I did that this morning. I typed in  personality testing, things like DISC profile   00:03:07.040 --> 00:03:14.040 and Enneagram and Myers Briggs showed up. Jenny: How does that type of personality   00:03:14.040 --> 00:03:19.400 testing connect to the type of personality  testing that one might do in the workplace,   00:03:19.400 --> 00:03:25.760 that they want to use a personality test to  hire or to select the best person for the job?  00:03:25.760 --> 00:03:31.280 Clinton: Yeah, that's a great question.  And there are some similarities, but there   00:03:31.280 --> 00:03:36.040 are important differences between those. Clinton: They are trying to measure traits   00:03:36.040 --> 00:03:42.320 or preferences of individuals. Some personality  components, however, tests like the Myers Briggs,   00:03:42.320 --> 00:03:48.600 for example, it was not developed with the purpose  of making hiring decisions. In fact, the creators   00:03:48.600 --> 00:03:52.600 of the Myers Briggs, they specifically say  on their website, our test is not designed   00:03:52.600 --> 00:03:56.760 to be used to help hire individuals. Clinton: It's not designed to predict   00:03:56.760 --> 00:04:02.000 performance in a job. It's more designed to  give feedback for the individuals,   00:04:02.000 --> 00:04:08.120 maybe preferences of careers they may enjoy, but  it's not designed to be used. In an employment   00:04:08.120 --> 00:04:12.960 context, neither is like the disc or strengths  finder. Some of those, they weren't designed   00:04:12.960 --> 00:04:18.960 with employee prediction with hiring in mind. Jenny: Okay. All right. It's interesting because   00:04:18.960 --> 00:04:24.200 one of my daughters recently applied for a job and  they asked in the online application, what's your   00:04:24.200 --> 00:04:31.600 Strength Finders profile? What Enneagram are you? Clinton: Yeah. I actually this morning I Googled   00:04:31.600 --> 00:04:35.560 Myers Briggs selection and I found  a number of websites explaining how   00:04:35.560 --> 00:04:40.760 to use the Myers Briggs within selection. Clinton: So people use it, but like I said,   00:04:40.760 --> 00:04:44.800 Myers Briggs themselves says, don't use  it for selection, but people do it anyway.  00:04:44.800 --> 00:04:50.200 Mike: That's very interesting, isn't it? We're  really focused on doing things the right way,   00:04:50.200 --> 00:04:54.800 following the Uniform Guidelines.  And there's, it seems like there's   00:04:54.800 --> 00:04:59.240 a lot of mavericks out in the selection world. Mike: At any rate, it's just very interesting.   00:04:59.240 --> 00:05:04.120 I'm actually surprised I didn't realize  that there were people that were asking for this   00:05:04.120 --> 00:05:10.120 kind of a profile. I would imagine that, in terms  of defensibility, that's a tough one to defend,   00:05:10.120 --> 00:05:15.400 right? If somebody says, what's your DISC profile  or something like that and you don't get hired.   00:05:15.400 --> 00:05:20.120 If I didn't get hired, I might challenge that. Clinton: Yeah, it can be problematic. Like I said,   00:05:20.120 --> 00:05:26.680 because when we talk about, tests and validity -  often misunderstood topics – often those tests are   00:05:26.680 --> 00:05:31.800 not valid for that intended purpose. And for the  purpose of selection, and yeah, you can definitely   00:05:31.800 --> 00:05:36.320 have some problems if it were challenged. Jenny: Okay, so a couple of questions come to   00:05:36.320 --> 00:05:42.080 mind. Mike, you mentioned Uniform Guidelines for  those who are watching or listening to this and   00:05:42.080 --> 00:05:46.460 have no idea what the Uniform Guidelines  are. Can you guys give a definition?  00:05:46.460 --> 00:05:50.920 Mike: Yeah, Clinton, go ahead. It's your  space. So I would I would get close,   00:05:50.920 --> 00:05:54.760 but I think you give it the best run. Clinton: Okay. Yeah, no, sure. I'd be   00:05:54.760 --> 00:05:59.240 happy to. So the Uniform Guidelines were  created in the late 1970s. So they've been   00:05:59.240 --> 00:06:03.360 around for a while and they are  the Uniform Guidelines on employee selection   00:06:03.360 --> 00:06:09.280 procedures. And they lay out the, like it says,  the guidelines or the requirements. If you're   00:06:09.280 --> 00:06:15.600 going to use tests in a hiring situation on what  you need to do to demonstrate that those tests   00:06:15.600 --> 00:06:21.440 are valid for use in that specific situation. Clinton: And the Uniform Guidelines when they,   00:06:21.440 --> 00:06:27.120 when we use the term test, that is, it's very  broad. It's not test in the traditional sense,   00:06:27.120 --> 00:06:32.800 maybe most people think of the multiple-choice  test. It’s anytime you are reducing your applicant   00:06:32.800 --> 00:06:39.360 pool. So if two people apply to the job and we  say yes to one person and no to another, whatever   00:06:39.360 --> 00:06:44.320 we did to make that decision, that was a test. Clinton: An interview is a test, a personality   00:06:44.320 --> 00:06:49.760 test is a test. If you're doing resume screens  and you make a pile of yes and a pile of no,   00:06:49.760 --> 00:06:54.440 however you, whatever the criteria used to make  that decision, it is a test under the Uniform   00:06:54.440 --> 00:06:58.880 Guidelines. And so the Uniform Guidelines lay out,  they lay out guidelines and say, if you're going,   00:06:58.880 --> 00:07:02.760 whenever you're going to reduce  your number of job applicants to make a hiring   00:07:02.760 --> 00:07:06.280 decision, there are certain requirements  that you need to comply with. If you don't,   00:07:06.280 --> 00:07:11.560 you can't - you could have problems legally. Mike: That goes back even to your job posting   00:07:11.560 --> 00:07:15.800 and your choice of where you put your job  posting, right? You put your job posting in   00:07:15.800 --> 00:07:24.440 a spot where you're not going to get a diverse  applicant pool. You have limited, essentially   00:07:24.440 --> 00:07:29.520 limited your selectability in that case. Mike: If you put invalid criteria in there,   00:07:29.520 --> 00:07:33.880 inother words, if you're posting a job and  it says a driver's license is required,   00:07:33.880 --> 00:07:38.240 but a driver's license isn't really required  for that job, you have reduced your population   00:07:38.240 --> 00:07:44.680 there as well. And so there's a lot of things in  our arena that go well beyond the traditional,   00:07:44.680 --> 00:07:50.440 written multiple choice type of test or  work sample test or personality test.  00:07:50.440 --> 00:07:55.040 Clinton: Yeah, good, great points. Mike: And I think another thing that's   00:07:55.040 --> 00:08:01.600 important to bring up is that I was taught early  on that it's not just hiring that is   00:08:01.600 --> 00:08:07.000 selection, it's hiring, it's promotion, it's  training opportunities that may be available   00:08:07.000 --> 00:08:11.240 to some and not available to others, like in  a union environment.If you're going to go,   00:08:11.240 --> 00:08:17.500 up to a journeyman level from, I forgot what the  basic term is, but those are all selection, right?  00:08:17.500 --> 00:08:22.400 Clinton: Yeah, you're getting put into what we  call maybe a high potential group. Sometimes   00:08:22.400 --> 00:08:28.280 organizations will identify high potentials that  are then put on like a track for management track   00:08:28.280 --> 00:08:33.800 that inherently comes with potential increased  earnings, other things that are benefit that   00:08:33.800 --> 00:08:39.440 tie into selection.So whatever you're using to  identify like high potential,those could also   00:08:39.440 --> 00:08:44.760 fall under the Uniform Guidelines requirements. Mike: Interesting. That's great. Thank you.  00:08:44.760 --> 00:08:50.440 Jenny: So is there a separate definition  for personality tests that's unique,   00:08:50.440 --> 00:08:58.120 that's specific for what you do? Clinton: Not so much a definition.   00:08:58.120 --> 00:09:03.960 I would say more that intended [00:09:00] use of  personality tests for selection.They are built and   00:09:03.960 --> 00:09:10.280 designed for use in selection. So that is their  intended purpose. The way I like to compare it as   00:09:10.280 --> 00:09:17.120 just like a test in general. Let’s just say I have  an accounting test used to hire accountants and   00:09:17.120 --> 00:09:22.080 it's a great test to hire accountants. It is valid  to hire accountants. And now let's say I give that   00:09:22.080 --> 00:09:27.960 test to help me decide who I should pick in my NBA  draft in my NBA lottery, who I should pick next.   00:09:27.960 --> 00:09:32.920 And I give, my NBA draft prospects this accounting  test. You'd probably say, what are you doing? That   00:09:32.920 --> 00:09:37.840 test is not valid. It's not that it's not a valid  test. It's valid for helping me hire accountants.   00:09:37.840 --> 00:09:44.160 It's not valid to help me hire NBA players. And  so that's similar to like these different types   00:09:44.160 --> 00:09:48.120 of personality tests. It's not that they're not  valid like Myers Briggs. It's just not valid to   00:09:48.120 --> 00:09:56.360 use in a hiring situation. Yeah, for that purpose. Jenny: So you've used that word valid a few times   00:09:56.360 --> 00:10:00.002 now. Do you want to give a definition  of what that actually means? 00:10:00.002 --> 00:10:04.160 Clinton: Yeah. And so validity,  there are different, what we call   00:10:04.160 --> 00:10:09.880 types of validity or evidences of validity. Clinton: And what the most common one for   00:10:09.880 --> 00:10:16.880 personality is criterion, what we call criterion  validity. To not go into lots of different types   00:10:16.880 --> 00:10:20.360 of validity, because I don't think this  is the purpose of today's conversation,   00:10:20.360 --> 00:10:24.600 we'll briefly cover a personality  test under the Uniform Guidelines.   00:10:24.600 --> 00:10:29.960 It requires criterion related validation. Clinton: So if you are using a personality test   00:10:29.960 --> 00:10:37.080 for selection, there should be some evidence of  criterion related validation. And if we pull up,   00:10:37.080 --> 00:10:41.840 I can pull up a slide here. We can show  a few here, criterion validity, what that   00:10:41.840 --> 00:10:49.100 is a mathematical relationship between how people  score on the And some measure of job performance.  00:10:49.100 --> 00:10:55.280 Clinton: So for example, if we say, higher scores  on the test, they're more likely to sell more of   00:10:55.280 --> 00:10:59.480 the product if it's a sales position. So if  we say higher scores on the test, they make   00:10:59.480 --> 00:11:04.480 more sales that would be, if we can  mathematically show the relationship between   00:11:04.480 --> 00:11:10.480 your score on the personality test and how much  product you sell, that is criterion validation.  00:11:10.480 --> 00:11:13.880 Jenny: Okay. Clinton: And this is a mathematical   00:11:13.880 --> 00:11:19.240 relation to show this kind of graph it out. You  can see if we plot here on the X axis, we have   00:11:19.240 --> 00:11:24.360 test scores that range in this case, 0 to 100.  And we have some sort of measure of performance on   00:11:24.360 --> 00:11:30.180 the side. We can graph this. You see this person  has a test score of 22, a job performance of 31.  00:11:30.180 --> 00:11:36.400 Clinton: Over here, test score of 85, job  performance of 55. You can visually see the   00:11:36.400 --> 00:11:41.840 relationship amongst these dots. What criterion  validity is it's you're coming up with, we call   00:11:41.840 --> 00:11:49.720 it validity coefficient, which is essentially just  a correlation coefficient in many cases that shows   00:11:49.720 --> 00:11:54.800 this linear relationship between how you score  on the test and how you perform on the job.  00:11:54.800 --> 00:11:58.680 Clinton: And so that's what we mean by validity  for personalities. We can show that this is a   00:11:58.680 --> 00:12:04.880 valid predictor of [00:12:00] success in the job. Mike: So that validity coefficient would be like   00:12:04.880 --> 00:12:10.440 the strength of the validity, right? Clinton: Yes, the strength and the   00:12:10.440 --> 00:12:14.240 higher that validity coefficient if it's  a traditional correlation coefficient,   00:12:14.240 --> 00:12:19.680 it's going to range from 0 to 1 with 1. Clinton: 0 being a perfect correlation,   00:12:19.680 --> 00:12:24.360 which that doesn't exist. But in theory,  you could get there. The higher that number,   00:12:24.360 --> 00:12:28.840 the stronger that relationship visually,  what that would look like a correlation   00:12:28.840 --> 00:12:34.000 of 1. Every blue dot here would be  perfectly on this red line. That would   00:12:34.000 --> 00:12:39.160 be a correlation of one. So the tighter  these blue dots are to this red line,   00:12:39.160 --> 00:12:42.880 the stronger that correlation coefficient. Clinton: The more spread out they are around   00:12:42.880 --> 00:12:48.280 the line, the lower that value is going to be.  The closer it's going to be to zero. Perfect.   00:12:48.280 --> 00:12:53.680 Correlation coefficient of zero would essentially  be saying, it doesn't matter how you score on the   00:12:53.680 --> 00:12:58.680 test, we have no idea how you score on this test  gives us no indication whatsoever of how you're   00:12:58.680 --> 00:13:00.640 going to perform on the job. Clinton: That would be a   00:13:00.640 --> 00:13:07.920 correlation coefficient of zero. Jenny: And where does the criteria come from?  00:13:07.920 --> 00:13:13.000 Clinton: And there are, that is flexible. The  Uniform Guidelines say they just need to be   00:13:13.000 --> 00:13:17.960 important criteria. And they mentioned some  examples. Some could be supervisor ratings of   00:13:17.960 --> 00:13:23.480 performance. It could be a sales and a sales job. Clinton: It could be turnover. So we could   00:13:23.480 --> 00:13:29.400 correlate with turnover like our people. Can  we predict maybe who's less likely to turn over   00:13:29.400 --> 00:13:35.320 by with this personality test? And so that is  open and it just needs to be what they call   00:13:35.320 --> 00:13:39.960 criteria that are important to the organization  to the job. So most typically, it's going to   00:13:39.960 --> 00:13:45.200 be some sort of measure of performance, if it's  sales supervisor ratings or turnover, those are   00:13:45.200 --> 00:13:52.200 the types of things that are very common to see. Mike: And, when you have a test like this you can   00:13:52.200 --> 00:13:59.320 assemble this test and then you can just, start  collecting all sorts of information and find out   00:13:59.320 --> 00:14:07.120 that a test correlates to some sort  of aspect that isn't necessarily valid. It wasn't   00:14:07.120 --> 00:14:12.800 intended to select for that purpose. So can  you talk a little bit about that philosophy   00:14:12.800 --> 00:14:17.360 of not just creating a test and throwing  everything up against the wall to see what   00:14:17.360 --> 00:14:23.600 sticks versus having some intention when you  design? It's a little bit risky, we've gone   00:14:23.600 --> 00:14:28.720 through this step with you folks many times. Mike: And there's a great deal of risk when you   00:14:28.720 --> 00:14:33.120 go through and you put this together because  you could potentially go through and not find   00:14:33.120 --> 00:14:38.360 what you're looking for. And that doesn't open up  the ability to find, some other unrelated thing,  00:14:38.360 --> 00:14:41.600 Clinton: Yeah, no, you need to be intentional  when you're designing these tests.  00:14:41.600 --> 00:14:44.800 Clinton: We don't just, like you said,  just throw a bunch of stuff against the   00:14:44.800 --> 00:14:50.800 wall and see what sticks. And so whenever we're  designing a test, a personality test for a job,   00:14:50.800 --> 00:14:55.000 we want to be intentional and we want to  do what we call a job analysis where we   00:14:55.000 --> 00:14:59.980 analyze. What is done on the job? What are  the requirements of the position? 00:14:59.980 --> 00:15:04.080 Clinton: What are individuals spending their  time doing? What are the most important parts,   00:15:04.080 --> 00:15:08.320 the most difficult parts? We're going to  interview employees, talk with them. And   00:15:08.320 --> 00:15:14.560 so you need to do your research and be intentional  with what you are doing. One of the, I'll share a   00:15:14.560 --> 00:15:18.680 slide here. Sorry, I'm going to skip forward. Clinton: This is an example for one we put   00:15:18.680 --> 00:15:23.760 together for a factory worker. And this doesn't  show everything. We did multiple interviews and   00:15:23.760 --> 00:15:28.960 we even went on site and watched employees, but  we took a look and we said, Hey, there's some   00:15:28.960 --> 00:15:34.600 attributes we've noticed in our research, in our  analysis of the job, the workers that appear to,   00:15:34.600 --> 00:15:38.300 in this case, stay on the job, here's  some characteristics that we're seeing.  00:15:38.300 --> 00:15:41.800 Clinton: These eight things that they're  self confident, analytical, forthright,   00:15:41.800 --> 00:15:46.120 and et cetera. It takes some time. So  you need, like Mike, like you said,   00:15:46.120 --> 00:15:50.040 we need to be intentional and purposeful.  So you're not just saying let's just throw   00:15:50.040 --> 00:15:57.160 some stuff out there and see if we get lucky. Mike: So we, when we've created personality   00:15:57.160 --> 00:16:03.760 tests we've worked together to create personality tests, we have used a concurrent   00:16:03.760 --> 00:16:10.440 validation strategy. And I know that, basically  I would say that, from my perspective, the two   00:16:10.440 --> 00:16:17.120 basic ways to do to validate a personality test  would be either a concurrent study or a predictive   00:16:17.120 --> 00:16:23.380 study. Why don't you talk a little bit about that  and, maybe what some advantages of either are.  00:16:23.380 --> 00:16:28.920 Clinton: Okay. So great. So those are the two  types, the concurrent or predictive. Concurrent,   00:16:29.480 --> 00:16:35.600 what that is done typically with your existing  employees. So if you are in a job and you have,   00:16:35.600 --> 00:16:40.760 a few hundred employees already employed in this  position, what we do is we develop the test or   00:16:40.760 --> 00:16:45.800 take an existing test from some vendor. Clinton: We administer it to your current   00:16:45.800 --> 00:16:50.920 employees. And then we correlate that with  measures of job performance. So if you have   00:16:50.920 --> 00:16:56.480 existing measures to job performance, or we  can create new measures of job performance.   00:16:56.480 --> 00:17:00.600 And so that is a concurrent one. As you take your  existing employees, have them complete   00:17:00.600 --> 00:17:05.880 the test and we correlate it with job performance. Clinton: Predictive is where we start to give   00:17:05.880 --> 00:17:11.320 the test to job applicants. And often you'll  not be scoring, it's collecting data in the   00:17:11.320 --> 00:17:15.720 background. So people are applying for this  job and you're giving 'em this new test. The   00:17:15.720 --> 00:17:19.300 applicants don't know that it's not being used,  but you're really not considering their score.  00:17:19.300 --> 00:17:25.120 Clinton: And as you hire new individuals on  the job you then, at a later point in time,   00:17:25.120 --> 00:17:29.320 correlate how they scored on the test with  measures of job performance. And so both   00:17:29.320 --> 00:17:33.240 you're trying to do the same thing- correlate test  scores with job performance. The main difference   00:17:33.240 --> 00:17:38.200 is concurrent is with existing employees. Clinton: Predictive, usually use it with job   00:17:38.200 --> 00:17:42.960 applicants and at a later date, you have to come  back. So the advantage, you can probably tell just   00:17:42.960 --> 00:17:47.120 from the way I'm describing it, the advantage of  concurrent is it's typically faster. You can take   00:17:47.120 --> 00:17:52.320 your existing employees. You don't have to wait  six months or a year to find how they panned out.  00:17:52.320 --> 00:17:58.120 Mike: Yeah, and your population is much more  under control as well, because obviously,   00:17:58.120 --> 00:18:02.200 if you're doing a predictive study, you're going to be testing all these applicants,   00:18:02.200 --> 00:18:07.880 and you might not hire 75 percent of them. So  you've actually collected data that you're not   00:18:07.880 --> 00:18:15.080 then going to be able to apply later. And  then I guess one so one advantage of the   00:18:15.080 --> 00:18:23.080 predictive study is that you get the broadest  range of people that in terms of knowledge,   00:18:23.080 --> 00:18:27.640 skill, ability, and personal characteristics  with the concurrent study, you have a little   00:18:27.640 --> 00:18:32.920 bit of range restriction. Is that not correct? Clinton: Yeah, we can't, you can get some range   00:18:32.920 --> 00:18:38.040 restriction because these individuals are current  employees and we assume they're performing at an   00:18:38.040 --> 00:18:42.920 adequate level or else they would not be current  employees or they would no longer be employed   00:18:42.920 --> 00:18:47.920 there. So that can be a downside is that range  restriction is because there are scores on your   00:18:47.920 --> 00:18:52.560 test. There's maybe not as much variability  or differences in how they score. One of the,   00:18:52.560 --> 00:18:56.720 one of the upsides of a predictive  study is you are getting, like you said,   00:18:56.720 --> 00:19:00.560 the range of responses and you can also  get, you're getting that real   00:19:00.560 --> 00:19:03.680 life situation from the job applicants. Clinton: They are really applying for a   00:19:03.680 --> 00:19:08.480 job. So you're mimicking what you intend the  test to be used for in a concurrent sample.   00:19:08.480 --> 00:19:12.920 When you're giving it to current employees,  they already have the job. And so they may   00:19:12.920 --> 00:19:17.120 complete the test from a different frame of  reference than your job applicants. And so   00:19:17.120 --> 00:19:21.520 that can be a potential downside of a concurrent. Clinton: And so there are benefits and negatives   00:19:21.520 --> 00:19:26.840 to both approaches, but both are equally  allowed under the uniform guidelines. And   00:19:26.840 --> 00:19:31.760 both have been shown from research perspective  to, to basically result in creating tests   00:19:31.760 --> 00:19:38.680 that are predictive of performance.. Jenny: So you're mentioning developing   00:19:38.680 --> 00:19:42.560 tests or validating tests that  are used for employee selection.  00:19:42.560 --> 00:19:45.240 Jenny: Are they ever use it  a post hire environment for   00:19:45.240 --> 00:19:52.800 perhaps employee development or training? Clinton: They can be, but that's often what   00:19:52.800 --> 00:19:57.360 you'll see, maybe with Myers Briggs or DISC  or Strengths Finder some of those. And so   00:19:57.360 --> 00:20:01.680 for the intended use, again, it comes back to  that intended use. Depends on what   00:20:01.680 --> 00:20:06.000 the test was designed to be used for. Clinton: Some tests have multiple,   00:20:06.000 --> 00:20:10.440 have been designed to be used for multiple types  of things. But this is something to be aware of.   00:20:10.440 --> 00:20:15.240 What is your test designed to do? And sometimes  I'll see this with clients who want to take a   00:20:15.240 --> 00:20:19.480 test. And not necessarily a personality test, but  let's say a technical skills test and they want   00:20:19.480 --> 00:20:23.780 to use it in a diagnostic way to diagnose where  someone is needs training or things like that.  00:20:23.780 --> 00:20:27.760 Clinton: And I'm like, that test wasn't designed  to do that. It gave you some initial information   00:20:27.760 --> 00:20:31.440 to make a hiring decision, but it really wasn't  designed to diagnose and tell you here's their   00:20:31.440 --> 00:20:36.280 training needs. And so you just need to be  aware of what your tests can and can't do.  00:20:36.280 --> 00:20:47.000 Mike: There's a a maxim in HR as well, that  says job performance trumps testing. One of   00:20:47.000 --> 00:20:54.040 the things that happens is when a person goes  from applicant to candidate, to employee is   00:20:54.040 --> 00:21:01.280 that you start to have these instances where,  job performance is being recorded.   00:21:01.280 --> 00:21:07.640 And so very often we have people who will  ask us, Oh, is it okay to give our employees   00:21:07.640 --> 00:21:12.720 these pre-employment tests and then, if they  can't pass them any longer to get rid of them.  00:21:12.720 --> 00:21:19.280 Mike: And generally speaking, that's a really  dangerous area to tread into because, you have a   00:21:19.280 --> 00:21:24.000 much better job-related measure of what's going  on, which is their performance on the job at   00:21:24.000 --> 00:21:29.000 that point. So at any rate, it's just an aside. Clinton: That's that's a great point and kind of   00:21:29.000 --> 00:21:32.440 shows in this, you can see that in this  graph, tests are by no means perfect.  00:21:32.440 --> 00:21:37.080 Clinton: Like you said, job performance trumps,  trumps the test scores. Like in this line here,   00:21:37.080 --> 00:21:41.240 you can see some of these blue dots, they're  not on the line. So in some cases, the test,   00:21:41.240 --> 00:21:46.560 a person may out predict what we outperform  and what we predict. In some cases, a person   00:21:46.560 --> 00:21:51.640 may underperform what we predict. Mike: And so just to be clear here,   00:21:51.640 --> 00:21:56.400 any one of these dots is the  intersection between an individual's   00:21:56.400 --> 00:22:01.280 test score and their performance on the job. Mike: So that one that's in the middle   00:22:01.280 --> 00:22:08.200 top there above the word test score. There's  somebody who scored approximately 50, but rated   00:22:08.200 --> 00:22:13.840 almost 60 on their performance measure, right? Clinton: Yeah, so in that case, they performed   00:22:13.840 --> 00:22:18.840 better than we thought they would on the job. Mike: And then conversely, down below the   00:22:18.840 --> 00:22:21.080 line to the left of the 40,   00:22:21.080 --> 00:22:25.240 there's somebody who scored, is that right? Mike: Am I saying this right? Yeah. They under   00:22:25.240 --> 00:22:29.440 tested and under performed, either way, right? Clinton: Yeah, but they were a lower test score,   00:22:29.440 --> 00:22:32.754 but even then, the performance  was lower than was anticipated.  00:22:32.754 --> 00:22:34.280 Mike: Yeah. Thank you. Clinton: And so that's   00:22:34.280 --> 00:22:40.000 something that the tests are, but the tests are  not perfect measures, but what I always say is   00:22:40.000 --> 00:22:43.600 it's, they're better than the alternative. Clinton: And so they are getting you,   00:22:43.600 --> 00:22:49.440 they are increasing your odds. And so it's like  when people sometimes will say this person is just   00:22:49.440 --> 00:22:54.080 not a great test taker. I often will say there's  also sometimes someone's great grandma who smoked   00:22:54.080 --> 00:22:59.840 40 cigarettes a day and lived until she was 97.  But you probably would not say in, in, in general,   00:22:59.840 --> 00:23:04.080 the trend is if you smoke a lot, you're  likely going to have some other health issues.  00:23:05.840 --> 00:23:11.480 Mike: We're going to expect everybody who smokes  several packs a day to live to a hundred. That's   00:23:11.480 --> 00:23:17.120 not the, that's not the case. Another side of  that's really important, at least I always try   00:23:17.120 --> 00:23:22.400 to bring this up when I have this conversation  with people, is that the alternative is that   00:23:22.400 --> 00:23:28.820 a human being is making a selection based  upon, in terms of the applicant, nothing.  00:23:28.820 --> 00:23:33.760 Mike: You have selected somebody who will be  hired and you have selected somebody who will   00:23:33.760 --> 00:23:39.320 not be hired, for apparently no reason  whatsoever, or no concrete reason. And   00:23:39.320 --> 00:23:44.320 so that's another reason why to have this  decision making process tied to something   00:23:44.320 --> 00:23:49.280 that's observable that you can find patterns in. Mike: Or if you can find that there's disparities,   00:23:49.280 --> 00:23:54.040 you can measure those things and see what  exists and then remediate that situation.  00:23:54.040 --> 00:24:00.840 Jenny: Yes, so if an employer would like  to begin using a personality   00:24:00.840 --> 00:24:08.560 assessment. How do they begin? Clinton: There's a, I guess there's   00:24:08.560 --> 00:24:13.120 a couple of different ways. There are,  you can, reach out to vendors out there.  00:24:13.120 --> 00:24:16.840 Clinton: There are two different approaches.  There are, is what I would call the build a   00:24:16.840 --> 00:24:22.000 custom test from scratch approach, and there  is the off the shelf approach. There are   00:24:22.000 --> 00:24:26.760 existing personality tests that are off the  shelf. And then there's the, Hey, we build   00:24:26.760 --> 00:24:31.520 it custom for this particular job approach. Clinton: What you're looking for may depend on   00:24:31.520 --> 00:24:37.640 your situation in your organization. If you're  a smaller organization and, or it's a role in   00:24:37.640 --> 00:24:42.160 your organization where you don't have that many  employees. Off the shelf is probably going to be   00:24:42.160 --> 00:24:47.140 the better route where you, a vendor has spent  the time to develop a test for that type of job.  00:24:47.140 --> 00:24:50.920 Clinton: And because you just don't have to  do a sort of study where you can show this   00:24:50.920 --> 00:24:55.400 mathematical relationship between test scores and  job performance, that requires data. And if you   00:24:55.400 --> 00:24:59.800 only have 10 employees, Iin a certain position  in the organization, you don't have the data and   00:24:59.800 --> 00:25:03.600 the necessary data to demonstrate a  relationship and have any confidence in that.  00:25:03.600 --> 00:25:07.680 Clinton: And so in that situation, you're going  to need to rely on an off the shelf test where   00:25:07.680 --> 00:25:11.960 they've done some of that research and can show  how it predicts performance for that type of   00:25:11.960 --> 00:25:16.560 job. If you have a large organization,  you may be in the spot where you say,   00:25:16.560 --> 00:25:20.880 let's custom develop this. If we have a few  hundred employees in a position, you can do that.  00:25:20.880 --> 00:25:24.200 Clinton: So those are the kind of the  2 approaches. And I'm sure there's,   00:25:24.200 --> 00:25:28.520 I can get into like, how can you continue to want  to get into how we do that once identified maybe   00:25:28.520 --> 00:25:33.860 which one, the custom or the off the shelf. Jenny: Yeah. Yeah. I think you should. Yeah.  00:25:33.860 --> 00:25:37.760 Clinton: So then when you get  a customer or off the shelf,   00:25:37.760 --> 00:25:43.480 there are different types out there of tests. Clinton: A lot of them, when it comes to   00:25:43.480 --> 00:25:50.000 personalities, they're typically statements. When  you take a personality test, you're often reading   00:25:50.000 --> 00:25:55.640 statements. And I'll show here a couple examples.  This is an example we'll call a forced choice   00:25:55.640 --> 00:26:01.120 personality test, where you say, Hey, I'm more  likely to, and this is a silly example   00:26:01.120 --> 00:26:05.280 here, but eat a salad or eat a hamburger. Clinton: And you have some points in the   00:26:05.280 --> 00:26:08.680 continuum. Sometimes you're just given two  options. You just pick one or the other.   00:26:08.680 --> 00:26:13.480 This point, you're given four choices on other  personality tests. You're just given a statement   00:26:13.480 --> 00:26:18.160 and you just slay like true or false, or that's  like me or not like me. For example, I enjoy   00:26:18.160 --> 00:26:24.760 team projects and you would say true or false. Clinton: Other times one of the most common still   00:26:24.760 --> 00:26:29.800 is we'll call it like your type skills that  strongly disagree to strongly agree. Again,   00:26:29.800 --> 00:26:35.760 I enjoy team projects or strongly disagree or  somewhat disagree or to the strongly agree. So   00:26:35.760 --> 00:26:41.600 personalities generally are in that kind  of flavor where they, you can see they   00:26:41.600 --> 00:26:46.960 have statements and you're making some sort of  agreement or disagreement with that statement.  00:26:46.960 --> 00:26:52.400 Clinton: And if you're looking to implement it,  so you need to go, if you're going to the vendor   00:26:52.400 --> 00:26:59.520 route, go to the vendor and you should ask the  vendor about what this test was designed for and   00:26:59.520 --> 00:27:04.840 its intended use to make sure they  built this for selection. And not only built it   00:27:04.840 --> 00:27:10.640 for selection, but they have validity for your job  type, going back to that accountant and NBA player   00:27:10.640 --> 00:27:14.720 example to make sure they're not giving you a  test that were designed to select NBA players   00:27:14.720 --> 00:27:18.640 and you're trying to use it to hire accountants.  Just what was this test? Where, what validity   00:27:18.640 --> 00:27:25.840 evidence do they have for your job type? Mike: So as well, Clinton, there are many,   00:27:25.840 --> 00:27:32.720 I may not be using the right word, but I would say  standardized personality tests where the battery   00:27:32.720 --> 00:27:41.000 like Hogan, for instance, they're going to ask the  same 110 questions of every person, but over time,   00:27:41.000 --> 00:27:50.800 they're going to do small studies with certain  job titles or certain groups of people in order   00:27:50.800 --> 00:27:58.200 to be able to take this standardized test and key  it or score it in such a way that it doesn't have   00:27:58.200 --> 00:28:03.280 meets with that particular population on the other side of the coin.  00:28:03.280 --> 00:28:10.480 Mike: There can be a custom test, which  wouldn't necessarily have standardized   00:28:10.480 --> 00:28:16.680 questions. They might have questions that are  written that have more face validity. They're   00:28:16.680 --> 00:28:24.160 more written directly towards this particular  job title that you're using the test for,   00:28:24.160 --> 00:28:29.200 is that not correct? And do you have any  recommendations about that, or any positives,   00:28:29.200 --> 00:28:34.320 negatives about either of those scenarios? Clinton: Yeah, that is correct. Most of the,   00:28:34.320 --> 00:28:38.760 a lot of the tests out there that measure  what we call the big five personality   00:28:38.760 --> 00:28:43.320 characteristics. And it, like you said, Mike,  it's the same exact test for every position.  00:28:43.320 --> 00:28:48.440 Clinton: The difference being what, where you  need to fall on the characteristics. So let's   00:28:48.440 --> 00:28:54.240 say openness to experience. You may, they may  say on this job profile or job type we found that   00:28:54.240 --> 00:28:58.360 people who are more likely to be successful are  high in openness to experience. And another job   00:28:58.360 --> 00:29:01.560 type people who are more likely  to be successful are in the middle range   00:29:01.560 --> 00:29:05.480 on openness to experience for the lower range. Clinton: And so they're given the same exact test,   00:29:05.480 --> 00:29:09.840 but there's a different, like constellation of  profiles of where you fall in these big five   00:29:09.840 --> 00:29:16.440 personality characteristics that they're saying  is indicative of success. And I don't know if   00:29:16.440 --> 00:29:21.600 there's really like a, say this one is better  than the other, but when you have a test that's   00:29:21.600 --> 00:29:27.400 custom designed for a specific position, the good  thing about that is that it was, like I said,   00:29:27.400 --> 00:29:31.320 that it was custom tailored for that job type. Clinton: And so that, again, the frame of   00:29:31.320 --> 00:29:35.760 reference, even the way the items are written from  what I've seen, just in my personal experience   00:29:35.760 --> 00:29:40.440 with work, I tend to see higher validity  coefficients with those tests when design,   00:29:40.440 --> 00:29:45.760 when they are well designed and put together.  Higher validity coefficients than with the   00:29:45.760 --> 00:29:49.720 general tests that are just Hey, we then custom  create a profile depending on your job type,   00:29:49.720 --> 00:29:54.240 but it's the same exact test for all jobs. Mike: Yeah. I think another aspect that's   00:29:54.240 --> 00:30:00.840 very important to maybe particularly important  now in this age of social media is   00:30:00.840 --> 00:30:06.280 the applicant satisfaction part of it.  When I was growing up and I had nothing   00:30:06.280 --> 00:30:11.880 to do with this particular career, I took  a lot of tests or did interviews or did   00:30:11.880 --> 00:30:17.440 physical ability tests, those kinds of things. Mike: And so I've often reflected back on those   00:30:17.440 --> 00:30:25.760 processes and thought about which ones were very  satisfying to me. They had the face validity and   00:30:25.760 --> 00:30:31.840 face validity is basically, how does it feel?  Does it feel to the applicant? Like it would be   00:30:31.840 --> 00:30:35.960 something that they might encounter on the job.  So it doesn't mean the test is valid or not.  00:30:35.960 --> 00:30:40.200 Mike: It just really has to do with my  own impression of it. And I do remember   00:30:40.200 --> 00:30:45.680 going back through in my mind, some of these  experiences I had and I was like, Oh, that   00:30:45.680 --> 00:30:51.040 was a valid test. That was not fair. That was a  very unfair experience. And I think some of these   00:30:51.040 --> 00:30:58.360 personality tests, where you're asking, would you  rather be, at a a noisy party with loud music or   00:30:58.360 --> 00:31:02.680 walking by yourself on a beach? Mike: What does that have to do with being in   00:31:02.680 --> 00:31:09.920 ABC job? There's a lot of things that don't feel  really good about that. Whereas if you're asking   00:31:09.920 --> 00:31:17.760 it in context of the job, do you prefer an active  work environment where your day goes by quickly   00:31:17.760 --> 00:31:23.240 versus a quiet environment where it's maybe a  little less stressful or those kinds of things   00:31:23.240 --> 00:31:28.720 that has a lot better feel to it than the other. Clinton: That's a great point that face validity,   00:31:28.720 --> 00:31:33.360 which is not what we always say. It's not like a  technical type of validity, but it is real in the   00:31:33.360 --> 00:31:39.160 sense that candidate perceptions of the experience  matter. If candidates feel a positive experience,   00:31:39.160 --> 00:31:43.160 they're less likely to, for example, to challenge  a test or beat it up or complain about it.  00:31:43.160 --> 00:31:46.120 Clinton: And that is one good thing of  the custom develop test too is that they   00:31:46.120 --> 00:31:50.680 tend to have a little more face validity  because the questions by written with that   00:31:50.680 --> 00:31:57.840 job in mind and so yeah that is one benefit  another benefit of custom developed tests.  00:31:57.840 --> 00:32:00.240 Mike: Perfect. Thank you. Jenny: So here's   00:32:00.240 --> 00:32:01.600 what we're going to do. Jenny: We're going to take   00:32:01.600 --> 00:32:06.020 a short break and then we'll be right  back with everyone in just a moment.  00:32:06.020 --> 00:32:11.520 Voiceover: And we'll be right back  after a word from our sponsor.  00:32:11.520 --> 00:32:16.320 Ready to revolutionize your HR strategies?  Head over to TestGenius.com to discover our   00:32:16.320 --> 00:32:20.040 latest tools and solutions designed  to streamline your hiring processes.  00:32:20.040 --> 00:32:26.280 Jenny: Welcome back everybody. And just a  reminder, we're with Dr. Clinton Kelly from   00:32:26.280 --> 00:32:32.720 IoPredict, and we're talking about personality  testing. So when someone takes a personality   00:32:32.720 --> 00:32:39.360 test or a personality assessment, a score is  spit out for that person, correct? Can you talk   00:32:39.360 --> 00:32:45.680 a little bit about personality test scoring? Clinton: Yeah. So when it comes to the scores,   00:32:45.680 --> 00:32:50.440 there are different scores that are done and  those, when we talk about those off the shelf   00:32:50.440 --> 00:32:56.400 tests that measure the big five personality  characteristics. You'll often get a scale, a   00:32:56.400 --> 00:33:01.120 score in each of those big five personality, what  we call sub skills. You'll get like an   00:33:01.120 --> 00:33:06.280 openness to experience score, an extroversion  score, conscientiousness score, and then you'll   00:33:06.280 --> 00:33:10.440 often get an overall fit recommendation. Clinton: And this is something that's on   00:33:10.440 --> 00:33:17.480 the screen. This is for a custom developed tests  that Biddle and TestGenius, where you give like   00:33:17.480 --> 00:33:22.440 an overall recommendation. And you can see here  on the screen here, in this case, we say, Hey,   00:33:22.440 --> 00:33:28.500 people are highly recommended. They have a 77  percent chance of being successful on the job.  00:33:28.500 --> 00:33:32.880 Clinton: And this is based on job performance data  collected from actual individuals. You can see   00:33:32.880 --> 00:33:37.560 the numbers here, the number of individuals.  And so what we give on the score, we give a,   00:33:37.560 --> 00:33:42.480 usually give an overall recommendation score. Most  tests you'll see like some sort of overall score.   00:33:42.480 --> 00:33:48.640 And then you'll see some sort of subscale scores. Clinton: The overall score is typically what is   00:33:48.640 --> 00:33:53.200 going to be the driving factor in whatever  decisions you are making or informing the   00:33:53.200 --> 00:33:57.880 decision. What you don't want to do,  and is tempting for many people to do,   00:33:57.880 --> 00:34:02.080 who are not super familiar  with personality tests, is take subscale   00:34:02.080 --> 00:34:07.520 scores. And let that drive your decision. Clinton: Because the subscales are a sub   00:34:07.520 --> 00:34:12.800 component. They are sub, like it says, they're  sub to the overall score. So less data points   00:34:12.800 --> 00:34:18.120 from the test are driving subscale scores 'cause  it's just a portion of the overall test. And   00:34:18.120 --> 00:34:22.920 so it really should be that overall score or  profile score you get from a personality that   00:34:22.920 --> 00:34:27.720 should drive your recommendation and those sub  skills can be informative pieces of information   00:34:28.440 --> 00:34:34.280 to then maybe dive into maybe in an interview or  another step of the selection process. But don't   00:34:34.280 --> 00:34:39.280 let a subscale score. I've seen this happen where  someone says there's some overall profiles like   00:34:39.280 --> 00:34:42.800 we highly recommend them based on the overall  profile, but they'll see a subscale score where   00:34:42.800 --> 00:34:46.960 they're fairly low on some subscales I don't  know if you should hire this person - I'm like,   00:34:46.960 --> 00:34:53.240 no, don't let that drive the decision. Mike: That's a really important point   00:34:53.240 --> 00:34:59.080 that you brought up about the subscale is  that you can use it to drive questions in   00:34:59.080 --> 00:35:04.720 the interview process. In fact, very  often the reports I know the reports that are in   00:35:04.720 --> 00:35:10.360 the test that we've created together, you've  included those drill down questions because   00:35:10.360 --> 00:35:17.040 that gives you a really great opportunity  to, a find out, did the test potentially   00:35:17.040 --> 00:35:25.560 really nail this subscale or did it maybe miss  the subscale or furthermore is this weakness,   00:35:25.560 --> 00:35:32.840 maybe something that the candidate is familiar  with and has the ability to work around it, which   00:35:32.840 --> 00:35:39.240 can turn a weakness into a strength. So is that  always something that's included with personality   00:35:39.240 --> 00:35:45.040 tests for selection or are we just lucky? Clinton: Yeah, not always. So Yeah,   00:35:45.040 --> 00:35:50.120 they're not always included, so that is, yeah, so  there are some where they just give you an overall   00:35:50.120 --> 00:35:54.720 score and and I've even seen some that just give  you an overall score, there are even those sub,   00:35:54.720 --> 00:35:59.560 sub scores, and then, but yeah, so it is just, it  is nice when you get more information like that,   00:35:59.560 --> 00:36:02.800 there are a number of tests out  there that will provide sub scores and even   00:36:02.800 --> 00:36:07.280 some additional information or potential  prodding questions, but yeah, it is,   00:36:07.280 --> 00:36:11.026 like I said, it's great when you can have that  additional information to prod into an interview.  00:36:11.026 --> 00:36:14.920 Clinton: Yep. But like Mike had said, just  don't let us, don't let a subscale score drive   00:36:14.920 --> 00:36:20.240 your overall decision. It's like letting, like one  quarter, one quarter of performance in a business   00:36:20.240 --> 00:36:23.600 drive your overall decision. If you don't look  at the yearly profit and loss, you're like, no,   00:36:23.600 --> 00:36:27.800 over a year we made a lot of money.But one  quarter one, it was slow. And now you say,   00:36:27.800 --> 00:36:30.600 Hey, we've got to close the company  because things are horrible. It's no,   00:36:30.600 --> 00:36:36.520 don't let the subscale score, the overall score. Mike: I want to tell a quick story because   00:36:36.520 --> 00:36:40.240 this was something that was so  impactful about 10 years or so ago.  00:36:40.240 --> 00:36:46.040 Mike: We created a testing process for  nurses and it included knowledge testing,   00:36:46.040 --> 00:36:50.960 video situational judgment testing, and  personality testing. And one of our.   00:36:50.960 --> 00:36:58.160 Clients told us about a instance where they were  reviewing the results with the applicant in real   00:36:58.160 --> 00:37:02.380 time. So they had tested and then they moved them onto an interview process.  00:37:02.380 --> 00:37:10.320 Mike: And the experience level of this applicant  was very good. The knowledge was very good. The   00:37:10.320 --> 00:37:16.280 situational judgment was good, but there was  this one subscale, which was hostility. And   00:37:16.280 --> 00:37:23.560 there was a red flag on the hostility scale. And  the interviewer asked the first prodding question,   00:37:23.560 --> 00:37:29.120 Hey, so tell me about a time, and it was some  question that had to do with when they were in   00:37:29.120 --> 00:37:34.920 a stressful situation, how they didn't handle it. Mike: And then they asked the second prodding   00:37:34.920 --> 00:37:40.760 question and then started, started to  feel that this was exposing something,   00:37:40.760 --> 00:37:47.840 asked the third prodding question, and at this  point, the candidate slammed her hand down on   00:37:47.840 --> 00:37:52.800 the desk, said, I've been to anger management  classes. I've worked through this. If you   00:37:52.800 --> 00:37:57.760 would just move on, I would appreciate it. Mike: There was a great instance where,   00:37:57.760 --> 00:38:02.160 you know, that this did bring up  something that should have been discussed   00:38:02.160 --> 00:38:08.080 and was discussed and herein is the real  value in collecting that data and having   00:38:08.080 --> 00:38:12.600 an opportunity to talk about it because,  boy, when you talk about dodging a bullet,   00:38:12.600 --> 00:38:16.040 that was an important one right there. Clinton: And that is a great example of how to   00:38:16.040 --> 00:38:20.680 use that. In some cases, those subscale scores, it  may be you may find something like you said that's   00:38:20.680 --> 00:38:26.760 yeah, this is a red flag, but more, more often  than not, I'll see individuals without prodding,   00:38:26.760 --> 00:38:30.560 like I said, without doing those follow up  questions, take a subscale and automatically maybe   00:38:30.560 --> 00:38:41.440 disqualify someone because of a subscale score. Mike: And we talked a little bit about how you   00:38:41.440 --> 00:38:46.920 use it for selection. There's a, we do a lot of  work in the 911 space, and I know we're going   00:38:46.920 --> 00:38:53.480 to do a follow up podcast where we talk a little  bit more about that. But there's a, there's fairly   00:38:53.480 --> 00:39:00.280 popular personality tests in that space that  are content based validated, which   00:39:00.280 --> 00:39:04.760 the Uniform Guideline specifically says,  don't content validate a personality   00:39:04.760 --> 00:39:09.080 test.You want a criterion validate it. Mike: So let's talk, why don't you talk   00:39:09.080 --> 00:39:13.840 a little bit about the defensibility aspect?  I think that the stuff we've talked about up   00:39:13.840 --> 00:39:19.160 to this point is really the utility aspect of  the validation. Here's why you go through this   00:39:19.160 --> 00:39:25.040 process so that it will do a good. Good job of  selecting, but why do you go through this process   00:39:25.040 --> 00:39:34.960 to make sure that if somebody, an applicant or  the DOL or the DOJ or the EEOC or some plaintiff's   00:39:34.960 --> 00:39:41.040 attorney has a problem with something that you  did, what, why does this, how does this come   00:39:41.040 --> 00:39:47.840 into play and how does this help somebody ease  out of an uncomfortable defensibility situation?  00:39:47.840 --> 00:39:53.440 Clinton: Yeah, great question. When it comes  to those situations, if you have a test that,   00:39:53.440 --> 00:39:57.480 for example, is just content valid or  you don't have the proper validation   00:39:57.480 --> 00:40:02.720 in place for a personality test to show that  it's predictive of performance,   00:40:02.720 --> 00:40:08.000 if you end up passing, let's say your personality  test ends up passing more males than females or   00:40:08.000 --> 00:40:13.280 more females than males or in one or just more  of one protected group over another, that's   00:40:13.280 --> 00:40:18.040 when the Uniform Guidelines kick into place. Clinton: The Uniform Guidelines say, Hey, if your   00:40:18.040 --> 00:40:25.520 test screens out a disproportionate amount of one  protected group status, you need to show that your   00:40:25.520 --> 00:40:30.920 test is a valid test and that it's screening those  people out because they can't do the job. And it's   00:40:30.920 --> 00:40:36.020 not screening them out just because they're female  or they're Hispanic or whatever group that may be.  00:40:36.020 --> 00:40:40.640 Clinton: And so if you are using a test that  doesn't have the proper validation in place,   00:40:40.640 --> 00:40:44.840 and let's say you're screening out a  disproportionate number of some protected group   00:40:44.840 --> 00:40:51.040 status, you will lose in court if you get sued. Mike: Yeah. Good point. That's good advice. It   00:40:51.040 --> 00:40:58.800 doesn't happen very often, but it does happen  that a process gets questioned and it can really 00:40:58.800 --> 00:41:06.348 financially impact an organization in  terms of legal decisions that might go towards the plaintiff, 00:41:06.348 --> 00:41:09.383 but also it hurts their reputation in  the space. 00:41:09.383 --> 00:41:14.055 You don't want to be that group that got sued because they did something incorrectly. 00:41:14.055 --> 00:41:25.080 Clinton: Yeah. So if you're a vendor out there and you're not doing your due diligence,  yeah, it can really hurt your reputation. Clinton: And that's what I tell with clients  and said, yeah, you can, we can put out a test   00:41:25.080 --> 00:41:30.000 that doesn't have the proper kind of validity  to back it up but I don't want to do that.   00:41:30.000 --> 00:41:34.300 Cause that's going to, that's going to hurt  my reputation as an individual in the field.  00:41:34.300 --> 00:41:41.280 Mike: Yeah. Yeah. And so you and I know, and  a lot of people may or may not know that,   00:41:41.280 --> 00:41:48.960 when criterion validity is established elsewhere,  the Uniform Guidelines in 7B does allow for   00:41:48.960 --> 00:41:57.880 validity to be transported from the environment  in which it was established over to an employer's   00:41:57.880 --> 00:42:01.400 own specific environment. Mike: Why don't you talk a little bit   00:42:01.400 --> 00:42:06.660 about what that means? Processes and  any recommendations that you have.  00:42:06.660 --> 00:42:11.320 Clinton: Yeah. And that is a  really great strategy for jobs,   00:42:11.320 --> 00:42:15.440 especially where you just don't have the sample  size to conduct these criteria types of criteria   00:42:15.440 --> 00:42:19.560 and validation studies in your own organization. Clinton: And so the Uniform guidelines allow us,   00:42:19.560 --> 00:42:22.840 let's say Mike, cause we've done, we'll talk  about this in a later one, but we've worked with   00:42:22.840 --> 00:42:28.440 the 911 space with personality testing.  And a lot of 911 dispatching agencies,   00:42:28.440 --> 00:42:32.720 they just don't have hundreds of dispatchers.  They have maybe a handful. There's a lot   00:42:32.720 --> 00:42:36.080 of smaller jurisdictions out there. Clinton: So if they want to implement   00:42:36.080 --> 00:42:41.400 a personality test and say, Hey, what profile  or what type of personality is more likely to   00:42:41.400 --> 00:42:45.680 be successful in this job? They just  logistically don't have the numbers to   00:42:45.680 --> 00:42:52.640 do such a study. And so what 7b, section 7b of  the Uniform Guidelines allows is it says, hey,   00:42:52.640 --> 00:42:57.280 if a validation study for let's say 911  dispatchers on the personalities has been   00:42:57.280 --> 00:43:03.800 conducted with a specific test, you as  a smaller group, you can adopt this test and not   00:43:03.800 --> 00:43:08.600 have to do your own criterion validation study. Clinton: If you can show that your dispatchers,   00:43:08.600 --> 00:43:14.840 your 911 dispatchers perform substantially the  same major work behaviors as those 911 dispatchers   00:43:14.840 --> 00:43:18.920 where this study was originally conducted. So  basically you have to show, hey, there's a match,   00:43:18.920 --> 00:43:24.760 there's overlap between my job where I want to use  this test and the job where they, what that they   00:43:24.760 --> 00:43:29.160 used to initially develop and validate this test. Clinton: And if you can show that the two jobs   00:43:29.160 --> 00:43:33.360 are substantially similar in the tasks  and the work behaviors they perform,   00:43:33.360 --> 00:43:37.320 you can transport that validity  under Section 7B of the Uniform   00:43:37.320 --> 00:43:41.760 Guidelines. And that fulfills  their validation requirements.  00:43:41.760 --> 00:43:47.880 Mike: My understanding is that when you're  looking at these work behaviors that were   00:43:47.880 --> 00:43:52.520 covered under the initial study, that's  the work behaviors that you're looking   00:43:52.520 --> 00:43:57.360 at in terms of transporting this validity. Mike: There may be some other work behaviors   00:43:57.360 --> 00:44:01.080 that weren't covered under that validation  study that aren't really a part   00:44:01.080 --> 00:44:05.560 of this conversation. But in terms of  being able to transport the validity   00:44:05.560 --> 00:44:12.160 from this original study and bringing it over  to your environment, is it not those particular   00:44:12.160 --> 00:44:19.000 performance dimensions that were purported to be  measured or in that original study by that test?  00:44:19.000 --> 00:44:22.800 Clinton: yeah, you do want to look at those.  So those original things that are measuring   00:44:22.800 --> 00:44:27.440 that test are those work behaviors also  important in your job? And and you say,   00:44:27.440 --> 00:44:32.440 that's the critical part is showing that overlap.  And it's not, it is not super difficult to do it   00:44:32.440 --> 00:44:38.800 is as long as I say as long as in the original  study, the vendor did their due diligence and how   00:44:38.800 --> 00:44:43.840 they showed what the work behaviors were in that  original study. So if they've done their homework,   00:44:43.840 --> 00:44:48.840 it's a very easy process. I have seen this in  the past where a client that I was working with   00:44:48.840 --> 00:44:51.760 was wanting to use a test. Clinton: And they had some   00:44:51.760 --> 00:44:57.440 criteria and validation, but they weren't able to  give me information on what the people in those   00:44:57.440 --> 00:45:00.720 jobs did or what exactly they did in that  original study. And so it was,   00:45:00.720 --> 00:45:04.120 we couldn't really transport the validity because  they didn't do their homework on that initial   00:45:04.120 --> 00:45:07.400 study. And so that's why it's important. Clinton: When you do that initial study,   00:45:07.400 --> 00:45:11.320 the vendors should have done a good  thorough job analysis of analyzing   00:45:11.320 --> 00:45:16.160 the job. So you can make sure there's a  match between where they did the study and   00:45:16.160 --> 00:45:20.840 your job where you want to implement the test. Mike: That's a really great point. And we might   00:45:20.840 --> 00:45:27.160 want to toot our horns a little bit right here. Mike: But, being a shop, both of our shops   00:45:27.160 --> 00:45:33.400 are shops where we lean very heavily on the  Uniform Guidelines and, the principles and   00:45:33.400 --> 00:45:41.200 the legislation and, laws that have been put  into place so that we are doing things right   00:45:41.200 --> 00:45:46.080 to begin with so that if somebody does determine  that they'd like to transport that validity over   00:45:46.080 --> 00:45:52.000 to their environment, they're able to do that. Mike: But you do really want to do your research   00:45:52.000 --> 00:45:57.880 or as much of the research as you can to make sure  that you are partnering up with an organization   00:45:57.880 --> 00:46:04.240 that has the same values and is doing  things right to begin with. Otherwise it's the   00:46:04.240 --> 00:46:09.320 garbage in garbage out philosophy, right? If  you start to transport over a validation study,   00:46:09.320 --> 00:46:13.620 that's a bad validation study.Then,  you're protected by nothing, right?  00:46:13.620 --> 00:46:18.280 Clinton: Yeah. And so that's one thing. If you're  looking to implement a study is ask that vendor   00:46:18.280 --> 00:46:22.800 to say, Hey, do you have a validation study? They  should have a technical report. If you're asking   00:46:22.800 --> 00:46:27.520 a vendor who's got a personality test and you  ask questions about validation or do you have a   00:46:27.520 --> 00:46:32.640 technical report that documents things and they  start to kind of him hot or dodge you or they   00:46:32.640 --> 00:46:38.280 send you like a one page more a marketing flyer. Clinton: Those would be red flags in my opinion   00:46:38.280 --> 00:46:41.720 to say, this person may not have done their  homework. You should be hearing things like   00:46:41.720 --> 00:46:45.400 Uniform Guidelines, like criterion  validation. There's certain terms,   00:46:45.400 --> 00:46:51.800 keywords that the vendor should be mentioning. Mike: Very good. We've mentioned Uniform   00:46:51.800 --> 00:46:54.800 Guidelines a lot of times. Mike: So we actually have a   00:46:54.800 --> 00:47:00.560 website set up. That's uniformguidelines.com where  the Uniform Guidelines are in there.   00:47:00.560 --> 00:47:06.360 Topically by heading hyperlinked. There's also the  Questions and Answers to the Uniform Guidelines,   00:47:06.360 --> 00:47:11.280 which came what, 10 or 20 years later, right? Or  I don't remember how many years, but that was,   00:47:11.280 --> 00:47:16.960 they circled back through and said, okay, we've  got this document, looking back now upon it,   00:47:16.960 --> 00:47:20.740 what are different scenarios that have been  encountered and how did this get interpreted?  00:47:20.740 --> 00:47:25.800 Mike: So both the Uniform Guidelines and the  questions and answers are very valuable in   00:47:25.800 --> 00:47:32.880 terms of knowing what the best practices are for  testing and selection. So uniform guidelines.com,   00:47:32.880 --> 00:47:38.200 you can go there and you can, take a look at it,  bookmark it and refer to it anytime that you'd   00:47:38.200 --> 00:47:42.260 like, that's a resource that we sponsor. Mike: Jenny, what else?  00:47:42.260 --> 00:47:48.000 Jenny: I actually think we're getting to a point  where we're going to start to wind down. Okay. And   00:47:48.000 --> 00:47:54.560 just want to let everybody know this is part  one or episode, session one with Dr. Kelly.   00:47:54.560 --> 00:47:59.880 And next time when we meet, we're going to be  talking about, specifically about personality   00:47:59.880 --> 00:48:06.320 testing in our CritiCall application. Mike: Okay. Which would be the dispatch realm.  00:48:06.320 --> 00:48:12.280 Jenny: 911 emergency dispatchers. So let  me do this. Let me change the screen here,   00:48:12.280 --> 00:48:18.560 bring us all back. And as we wind down Clinton,  if someone wants to get in touch with you,   00:48:18.560 --> 00:48:21.560 how do they do that? Clinton: Yeah. So you   00:48:21.560 --> 00:48:30.120 can reach out through our website at iopredict.com Clinton: And my email is just CKelly. So my name,   00:48:30.120 --> 00:48:36.880 first initial, last name, CKelly at ioPredict.com.  You can also reach out to Mike over at TestGenius   00:48:36.880 --> 00:48:40.880 and Biddle. And like I said, we have a great  relationship and we work together hand on hand   00:48:40.880 --> 00:48:44.860 and hand on a lot of validation projects.  And so those are some ways to get in touch.  00:48:44.860 --> 00:48:51.600 Mike: And you and Jason still are maintained  at Ckelly at biddle.com as well where we work   00:48:51.600 --> 00:48:58.400 together very closely and we continue to  really value this relationship and want to   00:48:58.400 --> 00:49:02.160 continue to be able to collaborate together for many years to come. It's been a,   00:49:02.160 --> 00:49:06.512 it's been really great. And I for one, would  like to thank you for being on with us today.  00:49:06.512 --> 00:49:11.640 Mike: , I think we maybe even went a little bit  longer than we had targeted. But it's been really   00:49:11.640 --> 00:49:19.880 valuable hearing from your experience regarding  this subject, which is, can be a little nebulous,   00:49:19.880 --> 00:49:24.480 it can be a little scary to tiptoe into.  So I've learned a lot and thank you so   00:49:24.480 --> 00:49:27.320 much for your time today. Clinton: Yeah. Thank you   00:49:27.320 --> 00:49:32.000 for having me. I really I enjoyed it. Jenny: Oh, it's been great. And we do have   00:49:32.000 --> 00:49:37.480 show notes that will show on the page where you  see this video. We'll have links to IoPredict's   00:49:37.480 --> 00:49:44.400 website to uniform guidelines.com and other  resources that were mentioned in this video. So   00:49:44.400 --> 00:49:49.700 thanks to everyone for listening and for watching. Mike: Great. Thank you.  00:49:49.700 --> 00:49:55.360 Thanks for tuning in to Testing,  Testing 123 brought to you by TestGenius and   00:49:55.360 --> 00:50:01.840 Biddle Consulting Group. Visit our website at  testgenius.com for more information.