Understanding Personality Testing with Dr. Clinton Kelly
Welcome to Testing, Testing 1, 2, 3, a podcast brought to you by TestGenius.
Jenny: Welcome everybody. My name is Jenny Arnez. I'm from TestGenius, and we're so glad you tuned
in today, or perhaps you're listening. And today we have Mike Callen with us. He's the
VP of Products from TestGenius. We also have Dr. Clinton Kelly from IoPredict. Clinton,
you want to tell us a little bit about yourself? Clinton: Yeah. Thanks for having me. I'd be happy
to tell you a little bit about me. I have a background in industrial and organizational
psychology. And so for those who aren't aware of that, I do not work with the
depressed people at work. That's the question I most often get from those who don't do this.
I think I'm a counselor for people. Clinton: In the workforce, but no,
I actually help people help companies decide who to hire and who to promote and within
organizations. So industrial organizational psychologists, we focus on improving
organizations, making things more efficient. I specifically focus on
hiring within organizations. And that's where I've spent the almost last 20 years of my
career doing is making tests for organizations. Clinton: Whether those be personality tests,
cognitive ability tests, multiple physical ability, interview questions. And so that's
what I do is for all sorts of different types of jobs and different types of organizations,
I help them make tests to decide who to hire. Jenny: Mike, why don't you say a couple of words,
let people know who you are as well. Mike: Sure. , I'm Mike Callen. I'm the VP of
Products at Biddle Consulting Group. Our product is TestGenius. It's a suite, a hiring suite,
which is a testing platform. And it contains a series of off the shelf office skills testing,
as well as a program called CritiCall, which is for police, fire, EMS, 911, and utility.
Mike: Dispatchers, call takers, those kinds of products. And then we have an
item banking testing platform system, which IoPredict Clinton's company actually uses
for some test administration. And I would add that Clinton and his partner, Jason, at IoPredict,
worked with us for several years here at Biddle Consulting Group and continue to work with us
here at Biddle Consulting Group, yet they have gone on and launched their own company. And we're
collaborating quite a bit as we continue forward in our own direction. Great collaborations.
Jenny: Yeah, for sure. So today we're going to focus on personality testing. And so I
have to tell you when I think of personality testing, because I'm not trained in industrial
organizational psychology, I'm new to this field. Jenny: Honestly, it wasn't until I joined Biddle
Consulting Group that I even knew that I was a job like yours existed,
so I'm new when I think of personality testing. If I were to go on to Google and
in fact, I did that this morning. I typed in personality testing, things like DISC profile
and Enneagram and Myers Briggs showed up. Jenny: How does that type of personality
testing connect to the type of personality testing that one might do in the workplace,
that they want to use a personality test to hire or to select the best person for the job?
Clinton: Yeah, that's a great question. And there are some similarities, but there
are important differences between those. Clinton: They are trying to measure traits
or preferences of individuals. Some personality components, however, tests like the Myers Briggs,
for example, it was not developed with the purpose of making hiring decisions. In fact, the creators
of the Myers Briggs, they specifically say on their website, our test is not designed
to be used to help hire individuals. Clinton: It's not designed to predict
performance in a job. It's more designed to give feedback for the individuals,
maybe preferences of careers they may enjoy, but it's not designed to be used. In an employment
context, neither is like the disc or strengths finder. Some of those, they weren't designed
with employee prediction with hiring in mind. Jenny: Okay. All right. It's interesting because
one of my daughters recently applied for a job and they asked in the online application, what's your
Strength Finders profile? What Enneagram are you? Clinton: Yeah. I actually this morning I Googled
Myers Briggs selection and I found a number of websites explaining how
to use the Myers Briggs within selection. Clinton: So people use it, but like I said,
Myers Briggs themselves says, don't use it for selection, but people do it anyway.
Mike: That's very interesting, isn't it? We're really focused on doing things the right way,
following the Uniform Guidelines. And there's, it seems like there's
a lot of mavericks out in the selection world. Mike: At any rate, it's just very interesting.
I'm actually surprised I didn't realize that there were people that were asking for this
kind of a profile. I would imagine that, in terms of defensibility, that's a tough one to defend,
right? If somebody says, what's your DISC profile or something like that and you don't get hired.
If I didn't get hired, I might challenge that. Clinton: Yeah, it can be problematic. Like I said,
because when we talk about, tests and validity - often misunderstood topics – often those tests are
not valid for that intended purpose. And for the purpose of selection, and yeah, you can definitely
have some problems if it were challenged. Jenny: Okay, so a couple of questions come to
mind. Mike, you mentioned Uniform Guidelines for those who are watching or listening to this and
have no idea what the Uniform Guidelines are. Can you guys give a definition?
Mike: Yeah, Clinton, go ahead. It's your space. So I would I would get close,
but I think you give it the best run. Clinton: Okay. Yeah, no, sure. I'd be
happy to. So the Uniform Guidelines were created in the late 1970s. So they've been
around for a while and they are the Uniform Guidelines on employee selection
procedures. And they lay out the, like it says, the guidelines or the requirements. If you're
going to use tests in a hiring situation on what you need to do to demonstrate that those tests
are valid for use in that specific situation. Clinton: And the Uniform Guidelines when they,
when we use the term test, that is, it's very broad. It's not test in the traditional sense,
maybe most people think of the multiple-choice test. It’s anytime you are reducing your applicant
pool. So if two people apply to the job and we say yes to one person and no to another, whatever
we did to make that decision, that was a test. Clinton: An interview is a test, a personality
test is a test. If you're doing resume screens and you make a pile of yes and a pile of no,
however you, whatever the criteria used to make that decision, it is a test under the Uniform
Guidelines. And so the Uniform Guidelines lay out, they lay out guidelines and say, if you're going,
whenever you're going to reduce your number of job applicants to make a hiring
decision, there are certain requirements that you need to comply with. If you don't,
you can't - you could have problems legally. Mike: That goes back even to your job posting
and your choice of where you put your job posting, right? You put your job posting in
a spot where you're not going to get a diverse applicant pool. You have limited, essentially
limited your selectability in that case. Mike: If you put invalid criteria in there,
inother words, if you're posting a job and it says a driver's license is required,
but a driver's license isn't really required for that job, you have reduced your population
there as well. And so there's a lot of things in our arena that go well beyond the traditional,
written multiple choice type of test or work sample test or personality test.
Clinton: Yeah, good, great points. Mike: And I think another thing that's
important to bring up is that I was taught early on that it's not just hiring that is
selection, it's hiring, it's promotion, it's training opportunities that may be available
to some and not available to others, like in a union environment.If you're going to go,
up to a journeyman level from, I forgot what the basic term is, but those are all selection, right?
Clinton: Yeah, you're getting put into what we call maybe a high potential group. Sometimes
organizations will identify high potentials that are then put on like a track for management track
that inherently comes with potential increased earnings, other things that are benefit that
tie into selection.So whatever you're using to identify like high potential,those could also
fall under the Uniform Guidelines requirements. Mike: Interesting. That's great. Thank you.
Jenny: So is there a separate definition for personality tests that's unique,
that's specific for what you do? Clinton: Not so much a definition.
I would say more that intended [00:09:00] use of personality tests for selection.They are built and
designed for use in selection. So that is their intended purpose. The way I like to compare it as
just like a test in general. Let’s just say I have an accounting test used to hire accountants and
it's a great test to hire accountants. It is valid to hire accountants. And now let's say I give that
test to help me decide who I should pick in my NBA draft in my NBA lottery, who I should pick next.
And I give, my NBA draft prospects this accounting test. You'd probably say, what are you doing? That
test is not valid. It's not that it's not a valid test. It's valid for helping me hire accountants.
It's not valid to help me hire NBA players. And so that's similar to like these different types
of personality tests. It's not that they're not valid like Myers Briggs. It's just not valid to
use in a hiring situation. Yeah, for that purpose. Jenny: So you've used that word valid a few times
now. Do you want to give a definition of what that actually means?
Clinton: Yeah. And so validity, there are different, what we call
types of validity or evidences of validity. Clinton: And what the most common one for
personality is criterion, what we call criterion validity. To not go into lots of different types
of validity, because I don't think this is the purpose of today's conversation,
we'll briefly cover a personality test under the Uniform Guidelines.
It requires criterion related validation. Clinton: So if you are using a personality test
for selection, there should be some evidence of criterion related validation. And if we pull up,
I can pull up a slide here. We can show a few here, criterion validity, what that
is a mathematical relationship between how people score on the And some measure of job performance.
Clinton: So for example, if we say, higher scores on the test, they're more likely to sell more of
the product if it's a sales position. So if we say higher scores on the test, they make
more sales that would be, if we can mathematically show the relationship between
your score on the personality test and how much product you sell, that is criterion validation.
Jenny: Okay. Clinton: And this is a mathematical
relation to show this kind of graph it out. You can see if we plot here on the X axis, we have
test scores that range in this case, 0 to 100. And we have some sort of measure of performance on
the side. We can graph this. You see this person has a test score of 22, a job performance of 31.
Clinton: Over here, test score of 85, job performance of 55. You can visually see the
relationship amongst these dots. What criterion validity is it's you're coming up with, we call
it validity coefficient, which is essentially just a correlation coefficient in many cases that shows
this linear relationship between how you score on the test and how you perform on the job.
Clinton: And so that's what we mean by validity for personalities. We can show that this is a
valid predictor of [00:12:00] success in the job. Mike: So that validity coefficient would be like
the strength of the validity, right? Clinton: Yes, the strength and the
higher that validity coefficient if it's a traditional correlation coefficient,
it's going to range from 0 to 1 with 1. Clinton: 0 being a perfect correlation,
which that doesn't exist. But in theory, you could get there. The higher that number,
the stronger that relationship visually, what that would look like a correlation
of 1. Every blue dot here would be perfectly on this red line. That would
be a correlation of one. So the tighter these blue dots are to this red line,
the stronger that correlation coefficient. Clinton: The more spread out they are around
the line, the lower that value is going to be. The closer it's going to be to zero. Perfect.
Correlation coefficient of zero would essentially be saying, it doesn't matter how you score on the
test, we have no idea how you score on this test gives us no indication whatsoever of how you're
going to perform on the job. Clinton: That would be a
correlation coefficient of zero. Jenny: And where does the criteria come from?
Clinton: And there are, that is flexible. The Uniform Guidelines say they just need to be
important criteria. And they mentioned some examples. Some could be supervisor ratings of
performance. It could be a sales and a sales job. Clinton: It could be turnover. So we could
correlate with turnover like our people. Can we predict maybe who's less likely to turn over
by with this personality test? And so that is open and it just needs to be what they call
criteria that are important to the organization to the job. So most typically, it's going to
be some sort of measure of performance, if it's sales supervisor ratings or turnover, those are
the types of things that are very common to see. Mike: And, when you have a test like this you can
assemble this test and then you can just, start collecting all sorts of information and find out
that a test correlates to some sort of aspect that isn't necessarily valid. It wasn't
intended to select for that purpose. So can you talk a little bit about that philosophy
of not just creating a test and throwing everything up against the wall to see what
sticks versus having some intention when you design? It's a little bit risky, we've gone
through this step with you folks many times. Mike: And there's a great deal of risk when you
go through and you put this together because you could potentially go through and not find
what you're looking for. And that doesn't open up the ability to find, some other unrelated thing,
Clinton: Yeah, no, you need to be intentional when you're designing these tests.
Clinton: We don't just, like you said, just throw a bunch of stuff against the
wall and see what sticks. And so whenever we're designing a test, a personality test for a job,
we want to be intentional and we want to do what we call a job analysis where we
analyze. What is done on the job? What are the requirements of the position?
Clinton: What are individuals spending their time doing? What are the most important parts,
the most difficult parts? We're going to interview employees, talk with them. And
so you need to do your research and be intentional with what you are doing. One of the, I'll share a
slide here. Sorry, I'm going to skip forward. Clinton: This is an example for one we put
together for a factory worker. And this doesn't show everything. We did multiple interviews and
we even went on site and watched employees, but we took a look and we said, Hey, there's some
attributes we've noticed in our research, in our analysis of the job, the workers that appear to,
in this case, stay on the job, here's some characteristics that we're seeing.
Clinton: These eight things that they're self confident, analytical, forthright,
and et cetera. It takes some time. So you need, like Mike, like you said,
we need to be intentional and purposeful. So you're not just saying let's just throw
some stuff out there and see if we get lucky. Mike: So we, when we've created personality
tests we've worked together to create personality tests, we have used a concurrent
validation strategy. And I know that, basically I would say that, from my perspective, the two
basic ways to do to validate a personality test would be either a concurrent study or a predictive
study. Why don't you talk a little bit about that and, maybe what some advantages of either are.
Clinton: Okay. So great. So those are the two types, the concurrent or predictive. Concurrent,
what that is done typically with your existing employees. So if you are in a job and you have,
a few hundred employees already employed in this position, what we do is we develop the test or
take an existing test from some vendor. Clinton: We administer it to your current
employees. And then we correlate that with measures of job performance. So if you have
existing measures to job performance, or we can create new measures of job performance.
And so that is a concurrent one. As you take your existing employees, have them complete
the test and we correlate it with job performance. Clinton: Predictive is where we start to give
the test to job applicants. And often you'll not be scoring, it's collecting data in the
background. So people are applying for this job and you're giving 'em this new test. The
applicants don't know that it's not being used, but you're really not considering their score.
Clinton: And as you hire new individuals on the job you then, at a later point in time,
correlate how they scored on the test with measures of job performance. And so both
you're trying to do the same thing- correlate test scores with job performance. The main difference
is concurrent is with existing employees. Clinton: Predictive, usually use it with job
applicants and at a later date, you have to come back. So the advantage, you can probably tell just
from the way I'm describing it, the advantage of concurrent is it's typically faster. You can take
your existing employees. You don't have to wait six months or a year to find how they panned out.
Mike: Yeah, and your population is much more under control as well, because obviously,
if you're doing a predictive study, you're going to be testing all these applicants,
and you might not hire 75 percent of them. So you've actually collected data that you're not
then going to be able to apply later. And then I guess one so one advantage of the
predictive study is that you get the broadest range of people that in terms of knowledge,
skill, ability, and personal characteristics with the concurrent study, you have a little
bit of range restriction. Is that not correct? Clinton: Yeah, we can't, you can get some range
restriction because these individuals are current employees and we assume they're performing at an
adequate level or else they would not be current employees or they would no longer be employed
there. So that can be a downside is that range restriction is because there are scores on your
test. There's maybe not as much variability or differences in how they score. One of the,
one of the upsides of a predictive study is you are getting, like you said,
the range of responses and you can also get, you're getting that real
life situation from the job applicants. Clinton: They are really applying for a
job. So you're mimicking what you intend the test to be used for in a concurrent sample.
When you're giving it to current employees, they already have the job. And so they may
complete the test from a different frame of reference than your job applicants. And so
that can be a potential downside of a concurrent. Clinton: And so there are benefits and negatives
to both approaches, but both are equally allowed under the uniform guidelines. And
both have been shown from research perspective to, to basically result in creating tests
that are predictive of performance.. Jenny: So you're mentioning developing
tests or validating tests that are used for employee selection.
Jenny: Are they ever use it a post hire environment for
perhaps employee development or training? Clinton: They can be, but that's often what
you'll see, maybe with Myers Briggs or DISC or Strengths Finder some of those. And so
for the intended use, again, it comes back to that intended use. Depends on what
the test was designed to be used for. Clinton: Some tests have multiple,
have been designed to be used for multiple types of things. But this is something to be aware of.
What is your test designed to do? And sometimes I'll see this with clients who want to take a
test. And not necessarily a personality test, but let's say a technical skills test and they want
to use it in a diagnostic way to diagnose where someone is needs training or things like that.
Clinton: And I'm like, that test wasn't designed to do that. It gave you some initial information
to make a hiring decision, but it really wasn't designed to diagnose and tell you here's their
training needs. And so you just need to be aware of what your tests can and can't do.
Mike: There's a a maxim in HR as well, that says job performance trumps testing. One of
the things that happens is when a person goes from applicant to candidate, to employee is
that you start to have these instances where, job performance is being recorded.
And so very often we have people who will ask us, Oh, is it okay to give our employees
these pre-employment tests and then, if they can't pass them any longer to get rid of them.
Mike: And generally speaking, that's a really dangerous area to tread into because, you have a
much better job-related measure of what's going on, which is their performance on the job at
that point. So at any rate, it's just an aside. Clinton: That's that's a great point and kind of
shows in this, you can see that in this graph, tests are by no means perfect.
Clinton: Like you said, job performance trumps, trumps the test scores. Like in this line here,
you can see some of these blue dots, they're not on the line. So in some cases, the test,
a person may out predict what we outperform and what we predict. In some cases, a person
may underperform what we predict. Mike: And so just to be clear here,
any one of these dots is the intersection between an individual's
test score and their performance on the job. Mike: So that one that's in the middle
top there above the word test score. There's somebody who scored approximately 50, but rated
almost 60 on their performance measure, right? Clinton: Yeah, so in that case, they performed
better than we thought they would on the job. Mike: And then conversely, down below the
line to the left of the 40,
there's somebody who scored, is that right? Mike: Am I saying this right? Yeah. They under
tested and under performed, either way, right? Clinton: Yeah, but they were a lower test score,
but even then, the performance was lower than was anticipated.
Mike: Yeah. Thank you. Clinton: And so that's
something that the tests are, but the tests are not perfect measures, but what I always say is
it's, they're better than the alternative. Clinton: And so they are getting you,
they are increasing your odds. And so it's like when people sometimes will say this person is just
not a great test taker. I often will say there's also sometimes someone's great grandma who smoked
40 cigarettes a day and lived until she was 97. But you probably would not say in, in, in general,
the trend is if you smoke a lot, you're likely going to have some other health issues.
Mike: We're going to expect everybody who smokes several packs a day to live to a hundred. That's
not the, that's not the case. Another side of that's really important, at least I always try
to bring this up when I have this conversation with people, is that the alternative is that
a human being is making a selection based upon, in terms of the applicant, nothing.
Mike: You have selected somebody who will be hired and you have selected somebody who will
not be hired, for apparently no reason whatsoever, or no concrete reason. And
so that's another reason why to have this decision making process tied to something
that's observable that you can find patterns in. Mike: Or if you can find that there's disparities,
you can measure those things and see what exists and then remediate that situation.
Jenny: Yes, so if an employer would like to begin using a personality
assessment. How do they begin? Clinton: There's a, I guess there's
a couple of different ways. There are, you can, reach out to vendors out there.
Clinton: There are two different approaches. There are, is what I would call the build a
custom test from scratch approach, and there is the off the shelf approach. There are
existing personality tests that are off the shelf. And then there's the, Hey, we build
it custom for this particular job approach. Clinton: What you're looking for may depend on
your situation in your organization. If you're a smaller organization and, or it's a role in
your organization where you don't have that many employees. Off the shelf is probably going to be
the better route where you, a vendor has spent the time to develop a test for that type of job.
Clinton: And because you just don't have to do a sort of study where you can show this
mathematical relationship between test scores and job performance, that requires data. And if you
only have 10 employees, Iin a certain position in the organization, you don't have the data and
the necessary data to demonstrate a relationship and have any confidence in that.
Clinton: And so in that situation, you're going to need to rely on an off the shelf test where
they've done some of that research and can show how it predicts performance for that type of
job. If you have a large organization, you may be in the spot where you say,
let's custom develop this. If we have a few hundred employees in a position, you can do that.
Clinton: So those are the kind of the 2 approaches. And I'm sure there's,
I can get into like, how can you continue to want to get into how we do that once identified maybe
which one, the custom or the off the shelf. Jenny: Yeah. Yeah. I think you should. Yeah.
Clinton: So then when you get a customer or off the shelf,
there are different types out there of tests. Clinton: A lot of them, when it comes to
personalities, they're typically statements. When you take a personality test, you're often reading
statements. And I'll show here a couple examples. This is an example we'll call a forced choice
personality test, where you say, Hey, I'm more likely to, and this is a silly example
here, but eat a salad or eat a hamburger. Clinton: And you have some points in the
continuum. Sometimes you're just given two options. You just pick one or the other.
This point, you're given four choices on other personality tests. You're just given a statement
and you just slay like true or false, or that's like me or not like me. For example, I enjoy
team projects and you would say true or false. Clinton: Other times one of the most common still
is we'll call it like your type skills that strongly disagree to strongly agree. Again,
I enjoy team projects or strongly disagree or somewhat disagree or to the strongly agree. So
personalities generally are in that kind of flavor where they, you can see they
have statements and you're making some sort of agreement or disagreement with that statement.
Clinton: And if you're looking to implement it, so you need to go, if you're going to the vendor
route, go to the vendor and you should ask the vendor about what this test was designed for and
its intended use to make sure they built this for selection. And not only built it
for selection, but they have validity for your job type, going back to that accountant and NBA player
example to make sure they're not giving you a test that were designed to select NBA players
and you're trying to use it to hire accountants. Just what was this test? Where, what validity
evidence do they have for your job type? Mike: So as well, Clinton, there are many,
I may not be using the right word, but I would say standardized personality tests where the battery
like Hogan, for instance, they're going to ask the same 110 questions of every person, but over time,
they're going to do small studies with certain job titles or certain groups of people in order
to be able to take this standardized test and key it or score it in such a way that it doesn't have
meets with that particular population on the other side of the coin.
Mike: There can be a custom test, which wouldn't necessarily have standardized
questions. They might have questions that are written that have more face validity. They're
more written directly towards this particular job title that you're using the test for,
is that not correct? And do you have any recommendations about that, or any positives,
negatives about either of those scenarios? Clinton: Yeah, that is correct. Most of the,
a lot of the tests out there that measure what we call the big five personality
characteristics. And it, like you said, Mike, it's the same exact test for every position.
Clinton: The difference being what, where you need to fall on the characteristics. So let's
say openness to experience. You may, they may say on this job profile or job type we found that
people who are more likely to be successful are high in openness to experience. And another job
type people who are more likely to be successful are in the middle range
on openness to experience for the lower range. Clinton: And so they're given the same exact test,
but there's a different, like constellation of profiles of where you fall in these big five
personality characteristics that they're saying is indicative of success. And I don't know if
there's really like a, say this one is better than the other, but when you have a test that's
custom designed for a specific position, the good thing about that is that it was, like I said,
that it was custom tailored for that job type. Clinton: And so that, again, the frame of
reference, even the way the items are written from what I've seen, just in my personal experience
with work, I tend to see higher validity coefficients with those tests when design,
when they are well designed and put together. Higher validity coefficients than with the
general tests that are just Hey, we then custom create a profile depending on your job type,
but it's the same exact test for all jobs. Mike: Yeah. I think another aspect that's
very important to maybe particularly important now in this age of social media is
the applicant satisfaction part of it. When I was growing up and I had nothing
to do with this particular career, I took a lot of tests or did interviews or did
physical ability tests, those kinds of things. Mike: And so I've often reflected back on those
processes and thought about which ones were very satisfying to me. They had the face validity and
face validity is basically, how does it feel? Does it feel to the applicant? Like it would be
something that they might encounter on the job. So it doesn't mean the test is valid or not.
Mike: It just really has to do with my own impression of it. And I do remember
going back through in my mind, some of these experiences I had and I was like, Oh, that
was a valid test. That was not fair. That was a very unfair experience. And I think some of these
personality tests, where you're asking, would you rather be, at a a noisy party with loud music or
walking by yourself on a beach? Mike: What does that have to do with being in
ABC job? There's a lot of things that don't feel really good about that. Whereas if you're asking
it in context of the job, do you prefer an active work environment where your day goes by quickly
versus a quiet environment where it's maybe a little less stressful or those kinds of things
that has a lot better feel to it than the other. Clinton: That's a great point that face validity,
which is not what we always say. It's not like a technical type of validity, but it is real in the
sense that candidate perceptions of the experience matter. If candidates feel a positive experience,
they're less likely to, for example, to challenge a test or beat it up or complain about it.
Clinton: And that is one good thing of the custom develop test too is that they
tend to have a little more face validity because the questions by written with that
job in mind and so yeah that is one benefit another benefit of custom developed tests.
Mike: Perfect. Thank you. Jenny: So here's
what we're going to do. Jenny: We're going to take
a short break and then we'll be right back with everyone in just a moment.
Voiceover: And we'll be right back after a word from our sponsor.
Ready to revolutionize your HR strategies? Head over to TestGenius.com to discover our
latest tools and solutions designed to streamline your hiring processes.
Jenny: Welcome back everybody. And just a reminder, we're with Dr. Clinton Kelly from
IoPredict, and we're talking about personality testing. So when someone takes a personality
test or a personality assessment, a score is spit out for that person, correct? Can you talk
a little bit about personality test scoring? Clinton: Yeah. So when it comes to the scores,
there are different scores that are done and those, when we talk about those off the shelf
tests that measure the big five personality characteristics. You'll often get a scale, a
score in each of those big five personality, what we call sub skills. You'll get like an
openness to experience score, an extroversion score, conscientiousness score, and then you'll
often get an overall fit recommendation. Clinton: And this is something that's on
the screen. This is for a custom developed tests that Biddle and TestGenius, where you give like
an overall recommendation. And you can see here on the screen here, in this case, we say, Hey,
people are highly recommended. They have a 77 percent chance of being successful on the job.
Clinton: And this is based on job performance data collected from actual individuals. You can see
the numbers here, the number of individuals. And so what we give on the score, we give a,
usually give an overall recommendation score. Most tests you'll see like some sort of overall score.
And then you'll see some sort of subscale scores. Clinton: The overall score is typically what is
going to be the driving factor in whatever decisions you are making or informing the
decision. What you don't want to do, and is tempting for many people to do,
who are not super familiar with personality tests, is take subscale
scores. And let that drive your decision. Clinton: Because the subscales are a sub
component. They are sub, like it says, they're sub to the overall score. So less data points
from the test are driving subscale scores 'cause it's just a portion of the overall test. And
so it really should be that overall score or profile score you get from a personality that
should drive your recommendation and those sub skills can be informative pieces of information
to then maybe dive into maybe in an interview or another step of the selection process. But don't
let a subscale score. I've seen this happen where someone says there's some overall profiles like
we highly recommend them based on the overall profile, but they'll see a subscale score where
they're fairly low on some subscales I don't know if you should hire this person - I'm like,
no, don't let that drive the decision. Mike: That's a really important point
that you brought up about the subscale is that you can use it to drive questions in
the interview process. In fact, very often the reports I know the reports that are in
the test that we've created together, you've included those drill down questions because
that gives you a really great opportunity to, a find out, did the test potentially
really nail this subscale or did it maybe miss the subscale or furthermore is this weakness,
maybe something that the candidate is familiar with and has the ability to work around it, which
can turn a weakness into a strength. So is that always something that's included with personality
tests for selection or are we just lucky? Clinton: Yeah, not always. So Yeah,
they're not always included, so that is, yeah, so there are some where they just give you an overall
score and and I've even seen some that just give you an overall score, there are even those sub,
sub scores, and then, but yeah, so it is just, it is nice when you get more information like that,
there are a number of tests out there that will provide sub scores and even
some additional information or potential prodding questions, but yeah, it is,
like I said, it's great when you can have that additional information to prod into an interview.
Clinton: Yep. But like Mike had said, just don't let us, don't let a subscale score drive
your overall decision. It's like letting, like one quarter, one quarter of performance in a business
drive your overall decision. If you don't look at the yearly profit and loss, you're like, no,
over a year we made a lot of money.But one quarter one, it was slow. And now you say,
Hey, we've got to close the company because things are horrible. It's no,
don't let the subscale score, the overall score. Mike: I want to tell a quick story because
this was something that was so impactful about 10 years or so ago.
Mike: We created a testing process for nurses and it included knowledge testing,
video situational judgment testing, and personality testing. And one of our.
Clients told us about a instance where they were reviewing the results with the applicant in real
time. So they had tested and then they moved them onto an interview process.
Mike: And the experience level of this applicant was very good. The knowledge was very good. The
situational judgment was good, but there was this one subscale, which was hostility. And
there was a red flag on the hostility scale. And the interviewer asked the first prodding question,
Hey, so tell me about a time, and it was some question that had to do with when they were in
a stressful situation, how they didn't handle it. Mike: And then they asked the second prodding
question and then started, started to feel that this was exposing something,
asked the third prodding question, and at this point, the candidate slammed her hand down on
the desk, said, I've been to anger management classes. I've worked through this. If you
would just move on, I would appreciate it. Mike: There was a great instance where,
you know, that this did bring up something that should have been discussed
and was discussed and herein is the real value in collecting that data and having
an opportunity to talk about it because, boy, when you talk about dodging a bullet,
that was an important one right there. Clinton: And that is a great example of how to
use that. In some cases, those subscale scores, it may be you may find something like you said that's
yeah, this is a red flag, but more, more often than not, I'll see individuals without prodding,
like I said, without doing those follow up questions, take a subscale and automatically maybe
disqualify someone because of a subscale score. Mike: And we talked a little bit about how you
use it for selection. There's a, we do a lot of work in the 911 space, and I know we're going
to do a follow up podcast where we talk a little bit more about that. But there's a, there's fairly
popular personality tests in that space that are content based validated, which
the Uniform Guideline specifically says, don't content validate a personality
test.You want a criterion validate it. Mike: So let's talk, why don't you talk
a little bit about the defensibility aspect? I think that the stuff we've talked about up
to this point is really the utility aspect of the validation. Here's why you go through this
process so that it will do a good. Good job of selecting, but why do you go through this process
to make sure that if somebody, an applicant or the DOL or the DOJ or the EEOC or some plaintiff's
attorney has a problem with something that you did, what, why does this, how does this come
into play and how does this help somebody ease out of an uncomfortable defensibility situation?
Clinton: Yeah, great question. When it comes to those situations, if you have a test that,
for example, is just content valid or you don't have the proper validation
in place for a personality test to show that it's predictive of performance,
if you end up passing, let's say your personality test ends up passing more males than females or
more females than males or in one or just more of one protected group over another, that's
when the Uniform Guidelines kick into place. Clinton: The Uniform Guidelines say, Hey, if your
test screens out a disproportionate amount of one protected group status, you need to show that your
test is a valid test and that it's screening those people out because they can't do the job. And it's
not screening them out just because they're female or they're Hispanic or whatever group that may be.
Clinton: And so if you are using a test that doesn't have the proper validation in place,
and let's say you're screening out a disproportionate number of some protected group
status, you will lose in court if you get sued. Mike: Yeah. Good point. That's good advice. It
doesn't happen very often, but it does happen that a process gets questioned and it can really
financially impact an organization in terms of legal decisions that might go towards the plaintiff,
but also it hurts their reputation in the space.
You don't want to be that group that got sued because they did something incorrectly.
Clinton: Yeah. So if you're a vendor out there and you're not doing your due diligence, yeah, it can really hurt your reputation. Clinton: And that's what I tell with clients and said, yeah, you can, we can put out a test
that doesn't have the proper kind of validity to back it up but I don't want to do that.
Cause that's going to, that's going to hurt my reputation as an individual in the field.
Mike: Yeah. Yeah. And so you and I know, and a lot of people may or may not know that,
when criterion validity is established elsewhere, the Uniform Guidelines in 7B does allow for
validity to be transported from the environment in which it was established over to an employer's
own specific environment. Mike: Why don't you talk a little bit
about what that means? Processes and any recommendations that you have.
Clinton: Yeah. And that is a really great strategy for jobs,
especially where you just don't have the sample size to conduct these criteria types of criteria
and validation studies in your own organization. Clinton: And so the Uniform guidelines allow us,
let's say Mike, cause we've done, we'll talk about this in a later one, but we've worked with
the 911 space with personality testing. And a lot of 911 dispatching agencies,
they just don't have hundreds of dispatchers. They have maybe a handful. There's a lot
of smaller jurisdictions out there. Clinton: So if they want to implement
a personality test and say, Hey, what profile or what type of personality is more likely to
be successful in this job? They just logistically don't have the numbers to
do such a study. And so what 7b, section 7b of the Uniform Guidelines allows is it says, hey,
if a validation study for let's say 911 dispatchers on the personalities has been
conducted with a specific test, you as a smaller group, you can adopt this test and not
have to do your own criterion validation study. Clinton: If you can show that your dispatchers,
your 911 dispatchers perform substantially the same major work behaviors as those 911 dispatchers
where this study was originally conducted. So basically you have to show, hey, there's a match,
there's overlap between my job where I want to use this test and the job where they, what that they
used to initially develop and validate this test. Clinton: And if you can show that the two jobs
are substantially similar in the tasks and the work behaviors they perform,
you can transport that validity under Section 7B of the Uniform
Guidelines. And that fulfills their validation requirements.
Mike: My understanding is that when you're looking at these work behaviors that were
covered under the initial study, that's the work behaviors that you're looking
at in terms of transporting this validity. Mike: There may be some other work behaviors
that weren't covered under that validation study that aren't really a part
of this conversation. But in terms of being able to transport the validity
from this original study and bringing it over to your environment, is it not those particular
performance dimensions that were purported to be measured or in that original study by that test?
Clinton: yeah, you do want to look at those. So those original things that are measuring
that test are those work behaviors also important in your job? And and you say,
that's the critical part is showing that overlap. And it's not, it is not super difficult to do it
is as long as I say as long as in the original study, the vendor did their due diligence and how
they showed what the work behaviors were in that original study. So if they've done their homework,
it's a very easy process. I have seen this in the past where a client that I was working with
was wanting to use a test. Clinton: And they had some
criteria and validation, but they weren't able to give me information on what the people in those
jobs did or what exactly they did in that original study. And so it was,
we couldn't really transport the validity because they didn't do their homework on that initial
study. And so that's why it's important. Clinton: When you do that initial study,
the vendors should have done a good thorough job analysis of analyzing
the job. So you can make sure there's a match between where they did the study and
your job where you want to implement the test. Mike: That's a really great point. And we might
want to toot our horns a little bit right here. Mike: But, being a shop, both of our shops
are shops where we lean very heavily on the Uniform Guidelines and, the principles and
the legislation and, laws that have been put into place so that we are doing things right
to begin with so that if somebody does determine that they'd like to transport that validity over
to their environment, they're able to do that. Mike: But you do really want to do your research
or as much of the research as you can to make sure that you are partnering up with an organization
that has the same values and is doing things right to begin with. Otherwise it's the
garbage in garbage out philosophy, right? If you start to transport over a validation study,
that's a bad validation study.Then, you're protected by nothing, right?
Clinton: Yeah. And so that's one thing. If you're looking to implement a study is ask that vendor
to say, Hey, do you have a validation study? They should have a technical report. If you're asking
a vendor who's got a personality test and you ask questions about validation or do you have a
technical report that documents things and they start to kind of him hot or dodge you or they
send you like a one page more a marketing flyer. Clinton: Those would be red flags in my opinion
to say, this person may not have done their homework. You should be hearing things like
Uniform Guidelines, like criterion validation. There's certain terms,
keywords that the vendor should be mentioning. Mike: Very good. We've mentioned Uniform
Guidelines a lot of times. Mike: So we actually have a
website set up. That's uniformguidelines.com where the Uniform Guidelines are in there.
Topically by heading hyperlinked. There's also the Questions and Answers to the Uniform Guidelines,
which came what, 10 or 20 years later, right? Or I don't remember how many years, but that was,
they circled back through and said, okay, we've got this document, looking back now upon it,
what are different scenarios that have been encountered and how did this get interpreted?
Mike: So both the Uniform Guidelines and the questions and answers are very valuable in
terms of knowing what the best practices are for testing and selection. So uniform guidelines.com,
you can go there and you can, take a look at it, bookmark it and refer to it anytime that you'd
like, that's a resource that we sponsor. Mike: Jenny, what else?
Jenny: I actually think we're getting to a point where we're going to start to wind down. Okay. And
just want to let everybody know this is part one or episode, session one with Dr. Kelly.
And next time when we meet, we're going to be talking about, specifically about personality
testing in our CritiCall application. Mike: Okay. Which would be the dispatch realm.
Jenny: 911 emergency dispatchers. So let me do this. Let me change the screen here,
bring us all back. And as we wind down Clinton, if someone wants to get in touch with you,
how do they do that? Clinton: Yeah. So you
can reach out through our website at iopredict.com Clinton: And my email is just CKelly. So my name,
first initial, last name, CKelly at ioPredict.com. You can also reach out to Mike over at TestGenius
and Biddle. And like I said, we have a great relationship and we work together hand on hand
and hand on a lot of validation projects. And so those are some ways to get in touch.
Mike: And you and Jason still are maintained at Ckelly at biddle.com as well where we work
together very closely and we continue to really value this relationship and want to
continue to be able to collaborate together for many years to come. It's been a,
it's been really great. And I for one, would like to thank you for being on with us today.
Mike: , I think we maybe even went a little bit longer than we had targeted. But it's been really
valuable hearing from your experience regarding this subject, which is, can be a little nebulous,
it can be a little scary to tiptoe into. So I've learned a lot and thank you so
much for your time today. Clinton: Yeah. Thank you
for having me. I really I enjoyed it. Jenny: Oh, it's been great. And we do have
show notes that will show on the page where you see this video. We'll have links to IoPredict's
website to uniform guidelines.com and other resources that were mentioned in this video. So
thanks to everyone for listening and for watching. Mike: Great. Thank you.
Thanks for tuning in to Testing, Testing 123 brought to you by TestGenius and
Biddle Consulting Group. Visit our website at testgenius.com for more information.