Got Tests?

Does your organization use mental alertness tests? You know, the kind that ask people to solve problems like, “If one ‘innie’ belly-button holds .03 grams of shirt fuzz, and one ‘outie’ belly-button holds .001 grams of shirt-fuzz, how much fuzz would seven innies and three outies collect in a week?” Although belly-button fuzz is probably not related to job success, mental alertness test scores frequently are. In fact, they are among the strongest predictors of job performance. It’s no mystery why this is so. We know that high producers are smart enough to do the job. And we know that dull people tend to be low producers. So if we test candidates, the smarter ones will probably do better, right? Well, maybe. The Three Bears Conundrum Although all good producers are smart enough for the job; not all smart people are good producers. For one thing, job intelligence tends to follow a bell-curve. Remember the story of the three bears? Bear #1 says the porridge is too hot; Bear #2 says the porridge is too cold; and Bear #3 says the porridge is just right. The same holds true of intelligence. Suppose, on a 1-to-10 scale, a job requires an intelligence level of 5. This level is necessary because there are problems to be solved and decisions to be made ó not difficult ones, but ones that require at least an average level of smarts. What would be the consequences of hiring someone who scored a 3? Yes, more mistakes. This might mean increased scrap, too many returns, too many lost customers, or wasted training dollars. The bottom line? Hiring people who are not bright enough for the job would threaten the viability of the business. Now, what would be the consequences of hiring only people who scored 7 or above? Would production go up? The answer is probably not! Why? People who are “too smart” tend to get bored, turn over quickly, expect faster promotions, or behave badly. Choosing only high-scoring applicants might seem like a good idea, but experience shows otherwise. Organizations frequently make this mistake when they hire MBAs to do ordinary tasks or insist on hiring people with college degrees to do general office work. What about “throwing the dice” and not testing anyone’s brains? Well, in that case, we would get a mixed bag ó some will be too cold (“Huh?”), some too hot (“When’s my next promotion?”), and some just right (“I’m really good at my job!”). Pass, Fail, or Who Knows? Intelligence often must be part of a good hiring system, but first we have to prove that intelligence is important to job performance. That may sound like a simple process, but it’s not. It takes a study of the job, the jobholders, the managers, and an anticipation of what the future job holds. If “smarts” are deemed necessary, it’s time to pick the right kind of test and set reasonable cut points that do not adversely affect protected groups. One way to do this is to test everyone, determine the mean score, and choose a cut-off point that is one standard deviation below the mean (remember, any hiring test must be based on job requirements and business necessity). Another way to set a cut-off score is to run a statistical correlation between test scores and job performance. Unfortunately, since not all job performance is related to job “smarts”, we will have to be careful which performance standards are used in our analysis ó otherwise we end up with GIGO (garbage in, garbage out). Flawed Logic and Testing Validation testing is not for the faint of heart. It’s deeper than it looks. For example, only wrong-headed people give tests to only high performers. Attempting to take group summary data and apply it to individuals is flawed logic. There is even a name for it ó “Aristotelian logic,” or the tendency to assign people to over-simplified groups. For example, after we divided a group of applicants into tall and short, we would observe more men in the tall group. Aristotelian logic would argue if an applicant was tall, he must be male. Galileo (the guy who discovered the sun) showed this kind of logic to be a product of teensy-weensy thinking. In our example, Galileo showed that only if we can prove the applicant is not tall can we say the applicant probably a member of the short group. Confused? You can read more by doing a web search, but for the purpose of this article, Aristotle didn’t know squat about validation. No cookies and milk for him. What does this mean for hiring tests? Group data cannot be used to validate individual performance, and high test scores do not predict high performance with the same accuracy that low scores predict low performance. Most tests that measure mental ability have some degree of overlap, but there are different kinds of mental alertness tests: the belly-button lint variety; situational judgment tests; case studies, abstract numerical and verbal, construct valid; criterion valid, and content valid, to name a few. Mental alertness tests come in every shape and form. The best ones parallel the kinds of decisions required of the job. Adverse Impact? Don’t forget adverse impact, or the practice of comparing an average score from one demographic group with another. As of today, adverse impact includes comparing races, ages, gender, religions, and disability (I expect facial hair, shoe size, and the politically incorrect to be added soon to the list). Anyway, if a disproportionate number of “protected” people are not hired (or promoted), the organization may be asked to show proof of job requirements, business necessity, and validation. In other words, they will be asked to show their placement decisions are actually based on evaluating job skills. Go figure! “Do mental alertness tests have adverse impact?” you might ask. The answer is, yes, mental alertness tests almost always have major adverse impact. Is adverse impact legal? That depends. It’s okay ó if your test follows the Uniform Guidelines on Employee Selection Procedures. It’s not okay if your test is arbitrary and undocumented. Corporate Legal to the Rescue? Probably Not! Law is like most fields: it is so large that people tend to specialize. Lawyers in the corporate lobby are generally contract specialists, not labor lawyers. In almost every meeting I attend where legal folks are invited, a few will quietly approach me after the meeting asking for a copy of the EEOC Guidelines “for their files.” If recruiting sees its job as sourcing, line management sees its job as managing, and corporate legal sees its job as contract review, who is in charge of the hiring system? One final note: Because they have to deal with complex issues and make difficult decisions, managers and professionals almost always need more “smarts” than the average employee. If an organization refuses to use mental alertness tests because of litigation fears, then there will always be fewer people with management potential in the pool. The inevitable results are shallow internal promotion opportunities and impaired career succession planning.