My last article, Part 1 of this series, outlined a few basics on personality traits directly related to job fit and job performance. Now we’ll move along to interpreting scores and identifying patterns. Setting Cut-Off Points Computing an average of high-producer scores might seem like a good idea, but it is a sure sign of a vendor who does not know how to validate a test. The whole purpose of test validation is to determine whether test scores predict job performance — NOT whether job performance predicts test scores. Test validation is like gravity: you can argue all you want against it, but that does not change the fact that your body is firmly stuck to the earth. There are several reasons why “producer averaging” is a poor practice:
- Test users need to identify significant differences between high and low producer scores, NOT just high producing averages.
- Because different AIMs are usually associated with different tasks, test users need to know what kind of “production” is being averaged (e.g., cold calls, satisfaction surveys, new dollar volume, repeat dollar volume, and so forth).
- Individual high producers usually do not meet their own group average, making the value of averages questionable at best.
- You need to work from causal data, not correlation data.
- Test “targets” should be based on job analysis, not data-fishing. If you don’t know how to do this, don’t use pre-employment tests.
If these reasons are not enough, self-reported tests are highly inaccurate. They are good at indicating likes, dislikes, ambivalences, and patterns, but they are not good at indicating whether a score of 55 produces better employees than a 40. Personality-test precision is loosey-goosey. But these test do have a purpose. Separating the Forest From the Trees Users can learn much more about candidates by examining response patterns. Here are some actual examples. Before we do anything, we need to discuss with job experts (managers and job holders) what AIMS levels are needed for the specific job. We’ll use a sales job for this example. These are the desired AIMs for the example:
- High scores desired for: Attitude toward work (likes working)
- Mid-range scores desired for: Self-centeredness (salespeople tend to be ego-centric), problem solving (every sales issue presents a moderate problem), idea generation/innovation (innovative, but not excessive), administration (follows most rules), expressiveness (outgoing, not overwhelming), impulsiveness (takes action, not pushy), perfectionism (cares about quality, not obsessive)
- Low scores acceptable for: Teamwork (work with others, but not co-dependent), resistance to change (tolerates frequent changes)
Candidate 1: Co-Dependent Conrad Conrad’s AIM Patterns: High scores: Administration 90, teamwork 72 (Translation: Conrad likes rules and being a team member.) Mid-range scores: Attitude 56 (Translation: Conrad thinks work is okay.) Low scores: Problem solving 38, self-centeredness 25, idea generation/innovation 17, resistance to change 11, expressiveness 12, impulsiveness 5, perfectionism 8 (Translation: Conrad dislikes solving problems or generating new ideas; he is easygoing but not very outgoing; he avoids making decisions, and he is not concerned with being accurate.) Truthfulness score: 8 (Scores are below the norm; be cautious.) Candidate 2: Anti-Social Annie Annie’s AIM Patterns: High scores: Attitude 90, idea generation/innovation 75, administration 76 (Translation: Annie likes working and generating new ideas, but also likes following rules.) Mid-range scores: Self-centeredness 59, impulsiveness 50, resistance to change 56 (Translation: Anne is very self-centered, but about average when it comes to being proactive or tolerating change.) Low scores: Expressiveness 38, teamwork 28, perfectionism 26, problem solving 11 (Translation: Anne is not very social and keeps to herself. She also dislikes solving problems and doesn’t care very much about being accurate.) Truthfulness score: 28 (Her scores are low, but within the norm.) Candidate 3: Faker Frank Frank’s AIM Patterns High scores: Problem solving 100, idea generation/innovation 100, administration 100, self-centeredness 100, attitude 100, expressiveness 77 (Translation: Frank loves solving problems and generating new ideas as much as he loves following rules and doing a good job. He is extremely outgoing, exceptionally self-centered, and either totally conflicted or lied on the test.) Mid-range scores: Impulsiveness 45 (Translation: Frank is about average.) Low scores: Resistance to change 25, teamwork 8, perfectionism 5 (Translation: Frank is easy going, doesn’t get very near to people, and could care less about quality.) Truthfulness score: 8 (His scores are low; be cautious.) Candidate 4: Peter Possible Peter’s AIM Patterns: High scores: Attitude 90, idea generation/innovation 75, administration 76 (Translation: Peter loves working, generating new ideas, and following rules.) Mid-range scores: Resistance to change 56, self-centeredness 59, impulsiveness 50 (Translation: Peter tolerates change, but he’ll be self-centered and average pushy.) Low scores: Expressiveness 38, perfectionism 26, problem solving 11, teamwork 8 (Translation: Peter is not very social and like to work alone, dislikes solving problems, and doesn’t care very much about quality.) Truthfulness score: 28 (Scores are low, but within the norm.) Peter Possible came pretty close to the job target, but he may need some supervision regarding quality and judgment. In addition, he might not be the most outgoing person in the room. Comments AIMS can be a valuable pre-hire resource. They often tell you much more about a candidate than an interview. However, you must follow a few guidelines:
- Set your target by interviewing people who do the job.
- AIMS must “cause” performance, not vice versa.
- AIMS are always self-reported. Always check the lie-scale.
- Only a few AIMS are related to job performance.
- Carefully examine patterns that show future behavior.
- Individual scores indicate trends, not absolute truth.
- Small AIMS-score differences are insignificant.
- Confirm patterns using behavioral-type interviews or reference checking.
- Good AIMS tests are developed expressly to predict performance.
- AIMS and skills are two different things.
In the next part of this article series, I’ll contrast the discussion above with how AIMs operate for call-center positions.