In the previous three articles in this series on the search for the perfect candidate, I made the case that three fundamental changes were required to hire more top performers:
- Redefine perfection (Part 1). A performance profile describes what a person taking the job must do to be successful. It summarizes the challenges and performance objectives of the position. Traditional skills-based job descriptions should be eliminated for hiring purposes ó since they exclude top people with different backgrounds who could do the work, and they are uninteresting to those top people with all of the skills.
- Use market-driven sourcing processes (Part 2). Finding top people should be based on how top people look for new careers, not on how average people look for jobs. This requires compelling online advertising, proactive employee referral programs and aggressive networking.
- Redo the interviewing, assessment and selection process (Part 3). Use a consultative interview process that is based on how top people gather information and make decisions. Part of this requires the use of the two-question performance-based interview process we recommend, plus a new technique to share information before assessing competency.
Collectively, these changes can profoundly improve the effectiveness of a company’s recruiting and selection methods. We have not found one situation where hiring results have not improved at least 25%-50% as a result of implementing performance-based hiring as described. In this article, I will present the theoretical validation for the process recommended. In 1998, an article by Frank Schmidt of the University of Iowa and John Hunter of Michigan State University appeared in the Psychological Bulletin (Volume 124, No. 2). The title pretty much tells it all: “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings.”
The authors examine 19 different selection procedures for predicting job performance and the impact of combining different processes together. Their summary conclusions are that the use of a test measuring general mental ability — in combination with a structured interview, a work sample test, an integrity test and a structured interview — dramatically improves the results of the job selection process. While the article would not be considered “easy reading,” there are some critical issues presented that impact how these tools should be used. Following is a table which summarizes the predictive validity of each selection method, plus the increase in using any one of the techniques in combination with a test for general mental ability (GMA).
| Selection method | Improvement on a coin flip (50/50) | Increase in combination with GMA test | Comments |
| GMA tests | 12% ? 62% | Alone 62% | General mental ability |
| Work sample tests | 14.5% ? 64.5% | 20% ?70% | Used to test skills |
| Integrity tests | 8.5% ?58.5% | 21% ? 71% | Measures counterproductive behaviors |
| Conscientiousness tests | 5% ? 55% | 18% ? 68% | Measures dependability |
| Employment interviews (structured) | 12% ? 62% | 20% ?70% | Pre-planned based on job and formal scorings |
| Employment interviews (unstructured) | 7% ? 57% | 15% ? 65% | No plan, any questions, no ratings |
| Job knowledge tests | 11.5% ?61.5% | 17% ? 67% | Requires job knowledge |
| Job tryout procedure | 9.5% ? 59.%5 | 17% ? 67% | Okay for entry-level positions |
| Peer ratings | 12% ? 62% | 17% ? 67% | Acceptable for internal moves |
| T & E behavioral consistency method | 10% ? 60% | 17% ? 67% | Compare job needs to past performance |
| Reference checks | 3.5% ? 53.5% | 16% ? 66% | Less useful in current legal environment |
| Job experience (years) | 1.5% ? 51.5% | 14.5% ? 64.5% | Has less value with more than five years of experience |
| Biographical data measures | 6% ? 56% | 13.5 ? 63.5% | Captured in GMA |
| Assessment centers | 7% ? 57% | 14% ? 64% | Costly and not much better than GMA |
| T & E point method | .5% ? 50.5% | 13.5% ? 63.5% | Not based on performance, too arbitrary |
| Years of education | 0.5% ? 50.5% | 13.5% ? 63.5% | Captured in GMA tests |
| Interests | .05% ? 50% | 13.5% ? 63 | No correlation whatsoever |
| Graphology | 0% ? 50% | 12% ? 62% | Handwriting is useless |
| Age | 0% ? 50% | 12% ? 62% | Slightly negative correlation |
This table summarizes how well each of the selection techniques predicts on-the-job performance. Column two shows this in comparison to a 50/50 coin flip. For example, a GMA (general mental ability) test improves random 50/50 odds to 62%. An unstructured interview by itself is 7% better than a coin flip, or 57%, and a structured interview by itself is 12% better than a coin flip, or 62%. Age, education, handwriting, and job experience don’t provide much predictive value. Other than the handwriting part, this is somewhat of a surprise. The point of the article is to demonstrate that using multiple selection tools, the odds of predicting on-the-job success increases. However, the results are not additive, since some of the impact of using one tool could overlap with another.
To address this, the authors used a statistical technique referred to as meta-analysis. This is shown in column two, where the results of each selection process were combined with the results of the GMA test. This yielded a combined predictive value. The biggest improvements were in areas using the structured interview, integrity tests, reference checks and job related tests. Using any one of these selections methods in combination with a GMA test improved the predictability of a new hire into the 65-67% range. Two other key points were made in the article which suggest that behavioral interviewing is not necessarily the ideal interviewing tool:
- Past performance is the best predictor of future performance. The authors clearly state this as the generally accepted psychological principle ó not “past behavior is the best predictor of future behavior,” as commonly stated. Behaviors and performance are not equivalent. A behavior is very situational. For example, a person can evidence a drive for results as a competency associated with a great supervisor, or as a project that clearly is self-motivating. The same person can be a unmotivated if the job is uninteresting, or if the boss is a weak leader. The failure to tie the behavior to a specific and comparable situation required for job success could result in under-performance if the person is assigned work that holds little interest.
- The behavioral consistency method compares a candidate’s past accomplishments to those required for job success. In this case, behaviors and competencies are tied to actual job needs. The authors clearly state that the examples of past performance chosen for comparison need only to be similar, not identical. This makes the process appropriate for less experienced people, as well as for more experienced people with different experiences.
The Schmidt and Hunter study didn’t determine which were the best overall suite of tools to use for selecting new employees. I’ll leave the actual meta-analysis to determine this to Charles Handler and Wendell Williams, but here’s my common-sense suggestion as to which five selection tools should be used:
- GMA test. Predictive value: 62% (based on the research). We recommend the Profiles International test be used for the short list of finalists. Using any type of test up front can be counter-productive, since the best people might opt out.
- Any type of structured interview. Predictive value in combination with a GMA test: 70% (based on research). My recommended two-question performance-based interview is a simply structured interview that directly links behaviors with performance.
- The T&E behavioral consistency method to conduct job analysis. This is very similar to the process I suggest using to prepare a performance profile. By itself, this improves predictive value by 10% from a 50/50 coin flip. The increase in predictive value in combination with the above two selection tools is estimated to be at least five to seven points, for a total predictive value of 75-77%.
- An integrity test. This test in combination with a GMA test increases the overall predictive value of the two tests alone to 71%, the highest combination on the list. Since the test measures bad behaviors, it allows the employer to avoid hiring people who could be disruptive. By itself, it improves the 50/50 predictive value by 8.5 points. It is estimated that its use would increase the predictive value of the three tools mentioned above by 5-6 points. Collectively, this would increase the total predictive value of these four selection tools into the 80-83% range.
- Some test of job knowledge. Job knowledge and skills tests by themselves add 10-12% to the predictive value of the overall assessment. Since some of this benefit is probably picked up in the behavioral consistency method and the GMA test, adding half of the value is probably high, but a 3-4 point improvement seems appropriate. This would increase the combined predictive value of using this suite of five selection tools into the 83-87% range. Collectively, this is remarkable.
The Schmidt and Hunter study is of invaluable importance to anyone involved in deciding how to make better hiring decisions. Most companies use some ad hoc approach which roughly cover the five recommended techniques. Few companies, however, modify them appropriately to meet the needs of top people. This is where performance-based hiring has a distinct advantage. Not only it is theoretically sound; equally important, it was developed to address the unique needs of the perfect candidate.