As a talent acquisition professional, you’re probably at least somewhat aware of pre-hire assessments, especially given that 82% of companies use them in their hiring process. Few, however, are using them in the right ways. That is, they are not truly investing in comprehensive assessment strategies based on best practices. Reasons include:
- Lack of support from the top of the organization
- Siloed talent practices, with little connection between nodes in the employee lifecycle
- Lack of awareness around assessment benefits
- Lack of expertise to guide development and execution of strategic testing programs
- Perceptions that testing is too complicated or can lead to legal issues
It doesn’t have to be this way. This is where football comes in.
Yes, Football
I recently attended the SIOP Leading Edge Consortium, a conference focusing on advanced personnel-selection techniques, where I saw some amazing presentations by I/O (industrial/organizational) psychologists involved in high-stakes hiring programs. Among the best was a presentation about a high-performance talent-assessment program created by the NFL.
Poor decisions on draft day can cost tens of millions and result in lost opportunities. In the past, player evaluation consisted of highly subjective judgments that relied on appearance (i.e., tattoos, piercings, hair styles, etc.) as a signal of character. While a simple cognitive test (the Wonderlic) was also part of the equation, the overall evaluation process was still off-target.
In 2013, a team of I/Os from Baruch College collaborated with the NFL to create a comprehensive player assessment tool (PAT) as a supplement to physical measures. The decision to develop the PAT was the direct result of a strategic initiative to use best practices from business to “elevate the game.” Here’s how the NFL used a proven four-step process to make that happen:
Step 1: Define the Performance Domain
Success starts with understanding a job’s strategic value to the organization and then identifying and defining a set of traits or characteristics associated with desired outcomes. This typically involves a job analysis in which subject-matter experts assist in the development and verification of a success profile or competency model. The results of this step provide a blueprint of what success looks like and how it will be measured.
For the NFL, the process of developing the PAT started at the top with a great deal of work by organizational leaders to shape the program policy. A rigorous job analysis started with in-depth interviews with coaches and general managers, but it didn’t stop there. Analysts spent hundreds of hours watching practices, interviewing players, and observing on the sidelines during games. The result was a clear definition of how to measure performance both on the field (like number of penalties and snaps) and off the field (jersey sales, negative tweets, community involvement, etc.).
The analysis also yielded a detailed model of required performance traits that was based on four dimensions:
- Intelligence: Complex reasoning, direct thinking, visual processing, mental speed, ability to learn, mental error resistance
- Personality: Team focus, dependability, mental toughness, grit, decisions under stress, preparedness, leadership, self-control, emotional stability, stress tolerance
- Needs/Motives: Need to compete, need to play, need to achieve, learning style
- Learning Strategies: Preferred learning context, performance orientation, learning motivation, confidence
Step 2: Measure the Human Elements That Drive Performance
This step is all about finding or developing psychometric assessments that systematically and objectively measure the traits that drive job performance. By completing these assessments, job applicants are measured against a standard of success that ensures apples-to-apples comparisons.
The team of I/Os developing the PAT followed a mandate that the test measure elements of performance specific to football. The result was an hour-long assessment that measured both personality and cognitive dimensions using standard types of test items, like multiple-choice questions, as well as innovative measures like dynamic motion tests that require predicting the trajectory of objects in motion.
Step 3: Use Results to Inform Decision-Making
Any assessment requires buy-in from those using results to make hiring decisions — the best test in the world is no good if people ignore or inconsistently apply results. In other words, people must have trust in the assessment and make a disciplined investment in using the results as key inputs when making hiring decisions.
The NFL made sure that the assessment process became an essential part of draft evaluations. All players who attend the NFL Scouting Combine are required to take the PAT, and most of the league’s 32 teams use the evaluation to inform draft decisions. Additionally, player scores are made available to all teams and are included in the information used by general managers and coaches to evaluate talent.
Step 4: Evaluate the Business Impact of Your Decisions
The true value of selection decisions using assessments can only be demonstrated via a validation process that evaluates the relationship between test scores and job performance.
Validation work on the PAT showed clear and strong correlations between test scores and many different player performance metrics. Ongoing analysis has further shown that players scoring in the top band were more than twice as likely as those in the bottom band to perform successfully. Executives also rated top-band players as having more than twice the value to their team as those in the bottom band. What’s more, the overall success of the PAT has led the NFL to invest in expanding the program.
Beyond the NFL
The tried-and-true methods used to develop the PAT aren’t just for the NFL. You can use them in your organization, too. Creating an effective talent assessment is the same whether you’re hiring baristas or quarterbacks.
Want to learn about the NFL’s testing program? Listen to my podcast interview with the team of I/Os who actually developed the PAT.