As we gear up for the upcoming NFL draft this weekend, teams (and their fans) are studying, analyzing, prognosticating, and deciding which talent they should hire to help them achieve their goals.
In my preparations for the draft I recently ran across a really great article by Field Yates about the role of cognitive testing as one of the many pieces of predictive data used to help teams make player personnel decisions.
As a rabid fan of both testing and football and a self-proclaimed expert in each, I feel we can all learn a lot from the situation discussed in this article.
The article uses the controversy around the fact that Morris Claiborne, a cornerback who is projected to be a high first-round pick, did very poorly on the Wonderlic Exam (a cognitive ability test that all draft entrees are required to complete). The author suggests that Claiborne’s seemingly impossibly low score may actually be the result of a learning disability that will likely not hamper his on-field performance and uses this story as a lead-in to discuss the issue of using tests as a surrogate predictor of on-field performance.
I really enjoyed the author’s take on the importance of testing as a predictor of performance.
Just like a near-perfect score doesn’t equate to guaranteed success, a far-from-perfect score does not signal impending failure.
The point is — and this is what has been lost in the recent Claiborne headlines — the Wonderlic exam will always be a part of the draft equation, at least until a better metric is derived to replace it.
The premium each organization places on a particular Wonderlic score will inevitably vary; consensus is a rarity in personnel evaluation.
But what will always remain true is that every available tool to measure a player’s ability — the Wonderlic, 40-yard dash, bench press, and most importantly his film—is a piece of the draft puzzle.
I could not agree more with the author’s take on the value of testing as one piece of the bigger picture, the value of which is determined by the situation and the goals of the individuals who are responsible for making decisions. In my own work with testing and assessment I tend to recommend a model that focuses on the collection of a variety of data points. They all tap into different things that are important for success. Some can weed applicants out at key points in the hiring process; collectively, they can be “added up” at the end of the process to provide the data needed to make an informed final decision between candidates.
Here are a few more thoughts about the parallels between the article’s main points about predicting success in sports and my own insights around predicting success in the workplace.
- The “whole person approach” is important: The Wonderlic is a test of basic cognitive ability and a very tried-and-true one. There is no doubt that it is provides very valuable information, but that there are other traits and abilities that are important too. It is often not a good idea to base all hiring on one test; understand what success looks like and build a measurement model that includes tools to measure all of the key things required. Just as the 40-yard-dash time is a critical component, so might be typing speed or skill with Java.
- Understand the characteristics that are most valued by key stakeholders: When designing a selection system I am always delighted to have the chance to start at the top and let the organization’s strategic goals drive the process. The more all stakeholders are given a say in what they value and see as critical for success, the better. The combination of traits that are valued will likely be different in different situations, and this is what should drive the development of selection tools. An organization that prides itself on defense may look at the draft and its data differently then one that is more offensive-minded. Similarly, one firm may value innovators, while another firm may value experience, or free-thinking, or raw ability, or people who can follow a traditional path.
- Provide applicants with a strong connection between the test content (or the hiring experience) and the real job: Tests are nothing but proxies for measuring an individual’s ability to perform in real situations. Real situations are often hard to model and evaluate, and thus we dissect the real experience into the basic traits required to perform, and then create tests to measure these things. The closer one is able to replicate the real situation, the more confidence one can have in its ability to identify those who will be successful in that situation. Tests that do not appear job related often require a leap of faith. In the world of sports, this is somewhat taken care of for us, as the author suggests film study of actual real-life situations is never going to take a back seat to a test score. The value is not in abstract skills like analytical ability, but in how those abilities are applied on the job (reading defenses when a giant lineman is trying to remove your head). While there is value in testing, there is even more value in making your test look like the job. The less it does so, the more its value may be questioned.
- Don’t underestimate the value of training and experience: While natural ability sets some individuals apart (Barry Sanders, Lynn Swann, etc.), there are plenty of regular Joes (and Janes) out there who have some innate gifts but who also can be developed into champions. We strive to identify the best talent and make the best predictions, but there are only so many persons who are operating on an exceptional level. Hiring for basic ability and coachability is a fine strategy as long as you know what you are starting from (the correct level of raw talent) and have good coaches on your staff. Sometimes you hire someone for the intangibles and work tirelessly to coach the other stuff (think TimTebow) to a useable level.
- Cast a wide net and don’t believe the hype: My favorite team has found some of its very best players in the furthest corners of the lower divisions of the NCAA. The moral is that raw talent is not always operating in the limelight. A good evaluation process will help mitigate the biases in perception about where one finds it. We have all lived through the many busts that have occurred when an individual is overly hyped and then fails to live up to expectations. A good evaluation process can also help cut through the crap and separate the pretenders from the contenders.
- No matter how good you are, you will never be right all the time: Get over it already.Humans are extremely unpredictable and our behaviors and choices often defy all logic and explanation. The best NFL scout and the best hiring managers and recruiters all share the burden of the imperfection inherent in predicting human behaviors. Failure is part of the equation. It’s how you deal with failure and what you learn from it that will make or break you.
As we gear up for the upcoming NFL draft this weekend, teams (and their fans) are studying, analyzing, prognosticating, and deciding which talent they should hire to help them achieve their goals.