What’s Wrong With Reference Checking? Pretty Much Everything (Part 2)

This article addresses five questions raised in response to Part 1 of this series published last week. It addresses the best ways to assess candidate performance pre-hire and when to use references.

Question 1 — What are the most accurate indicators of past, current and future performance?

Finding accurate real world predictors of future performance is difficult but not impossible. Professional sports teams find that the best predictor of a new hire’s potential performance is their performance on the field in practice and preseason games — i.e., a work sample. Google looks at a multitude of factors that can be combined by an algorithm to successfully predict both future on-the-job performance and retention risk. The U.S. military and numerous firms in industries with extreme operational risk like airlines and chemical production facilities rely on sophisticated simulations to assess how a candidate would react in various situations.

There are literally hundreds of potential tools and approaches that can be used. Unfortunately, the vast majority of research on the subject is questionable at best.

Rarely are academic researchers trusted to manipulate the assessment processes of organizations hiring in significant volume such that the process could provide valid data, and few if any organizations execute consistently without extreme oversight Commercially sponsored studies are flawed in that they are executed to prove a tool or approach works, not to test the extent to which it works.

That said, most studies generally conclude, and I tend to agree, that work samples are often the best predictors. Based on my observations and learnings, I routinely recommend the following.

Most Accurate Tools for Assessing Current Performance

Temporary hiring — The best way to identify top performers is to hire them into the job as temporary workers, contractors, consultants, or interns. Interns often have the highest success rate among college hires because you can rely on their internship track record as a predictor of their ability to perform in your environment.
Actual work cases — next to “playing in the game” is assigning them a case challenge based on the real problems that the candidate would face shortly upon accepting an offer. The problem should be a real current issue, so that even if you opt not to hire the candidate, you benefit from their insight into the issue. An alternative approach that has been used by firms like Raytheon is to give them a “broken” or flawed process and ask them to find the errors and source of problems.
A verbal simulation — this approach takes advantage of the traditional interview time period and asks candidates to walk you through how they would tackle an actual work problem. Firms using this approach generally use a case that has already been solved (so that the interviewer/assessor is familiar with all of the relevant information the candidate may need to inquire about and knows the likely outcome of each solution step introduced).
Samples of current work — in roles that produce physical products, one of the best ways to assess a candidate is to assess samples of their current work. If you want to find out if a chef is any good, you taste their food. In some cases candidates may not be able to provide existing work samples from their current employer due to confidentiality issues.

Most Accurate Tools for Assessing Future Performance

Ask them to forecast — if you need to assess candidates’ ability to perform in the future, consider asking them to predict future opportunities and problems your industry or firm will likely face and the skills and competencies that will be required to address them successfully. Consider whether their vision aligns with or is more/less robust than yours.
Future work case — give the candidate a problem or opportunity that is projected to occur two years out. Ask them to “walk you through the steps” that they would take and the skills that they would need to resolve it.
Demonstrate leading-edge learning — the most important competency in a fast-changing world is the ability to continuously learn and stay on the leading edge of knowledge. Even if candidates and employees are successful performers today, it is unlikely they will remain top performers unless they are continuous self-directed learners. You can assess whether they have that competency by asking them specific questions about what are the leading-edge “next practices” in their area, who are the benchmark firms, the key thought leaders, and what specifically do they do every day to remain on the leading edge of knowledge.

Question 2 — When do you recommend using reference and background checks?

As I said in the first part of this article, it’s okay to use one or more reference and background checking approaches provided that you understand why you are doing so and that you have designed a process that can be consistently executed to limit exposure to the vast majority of limitations found in the typical traditional process. If you have read my stuff for a while, you will know that I look at references as future candidates, so getting current candidates to identify their network through any process is a win, even if it wasn’t the intended focus of the process.

Two good reasons that would make doing references essential:

A correlation with quality of hire — any time you have supporting data showing that at your company there is a positive correlation between a candidate’s on-the-job performance and retention after one year (quality of hire) and a scale-based recommendation output by your reference/background checking process.
When there is a legal requirement — when you are hiring for roles that require due diligence to confirm proper licensure, certification, education, etc.

Note that 99.9% of all murderers, bank robbers, and major evildoers are first-time offenders, so the fact that they haven’t done anything in their past doesn’t prevent them from future missteps. If you really believe that criminal, driving, credit, licenses, etc. are valid indicators, you need to check these elements periodically post hire to ensure that your employees maintain their initial pristine status.

Four not-so-good but still-common reasons to do them:

Covering your butt — the most common reason to use reference checks are to shield yourself from potential verbal lashings (after a new hire turns out to be a real turkey) by being able to say “but I checked their references.” In addition, in the almost miniscule chance that you’ll be called in a negligent hiring suit, having checked the references will get you a handful of minor points from the judge.
When you want a particular candidate no matter what — if the recruiter or the manager is dead-set on a particular candidate, I understand the common practice of assigning an inexperienced and positive-thinking person (overly positive people will seek out positive attributes until they find them) to do the reference checking. You can almost guarantee that they will come back with the desired “I found no problems” answer.
When you need another excuse to fire a bad performer — if you are reluctant to fire a bad-performing new hire, I have seen many firms purposely redo reference and background checks in the hopes that they can find a lie or omission that could be used as an excuse to terminate.
You yourself need job security and are a bit selfish — if your job involves doing reference checks and recruiting in general, it’s in your best interest to continue executing a time-consuming, flawed, yet accepted process. Avoiding more effective candidate assessment methodologies will ensure that a greater percentage of new hires will fail and therefore require replacement recruiting; in other words, provide job security.

Question 3 — Are reference/background checks accurate assessments of past performance?

First of all we must establish that there is little to no consistency in what is considered a reference check. I cannot say that no reference check focuses on past performance, but based on my observation of practices in place at hundreds of organizations, the vast majority come nowhere close to remotely gathering intelligence on past performance. For example, many of the following elements often focused on during a reference/background check are not indicative of performance:

Job title
Dates of employment
Credit scores
Criminal history
Personality traits
Attainment of a degree
Knowledge of buzzwords
Information relating to job performance in another era (skills and knowledge grow obsolete too!)

All performance can be quantified, but rarely would a reference checker be granted access to real records of output/productivity from another employer. While candidates could request copies of past performance appraisals, we all know that most view that process as being worse than reference checking with regard to validity!

Question 4 — Are reference checks predictors of present or future performance?

If traditional reference checks are weak ways of assessing performance and especially past performance, do they provide any insight into present or future performance potential? Again, the answer is a sad no. As typically executed the traditional, haphazardly executed reference check is not a valid indicator for the following reasons:

The past isn’t today or tomorrow — the world has and continues to change, rendering information and skills obsolete at a fastening pace. Behaviors and actions that produced spectacular results yesterday may fail miserably if the environment changes, which we know it does.
Your culture is different — what makes one successful in one environment will not necessarily do so in another. Top performers can easily become average performers under a new manager, and vice versa. A lot of factors influence a candidate’s ability to perform.
They may have been bridled — a candidate with a poor “performance related reference” may have everything needed to perform in your environment, but been unable to do so in a previous environment due to lack of professional freedom, proper tools, poor management, etc.
The individual has changed — skill and motivation levels change over the years, so how someone previously performed in a different state of mind may not be indicative of how they would perform in a different state.

Question 5 — Are the current vendor offerings in the areas of reference and background checking worth examining?

Technologies and service offerings in the HR/talent management space are evolving at a rapid pace. Many emerging offerings are not bound by the traditions and ingrained practices of the corporate world, and bring valid arguments to the table that all professionals should be aware of. The hottest tools under development today, and others that have been around for a short time, focus on providing organizations with a more comprehensive look into a candidate’s professional network. Tools leveraging a 360-type approach increase the size of the assessor pool, and thereby increase the degree of structure afforded the process and the probability that more qualitative data will emerge. I can’t say without firm specific evaluation that any tool would produce valid results in your organization, but some criteria that I would recommend using when assessing top vendors include:

Proof — do they have proof demonstrating the correlation between reference process outcome and on-the-job performance in real client organizations that can be referenced themselves?
Integration — do they integrate the many different checks (i.e. credit, work history, personal references, criminal, educational, license, driving, Internet etc.) into a single system?
Metrics — do they have access to/knowledge of professional intelligence about the validity of component processes so that they can advise you using data on what practice improve/harm your candidate assessment efforts?
Global — do they present a solution that can be used around the globe and that would not be limited in adoption by variances in culture or language? (The labor market is quickly becoming more global, so it is increasingly unrealistic to assume than a candidate’s references will come from the same region or cultural background as the candidate.)

Final thoughts

This series has introduced 30+ reasons to be wary of traditional reference checking processes, as well as a host of alternative approaches that could lead to much more accurate assessment of a candidate’s past, current, and future performance. As the EEOC increases its scrutiny and focus on this subject, everyone in the field of recruiting should learn to be cynical and test their efforts and solutions being adopted.