What’s Wrong With Reference Checks (Part 1)

Employment reference checking and background screening should win recognition as the weakest of all corporate HR processes. A validity meta-analysis study conducted by Aamodt & Williams in 2005 found that the corrected validity coefficient for reference recommendations and actual job performance was a staggeringly low .29. Despite the facts, 96% of organizations use reference checks as a screening and selection tool, according to a recent survey by SHRM.

Anyone who’s been in the profession for more than a minute or two knows deep down that references suck as indicators.

That’s why it is a task often assigned to an unknowledgeable coordinator, outsourced, or even turned into a multiple choice online form. While there is a handful of people who swear that they do it right, the vast majority do it simply because it has always been done. Maybe reference checking is/was more accurate in smaller communities where everyone knows everyone, but for many organizations today, new hires are strangers. As we enter an era where organizations have years of data and access to sophisticated tools to better determine what does and doesn’t work as an indicator, recruiting leaders should expect more scrutiny of this process by everyone.

Government agencies and commissions that oversee equal opportunity and business dealings are paying more attention and increasingly holding organizations accountable for the actions of third-party services commissioned to carry out screening activities. If you are in a leadership role or influence someone who is, it is your professional obligation to ensure that what your organization is currently doing and plans to continue doing isn’t stupid!

The following is a long list of things that routinely plague reference checking processes broken into four categories. If you look for these weaknesses in your current approach and address them, just maybe your validity coefficient will improve.

The Top Five Most Serious Reference-checking Problems

Few provide detailed reference information — because former employers face legal action relating to defamation, many firms have a policy restricting reference conversations to factual information such as job title, dates of employment, and possibly salary history. Questions aimed at identifying if the applicant could be rehired by a former employer often go unanswered. The end result: some candidates may benefit from robust reference profiles, while others may not depending on the stance of their former employers.
Reference information provided on the wrong person — in larger organizations it is not uncommon for supervisors and coworkers to confuse past employees with the same/similar names, so it is possible the individual being described isn’t the individual in question.
Historical situations may not align with present conditions — references often focus on generic aspects of job performance specific to a different environment at a different time. Even if you are able to robustly identify how someone previously acted, you can only speculate that they will act the same way today in a completely different environment. Most firms have no hard data proving that in a fast-changing world where skills and knowledge become obsolete much faster, that past performance at another firm is an accurate predictor of future performance at your firm.
Knowledge of the applicant is limited — for a variety of reasons, the individual providing reference information may have extremely limited knowledge about the individual being referenced. For example, they may be able to confirm job responsibilities, but know nothing of actual performance if they were not the person’s actual supervisor. In large organizations, few in HR would have personal knowledge of the individual’s work history or traits.
Records are not reviewed — unfortunately, many references (especially telephone references) are provided by former employers who are relying 100% on their memory. You can be almost 100% certain that performance appraisals or attendance records will not be reviewed prior to the reference. Not relying on records and data almost guarantees an inaccurate reference.

Problems Related to a Changing Business Environment

Bankruptcies, mergers, and layoffs make getting references difficult — because some employers simply no longer exist, it is often impossible to contact previous employers. Large-scale layoffs might also mean that even if you find the firm, the applicant’s supervisor may no longer be there. Layoffs and lean staffs in HR may also slow responsiveness to reference requests.
Culture and language issues complicate — because different cultures and regions have unique ways of handling references, global consistency is difficult. In addition, language issues can make some reference checking extremely problematic.
Many have bad credit — firms that used to supplement employment reference checking with credit checks now find that so many people have credit issues stemming from bank actions that relying on past standards will cause you to lose too many candidates.
Identity theft issues — identity theft has grown beyond credit card fraud and has entered the employment arena. Unless you take and verify fingerprints, you may be checking the references of a real person who unfortunately is not actually the person you interviewed.
Illegal or private information is viewed — supervisors and managers all too often provide illegal or personal information that may unfortunately end up being used in the hiring decision. Reference checking that includes searching Internet or social network sites also increases the likelihood of you obtaining information that should not be part of an employment reference check.
Resume lies are more common — in a weakened economy where individuals are desperate to get a job, it is much more likely that they will stretch the truth on their resume, application, or during the reference process. This makes it even more important to get accurate reference information beyond the resume.

Problems Related to the Person Providing The Reference Information

Bias produces slanted references — a supervisor or coworker with a vendetta may slant the reference so that it comes out overly negative. Supervisors may provide a negative reference on a current employee in order to improve their chances of keeping them.
A positive slant — the majority of references are positive because people try not to say bad things. But when the reference is or was a close friend or colleague who wants to help you, the odds of finding negatives drops dramatically. Personal references are even more likely to be 100% positive because the individual selects only those individuals who they know will provide a positive reference. Managers may give a glowing reference in order to get rid of a problem employee.
Self-provided references may mislead or be fraudulent — many firms allow the applicant to provide the names, e-mail addresses, and phone numbers of their references, and it should be obvious that self-selected references are more likely to give positive answers. In addition, unfortunately the names that applicants provide may not have been the person’s actual supervisor. In some cases, the number or e-mail address provided might not even belong to a real firm.
No one is fired anymore — if you are using references to find out if someone has been previously fired from a job, you need to realize that because of legal issues, these days most individuals are allowed to resign. As a result, their employment record will not show a termination or the person will not divulge that fact.
Leniency effect — when corporate employees do reference checking, there is a measurable tendency to be “lenient” and to downplay negative information. This leniency is caused in part because finding too much negative information will mean that the reference checker will be asked to delve deeper (more work) or that they must reopen the position search. People providing references can also be overly positive because they know that a bad reference may cost the applicant (who may also be a friend) their next job.
Competitor firms will not cooperate — in a highly competitive business world, it can be increasingly difficult to get any form of information from a major product or talent competitor.

Administrative Problems With the Reference Checking Process

No company data to prove they work — because many firms execute their own reference-checking process, it would be a mistake to rely on external research to prove the accuracy of the process. Unfortunately, I have never come across an organization that routinely gathers and analyzes data on the predictive accuracy of their reference process.
Reliability is low — most corporate reference-checking processes (unlike vendor processes) are loosely designed and poorly controlled. Moreover, when reference checking is decentralized and many untrained individuals are conducting them, the execution (consistency and reliability of the process) slips even further.
The person doing the checks are weak — many times the employees who are given the assignment are not trained in reference checking or knowledgeable about the organization or job specifics, making it difficult for them to dive deeply into relevant matters.
References are often not completed until after the time of hire — due to the cost and time involved, some firms now complete reference checks after the individual is already on the job.
Lawsuits are real — final hiring decisions are made based on reference-checking results, and in a down economy, more rejected candidates resort to legal remedies. Unfortunately, during legal discovery you may find that your references have a sudden memory loss and they will not repeat what they initially said, leaving you in legal jeopardy (which is why written references are superior, even though they are slower). In direct contrast, failing to check references may result in the possibility of negligent hiring claims.
Reference checking is slow — in order to get documentation, it is important to use written references. However, written references take much longer, and a higher percentage of individuals are reluctant to participate in references when they must provide their signature. Reference checking is often not a high priority among recruiters or support staff even though slow reference checking may cause you to lose top candidates.
The use of leading questions — a significant percentage of the reference questions that I encounter are binary questions pertaining to positive traits that almost always elicit positive answers. Requiring the individual to force rank or pick from a list of both positive and negative answers is a superior approach. Open-ended questions result in different information, making comparisons between candidates difficult.
Poor documentation — telephone references often suffer from poor documentation and capture no signatures, which can be a problem should legal issues arise.
Automated reference-checking limits depth — web-based reference-gathering systems shift the burden of getting references to the candidate, added work that may cause some candidates to drop out. In addition, to make such tools possible, the scope of the reference is limited and the ability to dive deeper into specific responses is virtually non-existent.
The same process for all jobs — even though some jobs demand much more thorough reference checking (i.e. those who handle money, those with childcare responsibilities, etc.) most corporations use the same exact process for all jobs.
A checklist is not used — rather than using a checklist of standard questions, many supervisors who check references “wing it” or make up their reference questions as they go, resulting in no consistency. If standard questions are used, they are often not weighted based on their relative importance in predicting future job performance.
Mixed results make comparisons and conclusions difficult — the more references you check and the further back you go, you increase the likelihood that you will be faced with mixed references. Also, because an individual may receive several glowing and one extremely negative reference, making a decision based on consistent results may be difficult. When candidate slates include those recently out of school with little or no experience, managers are forced to make difficult comparisons between candidates with business experience and those who only have school related references.
All references shouldn’t receive equal weight — older references should be given a lower weight because the further back you go in a candidate’s work history, the less likely that the results will be valid predictors.
No permission signature is obtained — if you are using resumes instead of application forms, you may be checking references without written authorization. If you inadvertently call their current employer, you may get them fired, which could lead to them taking action against you.
It is not a continuous process — most references are done exclusively during the hiring process, but criminal, motor vehicle, or credit issues could occur well after the individual has been hired.
Inclusion requires flexibility — if your recruiting or diversity goals include attracting individuals from lower-income groups, it is important to realize that some of these individuals may be more likely to have criminal, driving, credit, or job-jumping issues that can result in weaker overall reference scores.
Contingent workers are treated differently — some organizations fail to require reference checks or allow vendors to use less stringent processes for temporary, part-time, and contract labor. As contingent workers become a larger percentage of a firm’s total labor, this inconsistency will need to end.