Every year I attend the annual Society for Industrial Organizational Psychology conference to learn and make sure I am in touch with the latest goings-on in my field. This past year I was very excited to walk away with an unpublished research paper titled Legal Risk in Selection: An analysis of processes and tools, by Kate Williams, a doctoral candidate at Clemson University. This article has direct and practical value for the members of the ERE community. If you are in any way involved in shaping the staffing strategy of you organization or if you really want to know the practical requirements for ensuring the EEOC and OFCCP stay out of your kitchen, you need to read this paper, or at least the short summary of its major points that I provide below.
Williams’ paper is basically an analysis of the data provided by the Bureau of National Affairs, a private company that publishes information and analysis for business and government. Believe it or not, the BNA offers paid access to a database called Employment Discrimination Verdicts and Settlements that can be searched to identify each and every employment discrimination case filed with the EEOC and OFCCP. This database is a literal gold mine for anyone who wants to know why companies are being sued for their hiring practices. It represents the reality behind all of the conjecture around what will get you strung up by the feds when it comes to hiring practices, and it’s data should serve as a good way for companies to gauge their level of risk.
What Was Analyzed
Williams’ paper analyzed the nature and outcome of legal proceedings involving the EEOC and OFCCP, including all cases related to external hiring (not promotions) that were settled both in and out of court during the time period between 1998 and 2010. The findings reported cover two key areas:
- Issues with a particular type of selection measure or test including: unstructured interviews, structured interviews, biodata measures, cognitive ability tests, psychomotor tests, and “other tests.”
- Issues with the way the selection tools were used within the hiring process, including the following key issues: lack of documentation, inconsistent process, quota for affirmative action programs, recruiting source violations, violations of the 4/5th rule, and “other” process issues.
Findings
I don’t have room to present a detailed review of the findings from Williams’ study in this article. Those interested in a full list of the findings should obtain a copy of the article itself (email her at mailto:kate.zaner.williams@gmail.com). However, a brief run through of the highlights still provides a ton of useful information and presents an excellent picture of what not to do when building a hiring process.
A total of 224 complaints were identified. Within this total 109 were settled out of court (73 by the EEOC and 36 by the OFCCP). Of the 224 total complaints, only 78 of these actually went to trial. Note that when examined over the 12 years covered by the study, the total of 224 cases breaks out to about 19 per year. The total number of cases that end up facing legal challenges is very small when one considers the massive number of hires made each year in the U.S.
Issues with Selection Measures
Of the 52 complaints about a selection instrument, a bit over half (52%) were settled before trial.
Issues with selection measures almost exclusively involved either interviews or cognitive and psychomotor testing.
Interviews
No cases involving complaints about interviews were settled out of court. In all cases employers decided to take their chances in court and prevailed more often with structured rather than unstructured interviews.
Of the cases that went to trial involving interviews, structured interviews held up better than unstructured, with only 13% of the structured type being ruled discriminatory and 50% of the unstructured ones being ruled as such.
The bottom line here is that if you get challenged on your interviewing process, you are likely going to court, and if you follow best practices and take the time to develop a structured interview process, you are more likely to prevail. If not, it is going to cost you big time. This makes sense given the fact that structured interviews create consistency across interviewers and usually provide excellent documentation of the rationale behind the interview scores that contribute to hiring decisions.
If you are not using a structured interview process, you have a problem. You should make every effort to move out of the stone age and adopt this easy and painless upgrade to your hiring process.
Testing
Two types of tests (cognitive ability and psychomotor tests) ended up accounting for almost all of the challenges related to testing. This makes sense given the fact that these tests consistently demonstrate adverse impact while also having some of the highest levels of validity (i.e., ROI). Welcome to the crazy, mixed up world of testing, where the most effective tests can land you in the most trouble!
Completely opposite of the results for interviews, all of the challenges related to cognitive ability tests and two-thirds of those related to psychomotor tests were settled out of court. This is likely due to the fact that if a test cannot be shown to be job related by the plaintiff, there is no possible way they can win a challenge. Cases that are settled for the plaintiff almost always relate to a failure to demonstrate the job relatedness of a test that demonstrates adverse impact.
For example, in EEOC vs NationsBank of Tennessee (2001) a cognitive ability test discriminated against Hispanic employees by requiring English proficiency, a competency that was not required on the job.
Cases related to psychomotor tests that were settled also showed serious issues with job-relatedness. For example, in EEOC vs. American Airlines (2002), the company used a pre-employment test for meter readers and janitors that had adverse impact against females and measured skills that were not required on the job.
The issue of job relatedness and adverse impact can be a bit complex, but the bottom line is that failure to demonstrate the job relatedness of any sort of test can land you in big trouble. Doing so with cognitive or psychomotor tests will significantly increase your risk factor because these tests are most likely to cause issues with minority score patterns.
Process Issues
While selection devices are a common source of litigation, this study shows that the legality of the selection process is a much more important factor. Cases that went to trial around selection devices were decided for the plaintiff only 28% of the time, vs 68% for those related to the selection process, meaning that process issues are more likely to land an employer in hot water.
This table (click to enlarge) summarizes the reason for legal challenges related to discriminatory selection processes.
The data clearly demonstrates that the most common reason for process cases is inconsistency in the hiring process. Cases related to inconsistent process accounted for the largest percentage of all process related cases and over half of these were settled prior to court. A whopping 91% of all inconsistent process cases were found to be discriminatory.
Some examples of process related cases that were lost by the plaintiff include:
In Dennis v Columbia Colleton Medical Center (2002), the U.S. Court of Appeals described the hospital’s selection process as “a peculiarly informal process” because their explanations for not hiring the plaintiff were different from the written job description, giving the decision “a flavor of post-hoc rationalizations.”
In Dunlap v Tennessee Valley Authority (2008), the court determined the company’s hiring process was discriminatory because they found 70 counts of manipulating test scores and changing interview and test scores in candidate rankings.
ERE readers should know that issues related to the selection process include the source of candidates and the way positions are advertised. In Allen v Tobacco Superstore (2007), the company relied on word of mouth to publicize open positions and had no consistent procedures for advancement; employees simply asked a supervisor to be considered. The court found the word-of-mouth hiring and promotion process — which resulted in a company-wide dearth of Black store managers despite operating in communities with large Black populations — was discriminatory.
Hiring processes must be standardized in terms of the information that is used to make employment decisions. Informal practices or doing an end-around and ignoring the documented process can lead to a heap of trouble. Those who think this is just a silly little nuisance should be sure to read on.
Costs of Legal Complaints
Settling out of court resulted in average fees per case of $590,266 for EEOC cases and $668,785 for OFCCP cases.
Cases that were settled by individual plaintiffs rather then a government body averaged $12,292,492. In comparison, cases that went to trial and were found in favor of the plaintiff cost organizations an average of $13,306,346.
Settling out of court is a much cheaper way to go when faced with a challenge to a selection process or tool. Complaints about selection processes can be very expensive and greatly outweigh the costs of taking the time to do things correctly.
Concluding Remarks
The most interesting thing I gleaned from Williams’ article is how much employers’ thinking about hiring practices reminds me of the Ford Pinto. Most of us are old enough to remember Ford’s exploding car bomb and the fact that Ford’s bean counters knew about the risks related to using faulty components in its fuel system, but decided that the estimated cost of litigation was less than the cost of fixing the defective parts.
Like the Pinto, while employment litigation or challenges can be expensive, the numbers show that it is highly unlikely that an employer will have to pay the price, making this an issue of risk tolerance. The odds are in the employers’ favor that they will get away with poor practices and as such the threat of legal action is not enough incentive to force them to action.
The good news is that there are two wonderful reasons to use job-related tests within a consistent and standardized process. The first is that companies have a moral and ethical obligation to do the right thing and treat all applicants fairly. The second is that the two key issues here — job relatedness and standardized process — are directly related to ROI. That’s right, folks: doing the right thing is also a direct path to profits!! We I/O psychologists have decades of data to support this fact, and almost every vendor out there can produce dozens of case studies to support this fact.
So it’s time to flip the risk equation and realize that doing the right thing can also lead to excellent profits. Use your hiring practice to build a Prius, not a Pinto!