As online screening and assessment continues to increase in popularity, more and more people are starting to talk about the value that these services can add to the staffing process. While this is music to the ears of those of us who have dedicated our careers to increasing awareness of the benefits of assessment tools, it has also been a source of endless frustration. Why you ask? Because many folks out there are creating needless confusion by refusing to acknowledge the differences between screening and assessment. This article is dedicated to discussing this problem, and hopefully putting an end to the confusion. The Origins of the Problem The origins of this problem are firmly rooted in the fact that screening and assessment each represent the perspectives of two different sets of players in the staffing game: 1) recruiters and 2) the employee-selection scientist-type geeks known as “industrial psychologists.” Recruiting is about sourcing qualified candidates and getting them into the staffing pipeline, while selection focuses on evaluating candidates after they’ve been identified. For this reason, recruiting tactics tend to be closer to those found in marketing and sales, while selection tactics tend to be more systematic and analytical. While these two approaches can and should complement each other, the reality is that they often don’t. For instance, selection system designers often don’t pay enough attention to the “emotional” appeal of the system and the important impacts of candidate reactions, while recruiting efforts often focus more on intuition and gut feeling as opposed to rigorous analysis. Please understand that I feel that the viewpoints of both recruiters and selection geeks are legitimate and can actually compliment one another quite well. In fact, the most well balanced and effective hiring systems are those that have been designed to include both perspectives. The real problem here is that, in many cases, members of these two camps do not understand the other’s perspective as well as they should, and thus end up making mistakes. For example, selection professionals write notoriously bad job postings (they are too detailed, have boring lists of qualifications, and do little to “sell” the job). On the other hand, recruiters tend to be lousy at creating assessment systems, because they are often unaware of the technical details needed to make these systems work effectively. Although both of these issues can create problems, the downside risks of a recruiter masquerading as a selection expert (e.g., hiring the wrong candidates, getting sued) are far worse than the downside risks of a selection experts masquerading as a recruiter (failing to generate enough qualified candidates, missing out on the “feel” of which candidate is right for the job). Again, I am not saying that one camp is any better than the other. I’m saying, rather, that effective hiring systems require a balanced input in which both groups respect one another’s viewpoint and work together to leverage their combined knowledge and skills to hire the best candidates. So, before all the recruiters out there begin to bombard me with hate mail, let me explain where I am going with this. The Crux of the Issue The reason that I decided to write this article is because I have noticed a disturbing trend lately. The advent of web-based integrated staffing systems is leading to the perpetuation of misinformation by technology platforms that have been created by recruiting-minded companies, who suggest that their systems have the ability to effectively “screen” candidates. While the rationale behind these systems ó providing the ability to collect information that can be used to determine which applicants are most qualified for a job ó is sound; the functionality and process that is provided to help users make screening decisions is often way off base. This is because the creators of these systems have confused screening with assessment. The major flaw when it comes to screening systems is that, because they are created by recruiters, these systems don’t take account the technical rigor that needs to go into staffing system design. This level of rigor was not created to add fun and excitement to the staffing process, rather it is an essential ingredient in ensuring the legal compliance (and effectiveness) of a staffing system. Furthermore, the lack of awareness of the technical requirements of the hiring process often leads to the creators of these systems making deceptive claims. Chief amongst these are:
- Screening is a substitute for assessment
- There is no need to validate or audit the quality of prescreening questions
The remainder of this article is dedicated to providing information that I hope will be helpful in clearing up these misconceptions. Screening Defined The purpose of screening is simply to examine very broad and basic qualifications of applicants in order to narrow down the applicant pool to a manageable number of qualified applicants, whose fit with the job can be examined in more detail during the remainder of the staffing process. This detailed examination takes time and resources, and thus there is great value in providing a mechanism that ensures recruiters can avoid wasting precious time barking up the wrong tree. Based on the needs that it fills, screening is most often characterized by a questionnaire administered very early on in the application process for the purpose of collecting data about an applicant’s most basic qualifications. Common screening criteria include willingness to relocate, number of years experience, highest degree obtained, salary requirements, and so on. There is no doubt that this is important information. But it is important to recognize that this information will only take you so far. While information gathered during screening is excellent for filtering applicants out based on very coarse qualifications, it does not tell you much at all about a candidate’s ability to perform the job in question. It certainly does not get at critical, below-the-surface information related to personality, ability, or fit. Assessment Defined Although it shares some basic similarities with screening (i.e., questions asked as part of the job application process), assessment (a.k.a. “scientific screening”) is quite a different animal than screening (for a more detailed discussion of these differences please read my earlier article about this subject. The purpose of assessment is to provide a measurement of specific knowledge, skills, abilities, traits, and competencies that have been clearly linked to job performance. The benefit of this is that properly constructed assessments look below the surface information presented by an applicants in order to systematically predict which applicants will be the best hires for a position. What assessment tools do not do is make high-level judgments about a candidate’s suitability for a job based on highly objective information. Instead, assessment is a systematic process that uses scientifically developed tools designed to measure specific aspects of job performance. Assessment is a data-driven process, both because data is required to construct assessment tools and because in an ideal situation, organizations using assessment tools will collect data in order to obtain metrics to verify that these tools are actually doing what they are supposed to do. At the end of the day, analysis of assessment data can tell us exactly what job performance is and how good of a job the assessment tool is doing at identifying successful candidates based on a working definition of job performance. Even better is the fact that the data obtained regarding the performance of assessments can be directly linked to critical monetary metrics such as ROI. Try that trick with a resume screen! Problems With Screening The major problem with the use of screening is that it is often sold as being able to do the same thing as assessment. This is a very dangerous proposition because, as I have shown, it is actually very different. Here are some of the problems that can result from this misunderstanding:
- Not based directly on job analysis. Screening questions often do not have direct, tight linkages to clearly specified aspects of job performance such as those provided by job analysis.
- Not validated. Screening systems are almost never validated, so there is no way to gather robust (i.e., durable) metrics about the effectiveness of a specific screening question.
- Lack of quality control. Systems that are set up to allow recruiters to create screening questions offer almost no quality control over contents of screening questions or the development of algorithms used to eliminate persons from the applicant pool.
- Lack of concern over legal issues. Most screening system vendors downplay the whole idea that screening questions may actually run afoul of EEOC rules and regulations. This is far from true if you consider the fact that most of these systems allow personnel who are not well versed in the legal aspects of selection systems to develop measures used to kick applicants out of the selection process.
I feel that all of these problems share a common underlying trait, namely a lack of rigor surrounding the deployment of questions used to make key decisions in the staffing process. As long as these tools lack this rigor they will place any selection at greater risk for potential legal problems and they most certainly cannot be called assessments. Problems With Assessment To show that I am not adopting a holier-than-thou attitude about assessment, I want to state for the record that assessment is not without its own problems. These problems include:
- Fear of legal issues. Assessments are bound by the same set of legal issues as screening is however legal concerns seem to be much more salient when it comes to assessment. Many folks are so worried about possible legal ramifications of these tools that they steer away from them, favoring simpler, but less effective, tools instead.
- Time consuming to set up. Although the Internet has greatly reduced the time it takes to set up an assessment system, there is still ground work involved in getting an assessment system up and running. I place this in the “no free lunch” category. As far as assessment goes it is very hard to get something for nothing. Accurate prediction requires a bit of sweat.
- Experts required. The technical rigor associated with assessment means that one of us I/O geeks should be involved somewhere in the process. In most cases this is the only way to ensure that assessments are being used in an effective and legally defensible manner.
- Perceptions that the cost of entry is too high. All of the issues related to the proper use of assessments mean that many folks balk at the costs of setting up an assessment system. What they don’t understand is the fact that, with a properly set up system, these costs are but a small fraction of the return on the investment made in the system.
Ironically, many of these problems stem from the rigor that must be applied to the assessment process. This is pretty much the opposite of the origin of problems related to screening. Conclusion: Let’s Learn From One Another The information that I have presented in this article should help to clarify why screening and assessment are not the same and show that both suffer from their own problems. Screening seems to be marked by a lack of rigor while the rigor associated with assessment seems to be a roadblock in the minds of those who are creating staffing systems. This information clearly suggests that those who use and promote screening can learn from those who use and promote assessment and vice versa. The recruiting camp needs to respect the importance of rigor while the assessment camp needs to continue to work to demystify assessment and provide more hard evidence of its effectiveness. This will result in more harmonious staffing systems that are a blend of the expertise of all parties involved in the staffing process.