2005 Online Screening and Assessment Survey Results

article by Charles A. Handler & Mark C. Healy In both 2003 and 2004, Rocket-Hire released findings from a survey of ERE readers who use online hiring tools. In order to continue tracking trends in online screening and assessment, we again asked people professionals to tell us about their use and opinions of these cutting edge tools. The results of this third annual survey are in, and the continuing story of the real world use of online screening and assessment in 2005 can now be told. This year’s survey contained most of the same questions as last year’s, and provides a glimpse into the ever-changing and growing world of online collection of job candidate information and qualifications. In addition, we have expanded our questionnaire to more completely cover some emerging areas of practice. Before we detail the findings, we would like to thank all those “People People” who filled out the survey. We realize that time is money, and we couldn’t have completed this study without the thoughtful responses to questions, wide breadth of information and enlightening comments they provided. Who Responded? A total of 90 people professionals completed the survey, which represents an increase of 15% over the 78 that responded last year. Sixty-five individuals responded to the first survey back in early 2003. We’ve grown the sample nearly 40% since we first began collecting this kind of information. At this point, we should highlight some potential limitations to the sample of individuals and organizations who participated in this study:

• This study represents the work and opinions of a sample of individuals interested in online screening and assessment, not a random sample of HR or recruiting departments or organizations in general.
• Individuals responding to the survey ó and regular readers of ERE ó are likely to be more technologically savvy and represent the concerns of those with a fairly mature understanding of the Internet, emerging hiring technology, and hands-on use of these online screening and assessment.
• As with last year’s survey, only about 10% of the previous years’ sample responded this year. Therefore, this paper cannot be considered a true longitudinal research study. However, the types of individuals responding to the survey ó and the kinds of organizations they represent ó are similar to year’s past.

We believe the insights gained in these results ó in light of the above limitations ó certainly outweigh most concerns about the relevance and applicability of this data. Clearly, some interesting trends and perceptions are emerging. This group of respondents represents a variety of different job titles and areas of hiring and recruiting. As can be seen in Figure 1, responsibility for online hiring systems and locations of this expertise may be found in all parts of an organization. The most common job titles were recruiter (22%), an HR/staffing manager or executive (31%) or HR generalist (10%). Others included various types of consultants, managers and executives. Figure 1. Job Titles of Respondents The size of the company described in this data varied quite a bit as well, as shown in Figure 2. Figure 2. Number of Employees in Organization Consistent with previous years’ results, companies of all sizes are involved in online screening and assessment. The largest groups were comprised of companies with between 51 and 500 employees (23%), large companies (28%), and those with 501-2,500 people (21%.) Small companies (1-50 employees) represent nearly 17% of the sample. Possibly more relevant to trends in online assessment than mere company size, we asked respondents to estimate the number of individuals their organization hired in a given year. Keep in mind that this data does not represent the number of applicants screened or assessed nor the number of resumes received in a given year. Rather, Figure 3 details actual placement activity. Figure 3. Number of Hires per Year In this year’s study, a majority of respondents (66%) made 500 or fewer hires per year. However, the rest (nearly 34%), hired more than 500, which represents an increase over last year’s data. Specifically, in the previous survey over 70% of respondents hired 500 or fewer new employees, with 60% hiring 150 or fewer. This slight trend upwards in hiring reflects a stronger economy and the upward tick in hiring reported by various research outlets. Looking deeper into this aspect of the data, it seems that hiring trends, essentially, reflect the size of the organization. Specifically, larger organizations are making a greater number of hires, with a commensurate lower amount of activity in smaller companies. In summary, our sample ó like previous reports ó reflected the great diversity of practitioners out there. Not only are large companies adopting online screening and assessment tools, but small companies are in the game as well. Using an ATS As with previous incarnations of the present study, the majority of companies use an applicant tracking system (ATS) to manage, track, and communicate with job candidates as well as link data with an HRIS or other technology. Specifically, 68% of companies are already using one, with 10% in the process of installing ATS technology. As can be seen in Table 1, when ATS usage is compared to company size, some trends emerge. Perhaps obviously, company size relates to use of an ATS. Table 1. Size of Company and use of an ATS

A similar trend emerges when ATS usage is mapped according to the estimated number of hires made by an organization. As with company size, it makes sense that the results presented in Table 2 reflect the volume of hiring activity and, consequently, the need for an ATS. Table 2. Number of Hires per Year and Use of an ATS

As we noted last year, use of applicant tracking systems is mature, with adoption of such systems a standard part of an HR operation today, in the early 21st century. Companies of all sizes are using ATS technology, and the generally high percentages indicate that ATS usage is fairly mainstream. The benefits of an ATS are fairly obvious to most organizations, particularly the low cost of entry, mature product offerings by leading vendors, and the salient benefits compared to a paper-based approach to managing a company’s hiring process. As part of the background and organizational context assessed in order to understand the use of online hiring systems, survey participants were asked: “Overall, what percentage of your company’s hires over the last two years would you consider to be ‘highly successful’ in terms of the employees’ subsequent performance and tenure?” No clear trend emerged, and, as shown in Table 3, a different trend (or lack thereof) is revealed when current use of an ATS is juxtaposed to perceptions of hiring success, as shown in Table 3. Table 3. Perceptions of Hiring Success and Use of an ATS

Overall, it appears that the use of an ATS is fairly standard. In addition, the fact that perceptions of hiring success have little to do with installation of an ATS makes perfect sense. The automated tracking of job seekers and employees effectively really shouldn’t have an effect on perceptions of quality hiring, as use of an ATS alone does not, in general, involve any evaluation of candidates. Using Online Screening Tools Some of the most important aspects of this study were to determine 1) the general rate of usage of automated screening, 2) specific types of tools in use; and, 3) whether prescreening tools are perceived as effective for the organization. First, it is probably necessary to define what is meant by “prescreening.” For purposes of this survey (and in line with an emerging, accepted distinction between different steps in a typical hiring process), prescreening tools are defined as: Tools that gather information about, or ask candidates to respond to questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process. In contrast, for the purposes of this survey online assessment tools (discussed in more detail in part 2 of our survey results to be released next month) were defined as “scientifically based screening tools that look more deeply into a candidate’s abilities, interests, and skills.” These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process. Types of Tools in Use Reflecting little change from last year, 61% of respondents indicated that they use a form of online prescreening tools to sort job candidates early in the hiring process. The table below captures the usage rates of each of the most common forms of prescreening. Table 4. Primary Type of Automated Screening Tool in Use

As can be seen in Table 4, qualifications screening was the most common form of automated prescreen in use. Again, as with previous surveys, usage rates for each of these tools are quite consistent. Part of this trend seems to be the relative simplicity with which basic job (or stated “minimum”) qualifications (such as education or years of experience) may be collected and evaluated. In general, qualifications screens don’t involve any complex scoring algorithms or reports; rather, they simply take the place of a stack of paper or notation in a database, making it easier to sort and move candidates through the recruitment and hiring process. The format and context of prescreening varies somewhat as well. Specifically, survey respondents were asked how their prescreening is provided. The following table indicates that these tools are usually part of an ATS or candidate management system, with few organizations utilizing a standalone tool/software or melding the prescreen with a more intensive form of assessment. Table 5. Who provides the prescreening functionality in the organization?

As can be seen in Table 6, the extent of the deployment of prescreening technology varied quite a bit as well, with some companies utilizing these tools for all jobs while some using it for one or two alone. Table 6. How is prescreening deployed in the organization?

It’s clear that the variation in the use of prescreening technology is wide, with 50% of companies indicating that the placement of individuals into either all domestic or all global jobs involves these tools. Although we did not delve into the planned extent of use or length of time prescreening has been deployed, it’s apparent from the data in Table 6 that some organizations have firmly entrenched prescreening into hiring systems, whereas some have selectively implemented these tools or are in the process of expanding their reach. Effectiveness of Screening Tools A key goal of this series of studies has been to document the formal evaluation of the effectiveness of these tools as well as the feelings people professionals have about their use. Therefore, we asked respondents to indicate their opinions about the effectiveness of different types of tools and tell us about any metrics they collect, track and utilize to evaluate their use. Although it is common for HR and recruiting organizations to track a variety of success indicators and metrics as part of an HRIS or ongoing reporting, a minority of companies in our survey actually evaluated the use of prescreening technology. Specifically, only 27% of users of prescreening tools collected metrics, 9% were unsure if the organization collected them, and 64% do not. This is similar to last year’s results, where we found that 60% of organizations were not formally evaluating their automated hiring tools. Of those organizations who did evaluate the effectiveness of their prescreening tools, specific metrics and indicators collected varied from the conventional (e.g., time to hire, tenure, adverse impact) to the scientific (e.g., validation studies). In addition, many solicited feedback in the form of opinions and impressions of hiring managers and applicants. Moreover, most companies who spent time assessing their hiring practices utilized several different metrics, studies, and opinions in their continued evaluation of their prescreening tools. Impressions of the usefulness of prescreening tools varied as well. Reflecting a small drop from last year’s results, only 41% of users felt their prescreening tools were effective. Nearly as many (37%) were not sure if their tools were effective, and the remaining 22% responded that their use of this technology was not effective. The distinction between organizations that collect metrics and evaluate the success of their hiring tools versus those that do not sheds light on these perceptions. Table 7 reveals this contrast: Table 7. Perceived effectiveness of prescreening tools vs. collection of metrics.

Clearly, collecting metrics and analyzing results helps to answer the question of effectiveness and also appears to lead to impressions of success in the use of prescreening tools. This is not particularly surprising, as formal, meaningful data should make it easier for hiring professionals to judge the success of any staffing intervention. Moreover, it is not unexpected that companies who methodically judge their hiring interventions should find them to be effective; scientific evaluations of recruitment, staffing, and hiring systems often reveal positive results, especially compared to not using any formal assessment or leaving hiring decisions subject to the informal impressions of individual hiring managers. In addition, direct evaluation of the effectiveness of hiring tools provides feedback on what works and what changes need to be made ó more easily leading to improvements to those systems. In Part 2 of this report, we will further investigate the perceived results and value of prescreening as well as more in-depth online assessment tools. Overall Trends Looking back over these results, a few trends and tendencies characterize the world of online hiring:

• The use of applicant tracking systems is ubiquitous, with a very high percentage of organizations adopting or soon-to-adopt an ATS.
• The use of an ATS does not appear to relate to perceptions of hiring effectiveness.
• Hiring activity does seem to have increased in 2004 and early 2005, as reported by various government reports and industry surveys.
• A majority of organizations are using some form of automated prescreening to sort applicants, eliminate candidates from the hiring pool, and move qualified candidates forward in the hiring funnel. Nonetheless, these tools are not as common as an ATS.
• A significant number of people professionals seem to feel that their use of prescreening tools is not effective for the organization.
• Few companies formally assess their prescreening tools or use metrics of any kind as a decision making aid. Those who do tend to be aware of the effectiveness of their hiring systems and perceive greater value in them.
• Adoption of prescreening tools is still in its infancy, with a lack of understanding of their value as well as tentativeness and excessive worry about their effect on hiring success.

Clearly, the world of online recruitment and selection is widely varying and continually evolving. We hope that presenting the activities and impressions of your colleagues and competitors adds value to your own use of hiring technology. In Part 2, we’ll take a look at the use of in-depth online assessment tools and the climate of use of both screening and assessment tools. Specifically, we’ll detail the extent of use of tools such as personality inventories and skills tests. Next, we’ll report on the types of tools that companies are considering using, and how they plan to choose these systems. Finally, the obstacles faced by people professionals in the adoption of such tools are discussed, and we ponder the future of online screening and assessment in light of overall data trends and these perceptions and obstacles.

About Mark C. Healy Mark C. Healy (MarkCHealy@aol.com) is a consultant, teacher, and writer based in Oakland, California. He assists companies with the development of their human capital strategy and designs hiring, leadership development, and training programs. His recent clients include Genentech, Market-Up, and Wachovia Bank. He is affiliated with Integral Talent Solutions of Palo Alto, California, and Rocket-Hire in New Orleans, Louisiana. Mark has published articles in journals such as Personnel Psychology and Human Performance and has presented research at conferences of the Society for Industrial/Organizational Psychology and the Decision Sciences Institute. He teaches General and Business Psychology at De Anza College in Cupertino, California. Mark received an MA in Industrial/Organizational Psychology from The University of Akron.