Article and research by Charles Handler and Mark C. Healy
For the last three years, Rocket-Hire has surveyed ERE readers who use Web-based screening and assessment products as part of their hiring process. The information we have been able to gather from this research has been a valuable part of our efforts to track important trends in the use of screening and assessment tools.
Thus, earlier this year, we again asked people professionals to tell us about their use and opinions of these tools. This brief report highlights the major findings of our fourth annual survey.
Please visit the ERE article archives to view the results of the previous three years’ surveys. The full version of our survey results will be available in mid-November; contact us at info@rocket-hire.com and we will be sure to email you a copy.
The information below provides an overview of the data we used to achieve our objectives, which included the following three goals:
- Understand the general rate of usage of Web-based screening and assessment
- Get information on usage rates for various types of tools
- Determine whether these hiring tools are perceived as effective for the organization
Who Responded?
This year, 136 people professionals completed the survey, representing an increase of 51% over last year. This increase is indicative of the fact that screening and assessment is continuing to gain momentum as a best practice among hiring professionals.
As we have seen in past surveys, respondents were evenly representative of recruiters, HR executives, and hiring managers, and represented a wide variety of organizations and hiring situations. Thirty-four percent hired more than 500 employees in a given year, but a majority of respondents (66%) made 500 or fewer hires per year.
ATS Usage
As with previous surveys, a majority of respondents use an applicant tracking system. Specifically, 72% already use one, with 8% in the process of installing applicant-tracking technology. As can be seen in the tables below, companies of all sizes are taking advantage of applicant tracking technology, and high rates of adoption indicate that use is fairly mainstream.
| Size of Company | % using or intending to use ATS |
| 1-50 | 56% |
| 51-500 | 71% |
| 501-2,500 | 82% |
| 2,501-5,000 | 85% |
| More than 5,000 | 100% |
This is important because the use of screening and assessment tools is often the next step in the creation of a technology-based hiring process. Thus, the higher the adoption rate for ATS, the greater the usage of screening and assessment tools will be.
Screening Tools
Strictly an administrative process, an applicant tracking system alone does not usually mean any in-depth evaluation of candidates. However, the following data on use of prescreening and assessment tools does underscore the growing importance of judging job applicants using systematic tools. For purposes of this survey, screening tools are defined as:
Tools that gather information about, or ask candidates to respond to questions about, their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.
Sixty-two percent of respondents indicated that they use at least one online screening tool to sort job candidates early in the hiring process. The table below captures the usage rates of each of the most common forms of screening.
| Type of Prescreen | % using as a primary automated prescreen |
| Resume-scanning systems | 15% |
| Biodata or personality questionnaires | 10% |
| Basic qualifications | 58% |
| Assessment of “fit” with company | 14% |
| Other tools | 3% |
As can be seen above, qualifications screening was the most common form of automated prescreen in use. Over the last several years, usage rates for each of these tools are quite consistent. Part of this trend seems to be the relative simplicity with which basic job (or stated “minimum”) qualifications (such as education or years of experience) may be collected and evaluated. As can be seen in Table 3, the extent of the deployment of prescreening technology varied quite a bit as well, with some companies using these tools for all jobs while some using it for one or two alone.
| Extent of Prescreening | % |
| All jobs within a business unit, but not all business units | 25% |
| All domestic jobs | 27% |
| All worldwide jobs | 19% |
| Specific local jobs only | 18% |
| Not sure | 11% |
Assessment Tools
In contrast to screening tools, online assessment tools were defined as:
Scientifically based screening tools that look more deeply into a candidate’s abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.
The overall usage rate of in-depth candidate assessment tools by organizations has grown when compared to the last few years. Although our current sample of people professionals represents a largely different group of individuals compared to last year, the substantial increase in the adoption of most types of tools is difficult to put aside. This trend alone is quite significant as it shows the growing increase in the popularity of these tools.
| Type of Assessment | 2006 | 2005 | 2003-04 | 2002 |
| Personality measures | 65% | 34% | 30% | 21% |
| Assessment of “fit” with company | 53% | 35% | 27% | 29% |
| Cognitive ability | 42% | 46% | 27% | 26% |
| Biodata | 16% | 15% | 10% | 14% |
| Skills/Knowledge | 66% | 53% | 40% | 12% |
| Background investigations | 65% | 54% | 30% | 31% |
| Simulations | 21% | 18% | 14% | 10% |
| Online interviews | 7% | 15% | 4% | 19% |
In line with previous surveys, checks of criminal background, specific skills and knowledge certifications, and personality inventories are becoming routine. However, this year’s results signify a substantial jump in the level of adoption of these instruments. Online interviews and simulations continue to be less often used. But how extensive are deployments of assessment tools across organizations?
| Variation in Deployment | % |
| All jobs within a business unit, but not all business units | 24% |
| All domestic jobs | 28% |
| All worldwide jobs | 8% |
| Specific local jobs only | 27% |
| Not sure | 13% |
As with prescreening, the variation in the deployment of tests and assessments is wide, with a total of 36% of users indicating that assessment is utilized in the placement of individuals into either all domestic (28%) or global jobs (8%). For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.
| Use Across an Organization | % |
| Entry level/hourly | 43% |
| Lower-level management/professional | 55% |
| Middle-level management/professional | 53% |
| Higher-level management (director) | 41% |
| Executive/Vice President | 26% |
Table 6 suggests that this aspect of online assessment has evolved. 2006 appears to be the first year that our data show a general pattern of use across all levels of an organization. But unlike job level, rates of use for different types of job settings appear to vary quite a bit. Table 7 displays these broad differences.
| Rates of Use for Different Types of Job Settings | % |
| Customer service | 41% |
| Manufacturing/Labor | 12% |
| Skilled trades | 24% |
| Account management | 32% |
| Call centers | 30% |
| Managerial/Supervisory | 53% |
| Administrative | 41% |
| Information technology | 38% |
| Retail | 12% |
| Consulting/Advising | 16% |
| Sales | 41% |
| Professional | 45% |
| Other | 22% |
As we noted last year, Web-based managerial, sales, IT, administrative, and customer-service assessments are common, in part, because so many different instruments and recruiting programs have been developed specifically for these sorts of jobs. Moreover, non-online, validated assessments of key skills and competencies required for success in these roles have been used for decades.
Effectiveness of Screening and Assessment Tools
A key goal of this series of studies has been to document the feelings people professionals have about the effectiveness of online recruiting and hiring systems. This year, 37% of prescreen users and 63% of assessment users felt their tools added value to their organization. The rest were evenly divided between either not having an opinion or feeling that Web-based hiring tools did not add value.
Understanding how these users have formed their opinions is of even greater interest. Apparently, these opinions have been formed through general impressions as opposed to actual evaluation, as only a minority of companies in our survey actually evaluated the use of hiring technology. Specifically, only 31% of users of prescreening tools and 30% of assessment users collected metrics to judge the quality of their interventions. Moreover, users of metrics tend to distinguish themselves from the majority. Tables 8 and 9 reveal this contrast:
| Are Prescreening Tools Effective? | Organizations collecting metrics | Organizations that do not collect metrics |
| Yes | 59% | 24% |
| No | 19% | 31% |
| Not sure | 22% | 31% |
| Did not answer | 0% | 14% |
| Does assessment have a positive impact on your organization? | Organizations collecting metrics | Organizations that do not collect metrics |
| Yes | 89% | 50% |
| No | 7% | 27% |
| Not sure | 4% | 23% |
Clearly, collecting metrics helps to answer the question of effectiveness and also appears to lead to impressions of success in the use of Web-based hiring systems. A variety of different metrics and measures are in use. Our respondents mentioned comparisons of assessment scores versus tenure and turnover; validation studies; diversity indicators; standard HR metrics (e.g., time-to-fill, cost-per-hire); and impressions and opinions of managers and job candidates.
The Future of Online Screening and Assessment
Of those companies not currently using screening or assessment instruments, 62% feel they will implement one or more in the future. For them, Table 10 summarizes the sorts of tools under consideration.
| Type of Screening/Assessment | % considering use |
| Resume-scanning tools | 24% |
| Qualifications – experience, education, etc. | 48% |
| Personality measures | 25% |
| Assessment of “fit” with company | 25% |
| Cognitive ability | 21% |
| Biodata | 5% |
| Skills/knowledge assessment | 32% |
| Background investigations | 22% |
| Simulations | 8% |
| Online interviews | 11% |
| Don’t know | 11% |
Consistent with experienced users, qualifications screening, assessment of cultural fit, skill/knowledge assessments, and personality inventories dominate the interest of organizations seeking to implement web-based hiring tools. But with such an optimistic outlook ahead, what is keeping organizations from adopting online screening and assessment tools?
Obstacles to the Use of Online Screening
Of those seeking to enter the world of online hiring, what is single biggest obstacle to the adoption of prescreening and assessment in their organization?
| Single-biggest obstacle to the adoption of online screening and assessment? | % |
| No obstacles right now | 22% |
| Lack of knowledge in organization | 17% |
| Too costly / lack of budget resources | 16% |
| Skepticism about the ability of screening to provide results | 13% |
| Decision-makers do not believe it is worth the cost | 10% |
| HR not interested in innovation, reluctance to change | 6% |
| Hesitation due to legal issues | 6% |
| Technology is still too new | 4% |
| Tools will negatively impact the candidate experience | 3% |
| Hesitation due to security issues | 3% |
The data in Table 11 are both encouraging and discouraging at the same time. It is very encouraging that the largest percentage of respondents indicated there are no obstacles at this time. This is a first and clearly shows an increasingly positive climate for the use of screening and assessment tools.
At the same time, it’s the same-old song and dance. Data suggest that typical concerns center around value of spending money and time on more advanced assessment, candidate reactions to hiring tools, and lack of knowledge regarding the positive implications of assessment tools.
Thus, in terms of skepticism about benefits and a lack of budget, companies continue to be penny-rich and dollar-poor when it comes to dedicating monetary and human resources to their recruitment and hiring processes. Cost of entry is low, and the value of the better assessment tools has been established through hundreds of studies.
Survey respondents also tended to mention candidate distortion of qualifications as the biggest threat to the success of their online hiring tools. But inflation of job skills has always been with us. Overall, most companies have few or no substantive reasons to limit their use of online recruitment and hiring techniques.
Summary
We offer the following as key findings that help clarify today’s trends in online hiring:
- Most medium-sized and large organizations have adopted or will adopt an applicant tracking system, but the use of an applicant tracking system does not relate to perceptions of hiring effectiveness.
- Some hiring tools, like qualifications screening, skills testing/certification and personality tests, are becoming standard features of a strategic hiring process. The use of some methods (e.g., online interviewing) is rare.
- Few companies formally assess their prescreening tools or use metrics of any kind as a decision making aid. Those who do tend to be aware of the effectiveness of their hiring systems and perceive greater value in them.
- People professionals report a lack of understanding, weak budgets, or a general lack of support for online tools as their primary obstacles to adoption or greater use of modern prescreening and assessment technology.
Overall, the results of this fourth version of our survey are the most encouraging yet. They clearly show a growing interest in these tools. Valid, Web-based approaches to recruiting, screening, and placing job candidates are fast becoming the norm at leading edge, people-focused organizations.
This trend is encouraging and, in our experience, not surprising. We feel these numbers will continue to grow as more organizations understand the value in properly evaluating the impact of screening and assessment tools.
Unfortunately, a major takeaway for us is that there is still a very widespread failure to adopt the business intelligence mindset required to evaluate the effectiveness of screening and assessment tools. We found a lack of understanding of the value these tools can return as well as budgeting issues the primary obstacles.
As long as this deficiency exists, it is not surprising that skepticism about the value of these tools remains strong. We will continue to champion the use of evaluation to “close the loop” and help ensure an increase in the understanding of how properly implemented screening and assessment tools can impact the bottom line.
(See next month’s Journal of Corporate Recruiting Leadership for expanded coverage of findings from the 4th Annual Rocket-Hire Online Usage Survey.)
Mark C. Healy also contributed to this article. Mark is a consultant, teacher, and business writer based in Oakland, California. He assists companies with the development of their human capital strategy and designs hiring, leadership development, and training programs. Mark teaches Industrial/Organizational Psychology at De Anza College in Cupertino, California.