article by Mark Healy & Charles Handler
Organizations have been deploying pre-employment tests and assessments on computers for at least two decades. Originally just pencil-and-paper tests adapted to a computer interface, modern hiring and employee development tools have surmounted difficulties with data capture and reporting, user access, and database management to bring a new level of technical sophistication to recruitment and hiring programs. Now, six years into the 21st century, human resources and recruiting departments are keenly focused on the dispersal of postings, applications, and assessment tools to job applicants via the Internet. This revolution in the way companies recruit and hire people is moving quickly. So for the third straight year, I’ve asked users of online screening and assessment technology about their current and future use of web-based hiring systems.
This year we were able to collect data from 90 hiring professionals. The data offered us an excellent snapshot of current organizational strategies, concerns, and insights around the use of technology based screening and assessment tools.
In Part 1 of this article series on our findings, we detailed current usage rates and feelings about screening systems such as online job applications, applicant tracking systems (ATS), qualifications screening, and resume scanning. Moreover, we investigated trends in tracking the effectiveness of these tools, and how organizations could do a better job of understanding the systems they have in place. Here, in Part 2, we dive deeper into automated hiring processes by considering the current and future use of online assessment tools.
Assessment vs. Screening
Although definitions vary, the term “assessment” has been distinguished from “screening” in today’s human resource environment. For the purposes of our survey — and in alignment with this trend — we defined online assessment tools as: “Scientifically-based screening tools that look more deeply into a candidate’s abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.” In other words, assessment tools evaluate job candidates much more thoroughly than prescreening devices, and are often used once an initial hurdle or multiple hurdles have been successfully passed by applicants. Assessment tools tend to evaluate the level or quality of a skills and abilities that are essential for job performance rather than the extent to which an applicant’s experience qualifies them to perform a specific position. As in past years, our survey revealed some very interesting information about the use of assessment tools. The remainder of this article provides a summary of these results.
The Use of Assessment
Of our 90 respondents, 68 companies (76%) utilize assessment tools — either online or in traditional paper and pencil format. Of these, 50% deploy one or more tools online, either in a proctored environment or over the Internet for access from any web-enabled computer. Organizations surveyed varied quite a bit by size, but this — and number of hires per year — had no bearing on whether or not a company used assessment tools. This is in contrast to last year’s sample, when organizations with 5,000 or more employees used assessment at much greater rate than smaller companies. The ubiquity of applicant tracking systems has engendered a slight increase in the rate of companies who integrate assessments into these common HR platforms. Specifically, 22% of companies using assessment integrate candidate results into their ATS, compared with 13% last year. This generally low rate of assimilation (under 25%), however, is not really surprising; although use of an ATS has become standard in most large HR operations, the full capabilities of such systems are not typically utilized. The breadth of assessment use throughout an organization varied quite a bit across the study as well.
As can be seen in Table 1, the typical organization deploys assessment tools as part of the hiring strategy for only a subset of jobs. Only 19% utilize online assessment for all jobs worldwide, and an additional 20% use assessment for all domestic jobs. These results are similar to last year’s findings, where 37% of organizations used assessment either world- or country-wide.
| All jobs within a business unit, but not all business units | 4% |
| All domestic jobs | 20% |
| All worldwide jobs | 19% |
| Specific local jobs only | 41% |
| Not sure | 17% |
For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.
| Entry level/hourly | 46% |
| Lower level management/professional | 49% |
| Middle level management/professional | 43% |
| Higher level management (director) | 37% |
| Executive/vice president | 26% |
| Other | 7% |
The results of this year’s survey suggest that this is one aspect of online assessment that has evolved. The in-depth evaluation and comparison of pre-qualified candidates using online assessment tools is no longer relegated to entry-level positions or technical professionals. At only the senior leader level does use remain under 30%; this is not unexpected, given the one-on-one, grassroots nature of executive recruiting and promotional systems. Unlike job level, rates of use for different types of job settings appear to vary considerably. Table 3 reveals these distinct trends.
| Customer service | 41% |
| Manufacturing/labor | 12% |
| Skilled trades | 35% |
| Account management | 34% |
| Call centers | 35% |
| Managerial/supervisory | 46% |
| Administrative | 35% |
| Information technology | 43% |
| Retail | 12% |
| Consulting/advising | 19% |
| Sales | 35% |
| Professional | 46% |
| Other | 26% |
These results reflect not only the sample of survey respondents, but a certain level of maturity in the development and implementation of assessment in the early 21st century. For example, use of online assessment tools for managerial, IT, and customer service hiring is common in part because so many different cognitive ability tests, personality inventories, and other in-depth assessments have been developed specifically for these sorts of jobs. Moreover, validated assessments of key skills and competencies required for success in these roles have been available for years in paper-and-pencil format. Not insignificantly, organizations tend to imitate others in their industry, so a bit of a “snowball” effect is reflected here as well. Completing an online assessment has become relatively standard for customer service, managerial, professional, and IT jobs, but manufacturing and retail hiring situations are still commonly handled via face-to-face evaluation of local, “walk-in” applicants. Online assessment of these candidates is still in its novel phase of evolution.
Types of Assessment Tools in Use
The rates of usage of various types of tools by organizations employing an assessment strategy provide insight into the growth of this area, especially when compared to our previous surveys. Although this year’s sample of people professionals represents a largely different group of individuals compared to last year, the substantial increase in the adoption of most types of tools is difficult to put aside. Even more telling, Table 4 displays the wide disparities among the rates of use of different types of assessments.
| Type of assessment | 2005 | 2003-04 | 2002 |
|---|---|---|---|
| Personality measures | 34% | 30% | 21% |
| Assessment of “fit” with company | 35% | 27% | 29% |
| Cognitive ability | 46% | 27% | 26% |
| Biodata | 15% | 10% | 14% |
| Skills/knowledge | 53% | 40% | 12% |
| Background investigations | 54% | 30% | 31% |
| Simulations | 18% | 14% | 10% |
| Online interviews | 15% | 4% | 19% |
In line with previous surveys, assessments of background, specific skills and knowledge, and cognitive ability are becoming routine. However, this year’s results signify a substantial jump in the level of adoption of these instruments. Biodata inventories, online interviews, and simulations continue to be less often used. Moreover, the most common types of assessments realized the largest gains in usage — particularly skills/knowledge assessments, background investigations, and cognitive ability tests.
Effectiveness of Assessment Tools
With the wider implementation of these tools, are companies finally realizing the value of structured, validated assessments to their people strategy? Table 5 displays overall responses to, arguably, the most important question surrounding the implementation of any people strategy or tactic. As can be seen below, whether an organization actually attempts to answer this question is critical to the eventual answer.
| Does assessment have a positive impact on your organization? | Yes | No | Not sure |
|---|---|---|---|
| Overall | 70% | 7% | 22% |
| Organizations collecting metrics | 88% | 0% | 12% |
| Organizations who do not collect metrics | 64% | 16% | 20% |
As with previous years, and with just about any people program or strategy, less than one-third (specifically 28%) of the companies who use assessment formally collected metrics to determine if their assessment strategy adds value to their organization. Of those that do, there seems to be strong confirmation of the value of assessment. Those who do not, in some cases, have determined that assessment does not positively impact their company, even if they have not directly measured its effect. These results match those for online screening tools, as detailed in Part 1 of this piece. Since the collection of metrics does seem to contribute to perceptions of value to the organization, we also asked if the collection and presentation of metrics help to make a business case for the continued use or expansion of assessment tools. According to Table 6, the majority of companies collecting metrics are able to use them to build a business case for the use of assessment.
| Yes | 71% |
| No | 11% |
| Not sure | 18% |
An evaluation of a company’s hiring system — or just parts of it — may involve a variety of different metrics and measures that consider “success” from a variety of angles. We asked survey respondents to indicate the metrics they use to evaluate the impact of their assessment systems. Typical measures include:
- Comparisons of assessment scores versus tenure and turnover
- Traditional validation studies, including statistical analysis of the relationship between prescreen and assessment scores and job performance measures
- Adverse impact and diversity indicators
- Standard HR metrics, such as time-to-fill position and cost-per-hire
- Formal and informal impressions and opinions of managers, HR and recruiting staff, and job candidates
These vary in terms of effort required and the time span required for data collection. Still, these strategies may be systematized and applied to all hiring strategies in the organization, and people professionals can easily build expertise in this area as use of metrics permeate their hiring processes.
The Future of Online Screening and Assessment
Despite the expansion and pervasiveness of online screening and assessment, many organizations have not yet begun their implementations, or even their purchase, of these tools. However, of those companies not currently using screening or assessment instruments, nearly three-quarters of them (73%) feel their organization will adopt them in the future. For these potential adopters of online hiring technology, Table 7 summarizes the sorts of tools under consideration.
| Type of screening/assessment | Percent considering use |
|---|---|
| Resume scanning tools | 26% |
| Qualifications (experience, education, etc.) | 46% |
| Personality measures | 23% |
| Assessment of “fit” with company | 34% |
| Cognitive ability | 17% |
| Biodata | 11% |
| Skills/knowledge assessment | 60% |
| Background investigations | 34% |
| Simulations | 17% |
| Online interviews | 23% |
| Don’t know | 6% |
Consistent with previous surveys, and with the large market space filled by the many competitors for your business, qualifications screening, assessment of cultural fit, skill/knowledge assessments, and background investigations dominate the interest of companies seeking to dip their toe into the ocean of online hiring tools. Companies will consider, compare, and purchase these systems using a variety of techniques. Table 8 details these methods for selecting recruitment and hiring technology.
| Shopping and purchasing method | % using method |
|---|---|
| Formal RFP process | 26% |
| Informal decision making | 18% |
| Recommendation from consultant | 6% |
| Via partnerships/services from existing vendors | 16% |
| Don’t know | 26% |
| Other | 8% |
Clearly, few trends emerge, with formal requests for proposal (RFP) the most popular — but by no means majority — decision-making tool. So what is keeping many organizations from adopting online screening and assessment tools or expanding their role in recruiting and hiring?
Obstacles to the Use of Online Screening
Greater usage of online prescreening and assessment would occur if it wasn’t for the obstacles and limitations perceived by organizational leadership and people professionals. These anxieties ó whether grounded in reality or not — present real barriers to the growth and enhancement of hiring success in every type of organization. Presented in order of endorsement, Table 9 highlights the concerns held by organizations seeking to adopt hiring technology.
| Too costly; lack of budget resources | 27% |
| Lack of knowledge in organization | 26% |
| Skepticism about the ability of screening to provide results | 22% |
| Decision makers do not believe it is worth the cost | 17% |
| Hesitation due to legal issues | 12% |
| Hesitation due to security issues | 11% |
| Tools will negatively impact the candidate experience | 11% |
| No obstacles | 9% |
| Technology is still too new | 9% |
| HR not interested in innovation; reluctance to change | 8% |
Another way to consider these feelings is to look at the “single biggest obstacle” perceived by survey respondents. For the most part, the data in Table 10 reflect the results in the previous table.
| Too costly; lack of budget resources | 19% |
| Skepticism about the ability of screening to provide results | 16% |
| No obstacles | 14% |
| Tools will negatively impact the candidate experience | 11% |
| Technology is still too new | 11% |
| Decision makers do not believe it is worth the cost | 7% |
| Lack of knowledge in organization | 7% |
| HR not interested in innovation; reluctance to change | 7% |
| Hesitation due to security issues | 5% |
| Hesitation due to legal issues | 4% |
In general, these concerns center around:
- The value of spending money and time on more advanced assessment
- Candidate reactions to hiring tools
- Lack of knowledge regarding the positive implications of assessment tools
With 14% indicating no obstacles, are these legitimate concerns or merely anxieties experienced by inexperienced, paranoid, or jaded decision makers? To answer that question and others, we will, in Part 3 of this article series, delve into a deeper analysis of the overall findings and general implications of these results.