Annual Online Screening and Assessment User Survey Results, Part 2

Towards the end of 2003, I invited ERE readers to participate in my second annual Online Screening and Assessment User Survey. The goal of this year’s survey was to pick up where last year’s survey left off by continuing to identify important trends in the usage of online screening and assessment tools and by providing data confirming that usage rates for online screening and assessment tools are on the rise. I’d like to say thank you to all of the ERE readers out there who were kind enough to spend some of their valuable time completing my survey. Thanks to you, I was able to collect a wealth of information. In fact, the survey provided so much good information that we had to present the results in two installments. To provide a quick recap, a total of 78 persons responded to my survey. They held a variety of staffing-related jobs and were spread relatively evenly across organizations of all sizes. Their responses revealed the following trends:

• Almost all organizations are using the web as a part of their staffing process.
• Hiring trends reveal that fewer hires are being made by companies of all sizes, but the slowdown is especially prevalent amongst mid-sized companies.
• ATS usage has become widespread amongst companies of all sizes. Almost all medium and large companies are using some form of applicant tracking software.
• There is little to no relationship between ATS usage and success of hires.
• Even though metrics are a critical for understanding the ROI associated with your hiring process, few companies are using them. Those that are using them seem to feel more positive about the effectiveness of their hiring processes.
• Screening tools are used by many organizations, but their usage is still not as widespread as that of other technology-based staffing tools, such as ATSs and employment portals.
• Many organizations report that the screening tools they are using are not effective. Screening tools that provide a more in-depth look at a candidate relative to job requirements are more effective than “on the surface” tools such as resume reviews.
• People are still passing over dollars to pick up pennies by refusing to expend the effort needed to properly configure and analyze the effectiveness of their screening tools.

With this information as a backdrop, it is now time to dig into the data for Part 2. Defining Online Assessment Tools In order to collect more detailed information about screening-and-assessment-related trends, this year’s survey provided separate questions for screening and assessment. In order to help ensure the collection of usable data, we provided operational definitions for both of these terms. In the survey we defined online screening tools as:

Tools that gather information about, or ask candidates to respond to, questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.

We defined online assessment tools as:

Scientifically based screening tools that look more deeply into a candidate’s abilities, interests, traits, and skills. These tools include personality measures, cognitive tests (e.g., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.

This section deals with trends related to the use of online assessment tools as defined above. The data revealed some very interesting results about the current usage of these tools. Current Usage of Assessment Tools Sixty-seven percent of respondents indicated that their organization uses some form of online assessment tools. This represents a slight increase over last year’s results. However, this is merely an anecdotal observation. There is no way to verify the magnitude of this increase since last year’s survey provided a slightly different definition and utilized a different respondent sample. Thirteen percent of respondents indicated that they use assessment tools, but only in paper and pencil format. When examining the usage of online assessment tools by size of the organization, a clear trend emerges. Usage rates are somewhat similar for all organizations of 5,000 employees or less. For instance:

• 15% of organizations with under 50 employees indicated using assessment tools
• 23% of organizations with 50-499 employees indicated using assessment tools
• 17% of organizations with 50-5,000 employees indicated using assessment tools
• 44% of organizations with over 5,000 employees indicated using assessment tools

This confirms the fact that larger companies are the biggest consumers of online assessment tools. The sample had approximately even numbers of respondents across all ranges of organizational size, so I feel confident that this trend is not an artifact of the sample (i.e., there were not a disproportionately large number of big companies in the sample). Only 13% of respondents currently using ATS systems and online assessment tools indicated that they have integrated assessment tools into their ATS system. This information indicates that assessment is still a relatively standalone component of the hiring process. Data on the manner in which assessments are deployed helps to support this conclusion. For instance, The data in Table 1 indicate that assessment tools are still deployed at a local, job-specific level rather then globally across many jobs within an organization. Table 1: Deployment of Assessment Tools

The data in this table demonstrate the clear fact that most organizations use assessment tools locally for specific jobs rather than as a core component of a strategic staffing program. This is consistent with the fact that these tools are still new to many organizations and are often adopted as part of an effort to fight fires rather then as part of a strategic fire prevention strategy. This indicates a huge growth potential for these tools, especially given the high percentage of respondents who indicated using assessment tools. However, realizing this growth potential will require using local implementations as case studies that can be used to demonstrate the value of a more broadly based assessment program. Unfortunately, most organizations are not currently following this type of strategy. The survey also collected data related to the specific type of assessment tools that are currently being deployed. The data in Table 2 summarizes this information and provides a summary of the data collected from my 2002 survey to help provide information on increases in usage. Table 2: Type of Assessment Tools Currently in Use

These data reveal that, except for skills assessments, which seem to have experienced a large increase in usage, the type of assessments being used have remained relatively stable over the last few years. It is important to remember that the samples differed between these two years, a fact that somewhat limits the value of these direct comparisons. The data also revealed that most companies who are using assessment tools used more than one tool. In fact, 67% of those using assessment tools listed above used more than one of these tools. This is consistent with last year’s results, indicating that very few organizations using assessment tools used only one type of assessment tool. The data do not indicate if these tools are used together as part of the hiring process for one specific job or if they are used separately for different jobs. The data in Table 3 provide an overview of the job levels for which online assessment used. Table 3: Assessment Usage by Job Level

These data indicate that online assessment is most commonly used for lower level or less complex jobs. There is a clear trend in the data indicating that as the complexity, and thus the value of the job, goes up, use of online assessment tools declines. This is consistent with the fact that higher value hiring decisions often include a longer hiring cycle with more in-depth assessment measures. The fact that a large amount of performance at most simpler jobs can be boiled down to a few key factors and the fact that lower level jobs will have the highest applicant volume make these types of jobs a perfect candidate for the use of online assessment tools. This is because these tools can quickly and remotely assess key competencies while also providing the organization with tools needed to manage applicant data. The data in Table 4 summarizes the percentages of responding organizations who are currently using assessment for specific job types. Table 4: Online Assessment Usage by Job Type

These data indicate that customer service, sales, call center, and admin jobs see the most common use of assessment tools. This is consistent with the information in Table 3 indicating that less complex jobs that have a few key performance factors seem to account for the bulk of online assessment usage. Given the large number of assessment tools available and the fact that assessment tools are useful for all types of jobs, not just lower level ones, there is huge growth potential for the use of assessments to make managerial/professional level hiring decisions. Effectiveness of Assessment Tools A look at respondents’ perceptions of the success of their hiring process provided some very interesting results. Table 5 summarizes respondents’ perceptions regarding the effectiveness of their hiring process. This table summarizes these perceptions for both companies that indicated using assessment and those that indicated that they do not use assessments. Table 5: Assessment Usage and Perceptions of Hiring Success

These results do not seem to indicate any significant trend demonstrating that there is a direct relationship between the use of assessment tools and perceptions of successful hiring. But before attempting to draw any conclusions from this data, it is important to understand that respondents were asked to indicate their perceptions of the effectiveness of their overall hiring process for all jobs rather than their perceptions of success related to the use of specific assessment tools. This fact makes a difference in the interpretations that can be drawn from this data. Also, the data on assessment usage clearly indicate that assessments are most commonly used for specific, local jobs. It is important to note that the survey question of interest here asked about perceptions of effectiveness for the overall hiring process, a fact that limits generalizations about the data in Table 5. Despite this, I think it is important to share this data because I feel it helps demonstrate the fact that companies are not taking the time to evaluate the effectiveness of their hiring processes. This is especially true when it comes to the evaluation of assessment measures. Survey data regarding respondents’ use of metrics to evaluate their assessment process help to reinforce this conclusion. For instance, only 42% of respondents indicated that their companies collect metrics on the effectiveness of assessment tools. This is consistent with the data presented in Part 1 indicating that only 40% of companies collect metrics to evaluate the effectiveness of their screening programs. It is clear that companies who do not collect metrics are missing out. For instance, survey data indicate that 86% of respondents whose companies use assessment and collect metrics felt that these metrics have been useful in helping to build a business case for the continued or expanded use of assessment tools. These data clearly highlight a trend I discussed in Part 1 of my survey results, which is that companies simply aren’t making the effort needed to collect evidence demonstrating the effectiveness of screening and assessment tools and that collecting evidence makes a difference when it comes to understanding the value of online assessment tools. Until companies can demonstrate a clear linkage between the use of these tools and ROI, it will be difficult for them to justify the use of assessment. For those readers interested in gaining a better understanding of what type of metrics companies are using to evaluate the effectiveness of their assessment tools, open-ended survey data indicated the following as the most popular type of metrics collected:

• Performance data
• Retention data
• Sales figures
• Customer service behaviors
• Statistics related to call handling (call center jobs)

Finally, 82% of respondents indicated that they felt the use of screening or assessment has had a positive impact on their organization. In my mind, when viewed along with data indicating the lack of assessment related metrics, this data provides clear evidence of a gap between the perceived value of assessment and the organization’s ability to capture this value in concrete terms. In order to fix this situation, organizations need to begin closing the gap between the perceived value of assessment and its actual applied value. Future Usage and Obstacles Survey results indicate that 60% of those not currently using screening assessment tools felt they would use these tools in the future. Table 6 provides a summary of the specific types of tools being considered. Table 6: Tools Under Consideration

These results are very consistent with the trends in usage over the past few years with qualifications screening, skills assessment, and personality assessment representing some of the most popular tools. The results in Table 7 summarize the manner in which respondents feel they will decide on which specific screening and assessment tools they will use. Table 7: Selection Methods

These results seem to indicate that there is no one method that is clearly used most; however, it seems that the formal RFP process is a popular option. Ninety-one percent of respondents indicated that they face obstacles to the adoption of screening and assessment within their companies. Table 8 summarizes some specific obstacles and the percentage of respondents indicating the presence of a particular obstacle within their organizations. Table 8: Obstacles to the Adoption of Screening and Assessment Tools

These results are consistent with the major trends in the adoption of screening and assessment that I have been discussing over the past few years and are almost identical to the data collected last year. Specifically, organizations are still firmly in the habit of passing over dollars to pick up pennies. Lack of understanding regarding screening and assessment tools and their potential upside in terms of adding ROI via a focus on applicant quality are still the norm. The complexity of these tools and confusion around their correct application is continuing to present a serious obstacle to the adoption of these tools. The results of my survey help to develop a profile for the use of online assessment tools. I feel this profile is as follows: The most common user of assessment is a very large organization that uses assessment locally for specific lower-level, high-volume jobs. These assessment tools are not integrated into the rest of the hiring process, but rather are used in a standalone fashion. Most consumers of assessment tools are probably using skills or personality assessment tools in a reactionary manner to help them solve a specific problem related to the job performance associated with these jobs. The organization probably does not collect metrics to evaluate the effectiveness of their assessment initiatives. Those organizations that are collecting metrics are much more likely to be able to see the value that these tools can provide. Beyond this, survey data related to the usage of assessment tools provides the following high level information: Assessment usage is slowly rising, but it is still not as widespread as that of screening. The fact that assessments are being used mostly for less complex jobs and are deployed locally reflects fact that assessment is being used in a reactionary manner to help put out fires or stop the bleeding rather than as a strategic part of the staffing process. This mindset is not going to help any organizations to build a winning talent acquisition strategy. I feel that companies should be using assessment to help bake quality into their hiring processes instead of gunning for a quick fix. There is a large gap between the perceived value of assessment and company’s ability to clearly demonstrate this value. As long as this gap exists, the adoption rate for assessment tools will be relatively slow. Information regarding obstacles related to the adoption of screening and assessment provides a pretty clear indication that the cause of this gap is a lack of understanding regarding how assessment works and how it can create value as well as the failure to establish programs to gather the metrics needed to effectively demonstrate ROI. Overall Trends Identified via Survey Data Taken together, the data discussed in Part 1 and Part 2 of this series reveal some very interesting trends about the current state of online hiring. Collectively, survey results provide a very clear picture of our current evolutionary state when it comes to the use of technology in the hiring process. Specifically, it seems that by now almost everyone is using the web to source candidates (although many companies still have a lot to learn about how to create a good web based branding and sourcing strategy). It also seems clear that ATS usage has moved well past the early adopter stage and is approaching a saturation point. At the present time companies are finding that the value of the ATS is beginning to shift from one of managing candidates to that of helping them focus on determining candidate quality. People are beginning to realize that a good website and an ATS will only take them so far. The ability to evolve the hiring process is going to require measures that help companies impart quality control into their hiring processes. I firmly believe that the next step in the ongoing evolution will be an increased focus on the adoption of quality focused tools such as screening and assessment measures. We currently stand at what I feel is still the early adopter phase when it comes to online screening and assessment tools. Survey results also indicate that most companies do not yet understand how to build a quality-focused hiring process. Organizations do not understand the role of assessment tools in helping them to turn the hiring process into a profit center. Instead, they are still fixated on the upfront costs associated with using screening and assessment tools. It is clear that the understanding of profit-based human capital decisions still lags far behind that of other types of capital expenditures. Until firms bite the bullet and make an initial investment in quality, this situation is not going to change. Those that are willing to make the investment now will find themselves with a distinct competitive advantage in the near future. At the present time, one of the biggest barriers to a quality-focused hiring process is education. The creation of a quality-focused hiring process requires the strategic use of scientific tools. Much of the knowledge required to execute a quality strategy is complex and lies outside the core competencies of most HR departments. This can be a difficult obstacle to overcome. In my opinion, education and experience are the way to break through barriers related to understanding. The best way to learn about how to use tools that provide a focus on quality is to begin using them. While the learning curve may be steep at first, efforts in this area can help organizations create a knowledge base that can provide a legacy as the organization moves forward with a continued focus on using human capital to increase profits via a more efficient and effective execution of the strategic initiatives that define their existence.