Introduction to this series by Charles Handler Many readers know me through my monthly articles about online screening and assessment. What many readers may not know is that over the past three years I have been working to learn as much as I can about vendors of Internet-based staffing systems and the tools they are selling. My objective has been to keep abreast of innovations in this constantly evolving field, and to eventually create a report that categorizes and reviews all of the web based screening and assessment tools that are currently available. About a year ago, I met up with Dr. Steve Hunt, another I/O Psychologist and employee selection specialist. I soon discovered that Steve had been involved in similar activities. Like me, Steve had spent many years building staffing systems and helping to create some of the very first web-based assessment measures. He eventually left the business of developing and selling assessments to accept a position assisting a Fortune 500 retail organization with designing and implementing a corporate staffing assessment strategy. As a result of this work, Steve had also been conducting an extensive review of online staffing tools and systems. Steve shares my passion and vision regarding the impact that Internet technology is having on the staffing industry, and we quickly agreed that it made sense for us to actively collaborate. In fact, Steve and I are now just a month away from launching the “Rocket-Hire Buyer’s Guide to Web-Based Screening and Staffing Assessment Systems.” (You can contact me at chandler@rocket-hire.com for more information on the launch of this product.) The vendors who are exploring new uses for technology and creating a sea change in the staffing process represent an important driving force in the evolution of online staffing systems. We applaud these folks and wish to recognize what they are doing. It is for this reason that we approached ERE and asked them if we could do a monthly column in which we pass along information about some of the more innovative products we have come across. We are happy to report that ERE liked the idea and that we will be publishing a monthly article in which we profile a specific online staffing product and discuss why we feel it is worthy of attention. The article you are reading today represents the first article in what we hope will be a long running and informative series. Please note that neither Steve nor myself are affiliated with any of the vendors whose products we are profiling, nor are we compensated in any way for our decision to write about a specific product. CC Audition, by FurstPerson For our first article we have chosen to review CC Audition, a web-based call-center simulation tool offered by FurstPerson. We feel that simulations, when properly configured, are one of the most powerful tools available both for predicting job performance and providing candidates with a meaningful and positive experience while they participate in the staffing process. The Internet provides the technology needed for the creation of a new generation of innovative simulations that are highly realistic and effective, but until recently relatively few companies have taken advantage of this capability. However, several companies, including FurstPerson, Qwiz, SkillCheck, DDI, Aon, and Shaker Consulting have begun to actively push the envelope on the use of simulations. These companies deserve recognition for working to take assessment to the next level. We have chosen to highlight FurstPerson in particular because of the extensive flexibility, depth of content, and validation research associated with their simulation. However, all of these companies have simulations that may be worth exploring should you be in the market for a highly realistic assessment system. Before we discuss the CC Audition tool, we’d like to provide some background information about simulations and their many benefits. What Are Simulations? Simulations include a variety of different types of assessments. The common element of all of these is the re-creation of situations or tasks that are critical to the performance of a specific job or type of job for the purpose of predicting an applicant’s ability to perform the job in question. Simulations are able to achieve three objectives:
- To assess an applicant’s aptitude for learning how to perform tasks required on a given job
- To assess a candidate’s present ability to perform job related tasks
- To provide applicants with a realistic preview of critical job tasks
The simulations used for each of these purposes differ slightly but function in the same basic way, in that they are based on the same set of scientific principles used to construct all assessments used to predict job performance. Why Are Simulations So Powerful? Simulations have been shown to be highly effective and valid predictors of job performance. There are a variety of reasons why simulations are one of the most powerful tools for available for selecting employees. These include:
- Job relatedness. The major reason for the effectiveness of simulations lies in their job relatedness. Since the start of time, the litmus test of the value of any assessment used for employee selection has been the job relatedness of that assessment. The closer the information gained via measurement approaches the skills required for successful job performance, the more confidence one can have in making predictions based on that assessment. This is known as the idea of “point-to-point correspondence” between a predictor of performance and the job in question. Simulations offer a very close match between assessment tasks and tasks required on the job. This relationship also means that simulations provide excellent legal defensibility when used as assessment tools (provided that they are properly constructed and validated).
- Face validity. The concept of face validity means that the contents of an assessment tool appear to be related to performance at a given job. Simulations have very high face validity because they include tasks that closely resemble those faced on the job. This is very important, because candidates are often upset by assessments that do not appear to be related in any way to the job they are applying for. Negative impressions due to a lack of face validity can cause a variety of problems, including damaging an employer’s brand image, causing candidates to withdraw from the selection process, and increasing the probability that an applicant will seek legal action due to dissatisfaction with the results of an assessment.
- Simulations are fun and engaging. Not only do simulations resemble the tasks required for a job, they can actually be fun for applicants. This fun factor can have a huge impact on an employer’s ability to use an assessment to help build an employment brand. The ability to offer an assessment that simultaneously provides value for the job seeker and the employer is a winning situation for both parties.
- A realistic job preview (RJP). The similarity of simulations to tasks found on the job means that they can actually provide applicants with a realistic view of what it is like to hold the job they are applying for. This is important because the information provided by the RJP allows applicants to better understand what will be required of them on the job. This allows candidates who do not feel qualified or who do not enjoy performing job-related tasks to remove themselves voluntarily from the selection process early on, a situation that can help to reduce turnover and at the same time provide huge savings.
With this background in mind, let’s examine FurstPerson’s CC Audition tool. FurstPerson The goal of Chicago-based FurstPerson, Inc. is to help organizations find, hire, and keep better hourly employees and first level supervisors. By integrating screening tools and selection tools with a strong recruiting focus, FurstPerson focuses on helping clients hire employees that will stay on the job longer, be more productive, and learn more and faster during training. FurstPerson’s core focus is on customer contact positions, with a strong focus on call centers. At the core of FurstPerson’s products and services is its FurstSelect web-based platform. This platform integrates FurstPerson’s screening tools and selection tools (such as CC Audition) into a web-based system that provides clients the ability to manage their hourly workforce hiring process and provides them with real-time reporting tools. Based on XML technology, the platform is flexible enough to allow FurstPerson the ability to create hiring systems using a building block approach that matches the tools included in the selection system with a client’s hiring needs and budget. The FurstSelect system also allows FurstPerson to build complex decision trees that can automatically route the job candidate to specific selection tests based on their skills and behaviors and the hiring firm’s open positions. CC Audition Product Overview FurstPerson’s CC Audition is a multi-media, web-based simulation tool that evaluates job candidates applying for contact center positions. The CC Audition provides:
- A highly realistic simulation of a contact center environment
- A powerful, Internet-based realistic job preview that provides an opportunity for candidates to determine if the position is suited for them early on in the hiring process
- Ten interactive scenarios that focus on contact center jobs
- Immediate scoring and results accessible via the Internet
- Validation evidence supporting the tool’s legal defensibility
- Validation evidence demonstrating the tool is predictive for inbound and outbound contact centers, including customer service, sales, collections, market research, and technical support
CC Audition is broken into 10 scenarios that measure the following competencies using five standard call types: Computer skills: Ability to use a mouse and keyboard and understand the logic of a Windows operating environment. General learning: Ability to understand, memorize, and integrate new information and then to apply it in an appropriate context Problem solving: Ability to promptly solve problems by gathering and analyzing appropriate information. Ability to understand customers’ needs and suggest the best-suited product or service. Influence, persuasion, and competitive positioning: Ability to build a strong argument, negotiate agreements, and convince others of an optimal solution. Ability to integrate knowledge of the competitive environment to achieve company objectives. Rapid information processing/multi-tasking: Ability to react promptly to a situation and to efficiently listen, think, type, or search simultaneously. Customer Focus: Ability to ensure a positive customer contact while adhering to company policies. The ideal candidate respects the customer while promptly responding to requests, soothes tensions, refocuses the discussion on the problem, and recognizes and apologizes for errors. The five standard call types include:
- Orders
- Billing Inquiries
- Complaints and problems
- Product service/inquiries
- Up-selling and cross-selling initiatives
CC Audition is broken into two main components: 1) a training section that includes an introduction, a training module, and a practice session and 2) the actual call scenarios. The average test time for interacting through the 10 different call scenarios is about 75 minutes. Does It Work? FurstPerson reports that clients who have utilized the CC Audition for hiring purposes are seeing results. For instance:
- One trainer that provides training for a major communications company noted that the new employees hired through the CC Audition tool were the best she had ever trained.
- An HR manager for a call center outsourcer noted that the CC Audition has definitely allowed them to better identify job candidates with better sales skills.
- One major communications company has seen a reduction in turnover of at least 10% while using CC Audition.
- Underwriter Laboratories hired FurstPerson for a major call center expansion project. While the CC Audition was part of a multi-hurdle hiring process, it was the anchor test used in the process. Peter Serrate, director of customer service for Underwriters Laboratories, told FurstPerson, “The quality of the people working in the three centers has been exceptional. We’ve recruited 70 to 80 people and have only lost one person.”
Why Does It Work? As we discussed earlier, there are several reasons why simulations are so effective. However, in order for a simulation to be effective, it must be constructed using a precise process that will ensure it is actually measuring what it is supposed to measure. FurstPerson used the following processes to ensure the validity of the CC Audition tool:
- Three thousand contact center incumbents were surveyed on a wide variety of factors related to their job performance.
- Sixteen focus groups ó comprised of incumbents, supervisors, trainers and coaches ó were conducted.
- Real call observations were conducted at 16 different contact center sites.
- Version 1 scenarios were developed. Version 1 was analyzed within a study group of 271 candidates.
- A face validity survey (see below) was conducted.
- Suggested changes were adopted for Version 2.
- A statistical analysis with a norm group of 988 was conducted.
- Version 2 was released. Most clients are now using Version 3, and Version 4 is set to come out soon.
As part of its validation process, FurstPerson also conducted a face validity analysis in order to ensure that the CC Audition tool was seen as being job relevant by candidates actually seeking call center jobs. A total of 261 job candidates who took the CC Audition were surveyed, and 63% agreed or strongly agreed that CC Audition correctly represented contact center jobs. Conclusion The CC Audition tool represents what we feel is the cutting edge when it comes to leveraging technology to help increase the effectiveness of the process used to help make hiring decisions. This is because it:
- Enables candidates to determine if they could enjoy a contact center environment and allows candidates to self-screen out of the hiring process.
- Increases the probability that persons hired will be a good fit for call center jobs.
- Allows employers to test candidates in a highly realistic simulated call center environment before making a hiring decision.
- Presents a very cost-effective solution when one considers the rate of turnover at call-center jobs and the cost of this turnover.
- Has been shown to be a highly effective method of selecting call center employees
- Can be linked to FurstPerson’s web-based backbone system in order to provide candidate management and reporting functionalities
Look for more reviews of screening tools from us in the coming months!