I continue to be impressed by the evolution of pre-employment assessment tools. This evolution is being driven by the continued growth of the value proposition assessment provides. As a result, an increasing number of new product include embedded assessments designed to help predict which applicants have the best chance of success.
This past year has brought a proliferation of firms that are using assessments to provide a new twist on matching online job applicants with job openings (and conversely allowing firms to match their job openings to candidate data residing in a database).
To understand the origin of Internet based matching, one has to turn the clock back about 15 years to the dawn of the job board. Job boards provided arguably the biggest overall change to the status quo for the way — both in the way people are hired because of the increased ability for individuals to find out about job opportunities, as well as for those hiring, to locate viable candidates.
Things have not changed much as even in the present day. The basic Internet job search equation involves a matching process in which each party provides information about who they are and what they are looking for. Behind-the-scenes algorithms living on servers evaluate the data provided by each party and calculate a match.
Although the job boards would argue otherwise, my basic description of the matching process used by most of them can be summed up with the phrase “garbage in, garbage out.”
I believe this is because most job boards do not really approach the matching process in a manner that uses what we I/O psychologists know about predictive science. The result is that while there are pockets of effectiveness, big job boards continue to be a source of noise for hiring personnel because they tend to deliver quantity over quality. This noise requires organizations to have strong capabilities for screening candidates at volume. Unfortunately, few companies have this capability, and even for those that do, the statistical chances of making a good hire go down each time an unqualified applicant enters into the hiring process.
I have ranted about the ineffectiveness of the matching process used by job boards again and again over the years and it seems that others are starting to agree with me. The difference between me and them is that they have begun to put their money where their (and my) mouths are by investing heavily in the creation of new businesses devoted to delivering candidate quality through matching that uses predictive assessments.
How Do These Sites Work?
An important difference between these companies and other assessment firms is that, for the most part, these companies provide matching that is part of the search process (as opposed to part of the application process). Their goal is to help companies stack the odds in their favor through a matching process that supplies candidates who have the best chance of being a good hire for a specific position.
There are a few variations on the basic theme, but these sites are all based on the concept of providing accurate matching using various pieces of information that help users to predict which persons are a match for a given situation, or vice versa.
One excellent analogy for the way these sites work is online dating sites. Back in 2005 I wrote about the parallel between online job matching and online dating and this analogy continues to work.
Both job matching sites and online dating sites work via the following process:
- Users are looking for a new situation of some sort
- Users define who they are (and what they are looking for in some cases) based on the information requested by the system. This is usually based on a profile creation process
- Users search for matches to their needs via the comparison of their profiles to those of others stored in the database.
- Results are returned electronically with some basic information about why the match was suggested (alignment in key areas indicated to be of a high level of importance).
In online dating all the matching process does is narrow down the database from millions to a few. Once this initial match is made, it is still up to the individual to communicate and evaluate the effectiveness of the match (usually by flirting and then going on some dates).
Hiring is no different. After the match is returned, the actual hiring process is used to evaluate mutual compatibility and to make important decisions about entering into a relationship.
The main way that online job matching sites are differentiated is by the information they use as inputs to their matching equations. Some use information that is provided by assessment tools that were developed using an accepted scientific process. Others do not, favoring the use of a variety of other types of information that may be relevant (past job experience, salary requirements, geographic location, favorite color, hobbies, etc.).
A thorough review of all the specific models being tried (some of which I’ve advised or consulted for) is beyond the scope of this article, as my focus is specifically on those models that use some form of assessment tool to help with the matching process. In these models, part of the profile creation process involves the individual completing assessment tools that provide a way to measure them on key attributes (personality, intelligence, skills, etc) to understand who they are and what they bring to the table. These sites may also use assessments to profile job seekers using a set of work-related values or preferences (autonomy, social responsibility, innovation, etc.) to determine what is most important to them in a job. Employers provide the other side of the matching data by indicating which attributes they feel are most important for performing a specific job or by profiling the values that the organization supposedly holds.
Are These Sites Really Effective?
I think that the jury is still out on the effectiveness of these sites. I am going to guess that as the data rolls in, some will be much more effective than others. Overall, many of these sites will eventually demonstrate a level of impact that is greater than that of sites using coarse keyword searches.
However, users of these sites should have realistic expectations. We have to take these sites for what they are: a relatively coarse matching tool that can be implemented quickly with a low cost. When it comes to assessments, you get what you pay for, and quick and easy matching is not going to tell you everything you need to know or ensure you make a good hire. Effective prediction requires precision; precision requires detailed effort. The more corners you cut, the less effective the results.
You still have to flirt and go on dates to figure out who you want to marry! The best value from these sites is allowing you to narrow down the pool from all the fish in the sea to those fish who have potential to thrive in your pond.
Here are a few factors that will impact the success of matches made using these sites:
- Quality of the assessment tool included: The better constructed the tool and the more research behind its effectiveness, the better the matching will be.
- Thoroughness of the definition process used by the company side: One potential shortcoming of this method is that the company-side profiles jobs based on the input of one or a few individuals. If these individuals are off base with their data, the accuracy of the whole process is compromised.
- Niche focus on one job or industry: Focusing on a niche allows for a deeper level of knowledge about what it takes to perform a specific job well and what predicts specific traits lead to success.
- Additional data collected: The quality, thoroughness, and relevance of the non-assessment related data used as part of the matching process will also impact the accuracy of matches.
- Ability to demonstrate that those hired using their system are “quality hires”? The more evidence that shows that matching impacts outcomes, the more effective the system at achieving its goals.
What You Need to Know
For job seekers: Go ahead and try. It can’t hurt. These sites all have some specific twist on the matching process, and there is value in interfacing with all of them as a learning experience. One of the big draws for job seekers is that these sites will give you information back to help compel you to make an investment in creating a profile. This information is usually in the form of a developmental report that is based on the assessment. This is valuable information for job seekers to help you understand themselves. It is unheard of for companies to share the results of assessments used in the hiring process with applicants, so as an applicant this may be your only shot at seeing your assessment results. Also, these sites can help you find jobs for which you are actually qualified — so it’s worth the time, but be ready for imperfect matches. Expect some error, albeit less than that of the big job boards. Don’t spend a dime to use these systems. They should be free.
Employers: Go ahead and try it. It can’t hurt. But remember that these sites will likely reduce your hiring error rate but they will not erase it. These sites are an excellent way to feed a more developed and focused assessment program that resides within your actual hiring process. Also be very aware of the importance of the accuracy of the profile parameters that you set. If this process is off base, the results will suffer. A few of matching companies help with this issue by using databases of information about the competencies that are critical for specific jobs based on information collected over the years by various sources. This removes the profile definition process from the employer, which can be helpful in some situations. If you are in a small- to medium-sized business, these sites can be an inexpensive way to find talent.
These sites will matter in the future, and make money, but it will take some time. When you boil it down, these sites create value in two ways. The first is in their candidate database. No companies will invest money to access an empty database. The big guys (Monster) can attest to the fact that the value of the business is in the size of the candidate database. The second way these firms generate value is via their impact on quality of hire. Unfortunately, most firms don’t measure quality of hire or trace it far enough back to understand the impact of their sourcing methods on it. So the real winners will be the ones who are able to figure out how to go viral so that they fill their database with candidate information, allowing them some bait to lure in companies who will pay for access. Unfortunately, quality of hire, while quite important, will likely always take a back seat to database size mostly because of how much harder it is to quantify.
Sit back and enjoy the coming onslaught of these companies. Look for one whose business model makes sense to you and give it a try. You have almost nothing to lose and some good employees to gain.