The Metrics of a Non-Event

Every now and then ó especially on anniversaries and Valentine’s Day ó my wife and I will discuss our first meeting and the series of events that resulted in that chance occurrence. There were no middle-persons who arranged that fateful day for us; it was all luck and timing. We met, fell in love, and 23 years later still seem to agree that we did a fairly good job of matchmaking all by ourselves. But somehow the ultimately successful outcome does not seem to be enough to quiet the concerns that still plague her:

• “What if you didn’t turn that corner on that day?”
• “What if I was seeing someone else then?”
• “What if you didn’t take that job where I also worked?”
• “What if you hadn’t lied to me about being rich?” (Hey, it worked.)

As my wife ponders these conundrums, I find my own thoughts wandering (although I continue to nod my head attentively). I, of course, consider the implications of her concerns from an HR/staffing point of view. That is to say, I think about the need to consider the metrics of a non-event. Think about it. We’ve developed all sorts of reports and measures to monitor cost and effectiveness in our recruiting and staffing efforts based on known occurrences. We compile metrics like:

• Received 200 resumes today from searches, search-agents, our website, and email.
• Rejected 150 outright using manual or automated pre-screening.
• Forwarded 50, and 35 were rejected by the hiring managers.
• Contacted 15 and screened out 10.
• Interviewed 5 and made 2 offers.
• Hired 1 and lost 1.

By establishing ratios we can come up with some nice “see how good I am doing” statistics that we can polish up and send to the next team meeting. But what about what is not being measured?

• How many of the 150 should not have been rejected?
• How many resumes rejected outright by hiring managers should not have been?
• How many resumes were never “captured” that should have been?

How Do You Track a Non-Event? If we mined for gold the same issue would plague us: how much gold is lost in processing the raw ore? But with a pile of previously processed ore sitting in the yard you can do a “random” sampling and determine:

• Are we throwing out gold?
• Is the amount of gold lost worth the cost of reprocessing the discarded ore?

The “Law of Diminishing Returns” exists for a reason. Every process is beset with inefficiencies. Sometimes the cost of attempting to achieve perfection exceeds the benefit to be derived. If you periodically review the 150 resumes rejected outright or the 35 resumes rejected by hiring managers in a typical day’s processing, you are taking a sampling from the “ore pile” ó a known event. Although this constitutes a worthwhile periodic quality control measure ó and one I highly recommend for determining screening process efficiency ó it does nothing to establish a metric for the unknown event. So, the question remains, “How do you measure an event that did not occur?” Simple, create a non-event event. Huh? Think about it: You already have tons of “gold” in your system ó your best employees. Run them through “the process” again and see what happens! Creating the Non-Event Event Here’s how to set up a testing scenario that will allow you to gather metrics on the “non-events” in your recruiting process:

1. Request that all hiring managers submit a list to you of the top 20% of their current staff.
2. Pull the original resumes of all these employees submitted at the time of hire. Do not update the resumes.
3. Submit a request to those employees, in writing, asking permission to use their background only ó not their identity ó in a staffing experiment to test on-line recruiting.
4. For each resume create new contact information to include:
   • Alias name
   • Alias email addresses.
   • Alias P.O. Box address
   • Alias phone number with a generic voice mail ó a machine voice giving phone number verification only, such as, “You have reached 555-1234. Please leave a message.”
5. Hire a human resources consultant to perform the following on a day of their own choosing:
   • Using resume(s), create a candidate job search strategy for backgrounds and see if your opportunities or company “pops up.”
   • Search those places where you post positions and submit “faux” resumes.
   • Enter the “faux” resumes into those third-party data banks that have agreed to allow the test to verify the efficiency of the search agents and web crawlers you currently use (offer to share data with the third parties as an inducement).
   • Enter resumes into your Web site as job applications for specific and general submissions.
   • Send email submissions as “blind responses.”
   • Monitor email aliases, voice mail, and P.O. box addresses for activity. This includes informing other contacts, other than your own staff and managers, that there is no interest.
6. Meanwhile, your staff should continue to do their own searches and recruiting efforts as per usual.
7. Ten working days after the resumes were first submitted into the process by the consultant, the consultant should contact the test project manager and the test should be frozen.
8. Using the candidate aliases, search your system ó automated or manual ó for the status of all resumes:
   • How many were contacted?
   • How many were by hiring managers?
   • How many were rejected by pre-screening?
   • How many were never captured in the process?
9. Pull all resumes off external data banks.

Now you have an idea of not only your efficiency in processing ore, also how much “ore” never makes it into your mill (in your answer to the “how many resumes were never captured?” question). Don’t forget, this “ore” is currently considered part of your most valued “gold.” There can be some issues if you do not prepare properly:

• This test must be done secretly to prevent an “alerted test group” ó one of the reasons for using an outside consultant.
• Members of your staff or even your hiring managers may resent using “faux” candidates to secretly test their abilities. Remind your staff that many federal and state agencies submit resumes into company recruiting processes to verify EEO/AA efforts. Quality control departments often insert defective material into the manufacturing process to screen practices and efficiencies. The finance department audits other departments, and often itself. Anybody ever hear of ISO standards? Maybe the time has arrived for HR/staffing to stop being so sensitive about uncovering flaws and hurt feelings (we can be such “wimps” sometimes).
• Using a consultant as the external tester insures that the efforts are not, intentionally or unintentionally, tilted in favor of the process in place. After all, no matter how poor the map, it is nearly impossible to get lost going home from a known place. You know too much to effectively simulate the efforts of a stranger or an applicant.
• Budget for enough alias email, phone, and P.O. boxes to insure that your staff and hiring managers do not notice a lot of candidates with different names all living at the same address. It is easy to rent PO boxes and create additional email accounts. Most companies have “spare” phone numbers for new sales and marketing hires.
• Insure the participating employees do not “spill the beans” by limiting the information you give them to include only your intention to use their background to test the applicant system. The extent of the test does not have to be revealed, as their identity is not at risk or being “used.”
• Be prepared to deal with the consequences óbetter known as “the truth.”

What is a good number in this experiment? Should you be happy with an 85% “recapture” or devastated with a 75% “escape” ratio? I don’t know, that really depends on how important the hiring of top quality candidate/employees is to you. But process improvement requires continued testing and evaluation. I would be more concerned with why obviously good employees backgrounds were not “re-captured” than with how many. I would also be especially concerned about who was passed over. Do they represent the most critical skills areas? Are they the most difficult to replace? There may be other surprises as well.

• Initially, how many of your managers’ top 20% were in fact “advocated candidates” (i.e., a candidate who entered the interview process based on another person’s recommendation) and not the result of your online search efforts? Do employee referrals and agency candidates still “pop up”?
• What other third-party sources send you “faux” resumes? A side benefit of testing the efforts of those who “search for you” and a good test of their ethics. After all, there is no way these could be pre-screened submissions.
• Based on other contact attempts made to the alias emails, voicemails, and P.O. boxes ó by those other than your hiring managers and recruiters ó you can see who else is looking for the same candidate profile as you. These may therefore be good sources for you, or other potential companies seeking your current staff.

There will be those who feel that process testing using “faux” backgrounds has ethical issues. But if the test is carried out with the permission of your employees, with advanced clearance of any of the resume databanks you use, and no contact attempt is followed up with any misrepresentation, the only ethic at risk is your own willingness to truly test your process against a known previously successful outcome. If you are worried about your relationship with your own staff or hiring managers after “the test,” I can only venture to say that the relationship must have already had serious issues to be that tenuous. By not using the data as a “hammer” to embarrass individuals, but rather as a review of total process, it will help those tested actually use the information rather than be threatened by it. If you have a manager or recruiter with a horrendous result, I guess the issue is the importance of their feelings as opposed to their effectiveness. But that is an environmental issue, not an issue of the test itself. It is said that, “The truth shall set you free.” That presupposes that you wish to seek the truth and not merely sit around waiting for it to find you by accident. I asked my wife if she had to do it all over again, would she make the same decisions as on the day we met, now knowing the ultimate outcome? She said, “Yes… probably.” I think she said “probably” to get even with me for all the times she has been aware that I am nodding my head when I am really not listening. Can’t get anything past her! Have a great day recruiting!