One of my readers recently wrote me with a very good question. He wanted to know how to verify all the claims and data flying around. What?s true and what?s false? What about statements such as:
- People use 10% of their brains,
- 70% of the resume data is false,
- it costs $70,000 to replace an employee,
- our test is validated for use,
- interview data is no better at predicting performance than chance,
- good employees out-produce poor employees by a factor of at least two to one,
- broccoli is good for you,
- personality tests predict job skills,
- or, politicians care more about you than they care about themselves?
Well, let?s look at this systematically. The first step in deciding whether these statements are facts or not is to examine the source. Step 1: Examine the Source
There are many sources of information. They include the ?Expert Opinion? (Expert), the ?I?m Not Really an Expert, but I Play One On TV?(Misinformed), the ?Everybody Really Ought To Know This? (Common Sense), and the ?I?ve Proven It Through Research? (Research, or as we say in the trade, ?Nanny-Nanny-Boo-Boo?). Let?s discuss how the sources differ and when you should use the data or politely drop it in the trash. Formal accomplishments, graduate education and practical experience are associated with being an Expert. A neurosurgeon, for example, may claim that people use less than 10% of their brain capacity. Neurosurgeons have lots of credentials, education, practical experience, and are fully qualified to present opinions about their specialty — brains. When it comes to stating opinions about brains, you can probably trust what they say. On the other hand, there is the Misinformed source. The Misinformed have opinions, but their opinions are seldom fact-based. For example, another reader told me with a straight face that it was not necessary for selection test vendors to belong to a professional group such as the American Psychological Association (APA). Belonging to the APA, she said, was only for psychologists. Psychological selection tools were illegal, she claimed, and APA membership would just attract lawsuits. Wrong. This is a case of being Misinformed. The person was probably well intentioned, but her statement bordered on being silly. We all are misinformed about something ? it?s just not good to be misinformed about the product you are selling. Experts almost always belong to exclusive (i.e., ?snooty?) professional associations. When my Misinformed reader stated her beliefs, she was also stating how little she knew about the selection field and the products she represented. Misinformed people tend to re-invent the wheel. People who listen to the Misinformed usually fall into two groups: people who trust and believe them; and, people who think they are silly. The claim that turnover costs $70,000 per year per employee falls into the Common Sense category. Well, almost. We know turnover costs money. There is the obvious recruiting expense, recruiting fees, lost productivity while the job is empty, training expense, lost salary, benefits, lost customers, legal costs, etc. But does it cost $70,000? In some organizations, it probably costs much more. And, in others, it probably costs much less. You have to do the math for yourself ? no one else can calculate your costs?and if you don?t know your costs, what are you doing in the business? ?Good employees out-produce poor employees by a factor of at least two to one. Can you believe that? You better! This data has been verified by many controlled studies. Controlled studies use an experimental design protocol that basically selects two groups of subjects at random. One group participates in the experiment while the other group acts as a control. At the end of the experiment, the two groups are statistically compared to see if there is a difference. Where do you learn how to do this? Graduate school, professional association conferences (snooty members only) and competent research studies (see Expert). Next: Examine the Claim
Our neurosurgeon claimed that we only use 10% of our brain capacity. Was this based on research? Probably not. Brains are very complex and it is unlikely that someone could set up a controlled experiment that showed the number to be exactly 10%. In fact, any percentage estimate of mental capacity would be purely an expert guess. Nice to know. May even contain some truth. But, is the figure really 10%? Why not 15%? 17.3%? We know the neurosurgeon is a trustworthy source. But, since no one has yet been able to discover exactly what the other 90% of our brain is actually doing, knowing the miniscule ?usable percentage of our brain capacity? is only going to either make us clinically depressed or turn us into hopeless party bores. Besides, who can prove the other 90% isn?t doing something useful, like trying to understand why some people still think interviews are accurate predictors of performance? Even claims based on research are estimates, albeit expert-based. For example, scores on mental alertness tests tend to explain from about 5% to 60% of the variance in performance. Does that mean all smart people are successful? No. That would be Misinformed. It only means that higher performers tend to be smarter than lower performers. Are mental alertness tests illegal? No. That would also be Misinformed. Mental alertness (MA) tests just have to be validated to see what, if any, relationship their scores have to do with job performance. No one is forcing you to hire dull people ? just be sure your MA test scores represent what you really need for the job. Where do you learn all this stuff? You guessed it ? in graduate school and technical seminars! Finally: Examine The Usefulness
Is a resume really 70% false? We all know the purpose of a resume ? get the interview. Everyone who drafts a resume is told to use key words, employ ?power phrases? and think of the resume as a personal advertisement. Duh! But, who is so ?simple? that his or her entire life experience can be captured on three crummy pieces of paper that will be skimmed by someone who thinks typos are reliable predictors of performance. Come on! This is a Common Sense area. Of course resumes contain false data ? it is the nature of the beast! Who cares if it is 70% (although I?d like to see the research study that backs up that claim) or 30% or even 100%? Organizations should expect resumes to be little advertisements ? equal parts fact and personal hype. That?s why we need tests, simulations, assessments and behavioral interview technology to probe deeper into job skills. Conclusion
I know. All this stuff gives you a headache. Is everything painted in shades of gray? Yes. Should you question every quoted fact? Yes. Do you always have to work to discover truth? Yes. Will absolute truth always be teens weensy? Yes. Make Dr. Williams? Three-Step Program of Truth your career credo:
- Does the speaker have the credentials, experience, education, and objectivity to be an unqualified expert on the subject for which he or she is giving an opinion?
- Does the speaker present hard data to back up his or her claim?
- Does the data have any practical use?
Oh, yes. Why do I keep pushing graduate formal education? That?s about the only time in school when you are forced to shift from reading and memorizing someone else?s work to doing your own work and research. It is marvel to observe the change from na?ve acceptance to critical evaluation. In case you are wondering, here are some answers:
- people use 10% of their brains (I?m pretty sure we use less than our full capacity, but so what?)
- 70% of resume data is false (so what else is new?)
- it costs $70,000 to replace an employee (check your own numbers, you might be astonished)
- our test is validated for your use (HAH! ? if you believe this, your attorney will soon be driving a new Porsche!)
- interview data is no better at predicting performance than chance (that depends — true for non-structured interviews, but false for behavioral or situational interviews ? based on research gathered from dozens and dozens of studies)
- good employees out-produce poor employees by a factor of at least two to one (true and confirmed by dozens of studies)
- broccoli is good for you (probably true, but may also be a part of a international broccoli growers? plot to overthrow the powerful cauliflower cartel)
- personality tests are good predictors of job skills (unfortunately, false, as confirmed by my own research)
- Politicians care more about you than they care about themselves (Duh!)