Every once in a while I take notice of a new company that seems to be flying under the radar. Smarterer is one such company I believe it is quietly going about the business of changing the testing industry as we know it.
What Is Smarterer?
Smarterer is a skills testing company that is very different from the other skills testing companies out there. It offers 400 individual skills tests across the following content areas:
- administrative skills
- finance skills, marketing skills
- design skills
- sales skills
- software development skills
Perhaps the biggest difference between Smarterer and other testing companies is that all of its test content is crowdsourced from the individuals who take the tests. Smarterer relies on advanced artificial intelligence and sophisticated algorithms to evaluate the suitability of items and to create a dynamic, fluid process in which test content is continually evaluated and optimized based on actual user data. What?!! Who ever heard of such a thing?
Even more insane is the fact that Smarterer’s tests are available for free to both employers and individuals. That’s right — it won’t take your money, no matter how hard you try! This alone is a radical concept for a testing company, but one that is easy to execute when you are funded by Google. This unique combination of elements: a free testing platform that relies on its users to create content and is staked by an information-age titan should provide some clue that something truly unique is going on here.
How Does Smarterer Work?
Smarter is a web-based platform that allows employers to log in and create a free account from which they can deploy any of Smarterer’s skills tests in a combination that will map onto the unique combination of skills required for performing a job. Employers can add Smarterer’s skills tests to their job postings, or require applicants to use a link and complete the assigned tests. Employers can add Smarterer test widgets to their employment sites and can even build their own leader boards to help track candidates’ progress and suitability.
Smarterer provides the ability for any individual to take any tests at any time and to add their own test questions to any given test. Test takers do not necessarily have to be asked to do so by an employer. Once an individual completes a test and receives a score, they are provided with a credential badge that they can add to the various profiles and documents they use in communication with employers.
Those taking tests are graded as: master, expert, proficient, familiar, and beginner. An interesting twist is the fact that applicants can also create their own questions, and as they create these questions their scores go up. In fact, scores in Smarterer’s world are very dynamic, and they actually decay over time if an individual does not come back and take additional test questions to prove that their knowledge is stable or increasing.
Smarterer relies heavily on computer-adaptive testing and analytics to evaluate responses, determine that items are functioning optimally, and continually adjust test content to ensure high levels of relevance to the intended subject matter.
Why Is Smarterer Useful?
Beyond the fact that Smarterer provides a free way to evaluate critical aspects of an individual required for job performance, Smarterer’s real value is in offering a new and very different way to create and manage test content. Smarterer’s platform makes it very easy for individuals running hiring programs to evaluate things that are not often or commonly measured by traditional employment tests. Just taking a look at Smarterer’s test categories shows some areas that those of us familiar with the testing world do not often see, but which may definitely have value for employers.
Smarterer’s value proposition is the same as that of many other companies: to help quickly and benefit evaluate large numbers of applicants at the top of the funnel in order to determine whom should be evaluated further.
But the real value proposition goes well beyond this realm, because Smarterer is really an entirely new kind of company that is doing some very radical things.
True Open Source
One of the most interesting things about Smarterer is that it’s free and the way it is used is entirely determined by its user base, effectively allowing the user to define how it works and how to apply it. This is truly the new face of technology and the Internet. It is truly open source testing that will help make it easy for us to explore and develop new ways to involve testing in the business of talent acquisition and talent management.
Of great interest to psychometric geeks like myself is the manner in which Smarterer evaluates and manages the millions of items that are created by its users. It has informed me that behind-the-scenes it has sophisticated algorithms that can evaluate many different parameters about an item and can make determinations about the effectiveness and relevance of that item. Its adaptive testing engine that ensures that no test-taker sees the same item twice.
This is a great way to manage security concerns while also requiring a high level of engagement by test takers.
Credentialing Is the Future
Another extremely innovative part of Smarterer, and one that offers a window to the near future of testing, involves the use of credential badges. Any individual can log onto Smarterer’s website and take any test for free at any time. Their score results in a credentialing badge that allows them to demonstrate a given level of proficiency around a given skill. These badges can then be provided to employers as part of the job application process to demonstrate that an individual has a given level of proficiency.
Of course this will only work if the employer accepts Smarterer’s badges as valid and legitimate ways to demonstrate competency or proficiency. This remains to be seen; however, if it is not Smarterer who pulls this off, mark my words, someone will do it soon. We are about to see intense competition and many different business models in the credentialing arena.
Coming Back to Earth — My Concerns
As you can tell by my words so far, I’m a huge fan of Smarterer and what it is doing. There are a few things about the system, however, that keep me up at night.
My main concern is that, for all the innovation and amazingly sophisticated use of artificial intelligence, Smarterer does not seem to have included the core foundation of testing-related psychometrics into its model. There are certain parameters and indices that are extremely important when creating a test. These help to ensure that the test is both reliable — that is, will return an equivalent score every time someone takes the test — and also that these items are relevant to the content area that they are supposed to measure.
There is no doubt that Smarterer’s sophisticated artificial intelligence provides a great deal of data about each item. But it would be even better if it were to employ psychometricians to work with their test content using both Smarterer’s advanced artificial intelligence as well as more traditional forms of test development and evaluation.
Another concern I have lies with the actual validation methods of the Smarterer tests. I’m sure that many employers, especially those larger employers who are risk averse, might have an issue using an employment test that does not have any actual validation documentation. Smarterer is asking the user to take a leap of faith that its test actually effectively, reliably, and validly measures the content that is that it is supposed to. No matter how complex the artificial intelligence, this is a leap that many employers will not be allowed to make by their HR and compliance depts. Smaller employers who are not as concerned about legal risks could easily benefit from Smarterer’s ability to quickly provide information about which candidates are most suited for the job.
But, for Smarterer to be truly dangerous, and I mean this in a good way, it really needs to add the insight provided by employment testing professionals. This is in some way analogous to many other situations we are beginning to find ourselves in these days — in which there is a strong temptation to be guided purely by the empirical data that is so abundant and accessible the a web-based applications without or while losing sight of the value of rational expert judgment and input as well.
Smarterer’s best value would be to fold in the rational expert judgment required to provide it with the credibility and foundation needed to be accepted by consumers of mainstream employment testing products.
Then again, inherent in its nature as a true open source product is the fact that Smarterer can get away with being completely agnostic and allowing its users to define its ultimate purpose. It will be interesting to see what Smarterer has in store as it matures and evolves.
Smarterer’s ease-of-use and the fact that it is a free service will result in it being built into many new talent acquisition platforms. For instance, Cangrade, another company doing great things right now, includes Smarterer assessments in its candidate evaluation process, allowing for skills testing to be added to other types of testing such as personality and cognitive ability, resulting in a more complete picture of the applicant and an easier way to sort applicants.
I definitely believe we have not seen the last of Smarterer, and I’m excited to see where this unique and interesting concept takes us.