The future of education is free — free content, courses and textbooks. However, without a way for students to validate their knowledge and prove their skills, this is only one step (albeit a significant one) towards reforming higher education. Many are now beginning to attack the problem of accreditation — helping to develop the next generation of certificates, badges and credentials that will allow self-learners to prove their knowledge in order to get employment and advance in their careers. The race is on to (slowly) replace the college degree as the primary way for companies to evaluate and validate talent.

The Degree

Having a degree means better earnings. That is a fact, and it is backed by substantial data. An associates degree will generate $200,000 more than just having a high school diploma over your lifetime, while a bachelors degree will provide a more robust $600,000 increase. Having a degree from a top institution is worth even more. Given such a clear increase in earnings, it is obvious why young people make the choice to take on substantial debt to get a degree from a traditional university. Yet if we are going to truly disrupt education, the time is going to come where someone who has the brains and skills to enter college decides against it because an online education is a true alternative. Given the data, what young person would make that choice today? Without alternative credentials that employers value, the idea of an “Online University” is just that – an idea.

This article is the first in a series that seeks to examine online credentials used to demonstrate accomplishments and skill sets. We will seek to answer the following questions:

• • Why do these platforms exist? What purpose do they serve?
  • How do they assess skills, knowledge and intellect?
  • What is the value for employers? Job-seekers? Self-learners?

We have focused on four types of online credential that are prevalent today:

• • Test-based credentials: Test-based credentials have gained popularity both in the online freelance market, as well as in Programming and highly technical tasks. These credentials are earned by taking multiple-choice or project-based tests in various skill areas.
  • Online Badges: Badges allow individuals to demonstrate job skills, educational accomplishments, online course completion or just about anything else that a badge creator decides. And they are still very nascent and in the very early stages of their growth – right now a ‘badge’ can mean almost anything.
  • Completion Certificates: Lynda.com, TeamTreehouse and Grovo, among others, offer certificates upon completion of courses. Sometimes short quizzes or challenges are used to ensure that the student is able to demonstrate real knowledge, although this is rarely (if ever) as robust as test-based credentials.
  • Online Certificates: The highest value credentials to a degree are currently online certificates. Earning an online certificate from an online college, a company or an industry-specific organization is typically much more  involved than the other credentials, and are often connected to specific job functions. Many of these certificates have been created by companies such as Cisco, IBM or Microsoft from their own needs or the needs of their customers.

In our first installment in this series we will be looking into Test-Based credential platforms from companies such as ExpertRating, Gild, and Smarterer, as well as getting a glimpse into the growing badge space.

ExpertRating

ExpertRating is an India-based web-based testing and certification platform, with over 300 40-question tests that are used by all of the major freelance platforms, including: Elance, oDesk, Freelancer.com and vCoder/Rentacoder (now acquired by Freelancer.com). Tests are offered to freelancers for free, or a low price ($5), as a mean to benchmark their skills and publish the results to their profiles.

Elance claims taking tests leads to a 67% increased chance of landing a project, perhaps the reason that over 20,000 tests are taken each month on the platform. Surveyed employers apparently appreciate these tests, with 76% saying they are more likely to hire those who have taken tests relevant to the job. Odesk hasn’t provided any similar data, but they have seen tens of thousands of contractors taking skill-based tests.

However, the most popular tests are the Readiness test (created by Odesk) as well as tests for English language skills, English vocabulary and Email Etiquette. The popularity of these particular tests highlights the workers that are prevalent on these platforms: foreign-based workers performing relatively low-skill tasks. That said, there are also plenty of tests in programming, design and more advanced skill areas.

ExpertRating operates as a monopoly on these platforms. Since 2000, the firm has tested 2.7 million people, and boasts over 10,000 corporate clients in over 60 countries. It offers an expansive range of training and testing – from personal training certification and first aid to SEO and Microsoft Excel, covering over 300 skill areas.

ExpertRating offers two ‘tiers’ of testing — certification courses (ranging from $50 – $2,000) that are instructor-led or self-paced and abbreviated “concept tests” that go for $9.99 a clip. It is these tests — all done via in a 40-question in 40 minutes format — that are are being taken thousands of times each month through the various platforms.

Okay Great. Multiple Choice. 
We took the a varied set of tests on different platforms, and it is evident that the tests used the same exact 40 question test bank, leaving the door open for cheaters. We were easily able to find answer keys for the various test modules available online.

The test do vary among platforms, with Elance using a single 40 minute clock for all 40 questions, whereas on oDesk, there was a 1 minute timer on each of the 40 questions. Platforms also treat re-tests differently. Elance allows retests every two weeks, whereas oDesk requires a month between re-tests. Both platforms allow users to delete and hide skills.

“You will receive your score after the test is completed and it will display on your Elance profile. At any time, you can choose to hide or delete the skill completely. You can retake the test as many times as you like, but there is a 14-day waiting period between test retakes.” -Elance

When we researched online, we found mixed reviews regarding the quality of ExpertRating’s testing. Quite a few users that we talked to have complained about the test mechanism, including a group of users who scored perfect pass rates but were instead were only give a badge displaying “Top 10%”, perhaps indicative that many test-takers are scoring perfectly.

A relational database expert who had developed assessments in the past that questions the accuracy and effectiveness in evaluating the skills in question. As he put it, “These so-called “skills assessment” tests were proof-positive of that; they’re a joke, and folks serious about testing the skills of others would be ashamed to have them as the representation of their own knowledge on a given subject.”

Despite a less than stellar online reputation (with a F-rating from the Better Business Bureau and plenty of bloggers complaining about them) , ExpertRating’s tests are used tens of thousands of times a month. Clearly, there is some perceived value to these tests. We conducted interviews of medium-sized employers on these platforms, and anecdotally, we found the tests are being used to eliminate and search amongst contractors rather than to decide whether one is worthy of hiring. Employers found the value of the tests greater for narrowing down huge groups of similar candidates, rather than actually choosing a specific one.

There is some evidence that the ExpertRating tests are decreasing in value. The more popular freelancers on newer platforms such as Guru and Freelancer.com tend to avoid these tests entirely, perhaps due to the cost (on these platforms, ExpertRating tests cost $5 each, whereas they are free on Odesk and Elance), but more likely it is because they are unnecessary for those with good portfolios to prove their skills.

Smarterer

Smarterer is a relatively new entrant in the field of online testing and credentialing. The founders describe it as “a simple, fun, and authentic way to show people what you know how to do”. Smarterer uses crowdsourced questions that are then reviewed by the Smarterer team. Adaptive testing techniques allow them to accurately assess one’s abilities in 10 questions and less than a few minutes. Whereas ExpertRating’s tests are identical, on Smarterer, no two tests are likely to be the same. “We’re out to fix the ‘skills’ box on everyone’s resume,” says Dave Balter, Smarterer cofounder. “It’s the most important resume element, but what does it mean when you say you’re ‘proficient’ at Excel?’ We’re going to be the gold standard of scoring skills.”

Tests are very short and are designed to take no longer than a couple of minutes. Cheating is very difficult as there is a 20-second timer on each question.  Expert users can both contribute and create their own tests, which is great for employers who are trying to test specific skills and knowledge. Additionally, the crowdsourced nature of the questions allows tests to grow and morph over time, reducing the potential issue of cheating.

Smarterer is clever in how they address scoring, which is difficult when everyone takes a different test. Simple multiple choice tests don’t do a great job in determining mastery, which is critical when it comes to evaluating skills. Smarterer uses the Glicko scoring system, a method that is used in professional chess and many video games. The score output is on an 800 point scale similar to the GMAT. The magic of Glicko is that it is based on test-takers rather than an external grading mechanism. As a result, if you take an early version of a test and score high, your score may actually lower over time, as more people take the test and it becomes clear that others have a higher degree of mastery than you. It’s unclear yet if this scoring system is appropriate, but it is certainly better than a system that doesn’t take into account what others are achieving.

Smarterer is both a site and a service. They started in 2010 with the vision of helping people get jobs and helping companies find applicants. This continues today with various partnerships that embed Smarterer tests within larger recruiting platforms. In our research, we came across a several different platforms using Smarterer within their own site. This enables companies to focus on what they do best, and use Smarterer to evaluate specific skills.

Smarterer may have a greater use in online education. “The deeper we got, the closer we came to the realization that people were using the system for much more intrinsic purposes… to understand if they were growing or not.”  said co-founder Balter.  Smarterer sees this as a future use case — to help self-learners to learn from their mistakes and zero in on areas that can use more development, better enabling online learners to get the feedback that is often missing in a one-directional video lecture.

It is unclear yet where Smarterer fits into the larger world of replacing the degree. While the platform was designed to share results, as of last September just 3% of active users shared a Smarterer score externally. One would surmise most people who are scoring below 700 on any given test are not rushing to share their results. It is interesting to note that for the most popular tests, each user is averaging 6 to 8 sessions, indicating that the platform is likely being used for measuring self-improvement.

Whether it will be used for self-learners to self-evaluate, or as a way for employers to evaluate potential employees, Smarterer’s crowd-sourced questions, unique scoring system, and large question banks represent a substantial step-up from the ExpertRating system.

Grade the Coders

There is one profession that is seeing a surge in online testing and credentialing — programming. With programmers in-demand in every corner of the world, and in an industry where languages are being introduced and updated on an annual basis, programming is one area where many, or even most, are self-taught, and the typical degree is of little value in gauging someone’s capabilities. With programmers, we have seen the rise of several assessment companies to grade and evaluate coders, including Gild, Codility, InterviewStreet, and CodEval, among others. Although programming is an area that naturally fits with online testing, the rise of these services is perhaps a precursor to more creative assessments in other subject areas.

Gild

Gild is a web-based platform that allows recruiters to efficiently locate developers all over the world and start relationships with them. It collects detailed insights about programmer’s abilities from developer forums, open source coding projects and details about their online “social life.” With this information in hand, Gild scores programmers based on their knowledge and accomplishments and provides recruiters with tools to reach out to these potential candidates.

To be clear, there is no specific test with Gild. If you are good, Gild will find you based on what you have posted online. Gild is also focused on helping recruiters and companies rather than individuals. As a coder, the best way to get “certified” by Gild is simply to post high quality code on Github or other popular programming hangouts online. Gild represents a new way of thinking — rather than have the individuals go through a specific series of tests — which opens up the possibility of cheating, and requires a higher cost component in terms of developing or purchasing test services, Gild instead takes what you’ve already done and grades you based on that.

Test Based Developer Graders

The rest of the players in the space are similar, in that they provide tests that allows employers to gauge a potential employee’s abilities. CodeEval is a platform that enables companies to create online coding challenges and have applicants show their direct skills by enrolling in the challenges. InterviewStreet’s model is a combination of the CodeEval challenge concept, and the Gild employer-focused concept. Rather than create an open platform where anyone can apply, InterviewStreet only allows client companies to invite candidates to take online challenges. Additionally, InterviewStreet has created a library of challenges and tests that any company can use, but also allows companies to create their own challenges. With a built-in console where coders can enter and evaluate their code, the entire challenge can be done in the browser window. With Facebook, Quora, Microsoft and Amazon among their client list, InterviewStreet is able to help companies save time and eliminate candidates that are not up to par.

In our research, we also found one innovate site from India. TechGig helps coders to find jobs, take courses, and evaluate themselves. They offer quarterly code contests where coders can work on a project and compare their results to thousands of others. Additionally, they offer skill tests that have already seen thousands of test-takers get to evaluate their skills. The platform is new, but just seeing the thousands who have already taken tests is indicative of a major need for both job-seekers (to prove their knowledge) and students (to self-evaluate and test their knowledge).

Applicant-Testing

Much of what we’ve discussed is publicly accessible and usable by anyone. It is user-driven. However, companies have used pre-employment testing for years (even decades). In this model, these tests are not open to the public, and the results are stored by the companies themselves.

Two such employer-based tests platforms are ProveIt and EmployTest, although there are countless others. ProveIt is from Kinexa, a subsidiary of IBM, and has over a thousand different assessments and has positioned itself as a talent management platform that can help companies attract and cultivate talented employees. In addition to testing job applicants, ProveIt is used to monitor employee engagement and gauge leadership potential.

EmployTest on the other hand is strictly focused on pre-employment testing, offering over 800 different tests. While the previously coding tests platforms have a very narrow focus, ProveIt and EmployTest offer a range of assessments that cover administrative, clerical, general computing, and industry specialties. Generally, these assessments seem to be less technical and less robust (multiple choice).

Regardless, both platforms are part of the job application process and help companies get a quick and accurate assessment of job skills, knowledge, behavioral tendencies and aptitude, and at the same time allow employers to quickly weed through large volumes of applicants. For behavioral and aptitude tests, there is not much an applicant can do to prepare for these assessments. While one can prepare for the skill tests, their private nature means it’s difficult or impossible to attempt to cheat. In addition, they are administered in the middle of the application process, giving little time to prepare.

What’s Next: Badges

Badges have been used as visual displays of skills, rank or achievement. However, outside of the gaming context, they have rarely been used in the professional or academic arena. This is slowly changing.

A brief history of Badges

It is difficult to trace the roots of badges but perhaps they date back ancient times when members of the royal court and military wore sashes and other accessories to denote their rank and file. We still see this tradition alive in well in modern militaries around the world. In the last century, the badge tradition was adopted by the Boy Scouts and Girl Scouts, and have been in use by the organizations as early as 1911. In the past ten years or so, it was gaming that brought badges online and World of Warcraft in particular, was one of the most popular games utilizing badges.

Open Badges

Mozilla, via the Open Badges platform, has created a set of tools to better enable badges to become something more than a gimmick in gaming. They’ve introduced a way to share badges across platforms easily, as well as a “backpack” or “trophy case,” where all of your accomplishments can be displayed in one place. According to Mozilla Executive Director Mark Surman, “Today, we learn things in a wide variety of ways, but there are fewer opportunities to gain formal recognition. Open Badges lets you take all those skills and show them off in one place, regardless of where you’ve earned them.”

After two years in development, Mozilla debuted its beta version of the platform last month, generating a substantial buzz in the media. The project has strong backing from Mozilla, the MacArthur Foundation, and the Humanities, Arts, Science, and Technology Advanced Collaboratory (HASTAC), who together funded a $2 million Digital Media and Learning Competition to encourage the development of digital badges that recognize lifelong learners’ knowledge and skills. Thus far, the project has enlisted 600 organizations, including the likes of NASA, Intel, Microsoft, and Carnegie Mello. Together these organizations have issued badges to over 23,000 learners. Even Secretary of Education, Arne Duncan, calls badges a “game changing strategy.”

We signed up for Open Badges and setting an account is easy. In about a minute you can have your very own Mozilla “digital backpack.” After that you are sent out into the wild to earn badges from the existing 25 or so organizations that are currently providing them. It’s up to you to figure out what is interesting and worthwhile in your field and each site has its own registration and badge certification process. Some are more focused on recognizing achievement (perhaps you finished on online course), whereas others are more about recognizing a skill, such as passing a test or challenge.

Badge Supporters

Several organizations have taken a lead role in integrating Badges into their platforms. We would expect many more to be added, but these are the trailblazers:

• • P2PU – Peer To Peer University is “Learning for the people, by the people. About almost anything.” It is an open online course platform that focuses on peer to peer information sharing where learners can be recognized for their achievements in the form of badges. Courses are free and all content is user generated.
  • Credly & BadgeOS – Credly is a startup that allows anyone to create badges, and then issue those badges online. They have worked with LearningTimes to create  a WordPress plugin, BadgeOS, that enables any site owner to begin to offer badges and create a community around gaining skills.

Private Badges

While it would be great if everyone who issues badges used Mozilla exclusively, companies are not waiting for their solution to come out of beta. One such example is CrowdFlower, a real-time crowd labor platform. As CrowdFlower has wanted to better enable its clients to find the right talent to complete crowd tasks, they decided that they needed a way to evaluate and ‘badge’ their workers. They, like many companies, are using their own, proprietary badges. There is some danger that without a single organization to verify and centralize the badge issuing and earning process, the entire ‘badge movement’ could sink under its own weight.

Replacing the Degree

There is a race underway to create the next set of online credentials to enable self-learners to both self-evaluate their knowledge to fill in the gaps and gain mastery, as well as prove their knowledge to gain employment. We’ll be back to discuss other types of credentials that are being created everyday.

The post Prove your Skills: Test-Based Online Credentials appeared first on SkilledUp.com.