As discussed in previous articles, gamified and game-based assessments are particularly suited to mobile devices such as smartphones. Using the inherent capabilities of smartphones, recruiters can access a wider potential talent pool, especially in developing countries such as South Africa where relatively few people have consistent access to laptops or tablets.
In this article, we’ll take a look at some exciting developments in game-based, smartphone-centric assessments from our assessment partner, cut-e (an Aon company).
Specifically, we’ll examine cut-e’s smartPredict collection of game-based assessments. These measures have been specifically designed with mobile delivery in mind and preliminary research on these assessments point to promising benefits for IO Practitioners interested in innovating their assessment practices.
smartPredict and gamification
Recall from previous articles that gamification is perhaps best seen along a continuum. On one end, we have games that have been designed as games first, albeit with some form of behavioural measure included (i.e. game-based assessments). On the other end are traditional psychometric assessments with some gamified aspects added on a more surface level, in response to an increased need for more engaging assessments (i.e. gamified assessments).
Where does cut-e’s smartPredict fall on this scale? To answer that question, recall that smartPredict comprises four different activities or games:
- switchChallenge: measuring logical reasoning
- digitChallenge: measuring numeracy
- motionChallenge: measuring complex planning capacity
- gridChallenge: measuring executive attention
Research conducted by Aon using more than 500 international, Millennial job-seekers, found that game-based assessments are most effective when they engage participants without being overly playful or irreverent.
Drawing on such findings, test designers at cut-e used existing, well-established psychometric tests and testing principles and combined them with strong gaming principles and technologies.
Thus, smartPredict falls neatly in the middle of the spectrum of gamification. It both contains methods that have long been established within psychometric measurement as reliable and valid, while drawing on actual gaming principles and outcomes to deliver a more engaging candidate experience.
Additional findings: Caveats and provisos
Followup research commissioned by Aon investigated the perceived fairness and impact of gamified assessments may have on organizational attractiveness. Using a sample of 200 participants who applied to a fictitious organization by completing both game-based and traditional assessments, researchers came to two important conclusions:
- Participants who enjoyed game-based assessments tended to have positive ratings of the organizational attractiveness. For those who did not enjoy the games, ratings of the organization were not negatively affected.
- Perceived fairness of game-based versus traditional assessments did not differ substantially
Important to note is that the smartPredict challenges were designed for use in important talent decision-making, and hence they resemble more traditional assessments while being delivered in a more game-based interface. This design consideration may well account for the findings of perceived attractiveness and fairness discussed above.
Another important consideration for IO Professionals is to understand when gamified or game-based assessments will unlock benefits and when they may not be appropriate. In general, cut-e and Aon’s research studies on the topic have found that game elements with universal appeal include:
- Completing designated challenges in a sequential fashion
- Unlocking of levels and tracking progress through achievements earned
- Immediate feedback on assessment performance
- Challenging and interactive assessment experiences
Gaming experiences that may be contra-indicated, or for which the costs may outweigh the ultimate benefits included:
- Assessments that are indistinguishable from games intended for leisure or fun only
- Overly noisy assessments that feature lots of background sound effects
- Any element that generally detracted from the seriousness of the assessment’s purpose
Next steps
From the preceding discussion, it’s clear that like most innovations in our field, game-based and gamified assessments require a disciplined, evidence-based approach that IO Psychologists are well positioned to deliver to the organizations they serve.
While gaming elements will no doubt continue to be incorporated into assessments (along with the increased appeal, engagement, and fairness that seem to be associated with them), IOs will still need to critically evaluate the psychometric robustness and validity of such instruments.
In addition, preliminary findings suggest that there is such a thing as too much gamification of assessments, and we should be cautious of having our enthusiasm for new technologies outstrip the value that they truly add to the quest of making better talent decisions.
If you would like to know how TTS helps clients bring innovation to their assessment practices, why not drop us a line at info@tts-talent.com?