In recent articles, we’ve reviewed not only the available mobile-capable and mobile-first assessments, but also what the world (and specifically the South African) mobile landscape looks like.
Recently, TTS conducted a study to investigate the perceptions of Millennials who are entering the technology and telecommunications job market. We were interested to know what this exciting new generation thought of conducting selection assessments on their smartphones, how they behaved, and what their impressions were of such assessments.
This represents some of the first data we know of that really describes the South African landscape in terms of perceptions of mobile (especially smartphone) assessments. While equivalence research is important and will follow, we were particularly interested to see if some of the claims regarding mobile assessments (e.g. enhanced engagement, perceptions of fairness, etc.) would be true of a relevant local sample.
The sample
We had the privilege of working with a client who wanted to innovate their recruitment process and using mobile assessments was part of the process. Our client wanted to recruit Millennial-aged job seekers who were new to the world-of-work but who were interested in the technology sector as employer.
Our study comprised 94 participants (50% female; 50% male). Their ages ranged from 20 to 26 years and ethnicity was composed of 80% Black, 7% Indian, 6% Coloured and 6% White candidates. Our sample had all applied online to take part in the client’s recruitment and selection process for this opportunity.
Assessments
Our candidates completed several behavioural and ability competency-based assessments as part of the selection process, and were encouraged to complete their assessments on their smartphones where possible.
For this project, we used cut-e’s suite of mobile-enabled and mobile-first assessments. As mentioned in previous articles, cut-e has long been at the forefront of developing highly engaging and novel measures that are especially suited to mobile and smartphone devices.
In addition to the assessments, candidates completed a mobile assessment perception survey designed by us. Post-assessment interviews by TTS consultants were also employed to collect qualitative data about the candidates’ experience of the selection process.
Results
We focussed on a few key questions in our analysis of the perception survey and interview results:
- What was participants’ experience of using mobile assessments?
- In which contexts did participants complete their assessments?
- Were there benefits, such as perceived fairness, derived from using mobile assessments?
- What were participants’ preferences regarding device-type and completing selection assessments?
Based on our survey data, we found that:
- Connectivity: 70% of our participants had no issues with the quality of their connection while completing the assessments, while the remaining participants experienced some challenges. However, all participants were able to complete their assessments fully.
- Location of testing: 70% of our participants completed their assessments at home, while the remainder completed their assessments at work.
- Ease-of-use: 72% of our participants found the testing interface easy to use, 24% didn’t find the interface easy or difficult, and 4% of participants had some difficulties with the interface.
- Preferred device: Interestingly, 50% of our participants preferred using laptops or desktops for completing assessments, while 45% preferred smartphones and only 5% chose tablets.
- Perceived fairness: 85% of our participants experienced the assessments as fair, 10% thought the assessments neither fair nor unfair, 3% perceived them as somewhat unfair while 2% experienced the assessments as very unfair.
We also interviewed participants to get a picture of their assessment and selection experiences. Strong themes identified were:
- Assessments enhance recruitment: In general, candidates experienced the assessment component of the recruitment process as very positive and interesting.
- Device type may be less important: Despite the interesting findings in our survey on device preference, when interviewed most candidates did not really have strong feelings about device type. For most, using their mobile phones was natural and unproblematic, but we got the impression that given Millennials’ comfort with technology, they are perhaps more “device-agnostic” than others.
- Using mobile assessments can affect brand perception: Our candidates commented on the alignment between the client’s brand within the technology sector and using what was perceived as advanced, technologically aligned assessments.
- Getting non-assessment factors of recruitment right is important: Many of our candidates commented on how they experienced non-assessment factors in the recruitment process. For instance, poor communications prior to interviews, vague instructions of what was to happen next, and similar process flaws were especially salient for them, and negatively affected their perception of the entire process.
Concluding thoughts
This study represents a preliminary reconnaissance of the perceptions of Millennial job-seekers regarding mobile, and especially smartphone-based assessments. Given the paucity of research on the topic in South Africa at present, it sketches a preliminary picture that we hope to flesh out over time with further research.
A few selected conclusions we can draw thus far are:
- Connectivity might still be an issue. Although all our participants managed to complete their assessments, a sizeable portion did experience some form of connection problem. This perhaps relates to the South Africa’s internet infrastructure still not being as robust and widespread as other countries. Employers should perhaps be cognizant of such limitations when using especially mobile-based assessment solutions.
- Smartphones are a viable means of assessment. Given the ease-of-use results combined with our impression we derived from interviews that Millennials especially are “device-agnostic”, we would argue that from a perceptual point-of-view, mobile and smartphone assessments pass a basic litmus test of credibility for candidates in selection situations.
- Recruitment and selection processes are brand-central. It seems very clear from the perceptions of our candidates that companies have a powerful opportunity to build their brand through professional, well-run recruitment and selection processes. Of course, the opposite also holds true, and much brand damage can be done if such processes are disorganised or below candidates’ expectations. If nothing else, assessments, and mobile-based assessments seem to result in a net gain in brand equity for organisations that use them.
If you would like to know more about our ongoing research on mobile and smartphone assessments, or if you’re interested in how TTS can help you make better talent decisions, why not drop us a line at info@tts-talent.com? We would love to hear from you!