At TTS, our assessment clients often want to know whether time of day or other, non-psychometric factors may influence their candidates’ performance on talent assessment measures.
For instance, our research team investigated if cognitive capability assessments are best completed early in the morning, a common belief among assessment end-users (and developers). In that study, based on close to 20,000 individual assessments, we found that there was no significant influence of time of completion on assessment results.
Therefore, at least some common beliefs about test-taking conditions turn out to be incorrect.
In a similar vein, our research team was intrigued by a study by James Bywater and SHL in 2005, titled Early Bird Graduates get the Worm, which investigated assessment results of graduates who applied early in a recruitment process to those who applied later and nearer to the completion deadline.
In this study, Bywater and his colleagues found that motivational personality traits, persuasiveness, forward-thinking traits, and cognitive abilities of so-called “early bird” applicants were significantly better than applicants who applied later. However, when it came to resilience and sensitivity to criticism, so-called “laggard” applicants showed better results.
Based on the study, our research team decided to investigate whether local graduate applicants showed similar trends in their talent assessment results.
Our study
Initially, our study focused on determining if there were significant differences between graduates who completed their talent assessments earlier within the allocated timeframe and those who completed their assessments closer to the deadline.
Our sample comprised 215 graduates who had applied for a consulting graduate position. As part of the assessment process, the applicants had 10 days (from a Monday to the following Wednesday) to complete the following award-winning personality and ability assessments from Aon Assessments:
- The ADEPT-15
- Scales clx
- Scales verbal mobile
- motionChallenge
- gridChallenge
- Scales numerical mobile
- gapChallenge
- Scales mt
Our team analyzed the graduates’ results and tested for the influence of time-based factors on assessment results similar to the research done by Bywater.
Results
When analyzing the data for proximity to the deadline period, we could not replicate Bywater’s findings. The results of our study showed no significant differences in ability or personality traits between applicants who completed their assessments early or nearer to the deadline day.
So at least in our applicant sample, the assessment results suggested that there are no benefits for “early worm” applicants over their “laggard” counterparts.
However, our team looked at various permutations of time of completion, and how such factors could have affected candidate results, and found a significant effect for completion time, just not the one they expected.
For our applicants, there were significant differences in personality traits between those who completed the assessments during the week (Monday to Friday) and graduates who completed their assessments on a weekend (Saturday and Sunday).
Graduate applicants who completed the assessments on a weekend scored higher on conscientiousness, in particular, on being structured, planful, detail-oriented, and rule-conscious (ADEPT-15 Structured, d = 0.44), as well as being dependable, hardworking, and accountable (ADEPT-15 Drive, d = 0.39), both of which are subsections of the Task style (d = 0.54) as shown in the graph below:
A further interesting finding was that graduate applicants who completed the assessments during the week scored higher on being outgoing and energetic, tending to be more sociable and friendly (ADEPT Lively).
Conclusion and final thoughts
Based on the findings of this study, it seems that when recruiting and selecting graduates, hiring organizations may be well served to have their assessment stages run across weekends and weekdays.
Only allowing completion during weekdays may result in less conscientious graduates completing their assessments, with a resulting lower selection of such applicants.
The reasons for our finding are not yet entirely clear, but several logical connections come to mind:
- Conscientious candidates may view their current working weekday activities, such as studies or part-time employment as important, and this commitment may be a causal factor in rather completing assessments in “down-time”, i.e. weekends.
- Alternatively, highly conscientious candidates may prefer reserving weekends for assessment completion, reasoning that they would be less likely to be distracted as would be the case while at work or engaged in their studies.
The above are only speculative explanations, and further research will be required to fully understand this effect. Our research team looks forward to investigating the link between weekend completion and conscientiousness in future studies – so, watch this space!
If you are interested in our work in graduate selection and talent assessments, why not drop us a line at info@tts-talent.com?