Our clients often want to know about the best time of day for candidates to complete ability tests (and by implication, whether time-of-day and ability assessment results will be related).
Often, guidelines such as those published by the Institute of Psychometric Coaching suggest that candidates are “well rested” before completing such assessments. However, the guidelines do not make reference to a specific time-of-day.
So, is there a better time-of-day to complete ability assessments? Our research team investigated TTS’s massive database of assessment results to find out.
Background to the study
There are conflicting research findings regarding time-of-day and its relationship with ability assessment results in psychometric tasks.
For instance, in a study of test performance in Danish children attending public schools over a four-year period, researchers found that for every hour later in the day, test performance decreased by 0.9% of a standard deviation. The authors of the study reasoned that as the day wears on, students become increasingly tired and are therefore more likely to underperform on standardised tests.
However, other studies found that it may benefit candidates if they complete their ability assessments later in the day rather than earlier.
To this point, researchers examining time-of-day effects on the basic cognitive processing of college students, found an advantage for fluency and digit symbol task performance in the afternoon and evening over morning sessions.
Based on these results, the researchers argued that executive functioning and processing speed may be at optimal levels in the afternoon and evening (at least for typical college students).
Yet another line of inquiry in this question focussed on circadian rhythms: unique activation cycles that vary across individuals.
Such studies have suggested that cognitive performance may depend more on individuals’ specific optimal period, as determined by their circadian rhythms.
Given the uncertain findings in the literature, the research team at TTS embarked on their own study of how (or if) the time-of-day of ability assessment completion may impact candidates’ measured performance.
The current study
In our research, we wanted to address the following 3 questions:
- Are ability scores different between candidates who completed their assessments in the morning compared to those who completed in the afternoon?
- Are ability scores different between candidates who completed their assessments during working hours versus those who completed outside of working hours?
- Combining (1) and (2) above, were ability performances affected by morning, afternoon, and working hours completion when viewed together?
To answer these questions, we used three years’ worth of assessment data from three commonly used ability tests:
- Verbal reasoning
- Numerical reasoning
- Deductive-Logical reasoning.
Fortunately, there was a lot of data to analyse, and our main sample contained 20 000 test results!
In addition to individual test results, we created a composite score by averaging each candidate’s verbal, numerical and deductive-logical reasoning results. Random sub-samples were then drawn from the different time-of-day completion groups to ensure equivalence on key demographic variables such as age and gender.
More than half (N = 10 475) of our candidates completed their ability tests in the morning (6 am to 12 pm), while the remainder (N = 8 831) completed their assessments in the afternoon (12 pm – 10 pm).
Due to the large sample sizes under study (N > 2 500 in each time completion group), even small differences between groups can flag as statistically significant.
Because of this risk, we considered both a test of statistical significance and practical significance (Cohen’s d-statistic effect size).
Question 1: Are ability scores different between candidates who completed their assessments in the morning compared to those who completed in the afternoon?
Although we observed statistically significant differences between the mean score for morning vs. afternoon/evening completions, the practical significance of that difference was very small (d-statistic of 0.15; see graph below).
This suggests that there are very few differences in ability performance between candidates who completed their assessments in the morning versus the afternoon.
This finding lends support to other studies that have found negligible covariance between time-of-day and ability assessment results.
Question 2: Are ability scores different between candidates who completed their assessments during working hours versus those who completed outside of working hours?
As in Question 1 above, we observed statistically significant differences in average scores between candidates completing their assessments during vs. outside working hours. But practically speaking, the mean difference between the two groups was very small (d-statistic of 0.11; see graph below).
Question 3: Combining (1) and (2) above, were ability performances affected by morning, afternoon, and working hours completion when viewed together?
Similar to the results for the previous two questions, statistically significant differences were detected between the average scores for completions before working hours, during morning working hours, during afternoon working hours, and after working hours.
But unlike our previous findings, we did observe a small effect size difference between completions after hours and completions before working hours (d=0.23), and between completions after hours and during morning work hours (d=0.27; see graph below).
What do these differences indicate?
Candidates who completed their assessments in the late afternoon, after work hours, may have enjoyed a slight improvement in their performance.
Contrary to the common belief that candidates should complete assessments earlier in the morning, our research findings suggest that later, after working hours (5pm – 10 pm) completion might be more conducive to better performance.
This finding still needs to be viewed with caution, mainly due to the small effect size observed.
In addition, we weren’t able to control for every conceivable intervening influence other than time-of-day on ability assessment results, so there may well be alternative explanations for this effect.
But perhaps the most interesting result is the general absence of time-of-day effects across our samples. Therefore, the best practice course of action might be to continue advising candidates to be well rested or prepared for their assessments, but not to prescribe a specific time-of-day for completion.
If you are interested in how TTS might be able to offer evidence-based answers to your IO Psychology and talent management questions, why not drop us a line at: email@example.com?
Institute of Psychometric Coaching. (n.d.). The Psychometric Guide: Top 10 Tips to Prepare for a Psychometric Test. Retrieved from https://www.psychometricinstitute.com.au/Top_10_Tips_to_Prepare_for_a_Psychometric_Test.html
Sievertsen, H. H., Gino, F., & Piovesan, M. (2016), Cognitive fatigue influences students’ performance on standardized tests. PNAS Proceedings of the National Academy Sciences of the United States of America. Retrieved from https://doi.org/10.1073/pnas.1516947113
Allen, P. A., Grabbe, J., McCarthy, A., & Bush, A. H. (2008). The Early Bird Does Not Get the Worm: Time-of-day effects on college students’ basic cognitive processing. American Journal of Psychology, 121,(4), 551-564.
May, C. P., Hasher, L., & Stoltzfus, E. R. (1993). Optimal time of day and the magnitude of age differences in memory. Psychological Science, 4(5), 326-330.