Must candidates complete ability tests in the morning?

Our clients often want to know about the best time of day for candidates to complete ability tests (and by implication, whether time of day and ability assessment results will be related).

Certain test publishers have even made recommendations regarding when the best times might be to complete their measures. For instance, aptitude and cognitive assessments are sometimes believed to be best completed in the morning, when, presumably, candidates are well rested.

While guidelines such as those published by the Institute of Psychometric Coaching suggest that candidates are “well rested” before completing such assessments, they do not make reference to a specific time of day.

So, is there a better time of day to complete ability assessments? Our research team investigated TTS’s massive database of assessment results to find out.

Background to the study

There are conflicting research findings regarding time-of-day and its relationship with ability assessment results in psychometric tasks.

For instance, in a study of test performance in Danish children attending public schools over a four-year period, researchers found that for every hour later in the day, test performance decreased by 0.9% of a standard deviation. The authors of the study reasoned that as the day wears on, students become increasingly tired and are therefore more likely to underperform on standardised tests.

However, other studies found that it may benefit candidates if they complete their ability assessments later in the day rather than earlier.

To this point, researchers examining time-of-day effects on the basic cognitive processing of college students, found an advantage for fluency and digit symbol task performance in the afternoon and evening over morning sessions.

Based on these results, the researchers argued that executive functioning and processing speed may be at optimal levels in the afternoon and evening (at least for typical college students).

Yet another line of inquiry in this question focussed on circadian rhythms: unique activation cycles that vary across individuals.  

Such studies have suggested that cognitive performance may depend more on individuals’ specific optimal period, as determined by their circadian rhythms.

Given the uncertain findings in the literature, the research team at TTS embarked on their own study of how (or if) the time-of-day of ability assessment completion may impact candidates’ measured performance.  

The current study

In our research, we wanted to address the following 3 questions:

  1. Are ability scores different between candidates who completed their assessments in the morning compared to those who completed in the afternoon?
  2. Are ability scores different between candidates who completed their assessments during working hours versus those who completed them outside of working hours?
  3. Combining (1) and (2) above, were ability performances affected by morning, afternoon, and working hours completion when viewed together?  

To answer these questions, we used three years’ worth of assessment data from three commonly used ability tests:

  • Verbal reasoning
  • Numerical reasoning
  • Deductive-Logical reasoning.

Fortunately, there was a lot of data to analyze, and our main sample contained 20 000 test results!

In addition to individual test results, we created a composite score by averaging each candidate’s verbal, numerical and deductive-logical reasoning results. Random sub-samples were then drawn from the different time-of-day completion groups to ensure equivalence on key demographic variables such as age and gender.

More than half (N = 10 475) of our candidates completed their ability tests in the morning (6 am to 12 pm), while the remainder (N = 8 831) completed their assessments in the afternoon (12 pm – 10 pm).

Due to the large sample sizes under study (N > 2 500 in each time completion group), even small differences between groups can flag as statistically significant.

Because of this risk, we considered both a test of statistical significance and practical significance (Cohen’s d-statistic effect size).

Results

Question 1: Are ability scores different between candidates who completed their assessments in the morning compared to those who completed in the afternoon?

Although we observed statistically significant differences between the mean score for morning vs. afternoon/evening completions, the practical significance of that difference was very small (d-statistic of 0.15; see graph below).

Ability results based on time-of-day completion

This suggests that there are very few differences in ability performance between candidates who completed their assessments in the morning versus the afternoon.

This finding lends support to other studies that have found negligible covariance between time-of-day and ability assessment results.

Question 2: Are ability scores different between candidates who completed their assessments during working hours versus those who completed them outside of working hours?

As in Question 1 above, we observed statistically significant differences in average scores between candidates completing their assessments during vs. outside working hours. But practically speaking, the mean difference between the two groups was very small (d-statistic of 0.11; see graph below).

Ability results for working hour vs. non-working hour completion
Question 3: Combining (1) and (2) above, were ability performances affected by morning, afternoon, and working hours completion when viewed together?  

Similar to the results for the previous two questions, statistically significant differences were detected between the average scores for completions before working hours, during morning working hours, during afternoon working hours, and after working hours.

But unlike our previous findings, we did observe a small effect size difference between completions after hours and completions before working hours (d=0.23), and between completions after hours and during morning work hours (d=0.27; see graph below).

Combined time-of-day and working hours ability results

What do these differences indicate?

Candidates who completed their assessments in the late afternoon, after work hours, may have enjoyed a slight improvement in their performance.

Final thoughts

Contrary to the common belief that candidates should complete assessments earlier in the morning, our research findings suggest that later, after working hours (5pm – 10 pm) completion might be more conducive to better performance.

This finding still needs to be viewed with caution, mainly due to the small effect size observed.

In addition, we weren’t able to control for every conceivable intervening influence other than the time of day on ability assessment results, so there may well be alternative explanations for this effect.

But perhaps the most interesting result is the general absence of time-of-day effects across our samples. Therefore, the best practice course of action might be to continue advising candidates to be well rested or prepared for their assessments, but not to prescribe a specific time of day for completion.

This finding (and recommendation) is perhaps not that surprising if we consider the substantial volume of research conducted on how circadian rhythms can result in diurnal variations in emotional expression and mood (see for instance, Caminada & De Bruijn, 1992 and Timmermans et al., 2020). Plainly put, there is strong evidence to suggest that some people are “morning people” whereas others are “evening people” (including a host of variations in between).

Future research will focus on this interesting possibility, possibly leading to more nuanced and tailored prescriptions for candidates who are about to complete ability assessments: Perhaps we first need to determine their optimal functioning time of day before recommending an ideal time for testing.

If you are interested in how TTS might be able to offer evidence-based answers to your IO Psychology and talent management questions, why not drop us a line at: info@tts-talent.com?


Sources

Allen, P. A., Grabbe, J., McCarthy, A., & Bush, A. H. (2008). The Early Bird Does Not Get the Worm: Time-of-day effects on college students’ basic cognitive processing. American Journal of Psychology, 121,(4), 551-564.

Caminada, H., & De Bruijn, F. (1992). Diurnal variation, morningness‐eveningness, and momentary affect. European Journal of Personality6(1), 43–69.

May, C. P., Hasher, L., & Stoltzfus, E. R. (1993). Optimal time of day and the magnitude of age differences in memory. Psychological Science, 4(5), 326-330.

Institute of Psychometric Coaching. (n.d.). The Psychometric Guide: Top 10 Tips to Prepare for a Psychometric Test. Retrieved from https://www.psychometricinstitute.com.au/Top_10_Tips_to_Prepare_for_a_Psychometric_Test.html

Sievertsen, H. H., Gino, F., & Piovesan, M. (2016), Cognitive fatigue influences students’ performance on standardized tests. PNAS Proceedings of the National Academy Sciences of the United States of America. Retrieved from https://doi.org/10.1073/pnas.1516947113

Timmermans, T., Van Mechelen, I., & Kuppens, P. (2010). The Relationship between Individual Differences in Intraindividual Variability in Core Affect and Interpersonal Behaviour. European Journal of Personality24(8), 623–638.