When faced with unique or difficult recruitment and selection challenges, organisations are sometimes reluctant to use traditional assessments to filter out unsuitable candidates.
This is, in part, because of the perceived cost and complexity of psychometric measures, but also because recruiting managers fear that traditional testing is too abstract to accurately communicate the company’s brand and values to prospective employees.
Although IO Professionals understand the power of psychometrics, it can be a difficult proposition for hiring managers who require greater face validity and a closer alignment to actual job tasks. In such cases, Situational Judgement Tests (SJTs) can bridge that divide. SJTs offer robust predictive capabilities that are comparable to those of psychometric measures. But, they have the added advantage of being closely aligned to work behaviours.
In addition, SJTs are often costed more competitively once developed, making them attractive for high-volume testing.
Selecting millennials: A Swedish case study
Recently, a Swedish resort group wanted to recruit around 2000 young, talented millennials for entry-level hosting jobs at their resorts around Scandinavia. The group had a very specific and unique brand offering that was expressed by their core values of safety, compassion, fun, and efficiency. Given the popularity of the resort brand in the youth market, recruiting managers were anticipating strong interest in the advertised positions and needed an effective filter that would be useful in screening out unsuitable candidates for the next selection hurdle, an interview round.
To achieve these goals, they partnered with TTS assessment partner, Aon Assessment Solutions, to develop a customized SJT solution. The main goal of the test was to screen out candidates who wouldn’t be likely to express the group’s core values in their everyday interactions with resort guests.
Developing the SJT
Aon assessments and the resort group started their development of the SJT by interviewing eight senior resort managers. The interviews generated 92 possible scenarios that entry-level workers would likely face in their daily interactions with guests and staff. Scenarios that were likely to generate values-driven behaviours were emphasized in this stage of development.
The initial 92 scenarios were converted into SJT items and tested using existing resort workers. Personality and ability tools provided by Aon were also used as validation instruments in the study.
Analyses focused on identifying SJT items that would predict: the number of compliments or complaints received by staff from resort guests, managers’ ratings, and alignment with personality attributes and ability scores. Based on the study, the 92 items were reduced to 25 items that were most predictive of the criteria.
The advantage of this validation approach is that SJT items are immediately shown to be predictive of actual work performance. Pairing the items with known psychometric measures helped to check for both discriminant and convergent validity.
Implementing the SJT
The now-reduced SJT items were delivered to candidates as part of the group’s online recruitment portal. As predicted, interest in the positions was strong, and more than 20,000 applicants applied. Using the SJT, the client was able to screen out around 15,500 applicants. The remaining 4,500 applicants were invited for interviews, which resulted in the target of 2,000 hires being reached.
Further investigation revealed that the SJT was an especially good predictor of actual work behaviour, as measured by manager ratings and guest feedback. In addition, SJT results added predictive validity to the standard personality and ability testing that the resort group used for certain positions.
Clearly, the SJT measured different, yet important, job-related variables compared to traditional psychometric instruments.
Conclusions
Although not new, SJTs have recently become more popular for organisations interested in conducting mass screening of applicants, especially in a niche or highly contested industry segment. When paired with mature online technologies, SJTs seem to offer strong utility for their users and come with the added advantage of being easy to justify, sell, and explain to hiring managers.
Candidates also benefit from SJTs because of their high face validity. Actual work behaviours are presented, so test-takers perceive SJTs as fair and practical measures of suitability to a job. In addition, the slick, well-designed technology that is paired with SJTs appeals to millennials, who expect a high level of design and online savviness when interacting with organisations. Therefore, a knock-on benefit of using SJTs is that they can effectively communicate a company’s brand promise to prospective applicants.
If you are interested in applying SJTs to your selection process or would like to know how TTS can help you make better talent decisions, why not drop us a line at info@tts-talent.com? We can’t wait to hear from you!