Skip to content
85,000 candidates completed assessments through the Amberjack Volume Assessment Platform in October 2024.
This number smashed all previous records. Whilst some of this increase can be explained by a record year of sales for Amberjack in 2023/24, this certainly doesn’t explain it all.
Martin Kavanagh, Amberjack’s Head of Assessment, believes that we’re seeing the results of the perfect storm of:
- The removal of unnecessary, and exclusionary, minimum requirements in early career role descriptions.
- Less barriers to entry with fewer organisations asking for CVs or lengthy application forms at the first stage of the process.
- Rapid advancements in tech and just how tech savvy candidates are.
Across October, we also closely monitored a high-volume campaign using video interviews scored manually by Amberjack assessors. We trained all our assessors to spot where candidates are using technology to create verbatim responses. We found that 8% of candidates were, without doubt, using generative AI to create answers for them, and were not even attempting to add their own authenticity to the content. This is a big challenge as, at the volumes we’re seeing, 8% is a significant minority and academic research (e.g. Canagasuriam and Lukacik, 2024) is showing that AI generated responses to video interviews score well and would likely pass a typical benchmark.
The Amberjack Insights report found that a surprising 60% of organisations didn’t change anything about their processes in response to the threat of generative AI in 2023/24. It is likely that number will – and should – go up in 2024/2025. Changes seem most likely in initial assessment stages, where CVs, Online Assessments, and Video Interviews are used. This is where the impact of AI was more widely reported by respondents of our Employer Survey.
To request the full Employer Trends Insight Report for 2024, visit this page.
If you are making changes in the 2024/25 campaign year, here are the things Martin would recommend introducing as a minimum:
- Be clear in your communications to candidates about expectations around technology use and consequences if it is clear they have completely plagiarised a response/gamed an assessment. We are constantly updating and improving our default candidate communications and it’s having an impact.
- Monitor your data. Make sure you are not seeing unexpected spikes in performance or significant cohort shifts. As a minimum, monitor whether assessment scores at each stage of the process are predicting performance at the next.
- Train those involved in your process, and the technology you use, to spot data anomalies and clear cheating behaviour from candidates.
- Use multi criterion assessments over single-criterion assessments.
What About the Candidate Experience?
It’s worth noting, the above recommendations are quite “greedy” – looking at what organisations should do to help themselves.
Let’s now think about this from the candidate’s perspective who will likely be applying for many more roles and have far more choice. We often hear candidates talking about which assessment provider’s assessment they completed, without necessarily knowing which organisation they were completing that assessment for. This is unsurprising given our Insights 2024 research suggests less than half of organisations are using assessment processes completely bespoke to them.
Organisations who tailor all stages of the assessment process to their brand, values and roles, and create a consumer-grade experience for candidates will have a competitive advantage. We have heard this directly from members of Amberjack’s Gen Z Advisory Board, made up of young people who participate in our research and events, who say that being provided with a realistic job preview and an opportunity to look into your organisation, during the assessment process, is a big green flag. This is why we are so clear that the assessment process needs to be an extension of a hiring organisation’s attraction activity.
