01.05.2024

Research finds that recruiters are not consistent when reviewing resumes

Research from Kickresume tasked six recruiters with matching a pool of 12,000 pairs of job adverts and candidate resumes. What they didn’t know was that they would see the same resumes twice.

Recruiters were found to be subjective. Recruiters didn't consistently agree on candidates' suitability for job positions, even when assessing the same candidates.
Recruiters were found to be inconsistent. When recruiters were shown the same resume twice, their decisions were inconsistent, particularly when assessing suitable candidates.
Recruiters performed similarly to AI. AI demonstrated a level of consistency similar to human recruiters, suggesting the potential for automation in recruitment.

Peter Duris, the CEO and Co-founder of Kickresume, comments: “We were fascinated to see the level of variance in the results. One of the most interesting discoveries we made in this research was that AI is already performing at a similar level to human recruiters, in terms of its ability to select the same candidates. AI technology is a fantastic time-saving tool - this research suggests that it might be able to assist recruiters in reading through resumes and selecting the best candidates for the job, or at least excluding those who are not suitable.

“For job seekers, these results might seem disappointing, but there is a positive way of looking at this. Having the knowledge that recruiters will not always agree on who to select based on their resume means that job seekers can be reassured that it doesn’t necessarily mean their resume is bad or their experience isn’t sufficient if they are not chosen to advance to the interview stage. It just means that, on this occasion, the recruiter did not choose them. So I’d advise job seekers to not be disheartened by this! Take another look at your resume, tweak it for the next job you’re applying for, and get back out there.”

Recruiters were found to be fairly inconsistent. When reviewing each resume and job advert, they could either answer ‘yes’ or ‘no’. Kickresume used a metric called Cohen’s Kappa which measures the level of agreement between different people’s answers to a question. Cohen’s Kappa can vary from 0, nearly random, to 1, total agreement. The average Cohen's Kappa score was 0.49, which falls roughly half-way between the two ends of the scale.

When they were shown the same resume twice, recruiters consistently identified the candidate as suitable only 40% of the time - which is to say the recruiter said 'yes' on both occasions. However, the recruiters did consistently identify the candidates whose resumes were unsuitable, with the percentage of those who they said ‘no’ to stable at 91%.

Lastly, Kickresume used an industry-leading AI job matching tool to assess the 12,000 resumes. Measuring the level of agreement between the AI and the human recruiters, the Cohen's Kappa score was 0.45 - only slightly lower than the level of agreement between the pool of six recruiters. The AI used was more advanced than the well-known ATS software which relies on filtering resumes based on keywords.

In response to these results, Kickresume advised job seekers to consider applying to a larger number of roles, casting a slightly wider net and putting their application in front of a higher number of recruiters.

While recruiters did not always agree on who the best candidates were, they consistently agreed on which candidates were unsuitable. Therefore, your resume should closely match the job description, and showcase the experience, skills and qualifications that make you a good pick.

Posted by: FMJ 0 comment(s)