Candidate screening: How to assess candidates 3x more accurately

Joe Caccavale

14

July

2020

|

8

minute read

X

We're rebuilding the hiring process.

Want to see more content like this? Subscribe to our fortnightly newsletter.

Give me more

You're elbow deep in CVs. 

You’ve managed to shortlist a handful of promising-looking candidates.

But how do you know you’ve chosen the right people?

What if we told you that there is a way to prove you’ve got the best people...

And that you could be interviewing 3x as many suitable candidates, just by switching your candidate screening method?

Why spend 63 hours screening resumes when you could find the same number of top candidates in just 18 hours?

At Applied, we’ve made it our mission to make hiring as data-driven and predictive as humanly possible…

And this is the candidate screening process we’ve found to be the most effective and time-efficient.

The candidate screening guide:

  • Work samples: a quick definition
  • The science behind human decision making and the problem with CVs
  • How to create work sample questions

Our candidate screening secret: Work samples

Here at Applied, we don’t use CVs to hire.

We use work samples - job-specific questions designed to test how a candidate would think and perform in the role. 

We didn’t axe CVs on a whim. Work samples have been proven to be 3x more predictive than CVs. They’re the single most predictive candidate screening method.

Work samples are intended to simulate the actual job as closely as possible by getting candidates to think through or actually have a go at tasks/situations that would legitimately need to be tackled should they get the job.

We recommend basing your work samples around specific skills needed for the role. 

If a job required 6-8 essential skills, then you’d have 3-5 work samples that test these skills.

*We’ll come back to the work sample creation process later on

Here’s an example of one of our own work samples:


Consider this stat: We found that 60% of people hired through our work sample process would’ve been missed in a traditional CV sift.

To understand how this happens and why work samples are so much more effective than a resume screening, it helps to have a little behavioural science context... 

Here’s the behavioural science behind human decision making (and why resume screening has to go)

The average CV review takes just 7.4 seconds.

With a looming pile of CVs to wade through, we’re forced to make a snap decision, but based on what?

The truth is: resume screening relies on unconscious bias

Unconscious bias is the term used to describe certain prejudices that we may be unaware of, and are therefore out of our direct control, often stemming from stereotypes that arise from our backgrounds, cultures, and personal experiences.

This bias is completely natural but you don’t want it to affect your hiring.

To get a better grasp of how unconscious bias works, let’s look at how the brain actually makes decisions…

We take 1000’s of decisions every day.

What to wear, what to have for breakfast, where to stand on the platform, when to get off the train… and that’s before you even arrive at work.

Our brains have two ‘systems’ for processing these decisions.

System 1: Quick, initiative thinking that relies on short-cuts and associations (like being on auto-pilot).

System 2: Slow, considered thinking that requires more effort and reasoning.

Source: Thinking Fast and Slow


Without system 1, we’d be having a meltdown before leaving the house in the morning due to all the decisions we need to make.

System 1 is absolutely necessary… but we sometimes use system 1 when we should be using system 2.

Given that we tend to spend less than 10 seconds reviewing a CV, it’s safe to assume system 1 is being used.

This means that we’re using fast, gut-driven thinking to make a decision that should be made by the more conscious and logical part of our brain.

This is why work samples are so effective: they force you to use system 2, with no excess information to base snap-associations and shortcuts on.

Think about the sort of information provided on CVs:

  • Name
  • Address
  • Previous employers 
  • Education

All of this information is grounds for bias and says nothing about someone’s ability to do the job.

From this information alone you’re supposed to take a wild guess at who the most qualified candidate might be.

Work samples were designed to skip the part where you guess, and instead, ask candidates to demonstrate the exact skills you’re looking for.

The impact of this bias: poor diversity and lost talent

Considering that much of our unconscious bias is based around gravitation towards the familiar and stereotypes, candidates miss out due to factors irrelevant to the actual job.

Those from minority backgrounds are disproportionately overlooked.

This doesn’t necessarily mean that hiring managers are bad people. They’re just human.

Stereotypes dictate who we expect to see in certain roles - this is bred into us as children and perpetuated throughout our lives.

Here’s an example:


When googling nurse, 81% of people pictured were women, and when searching for images of surgeons, 68% were male.

This sort of stereotyping trains us to think of nurses as women and surgeons as men.

Although this is a very basic example, you can begin to see how this could play out in hiring. 

We have a certain image in our head of the type of person we expect to see in a given role and naturally tend to look for someone who fits that mould.

… which can have a detrimental effect on minority groups.

We often refer to this German study to drive this point home.

The same resume was sent out, changing only the name and appearance of the candidate.

The result: Maryem Ozturk would have to send 40% more applications to get the same number of callbacks.



That’s why we use work samples - to screen candidates based only on skill

To use work samples to their full potential, we’d recommend doing them ‘blind’ to completely remove unconscious bias from decision making.

This means removing all identifying information from applications and letting candidates answers speak for themselves.

For more info about blind hiring, you can read our ultimate guide.

Yes, losing the CV seems like a daunting proposition but the evidence speaks for itself - work samples are actually a safer option than a resume screening.

Guess what - education and experience aren’t predictive of ability

Here’s another one of our go-to studies courtesy of Schmidtt & Hunter.


As you can see, work samples are the most predictive assessment method, whilst eduction and experience performed pretty poorly.

Sure, someone’s degree or years of hard-earned experience can (and often do) make them the best person for the job.

But instead of just assuming they’re the best based on this alone, work samples put candidates’ ability to the test… the best person for the job often isn’t who you think it is.

How to create work samples

Work samples can be created in 4 easy steps:

  1. Decide on the 6-8 skills you want to test. Ideally, these would be a mix of technical and soft skills/ characteristics.
  1. Think of real situations that the candidate would come across in the job that would test these skills.
  1. Turn these into questions, posing them hypothetically (ask ‘what would you do…’)
  1. Give yourself scoring criteria so you can stay objective. What would a 1-star and a 5-star answer look like?

Deciding skills

Before you can start coming up with questions, it’s best to start by thinking about the role itself, and the skills that would be required for the new hire to succeed.

They don’t all need to be job-specific, technical skills. They can also include working characteristics like teamwork or entrepreneurship.


Imagine real situations that might occur

The best work samples are the ones closest to real life. 

So, why not use real situations that have come up in the past to see how candidates would’ve reacted.

Similarly, if there’s an upcoming project or a daily duty the candidate would be tackling, why not use this as the basis for a work sample question?

Turn situations into questions

Since we’re not concerned with experience, simply take the situations you thought up above, and pose them as hypothetical, ‘what would you do’ questions.

It can be as straightforward as outlining the situation/ context and then asking what the candidate would do to address the situation.


A few other ideas:

  • List of tasks to prioritise 
  • Email to draft
  • Small blog post to write
  • Project to plan
  • Conflict to resolve 

If you have a combination of 3-5 of these, you should be able to test candidates on the key areas of the role before even interviewing them, so that you only need to interview those who have proven they can do the job.

Give yourself scoring criteria

Your review process should be as objective as the candidate screening method itself.

Instead of going with the answer that you ‘feel’ is the best, or jumps out the most, write down rough criteria for each question.

We use a 1-5 star scale for scoring work samples. It doesn’t need to be too detailed, just bullet point what a good, bad and average answer would look like.

What should candidates have taken into consideration? 

Should they be scored higher if they’ve done their research?

*You may also want to get other team members to score answers too. Having at least two other reviewers will draw on crowd wisdom - the rule of thumb that says collective judgement is generally more accurate than that of an individual.

Use our (free) work sample cheat sheet

Feel free to use this cheat sheet to get a head-start ion your work sample creation - we threw in some real examples that we used to hire Applied Team members which you can tinker with and repurpose for your own roles.

Applied was built to make hiring as predictive and fair as possible, by designing behavioural science into every step of the process. Browse our resources to get a better idea of how we do this, or go ahead and start your free trail.