3 Features Your Applicant Tracking System Should Have if You Want to Improve Diversity

Published by:
Joe Caccavale
March 11, 2022
6
min read

Debiased Hiring Bundle

1. Anonymous applications 

FACT: All humans are prone to unconscious bias. It doesn’t matter how pure your intentions may be, we’re all subject to biases that affect our judgment.

To deal with the 1000’s of micro-decisions we make each and every day, our brains use two systems to lighten the load. One is reserved for more effortful, one-off decisions whilst the other relies on shortcuts and patterns to draw quick conclusions - a theory popularized by Daniel Kahneman’s ‘Thinking Fast and Slow’.

System 1: used for everyday, intuitive decisions - like how much milk to put into your coffee. For the most part, this sort of decision making is a bit like being on autopilot.

System 2: used for bigger, more important decisions like planning a trip or working on a big presentation. System 2 thinking tends to be slower, considered, and more mentally taxing.

System 1 vs System 2 thinking

Since system 1 relies on mental shortcuts based on things like stereotypes, past experiences and what we’ve seen in the media, it has serious negative consequences when it comes to people-based decisions.

When it comes to hiring, for example, our natural tendency to resist the unfamiliar and warm to those most like ourselves result in candidates from underrepresented backgrounds being disproportionately overlooked.

Callback rates for different candidates (chart)


Awareness and good intentions alone won’t move the needle on diversity.

This is why unconscious bias/diversity training doesn’t work.

As the chart below shows, although training does increase awareness around diversity, this doesn’t result in reduced bias.

Effect of training on gender bias (chart)


The only way to make genuine change is to make sure our recruitment technology and practices are designed in a way that reduces bias.

We know that the following information can all trigger unconscious biases:

  • Names
  • Photos
  • Addresses
  • Dates of Birth
  • Hobbies

If you want to make measurable improvements when it comes to diversity, your applicant tracking system should have anonymization options - even if just to hide the basic information above.

2. Predictive assessments

To hire a diverse array of talent, you’ll need to make sure you’re testing for what matters… 

Not all talent assessments were created equal.

‘Predictive validity’ - a concept used in science and psychology - is a measure of how accurately an assessment can forecast future outcomes or behaviour. 

In terms of hiring, we’re essentially looking at how reliably a given assessment can predict someone’s performance on the job.

The more predictive an assessment is, the more effectively you’ll be able to identify talent. 

Keeping the emphasis on predictive validity means that there’s never a tradeoff between diversity and quality of candidate.

By reducing bias and putting the emphasis solely on skills, diversity will improve.

Predictive validity studies take a long time to complete and demand large sample sizes in order to draw any meaningful conclusions.

However, we can look to the Schmidt–Hunter meta-analysis for answers - which summarizes the findings of 100 years of research in personnel selection.

The purpose of this meta-study was to evaluate how accurately various assessment methods can forecast an individual’s future job performance.


Predictive validity of assessment methods (chart)


As you can see, the information CVs provide doesn’t say much about someone’s ability.

Work samples are the most predictive assessment you can use - and so here at Applied, we adapted for office-based jobs.

We use interview-style questions designed to test the specific skills required for the job. 

They’re not too dissimilar from ‘tell me a time when…’ interview questions - the key difference is that they pose questions hypothetically.

The basic idea is to simulate the tasks and challenges candidates would face, should they get the job.

Example work sample question


Whilst a work sample-based approach won’t be possible through most applicant tracking systems, the Schmidt-Hunter study should give you a rough overview of what works and what doesn’t.

The problem with most applicant tracking systems is that they simply make age-old, biased hiring practices slightly more efficient, rather than replacing them with something better.

Just ask yourself: can we prove that our assessment process actually identifies the best people for the job? And if so, are you able to quantify someone’s skills?

No matter which applicant tracking system you use, having review guides for your assessments is essential.

This will enable you to generate objective, number scores and make decisions using data.

Ideally, you’d use a 1-5 star scale for each work sample - with bullet points noting what a good, bad and mediocre answer would include.

Applied soft review screenshot


If every assessment is given a number score, then you can simply average out candidates’ scores to decide who to bring through to the next round.

Having even the most rudimentary scoring system will give you data points to use in decision making.

We know that what we might call our ‘gut instinct’ is actually just unconscious bias, which is why scoring criteria is so important to ensure that candidates are being measured against something concrete.

Applied candidate leaderboard screenshot


3. Diversity reporting 

You can’t change what you can’t measure.

To be able to make any meaningful changes to your organization’s diversity, you first need to be able to track the diversity of the candidates that are sourced and eventually hired. 

Yes, we want an anonymous process, but we also want to be able to track how fair and effective it is.

Applied diversity tracking screenshot


Once you know how different groups progress through your hiring process, you can begin to identify and tweak any stages or individual assessments that might favour one approach over another.

If you see candidates drop off at a particular stage or question, you can then take a closer look at whether the framing or review guide is disadvantaging a particular group. 

Applied diversity analytics screenhot


Corporate hiring processes in the U.S, for example, have been shown to prefer a masculine style of leadership

You’ll also want to ensure that you start the process with a diverse candidate pool.

The less diverse your initial pool is, the higher the chance you’ll hire someone from an overrepresented demographic.

Studies have shown that when there’s just one woman in the finalist pool, their chances of being hired are statistically zero.

Finalist pools vs hiring decisions (chart)


Before asking for any personal background information, you’ll want to communicate to candidates that this information is only ever going to be used at an aggregate level (meaning candidates will remain anonymous) and will never be used to identify individuals. 

There are plenty of valid reasons for a candidate to be sceptical… so sharing diversity information should never be a requirement.

This information can be collected via an equal opportunities form.

What is Applied? Why use our platform?

We built the Applied platform specifically to make hiring fair.

By anonymizing candidates and using skill-based assessments, our applicant tracking system helps organizations sustainably improve diversity and hire the best possible talent.

Instead of just automating practices we know are broken, we built a process with diversity baked into every step.
Applied job builder screenshot


Instead of asking candidates about their past achievements, we use a work sample-based process to find candidates with the skills and values needed for the job, regardless of their background.

Using objective, data-backed assessments, we’re even able to generate automated, personalized feedback for every single candidate (even those who are unsuccessful).

Applied feedback


What are the benefits of hiring the Applied way?

  • Up to 4x attraction and selection of ethnically diverse candidates
  • 3x as many suitable candidates
  • 66% reduction in time spent hiring
  • 93% retention rate after one year
  • 9/10 average candidate experience rating
Effect of skill-based hiring of STEM diversity (chart)


Key takeaways: Using your applicant tracking system to 

When it comes to making genuine improvements to diversity, the only way to see tangible results is by using a process that reduces bias by design.

Whether you’re looking for a new applicant tracking system or just starting to invest in hiring tech, there are some basic practices we’d recommend to make your process fairer and more diversity-driven:

  • Anonymous applications
  • Work sample questions
  • Scoring criteria
  • Diverse interview panels 
  • Diversity tracking 
How to hire ethically graphic

Applied is the essential platform for debiased hiring. Purpose-built to make hiring empirical and ethical, our platform uses anonymized applications and skill-based assessments to identify talent that would otherwise have been overlooked.

Push back against conventional hiring wisdom with a smarter solution: book in a demo