“It’s a Pandemic of Racism”: The Failure of Data, Implicit Bias and Systemic Discrimination

Published by:
Kate Glazebrook
June 11, 2020
min read

Debiased Hiring Bundle

Authors: Kate Glazebrook and Khyati Sundaram

Just over two weeks ago, an unarmed Black man - George Floyd - was killed while in police custody on a street in Minneapolis. 

The raw footage, made possible through the courage of strangers to continue filming even as police repeatedly told them not to, makes for horrifying viewing.  Listening to these same bystanders make impassioned pleas for the police to use less force and to check the unconscious man’s pulse, fills you with simultaneous despair at the inhumanity of the police response and hope at the humanity of random strangers. 

What we might think of as an unusual event, is sadly more usual than even those knowledgeable about social justice might believe.  In the US alone, over 1,000 citizens were killed by police in the US last year; and the victims were disproportionately from Black and minority ethnic communities.  

But we only know that thanks to people, not the transparency of governments. In one of the great deliberate acts of institutional ignorance, there’s no official data source for these incidents: yet again, the unrelenting effort of civil society has had to rise up in the face of systemic failure.

According to data collated by Sam Singyangwe and DeRay McKesson as part of the Mapping Police Violence project, there were just 27 days last year when there wasn’t at least one person killed by a member of a police force in America.  

Their data (which if anything, under-report) show that Black citizens are 3 times as likely to be killed by the police than White citizens are, and yet they’re also 1.3 times less likely to be armed than their White counterparts.  

Contrary to common belief, they show that there is little to no relationship between local crime rates and police killings, a finding which accords with other research indicating it is the culture of the specific part of the police force that shapes the frequency of violence.  

Data collected by the Washington Post in 2017 highlighted the compounding intersectional effects: while Black males made up 6 per cent of the US population that year, they represented 34 per cent of all killings of unarmed citizens.  

The other side of the criminal justice system is as grim: a Black man born in 2001 has a 1 in 3 chance of being incarcerated at some point in his lifetime, compared to 1 in 18 for a Black woman, 1 in 17 for a White man, or 1 in 111 for a White woman. 

And if that wasn’t depressing enough, communities of colour are also failed by a lack of accountability: fully 99 per cent of the police killings from 2013-2019 have resulted in no criminal charge (Mapping Police Violence).

Tragically, therefore, in a statistical sense, George Floyd’s death is anything but extraordinary.

But what has emerged in response to his death certainly is. 

Protests under the banner of Black Lives Matter have spread - and sustained - not only across the US but across the world.  At last count, tens of thousands of people from communities of colour and their allies have marched in towns and cities in the UK, Australia, South Africa, Portugal, Germany, Italy, France, Brazil, Hong Kong, Kosovo, Tunisia, South Korea, Belgium, Japan, Bulgaria, and Ireland.  

The second order impact of these led to Black Out Tuesday on social media platforms like Instagram, a renewed focus on Black Lives Matter, and a call for transparency and accountability regarding black talent and leadership. 

And thankfully, most media outlets are making efforts to explain that the overwhelming majority of these protests have been peaceful and respectful - at least avoiding many instances in the past where selective journalism scandalising protest movements undermines them.

Why now? The pandemic of racism

So if his death is just one of many, why now?  What makes this moment different? 

It’s a question scholars of social movements and civic progress will debate in blogs, journals and history books.  Was it the visceral nature of the death?  The fact it was captured on camera in full?  Was it fuelled by the data showing under-represented groups are more likely to be affected by coronavirus?  Or maybe the sheer unreality of this period of lockdown?  

At George Floyd’s memorial last week, Reverend Al Sharpton suggested that life spent in lockdown rendered salient realities that were previously ignored: “And maybe in God’s own way, the fact we were sheltered means we couldn’t watch sports, couldn’t watch nothing, we had to keep watching that tape. Maybe because we had no distractions, that finally we were ready to deal with this”.  

And the ‘this’ he refers to is the scourge of discrimination, disenfranchisement, oppression and inequality that cuts across domains, geographies, and languages for under-represented groups.  As the Reverend later put it: 

"George Floyd’s story has been the story of Black folks. Because ever since 401 years ago, the reason we could never be who we wanted and dreamed to be is that you kept your knee on our neck. We were smarter than the underfunded schools you put us in, but you had your knee on our neck. We could run corporations and not hustle in the street, but you had your knee on our neck. We have creative skills, we could do whatever anybody else could do, but we couldn’t get your knee off our neck. What happened to Floyd happens every day in this country, in education, in health services, and in every area of American life.”

And just as the stats presented above on race and criminal justice are tragic, so too are the parallel stories of racial inequality in other aspects of everyday life.  

In the world we know best - employment - a seminal study in 2004 by Marianne Bertrand and Sendhil Mullainathan revealed that African American candidates in the US needed to send 50 per cent more CVs/resumes out than their White counterparts to get the same rate of call-back for a job interview.  This result held across industries, organisation sizes, and occupations.  

They calculated this was the equivalent of asking African American candidates to have 5 more years of job experience to be considered equally qualified for the job.  

A meta-analysis of similar US studies conducted from 1989 to 2015 by Lincoln Quillian, Devah Pager, Ole Hexel and Arnfinn Midtbøen found that while there were some tentative signs of a reduction in discrimination against Latinx candidates, “[w]e find no change in the levels of discrimination against African Americans since 1989 … The results document a striking persistence of racial discrimination in US labor markets”.

Evidence from hiring bias experiments over recent decades (Source )

Depressingly - and perhaps by way of explanation for the widespread nature of the solidarity across other cultures - equivalent studies catalogued by Bertrand and Nobel Laureate Esther Duflo have been carried out in countries all over the world.  They show candidates from under-represented groups (be they by ethnicity, sexual orientation, religion, gender, and others) are systematically overlooked for jobs and other forms of application processes.  

Here in the UK, for example, a recent meta-analysis of UK-based studies conducted by Oxford’s Nuffield College found that exactly equally qualified ethnic minority candidates needed to send 60% more CVs to get the same rate of call-back for interview, and “[w]orryingly, discrimination is an enduring phenomenon. Comparing these results with those from previous field experiments conducted in Britain, we found no sign of progress for Caribbeans or for South Asians over the past 50 years.” 

Considering that your job opportunities can affect not only your income, your personal fulfilment, your job security and your ability to be a role model for others, this represents an inconceivable cumulative loss to Black and other under-represented individuals the world over, let alone the collective loss to us all through lost potential.  

Now, let’s be clear: what happened to George Floyd and to the scores of people before him is due to criminal justice inequality.  It wasn’t a missed job opportunity that killed Mr Floyd.  

But as Rev Sharpton and countless others have pointed out, reforming policing and justice is critical, but we can’t stop there, discrimination and oppression pervades other spheres as well.  These inequalities accumulate and exacerbate one another, resulting in massive disparities across racial and ethnic lines that become determinative for many people.  

Structural racism in the criminal justice system both creates and is reinforced by other forms of economic and social inequality. 

And because we are all shaped by the world around us, including who occupies what roles in it, for us as individuals and the organisations we work in, it’s not as simple as just saying “we don’t discriminate” and moving on.  

As one of those marching in the UK, Aaron Cikaya, put it when interviewed by the BBC:

“We want equality. We want the same opportunities as everyone else.  I’ve seen a lot of companies coming out with posts being like “Black Lives Matter, we support the movement”, but then when I go on your website and I look at the Board members, nobody looks like me.  I go a bit further down, but still nobody looks like me. And when I walk into a big company, or any company, the only people who seem to look like me are the ones cleaning” 

Explicit and implicit bias

In trying to make sense of all this, people have turned to the role that bias plays in creating the conditions for, and reinforcing, inequality.  Many argue that some police forces are inculcated in a system which perceives Black men in particular as a threat, resulting in conscious and unconscious actions that see them disproportionately targeted and harmed by police.  In effect, that their demography alone is worthy of a different response.  

As Baratunde Rafiq Thurston put it in his TED talk on deconstructing racism last year:

“I am asking people here to see the structure. Where the power is in it. And even more importantly, to see the humanity of those of us made targets by this structure. I am here because I was loved and invested in and protected and lucky. Because I went to the right schools, I’m semi-famous, mostly happy, and meditate twice a day. And yet, I walk around in fear. Because I know that someone seeing me as a threat can become a threat to my life. And I am tired. I am tired of carrying around this invisible burden of other people’s fears.  And many of us are.  And we shouldn’t have to.”

Without a doubt - though it pains us to admit it - there are people who are not only implicitly, but explicitly, biased against others.  

There are some people who genuinely do not believe that all people are born equal.  There’s also another, sometimes overlapping, group who stand to gain from the status quo, and who conflate the drive for equality with the loss of their privilege, their status, their power.  Sadly, some people in each of these groups sit in high offices.

But that’s not everyone.  And though it may include some of the (very) powerful, there is also power in numbers, as we have seen these past two weeks.  

When it comes to the data on societal attitudes, we see that we are, as a collective, becoming more tolerant and less likely to believe that the way a person looks or their background determines what they’re capable of (positively or negatively).  

Measures of bias are fraught with academic debate (and often limited to an American context), but longitudinal studies at the very least help us to identify some patterns of change.  The team that oversees the Implicit Association Test, including Tessa Charlesworth and Mahzarin Banaji, recently examined 10 years of US data in both explicit and implicit bias (that is the degree to which you would agree with a statement such as “I prefer men to women” versus the extent to which - unconsciously - you associate more positive terms with men than women (regardless of your stated preference)).  

Analysing data from over 4 million tests looking at bias on the basis of race, skin tone, gender, disability, age, sexual orientation, and body weight between 2007 and 2016, they found strong evidence of a decline in explicit bias and a movement toward neutrality across all 6 groups in the US.  For race, there was a 37% shift toward neutrality, and for skin tone, it was 21% (see Image 2 below).  Extrapolating, Americans should reach attitude ‘neutrality’ on these measures in 2-3 years.  On the face of it, this is good news.  

Change in explicit bias in attitudes over time - race and skin tone (Source)

But we’re imperfect, and even when we want to be unbiased, sometimes our brains - conditioned in an imperfect, structurally unequal world - reveal something different.  And that’s where implicit biases live.  

Over the same period, they found that the decline in implicit bias was only 17% and 15% respectively (see Image 3 below).  Plus, implicit biases were starting from a higher base - since we’re less aware of, and therefore unable to socially moderate our implicit biases.  

Their extrapolations show that on the current trend, it will take a staggering 58 and 139 years respectively for us to reach ‘neutrality’ in implicit racial and skin tone bias.  

On both explicit and implicit measures - the average change was driven almost entirely by younger generations (millennials and GenZs), with little or no change in attitudes for baby boomers or GenXs.

Change in implicit bias in attitudes over time - race and skin tone (Source)

This gap between explicit and implicit bias matters because even if we could shift everyone to actively supporting concepts of equality - removing the explicit attitudinal barriers to change - we would still have an uphill battle in the minds of the vast majority of us (including the authors of this article) who unconsciously fall prey to negative stereotypes.  

And as the field of behavioural science has documented over decades, implicit biases reign supreme in a host of daily experiences: they influence our actions when we’re thinking quickly, we’re cognitively overloaded, we’re tired and we have limited information. 

And if you think about it, that’s most of the decisions we take in a day.  They drive how we interact with others - especially strangers - and those interactions then reinforce everyone’s collective unconscious understanding of how the world actually works, creating a vicious cycle.  

It’s worth noting that there is some debate about how direct the connection between an IAT score is, and how biased your actions in real life are.  But putting all this together with the dire lack of progress in real-world hiring decisions discussed above, not to mention the continuing (and indeed growing) rates of income and social inequalities in education, health and other areas, and it’s clear that this rate of change is far too small and too slow. 

So what now?

Many well-meaning organisations keen to do more than issue blog posts or solidarity tweets will be reaching to bring in unconscious bias or diversity training right now. Indeed there’s already about USD8 billion spent by corporates in the US each year on them, and this spending goes up after major incidents, like Starbucks did in 2018 following the discriminatory treatment of a Black man waiting for his colleague.  

And while the urge to learn and to build a common understanding in an organisation of the reality of bias is commendable, we would urge those who do so to not confuse doing that with making real change.  Sadly the reality is that these programmes have no impact on future bias and in some unfortunate cases, they actually backfire.  This is because they give us an easy ‘checkbox’ in our minds which unconsciously leaves us to be more biased in the future.  

But that’s not the same thing as trying to better understand the problem in your organisation, from the inside out.  

That means taking the time to not do what the US police system does by not measuring what actually happens because the answers might make you uncomfortable.  Data can reveal truths we don’t want to know, but as research by Elizabeth Hirsch and Donald Tomaskovic-Devey has found they also spur action and accountability which we know definitely does help.  It helps to diagnose where issues lie and sometimes to sequence where to act first.

And while you’re going through the exercise of collecting and analysing data, make sure to look for it in words not just numbers.  People’s lived experiences offer a richness of insight that excel spreadsheets do not.  And they can be powerful and persuasive and memorable in ways numbers aren’t.  Take, for example, the following series of Instagram posts from Toni Okungbowa:

See more here

Scenarios like the ones Toni described are now being used - with some tentative signs of success - as a starting point for virtual reality experiences that give you an idea of what it can feel like to have people react differently to you in the street because of how you look.  

While sadly almost all the efforts to re-wire bias out of our unconscious brains have failed, there is some possible hope in using visceral, personal experiences to start to shake out stereotypes and build empathy.  And technologies like Debias VR developed by Clorama Dorvilias are part of that story.  (Note that we don’t put as much hope in most of the implementations of AI to dig us out of this hole, for reasons we’re covering separately.)

There are also some small possible rays of hope of learning from the significant reduction in anti-gay/lesbian bias found in the same IAT data, reported by Charlesworth and Banaji in a more recent article.  Over the same time horizon, implicit bias on that basis has declined by 33% (twice that of race), and is observed across age groups not just the young.  While theories differ on what has driven this substantial shift toward equality, some of them might offer opportunities to drive equality for other groups. 

But not all organisations have access to experimental virtual reality labs, or to the keys to changing societal attitudes, and most are keen to see change in their lifetimes, so what can people actually do?  

Prof Iris Bohnet - who, bias alert: has been one of Applied’s board members and has worked closely with us designing our own debiasing technology - puts it as simply as: “Start by accepting that our minds are stubborn beasts. It’s very hard to eliminate our biases, but we can design organizations to make it easier for our biased minds to get things right.”

As a team who has built a platform that uses behavioural science to debias hiring decisions, at Applied, we’re obviously big believers in the power of well-designed technology to drive equality through systematically changing the way humans evaluate others

We aim to use what we know about how the brain works to make sure hiring teams focus on what matters (actual skills and what a candidate can bring to the role) not what doesn’t (how that person looks, sounds, or what their name is).  The end goal is the best person gets the job, regardless of their background.

And we know that solutions that aren’t designed with empathy will always fall flat. Thankfully, though, tech’s history of (in)advertely reinforcing discrimination in product design is slowly changing.  And we’re not alone, there’s a growing band of tools out there to help individuals and organisations identify, mitigate and learn from bias, and designed by people who care about doing it inclusively.    

But we also know we alone are only one small part of what’s needed: there’s an entire ecosystem of forces of change required to overcome centuries of inequality.  From how we draw electoral maps to how we provide educational opportunities, from the faces and names we see portrayed in popular culture to whose startup ideas get funded.  

We aim to be part of the solution, but we are very far from having answers to all the important questions.  People will want or need to engage in these topics in different ways, and thankfully that diversity of approach should ultimately help us collectively drive the widespread change we need so that this past fortnight, and the horrific event that precipitated it, aren’t just lost in a sea of headlines.  

We’ve compiled a far from comprehensive, but hopefully valuable, list of places you might want to go to next. 

With sincere thanks to Aaron Cikaya and Toni Okungbowa for graciously allowing us to include their perspectives, and Andy Ayim and the wider Applied team for their comments

Some resources you might find useful (beyond those listed above)

For those who want to hear more about people’s experiences and great compilations of materials

Why I am no longer talking to white people about race - Reni Eddo-Lodge book and podcast (plus, she helpfully does her own podcast recommendation list here)

How to deconstruct racism, one headline at a time - Baratunde Rafiq Thurston (video), How to be Black (book) and Living While Black Headlines (repository)

The immortal life of Henrietta Lacks - Rebecca Skloot (book)

Black(ish): On race, identity and belonging - Afua Hirsch (book)

The Black cup of excellence: Being Black in specialty coffee - Michelle Johnson (article)

Various pieces - Mona Chalabi (data-art)

An open letter to leaders trying to improve diversity - Andy Ayim (article) and also his Minorities in Tech (newsletter)

An opinion piece - featuring COVID19, George Floyd, and Venture Capital - Vanessa Sefa and Joshua Olusanya (article)

Why White people stay silent on racism, and what to read first - Adam Grant (article)

For those who want some recommendations to folks who do forms of bias awareness in organisations we think are likely to be more effective

Fearless Futures

Move the Elephant

More than Now

For those who want to delve more into behavioural science, unconscious/implicit bias and equality in the workplace

What Works: Gender equality by design - Iris Bohnet (book)
(NB while her book focuses on gender bias efforts to address gender equality, much of the research and suggestions for organisations are valuable in the pursuit of for all forms of equality)

Thinking Fast and Slow - Danny Kahneman (book)

Nudge - Richard Thaler and Cass Sunstein (book)

Implicit association test - Project implicit (test)

Diversity and Inclusion Efforts that Really Work - David Pedulla (article)

Using behavioural science to build the police force of tomorrow - the Behavioural Insights Team (report)

Making the Cut: Hiring Decisions, Bias, and the Consequences of Nonstandard, Mismatched, and Precarious Employment - David Pedulla (book)

How diversity makes us smarter - Katherine Phillips (article)

The Inclusion Nudges Guidebook: 100 how-to behavioral designs to de-bias and make inclusive behavior, culture, and systems the default and norm - Lisa Kepinski and Tinna Nielson (book)

The Kirwan Institute for the Study of Race and Ethnicity: Ohio State University (multiple articles)

Invisible Women: Exposing data bias in a world designed by men - Caroline Criado Perez (book)

The Social Animal: A story of how success happens - David Brooks (book)

Diversity in venture capital: A practical toolkit for VC funds - Diversity.vc (toolkit)

Rebel Ideas: The power of diverse thinking - Matthew Syed (book)

For those who want to learn more about tech solutions (incl AI and the risk of algorithmic bias)

Human Decisions and Machine Predictions - Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, Sendhil Mullainathan (article)

AI: Ain’t I a Woman? - Joy Buolamwini (video)

Made by Humans: the AI condition - Ellen Broad (book)

Algorithms of Oppression: How search engines reinforce racism - Safiya Umoja Noble (book)

Gender Shades: Intersectional accuracy disparities in commercial gender classification - Joy Buolamwini and Timnit Gebru (article)

Using technology to increase fairness in hiring - Kelly Trindal, Frida Polli, Kate Glazebrook (article)