October 2015

This will come as a surprise to a lot of people, but in some cases it's possible to detect bias in a selection process without knowing anything about the applicant pool.

Which is exciting because among other things it means third parties can use this technique to detect bias whether those doing the selecting want them to or not.

You can use this technique whenever (a) you have at least a random sample of the applicants that were selected, (b) their subsequent performance is measured, and (c) the groups of applicants you're comparing have roughly equal distribution of ability.

You can sometimes detect bias in a selection process without seeing the applicant pool at all — so third parties can detect it whether the selectors like it or not.

You need only a random sample of those selected, a measure of their performance, and groups of roughly equal ability.

In some cases you can detect bias in a selection process without knowing anything about the applicant pool — so third parties can use the technique whether the selectors want them to or not.

How does it work?

Think about what it means to be biased.

What it means for a selection process to be biased against applicants of type x is that it's harder for them to make it through.

Which means applicants of type x have to be better to get selected than applicants not of type x. [1] Which means applicants of type x who do make it through the selection process will outperform other successful applicants.

And if the performance of all the successful applicants is measured, you'll know if they do.

Of course, the test you use to measure performance must be a valid one.

And in particular it must not be invalidated by the bias you're trying to measure.

But there are some domains where performance can be measured, and in those detecting bias is straightforward.

Want to know if the selection process was biased against some type of applicant?

Check whether they outperform the others.

This is not just a heuristic for detecting bias.

It's what bias means.

Bias against type x means it's harder for them to get through — so they have to be better, so the type-x applicants who do get through outperform the others. Measure everyone and you'll know.

Given a valid test, detecting bias is straightforward: check whether the type in question outperforms the others. This isn't just a heuristic. It's what bias means.

For a process to be biased against type x means x has to be better to get selected, so the selected x's outperform the others. Measure performance and you'll know. This isn't a heuristic — it's what bias means.

For example, many suspect that venture capital firms are biased against female founders.

This would be easy to detect: among their portfolio companies, do startups with female founders outperform those without?

A couple months ago, one VC firm (almost certainly unintentionally) published a study showing bias of this type.

First Round Capital found that among its portfolio companies, startups with female founders outperformed those without by 63%. [2]

Many suspect VCs are biased against female founders. Easy to detect: do their female-founded portfolio companies outperform the rest? One firm, almost certainly unintentionally, published exactly that — First Round Capital found theirs outperformed by 63%.

Many suspect VCs are biased against female founders — easy to detect. First Round Capital, almost certainly unintentionally, found that among its portfolio startups with female founders outperformed those without by 63%.

The reason I began by saying that this technique would come as a surprise to many people is that we so rarely see analyses of this type.

I'm sure it will come as a surprise to First Round that they performed one.

I doubt anyone there realized that by limiting their sample to their own portfolio, they were producing a study not of startup trends but of their own biases when selecting companies.

I predict we'll see this technique used more in the future.

The information needed to conduct such studies is increasingly available.

Data about who applies for things is usually closely guarded by the organizations selecting them, but nowadays data about who gets selected is often publicly available to anyone who takes the trouble to aggregate it.

We so rarely see such analyses that I doubt First Round realized they'd performed one: by limiting the sample to their own portfolio, they studied not startup trends but their own biases.

I predict we'll see this more. Data on who applies is closely guarded, but data on who gets selected is increasingly public to anyone willing to aggregate it.

We rarely see analyses like this; First Round didn't realize that by limiting the sample to its own portfolio it was studying its own biases. I predict we'll see it more, as data on who gets selected becomes public.

[1] This technique wouldn't work if the selection process looked for different things from different types of applicants—for example, if an employer hired men based on their ability but women based on their appearance.

[2] As Paul Buchheit points out, First Round excluded their most successful investment, Uber, from the study. And while it makes sense to exclude outliers from some types of studies, studies of returns from startup investing, which is all about hitting outliers, are not one of them.

This fails if the process judges types by different criteria — men on ability, women on appearance. And as Paul Buchheit notes, First Round excluded Uber, its biggest hit — wrong for studies of startup returns, which are all about hitting outliers.

The technique fails if the process judges different types by different criteria. And First Round excluded Uber, its biggest hit — a mistake in studies of startup returns, which are all about outliers.

Thanks to Sam Altman, Jessica Livingston, and Geoff Ralston for reading drafts of this.

Thanks to Sam Altman, Jessica Livingston, and Geoff Ralston for reading drafts.

Thanks to Sam Altman, Jessica Livingston, and Geoff Ralston for reading drafts.