In this Insights post, Frances Brooks Taplett, Broad’s Chief People Officer, talks with Shuba Gopal, Senior People Scientist on Broad's HR team, about the adjusted pay gap and how Broad analyzes and ensures pay equity.
Each year, the American workforce crosses an important and depressing threshold on August 13: this day represents how much farther into a given year a Black person who identifies as a woman would have to work to earn what a man earned the previous year. You’re probably familiar with the dismal numbers. Across the U.S., the average person who identifies as a woman makes 19 percent less than the average person who identifies as a man. Black/African Americans and Latinx workers of all genders make 33 percent less than all other workers, and as much as 50 percent less.
As we think about structural racism and sexism and how to address it at the Broad, a clear place to start is making sure everyone is paid equitably. Pay equity—equal pay for comparable work—is foundational to ensuring a diverse workforce. Besides, it’s the law! Massachusetts passed a pay equity law in 2018 (joining 42 other U.S. states).
With this in mind, I sat down with my colleague Shuba Gopal, Senior People Scientist on our HR team, to discuss what we can do about pay equity...
Shuba: First, we have to know what problem to solve. The metric cited most often in the popular press is known to economists as the unadjusted pay gap. It’s a simple ratio: the average salary of all the people of interest (let’s say people who identify as a woman, for example, but it could just as easily be any other demographic category) divided by the average salary of all the other people (people who identify as men, in our example). That’s a nice metric because it’s simple and easy to understand.
Frances: But, that’s the tricky part: it assumes people can freely choose to do every type of work possible. Sadly, we know that’s not true. Because of systematic biases and subtle cultural influences, some people end up more often in certain types of work than others. For example, 95 percent of preschool teachers in America identify as women, but 75 percent of software engineers identify as men. And as a society, we seem to believe that the work of software engineers is more valuable—we pay higher salaries for that work—than caring for preschoolers. The unadjusted pay gap ignores these social inequalities.
Shuba: Why some types of work are deemed more valuable than others is an important question, and no single organization can change the way society currently values different types of work. But, within Broad, we need to be able to account for why one person’s pay is different from another person’s pay, and to distinguish between any legitimate, external reasons for that difference—for example education, unique skill sets, or prior experience—and bias.
To do that, Broad’s compensation team used to apply the unadjusted pay gap within specific contexts. For example, they would look at the pay of all research scientists in a single program or platform and be confident that some of the external factors, like education and skill sets, were similar for all the people being compared. But from such narrow comparisons within specific contexts, we can’t extrapolate to understand the differences in pay across the institute. We use a different approach now, but the last time the compensation team assessed the pay gaps this way, in the fall of 2017, the unadjusted pay gaps were small within specific contexts (around one to three percent for people who identify as men versus women), but across all the jobs and employees of Broad, it was 11 percent. That is, the average Broadie who identifies as a woman made about 89 cents for every dollar that the average Broadie who identifies as a man earned.
This percentage, however, doesn’t properly account for legitimate reasons why there are differences in pay, and more importantly, it doesn’t account for the fact that Broadies who identify as women are more frequently represented in some types of work than others.
We use a different approach now: the adjusted pay gap. This approach uses linear regression. Essentially, it asks, “After accounting for the known reasons why pay varies—by type of work, education, previous experience, time at Broad, etc.—are there any pay differences remaining?” If there are, these are attributed to bias.
Frances: Yes, I’ve read about this, in fact, in my understanding, when such an analysis was conducted at the national level in the U.S., the adjusted pay gap for people who identify as women was 5.4 percent. That’s not great, but it’s much smaller than the more frequently cited 19 percent (the unadjusted pay gap).
Shuba: An adjusted pay gap analysis is not hard to do, but it does present some challenges in a smaller organization like Broad. When I started work on this analysis in the fall of 2017, I discovered we had just under 1,200 employees and nearly 400 unique job titles. How could I assess equal pay for comparable work if nearly a third of employees had unique jobs? A second challenge is that there are fewer people at the more senior levels of the institute, and smaller sample sizes make it harder to conduct a robust linear regression analysis. Many other organizations seem to sidestep these issues by excluding unique job holders and senior executives in their analyses, but that’s not satisfactory. We want a robust, statistically sound approach to evaluating all Broad employees’ pay.
As a former computational biologist, I had some tricks up my sleeve for handling the problem of small sample sizes. In this instance, I aggregated some of the data and applied permutation testing to assess pay equity for all Broad employees in all roles and at all levels of seniority.
For a highly simplified example to show how this works, let’s pretend we’re trying to assess female-identifying pay equity for Acme, a company that has only software engineers and exactly five levels of seniority. (See chart, “Software Engineering Roles.”) And to keep things simple in this example, we’ll use the binary “female/male” definition of gender, although we know it’s an inadequate metric in the real world.
Ideally, we would run an adjusted pay gap analysis on each seniority rank by itself, but that won’t work here because there aren’t enough people in each rank to yield a statistically robust result from regression analysis. So instead, we aggregate some of the ranks together. Let’s group Ranks 1 to 3 together as “junior software engineers” and Ranks 4 to 5 as “senior software engineers.” Now we have enough people in each category to run the regression analysis.
The result is a three percent pay gap in favor of people who identify as men.
We know there’s also a big representational skew. In our new “senior software engineers” category, there are 43 people who identify as males and three people who identify as females. Ten of the 43 people who identify as males make substantially more. That may be creating a bigger pay gap than actually exists since we’re comparing those ten people who identify as men’s salaries to three more junior, lower-paid people who identify as women.
So the second step is permutation testing, which can help us test the hypothesis that representational skews are driving part of the pay gap we observe. We randomly swap the male-identifying/female-identifying labels on the pay data while leaving all other variables the same, run the linear regression to compute the pay gap—and then repeat this procedure 10,000 times to compute an empirical distribution of pay gaps. (See chart, “Example Permuted Pay Gap Distribution.”)
The mean of this distribution indicates how much representation skews the adjusted pay gap. By permuting the male-identifying/female-identifying labels, we’ve erased any systematic bias that resulted in higher pay for people who identify as males over people who identify as females.
In this example, the mean of the empirical distribution is 1.6 percent. In other words, of the three percent adjusted pay gap observed, 1.6 percent is actually due to representational skews. Subtracting that out, we get an actual adjusted pay gap of 1.4 percent. (We describe all the details of this approach—in gory mathematical detail—in Assessing Our Workplace—Part I.)
Frances: Shuba, can you share what we’ve been doing to address this at the Broad?
Shuba: Since January 2018, Broad has implemented following pay equity analyses:
About every month, we run the adjusted pay gap analysis with permutation testing. We conduct four separate analyses: male-dentifying versus female-identifying, Black/African American versus others, Latinx versus others, and persons with disabilities versus others. When we find statistically significant pay gaps, we immediately make salary adjustments. As a result, our overall adjusted pay gaps continue to be small, not statistically significant and close to 0 percent.
The pay equity analysis described here provides a snapshot of the aggregate pay equity for the institute. That doesn’t mean discrepancies don’t still exist. The Broad compensation team is constantly monitoring pay equity as we hire people and as Broadies leave the institute, and we make salary changes as needed.
These processes help ensure that we pay everyone at Broad fairly and equitably. It’s not perfect, but it does account for all Broad employees (unlike other organizations that drop or skip some subsets) and provides a statistically robust assessment. It gives us some confidence that we’re being fair and equitable in this important area of the employee experience.
Still, there are groups that we can’t evaluate using this approach. At the moment, we don’t have a reliable way to evaluate pay equity for groups of less than 10 individuals. For example, there are fewer than 10 Broadies who identify as Indigenous. We aren’t able to use this approach to evaluate pay equity for them. We have a similar challenge evaluating pay equity for trans or gender fluid individuals.
Frances: How do you think we should monitor pay equity now, during the pandemic? There are worrisome trends in the workforce with more mothers reducing hours or leaving work altogether to care for children. And we know Black/African Americans and Latinx people have disproportionately worse outcomes than others if they contract COVID-19. How should we think about these factors when evaluating pay equity at Broad?
Shuba: It’s hard to say because, as everyone keeps pointing out, these are unprecedented times! I think there are a few things we’re doing now that might be helpful, and we should think carefully about how we might want to continue to monitor and modify our approach to pay equity assessments going forward.
For example, in the past, we’ve evaluated pay equity based on a person’s total salary (that is, their salary assuming they worked full time, even if they are currently working part time). But if we start to see more Broadies moving to part-time, we might want to evaluate their pay based on the actual pay they receive rather than what we project they would have made were they full time. That might capture the differential effect of some subsets of our community disproportionately moving to part time. Then we could consider ways to ensure equity in their pay given they are not working full time.
Frances: That’s right, and of course, pay equity is just one piece of the puzzle. We also need to monitor and make sure our other policies like paid time off, family leave, and stretch assignments are supportive of Broadies’ needs. Across HR, from BroadLife, to People Development, and the IDEA office we are striving to work on these issues. We hope you’ll join us next week at the Belonging@Broad Symposium next week to continue this important discussion.
