The UBC Faculty of Science Equity and Inclusion Scholars Reading Group is reading the paper Impact of equity models and statistical measures on interpretations of educational reform by Idaykis Rodriguez, Eric Brewe, Vashti Sawtelle, and Laird H. Kramer. The paper argues that educational research aimed at increasing equity should approach two factors more carefully: what “equity” means and how to report results so they are most usable. For the former, the paper presents three different models of equity. For the latter, the paper encourages reporting effect sizes with confidence intervals and not just statistical significance, both to make results more meaningful and to facilitate meta-analyses. They then reinterpret the results of Reducing the gender gap in the physics classroom by Mercedes Lorenzo, Catherine H. Crouch, and Eric Mazur under different models of equity with careful evaluation of effect sizes.

The paper’s discussion of statistical reporting and its re-analysis of the Lorenzo et al. results are both interesting, but they’re not what caught my attention. I read another paper recently—Student Perceptions of Fairness and Security in Versioned Programming Exams by Chinedu Emeka and Craig Zilles—where I found the paper results interesting but really got caught up in the models presented in related work. In that paper, it was different models of justice in grading practices by which students might judge a course.

In this paper, I got nerd-sniped by the models of equity. What is equity, and what do we mean by it?

I had a vague idea in my head from previous papers that matches the equity vs. equality graphics out there, although I hadn’t actually seen them before I discussed the paper with my partner and daughter and they both said it was “like the fence diagram”.

Two of this paper’s models generally match my understanding of the most common contrast drawn in those pictures. The third is an interesting alternative (or maybe two), and not really the same as the “liberation” or “justice” models presented in some of those fence diagrams.

Below, I’ll discuss the notions of equity the paper laid out, using completely fake data on pre-test and post-test results in some educational intervention for Indigenous and White students. In my examples, the Indigenous students enter with lower average scores because of the previous impacts of racism. If that already feels racist (assuming Indigenous students are worse than White students, insisting on comparing Indigenous students to White students), then keep reading to the third definition of equity.

Equity of Parity

First is equity of parity:

… parity is achieved when the distributions of achievement scores in two or more groups have the same postinstruction average despite differences in preinstruction averages. … one must acknowledge that the instruction benefits the ‘‘less prepared’’ students more than the ‘‘well prepared’’ students.

Loosely speaking, this matches my pre-existing definition of equity. We allocate resources to different “types” of students so that everyone achieves similar learning outcomes. We “close the achievement gap” as both the Rodriguez et al. paper puts it and many other resources online.

The paper does point out that this may not quite be what’s intended in those graphics, however. Equity of parity in outcomes in a single course is an episodic sort of equity. If successful, it closes the achievement gap in this one course in its learning outcomes, leaving untouched the inequities that led to the gap in the first place and the outcomes of other metrics outside the reform’s specific focus.

Equity of Fairness

Next is equity of fairness:

… defining equity as that which promotes justice and freedom from bias or favoritism [2]. … The equity of fairness perspective implies an impartial treatment that would be reflected in an equal gain (posttest score – pretest score) in conceptual understanding for all students groups, regardless of initial understanding or group.

The paper’s definition of equity of fairness loosely matches my pre-existing definition of equality. This approach is aiming for a “colour-blind” outcome. Everyone is treated equally in some measurable sense. There’s more to say to even truly nail down this definition. Should we say the “same gains” are achieved if the mean difference in score is the same? The mean percentage increase in score? The “normalized learning gain”?

It feels clear looking at this that it is a disappointing definition of equity, maybe even a “straw” argument. However, on a second read, I realized this actually is another notion of equity and not quite the notion I previously had for equality. That’s because despite differences in their circumstances coming in, both groups had the same gains. That may well mean that more resources are allocated to the group that had been disadvantaged coming in.

A more strict notion of “equality” might mean that resources are allocated to students entirely independent of which group they belong to, or perhaps independent of any other factors. Measuring the success of this equality approach doesn’t involve pre-test or post-test results at all; it involves measuring the distribution of resources without reference to outcomes.

Despite that, this model is problematic at best in that it perpetuates the initial inequity between groups.

Equity of Individuality

Last is equity of individuality:

…relying on a comparison group perpetuates the idea that marginalized students are not worth studying in their own right and therefore sustains upper-middle class whites as the normative group [5,6].

The equity of individuality model promotes individual excellence of students within groups.

It took me three readings and a quick read of A “Gap-Gazing” Fetish in Mathematics Education? Problematizing Research on the Achievement Gap by Rochelle Gutiérrez to feel like I understood what was going on here. I see this as a lens on equity that differs in two important ways from the ones above.

First, the focus is solely on the specific group for whom we seek justice. In my imaginary example, we have identified that Indigenous students have been and are subject to injustice. In both previous definitions of equity, the response is to place them in opposition to a “norm” of White students and to try to make the Indigenous students do as well as the White ones. Gutiérrez points out that that perspective bakes in marginalization and injustice. Why shouldn’t we instead focus on the circumstances, needs, and desires of the Indigenous students exclusively? Why are Indigenous students themselves not a group worthy of study?

Second, the focus is on individual achievement. Returning to Gutiérrez again, they point out that the distribution of achievement within each of these groups is going to dwarf the difference between groups. So, why not focus on trying to help individuals achieve excellent outcomes?

Entangled in this as well is a greater focus on context and explanation rather than simply success or failure. My knee-jerk reaction is to shout “qualitative methods” to the rooftops when I see that, but Gutiérrez again points out that while qualitative methods may well be important to this lens, the distinction is not really about qualitative vs. quantitative but about the two factors above: a focus on the specific group and a focus within that group on individual excellence.

As a lens, this model focuses attention on my choice in my fake data to compare (imaginary) Indigenous students to (imaginary) White students and my real choice in manufacturing the data to place the Indigenous students’ pre-test scores below the White students’ scores. Gutiérrez’s paper emphasizes how the achievement gap model of equity plays into this problematic narrative of deficiency. Why not invent this world:

In a sense, this model dodges the question of allocation of resources within the course across groups. The focus is on one group and how we allocate resources within that group, not on a comparison across groups.

On the other hand, instructors still will be allocating resources to groups, perhaps unequally or inequitably, and this model doesn’t really let us judge that. Furthermore, this model brings up a meta-question about how society should allocate resources to educational reform across groups if the educational reform’s focus is entirely within-group.

Which is Right? Which is Missing?

Yeah. I’m gonna cop out and tell you there is no Right Model here. I certainly prefer the parity and individuality models over the fairness one. However, the fairness model is actually a great tool for understanding the distinction between equity and equality. It emphasizes that even a “do no (more) harm” approach cannot necessarily blithely allocate the same resources to every student.

I’m also not going to brilliantly identify the gaps in these models. I will say that, by the nature of the Rodriguez et al. paper, they are all pretty narrowly focused on one course and one (set of) metrics. More context and a thicker understanding might be important to your model of equity. (That is where the “liberation” and “justice” pictures in the fence diagrams come in.) That seems especially important since the factors that lead to injustice often involve weaving discriminatory factors into unstated assumptions. (All my students come in knowing X. Everyone’s family includes Y. The non-technical word Z means the same thing to all my students.)

Why Not Equity?

I’ve approached equity as something obviously-to-be-desired. There are certainly critiques of equity as a goal, both recent and longstanding, aimed at avoiding equity becoming a drag on the success or excellence of individuals or groups.

I’m not convinced by the urgency of these critiques in my own current cultural and practical context. From my perspective, it seems clear that work is needed to overcome the impact of both historical and ongoing discrimination across many dimensions. Furthermore, we should be striving to create educational interventions that benefit students as a whole. So the nightmare scenario in the imaginary results graph above really is a failure of an educational intervention, regardless of whether our definition of equity might admit it as a “success”.

That said, these critiques highlight the danger of blindly applying a definition of equity and also the added risks to consider in a situation that is “zero-sum” such as selecting a fixed number of “winners” from a pool of candidates.

Skimpy Little Conclusion

I will come out of this paper trying to think more carefully about how my instructional design choices impact individual students with different personal circumstances and groups of students who face structural barriers to success.