In their paper titled "Measurement and Fairness," authors Abigail Z. Jacobs and Hanna Wallach introduce the concept of measurement modeling from the quantitative social sciences as a framework for understanding fairness in computational systems. They highlight that computational systems often deal with unobservable theoretical constructs like "creditworthiness," "teacher quality," or "risk to society," which cannot be directly measured and must be inferred through observable properties, operationalized via a measurement model. This process can lead to a potential mismatch between the theoretical understanding of the construct being measured and its operationalization, resulting in various harms discussed in the literature on fairness in computational systems. Furthermore, the authors point out that fairness itself is an unobservable theoretical construct that is essentially contested. This means it has multiple theoretical understandings depending on the context. They argue that recent debates about fairness definitions are not necessarily about contradictory operationalizations but rather disagreements about different theoretical understandings of the construct itself. By introducing the language of measurement modeling, Jacobs and Wallach provide the computer science community with a systematic approach to explicitly test assumptions about unobservable theoretical constructs. This methodology aims to facilitate the identification, characterization, and mitigation of fairness-related harms within computational systems. Through their work, they shed light on how understanding and addressing mismatches between theoretical constructs and their operationalization can contribute to more equitable outcomes in algorithmic decision-making processes.
Summary- Measurement modeling is a way to understand fairness in computer systems.
- Computational systems deal with ideas that can't be seen, like "creditworthiness" or "risk."
- If we don't match up our ideas with how we use them, it can cause problems.
- Fairness is a tricky idea that can change depending on the situation.
- People are talking about what fairness means, not just how to do it.
Definitions
- Measurement modeling: A method for understanding things by measuring and testing them.
- Computational systems: Programs and machines that process information.
- Theoretical constructs: Ideas or concepts that are not directly observable but are used to explain things.
- Operationalization: Turning abstract ideas into something practical or measurable.
- Fairness: Treating everyone in a way that is right and equal.
In recent years, there has been a growing concern about the fairness of computational systems and their impact on society. From predictive policing to credit scoring algorithms, these systems are often tasked with making decisions that have significant consequences for individuals and communities. However, as highlighted by Abigail Z. Jacobs and Hanna Wallach in their paper titled "Measurement and Fairness," the concept of fairness is not easily defined or measured in these systems.
Jacobs and Wallach argue that one key issue in understanding fairness in computational systems lies in the fact that they often deal with unobservable theoretical constructs such as "creditworthiness" or "risk to society." These constructs cannot be directly measured but must be inferred through observable properties, which are operationalized via a measurement model. This process can lead to a potential mismatch between the theoretical understanding of the construct being measured and its operationalization, resulting in various harms discussed in the literature on fairness in computational systems.
To address this issue, Jacobs and Wallach introduce the language of measurement modeling from quantitative social sciences as a framework for understanding fairness. This methodology aims to explicitly test assumptions about unobservable theoretical constructs within computational systems. By doing so, it facilitates the identification, characterization, and mitigation of fairness-related harms.
One key contribution of this work is highlighting that even the concept of fairness itself is an unobservable theoretical construct that is essentially contested. This means that different contexts may have different understandings of what constitutes fair decision-making processes. The authors argue that recent debates about definitions of fairness are not necessarily about contradictory operationalizations but rather disagreements about different theoretical understandings of the construct itself.
By introducing measurement modeling into discussions around algorithmic decision-making processes, Jacobs and Wallach provide a systematic approach for addressing issues related to fairness. This approach involves explicitly defining theoretical constructs at play within a system and testing them against observable data points to ensure alignment between theory and practice.
Furthermore, this methodology also allows for identifying and addressing potential biases within measurement models themselves. For example, if a model is trained on biased data, it may perpetuate those biases in its decision-making processes. By explicitly testing for these biases through the lens of measurement modeling, steps can be taken to mitigate their impact.
Through their work, Jacobs and Wallach shed light on how understanding and addressing mismatches between theoretical constructs and their operationalization can contribute to more equitable outcomes in algorithmic decision-making processes. This has important implications for not only computer science but also other fields that rely on computational systems such as healthcare or criminal justice.
In conclusion, "Measurement and Fairness" by Abigail Z. Jacobs and Hanna Wallach provides a valuable contribution to the ongoing discussions around fairness in computational systems. By introducing the language of measurement modeling, they offer a systematic approach for understanding and mitigating fairness-related harms within these systems. As technology continues to advance at a rapid pace, this work serves as an important reminder of the need for critical examination of underlying assumptions when it comes to issues of fairness in algorithmic decision-making processes.
We introduce the language of measurement modeling from the quantitative social sciences as a framework for understanding fairness in computational systems. Computational systems often involve unobservable theoretical constructs, such as "creditworthiness," "teacher quality," or "risk to society," that cannot be measured directly and must instead be inferred from observable properties thought to be related to them---i.e., operationalized via a measurement model. This process introduces the potential for mismatch between the theoretical understanding of the construct purported to be measured and its operationalization. Indeed, we argue that many of the harms discussed in the literature on fairness in computational systems are direct results of such mismatches. Further complicating these discussions is the fact that fairness itself is an unobservable theoretical construct. Moreover, it is an essentially contested construct---i.e., it has many different theoretical understandings depending on the context. We argue that this contestedness underlies recent debates about fairness definitions: disagreements that appear to be about contradictory operationalizations are, in fact, disagreements about different theoretical understandings of the construct itself. By introducing the language of measurement modeling, we provide the computer science community with a process for making explicit and testing assumptions about unobservable theoretical constructs, thereby making it easier to identify, characterize, and even mitigate fairness-related harms.
