This position paper addresses the lack of public trust in automated decision-making systems and the institutions that deploy them. It introduces the concept of fiduciary responsibility within a data science lifecycle (DSL) as a means to address this issue. The paper explores how bias can manifest in different stages of the DSL, including data acquisition, model building, and post-processing, and proposes distinct mitigation strategies for each stage. The authors argue that by operating transparently and involving stakeholders throughout the DSL, institutions can build trust and accountability in their decision-making processes. To illustrate these concepts, a case study on predictive policing technology used by the Los Angeles Police Department is examined. The paper concludes with specific examples of actions that police departments could take to enhance fiduciary responsibility, such as providing transparent communication through press releases and social media posts, holding town halls to address public concerns, and utilizing multiple perspectives in decision-making processes. Overall, this paper provides a framework for promoting fairness, accountability, and transparency in automated decision-making systems to foster public trust.
- Lack of public trust in automated decision-making systems and deploying institutions
- Introduction of fiduciary responsibility within a data science lifecycle (DSL) as a solution
- Manifestation of bias in different stages of the DSL: data acquisition, model building, and post-processing
- Distinct mitigation strategies proposed for each stage
- Operating transparently and involving stakeholders throughout the DSL to build trust and accountability
- Case study on predictive policing technology used by the Los Angeles Police Department
- Actions that police departments could take to enhance fiduciary responsibility:
- Providing transparent communication through press releases and social media posts
- Holding town halls to address public concerns
- Utilizing multiple perspectives in decision-making processes
- Framework for promoting fairness, accountability, and transparency in automated decision-making systems to foster public trust
Summary1. People don't trust machines that make decisions and the organizations that use them.
2. One solution is to have a person responsible for making sure the machine works properly.
3. Bias can happen at different stages of using machines, like collecting data and making models.
4. There are different ways to fix bias at each stage.
5. To build trust, everyone should be involved in using machines and be able to see how they work.
Definitions- Trust: Believing that something or someone is reliable and will do what they say.
- Automated decision-making systems: Machines or computers that make choices without human help.
- Fiduciary responsibility: When someone has a duty to act in the best interest of others.
- Bias: When something or someone shows unfair favoritism towards one group over another.
- Data acquisition: Collecting information or data from different sources.
- Model building: Creating a representation of something, like a computer program that predicts things based on data.
- Post-processing: Making changes or adjustments after something has been done or created.
- Transparently: Being open and clear about how something works so people can understand it easily.
- Stakeholders: People who have an interest or concern in something, like the public who might be affected by automated decision-making systems.
- Accountability: Taking responsibility for your actions and being held responsible for them if they go wrong.
Introduction
Automated decision-making systems have become increasingly prevalent in our society, from determining credit scores to predicting criminal behavior. However, these systems have also been met with skepticism and mistrust from the public. This position paper addresses this lack of trust and proposes a solution through the concept of fiduciary responsibility within a data science lifecycle (DSL).
The Concept of Fiduciary Responsibility
Fiduciary responsibility refers to the ethical obligation of an individual or institution to act in the best interest of their clients or stakeholders. In the context of automated decision-making systems, it means that institutions must prioritize fairness, accountability, and transparency throughout the entire process.
The Data Science Lifecycle (DSL)
The DSL is a framework for understanding how data is collected, processed, and used in decision-making processes. It consists of three stages: data acquisition, model building, and post-processing.
Data Acquisition Stage
Bias can enter into automated decision-making systems at this stage if there are disparities in the data being collected. For example, historical data used to train predictive policing technology may be biased due to past discriminatory practices by law enforcement agencies.
Mitigation Strategy: Institutions should ensure that their datasets are diverse and representative of all groups within society. This can be achieved through careful selection and curation of data sources.
Model Building Stage
During this stage, algorithms are developed based on the collected data. If there are biases present in the training dataset, they will be reflected in the resulting algorithm.
Mitigation Strategy: Institutions should regularly test their algorithms for bias using techniques such as fairness metrics or sensitivity analysis. If bias is detected, steps should be taken to mitigate it before deploying the system.
Post-Processing Stage
After an algorithm has been trained and deployed for use, its outputs may still exhibit bias due to changes in societal norms or shifts in patterns over time.
Mitigation Strategy: Regular monitoring and auditing should be conducted on automated decision-making systems to identify and address any potential biases that may arise. This can involve seeking feedback from stakeholders and making necessary adjustments to the system.
Case Study: Predictive Policing Technology in Los Angeles
To illustrate the concepts of fiduciary responsibility and bias in automated decision-making systems, the paper examines a case study on predictive policing technology used by the Los Angeles Police Department (LAPD). The LAPD has faced criticism for using this technology, which has been shown to disproportionately target communities of color.
Enhancing Fiduciary Responsibility
The paper concludes with specific actions that institutions, particularly police departments, can take to enhance their fiduciary responsibility and promote trust in their decision-making processes. These include:
1. Transparent Communication: Institutions should provide clear and transparent communication about their use of automated decision-making systems through press releases, social media posts, and other channels. This can help build understanding and trust among the public.
2. Town Halls: Holding town halls or community forums where stakeholders can voice their concerns about these systems can foster open dialogue and address any misconceptions or fears.
3. Multiple Perspectives: Decision-making processes should involve multiple perspectives from diverse groups within society to ensure fairness and accountability.
Conclusion
In conclusion, this position paper highlights the importance of promoting fairness, accountability, and transparency in automated decision-making systems through fiduciary responsibility within the DSL framework. By implementing mitigation strategies at each stage of the DSL and taking proactive measures to enhance trust with stakeholders, institutions can build public confidence in these technologies. It is crucial for institutions to prioritize ethical considerations when deploying automated decision-making systems as they have a significant impact on individuals' lives.
