Kozminski Techblog

Reading Time: 3 minutes

Machine bias and algorithmic injustice are two of the most pressing ethical concerns surrounding the development and use of artificial intelligence (AI). Machine bias is the tendency of AI systems to make decisions that favor or disfavor certain groups of people, even when there is no legitimate reason to do so. Algorithmic injustice is the harm that results from machine bias, particularly when it disproportionately affects marginalized and vulnerable groups.

The sources of machine bias

Machine bias can arise from a number of sources, including:

• Biased training data: If AI systems are trained on data that reflects or amplifies existing social biases, the systems will learn to make decisions that are also biased. For example, if an AI system is trained to predict recidivism using data from a police department that has a history of racial profiling, the system will likely learn to associate certain racial groups with crime, even if that association is not accurate.
• Biased algorithms: Even if AI systems are trained on unbiased data, the algorithms themselves may be biased. This can happen if the algorithms are designed to maximize profits for a company, for example, or if they are not properly audited for bias.
• Biased human input: Even if the training data and algorithms are unbiased, human input can introduce bias into AI systems. For example, if human experts are used to label data or to train the algorithms, their own biases can be reflected in the system’s outputs.

The dangers of algorithmic injustice

Algorithmic injustice can have a number of negative consequences for individuals and society as a whole. For example, machine bias can lead to people being denied jobs, housing, or loans, or being subjected to harsher criminal justice outcomes. It can also lead to the spread of misinformation and the erosion of trust in institutions.

Here are some specific examples of the dangers of algorithmic injustice:

• Criminal justice: Studies have shown that AI-powered risk assessment tools are more likely to recommend bail denials and harsher sentences for black defendants than for white defendants with similar criminal histories.
• Employment: Studies have shown that AI-powered resume filtering systems are more likely to screen out women and people of color than white men.
• Housing: Studies have shown that AI-powered mortgage lending systems are more likely to deny loans to black and Hispanic borrowers than to white borrowers.
• Online advertising: Studies have shown that AI-powered online advertising systems are more likely to show ads for high-paying jobs to men than to women.

What can be done to mitigate machine bias and algorithmic injustice?

There are a number of things that can be done to mitigate machine bias and algorithmic injustice. These include:

• Using unbiased training data: Developers of AI systems should carefully collect and audit their training data to identify and remove any biases.
• Designing AI systems with fairness in mind: Developers should consider the potential for bias at every stage of the design process and implement safeguards to mitigate bias.
• Auditing AI systems for bias: Developers should regularly audit their AI systems for bias and take steps to address any problems that are found.
• Increasing transparency and accountability: Developers and users of AI systems should be transparent about how the systems work and should be held accountable for the systems’ decisions.

Conclusion

Machine bias and algorithmic injustice are serious problems that threaten to undermine the benefits of AI. It is important to be aware of these dangers and to take steps to mitigate them. By working together, we can create a future in which AI is used to benefit all of humanity.

In addition to the measures listed above, there are a number of other things that can be done to mitigate machine bias and algorithmic injustice. For example, governments can develop regulations to ensure that AI systems are fair and transparent. Researchers can develop new methods for detecting and mitigating bias in AI systems. And consumers can demand more transparency and accountability from the companies that use AI systems.

By taking these steps, we can help to create a world where AI is used to benefit everyone, not just a select few.

References

1. Angwin, Julia, Jeff Larson, Surya Mattu, and Lauren Kirchner. “Machine Bias.” ProPublica, May 23, 2016. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
2. Buolamwini, Joy, Timnit Gebru, and Inioluwa Deborah Raji. “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.” arXiv preprint arXiv:1811.12231 (2018). https://arxiv.org/abs/1811.12231.
3. Choulde, Rashida. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” arXiv preprint arXiv:2005.10364 (2020). https://arxiv.org/abs/2005.10364.
4. O’Neil, Cathy. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Broadway Books, 2016.
5. Zuboff, Shoshana. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. PublicAffairs, 2019