AI bias happens when a computer system learns unfair patterns from the data it is trained on. An AI model does not automatically know what is fair, accurate, or important. It finds patterns in examples, so missing, unbalanced, or historically unfair data can lead to unfair predictions.
This matters because AI tools are used in areas like school, jobs, health care, banking, and public safety.
Key Facts
- AI models learn patterns from training data, so data quality strongly affects model behavior.
- Biased data in plus pattern learning can produce biased predictions out.
- Accuracy = correct predictions / total predictions.
- Group error rate = wrong predictions for a group / total predictions for that group.
- A model can have high overall accuracy but still perform poorly for a smaller group.
- Fairness checks compare model performance across groups before and after deployment.
Vocabulary
- AI model
- An AI model is a computer system that learns patterns from data and uses them to make predictions or decisions.
- Training data
- Training data is the set of examples used to teach an AI model how to recognize patterns.
- Bias
- Bias is a systematic unfairness or error that causes outcomes to favor or harm certain groups.
- Prediction
- A prediction is the output an AI model gives after analyzing input data.
- Fairness check
- A fairness check is a test that compares how well an AI system works for different groups of people.
Common Mistakes to Avoid
- Assuming AI is always objective. This is wrong because AI learns from human-made data that may contain missing examples, stereotypes, or past unfair decisions.
- Looking only at overall accuracy. This is wrong because a model can seem accurate on average while making many more errors for one group.
- Blaming the algorithm without checking the data. This is wrong because biased training data, labels, or sampling can create unfair results even when the code runs correctly.
- Thinking bias can be fixed once and ignored. This is wrong because real-world data changes, so models need repeated testing, monitoring, and updates.
Practice Questions
- 1 A face recognition system correctly identifies 450 out of 500 lighter-skin faces and 360 out of 500 darker-skin faces. What is the accuracy for each group, and which group has the higher error rate?
- 2 A hiring model is trained on 1,000 past employee records. If 800 records are from one group and 200 are from another, what percent of the training data comes from each group? Explain one reason this imbalance could affect predictions.
- 3 A school wants to use an AI tool to recommend students for an advanced class. Describe two fairness checks the school should do before trusting the tool.