A machine learning bias demonstration project helps students see how a model can perform well on average while still making unfair or unreliable predictions for some groups. In this project, you train a small classifier on a dataset, test it on different types of inputs, and compare the results across groups. The goal is not to blame the computer, but to understand how data, design choices, and evaluation methods affect performance.
This matters because machine learning systems are used in areas such as search, recommendations, translation, image sorting, and decision support.
Key Facts
- Accuracy = correct predictions / total predictions
- Group accuracy = correct predictions for one group / total examples in that group
- Error rate = 1 - accuracy
- A model can have high overall accuracy but low accuracy for an under-represented group.
- Training data should be separated from test data so the model is evaluated on examples it has not already seen.
- Bias can come from unbalanced data, missing features, mislabeled examples, or a model that is too simple for the task.
Vocabulary
- Classifier
- A classifier is a machine learning model that assigns an input to one of several categories.
- Training data
- Training data is the set of examples used to teach a model how inputs relate to labels.
- Test data
- Test data is a separate set of examples used to measure how well a trained model works on new inputs.
- Bias
- Bias is a systematic pattern of error that causes a model to perform differently for different kinds of inputs or groups.
- Under-represented group
- An under-represented group is a category that appears much less often than others in the dataset.
Common Mistakes to Avoid
- Using the same data for training and testing makes the model look better than it really is because it may only be repeating patterns it already saw.
- Only reporting overall accuracy hides unequal performance because a large group can dominate the final score.
- Assuming the model is neutral because it uses math is wrong because the model learns patterns from human-collected data and design choices.
- Changing the test set after seeing poor results can weaken the experiment because the evaluation should measure the original question fairly.
Practice Questions
- 1 A classifier correctly labels 84 out of 100 test examples. What is its accuracy and error rate?
- 2 A test set has 80 examples from Group A and 20 examples from Group B. The model gets 72 Group A examples correct and 10 Group B examples correct. Find the overall accuracy, the Group A accuracy, and the Group B accuracy.
- 3 A model is trained on a dataset where one type of image appears 900 times and another type appears 100 times. Explain why the model might perform worse on the second type, and describe one fair way to test for this problem.