Computer Science: AI Bias and Fairness Case Studies Worksheet

Question 1

A school uses an AI tool to flag students who may need academic support. The tool was trained mostly on data from large suburban schools, but it is now used in rural, urban, and online schools. Identify two possible fairness concerns and explain why they matter.

Accepted Answer

One fairness concern is that the model may not work as accurately for students from schools that were not well represented in the training data. Another concern is that the model may confuse differences in school resources or attendance patterns with student ability. These concerns matter because students could be incorrectly denied support or unfairly labeled as at risk.

Question 2

A hiring algorithm recommends applicants for interviews. In a test set, 80 out of 100 qualified Group A applicants are recommended, while 60 out of 100 qualified Group B applicants are recommended. Calculate the true positive rate for each group and explain what the difference suggests.

Accepted Answer

The true positive rate for Group A is 80 percent because 80 out of 100 qualified applicants were recommended. The true positive rate for Group B is 60 percent because 60 out of 100 qualified applicants were recommended. The difference suggests that the system is less likely to correctly recommend qualified applicants from Group B, which may be a fairness problem.

Question 3

A facial recognition system is tested on four groups. The error rates are: Group 1, 1 percent; Group 2, 3 percent; Group 3, 12 percent; Group 4, 18 percent. What fairness issue does this reveal, and what should developers investigate next?

Accepted Answer

This reveals unequal error rates across groups. The system makes far more mistakes for Groups 3 and 4 than for Groups 1 and 2. Developers should investigate whether the training data, lighting conditions, camera quality, labeling process, or model design caused these unequal results.

Question 4

A bank uses an AI model to approve loans. The model does not use race as an input, but it uses ZIP code, income, school attended, and employment history. Explain how bias could still enter the system.

Accepted Answer

Bias could still enter the system because variables like ZIP code, school attended, and employment history can act as proxies for protected characteristics such as race or socioeconomic status. Even if race is not directly included, the model may learn patterns from historical inequality and repeat unfair decisions.

Question 5

A hospital AI predicts which patients should receive extra care coordination. The model uses past health-care spending as a measure of need. Explain why this choice could create bias.

Accepted Answer

Using past health-care spending as a measure of need can create bias because spending does not always equal medical need. Some groups may have received less care in the past because of access barriers, cost, discrimination, or lack of insurance. The model could underestimate their needs and continue the unfair pattern.

Question 6

In a content moderation system, posts written in Standard English are incorrectly removed 2 percent of the time, while posts written in a dialect are incorrectly removed 14 percent of the time. Identify the type of error being described and explain why it is harmful.

Accepted Answer

The error being described is a false positive because acceptable posts are incorrectly removed. It is harmful because users who write in the dialect may be unfairly silenced, punished, or discouraged from participating online.

Question 7

A city uses an AI system to decide where to send more police patrols. The training data comes from historical arrest records. Explain one reason this data may be biased and one possible consequence.

Accepted Answer

Historical arrest records may be biased because they reflect where police were sent in the past, not necessarily where all crime occurred. A possible consequence is a feedback loop where the AI sends more patrols to the same neighborhoods, leading to more arrests there and reinforcing the original pattern.

Question 8

A company tests two speech recognition models. Model X has an overall word error rate of 8 percent. Model Y has an overall word error rate of 10 percent. However, Model X has much higher errors for speakers with certain accents. Which model might be fairer, and what additional information is needed?

Accepted Answer

Model Y might be fairer if its errors are more evenly distributed across accent groups, even though its overall error rate is slightly higher. Additional information is needed about the word error rate for each accent group, not just the overall average.

Question 9

An AI resume screener was trained on resumes from employees who were hired over the past 15 years. Most past hires were men. The system begins ranking resumes with women's colleges or women-focused organizations lower. Explain what likely happened.

Accepted Answer

The system likely learned biased patterns from historical hiring data. Because past hiring favored men, the model treated features associated with women as negative signals, even if those features were unrelated to job performance. This is an example of historical bias being reproduced by an AI system.

Question 10

A fairness audit compares approval rates for a scholarship AI. Group A has 50 approvals out of 200 applicants. Group B has 30 approvals out of 200 applicants. Calculate the approval rate for each group and state whether the results raise a fairness concern.

Accepted Answer

Group A has an approval rate of 25 percent because 50 divided by 200 equals 0.25. Group B has an approval rate of 15 percent because 30 divided by 200 equals 0.15. The results raise a fairness concern because Group B is approved at a lower rate, although more investigation is needed to understand why.

Question 11

A developer says, "Our AI is fair because the code treats every user the same way." Explain why this claim is incomplete.

Accepted Answer

The claim is incomplete because fairness depends on more than whether the code follows the same steps for every user. The training data, labels, input features, evaluation metrics, and real-world context can still create unequal outcomes. Equal treatment by code can still lead to unfair impact.

Question 12

A photo-tagging AI labels pictures of adults in business clothing as "manager" more often for men than for women, even when the images are similar. Name one possible source of the bias and one way to reduce it.

Accepted Answer

One possible source of the bias is training data that showed men labeled as managers more often than women. One way to reduce it is to rebalance or relabel the training data so that job labels are not strongly tied to gender when the visual evidence does not support that connection.

Question 13

An AI system for college admissions uses extracurricular activities as one input. Explain how this could disadvantage some applicants, even if the model is accurate on past data.

Accepted Answer

This could disadvantage applicants who had fewer opportunities for extracurricular activities because of work, family responsibilities, transportation limits, cost, disability, or school resources. If the model learns from past admissions patterns without considering unequal access, it may reward opportunity rather than potential.

Question 14

A team wants to improve fairness in an AI model. List three actions they could take before releasing the model to the public.

Accepted Answer

The team could test model performance separately for different demographic groups, review training data for underrepresentation or biased labels, and involve affected communities or domain experts in the design process. They could also document limitations and create an appeal process for people affected by the model.

Question 15

A judge is given a risk score from an AI tool that predicts whether a defendant might miss a court date. The tool's explanation only says "high risk" without showing important factors or confidence. Explain two reasons this lack of transparency is a problem.

Accepted Answer

The lack of transparency is a problem because the judge and defendant cannot understand which factors influenced the score or check whether the factors are fair and relevant. It is also hard to challenge mistakes, measure bias, or hold anyone accountable if the system gives only a label with no explanation.

Sign in to save

Sign in to save

Computer Science: AI Bias and Fairness Case Studies

Related Tools

Related Labs

Related Infographics

Related Cheat Sheets

More Computer Science Worksheets

More Grade 9-12 Worksheets