Explainable AI, often called XAI, is the study of how to make artificial intelligence decisions understandable to people. Many AI systems can classify images, recommend videos, approve loans, or detect medical problems, but their reasoning can be hard to see. Explainability matters because students, engineers, doctors, and users need to know when to trust a model and when to question it.
A good explanation shows which evidence influenced a decision, not just the final answer.
Key Facts
- Explainable AI helps answer: What did the model decide, and why did it decide that?
- Input data -> model -> prediction -> explanation is a common XAI workflow.
- Feature importance ranks which inputs had the biggest effect on a prediction.
- Accuracy = correct predictions / total predictions.
- Precision = true positives / (true positives + false positives).
- An explanation is useful only if it is understandable, faithful to the model, and relevant to the user.
Vocabulary
- Explainable AI
- Explainable AI is a set of methods that make an AI system's predictions easier for humans to understand.
- Model
- A model is a computer program trained on data to find patterns and make predictions or decisions.
- Feature
- A feature is an input variable, such as age, brightness, word count, or temperature, used by a model to make a prediction.
- Feature importance
- Feature importance is a score or ranking that shows which input features most strongly affected a model's output.
- Black box
- A black box model is an AI system whose internal reasoning is difficult for people to inspect or explain.
Common Mistakes to Avoid
- Assuming a correct prediction means the model used good reasoning. A model can be right for the wrong reason, such as using a shortcut or biased pattern in the data.
- Treating feature importance as absolute truth. Feature importance is an explanation tool, but it can change with the method used, the data sample, and the model design.
- Confusing correlation with causation. If a feature is strongly linked to a prediction, that does not prove it caused the outcome.
- Ignoring who needs the explanation. A student, programmer, doctor, or customer may need different details, so the explanation should match the audience and decision.
Practice Questions
- 1 A model makes 200 predictions and gets 170 correct. What is its accuracy?
- 2 A spam detector marks 60 emails as spam. Of those, 45 really are spam and 15 are not. What is the precision of the spam detector?
- 3 An AI system denies a student scholarship application and explains that the most important features were zip code, school name, and internet search history. Explain why this explanation might raise fairness or privacy concerns, and name one better type of evidence to use.