A machine learning model learns patterns from past data so it can make predictions on new data. Model drift happens when the real world changes, but the model keeps using patterns that are no longer accurate. This matters because AI systems are used in weather tools, shopping recommendations, fraud detection, health apps, and school software.
A model that was once useful can become misleading if no one checks whether its predictions still match reality.
Model drift is usually found by tracking model performance over time, such as accuracy, error rate, or false positives. Drift can happen because user behavior changes, sensors are replaced, seasons shift, prices rise, or new types of data appear. Teams respond by collecting fresh data, retraining the model, testing it again, and monitoring it after release.
The goal is not to build a model once, but to keep it accurate as the world changes.
Key Facts
- Model drift means a model's predictions get worse because the data it sees now is different from the data it learned from.
- Accuracy = correct predictions / total predictions.
- Error rate = wrong predictions / total predictions = 1 - accuracy.
- Data drift happens when input features change, such as age, location, price, or temperature patterns.
- Concept drift happens when the relationship between inputs and the correct answer changes.
- A common response is monitor performance, detect drift, retrain with new data, then redeploy the model.
Vocabulary
- Machine Learning Model
- A computer program that learns patterns from data and uses them to make predictions or decisions.
- Model Drift
- A decrease in model performance over time because the real-world data or relationships have changed.
- Training Data
- The example data used to teach a machine learning model before it is used on new cases.
- Data Drift
- A change in the kinds or amounts of input data a model receives compared with what it was trained on.
- Concept Drift
- A change in the true relationship between the inputs and the correct output the model is trying to predict.
Common Mistakes to Avoid
- Assuming a high launch accuracy lasts forever. This is wrong because accuracy can drop when people, seasons, technology, or habits change.
- Checking only the model's predictions and not the real answers. This is wrong because drift is measured by comparing predictions with actual outcomes whenever possible.
- Mixing up data drift and concept drift. Data drift means the inputs changed, while concept drift means the rule connecting inputs to answers changed.
- Retraining automatically without testing the new model. This is wrong because new data can contain errors or bias, so the retrained model must be validated before use.
Practice Questions
- 1 A model made 920 correct predictions out of 1000 last month. This month it made 810 correct predictions out of 1000. Calculate the accuracy for each month and the drop in accuracy percentage points.
- 2 A spam filter incorrectly labels 45 real emails as spam out of 900 real emails. What is the false positive rate as a decimal and as a percent?
- 3 A model predicts which students may need math help. It was trained during online school, but now most students are back in classrooms and study habits have changed. Explain whether this could cause model drift and describe one way the school should respond.