Transfer learning is a machine learning method where an AI model reuses patterns it learned from one task to help with a new task. It matters because training a large model from the beginning can take huge amounts of data, time, and computing power. With transfer learning, students can think of an AI model as bringing useful experience from one problem to another.
This makes many real-world AI tools faster and easier to build, such as image classifiers, speech tools, and language apps.
A common workflow starts with a pretrained model that has already learned general features from a large dataset. For example, an image model may learn edges, colors, shapes, and textures before it is adapted to recognize plant diseases or different types of rocks. Developers often freeze some early layers, train new final layers, and then fine-tune the model on the new dataset.
The goal is to keep useful general knowledge while adjusting the model to perform well on a specific new task.
Key Facts
- Transfer learning reuses a pretrained model instead of training every part from scratch.
- A source task teaches general patterns, and a target task uses those patterns for a new goal.
- Accuracy = correct predictions / total predictions.
- Error rate = 1 - accuracy.
- Fine-tuning means updating some or all pretrained weights using new training data.
- Freezing layers keeps selected model weights unchanged during training.
Vocabulary
- Transfer learning
- A machine learning method that adapts knowledge from a model trained on one task to improve learning on a different but related task.
- Pretrained model
- A model that has already been trained on a large dataset before being reused for a new problem.
- Feature
- A useful pattern in data, such as an edge in an image or a keyword in a sentence, that helps a model make predictions.
- Fine-tuning
- The process of training a pretrained model further on a new dataset so it becomes better at a specific task.
- Frozen layer
- A layer in a neural network whose weights are kept fixed while other parts of the model are trained.
Common Mistakes to Avoid
- Assuming transfer learning always works, because it can fail when the source task and target task are too different.
- Using a tiny or biased new dataset, because the model may learn patterns that do not generalize to real examples.
- Fine-tuning every layer immediately, because this can erase useful pretrained knowledge and cause overfitting.
- Judging only by training accuracy, because a model can look excellent on practice data but perform poorly on new data.
Practice Questions
- 1 A pretrained image model gets 84 correct predictions out of 100 test images after fine-tuning. What is its accuracy and error rate?
- 2 Training a model from scratch takes 20 hours. Using transfer learning takes 3 hours for fine-tuning plus 1 hour for testing. How many hours are saved?
- 3 A model pretrained on millions of animal photos is being adapted to classify medical X-ray images. Explain one reason transfer learning might help and one reason it might be risky.