A random forest is a machine learning model that uses many decision trees to make a better prediction than one tree alone. Each tree asks a series of simple yes or no questions about the data, such as whether a value is above or below a cutoff. By combining many trees, the model can reduce the chance that one unusual pattern leads to a wrong answer.
Random forests are used in tasks such as classifying emails, predicting house prices, detecting fraud, and helping computers recognize patterns.
Key Facts
- A random forest is an ensemble model made from many decision trees.
- Each tree is trained on a random sample of the training data, often using bootstrap sampling.
- At each split, a tree considers only a random subset of features, which helps trees become different from each other.
- For classification, the forest often predicts by majority vote: prediction = most common class among trees.
- For regression, the forest often predicts by averaging: prediction = (tree1 + tree2 + ... + treeN) / N.
- More trees can improve stability, but they also increase computing time and do not always improve accuracy.
Vocabulary
- Decision Tree
- A decision tree is a model that makes predictions by following a path of yes or no questions from a root to a final answer.
- Random Forest
- A random forest is a group of decision trees that combine their predictions to make a more reliable result.
- Feature
- A feature is an input variable used by a model, such as age, height, temperature, or number of clicks.
- Bootstrap Sample
- A bootstrap sample is a random sample made by choosing from the training data with replacement.
- Majority Vote
- Majority vote is a method where the class predicted by the most trees becomes the final classification.
Common Mistakes to Avoid
- Thinking one tree is the same as a random forest. A random forest uses many trees, so it can average out errors that a single tree might make.
- Using all features at every split. Random forests usually test only a random subset of features at each split, which helps the trees make different mistakes.
- Assuming more trees always means perfect accuracy. More trees can make predictions more stable, but bad data, weak features, or the wrong problem setup can still limit performance.
- Testing the model on the same data used for training. This can make the model look better than it really is because it may have memorized patterns instead of learning to generalize.
Practice Questions
- 1 A random forest has 9 trees for a classification problem. If 6 trees predict Cat and 3 trees predict Dog, what is the final prediction using majority vote?
- 2 A random forest regression model has 5 trees with predictions 12, 15, 14, 10, and 13. What is the final prediction using the average?
- 3 A single decision tree fits the training data almost perfectly but performs poorly on new data. Explain why a random forest might perform better on the same problem.