Machine learning models learn by practicing on examples, much like students improve by working through many problems. The training data is usually too large to use all at once, so it is split into smaller groups called batches. As the model sees each batch, it makes predictions, measures its errors, and adjusts its internal settings.
Understanding epochs, batches, and iterations helps you see how training is organized and why learning takes time.
In one iteration, the model processes one batch and updates its weights using the error from that batch. When the model has processed every training example once, it has completed one epoch. Training usually uses many epochs because the model often needs repeated passes through the same data to find better patterns.
If the batch size, number of epochs, or learning rate is poorly chosen, the model may learn too slowly, memorize the data, or fail to improve.
Key Facts
- Epoch = one full pass through the entire training dataset.
- Batch = a smaller group of training examples used in one model update.
- Iteration = one training step using one batch.
- Iterations per epoch = number of training examples / batch size, rounded up if needed.
- Total iterations = epochs × iterations per epoch.
- Loss measures prediction error, and training tries to make loss smaller over time.
Vocabulary
- Training data
- Training data is the set of examples a machine learning model uses to learn patterns.
- Epoch
- An epoch is one complete pass through all examples in the training dataset.
- Batch
- A batch is a smaller group of examples processed together during training.
- Iteration
- An iteration is one update step in which the model learns from one batch.
- Loss
- Loss is a number that tells how far the model's predictions are from the correct answers.
Common Mistakes to Avoid
- Confusing an epoch with an iteration: an epoch uses the whole dataset, while an iteration usually uses only one batch.
- Assuming a bigger batch is always better: very large batches can train faster per epoch but may need more memory and may not improve learning quality.
- Forgetting to round up iterations per epoch: if the dataset does not divide evenly by the batch size, the leftover examples still need another iteration.
- Training for too many epochs without checking validation performance: the model may memorize training examples instead of learning patterns that work on new data.
Practice Questions
- 1 A dataset has 1,000 training examples and the batch size is 50. How many iterations are in one epoch?
- 2 A model trains for 12 epochs. Each epoch has 80 iterations. How many total model update steps occur during training?
- 3 A model's training loss keeps decreasing, but its accuracy on new validation data gets worse after many epochs. Explain what might be happening and what change you could try.