A loss function is a rule that tells an AI model how wrong its prediction is. It compares the model's output to the correct answer and turns the error into a number called loss. A smaller loss means the model is doing better, while a larger loss means it needs more improvement.
Loss functions matter because they give machine learning systems a clear goal to minimize during training.
During training, the model makes a prediction, calculates the loss, and then adjusts its internal settings to reduce future loss. This process is repeated many times using data examples, so the model gradually improves. Different tasks use different loss functions, such as mean squared error for number predictions and cross-entropy for classification.
In practice, loss acts like a score that guides the model toward better decisions.
Key Facts
- Loss measures how far a model's prediction is from the correct answer.
- Prediction error can be written as error = predicted value - true value.
- Mean squared error: MSE = (1/n)Σ(y_pred - y_true)^2.
- Mean absolute error: MAE = (1/n)Σ|y_pred - y_true|.
- For classification, cross-entropy loss gives a large penalty when the model is confident but wrong.
- Training tries to minimize loss by updating model parameters, often using gradient descent.
Vocabulary
- Loss Function
- A mathematical rule that measures how wrong a machine learning model's prediction is.
- Prediction
- The output or answer produced by an AI model after it processes input data.
- True Value
- The correct answer used to compare against the model's prediction during training.
- Gradient Descent
- An optimization method that changes model parameters step by step to reduce the loss.
- Parameter
- A value inside a model, such as a weight or bias, that is adjusted during training.
Common Mistakes to Avoid
- Thinking loss is the same as accuracy is wrong because loss measures size and confidence of errors, while accuracy only counts correct versus incorrect answers.
- Using the wrong loss function is a mistake because regression and classification tasks often need different ways to measure error.
- Ignoring very large errors is wrong because some loss functions, such as mean squared error, strongly penalize big mistakes.
- Assuming zero loss is always realistic is wrong because real data often contains noise, measurement error, or patterns the model cannot perfectly learn.
Practice Questions
- 1 A model predicts 8 for a true value of 10. What is the error, the absolute error, and the squared error?
- 2 For three data points, the true values are 2, 4, and 6, and the predictions are 3, 5, and 4. Calculate the mean squared error.
- 3 A model classifies animals as cat or dog. It gives 99% confidence for cat, but the correct answer is dog. Explain why a cross-entropy loss function would give this a large loss.