Click image to open full size
Computer Science
How a Neural Network Recognizes Patterns
Layers of weighted sums and activations
Related Tools
Related Worksheets
A neural network is a computer model inspired by how brains process signals, and it is especially useful for finding patterns in messy data. In handwritten digit recognition, the computer starts with an image made of tiny pixels and tries to decide which number it shows. This matters because the same idea is used in phone cameras, speech recognition, medical scans, and recommendation systems. Instead of following one fixed rule, the network learns from many examples.
Key Facts
- An input image can be represented as pixel values, such as 0 for black and 1 for white.
- Each neuron computes a weighted sum: z = w1x1 + w2x2 + ... + b.
- An activation function changes the weighted sum into an output: a = f(z).
- Hidden layers detect useful features such as edges, curves, loops, and digit parts.
- The output layer gives scores or probabilities for each class, such as digits 0 through 9.
- Training adjusts weights to reduce error, often using loss, gradients, and many labeled examples.
Vocabulary
- Neuron
- A neuron is a small computing unit that combines inputs using weights and a bias, then passes the result through an activation function.
- Weight
- A weight is a number that controls how strongly one input affects a neuron's output.
- Bias
- A bias is an added number that helps a neuron shift its decision point.
- Activation function
- An activation function is a rule that turns a neuron's weighted sum into a signal for the next layer.
- Training
- Training is the process of showing a neural network many examples so it can adjust its weights and improve its predictions.
Common Mistakes to Avoid
- Thinking the network memorizes only exact pictures. This is wrong because a trained network learns patterns that can generalize to new images, although it may still fail on unusual examples.
- Ignoring the bias term in z = w1x1 + w2x2 + b. This is wrong because the bias can change when a neuron activates, even if the inputs stay the same.
- Assuming the largest output score is always perfectly certain. This is wrong because the network chooses the highest score, but the scores may still show uncertainty or confusion between similar digits.
- Believing more layers always make a better model. This is wrong because deeper networks can be harder to train and may overfit if there is not enough good data.
Practice Questions
- 1 A neuron has inputs x1 = 0.6 and x2 = 0.2, weights w1 = 3 and w2 = -1, and bias b = 0.5. Calculate z = w1x1 + w2x2 + b.
- 2 A digit classifier gives output scores for digits 0, 1, 2, 3, and 4 as 0.05, 0.10, 0.70, 0.12, and 0.03. Which digit does the network predict, and what is the score?
- 3 Explain why hidden layers in a neural network can help recognize a messy handwritten 5 better than a single rule like counting dark pixels.