Click image to open full size
Neural Networks Explained
Layers, weights, and how neural nets learn
Related Tools
Related Labs
Related Worksheets
A neural network is a computer model inspired by how connected neurons process signals. It learns patterns from data by passing numbers through layers of artificial neurons. Neural networks matter because they power image recognition, speech tools, language models, medical prediction systems, and many other modern AI applications. The core idea is that simple calculations, repeated many times, can learn complex relationships.
Data enters the input layer, is transformed by hidden layers, and produces a prediction at the output layer. Each connection has a weight, and each neuron usually adds a bias before applying an activation function. During training, the network compares its prediction to the correct answer using a loss function, then adjusts weights through backpropagation. Over many examples, these small updates help the network reduce error and improve performance.
Key Facts
- A basic neuron computes z = w1x1 + w2x2 + ... + b, then outputs a = f(z).
- Weights control how strongly one neuron influences another neuron.
- An activation function such as ReLU, f(x) = max(0, x), helps the network model nonlinear patterns.
- Training usually means minimizing a loss function such as mean squared error, MSE = (1/n)Σ(y - ŷ)^2.
- Gradient descent updates a parameter using w_new = w_old - α(dL/dw), where α is the learning rate.
- Backpropagation efficiently calculates how much each weight contributed to the final error.
Vocabulary
- Neuron
- An artificial neuron is a small computing unit that combines inputs with weights and a bias, then applies an activation function.
- Weight
- A weight is a learnable number that controls the strength and direction of a connection between neurons.
- Bias
- A bias is a learnable constant added to a neuron's weighted input to shift its activation threshold.
- Activation Function
- An activation function transforms a neuron's input signal and allows the network to represent nonlinear relationships.
- Backpropagation
- Backpropagation is the method used to compute gradients of the loss with respect to network weights so they can be updated during training.
Common Mistakes to Avoid
- Thinking more layers always make a better network. Extra layers can overfit, slow training, or fail if the data and architecture are not suitable.
- Ignoring data normalization. Inputs with very different scales can make gradient descent unstable and cause slow or poor learning.
- Using the test set during training decisions. This leaks information and makes the final accuracy look better than it will be on truly new data.
- Setting the learning rate without checking training behavior. A rate that is too large can overshoot good solutions, while a rate that is too small can make learning extremely slow.
Practice Questions
- 1 A neuron has inputs x1 = 2 and x2 = -1, weights w1 = 0.5 and w2 = -3, and bias b = 1. Compute z = w1x1 + w2x2 + b.
- 2 A model predicts ŷ = 6 for a true value y = 10. If the loss is squared error L = (y - ŷ)^2, what is the loss?
- 3 A neural network performs very well on its training data but poorly on new data. Explain what problem this suggests and name one method that could help reduce it.