Large language models are computer programs that learn patterns in human language from very large collections of text. They matter because they can summarize, translate, write code, answer questions, and help people explore information quickly. At their core, they do not store a perfect library of facts, but learn statistical relationships between tokens.
This makes them powerful tools, but also tools that can make mistakes when patterns are incomplete or misleading.
Most modern large language models use the transformer architecture, which processes tokens with attention layers. Attention lets the model compare each token with other tokens in the context so it can estimate which parts of the input are most relevant. During training, the model adjusts billions of numerical weights to make its next token predictions more accurate.
During use, it generates output one token at a time by repeatedly choosing likely next tokens from a probability distribution.
Key Facts
- Text is split into tokens before the model processes it, and tokens may be words, word parts, symbols, or punctuation.
- A language model estimates P(next token | previous tokens), the probability of the next token given the context.
- Self-attention compares tokens using attention weights, often written as Attention(Q, K, V) = softmax(QK^T / sqrt(d_k))V.
- Training reduces prediction error by updating weights with gradient descent, commonly using loss = -log(P(correct token)).
- A context window is the maximum number of tokens the model can use at one time.
- Temperature controls randomness during generation: lower temperature gives more predictable outputs, higher temperature gives more varied outputs.
Vocabulary
- Token
- A token is a unit of text, such as a word, part of a word, number, or punctuation mark, that a language model processes.
- Transformer
- A transformer is a neural network architecture that uses attention layers to process relationships between tokens in a sequence.
- Attention
- Attention is a mechanism that lets a model assign different importance to different tokens when making a prediction.
- Parameter
- A parameter is a learned numerical value inside a model that helps determine how inputs are transformed into outputs.
- Inference
- Inference is the process of using a trained model to generate predictions or answers from a new input.
Common Mistakes to Avoid
- Thinking the model understands text exactly like a person, which is wrong because it predicts token patterns using learned numerical relationships rather than direct human experience.
- Assuming the most likely answer is always true, which is wrong because a fluent prediction can still contain false or unsupported information.
- Ignoring the context window, which is wrong because tokens outside the window cannot directly affect the model's next prediction.
- Confusing training with inference, which is wrong because training changes the model's weights while inference uses fixed weights to generate output.
Practice Questions
- 1 A prompt contains 850 tokens and the model generates 300 tokens. If the model's context window is 1,024 tokens, how many tokens exceed the window limit?
- 2 A model assigns probabilities 0.50, 0.25, 0.15, and 0.10 to four possible next tokens. What is the probability that it chooses one of the two most likely tokens if sampling follows these probabilities?
- 3 Explain why a large language model can produce a grammatically correct answer that is still factually incorrect.