Machine translation is the computer science task of converting text from one language into another while preserving meaning. It matters because people use it to read websites, communicate across borders, and search information written in languages they do not know. Modern systems do not simply swap words one by one, because grammar, context, and idioms often differ between languages.
Instead, they learn patterns from large collections of translated examples.
Key Facts
- Machine translation maps an input sequence x = (x1, x2, ..., xn) to an output sequence y = (y1, y2, ..., ym).
- A tokenizer splits text into words, subwords, or symbols so the model can process it numerically.
- Word or token embeddings convert tokens into vectors, such as cat -> [0.2, -0.7, 0.1, ...].
- Attention weights often use softmax: attention = softmax(QK^T / sqrt(dk))V.
- The model predicts the next translated token using probabilities: P(y_t | y_1, ..., y_{t-1}, x).
- Training reduces error by minimizing loss, often cross-entropy loss = -sum y log(p).
Vocabulary
- Tokenization
- Tokenization is the process of splitting text into smaller units, such as words, subwords, or punctuation marks.
- Embedding
- An embedding is a numerical vector that represents a token so a computer model can compare and process language patterns.
- Neural network
- A neural network is a layered computing system that learns to map inputs to outputs by adjusting many numerical weights.
- Attention
- Attention is a method that lets a model focus more strongly on the most relevant input tokens when producing each output token.
- Training data
- Training data is the collection of example inputs and correct outputs used to teach a machine learning model.
Common Mistakes to Avoid
- Translating word by word, because languages often use different word order, grammar, and expressions.
- Ignoring context, because the same word can have different meanings depending on the surrounding sentence.
- Assuming the highest-probability word is always best, because good translation depends on the whole sentence and later choices can change the best output.
- Thinking the model understands language like a person, because it is mainly learning statistical patterns from examples rather than having human experience.
Practice Questions
- 1 A sentence has 8 input tokens, and a translation model compares each input token with each of 10 output positions using attention. How many input-output attention comparisons are made?
- 2 A tokenizer splits a 120-word paragraph into an average of 1.4 tokens per word. How many tokens are produced?
- 3 Explain why the sentence I saw her duck can be difficult for a machine translation system, and describe what kind of context could help.