Embeddings are a way for computers to represent information as lists of numbers. A word, image, sound clip, or student interest can be turned into a point in a mathematical space. This matters because computers cannot directly understand meaning the way people do, but they can compare numbers.
Embeddings help AI systems find patterns, make recommendations, search documents, and group similar items.
Key Facts
- An embedding is a vector, such as v = [0.2, -1.4, 3.1, 0.7].
- Similar items have embeddings that are close together in embedding space.
- Distance can be measured with formulas such as d = sqrt((x2 - x1)^2 + (y2 - y1)^2).
- Cosine similarity compares direction: cos(theta) = (A dot B) / (|A||B|).
- Machine learning models learn embeddings from many examples, not from hand-written rules alone.
- Embeddings can represent words, sentences, images, songs, videos, users, or scientific data.
Vocabulary
- Embedding
- An embedding is a numerical vector that represents the meaning or features of an item.
- Vector
- A vector is an ordered list of numbers that can describe a point, direction, or set of features.
- Embedding Space
- Embedding space is the coordinate system where vectors are placed so similar items are near each other.
- Similarity
- Similarity is a measure of how alike two items are based on their vectors.
- Cluster
- A cluster is a group of points that are close together because they share related features or meanings.
Common Mistakes to Avoid
- Thinking embeddings are the original data, but they are compressed numerical representations of the data.
- Assuming each number in an embedding always has an obvious meaning, but individual dimensions are often hard to interpret by themselves.
- Using only one coordinate to judge similarity, but embeddings usually need many dimensions working together.
- Believing close points are always exactly the same, but closeness means the model sees them as similar based on its training data.
Practice Questions
- 1 Two word embeddings are A = (2, 3) and B = (5, 7). Find the distance between them using d = sqrt((x2 - x1)^2 + (y2 - y1)^2).
- 2 A movie has embedding M = (1, 4). Student interest embeddings are S1 = (2, 4), S2 = (6, 8), and S3 = (1, 1). Which student interest is closest to the movie by straight-line distance?
- 3 Explain why an AI search engine might return results about puppies when a student searches for dogs, even if the exact word puppies was not typed.