A fixed-length vector for text, an image or audio, placed so similar meanings land near each other, which turns search into a nearest-neighbour lookup. Vectors only compare within one model, and closeness is not relevance, hence rerankers.
Card 1 of 292. Answer: A fixed-length vector for text, an image or audio, placed so similar meanings land near each other, which turns search into a nearest-neighbour lookup. Vectors only compare within one model, and closeness is not relevance, hence rerankers.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
An embedding is a fixed-length vector that a trained model assigns to text, an image or audio, so that similar meanings land close together. Closeness is usually cosine similarity, cos θ = a·b / (‖a‖ ‖b‖): 1 for vectors pointing the same way, 0 for perpendicular ones.
With unit-length toy vectors, cosine is just the dot product: the query "how do I get my money back" is (0.8, 0.6, 0), "refund policy" (0.6, 0.8, 0) and "GPU memory limits" (0, 0.6, 0.8). The refund page scores 0.96, the GPU page 0.36. At real sizes, a million 768-dimensional float32 vectors take 3.1 GB, so large collections use approximate nearest-neighbor indexes such as HNSW.
The idea took off with word2vec (Mikolov and colleagues, 2013), where king − man + woman lands near queen. Two models' vectors are not comparable, since each learns its own coordinates; switching models means re-embedding everything. For unit vectors ‖a − b‖² = 2 − 2a·b, so cosine and Euclidean distance rank alike. Closeness is not relevance: "cancel my plan" and "upgrade my plan" sit close together.
