Embeddings
In brief
An embedding represents a text, an image or another object as a list of numbers (a vector), so that items with similar meanings have similar vectors.
Sources[1]
Definition
An embedding is the representation of an object (word, sentence, document, image…) as a vector of numbers. A model is trained so that objects with similar meanings end up close together in this vector space.
What it is used for
- Semantic search: finding texts by meaning rather than by exact words. The query’s vector is compared with those of the documents, often using a measure such as cosine similarity.
- RAG: finding the relevant passages to give a language model before it answers.
- Classification, clustering, recommendation, duplicate detection.
Historical landmarks
The work of Mikolov et al. (2013) on word2vec popularised word embeddings. Current models produce embeddings of whole sentences or documents that take context into account.
Limits
An embedding compresses meaning into a fixed number of dimensions, so nuances can be lost. Quality depends on the model, the language and the domain; a general-purpose model may perform less well on highly specialised vocabulary.
Spotted a mistake or have an idea about this entry? Let us know