Embeddings

An embedding maps an item into a numerical representation used by a model or downstream task. Learned word vectors, for example, can capture patterns from language data without storing a dictionary definition in each coordinate. The original Word2Vec work studies efficient word-representation architectures and evaluates semantic and syntactic analogy questions. Such findings explain why a vector operation may recover a useful relation in a particular trained space; they do not guarantee that every analogy succeeds or every relationship is encoded consistently. Comparing vectors also requires specifying the metric. Cosine similarity divides the dot product by both vector norms, describing direction for nonzero vectors rather than raw magnitude. Inner product alone can produce different rankings when stored vectors have different norms. With both query and stored vectors normalized to unit length, inner product equals cosine similarity. Their squared Euclidean distance is then two minus twice the dot product. For instance, unit vectors with dot product 0.8 have squared distance 0.4, while their Euclidean distance is the square root of 0.4. Searching a large collection can use an approximate nearest-neighbor index such as HNSW. Its hierarchy of proximity graphs supports efficient search under tested configurations, with recall, memory, and latency depending on the actual data and settings. Approximation concerns recovering neighbors under the chosen metric, not whether those neighbors are useful answers. Learned representations can also reflect unwanted associations: Bolukbasi and colleagues studied gender stereotypes in particular word embeddings. That result motivates checking the representation and application rather than assuming that numerical features are neutral.